GoFilm: multi-source film collection in Go, Vue and MySQL

How gocolly and robfig/cron feed the vue3 front end

GoFilm ships as 2 programs inside one repository. Inside `client` sits a vite and vue3 application built with ElementPlus, while `server` answers the API through Gin, gorm and go-redis. Collection lives in that same backend: gocolly crawls the configured sites, robfig/cron schedules the runs, and the logic layer turns crawler output into rows the front end can page through.

GoFilm-main
├─ client
│  ├─ src
│  │  ├─ components
│  │  ├─ router
│  │  ├─ utils
│  ├─ package.json
│  ├─ tsconfig.json
│  └─ vite.config.ts

That tree shows where the work sits. `Spider.go` is the crawl entry point, `SpiderRequest.go` builds the outbound requests, `SpiderCron.go` owns the timing, `ProcessCategory.go` decides which categories survive, and `ProcessMovies.go` writes what is left afterwards. Request and response shapes live side by side in `RequestParams.go` and `ResponseJson.go`, and the gorm models sit in `Movies.go` and `Categories.go`.

Two storage tiers and the Sunday four in the morning sync

Update notes describe a storage change with a visible consequence. Main and secondary site data moved into MySQL, which cut Redis memory consumption, yet the collected cache still lives in Redis and only reaches MySQL through a scheduled job at four in the morning every Sunday, chosen to lower the rate of MySQL updates. So the 2 tiers disagree for most of the week, with a gap of up to 7 days between one sync and the next, and anything read from MySQL reflects the last Sunday run rather than a collection in progress. Both connections open inside one binary, in `mysql.go` and `redis.go`. The deployment directory ships a `redis.conf` for the cache and an `nginx.conf` for serving the built client, but no tuning note covers cache size, expiry or eviction, so defaults decide how much the cache holds. Assets under the nginx directory arrive as hashed bundles such as `index-984712d6.js`, which means a front-end change means building the client yourself rather than editing a file in place.

Deploying through the film directory and its Dockerfile

Setup happens through a directory, not a command. The root README carries no shell command for installation; Docker deployment is delegated to the `film` directory and the README inside it, and a visual 1Panel walkthrough is linked from the project blog. Even so, the repository shows the shape of that deployment.

├─ film
│  ├─ data
│  │  ├─ nginx
│  │  │  ├─ html
│  │  │  │  ├─ assets
│  │  │  │  │  ├─ 404-b813c94a.png
│  │  │  │  │  ├─ index-984712d6.js
│  │  │  │  │  ├─ index-de4c7ff5.css
│  │  │  │  │  └─ play-bb9c8990.png
│  │  │  │  ├─ favicon.ico
│  │  │  │  └─ index.html
│  │  │  └─ nginx.conf
│  │  └─ redis
│  │     └─ redis.conf
│  ├─ server
│  ├─ docker-compose.yml
│  ├─ Dockerfile
│  └─ README.md

A full second copy of the backend sits under `film`, in a `server` folder mirroring `config`, `controller`, `logic`, `model`, `plugin`, `router` and `go.mod` from the top level, which makes it easy to patch the wrong copy. Two addresses are given for a first look, the public site and the console:

http://127.0.0.1/index
http://127.0.0.1:3600/manage

Anyone changing backend code has to decide which copy the running image builds from, and the tree records no such choice.

Port 3600, the manage path and a shared default password

An admin group arrived with the new version, covering 6 areas: collection sites, scheduled updates, basic site information, image sync, film categories and film information. Notes beside it say some of those screens are still being completed and that existing functions are unaffected by that. Reaching the console requires a login, the published pair is `admin admin`, and the password is changed after the first sign-in from the drop-down menu in the top right corner. Documentation stays silent on how that credential is stored, how the session stays alive and whether repeated failures are counted, so an operator who puts port 3600 on a public address has to read that code before trusting it. Front-end content moved in the same direction: the site name and the playback source details are now database values edited from the console rather than constants compiled into the client bundle.

The server entry in the README points at the client docs

A close read of the directory listing turns up a link that resolves to the wrong file. Both the server entry and the client entry end at `./client/README.md`, so the single path a reader follows to reach the backend guide and its API notes lands on the front-end document instead. Earlier in the same file the deployment section links the backend to `GoFilm/server` and the front end to `GoFilm/client`, which leaves no doubt that the duplicate is a typo rather than intent. A second gap sits in the access addresses: the console is written with its port filled in as `服务器IP:默认端口/manage`, while the public site example gives the same prefix with no port at all, and 3600 is named in only 1 of the 2 examples. Neither gap blocks a deployment, and both cost time on a first run.

Matching one film across several collection sites

Collection was rewritten rather than patched. Categories are configured before a main site gets crawled, and anything marked as not displayed is skipped, which trims the volume pulled in a single pass. The matching logic changed at the same time, because two sites describe the same film with different identifiers and one comparison stopped holding. Playback moved to `xgplayer`, with playlist and next-episode controls added, and a duplicated rendering problem in the user interface lists was fixed. Still on the plan rather than shipped: aggregated collection that searches the configured sources by film name and adds matches on demand, finer control over what gets collected, and further player bug fixing. Requests for new features and fixes are gathered under issue 71, and the note attached to that thread ties the reply schedule to one maintainer's holiday in mid-January.

An old Dockerfile that stopped building after a Go upgrade

Upgrades carry real work here. The Dockerfile was replaced because a newer Go version and raised dependencies meant the old file could no longer build the image, so an older fork has a rebuild step ahead of any front-end refresh. Storage was swapped out at the same time and redundant fields dropped to save memory, which yields a fresh MySQL schema rather than an in-place migration. No migration script, rollback path or compatibility note for existing rows appears anywhere in the repository, and the console is flagged as partly unfinished. Taken together, that makes an upgrade a re-deploy plus a re-seed rather than a patch. The last push was on 2026-04-23.

The alternative is a catalogue you control, not another crawler

A comparison against another scraper misses the real difference, which is ownership. GoFilm assumes the links live on sites an operator configures in the `采集站点` group and that each crawl will keep finding them, which is exactly why category filtering, per-site matching and a scheduled sync exist. Media already on your own disk, or a fixed list of sources you are allowed to serve, makes all of that dead weight. What stays useful either way: the Gin and gorm API, the vue3 client and the ElementPlus interface, and those sit on top of any catalogue you feed them. So does the reverse, since a hand-written film database rarely arrives on day one with a browsable catalogue, a multi-source player and a console for editing both. The MIT licence covers the repository files, and it says nothing about the media a deployment links to.

Editorial conclusion

GoFilm suits an operator who wants one deployment carrying a browsable film catalogue, several playback sources, a scheduled crawler and an admin console, and who accepts running the whole stack. It is the wrong pick when the media already sits on local disk, since every feature past the API and the interface exists to pull links from sites you configure. Check three things before exposing it: change the admin password published in the README, confirm the port the client binds rather than trusting the address written without one, and read how the two storage tiers interact before relying on a weekly sync.

Frequently asked questions

Official sources

1. License: MIT
2. Project website
3. ProudMuBai/GoFilm on GitHub
4. README
5. Releases