Node-SpliderApi: an Express and MySQL service serving Chinese content data

One Express service for a pile of Chinese content domains

Node-SpliderApi is a Node and Express project that wraps a set of scraped data sources in a single API. The content domains it covers are listed in the repository description and are worth reading as a list of use cases: information about universities across the country, idioms and poetry, horoscope fortune, today in history, music data endpoints, image wallpapers, funny videos, and hot news with detail endpoints.

That mix is characteristic of the genre. This is a back end for a small front end or a mobile client that needs several kinds of reference content in one place, and the value is in the plumbing rather than in any single dataset.

The stack is declared in the README as a checklist: express, sequelize, mysql, express-validator for parameter and form validation, cheerio for parsing scraped pages, ejs for templates, Swagger UI for the API documentation, nodemon for hot reload during development, dotenv for configuration, request signing for verification, and log4js for error log collection.

The package manifest names the project vtools-server, marks it as private with version 0.0.1, and keeps the scripts short. The start script runs the conventional Express entry point, and the dev and prd scripts differ only by the NODE_ENV value they set, both launching the app through nodemon.

$ git clone https://github.com/ecitlm/Node-SpliderApi.git
$ npm install
#  start project dev
$ npm run dev
# starting prd
$ npm run prd
#localhost:3001

The port is 3001, and the service listens there once the script is running. The project is MIT licensed, the default branch is master, and the last push was on 2026-06-18.

Most of the data ships with the project, one domain does not

The database section of the README explains where the content comes from, and the answer differs by domain.

The MySQL database that comes with the project contains 300 Tang poems, a set of idioms, the today in history data, and a user table. The project directory has a bak-file folder where these can be inspected, and the README shows a screenshot of the API documentation there as well.

Horoscopes are the exception. The data is aggregated from another platform, and using it requires applying for your own APPCODE from that provider. In other words, the horoscope content is a proxy in front of somebody else's API, with a key you have to obtain yourself, while the bundled datasets are data you already have.

That distinction matters for deployment planning. The bundled datasets need nothing beyond the database import, while the horoscope endpoints need a credential, and the credential is a dependency on an external service that the project does not control.

The other direction of data flow is the scraper side. Cheerio is used to parse pages, and axios, request, superagent and node-fetch are all in the dependency list, which means the service both serves stored data and fetches and parses live pages. The endpoints for music data, wallpapers, funny videos and hot news are the ones likely to depend on that scraping path, and that is also where the fragility sits, since a page that changes its markup breaks a parser that nobody owns.

The supporting dependencies hint at the data work. iconv-lite handles character encoding, lunar-calendar-fix exists for calendar conversions, dayjs for dates, and idcard for Chinese identity card checks, which suggests some endpoint validates an identity number.

Swagger docs, a request signature, and a documented error code table

Once the project is started, the API documentation is available at the api-docs path on port 3001. The tooling behind it is express-jsdoc-swagger, which generates the specification from the source comments, with express-routemap available for listing the routes.

Requests are signed. The README lists a sign request verification feature among the checklist items, and the dependency list shows the pieces: md5 for the digest, jsonwebtoken for tokens, and express-validator for validating parameters before the handler runs. The error code table then tells you what a client is expected to handle, and it is a good example of an API that documents its failure modes rather than returning a bare 500. The table lists 200 for a normal response, 1001 for an invalid parameter such as a non-existent id, 1002 for an empty, failed or wrongly typed parameter, 1003 for an invalid signature, 404 for a missing route, 405 for a wrong method, 9999 for a failed third party call, and 500 for a system error.

Two of those codes carry the design. The 1003 code for an invalid signature means the signing scheme is enforced on the routes that need it. The 9999 code for a third party request error means the service acknowledges that some of its data comes from elsewhere, and it separates that failure from its own 500. A client can therefore tell a bug in the service from a provider being unavailable, and can retry the second kind.

The middleware list adds the operational layer. express-rate-limit is in the dependencies, apicache is there for caching responses, express-session for sessions, morgan for request logging, and log4js for error log collection. For a service that proxies scraped content, rate limiting is not a nicety, since the upstream sites and the aggregator platform both have limits of their own.

The deployment story is pm2 in front of a nodemon process

The README is specific about how the service is run on a server, and the recipe is short:

pm2 start npm --watch --name tools -- run prd
pm2 restart tools
pm2 stop tools
pm2 delete tools

pm2 is the process manager, the process is named tools, and the command starts the npm prd script in watch mode.

There is a wrinkle in that arrangement, and it is visible in the package manifest rather than hidden. The prd script runs nodemon under NODE_ENV=production, and the pm2 recipe adds --watch on top. So the production process is a file watcher that restarts the application when a source file changes, supervised by a second supervisor. It works, and it is a common pattern in this kind of project, but it means the production process is not what the script name suggests, and the two watch layers are redundant with each other.

The environment configuration comes from dotenv, and the repository root contains both a .env file and a .env.development file alongside a .gitignore. Having the environment file in the tree is the detail to look at before you deploy anything: if your instance has real keys, real database credentials or an aggregator APPCODE, they need to be kept out of the tree and out of any copy of it.

The layout also shows how the project is organised. The application code is in src/, the templates in views/, the Express entry point in bin/, and the data in bak-file/. A jsconfig.json and a webstorm.config.js suggest an IDE-oriented workflow, and the package manifest maps the alias @ to the project root and points the module directories at a custom node_modules_custom folder, which is a small sign of a project that has been reorganised by hand more than once.

The version trail is thin, and the code is old in places

Two dates deserve attention before you build on this.

The tagged releases listed for the repository are 2.0.0 from 2020-11-02 and release_20220814 from 2022-08-14. The last push was on 2026-06-18. So the code is being touched, but there is no release tag newer than 2022, and the package is marked private with version 0.0.1, which means it is not published to a registry. Adopting it means adopting a copy, not a dependency, and you own the upgrades from the first commit.

The second thing is the dependency versions. axios sits at 0.26.0 and the express line is 4.17.3, sequelize 6.17.0, mysql2 3.5.1, express-validator 6.14.0. Several of those are old branches of actively developed libraries, and the project also still depends on request alongside axios, superagent and node-fetch, which is three HTTP clients and a legacy one at that. None of that blocks a deployment, but it does mean a security review of the transitive tree is part of the work rather than an afterthought.

There is also no homepage recorded for the repository and no project documentation site, so the README is the whole story, and it is a good but short one. The Swagger page generated from the source is the real reference, and the error code table plus the screenshots in bak-file are the most useful artefacts in the repository.

The offsetting point is that the project is small, readable and complete. It is a service you can read in an afternoon, with the data, the validation layer, the signing, the documentation and the deployment recipe all in one place, which is a genuinely useful reference even for teams who will not run it.

Editorial conclusion

Adopt Node-SpliderApi if you need a small Chinese content API to build a front end against, and you want the data, the validation, the signing and the documentation in one small service rather than three services. Do not adopt it as a dependency you can upgrade over time, because the tagged releases stop at 2022 and the package is marked private, so you own it the moment you clone it. Before you deploy it, replace the committed environment files, obtain your own aggregator key for the horoscope endpoints since the data is proxied from a third-party platform, and check that the scraper endpoints still work against the source sites, because page parsing is the part of the design that breaks first.

Frequently asked questions

Official sources

1. ecitlm/Node-SpliderApi on GitHub
2. Issues
3. License: MIT
4. README
5. Releases