douyin-hot-hub archives the Douyin leaderboards every hour

Five leaderboards, one repository

douyin-hot-hub records the Douyin hot search leaderboard together with four related ones: the celebrity leaderboard, the live streaming leaderboard, the music leaderboard and the brand leaderboard. The subject is a single commercial platform, Douyin, and the unit captured is the ranked list of trending topics at a point in time. That framing matters, because it is a different kind of artifact from a general trend tracker. Each entry is a topic name with a link to a Douyin search page for that term, so the repository records what people were searching on the platform rather than what was published anywhere else. The repository description in Chinese states the same scope: it records the Douyin hot search, celebrity, live, music and brand lists, updating them every hour and archiving them by day. For an engineer, the useful question is what a data analyst can do with that. A dated archive of leaderboard positions is a research input: you can diff two days to see what entered and left the list, measure how long a topic stayed on it, or compare a marketing campaign against the brand board. The repository is the raw material for that kind of analysis rather than a finished dataset.

The archive starts on 2021-01-11 and is organised by day

The README states that the records run from 2021-01-11 to the present, and the archive is organised per day in a directory called archives. The top level of the repository matches that structure, with an archives/ directory alongside a raw/ directory, and the README links to the archive with the word archives as the link text. So there are two levels of data kept, a raw directory and a dated archive, and the distinction between them is not explained in the README. That is worth flagging for anyone writing a parser: the README does not document what separates raw/ from the per-day archive, nor does it document the file format of either. The repository also carries a template/ directory, which suggests the per-day files are rendered from a template rather than written from scratch, and the presence of the template makes the shape of a daily file guessable even where the README is silent. If your work depends on the exact schema of an archived file, plan to read a recent archive and the template yourself before writing code against it, because the repository does not promise a stable format.

One request library and three scripts

The implementation is small and easy to survey from the repository layout. At the top level there are douyin.py, main.py and util.py, alongside the raw/ and template/ directories and the archives directory. douyin.py is presumably the module that talks to the platform and parses the leaderboards, main.py is the entry point that assembles the output and drives the update, and util.py holds shared helpers such as fetching and file writing. The only declared dependency is in requirements.txt, which lists a single pinned package:

requests==2.25.0

There is no async client, no browser automation library and no database driver in that list. That is a meaningful signal about how the collection is done. The data comes from ordinary HTTP requests, which means the scraper depends on whatever endpoints the platform currently serves and on the headers and parameters those endpoints expect. It also means the refresh loop is a scheduled job that runs a Python script on a timer, and the archived result lands in the repository as files. If you want to run your own copy of the collection, the shape of the work is: fetch with requests, parse the returned leaderboard data, render a dated file from the template, and commit it. The parsing specifics are not in the README, and they are the part that would need the most work to redo.

The README is the current snapshot, so it doubles as a status page

One structural detail of this repository is unusual and, in a small way, clever. The README is not just documentation. It carries the update time line, giving a timestamp with the date, the time to the second and the +0800 offset, and then it lists the current Douyin hot search leaderboard as a numbered list of entries with Douyin search links, followed by further sections for the other boards. In other words, the README is regenerated on every run and acts as a live view of the current leaderboards. If you visit the repository page, you see the latest state without reading any file, and if the automation stops, the README stops moving. The timestamp makes that visible at a glance, which is a good pattern for a data repository because staleness is otherwise hard to spot in a git history. It also means the README grows, so anyone automating against it should read the file rather than scrape the rendered page, since the list is a snapshot rather than a stable interface.

One of a family of hot list repositories

The README lists related projects, and they are siblings rather than dependencies. The same author maintains hot hub repositories for Zhihu, Weibo, Toutiao, GitHub and V2EX. That set tells you something about the approach. The hard part of each of these platforms is different, the Douyin hot search list is one API surface, Weibo's is another, GitHub Trending is a page rather than an API, and V2ex has its own hot endpoint, so each repository solves its own access problem while sharing the same output convention: a current list in the README, a dated archive in the repository, and an hourly refresh. The practical advantage for a user is that the shape of the data is familiar across boards, so a script written to diff one repository can be pointed at another. The corresponding limit is that the shared convention is a convention, not a contract. Each sibling has its own cadence, its own quirks and its own update time, and the README of each documents only its own platform. If your analysis spans several platforms, expect to normalise the formats yourself rather than finding a common schema.

What the refresh does not guarantee

The limits of this project come from its design rather than from a documented caveat list, so it is worth naming them. First, hourly is not real time. The README states the data is scraped once an hour, and every entry in the archive is a sampled snapshot, so a short lived topic that rose and fell inside one hour can be missed entirely. Second, the archive is a record of what Douyin chose to promote, which is a curated surface rather than a measure of attention. The leaderboards are the platform's own ranking, so the data inherits whatever ranking and filtering the platform applies, and the repository does not describe those rules. Third, there is no API and no service contract. The README does not describe an endpoint, a schema version, a rate limit or a quota, and the repository ships no changelog or release history, so a consumer has no way to be told when the shape of the data changes. The dependency on a live platform endpoint also means the collection can break without notice when that endpoint changes, which is the main operational risk in adopting a scraper that stores its output in version control.

Licence, cadence and what is in the repository

The project is licensed under MIT, and the LICENSE file sits at the repository root, which is the permissive case for anyone planning to analyse, republish or build on the collected data. The repository is not archived and the last push was on 2026-10-20, so the collection is still running, and the update time shown in the README reflects a refresh on that day. There are no GitHub releases, so there is no version history to consult, and the top level carries a .github/ directory along with .gitignore, the README, requirements.txt, the scripts and the data directories, which is a minimal footprint. The archives directory is the part that grows without limit, since it holds one set of files per day across five boards going back to 2021-01-11. If you clone the repository you are cloning that history, so a shallow clone or a sparse checkout of recent archives is the practical way to get started rather than pulling years of daily snapshots in full.

Editorial conclusion

Adopt douyin-hot-hub when you need a dated, diffable history of Douyin leaderboards rather than a live API, and when the roughly hourly refresh and the Git history are good enough for your resolution. Do not adopt it for real time monitoring, for anything that needs a guaranteed API contract, or as a substitute for a licensed data feed, since the README does not document an API, a rate limit policy or the request path used for scraping. Verify first that the archives directory holds the history you need for your date range, that the refresh time recorded in the README is recent enough for your purposes, and that the scraping approach still works before you rely on the hourly cadence.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. lonnyzhang423/douyin-hot-hub on GitHub
4. README