weekly.howie6879: two years of curated tech links, and the database behind them

A reading habit that happens to be a repository

The project describes itself as Lao Hu's information technology weekly, recording the valuable information the author saw that week, aimed at projects, software, tutorial material and websites in the computing world, with the content themes heavily guided by the author's own preferences. The stated core purpose is to record information that made an impression, in order to keep it and to share it.

That framing matters, because it explains what kind of artefact this is. It is not a news aggregator and not a link dump service. It is one person's filter, applied weekly, published in a format that survives the week.

The scale is stated too. The weekly began on 2021-08-16, and by the time of writing it had been going for more than two years. The distribution channels are a website at weekly.howie6879.com, an RSS feed, a Telegram channel, a WeChat public account, and a recommendation page that surfaces a subset of items separately. The author notes that the WeChat account is where each issue is published first, and the blog and the account are the places to follow along.

The audience is therefore mostly Chinese-speaking developers, and the secondary audience is anyone who wants the accumulated material. The repository's own metadata lists the topics as awesome, backend, front-end, github, rss and weekly, which is an honest description of a personal link roundup rather than a technical project.

Filtering and aggregation are the stated core problem

The introduction section explains the motivation in a way that is more than a hobby justification, and it is the part to read if you are deciding whether the archive is worth anything to you.

In the internet era, the filtering and aggregation of information are very important. As a programmer, the author says, he often comes across interesting projects, resources, software and websites while browsing, and if the browsing target is striking then it means he has filtered something interesting, and he usually records the corresponding software and aggregates it to look at slowly.

The problem he then names is forgetting. Over time, good resources are always missed, or at some moment you remember something and do not know where to look for it. The feeling of having seen something and forgotten it is not good, so why not build your own resource aggregation, and the weekly was born.

What follows is the part that makes the project interesting to read as a piece of engineering rather than as a list. The weekly is described as one experiment, a sharing format, and the plan is staged in three steps. First, to build a resource sharing format based on the weekly and to find out what happens when accumulated resources are shared continuously, since resources need not only accumulation but effective sharing to produce value. Second, to build a resource aggregation and sharing process that produces more interesting ways to share, so that resources move and the search time of the people on the internet who need them is shortened. Third, and eventually, to produce a script or a product.

The named project behind the third step is FRE123, Free Resource for Everyone, whose stated purpose is to use the resources collected by the weekly to break the information asymmetry for non-geek users and reduce the information gap.

Every issue is a static page with an issue number in its name

The archive structure is visible in the README and it tells you how the site is built.

Each year has an index page, and each issue is a separate HTML document whose URL encodes the date range it covers and the issue number, in the pattern of a start date and end date followed by the title of the issue and a parenthesised issue ordinal. The first issue, in August 2021, is numbered 001, and the numbering continues: the archive links show issue 021 for early January 2022, issue 073 for the first week of January 2023, and issue 123 for the first week of January 2024.

So the archive is not paginated in a database sense. Each issue is a file, named after the period it covers, and the year pages are indexes into those files. That is a deliberate consequence of a static site generator, and it has an obvious property: a URL for a specific week will keep working as long as the file exists, and an issue can be linked to directly without a search step.

The URL naming also means the file names are not stable identifiers in a data sense, because they embed the title, and the titles are written in Chinese. A tool that wants to track issues over time should use the issue number rather than the URL.

The recommendation page is the other piece of the public surface, and it is a separate route rather than a section inside the weekly, which suggests the site is trying to serve two different readers: the person who wants the week's reading and the person who wants a rotating shortlist.

weekly.db is the artefact other people can actually use

The most concrete thing the repository offers to anyone else is the database.

The README says that if you find the weekly content interesting and want to use it in your own product, you can take the processed data directly from weekly.db, and that you should declare the source when you use it. There is a screenshot of what that data looks like, and the file is committed to the repository root, so it is available without building anything.

The wording, 处理过的数据, is worth taking seriously. This is not a raw scrape of every link the author saw. It is a processed set, and processing is where the value and the coupling to the author's own categorisation live. A schema designed for one person's reading habits is unlikely to match a different product's model, and the README does not document a schema version or a compatibility promise, so the file is best treated as a snapshot rather than a stable interface.

There is a second way to contribute, which is worth noticing because it tells you how the weekly stays fresh. If you have resources worth sharing, the author invites you to get in touch by opening an issue for a self recommendation, with a link to a dedicated issue for that purpose. The site and the Telegram channel then become the distribution path for what the community submits.

Together, the database and the issue workflow are the two parts of this project that a stranger can actually build on, and both of them are described in a few lines in a README that is mostly links.

The site is MkDocs, with a Chinese segmenter in the requirements

The requirements file shows what actually builds the site, and it is a short, specific list:

mkdocs~=1.5
mkdocs-material==9.2.8
mkdocs-git-revision-date-localized-plugin==1.0.1
jieba==0.42.1
pyyaml
mkdocs-open-in-new-tab==1.0.3

MkDocs builds the site, the Material theme gives it the look, and the git-revision-date-localized-plugin puts a date on each page based on the commit, which is the right mechanism for a site whose pages are edited weekly. The open-in-new-tab plugin changes how external links behave.

The interesting entry is jieba, the Chinese word segmenter. It is there because the content is Chinese, and the tool that splits Chinese text into words is the one you need for anything word-based, whether that is keyword extraction, indexing or a recommendation heuristic. Its presence next to a project that publishes a recommendation page is a reasonable indication of how that page is built, though the README does not describe the algorithm.

The repository layout matches a small static site with a generation script. There is a main.py, a random_weekly.py, an mkdocs.yml, a Pipfile alongside the requirements file, a .pylintrc and a setup.cfg, a deploy.sh, an nginx.conf, and the directories you would expect: docs, src, public, archive and overrides, where overrides is the standard MkDocs place to replace theme templates.

The deploy script and the nginx configuration are the parts worth reading if you are thinking of running your own instance, because they encode the deployment assumptions that the README does not describe.

There is no licence, which is the constraint on reuse

The repository has no licence file in its top level listing, and the licence metadata is not classified either. That is the single most important fact for anyone considering reuse.

In most jurisdictions, code published on a public repository without a licence is not open source in any usable sense. The default is exclusive copyright, and the absence of a grant means the author has not given permission to copy, modify or redistribute. The project is public and readable, and that is all the permission it carries.

The one explicit request the project makes is about attribution: use the data from weekly.db and declare the source. That is a request rather than a licence, and it does not by itself create a grant.

The project also asks for a Star, which is a different kind of support request, and it invites contributions through the self recommendation issue and through pull requests, with the invitation for patches stated in the support section. Contributing is a normal route into the project and would normally be covered by a contribution licence, but the README does not name one.

The weekly also links to a personal blog and a WeChat account, and the FRE123 project is presented as a separate effort with its own announcement article, which suggests the project may grow into something with its own terms. For a reader, the practical advice is short. Read it, subscribe to the RSS feed, and if you want the data, take the processed weekly.db and declare the source, which is the only condition stated.

Editorial conclusion

The weekly fits a developer who wants one curated slice of the week instead of a firehose, and it fits anyone who wants the two-year archive as a queryable data set rather than a pile of links. It does not fit a product that needs a supported feed, a stable schema or a licence grant, because the repository ships no licence file at all. Before you use anything from it, read the archive pages to see the format, check whether weekly.db suits your schema, and, if you publish the data, attribute it, since that is the only request the project makes.

Frequently asked questions

Official sources

1. howie6879/weekly on GitHub
2. Issues
3. Project website
4. README