hacker-news-digest publishes a static digest, and its Makefile runs things nobody documents

A static site on GitHub Pages, with a Flask app and a database behind it

The README describes five periodic actions: parse the Hacker News page for a list of articles, extract the main content of each with a score algorithm, find the best illustration and copy it locally, generate summaries, then render a template and deploy to GitHub Pages. The dependency file underneath tells a different story about what has to run for that to happen:

Flask==2.3.3
sqlalchemy==2.0.21
psycopg2-binary==2.9.9
werkzeug==2.3.7

A Flask app plus SQLAlchemy plus the Postgres driver means there is a server and a database in the picture, and the Makefile confirms both. `initdb` runs `python -c 'from db import init_db; init_db()'` and `dropdb` runs `python -c 'from models import db; db.drop_all()'`. Both run targets depend on `initdb`, so creating the database is a precondition for the test target as well as for serving. The published artefact is static, but the machinery that produces it is a web application with persistent storage.

make run points at index.py, which is not in the top level listing

The run targets name two modules that do not appear among the repository's top level entries:

run: initdb
	# DEBUG=1 python index.py
	python index.py

run-in-docker: initdb
	gunicorn -b 0.0.0.0:5000 -c config.py index:app

The top level holds `.github/`, `.gitignore`, `LICENSE`, `Makefile`, `README.md`, a tutorial notebook, `config.py`, `config/`, `db/`, `hacker_news/`, `output/`, `page_content_extractor/`, `probe.py`, `publish.py`, `requirements.txt`, `static/`, `templates/` and `test/`. There is no `index.py` there, and `dropdb` imports a `models` module that is also not at the top level, though either could sit inside one of those directories. What is certain is that `make run` executes `python index.py` and `make run-in-docker` hands gunicorn the application object `index:app` on port 5000, so the Flask side is only usable if you can find where that module lives. `config.py` is the gunicorn config in that command, which makes the lone config file at the root do double duty.

setcron posts to an update endpoint that the README never mentions

One target is a loop that calls the application itself:

setcron:
	while true; do sleep 600; curl -s -H "User-Agent: Update from internal" -L "http://localhost:$(PORT)/update" -X POST `[ -z $${HN_UPDATE_KEY} ] && echo '' || echo -d key=$${HN_UPDATE_KEY}`; done &

Every ten minutes it POSTs to `http://localhost:$(PORT)/update`, identifies itself with a fixed User-Agent string, and appends `key=$HN_UPDATE_KEY` to the form body only when that variable is set. Two things follow. The refresh trigger is an HTTP endpoint on the app, not a cron entry in the repository's own scheduler, and the key travels as a form field in the body rather than as a header, so anything that can reach that port can post to it. The README describes the pipeline as periodic and never mentions `/update`, `PORT` or `HN_UPDATE_KEY`, so this target is the only place the trigger is written down.

Publishing calls sudo apt-get to install minify

The home page target rebuilds the output directory, publishes, copies four static files by name, creates two symlinks and then hashes and minifies:

gh_home_page:
	#find output -maxdepth 1 -type f -delete
	rm -rf output/static
	python publish.py home
	cp -r static output/static
	cp static/ads.txt static/robots.txt static/llms.txt static/404.html output/
	ln -sf index.html output/hackernews  # backward compatibility
	ln -sf feed.xml output/feed
	ln -sf static/favicon.ico output/favicon.ico
	$(MAKE) hash_static
	$(MAKE) minify_static

The asset cache busting reads a checksum of the source file and keeps ten characters of it: `cssname = $(shell md5sum static/css/style.css | cut -c1-10)`, with the same for `static/js/hn.js`. The minify step starts with `@which minify > /dev/null || { sudo apt-get update; sudo apt-get install --no-install-recommends --yes minify; }`, so a build of the static site installs a Debian package as root when the tool is missing. `llms.txt` is copied alongside `robots.txt` and `ads.txt`, which says the output directory is built to be crawled as well as read. The visible Makefile ends mid-recipe, on the line `cp out`, in the hashed JavaScript rule.

It scrapes the Hacker News page, and switching to the API is still open

Step one of the pipeline is parsing the Hacker News page to obtain the list of articles, and the open TODO list still contains a checkbox for switching to the Hacker News API. That single line sets the architecture: the item list comes out of the page rather than the documented API, and the closed items show what the project considered done, sorting by points, comments and time, filtering the top N by points, RSS support and the GitHub Pages deployment, alongside one item reading Have a good sleep !important. The extraction step points at a machine learning tutorial notebook, and that notebook is checked in at the repository root with a filename that the README links to in percent-encoded form. The remaining open items name PhantomJS and Selenium as a better way to scrape sites and HTML9 as the answer to a home page that is not beautiful enough. There is also an open item to summarise comments rather than just articles.

openai 0.28.1 sits next to llama-cpp-python and huggingface-hub

The pinned dependency list is the clearest statement of how the summaries are produced. Summaries come primarily from a remote LLM service, and the README says a local Qwen model is the fallback when remote inference is unavailable. The packages line up with exactly that shape:

openai==0.28.1
llama-cpp-python==0.3.34
huggingface-hub==0.36.2
tiktoken==0.5.2

`openai` is pinned to a 0.x release, the generation era of that SDK, and `huggingface-hub` is what a local model would be fetched with, while `llama-cpp-python` is the runtime that executes it. Translation works the same way, done by the model with one extra step in the prompt, and Chinese is the only language listed, served from `zh.html`. Around those sit the crawler dependencies, `beautifulsoup4`, `lxml`, `chardet`, `fake-useragent`, `pdfminer.six` and `requests`, plus `Pillow` for the copied illustrations, `feedwerk` for the RSS feed and `humanize` and `python_slugify` for rendering.

No releases, an http homepage and an LGPL license on a scraped-content site

The repository publishes no GitHub releases, so there is nothing to pin, while the published site and the README link use two different forms of the same address, an http homepage and https links. The README also carries a link to a Chinese version of the write-up on a personal blog, and the description on the repository page is a single line about letting ChatGPT summarise Hacker News. The license is LGPL-3.0 with the LICENSE file at the root, and nothing on the page says what license applies to the generated output, which is the part most visitors see. The interface leans on a legend of markers rather than labels: the README explains one marker for points, one for the submitting user, one for submission time, one for comment count, one for the source and one for which model wrote the summary, and the feature list adds sorting by any of points, comments or time, filtering to the top N by points, RSS, and embedding for common video sites, PDFs and GitHub gists. The last commit is dated September 30, 2026.

Editorial conclusion

Use it as a reference implementation if you are building the same thing: the pipeline is legible, each stage lives in its own directory, and the pinned dependency list shows exactly how little it needs beyond a crawler and a model. Do not adopt it expecting a maintained service, because there are no releases to pin, the Makefile references a module that is not in the repository listing, and the two ways of running it, a static GitHub Pages build and a Flask app bound to port 5000, are documented to very different degrees. Three things to check first: whether the model path you want is the remote API or the local Qwen fallback, since that decides whether your summaries depend on an outside service; that `make gh_home_page` will call sudo apt-get to install minify, which is not going to work the way you expect on a machine without apt; and what `HN_UPDATE_KEY` protects, since the cron loop posts to an update endpoint on localhost and the README never says that endpoint exists. The license is LGPL-3.0 and nothing on the page says what license the generated summaries carry. The last commit is dated September 30, 2026.

Frequently asked questions

Official sources

1. Issues
2. License: LGPL-3.0
3. polyrabbit/hacker-news-digest on GitHub
4. Project website
5. README