xikipedia: Wikipedia as a feed, with a local scoring algorithm

A feed for Simple Wikipedia, with the algorithm on the client

Xikipedia is a pseudo social media feed that algorithmically shows you content from Simple Wikipedia, and the demonstration purpose is stated clearly in the project's own words.

It is made as a demonstration of how even a basic non-ML algorithm with no data from other users can quickly learn what you engage with to suggest more similar content. The algorithm runs locally and no data leaves your device.

That last pair of sentences is the whole design. There is no server, no account, no leaderboard, no shared popularity signal. Every click and like modifies a score in your own browser, and the suggestion is derived from your own behaviour alone. The interesting part for an engineer is that it shows a recommendation system working without the usual crutch of collective data, and it does so with a data set anyone can read.

The deployment is correspondingly simple. The whole application is a handful of files at the top level of the repository: `index.html` for the app itself, `sw.js` for the service worker, `app.webmanifest` for the installability metadata, `favicon.ico` and the two PNG favicons, `version.json`, `process_data.py` for the data generation, and `smoldata.json.br`, the compressed Simple Wikipedia data set.

Once loaded, the app is available fully offline, and it can be installed as an app by clicking the install button, which makes it a PWA rather than a website. The live version is at xikipedia.org, and the default branch is `mane`, which is a small indication that this is a personal project rather than a product.

Four actions with four scores, and a base score that punishes repetition

The algorithm is described in enough detail to be reproducible, which is the point of a demonstration.

Each post has a set of categories, consisting of the post's Wikipedia category tree and the pagelinks in the post, and these categories carry point scores.

The actions and their scores are:

- Scrolling past a post: -5 - Liking a post: 50 + 4*posts_since_last_like - Clicking on an article: 75 - Clicking on an image: 100

The scores are applied through the `engagePost` function in the code.

The like formula is the one worth pausing on, because it encodes something. A like is worth 50 plus four times the number of posts since your last like, so liking something after a long pause is worth much more than liking something in a rapid scroll. The effect is a recency-decayed reward, which pushes the algorithm toward surfacing things that are new to you rather than things you have been liking continuously.

Each post also has a base score, which is 0 by default. If a post has an image, it gets +5 on its base score, so visual posts start slightly ahead. And if you have already seen a post, its base score is `(3**(post_seen_times)-1) * -5000`, so the second sighting is worse than the first, the third much worse, and the curve grows geometrically. That is the mechanism that stops a feed from looping, and the use of a power rather than a linear penalty means a post you have seen five times is effectively suppressed rather than merely discouraged.

There is one hand-tuned correction. The categories `given names` and `surnames` start off with a base score of -1000 due to how prevalent they would be otherwise. On Wikipedia, name categories are enormous and would otherwise dominate any score derived from the category tree, so the author suppressed them by hand.

The selection step is deliberately not greedy

How the next post is chosen is the part of the algorithm that most shapes the experience, and it is not the obvious implementation.

To get the next post in the feed, 10000 random posts are picked out of the data set. Then one of three things randomly happens.

With a 40 percent chance, the scores of all posts are summed together and a random value is picked, which is described as being kind of like picking a random value, except that posts with higher scores have a higher likelihood of getting picked. That is a roulette-wheel selection over the 10000 candidates, weighted by score.

With a 42 percent chance, the post with the highest score is shown. That is the greedy pick, and it is the one a naive implementation would always make.

With an 18 percent chance, a completely random post is shown.

So the top-scoring post appears most of the time, roughly 42 percent of the time, and a large fraction of the remaining goes through the weighted random selection, and a slice of the feed is genuinely random. The effect is that the feed does not become monotonous, because a purely greedy selection over a scoring function that increases monotonically with engagement will converge on a small set of posts and keep showing them.

The 10000-post sample is the other half of that design. Sampling the whole data set for every decision would be expensive, and sampling a large subset preserves the weighted distribution approximately while keeping the per-step work bounded. It also means a post that is not in the sample cannot be shown, so the algorithm's reach is limited by the sample size.

The whole thing is a weighted-random-with-exploration recommender, built without a machine learning library, and it is a decent illustration of the argument in the project's description that a basic algorithm with no data from other users can still learn quickly what you engage with.

Generating your own data set from a WikiMedia dump

Running Xikipedia needs the JSON file that contains the required data, and the repository ships one for Simple Wikipedia, so the app works out of the box.

If you want a different corpus, you can make your own by replacing the files in the `process_data.py` script with your own WikiMedia data dumps, from the Wikimedia dumps site. That is the whole procedure, and it is a good example of a project where the data pipeline is a single readable Python file rather than a framework.

The shipped data file is `smoldata.json.br`, a Brotli-compressed JSON. The compression matters for the offline-first design, because the file has to be fetched once and then cached by the service worker, and Brotli is what a modern browser can decompress natively. The `version.json` file alongside it is what lets the app check whether its cached copy is current, and the service worker is what makes the app work with no network at all after the first load.

The `sw.js` service worker and the `app.webmanifest` together are what make the install button work, and the changelog records both arriving on 2026-02-06, when the project became available as an app and fully available offline through service workers, alongside algorithm data persistence, a settings menu, a light theme, the ability to choose between English and Simple Wikipedias for links, profiles, statistics and an about page.

The app also warns iOS users on the start screen, which the changelog for 2026-02-03 records, and that same entry notes the algorithm script was added to the repository and more information was put in the readme. The initial release was 2026-02-02.

What the project still has not done

The TODO list in the readme is short and gives an honest picture of where the project stands.

- Handle back button on phones. - Maybe an iframe view that doesn't open a new tab. - Renaming profiles. - Reset algorithm without resetting time statistics. - Handle updating the dataset in settings or something. - Export/import profiles (algorithm data).

Each of those is a small thing, and together they describe a project that works but has not been polished. The back button on phones is the one that matters most, because a feed you cannot leave with the browser's back button is broken in a way users notice immediately. The dataset update in settings is the other one, since the data set is a file the app downloads and the user currently has no way to refresh it from inside the app.

The export and import of profiles is the one a power user would want, because the algorithm data is the only thing in the app that is genuinely the user's. The changelog lists algorithm data persistence as optional, which is the right default for a privacy-focused project, and export would be the natural companion.

One detail in the readme worth noting for anyone tracking the project's history. The changelog is described as loose, with a note to check the commit history for more detail. The default branch is `mane`, so the commit history on that branch is where the detail lives.

The project is licensed AGPLv3, and the licensing section makes a distinction that most projects do not: the licence applies to the project itself, but not to the included JSON file that contains data from Wikipedia. The author also says that if you would like to use the project but cannot due to its current licence, to get in touch, because they might relicense it. That is a courtesy worth noting for an AGPL project, since the AGPL is exactly the kind of licence that makes adoption conversations harder.

Editorial conclusion

Xikipedia fits a reader who wants to browse Wikipedia the way they browse a feed, and it fits a developer who wants to see a recommendation algorithm demonstrated on top of a public corpus with no server component at all. It does not fit a reader who wants the full Wikipedia, since the bundled data is Simple English and the item count is limited by the data file. Before you build your own dataset, read `process_data.py`, because that is where the WikiMedia dumps are turned into the JSON the app consumes, and note that the AGPL-3.0 licence covers the project and not the included Wikipedia data.

Frequently asked questions

Official sources

1. Issues
2. License: AGPL-3.0
3. README
4. rebane2001/xikipedia on GitHub