Learning Beyond Gradients: the artifact repository behind a bilingual article

A repository of evidence, not a package

The repository description calls itself a Heuristic Learning Blog Post, and the README is explicit about scope. It contains the public artifacts for Learning Beyond Gradients, and it links both the published article and the artifact repository. The article is bilingual, with the rendered HTML defaulting to English and including a Chinese switcher. What that means in practice is that the substance of this project is a document, and the code in the tree exists to support the document and to let a reader check its claims. The audience is therefore a reader who has read the post and wants to look at the policy scripts, the trial summaries and the figures, or a reader who wants to reproduce the representative results. The README says the article appendix contains reproduction commands for several representative results, and that those commands assume they are run from the repository root after cloning. That assumption tells you how the repository is meant to be used.

Two Markdown sources and one renderer

The source layout is small and legible. There is learning-beyond-gradient.en.md as the English article source and learning-beyond-gradient.md as the Chinese one, plus the rendered learning-beyond-gradient.html and a single script, render_learning_beyond_gradient.py, described as the local renderer. The relationship is stated plainly: the renderer reads the English and Chinese Markdown files and rewrites learning-beyond-gradient.html in place, and the deployed article is that HTML file. The rendering dependency is minimal. The requirements file contains one entry:

python3 -m pip install -r requirements.txt

which installs markdown 3.5 or newer and nothing else. That is a deliberate choice for a single document, and it means the rendering step has no heavy toolchain attached to it. The cost is that the renderer is the one piece of custom logic that everything else depends on, so anyone changing the article format has to read that script rather than swap in a general static site generator.

Previewing the page locally before you publish it

The README gives a local preview that is close to how the page is served in production, and it prefers that over opening the file directly:

python3 -m http.server 8000

and then opening:

http://127.0.0.1:8000/learning-beyond-gradient.html

The README notes that opening the HTML file directly also works in most browsers, but that using http.server is closer to how the page is served. That distinction matters for a bilingual page, because the switcher and any relative asset references behave differently under a file protocol than under HTTP. Re-rendering after editing the Markdown is a separate step:

python3 render_learning_beyond_gradient.py

The script rewrites the HTML in place rather than writing to a build directory, which keeps the local loop simple but also means the HTML file in the repository is a generated artefact that has to be committed alongside its sources.

GitHub Pages deploys a small _site directory, not the whole repository

Deployment is handled by the workflow .github/workflows/deploy-pages.yml, which runs on every push to main. The README makes one point about it in unusual detail: the workflow does not publish the whole repository as the website root. Instead it builds a small _site directory that contains index.html copied from learning-beyond-gradient.html, a .nojekyll file, and the local files the article references through src or href, such as figures, videos, scripts, CSVs and prompt files. That list is the whole reason the repository layout looks the way it does. The assets are not decoration beside the article, they are referenced by it, so the deployment has to collect them, and the presence of CSVs and prompt files in that list tells you the article points readers at data and at the exact text used for the batch runs.

What is in the artifact tree, environment by environment

The tree is organised by experiment environment, and the README lists what each directory holds. The atari/pong directory has the Pong policy script. atari/breakout has the Breakout policy, trial summaries, a sample-efficiency figure and checkpoint videos. atari/montezuma is the richest, with exploratory policies, state and archive search scripts, summaries, probe images and replay artifacts, which is what a hard-exploration game would need. atari/atari57 holds the aggregate and per-game figures, CSV summaries, and the batch prompt template used for unattended Codex CLI runs, which is the piece that connects the article to a specific tool. The MuJoCo side has mujoco/ant with the Ant policy, a minimal extracted Ant policy, trial summaries, the MuJoCo XML, a sample-efficiency figure and a final-policy video, and mujoco/halfcheetah with a policy script, an iteration log and a sample-efficiency figure. The vizdoom directory has D1 and D3 heuristic scripts plus 35fps ten-seed render videos. The file names tell you what kind of evidence is offered for each claim: a figure, a summary table, a video, or a script to rerun.

Runtime notes that decide whether a rerun works

Two runtime details are called out, and both will decide whether a reproduction attempt succeeds. The first is that the experiments were written against EnvPool 1.1.1, and that the article commands assume the relevant Python environment already has EnvPool and the Atari and MuJoCo runtime dependencies installed. There is no requirements file for the experiments, only the one for the renderer, so the environment for the actual runs has to be assembled separately. The second detail concerns a file path. For Ant, the ant_envpool.xml file stays next to heuristic_ant.py under mujoco/ant/, and the reproduction command references it as:

--mujoco-xml-path mujoco/ant/ant_envpool.xml

The reason for mentioning it is that the path is relative to the repository root, which is the same assumption the appendix commands make. There is also a heuristic_learning directory at the top level of the tree, alongside the figures stored under long generated names.

How to cite it, and the neighbouring project

The README ends with a BibTeX entry for the article, which is what you would use if you want to reference the work:

@misc{weng2026learning_beyond_gradients,
  title = {Learning Beyond Gradients},
  author = {Weng, Jiayi},
  year = {2026},
  month = may,
  howpublished = {\url{https://trinkle23897.github.io/learning-beyond-gradients/}},
  note = {Blog post}
}

The community section points at HL-ImageNet, a project that explores heuristic learning in symbolic visual classification using a non-neural code pipeline, to test how far code-based heuristic updates can go on a constrained ImageNet-style perception task. Its stated purpose is instructive: it is meant as a perception-domain boundary case for the approach, on the grounds that train-only optimisation over symbolic code can still become a memoriser, so the central problem becomes reusable visual representation and generalisation rather than fitting alone. That framing is more useful than the link itself, because it tells you where the author thinks the open problem is. The project has no GitHub releases, the last push was on 2026-05-25, and the repository reports no standard licence identifier, so the terms for reusing the scripts and figures are not stated in the README and have to be checked at the source.

Editorial conclusion

This repository fits a reader who wants to inspect or rerun the experiments behind the Learning Beyond Gradients article rather than just read it, and a writer who wants to see how a bilingual Markdown article is rendered and published. It does not fit someone looking for a reusable library, since there is nothing to install and no public API, and it does not fit someone who wants to run the experiments immediately, because the runtime dependencies are assumed rather than installed. Before using it, read the runtime notes, because the commands assume an environment that already has EnvPool and the Atari and MuJoCo runtimes, and check the licence position yourself, since the repository reports no standard licence identifier.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. Trinkle23897/learning-beyond-gradients on GitHub