# mini-AGI: continual byte-level learning that outgrows its VRAM budget > mini-AGI is a Python project that trains a byte-level language model from scratch on a single 8GB GPU, keeps its weights on disk and pages them onto the card as needed, and grows or prunes its own capacity while it reads a single stream of data. It suits someone who wants to watch continual learning work without a cluster, and the author is explicit that it is a toy rather than a capable model. **volotat/mini-AGI** — Continual learning model trained from scratch on 8GB VRAM laptop with batch-1 stream of data. - Repository: https://github.com/volotat/mini-AGI - Stars: 1,267 · Forks: 203 - Language: Python - License: MIT - Published: 2026-10-19 · Updated: 2026-10-19 · Language: en - Canonical page: https://hysenlabs.com/projects/volotat-mini-agi ## The architecture question is about growth and pruning mini-AGI is a continual learning byte-level language model that assembles its own architecture, trains from scratch on a single 8GB VRAM GPU, and keeps learning from everything it reads. The unusual part is the second half of that sentence. It stores its weights as ordinary files on disk and pages them onto the card as it needs them, so the parameter count is bounded by free disk space rather than by VRAM. It grows new capacity while training when it runs short, prunes what nothing asks for, and reads through exactly the same code path it serves on. Three mechanisms are doing the work here, and each is worth separating. Paging the weights means the model can be larger than the card that trains it, which is the usual reason a model stops growing. The weights are files, so the constraint becomes disk, and disk is cheap. Growing new capacity while training when it runs short is the interesting one. The model does not have a fixed architecture decided in advance. It adds capacity when it needs it. And pruning what nothing asks for is the counterpart, because growth without pruning is just a memory leak with a training loop attached. Together they make the parameter count a function of the data and the demand rather than a constant chosen at the start. The phrase that reads through exactly the same code path it serves on is the one to notice. The model learns from the corpus by doing the same thing it does when it generates, so there is no separate training pipeline and no train and serve skew, no conversion of the input distribution into a different format for the training path. The name is a half-joke and not a statement of the current capabilities of the model, but rather the potential and traits it has. Continual single-stream learning and natural size and processing adaptation to the available resources exactly what the author expects from an AGI to have. It has just a small toy-level memory footprint and hence it is mini-AGI. ## The author is clear that this is an experiment, not a model The most important paragraph in the documentation is the one that says no, and it is worth reproducing in substance because it changes how the whole project should be read. The note says that as of now this is a small toy-level model, that you should not expect frontier level capabilities, and that this is rather a small experiment to show that continual learning from the single stream of data without catastrophic forgetting is possible. Furthermore it is possible on modest hardware, which means that almost everyone could train their own version of the model, or simply continue training this one, exactly as they see it fit. And the capabilities would be bounded by the actual hardware, scale and quality of the data available, and the amount of time one is willing to spend on training the model. So the claim being made is not about model quality. It is about a capability, continual single stream learning without catastrophic forgetting, and about the fact that it can be demonstrated on a laptop rather than in a data centre. The practical consequence for a reader is that the interesting artefacts are the mechanism and the run history, not the outputs. The dashboard screenshot and the training progress graph are the evidence of a working system, and the sample history is where you can see the model actually improving. The project is targeted at a PC or laptop with at least an 8GB VRAM GPU on the board, and the single 8GB figure appears in both the description and the targeting statement, so the hardware floor is one consumer card. The repository layout backs the experimental framing. There is a `minagi/` directory for the implementation, a `corpora/` directory for the data, a `runs/` directory for the run history, a `replication/` directory, a `tools/` directory, plus `train.py`, `serve.py` and a `config.yaml` at the top level. The presence of a `replication/` directory is interesting, because it suggests the author is treating the ability to reproduce a run as part of the artefact rather than an afterthought. ## Reading a run dashboard, in the numbers The README includes a run dashboard and a block of statistics, and those numbers are the most concrete description of how the system works. Every sample round of the run to date: 2,048.8M characters over 2,846 evaluations. And the detail block, which reads like a training log frozen at one moment: ``` step 994,692 2036.4M of 8,037M characters (25.34%) 282 min 397 experts context 4,096 characters of 4,096 reading 1,483 char/s writing 32.3 char/s still gaining -0.0031 deep into it grad norm 1.59 against a clip of 1 clipping train loss 0.4954 lr 1.34e-05 evidence t +1.91 over 65.7 (effect +0.0424) rate x0.045 held-out loss 0.6136 +/-0.0280 nats 0.8852 bits/char perplexity 1.85 gap +0.1182 arithmetic 0.618 chat 0.586 chat_hermes 0.862 chess 0.389 code 0.506 reasoning 0.529 stories 0.401 wikipedia 1.018 repeats 27% of 8-grams, greedy with no guard ``` Several of these fields describe the architecture directly. 397 experts is the mixture of experts count at that point, so the model is a sparse mixture and the expert count grows as training proceeds. A context of 4,096 characters of 4,096 is a full context window being used, and the reading rate of 1,483 characters per second against a writing rate of 32.3 characters per second gives you the read and write asymmetry of a continual learner that spends most of its time consuming. The grad norm of 1.59 against a clip of 1 with clipping noted shows gradient clipping is active and being hit. The evidence t of +1.91 over 65.7 with an effect of +0.0424 is some kind of learning-signal statistic over a window, and the rate multiplier of x0.045 suggests the update rate is being scaled down. The per-domain held-out losses, with chess at 0.389 and wikipedia at 1.018, show the model is much better at the structured domains than at wikipedia, which is not surprising for a byte-level model at this scale. The last line is the one that matters most for the samples, and it says repeats 27% of 8-grams, greedy with no guard. The model repeats itself, and the raw samples show exactly how badly. ## The repetition trace, and what the samples show The samples block in the documentation is presented as a comparison between two readings of the same prompt, and the comparison is the most useful thing in the project. The round with the lowest held-out loss so far is 0.6136 nats at 2,036.4M characters. Two readings of each prompt are given, `raw` is plain greedy with no guard at all, and `adapted` is the same with the repetition trace on. The repetition trace is a decoding intervention rather than a model change. It detects that the model is about to repeat an n-gram it has already produced and steers away from it, and the effect on the samples is dramatic. On the code prompt, the raw reading repeats 71% of 8-grams and produces a comment line that reads Convert a sorted array to a sorted sorted sorted sorted sorted sorted sorted sorted sorted sorted sorted sorted sorted sorted sorted, and then the sample stops. The adapted reading of the same prompt repeats 1% of 8-grams and produces a coherent comment, a plausible docstring and a function that starts to make sense, even though it still has a type error further down. On the arithmetic prompt, the raw reading repeats 59% of 8-grams and gets stuck in a loop of add 7 + 8 = 15 after one correct answer, and eventually emits a think block. The adapted reading repeats 0% and answers three further additions correctly, with the last one wrong but the behaviour is that of a model doing arithmetic rather than a model stuck in a loop. The chat prompt shows the same pattern, with the raw reading producing garbled output at a 1% repeat rate, and the adapted reading giving a self-description that actually mentions single characters and byte values. The reasoning prompt is the interesting exception, where both readings repeat little, and the adapted version actually breaks down mid-sentence. So the trace is a mitigation, not a cure, and it can suppress a repetition that was not the problem. The whole history is in the samples file, and the documentation says you can inspect it yourself to see how the model improved over the course of training and reading the corpus. That is the artefact to spend time on, because the quality of a model that learns from one pass over a corpus is best judged by watching it improve rather than by reading a final number. ## No final weights, and a run measured in weeks The weights situation is stated early and it is the practical constraint on using this project. The final weights are not published yet. The run is still reading its first pass over the corpus, the weights go up once it has been through all of it, which is several weeks away at the current rate. If you would like to play with undertrained weights as they are right now, you can find the most recent snapshot, dated 12 Oct 2026, on a Hugging Face repository under the name Volotat/mini-AGI-undertrained. So the honest position is that the published artefact today is an undertrained checkpoint and a run history, not a finished model. The corpus is 8,037M characters in total, and the run was at 25.34% of it when the statistics were captured, at a reading rate of 1,483 characters per second. That gives the scale of the remaining time, and the documentation says several weeks rather than days. The dashboard image is described as a min-run dashboard, and the model is pointed to the corpus to constantly read and learn from it. So the running system is the project, and the screenshots and the samples are the output. There is also a graph of the whole run so far, described as every sample round of the run to date, 2,048.8M characters over 2,846 evaluations. A graph of every evaluation round is a lot of data points to plot, and it is the kind of artefact that tells you about the training dynamics rather than the final result. For anyone wanting to run it themselves, the visible entry points are `train.py` and `serve.py` at the top level, with `config.yaml` for the configuration, `corpora/` for the data, and `tools/` for the rest. The replication directory is the one to look at first, because a continual learning experiment that cannot be reproduced is much less useful than one that can, and the author has clearly thought about that. The project is MIT licensed, the repository is not archived, and the last push was on 2026-10-14, so the run is ongoing and the documentation is a moving target. ## Conclusion mini-AGI fits a developer who wants to understand continual learning from a single stream without catastrophic forgetting, running on hardware they already own, because the whole point is that the weights live on disk and the architecture adapts to what the machine can hold. Skip it if you want a capable model, since the author states plainly that this is a small toy-level model with no expectation of frontier level capabilities, and note that the final weights are not published yet. Before you start a run, read the sample history in runs/samples.txt, because the current generations show the repetition problem that the repetition trace addresses, and check the corpus and the reading rate, since the run is a single pass and a full pass takes several weeks at the current rate. The last push was on 2026-10-14. ## FAQ ### What is mini-AGI? It is a continual learning byte-level language model that assembles its own architecture, trains from scratch on a single 8GB VRAM GPU, and keeps learning from everything it reads, with its weights stored as ordinary files on disk and paged onto the card as needed. ### How does mini-AGI handle being larger than the GPU? It stores its weights as files on disk and pages them onto the card as needed, so the parameter count is bounded by free disk space rather than by VRAM. It also grows new capacity while training when it runs short and prunes what nothing asks for. ### Are the trained mini-AGI weights published? Not the final ones. The run is still on its first pass over the corpus and the final weights go up once it has been through all of it, which is several weeks away. An undertrained snapshot dated 12 Oct 2026 is on Hugging Face under Volotat/mini-AGI-undertrained. ### What is the repetition trace in the mini-AGI samples? It is a decoding guard applied on top of the model that steers away from repeating n-grams the model has already produced. The samples compare raw greedy decoding against the same decoding with the trace on, and the repeat rate drops sharply, for example from 71% of 8-grams to 1% on the code prompt. ### How capable is the mini-AGI model? The author is explicit that it is a small toy-level model with no expectation of frontier level capabilities, and that it exists to show continual single stream learning without catastrophic forgetting is possible on modest hardware. The licence is MIT and the last push was on 2026-10-14. ## Sources - [Issues](https://github.com/volotat/mini-AGI/issues) - [License: MIT](https://github.com/volotat/mini-AGI/blob/main/LICENSE) - [README](https://github.com/volotat/mini-AGI/blob/main/README.md) - [volotat/mini-AGI on GitHub](https://github.com/volotat/mini-AGI) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/volotat-mini-agi