GuppyLM: a 9M parameter fish you can train in a Colab notebook

The problem GuppyLM actually solves

Most introductions to language models start with a pretrained checkpoint. You download weights, wrap them in an API, and never see a tokenizer get trained or a loss curve move. GuppyLM takes the opposite route. The README states the project exists to show that training your own language model is not magic, and the deliverable is a complete path: data generation, tokenizer training, model definition, training loop, and inference, all in one repository you can read in an afternoon. The target reader is someone who can run a Colab notebook and wants to see every stage of the pipeline rather than call someone else's endpoint. The fish persona is the pedagogical device. Because the model is small and the domain is narrow, its failures are legible. When it produces a bad sentence you can trace it back to the template data or the 128-token window instead of shrugging at emergent behaviour.

A vanilla transformer with no modern shortcuts

The architecture table is deliberately unremarkable: 8.7M parameters, 6 layers, hidden dimension 384, 6 attention heads, a 768-wide ReLU feed-forward block, a 4,096-entry BPE vocabulary, and a maximum sequence length of 128 tokens. LayerNorm, learned positional embeddings, and an LM head weight-tied to the embedding matrix. The README says this plainly: no GQA, no RoPE, no SwiGLU, no early exit. That is the point of the project. Every component maps to a few dozen lines in model.py, and there is no configuration flag that changes the attention scheme underneath you. The data flow is equally direct. generate_data.py composes conversations from templates across 60 topics, the tokenizer is trained on that corpus, dataset.py batches it, and train.py runs a cosine learning rate schedule with automatic mixed precision. The published dataset, arman-bd/guppylm-60k-generic, holds 60,000 samples split into 57K train and 3K test, each row shaped as input, output, and category. The categories are the 60 topics: greetings, temperature, food, light, bubbles, cats, rain, and so on. Because every response is assembled from templates, the model learns a narrow distribution well and has nothing to say outside it.

Running it: three commands and a notebook

The browser demo needs no installation. It downloads a quantized ONNX model of roughly 10 MB and runs inference locally through WebAssembly, with no server and no API keys. For local use, the README gives two commands. Install the dependencies with pip install torch tokenizers, then start an interactive session with python -m guppylm chat. A single-turn variant exits after one reply: python -m guppylm chat --prompt "tell me a joke". The training path is a Colab notebook rather than a CLI. You set the runtime to a T4 GPU and run all cells, which downloads the dataset, trains the tokenizer, trains the model, and tests it, then optionally uploads the result to HuggingFace. The README claims about five minutes on a single GPU. That figure is the author's, and it depends on Colab's hardware allocation, so treat it as an order of magnitude rather than a guarantee. Loading the dataset directly is one call: load_dataset("arman-bd/guppylm-60k-generic"), which returns rows such as an input of "hi guppy" with an output of "hello. the water is nice today." and a category of "greeting".

The 128-token window is the real ceiling

The README does not hide this, which is worth crediting. In interactive chat mode the conversation grows and quickly runs into the 128-token limit, reducing quality. There is no retrieval, no summarisation, no sliding window, and no truncation strategy documented. The model simply sees a bounded prefix and degrades as the transcript fills it. That makes the interactive mode a demo, not a usable chat loop; the single-prompt invocation is the more reliable interface. The second limitation is the data. Sixty thousand conversations generated by template composition across 60 topics means the model has seen a finite set of phrasings. It generalises within that distribution and produces confident nonsense outside it. The README is explicit that Guppy does not understand human abstractions like money, phones, or politics, and that this is intentional. If your goal is a small model that answers arbitrary questions, this is the wrong tool, and the wrong tool for a specific reason: the training signal contains no such answers. The third gap is documentation. The project structure listing shows config.py, model.py, dataset.py, train.py, generate_data.py, eval_cases.py, and prepare_data.py, but the README does not describe what prepare_data.py does or how eval_cases.py is scored. Anyone who wants to measure the model beyond reading sample outputs will have to open those files.

Template generation versus a real corpus

The obvious alternative is fine-tuning a small pretrained model such as a distilled GPT-2 class checkpoint on a hand-written persona corpus. The difference is in where the knowledge comes from. GuppyLM starts from random weights and a synthetic dataset, so the tokenizer and every parameter are fitted to a distribution the project itself authored. Fine-tuning starts from a model that already encodes general English and nudges its style. The trade-off is legibility against capability. GuppyLM's approach means you can regenerate the data, retrain the tokenizer, and watch the effect end to end, which is the whole value proposition. The fine-tuning approach gives you a model that handles a stray question about the weather and still talks like a fish, but you inherit a checkpoint you did not build and a tokenizer you did not train. For teaching, the synthetic route is the better one. For anything a user might actually type, the fine-tuned route wins. A third option, if the goal is only the fish persona and not the learning exercise, is a template generator with no model at all: the same generate_data.py logic served directly. It would be faster, fully deterministic, and never hit a context limit. GuppyLM exists precisely because that is not the interesting part.

Licence, maintenance, and what to check first

The README badge points to a LICENSE file and labels the project MIT, and the repository is not archived, with a last push in April 2026. The badge is not the licence; the text in that file is, and the supplied material does not include it, so read it before you redistribute the weights or the dataset. The maintenance surface is small, which cuts both ways. There are no releases retrieved, so there is no versioned artifact to pin, and the Colab notebooks track the main branch. A change to model.py or config.py can silently invalidate a checkpoint you trained earlier, because nothing in the material suggests a compatibility contract between architecture and weights. If you fork it for a course or a workshop, pin a commit hash rather than pointing at main. The dependency list is short, torch and tokenizers, so the upgrade cost is mostly the cost of following PyTorch across major versions. The dataset on HuggingFace is a separate artifact with its own lifecycle; regenerating it locally with generate_data.py is the way to stay independent of it.

Who should pick this up

Use GuppyLM as a teaching artifact. It fits an engineer who has used language model APIs and wants to see the whole pipeline without renting a cluster, and it fits an instructor who needs a complete example that finishes inside a class period. The 60-topic dataset, the 4,096-token BPE vocabulary, and the 8.7M parameter model are all small enough to reason about individually. Skip it if you need a chat product, a model with long context, or anything that answers questions outside tank life. Skip it too if you need a stable released version to depend on, since the material shows no releases and no compatibility guarantee between checkpoints and code. The first thing to verify is the LICENSE file, because the badge is not the licence. The second is prepare_data.py, which the structure listing includes but the README never explains. The third is whether the five-minute training claim holds on your own Colab allocation before you build a lesson plan around it.

Editorial conclusion

Adopt GuppyLM if you want a complete, readable pipeline that goes from synthetic data generation through a 4,096-token BPE tokenizer to a trained 8.7M parameter checkpoint, and you are willing to accept a model that only talks about tank life. Do not adopt it as a general assistant, a production chat backend, or anything that needs to remember more than 128 tokens of context. Before you rely on it, verify two things yourself: the LICENSE file in the repository root, since the badge says MIT but the licence text is the authority, and the contents of prepare_data.py, which appears in the project structure listing but is not described anywhere in the README.

Official sources

1. arman-bd/guppylm on GitHub
2. Issues
3. README