talkie: a 13B language model that only knows the world before 1931

A 13B model that only knows the world before 1931

Talkie is an inference library for the talkie 13B language model family developed by Alec Radford, Nick Levine and David Duvenaud, and the model family is described in the README as a 13B vintage language model from 1930.

The design is a controlled experiment rather than a product. `talkie-1930-13b-base` is a 13b language model trained on pre-1931 English-language text. Every fact, every idiom, every association the model has comes from a corpus that stops before 1931, which means the model has a period view of the world. It will reason about automobiles and public morality in the vocabulary of people who wrote about them before the Second World War.

That framing explains the whole project, including the odd package design. A 13B parameter model is small enough to run on a single modern GPU, which is what makes the comparison affordable, and the point of the exercise is that the same architecture, the same size and the same training budget produce a model whose knowledge is bounded in time.

The intended user is a researcher in language model behaviour, in interpretability, in historical linguistics or in creative writing, and the intended use is comparison. You run the 1930 model and the modern model on the same prompt, and you look at the difference.

The primary language is Python, the licence is Apache 2.0 with the file at the repository root, the homepage is talkie-lm.com, the default branch is `main`, and the last push was on 2026-05-19. The repository has no GitHub releases, and the repository layout is small: `src/`, `assets/`, a `pyproject.toml` and the README.

The instruction-tuned variant built from etiquette manuals

`talkie-1930-13b-it` is the instruction-tuned model, and the way it was tuned is the most interesting part of the release.

It has been instruction-tuned using a novel instruction-following dataset built from pre-1931 reference works including etiquette manuals, letter-writing manuals, encyclopedias, and poetry collections. It has also undergone reinforcement learning using online DPO to improve instruction-following capabilities.

That is a neat constraint. A modern instruction dataset is mostly synthetic, written by a model, describing how a helpful assistant should behave. This one was assembled from the period's own self-help literature, which is a large body of text about how to conduct yourself, how to write a letter and how to answer a question well. The behaviour the model learns to follow is the behaviour that people in 1930 were taught, which is a different thing from modern helpfulness, and it is a dataset that could only be built because the archive is public domain.

The poetry collections in that list are doing quiet work too. Instruction tuning on verse teaches the model a lot about form, rhythm and compression, which is why the README's own example prompt asks the model to write an essay predicting what life will be like in the year 1960, a question a period writer might have answered.

The chat API mirrors the usual shape, with a `Message` type carrying a role and content, a `chat` method for a full completion and a `chat_stream` method for token-by-token output. The instruction-tuned model is the one to reach for if you want the model to follow a request, and the base model if you want raw completion behaviour.

The modern control model, and the caveat the README insists on

The third model is the scientific control. `talkie-web-13b-base` has the same architecture and training FLOPs as `talkie-1930`, but is trained on FineWeb, and it exists to allow controlled comparisons between modern and vintage models.

Holding the architecture and the compute constant is what makes the comparison interpretable. If you compared a 1930 model with a modern large model, you would be measuring size, data and era at once, and the result would tell you very little. With the control model, the only systematic difference left is the corpus.

The README then does something that is unusual and worth quoting, because it is a research team narrowing its own claims. It says we need to be careful about the claims we make contrasting the behavior and capabilities of the models, because temporal coverage is not the only difference in the pretraining corpora. For example, the distribution of subject matters differs significantly.

That is a real caveat. A pre-1931 corpus is not a modern corpus with the recent decades removed. It is a corpus of a different kind of writing, from a different set of publishers, about a different distribution of subjects, and the model that comes out of it differs in ways that have nothing to do with the calendar. A study that attributes a behaviour difference to knowledge cut-off rather than to subject matter distribution would be making a mistake, and the maintainers say so before a reviewer can.

There is also a modern base model rather than a modern instruction-tuned one, which is the right control for the base comparison and leaves the instruction side of the experiment to the 1930 data alone.

The Python API in four calls

The API is small enough to read in full, which is a deliberate choice for a research tool.

from talkie import Talkie

# Load a base model (downloads from HuggingFace on first use)
model = Talkie("talkie-1930-13b-base")

# Generate a completion
result = model.generate("If scientists discover life on other planets,", temperature=0.7, max_tokens=300)
print(result.text)

# Stream tokens
for token in model.stream("The effects of the automobile on public morality have"):
    print(token, end="", flush=True)

The chat path uses the same object with a message list:

from talkie import Talkie, Message

model = Talkie("talkie-1930-13b-it")

# Single-turn
result = model.generate("Write an essay predicting what life will be like in the year 1960.", max_tokens=600)
print(result.text)

# Multi-turn chat
messages = [
    Message(role="user", content="What were the causes of the French Revolution?"),
]
result = model.chat(messages, temperature=0.7)
print(result.text)

Two details are worth noting for a research workflow. The `temperature` and `max_tokens` parameters are the standard sampling controls, and the presence of `temperature=0.7` in the examples says something about how the maintainers expect the model to be used. And the model is downloaded from HuggingFace on first use, which means the first call in a script pays a download cost.

That is what `download_model` is for:

from talkie import download_model

# Download before loading (useful for setup scripts)
download_model("talkie-1930-13b-base")

The comment names the use case, a setup script, which is the obvious one for a cluster: fetch the weights once onto shared storage, then start inference jobs without every node downloading them.

The CLI, and where the models actually live

The command line interface covers the same ground as the Python API, and its five subcommands map onto the workflow a researcher actually has.

# Generate text
uv run talkie generate "Once upon a time" --model talkie-1930-13b-base -t 0.8

# Interactive chat
uv run talkie chat --model talkie-1930-13b-it

# Download a model
uv run talkie download talkie-1930-13b-base

# Download all models
uv run talkie download all

# List available models
uv run talkie list

The `download all` command matters more than it looks. There are three models, each tens of gigabytes, and a comparison study needs at least two of them, so the download step is a first class part of the workflow rather than an afterthought.

The installation is a clone and a sync:

git clone https://github.com/talkie-lm/talkie.git
cd talkie
uv sync

The `uv sync` call does more than install three Python packages. The project metadata declares an explicit uv index named `pytorch-cu128` pointing at the PyTorch wheel index for CUDA 12.8, and a matching source that routes `torch` to it. That is how the project gets a CUDA build of PyTorch without asking you to configure an index by hand, and it is also why the tool assumes an NVIDIA machine.

The package metadata itself is minimal. The project is named `talkie` at version 0.1.0, it requires Python 3.11 or newer, and its dependencies are `torch>=2.1.0`, `tiktoken>=0.6.0` and `huggingface-hub>=0.20.0`. The `tiktoken` dependency is the tokeniser, and `huggingface-hub` is the download path. The build backend is Hatchling, the wheel packages `src/talkie`, and the console script is `talkie = talkie.cli:main`.

What the hardware requirement does to your experiment plan

The requirements section is short and it constrains everything else.

Python 3.11 or newer, PyTorch 2.1 or newer, a CUDA GPU with at least 28 GB of VRAM for bfloat16 inference, and roughly 26 to 50 GB of disk space per model.

The 28 GB figure is a consequence of the model size. A 13B parameter model in bfloat16 needs about 26 GB for the weights alone, and inference needs room for the activations on top, which is why the floor is stated at 28 GB rather than at the weight size. That is an A100 or an H100 class of card, or a consumer card with the memory. It excludes the smaller 16 GB and 24 GB cards that most people have, which means the experiment is not something you can run on a laptop.

The disk figure compounds it. Three models at 26 to 50 GB each is a hundred gigabytes or more of weights, and if you are running a comparison you need at least the vintage base model and the web control model resident at the same time on the node that runs the experiment.

That is a real constraint on who can do the research, and it is worth stating plainly: this project is reproducible, and it is not cheap to reproduce. A group with one good GPU can do it. A group with a laptop cannot, and would have to use a hosted inference service or a different, smaller model to explore the same question.

The alternative approach, and the one to weigh against, is to use a smaller vintage model, or to use a model with a time-limited corpus that fits in the memory you have, or to work from the published outputs rather than the weights. The difference in approach is between running the exact experiment and approximating it, and for a research project that wants a controlled comparison, the approximation is often not worth much.

Editorial conclusion

Adopt talkie if you are doing research on what a language model looks like when its training data stops in 1930, or if you want a model whose output cannot contain post-war knowledge for an evaluation or a creative constraint, because that is what the project is for and the two-model comparison with the FineWeb base gives you a control. Do not adopt it for production use, for anything where factual accuracy about the modern world matters, or for a machine without a large CUDA GPU. Verify three things first: that your GPU has the 28 GB of VRAM the requirements ask for, because bfloat16 inference on a 13B model does not fit on smaller cards, that you have the disk space for the model you intend to load, since each one is 26 to 50 GB, and that the subject matter distribution caveat is acceptable for your research, since the README warns that temporal coverage is not the only difference between the pretraining corpora.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. Project website
4. README
5. talkie-lm/talkie on GitHub