create-llm: a scaffolder for PyTorch language model projects

The setup work create-llm removes, and the setup work it does not

Starting a language model training run from an empty directory means assembling several pieces that have nothing to do with your research question: a transformer definition, a byte pair encoding implementation, a tokenizer training script, a dataset preparation step, a training loop with checkpointing, and something to sample text from afterwards. create-llm exists to collapse that assembly into one command. The README frames the pitch directly: training a model from scratch requires architecture, preprocessing, tokenizer training, a training loop with callbacks, checkpoint management, evaluation metrics, text generation and deployment tools, and the project claims to provide all of it in one command.

The audience is narrow and identifiable. If you already have a training harness you like, the generated code is redundant. If you have never written a training loop, the generated code is useful precisely because it is small enough to read. The four templates map to that audience division: NANO at roughly 1M parameters is described as a two-minute run on any laptop with 2GB of RAM and 100 or more examples of data; TINY at roughly 6M parameters is a 5 to 15 minute CPU or small-GPU run with 1,000 or more examples. SMALL and BASE move into territory where the scaffolding is the easy part: SMALL is listed at roughly 100M parameters, an RTX 3060 or better with 12GB of VRAM, 1 to 3 hours and 10,000 or more examples, while BASE is roughly 1B parameters, an A100 or multi-GPU, 1 to 3 days and 100,000 or more examples.

That last column is the honest part of the README. It tells you the data volume each template expects, which is the constraint most people underestimate. A 1B parameter model trained on a few thousand examples will not produce anything useful, and the documentation says so by implication rather than by warning.

What the generator actually writes to disk

The CLI is written in TypeScript and distributed on npm. Running it produces a project directory containing a Python codebase, not a JavaScript one. The two languages never mix at runtime: Node runs the scaffolder once, then you work in Python. From the workflow section, the generated layout includes data/raw/ for source text, a tokenizer/ directory with a train.py, a data/ directory with a prepare.py, and a training/ directory with train.py. Configuration and dependency pinning live in requirements.txt at the project root.

The data flow is a four-stage pipeline. You drop plain text files into data/raw/. You run python tokenizer/train.py --data data/raw/, which trains a vocabulary from that corpus. You run python data/prepare.py, which the README describes as tokenizing and preparing the data for training. Then python training/train.py starts the run. Each stage consumes the artifact of the previous one, so the tokenizer must be trained before preparation and preparation must finish before training. This is the standard from-scratch arrangement, and it is worth noting what it implies: the vocabulary is derived from your corpus, so there is no pretrained embedding table to inherit and no checkpoint to resume from.

The README lists a set of behaviours under the heading Smart Defaults. Among them: auto-detecting vocabulary size from the tokenizer, handling sequence length mismatches, warning about model and data size mismatches, detecting overfitting during training, suggesting hyperparameters, handling cross-platform paths, and producing diagnostic messages for errors. These are described as features of the generated code, and the README does not document the thresholds or heuristics behind any of them. Treat the list as a set of claims about the template internals that you should verify by reading the generated files rather than as guarantees.

Commands, flags and the Docker path

The fastest route is npx create-llm my-llm, which needs no global installation. A global install works too: npm install -g create-llm followed by create-llm my-llm. Run without arguments, npx create-llm enters an interactive mode that prompts for project name, template, tokenizer type (BPE, WordPiece or Unigram) and optional plugins. Quick mode skips the prompts: npx create-llm my-llm --template tiny --tokenizer bpe --skip-install. Note the flag names: --template and --tokenizer take lowercase values, and --skip-install suppresses dependency installation.

After scaffolding, the sequence in the README is pip install -r requirements.txt, then data preparation, then training. The README suggests starting with at least 1MB of text for meaningful results and gives a concrete example of fetching a corpus: curl https://www.gutenberg.org/files/100/100-0.txt > data/raw/shakespeare.txt.

There is a second, parallel way to run everything, and it is the more interesting part of the documentation. The Docker route builds an image from a clone of the repository with docker build -t create-llm ., then mounts a host directory at /workspace. Scaffolding inside the container is docker run -it -v $(pwd):/workspace create-llm scaffold my-llm --template tiny. Training is docker run --gpus all -v $(pwd):/workspace create-llm train. The chat interface is docker run -p 7860:7860 -v $(pwd):/workspace create-llm chat, which exposes port 7860. A Makefile wraps these: make build, make compose-cli, make compose-train, make compose-chat and make dev, with the chat interface reachable at http://localhost:7860.

The container subcommands (scaffold, train, chat) are a separate interface from the npx flags. If you learn the CLI from the quick-start section and then switch to Docker, the argument shapes change. That inconsistency is not explained in the README.

Where the scaffolding stops being the hard part

The templates are honest about hardware but the README does not discuss what happens when a run fails partway through. Checkpoint management with auto-save is listed as a feature, and the README does not document rollback, resume semantics, or what happens if a checkpoint is written mid-epoch. If you are planning a multi-day BASE run, that omission matters more than any convenience the scaffolder offers. You would need to read the generated training loop to determine whether an interrupted run can be resumed without restarting from step zero.

The second limitation is conceptual. The project trains models from scratch. There is no mention of loading pretrained weights, no mention of fine-tuning an existing checkpoint, and no mention of adapting a Hugging Face model. The HuggingFace plugin is described only as model sharing, which is an output path, not an input path. If your goal is a model that understands English, starting from a random initialization and training on your own corpus is the wrong tool, regardless of template size. The BASE template at roughly 1B parameters and 100,000 or more examples is still a small pretraining run by any external standard, and the README makes no claim about output quality.

The third limitation is the plugin system. WandB for experiment tracking and HuggingFace for model sharing are the only two listed. That is a thin integration surface, and the README does not describe how plugins are wired into the generated code or whether removing one after generation leaves dangling imports. The generated project is a snapshot, not a managed dependency: once created, upstream fixes to the templates do not reach it.

The alternative that most teams should weigh first

The realistic alternative is the nanoGPT-style single-file training script, or one of the Hugging Face example training directories. The difference in approach is not cosmetic. create-llm generates a multi-directory project with separate stages for tokenizer training, data preparation and training, plus deployment scripts, a comparison tool and a chat interface. A single-file script does the opposite: it inlines the model definition, the data loading and the training loop into one readable file, and it assumes you will supply a tokenizer that already matches a pretrained checkpoint.

That distinction decides the choice. If you want to understand every line of the training loop and you intend to modify the architecture, the generated multi-file layout adds indirection between you and the code you care about. If you want a project that already has a data pipeline and a sampling script wired together, and you would rather not write the tokenizer training step yourself, create-llm saves real time. The README's own framing supports this reading: it positions the tool against the list of components you would otherwise assemble by hand, not against any specific training framework.

A second alternative worth naming is simply writing the pipeline yourself for the NANO case. At roughly 1M parameters and a two-minute training time on any laptop, the cost of a wrong architectural decision is low, and the pedagogical value of having written the loop is high. The scaffolder is most defensible at the SMALL and BASE tiers, where the surrounding infrastructure is genuinely tedious and the training run is long enough that you want checkpointing and TensorBoard integration from the start.

Licence, maintenance and the cost of upgrading

The project is MIT licensed, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are retained. No legal advice follows from that; the practical implication is that the generated project carries no copyleft obligation and you can ship models trained with it under whatever terms you choose. The generated code inherits whatever licence the templates carry, and the README does not state whether the template files include a separate notice.

The repository is not archived, and the last push was on 2026-09-13. No releases were retrieved, so there is no version history to inspect and no changelog to read before upgrading. That is the upgrade cost in concrete terms: because the tool writes files rather than installing a runtime dependency, there is no package to bump. Updating means re-running the scaffolder into a fresh directory and diffing against your modified project, or manually porting template changes. The npm package name in the README badge is @theanikrtgiri/create-llm, which differs from the unscoped create-llm used in the npx examples; verify which package resolves on your registry before scripting an install.

For a project you intend to maintain for a year, the absence of releases is the thing to check first. A single last-push date tells you the repository changed recently. It does not tell you whether the templates are stable, whether breaking changes are announced, or whether issues are triaged.

Editorial conclusion

Adopt create-llm if you want a working PyTorch training directory on disk in one command and you accept that you will read the generated code to understand it. Do not adopt it if you need a pretrained model or a tokenizer that is already aligned to a checkpoint; the templates train from scratch on your own corpus. Before committing, run the NANO template end to end and inspect training/train.py and data/prepare.py, because the README documents the workflow but not the contents of those files.

Official sources

1. Issues
2. License: MIT
3. Project website
4. README
5. theaniketgiri/create-llm on GitHub