bumblecore pins fifteen dependencies to exact versions and writes the training loop out by hand

Fifteen exact pins make the install reproducible and the upgrade hard

The dependency list is the most opinionated thing in the manifest, and it uses exact versions rather than ranges for everything. The stack runs from a pinned framework version through a pinned transformer library, a pinned distributed training library at a zero-series version, a parameter-efficient fine-tuning library, a tensorboard build, two maths libraries, and then the serving and interface stack: a web framework, an asynchronous server with the standard extra, a validation library, a YAML parser, a console formatting library and a test runner. The project is classified as Alpha, its version is 0.1.0, and there is a single tagged release from December 2025. An optional extra pins an attention implementation to a post-release build and has to be installed with build isolation disabled, which is a small hint about how hard that wheel is to build.

There is no trainer library, and that is the entire premise

The first core feature states that the framework does not rely on any high-level trainer libraries and that every core component is built from the ground up. The list underneath is what that means concretely. Custom data loaders and preprocessing pipelines. Manual configuration of the distributed training environment with what the documentation repeatedly calls a deep integration of the distributed library. Fully controllable forward propagation, backward propagation and parameter update flow, which means the update step is code you can read rather than a callback you register. A flexible loss implementation with multi-task learning support. And an inference generation mechanism written out by hand, including top-p and top-k sampling and the key-value cache. The stated reason is that manual implementation lets you understand the purpose of every line, which makes debugging, optimisation and innovation easier.

Seven model sizes, and the vocabulary changes at 7B

The built-in architecture is called Bumblebee and the documentation says it is inspired by the design of an existing open model family. Configurations are provided from 0.5B up to 72B parameters, in seven steps, and the interesting part is that they are not a clean scale-up of each other. Hidden size goes 896, 1536, 2048, 3584, 5120, 5120, 8192, so the 14B and 32B configurations share a hidden dimension while differing in layer count and intermediate width. Head counts move independently too, and the key-value head count stays at two for the three smallest sizes before stepping up to four and then eight. Vocabulary is 151936 for the three smallest configurations and 152064 from 7B upwards, and the embedding tying flag is true only for those three. The configuration lives in a file under the model directory, and the documentation also gives a separate page for every training parameter.

Two model series are verified, and both through the same three stages

The verified-support table is short and specific. Two series are listed, and for both the verified stages are pretraining, supervised fine-tuning and preference optimisation, both marked as fully tested. Everything else is left as an expectation rather than a claim: the documentation says other models with similar architectures should in general be compatible, invites users to test and integrate additional models, and asks for compatibility problems to be reported as issues. That is a reasonable position for an Alpha project, but it means the compatibility surface is two model families, and the stages are not the full range the framework claims to support. Elsewhere the feature list also names continual pretraining, instruction fine-tuning and reinforcement learning as supported parts of the pipeline, so the verified table is narrower than the feature list.

One data format per stage, and tool calling is supported

Data preparation is specified as three formats, one for each training stage, and every format is accepted in either JSON or JSONL with automatic recognition rather than a flag. Pretraining takes a single text field. Supervised fine-tuning takes the Alpaca shape, the ShareGPT shape or an OpenAI-style messages shape, and the messages option is the one that supports tool calling. Preference optimisation takes the same three shapes, augmented with a chosen and rejected pair, and its messages variant supports full tool trajectories rather than a single call. A worked example shows the fine-tuning shape with an instruction, an empty input and an output, which is the Alpaca convention where the input is optional. Full format documentation is a separate document, so the table here is the summary rather than the specification.

Configuration priority runs from the command line down to defaults

There are two ways to configure a run and one rule that decides between them. The rule is explicit: command-line arguments take priority over a YAML configuration file, which takes priority over the defaults in the training configuration class. The YAML route is one command that launches the distributed launcher with a specific set of local ranks and passes a configuration file path under the supervised fine-tuning configurations directory:

deepspeed --include localhost:0,1 src/train.py \
    --yaml_config ./configs/sft/sft_full.yaml

The pure command line route uses the same launcher and the same rank list, and then spells out the arguments individually, starting with the training stage and the fine-tuning type, followed by the path to the model, the path to the dataset and the output directory. So the two methods are not two tools, they are two ways of expressing the same settings, and the arbitration between them is documented rather than left to argument parser order.

Memory optimisation is built in rather than left to the launcher

The universal-framework feature claims four things and the third is the one that matters on a single machine. The framework integrates deeply with the distributed training library, supporting the ZeRO optimisation and mixed precision training. The pipeline covers pretraining, continual pretraining, instruction fine-tuning and the preference stages. And the memory techniques are built in: gradient accumulation, gradient checkpointing and activation recomputation. That trio is what makes a 7B configuration reachable at all on hardware that is not a cluster, and having them in the framework rather than in a launcher configuration means a quick experiment and a long run share the same code path. The fourth claim is modularity, with a design intended to make new architectures and strategies easy to add, which is a statement about intent rather than something you can verify from the table.

The quick start goes through the distributed launcher, not a training script

Both quick-start methods invoke the distributed launcher directly against a training script in the source directory, passing an explicit list of local ranks. That is the detail that will surprise anyone arriving from a framework with its own trainer entry point, because there is no command of the form train with a config flag: you are being asked to launch the distributed runtime yourself and name the script. Installation follows the same manual pattern. Clone the repository, create a named environment with a specific interpreter version and activate it, then install the package in editable mode, with a separate command for the optional attention extra. The stated requirements are a Python version and Linux. The repository carries configuration and dataset directories as well as the source, documentation, scripts, tools and tests, and a frontend directory that matches the web framework and server in the dependency list.

Editorial conclusion

Use this framework if the point is to read and modify the training loop rather than to get a model trained this afternoon. That is the bet the project makes: no high-level trainer library, its own data loading, distributed setup, sampling and cache implementation, and three stated principles of transparency, flexibility and efficiency that follow from that choice. Three things to check before committing. Every dependency is pinned to an exact version, which makes the install reproducible and makes upgrading your problem, and the framework is classified as Alpha with a single 0.1.0 release. Only two model series are marked as verified, and both only through pretraining, supervised fine-tuning and preference optimisation, so anything outside that needs your own compatibility work. And the requirements name Linux and Python 3.10 or newer, with the classifier listing 3.10 and 3.11.

Frequently asked questions

Official sources

1. Official README
2. Project repository
3. Release notes