Vime: slime's RL training loop with vLLM as the rollout backend
An LLM post-training framework with vLLM for RL Scaling
At a glance
- What is it?
- Vime is a post-training framework for reinforcement learning on LLMs that keeps slime's Megatron training stack and data-generation design while swapping in vLLM and vllm-router for generation. It is an integration project, not a new algorithm, and its value depends on whether you already want slime's paradigm inside the vLLM release cycle.
- Who is it for?
- Adopt Vime if your team already runs Megatron-based RL training and wants vLLM plus vllm-router as the rollout backend under a single orchestration layer, or if you need custom generate functions for multi-turn and sandboxed agentic rollouts.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap Vime fills between slime and the vLLM ecosystem
slime established a training paradigm: a Megatron training process, a rollout process that generates samples, and a data buffer that sits between them. Vime takes that structure and replaces the default rollout backend with vLLM and vllm-router. The README frames this as bringing slime's training paradigm into the vLLM ecosystem and aligning both projects' release cycles, which is a maintenance argument as much as a technical one.
The intended user is a team that already accepts Megatron as the trainer. Vime does not remove that dependency. It reads Megatron arguments directly, so configuration like --tensor-model-parallel-size 2 passes straight through. If your organization has no Megatron deployment and no intention of building one, the framework's centre of gravity is somewhere you are not standing. The README lists Qwen3.6, Qwen3.5, Qwen3Next, Qwen3MoE, Qwen3, Qwen2.5, DeepSeek V3, V3.1, R1 and Llama 3 as inherited model support, so the practical question is whether your checkpoint is in that set.
Three processes, one data buffer: the architecture as documented
The architecture diagram in the README splits the system into three modules. The training module runs Megatron, reads data from the data buffer, and synchronizes parameters to the rollout module after each training step. The rollout module launches vLLM inference engines and routes generation requests through vllm-router. The data buffer handles prompt initialization, custom data, and the generation methods themselves.
That third module is where the framework's flexibility lives. The README describes it as managing rollout generation methods, including agentic workflows that produce samples through the same interface. Agentic RL is therefore not a separate subsystem with its own control flow. Multi-turn loops, tool calls, environment interaction and verifier-based rewards are wrapped by custom generate functions plugged into the ordinary rollout path. The default entry point is vime.rollout.vllm_rollout.generate_rollout, and --custom-generate-function-path is the switch that replaces it.
The code reading path in the README is unusually explicit, which is a good sign for a project at v0.3.2. It starts at train.py: train, fans out to vime/ray/placement_group.py for Ray resource and worker initialization, then to vime/ray/rollout.py for RolloutManager.generate, down into vime/rollout/vllm_rollout.py for sample generation and reward computation, and separately to vime/ray/actor_group.py for RayTrainGroup.async_train, which dispatches to vime/backends/megatron_utils/actor.py, model.py and loss.py. Ray is the orchestration substrate throughout. The README advises treating vime/utils/arguments.py as the configuration entry point on a first pass and deferring vime/backends/vllm_utils/ and the weight-sync code under vime/backends/megatron_utils/update_weight/ until you need to modify them.
Argument prefixes tell you where each knob actually belongs
Vime divides configuration into three categories, and the prefixes are the only reliable way to tell which component will consume a flag. Megatron arguments are read verbatim, so --tensor-model-parallel-size 2 goes to the trainer. vLLM server and engine options carry a --vllm- prefix, for example --vllm-gpu-memory-utilization. Then the router splits into two prefixes that are easy to confuse: vllm-router's own options use --router-, such as --router-policy round_robin and --router-request-timeout-secs, while Vime-side orchestration knobs that tell Vime where the router lives use --vllm-router-, such as --vllm-router-ip and --vllm-router-port. Misplacing a flag between those two prefixes is a plausible first-run failure, and the README does not describe an error message for it.
--rollout-num-gpus-per-engine sets the tensor parallel size of each vLLM engine, which is the knob that decides how a fixed GPU pool is divided between training and generation. The README does not state a default for it or give guidance on choosing a value, so this is one to determine empirically against your own hardware. The full vLLM-side argument surface is listed in vime/backends/vllm_utils/arguments.py, and shared Vime orchestration flags covering rollout GPUs, data paths and RL algorithms are in vime/utils/arguments.py. Both files are named as references rather than reproduced, which means the README alone is not sufficient to enumerate what you can configure.
Getting it running: the quick start is a separate document
The README does not contain installation or launch commands. It points to docs/en/get_started/quick_start.md for environment setup, data preparation, training startup and key code analysis, and to docs/en/get_started/usage.md for complete usage instructions. Examples for cases outside the quick start live under examples/. The only shell commands printed in the README itself are the pre-commit setup for contributors:
apt install pre-commit -y pre-commit install pre-commit run --all-files --
That is a developer-workflow detail, not a deployment instruction. If you are evaluating Vime for adoption, the honest first step is to read the quick start document, because nothing in the repository README tells you which Python version, CUDA build, Megatron fork or vLLM version the current release expects. The release history offers one data point: v0.3.2 was tagged on 2026-09-07 and the last push to main was 2026-09-10, three days later. That cadence suggests main moves quickly relative to tagged releases, so pinning to a tag rather than tracking main is the safer default when you are trying to reproduce a training run.
Agentic rollouts and the sandbox contract you have to implement
Three examples cover agentic workloads, and the README is explicit that they use the standard rollout and data buffer loop rather than a separate framework. examples/multi_agent handles multi-agent generation through --custom-generate-function-path. examples/fully_async covers fully asynchronous rollout for long-tail agent generation, which addresses the case where some trajectories finish far later than others and a synchronous loop would idle. examples/coding_agent_rl is an end-to-end coding-agent RL setup using Claude Code or Codex, with sandboxed tool use, test-based rewards, and token-correct trajectory segments.
The sandbox detail is the practical constraint. The coding-agent example ships an E2B-compatible backend, and the README states that the shared vime.agent.sandbox.Sandbox contract can be implemented for Docker, Modal, or local VMs. Those implementations are not provided. If your infrastructure is not E2B-compatible, you are writing an adapter against that contract before the example is useful to you. The token-correct trajectory segments claim matters for anyone doing agentic RL, because reward attribution in multi-turn tool use depends on mapping generated tokens back to the turns that produced them, but the README does not explain the mechanism behind that mapping.
What Vime is not, and where the documentation stops
The README's positioning section names the other vLLM-community post-training frameworks in alphabetical order: NeMo RL, OpenRLHF, prime-rl, SkyRL and verl. That list is a statement of coexistence rather than competition, and the README does not benchmark Vime against any of them. The concrete difference from those projects is the slime lineage. Vime keeps slime's training stack and data-generation design, which means a team migrating from slime faces a rollout-backend swap rather than a rewrite, while a team coming from a different framework inherits Megatron and the data buffer abstraction whether or not they wanted them.
The gaps worth naming: there is no stated default for --rollout-num-gpus-per-engine, no version compatibility matrix in the README, no description of failure modes when the router and the vLLM engines disagree about configuration, and no worked example of implementing vime.agent.sandbox.Sandbox for a non-E2B backend. The README also does not describe how weight synchronization between Megatron and the vLLM engines is scheduled, beyond noting that parameters are synchronized after training and that the implementations live under vime/backends/megatron_utils/update_weight/. For a framework whose central claim is high-performance training by connecting Megatron with vLLM, the synchronization path is the part most likely to determine your throughput, and it is the part the README defers.
Licence, upgrade cost and who should wait
Vime is Apache-2.0, the same licence family as vLLM and Megatron-LM. Apache-2.0 permits commercial use and modification and includes an express patent grant, but it also carries attribution and notice requirements for redistributed derivative works, and it does not grant trademark rights. The README does not discuss licence obligations for the dependencies you will pull in, and it does not mention any model weights licence, which is a separate question from the framework's own licence. That is a question for your legal team, not something this review can settle.
Upgrade cost is driven by the argument surface. Because Vime reads Megatron arguments directly, a Megatron version bump can change your configuration even when Vime's own release notes say nothing. The two files the README identifies as authoritative, vime/utils/arguments.py and vime/backends/vllm_utils/arguments.py, are the ones to diff between tags. The router's dual prefix scheme means an upstream vllm-router change to --router- options can require edits on your side that look like Vime changes but are not.
Wait if any of the following is true: your checkpoint is outside the listed Qwen, DeepSeek V3 or Llama 3 families; you have no Megatron deployment; your sandbox is not E2B-compatible and you are not prepared to implement the Sandbox contract; or you need a documented compatibility matrix before you can plan a rollout. The project is at v0.3.2 with a tag three days behind main, which is a normal cadence for an integration project tracking two fast-moving upstreams, and it means the documentation will lag the code in exactly the areas where the two upstreams diverge.
Editorial conclusion
Adopt Vime if your team already runs Megatron-based RL training and wants vLLM plus vllm-router as the rollout backend under a single orchestration layer, or if you need custom generate functions for multi-turn and sandboxed agentic rollouts. Do not adopt it if you want a self-contained trainer with no Megatron dependency, or if you expect a stable argument surface: the README points at vime/utils/arguments.py and vime/backends/vllm_utils/arguments.py as the authoritative lists, and those are the files to diff on every upgrade. Before committing, verify that your model family appears in the supported list and that your sandbox backend satisfies the vime.agent.sandbox.Sandbox contract, since the shipped coding-agent example only provides an E2B-compatible backend.
Community notes