# vLLM Vime review: slime's RL training loop with vLLM as the rollout backend > Vime is an LLM post-training framework for RL scaling that keeps slime's training and data-generation design and swaps in vLLM plus vllm-router for rollout. It is for teams already running Megatron and vLLM who want agentic data generation without a second framework. **vllm-project/vime** — An LLM post-training framework with vLLM for RL Scaling - Repository: https://github.com/vllm-project/vime - Stars: 478 · Forks: 98 - Language: Python - License: Apache-2.0 - Published: 2026-09-10 · Updated: 2026-09-10 · Language: en - Canonical page: https://hysenlabs.com/projects/vllm-project-vime ## What Vime actually adds on top of slime Vime is a fork-shaped project: it keeps slime's training stack and data-generation design and replaces the rollout side with vLLM and vllm-router. The README states this directly, and the positioning section frames the project as a bridge that brings slime's training paradigm into the vLLM ecosystem. That is a narrower claim than "new RL framework", and it is the right way to read the repository. The practical consequence is that Vime's value is concentrated in two places. First, the connection between Megatron training and vLLM generation, including how weights move between them. Second, the data generation interfaces, which let you write custom generate functions that wrap generation in multi-turn loops, tool calls, sandbox interaction and verifier-based rewards. Everything else, including the model support list (Qwen3.6 down to Qwen2.5, the DeepSeek V3 series, Llama 3), is inherited from slime. Who this is for: engineers who already run Megatron for training and vLLM for serving, and who now need rollout data that a plain generate call cannot produce. If your RL loop is single-turn prompt-in, completion-out, Vime's customization layer is surface area you will pay for and not use. ## Training, rollout and the data buffer in between The architecture diagram in the README names three modules. The training module runs Megatron, reads data from the Data Buffer, and synchronizes parameters to the rollout module after training. The rollout module launches vLLM inference engines and routes generation requests through the router. The Data Buffer is the bridge: it manages prompt initialization, custom data, and rollout generation methods. The README's code reading path makes the control flow concrete. `train.py` calls `train`, which initializes Ray resources and workers through `vime/ray/placement_group.py`. Rollout orchestration lives in `vime/ray/rollout.py` (`RolloutManager.generate`), which reaches `vime/rollout/vllm_rollout.py` for sample generation and reward computation. Training dispatch goes through `vime/ray/actor_group.py` (`RayTrainGroup.async_train`) into `vime/backends/megatron_utils/actor.py`, then `model.py` for Megatron execution and `loss.py` for RL losses and advantages. Two things stand out. Ray is not optional plumbing here; it is the scheduling substrate, and the placement group file is the second thing the reading path tells you to open. And the default rollout entry is named explicitly as `vime.rollout.vllm_rollout.generate_rollout`, so replacing rollout behavior means replacing or wrapping that function rather than configuring a plugin. The README also notes that agentic workloads are not a separate framework: multi-agent and coding-agent examples run through the same rollout and Data Buffer loop via `--custom-generate-function-path`. ## Installing Vime and running the pre-commit setup The README does not give a pip install line for Vime itself. It points to the Quick Start Guide at `docs/en/get_started/quick_start.md` for environment setup, data preparation and training startup, and the repository ships a `docker/` directory and a `scripts/` directory alongside `requirements.txt`. Treat the quick start document as the install path; the README is a map, not a manual. The one command sequence the README does spell out is the developer setup for code style, which is what you run if you intend to send a patch: ```bash apt install pre-commit -y pre-commit install # run pre-commit to ensure code style consistency pre-commit run --all-files -- ``` After `pre-commit install`, the hook runs on each commit; `pre-commit run --all-files --` forces a pass over the whole tree. Note the trailing `--` in the README's example, which is what gets copied into the repository and into this article. The package metadata is worth checking before you build anything. `setup.py` declares the name `vime`, version `0.3.2`, and `python_requires=">=3.10"`, with classifiers for Python 3.10 through 3.12 and an NVIDIA CUDA environment. The custom wheel class sets `root_is_pure = False` and returns a platform tag of `manylinux1_x86_64` on Linux, so the distributed artifact is platform-specific rather than a pure-Python wheel. `requirements.txt` pulls in `ray[default]`, `vllm-router>=0.1.15`, `transformers`, `wandb`, `e2b`, `mcp[cli]` and `openai-agents`, among others, which tells you the intended deployment is a GPU cluster with Ray, not a laptop. ## Where the configuration surface will bite you Vime splits arguments into three categories, and the split is the most useful thing in the README for anyone trying to reason about a run. Megatron arguments are read in full, so flags like `--tensor-model-parallel-size 2` pass straight through. vLLM server and engine options get a `--vllm-` prefix, for example `--vllm-gpu-memory-utilization`. Framework-specific orchestration flags live in `vime/utils/arguments.py`. The router is where the naming gets confusing. Router options carry two different prefixes with two different meanings. Native vllm-router options use `--router-`, as in `--router-policy round_robin` or `--router-request-timeout-secs`. Vime-side knobs that tell Vime where the router lives use `--vllm-router-`, as in `--vllm-router-ip` and `--vllm-router-port`. The README points to `vime/backends/vllm_utils/arguments.py` for the full surface. If you have used vLLM directly, expect to relearn flag names rather than reuse them. One flag is worth internalizing early: `--rollout-num-gpus-per-engine` sets the tensor parallel size of each vLLM engine. That single number determines how a rollout engine is sharded, and it has to be reconciled with whatever tensor parallelism you chose on the Megatron side. The README does not document how to choose it, only what it does. ## The trade-offs the README does not resolve The clearest limitation is the dependency floor. Vime inherits slime's model support and Megatron's configuration model, then adds Ray for scheduling and vLLM plus vllm-router for generation. That is a large stack to keep in step, and the README's own positioning note says the project exists partly to align both projects' release cycles, which is an admission that version drift between Megatron, vLLM and the router is a real operational concern rather than a hypothetical one. Second, weight synchronization is the risky seam. The README defers the deployment details under `vime/backends/vllm_utils/` and the weight-sync implementations under `vime/backends/megatron_utils/update_weight/` to a later reading pass. That is honest advice for a first read, but it also means the part of the system most likely to fail silently (training weights not matching what the rollout engine is serving) is the part with the least coverage in the top-level document. `requirements.txt` includes `xxhash` with the comment "disk delta weight sync checksum", which suggests checksumming exists for the disk delta path; the README does not describe a rollback or recovery procedure if a sync step fails mid-run. Third, the sandbox layer is an interface, not a product. The README states that the coding-agent example ships an E2B-compatible backend and that the shared `vime.agent.sandbox.Sandbox` contract can be implemented for Docker, Modal, or local VMs. That is a contract you implement, not a feature you enable. If you expected a managed sandbox, you will be writing it. Finally, Vime is the wrong tool if you are not already invested in Megatron. The training path runs through `vime/backends/megatron_utils/`, and nothing in the README suggests a way to substitute a different trainer. ## How Vime differs from verl and the other vLLM-ecosystem options The README names its neighbors directly: NeMo RL, OpenRLHF, prime-rl, SkyRL and verl, described as frameworks the vLLM community supports horizontally. The stated reason for building Vime anyway is that slime's training paradigm was worth carrying into the vLLM ecosystem. The real difference is lineage. Vime's training stack and data-generation design come from slime, and the customization interface is the Data Buffer plus custom generate functions reached through `--custom-generate-function-path`. verl is a separate codebase with its own trainer and its own rollout abstraction; choosing between them is closer to choosing a training stack than choosing a feature set. If your team already knows slime's interfaces, Vime removes a rewrite. If your team knows verl, Vime asks you to learn a different set of extension points for the same class of problem. A second difference is where the examples point. Vime ships `examples/coding_agent_rl/`, which the README describes as end-to-end coding-agent RL with Claude Code or Codex, sandboxed tool use, test-based rewards, and token-correct trajectory segments. It also ships `examples/fully_async/` for long-tail agent generation and `examples/multi_agent/`. These are agentic workloads expressed in the same rollout loop, not a bolted-on agent runtime. If your RL work is agent-shaped, that is the concrete thing to compare against another framework's agent support. ## Conclusion Adopt Vime if your stack is already Megatron for training and vLLM for inference, and you need custom generate functions for multi-turn or tool-using rollouts; the shared Sandbox contract and the coding-agent example are the parts worth reading first. Do not adopt it if you want a pip-installable library, a documented rollback path, or a framework that hides Ray and Megatron behind an API. Before committing, verify the weight-sync path under vime/backends/megatron_utils/update_weight/ against your parallelism layout, and read docs/en/get_started/quick_start.md to confirm the environment it assumes matches the one you have. ## FAQ ### What is vLLM Vime and what is it built on? Vime is an LLM post-training framework for RL scaling. The README states it is built on slime, keeps slime's training stack and data-generation design, and uses vLLM with vllm-router as the default rollout backend. ### How do I install Vime? The README does not give a pip install command. It directs readers to the Quick Start Guide at docs/en/get_started/quick_start.md for environment setup, data preparation and training startup, and the repository includes a docker directory and a requirements.txt. ### Which models does Vime support? The README says Vime inherits broad model support from slime, listing the Qwen series (Qwen3.6, Qwen3.5, Qwen3Next, Qwen3MoE, Qwen3, Qwen2.5), the DeepSeek V3 series (DeepSeek V3, V3.1, DeepSeek R1) and Llama 3. ## Sources - [Issues](https://github.com/vllm-project/vime/issues) - [License: Apache-2.0](https://github.com/vllm-project/vime/blob/main/LICENSE) - [README](https://github.com/vllm-project/vime/blob/main/README.md) - [Releases](https://github.com/vllm-project/vime/releases) - [vllm-project/vime on GitHub](https://github.com/vllm-project/vime) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/vllm-project-vime