NeMo Gym: A Server-Based Runtime for Stateful Agent Evaluation and RL Rollouts
Evaluate and improve models and agents using environments
At a glance
- What is it?
- NeMo Gym is NVIDIA's Apache-2.0 Python library for running agents against environments that keep state, score task completion, and feed reinforcement learning. It is worth adopting when a stateless scoring script has stopped being enough, and it is explicitly unfinished.
- Who is it for?
- Adopt NeMo Gym if you are running stateful agent tasks at scale, need one verifier shared across teams, or want evaluation rollouts to feed SFT and RL training through NeMo RL, Unsloth, or VeRL. Do not adopt it if a stateless check answers your question; the README says a script is probably sufficient in that case, and the project itself warns of evolving APIs, incomplete documentation, and occasional bugs.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository received new commits within the last day.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem NeMo Gym solves: state that a scoring script cannot hold
Most model evaluation reduces to a function: take an output, compare it to a reference, return a number. That works until the task involves code execution, tool calls, or a sandbox that changes between turns. At that point the score depends on what happened earlier in the episode, and a script that only sees the final string cannot reproduce it. NeMo Gym's answer is the environment, which the README defines as four parts: a dataset of tasks, an agent harness describing how the model interacts with the world, a verifier that scores task completion, and per-task execution context. The audience is teams doing agent evaluation who also intend to train. The README is direct about the boundary: if you are scoring model outputs with a stateless check and do not need scale or training, a script is probably sufficient. That sentence is the most useful line in the repository, because it tells you when to close the tab.
Servers, harnesses and verifiers: the architecture the README exposes
The mechanism is a set of cooperating servers. The repository layout separates resources_servers, which hold environments and benchmarks, from responses_api_agents, which hold agent harnesses. A run therefore pairs a harness with a resource server, and the verifier scores the resulting trajectory. This split is what allows the same environment to be driven by OpenHands, Mini SWE Agent, or LangGraph, all of which the README lists as out-of-the-box harnesses. It also explains the interoperability claims: Aviary, Harbor, OpenEnv, Reasoning Gym, and Verifiers appear as resources_servers or responses_api_agents entries, so environments from other libraries can be combined with NeMo Gym's own. The v0.5.0 release notes describe a standardized ng_trajectory schema and end-to-end rollout observability covering model-call capture and agent observations. The v0.6.0 notes add task-level harness routing, meaning a single run can evaluate multiple agents and datasets, and Switchyard for comparing fixed and routed model strategies on one benchmark. If your evaluation pipeline currently concatenates logs from three tools, that schema is the part worth examining first.
Getting it running: the gym CLI and what the material actually specifies
The README publishes the package as nemo-gym on PyPI, so installation goes through pip. Beyond that, the material is thinner than the feature list suggests. The unified gym CLI arrived in v0.4.0, and one subcommand is named explicitly in the v0.5.0 notes: gym eval reverify, which recomputes rewards from stored rollouts without re-running inference. That is a real operational detail, because it means a verifier change does not force you to pay for inference again. The v0.6.0 notes mention scaling vLLM evaluation jobs across GPUs or Slurm nodes. Requirements are stated concretely: Python 3.13.14 or higher, Linux Ubuntu 20.04 or equivalent, macOS 11.0 for x86_64 and 12.0 for Apple Silicon, or Windows through WSL2. No GPU is required for the library itself, though individual resource servers or model inference may need one. Configuration keys are not listed in the supplied README, so I cannot give you environment variable names or YAML fields without inventing them. Treat the docs site at docs.nvidia.com/nemo/gym as the source for those.
Sandboxes and harnesses: breadth that comes with a versioning cost
The v0.5.0 release notes list seven sandbox providers: Docker, Daytona, ECS Fargate, Enroot, OpenShell, OpenSandbox, and Apptainer. The same release added Codex CLI, KiloCode, RemoteAgent, and anyswe_agent as harnesses, on top of Claude Code and Hermes from v0.3.0 and OpenCode, OpenClaw, and Pi from v0.4.0. Breadth here is not decoration. A sandbox provider determines where untrusted agent code actually executes, and the difference between Docker on a workstation and ECS Fargate is the difference between a laptop experiment and a job that runs without you watching it. The cost is that each provider is a separate integration surface. The notes single out large-scale OpenSandbox reliability as having been significantly improved in v0.5.0, which implies it was a problem before. If you pick a provider, check whether its reliability work has landed in the version you install rather than assuming parity across all seven.
Where NeMo Gym is the wrong tool, and what to use instead
The clearest failure mode is scope mismatch. A team that wants to know whether a model writes correct SQL should not stand up resource servers, harnesses, and verifiers to find out; the README's own guidance points them at a script. The second limitation is stated by the project: early development, evolving APIs, incomplete documentation, occasional bugs, and a request to open an issue before making changes. That last point matters for anyone planning to fork or patch. A third is the Python floor of 3.13.14, which is high enough that some managed notebook environments will not satisfy it. For a narrower alternative, consider lm-evaluation-harness, which takes a different approach: tasks are defined declaratively and the harness owns the whole loop, so you get reproducibility across a fixed task set without running separate servers. The trade is that it is built around static task definitions rather than a stateful environment an agent can act inside, and it does not carry rollouts into an RL trainer. NeMo Gym is the heavier option because it is solving a different problem, not because it is strictly better.
Training integration is the real reason to accept the moving APIs
Evaluation and training share the same rollout in NeMo Gym, and that is the design decision that justifies the complexity. The README lists NeMo RL, Unsloth, and VeRL as training frameworks that consume environments for SFT and RL. The v0.6.0 notes add a specific capability: using supported external agent harnesses during RL training while preserving exact token IDs across multi-step runs. Token ID fidelity across a multi-step agent run is a genuine constraint, since a harness that re-tokenizes or loses the mapping between what the model emitted and what the trainer sees will corrupt the gradient. If you already train with one of those three frameworks, the integration is the argument. If you only evaluate, you are paying for machinery you will not use, and the stateless-script advice applies with more force.
Maintenance, licensing and what to check before you commit
NeMo Gym is Apache-2.0, which permits commercial use and modification, and the licence text is the authority rather than this summary. The maintenance picture from the material is a steady cadence: v0.4.0 in July 2026, v0.5.0 in August, v0.5.1 in early September, and v0.6.0 on 2026-09-09, with the last push to main on 2026-09-10. Frequent minor releases are good for fixes and awkward for stability, and the README's own warning about evolving APIs tells you which way to read it. Pin your version. The upgrade cost is concentrated in the two places the release notes keep touching: sandbox providers and agent harnesses. Both are integration surfaces you do not control, so a bump from v0.5.1 to v0.6.0 can change how a harness emits tokens or how a sandbox reports failures. The concrete next step is to install nemo-gym, run a single environment through the gym CLI, and confirm the ng_trajectory output contains the token and tool-call diagnostics the v0.6.0 notes describe before you connect a trainer.
Editorial conclusion
Adopt NeMo Gym if you are running stateful agent tasks at scale, need one verifier shared across teams, or want evaluation rollouts to feed SFT and RL training through NeMo RL, Unsloth, or VeRL. Do not adopt it if a stateless check answers your question; the README says a script is probably sufficient in that case, and the project itself warns of evolving APIs, incomplete documentation, and occasional bugs. Before committing, verify that Python 3.13.14 or higher is available on your hosts, that your chosen sandbox provider is among the seven the release notes list, and that the harness you depend on appears under responses_api_agents. Then run one environment end to end and inspect the ng_trajectory output before wiring anything into a training loop.
Community notes