# Polar turns an existing agent harness into a reinforcement learning environment > Polar, in the NVIDIA-NeMo ProRL Agent Server repository, is a rollout server for reinforcement learning over real agent harnesses: it dispatches tasks to distributed gateway nodes, runs the agent against an inference server, builds trajectories and evaluates them, and returns the result to any training framework that speaks HTTP. The point of the design is that you keep your harness and change the surrounding infrastructure, not the other way round. **NVIDIA-NeMo/ProRL-Agent-Server** — Agentic RL on Any Harness at Scale - Repository: https://github.com/NVIDIA-NeMo/ProRL-Agent-Server - Stars: 862 · Forks: 95 - Language: Python - License: Apache-2.0 - Published: 2026-10-20 · Updated: 2026-10-20 · Language: en - Canonical page: https://hysenlabs.com/projects/nvidia-nemo-prorl-agent-server ## Harness as environment, without a code change The README states the development goal for Polar directly, and it is a constraint rather than a feature list. The goal is low-intrusion and neutral, finding the lowest common ancestor that can cover and support diverse training and inference frameworks. The problem it addresses is specific to reinforcement learning for agents. An agent harness is normally written to be used: it has an interface, some tools, a loop, and a way to talk to a model. In reinforcement learning that same harness has to be executed thousands of times per step, in parallel, across many environments, with every rollout recorded as a trajectory and then scored. Rewriting a harness into an RL environment is a large and usually wasted effort, because the harness is usually already the best version of itself. So Polar's first listed capability is to bring agent harnesses as RL-ready environments without a code change, and the usage guide points at a generic `shell` harness or a preset shortcut for expressing an agent. The second capability is a smart rollout pipeline with parallel rollout staging and runtime pooling, which the README frames as a way to save GPU hours. The third is rollout as a service, with server mode by design, for scaling asynchronous RL with any training framework. The three are related. Pooling is what makes many concurrent rollouts affordable, the service boundary is what makes the trainer independent of the harness, and the harness independence is what makes an existing harness usable for training in the first place. The project lives in the NVIDIA-NeMo organisation, the package on the repository is `polar` at version 0.1.0, and the licence is Apache-2.0. ## The rollout server, the gateway nodes and the proxy in between The architecture paragraph in the README is dense, and it is the whole design in a few sentences. The Rollout Server manages and dispatches client requests into distributed Gateway Nodes. Those nodes asynchronously prepare the runtime, execute the agents, build the trajectories and evaluate them. And the agent harnesses are listened to by a proxy that sits between the agnostic agent execution processes and the inference servers. Three design points follow from that description. First, the split between the server and the gateway nodes is what makes the system scale and what makes it deployable. The server is the scheduler and the front door, and the gateway nodes are the workers that actually run agents, prepare runtimes and hold the trajectories. The rollout is asynchronous, so a submission returns an identifier and the result arrives when it is ready, which is what lets a training loop keep the GPUs busy rather than waiting on each step. Second, the trajectory and evaluation stages live on the gateway, not in the trainer and not in the harness. The usage guide points at separate builder and evaluator guides for registered strategies, and the roadmap lists more built-in evaluators, such as self distillation with textual feedback, as still to come. That placement is what keeps the harness untouched: the harness produces actions, and the surrounding machinery records and scores them. Third, the proxy is the piece that lets the same harness talk to different inference servers. It sits between the agent execution processes and the inference servers, and it is agnostic on the agent side, which is what allows vLLM and SGLang to be used interchangeably, and why the roadmap lists vLLM dual inference support as done. The topology itself is described in a configuration guide, and every CLI command takes the same file, so the deployment shape is declared once rather than negotiated per process. ## Three installs: the server, an inference server and a trainer The installation is split into three pieces, and the README is explicit that the inference server choice is a decision, not a detail. The rollout server is installed from a checkout with uv. ```bash uv venv --python 3.12 uv pip install -e . source .venv/bin/activate ``` The inference server is either vLLM or SGLang, and the README says to pick one that your trainer supports, and to avoid installing both under the same environment given dependency conflicts. ```bash uv pip install vllm --torch-backend=auto ``` ```bash uv pip install "sglang==0.5.13" ``` The third piece is the training framework, and here the wording is careful: Polar is trainer agnostic, and the choice of trainer and training backend is highly flexible given the HTTP server boundaries. What exists today is a demo-purpose Slime integration, with the instructions in the Slime bridge installation guide inside the source tree. The roadmap lists more trainer bridges, naming NemoRL and VERL, as unfinished. The dashboard is optional and needs a one-off frontend build. ```bash cd web && npm install && npm run build ``` The dependency list in the package metadata is short and worth noting, because it explains the design. It is fastapi and uvicorn for the HTTP surface, httpx for the client side, pydantic for the data model and pyyaml for the topology file, with swebench and datasets as an optional extra. There is no deep learning framework in the runtime dependencies, because the model serving happens in a separate process, and the trainer's own stack is the trainer's business. The package also installs a `polar` console script pointing at `polar.cli:main`, and the distribution includes both the `polar` package and the `slime_bridge` package. ## Five commands and one topology file The CLI section is the part of the README you can use directly, and it is short enough to take whole. ```bash polar serve_rollout -c topology.yaml # central orchestrator (port 8080) polar serve_gateway -c topology.yaml --node-id # one per gateway node (port 8100+) polar dashboard -c topology.yaml [--port 8090] # observability & monitoring dashboard polar submit -c topology.yaml # submit a task and tail it polar status -c topology.yaml # one-shot health / topology check ``` A typical local run uses all five, and every one of them reads the same `topology.yaml`. That is the important detail. The topology file is the single declaration of the deployment, so the orchestrator, each gateway node and the dashboard all agree on the shape of the system because they are reading the same file rather than being told about each other. The commands divide into roles. `serve_rollout` is the central orchestrator on port 8080, and `serve_gateway` is started once per gateway node with a node identifier, on port 8100 and up. The dashboard is a separate observability surface on port 8090 by default. `submit` takes a task in JSON or YAML and tails it, so you watch a rollout rather than polling for a result. And `status` is a one-shot health and topology check, which is the command to run when something is not working and you need to know whether the pieces are talking to each other. The gateway nodes being separately addressable is what allows a deployment to spread across machines, and the node identifier is what lets the orchestrator place work on a specific one. ## The examples move from smoke test to training loop The quick start is a set of examples, and the README annotates them in the order you should try them. The calculator example is a minimal smoke test, and it says start here. The count stars example is a minimal test for a vision language model. The SWE-bench Verified example is a benchmark-style evaluation on SWE-bench Verified tasks, and the SWE-Gym Slime GRPO example is a training path that connects Polar rollouts to Slime. That ordering is a description of the maturity ladder in the project. The first example proves that a harness runs inside the server and returns a trajectory. The second proves the same path works when the model has to see something, which is the case where the infrastructure gets harder, because the rollout now carries images through the gateway and the evaluator. The third puts a known benchmark behind the system so you can compare against something. The fourth closes the loop with a trainer. The repository also carries an `examples/tmax-15k/` directory, and the optional dependency group in the package metadata includes `swebench` and `datasets`, which is the packaging side of the benchmark example. The tests are structured to match this layering. The pytest configuration defines markers for unit checks that are fast and CPU only, for integration checks that need local services or external programs, and for checks that need GPU backed model serving, and the exclusion list keeps the examples, the third party training checkouts and the output directories out of test collection. A contributor reading the tree will find the guidance for each of these areas under `src/polar/`, with separate READMEs for the agent harnesses, the trajectory builder and evaluator, the deployment topology and the rollout API, and the web frontend under `web/`. ## Neutral by design, with the gaps listed in public The roadmap is a checklist rather than a promise, and it is worth reading because it tells you where the project is honest about itself. Done are the initial release and the tech report, the Slime bridge and the RL example, support for CUA, meaning vision language and vision language action models, and vLLM dual inference support. Not done are more built-in evaluators, with self distillation with textual feedback given as the example, and more trainer bridges, with NemoRL and VERL named. That list is consistent with the stated goal. If the design goal is the lowest common ancestor across training and inference frameworks, then the first-class integrations are the thing you would expect to grow last and slowest, because the whole point is that you do not need them. The neutral framing has a cost, though, and it is worth naming it plainly. A framework that is agnostic about trainers and inference servers spends its complexity budget on the boundary between them, so the parts that are framework specific, the evaluators and the trainer bridges, are the parts that arrive last. If your workflow depends on a particular reward function or a particular trainer integration, that dependency is not covered today, and the two unchecked roadmap items are exactly the ones that would cover it. The project also asks to be cited, with two references given. The first is the Polar paper, Agentic RL on Any Harness at Scale, and the second is the ProRL Agent paper on rollout as a service for reinforcement learning of multi-turn LLM agents. Both are arXiv preprints from 2026, and the README frames citing them as the way to support the work. The repository is not archived, and the last push was on 2026-10-07, on a default branch named `stable` rather than `main`. ## Conclusion Polar fits a team that already has an agent harness and a training loop and wants to move the rollout side onto a pooled, served infrastructure without rewriting the harness. Skip it if you have no multi-turn agent to train, or if you are looking for a single-process RL library rather than a distributed service. Verify first which Python version the training environment needs, since the README and the package metadata disagree, confirm that your inference server choice is the one your trainer supports, and try the calculator example before building your own topology. ## FAQ ### What is Polar in the NVIDIA-NeMo ProRL Agent Server repository? Polar is a reinforcement learning rollout framework for real-world agent harnesses, published as the `polar` Python package at version 0.1.0. Its goal is to bring existing agent harnesses into reinforcement learning as ready environments without a code change, with pooled parallel rollouts and a server boundary that lets any training framework drive it. ### How do I install the Polar rollout server? From a checkout, the README gives `uv venv --python 3.12`, `uv pip install -e .` and `source .venv/bin/activate`. You also need one inference server, vLLM or SGLang, and a training framework, with a demo Slime integration provided. ### Which inference server should I use with Polar? The README says to pick one that your trainer supports, and to avoid installing both under the same environment given dependency conflicts. The documented commands are `uv pip install vllm --torch-backend=auto` for vLLM and `uv pip install "sglang==0.5.13"` for SGLang, and the roadmap lists vLLM dual inference support as complete. ### What commands run a local Polar deployment? Five commands, all taking the same `topology.yaml`: `polar serve_rollout` for the central orchestrator on port 8080, `polar serve_gateway` once per gateway node with a node id on port 8100 and up, `polar dashboard` on port 8090 by default, `polar submit` to submit a task and tail it, and `polar status` for a one-shot health and topology check. ### Where should I start when trying Polar? The README marks the calculator example as a minimal smoke test and says to start there, followed by the count stars example for a vision language model, then SWE-bench Verified for a benchmark-style evaluation, and the SWE-Gym Slime GRPO example for a training path connected to Slime. ## Sources - [Issues](https://github.com/NVIDIA-NeMo/ProRL-Agent-Server/issues) - [License: Apache-2.0](https://github.com/NVIDIA-NeMo/ProRL-Agent-Server/blob/stable/LICENSE) - [NVIDIA-NeMo/ProRL-Agent-Server on GitHub](https://github.com/NVIDIA-NeMo/ProRL-Agent-Server) - [README](https://github.com/NVIDIA-NeMo/ProRL-Agent-Server/blob/stable/README.md) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/nvidia-nemo-prorl-agent-server