OpenManus-RL's root packaging belongs to the verl submodule, and three files disagree on vLLM
A live stream development of RL tunning for LLM agents
At a glance
- What is it?
- OpenManus-RL is a research programme in public: an agent tuning effort run by Ulab-UIUC with MetaGPT, documenting its exploration of rollout strategies, reasoning formats and reward design for reinforcement learning on LLM agents. The packaging at the root is not its own, since the root manifest and setup script both describe the verl submodule from a different project, and the dependency files pin three different versions of vLLM and two of transformers.
- Who is it for?
- OpenManus-RL is worth reading as a record of how an agent tuning programme is being run in public, since the method section names its rollout strategies, reasoning formats and reward types rather than hiding them behind a result table, and the roadmap is honest about what has not been built yet. It is not a library to install today.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Activity is slowing. The repository last received commits 6 months ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.
Star history
- OpenManus/OpenManus-RL4,178 ★ · +22
| Repository | 2026-09-13 | 2026-11-15 |
|---|---|---|
| OpenManus/OpenManus-RL | 4.2k | 4.2k |
22 Stars in the last 63 days, measured from the daily GitHub snapshot. Compare with other projects
Editorial analysis
The root pyproject.toml is verl's, not OpenManus-RL's
The most confusing thing about this repository is that its own packaging describes a different project. The manifest at the root declares the package name as `verl`, describes it as reinforcement learning for LLMs from the Volcano Engine, and lists the authors as a ByteDance Seed team with two contact addresses. Its homepage points at the `volcengine/verl` repository, not at OpenManus. The version is dynamic, read from a file inside a `verl/version/version` path.
That is because the readme says the `verl` submodule has been integrated for enhanced RL training capabilities, and the tree carries both a `.gitmodules` file and a `verl` directory. The manifest therefore describes the submodule, while the project's own code sits in an `openmanus_rl/` directory that no packaging file in the visible tree refers to.
The practical consequence is that there is nothing here to `pip install` as OpenManus-RL. What you can read is a research programme, a dataset link, and the upstream training framework it borrows.
Three vLLM constraints and two transformers pins across three files
The dependency files do not agree, and the differences are the kind that decide whether an environment resolves at all. The root manifest asks for `transformers<4.48` and `vllm<=0.6.3`. The `setup.py` at the root asks for `transformers<=4.51.1`, keeps the inference extra separate at `vllm<=0.8.5`, and adds a SGLang extra pinned to a single post release together with `torch==2.6.0` and a torch memory saver. The `requirements.txt` pins `transformers==4.51.1` exactly and leaves the vLLM line commented out entirely.
So the manifest and the setup script contradict each other on transformers, and the two of them plus the requirements file give three different answers about vLLM. The manifest also predates the setup script in shape, since it omits several packages the setup script requires, including pandas, peft, pyarrow, wandb and the vision utilities.
All three files belong to the submodule rather than to the project, which is the likeliest explanation. Anyone building an environment here has to pick one stack deliberately instead of trusting the files to agree.
setup.py describes itself as the fallback for when the manifest fails
The setup script at the root opens with a comment saying it is the fallback installation script for when `pyproject.toml` does not work, then reads its version from a file, and then defines the dependency lists as module-level variables. That structure is more informative than the names suggest.
The lists are grouped by purpose rather than merged: a test group with pytest, pre-commit and py-spy; a prime group with a single package; a geography group with a single package; a GPU group with a Triton-fused kernel library and flash attention; a maths group with a verifier; a vLLM group; and an SGLang group that pins an exact engine release, a torch memory saver and torch itself. The setup script reads the version from a plain text file before doing anything else, which is the classic reason a fallback installer exists when a manifest declares its version dynamic.
There is also a separate `requirements_docker.txt` in the tree, so the container path has its own dependency list again, and the documentation directory and a scripts directory sit beside them.
The news log stops in March 2025 while pushes run to May 2026
The visible timeline and the commit record do not line up. The news section has three entries and all three are from March 2025: the project was announced on 2025-03-06 as a live-streaming effort, collaboration with the OpenManus team at MetaGPT was announced on 2025-03-08, and the agent supervised fine-tuning dataset was published on Hugging Face on 2025-03-09. The readme separately states that code and dataset are now available and that the verl submodule has been integrated.
Against that, the branch was last pushed on 2026-05-05, and the single published release carries the literal tag `version` with the name OpenManus 0.0.2, dated 2025-05-09.
So the narrative the readme tells stopped fourteen months before the last commit, and the release tag is a placeholder word rather than a version scheme. For a project whose selling point is continuous public updates, that gap is the first thing a reader should notice. The repository description itself also spells tuning with an extra n, a small thing that suggests the text has not been revisited much.
Four rollout strategies and two reasoning formats, borrowed from RAGEN
The method section is a list of what will be tried rather than a description of what works, and it credits its starting point. The approach draws on the Reasoning-Interaction Chain Optimization work from RAGEN, and from there explores algorithmic structures, reasoning paradigms, reward strategies and benchmark environments.
Three reasoning models are named as the things being benchmarked to begin with: GPT-O1, Deepseek-R1 and QwQ-32B, with the claim that each brings different reasoning capability that informs later optimisation.
The rollout strategies are four familiar search shapes applied to agent planning: tree-of-thoughts for branching paths, graph-of-thoughts for dependency structures, depth-first search decision trees for long-horizon action selection, and Monte Carlo tree search for the exploration and exploitation balance. The reasoning formats compared are narrower, only two: ReAct, which interleaves reasoning and action explicitly, and outcome-based reasoning, which optimises toward explicit outcome predictions. Two formats is a small axis to draw a conclusion from, and the readme does not say which wins.
Rewards split into format and outcome, and the list stops mid-word
The post-training section names two strategies. Supervised fine-tuning is the first, described as initialising reasoning capability from human-annotated instructions. The second is generalized reward-based policy optimization, and the readme begins to break its reward types down into format-based rewards, which reward adherence to a specified reasoning structure, and then stops partway through the second bullet, at the word `Outcom`.
That truncation is not cosmetic, because it is exactly where the reward design lives, and the rest of the document depends on it. The table of contents promises sections that the visible text never reaches: a section on training an agent reward model, one on test-time scaling of trajectories, one on action space awareness and strategic exploration, and one on integration with RL tuning frameworks. The same applies to the dataset sections, which promise an overview and data instances, and to a running section.
The repository description and the roadmap also use the phrase online reinforcement learning tuning for setting up the agent environment, which implies an environment loop the visible text does not describe.
Four provider keys and a Google custom search key in the example environment
The example environment file is short and tells you what the agent is expected to reach. It asks for an OpenAI base URL and key, a Together key, a Google API key, and a Google custom search key. That last pair is the interesting one: a programmable search engine credential alongside three model providers suggests the agent's tool set includes web search through Google's programmable search rather than only model calls.
There is no database URL, no object store and no queue in the file, so whatever state the training run keeps is not configured through these variables. The tree has a `data/` directory, an `examples/` directory holding only a placeholder file and a data preprocessing directory, a `scripts/` directory, a `docs/` directory and a `test/` directory, so the layout expects data preparation to happen through those directories rather than through environment configuration.
For a project whose stated contribution path includes environment setup and computing resources, the gap between this five-line file and the training stack it is meant to drive is the thing to ask about.
Editorial conclusion
OpenManus-RL is worth reading as a record of how an agent tuning programme is being run in public, since the method section names its rollout strategies, reasoning formats and reward types rather than hiding them behind a result table, and the roadmap is honest about what has not been built yet. It is not a library to install today. The packaging at the root belongs to the verl submodule, so there is no installable package for this project in the visible tree, the three dependency files disagree about which inference stack to use, and the one published release carries the tag `version` at 0.0.2 from May 2025. Before trying to reproduce anything, read the news log and the roadmap rather than the method, since both are dated March 2025 while the branch itself was pushed as late as May 2026.
Frequently asked questions
How does OpenManus work?
The repository does not describe how the original OpenManus works; it describes OpenManus-RL, an extended version of that initiative led by Ulab-UIUC with MetaGPT. What it sets out is an agent tuning programme: reasoning models GPT-O1, Deepseek-R1 and QwQ-32B as a starting point, four rollout strategies, two reasoning formats, and post-training with supervised fine-tuning plus generalized reward-based policy optimization using format-based and outcome-based rewards.
Is Manus AI open-source?
The repository says nothing about Manus as a product. It describes itself as an open-source initiative, collaboratively led by Ulab-UIUC and MetaGPT, licensed under Apache-2.0, with an agent supervised fine-tuning dataset published on Hugging Face and a code release described as available.
What is the verl submodule used for in OpenManus-RL?
The readme says the verl submodule has been integrated for enhanced reinforcement learning training capabilities. The tree carries a `.gitmodules` file and a `verl` directory, and the root `pyproject.toml` and `setup.py` are verl's own packaging, naming the package verl, crediting a ByteDance Seed team and pointing its homepage at the separate verl repository.
Which agent benchmarks does OpenManus-RL evaluate on?
GAIA, AgentBench, WebShop and OSWorld. They are named in the introduction as the agent benchmarks the project intends to test on rigorously, and Webshop, GAIA, OSWorld and AgentBench appear again as the fourth item of the roadmap, which is to evaluate the framework on agentic benchmarks.
How can I contribute to OpenManus-RL?
The listed routes are contributing to the fine-tuning codebase, the tuning dataset, environment setup, or computing resources, plus opening issues for feature requests and bug reports, submitting pull requests, or reaching out directly for collaboration. The readme also states that important contributors will be listed as co-authors on the paper.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/openmanus-openmanus-rl)Star history chart for your README
[](https://hysenlabs.com/projects/openmanus-openmanus-rl)Was this useful?