HY-Embodied VLM 1.0: a 3B active embodied model with an 86 GB floor

A vision-language model aimed at agents that act beyond mere perception

The stated problem is that building an agent for the physical world needs more than multimodal perception and understanding. It also needs agentic capabilities: reasoning about actions, adapting to a situation that changes, and interacting with the world. HY-Embodied VLM 1.0 is built for that specific combination, and the name on the release badge and in the Hugging Face repository is Hy-Embodied-VLM-1.0.

The release history shows a family rather than a single model. An earlier release on 2026-04-09 opened the Hy-Embodied-0.5 weights, described as HY-Embodied-0.5 MoT-2B, together with its inference code and documentation under the Hy-Embodied-0.5 directory in the repository. A second release on 2026-06-15 covered HY-VLA-0.5, with official code, UMI-trained weights and more than 2000 hours of high-fidelity UMI data, hosted in separate repositories. The 1.0 release, dated 2026-07-15 in the project update log, is the model with weights on Hugging Face and inference code for both the Hugging Face transformers library and vLLM.

So the audience is embodied AI researchers and robotics teams who need a checkpoint they can serve today, and who accept that a general purpose vision-language model with an action-oriented training regime may suit them better than a pipeline-specific model.

Sparse activation: about 3B of roughly 30B per token

The architectural claim is efficiency through sparsity. The model is built on the Hy3-A3B language backbone and the Hy-ViT2 vision encoder, combined in a mixture-of-experts arrangement. According to the project, only about 3B parameters are activated per token out of a total of roughly 30B.

The comparison the documentation makes is with its own previous generation rather than with a generic baseline. The activated parameter count is described as approximately one tenth of the previous generation A32B system, which the abstract says had 32B activated parameters, and the new model still reaches close to that previous model's performance while improving average performance by 8.4 percent.

That ratio is the interesting engineering claim, and it also has an obvious shadow: the total parameter count is not 3B. Anyone budgeting memory for a deployment should read the 86 GB figure in the prerequisites as the real number, not the 3B figure, because the first is what the full BF16 weights occupy and the second is only the slice touched per token.

The MoE structure also implies a routing layer that decides which experts run. The project does not document the routing granularity or the expert count, so anyone trying to predict latency or shard the model across devices has no public basis for it beyond the throughput characteristics implied by a sparse forward pass.

The action-centric taxonomy that shapes the data mixture

The most distinctive part of the project is not the architecture but the training design. The team defines an action-centric capability taxonomy with three progressive dimensions, and states that the data pipeline and the pre-training and post-training mixtures were designed around it.

The first level is Action-Relevant State Understanding, which covers understanding the state of the agent and of its environment. The second is Action-Transition Reasoning, meaning understanding actions, planning them and reasoning about their consequences. The third is Sequential and Adaptive Reasoning, covering long-horizon planning, reflection, repair and recovery. The order matters because it is a progression: a model that cannot read the current state cannot plan a transition, and a model that can plan one transition cannot be trusted over a long horizon.

This is a data and curriculum claim rather than an architectural one, and it is also the least independently verifiable part of the release. The taxonomy is illustrated in a figure in the repository and described in the technical report PDF, but the training mixture itself is not published in the repository, so an outside team cannot audit how much of the improvement comes from the taxonomy versus from scale and post-training. What you can adopt is the vocabulary, which is specific enough to be useful for labelling your own evaluation data.

Post-training that samples, rejects and fuses reward policies

The second training component is described as a self-evolving post-training loop, and the mechanism is specific enough to evaluate. Embodied agentic reasoning is cultivated through a loop that couples reinforcement learning with rejection-sampling fine-tuning, seeded from a small curated set of high-quality thinking traces. The word rejection-sampling means candidate outputs are generated and then filtered, so the model trains on its own acceptable solutions rather than on human demonstrations alone.

A final stage then specialises on reward. It trains continuous-reward and discrete-reward reinforcement learning policies separately and fuses them, with the stated aim of combining geometric precision from the continuous policy with the decision-making and reflection behaviour of the discrete one. Training two policies and merging them is a pragmatic way to get behaviour that a single reward function tends to blur together.

For an adopter, the relevant question is whether you can reproduce any of this, and the answer from the repository is no. The release plan lists fine-tuning code as an open item, so the reinforcement learning and rejection-sampling loop described here is documented in the report rather than shipped as runnable code in the repository.

Serving it: the in-tree vLLM plugin and a pinned dependency set

The install path uses uv, and the versions are pinned to what was validated end to end: vllm==0.14.1 with transformers==4.57.6 and torch==2.9.1. The reasoning for uv is practical, the documentation says the cleanest way to install these, with the CUDA build matched to your driver, is uv.

The first block installs uv itself if it is not present, then creates a fresh virtual environment on Python 3.12 and activates it:

# Install uv once (skip if you already have it)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create a fresh venv (Python 3.10+)
uv venv --python 3.12
source .venv/bin/activate

The vLLM path is the recommended one for serving, and it comes in two commands. The first installs vllm and lets the toolchain pick the CUDA wheel that matches your driver, the second installs the project's own plugin from the repository as an editable package:

uv pip install vllm==0.14.1 --torch-backend auto
uv pip install -e Hy-Embodied-VLM-1.0/inference/vllm/    # our plugin

The editable install is the part that matters. The vLLM integration is in-tree, and the planning checklist records vLLM inference as done, with the plugin noted in the item itself. A second installation option exists for the Hugging Face transformers path, since the release notes state that inference code is available for both transformers and vLLM, and the plugin is the mechanism by which the model reaches the vLLM serving stack. Expect to work inside the Hy-Embodied-VLM-1.0 directory structure, because that is where the plugin path is rooted.

The evaluation spans 38 benchmarks and stops at static tasks

The reported evaluation covers 38 benchmarks spanning embodied perception, physical-world understanding and embodied reasoning. The project states the model achieves the best performance among similarly sized models on 19 of the 38 and ranks second on another 11, and it lists Qwen3.6-A3B, Cosmos 3-8B and Embodied-R1.5-8B among the comparison models, with an average advantage of 4.4 percent over Qwen3.6-A3B. Two specific results are named: first place on R2R-CE vision-and-language navigation in the RGB-only setting, and zero-shot performance on Matterport3D Object Goal Navigation.

Two things are worth pressing on. The first is that these are the authors' own numbers on their own evaluation suite, presented in the repository's own write-up, and the repository does not publish the evaluation harness. The second is scope: the report also claims performance on embodied agentic tasks needing multi-turn interaction and long-horizon reasoning, but no separate benchmark scores are given for that category, so the interactive claim is harder to compare than the static one.

For a reader, the useful signal is the shape of the result rather than the exact percentages. Leading on 19 of 38 and second on 11 suggests a model that is competitive across the board rather than one that wins a single task spectacularly, which matches a design that spreads capability across a three-level taxonomy.

An 86 GB memory floor and a licence badge that needs reading

Two practical constraints should stop a prospective user before anything else. The first is memory. The prerequisites list Linux as the recommended operating system, Python 3.10 or newer, CUDA 12.x with H100, H20 and A100 tested, PyTorch 2.4 or newer, and NVIDIA GPUs, with the note that the full BF16 model requires roughly 86 GB across GPUs and that a single 8x80 GB node is sufficient. There is no smaller configuration described, and no quantised or lower precision variant listed in the release plan, so a single consumer card is not a target.

The second is the licence, where the two available signals disagree. The repository metadata classifies the licence as other, a custom licence that GitHub cannot classify, and points to the LICENSE file. The README badge, however, reads Apache 2.0 and links to that same file. Both statements cannot be taken at face value, and the project page for Hy-Embodied-0.5 in the same family carries a different model name on its weights link, so the licence file should be read line by line before any of this is used in a product. Anyone with a legal reviewer should have that reviewer read the LICENSE file directly, since a badge is a label and the file is the text.

Finally, on maintenance. The last push to the master branch was on 2026-07-15 and the repository publishes no GitHub releases, so there is no tagged version to pin. The update log inside the README is, for now, the only version history available.

Editorial conclusion

Adopt HY-Embodied VLM 1.0 if you need a vision-language model that reasons about actions and physical state and can serve it through vLLM or transformers, and if you have node class hardware, since the full BF16 model needs about 86 GB of GPU memory across an 8x80 GB node. It is the wrong starting point if you intend to fine-tune, because the fine-tuning code and the online Gradio demo are both still unchecked items in the project plan. Before anything else, confirm the licence in the LICENSE file, since the README badge reads Apache 2.0 while GitHub classifies the repository licence as a custom one, and note that the last push was on 2026-07-15 with no tagged release to pin against.

Frequently asked questions

Official sources

1. Issues
2. README
3. Tencent-Hunyuan/HY-Embodied on GitHub