Tencent-Hunyuan/Hy3: a 295B MoE model you host yourself

The problem Hy3 is aimed at: flagship-class reasoning without flagship-class serving cost

Hy3 is a Mixture-of-Experts language model with 295B total parameters, 21B activated parameters, and a further 3.8B of MTP layer parameters. The README states it "outperforms similar-size models and rivals flagship open-source models with 2-5x parameters." That claim is the project's whole pitch: you pay for 21B of compute per token while the checkpoint holds 295B of weights.

The intended audience is not someone who wants a chat tab. It is an engineering team that already runs inference infrastructure and wants an instruct model with agentic and long-context behaviour. The README lists coding, office work, financial modeling, frontend design, and game development as productivity targets, and the repository ships finetune/ and rl/ directories alongside the weights, so post-training is treated as part of the workflow rather than an afterthought.

One structural detail matters more than the parameter count. The architecture table gives 192 experts with top-8 activation, 80 layers excluding the MTP layer, 64 attention heads with 8 KV heads under GQA, and a 256K context length. That combination is what makes a 295B checkpoint plausible on a realistic node count, and it is also what makes the memory math non-obvious: total weights dominate your VRAM or HBM budget, not your per-token FLOPs.

How the MoE routing and the reasoning switch actually work

Each token passes through 80 transformer layers. At each layer the router selects 8 of 192 experts, so the active path is a small slice of the network even though every expert's weights must be resident or swappable. Attention uses grouped-query attention with 64 query heads sharing 8 KV heads at head dimension 128, which keeps the KV cache smaller than a full multi-head setup at the same context length. The single MTP layer adds 3.8B parameters on top and is excluded from the 80-layer count.

Reasoning depth is a runtime switch, not a separate checkpoint. The quickstart passes reasoning_effort through chat_template_kwargs, and the README documents three settings: no_think for a direct response, low, and high for deep chain-of-thought on math or coding. The default is no_think. That means the same deployed model serves both a fast chat path and a slower reasoning path, and the choice is made per request by the caller.

The README also describes training-side work that shows up as behaviour rather than as a flag: joint SFT and RL optimization for coreference, ellipsis recovery, and multi-turn constraint inheritance, plus data cleaning aimed at grounding. The reported internal figures are a hallucination rate falling from 12.5% to 5.4%, commonsense error rates from 25.4% to 12.7%, and a multi-turn issue rate from 17.4% to 7.9%. These are internal evaluations, not third-party reproductions, and the README does not publish the harness.

Installing Hy3 and making a first request against vLLM

There is no pip install for the model itself. The README's path is to deploy with vLLM or SGLang first, then call the resulting server through an OpenAI-compatible client. Weights come from Hugging Face, ModelScope, GitCode, or CNB; the model table lists Hy3 as the instruct model and Hy3-FP8 as the FP8 quantized variant.

The client side is a standard OpenAI SDK call. Note the base URL and the placeholder key, which are exactly what the README shows:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="hy3",
    messages=[
        {"role": "user", "content": "Hello! Can you briefly introduce yourself?"},
    ],
    temperature=0.9,
    top_p=1.0,
    extra_body={"chat_template_kwargs": {"reasoning_effort": "no_think"}},
)
print(response.choices[0].message.content)

If the server is up and the model name matches, this prints the assistant's reply. The README recommends temperature=0.9 and top_p=1.0. Reasoning mode is controlled through the same extra_body field, and the README lists no_think, low, and high as the accepted values for reasoning_effort.

The README does not print the vLLM or SGLang launch command in the excerpt available here, so treat the server invocation as something to read from the vLLM and SGLang sections of the repository before you size a machine. What is documented is the contract: an OpenAI-compatible endpoint on 127.0.0.1:8000, model name hy3, and a per-request reasoning_effort value.

Where Hy3 is the wrong tool

The supported precision is BF16. A 295B-parameter checkpoint in BF16 is roughly 590GB of weights before you account for KV cache, activations, and the MTP layer, which puts it in multi-node or multi-GPU territory. The README points to Hy3-FP8 for a quantized variant, but that is a separate model repository, not a flag on this one. If your target is one workstation GPU, Hy3 is not the model you are looking for, and no amount of expert sparsity changes the resident weight footprint.

The README's strongest quality claims are internal. The blind evaluation with 270 experts scoring Hy3 at 2.67/4 against GLM-5.1 at 2.51/4 is described as run by the project, and the hallucination and multi-turn figures come from internal evaluations. The scaffold-variance claim, that SWE-Bench Verified accuracy varies within 4% across CodeBuddy, Cline, and KiloCode, is the kind of result that would be more convincing with the raw runs attached. None of this makes the numbers false, but it means an independent evaluation on your own workload is the only number that should drive adoption.

Finally, the license situation needs care. The README badge says Apache 2.0, and a LICENSE file exists in the repository root, but the repository metadata reports the license as NOASSERTION. Those two signals disagree, and the README does not reconcile them. Read the LICENSE file before you ship anything built on these weights.

Hy3 against GLM-5.1: same category, different trade

The natural comparison is with other open-weight MoE instruct models of similar ambition. The README itself names GLM-5.1 as the baseline in its blind evaluation, and that comparison is the most useful one to reason about because both are open-weight MoE systems aimed at the same buyers.

The difference is not the architecture family, since sparse routing with a small active parameter count is now the standard shape at this scale. The difference is in what the project optimizes for after the base model. Hy3's README devotes most of its length to tool-call reliability, output-format stability, anti-hallucination training constraints, and multi-turn intent tracking, with explicit before-and-after internal numbers. That is a production-reliability pitch rather than a benchmark-topping pitch, and the README says so directly: "We don't think public benchmark scores tell the full story."

If your workload is single-turn generation, that emphasis buys you little, and a smaller dense model may be cheaper to serve and easier to reason about. If your workload is an agent loop with tool schemas, retries, and long conversations, the properties Hy3 claims to have fixed are exactly the ones that break agent pipelines in practice, and they are worth testing on your own traces rather than on a leaderboard.

Maintenance, finetuning surface, and upgrade cost

The repository was last pushed on 2026-07-17 and is not archived. There are no releases listed, so there is no versioned artifact to pin against and no changelog to read between checkpoints. For a model project that is normal, but it means your upgrade unit is a weight revision on Hugging Face or ModelScope, not a tagged release.

The repository layout is small: LICENSE, README.md, README_CN.md, assets/, finetune/, and rl/. The presence of finetune/ and rl/ indicates the project expects downstream teams to post-train rather than only serve, and the README has dedicated sections for finetuning, RL post-training, and quantization. Those sections are where the real integration cost lives, because a 295B MoE checkpoint is not something you fine-tune on a single node without sharding strategy, and the README does not document rollback or checkpoint-compatibility guarantees between revisions.

On licensing, the Apache 2.0 badge is the only permission signal in the README, while the repository metadata says NOASSERTION. The README does not state whether the weights carry additional terms beyond the LICENSE file. This is not legal advice; the practical step is to read LICENSE in the repository root and confirm it matches the badge before you build a product on top of the weights.

Editorial conclusion

Adopt Hy3 if you already operate multi-GPU inference and want a 21B-active MoE with a 256K context and an OpenAI-compatible endpoint. Do not adopt it if you need a managed API, a small single-GPU footprint, or a license you can read without a lawyer, since the LICENSE file is present but GitHub reports the license as NOASSERTION. Before committing, verify the exact vLLM or SGLang invocation and the weight precision you can actually serve, because the README documents BF16 as the supported precision and points to a separate Hy3-FP8 repository for the quantized variant.

Frequently asked questions

Official sources

1. Issues
2. README
3. Tencent-Hunyuan/Hy3 on GitHub