# Orca-4B wins the four benchmark average and still loses MVBench > orca-wm/Orca is a BAAI world foundation model whose README publishes a text generation table, a partly visible image prediction table, and a repository top level holding only a README, assets and an evaluation directory. Reading the numbers closely shows a 2B model outscoring a 34B one, a released row for an unreleased checkpoint, and an action readout with no code behind it. **orca-wm/Orca** — Orca: The World is in Your Mind - Repository: https://github.com/orca-wm/Orca - Stars: 1,019 · Forks: 58 - Language: Python - License: not declared - Published: 2026-09-18 · Updated: 2026-09-18 · Language: en - Canonical page: https://hysenlabs.com/projects/orca-wm-orca ## Orca-4B takes three of four columns, and Qwen3.5 takes MVBench The text generation evaluation runs on TemporalBench, MVBench, SWITCH, and 3DSRBench, and the published table puts Orca-4B at 34.2, 52.1, and 55.6 on TemporalBench, 3DSRBench, and SWITCH, with an average of 51.8. Those are the best three numbers in the table. MVBench is the exception. There Orca-4B scores 65.3, and Qwen3.5 at 4B scores 67.1. Both models are 4B, so the same-size generalist leads on that column while Orca leads the average 51.8 to 46.7. The smaller Orca row reads 0.8 for size, 53.6 on MVBench, 22.6 on TemporalBench, 43.4 on 3DSRBench, 43.7 on SWITCH, and 40.8 average. So the smaller backbone beats the 4B model on MVBench by 11.7 points while losing to it everywhere else, which is the shape you would expect from a column that rewards general visual understanding rather than temporal modelling. Both Orca rows are marked with the same model name in the table, so the 0.8 and 4 rows are distinguished by the size column rather than by a separate name. ## A 2B model outscores a 34B model on the same four benchmarks The comparison rows are worth reading as carefully as the Orca rows. MiniCPM-V-4.6 at 2B averages 37.9. Emu3.5 at 34B averages 29.8. The smaller model beats the larger one on the average by more than eight points, and it wins three of the four columns outright: 41.4 against 39.5 on MVBench, 21.2 against 9.5 on TemporalBench, 47.7 against 31.3 on 3DSRBench, with SWITCH the only column Emu3.5 holds, 38.9 against 41.2, and by 2.3 points. Emu3.5 also lands below Emu3. Emu3 at 8B averages 30.4, so going from 8B to 34B in that family drops the average by 0.6, and the drop comes entirely from 3DSRBench, which falls from 39.1 to 31.3 while MVBench rises from 35.2 to 39.5. TemporalBench is the column that separates the field most sharply. Emu3 and Emu3.5 both score 9.5 there while Orca-4B scores 34.2, more than three times either. Whatever the average column rewards, Orca's advantage sits in temporal reasoning, and the other four models in the table cluster near the bottom of it. ## The 0.8B row is in the results table, and the model zoo says coming soon The model zoo has two rows. Orca-4B points at a Hugging Face checkpoint and is described as the larger backbone with stronger downstream readout performance. Orca-0.8B, described as a lightweight backbone for world latent learning, is listed as coming soon. The task list agrees on the 0.8B side. Three items are checked: release the technical report, release the Orca-4B checkpoint for world latent learning and downstream readouts, and release inference code for text and image. Two are not: release the Orca-0.8B checkpoint for lightweight research and reproduction, and release downstream fine-tuning code for modality-specific readout adaptation. So the table's 0.8B row, with 53.6 on MVBench and 40.8 average, is not attached to an artifact a reader can download. The numbers are published, the checkpoint is not, and the two documents that could have kept them in step did not. For anyone planning an ablation, that matters more than the missing file does. The interesting comparison in this project is between a 4B and a 0.8B backbone reading out the same latent, and the smaller side of it is exactly what you cannot check. ## Action is an evaluated readout with no released code behind it The overview names three kinds of readout: lightweight decoders for text, images, and actions. The architecture section says the frozen encoder's latent is read out into language, vision, and action. The evaluation section says Orca is evaluated through three representative downstream readouts, text generation, image prediction, and action generation. The released items do not line up with that list. Inference code is marked released for text and image only. Code for fine-tuning modality-specific readout adaptation is an open task. Nothing in either line mentions the action path. So the third readout is described in the same breath as the other two and is evaluated in the same breath as the other two, and it is the one with no code path named. The architecture reasoning does explain why it exists, since implicit dynamics cover physical laws, object properties, scene dynamics, and environmental forces, and a world model that cannot read out an action has learned the dynamics for its own sake. That is an argument for the design, not a substitute for the code. If action is what you came for, ask where that decoder lives before planning around it. ## The top level holds a README, assets and an evaluation directory The repository's top level entries are README.md, assets/, and evaluation/. There is no source directory, no training tree, and no inference tree at the root. That sits against a project whose news entries record a technical report released on 2026-06-29 and an Orca-4B checkpoint released on Hugging Face on 2026-07-14, and whose completed task items include inference code for text and image. The evaluation directory is where the only code path named anywhere in the project appears: the PRICE evaluation toolkit at evaluation/image_gen/PRICE, used to evaluate future-state prediction on real-world interactions. The benchmark data itself is not in the repository. It is hosted as a dataset at BAAI/PRICE on Hugging Face, so a reproduction has to pull the data from outside the tree and the toolkit from inside it. What that means for a reader is concrete: the model is a download, the weights are on a model hub under the BAAI organisation, and the repository is a document plus assets plus evaluation code. Anyone expecting training scripts or a loadable model class in this repository will not find one here. ## No license file sits next to a 4B checkpoint The repository's license is listed as unknown, and there is no LICENSE file among the top level entries. The README carries no license section. The only terms-adjacent text in it is a note inviting questions through WeChat, alongside the project page, the Hugging Face link, and the technical report. That is a real gap for a project whose central artifact is a downloadable 4B checkpoint. Language for what the model may be used for, whether weights can be redistributed or fine-tuned commercially, and what obligations attach to the training data are all absent from the repository. The training mixture is described in volume terms, 125K hours of video, 160M event annotations, and general VQA data, and nothing in the visible text addresses where that video came from or under what terms. The distinction to hold on to: an unavailable license is not the same as a permissive one, and the fact that a checkpoint is downloadable says nothing about the rights attached to it. Check the terms on the model hub before you use Orca-4B for anything. ## Two data components are counted, and the third is not The pre-training inventory is built from visual signals and language signals, and the mixture has three parts. Video data covers observation-only state transitions and is given a volume: 125K hours covering egocentric interaction, exocentric manipulation, robot execution, and natural dynamics. Event data covers event-conditioned state transitions and is given a count: 160M event annotations with fine- and coarse-grained captions. The third component has neither. General VQA data is named for its purpose, aligning world latents with language understanding and response generation, with no size and no source. So two thirds of the recipe are quantified and one is not, which makes the mixture impossible to reconstruct or to compare against another model. The scaling claim is stated in the same shorthand. Stronger world modeling, stronger downstream readouts, with no curve, no sweep, and no axis. The one scaling-related thing in the visible material is the size pair in the results table, 0.8 and 4, and the 0.8B checkpoint is not the one that is available. ## Conclusion Orca is worth an evening if you want to understand what a next-state formulation claims, and the four numbers in the text table are the honest way to size it: a 4B model that wins three of four columns and the average, behind a same-size generalist on MVBench. Verify four things before you plan around it. Whether the 0.8B checkpoint is actually downloadable, since its row appears in the results but the model zoo says otherwise. Whether action readout code exists, since the evaluation names it and the release list does not. What license applies to the checkpoint, because the repository has none and the only stated terms are a WeChat contact. And whether the benchmark numbers come from a run you can point to, since no code path in the tree is named for reproducing them. ## FAQ ### What is the Orca world foundation model from orca-wm? It is a general world foundation model from the Orca Team at the Beijing Academy of Artificial Intelligence, centered on Next-State-Prediction rather than next-token, next-frame, or next-action prediction. It learns a unified world latent from visual and language signals and exposes it through lightweight frozen-backbone readouts for text, images, and actions. ### How did Orca-4B score on the text generation benchmarks? On MVBench, TemporalBench, 3DSRBench, and SWITCH it scored 65.3, 34.2, 52.1, and 55.6, for an average of 51.8. It leads the table on the last three columns and on the average, while Qwen3.5 at the same 4B size leads MVBench with 67.1. ### Is the Orca-0.8B checkpoint available for download? No. The model zoo lists Orca-0.8B as coming soon, and the task list still has releasing the 0.8B checkpoint unchecked. Only Orca-4B is linked, on Hugging Face under BAAI. The 0.8B row does appear in the text generation results table. ### What data was Orca pre-trained on? 125K hours of video covering egocentric interaction, exocentric manipulation, robot execution, and natural dynamics, plus 160M event annotations with fine- and coarse-grained captions for event-level transition learning, plus general VQA data. The VQA portion is given without a size. ### Is inference code released for all three Orca readouts? No. The completed task items name inference code for text and image. Downstream fine-tuning code for modality-specific readout adaptation is still unchecked, and the action readout is described and evaluated without a code path named for it. ### What license applies to Orca? The repository lists its license as unknown and has no LICENSE file at the top level, and the README carries no license section. Check the terms on the Hugging Face checkpoint at BAAI/Orca-4B before use, since the repository states none. ## Sources - [Issues](https://github.com/orca-wm/Orca/issues) - [orca-wm/Orca on GitHub](https://github.com/orca-wm/Orca) - [README](https://github.com/orca-wm/Orca/blob/main/README.md) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/orca-wm-orca