Cambrian-S ships weights, data and a benchmark for spatial video reasoning

Asking video models to hold on to space, not just frames

Cambrian-S is a video multimodal model release from NYU's vision group, aimed at what the authors call spatial supersensing: reasoning about space across a video rather than answering questions about individual frames. It arrives with model weights, a training dataset, a benchmark and the training and evaluation code, which is a more complete release than most papers manage.

The gap it targets is genuine. Video models have become competent at describing what appears on screen and remain weak at questions requiring a persistent sense of the space being moved through: how many distinct objects were in a room, what order things were encountered in, where something was relative to something else several minutes earlier. Those require carrying state across time rather than summarising a window.

The audience is researchers working on video understanding and teams evaluating whether spatial reasoning is usable yet. This is a research artifact, with the model sizes and the packaging to match.

Four sizes, one recipe, and a separate predictive variant

The model card is refreshingly plain about construction. Four standard models are published at 0.5, 1.5, 3 and 7 billion parameters, each pairing a Qwen2.5 Instruct language model of the corresponding size with the same siglip2 vision encoder. Holding the vision encoder fixed across the range is what makes the sizes comparable, since the only variable is the language model.

A fifth model sits apart. Cambrian-S-7B-LFP is described as trained with predictive sensing, and it is the one evaluated on the project's own spatial benchmark. That separation is the interesting structural claim: the standard models are the conventional recipe at four scales, and the predictive variant is where the contribution is meant to show.

Publishing a small model alongside a large one is more useful than it appears. A 0.5 billion parameter version is a way to run the pipeline end to end cheaply and confirm the data path works before committing hardware to the 7 billion parameter model, and it gives anyone studying scaling behaviour four points instead of one.

The dataset and benchmark matter as much as the weights

Two artifacts alongside the models carry a good deal of the release's value.

VSI-590K is a video instruction-tuning dataset curated for spatial understanding, released with the models in November 2025, with the meta information for the curated set published later on 2026-04-03. A separate collection of three million open-sourced video instruction tuning examples was released in December 2025. Training data released with a model is the part that makes results reproducible rather than merely repeatable, and it is the part most releases keep back.

VSI-SUPER is the benchmark, designed for spatial supersensing and released at the same time as the models. Here a caution applies that applies to every paper shipping its own benchmark: the team that defines the evaluation, curates the training data and trains the model controls all three, and strong results on such a benchmark demonstrate that the approach addresses the problem as the authors framed it. That is worth something and it is not the same as generalisation.

The README supports this reading, stating that the models remain competitive on standard video benchmarks while excelling at spatial tasks. Competitive rather than better on the general benchmarks is the honest framing of a specialised model, and the specific comparisons live in figures and in the paper.

Building the environment is the first real obstacle

Packaging tells you what running this involves. The manifest sets a low interpreter floor.

requires-python = ">=3.9"

The dependency list is where the difficulty sits, because almost everything is pinned to an exact version rather than a range: the transformers library at 4.37.0, tokenizers at 0.15.0, accelerate at 0.23.0, the parameter-efficient tuning library at 0.4.0, numpy at 1.26.4, and several others in the same style. Exact pins make the authors' environment reproducible, which is correct for research, and they make the project difficult to install beside anything else with opinions about those libraries. A dedicated environment is not optional here.

Two files in the tree describe the intended scale of operation. A fully sharded data parallel configuration sits at the repository root, which indicates training was designed for multiple GPUs rather than one. An evaluation harness is vendored into the repository rather than declared as a dependency, which pins the evaluation code alongside the training code and is the right choice when benchmark numbers must be reproducible.

Weights are distributed through Hugging Face rather than the repository, so the code here is for training and evaluation while inference starts with a download.

Where this is the wrong thing to reach for

The first limit is that this is research code released around a paper. There are no tagged releases, the last push was on 2026-04-03, and activity in projects of this kind tracks publication rather than user demand. Anyone depending on it should pin a commit.

The second is the pinned dependency set described above. A version of the transformers library pinned exactly is the sort of constraint that decides whether this can coexist with the rest of your stack, and the answer is usually that it cannot, so plan on isolation.

The third is scope. Spatial supersensing is a specific capability, and the release is explicit that general video performance is competitive rather than leading. A team that needs the strongest general video understanding should not adopt this for that purpose.

The fourth is evaluation independence. The training data, the benchmark and the model come from one group, and until the benchmark is adopted by others the results describe performance on a test its authors designed. That is a normal position for new work and it is the thing to hold in mind when reading the figures.

A general video model is the alternative, and the split is specialisation

The realistic alternative is a general purpose video multimodal model, whether open weights or a hosted service, used directly on your task.

The difference in approach is what the training was optimised for. A general model is trained broadly, so it handles description, question answering and captioning across many kinds of video, and it will be weaker where the question requires holding spatial structure across a long sequence. That weakness is precisely what this project set out to measure and address, and the tuning data was curated for it.

The practical consequence for a reader is a test rather than a preference. Take a handful of your own videos and ask the questions you actually care about: counting distinct objects across a walkthrough, recalling the order things appeared, relating positions separated by minutes. If a general model handles them, its breadth and easier packaging win and there is no reason to accept exact dependency pins and a multi-GPU training configuration. If it fails on exactly those questions, this release is aimed at your problem and the specialisation is the point.

The one thing not to do is choose between them on the published tables, since those were produced on a benchmark built alongside the model.

Apache terms and what a reproduction would cost

Cambrian-S carries an Apache-2.0 licence with the file present in the repository, which includes an express patent grant and permits commercial use. That is the permissive end for research code. The weights and the datasets are distributed separately through Hugging Face and carry their own terms, which are a separate question from the licence on this code, and anyone shipping a product should read those directly. This is not legal advice.

Reproduction cost is the figure to estimate before committing. The release includes the training code, the tuning dataset and the evaluation suite, so a reproduction is possible in a way it usually is not. What it requires is the hardware implied by a sharded parallel training configuration and a 7 billion parameter model, plus the storage for a video instruction dataset in the hundreds of thousands of examples.

For most readers the sensible entry point is not reproduction at all. Pull the smallest published checkpoint, run the vendored evaluation harness against your own clips, and decide from that whether the predictive variant at 7 billion parameters is worth the environment it demands.

Editorial conclusion

Cambrian-S is worth taking up if your questions involve spatial structure held across a long video, counting distinct objects through a walkthrough or recalling the order things were encountered, and you can give it an isolated environment. It is the wrong choice when you need the strongest general video understanding, which the release itself describes only as competitive, or when exact pins on the transformers, tokenizers and accelerate libraries cannot coexist with your stack. Judge it on your own clips rather than the published figures, because the model, the tuning dataset and the benchmark all come from the same group, and start with the 0.5 billion parameter checkpoint before committing hardware to the 7 billion parameter predictive variant.

Frequently asked questions

Official sources

1. cambrian-mllm/cambrian-s on GitHub
2. Issues
3. License: Apache-2.0
4. Project website
5. README