NEO builds a vision-language model without a separate image encoder

What it means to be native and encoder-free

NEO is described as a series of native vision-language models, and the word native is doing specific work in that description. The highlights section states that NEO innovates a native VLM primitive that unifies pixel-word encoding, alignment and reasoning within an encoder-free dense model architecture.

Most multimodal systems in production are modular: a vision transformer or similar encoder turns an image into a sequence of vectors, and a language model consumes those vectors as if they were text tokens. The image side and the text side are separate models that were trained separately and then connected. NEO removes that split. Encoding, alignment and reasoning all live inside one dense model, and there is no image encoder in the pipeline at all.

The motivation section frames this as a research question rather than a feature list. It asks what constraints set native VLMs apart from modular ones, and to what extent they can be overcome, and separately how to make native VLMs more accessible so their progress accelerates. The second question explains why the repository publishes training code and evaluation harnesses rather than weights alone.

The claimed result is efficiency rather than a new ceiling. The README states that using under 10% of image-text data, NEO develops strong visual perception from scratch, rivaling top-tier modular VLMs and outperforming native encoder-free ones. That is the project's own claim, stated in its own words, and the charts that back it are images in the docs directory rather than numbers anyone can check in the text.

Two model families and a timeline of releases

The model zoo lists two entries. NEO is the first, with its weights on Hugging Face under the neo1-0 collection, and NEO-ov is the second, under neo1-5. The README states that 2B and 9B NEO-series models are released across different training stages, so the naming differences are about the training stage and the vision scope, not a single checkpoint.

The news entries give a chronological picture. The project reports NEO as completed in September 2025, with the paper, weights and evaluation code following in October 2025. NEO-ov was reported completed in December 2025, with its paper, weights and evaluation code released in May 2026. Training code for the NEO series was released in January 2026.

The paper trail runs alongside. The entries point to an arXiv paper on native vision-language primitives at scale from September 2025, an arXiv paper on native one-vision models at scale from December 2025, a Hugging Face blog post on NEO-unify from February 2026, and a technical report from 2026 on SenseNova-U1 with the NEO-unify architecture from April 2026. The two arXiv identifiers are cited in the BibTeX block at the end of the README, together with the blog post and the SenseNova-U1 report, and the author list on the NEO-ov paper is longer than on the first.

The repository itself is not where the papers live. The README is the index, and it points outward to arXiv, to Hugging Face for both weights and the blog, and to the SenseNova-U1 report for a system built on the same architecture.

Three directories, three jobs

The top-level layout of the repository is unusually small for a project of this scale, and it maps directly onto the three jobs the project has to do. There are three directories, VLMEvalKit, VLMEvalKit_ov and VLMTrainKit, plus a docs directory and the README and licence.

The two evaluation kits are named after the two model families. The todo list records an evaluation guide for NEO-ov at VLMEvalKit_ov/README.md and an evaluation guide for NEO at VLMEvalKit/README.md, and both are marked done. The news entries point at the same two files as the released evaluation code for the respective models.

VLMTrainKit is the training side, and its README is linked from the January 2026 news entry as the released training code. Its todo item, a training guide, is also marked complete. The fact that all three guides exist and are complete is the most useful fact in the repository for anyone deciding whether to try it, because the three tasks a researcher actually needs, training, evaluation and reproduction, each have a documented entry point.

The separation is by model family rather than by function, which means a user working with NEO-ov uses the ov kit for both evaluation and training and leaves the original kit alone. That is a reasonable arrangement, though it does mean a cross-family comparison means running two harnesses rather than switching a flag.

Getting the weights and running a first evaluation

There is no install command in this repository, and that is not an omission to work around. NEO is a set of models with training and evaluation code, not an installable package with a command line, and the README points to where things actually live.

The weights are on Hugging Face. The model zoo table links NEO to the neo1-0 collection and NEO-ov to the neo1-5 collection, and the news entries point at the same places when each release was announced. Nothing about the licence of those weights is stated in the README, and the licence paragraph covers the content of the project itself, which is worth reading carefully before you load a checkpoint.

The route for a first run is the evaluation guide. Following the link in the todo list to VLMEvalKit/README.md for NEO, or VLMEvalKit_ov/README.md for NEO-ov, gives the procedure the project intends for its own checkpoints, and using the same harness the authors used is the only way the reported comparisons can be reproduced rather than merely approximated. The performance section of the README points at three charts for single-image understanding, multi-image and video understanding, and spatial intelligence, all of which live as images under docs.

What you should expect to read is a benchmark procedure, not a service contract. There is no inference API described, no serving configuration, and no stability promise around the weights, so anyone planning to put this in production has to build the serving layer themselves and should treat the model as a research artefact with a licence and a provenance to check.

What the repository does not settle

Two gaps are visible in the README and both are structural rather than temporary.

The first is reproducibility detail. The repository has no GitHub releases, so there is no tagged version to pin to, and the last push was on 2026-07-27. Anyone planning to run the evaluation kits has to track the branch rather than a release, and a change to a harness between two runs is a real possibility that no tag would warn them about. The CHANGELOG is not part of the repository either, since the top-level entries are the LICENSE, the README and the three kits plus docs.

The second is the boundary between the project and the systems built on it. The README documents a NEO-unify architecture and points to a SenseNova-U1 report that uses it. It is clear that NEO-unify is the architecture and SenseNova-U1 is a system built on that architecture, but the repository does not contain the SenseNova-U1 code, and the Model Zoo lists only the NEO and NEO-ov checkpoints. A reader should not assume that the training code here reproduces that system.

The charts are the third gap, in a smaller way. Every performance claim in the README is an image. That means the numbers cannot be checked from the text of the repository, and a reader who wants to compare against a different evaluation setup has to rely on the evaluation guides and run them.

Editorial conclusion

NEO suits a research group that wants to study or reproduce native multimodal architectures, since the repository ships the training code, two evaluation kits and downloadable weights for 2B and 9B models. It is the wrong tool for a product team expecting a hosted endpoint or a stable inference contract, because the repository has no GitHub releases and the README documents training and evaluation rather than serving. Before you adopt it, read the evaluation guides in VLMEvalKit and VLMEvalKit_ov to see which benchmarks the reported comparisons cover, confirm that the Apache-2.0 licence file covers the weights you intend to use rather than only the repository contents, and check the last push on the repository, which was on 2026-07-27, against your own timeline for tracking upstream changes.

Frequently asked questions

Official sources

1. EvolvingLMMs-Lab/NEO on GitHub
2. Issues
3. License: Apache-2.0
4. README