# VideoChat-Flash compresses video into 16 tokens per frame > VideoChat-Flash is an MIT licensed research codebase for long-context video understanding, where hierarchical compression encodes each frame into 16 tokens so that very long sequences fit into an LLM context. It ships model weights, evaluation and training code, and a needle-in-a-haystack suite, with the paper accepted at ICLR 2026. **OpenGVLab/VideoChat-Flash** — [ICLR2026] VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling - Repository: https://github.com/OpenGVLab/VideoChat-Flash - Stars: 531 · Forks: 21 - Language: Python - License: MIT - Published: 2026-10-22 · Updated: 2026-10-22 · Language: en - Canonical page: https://hysenlabs.com/projects/opengvlab-videochat-flash ## The compression idea, stated in one number The mechanism the project is built around is a single figure: each video frame is encoded into 16 tokens. The README describes the architecture as highly efficient, with exceptional inference speed, and says this makes it 5 to 10 times faster than the previous model. Everything else in the repository follows from that choice. A transformer language model has a fixed context budget, and a video tokenises into far more tokens per second of footage than text does, so long video has historically been out of reach for multimodal LLMs. Hierarchical compression attacks the cost at the frame level rather than by subsampling frames away, which is why the README pairs it with support for ultra-long video inputs and a needle-in-a-haystack evaluation accuracy of 99.1% on 10,000 frames, with the ability to process videos up to three hours long. The README also claims state-of-the-art performance in short and long video understanding with temporal localization capabilities comparable to expert models, though that is the project's own claim and the evaluation section points elsewhere for the numbers. ## What the repository actually contains The top-level layout shows four working directories, each one a different task. llava-train_videochat holds the training codes based on LLaVA for VideoChat-Flash, and xtuner-train_internvideo2_5 holds training codes based on XTuner for finetuning InternVideo2.5. lmms-eval_videochat holds the evaluation codes for the QA and Grounding benchmarks, and xtuner-eval_niah holds the evaluation code for the Single-Hop NIAH-Video and Multi-Hop NIAH-Video suites. The repository is therefore less a library than a set of recipes, and the primary language is Python. Weights and data live outside the repository on Hugging Face, under the OpenGVLab organisation, with a collection for the models and data and a separate dataset for the training data. The paper is on arXiv at 2501.00574, there is a blog post on the InternVideo site, and a demo link is present in the header. The project is MIT licensed, the default branch is main, the repository is not archived, and the last push was on 2026-07-19. ## The model family: five checkpoints with different trade-offs The January 2025 release introduced VideoChat2-Flash, built on a video encoder, InternVideo, and an LLM, Qwen, and offered five models. The naming encodes the axis of variation: the size of the language model and the input resolution. There is a 2B model at 224 resolution for the small LLM case, a 7B model at 224, a 7B model at 448 which the README marks as the overall best, a 7B model with InternVideo2 as the encoder and a 1M token context for super long video input, and a 7B model with the InternVideo2 1B encoder, which the README marks as stronger for short-term temporal understanding. That last pair is the interesting design choice. The 1M-context variant buys length by giving the language model a very large window, while the InternVideo2 variant buys temporal precision with a stronger encoder over a shorter window. Choosing between them is a decision about whether your video is long or whether your questions are about fine temporal detail. ## Getting the weights and running inference There is no install command in this repository, and the README is explicit about where the inference steps live: it refers you to the Hugging Face model README for the model you want, with the linked example being VideoChat-Flash-Qwen2_5-2B_res448. The same pattern holds for the other pieces. Evaluation goes through the lmms-eval_videochat directory in this repository, and the README notes that lmms-eval itself supports the model, so you can use that to run it on various benchmarks. Training for VideoChat-Flash uses the LLaVA-based codes in llava-train_videochat, and finetuning InternVideo2.5 uses the XTuner-based codes in xtuner-train_internvideo2_5. The README also records a sequence of weight releases for each training stage, published so that people could try to build VideoChat-Flash on them. That staging matters if you plan to train rather than only to evaluate, since the released checkpoints correspond to points in the training pipeline. ## The evaluation story, including the needle-in-a-haystack results The evaluation section is where the project's claims are checkable. The code for the QA and Grounding benchmarks was released in January 2025, and the needle-in-a-haystack code covers both single-hop and multi-hop variants. The claimed result is 99.1% accuracy on 10,000 frames, and the README reports promising results on the VideoEval-Pro benchmark for long video understanding in June 2025. The dataset and evaluation codes for single-hop and multi-hop needle-in-a-haystack were released in March 2025, and most of the training data was released in May 2025. The February 2025 entry records that VideoChat-Flash-7B@448 achieved first place on the AuroraCap video detail caption benchmark. Taken together, these give you two distinct evaluation surfaces: a synthetic retrieval test that isolates long-context memory, and captioning and question answering benchmarks that test whether the compressed representation still carries meaning. The single-hop and multi-hop distinction is the more demanding of the two, since a multi-hop question requires the model to combine facts from separate positions in the video. ## The project has already moved on to VideoChat3 The July 2026 update is the most recent entry and it points somewhere else. The authors released VideoChat3, a fully open 4B video MLLM for general, long-form and streaming video understanding, with model weights, code, training recipes and complete datasets, and with improvements over Qwen3-VL-4B on 18 of 19 offline metrics and 10 of 11 streaming metrics. That is a separate repository, MCG-NJU/VideoChat3, with its own paper and homepage. The practical consequence for anyone picking this up is that VideoChat-Flash is the earlier generation, and the team has moved on to a successor that covers streaming video, a capability VideoChat-Flash does not address. The README's future plan also lists lmdeploy and vllm support for VideoChat-Flash and InternVideo2.5 as unchecked, LoRA finetuning training code as unchecked, mixed image and video training code for InternVideo2.5 as unchecked, and faster training code with XTuner as unchecked. The author then asks the community to submit pull requests, citing being busy with work. ## Where VideoChat-Flash is the wrong choice Three limits, and they are the ordinary limits of research code rather than failures of the method. The first is deployment. The README's own plan has lmdeploy and vllm support unchecked, and the inference instructions live on a Hugging Face model card rather than in a packaged runtime, so turning this into a service is work you own rather than something the repository gives you. The second is the dependency on upstream projects. The acknowledgements list InternVideo, UMT, Qwen, LLaVA-VL, lmms-eval, Ask-Anything, ToMe, LongVLM, FastV, LLaVolta, PyramidDrop and LongVA, and the model itself combines an InternVideo-family encoder with a Qwen-family LLM. Reproducing a result means keeping a dozen upstream projects in compatible versions, and the compression scheme is only as good as the encoder you point it at. The third is the comparison set. Qwen3-VL-4B and the newer VideoChat3 are the baselines a reviewer will ask about, and VideoChat-Flash predates both, so its numbers are a moving target rather than a final claim. The realistic alternative for most teams is to start from VideoChat3 directly, since it is described as fully open with training recipes and complete datasets, and treat VideoChat-Flash as the reference implementation for the hierarchical compression idea and as a source of the needle-in-a-haystack evaluation suite, which remains useful regardless of which model you evaluate. ## Conclusion Adopt VideoChat-Flash if you are working on long-video understanding and want the model, the training recipes and the needle-in-a-haystack evaluation in one place. Skip it if you need a serving deployment, because the README's own future plan lists lmdeploy and vllm support as still unchecked. Before you build on it, read the model card on Hugging Face for the inference steps, and note that the pipeline for long video still depends on XTuner for finetuning InternVideo2.5. ## FAQ ### What is VideoChat-Flash? It is a research codebase for long-context video understanding, built on hierarchical compression that encodes each video frame into 16 tokens. The paper, VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling, is on arXiv at 2501.00574 and was accepted at ICLR 2026. The repository is MIT licensed. ### Which VideoChat-Flash checkpoints are available? Five VideoChat2-Flash models are offered: a 2B at 224 resolution, a 7B at 224, a 7B at 448 described as overall best, a 7B with the InternVideo2 1B encoder and a 1M token context for super long video input, and a 7B with the InternVideo2 1B encoder for stronger short-term temporal understanding. The weights are on Hugging Face. ### How do I run inference or evaluation with VideoChat-Flash? The README directs inference to the Hugging Face model README for the model you want, with VideoChat-Flash-Qwen2_5-2B_res448 given as the example. Evaluation codes for the QA and Grounding benchmarks are in the lmms-eval_videochat directory, and lmms-eval supports the model for running it on various benchmarks. ### How is long video context tested in VideoChat-Flash? Through the needle-in-a-haystack evaluation, with separate single-hop and multi-hop suites and the code in xtuner-eval_niah. The README reports 99.1% accuracy on 10,000 frames and the ability to process videos up to three hours long. ## Sources - [Issues](https://github.com/OpenGVLab/VideoChat-Flash/issues) - [License: MIT](https://github.com/OpenGVLab/VideoChat-Flash/blob/main/LICENSE) - [OpenGVLab/VideoChat-Flash on GitHub](https://github.com/OpenGVLab/VideoChat-Flash) - [README](https://github.com/OpenGVLab/VideoChat-Flash/blob/main/README.md) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/opengvlab-videochat-flash