# Vidi: ByteDance video understanding models with three evaluation benchmarks

> Vidi is a family of large multimodal models from ByteDance for video understanding and creation, released in stages from a 7B model to Vidi2 and Vidi2.5 with a 9B checkpoint and finetune code. The repository ships three evaluation suites, VUE-STG, VUE-TR-V2 and VUE-PLOT, and a hosted demo covering grounding, retrieval, chaptering, highlights and question answering.

**bytedance/vidi** — The official repo for "Vidi: Large Multimodal Models for Video Understanding and Editing"

- Repository: https://github.com/bytedance/vidi
- Website: https://bytedance.github.io/vidi-website/
- Stars: 660 · Forks: 46
- Language: Python
- License: NOASSERTION
- Published: 2026-10-21 · Updated: 2026-10-21 · Language: en
- Canonical page: https://hysenlabs.com/projects/bytedance-vidi

## A model family built up in public, release by release

Vidi is a family of Large Multimodal Models for a wide range of video understanding and editing scenarios, and the README explains the trajectory. The first release focused on temporal retrieval, meaning identifying the time ranges in an input video that correspond to a given text query. The second release evolved toward a foundation model with state-of-the-art spatio-temporal grounding and temporal retrieval capability, while maintaining basic open-ended video question answering performance. The current headline is Vidi2.5, described as large multimodal models for video understanding and creation, and the paper is on arXiv as 2511.19529. The release history in the README shows the pace. The first release on 04/21/2025 consisted of a tech report and the VUE-TR evaluation benchmark, with a 7B demo and weights promised. A 7B demo followed on 06/06/2025, a Vidi1.5-9B demo on 08/29/2025, Vidi2 on 11/25/2025 with a report, code, homepage and demo, and Vidi2.5 on 01/20/2026 with an updated report, the repository, the demo, the VUE_PLOT benchmark and the Vidi1.5-9B weights with finetune code.

## The demo exposes six video analysis modes plus an edit page

The hosted demo has two pages, a vidi base page and a vidi-edit page, and the base page is organised as a segmented button of six modes. Grounding takes a text query naming an object and returns the matching clips with bounding boxes on that object. Retrieval takes a text query and returns the clips that correspond to it. Chapter needs no query and outputs a set of chapters with titles, and Highlight likewise needs no query and outputs highlight clips with titles. VQA takes a question or instruction about the video and answers it, and Thinking takes a question and thinks before answering. The README asks for English queries for the best experience, and for the upload step it recommends 480p for fast uploading and decoding, warns that the video should not be corrupted and that the resolution should not be too high, and says that results can take several minutes for a long video. The edit page is different in kind: you upload multiple videos and click generate, and it outputs an edited video with storyline, music and effects.

## VUE-STG evaluates spatio-temporal grounding with released annotations

The repository ships three evaluation suites, and the first covers spatio-temporal grounding. The video ids, the ground truth annotation and the evaluation results are released as csv files, and the instruction lives in the VUE_STG README. Running the published evaluation is two commands:

```bash
cd VUE_STG
python3 evaluate.py
```

To evaluate your own model the process has two steps. First you download the videos using the ids in the vue-stg-benchmark video.csv file, and the README points at yt-dlp as the way to do it. Then you generate results in the format shown by the released tubes.csv file for vidi2 and run the same evaluation script. The design choice worth noting is that the videos are not redistributed. The ids and the annotations are, and the corpus comes from YouTube, which means the benchmark can be regenerated but also that it depends on those videos staying reachable, and on a tool that can fetch them.

## VUE-TR-V2 measures temporal retrieval with an IoU-style metric

The second suite, VUE-TR-V2, covers temporal retrieval and is the one with the most explicit result format. The ground truth annotation and the evaluation results come in five json files, and the standalone evaluation runs a script:

```bash
cd VUE_TR_V2
bash install.sh
python3 -u qa_eval.py --pred_path results_Vidi.json
```

The result figures are written to an output folder, ./results by default, and the README shows the radar plot for IoU and the overall IoU plot. For evaluating a new model, you download the videos from the ids listed in the video_id.txt file, again with yt-dlp, run your own inference and save the results in the documented shape. Each entry carries a query_id, a video_id, a duration, the natural language query, an answer array of time ranges and a task field, with the example showing a single time range for a temporal_retrieval task. The previous version, VUE-TR, keeps its own instruction and data in a separate directory, which is convenient for anyone who ran the earlier numbers and wants to reproduce them.

## VUE-PLOT adds plot understanding with character and reasoning tracks

The third benchmark, VUE_PLOT, is about plot understanding, and it ships with two tracks, character and reasoning. The instruction is in the VUE_PLOT readme, and the evaluation is a separate script. Here the video acquisition has an alternative route that the other two suites do not offer. You can obtain the raw videos either using the YouTube video ids or, alternatively, by downloading them from the Condensed Movies dataset homepage at Oxford. That is a meaningful difference in reliability, since a dataset hosted on an institutional site does not depend on a video staying up on YouTube. The evaluation of your own model follows the same pattern as the others: generate the results and follow the instruction in the VUE_PLOT readme to finish. The benchmark came in with the Vidi2.5 release on 01/20/2026, which is the most recent release in the history and the one that added the report, the repository, the demo and the finetune code together.

## Weights and finetuning live in separate directories per model

The repository layout is organised by evaluation suite and by model, with VUE_PLOT, VUE_STG, VUE_TR and VUE_TR_V2 directories for the benchmarks and Vidi1.5_9B and Vidi_7B directories for the models. For inference and finetuning with Vidi1.5-9B, the README points at the instructions in the Vidi1.5_9B README, and for inference with Vidi-7B it points at the Vidi_7B README, with the model weights themselves hosted on Hugging Face under the bytedance-research organisation. The model documentation is not in the main README, which is a deliberate split: the main file is an index of what exists and where, and the per-model directories hold the operational detail. The top-level entries are otherwise minimal, a LICENSE.txt, the README and those six directories, so the repository is a container for artefacts rather than a Python package, and the only primary-language code is the evaluation scripts and the finetune code inside the model directories.

## Licensing, releases and the state of the tree

Two things to note about the project as an artefact. The repository reports a custom licence that GitHub cannot classify, and the README does not restate the terms, so the LICENSE.txt file at the root is the document to read before you use the weights or the code in a product. There are no GitHub releases, which means there are no tagged artefacts and no release notes attached to tags, and the version history lives in the README news list and in the arXiv paper. The last push to the repository was on 2026-08-03, a little under three months before this writing, so the tree is not abandoned, but the most recent published model release named in the README is from 01/20/2026. The project is a research release with a demo, weights, finetune code and public benchmarks, which makes it reproducible in the way a paper artefact should be, and it makes it a moving target in the way a paper artefact usually is. Anyone building on it should treat the checkpoints as snapshots and read the version-specific README before assuming an API.

## Conclusion

Vidi fits a research team that needs a video language model with published weights, a public demo and reproducible evaluation suites, especially one working on temporal retrieval or spatio-temporal grounding. It does not fit an application that needs a stable production API, since the releases are point-in-time checkpoints with no GitHub releases, and the model runs are slow enough that the demo documentation warns about minutes of waiting for long videos. Before adopting it, read the inference instructions for the checkpoint you intend to use, run the VUE-TR-V2 evaluation script on the published results to see the numbers yourself, and check the LICENSE.txt file, since the repository reports a custom licence rather than a standard one.

## FAQ

### What is Vidi and what tasks does it support?

Vidi is a ByteDance family of large multimodal models for video understanding and creation. The first release focused on temporal retrieval, the second added spatio-temporal grounding, and the current Vidi2.5 release adds creation alongside understanding.

### How do I evaluate Vidi on the released benchmarks?

The VUE-STG suite runs cd VUE_STG followed by python3 evaluate.py, and the VUE-TR-V2 suite runs cd VUE_TR_V2, bash install.sh and python3 -u qa_eval.py --pred_path results_Vidi.json, with results written to ./results by default. VUE_PLOT has its own readme and script.

### Where do the Vidi model weights and finetune code live?

The weights are hosted on Hugging Face under bytedance-research for Vidi1.5-9B and Vidi-7B. The README points inference and finetuning instructions at the Vidi1.5_9B README and the Vidi_7B README directories inside the repository.

### Which video resolutions does the Vidi demo handle best?

The README recommends 480p for fast uploading and decoding, and advises that the video should not be corrupted and the resolution should not be too high. Results for long videos can take several minutes to appear.

## Sources

- [bytedance/vidi on GitHub](https://github.com/bytedance/vidi)
- [Issues](https://github.com/bytedance/vidi/issues)
- [Project website](https://bytedance.github.io/vidi-website/)
- [README](https://github.com/bytedance/vidi/blob/main/README.md)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/bytedance-vidi