SALMONN Is a Branch Farm, Not a Package: What the bytedance/SALMONN Repo Actually Contains
SALMONN family: A suite of advanced multi-modal LLMs
At a glance
- What is it?
- The default branch of bytedance/SALMONN is an index page for a family of audio and audio-visual LLMs, each living on its own branch or in a sibling repository. Adopting SALMONN means picking a branch, not installing a library.
- Who is it for?
- Adopt SALMONN if your work is research-oriented audio or audio-visual modelling and you are willing to clone a specific branch and reconcile its dependencies yourself. Do not adopt it if you need a versioned package with a stable API, a release channel, or a support commitment, because the repository provides none of those on the default branch.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 23 days ago.
- What is it written in?
- GitHub does not report a main language for this repository.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What the default branch of bytedance/SALMONN actually is
The repository's default branch is a landing page. Its README opens with the line "The SALMONN model family consists of a series of advanced multi-modal large language models" and then does almost nothing except point elsewhere: SALMONN 2, ELLSA, video-SALMONN 2, F-16, video-SALMONN-o1, the speech quality assessment work, video-SALMONN, and the original SALMONN. Each entry is a link to a branch inside this repository or to a separate bytedance repository. If you clone the default branch expecting runnable inference code, you will not find it there. The material does not describe the default branch as containing a shared library, a common Python package, or a unified entry point, and nothing in the README suggests the branches share code. Treat the repository as a directory of research projects that happen to live under one URL.
Who each branch is aimed at, and who it is not
The audience is narrow and academic. The paper list names ICLR 2024 for the original SALMONN, ICML 2024 for video-SALMONN, ICML 2025 for F-16 and video-SALMONN-o1, ICASSP 2025 and ACL 2025 for the speech quality assessment line, ICLR 2026 for ELLSA, plus an arXiv preprint for video-SALMONN 2. That citation pattern tells you the intended consumer: someone reproducing or extending a paper, not someone shipping a voice interface. The tasks are specific. SALMONN targets what the ICLR paper title calls "generic hearing abilities". video-SALMONN and video-SALMONN 2 add the visual channel, with the news entry describing video-SALMONN 2 as generating "high-quality audio-visual video captions". ELLSA is described as "the first end-to-end model that unifies vision, speech, text and action in a streaming full-duplex framework". The speech quality assessment branch is the outlier with a practical framing: the QualiSpeech dataset is offered so you can "develop your own audio LLM for speech quality assessment or to evaluate the low-level speech perception capabilities of existing audio LLMs". If your problem is transcribing a call centre recording, none of this is addressed to you.
How the family is organised: branches and sibling repositories
There are two distribution mechanisms in play, and the distinction matters when you go to fetch code. Some models live on branches of this repository: salmonn2, ELLSA, video-salmonn-o1, speech_quality_assessment, videosalmonn, and salmonn. Others live in separate bytedance repositories entirely: video-SALMONN 2 and F-16. The README links them all in one flat list, so the first practical step is to determine which of the two you are dealing with. Branch-based distribution means the code for a given model is frozen at whatever state that branch is in; there is no tag or release in the supplied material, and the releases field for this repository came back empty. The news entries give dates for when code and checkpoints appeared (2023-10-08 for the SALMONN-13B checkpoint and inference code, 2024-04-07 for "all the codes you need to train your own SALMONN", 2024-09-04 for video-SALMONN, 2026-04-20 for ELLSA), which is the closest thing to a changelog here. Plan on reading the branch you need rather than the default branch.
Getting it running: what the material specifies and what it does not
The README does not contain installation instructions, a requirements file listing, or an inference command. It gives you links and nothing else. What it does specify concretely is where artefacts live. SALMONN-13B's model checkpoint and inference code were released on 2023-10-08, and the 7B version is at tsinghua-ee/SALMONN-7B on Hugging Face, with a Gradio demo at tsinghua-ee/SALMONN-7B-gradio. The QualiSpeech dataset is at huggingface.co/datasets/tsinghua-ee/QualiSpeech. Training annotations, described as "600k SQA/AQA data and 50k audio-based storytelling data" for the 3-stage training of SALMONN, are distributed through a Google Drive link. That is the full extent of the deployment surface described here. Any command you run, any config key you set, comes from the individual branch README, not from this page. I have not run any of it, and I am not going to invent a pip line or a Python entry point that the material does not contain.
The three-stage training pipeline and its data dependency
The one architectural detail the default branch states outright is that SALMONN was trained in three stages, and that the annotations for all three were released together on 2024-05-28. The composition given is 600k SQA/AQA examples and 50k audio-based storytelling examples. SQA and AQA are not expanded in the README, so if you are trying to reconstruct the recipe, you are reading the paper and the released annotation files to work out what those acronyms cover. The practical consequence is that training your own variant is gated on a Google Drive download rather than a package index, and the data volume implies a training budget that the README never quantifies. Nothing here states GPU counts, hours, or memory requirements for any stage. If you are capacity planning, that information is not in this repository's default branch and you should not assume it is small because the parameter count is 7B or 13B.
Where SALMONN is the wrong choice
The clearest failure mode is expecting a maintained dependency. There are no releases retrieved for this repository, the default branch is an index, and the code for each model sits on a branch that was written to accompany a paper. If a branch's pinned dependency versions conflict with your environment, you own that conflict. A second limitation is scope creep in the opposite direction: because the family spans speech, music, audio-visual video, speech quality assessment, and in ELLSA's case action, it is tempting to assume one branch handles all of it. It does not. The README presents them as separate models with separate papers, separate checkpoints, and in two cases separate repositories. Third, the streaming full-duplex claim attached to ELLSA is the newest entry in the list, dated 2026-04-20, and the README offers one sentence about it. If your use case depends on the streaming behaviour, that sentence is all you have to evaluate before cloning.
How this differs from a general-purpose audio model stack
The obvious comparison is a speech recognition toolkit such as Whisper or a general audio tagging pipeline, and the difference is not accuracy, it is the shape of the output. Those systems are built to emit a transcript or a label. SALMONN's premise, per the ICLR paper title, is hearing abilities routed through a language model, so the output is generated text conditioned on audio, and in the video branches, on audio plus frames. That is what makes the speech quality assessment branch interesting as a contrast: instead of predicting a MOS score, QualiSpeech pairs quality assessment with "natural language reasoning and descriptions", so the model explains a judgement rather than emitting a number. If your downstream system needs a scalar, you are adding a parsing layer that a dedicated regression model would not require. If your downstream system needs a reason, the generative framing is the point. Pick accordingly rather than by benchmark tables you cannot reproduce.
Licence, maintenance, and what you are signing up for
The repository is Apache-2.0, which is a permissive licence and, on its face, friendly to commercial use. Two caveats follow from the structure rather than the licence text. First, the licence covers what is in this repository; the checkpoints and datasets hosted on Hugging Face and the annotations on Google Drive are separate distributions with their own terms, and the README does not state what those terms are. Second, two of the models in the family are not in this repository at all, so their licensing is whatever their own repositories say. On maintenance: the last push recorded for this repository is 2026-08-24, and the news list shows a steady cadence of releases stretching from October 2023 to April 2026, so the project is active. Active is not the same as supported. There is no versioned release channel in the supplied material, which means upgrading means re-reading a branch and diffing it yourself. Budget for that, and check the licence files attached to the specific checkpoint and dataset you plan to use before you build on them. This is a description of what the material says, not legal advice.
Editorial conclusion
Adopt SALMONN if your work is research-oriented audio or audio-visual modelling and you are willing to clone a specific branch and reconcile its dependencies yourself. Do not adopt it if you need a versioned package with a stable API, a release channel, or a support commitment, because the repository provides none of those on the default branch. Before committing, verify three things: which branch or sibling repository matches your modality, whether the checkpoint you intend to use is published on Hugging Face for that specific model, and what the branch's own README says about training data and compute, since the default branch does not restate any of it.
Community notes