Speech Trident: a curated index of speech representation, codec, and speech LLM papers
Awesome speech/audio LLMs, representation learning, and codec models
At a glance
- What is it?
- Speech Trident is not a library or a model. It is a survey repository that splits speech and audio language model research into three tracks (representation learning, neural codecs, speech LLMs) and lists papers under each. This article covers what it actually contains, how the listing is organised, where it stops being useful, and what to check before you rely on it.
- Who is it for?
- Adopt Speech Trident if you need a starting point for reading in speech representation learning, neural codecs, or speech LLMs, and you are willing to follow the arXiv links yourself. Do not adopt it if you need runnable code, versioned releases, or an installable package; the repository supplies none of those.
- Can I use it commercially?
- Not without permission. GitHub finds no licence file in the repository, and without a licence all rights are reserved by default: you may read the code but not reuse it. Check the README, or ask the authors, before using it.
- Is it still maintained?
- Yes. The repository last received commits 67 days ago.
- What is it written in?
- GitHub does not report a main language for this repository.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What problem a paper index solves, and for whom
Speech and audio language model research moves across three communities that publish in different venues and use different vocabulary. Representation learning papers talk about semantic tokens. Codec papers talk about acoustic tokens and bitrate. Speech LLM papers talk about decoding patterns and instruction tuning. A reader trying to build a speech LLM has to pull from all three, and the README frames the repository as a survey of exactly those three areas. The stated audience is implicit rather than spelled out: the contributors are academic researchers (Kai-Wei Chang, Haibin Wu, Wei-Cheng Tseng, Kehan Lu, Chun-Yi Kuan, Hung-yi Lee), and the artefact is a reading list, not a toolkit. If you are a graduate student or an engineer starting a speech LLM project and you need to know which papers define the current design space, the repository is aimed at you. If you want to fine-tune a model this afternoon, it is not.
The three-track structure and how the paper tables are built
The README defines three tracks. Speech representation models learn structural speech representations that can be quantized into discrete tokens, which the README calls semantic tokens. Speech neural codec models learn discrete tokens for speech and audio, called acoustic tokens, while keeping reconstruction ability at low bitrate. Speech large language models are trained on top of speech and acoustic tokens in a language modeling approach, and the README attributes proficiency in speech understanding and speech generation to them. The main body is a table with four columns: date, model name, paper title, and a link column that sometimes holds a paper link and sometimes also a code link. Entries are ordered newest first, starting at 2025-07 with Audio Flamingo 3 and DeSTA2.5-Audio and running back through SLED, BALSa, Fun-ASR-Nano, Kimi-Audio, Qwen2.5-Omni, Yue, CSM, Phi-4-Multimodal, Baichuan-Audio, and DiTAR. The link column is the only navigation mechanism. There is no per-track table, no tag column, and no indication of which of the three tracks a given row belongs to. That is the central structural weakness: the three-track taxonomy is described in prose at the top and then abandoned in the table itself.
The 2025 survey paper is the deeper version of this repository
The README devotes a section to a 2025 arXiv paper, On The Landscape of Spoken Language Models: A Comprehensive Survey (2504.08528), by Arora, Chang, Chien, Peng, Wu, Adi, Dupoux, Lee, Livescu and Watanabe. The repository's own text says the paper covers a lot of the speech and audio language models surveyed in Speech Trident, but with more detailed and technical discussion. The paper uses a different three-way split: pure speech LM, speech-aware text LM, and speech plus text LM. It also covers training strategies, speech and text token decoding patterns, duplex speech dialogue, and benchmarks. This matters for anyone deciding how to spend their time. The repository is a set of links with dates and titles. The paper is where the taxonomy, the comparisons, and the benchmark discussion live. If you only read the repository, you get the list without the reasoning behind it. The README points you to the paper directly, which is an honest arrangement, but it also means the repository's own contribution is thinner than its framing suggests.
There is nothing to install, and that changes the evaluation
The repository has no primary language listed, no license listed, no homepage, and no releases retrieved. The README contains no installation instructions, no setup commands, no configuration keys, and no CLI. That is consistent with what it is: a Markdown file with tables and image assets (the README references assets/Speech-Trident-v4.png and links to a timeline image in a sibling repository). So the usual adoption questions do not apply. You cannot pin a version, you cannot run a test, and you cannot check a changelog. What you can check is whether the links resolve and whether the listed papers match the titles given. The absence of a license is a real consideration if you intend to reuse the tables or the survey text in your own material, because with no license file the default position is that no reuse rights are granted. That is a factual observation about the repository state, not legal advice; if reuse matters, ask the maintainers.
Where the coverage stops
The table is a snapshot of models the contributors chose to include, and the README gives no inclusion criteria. There is no stated cutoff date, no explanation of why one 2025 paper appears and another does not, and no marking of which entries have public code. The link column mixes paper-only rows with rows that carry both a paper and a code link, as with Fun-ASR-Nano, but nothing in the table distinguishes them at a glance. Rows with an empty model name exist, such as the 2025-04 entry for the survey paper itself, which sits inside the model table even though it is not a model. The README is also truncated in the supplied material at the Slamming entry, so the lower portion of the table cannot be assessed here. Treat the list as a starting bibliography, not a completeness claim. If your work depends on a specific model family and it is not in the table, that tells you nothing about whether the model exists or matters.
Compared with an arXiv listing or a papers-with-code style index
The obvious alternative is to search arXiv directly, or to use a paper index that attaches code repositories and reported results to each entry. The difference in approach is editorial. Speech Trident applies a human filter: a small group of researchers decided these papers belong together under a speech LLM framing, and the three-track description at the top explains the reasoning. An automated listing applies no such filter; it returns everything matching a query, in date order, with no grouping. The trade-off runs both ways. The curated list gives you a coherent starting set and saves you from sorting signal from noise on a first pass. The automated index gives you coverage and, in some cases, code links and benchmark tables that Speech Trident does not maintain. For a first reading list, curation wins. For a literature review that has to be defensible, you need the broader index as well.
Maintenance signals and what to verify before you rely on it
The repository is not archived and the last push is dated 2026-07-10, so it is being touched. The README carries 2026 and 2025 news sections, including a pointer to a separate spoken dialogue model survey repository and a 2026 citation (chang2026tico, arXiv 2603.22267), which indicates the maintainers keep adding material. There are no releases, so there is no version to track and no upgrade path to plan. The cost of using it is the cost of re-reading the README when you return, because the table changes in place. Two things are worth verifying before you build anything on top of it. First, whether each link still resolves, since link rot is the standard failure mode for this kind of repository and nothing here validates the URLs. Second, whether a given row points to a paper, to code, or to both, because the table does not label that distinction. Both checks take minutes and prevent the common mistake of citing a model whose code you never actually located.
Editorial conclusion
Adopt Speech Trident if you need a starting point for reading in speech representation learning, neural codecs, or speech LLMs, and you are willing to follow the arXiv links yourself. Do not adopt it if you need runnable code, versioned releases, or an installable package; the repository supplies none of those. Before relying on it, open the table entries for the models you care about and confirm each link still resolves, then check whether the 2025 arXiv survey it points to (2504.08528) already covers the same ground in more depth, since the README itself says the paper goes further than the repository does.
Community notes