MOSS-Transcribe-Diarize: one negative number in a table of losses

One pass instead of a stitched pipeline

The design decision is stated in the introduction as a rejection of the usual arrangement. Rather than running a speech recognition system and a separate speaker diarization system and joining their output afterwards, this model performs both jointly, so the text, the timing and the speaker labels come out of a single forward pass and cannot drift against each other. The intended inputs are the awkward kind: meetings, calls, podcasts, interviews, lectures and video, described as long and messy with several speakers. Speaker labels are written inline as bracketed tags that increment, so the first voice is one tag and the second another, and the model can additionally emit acoustic event annotations, which is what lets a downstream system know that something happened rather than only that somebody said it. Fifty or more languages are supported.

[start_time][Sxx]transcribed speech[end_time]

Two bracketed numbers in a row are your silence

The canonical format is worth reading closely because it encodes timing in a way that costs the parser work. Each segment is a start time, a speaker tag, the text, and an end time, with all times in seconds, and adjacent segments are concatenated into one unbroken stream with no separator between them. The worked example makes the consequence visible.

[0.48][S01]Welcome everyone[1.66][12.26][S02]The new transcription pipeline is ready for evaluation[13.81][14.36][S01]Great, include the diarization results in the report[18.76]

The first segment ends at 1.66 and the second begins at 12.26, so ten seconds of silence appear as two numbers sitting next to each other with no text between them. Nothing marks the gap, so any consumer has to compute it, and because there are no spaces in the stream, a parser has to split on the brackets rather than on whitespace. The first segment also ends at 1.66 while the second starts at 12.26, meaning timestamps are per segment and not contiguous, which is what makes a gap detectable at all.

A Whisper encoder joined to a Qwen3-scale decoder

The architecture table is six rows and it reads as a shopping list of parts rather than a novel design, which is a compliment for a 0.9B model that has to handle hour-long recordings. The text backbone is a Qwen3-0.6B style causal decoder. The audio encoder uses a Whisper-Medium encoder configuration, and the frontend is the standard Whisper feature extractor at 16 kHz with 80 mel bins and 30-second chunks. The two halves meet at an audio-text bridge built from a four-times temporal merge and a multilayer perceptron adaptor, and the fusion step is explicit: audio features take the place of the decoder's audio placeholder embeddings through a masked scatter operation. That four-times merge is the mechanism that makes 30-second chunks affordable to feed through a small decoder, and it is why the frontend can stay at the standard Whisper settings.

The gap metric goes negative for exactly one row

Three numbers are reported for every model on every test set, and the third one is derived from the first two rather than measured separately, which is worth working out because it changes how the table reads. Subtracting the plain character error rate from the permutation-corrected version gives the third column in every cell of the table, and a positive value means that reassigning speakers made the transcription worse than ignoring speakers altogether. Most rows are positive, often badly so: one competitor goes from 21.40 to 48.30 on the podcast set for a gap of 20.36. The open MOSS row is the exception, and on the meeting set its corrected score is better than its plain score, which is a gap of minus 2.69: 24.86 plain against 22.17 corrected. On the first test set the same model sits at 14.84 plain and 15.83 corrected, a gap of just under one point.

The open model wins one cell against its own stronger sibling

The table compares eight rows across four test sets, and bold marks the best result with an underline marking second, which makes the shape of the comparison easy to read. The stronger Pro model takes the best score in eleven of the twelve cells. The exception is the podcast diarization gap, where the open 0.9B model is best at 1.40 and the Pro version is second at 2.51, so on the one measure that isolates speaker assignment on conversational audio the smaller released model wins. The open model is also second on six other cells and best nowhere else. Two rows are incomplete in ways that matter for reading the table: one frontier model has dashes for the first three test sets and only reports the movie set, and another has dashes for the podcast set entirely, so the comparison covers an uneven set of systems per column.

A first place in a fourteen-language challenge, five days after release

The news section is three dated lines and they tell a compressed story. The model was open-sourced on 2026-07-09. Five days later, on 2026-07-14, it took first place in the second edition of a challenge held at a speech conference, and that challenge covered fourteen languages. Two weeks after that, on 2026-07-22, the subtitle web interface added Simplified Chinese alongside English. The gap between fourteen languages in the competition and fifty or more in the model is worth keeping in mind when reading a challenge result as a general quality claim, since the breadth claim is not what was measured. The other thing the page states plainly is that a stronger model exists and is available only through an online playground, which is the honest framing for an open release that is not the best thing the team has.

Torch is an optional extra, and the web server is not

The packaging tells you what a user actually has to assemble. Torch and torchaudio are not dependencies, they sit in an optional extra named for a torch runtime at version 2.8 or newer, and an accelerated attention package is a second extra of its own. So a plain install gives you the transformers library and no way to run the model. The dependencies that are marked required tell the other half of the story: a web framework, an application server and a multipart parser are all core, because the project ships two console entry points and one of them serves a subtitle application over HTTP. The package also ships its static files and its locale JSON files as package data, which is how the bilingual interface in the news section is delivered.

[project.scripts]
mtd-subtitle = "moss_transcribe_diarize.app.cli:main"
mtd-subtitle-web = "moss_transcribe_diarize.app.web_cli:main"

Python 3.10 is the floor, transformers is capped below its next major, and numpy is capped below 3.

The page ends before the quickstart starts

The contents list promises ten sections and the document delivers four of them. Introduction, architecture and the evaluation are there in full, and then the file stops on a heading reading Quicksta, mid-word. Everything a new user needs is in the missing part: environment setup, Python usage, serving with one inference engine and another, use inside a third-party speech ecosystem, custom prompts and hotwords, and the subtitle web application. What survives of that work is visible in the repository instead, where a finetuning script and a finetuning document sit at the root and an examples directory holds a prompts document, so the promised prompt and hotword features exist in some form. The model weights are not in the repository at all: the recorded homepage is the model page on a model hub, and the repository itself has never cut a release.

Editorial conclusion

This is a model worth trying if your recordings are the hard kind, long and multi-speaker, because doing transcription and speaker assignment in one pass is the right architecture for that and the diarization metric in its own table is the evidence. Three caveats to weigh. The strongest version of the model is not open, and the one you can download is measured against it on the same table, where it loses eleven of twelve cells. Torch is not a dependency, it is an extra, so plan your install around that rather than discovering it at first run. And every number on the page is self-reported by the team, with two frontier models left as dashes in several columns.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. OpenMOSS/MOSS-Transcribe-Diarize on GitHub
4. Project website
5. README