liquid-audio: a 1.5B speech-to-speech model you can fine-tune

Two generation modes, and why the difference matters

LFM2-Audio-1.5B is described as Liquid AI's first end-to-end audio foundation model, built with low-latency in mind, with a lightweight LFM2 backbone that the README says enables real time speech-to-speech conversations without sacrificing quality.

The interesting design choice is that the model supports two generation modes, interleaved and sequential, to maximize performance and quality across different tasks.

Interleaved generation outputs text and audio tokens in a fixed interleaved pattern. The README says this minimizes time to first audio output and the number of tokens generated, and that it makes the model ideal for naturally flowing real-time speech-to-speech interactions on resource constrained devices.

Sequential generation is the opposite in structure. There the model decides when to switch modalities via special tokens, and the README says it is suitable for non-conversational tasks, such as speech-to-text or text-to-speech.

That split is the engineering content of the release. For a voice agent, the cost that matters is how long the user waits before hearing anything, and fixed interleaving is the cheapest way to get that down, because the model does not have to decide it is done thinking before it starts speaking. For an ASR pipeline, that latency does not matter and the sequential mode gives a cleaner separation, because the whole output is text and the special token tells the model when to hand over.

Both modes are reached through the same class, with the methods LFM2AudioModel.generate_interleaved and LFM2AudioModel.generate_sequential, and both are generators that yield torch.Tensor objects.

Tokens, tensors and the split between model and processor

The API divides cleanly into a model that works on tokens and a processor that converts between tokens and data, and the detail that makes this concrete is the tensor shapes.

Text tokens are represented by tensors with 1 entry, and audio tokens are tensors with 8 entries, corresponding to 8 Mimi codebooks. So a single yielded tensor tells you which modality it came from by its size alone, and the example code branches on exactly that, checking whether t.numel() is 1 to decide whether to decode text or to collect it as audio.

The LFM2AudioModel class operates on tokens only, and the LFM2AudioProcessor class is used to convert between tokens and data. For text that means the conversion from string to tokens and back. For audio inputs it handles the conversion of waveforms to log-mel features, and for audio outputs it handles the detokenization of audio tokens to waveform.

Separating those two concerns is what makes the library usable for both training and inference, because a fine-tuning script needs the token side and a deployment needs the data side.

The third piece is the ChatState helper class, which exists to facilitate the creation of inputs for the generation methods and to apply the correct chat templates. It is the object that tracks the turns, and the API around it is turn based, with new_turn, add_text, add_audio, end_turn and an append call for writing generated tokens back into the history. The system prompt for a multi-modal conversation is set to a specific string, Respond with interleaved text and audio, which tells the model which mode it is in.

The result of a generation loop is a list of tensors for the text, a list for the audio and a list of LFMModality values recording which was which, and the history append takes those three.

Finetuning, the Japanese variant, and the detokenizer change

Three changes define the current release line, and they matter for what the package can be used for.

The Japanese version of the model, LiquidAI/LFM2.5-Audio-1.5B-JP, is released, and the README calls it the first Japanese speech-to-speech model of this scale, with instructions in a separate README_JP.md. The version 1.3.0 release is titled JP model support.

Finetuning is now supported in both interleaved and sequential generation modes. Version 1.2.0 introduced data preparation tools and a lightweight trainer, and the README says this enables users to fine-tune models for a broad range of tasks, from ASR and TTS to function calling and end-to-end speech-to-speech chat. The repository carries examples/train.py and examples/preprocess_jenny_tts.py, so the preprocessing and the training loop are both shown rather than just described.

The third change is the LFM2.5-Audio-1.5B model, which is based on the stronger LFM2.5-1.2B base and comes with a lightning fast LFM2 based audio detokenizer, stronger ASR and better TTS voices. The detokenizer change has a practical consequence for existing code, since to use it you simply call processor.decode, and the example comment shows why the change matters. The code stacks the audio output tokens, drops the last one because it is an end-of-audio code, unsqueezes it and decodes, and the comment notes that Mimi returns audio at 24kHz.

That last detail is the kind of thing that silently breaks a pipeline if it is wrong. The write call uses 24_000 as the sample rate, and if the detokenizer changes the output rate, the file is written with the wrong metadata and plays back at the wrong speed.

Installing, the demo, and the AMD path

The package installs through pip, and there are three optional extras worth knowing about.

pip install liquid-audio
pip install "liquid-audio [demo]" # optional, to install demo dependencies
pip install flash-attn --no-build-isolation  # optional, to use flash attention 2. Will fallback to torch SDPA if not installed

The third line matters if you care about latency. flash attention 2 is optional, and the note says it will fall back to torch SDPA if it is not installed, so the package works without it but the attention implementation differs.

For AMD ROCm installations, the README says not to forget to specify the correct pytorch version and index:

pip install liquid-audio torch==2.12.0+rocm7.2 --extra-index-url https://download.pytorch.org/whl/rocm7.2

The Gradio demo is the fastest way to see the model working. You install the extra dependencies in the demo group, then run the liquid-audio-demo command on the terminal, and the interface is available at http://localhost:7860. The console script is declared in the manifest as liquid-audio-demo = "liquid_audio.demo.chat:main [demo]", and the demo extra pulls in fastrtc with the vad option, which is what handles voice activity detection in the live audio path.

The runtime dependencies are the usual speech model stack: torch and torchaudio at 2.8.0 or newer, transformers at 4.55.4 or newer, librosa for audio loading, sentencepiece for the text tokenizer, einops for tensor manipulation, datasets for the finetuning path, and accelerate for the training loop. Python 3.12 or newer is required, and the build backend is uv_build.

A custom licence and a model that lives on Hugging Face

The licensing here is not an open source licence, and that is the first thing to settle before anything else.

The manifest records the licence as LicenseRef-LFM-Open-License-v1.0, with the LICENSE file listed under license-files, and the repository is marked as carrying a custom licence that GitHub cannot classify. The name is what you would expect for a model release from a commercial lab, and the practical effect is that the terms are whatever that document says rather than one of the standard permissive licences, so it needs reading before use in a product.

The weights themselves are not in this repository. The models are on Hugging Face, with the base model referenced as LiquidAI/LFM2.5-Audio-1.5B, and the code loads them with the familiar from_pretrained calls followed by eval.

The repository layout is small and conventional for a Python package. There is src/ for the library, examples/ with the training and preprocessing scripts, assets/ with the audio files the examples read, a README.md and a README_JP.md, a .python-version file, a pyproject.toml and a uv.lock. The .github/ directory holds the workflows.

Two details in the tooling configuration are worth noting. The ruff configuration sets a line length of 127, with a comment explaining that the GitHub editor is 127 characters wide, which is a small sign of how the project is developed. And the lint selection is a long explicit list of rule families, from pycodestyle and Pyflakes through pyupgrade, bugbear, simplify, isort, tryceratops, perflint, refurb and ruff itself, with a documented ignore list for rules that conflict with the formatter.

The release history is v1.1.0 on 2026-01-06, v1.2.0 on 2026-05-08 and v1.3.0 on 2026-06-05, and the last push was on 2026-10-03, so the package is being maintained at roughly a release every two to four months.

Editorial conclusion

liquid-audio suits a research team that needs an end-to-end speech-to-speech model small enough to fine-tune and deploy on a single accelerator, and the interleaved mode with its low time to first audio is what makes a real-time voice agent plausible rather than a demo. It does not suit you if you need a streaming-first architecture with the scale of a frontier API, since a 1.5B model trained on a fixed corpus will not match one, and the model weights are on Hugging Face under a custom licence rather than an open one. Before you download anything, read the LFM Open License in the repository and check the Python floor, which is 3.12.

Frequently asked questions

Official sources

1. Issues
2. Liquid4All/liquid-audio on GitHub
3. Project website
4. README
5. Releases