FlashLabs Chroma: a speech-to-speech model with zero-shot voice cloning

What the model is

Chroma 1.0 is described in the repository title as a real-time end-to-end spoken dialogue model with personalized voice cloning, and the README's own description is an advanced multimodal model developed by FlashLabs, designed to understand and generate content across multiple modalities including text and audio. As a virtual human model, Chroma can process auditory inputs and respond with both text and synthesized speech, which is what enables natural voice interactions.

The word end-to-end matters here. The alternative architecture is a pipeline: speech to text, then a language model, then text to speech. That pipeline is easier to build and to control, and it is what most voice assistants are. End-to-end means the audio is the model's native input and output, so the whole exchange is one model producing one continuation, and the latency of the pipeline's boundaries disappears.

The model description block gives the formal details. The model type is a multimodal causal language model, developed by FlashLabs, with English as the language and Apache-2.0 as the licence. Then the architecture, which is the part to read carefully because it explains what the model actually is built from.

The reasoner is based on Qwen2.5-Omni-3B. The backbone is based on Llama3 with 16 layers and a hidden size of 2048. The decoder is based on Llama3 with 4 layers and a hidden size of 1024. And the codec is Mimi at a 24 kHz sampling rate.

So the design is a reasoner that handles the multimodal input and reasoning, a Llama3 backbone that carries the main computation, a much thinner Llama3 decoder for the output, and a Mimi codec to turn between audio tokens and waveform. The asymmetry between the 16-layer backbone and the 4-layer decoder is the interesting design decision, and it says the heavy work happens in understanding and reasoning, while generation needs less capacity.

The primary language of the repository is recorded as Jupyter Notebook, which is accurate in the sense that the example is a notebook. The licence is Apache-2.0, the default branch is main, and the homepage is the FlashAI Voice Agents product page. The last push was on 2026-09-04 with the project not archived, and the repository publishes no GitHub releases. The model itself is on Hugging Face as Chroma-4B, and there is a technical report on arXiv.

The capabilities, and what voice cloning actually means here

Three capabilities are listed, and they are the three that matter for a voice agent.

Speech understanding is processing user audio input directly. Multimodal generation is generating coherent text and speech responses simultaneously. And voice cloning is utilizing reference audio prompts to guide speech generation style.

The phrasing of the third one is precise and worth not glossing over. It is reference audio prompts, and the guide is speech generation style. The usage example makes it concrete: the process takes a reference audio file and a matching text file, both loaded from disk by a helper function that reads the prompt text for a named speaker and the prompt audio for the same speaker.

So the reference is a paired clip and transcript, and the model learns from that pair how to speak. There is no training step, no per-speaker weights, no fine-tuning run. The topic list on the repository includes zero-shot-voice, which is the term for exactly this: a voice that was never seen during training.

That is the capability with the most care needed around it, and the README does not address it at all. Voice cloning from a short reference clip is the kind of feature that can be used to build a convincing impersonation of somebody, and the fact that the model is Apache-2.0 and freely downloadable means the safety question is not handled by access control. Anyone running this is responsible for what they do with it, and the reference clip for the example is a speaker in the repository's own example directory, which is the right default.

One more thing to notice about the reference mechanism. Because the reference is a text-and-audio pair rather than audio alone, the model is being given both what was said and how it was said, which is a stronger conditioning signal than audio alone and explains why the paired files exist at all.

The other capability, simultaneous text and speech output, is what makes the model useful for a chat interface rather than only for a voice one. A voice agent that also emits text can show a transcript, and a text agent that also emits audio can speak, and one model doing both avoids the need to run two.

Installing it, and the dependency ordering trap

The requirements are Python 3.11 or higher and CUDA 12.6 or a compatible version for GPU support, and the install is a clone, an optional conda environment and a pip install:

# Clone the repository
git clone https://github.com/FlashLabs-AI-Corp/FlashLabs-Chroma.git
cd FlashLabs-Chroma

# Optional: Create a new conda environment with Python 3.11
conda create -n chroma python=3.11 -y
conda activate chroma

# Install dependencies in the correct order
pip install -r requirements.txt

The requirements file is where the real constraints are, and the versions are specific in a way that matters:

# Core dependencies
torch==2.7.1
torchvision==0.22.1
torchaudio==2.7.1
transformers==5.0.0rc0

# Audio processing
av>=14.0.0
librosa>=0.11.0
audioread>=3.0.0
soundfile>=0.13.0

# Other dependencies
pillow>=11.0.0
accelerate>=1.7.0
numpy>=2.2.0
safetensors>=0.5.0
huggingface-hub>=1.3.0

Torch, torchvision and torchaudio are all pinned to the 2.7.1 line, and transformers is pinned to 5.0.0rc0, which is a release candidate. That is a slightly unusual choice for a released model, but it is not surprising for a model that uses trust_remote_code to run custom modelling code, since the custom code may target the newest library API.

The audio stack is av at 14 or newer, librosa at 0.11 or newer, audioread at 3 or newer and soundfile at 0.13 or newer. So there are four separate audio libraries, which is more than one would expect, and the reason is that they cover different gaps: av for decoding containers, librosa for analysis, audioread for a fallback decoder and soundfile for writing.

The troubleshooting section explains why the ordering matters, and the explanation is the part to take away. A TypeError: argument of type 'NoneType' is not iterable occurs when loading the processor if torchvision is not properly detected during the transformers module initialisation. The solution is to ensure you install packages in the correct order, PyTorch and torchvision before transformers, and if you have already installed them, to reinstall in the correct order:

   pip uninstall transformers torchvision torch -y
   pip install torch==2.7.1 torchvision==0.22.1 torchaudio==2.7.1 --index-url https://download.pytorch.org/whl/

So the failure mode is a confusing TypeError during processor loading, and the fix is an uninstall and reinstall from the PyTorch index URL. This is the classic conflict between a library that does a feature detection at import time and an optional dependency that was installed too late or not at all, and it is worth knowing about before you hit it.

The requirements file being grouped into core dependencies, audio processing and other dependencies, with comments in each, is a small sign of care that makes the ordering constraint visible in the file rather than only in the troubleshooting section.

Loading the model and running inference

The usage section gives two code blocks, and they show the shape of the API.

Loading is a standard Hugging Face pattern:

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

model_id = "FlashLabs/Chroma-4B" # Or local path

# Load model
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    trust_remote_code=True, 
    device_map="auto"
)

# Load processor
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)

The model is loaded with AutoModelForCausalLM.from_pretrained with trust_remote_code=True and device_map="auto", and the processor with AutoProcessor.from_pretrained and the same trust_remote_code flag. The model identifier is FlashLabs/Chroma-4B, with a comment that a local path also works.

The trust_remote_code flag is the significant detail. It means the model repository contains Python that will be executed on your machine, so the model is not a set of weights you can load safely without trusting the publisher. For an Apache-2.0 model from a company with a commercial product built on it, that is a reasonable trade, but it is worth understanding that you are trusting it, and it is worth remembering that the model card is the place to check what that code does.

device_map="auto" is what handles the hardware, letting the library place the model across available devices.

The inference block is longer and shows the conversation format, which is a list of messages where each message has a role and a content list of typed parts. The system message carries text, and the user message carries an audio entry with a file path. Then a generation prompt is added, and the processor is called with the conversation plus the reference prompt audio and prompt text, and the inputs are moved to the model's device.

Generation uses max_new_tokens of 100, do_sample=True, temperature 0.7, top_p 0.9 and use_cache=True. Then the key step, decoding the audio. The comment says the model outputs raw tokens and the audio part is decoded using the codec:

audio_values = model.codec_model.decode(output.permute(0, 2, 1)).audio_values

And the result is played with the IPython display Audio helper at a rate of 24_000, which matches the Mimi codec's sampling rate.

The three-step structure, generate, decode the codec, play, is a good illustration of what an end-to-end audio model looks like in practice. The model does not output a waveform, it outputs tokens that the codec turns into one.

How to evaluate it, and what to check first

There are four things to consider.

First, the hardware and the dependency stack. CUDA 12.6 or compatible for GPU support, Python 3.11 or newer, the Chroma-4B model with a separate codec, and a transformers release candidate pinned to 5.0.0rc0. This is a research codebase rather than a packaged library, so expect to be reading the README and the notebook to work out what is going on, and expect the troubleshooting section to matter.

Second, the security question, and it is not a small one. The model clones a voice from a short reference clip with no training, the code is Apache-2.0 and freely downloadable, and the model weights are on Hugging Face. The README does not discuss misuse at all. That is a choice the maintainers have made, and you should know it is a choice before you build something with it, particularly if your use case involves voices that are not the user's own. The reference mechanism does at least require a text and audio pair, which is a slightly higher bar than a single clip, but the underlying capability is general.

Third, the model card and the technical report. The repository links both, and the arXiv paper at 2601.11141 is where the training details, the evaluation and the limitations will be, and it is also where the honest reporting lives that a README of this length does not attempt. The README here is a getting-started document, and for a model like this the paper is the real documentation.

Fourth, what is actually in the repository. The listing is short: a chroma/ directory for the model code, an example/ directory with the make_taco.wav audio and the prompt_audio and prompt_text reference directories, an example.ipynb notebook, a local_run/ directory, requirements.txt, and figures/ for the architecture diagram. The presence of local_run/ suggests an offline or local inference path, and the notebook is the entry point that most people will actually use.

And then the commercial context, which you should be aware of without being distracted by. The README leads with a link to FlashAI Voice Agents, described as production-ready voice AI solutions powered by Chroma, alongside the open-source model for developers and researchers. So the open model is the research release and the product is the commercial service on top of it, and the relationship between them is worth understanding before you build something that depends on the model rather than the service.

What you get is a genuinely capable open speech-to-speech model with a documented architecture, a worked example, a troubleshooting note for the one install problem everybody hits, and a licence that lets you use and modify it. The model card, the paper and the notebook are where the judgement calls are, and the README is the part that gets you running.

Editorial conclusion

Adopt FlashLabs Chroma if you are building a voice agent and want an end-to-end speech-to-speech model that clones a voice from a short reference pair with no per-speaker training, and you are comfortable running research code with a release candidate of transformers pinned. Do not adopt it without reading the model card and the technical report, because the README is a getting-started document that says nothing about the limitations or the misuse of a general voice cloning capability, and a freely downloadable Apache-2.0 model that clones voices from short clips is a responsibility rather than a feature you inherit silently. Before you commit, check the hardware and the dependency ordering, since the install fails with a confusing TypeError if torchvision is not present when transformers initialises, and note the relationship between the open checkpoint and the company's commercial voice agent service before you build on the model rather than the service.

Frequently asked questions

Official sources

1. FlashLabs-AI-Corp/FlashLabs-Chroma on GitHub
2. Issues
3. License: Apache-2.0
4. Project website
5. README