C2C: letting language models talk through their KV caches

The idea: exchange latents instead of words

Cache-to-Cache, abbreviated C2C, enables large language models to communicate directly through their KV caches, bypassing text generation. By projecting and fusing KV caches between models, the reported results are 8.5 to 10.5 percent higher accuracy than individual models and 3.0 to 5.0 percent better performance than text-based communication, with a 2.0 times speedup in latency.

The mechanism is the interesting part. In a normal pipeline, model A generates text, that text is tokenised, and model B reads it. The text is the only channel, and it is a lossy one because everything the first model knew has to survive a round trip through tokens. C2C instead takes the first model's KV cache, projects it into the second model's representation space layer by layer, and lets the second model continue from there.

The package is called rosetta, and the README explains why with a good analogy. The name comes from the Rosetta Stone, the artefact that opened up the translation of Egyptian hieroglyphs by presenting the same text in multiple scripts. Likewise, C2C translates KV-cache representations between otherwise independent language models, allowing them to speak a common language in a richer and more direct way.

There is a demo that shows the point in concrete terms. The Combined Wisdom example asks a philosophical question where the stated answer is that only by combining latent semantics from both Qwen2.5 and Qwen3 can the question be correctly answered, and the demo can be reproduced with the script at script/playground/gradio_demo.py.

The project is the official code implementation for the paper Cache-to-Cache: Direct Semantic Communication Between Large Language Models, which was accepted by the ICLR 2026 conference. The licence is Apache-2.0, the default branch is main, and the last push was on 2026-10-09.

Installing rosetta and loading a published fuser

The environment setup is a conda environment on Python 3.10:

conda create -n rosetta python=3.10
conda activate rosetta

Then the package itself, installed in editable mode:

pip install -e .

And for training and evaluation, the extra groups:

pip install -e ".[training,evaluation]"

The extras are meaningful. The training group pulls in datasets, accelerate, wandb and peft, and the evaluation group pulls in scikit-learn, matplotlib, seaborn, jsonlines, openai, math_verify, latex2sympy2_extended and spaces, which is the tooling for scoring model answers including mathematics answers.

The package metadata pins the two core dependencies exactly. torch is 2.6.0 and transformers is 4.52.4, and the project requires Python 3.10 or newer with classifiers listed from 3.8 through 3.11. The development status is marked as 3, Alpha, and the intended audiences are Developers and Science/Research, which tells you what kind of project this is.

The published weights live on Hugging Face in the C2C_Fuser collection, and the full list of available fuser pairs is in the README, and loading one is a hub download with a pattern filter:

import torch
from huggingface_hub import snapshot_download
from script.playground.inference_example import load_rosetta_model, run_inference_example

checkpoint_dir = snapshot_download(
    repo_id="nics-efc/C2C_Fuser",
    allow_patterns=["qwen3_0.6b+qwen2.5_0.5b_Fuser/*"],
)

The model configuration names a base model and a teacher model, so a fuser pair is defined by which receiver and which sharer it bridges, and the model is then loaded onto a CUDA device with a rosetta config dictionary.

The kv_cache_index is the control surface

The inference script contains the mechanism that makes the whole thing work, and it is a single argument.

The generate call takes a kv_cache_index, and the README's own example builds it from two tensors. An instruction_index of [1, 0] and a label_index of [-1, 0] are created with torch.tensor, repeated across the input length and unsqueezed, and the two are put in a list as kv_cache_index. Then the model generates with that index passed in, and the output is decoded from the position after the instruction length.

The semantics of the index are given in the wrapper example later in the README. The comment reads that kv_cache_index controls when to apply C2C projection, with [1, 0] meaning apply projection from sharer 1 and [-1, 0] meaning no projection.

That is a compact and readable control. A value of -1 at a position means leave that position alone, and a positive value names the sharer whose cache is projected into the receiver at that point. So a prompt that begins with the receiver model's own tokens and then merges in a sharer's cache is expressed as a list of small index pairs, one per position, rather than as a separate training example per combination.

The generation call also takes ordinary sampling parameters, and the example uses do_sample False with max_new_tokens 256, so the output is deterministic. That matters when you are comparing a fused run against a baseline run, which is the main thing the demo does.

The RosettaModel wrapper applies C2C to models you bring

The package provides a universal RosettaModel wrapper so C2C can be applied to your own models rather than only to the published fuser pairs.

The example is short and explicit. You load a target, the receiver, with AutoModelForCausalLM.from_pretrained and a source, the sharer, the same way. Then you build a list of C2CProjector objects, one per layer of the target model, each taking source and target dimensions, source and target head counts, a hidden dimension and a number of layers. The example uses 128 and 128 for the dimensions, 8 and 8 for the heads, 1024 for the hidden dimension and 3 layers.

The models and the projectors are then handed to RosettaModel with a model_list, a base_model_idx and the projector list. The base_model_idx is 0, which is the receiver in this example, and the source is the second entry.

The layer-wise mapping is set separately with set_projector_config, and the example walks the target model's hidden layers, calling it for each one with a source model index, a source model layer index, a target model index, a target model layer index and a projector index. The comments note that a pretrained projector can be loaded from the checkpoint directory instead of being constructed fresh.

The design point is that this is a layer-by-layer correspondence rather than a whole-model translation. A projector is bound to one source layer and one target layer, and the mapping is explicit in the configuration. That is a lot of plumbing for a user, and it is the price of the generality, because a fuser trained for one specific pair of models does not automatically apply to another.

Multi-sharer support, and the roadmap that is still a roadmap

The news section in the README is a good way to read the project's maturity, and it is dated.

In October 2025 the paper was featured as the number one paper of the day on Hugging Face Daily Papers. In November 2025 a live demo on Hugging Face Spaces became available, with side-by-side model comparison. In December 2025 multi-sharer support arrived, the ability to fuse KV caches from multiple sharer models into a single receiver, and the README is explicit that this feature is in preliminary stages and still being worked on. The usage is in the file live_chat_example.py.

In January 2026 the paper was accepted by ICLR 2026, and the announcement invites discussion in Brazil. And in September 2026 the project posted that agent-managed KV-cache will soon be released in the repository, along with the corresponding serving system, as the next step forward.

That last entry is the one to read carefully if you are thinking about deploying this. Agent-managed KV-cache and a serving system are announced as forthcoming, not as shipped. What exists today is a research codebase, a set of published fuser weights, a live demo and a wrapper.

The interactive chat example is the way to see the multi-sharer path, and it takes one or more checkpoint directories:

# Single sharer
python script/playground/live_chat_example.py --checkpoint_dir path/to/checkpoint

# Multiple sharers (teacher models read from each checkpoint's config.json)
python script/playground/live_chat_example.py --checkpoint_dir path/to/ckpt1 path/to/ckpt2

The comment on the second form is a useful hint about how the design works. With multiple sharers, the teacher models are read from each checkpoint's own config.json, so a multi-sharer setup is a set of self-describing checkpoints rather than a single merged one.

The repository layout reflects a research project: a rosetta/ package for the library, script/ for the playground and examples, a recipe/ directory, a bash/ directory, a resource/ directory, an environment.yml and a pyproject.toml.

Where C2C is the wrong choice

The first wrong case is a production system. The paper's numbers are research numbers, on benchmarks the authors chose, and the serving path the project itself calls the next step forward does not exist yet. Adopting a research codebase with an Alpha development status and a preliminary multi-sharer feature is a different decision from adopting a library.

The second is a task where text communication is sufficient. The reported gain of 3.0 to 5.0 percent over text-based communication is real but bounded, and it comes with the cost of running two models and a projector instead of one. If your workload is a simple prompt to a single model, none of that pays back.

The third is a hardware constraint. The published example loads onto a CUDA device, and the whole point of the method is that the KV cache is a large tensor that has to be projected layer by layer. Running the receiver and the sharer side by side, plus the projector, costs memory and compute that a text-based pipeline does not.

The fourth is a model pair with no published fuser. The Hugging Face collection has fusers for specific pairs, and the README points at a list of supported model pairs. For anything else you build and train the projectors yourself, with the training extras, wandb and peft, and the gap between that and using a published checkpoint is a research project.

The alternative worth weighing is an ensemble at the text level, where you query two models and combine their outputs, or use the teacher as a source of context for the receiver. That costs you the 2.0 times latency speedup and the ability to combine information that never survives tokenisation, and it is far simpler to build, debug and maintain. The C2C approach is worth it when the information you need to move between models is genuinely unrepresentable in text, which is a narrower condition than it first appears.

Editorial conclusion

Adopt rosetta if you are doing research on multi-model ensembles and want to try cache-to-cache communication with pre-trained projectors rather than training your own, since the fuser weights and a live demo are published and the wrapper works with any pair of models. Keep away if you need a production serving path, because the project describes the agent-managed KV-cache and its serving system as a next step rather than something shipped. Before you build on it, read the news entries in the README, since the multi-sharer support is explicitly preliminary, and note that the package pins torch 2.6.0 and transformers 4.52.4 exactly and requires Python 3.10 or newer.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. Project website
4. README
5. thu-nics/C2C on GitHub