kani: a chat framework with one runtime dependency and everything else optional

One required dependency, and every engine behind an extra

Install kani for one model or for all of them; the extras decide which world your environment ends up in:

# for OpenAI models
$ pip install "kani[openai]"
# for Hugging Face models
$ pip install "kani[huggingface]" torch
# for multimodal inputs
$ pip install "kani[multimodal]"
# or install everything:
$ pip install "kani[all]"

What sits in the base install is one package: pydantic, pinned to the 2.x series. Everything else is behind a bracket. That is the whole thesis of the project, that a chat framework should not decide your model provider for you, and it is why the model table in the docs rather than a default engine is the thing to read before installing.

There is also a development install straight from the repository's main branch, for when you need a fix that has not been cut into a release:

$ pip install "kani[all] @ git+https://github.com/zhudotexe/kani.git@main"

Python 3.10 or newer is the floor, stated in both the quickstart and the project metadata.

The extras carry comments naming the reason for each bound

The optional dependency groups are more informative than the version numbers alone, because several of them carry a comment explaining the bound. The multimodal extra pulls in `kani-multimodal-core` under 2.0.0 and `transformers` from 4.56.1, with a note that 4.56.1 is where `ProcessorMixin.__call__` changed. The MCP extra requires `mcp` from 1.15.0, noted as the release that introduced `ClientSessionGroup`.

The provider groups are equally specific. The OpenAI extra is `openai` under 3.0.0 plus `tiktoken`, Anthropic is `anthropic` under 1.0.0, and Google is `google-genai` from 1.0.0. Hugging Face adds `accelerate` and `huggingface-hub`. The llama.cpp path adds `llama-cpp-python`, and the vLLM path resolves to a separate package called `kani-ext-vllm` rather than to vLLM itself.

Read those comments as warnings. An upper bound with a reason attached is a place where a silent upgrade turns into a runtime error.

Streaming is a drop-in superset, not a rewrite

Streaming responses token by token works even when the turn ends in a function call, and the design goal was explicitly to avoid a migration. `chat_round_stream` and `full_round_stream` are supersets of `chat_round` and `full_round`, so you can move one call site at a time and never leave the code broken in between.

The shape is the same in both variants. You iterate the stream for tokens, then await the finished message object, which also means awaiting the stream directly works:

async def stream_chat():
    stream = ai.chat_round_stream("What does kani mean?")
    async for token in stream:
        print(token, end="")
    print()
    msg = await stream.message()  # or `await stream`

async def stream_with_function_calling():
    async for stream in ai.full_round_stream("What's the weather in Tokyo?"):
        async for token in stream:
            print(token, end="")
        print()
        msg = await stream.message()

The two differ in what a stream represents. A single chat round is one model response, while a full round is the loop in which the model calls a tool and gets a result back, which is why the full variant yields streams in a loop.

@ai_function is the whole tool interface

Tools are ordinary Python. You subclass `Kani`, decorate a method with `@ai_function()`, and the method becomes something the model can choose to call based on its own documentation. Extra parameter descriptions come from `Annotated` plus `AIParam`, which is how you tell the model what a parameter means rather than leaving it to guess from the type.

The guarantee the project makes is about the moment of truth: kani validates that a function call is well-formed before it reaches your method. Your function body does not need a defensive parsing layer for missing arguments, because the argument handling happens upstream of the call.

That leaves the interesting work where you want it, which is the control flow. The examples directory is organised by exactly that, with `3_customization_exception_prompt.py`, `3_customization_last_four.py`, `3_customization_log_messages.py` and `3_customization_track_function_calls.py` each changing one specific thing about what the model sees and what gets logged.

MCP servers enter through the same tool path

There is a note in the function calling documentation pointing at MCP support, and kani supports both local and remote MCP servers. That matters for how the project is positioned: MCP tools are not bolted on as a separate subsystem with their own loop, they arrive through the tool-calling machinery the `@ai_function` decorator already uses.

The example file names make the layering clear. `2_function_calling_weather.py` is the plain case with a hand-written function, `2_function_calling_fewshot.py` adds few-shot examples, and `2_function_calling_mcp_tools.py` swaps in MCP. All three sit at the same level of difficulty, so the difference you are seeing is the tool source and nothing else.

The dependency explains what is underneath. The MCP extra requires `mcp` from 1.15.0, a version chosen for `ClientSessionGroup`, which is the piece that lets several MCP sessions be managed together.

Multimodal turns a string into a list of parts

Multimodal input is optional and lives in an extension package, installed either as `pip install "kani[multimodal]"` or by taking `kani-multimodal-core` directly, with its own documentation site. What it changes in your code is the shape of a message rather than the shape of your program: `chat_round_str` accepts a list of parts instead of one string.

from kani import Kani
from kani.engines.openai import OpenAIEngine
from kani.ext.multimodal_core import ImagePart

engine = OpenAIEngine(model="gpt-4.1-nano")
ai = Kani(engine)

# notice how the arg is a list of parts rather than a single str!
msg = await ai.chat_round_str([
    "Please describe these images:",
    ImagePart.from_file("path/to/image.png"),
    await ImagePart.from_url(
        "https://upload.wikimedia.org/wikipedia/commons/thumb/5/53/Whitehead%27s_Trogon_0A2A6014.jpg/1024px-Whitehead%27s_Trogon_0A2A6014.jpg"
    ),
])
print(msg)

Local files use `ImagePart.from_file` and remote ones use an awaited `ImagePart.from_url`, so the awaiting is part of the API rather than an accident of the example.

The examples are numbered by how deep they go

The examples directory is the clearest map of the library's surface. Files beginning 1_ are entrypoints and quickstart territory: `1_quickstart.py`, `1_async_entrypoints.py`, `1_fewshot.py`. Files beginning 2_ add function calling, in its plain, few-shot and MCP forms. Files beginning 3_ are the customisation layer. Files beginning 4_ and 5_ are the engine zoo and the advanced API, including `4_engines_fp4_quant.py`, `5_advanced_retrieval.py`, `5_advanced_subkanis.py` and the message-parts experiments.

Two Colab notebooks, `colab_quickstart.ipynb` and `colab_examples.ipynb`, let you try the library without installing it. Two standalone scripts, `dice.py` and `mistral_gguf.py`, show a complete small program and a local GGUF model. An `old/` directory holds retired examples, which is the honest signal that the numbering scheme gets revised rather than appended to forever.

Beta status, and a five month gap between 1.9.1 and 1.10

The project metadata classifies kani as Development Status 4, Beta, with an intended audience of Science and Research. It came out of the NLP-OSS workshop at EMNLP 2023 and has a paper on arXiv, and the repository carries a `citation.cff` alongside a Chinese and English documentation build served through Read the Docs.

Releases are uneven. v1.9.0 shipped on 2026-03-04, v1.9.1 on 2026-03-17, and v1.10.0 on 2026-08-27, five months after the patch, with the last push to main on that same day.

The development requirements file is the other place worth reading before you pin anything. It installs the all extra and adds torch, torchvision and torchcodec for multimodal end-to-end tests, and it caps pytest below 9.1.0 with a comment saying that version breaks pytest-lazy-fixtures. Those comments are the maintenance record of a project that has run out of road a few times and written down why.

Editorial conclusion

Use kani when you want to see and change the prompt, the tool schema and the underlying model, rather than hand control to an agent framework. It fits research code, teaching, and small tools where you will be reading the request the model received. Pass if you want a batteries-included agent loop, since kani does not provide one, or if you are on Python 3.9. Before pinning versions, read the comments in the extras: several upper bounds exist for a named reason, and one of them is why a naive upgrade can break your code.

Frequently asked questions

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. zhudotexe/kani on GitHub