ollama-rs is a typed wrapper over the Ollama HTTP API, and its test suite needs a running Ollama with three models

The API surface mirrors the upstream documentation

ollama-rs is described as a simple and easy-to-use library for interacting with the Ollama API, and the README says it was created following the Ollama API documentation.

That last sentence is the design statement. The library is a typed mapping of the documented endpoints, not an opinionated abstraction over them, and the table of contents in the README is a list of those endpoints: completion generation, completion generation streaming, completion generation with options, chat mode, list local models, show model information, create a model, create a model streaming, copy a model, delete a model, generate embeddings, generate embeddings in batch, a system one endpoint, make a function call, create a custom tool, and completion generation with thinking.

Every one of those corresponds to something the Ollama HTTP API exposes, and the request types are named after the operation. `GenerationRequest::new(model, prompt)` for a completion, `CreateModelRequest::new(my_model_string).into()` for a model, with the name escaped as a Rust string, `ChatMessageRequest` for a chat turn, and `ModelOptions` for the sampling parameters.

The result of the first example makes the shape obvious.

use ollama_rs::generation::completion::request::GenerationRequest;

let model = "llama2:latest".to_string();
let prompt = "Why is the sky blue?".to_string();

let res = ollama.generate(GenerationRequest::new(model, prompt)).await;

if let Ok(res) = res {
    println!("{}", res.response);
}

The sampling options are a separate builder rather than constructor arguments.

let options = ModelOptions::default()
    .temperature(0.2)
    .repeat_penalty(1.5)
    .top_k(25)
    .top_p(0.25);

let res = ollama.generate(GenerationRequest::new(model, prompt).options(options)).await;

That separation is a good choice, because the same request type is used for a plain completion and one with tuned parameters, and the option set is the part that grows as the upstream API adds sampling controls.

The library is MIT licensed, the primary language is Rust, and the recent releases are v0.3.6 from 2026-07-24, v0.3.5 from 2026-06-16 and v0.3.4 from 2026-02-12, with the last push to the repository on 2026-10-14.

Chat history is a trait, not a Vec

The chat mode is the part of the API with the most design in it, and the design decision is that history is a trait you implement.

use ollama_rs::generation::chat::{request::ChatMessageRequest, ChatMessage};
use ollama_rs::history::ChatHistory;

let model = "llama2:latest".to_string();
let prompt = "Why is the sky blue?".to_string();
// `Vec<ChatMessage>` implements `ChatHistory`,
// but you could also implement it yourself on a custom type
let mut history = vec![];

let res = ollama
    .send_chat_messages_with_history(
        &mut history, // <- messages will be saved here
        ChatMessageRequest::new(
            model,
            vec![ChatMessage::user(prompt)], // <- You should provide only one message
        ),
    )
    .await;

A `Vec<ChatMessage>` implements `ChatHistory` out of the box, but you can implement the trait on your own type. That matters because a real application usually wants history somewhere other than a vector: a database, a file, a session store, or a bounded buffer that trims old turns.

The comment on the call is also a warning. You should provide only one message, and the README notes in a separate line that every message sent and received will be stored in the library's history. So the caller sends the new turn and the library appends both directions to the history object it was given. The `&mut history` borrow is how the library gets that write access, and it is why the history has to be mutable and owned by the caller rather than hidden inside the client.

This is a small API with a real consequence. The library holds no conversation state of its own, which means the same `Ollama` client can serve several conversations, and it means the caller decides the lifetime of a conversation, which in a server is usually a database row.

The completion endpoints, by contrast, are stateless. The `generate` and `generate_stream` calls take a model and a prompt and return a response, with no memory between calls.

Streaming is behind a feature flag, and the flag also gates model creation

Not everything is available by default, and the feature flags are the first thing to check when a method does not exist in your build.

Streaming completions require the `stream` feature, and the streaming model creation call requires it too. The streaming example pulls in two extra dependencies, `tokio_stream` for the `StreamExt` trait that gives you `next()` on the stream, and `tokio::io` for writing to standard output and flushing it.

let mut stream = ollama.generate_stream(GenerationRequest::new(model, prompt)).await.unwrap();

let mut stdout = io::stdout();
while let Some(res) = stream.next().await {
    let responses = res.unwrap();
    for resp in responses {
        stdout.write_all(resp.response.as_bytes()).await.unwrap();
        stdout.flush().await.unwrap();
    }
}

The inner loop over `responses` is worth noting, because it says each item from the stream is itself a collection of responses. That is how the Ollama API delivers a chunked response, where a single network event can carry more than one response object. A consumer that assumed one response per item would drop data.

The same pattern appears in the streaming model creation, which yields a `CreateModelStatusStream` that will stream every status update of the model creation, so the progress of a long model build is observable rather than a blocking call.

The feature flag arrangement is a reasonable choice for a library where most users do not stream. The alternative is a dependency on the streaming machinery for everyone, and the cost is that the API differs between builds, which is a real source of confusion when you read an example and it does not compile.

There is one more feature-flag-adjacent detail. The `master` branch can be used directly from git, with the note that it may not be stable and may contain breaking changes. The crates.io release at 0.3.6 is the version to pin.

Two crates, and a test suite that provisions its own models

The repository is a Cargo workspace with two members, `ollama-rs` and `ollama-rs-macros`, and the macro crate is the one that is interesting. It is a separate crate because Rust proc macros have to be compiled for the host, and keeping them in their own package is the standard arrangement. The workspace pins the version to 0.3.6 in `workspace.package`, so the two crates cannot drift apart.

The Makefile is where the testing story is, and it is unusual enough to describe in full. The `prepare` target runs before `test`, and it provisions models rather than assuming they exist.

prepare:
	@$(MAKE) _ensure_ollama_create NAME=mario MODELFILE=ollama-rs/tests/model/Modelfile.mario
	@$(MAKE) _ensure_ollama_create NAME=test_model MODELFILE=ollama-rs/tests/model/Modelfile.test_model
	@$(MAKE) _ensure_ollama_model MODEL=llama2:latest
	@$(MAKE) _ensure_ollama_model MODEL=granite-code:3b
	@$(MAKE) _ensure_ollama_model MODEL=llava:latest

Two models are created from Modelfiles checked into the repository, `mario` and `test_model`, and three are pulled by name, `llama2:latest`, `granite-code:3b` and `llava:latest`. The choice of models is a description of the test coverage: a general model, a code model, and a vision model, which corresponds to the generation, the function calling and the embeddings or image paths in the API.

The helper targets are idempotent, which is what makes the test suite runnable in CI.

_ensure_ollama_model:
	@ollama list 2>/dev/null | awk 'NR>1 {print $$1}' | grep -qxF '$(MODEL)' || ollama pull '$(MODEL)'

The check lists the installed models, greps for an exact match, and only pulls if the model is not already there. So the test run requires a working Ollama installation with network access to the model registry on a cold machine, and needs nothing extra on a warm one.

That is a legitimate choice for an integration test suite against a local model server, and it is also a constraint on anyone trying to run the tests. The `test` target itself is just `cargo test`, and `fmt` runs `cargo fmt` plus `cargo clippy --fix --allow-dirty`.

The client is a builder, and it works with any server that speaks the API

Initialization is the simplest part of the API, and the builder exists for one reason.

use ollama_rs::Ollama;

// By default, it will connect to localhost:11434
let ollama = Ollama::default();

// For custom values:
let ollama = Ollama::new("http://localhost".to_string(), 11434);

The default is the standard Ollama port on localhost, which is what the `Ollama::default()` constructor gives you. `Ollama::new` takes a host string and a port.

The builder is the third form, and the README gives a concrete reason for it.

use ollama_rs::Ollama;

// llmman listens on 127.0.0.1:17434 by default
let ollama = Ollama::builder()
    .host("http://localhost")
    .port(17434)
    .build();

llmman is a local model runner that serves the Ollama API on port 17434, so `ollama-rs` works with it unchanged, and the only thing you have to change is the port. The server is started with `llmman serve` and a model is pulled with `llmman pull gemma4`, or `llmman pull hf.co/unsloth/Qwen3.5-0.8B-GGUF` to pull from Hugging Face.

That example is the clearest statement of the library's scope. The client speaks the Ollama HTTP API, and anything that serves that API is a valid backend, whether it is Ollama itself or a different runner that happens to be compatible. That also tells you the limitation: the library is only as good as the compatibility of the server, and a server that implements the API partially will give you errors from the parts it does not.

The README also points at a chatbot example that shows how to use the library to create a simple chatbot in less than 50 lines of code, and it notes the examples use poor error handling for simplicity, but you should handle errors properly in your code. The error handling in the snippets above follows that advice, since every call returns a `Result` that has to be unwrapped or matched.

Editorial conclusion

Adopt ollama-rs if you are writing Rust against a local Ollama instance and want typed requests rather than hand rolled HTTP calls. Do not adopt it if you need a stable client interface across Ollama versions, because the library is explicitly written to follow the upstream API documentation and its version is well below 1.0. Verify first that the Ollama version on your machine matches the endpoints the library expects, that you enable the stream feature if you want streamed responses, and that your test environment can host the models the test suite pulls.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. pepperoni21/ollama-rs on GitHub
4. README
5. Releases