genai, one Rust API over two dozen AI provider protocols

One request type, twenty-seven provider protocols

genai is a Rust library for native-protocol multi-AI provider access. The word native is the important one in that description, and it distinguishes genai from a family of libraries that normalise every provider onto one vendor's wire format and lose the provider-specific behaviour in the process.

Here each provider is spoken in its own protocol. Anthropic is called as Anthropic, Gemini as Gemini, xAI as xAI, Ollama as Ollama, and so on. The README states the scale as over 200 LLM models and 27 or more LLM providers out of the box, and the provider list in the documentation names them individually: openai, openai_resp, anthropic, gemini, omlx, ollama, ollama_cloud, vertex, bedrock_api, bedrock_sigv4, github_copilot, opencode_go, groq, together, fireworks, cohere, nebius, mimo, deepseek, mistral, minimax, zai, zai_coding, bigmodel, aliyun, qwen_cloud, baidu, moonshot, kimi, aihubmix, open_router, requesty, api_route, heabsy, atlascloud and xai.

The value of that list is not the count. It is that the awkward ones are covered. Bedrock appears twice, once for plain API access and once for SigV4 request signing, Vertex carries both Gemini and Anthropic support, and Ollama has both a native adapter and a cloud one.

The request surface is small. A chat request is a ChatRequest built from a vector of ChatMessage values, with system and user messages as constructors, and it is executed with exec_chat against a model name:

let client = Client::new()?;

let question = "Why is the sky red?";

let chat_req = ChatRequest::new(vec![
 ChatMessage::system("Answer in one sentence"),
 ChatMessage::user(question),
]);

let chat_res = client.exec_chat("gpt-5.4-mini-high", chat_req, None).await?;

println!("{}", chat_res.first_text().unwrap_or("NO ANSWER"));

The result carries the text through first_text(), and the whole call is one await with a Result, so error handling is ordinary Rust error handling.

The model name carries the routing decision

genai resolves a model name to a provider and an adapter, and the README describes two levels of that decision.

The first is convenience. Common models like OpenAI, Anthropic and Gemini can be accessed with their simple name, for example gpt-5.6-luna, claude-... or gemini-..., and the right adapter and provider are selected for you.

The second is explicit control. A namespaced model name of the form [name_space]::[model_name] forces a specific provider, for example qwen_cloud::qwen3.8-max. That is the form to use when two providers host a model with the same name, or when you need a vendor's specific endpoint rather than the default.

There is a third convention that is worth understanding before it surprises you. Model names can carry a reasoning effort suffix, such as -high, which is set and then removed from the name when the request is sent to the provider. The example above uses gpt-5.4-mini-high, and the model that reaches the provider is gpt-5.4-mini with a high reasoning effort. The provider family in the v0.7 notes includes ReasoningEffort::Zero replacing ReasoningEffort::None with the "zero" keyword and "none" kept as a backward compatible alias, plus the Anthropic Max and OpenAI XHigh values added in 0.6.

Together these three conventions give a routing model that is compact but has a sharp edge. Because the routing is encoded in a string, a typo in a namespace is a runtime failure rather than a compile error, and the same model name can resolve to different providers depending on the suffix.

Custom OpenAI-compatible endpoints without writing an adapter

A large share of self-hosted model servers speak the OpenAI wire format, and genai has a built-in adapter for them through the genai_n namespace.

The configuration is two environment variables, an endpoint and an optional key:

GENAI_1_ENDPOINT=https://my-host.example.com/v1/
GENAI_1_API_KEY=sk-your-api-key

After that, the model is addressed with the matching namespace:

let client = Client::new()?;
let chat_res = client.exec_chat("genai_1::my-model", chat_req, None).await?;

The numbered namespace exists so several custom endpoints can coexist in one process, which is the case where you run an internal gateway alongside a commercial API and want one client to reach both.

OMLX gets a first-class treatment rather than being forced through the generic path. For an OMLX deployment you set OMLX_ENDPOINT to the OpenAI-compatible base URL before creating the client, and OMLX_API_KEY when authentication is required, then target models with the omlx:: namespace.

For providers that are not OpenAI-compatible at all, the library offers ServiceTargetResolver, which lets an application resolve a model to a target and its credentials, with an example in examples/c06-target-resolver.rs. That is the extension point for a provider the library has never heard of. The trade-off is that you are writing the request and the response handling yourself, and the abstraction only saves you the client plumbing.

Version 0.7 broke a few things on purpose

The v0.7.0 release, dated 2026-10-19, is described as a big release with many fixes, new capabilities, new providers and minimal API changes. The breaking list is short and specific, and it is the kind you find at compile time rather than in production.

Client initialisation is now fallible. Client::new()? and ClientBuilder::build()? return a Result<Client>, and Client::default() is gone. ReasoningEffort::Zero replaces ReasoningEffort::None, with the "zero" keyword and "none" kept as a backward compatible alias. Error::HttpError now carries the headers of the failed response, which is the piece you need when a provider returns a useful error header.

Tool gains a custom_format field, and ChatOptions gains raw_frame_sink, stream_header_timeout and stream_read_timeout, which means struct literals need the new fields or ..Default::default(). Optional None fields are now omitted from the serialized JSON instead of being sent as null, a change that matters for strict providers.

One removal is worth calling out because it changes behaviour rather than breaking compilation: JsonSpec::schema_with_additional_properties_false has been removed, and the adapters now sanitize schemas themselves. The Gemini and Vertex work in 0.6 already did something similar for structured output and tool schemas, normalising common JSON Schema shapes including const, nullable schema patterns, additionalProperties, and JSON Schema-only keywords that Vertex rejects.

The repository carries a CHANGELOG.md and a v0.6 to v0.7 migration guide under docs/migration/, which is where to look before upgrading an existing integration. The Cargo manifest shows the crate version as 0.7.1-WIP while the published release is the 0.7 line.

Streaming, tool use and the observability hooks

The examples directory shows the feature surface, and it is a long one: c00-readme, c01-conv, c02-auth, c03-mapper, c04-chat-options, c05-model-names, c06-model-spec, c06-target-resolver, c07-image, c08-image-gen, c09-cache-ttl, c10-aihubmix, c11-openai-resp-reasoning-roundtrip, c12-otel, c13-gemini-interactions, and then a run of tool examples from c20 to c25 covering tool use, tool use streaming, deterministic tool use, a web search tool and a custom grammar tool, plus a frame sink example and several provider-specific ones for Baidu, both Bedrock adapters and Gemini search citations.

Several of those map to features the v0.7 notes highlight. Anthropic prompt caching on tools is enabled through Tool::with_cache_control. SSE heartbeat events are surfaced as ChatStreamEvent::Heartbeat, which matters for long streams where a proxy would otherwise time out an idle connection. Stream frame inspection is available through ChatFrameSink and ChatOptions::with_raw_frame_sink, giving access to the raw frames as they arrive rather than only the parsed events.

Thought signatures and signed thinking blocks are preserved across tool turns for Anthropic, Gemini and OpenRouter, and the Bedrock Converse adapter preserves streaming usage from metadata after the stop event and replays signed and redacted reasoning across tool turns.

Two additions change the shape of a program rather than adding a detail. The Gemini Interactions API adapter, gemini_ix::, supports stateful conversation sessions through previous_response_id and store. And OpenTelemetry instrumentation following the GenAI semantic conventions is available behind the otel feature flag, which is the difference between a provider client you can observe in production and one you cannot.

TLS, proxying and an outright ban on unsafe code

The Cargo manifest is unusually explicit about the transport layer, which is where multi-provider clients usually hide their sharpest edges.

There are two TLS backends and they are mutually exclusive. rustls-tls is the default and forwards to reqwest with the rustls feature, using aws-lc-rs and the OS trust store through rustls-platform-verifier, and the comment recommends it because it needs no system OpenSSL, works on musl and static builds, and honours enterprise certificate authorities. native-tls uses OpenSSL on Linux, SChannel on Windows and Secure Transport on macOS. The manifest explains the failure mode if you get it wrong: with default features left on, both backends compile and the build fails with a compile_error! in src/lib.rs telling you to set default-features = false. If you select native-tls, you also have to re-add the system-proxy feature, because it is off with default-features = false and you lose the macOS and Windows proxy settings.

For anything outside those two, the library steps aside. A bring-your-own crypto provider, whether that is FIPS or ring, static OpenSSL, custom root certificate authorities or mutual TLS, is described as not a genai feature: you build your own reqwest::Client and inject it with Client::builder().with_reqwest(...).

Proxy support is on by default through the system-proxy feature, which reads the macOS and Windows system proxy settings when the reqwest client is built, and the standard HTTP_PROXY, HTTPS_PROXY and NO_PROXY environment variables are honoured without it.

The last line of the lints section is a design commitment: unsafe_code = "forbid".

When one vendor's SDK is the better answer

The alternative to genai is to call the provider's HTTP API directly, or to use that vendor's own SDK, which is what most Rust teams do for a single provider.

The difference in approach is what happens when you need something unusual. A vendor's own SDK gives you every feature that vendor ships, including the ones with no equivalent anywhere else, and it gives them to you the day they appear. genai gives you a common shape across twenty-seven providers, which means a feature only appears once it can be expressed in that shape, or once an adapter has been written for it.

That is the correct trade for a service that has to run against more than one provider, where failover is a requirement and where the model choice changes with cost and latency. It is the wrong trade for an application that is deliberately bound to one provider and wants its newest capability the day it ships.

There is a second cost that shows up in code review. genai is an abstraction layer, so every call goes through an adapter that may transform the request on the way out and the response on the way back. When something behaves unexpectedly, you have a second place to look, and the CHANGELOG entries about schema sanitisation and reasoning-effort mapping are reminders that transformations exist.

The library's own documentation includes a file aimed at language models, docs/for-llm/api-reference-for-llm.md, which is a sign that the maintainer expects a lot of the usage to be written by an assistant rather than typed by hand. That is convenient, and it is also a reason to pin the version and read the migration guide before letting anything else write the integration.

Editorial conclusion

genai suits a Rust service that has to talk to several model providers behind one interface, to a local Ollama or OMLX deployment, or to a self-hosted OpenAI-compatible gateway, without writing a provider client per vendor. It does not suit a project permanently bound to one vendor's bleeding-edge features, since the abstraction can only expose what fits the common shape. Before you adopt it, read the v0.6 to v0.7 migration guide, since Client::new() became fallible and the option structs gained fields, and decide the TLS backend early, because rustls-tls and native-tls are mutually exclusive at compile time.

Frequently asked questions

Official sources

1. Issues
2. jeremychone/rust-genai on GitHub
3. License: Apache-2.0
4. README