teilomillet/gollm: one Go interface for OpenAI, Anthropic, Groq and Ollama

The problem gollm solves for Go services

Calling two LLM vendors from one Go program normally means two SDKs, two error types, two retry stories and two ways of expressing a prompt. gollm's stated goal is to collapse that into one interface. The README describes it as a package that "simplifies and streamlines interactions with various LLM providers, offering a unified, flexible, and powerful interface for AI engineers and developers". The audience is Go developers building services, not researchers: the repository root holds gollm.go, config.go, prompt.go, stream.go, validation.go and moa.go, with providers/, optimizer/ and examples/ as subdirectories. The provider list in the README covers OpenAI, Anthropic, Groq, Ollama, Mistral and OpenRouter, and the .env.example file adds keys for Cohere, DeepSeek, Gemini, DashScope, Azure OpenAI and Lambda. That spread is the point. A team that wants to try GPT-4o-mini for cheap classification and Claude for long-context work can do it without rewriting the call site, because the option list is the same in both cases. The trade-off is that gollm sits between you and the vendor SDKs. When a provider ships a new parameter, you get it through gollm's option set or not at all until the package catches up.

How the abstraction actually works: options, prompts and providers

Construction goes through one function. gollm.NewLLM takes variadic option functions, and the README's quick reference shows SetProvider, SetModel, SetAPIKey, SetMaxTokens, SetTemperature, SetMemory, SetMaxRetries, SetRetryDelay and SetLogLevel. The provider string selects an implementation from the providers/ directory; the model string is passed through to that provider. Beyond the typed setters there is a generic escape hatch: llm.SetOption("fallback_models", []string{...}) and llm.SetOption("auto_route", true), both shown in the OpenRouter section. That generic path is how provider-specific features reach the wire without a new setter per feature, and it is also where you lose compile-time checking.

The prompt side is a separate type. gollm.NewPrompt takes the prompt text plus options such as WithContext, WithDirectives and WithOutput, so the instruction, the background and the expected shape are carried as fields rather than concatenated into one string. Generation returns a plain string via llm.Generate(ctx, prompt). Configuration can come from environment variables, from code, or from a file, and the .env.example file lists LLM_PROVIDER, LLM_MODEL, LLM_TEMPERATURE, LLM_MAX_TOKENS and LLM_LOG_LEVEL as defaults, with local servers addressed through OLLAMA_ENDPOINT (http://localhost:11434) and LMSTUDIO_ENDPOINT (http://localhost:1234/v1). The go.mod file shows the supporting cast: invopop/jsonschema and go-playground/validator for structured output, pkoukk/tiktoken-go for token counting, caarlos0/env for environment parsing, and golang.org/x/time for rate limiting. Those dependencies tell you more about the design than the feature list does. This is a library that expects to validate and count, not just forward bytes.

Installing gollm and making a first call

Installation is a single go get against the module path. The README gives one line:

go get github.com/teilomillet/gollm

After that, the basic usage example builds an LLM with an API key read from the environment, sends a prompt and prints the response. The example below is the README's own, shortened to the call itself:

llm, err := gollm.NewLLM(
    gollm.SetProvider("openai"),
    gollm.SetModel("gpt-4o-mini"),
    gollm.SetAPIKey(os.Getenv("OPENAI_API_KEY")),
    gollm.SetMaxTokens(200),
    gollm.SetMaxRetries(3),
    gollm.SetRetryDelay(time.Second*2),
)

Note that the README's example passes the API key explicitly with SetAPIKey rather than relying on the environment variables in .env.example. Both paths appear in the README, and which one applies depends on whether you load a config file. If the key is missing, the README's example calls log.Fatalf with the message "OPENAI_API_KEY environment variable is not set", so a misconfigured run fails at startup rather than at the first request.

For a local model instead of a hosted one, the .env.example file points Ollama at http://localhost:11434 and LM Studio at http://localhost:1234/v1, which the file marks as OpenAI-compatible. Pointing gollm at either means no API key and no per-token cost, at the price of running the model yourself. The repository also ships examples/basic_usage/, examples/providers/ and examples/custom_config/ if you want a file that compiles as-is rather than a snippet to paste.

Structured output and the validation dependency

The feature most likely to justify the dependency is JSON output validation. The README lists "Structured Output and Validation" with JSON schema generation and validation, and the go.mod file confirms both halves: invopop/jsonschema generates schemas, go-playground/validator checks values. There are examples for json_handling, data_extraction and compare_structured_output, plus a caching_structured example. The pattern implied by those pieces is that you describe the shape you want, gollm asks the model for JSON, and the result is validated before it reaches your code. That matters because a model that returns plausible-looking JSON with a missing field is worse than one that errors, and the validator dependency is what turns the first case into the second.

I would treat the schema path as the part to test hardest. Schema generation from Go types is where library behaviour and model behaviour interact: a type that generates a perfectly good schema can still be answered with a string where an integer was expected, and the README does not describe what gollm does at that boundary, whether it retries, repairs or returns the validation error. The examples/compare_structured_output/ directory is the place to look before you commit to it.

Chain of thought, memory, and the mixture-of-agents file

Above the transport layer, gollm offers pre-built task functions. ChainOfThought is the named one, described in the README as a pre-built function for complex reasoning tasks, with an examples/chain_of_thought/ directory. Memory retention is an option: SetMemory(4096) appears in the quick reference and there is a gollm_memory_test.go at the repository root, so the behaviour is covered by tests even though the README does not spell out the eviction policy. The moa.go file at the root, plus examples/mixture_of_agents/, indicates a mixture-of-agents path that combines responses from multiple providers, which the README lists under real-world applications.

The prompt optimizer is the most ambitious piece and the least documented in the README. It is described as automatically refining prompts "with support for custom metrics and different rating systems", and there is a dedicated optimizer/ directory plus examples/prompt_optimizer/ and examples/batch_prompt_optimizer/. What the README does not give is the algorithm: how many iterations it runs, what it does with a metric that never improves, or what it costs in API calls. If you are considering it, read optimizer/ directly. A prompt optimizer that silently multiplies your token spend is a different proposition from one that runs a fixed number of rounds, and the README does not tell you which this is.

Where gollm is the wrong choice

The clearest case against gollm is a single-provider Go service. If you only ever call OpenAI, the official Go client gives you every parameter on day one, and gollm adds an indirection layer plus the invopop/jsonschema, go-playground/validator, tiktoken-go and caarlos0/env dependencies on top. Unified interfaces earn their keep through the second provider; with one provider they are pure overhead.

The second case is bleeding-edge provider features. Anything reached through llm.SetOption with a string key is unchecked at compile time, and a typo in "fallback_models" produces a runtime surprise rather than a build failure. Providers that ship parameters faster than gollm's option set grows will leave you waiting or bypassing the abstraction.

The third case is non-Go codebases. This is a Go package with a go.mod declaring go 1.25. Python and TypeScript teams get nothing from it, and the DSPy topic tag in the repository metadata points at the inspiration rather than at interoperability: there is no Python binding in the repository. Finally, the README's Project Status section is not reproduced here, so there is no stated stability guarantee. The last push to the repository was on 2026-03-21, roughly six months before this writing, which is consistent with a project that is maintained but not moving quickly. If you need a provider added this month, that cadence is a real constraint.

gollm against calling provider SDKs directly

The honest alternative is not another Go LLM wrapper. It is the vendor SDKs themselves, or a router that runs outside your process. Calling the OpenAI and Anthropic Go clients directly gives you first-class access to each vendor's features, their own retry and streaming behaviour, and one fewer party in the upgrade path. The cost is that your application code carries the branching: two client types, two request shapes, two error taxonomies, and a prompt that has to be assembled twice. gollm's SetProvider and SetModel replace that branching with a string, and its WithDirectives and WithOutput prompt options replace hand-built message arrays.

The difference shows up most clearly in retries and validation. With raw SDKs you write your own backoff and your own JSON checking, or you pull in a validator anyway. gollm has SetMaxRetries, SetRetryDelay and the validator already wired, and the go.mod file shows golang.org/x/time present for rate limiting. That is a genuine head start for a service that talks to several providers. It is also the reason the dependency is hard to remove later: once your prompts are gollm.Prompt values and your outputs go through its schema path, swapping back means touching every call site. Choose accordingly at the start.

Editorial conclusion

Adopt gollm if you are writing Go services that call more than one LLM provider and you want provider switching, structured output validation and retry handling in one dependency rather than three SDKs. Skip it if you need only one provider and prefer that vendor's official Go SDK, or if you need a documented prompt-optimizer algorithm before you depend on it. Verify first that the provider and model strings you plan to use appear in the supported list, and check the optimizer/ directory and examples/prompt_optimizer/ against your Go version.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. Project website
4. README
5. teilomillet/gollm on GitHub