# search: exact cosine search in Go with optional llama.cpp embeddings

> The search library is a small Go package that indexes vectors and finds nearest neighbours by exact cosine similarity, with local GGUF embeddings through llama.cpp and cloud embeddings through any OpenAI-compatible endpoint. The README is precise about the limits, and the biggest one is that search scans every vector.

**kelindar/search** — Go library for embedded vector search and semantic embeddings using llama.cpp

- Repository: https://github.com/kelindar/search
- Stars: 561 · Forks: 24
- Language: Go
- License: MIT
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/kelindar-search

## Three modules, and the dependency boundary between them

The library is split into three packages, and the split is the interesting design decision. The core github.com/kelindar/search package does vector indexing, search and persistence with no native library required. The llama module adds local GGUF embeddings through llama.cpp and purego, and the openai module adds text embeddings through OpenAI-compatible APIs, including OpenRouter. Each library has its own go.mod and go.sum, and the core search package has no dependency on either embedding module or on purego. The openai module uses only the standard library at run time, while the llama module owns the native inference dependency. Testify and its YAML dependency are used only by tests. That arrangement means an application that already has vectors from elsewhere can use the index without pulling in a native inference stack at all, which is the difference between a library you can drop into any Go binary and one you cannot.

## The index is exact, and the README says so plainly

The README's first line under the heading describes it as an exact cosine similarity index for small and medium datasets, and the usage example is four lines.

```go
index := search.NewIndex[string]()
index.Add([]float32{1, 0}, "cat")
index.Add([]float32{0, 1}, "dog")
results := index.Search([]float32{1, 0}, 1)
fmt.Println(results[0].Value) // cat
```

The generic parameter is the value type, so the index holds a string alongside each vector and you get your own data back from the results. The semantics are worth reading closely. Add normalizes the vector in place and retains its backing array, so you must not modify it afterwards, and Search normalizes the query in place as well. Results carry cosine similarity in descending order, so larger Relevance values are better. The practical rule that follows is that you need nonzero, finite vectors from the same model with the same dimensions for all documents and queries, and that changing models means re-embedding the dataset even if the dimensions happen to match.

## The scaling limit and the concurrency rules are stated, not hidden

Two constraints are documented in the same short section, and both are the kind of thing that saves you from discovering them in production. The first is scale. Search scans every vector, and the README says large datasets may need a different indexing algorithm. That is an honest statement of what a brute force exact cosine search costs, and it means the library is aimed at corpora you can hold in memory and scan per query. The second is concurrency. Concurrent searches require separate query slices and no concurrent index writes, because the query is normalised in place. The existing file formats, ReadFile, WriteFile, ReadFrom and WriteTo, are unchanged, so persistence is not a constraint. The combination is a library that is small enough to read in an afternoon and that tells you exactly where its edges are.

## Local embeddings through llama.cpp, with the native library built by hand

The llama module loads a GGUF model and embeds text, and the constructor is where the native library is touched.

```go
model, err := llama.New("dist/MiniLM-L6-v2.Q8_0.gguf", 0)
if err != nil {
    return err
}
defer model.Close()
vector, err := model.EmbedText(ctx, "Your text here")
```

The lifecycle rules are specific and worth following. The library initialises on the first valid constructor call and never on import, initialisation failures are returned as errors and cached for the process lifetime, and library paths must be configured before constructing a model. Zero GPU layers means the work happens on the CPU. Embedding calls on a model may run concurrently, but all calls must finish and any explicitly created contexts closed before the model is closed, individual contexts must not be shared concurrently, and contexts retain their token accounting. Cancellation is checked around native inference and between batch items, and it cannot interrupt native inference that is already running.

## Cloud embeddings, and a provider interface you can implement yourself

The openai module talks to any OpenAI-compatible endpoint, and OpenRouter is the explicit example.

```go
client, err := openai.New(os.Getenv("OPENAI_API_KEY"), "text-embedding-3-small")
if err != nil {
    return err
}
vector, err := client.EmbedText(ctx, "Your text here")
vectors, err := client.EmbedBatch(ctx, []string{"first document", "second document"})
```

For OpenRouter you pass its base URL and model identifier, and the optional configuration takes dimensions, headers and an HTTP client, with defaults of the OpenAI /v1 endpoint, provider-default dimensions, no extra headers and a 60-second HTTP timeout. Headers are copied and cannot override authorization or content type, and clients may be shared concurrently as long as the supplied HTTP client is not mutated during use. Batches use one HTTP request and return vectors in input order, empty batches do no work, failures return no partial results, and there are no automatic retries or batch splitting. Cloud support covers text only, not multimodal inputs or provider-specific routing. Any function returning a float slice can supply vectors to the index without importing either provider package, through a small interface.

```go
type Embedder interface {
    EmbedText(context.Context, string) ([]float32, error)
}
```

Both built-in providers satisfy it.

## Building the native library, and the fixtures that come with it

Local embeddings need the native library built, and the README gives the sequence. You need a C and C++ compiler and CMake, and the repository is cloned with its submodules and the model and vector fixtures retrieved.

```bash
git submodule update --init --recursive
git lfs pull
```

The submodule step checks out the pinned llama.cpp source, and the LFS step pulls the model and vector fixtures. The native source is pinned to llama.cpp v0.4.1, and there is a warning worth heeding: the binaries in the dist directory predate that pin, so you should build the wrapper from source for the updated version. The build instructions apply only to local embeddings. On Ubuntu the compiler and CMake come from the package manager, and the commands the README gives are an apt update followed by an apt install.

## Module tags, isolation in CI, and a v0.5 release

The release scheme is worth knowing if you depend on a specific module. Nested modules are released with their own tags, in the form llama/vX.Y.Z and openai/vX.Y.Z, while root releases continue to use the plain vX.Y.Z form. Tests and module maintenance run with GOWORK=off in CI specifically to verify that the modules are isolated from each other. The version line is 0.5.0, released on 2026-09-19, with 0.4.1 in March 2026 and 0.4.0 in March 2025, and the last push was on the same day as the 0.5.0 release. The module requires Go 1.27 with a toolchain line pinning 1.27.1, and the go.mod pulls in a small set of direct dependencies, an IO stream helper, a CPU feature detection library and testify, with the YAML and system packages indirect. The library is MIT licensed.

## Conclusion

search fits a Go project that needs semantic search over a few thousand or a few hundred thousand vectors and wants the whole thing in one binary with no Python in the loop. It does not fit a large corpus, because the README says search scans every vector and that large datasets need a different indexing algorithm, and it does not fit a project that needs multimodal embeddings. Verify first that every document and query comes from the same model with the same dimensions, since changing models means re-embedding the dataset even when the dimensions match, and read the concurrency rules before you run searches and writes on the same index concurrently.

## FAQ

### What does the kelindar/search library do?

It is an exact cosine similarity index for small and medium datasets, with optional local and cloud embedding providers. It provides vector indexing, search and persistence in the core package, local GGUF embeddings through llama.cpp in the llama module, and text embeddings through OpenAI-compatible APIs in the openai module.

### Does search work with large datasets?

Not well, and the README says so. Search scans every vector, and large datasets may need a different indexing algorithm. The library is aimed at small and medium datasets that fit in memory.

### How do I use local embeddings with search?

Import the llama module, build the native library so that the pinned llama.cpp source and the model and vector fixtures are available, then construct a model with llama.New pointing at a GGUF file. The library initialises on the first valid constructor call, never on import, and zero GPU layers uses the CPU.

### Can I use a custom embedding provider with search?

Yes. Any function returning a slice of float32 can supply vectors to the index without importing either provider package, by implementing a small Embedder interface with an EmbedText method taking a context and a string. Both built-in providers satisfy the same interface.

## Sources

- [Issues](https://github.com/kelindar/search/issues)
- [kelindar/search on GitHub](https://github.com/kelindar/search)
- [License: MIT](https://github.com/kelindar/search/blob/main/LICENSE)
- [README](https://github.com/kelindar/search/blob/main/README.md)
- [Releases](https://github.com/kelindar/search/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/kelindar-search