llm-ls: A Rust LSP Server That Puts the LLM Behind the Editor Protocol

The duplication problem llm-ls is built to remove

Every editor has its own extension model. Neovim extensions are written in Lua, VS Code and IntelliJ extensions in TypeScript and Kotlin respectively. If each of those extensions talks to an LLM directly, each one reimplements the same four things: assembling a prompt from the surrounding file, counting tokens so the request fits the model's context window, deciding whether a completion should be a single line or several, and translating the provider's HTTP response into something the editor can render. That is four places to fix the same bug.

llm-ls takes those responsibilities and moves them behind the Language Server Protocol. The README states the goal directly: to provide a common platform for IDE extensions to be built on, with llm-ls handling the heavy lifting of interacting with LLMs so extension code can stay lightweight. The audience is therefore not end users looking for an AI autocomplete plugin. It is extension authors, and the people who maintain the integrations that already exist: llm.nvim, llm-vscode and llm-intellij are all listed as compatible, while jupytercoder is listed as not yet done.

What actually runs when you type

The server is written in Rust and speaks LSP. An editor extension connects to it as it would to any language server, and the completion work happens inside that process rather than in the extension host.

Prompt construction uses the current file as context. The README says the server can use fill-in-the-middle or not, depending on your needs, which matters because FIM models expect the text after the cursor as a suffix, and non-FIM models expect a plain prefix. Before sending anything, llm-ls tokenizes the prompt and checks it against the context window so the request is not rejected for length. That tokenization step is the reason the server owns prompt assembly at all: the extension cannot know how many tokens its context will consume without knowing the model's tokenizer.

Completion shaping is the more interesting part. According to the README, llm-ls parses the AST of the code to decide whether a completion should be multi line, single line, or empty. That is a real design decision rather than a formatting preference. A server that always returns multi-line completions will produce suggestions that duplicate closing braces the user already typed. A server that always returns single-line completions cannot express a whole function body. Gating on the syntax tree means the decision is made from the code's structure, not from a heuristic over the model's output.

Telemetry is local. The README is explicit that llm-ls does not export data anywhere, apart from setting a user agent when querying the model API, and that everything is stored in a log file at ~/.cache/llm_ls/llm-ls.log when the log level is set to info. Requests and completions are recorded there with retraining in mind. That is a log file on disk, not a pipeline, and the README does not describe any upload path.

Backends: one interface, four very different operational profiles

The README lists four compatible targets: Hugging Face's Inference API, Hugging Face's text-generation-inference, ollama, and OpenAI-compatible APIs such as the python llama.cpp server bindings.

Those are not interchangeable in practice. The Inference API is a hosted endpoint, so the request leaves the machine and you depend on someone else's latency and rate limits. text-generation-inference and ollama are things you run yourself, which means the machine running the language server needs the capacity to serve the model, and the completion latency is now your hardware's problem. The OpenAI-compatible path is the widest, because it covers llama.cpp's server bindings and anything else that speaks that schema.

What the README does not give is a per-backend support matrix. It does not say which backends support fill-in-the-middle, which ones accept a suffix parameter, or whether tokenization is done with a model-specific tokenizer or an approximation. If you are choosing a backend for a FIM model, that gap is the first thing to resolve in the source rather than in the README.

Getting it running and the config surface you will actually touch

The README does not include an install section. There is no cargo install line, no prebuilt binary link, and no sample configuration block. What it does give is one concrete path and one concrete key.

The path is the log file: ~/.cache/llm_ls/llm-ls.log, written when the log level is set to info. That is the first thing to check after the server starts, because it tells you whether requests are reaching a backend at all.

The key is suffix_percent, which appears in the roadmap rather than in the feature list. The roadmap describes it as a setting that determines the ratio of the number of tokens for the prefix versus the suffix in the prompt. Because it is listed as planned, you should confirm whether it exists in the version you install before relying on it. The same applies to the other roadmap items: multi-file workspace context, changing context_window to max_tokens, filtering bad suggestions such as repetitive output or suggestions identical to the text below the cursor, and OLTP traces. These are stated intentions, not shipped behaviour.

If you need installation instructions, build steps or the full settings schema, they are not in the README as supplied. Plan to read the repository directly.

Where it breaks: the AST dependency and the pre-1.0 warning

The completion mode decision depends on parsing the file's AST. That is a hard dependency on language coverage. For a language the parser handles well, the server can tell a multi-line completion from a single-line one. For a language it handles poorly, or for a file that is mid-edit and syntactically invalid, which is exactly the state a file is in when you are asking for a completion, the parse may not produce the structure the decision needs. The README does not describe a fallback for that case. If you edit templating languages, dialects, or files with embedded languages, this is the failure mode to test first.

The second constraint is stated at the top of the README in a callout: this is currently a work in progress, expect things to be broken. That is the maintainers' own characterisation, and it should be read alongside the release history. The three most recent releases listed are 0.5.3 in May 2024, 0.5.2 in February 2024, and 0.5.1 one day earlier in February 2024. The version number has not reached 1.0, and the README does not promise API stability. An extension that pins to a specific llm-ls version is safer than one that tracks the latest build.

There is also a scope limit worth naming. The roadmap lists multi-file workspace context as unsupported. Prompt context today comes from the current file. For repositories where the useful completion depends on a type defined three files away, that is a ceiling on suggestion quality, not a bug to be worked around.

The alternative: call the model from inside the extension

The obvious alternative is to skip the server and have each extension talk to the model API directly. That is what many editor AI plugins do, and for a single-editor project it is a reasonable choice: no extra process to install, no LSP handshake, no version skew between the extension and a server binary.

The difference in approach is where the logic lives. Direct calls put prompt assembly, token counting and completion shaping in the extension's language, which for llm.nvim means Lua and for llm-vscode means TypeScript. llm-ls puts them once in Rust and exposes them over LSP. The trade is clear: the direct approach duplicates work per editor but has no additional moving part; the server approach removes the duplication but adds a process the user must have installed and running, and adds a protocol boundary between the extension and the logic. If you maintain one extension for one editor, the server buys you little. If you maintain or plan to maintain several, the duplication cost is the thing llm-ls exists to remove.

Maintenance cost, licence, and what the version number implies

The last push to the repository is dated 2026-05-26, while the newest release listed is 0.5.3 from 2024-05-24. The material does not explain that gap, so the honest reading is that development activity and tagged releases are not moving together, and you should check the commit history before assuming a release reflects current code.

Upgrading between 0.x versions carries the usual pre-1.0 risk. A minor version bump can change behaviour, and the README's own warning about things being broken means the changelog, not the version number, is what tells you whether an upgrade is safe. Because the server is a separate binary from your extension, you also own the compatibility question in both directions: an old server with a new extension, and the reverse.

The licence is Apache-2.0. That is a permissive licence, which generally means you can use, modify and redistribute the code, including in commercial products, provided you meet its notice and attribution conditions. It also includes an explicit patent grant. This is a description of the licence, not legal advice; if you are shipping llm-ls inside a commercial product, have your own counsel review the notice requirements and any modifications you distribute.

Editorial conclusion

Adopt llm-ls if you are building or maintaining an editor extension and want the LLM transport, prompt assembly and token budgeting handled outside your extension code, and if you are willing to read the source when the README runs out. Do not adopt it as a turnkey coding assistant for a team of mixed editors, and do not treat the 0.5.3 release as a stable API surface. Before wiring it into anything, verify three things against the current tree: the exact config keys it reads at startup, whether your chosen backend path works without the Hugging Face Inference API, and whether the AST completion logic covers the languages you actually edit, since a parse failure there changes what the server returns.

Official sources

1. huggingface/llm-ls on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases