huggingface/llm-vscode: Ghost-Text Completion With a Pluggable Backend
LLM powered development for VSCode
At a glance
- What is it?
- This VSCode extension pairs a TypeScript client with the llm-ls language server and lets you point completions at Hugging Face Inference API, Ollama, OpenAI-compatible endpoints, or TGI. The interesting part is the tokenizer configuration and the attribution check, not the ghost text itself.
- Who is it for?
- Adopt llm-vscode if you already run a completion-capable endpoint (Ollama, TGI, or an OpenAI-compatible server) and want VSCode ghost text without a hosted vendor. Skip it if you need a polished out-of-the-box experience; the free Inference API tier is explicitly rate limited and the backend URL rules require care.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 112 days ago.
- What is it written in?
- Mainly TypeScript, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap llm-vscode fills: ghost text without a vendor lock
Most inline completion tools in VSCode assume you will send your code to one hosted service. llm-vscode takes the opposite position. The README states that requests for code generation are made via an HTTP request, and that you can use the Hugging Face Inference API or your own HTTP endpoint provided it adheres to the listed backend APIs. That single sentence is the whole pitch. The extension is a client; the model is whatever you point it at.
The audience is therefore narrow but real. It is for engineers who already have a completion-capable model running somewhere (a local Ollama instance, a Text Generation Inference deployment, a llama-cpp-python server) and want it wired into VSCode. It is also for people who want to try StarCoder through the Hugging Face Inference API with minimal setup, since the README says the default is bigcode/starcoder plus the Inference API. What it is not is a managed product. There is no hosted control plane, no account beyond an HF token, and no promise that the free tier will keep up with your typing.
How llm-vscode routes a completion request through llm-ls
The extension does not talk to the model directly. It uses llm-ls as its backend, and by default that binary is bundled with the extension. The README describes the request shape with a worked example. Given a Python buffer with the cursor between two imports and a function definition, the prompt sent to the model is assembled from start, end, and middle tokens, with the text before the cursor, the text after the cursor, and the cursor position itself slotted between them. The resulting inputs string is placed into a request body alongside whatever is in configuration.requestBody, and posted to an endpoint built from configuration.backend and configuration.modelId.
The response is read as JSON and typed as an object with a generated_text field. That field name matters: any backend you configure must produce it, because the client expects that shape. The README notes the example is simplified, but the data flow is clear enough. Editor buffer to prompt assembly to HTTP POST to generated_text back to ghost text. There is no local inference in the extension itself and no model weights shipped with it.
URL construction is handled for you, with caveats. Depending on the backend, the extension appends the correct path to the base URL, for example {url}/v1/completions for the openai backend. If no URL is set for the huggingface backend it falls back to a default; other backends error instead, because there is no sensible default. If you already included the correct path, it will not be added twice. The llm.disableUrlPathCompletion setting turns this behavior off entirely. That is a small feature, but it is the difference between a working custom endpoint and a 404 you spend an afternoon debugging.
Configuring backends, tokenizers, and the token that pays for it
Installation is the normal VSCode extension flow from the marketplace listing. After that, the first thing to set is the Hugging Face token, which the README says you supply through the command palette: open it with Cmd/Ctrl+Shift+P, type Llm: Login, and paste a token from hf.co/settings/token. If you have previously run huggingface-cli login on the same machine, the extension reads the token from disk instead.
The backend setting accepts four values: huggingface (the default), ollama, openai for any OpenAI compatible API, and tgi for Text Generation Inference. The tokenizer configuration is where the extension earns its keep. The README gives four forms for llm.tokenizer. Setting it to null disables tokenization and llm-ls counts characters instead. Setting a path points at a local tokenizer.json. Setting a repository and optional api_token downloads tokenizer.json from the root of a Hugging Face repo; when api_token is null it reuses the token from Llm: Login. Setting a url and a to path downloads the tokenizer over HTTP GET to a local file. The stated purpose is that the prompt will always be sized to fit the context window, with token counts produced by tokenizers.
Two more settings shape when suggestions appear. llm.enableAutoSuggest toggles suggest-as-you-type. llm.documentFilter takes a DocumentFilter or an array of them, so you can scope completions to { pattern: "**/*.{py,rs}" } or restrict them to a project directory. On the keyboard side, Cmd+shift+l triggers editor.action.inlineSuggest.trigger and Cmd+shift+a runs the llm.attribution command.
The attribution check is a Bloom filter, and it says so
The code attribution feature is the most distinctive thing here and also the most honestly documented. Pressing Cmd+shift+a checks whether generated code appears in The Stack, using stack.dataportraits.org as a rapid first pass. The README is explicit about the mechanism: sequences of at least 50 characters are matched against a Bloom filter. It then states the consequences directly. False positives are possible, and long enough surrounding context is necessary. For a complete second pass it points at the dedicated Stack search tool, described as a full dataset index.
That framing is worth taking at face value. A Bloom filter answers membership with no false negatives but with false positives, so a hit is a signal to look closer rather than a verdict. The 50-character floor also means short or heavily reformatted matches can slip past. If your reason for running this extension is licence hygiene around generated code, treat the built-in check as a triage step and budget for the second pass. The README does not describe any automatic blocking or filtering based on the result; it is a check you invoke.
Where llm-vscode stops being the right tool
The clearest limitation is stated by the project itself. A note in the README warns that when using the Inference API you will probably encounter some limitations, and suggests subscribing to the PRO plan to avoid getting rate limited in the free tier. For anyone who leaves llm.enableAutoSuggest on and types continuously, a rate-limited endpoint means suggestions that arrive late or not at all. The fix is either a paid plan or a self-hosted backend, and the second option is the one this project is actually built around.
The second constraint is the response contract. Because the client reads generated_text, an endpoint that returns a different JSON shape will not work no matter how capable the model is. The README frames this as adherence to the listed backend APIs, which is a polite way of saying you may need a translation layer in front of your server.
The third is platform support. llm-ls is bundled by default, but the README anticipates that it may not cover your platform: when developing locally, or if you built your own binary because your platform is not supported, you set llm.lsp.binaryPath to the binary's location. That is a working path, not a trivial one. It means compiling llm-ls yourself and keeping it in sync with the extension. There is also only a single release listed, v0.1.0 from September 2023, so version churn is not something you can plan around from release notes alone.
How this differs from Copilot-style completion
The obvious comparison is GitHub Copilot, which is a hosted service with a fixed model and no backend configuration. llm-vscode inverts that: the model is your choice, the endpoint is your choice, and the trade is that you own the operational side. If your completion server is down, your ghost text is down, and there is no vendor to escalate to.
A closer comparison is llm.nvim, the Neovim extension from the same organization. Both use llm-ls as the backend, so the request assembly, tokenizer handling, and backend list are shared. The difference is the host editor and the integration surface: llm-vscode is TypeScript and hooks into VSCode's inline suggestion and command APIs, while the Neovim plugin lives in that editor's ecosystem. If your team is split across both editors, the shared llm-ls layer means your endpoint and tokenizer configuration translate, but the settings keys and keybindings do not. The README also lists a jupyter extension and an intellij extension, so the same backend can serve several editors under one deployment.
Licence, maintenance, and what to check before you install
The repository is Apache-2.0, which permits commercial use and modification and includes an explicit patent grant. That is the permissive end of the spectrum, and it matters here because you may be running this against proprietary code. The licence covers the extension code; it says nothing about the model weights you point it at, and those carry their own terms. bigcode/starcoder, the default, is a separate download with a separate licence, and the README does not address that distinction. Check the model licence separately before deploying this on a commercial codebase; that is not legal advice, just a boundary the repository does not resolve for you.
On maintenance cost, the honest answer from the material is that this is a thin client over an external server. Your recurring work is keeping the endpoint reachable, keeping the tokenizer file matched to the model (a mismatch silently changes how the prompt is truncated), and rebuilding llm-ls if you are on an unsupported platform. The bundled binary removes most of that for common platforms, which is why llm.lsp.binaryPath exists as an escape hatch rather than a default. The project lists a single release and a last push in May 2026, so treat the extension as stable-but-quiet rather than actively evolving, and pin your own llm-ls build if you depend on the custom-binary path.
Editorial conclusion
Adopt llm-vscode if you already run a completion-capable endpoint (Ollama, TGI, or an OpenAI-compatible server) and want VSCode ghost text without a hosted vendor. Skip it if you need a polished out-of-the-box experience; the free Inference API tier is explicitly rate limited and the backend URL rules require care. Before committing, verify three things: that your endpoint returns a generated_text field, that llm.tokenizer points at a tokenizer.json matching your model, and that the bundled llm-ls binary runs on your platform or that llm.lsp.binaryPath is set.
Community notes