llm.nvim: Ghost-Text Completion in Neovim via llm-ls and Your Own Endpoint

The gap llm.nvim fills: completions without a hosted subscription

Copilot-style completion in Neovim is normally a hosted product. llm.nvim takes the opposite position: the plugin is a client, and the model behind it is whatever you can reach over HTTP. The README describes it as a plugin for all things LLM that uses llm-ls as a backend, and notes it was influenced by copilot.vim and tabnine-nvim. That lineage matters, because it tells you the interaction model is ghost text, not a chat panel. The target reader is a Neovim user who already has, or is willing to run, an inference endpoint: a local Ollama daemon, a llama-cpp-python server exposing an OpenAI-compatible route, a TGI container, or the Hugging Face Inference API. The project was formerly named hfcc.nvim, so older configuration snippets circulating under that name refer to the same plugin. One caveat sits in the README itself: on the Inference API, the note warns that you will probably encounter limitations and points at the PRO plan to avoid free-tier rate limiting. If your plan is to use Hugging Face's hosted endpoint heavily, budget for that or run your own server.

Architecture: a Lua client talking to a Rust language server

The split is the most important thing to understand before installing. llm.nvim does not call a model directly. It talks to llm-ls, a separate binary, and llm-ls makes the HTTP request. Generation requests go out over HTTP, and the backend is selected by name: huggingface, ollama, openai or tgi. Each backend maps to a different URL path, and the README documents which one: llm-ls uses /api/generate for Ollama, /v1/completions for the OpenAI-compatible backend, and /generate for TGI. The plugin will try to append the correct path if your configured url does not already end with it, and you can turn that off with disable_url_path_completion. Authentication is a single header: when api_token is set, it is passed as Authorization: Bearer <api_token>. Model routing differs per backend. On Hugging Face the model identifier is appended to the URL as {url}/model/{model}. On Ollama and the OpenAI-compatible backend the model value is placed in the request body instead. That asymmetry is a real source of confusion when switching backends, and it is worth reading the note under each example before copying a config. The backend URL can also be overridden wholesale with the LLM_NVIM_URL environment variable, and when url is nil it falls back to the Inference API default.

The context window is sized with a real tokenizer, not a character count

The README states that the prompt sent to the model will always be sized to fit within the context window, with the number of tokens determined using tokenizers. This is the feature that separates llm.nvim from naive completion scripts that slice the buffer by characters. Model context windows differ by an order of magnitude across the configurations shown: Starcoder is listed at 8192 and CodeLlama-13b-hf at 4096. If you swap models without swapping context_window, the prompt sizing is computed against the wrong budget. The tokenizer block takes a repository field, and the examples point it at the same repository as the model. There is a documented escape hatch: setting tokenizer = nil makes llm-ls count characters instead. That is a fallback for models with no usable tokenizer, and it will over- or under-fill the window depending on the language and the density of the code, so treat it as a last resort rather than a default. The README also shows a tokens_to_clear list per model, for example <|endoftext|> for Starcoder and <EOT> for CodeLlama, which strips model-specific control tokens from the returned text before it reaches the buffer.

Fill-in-the-middle is configured per model, and the spacing is load-bearing

Completion here is not plain left-to-right generation. The fim block is enabled with prefix, middle and suffix markers, meaning the model receives text from before and after the cursor and returns the span between them. For Starcoder those markers are <fim_prefix>, <fim_middle> and <fim_suffix>. For CodeLlama they are <PRE> , <MID> and <SUF>, and the README attaches an explicit warning under that example: spaces are important here. Note the markers as written carry leading or trailing spaces, and dropping them changes the prompt the model sees. This is a configuration detail that will silently degrade output quality rather than raise an error, which makes it the kind of thing you debug by reading the config twice. The README states that the Starcoder values are the default config values, so a user who installs the plugin and changes nothing gets Starcoder's markers, token list, 8192 context window and tokenizer repository. If you point the backend at a different model family and leave the defaults in place, you are sending one model's fill-in-the-middle tokens to another model. The per-model sections exist precisely because that mismatch is easy to create and hard to notice.

Getting it running: token resolution order and the llm-ls binary

Token handling has a documented precedence. In order: api_token in plugin opts, which the README flags as not recommended if you version your config files; the LLM_NVIM_HF_API_TOKEN environment variable; a file at $HF_HOME/token; or running huggingface-cli login, which writes the token to the right path. Model selection has its own precedence: LLM_NVIM_MODEL first, then model in plugin opts. llm-ls itself is installed by llm.nvim on first load, downloaded from the release page into vim.api.nvim_call_function("stdpath", { "data" }) .. "/llm_nvim/bin". If your platform has no published binary, or you want a managed install, the README gives a Mason route: run :MasonInstall llm-ls, then set lsp.bin_path to vim.api.nvim_call_function("stdpath", { "data" }) .. "/mason/bin/llm-ls". Two more keys matter. lsp.version is used only when llm.nvim downloads llm-ls from the release page, so it has no effect on a Mason or self-built binary. lsp.cmd_env sets environment variables for the llm-ls process. For debugging, the README notes you can start llm-ls over TCP with the --port option.

Where it stops being the right tool

The clearest limitation is that llm.nvim is a completion client. The README's feature list covers ghost-text code completion, model choice and context-window sizing. There is no chat interface, no instruction-following edit command and no multi-file refactoring workflow described. If your need is "explain this function" or "rewrite this module", this plugin does not address it. The second limitation is operational. Because llm-ls is a separate binary downloaded at first load, an air-gapped machine or a platform without a release artifact will not get a working install from the default path, and the README's answer is to build your own and point lsp.bin_path at it. That is a build toolchain dependency, not a config toggle. Third, the hosted path has a stated ceiling: the README's own note says Inference API users will probably hit limitations and directs them to the PRO plan to avoid free-tier rate limiting. Fourth, there is no release history to reason about. The repository shows no retrieved releases, so the plugin is consumed from the main branch, and lsp.version is the only lever you have to pin the backend binary. If you need a versioned, changelog-backed upgrade path for the client side, this project does not currently offer one.

How it differs from Copilot and Tabnine in Neovim

The README names copilot.vim and tabnine-nvim as influences, and the difference is worth stating precisely because the surface looks the same. All three render ghost text in the buffer. Copilot and Tabnine route your code to a vendor-operated service and the model is not yours to choose. llm.nvim inverts that: the request goes to a URL you configure, with a token you supply, and the model identifier is a string you set. The practical consequence is that a local Ollama instance with codellama:7b, configured with url = "http://localhost:11434" and a request_body carrying temperature and top_p under options, keeps the code on your machine. The cost of that inversion is everything the hosted products absorb for you. You choose the model, you match the fill-in-the-middle markers to it, you set context_window to the model's actual size, you run or pay for the endpoint, and you keep llm-ls updated. The trade is control for operational work, and it is a fair trade only if you actually wanted the control.

Licence, maintenance and what to check before you commit

llm.nvim is Apache-2.0. That is a permissive licence with an explicit patent grant, and it is compatible with the usual practice of vendoring a plugin into a dotfiles repository. It covers the Neovim plugin; llm-ls is a separate repository with its own licence, and the models you point at carry their own terms, which for some Hugging Face Hub models are not Apache-2.0. None of that is legal advice, and if you are shipping a locked-down developer image you should read the licence of each component rather than assuming the plugin's licence covers the stack. On maintenance: the repository is not archived and the last push recorded is 2026-05-26, so the project is active, but with no retrieved releases the client is consumed from main and lsp.version controls only the downloaded llm-ls binary. Practically, that means your upgrade path is a git pull plus a possible llm-ls version bump, and your rollback path is a commit hash. Before adopting, verify three concrete things: that your chosen model's fill-in-the-middle tokens match the fim block you configure, that context_window equals that model's real window, and that llm-ls has a binary for your platform or that you can build one and set lsp.bin_path.

Editorial conclusion

Adopt llm.nvim if you already run a local inference server or hold a Hugging Face token and want completions inside Neovim without a hosted subscription, and if you accept that llm-ls is a separate binary you install and update yourself. Skip it if you want a single managed service with zero local process management, or if you need chat-style refactoring rather than inline ghost text. Verify two things before committing: that a model matching your backend exposes the fill-in-the-middle tokens listed under Models, and that llm-ls ships a binary for your platform, since the README directs unsupported platforms to build their own and set lsp.bin_path.

Official sources

1. huggingface/llm.nvim on GitHub
2. Issues
3. License: Apache-2.0
4. README