LLM VRAM calculator
Estimate the GPU memory an open model needs under llama.cpp or Ollama — weights, KV cache and overhead — and which graphics cards it fits.
Result
The result will appear here.Whether a model runs on your graphics card comes down to three numbers: the quantised weights, the KV cache that grows with the context length, and the runtime's own buffers. This calculator works them out for 24 popular open-weight models — Llama, Qwen, DeepSeek-R1 distills, Mistral, Gemma and Phi — at any GGUF quantisation, context length and number of parallel sequences, the way llama.cpp (ggml-org/llama.cpp, MIT) and the tools built on it, such as Ollama and LM Studio, allocate memory. The architecture numbers are read from each model's config.json on Hugging Face.
How it works
- Weights are the parameter count times the quantisation's effective bits per weight, taken from the file sizes llama.cpp's quantize tool lists: Q4_K_M averages 4.9 bits, not 4, because K-quants keep some tensors at higher precision.
- The KV cache is 2 × layers × KV heads × head size × cached tokens × bytes per element, so grouped-query attention, the cache type and the context all show up in it. On Gemma 2 and 3 the sliding-window layers only ever cache their window.
- Overhead is half a gigabyte for the CUDA or Metal context plus a compute buffer sized by llama.cpp's default 512-token micro-batch, which is why a large vocabulary adds memory.
- The GPU table marks a card ✓ below 90% use, ≈ between 90 and 100%, and ✗ ×N with the number of such cards needed to split the model; Macs are counted at about three quarters of their unified memory, the share macOS lets the GPU use by default.
Where your data goes
Nowhere. This tool runs entirely in your browser: the text you paste is processed by the page and is never transmitted to a server or written to a log.
This tool is free and needs no account. Its results exist only in your open page and are not saved anywhere.
What it costs
This tool is free, with no sign-in and no points.
Common questions
- How accurate is the estimate?
- For a single GPU with every layer offloaded, the weights and KV cache match what llama.cpp allocates to within a few percent; for Llama 3.1 8B at Q4_K_M and 8,192 tokens the tool gives 4.58 GiB of weights and exactly 1 GiB of KV cache, the figures llama.cpp reports. Leave 5–10% headroom for the driver, the desktop and other programs.
- What does parallel sequences mean, and how does it relate to Ollama's context?
- It is how many conversations the server holds at once, each with its own KV cache: OLLAMA_NUM_PARALLEL in Ollama, or -np in llama-server. The context field is per sequence, so 8,192 × 4 caches 32,768 tokens; in llama-server terms that is -c 32768 -np 4.
- Should I quantise the KV cache?
- q8_0 halves the cache with a loss that is hard to measure, and it is a good trade for long contexts; q4_0 quarters it but can hurt quality noticeably. In llama.cpp a quantised V cache needs flash attention, which is turned on automatically on GPUs that support it; the estimate assumes flash attention throughout.
- Why does it warn about the native context?
- Every model is trained up to a certain length — 40,960 tokens for Qwen3, 131,072 for Llama 3.1. Beyond that the memory estimate still holds, but the model needs RoPE scaling such as YaRN to work at all, and quality usually drops, so the tool tells you when you have gone past it.
The open-source behind it
This tool is a self-contained implementation. ggml-org/llama.cpp (MIT) does the same job as a library — if you need this behaviour inside your own program, start there rather than calling a web page.
ggml-org/llama.cppAlso known as
- llm vram calculator
- gpu memory calculator llm
- how much vram for llama
- gguf vram
- kv cache size calculator
- ollama vram requirements