hf-mem estimates model memory before you download the weights

The sizing problem hf-mem exists to solve

Picking a checkpoint for inference usually starts with a guess. The repository page shows you how large the weight files are, but file size is not runtime footprint: the framework allocates activations, and for language models it also reserves a key value cache whose size depends on sequence length and batch size. Reading a model card and guessing produces either a wasted GPU or an out of memory failure on the first request.

hf-mem is a CLI written in Python that estimates inference memory requirements for Hugging Face models. The project states that it covers Transformers, Diffusers and Sentence Transformers models, and more generally any Hub repository that carries Safetensors or GGUF weights. The intended user is someone with a hardware budget and a list of candidate models who needs a number per candidate before committing to a download.

The design constraint that shapes everything else is stated plainly: the tool is lightweight and depends only on httpx2, because it never downloads weights. That makes it cheap to run against many candidates, which is the scenario it is built for.

Metadata over HTTP range requests instead of a full download

The mechanism is the interesting part. Both Safetensors and GGUF keep their structural information at the front of the file, so hf-mem issues HTTP Range requests to fetch only those leading bytes. The metadata is then parsed locally to work out the tensor inventory and the shape of the model, without pulling the tensor data itself.

The packaging reflects that design. The dependency list in pyproject.toml contains a single entry, httpx2 with the http2 extra, and the project requires Python 3.10 or newer. The build backend is uv_build, and the console entry point is declared as hf-mem mapping to hf_mem.cli:main. There is no local model cache to warm up and no index to download, so a first run against a fresh repository is cheap.

The output is a Result object rather than a printed table of numbers. The documented example shows the fields you get back: model_id, revision, filename, memory, kv_cache, total_memory and details. Keeping the estimate in a typed object is what makes the Python entry point useful for scripting, since the same values are available without scraping terminal output.

The KV cache estimate lives behind the experimental flag

By default you get the weight side of the picture. The more interesting number for a served language model, the key value cache, requires the --experimental flag, which is also where the caveats start.

With that flag enabled, the cache estimate applies to causal language models and to conditional generation models, that is architectures named ...ForCausalLM and ...ForConditionalGeneration. Three options shape the result. The --max-model-len option defaults to the value found in config.json, --batch-size defaults to 1, and --kv-cache-dtype defaults to auto, which means it uses the data type set in config.json under torch_dtype or dtype, or the value from quantization_config when one applies. So the cache line is a function of declared configuration, not of what a server will actually allocate under a real request mix.

For Mixture of Experts models the same flag adds a weights only breakdown of the base model and of all the experts combined, which is the number you want when a single expert is loaded at a time but the file lists many.

GGUF repositories with many quantisations

Quantised GGUF repositories behave differently from single-file safetensors repositories. If a repository contains GGUF weights, they are listed by default and the memory is estimated for each one. That listing only happens when the repository has no Safetensors weights; where both are present the GGUFs are ignored, so the estimate covers the safetensors path and stops there.

When you want one specific quantisation rather than every one in the repository, the --gguf-file option narrows the target to a single file. This matters because a GGUF repository usually holds many quantisations of the same model at very different sizes, and the useful question is usually about one of them.

The same applies to the safetensors side of a repository, where a single model can be split across several weight files. Estimating the whole set gives a repository level answer; pointing at one file gives a file level answer. In both cases the tool is reporting metadata arithmetic, so a repository that stores its weights in an unusual layout will get an answer that reflects the metadata rather than a runtime trace.

First run with uvx, then the same call from Python

The documented CLI path runs the tool without installing anything into a project environment:

uvx hf-mem --model-id MiniMaxAI/MiniMax-M2

That prints an estimate for a Transformers model. The same form works for a Diffusers repository and for a Sentence Transformers embedding model, for example:

uvx hf-mem --model-id Qwen/Qwen-Image

uvx hf-mem --model-id google/embeddinggemma-300m

If you want the cache estimate as well, add the flag:

uvx hf-mem --model-id MiniMaxAI/MiniMax-M2 --experimental

From Python the same estimate comes back as an object. There is a synchronous call and an awaitable variant for code already running inside an async application:

from hf_mem import run

result = run(model_id="MiniMaxAI/MiniMax-M2", experimental=True)
print(result)

from hf_mem import arun

result = await arun(model_id="MiniMaxAI/MiniMax-M2", experimental=True)
print(result)

Two further integrations are described. The tool can be registered as a Hugging Face Hub CLI extension so that it runs as hf mem alongside your other extensions, and it can be exposed to a coding agent as a skill by placing the SKILL.md file at .claude/skills/hf-mem/SKILL.md.

Where the estimate stops being the whole picture

The honest limits are visible in the documentation. The project carries a warning that it is still experimental and subject to major changes across releases, so breaking changes may occur until v1.0.0. A tool whose output you put into a capacity plan can therefore change shape between releases, and the release list shows how often: 0.5.4 on 2026-05-11, 0.5.5 on 2026-05-12 and 0.5.6 on 2026-09-07.

The arithmetic itself is narrow. Weights plus a KV cache computed from declared config values is not the same as measured allocator behaviour. The default batch size of 1 means a cache number that is easy to underestimate once you serve concurrent requests, and the documentation does not say how activations, allocator fragmentation or attention implementation choices are handled. Nor does it describe a way to validate the figure against a running process.

The alternative is to do the arithmetic yourself: read the tensor shapes from config.json and the safetensors header, sum them by dtype, and add a cache term you compute under your own length and batch assumptions. That approach costs more effort and gives you no support for GGUF files or MoE breakdowns, but every assumption stays visible to you. A third option, loading the model and watching peak usage, is the ground truth and also the slowest.

Licence terms and what an upgrade actually costs

hf-mem is released under the MIT licence, and pyproject.toml points the licence file at LICENSE in the repository root. That means the code can be read, modified and reused inside your own tooling, including the small wrapper scripts that turn an estimate into a deployment check. The README itself notes a short form post at alvarobartt.com/hf-mem and warns that it was written in January 2026 and is not up to date, so treat that page as history and the repository as the reference.

The repository is not archived, and the last push was on 2026-09-07, with the 0.5.6 release published the same day. The gap between 0.5.5 in May and 0.5.6 in September suggests releases are not on a fixed monthly cadence, so you should not plan around a schedule.

The practical upgrade cost for a pre 1.0 project is that the output format and the flag names may move. If a team wires hf-mem into a deployment script, pin the version that the script was validated against and recheck the printed fields after each release, rather than tracking main and assuming the interface is stable.

Editorial conclusion

Adopt hf-mem when you are comparing several Hub checkpoints against a fixed GPU budget and want a fast first pass; skip it if you need a guaranteed fit, because the KV cache number moves with the flags you choose and the project warns that breaking changes may land until v1.0.0. Before trusting a result, run the same command against a model you already have resident and compare the printed memory figure with the real allocation, then pin the exact release you validated.

Frequently asked questions

Official sources

1. alvarobartt/hf-mem on GitHub
2. License: MIT
3. Project website
4. README
5. Releases