ggml: A Dependency-Free Tensor Library Where the Backend Is the Product

The Constraint That Explains Everything Else

The README lists six properties, and one of them governs the rest: plain C/C++ implementation without any dependencies. That single sentence explains why the project ships its own CMake build rather than a Python packaging story, why the supported targets are enumerated as x86, ARM, RISC-V, LoongArch, PowerPC, s390x and WebAssembly instead of being described as broadly portable, and why the tensor format is documented as a file specification rather than a serialization helper. A dependency-free library cannot call cuBLAS, cannot call oneDNN, and cannot call Accelerate. Everything has to be written down in the repository.

The second constraint is stated just as plainly: zero memory allocations during runtime. For an inference loop this means the graph and its buffers are sized ahead of execution, not grown during it. The practical consequence is that a model's memory footprint is a decision made at load time. The cost is that the library is a poor fit for workloads whose shapes change per call, because a dynamic allocation strategy is exactly what has been designed out.

This is a library written for the person embedding a model inside a larger C program, not for the person exploring a model in a notebook.

How the Backend Split Works in Practice

The README describes broad backend support across CPU, GPU, NPU and browser, and separately states that SIMD-optimized kernels exist for x86, ARM and RISC-V. Those two sentences describe different layers. The kernels are the arithmetic; the backend is the dispatch target. When you build the project, CMake is where that split becomes visible, because backend selection is a build-time decision rather than a runtime one. That is a deliberate trade: it keeps the binary small and the dispatch path short, but it means a single build artifact does not cover every accelerator on the machine.

Quantization sits alongside the backend layer. The README claims 2- to 8-bit integer quantization plus MXFP4 and NVFP4 microscaling formats. Note that the quantization formats are a property of the weights, while the kernels are a property of the hardware. A format can exist in the library without having a tuned kernel on every backend, and the README does not publish a compatibility matrix mapping formats to backends. If your plan depends on a specific low-bit format on a specific accelerator, that pairing is the first thing to confirm from the source tree rather than from the feature list.

Getting a Build and a First Graph

The quick start is four commands and no configuration file:

git clone https://github.com/ggml-org/ggml cd ggml mkdir build && cd build cmake .. cmake --build . --config Release -j 8

The README does not list any CMake options, so the material does not support naming flags for enabling a GPU backend or a specific SIMD path. Treat the bare `cmake ..` as the documented default and read the CMake files themselves for the available switches.

For a first program, the README points at examples/simple, described as a minimal, fully commented example doing matrix multiplication. That is the right starting point because it exercises the parts that matter: tensor creation, a graph, and execution. The documentation section links three further resources: docs/gguf.md for the GGUF file format, a Hugging Face blog post titled Introduction to ggml, and a GGML Tips & Tricks wiki page hosted in the llama.cpp repository. The wiki living in a different repository is worth noting. It means the practical guidance for this library is maintained next to its most prominent consumer, not next to the library itself.

What the Repository Does Not Promise

There is no autograd in the description. No optimizers, no training loop, no dataset abstraction. The one-line summary is tensor library for machine learning, and the feature list is about portability, kernels, backends and number formats. If your requirement is to fine-tune a model, nothing in the supplied material indicates ggml is the tool for it.

The second gap is documentation depth. The README links an external blog post as the introduction and an external wiki for tips. There is a GGUF format document in docs/, but the material does not show an API reference, a function-level guide, or a compatibility table for backends and quantization formats. For a library whose selling point is that it runs on seven CPU families plus accelerators, the absence of a published format-by-backend matrix is the sharpest limitation visible here. You will be reading C++ headers.

The third gap is the contribution path. The README asks that changes to the core library, including the CMake build system, be opened as pull requests in llama.cpp rather than here, on the stated grounds that this makes them more visible, better tested, and more likely to be reviewed. That is an honest description of how the project actually works, and it is also a warning that patches to this repository may sit outside the main review flow.

Where ggml Differs From PyTorch and ONNX Runtime

PyTorch is the obvious comparison and the difference is not performance, it is the contract. PyTorch gives you a Python-first imperative API with autograd, dynamic shapes and a large operator surface, and it accepts a heavy dependency tree in exchange. ggml gives you a C API with no dependencies, a fixed set of number formats, and memory planned before execution. If you need to differentiate through your computation, PyTorch is the answer and ggml is not in the conversation.

ONNX Runtime is the closer alternative, because both projects exist to execute a model inside another application. The split is in what you bring. ONNX Runtime consumes a graph defined by an external specification and provides execution providers as plugins, so the model format and the runtime are separable concerns. ggml defines its own format, GGUF, documented in docs/gguf.md, and compiles its backends in. Choosing ggml means adopting its file format as well as its runtime. That is a tighter coupling, and it is the reason the ecosystem around this library looks the way it does.

Versioning, Licence and Upgrade Surface

The release cadence visible in the material is fast: v0.21.0 on 2026-08-21, v0.22.0 on 2026-08-25, v0.23.0 on 2026-09-04. Four days between the first two, ten days to the third. Pre-1.0 version numbers plus a release every week or two means the API is not being held stable for you. Pinning to a tag and reading the diff before moving is the only defensible upgrade policy the material supports.

Building from source is the documented path, and the README gives no package-manager instructions for any language. That is consistent with a dependency-free C library, but it shifts the integration cost onto you: your build system has to vendor or fetch the source, and your CI has to rebuild it. There is no published ABI stability statement here.

The licence is MIT, per the README badge and the repository metadata. MIT is permissive, so redistribution inside a closed product is generally workable, but the usual obligations apply: keep the copyright and permission notice with the distribution, and check how the licence of any model weights you load interacts with it. That is a question about the weights, not about ggml, and it is not something this material can answer. Nothing here is legal advice.

Who This Is For, and What to Check Before You Commit

The fit is narrow and specific. You are writing C or C++, you need to run a model on hardware you do not fully control, and you want one source tree rather than a stack of vendor libraries. The supported target list and the zero-allocation runtime are aimed squarely at that situation.

The misfit is equally specific. If you want to train, if you want Python, if you need a stable API across releases, or if your model shapes change per call, the design choices here work against you rather than for you.

Before writing production code, confirm the pairing that the README does not publish: which quantization formats have kernels on the backend you intend to ship. Start from examples/simple, then read docs/gguf.md if you are producing or consuming weights, and treat the llama.cpp wiki as the practical reference it appears to be. The project's own contribution policy is the clearest signal of where the center of gravity sits: core changes go to llama.cpp, so plan to track both repositories, not just this one.

Editorial conclusion

Adopt ggml if you are shipping a C or C++ binary that must run on hardware you do not control, and if 2- to 8-bit quantization plus MXFP4 or NVFP4 microscaling is part of your inference plan. Do not adopt it if your team lives in Python and expects autograd, because ggml is a tensor library and the README makes no claim of a training stack. Before committing, verify three things yourself: that your target architecture appears in the supported list, that the quantization format you need is actually implemented for your backend, and that you can build the repository cleanly on your toolchain, since the README's only build instructions are the four CMake commands above.

Official sources

1. ggml-org/ggml on GitHub
2. License: MIT
3. Project website
4. README
5. Releases