# safetensors: a tensor format that refuses to execute code

> The safetensors library stores tensors in a header plus a raw byte buffer, so a downloaded file can be memory-mapped and read without running anything. This is a practical look at its format, its Python and Rust install paths, and where it is the wrong choice.

**safetensors/safetensors** — Simple, safe way to store and distribute tensors

- Repository: https://github.com/safetensors/safetensors
- Website: https://huggingface.co/docs/safetensors
- Stars: 3,905 · Forks: 379
- Language: Rust
- License: Apache-2.0
- Published: 2026-09-23 · Updated: 2026-09-23 · Language: en
- Canonical page: https://hysenlabs.com/projects/safetensors-safetensors

## The problem safetensors solves is pickle, not serialization speed

PyTorch's default checkpoint path goes through pickle, and the README states the main rationale for this crate plainly: remove the need to use pickle on PyTorch. Pickle files can run arbitrary code when loaded, which makes a checkpoint from an untrusted source a code execution vector rather than a data file. safetensors is aimed at anyone who downloads weights from a model hub, mirrors a checkpoint between machines, or serves weights to other people. The README's comparison table scores pickle as unsafe, not zero-copy, not lazy-loading and without layout control, while safetensors scores positively on all four. The second audience is less obvious: people who need to inspect a checkpoint without loading it. Because the header is JSON and the tensor data sits in one contiguous buffer, a reader can list tensor names and shapes, then pull a single tensor out, without scanning the whole file. That matters in distributed setups where each worker wants a slice of a large checkpoint. The project is not trying to be a universal model container. It stores tensors, and the README's own table marks Flexibility as the one column where safetensors loses to pickle.

## Inside the safetensors file format: 8 bytes, JSON, then raw data

The layout is short enough to describe in full. The first 8 bytes are an unsigned little-endian 64-bit integer N, the size of the header. The next N bytes are a JSON UTF-8 string that must begin with the character { and may be padded with whitespace. That JSON is a dict mapping tensor names to objects with dtype, shape and data_offsets. The offsets are relative to the beginning of the byte buffer, not the file, and END is one-past the last byte, so a tensor's size is END minus BEGIN. Everything after the header is the byte buffer itself. A special key, __metadata__, holds a free-form string-to-string map; arbitrary JSON is not allowed there, and every value must be a string. Several constraints follow from that design. Duplicate keys are disallowed, though the README warns that not all parsers respect this. The byte buffer must be entirely indexed with no holes, which the README says prevents the creation of polyglot files. Endianness is little-endian and order is C or row-major. Tensor values are not validated, so NaN and plus or minus infinity can be present in a file. Empty tensors with a zero dimension and 0-rank scalar tensors are both accepted. The README also notes that sub-byte dtypes have appeared and make alignment tricky, possibly requiring non-traditional APIs.

## Installing safetensors with pip and reading your first file

The documented install is a single pip command. There is no configuration step and no service to start.

```bash
pip install safetensors
```

For the Python bindings from source you need Rust first. The README gives this sequence, which installs Rust, updates to the stable channel, clones the repository and builds the binding in place.

```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup update
git clone https://github.com/huggingface/safetensors
cd safetensors/bindings/python
pip install setuptools_rust
pip install -e .
```

The getting started example writes two 1024 by 1024 zero tensors and reads them back through safe_open. Note that save_file comes from safetensors.torch, while safe_open comes from the top-level safetensors package.

```python
import torch
from safetensors import safe_open
from safetensors.torch import save_file

tensors = {
   "weight1": torch.zeros((1024, 1024)),
   "weight2": torch.zeros((1024, 1024))
}
save_file(tensors, "model.safetensors")

tensors = {}
with safe_open("model.safetensors", framework="pt", device="cpu") as f:
   for key in f.keys():
       tensors[key] = f.get_tensor(key)
```

After running it you should have a file named model.safetensors in the working directory, and the second half of the snippet repopulates the tensors dict with the same two keys. The framework argument selects the tensor library and device selects where the tensor lands, which is the mechanism that lets you read a checkpoint onto CPU before deciding what to move to a GPU.

## What safetensors will not do, and when to pick something else

The format's restrictions are deliberate, and they bite in specific situations. There is no place for custom code, so a checkpoint that needs a Python class definition to reconstruct a model cannot be expressed. The README's comparison table gives safetensors a negative mark for Flexibility, the only column where it loses to pickle, and that is the honest summary: you trade the ability to store more than pure tensors for safety and zero-copy reads. The __metadata__ map is string-to-string only, so you cannot stash a nested config there. Values are not checked, so a file can contain NaN or infinity and safetensors will not tell you. Duplicate keys are disallowed by the format but the README warns that not all parsers enforce it, which means a file that reads fine in one tool may behave differently in another. Sub-byte dtypes are a real edge: the README notes they make alignment tricky and may require non-traditional APIs, so quantization schemes built on them are not a drop-in case. Finally, if your problem is a single model file that also carries architecture, tokenizer configuration and quantization metadata in one artifact, safetensors is the wrong layer. It is a tensor container, and the surrounding format has to come from somewhere else.

## safetensors against H5, SavedModel and the rest

The README's table compares safetensors with pickle, H5, SavedModel, MsgPack, Protobuf, Cap'n'Proto, Arrow, NumPy's npy and npz, and Paddle's pdparams, scoring each on safety, zero-copy reads, lazy loading, file size limits, layout control, flexibility and native bfloat16 or fp8 support. H5 is the closest in spirit: the README calls it safe and able to lazy-load, but not zero-copy, and says it is now discouraged for TF and Keras. It also points at a history of use-after-free issues in the HDF5 CVE list and contrasts 210k lines of code with roughly 400 lines for this library. SavedModel is safe but neither zero-copy nor lazy-loading. MsgPack, used by flax, is safe and zero-copy but not lazy-loading and not flexible. Protobuf, used by ONNX, is safe but fails zero-copy, lazy loading, file size limits and layout control. Arrow is marked with question marks across the board in the README's own table, which is worth noticing: the project declines to score it rather than claiming a win. The practical difference is layout control. A format can be lazily readable in principle and still force you to touch most of the file to reach one tensor if its metadata is scattered. safetensors puts every offset in one JSON header and every byte in one buffer, which is what makes single-tensor reads cheap.

## Licence, releases and the cost of keeping up

The repository is Apache-2.0, which permits commercial use and modification and includes a patent grant, but it also carries notice and attribution conditions that a redistributing product has to satisfy. That is a summary of the identifier, not legal advice; check the LICENSE file and your own obligations before shipping. On maintenance, the last push to the default branch was on 2026-09-22, one day before this writing, and the repository is not archived, so the project is being worked on. The release cadence visible in the tags is roughly quarterly at the stable level, with release candidates in between: v0.8.0-rc.1 on 2026-06-01, v0.8.0 on 2026-06-09, and v0.9.0-rc.0 on 2026-09-09. Upgrade cost is mostly about the format's stability rather than API churn. The README states that the subset of JSON accepted is implicitly decided by serde_json for this library, and that obscure representations of integers, newlines and escapes in UTF-8 strings might be modified later for safety reasons. That is a narrow warning, but it means a file that depends on unusual JSON encoding is the thing to watch across upgrades. The Makefile's doc target regenerates both README files with cargo readme, so the README you read on the repository root is generated from the crate, not hand-maintained.

## Conclusion

Adopt safetensors if you ship or download model weights and want the file to be readable without executing anything, and if your tensors come from PyTorch, TensorFlow or NumPy style libraries. Do not adopt it as a general model container: the README's own comparison marks Flexibility as its one missing column, so architectures, optimizers or custom Python objects have no place in the format. Before committing, verify two things yourself: that your loader keys match the names stored in the header, and that your dtype set is covered, since the format notes that sub-byte dtypes make alignment tricky and may need non-traditional APIs. If your pipeline already depends on pickle-saved checkpoints, the migration is a rewrite of the save and load calls, not a flag.

## FAQ

### What are safetensors for?

They store tensors in a file that can be read without executing code, replacing pickle as the default checkpoint format for PyTorch. The README's stated rationale is exactly that: remove the need to use pickle on PyTorch. They are also designed for zero-copy reads and lazy loading of individual tensors.

### Why are they called safetensors?

The name reflects the safety property in the README's comparison table: a randomly downloaded file can be loaded without running arbitrary code, which pickle cannot promise. The format also disallows duplicate keys and requires the byte buffer to be entirely indexed with no holes, which the README says prevents polyglot files.

### How do I open a safetensors file?

In Python, use safe_open from the safetensors package with a framework and device argument, then iterate f.keys() and call f.get_tensor(key) for each name. The README's example reads a file saved with save_file from safetensors.torch. The header is JSON, so a reader can also list names and shapes without pulling tensor data.

### Are safetensors files safe to use?

The format is designed so that reading a downloaded file does not run arbitrary code, which is the safety column the README scores positively. It does not validate tensor values, so NaN and plus or minus infinity can be present, and the README warns that not all parsers respect the rule against duplicate keys.

### How do I install safetensors?

The documented install is pip install safetensors. Building the Python bindings from source requires Rust: the README installs it with the rustup script, updates to the stable channel, clones the repository and runs pip install -e . inside bindings/python. There is also a crates.io package for the Rust side.

## Sources

- [License: Apache-2.0](https://github.com/safetensors/safetensors/blob/main/LICENSE)
- [Project website](https://huggingface.co/docs/safetensors)
- [README](https://github.com/safetensors/safetensors/blob/main/README.md)
- [Releases](https://github.com/safetensors/safetensors/releases)
- [safetensors/safetensors on GitHub](https://github.com/safetensors/safetensors)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/safetensors-safetensors