llfio: a filesystem library that never copies the whole file

Written for the standard library, then abandoned by it

llfio is a low level file I/O and filesystem library for the C++ standard, and its history explains both its design and its current state.

It is a complete rewrite after a Boost peer review in August 2015. It was submitted for C++ standardisation in 2018, and that process was abandoned in 2025. So the library that began as a proposal has lived for seven years as an independent project, and the standards track that would have given it a long life and a slow-moving compatibility promise is gone.

That is the central fact about the project and it colours everything else. A standards-track library and a research library in production have different risk profiles, and llfio is now the second kind.

The target hardware explains the rest of the design. The library was originally intended for storage devices with about a microsecond 4Kb transfer latencies, and for those supporting Storage Class Memory or Direct Access Storage. The stated i/o overhead, including syscall overhead, has been benchmarked to 100 nanoseconds on Linux, which the README puts in terms of a theoretical maximum of 10M IOPS at queue depth 1, or approximately 40GB per second per thread. It has particularly strong support for writing portable filesystem algorithms that work well with directly mapped non-volatile storage such as Intel Optane.

The README is careful about the maturity of the code. It notes that most of this code is mature quality, that it has been shipping in production with multiple vendors for some years, and that amongst many big data solutions it powers the low level custom database component of the US Security and Exchange Commission's MIDAS solution, which ingresses terabytes of trade data per day. It also says it is considered quite reliable on Windows and Linux, and less well tested on Mac OS.

The handle hierarchy is the API

The v2 architecture is described in a table where each row is a change and the two columns are whether it is new in v2 and whether it came from the Boost peer review feedback. The handle hierarchy is the row that defines the whole design.

LLFIO v1 had a single handle type, and v2 splits it into a hierarchy of types. The list is worth reading as a description of what a filesystem library actually has to do.

A handle provides open, close, get path, clone, set and unset append only, change caching, and characteristics. An fs_handle is a handle with an inode number. A path_handle is a race free anchor to a subset of the filesystem. A directory_handle enumerates the filesystem. An io_handle adds synchronous scatter-gather I/O and byte range locking. A file_handle adds open and create file, and get and set maximum extent. And a mapped_file_handle adds low latency memory mapped scatter-gather I/O.

Each level adds exactly one capability, which is what makes the hierarchy usable. A caller who only wants to enumerate a directory takes a directory_handle and nothing more, and a caller who wants range locks takes an io_handle.

The other rows in the table explain the rest of the v2 direction. There is a universal native handle and file descriptor abstraction instead of void *. There is perfectly or ideally low memory allocation and deallocation per operation, usually none. The API is noexcept throughout, returning error_code for failure instead of throwing exceptions. Cancelable I/O was made possible by dropping XP support. All shared_ptr usage was removed because all use of multiple threads was removed. The use of std::vector to transport scatter-gather sequences was replaced with C++ 20 span borrowed views. And completion callbacks became an arbitrary type U&& instead of a future continuation, with type erasure for its storage bound into the one single memory allocation needed to execute the operation, so the overhead is optimal.

The design promises: no races, no copies, no exceptions

The characteristics list is a set of hard claims, and they are the reason to look at this library at all.

A race free filesystem design is used throughout, meaning no TOCTOU. That is a structural guarantee rather than a locking convention, and it comes from the path_handle concept, which is an anchor to a subset of the filesystem rather than a path string that is re-resolved on each call. The example directory has path_view_openat.cpp and wg21_path_view_equivalent.cpp, which suggests the comparison with the standard library's proposal is an explicit part of the work.

The allocation and copy promises are stated together: zero malloc, zero exception throw and zero whole system memory copy design used throughout, even down to paths, which can hit 64Kb. That last clause is the interesting one, because a path is exactly the thing a conventional filesystem API copies around on every call, and the design avoids it.

The original error code is always preserved, even down to the original NT kernel error code if an NT kernel API was used. The configuration for that is optional and based on P1028, the SG14 status_code and standard error object for P0709 zero-overhead deterministic exceptions. So the library is deliberately compatible with the deterministic exceptions proposal, which is a forward-looking choice for code that must run with exceptions disabled.

Other characteristics round it out. It is portable to any conforming C++ 17 compiler with a working Filesystem in its STL, and it works with C++ exceptions and RTTI globally disabled. It is fully clean with C++ 20, and will make use of Coroutines, Concepts, Span and Byte if you have them, otherwise swapping in C++ 17 compatible alternatives.

Platform targets are Microsoft Windows, Linux, Android, iOS, Mac OS and FreeBSD, with best effort support for older kernels up to their end of life, which as of July 2020 means Windows 8.1 or newer, Linux 2.6.32 or newer which is the RHEL EOL, Mac OS 10.13 or newer, and FreeBSD 11 or newer.

The filesystem algorithms were broken out into their own library

One of the v2 changes has consequences beyond the library itself, and it is the row about filing system algorithms being made generic and broken out into a public llfio::algorithms template library, described as the LLFIO FTL.

Breaking it out means the filesystem algorithms that llfio ships are usable on their own, which is exactly the component a database needs when it wants to place its own structures on raw flash or on a DAX device rather than going through a conventional file system.

That is the context for the MIDAS reference in the README. A system that ingests terabytes of trade data per day and needs a low level custom database component is doing something quite specific with storage, and the fact that a public project is credited with powering it tells you the level at which this code operates. It is not a convenience wrapper around the platform API; it is a replacement for it at the layer where the platform API is the bottleneck.

The rest of the v2 work is more ordinary but still substantial. There are four choices of algorithm implementing a shared filing system mutex, so the locking strategy is pluggable. Storage profiles are a YAML database of behaviours of hardware, OS and filing system combinations, which is a way of recording what was learned from testing across the combinations in the CI matrix. The dependency on the ASIO and Networking TS was removed completely, and absolute and interval deadline timed I/O was added throughout.

There is also a much more comprehensive set of memory map and virtual memory facilities, with support for SCM and DAX and page cached storage including large, huge and super pages, and much more granular micro level unit testing of individual functions.

Building it, and the examples that show the intent

The build instructions live in a separate Build.md at the repository root, linked from the README, and the top level gives a good picture of the machinery.

The project uses CMake, CTest, CDash and CPack, with automatic usage of C++ Modules or precompiled headers where available. There is a .ci.cmake, a .docs.cmake, a CTestConfig.cmake, a Doxyfile for the documentation, and a .clang-tidy along with a .clang-format for the code style. The CDash project is registered as Boost.AFIO, which is the name the project used when it was going through the Boost review process, and the test dashboards are public.

There are separate GitHub Actions workflows for unit tests on Linux, Mac OS and Windows, and prebuilt binaries are published on the releases page.

The examples directory is the clearest statement of what the library is for, because the file names describe the use cases. There is map_file.cpp and mapped_file.cpp for memory mapping a file for read, sparse_array.cpp for a sparsely stored temporary disposable array, read_stats.cpp for reading metadata about a file and its storage, read_directory.cpp, read_entire_file1.cpp, scatter_write.cpp, malloc1.cpp and malloc2.cpp, and path_view_openat.cpp for the race free path handling. There are also wg21_path_view_benchmark.cpp and wg21_path_view_equivalent.cpp, which benchmark the path view against the equivalent standard proposal.

The snippet references in the README point at three of these, mapping a file for read, a sparse array, and reading file statistics, and the docs site at ned14.github.io/llfio carries the API documentation plus a todo list of the already implemented parts.

For a sense of what the library can do, the repository also links a database of latencies for various previously tested operating systems, file systems and storage devices in the fs-probe program's results, which is a rare thing to find in a filesystem library and a useful reference in its own right.

Where llfio is the wrong choice

The first and simplest answer is that most applications do not need this. If you open a file, read it and close it, the C++ standard library and the platform APIs already do that, with less code, fewer concepts and a support commitment that does not depend on one maintainer's time. llfio is a library for the layer below that, and its complexity is only justified when you are building a file system, a database, or something else that has to control the storage layer itself.

The second is the state of the standardisation. The submission was abandoned in 2025, which means the compatibility and review benefits that a standards-track library would have accumulated are not going to arrive. A standard filesystem library exists; llfio is a separate thing with a different shape, and it will not converge with the standard.

The third is the dependency risk that comes with a single-maintainer project at this altitude. The README says the project has been shipping in production with multiple vendors and powers a component of a government system, which is reassuring in one sense and a concentration of reliance in another. A library that sits underneath a securities database is not going to be abandoned casually, but it is also not going to accept a broad set of feature requests from outside that use case, and the maintenance burden is one person.

The fourth is the licensing, and it is the first thing to check. The repository's licence field is one that GitHub cannot classify, with the terms in the Licence.txt file at the repository root. A library intended to be linked into commercial products needs terms you can point a lawyer at, and a custom licence that a package manager cannot classify is exactly the kind of thing that stalls a procurement review. Read Licence.txt before you build on it, because the code is good enough that people will be tempted to skip that step.

There is also a platform caveat worth weighing: the README says the library is considered quite reliable on Windows and Linux, and less well tested on Mac OS. If your target is a Mac, that is the platform to think about first.

Editorial conclusion

llfio is a good fit for a C++ system doing its own file system work on fast storage, especially on SCM or DAX where a conventional read of a large file is the bottleneck, because the library is built around handles, ranges and mappings rather than around whole file buffers, and the error handling is noexcept and returns error codes; it is the wrong fit for an application that just needs to read a file, because the standard library and the platform APIs already cover that, and before you adopt it you should read Licence.txt at the repository root, because the licence field is a custom one that GitHub cannot classify and this is code you are likely to link into a product.

Frequently asked questions

Official sources

1. Issues
2. ned14/llfio on GitHub
3. Project website
4. README
5. Releases