# simdutf8 trades error detail for speed, in two API flavours

> simdutf8 is a Rust crate that validates UTF-8 with SIMD, dispatching at runtime to the fastest implementation the CPU supports and falling back to the standard library when none is available. It offers a basic API optimised for valid input and a compat API that mirrors std::str::from_utf8() exactly, and the two differ in a way that decides where you can use it.

**rusticstuff/simdutf8** — SIMD-accelerated UTF-8 validation for Rust.

- Repository: https://github.com/rusticstuff/simdutf8
- Stars: 606 · Forks: 37
- Language: Rust
- License: NOASSERTION
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/rusticstuff-simdutf8

## Two APIs, and the choice between them is the design

simdutf8 is a high-speed, API-compatible UTF-8 validation library for Rust using SIMD extensions, based on the implementation from simdjson. It was originally ported to Rust by the developers of simd-json.rs and has since been heavily improved.

The API comes in two flavours, and the difference between them is the main thing to understand.

The basic API is for the fastest validation, optimised for valid UTF-8. It only checks for errors after processing the whole byte sequence, and it does not provide detailed information when the data is not valid UTF-8. Its error type, simdutf8::basic::Utf8Error, is a zero-sized struct, which is the point: there is nowhere to put a position.

The compat API is a fully API-compatible replacement for std::str::from_utf8(). It returns a simdutf8::compat::Utf8Error with valid_up_to() and error_len() methods. The first is useful for verification of streamed data, the second is useful for example when replacing invalid byte sequences with a replacement character.

The compat flavour also fails early: errors are checked on the fly as the string is processed, and once an invalid UTF-8 sequence is encountered it returns without processing the rest of the data. That comes at a slight performance penalty compared to the basic API even when the input is valid UTF-8.

So the rule is simple. If the data is trusted, use basic. If a caller can send you garbage and you need to say where it went wrong, use compat.

## Adding the dependency and the two call sites

The quick start is a Cargo dependency and a call.

```toml
[dependencies]
simdutf8 = "0.1.5"
```

The basic API is a drop-in replacement for std::str::from_utf8():

```rust
use simdutf8::basic::from_utf8;

println!("{}", from_utf8(b"I \xE2\x9D\xA4\xEF\xB8\x8F UTF-8!").unwrap());
```

If you need detailed information about a validation failure, the compat API is the one to use instead:

```rust
use simdutf8::compat::from_utf8;

let err = from_utf8(b"I \xE2\x9D\xA4\xEF\xB8 UTF-8!").unwrap_err();
assert_eq!(err.valid_up_to(), 5);
assert_eq!(err.error_len(), Some(2));
```

Both functions return a Result, so the unwrap and unwrap_err in the examples are the same shape as the standard library's, which is the point of the compat flavour. The error case in the example is a deliberately broken byte sequence, and the assertions show what you get back: the valid prefix length and the length of the invalid run.

The crate is written in pure Rust, and it has no-std support. The minimum supported Rust version is 1.38.0, and the manifest is on edition 2018.

The only runtime dependency is flexpect, and the build step uses rustversion, which is how the crate detects the compiler version and the target features at build time.

## Runtime dispatch on x86, compile-time selection elsewhere

The implementation selection rules are documented per architecture, and the reason for the differences is not about speed but about what each platform can tell you at run time.

On x86 and x86-64, the fastest implementation is usually selected at runtime using the std::is_x86_feature_detected! macro. The AVX 512 implementation requires Rust 1.89 or later and is only selected if the CPU supports the VBMI2 features, to avoid the throttling that happens with CPUs before Intel's Ice Lake microarchitecture. That is a careful default: shipping an AVX 512 path to a processor that downclocks under sustained wide-integer work would make the library slower than the code it replaced.

If you compile with RUSTFLAGS="-C target-cpu=native" on a recent x86-64 machine that supports AVX 512 with Rust 1.89 or later, the AVX 512 implementation is selected at compile time and runtime selection is disabled.

For no-std support, compiled with --no-default-features, the implementation is always selected at compile time based on the targeted CPU, and you pick it with RUSTFLAGS, using -C target-feature=+avx2 for AVX 2, -C target-feature=+sse4.2 for SSE 4.2, and -C target-feature=+avx512f,+avx512bw,+avx512vbmi,+avx512vbmi2 for AVX 512 with Rust 1.89 or later.

On aarch64, the SIMD implementation is used automatically since Rust 1.61. On armv7, the armv7_neon feature has to be enabled and a recent nightly compiler is required, with runtime selection through std::arch::is_arm_feature_detected! unless the target features are enabled, and some targets such as thumbv7neon-linux-androideabi have NEON on by default.

For wasm32, the implementation is selected at compile time based on the presence of the simd128 target feature, and you enable it with RUSTFLAGS="-C target-feature=+simd128". The README explains why: WASM has no way to detect SIMD through WASM itself, so although host environments offer something like wasm-feature-detect in the browser, there is no portable way to detect it from inside the library.

Where no SIMD is available, the library falls back to the standard library implementation.

## The speed claims and the hardware behind them

The README states two sets of figures, and the conditions are as important as the numbers.

On x86-64, the library is up to 23 times faster than the standard library on valid non-ASCII input, and up to four times faster on ASCII. On aarch64, it is up to eleven times faster than the standard library on valid non-ASCII input and up to four times faster on ASCII, with the aarch64 figures described as Apple Silicon. The README also claims it is faster than the original simdjson implementation.

The word up to matters, and so does valid. The gains are on valid non-ASCII data, which is the case where a byte-at-a-time validator does the most work, and the ASCII case is the conservative one, since ASCII is a single fast path in the standard library.

The benchmarks are described as done with criterion, with the tables created with critcmp, and the source code and data are in the bench directory. The naming schema is id-charset/size, where 0-empty is the empty byte slice and x-error/66536 is a 64KiB slice where the very first character is invalid UTF-8, and the library versions compared are simdutf8 v0.1.2 and simdjson v0.9.2, with simdutf8 compiled with inline(never) when comparing with simdjson.

Two configurations are named. The x86-64 one is a PC with an AMD Ryzen 7 PRO 3700 CPU, Zen2, on Linux with Rust 1.52.0. The aarch64 one is a Macbook Air with an Apple M1 CPU on macOS with rustc 1.54.0-nightly.

The benchmark tables are images in the README, so the numbers are not machine readable, and the older toolchain in the benchmark configuration is worth keeping in mind: the AVX 512 path and the aarch64 improvements since Rust 1.61 are not in those measurements.

## no-std, the public_imp escape hatch and the feature flags

The feature list in the manifest is a good map of the project's history, because two of the entries exist only to say they no longer do anything.

The default feature is std, which enables the CPU feature detection. Turning it off, by compiling with --no-default-features, is what switches the crate to no-std support, and that in turn forces compile-time implementation selection as described earlier. The reason the detection is tied to the std feature is that the detection macro lives in the standard library.

The public_imp feature exposes the validation implementations in the simdutf8::{basic, compat}::imp hierarchy, and the traits facilitating streaming validation are available there as well. That is the low-level door: if you want to call a specific SIMD implementation directly rather than letting the crate choose, this is how. The portable_public_imp feature builds on it and makes the portable SIMD implementation public, and it is marked experimental and nightly only.

Two features are marked deprecated and do nothing, aarch64_neon and hints. The aarch64_neon_prefetch feature enables aarch64 prefetching for a minor speedup and requires nightly.

There is one optimisation warning that matters more than it looks. Do not use opt-level = "z", which prevents inlining and makes the code quite slow. That profile is for size, and a SIMD validator that is inlined across the crate is a large part of where the speed comes from.

The repository also carries fuzzing/ and tests/ directories and an examples/streaming.rs, and the README states that the library has been thoroughly tested with sample data as well as fuzzing and that there are no known bugs.

## Where simdutf8 is the wrong tool

Three cases stand out, and one of them is a matter of picking the right flavour rather than the right library.

The first is that the basic API cannot tell you what went wrong. If your input can be malformed and the caller needs a byte offset, the basic API is the wrong choice, and the compat API costs you a little speed for a lot of usability. The two are not interchangeable in that situation, and the README is explicit that compat carries a slight performance penalty even on valid input.

The second is wasm32, where the library cannot detect SIMD by itself. The implementation is chosen at compile time from the simd128 target feature, so the decision moves to your build configuration, and a bundle that ends up running in a host without SIMD will take the non-SIMD path with no error. That is a workable arrangement, but it is one more thing to get right in a build you may not control end to end.

The third is the armv7 route, which needs a recent nightly compiler and an explicit feature flag, and for the no-std case, x86, which loses runtime dispatch entirely and needs the right target-feature flags.

There is a versioning consideration as well. The crate is published as 0.1.5, and the recent releases are v0.1.5 from 2024-09-22, v0.1.4 from 2022-04-02 and v0.1.3 from 2021-05-14. The version has moved slowly, which for a small, focused crate is a reasonable pattern, but it does mean the 0.1.x line carries the compatibility expectations of a pre-1.0 package, and that the compiler features it relies on, Rust 1.89 for AVX 512 and Rust 1.61 for aarch64, are floors that only some of your build targets will meet.

## Licence, maintenance and how to evaluate it yourself

The manifest declares the licence as MIT OR Apache-2.0, and the repository root carries both LICENSE-MIT and LICENSE-Apache. The repository listing, however, records the licence as a custom one that GitHub cannot classify, so the two licence files are the documents to read. Either way the terms are permissive, and the dependency is small: one runtime dependency and one build dependency.

The maintenance picture is the part to weigh. The repository is not archived and the last push was on 2026-06-22, so there is still work happening, but the last published release is v0.1.5 from September 2024. A project can be alive in the repository and stale in the registry at the same time, and for a dependency that is exactly the distinction that decides whether a security fix reaches you automatically.

The CHANGELOG.md and TODO.md files at the root are where the state of the work is recorded, and the build.rs file is where the compile-time feature detection happens, driven by rustversion.

If you want to judge the library rather than take the README's word for it, the repository gives you the means. The bench directory holds the criterion benchmark sources and the data, and the README names the machine and the compiler, so you can reproduce the comparison against std::str::from_utf8() on your own hardware with your own data. The fuzzing directory tells you what kind of adversarial input the validation has been tested against, and the streaming example shows the traits for validating data that arrives in pieces, which is the case where the compat API's valid_up_to() earns its keep.

For a library this small, that is a better evaluation path than any summary of the numbers in the README.

## Conclusion

simdutf8 fits a Rust service that validates large volumes of text on the hot path, where the standard library's byte-at-a-time scan shows up in a profile, and the compat API lets you adopt it without changing error handling at all. It is the wrong choice if you need the position of an invalid byte and take the basic path, if you are on wasm32 and cannot detect SIMD from inside the library, or if you want a dependency with a settled version line. Verify first that the implementation your CPU selects is the one you expect, since AVX 512 is only chosen on processors with VBMI2, and pin the version you tested, because the crate is still published as 0.1.x with releases spaced years apart.

## FAQ

### What is the difference between the simdutf8 basic and compat APIs?

The basic API is fastest on valid UTF-8 but only checks for errors after processing the whole input and provides no detail, with a zero-sized error type. The compat API is fully API-compatible with std::str::from_utf8(), returns an error with valid_up_to() and error_len() methods, and fails early, at a slight performance cost even on valid input.

### Which platforms does simdutf8 support?

It supports x86 and x86-64 with AVX 2 and SSE 4.2, and AVX 512 on modern CPUs since Rust 1.89, aarch64 since Rust 1.61, WASM with the simd128 target feature, and armv7 NEON on nightly Rust with the armv7_neon feature. Where SIMD is not available it falls back to the standard library implementation.

### How do I make simdutf8 use AVX 512 or AVX 2?

On x86 the fastest implementation is normally selected at runtime, and AVX 512 is only chosen when the CPU supports VBMI2. Compiling with RUSTFLAGS="-C target-cpu=native" on a recent x86-64 machine with Rust 1.89 or later selects AVX 512 at compile time, and for no-std builds you set -C target-feature=+avx2 or +sse4.2 explicitly.

### Is simdutf8 still on a 0.1 release line?

Yes. The latest release listed is v0.1.5 from 2024-09-22, with v0.1.4 from 2022 and v0.1.3 from 2021, while the repository itself is not archived and the last push was on 2026-06-22. The minimum supported Rust version is 1.38.0.

## Sources

- [Issues](https://github.com/rusticstuff/simdutf8/issues)
- [README](https://github.com/rusticstuff/simdutf8/blob/main/README.md)
- [Releases](https://github.com/rusticstuff/simdutf8/releases)
- [rusticstuff/simdutf8 on GitHub](https://github.com/rusticstuff/simdutf8)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/rusticstuff-simdutf8