bstr gives Rust byte strings that do not have to be valid UTF-8

Extension traits, not a new string type

The bstr crate takes an unusual approach for a string library. Rather than asking you to switch to a new owned type, it provides extension traits for &[u8] and Vec<u8>. Once the trait is in scope, a byte slice behaves like a string for the operations the crate implements, and you keep working with the types the standard library already gives you.

The distinction the crate draws is between conventionally UTF-8 and validly UTF-8. The README says byte strings are conventionally UTF-8, differing from String and str in that they are not required to be valid UTF-8 but may be fully or partially valid UTF-8. The standard library's String cannot hold bytes that are not valid UTF-8 at all, so anything that comes from a file, a socket or another program has to be validated, replaced with U+FFFD, or handled as raw bytes before it can become a String.

That is the gap bstr occupies. The short version the README gives is that byte strings are useful when it is inconvenient or incorrect to require valid UTF-8. The crate's own description is blunter still: a string type that is not required to be valid UTF-8.

For a text-processing library, the decision to add methods to existing types rather than introduce a parallel type has consequences you will feel in your code. You can call a bstr method on a &str with a trivial conversion, and you can call it on a &[u8] that is definitely UTF-8 without any conversion at all.

Searching is per-algorithm instead of generic

One design decision in the README is worth pulling out, because it explains why bstr has methods the standard library does not. A large part of the crate's API was taken from the standard library, so much of it sits on solid ground. The main difference lies in how the substring search routines work. The standard library provides generic infrastructure for supporting different types of searches with a single method, while this library prefers to define new methods for each type of search and drop the generic infrastructure.

In practice that means bstr offers separate methods rather than one method with a configuration argument. A reader coming from std will notice the difference in method names, and anyone porting code will spend a moment mapping old calls onto new ones. The payoff is that each search is a direct call with no runtime dispatch on a strategy enum, and no wrapper type to unwrap afterwards.

The crate is also explicit about scope. Regular expressions and Unicode collation are listed as probably out of scope, while Unicode normalization and more sophisticated Unicode case handling are listed as probable future considerations. The latter would likely draw on the caseless and unicase crates. The author notes that the exact scope is not quite clear but expects to be able to iterate on it, which is an unusual thing for a README to admit about a stable 1.x crate.

The dependency story supports the same low-weight goal. The README says every dependency of bstr except for memchr is optional, and Cargo.toml confirms it: memchr is required, while serde_core and regex-automata are marked optional. The unicode feature is what pulls in regex-automata, and it is what gives you the grapheme, word and sentence segmenters.

The four cargo features decide what you compile in

Feature selection is where the real trade-off with bstr lives, and the README documents all four.

The std feature is enabled by default and provides APIs that need the standard library, such as Vec<u8> and PathBuf, and enabling it also enables alloc. The alloc feature is also enabled by default and covers APIs that allocate through the alloc crate. The unicode feature is enabled by default and provides APIs that require sizable Unicode data compiled into the binary, including grapheme, word and sentence segmenters; with it disabled, basic support such as UTF-8 decoding is still included.

The unicode feature carries a caveat. Enabling it currently also requires enabling the std feature, and the README says this limitation is expected to be lifted at some point. So a no_std user who wants grapheme segmentation cannot have it today without also taking std.

The serde feature enables implementations of serde traits for BStr, and for BString when alloc is enabled. Cargo.toml shows the mechanism, with the feature declared as serde = ["dep:serde_core"] and the default set pulling in std and unicode.

[features]
default = ["std", "unicode"]
std = ["alloc", "memchr/std", "serde_core?/std"]
alloc = ["memchr/alloc", "serde_core?/alloc"]
unicode = ["dep:regex-automata"]
serde = ["dep:serde_core"]

The doc comment in Cargo.toml notes that the feature setup is complex enough to be worth opting into a nightly unstable option, and the docs.rs metadata sets all-features = true with the doc_cfg feature so every feature combination is documented.

Reading lines quickly and counting words without decoding

The README's examples show the two halves of the crate in use at once: the byte string API and the I/O helpers. The line iterator comes from bstr's io module, through the BufReadExt trait, and the per-line methods come from ByteSlice.

The first example reads stdin and prints only the lines containing a particular substring, using for_byte_line_with_terminator and contains_str:

use std::{error::Error, io::{self, Write}};
use bstr::{ByteSlice, io::BufReadExt};

fn main() -> Result<(), Box<dyn Error>> {
    let stdin = io::stdin();
    stdin.lock().for_byte_line_with_terminator(|line| {
        if line.contains_str("Dimension") {
            stdout.write_all(line)?;
        }
        Ok(true)
    })?;
    Ok(())
}

The second counts words in a Unicode-aware way, line by line, using the words method and its count. The third uppercases a stream without validating UTF-8 and without allocating per line, by calling to_uppercase_into with a buffer that is cleared and reused:

let mut upper = vec![];
stdin.lock().for_byte_line_with_terminator(|line| {
    upper.clear();
    line.to_uppercase_into(&mut upper);
    stdout.write_all(&upper)?;
    Ok(true)
})?;

The README states the point of that example directly: converting a stream to uppercase without performing UTF-8 validation and amortizing allocation is quite a bit faster on standard ASCII text than what the standard library APIs can easily do, and any invalid UTF-8 bytes are passed through unchanged.

The fourth example shows the grapheme cluster path, taking the first 10 visual characters from each line using grapheme_indices, where invalid UTF-8 sequences are generally treated as a single character and passed through correctly. The examples directory carries the paired standard library versions, graphemes-std.rs next to graphemes.rs, lines-std.rs next to lines.rs, uppercase-std.rs next to uppercase.rs and words-std.rs next to words.rs, so the difference in behaviour is meant to be read directly.

MSRV 1.65, a conservative release policy, and a custom licence file

The version policy is stated plainly. The minimum supported rustc version is 1.65, the crate says it will be conservative with respect to that floor, and that the MSRV may be bumped in minor version releases. Cargo.toml confirms rust-version = "1.65" with edition = "2021".

The commitment on major versions is stronger still. The README says that since some types such as BStr and BString might end up in your public API, the author will commit to being very conservative with respect to new major version releases, and that unless there is a major issue with the 1.0 release, a 2.0 release is not expected sooner than some period of years.

That is a meaningful signal for a library that other people expose in their own APIs. A crate that promises not to break your public types for years is worth more than a faster moving one, and the trade is that you wait longer for new capabilities.

The licensing needs a small clarification. GitHub classifies the repository licence as a custom one, and the top level of the tree contains a COPYING file alongside LICENSE-APACHE and LICENSE-MIT. Cargo.toml is unambiguous though, with license = "MIT OR Apache-2.0". The repository also carries an AI_POLICY.md, which is a file the ecosystem around this author maintains. A bench/ directory is a workspace member and there is a scripts/ directory, so the repository is set up for benchmarking and release tooling, not just for the library itself.

There are no GitHub releases for the project. The version lives in Cargo.toml, where it currently reads 1.13.1, and the last push was on 2026-08-10 with the repository not archived.

Where the standard library is the better answer

The README includes a high level motivation section that is unusually honest about the crate's value proposition. Strictly speaking, it says, bstr provides very little that cannot already be achieved with the standard library Vec<u8> and &[u8] APIs and the ecosystem of library crates. It gives two examples: the standard library's Utf8Error can be used for incremental lossy decoding of &[u8], and the unicode-segmentation crate can be used for iterating over graphemes or words, although it is only implemented for &str types.

That framing is the honest one. bstr is not doing something the ecosystem cannot do at all. It is doing several things at once with a low dependency count, and it is explicit that the goal is bringing many related APIs together into a single crate while keeping the total number of dependencies low.

There are situations where it is the wrong tool. If your data is known to be valid UTF-8, then String and str give you the standard library's own methods, the ecosystem's serde and unicode-segmentation integrations, and none of the cognitive overhead of deciding which feature set you need. The comparison examples in the examples/ directory exist precisely to make that difference legible, and the ones with the -std suffix are the ones to read first if you are weighing whether the extra crate is justified.

Where bstr earns its place is a tool that has to survive whatever arrives on stdin. A line-oriented filter that must not choke on a stray byte, a parser that needs to find delimiters in data that may be partially valid, a case-mapping routine that has to pass invalid sequences through unchanged: these are cases where String is not an option and lossy decoding is not acceptable, because lossy decoding silently corrupts the data you are about to transform.

Editorial conclusion

bstr is the right crate when you are processing bytes that are only incidentally text, such as lines from a pipe, identifiers in a binary format, or input that may contain invalid sequences, and it is the wrong crate for ordinary text where a String and the standard library already work. Before adopting it, check whether the extra dependency weight is acceptable, since the unicode feature compiles Unicode tables into your binary and the crate is not free of that cost. Verify first that your MSRV is at least 1.65, because the crate states that floor and bumps it in minor releases.

Frequently asked questions

Official sources

1. BurntSushi/bstr on GitHub
2. Issues
3. README