# utf8proc: Unicode normalization and case folding in C

> utf8proc is a small C library that does Unicode normalization, case folding and codepoint handling over UTF-8 data, and it is what the Julia language uses for its own Unicode support. It builds with make or cmake, ships a CMake package, and carries two version numbers, one for the API and one for the shared library ABI.

**JuliaStrings/utf8proc** — a clean C library for processing UTF-8 Unicode data

- Repository: https://github.com/JuliaStrings/utf8proc
- Website: http://juliastrings.github.io/utf8proc/
- Stars: 1,306 · Forks: 187
- Language: C
- License: NOASSERTION
- Published: 2026-10-19 · Updated: 2026-10-19 · Language: en
- Canonical page: https://hysenlabs.com/projects/juliastrings-utf8proc

## A small library with a long history of ownership

utf8proc is a small, clean C library that provides Unicode normalization, case folding and other operations for data in UTF-8. The history explains why the project sits where it does. It was initially developed by Jan Behrens and the rest of the Public Software Group, who the readme credits with nearly all of the credit for the package. With the blessing of that group, the Julia developers took over development, since the original developers moved to other projects. The stated reason the Julia developers got involved was that they wanted to add Unicode 7 support and other features. Today utf8proc provides basic Unicode support in the Julia language, and it is regularly updated to keep up with recent Unicode releases, which is the only sensible maintenance policy for a library whose entire job is to track a moving standard. There are wrappers for Ruby, Rust and Swift, and the original package also included PostgreSQL bindings. There is even an independent Lua translation, lua-mojibake, which takes a different approach from the wrappers.

## Building with make, with cmake, and on the awkward platforms

Compilation is a single make for the C library. You can also install the library and header file with make install, which by default puts them in /usr/local/lib and /usr/local/bin, and the location can be changed with make prefix=/some/dir. make check runs the tests and make clean deletes the generated files. The included Makefile supports GNU/Linux flavours and macOS with gcc-like compilers, and the readme is clear that Windows users will typically use cmake instead, because the makefile assumes a Unix shared library convention. For other Unix-like systems and other compilers you may need to pass modified settings to make so the shared library is built with the right flags. The HP-UX example is the extreme case, with a gmake invocation that overrides the compiler, the optimisation flags, the position independent code flag, the C99 flag and the shared library flags:

```sh
gmake CC=/opt/aCC/bin/aCC CFLAGS="+O2" PICFLAG="+z" C99FLAG="-Ae" WCFLAGS="+w" LDFLAG_SHARED="-b" SOFLAG="-Wl,+h"
```

The cmake path is the portable one, and it is a normal out-of-source build:

```sh
mkdir build
cmake -S . -B build
cmake --build build
```

After a successful compilation the library is in the working directory as libutf8proc.a for the static build and libutf8proc.so for the dynamic one.

## Consuming it from CMake

A CMake Config-file package is provided, so a project that already uses cmake does not need pkg-config plumbing. The integration is three lines:

```cmake
add_executable (app app.c)
find_package (utf8proc 2.9.0 REQUIRED)
target_link_libraries (app PRIVATE utf8proc::utf8proc)
```

The version in the find_package call is a minimum, and the imported target name is utf8proc::utf8proc, following the usual namespaced target convention. The repository also carries a pkg-config template, libutf8proc.pc.cmakein, and a plain libutf8proc.pc.in, so a build that does not use cmake can still find the library through pkg-config. The install layout is conventional too: prefix, libdir, includedir and pkgconfigdir are set in the Makefile, and the shared library gets a soname with the major, minor and patch numbers, so a system package manager can track the ABI.

## The API is small, and the header is the documentation

The documentation for the C library is the utf8proc.h header file, and the readme names one function as the one you will use most, utf8proc_map, which maps UTF-8 strings unless you want to allocate memory yourself. The examples show the rest of the surface. A codepoint becomes a string with utf8proc_encode_char, and a string becomes a codepoint with utf8proc_iterate, which is the function that walks a string and reports the next codepoint:

```c
// Convert string `str` to pointer to codepoint `a`
utf8proc_uint8_t str[] = "ß";
utf8proc_int32_t a;
utf8proc_iterate(str, -1, &a);
printf("%d\n", a);
// 223
```

Case folding goes through the same map function with a flag:

```c
// Convert "ß"  (U+00DF) to its casefold variant "ss"
utf8proc_uint8_t str[] = "ß";
utf8proc_uint8_t *fold_str;
utf8proc_map(str, 0, &fold_str, UTF8PROC_NULLTERM | UTF8PROC_CASEFOLD);
printf("%s\n", fold_str);
// ss
utf8proc_free(fold_str);
```

The memory ownership is explicit: the result comes back as a pointer you release with utf8proc_free, and the UTF8PROC_NULLTERM flag tells the library the string is null terminated. Helpers exist for the common normalization forms, with utf8proc_NFD decomposing and utf8proc_NFC composing:

```c
utf8proc_uint8_t input[] = {0xc3, 0xa4, 0xc3, 0xb6, 0xc3, 0xbc}; // "\u00e4\u00f6\u00fc" = "äöü" in UTF-8
utf8proc_uint8_t *nfd= utf8proc_NFD(input); // = {0x61, 0xcc, 0x88, 0x6f, 0xcc, 0x88, 0x75, 0xcc, 0x88}
utf8proc_uint8_t *nfc= utf8proc_NFC(nfd);
utf8proc_free(nfd);
utf8proc_free(nfc);
```

That decomposing and recomposing pair is the practical way to test that your own handling of combining marks matches the standard.

## Unicode 18.0.0 and the four normalization forms

The Unicode version supported is 18.0.0, which is the number to check when you compare a utf8proc build with another implementation. The library documents which options each normalization form uses, and the list is short enough to memorise. Normalization Form C is STABLE and COMPOSE, Form D is STABLE and DECOMPOSE, Form KC adds COMPAT to C, and Form KD adds COMPAT to D. The compatibility forms are the ones that change characters rather than only rearranging them, and choosing them by accident is a common way to mangle text, because a compatibility decomposition can turn a ligature or a superscript into something that looks different to a human reader while comparing equal to a machine. If your use case is comparing identifiers or searching text, canonical forms are usually what you want. The point of having one library do all four is that the difference between them is a flag, while the difference between two implementations is a bug.

## Two version numbers, and what they mean for packaging

The build files carry two version numbers and the comments in them explain why, which is unusual and useful. The shared library version is MAJOR.MINOR.PATCH, and the Makefile states that this may be different from the utf8proc version number because it indicates ABI compatibility, not API compatibility: MAJOR should be incremented whenever binary compatibility is broken, even if the API is backward compatible, and the API version number is defined in utf8proc.h. In the current Makefile the ABI version is 3.3.0 while the API version is 2.12.0, and the same file warns you to update these ABI versions in MANIFEST and CMakeLists.txt as well. The consequence for a packager is concrete. A distribution that ships utf8proc 2.12.0 with a shared library labelled 3.3.0 is not a mistake, it is the design, and the soname reflects the ABI so that a package upgrade replaces the library rather than leaving two incompatible copies installed. A second number in the same family is the minimum version accepted by the cmake find_package call, 2.9.0, which is an API version and is what you should require if your code depends on newer entry points.

## The licence, the data files and the alternatives

The package is licensed under the free and open source MIT expat licence, plus certain Unicode data governed by the similarly permissive Unicode data licence, and the readme points at the included LICENSE.md for the details. That split matters in practice, because the data tables are not really code, and a licence that covers only the code would leave the tables in an unclear state. The tree shows how the library is assembled: utf8proc.c and utf8proc.h hold the implementation and the API, utf8proc_data.c holds the generated tables, and a data directory, lump.md and utils.cmake sit alongside the build files, with a bench directory for performance work and a test directory holding the test programs the make check target builds. There is also a build.zig and build.zig.zon, so the library can be built with the Zig build system as well as with make and cmake. On the alternatives, there are two kinds. The Ruby, Rust and Swift wrappers bind the same C library, so they inherit its correctness and its Unicode tables. The Lua port, lua-mojibake, is an independent translation rather than a binding, which means it can follow a different path and a different release cycle. A project in Lua has a genuine choice here, and everyone else is choosing a binding.

## Conclusion

utf8proc is the right choice for a C or C++ project that needs correct normalization or case folding and would rather not carry a large text library, especially since the same code is available through wrappers in Ruby, Rust and Swift. It is the wrong choice if you need support for scripts beyond the Unicode tables it ships, or if you need a locale sensitive comparison that the Unicode default caseless matching does not provide. Before you build, read the utf8proc.h header, which is the real documentation, decide whether you want the static or the shared library, and remember that the library version and the ABI version move independently, so pin the one your distribution ships.

## FAQ

### What is utf8proc?

It is a small C library that provides Unicode normalization, case folding and other operations for data in UTF-8. It was originally developed by the Public Software Group and is now maintained by the Julia developers, and it is used for Unicode support in the Julia language.

### How do I build and install utf8proc?

Run make to build the C library, make install to install the library and header, by default into /usr/local/lib and /usr/local/bin, and make check to run the tests. You can change the location with make prefix=/some/dir, and cmake -S . -B build is the portable alternative.

### How do I use utf8proc from a CMake project?

Use find_package (utf8proc 2.9.0 REQUIRED) and link the imported target utf8proc::utf8proc. A CMake Config-file package is provided, and a pkg-config template is included for builds that do not use cmake.

### Which Unicode version does utf8proc support?

Unicode 18.0.0. The library is regularly updated to keep up with recent Unicode releases, and the project ships the four normalization forms, NFC, NFD, NFKC and NFKD.

### How do I casefold a string in C with utf8proc?

Call utf8proc_map with the UTF8PROC_NULLTERM and UTF8PROC_CASEFOLD options to get a mapped string, then release it with utf8proc_free. The header file utf8proc.h is where the API is documented.

## Sources

- [Issues](https://github.com/JuliaStrings/utf8proc/issues)
- [JuliaStrings/utf8proc on GitHub](https://github.com/JuliaStrings/utf8proc)
- [Project website](http://juliastrings.github.io/utf8proc/)
- [README](https://github.com/JuliaStrings/utf8proc/blob/master/README.md)
- [Releases](https://github.com/JuliaStrings/utf8proc/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/juliastrings-utf8proc