golang/text: the Unicode and CLDR data behind Go's text handling

A package collection, not a single library

golang.org/x/text holds supplementary Go packages for text processing, many of them involving Unicode, and the repository description calls it a mirror. The module path is golang.org/x/text, the canonical git repository is at go.googlesource.com/text, and the licence is BSD-3-Clause, with a PATENTS file alongside the LICENSE file.

What is inside is best understood from the directory list, because the module is a set of independent packages rather than one library with a single entry point. There is a unicode directory holding the Unicode tables themselves, plus collate for collation, currency for currency formatting and symbols, number for number formatting and parsing, message for message catalogues, language for language matching, width for converting between full, half and narrow characters, runes for character classes and categories, feature for language feature detection, search for string search, secure for secure transforms, transform for the streaming transformation pipeline, encoding for character encodings, and internal for shared machinery. There are also a cases directory, a cmd directory for the generators and command line tools, a gen.go at the root, and a doc.go.

The relationship to the standard library is the point. The Go standard library handles UTF-8 text, but the heavier internationalisation work lives here, and the packages are separate so that a program only pulls in what it imports.

The reference implementation the data is checked against is ICU, and the module uses it as a conformance oracle rather than as a dependency, which is the interesting architectural choice and shows up in the test setup.

The Unicode version has to match your Go compiler

The README opens with a warning that deserves to be read before anything else. It is important that the Unicode version used in x/text matches the one used by your Go compiler, and the repository supports multiple versions of Unicode and will match the version of Unicode to that of the Go compiler. This is supported for Go compilers from version 1.7.

The mechanism behind that is the table generation. Tables are generated by default for the Unicode version in core, meaning the version the Go toolchain itself carries, and for the CLDR version defined in golang.org/x/text/unicode/cldr. The result is a module whose behaviour is tied to the compiler you build it with, which is a reasonable design for a table driven library and an unusual one to depend on.

The practical consequence is that x/text is not a static dependency. Two binaries built with different Go toolchains can carry different Unicode data from the same module version, and an upgrade of the Go toolchain can change the tables you get. If your product depends on a specific Unicode behaviour, that is worth a test rather than an assumption.

The module file is small, which is itself notable. The go.mod declares the module as golang.org/x/text with a go directive of 1.26.0, and its only requirements are golang.org/x/tools at v0.51.0, marked as tagx:ignore, and indirect dependencies on x/mod and x/sync. So the runtime dependency footprint of the text packages themselves is essentially nil; the tool dependency exists for the generators.

go generate rebuilds the tables and leaves a DATA cache

The tables in the repository are generated artifacts, and the README explains how to rebuild them. To generate the tables, except for the encoding tables, you run go generate from the directory, and by default the tables are generated for the Unicode version in core and the CLDR version defined in golang.org/x/text/unicode/cldr.

Running go generate has a side effect that surprises people the first time they see it. It creates a DATA subdirectory in the directory, which holds all the files used as a source for generating the tables, and that directory also serves as a cache. So the working tree gains a directory of downloaded data on the first run, and that directory is not tracked in git, since the files in DATA/iana, DATA/icu, DATA/w3, and DATA/whatwg are currently not versioned.

The distinction between the data sources and the generated tables is the interesting part of the design. There are separate inputs for the IANA registry data, the ICU data, the W3C data, and the WHATWG data, and each generated table has a source in one of them. The encoding tables are excluded from this process and are maintained separately.

In practice this means two distinct workflows. A user of the library never runs the generator. A contributor changing a generator, or a maintainer moving to a new data version, runs it, and the result is a diff across a large number of generated files.

The README also points out that the code gets adapted over time to changes in the data, which is the warning that the version numbers alone do not tell you everything about what changed.

Moving the Unicode or CLDR version, and the compatibility warning

Two environment variables control which data the generator consumes, and both versions are pinned to directories that have to exist upstream.

To update the Unicode version you set the version in the variable and run the generator.

    UNICODE_VERSION=x.x.x go generate

The version has to correspond to a directory in the Unicode public data repository. There is a side effect worth knowing: if this version is newer than the version in core, it also updates the relevant packages in the standard library, and the idna package in x/net is always updated.

The CLDR version follows the same pattern.

    CLDR_VERSION=version go generate

Here the version has to correspond to a directory in the CLDR directory of the Unicode public data repository.

Then there is the sentence that decides how you should treat any of this. The README notes that the code gets adapted over time to changes in the data, that backwards compatibility is not maintained, and therefore that updating to a different version may not work.

That is an unusual and honest warning to find in a Go package, and it is the single most important thing for a user to know. A version bump of the data is not a patch release with a changelog, it is a rebase of generated tables onto new upstream data with hand adapted code, and the result can change behaviour in ways that are not described in release notes. Practically, that means you should treat a data version change the way you would treat a change in a generated parser: regenerate, run the full test suite, and expect to look at the diff.

It also means the three tagged releases in the repository, v0.1.0 as the first tagged release in September 2017, v0.2.0 adding support for multiple Unicode versions and including Unicode 10 in December 2017, and v0.3.0 upgrading to CLDR 32 in February 2018, tell you about the shape of the project rather than about the current state of the data.

The test suite has an ICU conformance layer behind a build tag

Running the tests is a single command from the directory.

    go test ./...

There is a second layer, and it is the interesting one. Adding the -tags icu flag also runs the ICU conformance tests, if available, and doing so requires that you have the correct ICU version installed on your system. So the module can check itself against an independent implementation of the same Unicode and CLDR data, but only on a machine that has the right ICU version present.

That is the difference in approach between this module and the reference libraries it draws on. ICU is a C library with its own data and its own conformance suite, and x/text reimplements the same specifications in Go, with the conformance tests as the check. A pure Go module with no runtime dependencies gets a different result from a Go module linked against a large C library, and the tradeoff is the one the standard library usually makes.

The practical advice is to run the conformance layer in continuous integration on a machine that has ICU installed, not just the default test command, since the default command is the one that skips it. The rest of the repository is conventional Go: a codereview.cfg file for the Gerrit style review setup that the Go project uses, a gen.go at the root, and the internal and cmd directories holding the generators.

There is also a TODO in the README, one item, updating unversioned source files, which is a small window into how much of the data is still not pinned to a version.

Commits continue, tags stopped, and issues go to the Go tracker

The maintenance picture needs two dates read together. The repository is not archived and the last push was on 2026-10-09, so the module is being worked on. The most recent tagged releases are v0.3.0 from 2018-02-21, v0.2.0 from 2017-12-14, and v0.1.0 from 2017-09-14, so the tag list stops years before the last commit.

There is nothing alarming about that for an x/ module, where the import path, not a tag, is the stable identifier, but it does mean that if you pin a version in a go.mod you are pinning a snapshot of generated tables rather than tracking a release stream with notes attached. The consequence is the one already described: a data version change is not visible in a changelog, and the README says plainly that backwards compatibility is not maintained across such a change.

Contributions go through the Go project's own process. To submit changes you are pointed at the contribute documentation at go.dev/doc/contribute, and the git repository is hosted at go.googlesource.com/text rather than on GitHub, which is consistent with a mirror being the GitHub copy.

Issues are reported in the main Go issue tracker at go.dev/issues rather than in a repository specific tracker, and the README asks that the subject line be prefixed with x/text: so that reports are easy to find. That arrangement suits a module whose bug reports are often shared with the standard library, since the data and the compiler versions move together.

On the licence, the module is BSD-3-Clause with a PATENTS file, which is the permissive end of the scale and carries no copyleft obligation for a program that merely imports these packages.

Editorial conclusion

golang.org/x/text is a fit when you need Unicode aware behaviour that the Go standard library does not provide, such as collation, locale aware number or message formatting, casing, or width handling, and you are willing to depend on a module whose data tables are regenerated periodically. It is not a fit when you need a stable, semver pinned API across Unicode version bumps, because the project states plainly that backwards compatibility is not maintained when the data version changes. Verify first that the Unicode version in the module matches the one in the Go compiler you build with, that you regenerate the tables with go generate after any data change, and that the version of CLDR you select corresponds to a real directory under the Unicode public data repository.

Frequently asked questions

Official sources

1. golang/text on GitHub
2. License: BSD-3-Clause
3. Project website
4. README
5. Releases