# tinylib/msgp: A File-Based Go Generator for MessagePack

> tinylib/msgp generates MessagePack serialization code from Go type declarations. It suits Go services that want typed encoding methods, stream support, JSON interoperability, or low-allocation paths without maintaining a separate schema language.

**tinylib/msgp** — A Go code generator for MessagePack / msgpack.org[Go]

- Repository: https://github.com/tinylib/msgp
- Stars: 1,956 · Forks: 203
- Language: Go
- License: MIT
- Published: 2026-10-16 · Updated: 2026-10-16 · Language: en
- Canonical page: https://hysenlabs.com/projects/tinylib-msgp

## Why tinylib/msgp exists in a Go codebase

tinylib/msgp solves a specific source-code problem: giving Go types MessagePack serialization methods without introducing a separate schema language. The generator reads Go declarations, then emits methods that encode and decode those types. The intended users are Go developers working with MessagePack services, local storage, binary APIs, or transports where JSON is not the required wire format.

The README lists type safety, encoding flexibility, JSON interoperability, user-defined extensions, and performance as reasons to use the project. The design also keeps the schema close to implementation code. A struct definition, field tag, and generated method set can be reviewed together in the same language and module.

That choice creates an important operational boundary. Generation is file-based, not based on a global schema directory. If a struct refers to a type declared in another file, that file must also be processed. This can make generation predictable for a package with many source files, but it also means that adding a dependency can require an additional generator invocation or an expanded generation setup.

The project generates more than one useful shape of API. It targets interfaces associated with sizing, encoding, decoding, byte-oriented marshaling, and unmarshaling. It also generates reader and writer methods, so an application can choose between a compact byte slice path and a streaming path.

## How tinylib/msgp turns Go declarations into methods

The generator uses Go itself as its schema language. A source file contains a directive that invokes `go tool msgp`, and the command processes exported type declarations in that file. Field behavior is controlled with `msg` tags, which resemble the field naming tags used by the standard `encoding/json` package.

The generated methods implement several `msgp` interfaces. The README identifies `msgp.Sizer`, `msgp.Encodable`, `msgp.Decodable`, `msgp.Marshaler`, and `msgp.Unmarshaler`. The byte-oriented interfaces support marshaling and unmarshaling operations. The encoding and decoding interfaces are intended for streams, represented by `*msgp.Writer` and `*msgp.Reader`, which the project describes as protocol-aware counterparts to buffered writers and readers.

The dependency model is based on declarations visible to the generator. The example includes a named integer, a named byte slice, a map of pointers to the named integer, and a fixed-size array. Because `MyInt`, `Data`, and `Struct` appear in the same file, the parser can determine how to pass the type information into the generated `Struct` methods.

There is also extension support. MessagePack extensions are represented as a data type, such as `int8`, paired with raw binary data. The repository links to a separate wiki page for a worked example. This is an escape hatch for custom wire representation, but the README does not specify every compatibility constraint, so applications should inspect the documented extension format before depending on it.

## Installing the generator and creating a first type

The README gives the quickstart using Go tool directives. Add the `msgp` generator command to the module, then add a generation directive to the source file containing the types you want to process. This installs the project as a module tool rather than requiring a separate binary-management workflow.

Run these commands in the Go module that contains the source file:

```bash
go mod edit -tool=github.com/tinylib/msgp&&go mod tidy
```

The command updates the module configuration and resolves dependencies. The repository's `go.mod` declares Go 1.24 and dependencies including `github.com/philhofer/fwd` and `golang.org/x/tools`.

Add this directive to the Go source file:

```go
//go:generate go tool msgp
```

The directive generates serialization methods for exported declarations in that file. A minimal declaration can use tags that define wire names and omit fields:

```go
package main

type Person struct {
	Name    string `msg:"name"`
	Address string `msg:"address"`
	Age     int    `msg:"age"`
	Hidden  string `msg:"-"`
}
```

After generation, the package should contain the generated methods for `Person`. The README says that the generator also generates tests, so the first useful verification is to run the normal Go tests for the package after regeneration. The repository's Makefile uses `go generate` for its own generated integration and unit-test files, but that is for developing `msgp` itself. For a consuming module, the documented `go tool msgp` directive is the relevant path.

## Choosing byte APIs or stream APIs for MessagePack

The two generated API families serve different workloads. Byte-oriented methods work with `[]byte` and are presented as the faster option for small objects. Stream methods use `*msgp.Reader` and `*msgp.Writer`, which can avoid buffering an entire object and can become more memory efficient for larger payloads.

That distinction matters for service design. A short control message or a value stored in a cache may benefit from a compact byte slice. A larger object moving through a connection may be better handled through the stream APIs. The README says stream methods are generally more memory efficient and can eventually be faster for objects larger than 2KB, but it does not provide a benchmark result in the repository description, so the threshold should not be treated as a universal performance guarantee.

JSON interoperability provides a separate path. The project exposes `msgp.CopyToJSON()` and `msgp.UnmarshalAsJSON()`. This can help systems that receive MessagePack at one boundary and expose JSON at another. It does not mean every MessagePack value maps cleanly to JSON. The map-key restriction, discussed below, is one concrete boundary.

## Known restrictions in msgp type generation

The generator has several limitations that can produce surprising results. Identifiers referenced from outside the processed source file are assumed to satisfy the generator's interfaces. If that assumption is false, compilation fails rather than the generator silently adding the missing behavior. This makes cross-file changes a code-generation concern, not merely a Go import concern.

`chan` and `func` fields are ignored, as are non-exported fields. Encoding an `interface{}` is limited to built-in types or types with explicit encoding methods. These rules reduce the set of values that can be serialized automatically, and callers must encode unsupported dynamic values before passing them to the generated methods.

Map support is narrower than the MessagePack standard. Maps must have `string` keys, intentionally preserving JSON interoperability. The deserializer can read some `bin` map keys produced by legacy encodings, but it casts them to Go strings and later re-encodes them as `str`. The user remains responsible for UTF-8 safety. The README applies the same rules to JSON translation.

The file-based model also means that a type cannot simply be discovered from an arbitrary file elsewhere. Generated output can also change between versions for performance reasons. The README describes the library as mostly stable and notes that commercial applications use it, while also warning readers to verify their own build.

## Comparing tinylib/msgp with hand-written MessagePack code

A real alternative is hand-written MessagePack serialization with a MessagePack library or direct encoding logic. The difference is not that one format is supported by the other. The difference is where the type mapping and repetitive method code live.

tinylib/msgp reads Go declarations and generates typed methods, tests, and both byte-oriented and stream-oriented interfaces. That reduces manual implementation work for ordinary structs and named types, but it also introduces generated files, generation directives, and a dependency on the generator's assumptions. Cross-file dependencies require explicit processing, and unsupported field types are omitted or constrained.

Hand-written code gives the application direct control over each wire decision. It can accommodate unusual map keys, dynamic values, custom field behavior, or protocol-specific framing without asking the generator to model them. The cost is more code to maintain and more opportunities for inconsistent size, encoding, and decoding behavior across a large set of types.

The choice therefore depends on project shape. A Go codebase with many regular structs and repeated binary serialization is a plausible fit for code generation. A small service with a few unusual wire contracts may gain little from adopting the generator and may prefer explicit encoding code.

## Maintenance, releases, and adoption boundaries

The repository is not archived. Its last push was on 2026-09-30, and releases v1.6.3, v1.6.4, and v1.6.5 were published on 2026-01-02, 2026-04-14, and 2026-09-30. That recent activity is evidence of current repository movement, but the README still warns that newer generator versions may produce different code for performance reasons.

The project uses the MIT license. For a Go application, the practical upgrade cost includes reviewing regenerated diffs, rebuilding packages, and rerunning tests whenever the generator changes. This is more involved than changing a runtime import alone because generated methods can change with the tool version.

The repository includes `gen`, `parse`, `printer`, and `msgp` directories, plus generated fixtures and regression test files. The Makefile also includes CI preparation commands such as `go install .`, `go generate ./msgp`, and `go generate ./_generated`. Consumers who rely on generated output should inspect those diffs and compile the entire affected package set, especially when types are spread across files.

The clearest adoption boundary is regeneration. Before adopting `msgp`, confirm that the Go version requirement fits the project, add the tool directive in a reproducible module configuration, process every source file containing referenced declarations, and review the generated code for ignored fields and interface assumptions.

## Conclusion

Adopt tinylib/msgp when Go source files are already the schema source and generated MessagePack methods are useful, especially for typed services with small-object or stream workloads. Do not choose it when non-string map keys, interface values without explicit encoding methods, or cross-file generation assumptions are unacceptable. Verify first that every referenced type is processed and that regenerated code passes the project test suite.

## FAQ

### How do I install the tinylib/msgp generator?

The README uses the command `go mod edit -tool=github.com/tinylib/msgp&&go mod tidy`, then adds `//go:generate go tool msgp` to a Go source file. The `go.mod` file requires Go 1.24.

### Does tinylib/msgp support maps with non-string keys?

Generated maps must use `string` keys. The README says the restriction preserves JSON interoperability. During decoding, some legacy `bin` keys may be accepted and converted to Go strings, but UTF-8 safety remains the user's responsibility.

### What happens when a msgp struct uses types declared in another file?

The referenced file must also be processed because tinylib/msgp operates on individual files. External identifiers are assumed to satisfy the generator's interfaces, so an incorrect assumption can lead to compilation failure.

## Sources

- [Issues](https://github.com/tinylib/msgp/issues)
- [License: MIT](https://github.com/tinylib/msgp/blob/master/LICENSE)
- [README](https://github.com/tinylib/msgp/blob/master/README.md)
- [Releases](https://github.com/tinylib/msgp/releases)
- [tinylib/msgp on GitHub](https://github.com/tinylib/msgp)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/tinylib-msgp