hyperpb trades a compile step for a Protobuf parser that runs reflection code fast

The parser is a VM, and the instruction set comes from UPB

The mechanism is unusual enough to be the whole story. The README states that the hyperpb parser is an efficient VM for a special instruction set, a variant of table-driven parsing, pioneered by the UPB project. That is not a description of a faster reflective decoder, it is a description of a second compiler. The message descriptor is compiled into a program for that VM, and the program is what runs when bytes arrive. The consequences show up everywhere in the API: you cannot parse without first compiling, and the compilation is per message type rather than per message. The README frames it as the core conceit, comparing it to how regular expressions need regexp.Compile before use. The stated benefit is that the compile step lets hyperpb run optimization passes over the message and that deferring it to runtime lets the layout optimizations improve over time without source-breaking changes.

Two ways to compile a type, and both want caching

There are two entry points for producing a compiled type, and the difference is where the schema comes from. hyperpb.CompileMessageDescriptor takes a protoreflect descriptor that you already have, which in the README's example comes from a compiled-in Go type via ProtoReflect().Descriptor(). hyperpb.CompileFileDescriptorSet takes a schema as a descriptorpb.FileDescriptorSet plus a message name, and that is the path for types you only learn about at runtime. Both return an error on failure, and the README repeats the same reminder in both places: cache this. Compiling on every request would replace the cost you were trying to remove, so the compiled type belongs in a map or a sync.Once in your service. The first compile also pays for the optimization passes, which is why the README describes the compile step as something that delays to runtime in order to let hyperpb improve layout optimizations continuously.

func main() {
    // Compile a type for your message. Make sure to cache this!
    // Here, we're using a compiled-in descriptor.
    msgType := hyperpb.CompileMessageDescriptor(
        (*weatherv1.WeatherReport)(nil).ProtoReflect().Descriptor(),
    )

    // Allocate a fresh message using that type.
    msg := hyperpb.NewMessage(msgType)

    // Parse the message, using proto.Unmarshal like any other message type.
    if err := proto.Unmarshal(weatherDataBytes, msg); err != nil {
        // Handle parse failure.
        log.Fatalf("failed to parse weather data: %v", err)
    }

The compile-and-parse loop is short enough to read in full:@@C1@@

Reflection in, reflection out, and the panic boundary

The read path is entirely protoreflect, and the README is explicit that hyperpb currently only supports manipulating messages through the reflection API. Access is by field name, through msg.Get(fields.ByName("region")), or by index through fields.Get(), which the README says is less legible but does not hit a hashmap. Repeated fields come back as a list you index with List(), and nested messages are reached through Get().Message(). The limitation that matters most is on the other side: mutation is currently not supported, and any operation that would mutate an already-parsed message panics, with the list of panicking methods of *hyperpb.Message included in the documentation. So the library is a reader, not a writer. A service that only decodes incoming payloads is a good fit; a service that decodes and then edits fields is not, and the failure mode is a panic rather than an error, which is the kind of thing you want to know before it happens in production rather than after.

Where it fits: generic services that fetch types off the network

The README names the audience plainly: it shines best when you need a very generic service that downloads types off the network and parses messages using those types, which forces you to use reflection. That is a specific architecture, and it explains the design. A plugin host, a multi-tenant gateway that serves schemas per tenant, or a proxy that has to understand a message it was not compiled against all need a dynamic parser, and the usual answer is dynamicpb, which is pure reflection and therefore slow. The second use the README describes is transcode rather than interpret: since any generic, non-mutating operation works with hyperpb messages, they can serve as an efficient transcoding medium from the wire format for runtime-loaded messages, and the example decodes into a hyperpb message and then hands it to protojson.Marshal, which works without any adaptation:

func dynamicMessageToJSON(
    schema *descriptorpb.FileDescriptorSet,
    messageName protoreflect.FullName,
    data []byte,
) ([]byte, error) {
    msgType, err := hyperpb.CompileFileDescriptorSet(schema, messageName)
    if err != nil {
        return nil, err
    }

    msg := hyperpb.NewMessage(msgType)
    if err := proto.Unmarshal(data, msg); err != nil {
        return nil, err
    }

    // Dump the message to JSON. This just works!
    return protojson.Marshal(msg)
}

@@C2@@

Generated code is the alternative, and it is still close

There are two honest alternatives, and the README compares against both. The first is dynamicpb, protobuf-go's canonical solution for completely dynamic messages, which hyperpb is designed to be a drop-in replacement for. The difference is architectural rather than incremental: dynamicpb interprets the descriptor on every message, while hyperpb compiles it once into a VM program, and the README claims a 10x improvement over dynamicpb. The second alternative is generated code, and here the claim is narrower and more interesting: hyperpb often beats protobuf-go's generated code by a factor of two to three, especially for workloads with many nested messages. The caveat is the word often, and the direction of the comparison. If you already have generated code, you are comparing a runtime compile against a build-time compile, and the generated path also gives you a concrete Go type rather than a reflective one, which is the part that matters for code you have to write against.

The benchmark story, and what make bench actually runs

The README presents two benchmark variants for hyperpb, and the distinction is a technical one rather than a marketing one. The first is out-of-the-box performance with no optimizations turned on, and the second is real-time profile-guided optimization with all the optimizations hyperpb currently offers enabled. PGO means the profile of a representative workload feeds back into the build, so the second number is a best case that requires a representative profile, not a default. The README says these benchmarks are a subset of the ones you can run using make bench, and the Makefile shows what that involves: a pinned Go toolchain at go1.26.0, a pinned Buf CLI at v1.56.0 that the file says to keep in sync with the CI workflow, and a pinned linter at v2.11.4 with the same instruction. Benchmark flags are separated from test flags, with BENCHFLAGS defaulting to -test.benchmem.

The dependency list tells you what the implementation is made of

The go.mod is unusually informative for a small library, and it shows the approach rather than just the version numbers. The module is buf.build/go/hyperpb and it requires Go 1.25.0, and the require block says so directly:

module buf.build/go/hyperpb

go 1.25.0

require (
	al.essio.dev/pkg/shellescape v1.6.0
	buf.build/gen/go/bufbuild/hyperpb-examples/protocolbuffers/go v1.36.7-20250725192734-0dd56aa9cbbc.1
	buf.build/gen/go/bufbuild/protovalidate/protocolbuffers/go v1.36.7-20250717185734-6c6e0d3c608e.1
	buf.build/go/protovalidate v0.14.0
	github.com/google/uuid v1.6.0
	github.com/melbahja/goph v1.4.0

@@C3@@ Alongside the expected google.golang.org/protobuf at v1.36.9 and buf.build/go/protovalidate at v0.14.0, two entries stand out. github.com/planetscale/vtprotobuf at v0.6.0 is the source of the generated marshalling and unmarshalling code the VM runs, and github.com/protocolbuffers/protoscope is the wire-format debugging tool, which tells you the library is built around inspecting bytes at a low level. There is also github.com/melbahja/goph, a Goroutine profiler, and github.com/tiendc/go-deepcopy, which is what a reflection-based message library needs to make copies. The module also depends on buf.build/gen/go for the generated example and validate types, so the examples in the README are compiled against real generated code.

Source layout, validation integration and release state

The repository root is small and readable, which suits a library that is still finding its shape. There are compile.go, message.go, message_type.go, options.go, shared.go and doc.go at the top level, with the implementation under internal/, plus a DESIGN.md that the README does not summarise, so it is the place to look for the reasoning behind the VM. The test surface includes example_test.go, parse_test.go and parse_fuzz_test.go, and the fuzz test is worth noting because a parser that consumes untrusted bytes is exactly the kind of code that needs one. Validation is integrated rather than bolted on: the README states that protovalidate works directly on reflection, so it works out of the box with hyperpb messages. The project is licensed Apache-2.0, the default branch is main, and the recent releases are v0.1.3 from 2025-09-24, v0.1.2 from 2025-09-23 and v0.1.1 from 2025-09-09, while the last push to the repository was on 2026-07-28, so the code moves and the version tags do not.

Editorial conclusion

hyperpb suits a Go service that parses Protobuf it does not know at compile time, such as a gateway that fetches descriptors over the network, and that is willing to pay a one-off compile cost per message type and cache it. It does not suit a codebase that already has generated code for every message it handles, because the README's own comparison puts generated code within a factor of two to three, and it does not suit code that mutates messages after parsing, since those operations panic. Before adopting, read DESIGN.md and the message.go documentation for the list of panicking methods, confirm your workload is read-only, and note that the module path is buf.build/go/hyperpb under Apache-2.0 while the newest tagged release is v0.1.3 from 2025-09-24.

Frequently asked questions

Official sources

1. bufbuild/hyperpb-go on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases