blackboxprotobuf: reading protobuf traffic without the schema

The problem: protobuf is efficient precisely because it is unreadable

Protobuf is a binary serialization format from Google which can be used as a more efficient alternative to formats like JSON or XML. Developers can define the message format in a .proto file and use the protobuf compiler to generate message handlers in their language of choice.

Then the description explains why this is hard work for anyone who does not have the schema. The protobuf encoding is binary, and unlike JSON or XML it is not human readable or easy to modify by hand. The format also takes advantage of both sides having the message definition and strips out much of the type information.

That last clause is the technical heart of the problem. The wire format does not carry the field types, so a message on the wire is a set of field numbers and values with no indication of whether field 3 is a string, a nested message, a repeated field or an enum. Both the sender and the receiver need the .proto file to make sense of it.

So when the .proto file does not exist, or is not the one actually deployed, you have a stream of bytes and a set of plausible interpretations. The format is good for efficiency, and that good for efficiency increases the difficulty of analyzing or modifying network traffic.

Blackbox protobuf is designed to allow working with protocol buffers without the message definition. The name is the thesis: you work on the message as a black box, and you infer the types.

The licence is MIT, the repository is under nccgroup, and the release list has a single entry, v1.4.2 from 2025-03-14, with the last push to the repository on 2026-04-13.

Four interfaces, chosen for where you happen to be working

The repository contains several interfaces for working with protocol buffers, and the choice of four is a design decision about where the analyst is standing when they need to read the bytes.

The first is a jython burp extension in the burp/ directory. The second is a python library that can be used in other applications, in lib/. The third is a python-based CLI embedded in that library, with its own documentation at lib/CLI.md. The fourth is a mitmproxy addon in the mitmproxy directory.

Two of those are proxies, and they are there because the two most common places to see unknown protobuf traffic are inside a proxy you already run. If your work already involves Burp, the extension drops into that session. If your work involves mitmproxy, the addon does the same job there. Neither requires you to re-architect your workflow, which matters because the hardest part of this kind of work is rarely the tool, it is getting the bytes out of the application in the first place.

The library is the part with the widest reach, because anything you can write in Python can import it, and that is what makes the forensics use case work. If you have a memory dump or a captured pcap and a set of hypotheses, you want to write a script that tests them, not to click through a GUI. The CLI is the same library exposed for the cases where a script is overkill.

The repository layout is consistent with that split, with a top-level burp/, lib/, mitmproxy/ and docs/ directory, plus a Vagrantfile and a .gitmodules file. The Vagrantfile suggests the Burp environment, which needs a specific Java and browser setup, is provided as a provisioned virtual machine rather than left to the user to assemble.

The .gitmodules entry indicates at least one component is a submodule rather than vendored code, which is worth noting if you plan to build from source.

The package rename, and which one to install

The first thing the README tells you is a package name change, and it is worth reading carefully because the old name still resolves on PyPI.

Blackbox Protobuf now has an official package on PyPi under the name bbpb. The blackboxprotobuf package is an older fork.

That sentence contains two separate facts that are easy to conflate. The project has an official distribution called bbpb, and there is a separately published package called blackboxprotobuf that is a fork rather than the original. So installing blackboxprotobuf does not give you the current upstream release, and pip resolving that name will quietly hand you someone else's fork of an older codebase.

This is a common enough hazard in the security tooling world that it is worth stating as a rule rather than a footnote. When a project moves its distribution name, the old name becomes available and gets claimed, and there is no way for a user to tell from the package name alone which one they installed.

The practical advice is short. Install bbpb, and if you have an environment where blackboxprotobuf is already installed, work out which one is actually in use before you trust the output, because the two will not behave identically and the failure will look like a decoding bug rather than a version mismatch.

The same pattern shows up in the release history too, where the single tag v1.4.2 is named the same way as the package rather than following a distinct versioning scheme, so the tag and the distribution name are easy to confuse when you are pinning a version.

Typedefs, and why the type guessing is the real work

The documentation section lists one extra document beyond the per-tool READMEs: a Type Definition Guide at docs/TypeDefs.md, described as a guide for editing typedefs to fix types and improve readability.

That file is the centre of the project, and understanding why tells you how to use the tools well.

Because the wire format strips out the type information, a black box decoder has to infer it. A field number alone does not say whether the value is a varint, a length-delimited string, a nested message, a fixed 32 or 64 bit value, or a repeated field. The decoder can often tell from the encoding itself, since a nested message and a length-delimited string are byte-for-byte ambiguous, and it can tell a great deal from context, since a field that is always present with the same shape is probably not a nested message that occasionally appears.

So the output of a first pass is a set of guesses, and a type definition is the document in which those guesses are recorded and corrected. Fixing a type is what turns a field the decoder was reading as an opaque blob into a nested structure you can step through, and improving readability is what turns the decoded output into something you can actually reason about while hunting for a flag or a length field.

That is the workflow the project implies. Decode, look at what came out, decide whether the inferred types make sense, correct the typedef, decode again. It is the same loop as decompiling, and the guide exists to make the second iteration cheap.

The two other tools named in the future section give a sense of where the project would like to go. A protobuf type discovery tool is listed as something that may be built on top of blackboxprotobuf in the future, which is the automation of the guessing step, and a separate type discovery tool in the same list suggests the project is aware that the manual loop is the bottleneck.

The subject matter is also worth naming plainly. The tool was originally implemented as a Burp extension for decoding and modifying messages during mobile pentests, and the modification capability is the reason it exists at all. Reading an unknown message tells you a protocol exists. Changing a field in a message that the server then trusts is a different capability, and that is the capability the shared drive gear of the type definitions is pointed at.

Maintenance, packaging and who this is for

The maintenance picture is modest and honest. The last push to the repository was on 2026-04-13, the repository is not archived, and the release list contains exactly one release, v1.4.2 from 2025-03-14. So the project is receiving work while its version history stays short.

For a security research tool that is a normal shape, and there is a reason for it. The tool operates on a wire format, and the formats it has to handle change. An application that switches from a legacy protobuf encoding to a new one breaks the assumptions in the decoder, and the fix is a code change rather than a release, because the library is a research instrument rather than a piece of infrastructure with a compatibility promise.

The subjects it targets are also worth naming. The description says it was originally implemented as a Burp extension for decoding and modifying messages during mobile pentests, and that it has also been used for reverse engineering and forensics tooling. The mobile pentest origin explains the Burp extension being first in the list of interfaces, and the forensics use explains the library and the CLI being there at all, because forensics work is scripting rather than interactive.

So the tool has three audiences. A penetration tester working inside a Burp session gets the extension. A reverse engineer working on a binary or a capture gets the library and the CLI. And a forensics team writing a repeatable analysis gets the library, because a script that runs against a thousand captured files is worth more than a thousand manual decodings.

The MIT licence makes all three easy to adopt and to modify. For a tool whose output feeds into findings in a report, the permissiveness is what lets an organisation put it in its own tooling without a legal review.

Editorial conclusion

Adopt blackboxprotobuf if you need to read or modify protobuf traffic and the .proto file is not available to you, because the whole point of the project is that the schema is missing and the type information is stripped out. Do not adopt it if you have the schema, because a real protobuf compiler is a better answer when you can generate one. Before you start, note that the maintained package is on PyPI as bbpb and the blackboxprotobuf package is an older fork, and read the Type Definition Guide, because the typedefs are where the type guesses live and getting them right is most of the work.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. nccgroup/blackboxprotobuf on GitHub
4. README
5. Releases