Buf: a Protobuf toolchain that replaces the protoc shell script

The problem Buf targets: Protobuf work encoded in shell scripts

If you drive Protobuf with protoc, most of the operational knowledge lives outside the schema. Include paths are assembled per command. Plugin binaries have to be installed and kept at matching versions on every developer machine and CI runner. Style rules exist only as review comments. Compatibility is discovered after generated code fails or a serialized message will not parse. The README frames Buf as the replacement for exactly this: "If you are still driving Protobuf with shell scripts around protoc -I ..., Buf is the upgrade you want."

The audience is therefore not someone writing their first .proto file. It is a team that already has a schema directory, already generates code, and has started to feel the maintenance cost. Buf's pitch is that the same schema language and the same plugin model stay in place, while the surrounding workflow moves into versioned configuration files that a new contributor can read.

Modules, workspaces and the buf.yaml contract

Buf's central design choice is to treat a directory tree of .proto files as a module, and a project as a workspace. That single abstraction is what makes the rest of the commands agree on the same input: build, lint, breaking-change detection, generation, dependency resolution and publishing all read the same declaration instead of being handed a different set of flags each time.

The README calls the resulting buf.yaml "small" and shows this shape:

version: v2
modules:
  - path: proto
lint:
  use:
    - STANDARD
breaking:
  use:
    - FILE

The modules entry tells Buf where the schema lives. The lint and breaking blocks select rule sets. Because these are declared rather than passed on the command line, the same policy applies on a laptop, in CI and in release automation.

The README's comparison table is blunt about the trade-off being made. With protoc and scripts you "maintain -I paths and hope import order does not change behavior"; with Buf you declare modules once and Buf "discovers files and rejects ambiguous imports." Rejecting ambiguous imports is a constraint, not a convenience, and it is the kind of thing that will surface immediately when you point Buf at a legacy tree with duplicated or shadowed paths.

Installing Buf and running the first checks

The README gives Homebrew as the install path:

brew install bufbuild/buf/buf

After that, the documented sequence initialises a workspace and runs the checks the README says "every Protobuf repository" should pass:

buf config init
buf build
buf format -w
buf lint
buf breaking --against '.git#branch=main'

buf config init writes the starting configuration. buf build compiles the workspace. buf format -w rewrites files in place, so run it on a clean working tree the first time. buf lint applies the configured rules. buf breaking compares the current schema against the given reference, here a Git branch, and flags incompatibilities. The README notes that --against also accepts a BSR module, a tarball, a zip file, a local directory or a prebuilt Buf image, which is why the same command works locally and in CI.

Code generation moves out of the command line and into a checked-in template. The README's example uses remote plugins hosted on the BSR and managed mode:

version: v2
clean: true
managed:
  enabled: true
  override:
    - file_option: go_package_prefix
      value: github.com/acme/weather/gen/go
plugins:
  - remote: buf.build/protocolbuffers/go
    out: gen/go
    opt: paths=source_relative
  - remote: buf.build/connectrpc/gosimple
    out: gen/go
    opt:
      - paths=source_relative
      - simple
inputs:
  - directory: proto

With that file in place, generation is one command:

buf generate

The README states that remote plugins remove the need to install and maintain generator binaries on every developer machine or CI runner, and that local plugins work too if they speak the standard Protobuf plugin protocol. The README also points to a CLI quickstart for a guided walkthrough from an empty workspace to a working Connect service.

buf breaking and the four compatibility categories

The most useful idea in the documentation is that Protobuf compatibility is not one thing. Renaming a field can break generated source code while leaving the binary wire format intact. Changing a field from int32 to string breaks every serialized message already in the wild. A tool that reports both as "breaking" trains people to ignore it.

Buf separates them into rule categories named FILE, PACKAGE, WIRE_JSON and WIRE. You pick the level your consumers actually depend on, and the README's starting configuration uses FILE. That choice is a real decision, not a default to accept blindly: FILE is the strictest of the four, so it will report changes that would not affect anyone decoding bytes, while WIRE would let source-level breakage through. Teams that publish schemas to external consumers and teams that only use Protobuf for internal RPC traffic should not land on the same setting.

The --against flag is where this becomes practical. Pointing it at '.git#branch=main' means the check runs against whatever is on main, so a pull request is compared to the branch it will merge into without any export step.

What the Buf Schema Registry adds, and what it costs you

The BSR is a Protobuf-aware registry. According to the README it stores modules, verifies they compile, renders documentation, resolves dependencies, hosts remote plugins, produces generated SDKs, and can enforce schema checks before a breaking change reaches consumers. buf push publishes named modules to it.

The README is explicit that this is a separate tier: "Core CLI features work without a BSR account. Signing in to the registry adds distribution, remote plugins, generated SDKs, hosted docs, dependency resolution for private modules, and server-side checks when you need them." That sentence is the honest boundary of the project. Everything in this article up to this point works locally. The remote plugin example above does not, because remote plugins are a registry feature.

So the decision splits in two. The CLI is a local tool you can adopt without a vendor relationship. The registry is a hosted service that introduces a dependency on an external system for distribution and policy enforcement. Teams that only need linting, breaking-change detection and local generation can stop at the CLI. Teams that want consumers to install generated SDKs with go get, npm install, Maven, Gradle, pip install, NuGet, Cargo, SwiftPM, CMake or an archive are choosing the registry, and with it a place their schema has to be pushed to.

Where Buf is the wrong tool

Buf assumes your schemas live in a repository and that a workspace definition is a reasonable thing to maintain. If your .proto files are generated at build time from another source of truth, or arrive as a tarball from a vendor with no repository around them, the module and workspace model has nothing to attach to. The README does note that --against accepts tarballs, zip files and Buf images, but that is a comparison input, not a substitute for owning a schema tree.

A second boundary is the rule sets. Lint rules and breaking-change categories encode opinions about API shape. Pointing buf lint at an established schema for the first time will produce findings, and the response has to be either fixing them or narrowing the rule set in buf.yaml. Buf does not decide that for you, and the README does not describe a migration path for a large legacy tree. That work is yours.

Finally, the plugin model is deliberately the standard Protobuf one. If your generation pipeline depends on a plugin that does not speak that protocol, Buf's code generation does not cover it, and you keep that step outside the tool.

Buf compared with staying on protoc

The real alternative is not another tool. It is the protoc script you already have. The difference is where the configuration lives and what it can express.

A protoc invocation is a statement about one compilation: these include paths, these files, these plugins, these outputs. It has no notion of a module, so it cannot tell you that two imports resolve to the same file, and it cannot compare today's schema to last release's without you exporting a descriptor set first. Buf's buf.yaml is a declaration about a project, which is what lets buf lint, buf breaking and buf generate share an input and share a rule set.

The second difference is the compiler itself. The README says Buf uses an internal compiler, "tested against protoc descriptor output and built for deterministic parallel compilation." That testing claim is the one to hold the project to, because the value of replacing protoc depends entirely on the descriptors matching. If they diverge on your schemas, the replacement is not a drop-in.

The third difference is plugin installation. With protoc you install and version plugin binaries yourself. With Buf and remote plugins, the README says that maintenance disappears, at the cost of depending on the registry.

Maintenance, versioning and licence

Buf is a Go module, github.com/bufbuild/buf, and the repository is not archived. The last push was on 2026-09-21, and the most recent release listed is v1.73.0 on 2026-09-11, following v1.72.0 on 2026-07-17 and v1.71.0 on 2026-06-16. That is a steady minor-release cadence rather than a frozen tree.

The practical upgrade cost sits in two places. buf.yaml carries a version field, shown as v2 in the README's examples, which means configuration has a schema of its own and can change between major lines. buf.gen.yaml carries the same version field. buf.lock pins module dependencies, so dependency resolution is reproducible but also something you update deliberately.

The licence is Apache-2.0, as stated in the repository and shown by the licence badge in the README. Apache-2.0 is a permissive licence with an explicit patent grant. That covers the CLI source. It says nothing about the Buf Schema Registry, which is a hosted service governed by its own terms rather than by the repository licence. Anyone evaluating the registry tier should read those terms separately; this article does not interpret them.

Editorial conclusion

Adopt Buf if your Protobuf work already lives in a repository and you are tired of maintaining -I paths, plugin binaries and long protoc commands; the CLI's core features work without a BSR account, so the cost of trying it is one Homebrew install and a buf.yaml. Do not adopt it expecting the registry's distribution, remote plugins and server-side policy enforcement for free, since those require signing in. Before committing, verify that your existing .proto tree passes buf lint under the STANDARD rule set, and run buf breaking against your current release branch to see whether it reports incompatibilities you did not know you had.

Frequently asked questions

Official sources

1. bufbuild/buf on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases