certificate-transparency-go, and why it keeps its own ASN.1 parser

What the repository covers, and where the documentation lives

The repository holds Go code related to Certificate Transparency, and the documentation is split between the repository and a separate site, with the project home page at certificate.transparency.dev and a Slack workspace for support.

The scope is broad, and the repository documentation walks the layout deliberately. There are encoding libraries, CT client libraries, a Trillian personality, command line tools and other libraries related to CT. Underneath that are directories for the certificate and ASN.1 work, the protocol clients, the log scanner, the CT utilities, the log policy code, the submission and gossip directories, the log identifier code, the preload and schedule directories, the integration tests, and the third_party directory.

The top level also has the files you would expect in a Go library of this size, with a serialization.go, a signatures.go and a types.go, each with its own test file, and a proto_gen.go that drives code generation.

One small thing to notice first. The project documentation states that the repository requires Go version 1.24, while the go.mod file declares go 1.26.0 with a toolchain directive of go1.26. The module file is the authoritative answer for a build, and the prose in the project documentation is out of date. That is a trivial detail on its own, and it is a useful signal about the general shape of the project: this is a large, working, long-lived repository rather than a carefully curated one.

The module is github.com/google/certificate-transparency-go, and the licence is a custom one that GitHub cannot classify, so the LICENSE file is the text that matters.

Forks of encoding/asn1 and crypto/x509, and the observatory argument

The single most important design decision in the repository is the set of forks at its centre, and the reason is given in the project description in a single sentence.

The asn1 and x509 directories are forks of the upstream Go encoding/asn1 and crypto/x509 libraries. The project maintains separate forks because CT is intended to act as an observatory of certificates across the ecosystem, and as such it needs to be able to process somewhat malformed certificates that the stricter upstream code would, correctly, reject.

That word correctly is doing the work. The upstream parsers are strict for good reason, because a validator that accepts a certificate nobody else accepts is a security problem. A log that is an observatory has the opposite requirement, because the whole point of Certificate Transparency is to record what was issued, including the things that are wrong, so that somebody can find them later.

So the two forks relax exactly the checks that a validator needs and an observer does not. That is a defensible boundary, and it is also the boundary you have to know about when you use the library, because a permissive parser is a liability if you feed its output to a security decision.

The x509 fork also includes code for working with the pre-certificates defined in RFC 6962, which is the construct used to prove that a certificate was submitted to a log before it was issued, and the tls directory holds a library for processing TLS encoded data as described in RFC 5246. The x509util directory adds utilities for working with x509.Certificate values.

Four ways to reach a log: ct, client, jsonclient, dnsclient, scanner

The client libraries are split by transport, and the split is more interesting than a list of packages suggests.

The top-level ct package holds the types and utilities for working with the CT data structures defined in RFC 6962, which is the shared vocabulary everything else is built on: the structures themselves, their encoding, and the operations you can perform on them.

The client and jsonclient directories hold libraries that allow access to CT logs via the HTTP entry points described in section 4 of RFC 6962. The dnsclient directory does the same over DNS, following the draft on CT over DNS. That last one is the odd one out and the one worth understanding, because a DNS based channel exists for a reason. A log that you can only reach over HTTPS is a log you can only reach from networks that allow HTTPS to that host, and a DNS channel gives the log a second, independent path for publishing its data.

The scanner directory holds a library for scanning the entire contents of an existing CT log, which is a different operation again. A client fetches what it needs, a scanner walks the whole log.

The loglist3 directory adds a library for reading the v3 JSON lists of CT logs, which are the lists that describe which logs exist and which are acceptable for which purposes. Reading them is a separate job from talking to a log, and it is the job that a client has to do before it can decide where to submit anything.

So the surface is: structures, four transports, a whole log scanner, and the log lists. That is enough to build a monitoring tool, an audit tool, a submission client or a log operator's dashboard, which is why this repository is the starting point for most Go work in the CT ecosystem.

Five command line tools, and a request to be polite

The command line tools are the fastest way to see what the library does, and there are five of them with clear division of labour.

The client directory contains ctclient, which allows interaction with a CT log. The ctutil directory holds sctcheck, which allows signed certificate timestamps from a log to be verified. The scanner directory holds scanlog, which allows an existing log to be scanned for certificates of interest, and the instruction attached to it is worth repeating: please be polite when running this tool against a log. Scanning a log means pulling its entire contents, and the log is somebody's infrastructure, so the request is a throttle in prose form.

The x509util directory holds certcheck, which allows display and verification of certificates, and crlcheck, which does the same for certificate revocation lists.

Between them, the tools cover the workflow an analyst actually performs. Pull a certificate out of a log and look at it with certcheck, confirm the SCT on it with sctcheck, look up the revocation list with crlcheck, and search a log for the certificates matching a pattern with scanlog.

The x509util directory is also where the library utilities for certificates live, so the command line tools and the library API are in the same place, which is convenient when you are working out what the library can do before you write any Go.

The rest of the code is infrastructure for running a log and for policy: the trillian directory, the ctpolicy directory, the schedule directory that handles log scheduling, the submission directory, the gossip directory for cross log communication, the logid directory for log identifiers, and the preload directory that holds precertificates.

Running a log on Trillian, and the code generation workflow

The trillian directory holds code and scripts for running a CT log using the Trillian general transparency log as its back end, and it is documented separately in its own README. The architectural point is that this repository does not implement a Merkle tree log. It provides the CT personality, the parts of a log that are specific to certificate transparency, and delegates the append only Merkle tree machinery to Trillian, which is a general purpose transparency log.

The module dependencies bear that out. Trillian itself is a direct dependency at v1.8.0, and there is a merkle library from the transparency-dev organisation alongside etcd, which is what Trillian stores its data in, plus a Postgres driver and gRPC tooling, a Prometheus client for metrics, cobra for the command line, and grpcurl for talking to the Trillian API.

The development workflow is the other thing worth knowing about, and it is a modern one. A presubmit script runs the various tools and tests over the codebase, and the documentation asks you to make sure it passes before sending a pull request.

# Install golangci-lint
go install github.com/golangci/golangci-lint/v2/cmd/[email protected]

# Run code generation, build, test and linters
./scripts/presubmit.sh

# Run build, test and linters but skip code generation
./scripts/presubmit.sh  --no-generate

Part of the code is autogenerated, from protocol buffer message definitions, from a GoMock implementation of the Trillian gRPC API, and from stringer methods for enums. The recommended way to regenerate is inside the same Docker image the Cloud Build uses.

docker build -f ./integration/Dockerfile -t ctgo-builder .
docker run -it --mount type=bind,src="$(pwd)",target=/src ctgo-builder /bin/bash -c "cd /src; ./scripts/install_deps.sh; go generate -x ./..."

The reason to use the image is stated plainly: the correct versions of the tools are determined using the go.mod file, so the generation is reproducible. The alternative is installing a list of tools by hand, plus protoc at version 3.20.1 with its bin directory on the PATH, and then running go generate to hunt for the go:generate comments and run them.

Editorial conclusion

Adopt this repository if you need to work with Certificate Transparency data in Go, whether you are running a log, scanning one, verifying signed certificate timestamps, or building tooling on top of CT, since it contains the encoding, the protocol clients and the log operations in one place. Do not adopt it as a general purpose x509 library, because the forks are deliberately more permissive about malformed input than the standard library and that behaviour is wrong for a validator. Before you build against it, read the fork rationale, check the Go version you have against the one in go.mod, and note that the licence is a custom one rather than a standard identifier, so read the LICENSE file before you ship it inside a product.

Frequently asked questions

Official sources

1. google/certificate-transparency-go on GitHub
2. Issues
3. Project website
4. README
5. Releases