palantir/stacktrace: stack traces for Go errors, with context you chose

The joke, and the claim underneath it

The readme opens with a joke, and the joke is the point.

Look at Palantir, such a Java shop. I can't believe they want stack traces in their Go code.

Then the section heading asks the question that the library exists to answer: why would anyone want stack traces in Go code?

And the answer is given as a contrast between two error messages. The first is a single line:

Inverse tachyon pulse failed

The second is the same failure with its context attached, and the difference between the two is the entire argument for the library.

That second message has a shape worth reading carefully, because every part of it is a decision. Each entry names a file, a line number and a function, in that order, between a pair of rules. Each failure is introduced by a phrase saying what was being attempted, and each failure that has an underlying cause is introduced with the words caused by. The chain reads downwards, from the outermost operation to the innermost cause, which is the direction a reader wants: what were we doing, and what actually went wrong.

The names in that example are a joke too, a parody of a codebase that is all hardware and jargon, and the point of the joke is that the message is still readable. The example is a small program that fails for a mundane reason, and the output tells you the three layers it passed through and the one that broke.

And then there is the paragraph that tells you when not to use this, and it is more important than the example.

Stack traces are not designed to be user-visible. The readme says the authors have found them to be valuable in the log files of server applications, and that nobody wants to see these in command line output, in a web interface, or in a return value from library code.

That is an unusually clear boundary, and it is a boundary about the audience rather than about the technology. A Go error that crosses an API boundary should be a message and a code. The same error, three layers down inside a server, is something you want explained in the log. The same library helps in one case and hurts in the other, and the readme is telling you which case you are in before you install it.

What it is not

The intent section is short and it draws a line that most stack trace libraries do not.

The intent is not to capture the exact state of the stack when an error happens, including every function call. For a library that does that, the readme points at another project. The intent here is to attach relevant contextual information, messages and variables, at strategic places along the call stack, keeping stack traces compact and maximally useful.

The two sentences are a rejection and a replacement, and both matter.

The rejection is of the captured machine stack. A Go runtime stack trace is a list of every function that was on the call stack, which for a server handling a request through a dozen layers of framework code is a wall of text that tells you the request went through the framework, which you already knew. It is also a snapshot of a moment rather than a record of a decision, and it does not contain anything the programmer wrote. Capturing it also costs: the runtime has to walk the stack, and the information it produces cannot be curated.

The replacement is the interesting phrase: at strategic places along the call stack. Strategic means chosen. The programmer decides at which points an error deserves a sentence of explanation, and those points are the ones where the function knows something the caller does not. A function that loads a configuration file from a path knows the path. A function that calls out to a service knows which service and what it asked for. Those are the places where context is added, and everywhere else the error passes through untouched.

So the resulting trace is not a stack capture at all. It is a chain of annotations, and the programmer is the author of the chain. The consequence is that the output is compact, because the number of lines is the number of places somebody decided to annotate rather than the depth of the call stack, and the content is meaningful, because every line says something the caller would otherwise have to guess.

That is a fundamentally different philosophy from the captured-stack libraries, and it explains the shape of the API. The library is a set of functions you call at the points where you have something to say, not a wrapper you install once and forget.

Propagate, and the discipline of writing the message

The most common call is the propagation function, which wraps an error to include line number information, and the readme spends more space on how to write the message than on the signature.

The message and its arguments work like the formatting function everyone knows, so there is nothing new to learn there. What is new is the content, and the readme gives a test for it: ask yourself what does this call do?

If the call processes an argument, then the message is that we failed to process that argument. The canonical shape is a call, an error check, and a return with the wrapped error and a message.

Then the rule about redundancy, which is the part that takes practice. The message must not repeat what the error already says. The readme's example is a directory creation: the error from the operating system already contains the path it failed to create, so putting the path in your message too is noise. But that error will not say that this path is the base directory rather than any other path, so that is the information worth adding.

So the test is: does the error already say this, and does the error fail to say the part you know? The first is the reason not to write it, the second is the reason to.

Then there is the empty message case, and the readme's explanation of it is one of the best lines in the documentation. If it is not possible to add any useful contextual information beyond what is already included in an error, the message can be an empty string. And then: the purpose of the empty string, as opposed to a separate function, is to make you feel a little guilty every time you do this.

That is a lovely piece of API design. A separate function for the no-message case would be a legitimate alternative, and it would be invisible; a caller would use it without thinking. By making the empty message the same function with an empty first argument, the author makes the absence of context a visible decision that you type and see, and the joke does the work that a lint rule would otherwise do. A reader of the code can see at a glance where the author did not bother to explain anything, and the guilt is a reasonable proxy for review.

The empty message case is also correct from the library's point of view, because the empty case still gets a line number and a function name, which is often the thing you actually wanted. The trace still says where the error came from; it just does not add a sentence to it.

The other behaviour worth noting is the nil case. The propagation function returns nil when the cause is nil, so there is no need to check the error before wrapping it. That removes a class of boilerplate from every call site, and it removes a class of bug where a nil error is wrapped into a non-nil one.

Error codes, and why they survive the unwinding

The second half of the library is a numeric code attached to an error, and the readme is careful about one property of it: an ordinary propagation preserves the error code of the error it wraps.

That is the whole design in one sentence. If a low-level function attaches a code and an intermediate function wraps the error with a message, the code survives, and the code that reaches the top of the stack is the one attached at the bottom. Which is the opposite of what happens with a message, where every layer typically adds its own text and the top-level message is the one you see.

The type is a typedef for an unsigned sixteen bit integer, and you name the set of codes that are relevant to your application, in a constant block built from an incrementing value.

const (
    EcodeManifestNotFound = stacktrace.ErrorCode(iota)
    EcodeBadInput
    EcodeTimeout
)

The readme explains the motivation with a concrete case: a RESTful API may use the error code to set the HTTP status code. That is the reason a code is a number rather than a string, and it is a good reason, because a number can be compared, mapped, and translated into a status without parsing.

There is one reserved value, and the readme is specific about it. The special value for no code is equal to the largest sixteen bit unsigned integer, so avoid using that, and it is the code of errors that have no code explicitly attached. So the zero value of a code is a perfectly ordinary code, and the maximum is the one you must not use, which is an unusual arrangement and one that is documented rather than left to be discovered.

The code-aware functions come in four forms, and the differences between them are the differences between the two base functions plus one more axis. The two base functions are propagation, which wraps an existing error, and creation, which makes a new one and is described as a drop-in replacement for the standard formatting function, with line number information added.

The two with codes do the same and attach a code. And then there is a fifth variant, which creates a message with a code but no line number, printing exactly like the standard formatting function with the code added. The readme says the error code mechanism can be useful by itself where stack traces with line numbers are not required, which is the case for an API layer that wants to map errors to status codes and does not want stack information in a response.

And the extraction function completes the set: it pulls the code off an error, returning the reserved value if the error is nil or has no code attached. The readme's example uses it in a retry loop, where a particular code means try again and any other code means give up.

The repository, and what the file names say

The repository is small, and the file names describe the design.

There is a file for the core of the library, a file for the cause chain, which is the mechanism that produces the lines beginning with the words for an underlying cause, a file for the formatting, and a documentation file that holds the package documentation. Then there is a file for the path handling, in its own directory, whose name suggests it holds the code that tidies the file paths that appear in the output.

That last one is worth a moment, because it explains something visible in the example. The paths in the trace are full paths into the source tree, and a trace that a person reads in a log file is easier to read if the path is shortened to the part that identifies the file. It is a small piece of presentation logic, and it is in a separate file because it is separable from the logic that builds the chain.

The test files follow the same shape as the source files, with two exceptions that are worth noting. There is a file of error codes used by the tests, which is what you would expect: the error code mechanism has a set of reserved and specific values, and the test suite needs a stable set of them to assert against. And there is a file of functions used by the tests, which is the same idea for the message formatting functions, whose exact output has to be pinned down by a test that will break when the format changes.

Both of those files exist so that the assertions are about behaviour rather than about literals scattered through the test code. When the format of a trace changes, there is one file to update and one place for the diff to show up.

The build and repository metadata is from an earlier era, and that is worth noting for a library that is still in use. There is a continuous integration configuration for a service that has since been folded into another, and badges for two different continuous integration systems, one of which is the successor of the other. The ignore file is the only other thing at the root besides the readme and the licence, and the readme points at the generated documentation on the package documentation site as the place for the full API reference.

So the shape is: a library with two ideas, a formatted chain of annotations and a numeric code that survives the chain, implemented in a handful of files with a test file beside each one.

Licensing, contribution, and the corporate layer

The licence is the Apache 2.0, released by the company whose name is on the repository, and the readme points at the licence file for the details. That is the permissive licence you would expect for a small library that any Go project can take a dependency on.

The contribution section is short and has a specific shape. Contributions of backward-compatible changes are welcome, and the list of what to do is four items: write the code, add tests for the new functionality, run the test command and verify the tests pass, and fill out the individual contributor licence agreement.

That last item is the one that tells you the most about the project. The repository contains two contributor licence agreement files, one for individuals and one for corporations, and the fact that both are there is a deliberate structure rather than an oversight. The individual agreement covers a person contributing in their own time. The corporate one covers an employee contributing on behalf of an employer, and it is the mechanism by which the company can receive the contribution without their employer claiming a position on it.

For a library maintained by a company with a legal department, that arrangement is the price of accepting outside contributions, and publishing both files is the way it is made clear to a contributor before they start rather than after they submit a pull request. It is also a mild constraint: a contribution from a company with a stricter policy on outbound licensing may need legal review, and a contributor who assumed the process was the permissive one is going to be surprised by a form.

The other detail is the requirement to add tests. For a library whose entire output is a formatted string, that requirement is not bureaucratic. The output of the trace is a contract: log parsers, alert rules, and human readers all depend on the format. A change that alters the format without anyone noticing is a change that breaks a downstream consumer, and requiring a test for new functionality is how the project keeps the format stable.

And the readme's last word on the matter is about scope. This is a small library with one idea, and the ideas it does not have are the point: it does not capture the machine stack, it does not try to replace the standard error package, and it does not attempt to be a general error handling framework.

Editorial conclusion

stacktrace fits a Go service whose logs need to say where a failure happened, and a codebase that wants error codes rather than string matching for control flow. It does not fit a library whose errors are returned to a user, because the readme is explicit that these traces are for logs. Verify first where your errors end up, since the design assumes they are logged rather than displayed, that the messages you add are worth the bytes, because every propagation is a line in the output, and that you follow the empty-message convention rather than filling every call site, since a trace with a line of context on every frame is the log noise the library is meant to avoid.

Frequently asked questions

Official sources

1. Issues
2. palantir/stacktrace on GitHub
3. Project website
4. README