iovisor/bcc: writing eBPF tracing tools in C and Python

The gap BCC fills between a kernel module and a shell one-liner

Instrumenting a live Linux kernel used to mean building and loading a module, or accepting whatever fixed counters the kernel already exposed. eBPF changed that: the kernel can run user-defined, sandboxed bytecode attached to kprobes, and as the README quotes Ingo Molnár describing it, this allows user-defined instrumentation on a live kernel image that can never crash, hang or interfere with the kernel negatively. The problem is that writing that bytecode directly, and getting it loaded and wired to the right probe, is its own project.

BCC is aimed at the people who would otherwise stall at that step. The README describes it as a toolkit for creating efficient kernel tracing and manipulation programs, with kernel instrumentation written in C and front-ends in Python and Lua. The audience is not kernel developers. It is SREs, performance engineers and application teams who can write a Python script and want to know which process is issuing which syscall, how long block I/O is taking, or what a lock is doing.

The repository reflects that split. There is src/ for the C library, tools/ for the ready-made Python tools, examples/ for smaller programs, and docs/reference_guide.md for the bcc and bcc/BPF APIs. A separate libbpf-tools/ directory exists for the newer libbpf-based rewrites, which is worth noticing: the project itself carries two generations of tooling side by side.

How a BCC program actually moves data from kernel to user space

The mechanism is visible in the README's own worked example. bitehist.py traces a disk I/O kernel function and populates an in-kernel power-of-2 histogram of the I/O size. The README is explicit about why: for efficiency, only the histogram summary is returned to user-level. That is the core design idea. Aggregation happens inside the kernel in BPF maps, and user space reads a summary rather than a stream of events.

The sample output shows a bimodal distribution where the largest mode of 800 I/O was between 128 and 255 Kbytes in size. Reading that required no per-event transfer.

A BCC program is usually a single file containing both the C that runs in the kernel and the Python that drives it, which is why the README notes that some entries are single files containing both C and Python, others have a pair of .c and .py files, and some are directories. The C side is compiled at load time; the README says BCC includes a C wrapper around LLVM. That compile-on-load is the trade-off you inherit: you get to write C with helper macros instead of raw BPF instructions, and you pay for a compiler in the process.

The README also lists what can be attached to. Examples cover kernel tracepoints (urandomread.py traces random:urandom_read), kprobes (stacksnoop traces a kernel function and prints kernel stack traces), and USDT probes for user space, with mysqld_query.py and nodejs_http_server.py tracing MySQL and Node.js respectively. That range is the reason the same toolkit shows up in IO analysis, networking and monitoring.

Installing BCC and running your first trace

The README does not inline installation steps. It says to see INSTALL.md for installation steps on your platform, and the repository confirms that file exists at the top level alongside QUICKSTART.md and FAQ.txt. So the first real step is reading INSTALL.md for your distribution rather than following a command from this article.

The tools ship in the repository under tools/. If you have a working installation, the README's own screenshot example is run from the examples directory:

# ./bitehist.py
Tracing... Hit Ctrl-C to end.
^C

After you interrupt it with Ctrl-C, the histogram is printed. The README's captured output looks like this, with a bimodal shape and the largest bucket between 128 and 255 Kbytes:

     kbytes          : count     distribution
       0 -> 1        : 3        |                                      |
       4 -> 7        : 211      |**********                            |
     128 -> 255      : 800      |**************************************|

For a first probe that is easier to reason about, the README points at examples/hello_world.py, which prints "Hello, World!" for new processes. That is the smallest thing to run to confirm the toolchain works end to end before you trust a longer script.

If something fails, the README directs you to FAQ.txt for the most common troubleshoot questions, and docs/reference_guide.md for the API reference. Both are in the repository root and docs/ respectively.

Where BCC stops being the right tool

The kernel version floor is the first constraint, and the README states it plainly: eBPF was first added to Linux 3.15, and much of what BCC uses requires Linux 4.1 and above. On older kernels you are not debugging a configuration problem, you are below the floor.

The second constraint is the compile-at-load model. Because the C side is compiled when the program runs, the tracing host needs a compiler and the matching headers for the kernel it is tracing. That is a heavier dependency than a pre-compiled probe, and it is the reason the project maintains a parallel libbpf-tools/ directory: those tools take the other approach. If your environment is a locked-down production host where you cannot install a toolchain, the Python-and-LLVM path in tools/ is the wrong half of the repository for you.

The third is scope. BCC gives you a library and a set of tools; it does not give you a dashboard, retention, or alerting. The README describes tracing and manipulation programs, not an observability platform. If what you actually need is a metric that survives a restart and pages someone, you are looking at the wrong layer.

Finally, the README warns indirectly about the tools themselves: reset-trace is labelled a maintenance tool only. That label is a reminder that some entries in tools/ are not analysis tools at all, and reading the per-tool example files before running something is the difference between a measurement and a change to system state.

BCC against bpftrace, and against the libbpf tools in the same repository

The most direct alternative for ad hoc probing is bpftrace, which appears in the related searches for this project because the two get compared constantly. The difference in approach is the front end. BCC expects you to write kernel-side C plus a Python or Lua driver, which gives you full control over maps, histograms and output formatting but means a script is a small program. bpftrace is a single-language, awk-like tool for one-off probes, so it wins when the question is short and loses when you need structured aggregation logic or a tool you will maintain and extend.

A more interesting comparison is inside the repository. The libbpf-tools/ directory holds a second set of tools built on libbpf instead of the BCC runtime. The practical difference is when the BPF program is compiled: BCC compiles at load time through its LLVM wrapper, while the libbpf approach compiles ahead of time. That is the same axis as the toolchain dependency above. If you are choosing where to invest, the README's own layout is telling you the project considers both viable, and the tools/ directory is the one with the longer history and the broader catalogue.

There is also the plain-syscall route: reading /proc, using perf, or writing a kernel module. BCC's advantage over a module is the safety property quoted in the README, that the instrumentation can never crash, hang or interfere with the kernel negatively. Its advantage over perf is that you get to define the aggregation rather than pick from a fixed set.

Editorial conclusion

Adopt BCC if you need to attach eBPF programs to kprobes and tracepoints and want the tooling to compile them for you, and if a Python or Lua front end fits your team better than writing raw BPF C. Skip it if your target kernels are old enough that much of what BCC uses is unavailable, or if you only need a one-line ad hoc probe, where bpftrace is the lighter choice. Before committing, read INSTALL.md for your distribution, check the kernel version on the machines you intend to trace, and confirm which of the tools in tools/ cover the subsystem you actually care about. The licence is Apache-2.0, and the repository layout puts the Python tools and the C library under the same licence file.

Frequently asked questions

Official sources

1. iovisor/bcc on GitHub
2. Issues
3. License: Apache-2.0
4. README
5. Releases