mtail: Turning Application Logs into Prometheus Metrics Without Touching the App

The niche mtail fills: logs as the only window into application state

Many applications do not export their own internal state. They write logs, and that is all an operator has. mtail exists to close that gap. It reads log files, applies pattern-based programs, and produces timeseries metrics that a collector can scrape. The README is explicit about the target user: system operators who do not want to patch applications or write custom extraction code for each one. This is a specific, narrow job. It is not a general log aggregator, and it is not a metrics server. It is the glue between a log stream and a timeseries database. If you control the application source and can add a metrics endpoint, mtail is probably the wrong tool. If you cannot, it may be the only option short of writing a bespoke parser.

How mtail programs work: pattern matching with counters

The core of mtail is its program language. A program defines metrics and the patterns that update them. The README gives a minimal example: a counter named lines_total, a pattern that matches every line, and an action that increments the counter. The pattern is written in a syntax derived from awk and regex. The action is a set of statements that modify metrics. The language supports more than counters; the documentation references managing internal state, which implies the ability to track values across log lines, such as timing or state transitions. The extraction is driven by the program, not by the log content itself. This means the operator decides what to extract and how. The trade-off is that you must learn the language and maintain the programs as log formats evolve. The README points to a Programming Guide and a Language Reference for that learning curve.

Export paths: Prometheus, JSON, and push-based collectors

Once mtail has extracted metrics, it must get them somewhere. The README lists two export modes. The first is HTTP scraping, serving metrics in Prometheus text format or JSON. Prometheus scrapes that endpoint. The second is periodic sending to a collectd, StatsD, or Graphite collector socket. This covers two common monitoring architectures: pull-based (Prometheus) and push-based (StatsD, Graphite). The README recommends pairing mtail with a timeseries-based calculator and alerting tool like Prometheus. That pairing makes sense because Prometheus can scrape mtail's endpoint and evaluate alerting rules. The JSON export is for systems that do not understand Prometheus format. This flexibility is useful, but it also means you need to configure the export target correctly per deployment. The documentation for interoperability and deployment is listed in the README, so the details live there, not in the main text.

Getting mtail running: binaries, Go modules, and Docker

Installation has three documented paths. The recommended one is to download a precompiled binary from the Releases page. Windows, OSX, and Linux binaries are available. The second path is building from source. The simplest command is go get github.com/google/mtail/cmd/mtail. The README notes that you must enable Go Modules with GO111MODULE=on to fetch the full source tree, and then run make install. The third path is a Dockerfile included in the repository, which handles build dependencies. The README warns that the compiler development requires extra tools like goyacc to rebuild the parser. For most users, the precompiled binary is the right choice. The source build is for those who want to modify mtail itself, not for deploying it. The Dockerfile is for local development, not necessarily for production deployment, though it could be adapted.

A real limitation: log format stability and stateful programs

mtail's approach has a clear failure mode. The extraction logic depends on the exact format of the log lines. If an application changes its log format, the mtail program must be updated. The README does not mention any automatic adaptation or fuzzy matching. The language is precise, which is good for correctness but brittle for evolving logs. Another limitation is the stateful nature of some programs. The documentation mentions managing internal state, which implies that mtail tracks values across lines. That state can be corrupted or reset if the log stream is interrupted or if the process restarts. The README does not describe how state persists across restarts. This is a genuine gap. If you need to extract metrics from logs that are not line-oriented or whose format changes frequently, mtail will require constant maintenance. It is also a single-process tool; the README gives no indication of horizontal scaling, so very high log volumes could overwhelm it.

Alternatives: general log pipelines versus dedicated extractors

The main alternative to mtail is a general-purpose log processing pipeline like Fluentd or Logstash, combined with a metrics plugin. These tools can parse logs, extract fields, and forward metrics to a timeseries database. The difference is in approach. mtail is a dedicated extractor with a custom language designed for metrics. Fluentd uses configuration with built-in parsers and plugins, so you do not learn a new language, but you do configure a pipeline with more moving parts. mtail runs as a single process that reads files directly. Fluentd often runs as an agent with its own buffer and retry logic. If you already run Fluentd for log collection, adding a metrics plugin might be simpler than introducing mtail. If you need precise per-line extraction with minimal overhead, mtail's language is more direct. The README does not compare itself to these tools, so this is a practical judgement based on the described architecture.

Maintenance and licensing: Apache-2.0 and a slow release cadence

mtail is licensed under Apache-2.0, which permits commercial use, modification, and redistribution with attribution. That is a permissive license with no copyleft obligations. The repository is not archived, and the last push was August 2024, with version 3.0.8 released then. The release cadence appears steady, with three releases in the summer of 2024. The README recommends using the latest production release binary, which suggests that releases are considered stable. Maintenance cost comes from writing and updating mtail programs. The documentation includes a testing guide, which implies that testing programs is expected. There is no mention of a migration path between mtail versions, so upgrades may require revalidating programs. The project has an active issue tracker and a mailing list, so support channels exist. Overall, the maintenance burden is on the program language, not the tool itself.

Who should adopt mtail and what to verify first

mtail is for operators who cannot instrument applications but have logs. It is not for greenfield projects where you control the application code. Before adopting it, verify three things. First, confirm that your log lines are parseable with the mtail language. The README gives a simple counter example, but complex extractions may require constructs you must check in the Language Reference. Second, test how mtail handles log rotation and restart. The README does not describe this, so you must read the Deploying and Troubleshooting docs. Third, measure the log volume against mtail's throughput. The README gives no numbers, so you need to run your own load test. If those checks pass, mtail can be a reliable bridge from logs to metrics. If any fail, consider a general log pipeline or patching the application. The final judgement: mtail is a focused tool that solves a real problem, but its success depends on your log formats being stable and your willingness to maintain its programs.

Editorial conclusion

Adopt mtail if you operate applications that only expose internal state via logs, you cannot patch them to emit metrics, and you are willing to maintain a set of mtail programs in a dedicated language. Do not adopt it if your applications already expose metrics endpoints or if you prefer a general-purpose log processing pipeline like Fluentd with a metrics plugin, which requires no custom language but offers less precise per-line extraction. Before deploying, verify that your log lines have stable, parseable formats, that the mtail program language supports the patterns you need, and that your log volume does not exceed mtail's single-process throughput, which the documentation does not quantify.

Official sources

1. Official README
2. Project repository
3. Release notes