prometheus-msteams forwards Alertmanager notifications into Teams

It is a webhook relay, and the whole design follows from that

The overview says it plainly. This is a lightweight Go web server that receives POST alert messages from Prometheus Alert Manager and sends them to a Microsoft Teams Channel using an incoming webhook url. The reason it exists is a gap in the upstream project, because Alertmanager does not support sending to Microsoft Teams out of the box, but it does allow a generic webhook_config for cases like this. The project was inspired by idealista's prom2teams, which was written in Python, so the lineage is a Python tool rewritten in Go. The scope is deliberately narrow. There is no rule engine, no routing, no grouping, no deduplication, because Alertmanager already does all of that before the notification reaches this server. What the server does is translate the Alertmanager payload into a Teams message card and post it to the webhook, which is a small enough job that the interesting question becomes how small the deployment can be.

A static binary and a scratch image

The README has a section headed Why choose Go, and the answer is about deployment rather than about the language. A Go binary is statically compiled, unlike the other simple languages, Python, Ruby, Node, and having a static binary means there is no need to install the program's dependencies, and those dependencies take up a lot of space in your Docker image. The Dockerfile takes that argument to its conclusion. It starts from alpine:latest as certs, installs ca-certificates and tzdata, and then uses FROM scratch, so the final image contains no shell and no package manager. It copies the certificate bundle and the zoneinfo directory out of the first stage, copies the two default card templates, the default-message-card.tmpl and the default-message-workflow-card.tmpl, copies the compiled binary to /promteams and sets that as the entrypoint. Exposed port 2000. The Makefile builds for linux, darwin and windows with CGO_ENABLED=0, cross compiling the cmd/server package and stamping the version, commit, branch and build date into the binary through linker flags. The result is an image with a few megabytes of content rather than a few hundred.

The container run is a single command:

docker run -d -p 2000:2000 \
    --name="promteams" \
    -e TEAMS_INCOMING_WEBHOOK_URL="https://example.webhook.office.com/webhookb2/xxx" \
    -e TEAMS_REQUEST_URI=alertmanager \
    quay.io/prometheusmsteams/prometheus-msteams

Two ways to point Alertmanager at it

By default the server creates a static URI handler at /alertmanager and a dynamic URI handler at /_dynamicwebhook/*. The static handler is the simple case, where the incoming webhook URL is configured on the server and Alertmanager posts to a fixed path, and the receiver configuration is a route with a group_by of alertname and a receiver named prometheus-msteams whose webhook_configs point at the local proxy with send_resolved set to true. The dynamic handler is the interesting one, because it lets you pass the webhook URL to the proxy directly from Alertmanager, so the target Teams channel is decided per notification rather than per process. The README is explicit that by default the passed URL is not validated, and that if validation is needed you pass the flag -validate-webhook-url on start. A valid URL matches a specific regular expression requiring a host of lowercase letters and digits under webhook.office.com, the webhookb2 path, a user and team identifier, and an IncomingWebhook section. That default is the thing to be careful about, because an unvalidated URL in a request path is a request to an address the server will then post to.

The webhook URL as a credential, hiding it from the UI

The third configuration is the one that solves a problem most people hit, and it uses the webhook as a means of authorization. Instead of putting the URL in the path, you set an http_Config authorization block with type webhook and the credentials holding the webhook URL without the https:// prefix, and the url points at the plain /_dynamicwebhook/ path. The README gives the added benefit plainly: this provides the benefit that Alertmanager will treat the Incoming Webhook as a credential and hide it from view in the Web UI. That is a real operational benefit, because the Alertmanager web interface shows receiver configurations, and a URL containing a Teams incoming webhook is a credential that anyone with read access to Alertmanager can then use to post to your channel. The trade-off is that the server still has to read the credential from the request header, so the protection is at the Alertmanager layer rather than inside the proxy. The README also points at stefanprodan's Prometheus in Docker project for setting up a local Prometheus to try the flow on a single machine.

Card templates are where the message content is decided

The customisation sections cover both the message format and the per-channel format. The default templates ship in the repository as default-message-card.tmpl and default-message-workflow-card.tmpl, and the binary option says you should run the binary in the same directory as the stored default-message-card.tmpl, which is a small operational constraint but a real one, since a container that does not copy the template into the working directory will fail to render. The examples directory holds a README, a message-cards directory and a templates directory, which is where the format of the card is documented. The template functions section is worth noting because it is what makes the cards configurable rather than hard-coded, and the examples/templates directory is where you would look at what a template can reach for. Sending to multiple Teams channels is a separate feature, driven by a configuration file, and there is a section on creating that file and a second Alertmanager setup for it. The repository also carries a chart/ directory for a Helm chart, docs/, an e2e/ directory, ct.yaml for chart testing and examples/.

The dependency set shows what the server actually is

The go.mod is instructive because the project imports the Alertmanager module itself, github.com/prometheus/alertmanager at 0.34.1, so the payload types are the real ones rather than a copy of them, and the notification types are the ones that come with a matching API version. Alongside that are labstack/echo/v4 as the HTTP framework, hashicorp/go-retryablehttp for the outbound calls to Teams, and oklog/run for the lifecycle of the process signals. The configuration side is peterbourgon/ff for flags, gopkg.in/yaml.v2 for the config file, and prometheus/client_golang for its own metrics. The observability side is interesting, because there are two tracing stacks present, contrib.go.opencensus.io/exporter/jaeger and contrib.go.opencensus.io/exporter/prometheus alongside go.opencensus.io itself, so the server traces its own request handling and exports spans to Jaeger as well as metrics to Prometheus. The k8s.io/helm dependency at 2.17.0 is for the chart in the chart/ directory, and github.com/go-kit/kit is there for the service and transport helpers.

Editorial conclusion

Adopt prometheus-msteams when your on-call engineers read alerts in Teams and you want a small, static binary to sit between Alertmanager and an incoming webhook. Do not adopt it if you need the notification content to arrive richer than the card templates allow, since the Teams message format is the limit. Verify first which handler you use, because the dynamic handler puts a webhook URL in the path unless you use its authorization mode, that your webhook URL matches the validation pattern if you enable -validate-webhook-url, and that the teams-request-uri you configure in Alertmanager matches the route the server exposes.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. prometheus-msteams/prometheus-msteams on GitHub
4. README
5. Releases