PMM: find the slow query, not just the slow server

The pitch is a query, not a dashboard

The README's headline is the argument in one line: Find the slow query, not just the slow server. The subtitle describes the product as open source monitoring and query analytics for MySQL, PostgreSQL, MongoDB and Valkey/Redis, self-hosted, and every feature is open source, there is no paid edition.

That last clause is the unusual one. Most products in this space split into a free monitoring tier and a paid query analytics tier, and a database vendor's monitoring tool in particular is a common way to sell a platform. The README states plainly that no such split exists here, and the licence badge reflects a dual grant of AGPL-3.0 and Apache-2.0.

The database coverage is wider than the four names in the subtitle once the badge list is read. Alongside MySQL, Percona Server, MariaDB, PostgreSQL, MongoDB, Valkey and Redis, the documentation links cover ProxySQL, HAProxy, Amazon RDS and Aurora, Azure and Google Cloud. A note under the badges says that supported versions follow the Percona services lifecycle policy, which is the document that decides when a given database version stops being monitored.

The main components of the repository follow that scope. There is an admin/ directory for the server side, an agent/ directory for the client that sits on the database host, a qan-api2/ directory for the query analytics service, an api/ directory and api-tests/ for the API surface, dashboards/, ui/ for the frontend, managed/ for the managed services, a vmproxy/ directory, documentation/, and dev/ and build/ for the development setup. The primary language is Go, and the module declares go 1.27.2.

The quickstart is one curl and two pmm-admin commands

The installation path is unusually short, and it is worth reading as a whole because the security-relevant decisions are embedded in it.

Step one starts the server, and the script installs Docker if it is missing:

curl -fsSL https://www.percona.com/get/pmm | /bin/bash

Then you open the server address over HTTPS and log in as admin with the password admin, and set a new password. The README is explicit that the script assumes a fresh server with its self-signed certificate, and for production it says to use a trusted certificate, which means dropping the --server-insecure-tls flag, and to register with a service account token instead of the admin password.

Step two runs on the database host, and the example shown is Debian and Ubuntu:

wget https://repo.percona.com/apt/percona-release_latest.generic_all.deb
sudo dpkg -i percona-release_latest.generic_all.deb
sudo percona-release enable pmm3-client release
sudo apt update && sudo apt install -y pmm-client

sudo pmm-admin config --server-insecure-tls --server-url=https://admin:<password>@<server-ip>:443
sudo pmm-admin add mysql --username=pmm --password=<pass> --query-source=perfschema

The last two commands are the whole configuration surface. pmm-admin config registers the client against the server, and pmm-admin add creates the monitoring target with a database user and a query source, perfschema in this case, which is the MySQL instrumentation that lets PMM see the statements rather than only the server state.

After that you open Query Analytics and your queries appear within a minute, according to the README. The pmm database user needs a specific set of privileges, and the documentation link is given for the exact list, which is the kind of detail worth reading before you create the account rather than granting broad rights.

Guides for RHEL, Docker, PostgreSQL, MongoDB, Valkey/Redis, Amazon RDS/Aurora, Azure and Google Cloud are all in the full quickstart, and the PMM Server itself can be run as Docker, Podman or Kubernetes with Helm.

The shipped defaults are development defaults

The environment file in the repository is a good place to see the shape of the configuration, because every important knob is a variable and every variable has a default that assumes a development machine.

PMM_SERVER_IMAGE=percona/pmm-server:3
PMM_ENABLE_ACCESS_CONTROL=1
PMM_ENABLE_INTERNAL_PG_QAN=1
PMM_ENABLE_NOMAD=0
PMM_DEBUG=0
GF_SECURITY_ADMIN_USER=admin
GF_SECURITY_ADMIN_PASSWORD=admin
PMM_PUBLIC_ADDRESS=localhost
PMM_PORT_HTTPS=443

Two of those deserve a second look. GF_SECURITY_ADMIN_PASSWORD is admin, and the comment above it says to set a strong admin password for production environments, which is the same instruction the quickstart gives about changing the password after the first login. And PMM_ENABLE_INTERNAL_PG_QAN is 1, with a comment saying that enabling QAN for the internal PostgreSQL database is not recommended for production environments, which tells you that the supported production arrangement puts the query analytics storage somewhere else.

That somewhere else is ClickHouse, and the file shows two ways to reach it. There is a set of PMM_CLICKHOUSE_ variables for an external ClickHouse, including PMM_CLICKHOUSE_ADDR, DATABASE, USER and PASSWORD, plus a set for the cluster case, PMM_CLICKHOUSE_IS_CLUSTER and PMM_CLICKHOUSE_CLUSTER_NAME, and PMM_DISABLE_BUILTIN_CLICKHOUSE=1 to turn off the embedded one.

The one detail that is easy to get wrong is the second ClickHouse user. PMM_CLICKHOUSE_DATASOURCE_USER=grafana with its own password is described as a read-only user for the Grafana ClickHouse datasource, and the comment says it must NOT be the user above. The two are different jobs, one writes and one reads, and the file is telling you to keep them apart.

The same pattern repeats for the external PostgreSQL, with PMM_POSTGRES_ADDR, PMM_POSTGRES_DBNAME=pmm-managed, PMM_POSTGRES_USERNAME, PMM_POSTGRES_DBPASSWORD, PMM_POSTGRES_SSL_MODE=verify-full and PMM_POSTGRES_SSL_CA_PATH for the certificate path.

The compose file is a local test harness with profiles

The docker-compose.yml in the repository is not the production deployment. The comment at the top of the file says its purpose is to spin up a fully wired PMM stack on a local machine with one command, including a monitored database workload, to get a sense of PMM working end to end without any manual configuration, and it documents three ways to use it:

#  1. Minimal suite (server, client, mysql + sysbench), where sysbench is used to generate load on MySQL:
#     docker compose -f docker-compose.yml --profile pmm up -d
#  2. Full suite (server, client, postgres, mongo, mysql + sysbench):
#     docker compose -f docker-compose.yml --profile pmm --profile postgres --profile mongo up -d
#  3. Full suite + external clickhouse (server, client, postgres, mongo, mysql + sysbench, external ClickHouse):
#     docker compose -f docker-compose.yml -f qan-api2/docker-compose.yml --profile pmm --profile postgres --profile mongo up -d

The profiles are pmm, postgres and mongo, and the sysbench workload generator is what makes the local stack produce the queries the tool exists to analyse. A third variant swaps in the query analytics compose file for an external ClickHouse, which mirrors the production arrangement described in the environment file.

The server service in that file is image percona/pmm-server:3 with the tag coming from PMM_SERVER_IMAGE, platform linux/amd64, restart always, and an environment block that repeats the same variables as the .env file, including the Grafana, ClickHouse and access control switches.

There is one deliberate choice in the file worth noticing, and it is a good one. The comment says the database ports are intentionally not mapped to the host since the communication takes place on a shared bridge network. A development stack that publishes its database ports is a development stack that gets confused with a production one, and this one does not.

The Makefile targets are all developer oriented for the same reason. env-up starts the devcontainer with the pmm profile and waits for it, env-pg-config tweaks the embedded PostgreSQL for development, env-down stops the stack and env-remove takes the volumes with it. There is also a docker-compose.dev.yml and a .devcontainer/ directory for the editor environment, and a get-pmm.sh script at the root that is what the quickstart curl pipe fetches.

What the Go module list says about the architecture

The dependency list in go.mod is the closest thing the repository has to an architecture diagram, and it describes a service made of several specialised parts.

There is a ClickHouse client, github.com/ClickHouse/clickhouse-go/v2, which is the query analytics store, and a Grafana OpenAPI client, which is how the dashboards read it. There are cloud SDKs for Amazon, Azure and Google, which is how PMM reaches a managed database, and a set of go-openapi packages for errors, runtime, strfmt, swag and validate, which together are the OpenAPI tooling the API surface is generated with. There is golang-migrate for schema migrations, grpc-gateway and the gRPC middleware for the RPC layer, a Prometheus client and the Prometheus server itself, and a MySQL driver.

Two details stand out because they are not ordinary dependencies. The module replaces github.com/go-openapi/spec with a Percona fork, and it replaces alecthomas/kingpin/v2 with a Percona fork of a specific commit. Forking the OpenAPI specification package and the command line parser is the sort of change that only makes sense when the upstream project cannot take a patch, and it is a maintenance cost that a self hosted deployment inherits.

The go version declared is 1.27.2, which is a recent toolchain requirement, and the tooling around the repository is correspondingly modern. There is a .golangci.yml for linting, a .mockery.yaml for mock generation, a .licenserc.yaml for the licence headers, a codecov.yml for coverage, and a Makefile.include that the main Makefile pulls in.

So the shape is a Go backend with an API, an agent that lives on the database host, a query analytics service backed by ClickHouse, a Grafana based dashboard layer, and a frontend in ui/. It is a large surface for a monitoring tool, and the trade is that you get statement level visibility into the databases you care about.

Editorial conclusion

Adopt PMM if you run MySQL, PostgreSQL, MongoDB or Valkey and Redis in production and you need statement level query analytics next to the usual metrics, and if you want every feature available without a commercial contract. Do not adopt it as a general purpose metrics platform, since its scope is database monitoring, and do not run it with the shipped defaults in production, because the environment file sets the admin password to admin. Verify first that your database version is covered by the Percona lifecycle policy, that the pmm database user you create holds exactly the privileges listed in the documentation, and that your ClickHouse datasource account is a separate read only user from the one PMM writes with.

Frequently asked questions

Official sources

1. License: AGPL-3.0
2. percona/pmm on GitHub
3. Project website
4. README
5. Releases