HouseWatch puts ClickHouse introspection in one web interface

PostHog's debugging knowledge, packaged as a tool

The motivation section explains where HouseWatch came from. At PostHog the team manages a few large ClickHouse clusters and needed a tool to monitor and manage them more easily.

The starting observation is that ClickHouse is excellent at introspection. It exposes a lot of metadata about the system in its system tables, so that metadata can be queried. The hard part is that knowing how to query and parse that information is a difficult task, and over the years the team built intuition for debugging ClickHouse issues using ClickHouse itself. HouseWatch is the compilation of that knowledge into a tool.

The second half of the motivation is about tool sprawl. The team had internal systems for managing the clusters spread across platforms: Grafana for metrics, SSH into nodes for operations, Metabase for digging into system table data and building dashboards, plus a combination of tools inside the PostHog product for debugging and for efficient operations such as their async migrations tool, and internal views for listing queries and analysing their performance.

Putting those in one place is the pitch. The stated vision is that HouseWatch can serve the purpose of a pganalyze for the ClickHouse ecosystem while also including tooling for taking action on the insights derived from the analysis, so it is both an analysis console and an operations console.

What the interface actually covers

The feature list at the top of the readme is the quickest way to see the scope. You get an overview of cluster load and performance, the ability to drill down into your queries and understand the load they put on the cluster, search through logs and errors, and the ability to monitor and kill running queries with the click of a button.

There is also disk accounting per node, with a breakdown of how much space tables, columns and parts take up, and a query editor so you can run your own queries from the interface to dig further into performance and cluster issues.

The screenshots in the readme fill in the detail on the analysis side. There are pages for query performance, including a slow queries view, a normalised query view, query statistics and an explain view. There is a schema stats page with a drill-down. There is a query benchmarking section with two views. There are dedicated pages for logs, for the query editor, for disk usage, for errors and for operations.

That is a wide surface, and it is the reason the project needed a real application stack rather than a set of shell scripts. The analysis pages are read heavy, while the operations and query killing pages are write heavy, and both need to talk to ClickHouse safely.

Deploying with docker compose and environment variables

Deployment is a clone and a compose invocation with the environment substituted for one of your ClickHouse instances:

SITE_ADDRESS=<SITE_ADDRESS> \
CLICKHOUSE_HOST=localhost \
CLICKHOUSE_CLUSTER=mycluster \
CLICKHOUSE_USER=default \
CLICKHOUSE_PASSWORD=xxxxxxxxxxx \
docker compose -f docker-compose.yml up

SITE_ADDRESS is the address the UI will run on, and it can be a domain name or simply a port like :80. After the command completes, the UI is running on the address you specified.

There is a warning in the readme that is worth repeating to anyone deploying this. The author says they would think twice about exposing it to the internet, as it is not currently secured in any way. There is no login, no session handling and no role model in the described configuration, so anything the interface can do, anyone who can reach the port can do.

For local development there is a separate path. Running HouseWatch locally along with a local ClickHouse instance is a single command:

docker compose -f docker-compose.dev.yml up -d

and then the interface is at http://localhost:8080.

The environment variable list tells you the trust boundary

The documented configuration is short, and it is effectively a description of what HouseWatch needs from ClickHouse.

Four are required. CLICKHOUSE_HOST is the hostname of the instance to connect to, CLICKHOUSE_USER is the username used to access ClickHouse, and CLICKHOUSE_PASSWORD is the password for that user. CLICKHOUSE_DATABASE is optional and sets the database to connect to by default, and CLICKHOUSE_CLUSTER is optional and names the cluster to analyse data from across the whole cluster rather than a single node.

Three more are optional and point at the ClickHouse driver documentation, which is where their semantics live. CLICKHOUSE_SECURE, CLICKHOUSE_VERIFY and CLICKHOUSE_CA cover the transport security options of the driver.

The last two are the interesting ones. OPENAI_API_KEY enables the experimental AI Tools page, which currently features a natural language query editor, and OPENAI_MODEL selects a model you have access to with that key, for example gpt-3.5-turbo or gpt-4.

That natural language query editor is the experimental corner of the project, and it is worth thinking about what it means before enabling it. A feature that turns a sentence into a query against your cluster, backed by a third party model, sends schema and question text outside your environment. The readme marks it experimental, and it is off unless you supply a key.

The stack behind the interface

The compose file describes the application, and it is a conventional Django plus React deployment.

The app service builds from the repository root image, mounts the code, runs python manage.py migrate and then python manage.py runserver on port 8000. Its environment block sets a Postgres DATABASE_URL, a RabbitMQ RABBITMQ_URL, a Redis REDIS_URL and the ClickHouse variables, along with AWS credentials and a region, which points at the async migrations work coming from PostHog's own tooling.

The rest of the services fill in the rest of the stack. A web service is built from the ./frontend directory, a worker service runs ./bin/celery, and there are Postgres, Redis, RabbitMQ and ClickHouse services, with Caddy in front on ports 80 and 443 configured from a Caddyfile and the SITE_ADDRESS variable.

The Dockerfile shows the Python side. It starts from python:3.10, installs build tools and libraries such as libpq-dev, libxmlsec1 and libffi-dev, installs the pinned requirements, copies manage.py, the housewatch package and the bin directory, and runs collectstatic at build time. The pinned requirements file is generated by pip-compile from requirements.in, and the dependency list shows Django 4.1.1, Celery 5.2.7, the clickhouse-driver and clickhouse-pool packages, and the OpenAI client, among many others.

Operations with automatic rollback, and the Celery dependency

One feature stands apart from the analysis pages: setup operations that run in the background with automatic rollbacks for failures. That is the difference between a monitoring dashboard and a management tool, and it is where the risk lives.

Background execution comes from Celery, with the worker service running the celery binary against RabbitMQ. The to-do list on the readme includes making operations resilient to Celery going down, in the same way PostHog handles it with async migrations, and the readme also lists a read-only mode as a to-do item.

Both of those to-do entries are worth reading as a risk statement rather than a feature wishlist. A background operation that fails part way through and cannot roll back is a way to end up with a half applied change on a production cluster, and a Celery worker that dies mid operation is a way to lose track of what is running. Neither problem is solved in the current state of the project.

The read-only path is the mitigation that exists today. The readme says that if you connect HouseWatch with a read-only user, the cluster management features will not work, for example operations and the query editor, but the analysis toolset will function normally. The analysis pages are where the bulk of the value is, so a read-only connection gets you most of the tool with a much smaller blast radius.

Beta software, and a to-do list that describes the roadmap

The status section is unusually candid. HouseWatch is in its early days, there are a lot more features in mind, the code could use some cleaning up, and as of right now it is considered Beta software and you should exercise caution when using it in production.

The to-do list is published as a public list of things the maintainers intend to do in the near future, and it reads as a candid list of gaps. On the features side there is a system issues tab, an EXPLAIN visualizer, multiple instance support, stats on page cache hit percentage, a button to force refresh the running queries list, logs pagination, the ability to copy example queries, configurable time ranges and whole cluster schema stats, plus more operation controls for viewing, deleting and editing.

The to-do list also tells you what is not finished in a way a feature list would not. Read-only mode is still a to-do, so today the way to be safe is the read-only database user rather than a setting in the interface. Resilient operations are a to-do, so a Celery outage during an operation is a scenario the current code does not handle. And the EXPLAIN visualizer is a to-do, so the explain view in the screenshots is not yet a graphical plan renderer.

Contributions are welcome, with the caveat that you should open an issue first if you want to build a new feature.

MIT licensed, with releases that stopped in 2024

The licence is MIT, with the LICENSE file at the top of the repository, so there is no copyleft obligation on a deployment and no restriction on commercial use of the tool itself.

The release history is the number to look at. The tags are housewatch-0.1.9 from 2024-06-04 and housewatch-0.1.8 from 2024-05-27, with an earlier commit tagged with a full hash from 2023-10-04. The last push was on 2026-08-05 and the repository is not archived, so the code is being touched, but the published versions stopped in mid 2024 and the version numbers are still in the 0.1 range.

That combination, an active repository with a stagnant release line, is the reason to check what you are actually deploying. A container built from main is a different artefact from the 0.1.9 image, and the readme's own beta warning applies to the code as it stands rather than to a specific tag.

The project was created and is maintained by PostHog and by yakkomajuri, which means the questions to ask are practical ones about the ClickHouse introspection queries and the operations, since those are the parts that carry the operational risk, and about how a read-only connection behaves in your environment.

Editorial conclusion

Adopt HouseWatch if you run ClickHouse clusters and want the system table queries already written for you, particularly the query performance views, the disk usage breakdown and the operations runner with automatic rollback. Skip it if you need a hardened admin panel, since the readme warns that it is beta software, that it is not secured in any way, and that the code still needs cleaning up. Before you expose it, put it behind your own authentication, run it with a read-only ClickHouse user if you only need the analysis pages, and verify the version you are running, since the last published release is 0.1.9 from 2024-06-04 while the last push was on 2026-08-05.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. PostHog/HouseWatch on GitHub
4. README
5. Releases