Gigapipe: A ClickHouse-based alternative for LGTM observability

Why Gigapipe targets the self-hosted LGTM gap

Gigapipe addresses the operational cost of assembling separate observability systems. Its stated scope covers logs, metrics, traces, and profiling in one stack, with interfaces compatible with Loki, Prometheus, Tempo, and Pyroscope. The intended users are teams operating Grafana or compatible clients on premises, especially organizations that want a self-controlled backend rather than a vendor-controlled observability service.

The project does not require applications to abandon OpenTelemetry instrumentation. Its documented OpenTelemetry integration accepts log, trace, and metric data, while native ingestion also covers Loki, Prometheus, Tempo or Zipkin, Pyroscope, Influx, Datadog, and Elastic. That combination matters when an organization has several generations of telemetry clients. A single query side can receive data from older protocols without first routing every producer through an OpenTelemetry Collector.

This is not merely a metrics database with Grafana panels attached. Gigapipe exposes query surfaces for LogQL, PromQL, TempoQL, and TraceQL, and the README says Grafana's Loki, Prometheus, and Tempo datasources can query it without a plugin. That compatibility is the main reason to evaluate the project over a conventional ClickHouse schema or a purpose-built single-signal backend. The tradeoff is that the project centralizes several API compatibility claims in one service, so one failed ingestion or query path can affect more than one observability signal.

Gigapipe ingestion and query paths

The documented architecture separates ingestion from exploration. Data can enter through OpenTelemetry, including OTLP metrics at `/v1/metrics`, profiles at `/v1development/profiles`, and an OTLP over gRPC receiver. Native protocol endpoints support clients associated with Loki, Prometheus, Tempo, Zipkin, and Pyroscope. Once accepted, the README states that ingested data can be queried through Gigapipe APIs using LogQL, PromQL, or TraceQL.

Log traffic follows Loki-oriented paths. The README documents `/loki/api/v1/index/stats` for stream count, entry count, byte volume, and chunk count, including filtered queries based on LogQL stream selectors. Its tail implementation accepts `limit` and `start`, while WebSocket responses include `dropped_entries` when backpressure causes data loss. Those details expose operational behavior that a simple API-compatibility statement would hide.

Metric querying uses Prometheus compatibility and is implemented through WASM according to the README. Trace querying follows Tempo and TraceQL interfaces, with documented metrics endpoints at `/api/metrics/query_range` and `/api/metrics/query` for calculations over trace data. This creates a useful bridge between signals, for example deriving latency or error measurements from traces, but it also means the query engine is part of the adoption boundary. API compatibility does not by itself establish identical storage behavior, retention controls, cardinality handling, or failure semantics with every upstream LGTM component. Those areas need validation against the project's documentation and the workloads they will carry.

Building Gigapipe from the repository

The repository exposes a Make target that builds a local Docker image tagged `gigapipe`. The deployment Dockerfile compiles `cmd/gigapipe/main.go` with Go 1.27.1 and can optionally include the View build through the `VIEW` build argument. The image declares port 3100, handles `SIGTERM`, and starts `/gigapipe`.

The following command performs the documented local image build. It uses the deployment Dockerfile under `scripts/deploy/docker`, not the repository-root Dockerfile. No container launch command is included because the README links to `docs/configuration.md` for Docker quickstart and environment variables rather than providing a complete run command.

make docker

After a successful build, the image should be available locally under the tag `gigapipe`. The next required step is configuration. The README links to `docs/configuration.md`, so it would be unsafe to invent environment variables, volume mounts, credentials, ClickHouse connection settings, or startup flags. Follow that document for the selected storage mode and deployment topology.

A first useful validation should use telemetry that already exists in the target environment. Confirm that OpenTelemetry reaches the documented OTLP endpoint, then query the same data through Gigapipe and the matching Grafana datasource. The expected result is not merely a successful container start: the application must receive telemetry and return a usable response from the signal-specific API.

Storage choices and Grafana integration

Storage control is Gigapipe's clearest architectural distinction from a managed Grafana stack. The README names ClickHouse, DuckDB, and GigAPI with S3 object storage as retention options. That does not mean the backends are interchangeable without qualification. ClickHouse is also central to the repository topics, while the separate table-ordering documentation discusses ClickHouse sort keys and `ADVANCED_SAMPLES_ORDERING`. Deployments already holding data may therefore have schema-ordering implications when changing configuration.

DuckDB and S3-backed GigAPI may appeal to operators who do not want every deployment to use the same database arrangement. The README does not state their performance characteristics, operational requirements, feature parity, or recommended production scenarios. It would therefore be a mistake to select one solely because it appears in a feature list. Check `docs/configuration.md` and the relevant backend documentation before deciding which storage mode fits the workload.

Grafana integration is based on native datasources. The README identifies Loki for logs, Prometheus for metrics, and Tempo for traces, with no plugin required. A Grafana deployment still needs correct datasource URLs and query behavior. The absence of a plugin reduces one integration layer, but it does not remove configuration or compatibility testing. Service graphs, TraceQL metrics, live-tail behavior, and profiling are separate features that should be checked rather than inferred from a successful panel query.

Where Gigapipe is the wrong operational choice

The clearest limitation is the absence of a complete deployment procedure in the README excerpt. Installation points to external documentation, and the project combines several storage backends, APIs, query languages, and optional components. A team that needs one prescribed, copy-and-run production topology will find more design and verification work here than it would with a narrowly scoped single-purpose service.

The WebSocket live-tail response also documents an explicit failure mode. `dropped_entries` appears when entries are dropped because of backpressure. That is useful for observability because the client can detect loss, but it means tailing is not a guaranteed lossless transport. Teams with strict log-tail delivery requirements must account for this boundary.

Another caution is compatibility scope. The project describes itself as drop-in compatible with several established systems and supports many ingestion protocols, but the README does not enumerate complete parity for every endpoint, option, or edge case. Query-language support should not be confused with feature parity in retention, alerting, recording rules, cardinality limits, tenancy, or operational tooling. The Makefile does expose a `QRYN_RULER_ENABLED` setting for end-to-end tests, but that internal setting is not evidence that every ruler feature is production-ready for a given deployment.

Profiles add another verification surface. The dedicated endpoint is `/v1development/profiles`, a development-oriented path documented for OTLP profile ingestion. Teams whose profile pipeline depends on a specific production protocol or exporter behavior should confirm that this endpoint and its downstream storage format meet their requirements.

Maintenance, release cadence, and AGPL-3.0

The repository is not archived, and its last push was on 2026-10-14. Recent releases include v5.5.3 on 2026-09-30, v5.5.2 on 2026-09-24, and v5.5.1 on 2026-09-17. That release sequence provides evidence of recent project activity, but it does not remove the need to read upgrade notes and configuration changes before moving between versions.

The maintenance cost extends beyond the Go binary. Operators need to manage telemetry protocol compatibility, Grafana datasource settings, storage schema, S3 or database access where selected, and any external systems used for collection and visualization. The project depends on a large set of Go modules, including ClickHouse clients, Prometheus packages, OpenTelemetry packages, gRPC, and Pyroscope libraries. Dependency and version changes are relevant during upgrades even when the application command remains `/gigapipe`.

Gigapipe is licensed under AGPL-3.0. That identifier matters to organizations reviewing open-source obligations, especially when the software is modified, networked, or distributed. This review does not provide legal advice. Teams should have their own compliance process review the AGPL-3.0 terms against their intended deployment and integration model before adoption.

Alternatives: separate LGTM services or a managed backend

The most direct alternative is to operate Loki, Prometheus, Tempo, and Pyroscope as separate services. That approach gives each system its own configuration, release cycle, storage design, and scaling boundary. It can also make failure domains easier to identify. The cost is additional deployment, maintenance, and cross-signal querying work. Grafana can still provide a common visualization layer, but the backends remain distinct.

Another alternative is a managed observability backend. That moves ingestion infrastructure, upgrades, and service operations to the provider, but changes the data-control and cost model. Gigapipe's distinction is self-hosted operation with explicit storage choices and compatibility with familiar Grafana datasource interfaces. It is most compelling for teams that value direct control over where telemetry is stored and are willing to validate compatibility across several signals.

The decision should not be reduced to feature count. Separate LGTM services are easier to reason about when one signal has distinct scaling or retention needs. A managed backend is easier to adopt when operational capacity is the main constraint. Gigapipe is the more relevant candidate when consolidating those services is a priority and the team can verify the storage, protocol, and query behavior before sending production traffic.

Editorial conclusion

Gigapipe is a reasonable candidate for engineers consolidating self-hosted logs, metrics, traces, and profiles without operating four separate services. It is a poor fit for teams that need a documented turnkey deployment, a non-AGPL licensing model, or verified parity with every Grafana LGTM feature. Before adoption, verify the documented configuration for the intended storage backend, send representative OpenTelemetry traffic through `/v1development/profiles`, and test the required Grafana datasource queries against the current release.

Frequently asked questions

Official sources

1. License: AGPL-3.0
2. metrico/gigapipe on GitHub
3. Project website
4. README
5. Releases