datacenter: a go-zero data platform with ten services behind one gateway

Ten services behind a single gateway on port 8887

datacenter is described in the repository as a data platform built on the go-zero framework, and it is a microservices project in the go-zero sense. Each capability is its own service with its own .api file and its own rpc directory, and the top level of the repository shows the full set: datasource, metadata, datasvc, catalog, quality, metric, tag, job, ide and ai, alongside the legacy user, common, votes, search and questions services.

The gateway is the entry point. The project structure table describes datacenter.go with the internal directory as the gateway on port 8887, responsible for route aggregation, JWT login verification, an anti-privilege-escalation middleware, and cross-service orchestration. Everything else talks to the gateway, and only the gateway is exposed.

That topology is the point of the design and it is visible in the compose file, which brings up four dependencies, MySQL, Redis, etcd and Elasticsearch, plus eleven services, and a scheduler process. The comments in the compose file are explicit that only the gateway's 8887 is reachable from outside and the rest communicate inside the container network.

The gateway's request pipeline for the data service API is the most detailed piece of documentation in the project, and it reads as a security chain rather than a feature list: HMAC signature verification, then application rate limiting, then authorisation checking, then parameter whitelisting, then result caching, then field masking on the way out.

The go.mod file shows the framework level. The module is datacenter, it declares Go 1.26.0, and the direct dependencies include the JWT library, protobuf and gRPC, the olivere elastic client, the Qiniu Go SDK, the wechat SDK, and the golang.org/x/time package. Everything else in the require block is indirect.

The documented workflow, from registering a data source to executing an API

The quick start section gives a three step path, and then a curl walkthrough that shows the actual API surface.

The three steps are a shell script called server.sh that starts the dependencies in Docker, a manual schema import into MySQL through the mysql client, and a restart script that compiles and restarts all the services. The dependencies are MySQL 8.4, Redis 8, etcd 3.5 and Elasticsearch 7.17, and the gateway listens on port 8887 once the services are up.

The curl examples then walk the whole lifecycle, and the order matters: register a data source, collect metadata, create an API, register an application, grant authorisation, publish, then execute.

The two calls in the middle of that chain are worth reading closely.

curl -XPOST http://127.0.0.1:8887/metadata/collect -d '{"ds_id":1}'

curl -XPOST http://127.0.0.1:8887/datasvc/app/register -d '{"name":"demo-app","rate_limit_qps":10}'

Metadata collection takes the data source id and, according to the documentation, reads information_schema from the target database to populate the metadata entities, which then feed the lineage registration and the versioned snapshots. Application registration returns an app_key and an app_secret, and the documentation notes that the secret is returned only once, which is the usual shape for an API credential.

The execute call signs the request, and the documented formula is given in the comment: sign equals hex of hmac-sha256 over app_secret and app_key plus ts, and the response carries rows_json, cost_ms and cache_hit. That last field is a useful detail for anyone evaluating this as a platform, because it means the cache layer is observable from the client rather than being invisible.

Ten capability groups, and which of them are actually finished

The repository documentation is organised in phases, and the phase structure is the clearest statement of what exists.

Phase 1 covers four services and is described as delivered. Data source management registers each type of data source, stores credentials with AES-GCM encryption, and supports connection probing with latency and error information. Metadata management collects database and table structures automatically from information_schema, registers and queries table level lineage, and keeps versioned metadata snapshots with multi-version comparison. The data service API uses a OneService model where an API is configured out and centrally governed, with the chain of signature, rate limit, authorisation, cache and masking described earlier. The asset catalogue handles asset classification, registration, and application with approval.

Phase 2 adds three. Data quality covers rule definition, task execution and result viewing. Data metrics covers atomic and derived metric definitions and their relationships, and the documentation notes that a metric can be referenced by the data service as an API. Data tags covers tag objects, tag definitions and tag values, with rule based grouping and profiling.

Phases 3 and 4 add three more, and these are the ones with caveats. Scheduling integration registers synchronous tasks and connects to the DolphinScheduler control plane, with real environment integration tracked as issue 21. The SQL IDE is online script development with version management. The AI querying service orchestrates natural language to metric through a semantic layer and logs the queries, with the LLM dialogue awaiting real environment integration, tracked as issue 23, and its table structure in the phase3 schema.

The project is also candid that it is mid-migration in a bigger sense. A note at the top of the open source section says the project is evolving from an event business platform into a true data platform, with the roadmap in docs/ROADMAP.md, and a separate note points at a commercial source sale for a Doris based data warehouse implementation with its own fourteen module list.

Two ways to run it, and the difference between them

The documentation gives two deployment routes, and they differ in more than convenience.

The recommended route is docker compose. There are three commands, and the comments describe the behaviour precisely.

FROM golang:1.26-alpine AS builder

RUN apk add --no-cache git ca-certificates

WORKDIR /src

ENV CGO_ENABLED=0 GOOS=linux

The documented commands are docker compose build, docker compose up -d and docker compose ps, with logs read through docker compose logs -f datacenter-api. The first build takes a while because the Dockerfile compiles twelve binaries in a builder stage, from the gateway through each of the ten RPC servers to the scheduler, and the runtime stage is a plain alpine image with no libc dependency. The builder stage is the following, and each service starts from the same image with a different command.

The behaviour that matters for a first run is the schema handling. On an empty database the initialisation SQL is executed automatically in filename order: sql.sql, then the phase1, phase2 and phase3 schema files, with no manual import needed. In the other route, that import is manual, which is the sequence below.

mysql -uroot -padmin
mysql > set names utf8
mysql > create database datacenter
mysql > use datacenter
mysql > source sql.sql
mysql > source docs/schema/phase1.sql
mysql > source docs/schema/phase2.sql
mysql > source docs/schema/phase3.sql

The second route is the local compile and start path, where the dependencies come up in Docker with server.sh and the services are built and started with restart.sh.

There is one gap to know about. The user, common, votes, search and questions legacy RPCs are not part of the compose orchestration, because etcd discovery is lazy and the gateway starts regardless. The documentation is clear that if you need endpoints such as /user/login you have to start those services separately using the second route.

Configuration files, Elasticsearch credentials, and the leftover business features

Each service has its own configuration file, and the documentation explains the setup pattern simply: each directory contains a corresponding rpc.example.yaml file, so you copy it and the setup is essentially done.

The list covers the gateway configuration in etc/datacenter-api.yaml and the RPC configurations for user, common, votes, search and questions, each under its own rpc/etc directory with a name matching the service.

The Elasticsearch credentials are baked into the server.sh script, which sets the password to admin, and the documentation tells you to put the account elastic and the password admin into search/rpc/etc/search.yaml. The same script prints a block of output for each dependency as it comes up, showing the container name and an identifier for mysql, redis, etcd and es.

The legacy features are worth understanding because they show what the project actually grew out of. The completed feature list includes WeChat official account login, Qiniu upload token retrieval, a voting feature with registration, a registration list and the vote itself, a question and answer lottery with activity information, the answer list, submitting an answer, getting a score, the draw, and filling in winner information, and search built on Elasticsearch. The unfinished items are WeChat and Alipay login and WeChat and Alipay payment.

So the origin is a WeChat based event platform with voting and a question draw, and the data platform is the second act. That history explains both the strength and the gap: the project has working go-zero service patterns, JWT login and cross-service orchestration, and it has the structure of a data platform, but the newest data services are the ones still marked as awaiting real environment integration.

Toolchain requirements, documentation language, and licensing

The environment requirements are stated at the top of the open source section, and they are stricter than the Go version alone suggests.

Go 1.24 or later is required, with go.mod declaring 1.26. The goctl tool at version 1.10.2 is needed only when regenerating code, and protoc 36.x with protoc-gen-go and protoc-gen-go-grpc are needed only when regenerating RPC definitions. The Dockerfile uses golang:1.26-alpine as the builder, so the containerised build is the path with the least friction on the version question.

A full stack upgrade is announced for October 2026, combining Go 1.26, go-zero v1.10.3, MySQL 8.4, Redis 8, etcd 3.5 and Elasticsearch 7.17, with the details in docs/UPGRADE.md. The Go Report style go.mod matches that, with olivere/elastic v7 for the 7.17 line and the go-redis v8 client for Redis 8.

The documentation language is a practical factor. The README, the roadmap, the upgrade notes, the phase contract and the deployment manual are all in Chinese, and the inline comments in the shell scripts are Chinese as well. A team that does not read Chinese will be working from the directory structure and the code rather than from the guidance.

The licensing is the one item a reader cannot resolve from the repository. The licence recorded for the repository is unknown, and there is no LICENSE file in the top level directory listing. The repository also contains an AI section in its README offering a commercial source sale for a Doris based implementation, with a contact path for a demonstration environment and a solution document.

That is not a licence grant, and a team considering the code for anything beyond evaluation needs to establish the terms directly with the author. The last push was on 2026-10-12, and the single release is v2.0.0, tagged as the official data platform version on 2026-10-10, so the code is new enough that the licence question is live rather than settled.

Editorial conclusion

datacenter fits a team that wants a running multi-service data platform to study and modify, since the compose file brings up eleven services and four dependencies with the schema applied automatically, and it fits a Chinese speaking team that reads its documentation. It does not fit a team that wants a finished data platform with an active community behind it, since the licence listing on the repository is unknown, several services are marked as awaiting real environment integration, and the five legacy RPCs are not part of the compose orchestration. Before running it, read docs/PHASE1.md for the service ports and schema contract, and check the two referenced integration issues, because the scheduling and natural language query services are the two the documentation describes as still being wired up against a real environment.

Frequently asked questions

Official sources

1. Issues
2. jackluo2012/datacenter on GitHub
3. README
4. Releases