RelayPulse: LLM Relay Availability Monitoring That Spends Real Tokens

The false-alive problem RelayPulse was built to catch

Uptime Kuma and similar tools answer one question: did the HTTP request complete. In the LLM relay market, that answer is close to useless. The README states the failure mode plainly: an endpoint can return HTTP 200 while the body carries an empty completion or an error code. A relay that has run out of upstream credit, or whose key pool has been rotated badly, still passes a connectivity check.

RelayPulse targets the people who resell or resell-adjacent consume those relays: teams that buy LLM access through a middleman and need to know when the middleman degrades. The README lists three scenarios: tracking a self-hosted or purchased relay over time, comparing several LLM providers on latency and error rate, and watching an external API dependency so that a silent failure does not become an incident. The audience is narrow and operational. This is not a tool for someone who just wants a status page for their own web app.

Probing with max_tokens 1 and validating the response body

The mechanism is a scheduled API call that costs money. The README puts the probe cost at roughly 20 input tokens plus 1 output token per check, with max_tokens set to 1. At the default interval of one minute, that works out to about 30,000 tokens per day per monitored service. The README frames this as "极低" cost, and for a handful of services that is defensible. Scale to fifty relays and you are paying for 1.5 million tokens a day just to watch them.

A monitor entry names a provider, a service, a category, a base_url, and a template. The template supplies the model and request_model, so the probe prompt is not written inline in config.yaml. Templates live in the templates/ directory, and the Dockerfile notes that template JSON is baked into the image at /app/templates and symlinked to /config/templates at startup. That is the data flow: scheduler reads config.yaml, resolves a template, calls the provider, validates the content, writes a result row, and the API layer aggregates rows into the matrix the dashboard renders.

Configuration reloads through fsnotify, so editing config.yaml does not require a restart. The storage layer is either modernc.org/sqlite for single-node deployments or pgx-backed PostgreSQL for multi-replica setups. Both are real dependencies in go.mod, not aspirational roadmap items.

Installing RelayPulse with Docker Compose

The README's recommended path is Docker Compose against a prebuilt image, and it is genuinely short. The compose file pulls ghcr.io/prehisle/relay-pulse:latest, maps port 8080, and mounts ./config and a named volume at /data. Start by fetching the compose file and the example config:

curl -O https://raw.githubusercontent.com/prehisle/relay-pulse/main/docker-compose.yaml
curl -O https://raw.githubusercontent.com/prehisle/relay-pulse/main/config.yaml.example
mkdir -p config && cp config.yaml.example config/config.yaml

Then edit config/config.yaml. A minimal monitor block looks like this, with the template supplying the model so you do not set it here:

interval: "1m"
slow_latency: "5s"
monitors:
  - provider: "88code"
    service: "cc"
    category: "commercial"
    base_url: "https://api.88code.com"
    template: "cc-haiku-arith"
    api_key: "sk-xxx"

The README recommends keeping the key out of the file and passing it as an environment variable instead, using the pattern MONITOR_<PROVIDER>_<SERVICE>_API_KEY with the provider and service uppercased and hyphens replaced by underscores. Bring the stack up and open the dashboard:

docker compose up -d
curl http://localhost:8080/health
curl http://localhost:8080/api/status

The health endpoint should return a success response, and /api/status should return the 24-hour window. If you are building from source instead, run ./scripts/setup-dev.sh first. The README warns that skipping it produces a build error, pattern frontend/dist: no matching files found, because the script copies frontend/dist into internal/api/frontend/dist for the go:embed directive, and that path does not accept symlinks.

Sliding windows and the sampling you have to do yourself

The API section documents a design choice that will surprise anyone wiring RelayPulse into a fixed-schedule report. Periods are sliding windows: period=24h means the 24 hours counting back from the moment of the request. The README is explicit that bucket boundaries shift with each request and that provider rankings always reflect the most recent 24 hours. For a human looking at a dashboard, this is the right behavior. For an integration that compares yesterday's numbers to today's, it is not, and the README's own suggestion is to sample at a fixed cadence such as the top of each hour.

There is a second subtlety in the status query API. The board field returned by /api/status is per monitor item and can be hot, secondary, or cold. The board field returned by /api/status/query is collapsed at the channel level and is binary: hot or cold. A channel is hot if any non-disabled item under it is hot or secondary. The README calls this out directly, which is good, but it means code written against one endpoint will misread the other if nobody reads the note. The query endpoint accepts up to 20 groups via repeated q parameters and up to 50 via POST to /api/status/batch.

Where RelayPulse is the wrong instrument

The probe model is the limitation. Every check consumes tokens against the very service you are measuring, so a relay that is failing may still be billing you for the failed probes, and a relay that is rate-limiting you will look worse than it is because your own monitoring traffic is competing for quota. A provider with a low request ceiling can be pushed into throttling by a one-minute interval, which turns the monitor into part of the problem.

The README's disclaimer is also worth taking literally. It states that displayed status and trends come from automated technical probes and may be affected by network variation, regional differences, and cache delay, and that results are not a judgement on any service's compliance, legality, or commercial standing. If you need a contractual uptime figure, this is not it. It is an observation tool, and the methodology document is where the error characteristics are supposed to be described.

Finally, the project is self-hosted by design. The README states that configuration and keys stay local and that monitoring data is not sent back. That is a privacy property, and it is also an operational burden: you own the database, the backups, and the uptime of the thing that watches uptime.

RelayPulse against a generic uptime monitor

The obvious alternative is Uptime Kuma, which the README names as the comparison point. The difference is what counts as success. Uptime Kuma asserts on transport and status code, which is cheap, requires no credentials, and works for any HTTP target. RelayPulse asserts on the content of a completion, which requires a real API key, costs tokens per check, and only works for LLM-shaped endpoints.

That trade is not close. If your target is a web application, RelayPulse has nothing to offer and Uptime Kuma will do more for less. If your target is an LLM relay, Uptime Kuma will report green through exactly the outage you care about. The second-order difference is data shape: RelayPulse stores per-check latency and error classification and renders 24h, 7d, and 30d availability heatmaps, because the question it answers is about service quality over time rather than up versus down at this instant. A generic monitor can be made to do a content assertion with a custom script, but you would be building the aggregation, the templates, and the dashboard yourself.

Maintenance, releases, and what the MIT licence leaves you

The repository is not archived, and the last push was on 2026-09-10. Three releases landed that day: v2.89.0, v2.88.0, and v2.87.0. That release cadence is the practical upgrade consideration. Pinning ghcr.io/prehisle/relay-pulse:latest means you receive whatever shipped most recently on the next docker compose pull, and given three tags in a single day, the version number is not a stability signal. Pin a digest or a specific tag if you want upgrades to be a decision rather than a side effect.

The upgrade path itself is not documented in the README. There is no rollback procedure and no migration note for the SQLite or PostgreSQL schema, so the safe reading is that a downgrade is untested. Back up the /data volume before pulling. The Go toolchain requirement is 1.25.0 in go.mod, and the Dockerfile builds the backend on golang:1.27-alpine, so source builds need a recent Go.

The MIT licence is permissive and imposes no obligation on what you do with your own deployment. It also carries no warranty, which matters more than usual here: the disclaimer states the author is not responsible for third-party providers shown on sites built with this software, including relaypulse.top. If you run RelayPulse publicly against named providers, the accuracy and framing of what you publish is your problem, not the project's.

Editorial conclusion

Adopt RelayPulse if you buy or resell LLM relay capacity and need to know whether a provider's endpoint is genuinely answering, not merely returning 200. Skip it if you cannot fund token spend per check or need a hosted service with no local data. Before committing, read docs/user/methodology.md to understand how a probe is judged, and confirm that your config.yaml monitor entries map to keys you can rotate through MONITOR_<PROVIDER>_<SERVICE>_API_KEY.

Frequently asked questions

Official sources

1. License: MIT
2. prehisle/relay-pulse on GitHub
3. Project website
4. README
5. Releases