Skyline: anomaly detection for a high volume of metrics without per-metric thresholds

Passive monitoring is the design goal, not threshold configuration

Skyline is described as a real time Python based anomaly detection, time series analysis and performance monitoring system, and the reason for that framing is in the same sentence: it is built to enable passive monitoring on metrics without the need to configure a model or thresholds for each one.

That is a specific and unglamorous problem. A team running a large estate does not have a handful of dashboards, it has thousands of series, and a static threshold per metric does not scale. A threshold that is right for a request counter at 3am is wrong for the same counter during a batch job, and nobody is going to hand-tune a thousand of them. Skyline's answer is to let the algorithms work out what it means for each metric to be anomalous, and then to let a user train it what is not anomalous on a per metric basis once it is running.

The intended environment is described as wherever there are a large quantity of high-resolution time series which need constant monitoring, and the repository description adds the multi-algorithm ensemble, the semi-supervised and unsupervised pattern recognition, the correlation analysis, and the claim that it learns and gets better over time. The description also says no GPU is required and that it can run on a VPS, which is a meaningful constraint for anyone who has considered a machine learning approach and backed away from the hardware cost.

One operational property is worth noting early. Once a metrics stream is set up, additional metrics are automatically added to Skyline for analysis, so there is no per-metric onboarding step for the metric itself.

Four metric sources, and the InfluxDB path goes through Telegraf

The ingestion side is short and specific. Skyline can ingest metrics from Graphite, InfluxDB via Telegraf, Prometheus and VictoriaMetrics.

The Telegraf detail is the one to understand rather than gloss over. Skyline does not talk to InfluxDB directly in that path; metrics are collected by Telegraf and delivered onward. That matters operationally, because it means the InfluxDB route depends on a Telegraf deployment being configured correctly, with its own output plugin and its own line protocol, before Skyline sees anything. The same applies to the Graphite route, which relies on the Graphite line protocol and its aggregation behaviour.

Prometheus and VictoriaMetrics are the paths that fit most naturally into a modern stack, and the requirements list contains prometheus-client at 0.22.1, along with the OpenTelemetry packages, including the API, the SDK, the semantic conventions and instrumentations for Flask, Redis, SQLAlchemy, PyMemcache and WSGI. There is also a Jaeger exporter set, so the tracing side is covered by OpenTelemetry with the Jaeger backend as the destination.

The topic tags on the repository match those names, which is a reasonable signal that the four ingestion paths are the supported ones rather than aspirations.

The dependency list shows what the detection stack actually is

The requirements file is long, and reading it tells you more about the internals than any architecture diagram would.

The statistical core is visible in numpy, scipy, pandas, statsmodels, statsforecast at 2.0.3, Prophet at 1.3.0, and the pattern-matching tools mass-ts for matrix profile work and stumpy for the same family of algorithms. scikit-learn at 1.8.0 covers the supervised parts, pywavelets handles signal decomposition, and kneed is the knee point detection library used for finding elbow points in sorted curves.

The infrastructure layer is just as visible. Redis at 7.4.0 with hiredis is the queue, Flask with Jinja2 and Werkzeug provides the web app, SQLAlchemy at 2.0.49 and mysql-connector-python cover relational storage, and pymemcache plus msgpack-python handle fast caching and serialisation. There is a long tail of integrations: pygerduty for PagerDuty, slack_sdk for chat notifications, boto3 for AWS, prometheus-client, PyJWT, gunicorn and gevent for serving, and gevent's presence alongside gunicorn suggests an async path for the high-throughput parts.

A few of these deserve a note. memray and numba are memory profiling and JIT compilation tools, which points to attention on the performance of the analysis path. bottleneck is a fast numeric library, and the presence of a croniter and the croniter dependency hints at scheduled jobs inside Skyline itself rather than leaving scheduling entirely to cron.

The requirements file also shows Python 3.8 as the declared target in setup.py's classifiers, alongside a 3.10 or later requirement, so the project has been running against a range of interpreter versions.

Packaging, structure and the shape of the tree

The repository layout suggests a long-lived system with several cooperating processes rather than a single script. There is a skyline/ package, a bin/ directory for entry points, utils/ for helper tooling, updates/ for the data and configuration refresh machinery, etc/ for configuration templates, docs/ for the documentation tree, examples/ with a data directory, and tests/.

The setup.py is a plain setuptools script. It declares the distribution name as skyline, the version as 4.0.0, the author as Abe Stanway and the maintainer as Gary Wilson, and the classifiers list Development Status :: 5 - Production/Stable, an intended audience of system administrators and developers, and the topic areas of system monitoring and information analysis. The licence field in setup.py reads MIT License, while the repository licence field on GitHub shows a custom licence that GitHub cannot classify, so the LICENSE file in the tree is the one to read before redistribution.

There are separate requirement files for development, documentation and something labelled bq-requirements, plus a requirements-torch-cpu.txt, which suggests an optional PyTorch-based analysis path with a CPU-only variant, and a ruff.toml for linting alongside setup.cfg.

The documentation is available online at http://earthgecko-skyline.readthedocs.io/en/latest/, and the same docs are viewable locally in a clone at file://<PATH_TO_YOUR_CLONE>/docs/_build/html/index.html, and through the docs tab of the Skyline Webapp frontend. Being able to read the documentation in the same web interface you use to view anomalies is a small but real quality of life detail for an on-call engineer.

Why the automatic approach is also the main limitation

The design decision that makes Skyline useful is the one that will generate false positives for you, and it is worth being blunt about.

An algorithm that infers what is anomalous for a metric it has never seen is inferring from the shape of the data, and a shape that genuinely is unusual is not necessarily a fault. A deploy that changes the request profile, a batch job that starts running twice as long, a cache that starts missing, a traffic shift to a new region: these all look like anomalies in a distribution. Skyline's answer to that is the training mechanism, where you tell it what is not anomalous on a per metric basis, and the correlation analysis, which is what lets it propose a common cause across several series rather than firing a page for each one independently.

That works, but it is a workflow rather than a switch. Someone has to look at the anomaly, decide, and teach the system, and the value only accrues if that loop is closed. On a team with no on-call ownership of the monitoring itself, the likely outcome is a large stream of anomalies that nobody reads, which is worse than no system because it produces noise that people learn to ignore.

The other limitation is the resource profile. The dependency list is heavy, and the described workload is a large quantity of high-resolution time series needing constant monitoring. That is a server workload, not a laptop workload. The repository description says no GPU is required and it can run on a VPS, which is a genuine advantage over a deep learning approach, but the requirements are pinned to exact versions, including numpy 2.4.4, pandas 3.0.2 and SQLAlchemy 2.0.49, so the install is not going to be flexible about what is already on the machine.

There is also a naming trap in the documentation itself. The readme image is labelled skyline.v4.1.0.png while setup.py declares the version as 4.0.0 and the tagged releases are v4.0.0, v3.0.4 and v3.0.3, so the version numbers in the docs, the packaging and the tags do not line up neatly. If you pin a version, pin a tag rather than trusting a version string in an image name.

The managed service and the alternative approaches

There is a related commercial path worth naming, because it changes the calculation for teams who want the detection without the operations. The README describes Anomify as a newer version of Skyline, built and managed by the team behind Skyline, with a new dashboard, a full spec API and an intuitive UI, currently offered as a free service for Skyline users at https://anomify.ai/skyline.

That is a genuine alternative to running the system yourself, with a different balance: you get the detection behaviour and a maintained interface, and you give up the control over the pipeline and the data.

The other alternatives are the ones most teams compare against. A hosted metrics product with built-in anomaly detection gives you detection without any infrastructure, but it constrains you to the data model that vendor supports. A hand-rolled threshold alerting setup, which is where a lot of teams actually start, is trivial to build and gives exact control over what pages, at the cost of not scaling past a handful of series and not learning anything.

Skyline sits in the space between those two. The difference in approach is that it is a self-hosted system designed around many high-resolution series with automatic per-metric judgement, rather than a notification configuration on top of a metrics store. That is a meaningful difference at a thousand series and a poor one at ten, which is the test to apply.

One more thing the README does not discuss, and that you should check for yourself, is how Skyline behaves when its own inputs stall. There is no mention of what happens if the Redis queue backs up, or how the system distinguishes a metric that has genuinely gone flat from a pipeline that has stopped delivering, and in an alerting system that failure mode is the one that will wake you up.

Editorial conclusion

Adopt Skyline when you run a large number of high-resolution metrics and cannot hand-tune thresholds for each one, since that per-metric configuration burden is the specific problem it targets. Do not adopt it for a handful of series or when you need a guarantee of catching a known failure mode, because the design trades explicit thresholds for automatic judgement. Before installing, read the docs at http://earthgecko-skyline.readthedocs.io/en/latest/, check the pinned dependency set in requirements.txt against your platform, and note that the last tagged release is v4.0.0 from 2023-06-28 even though the last push was on 2026-10-20.

Frequently asked questions

Official sources

1. earthgecko/skyline on GitHub
2. Issues
3. Project website
4. README
5. Releases