DeepADoTS: A Benchmarking Pipeline for Deep Time Series Anomaly Detection

The Gap DeepADoTS Fills: Seven Methods, One Interface

Comparing deep anomaly detection methods on time series is normally an exercise in reimplementation. Each paper ships its own training loop, its own scoring convention, and its own idea of what counts as an anomaly. DeepADoTS addresses that by collecting seven published methods in one repository: LSTM-AD from ESANN 2015, LSTM-ED from ICML 2016, the Autoencoder from DaWaK 2002, Donut from WWW 2018, REBM from ICML 2016, DAGMM from ICLR 2018, and LSTM-DAGMM, which the README describes as an extension of DAGMM using an LSTM autoencoder in place of a neural network autoencoder. The intended audience is narrow: researchers and engineers who need to run several of these methods against the same series and compare the numbers, not teams looking for a production detector. The README states the goal directly: it exists to provide a benchmarking pipeline for anomaly detection on time series data for multiple state-of-the-art deep learning methods.

How Scoring Works: fit, predict, and a Higher-Is-More-Anomalous Convention

The API follows the scikit-learn convention. The README says the project offers the interface methods fit(X) and predict(X). fit estimates the data distribution in an unsupervised way, so no labels are consumed during training. predict returns an anomaly score per instance, and the README fixes the direction of that score: the higher it is, the more certain the model is that the instance is an anomaly. That single convention is what makes cross-method comparison possible, because a method that scored the other way around would invert every downstream metric. Performance is compared with ROC AUC, using sklearn.metrics.roc_auc_score. The data side is handled by a Dataset base class in src.datasets, which subclasses extend. In the README example, a MNIST subclass calls super().__init__ with name="MNIST" and file_name='', then implements load() to populate self._data with a tuple of four pandas DataFrames. The comment in that constructor explains the empty file name: no data is loaded from a file. So the contract is a four-part tuple, and the example orders it as training features, training labels, test features, test labels.

The README Demo Uses MNIST, and Its Author Says So

The example is deliberately not a time series. The README acknowledges this in parentheses: it uses MNIST because the data is already available in TensorFlow and requires no external download, even though the data has no temporal aspect. That is an honest disclosure, but it also means the demo does not exercise the sequence handling that the LSTM-based methods depend on. The example constructs an AutoEncoder with sequence_length=1, num_epochs=40, hidden_size=10, and lr=1e-4, trains on 1000 instances, predicts on 100, and prints the ROC AUC, with 0.8614 given as an example value. The visualization snippet normalizes the error to [0, 1] and plots each test image at coordinates (anomaly score, random y), with the README noting that global outliers (zeros) and local outliers (strangely written digits) receive high anomaly scores. Read that as a sanity check on the scoring direction, not as evidence about temporal anomaly detection. If you want to know how LSTM-AD or Donut behave on seasonal data, the README does not tell you, and you would have to supply your own Dataset subclass to find out.

Getting It Running: Clone, venv, pip, main.py

The usage block is short and leaves little room for interpretation. Clone with git clone git://github.com/KDD-OpenSource/DeepADoTS.git, create a virtualenv with virtualenv venv -p /usr/bin/python3, activate it with source venv/bin/activate, install with pip install -r requirements.txt, then run python3 main.py. Note that main.py is the pipeline entry point, not the README example; the example is a standalone script you would assemble yourself. There is a Docker path as well: docker build -t deep-adots . followed by docker run -ti deep-adots /bin/bash -c "python3.6 /repo/main.py". That command pins Python 3.6 inside the container, while the virtualenv instructions point at /usr/bin/python3 without a version. The two paths may not resolve to the same interpreter, and the repository topics list both pytorch and tensorflow, so the dependency set is not obviously single-framework. The README does not state which framework each of the seven methods uses, and I cannot confirm that from the supplied material. Treat requirements.txt as the authoritative list before you assume a lightweight install.

No Releases, No Version Pin, and a CI Badge That Does Not Cover Your Setup

The repository has no releases. There is nothing to pin against, no changelog to read before upgrading, and no semantic version to signal whether a commit breaks the fit/predict contract. If you vendor this code, you are tracking master. The README carries a CircleCI badge, which tells you some configuration is exercised on every push, but the README does not say which Python version, which framework, or which of the seven algorithms the CI actually runs. A green badge on a pipeline that trains one small model is not the same as coverage of Donut or REBM. The practical consequence is that a working install on your machine is something you establish yourself, and once established you should record the exact commit hash, because the next push to master gives you no version boundary to reason about. The MIT licence is permissive and places few constraints on reuse, but it also means no warranty accompanies the code. That is a statement about the licence text, not legal advice; check it against your own obligations.

Where DeepADoTS Is the Wrong Tool

This is a benchmarking pipeline, and the README never claims otherwise. It has no serving layer, no streaming interface, no model persistence story, and no drift handling. The training loop is unsupervised by design, so if you have labelled anomalies and want to use them during training, the fit(X) signature gives you nowhere to put them; labels only enter at evaluation time through ROC AUC. The Dataset abstraction assumes you can produce four DataFrames in memory, which is workable for benchmark-sized series and awkward for long histories you would rather window lazily. There is also a subtler mismatch: the README's demonstration is a static dataset with sequence_length=1, while the method table is dominated by recurrent and sequence-aware models. Nothing in the supplied material shows a documented end-to-end time series run, so the path from a raw CSV of timestamps and values to a scored series is something you construct from the Dataset base class yourself. If you need a detector running tonight against a live feed, this repository is the wrong starting point.

The Alternative: A Single Maintained Detector Library

The obvious alternative is to pick one library that implements one family of detectors well and maintain it, rather than a benchmark harness that implements seven. The difference in approach matters. DeepADoTS optimizes for comparability: a shared Dataset contract, a shared fit/predict signature, a shared ROC AUC metric, and a shared main.py that runs the comparison. A single-method library optimizes for the operational path around one model: persistence, configuration, and an interface you can call from a service. You cannot get both from the same repository, because the benchmarking pipeline's value comes precisely from holding the methods at arm's length behind a thin interface, and that thinness is what makes it a poor production dependency. If your question is which method to use, DeepADoTS is the better fit. If your question is how to run the method you already chose, it is not.

Maintenance and Upgrade Cost in Practice

The repository was pushed recently and is not archived, so the code is not abandoned. But with no releases, the upgrade unit is a commit, and the upgrade cost is whatever changed between your recorded commit and the new head of master. The pinning problem is real: requirements.txt is the only dependency specification the README mentions, and it does not distinguish the TensorFlow methods from the PyTorch ones even though both appear in the repository topics. If you depend on DeepADoTS, the cheapest defensible practice is to record the commit you validated, keep your own Dataset subclasses in a separate directory so a rebase does not touch them, and re-run python3 main.py after any pull to confirm the pipeline still completes. The MIT licence permits that kind of vendoring. It does not supply anyone to ask when the pipeline breaks.

Editorial conclusion

Adopt DeepADoTS if you are evaluating deep anomaly detection methods for time series and want a shared fit/predict interface, a ROC AUC comparison, and a Docker path to run the pipeline. Avoid it if you need a maintained, versioned library for production scoring, or if your data is not already prepared as the Dataset subclasses expect. Before committing, verify that main.py and requirements.txt install cleanly on your Python version, decide whether you want TensorFlow or PyTorch (the topics list both), and check the CI configuration to see which combination is actually exercised.

Official sources

1. Issues
2. KDD-OpenSource/DeepADoTS on GitHub
3. License: MIT
4. README