ClearML: An Open-Source MLOps Suite for Experiment Tracking, Orchestration, and Model Serving

The five-module structure of the ClearML suite

ClearML is organized into five primary components, each addressing a different phase of the ML development lifecycle. The Experiment Manager handles automatic tracking of training runs. The MLOps component, implemented as the separate clearml-agent package, provides an orchestration and automation layer for running jobs on Kubernetes, cloud instances, or bare-metal machines. Data Management delivers a version-controlled dataset layer built on top of object storage services including S3, Google Cloud Storage, Azure Blob, and NAS. Model Serving, implemented as the separate clearml-serving package, provides scalable model endpoint deployment with Nvidia-Triton for optimized GPU inference and built-in model monitoring. Reports enable the creation of shareable Markdown documents with embedded content.

Beyond these five, the README lists an Orchestration Dashboard for monitoring compute clusters across cloud, Kubernetes, and on-premises deployments, and a Fractional GPUs feature via the clearml-fractional-gpu package, which uses container-level, driver-level GPU memory limitation.

All five components share the ClearML server as the backend. The server can be self-hosted following the clearml-server deployment guide or used through the free-tier hosted service at app.clear.ml. The README states that sign-up and first use with the hosted service takes under two minutes.

The Experiment Manager: automatic capture without instrumentation overhead

The Experiment Manager is the part of ClearML that most teams encounter first. The README states that adding only two lines to existing training code triggers automatic experiment capture. Without rewriting the training loop, ClearML records the complete experiment setup: full source control information including non-committed local changes, the execution environment down to individual package versions, and all hyper-parameters. It captures hyper-parameters from argparse, Click, PythonFire, Hydra, explicit parameter dictionaries, and TensorFlow Defines.

Output capture is also automatic. ClearML captures stdout and stderr, resource monitoring data including CPU and GPU utilization, temperature, I/O, and network activity, model snapshots with optional upload to central storage, and artifacts such as intermediate files. For visualization, it integrates with TensorBoard and TensorboardX, capturing scalars, metrics, histograms, images, audio, and video samples. Matplotlib and Seaborn plots are captured as well.

Framework support spans PyTorch, including the Ignite and Lightning integrations, TensorFlow, Keras, AutoKeras, FastAI, XGBoost, LightGBM, MegEngine, and Scikit-Learn. The README also mentions Jupyter Notebook integration and a PyCharm remote debugging plugin. The coverage is broad, but engineers working outside these frameworks will need to use the ClearML Logger API for manual logging.

Installing ClearML and connecting to a backend

The package is available on PyPI under the name `clearml` and on Anaconda under the clearml channel:

pip install clearml

The setup.py classifies the package as `Development Status :: 5 - Production/Stable` and targets Python versions from 2.7 through current releases, though the requirements.txt shows NumPy version pins that imply practical support starts at Python 3.7. The Apache-2.0 license covers the client library; running the server components introduces their own deployment requirements.

After installing the package, the next step is connecting to a ClearML server. The README presents two paths: sign up at app.clear.ml for the free-tier hosted service, or deploy clearml-server on your own infrastructure following the self-hosting documentation at clear.ml/docs. The hosted service provides the backend without any server administration. Self-hosting requires additional setup that the clearml-server repository documents separately.

The README provides three tutorial notebooks covering experiment management, remote execution agent setup, and running tasks remotely. These are available both in the repository at docs/tutorials/ and on Google Colab.

Remote execution with clearml-agent and the orchestration model

clearml-agent is the component that turns ClearML from an experiment tracking tool into a full orchestration layer. An agent polls a queue for tasks and executes them on the machine where the agent is running. This means a developer can clone a previous experiment from the ClearML web interface, change its hyper-parameters, and send it to a GPU machine or a Kubernetes cluster without SSH access or manual job submission.

Agents support Kubernetes, cloud environments, and bare-metal machines. A compute cluster with agents becomes a managed execution environment where job queues replace manual resource allocation. The autoscaler component, visible in the web interface under Workers and Queues, handles scaling agents based on queue depth. This model differs from a typical job scheduler because the task definition, including the code commit and the dependency list, is captured by the Experiment Manager at run time and is reproduced by the agent at execution time. The practical limitation is that agents require network access to the ClearML server and object storage backend.

Data management: differentiable versioning over object storage

The Data Management component provides version-controlled datasets stored in object storage. The README describes it as a "fully differentiable data management and version control solution" built over S3, Google Cloud Storage, Azure Blob, and NAS. Engineers register datasets as versioned artifacts that the Experiment Manager can reference by version ID, so a training run and its dataset are linked in the experiment record.

This approach has a specific limitation. The data versioning system stores references and diffs rather than full copies, but the storage backend still needs to be accessible to every machine that runs experiments against the dataset. A team without shared object storage across training machines cannot use this feature as documented. The system is not a replacement for a data lake or a feature store; it tracks what data was used in each experiment and where it lives, not the data itself.

ClearML versus Weights and Biases: what the open-source model changes

Weights and Biases is a cloud-hosted experiment tracking and visualization service commonly compared to ClearML. The fundamental difference is the operating model. Weights and Biases is a SaaS product with a self-hosted enterprise offering; ClearML is open-source first, with the client library under Apache-2.0 and a free-tier hosted backend at app.clear.ml. Teams with data residency requirements can run the full ClearML stack on private infrastructure without a commercial license; the same is not true for Weights and Biases at the free tier.

ClearML also bundles more components in one package. The suite covers orchestration, data versioning, and model serving in addition to experiment tracking. Weights and Biases focuses primarily on the experiment tracking and visualization layer. The tradeoff is operational complexity: running ClearML self-hosted requires maintaining the server, object storage, and agent infrastructure. Teams that want experiment tracking with minimal operational overhead and are comfortable with a SaaS backend may find Weights and Biases a simpler starting point. Teams that need the full orchestration and data pipeline integration or that require on-premises deployment should look at ClearML's suite.

Editorial conclusion

ClearML is a practical choice for teams that want a single open-source package covering experiment tracking, remote execution, dataset versioning, and model serving without assembling multiple separate tools. It fits best when the team can operate the ClearML server themselves or accepts the free-tier hosted service, and when the target frameworks are among the supported ones including PyTorch, TensorFlow, Keras, XGBoost, LightGBM, and Scikit-Learn. It is a poor fit for teams that need a lightweight observability add-on only: the full suite introduces PostgreSQL, Redis, object storage, and the ClearML server as operational dependencies. Before adopting it, verify that the free-tier hosted service meets your data residency requirements, or plan to deploy clearml-server on your own infrastructure.

Frequently asked questions

Official sources

1. clearml/clearml on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases