# XManager: packaging, running and tracking ML experiments from one API > XManager is a DeepMind Python framework for packaging, running and keeping track of machine learning experiments, locally or on Google Cloud, through launch scripts. It is aimed at research teams who want experiment bookkeeping and job dispatch in one library rather than a pile of scripts and a spreadsheet. **google-deepmind/xmanager** — A platform for managing machine learning experiments - Repository: https://github.com/google-deepmind/xmanager - Stars: 925 · Forks: 72 - Language: Python - License: Apache-2.0 - Published: 2026-10-20 · Updated: 2026-10-20 · Language: en - Canonical page: https://hysenlabs.com/projects/google-deepmind-xmanager ## One API across a local laptop and Vertex AI XManager is described as a platform for packaging, running and keeping track of machine learning experiments, and the project description calls it a platform for managing machine learning experiments. The README is precise about what that means: you can launch experiments locally or on Google Cloud Platform, and interaction with experiments happens through XManager's APIs via Python launch scripts. The interesting property is that the same code path serves both. The snippet in the README makes the point explicit: the local backend means all executables are sent for execution from this machine, independently of whether they are actually executed on that machine or on GCP. A launch script therefore describes what to run and where, not how to reach the machine that runs it. That is the difference from a typical training script that hardcodes its environment. Here the experiment is a first class object with an identity, the metadata is written to a database when the experiment is created, and it can be listed later, for example through `xm_local.list_experiments`. The README points to a set of slides for a more detailed introduction, alongside a Colab notebook and a Jupyter notebook codelab. The repository is a client library. The README states plainly that it only contains the client code, and not the server and UI code required for full integration, and that the private preview of XManager on Cloud, XMC, is reachable by contacting a GCP account team. ## An experiment context that persists its metadata The first thing a launch script does is open an experiment context, and the mechanism is a context manager: ```python # Contains core primitives and APIs. from xmanager import xm # Implementation of those core concepts for what we call 'the local backend', # which means all executables are sent for execution from this machine, # independently of whether they are actually executed on our machine or on GCP. from xmanager import xm_local # # Creates an experiment context and saves its metadata to the database, which we # can reuse later via `xm_local.list_experiments`, for example. Note that # `experiment` has tracking properties such as `id`. with xm_local.create_experiment(experiment_title='cifar10') as experiment: ``` Three things are visible here. The library is split into core primitives in `xm` and backend implementations in `xm_local`. The context manager creates the experiment and saves its metadata to the database at creation time, not at the end. And the experiment carries tracking properties such as an `id`, which is what makes later retrieval possible. The metadata being written on entry rather than on exit is the detail that matters for reliability. A run that crashes or is killed halfway still leaves a record that it existed, which is exactly the case where a spreadsheet based workflow tends to lose information. The database itself is not detailed in the README excerpt, but the packaging configuration lists `alembic` as a dependency, which is the schema migration tool, and package data includes `*.sql` files, so the metadata store is SQL backed and versioned. ## Package first, then add jobs to the experiment The API has two steps: packaging, and adding execution units. Packaging turns an executable specification into something that can be run, using a concrete executor specification. ```python [executable] = experiment.package([ xm.binary( # What we are going to run. path='/home/user/project/a.out', # Where we are going to run it. executor_spec=xm_local.Local.Spec(), ) ]) ``` The README explains what packaging involves depending on the combination of executable and executor. A `xm.python_container` designed to run on Kubernetes will be built via `docker build` and the new image uploaded to the container registry. For a simple prebuilt Linux binary designed to run locally, only some validations are performed, for example that the file exists. The `executable` that comes back contains everything needed to launch the packaged blob through `add`: ```python experiment.add(xm.Job( # The `a.out` we packaged earlier. executable=executable, # We are using the default settings here, but executors have plenty of # arguments available to control execution. executor=xm_local.Local(), # Time to pass the batch size as a command-line argument! args={'batch_size': batch_size}, # We can also pass environment variables. env_vars={'HEAPPROFILE': '/tmp/a_out.hprof'}, )) ``` An `add` call creates an experiment unit, usually a collection of semantically united jobs, and sends it for execution. Job groups exist for the cases where a collection is the right unit rather than a single job. One behaviour is worth knowing before you rely on it. The context waits for locally run things, but not for remote things such as jobs sent to GCP, although those can be awaited explicitly with `wait_for_completion`. A script that ends immediately after `add` will not wait for cloud jobs to finish. ## The examples folder is the feature catalogue The repository carries an `examples/` directory that maps out what the framework can target, and it is the best documentation the README points to beyond the external slides. The examples are named after the target rather than the technique. There are `cifar10_tensorflow`, `cifar10_torch` and `cifar10_torch_xla` as the base cases, then a cluster of Kubernetes variants of the same task: `cifar10_tensorflow_k8s_multiworker`, `cifar10_tensorflow_k8s_ps` for parameter server style training, `cifar10_tensorflow_k8s_tensorboard` and `cifar10_tensorflow_k8s_tpu`. The local ones are `local_arg_printer`, `local_container_gpu`, `local_container_links` and `local_container`, plus a plain `dockerfile` example. Two names stand out as being about more than training: `parameter_controller`, which is about tuning hyperparameters programmatically, and `vizier`, which points at Vizier, a Google service for hyperparameter and black box optimisation. There is also `dopamine`, which suggests an example built on Dopamine, a reinforcement learning research codebase, and `cifar10_torch_xla` for XLA compilation of a PyTorch model. The spread tells you the design. The same experiment object, the same package and add calls, with the difference being which executor spec is used, which is why the local and cloud variants of the same CIFAR-10 task exist as separate directories rather than one directory with a flag. ## Installing the client and the Debian setup script Installation is a pip install, and there are two routes: ```bash pip install git+https://github.com/deepmind/xmanager.git ``` or the PyPI project: ```bash pip install xmanager ``` For Debian based systems, the README describes a different path, cloning the repository and running a setup script that installs XManager and all its dependencies: ```sh cd xmanager/setup_scripts && chmod +x setup_all.sh && . ./setup_all.sh ``` The codebase assumes Python 3.10 or newer, and the packaging configuration agrees, with `python_requires='>=3.10'`. The distribution is named `xmanager` at version 0.8.3, it installs a console script named `xmanager` bound to `xmanager.cli.cli:entrypoint`, and it conditionally installs a second command, `xdash`, if an `xdash` directory is present in the source tree. The dependency list in setup.py is long and tells you where the framework touches. Alongside `absl-py`, `attrs`, `async_generator` and `etils[epath]`, there is `alembic` for schema migrations, `docker` for the container executor, `GitPython`, `cloud-sql-python-connector`, and the Google client stack including `google-auth`, `google-api-python-client`, `google-cloud-aiplatform` and `google-api-core`. Two optional pieces of tooling are tied to specific executors. Using `xmanager.xm.PythonDocker` requires Docker, with the README noting that Linux users should follow the steps to enable sudoless Docker. Using `xmanager.xm_local.BazelContainer` or `xmanager.xm_local.BazelBinary` requires Bazel. Optional prerequisites are therefore tied to the backend rather than to the framework as a whole. ## Vertex AI runs need a project, a bucket and three APIs The GCP path has the longest setup, and the README walks through it step by step. It applies if you use `xm_local.Vertex`, the Vertex AI executor, since you need a GCP project to access Vertex AI to run jobs. You create the project, install `gcloud`, and associate your Google account with the project: ```bash export GCP_PROJECT= gcloud auth login gcloud auth application-default login gcloud config set project $GCP_PROJECT ``` Then you let gcloud work with Docker: ```bash gcloud auth configure-docker ``` Three APIs have to be enabled: IAM, the Cloud AI Platform API, and Container Registry. The container registry matters because the Kubernetes executor builds and pushes images, which is the packaging step from the previous section. Finally you need a staging bucket in us-central1, which the README says is used to save experiment artifacts such as TensorFlow log files that can be read by TensorBoard, and may also be used to stage files to build a Docker image if you build images remotely: ```bash export GOOGLE_CLOUD_BUCKET_NAME= gcloud storage buckets create --location=us-central1 gs://$GOOGLE_CLOUD_BUCKET_NAME ``` The variable is then added to the environment or to your `.bashrc`. The location is fixed to us-central1 in the documented command, which is worth knowing before you plan where the bucket lives. ## Apache-2.0 terms, frequent patch releases, and the missing server The licence field on the repository says Apache-2.0, and setup.py carries the Apache 2.0 header with a copyright line naming DeepMind Technologies Limited and the year 2021. Apache-2.0 is permissive and includes an explicit patent grant, which for a Google project used inside a larger research codebase is the straightforward arrangement. The release history is active and recent. The tags are v0.8.3 on 2026-10-15, v0.8.2 on 2026-09-18 and v0.8.1 on 2026-09-17, with the last push on 2026-10-15, the same day as the current tag. Three releases in four weeks suggests a project being fixed and released as needed rather than on a schedule. The real limitation is the one the README states at the top. This repository contains only the client code, and not the server or UI code required for full integration. The hosted experience, XManager on Cloud, is in private preview and requires contacting a GCP account team or the given email address. That splits the audience. The library itself is usable today, and the launch script API is the whole of what a local or Vertex based workflow needs. But if your requirement is a web interface for browsing experiments across a team, that interface is not in this repository and not open source here, so the client-only parts are what you can actually adopt. ## Conclusion Adopt XManager when your team runs many machine learning jobs across a local machine and Google Cloud and needs a single API that records what was run, with executors for local binaries, containers, Kubernetes, TPU and Vertex AI. Skip it if you need the hosted server and UI, because the README states this repository contains only the client code, not the server or UI required for full integration. Verify first that Python 3.10 or newer is available and that the executor you need, `xm_local.Local`, `xmanager.xm.PythonDocker`, `xm_local.BazelContainer` or `xm_local.Vertex`, has its prerequisites in place. The licence field on the repository says Apache-2.0, the last push was on 2026-10-15, and the current version is 0.8.3. ## FAQ ### How do I install XManager? The README gives `pip install git+https://github.com/deepmind/xmanager.git` or the PyPI route `pip install xmanager`. On Debian based systems it also describes cloning the repository and running `cd xmanager/setup_scripts && chmod +x setup_all.sh && . ./setup_all.sh`. The codebase assumes Python 3.10 or newer. ### What do XManager launch scripts look like? A launch script opens an experiment context with `xm_local.create_experiment(experiment_title='cifar10')`, packages an executable with `experiment.package`, and then adds jobs with `experiment.add(xm.Job(...))` passing arguments and environment variables. The README also offers a Colab notebook and a Jupyter notebook codelab. ### Where does XManager store experiment metadata? The experiment context saves its metadata to the database when the experiment is created, and the experiment carries tracking properties such as an `id`. The same data can be retrieved later, for example with `xm_local.list_experiments`. The packaging configuration includes `alembic` and `*.sql` package data. ### Which backends does XManager support? The README names `xm_local.Local` for running locally, `xmanager.xm.PythonDocker` which requires Docker, `xmanager.xm_local.BazelContainer` and `xmanager.xm_local.BazelBinary` which require Bazel, and `xm_local.Vertex` for Vertex AI, which needs a GCP project. The examples directory includes Kubernetes, TPU and TensorBoard variants. ### Does XManager include a server or web interface? No. The README states that this repository only contains the client code, and not the server or UI code required for full integration. The private preview of XManager on Cloud, XMC, is reached by contacting a GCP account team or the address given in the README. ## Sources - [google-deepmind/xmanager on GitHub](https://github.com/google-deepmind/xmanager) - [Issues](https://github.com/google-deepmind/xmanager/issues) - [License: Apache-2.0](https://github.com/google-deepmind/xmanager/blob/main/LICENSE) - [README](https://github.com/google-deepmind/xmanager/blob/main/README.md) - [Releases](https://github.com/google-deepmind/xmanager/releases) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/google-deepmind-xmanager