# distributed: the scheduler that turns a Dask cluster into a running system

> The distributed package is the scheduler and worker runtime behind Dask, and its manifest says a lot about how it fits into a cluster: it is version-locked to the Dask core, it registers four command line entry points, and it depends on process, network and locking libraries rather than on a broker.

**dask/distributed** — A distributed task scheduler for Dask

- Repository: https://github.com/dask/distributed
- Website: https://distributed.dask.org
- Stars: 1,695 · Forks: 783
- Language: Python
- License: BSD-3-Clause
- Published: 2026-10-16 · Updated: 2026-10-16 · Language: en
- Canonical page: https://hysenlabs.com/projects/dask-distributed

## This is the scheduler half of Dask, not Dask itself

The project describes itself in one line as a distributed scheduler for Dask, and the README calls it a library for distributed computation with the documentation hosted on its own site. The distinction matters when you pick dependencies, because the Dask core library is a separate package and this one depends on it. The manifest names the package `distributed`, gives the description as Distributed scheduler for Dask, and lists Matthew Rocklin as the maintainer. The source tree is shaped like a long-lived scientific software project rather than a small library: a `distributed` package directory, a `docs` directory, a `continuous_integration` directory, a conftest at the root, CODEOWNERS and AUTHORS files, a pre-commit configuration, and both AGENTS.md and CLAUDE.md files for tooling instructions. The licence is BSD-3-Clause, with the licence text shipped as a separate file.

## The scheduler is version-locked to the Dask core

The dependency list contains one entry that behaves differently from the rest, and it is the first thing to notice in the manifest.

```toml
dependencies = [
    "click >= 8.0",
    "cloudpickle >= 3.0.0",
    "dask >=2026.8.0,<2026.8.1",
    "jinja2 >= 2.10.3",
    "locket >= 1.0.0",
    "msgpack >= 1.0.2",
]
```

Every other dependency has a floor and an open upper bound, but the Dask core dependency is pinned to a narrow range, and the range matches the version of the distributed release itself. A distributed release is therefore published against one specific Dask core release, and mixing versions is not something the manifest invites. The release list shows the versioning style, calendar based: 2026.8.0 on 2026-08-24, then 2026.1.3 and 2025.9.3, both dated 2026-07-13. The last push to the repository was on 2026-10-18, so the project is still moving.

## Four commands are registered with the Dask CLI

The package contributes four entry points under the `dask_cli` group, which is how a user starts a cluster from a terminal.

```toml
[project.entry-points.dask_cli]
scheduler = "distributed.cli.dask_scheduler:main"
worker = "distributed.cli.dask_worker:main"
ssh = "distributed.cli.dask_ssh:main"
spec = "distributed.cli.dask_spec:main"
```

There is a scheduler, a worker, an ssh helper and a spec command, and each maps to a module under the package's cli directory. The naming is worth internalising because it is how the pieces are launched separately, which is the normal shape of a cluster deployment: a scheduler process, one or more worker processes, and configuration expressed as a spec. The presence of an ssh entry point also hints at a mode where workers are started on other machines rather than on the same host, which is the difference between a single machine setup and a distributed one.

## The dependency list is a map of what a cluster touches

Read the rest of the requirements and the architecture becomes visible. `locket` handles file locking, `msgpack` is the serialisation format on the wire, `tornado` provides the asynchronous HTTP and websocket layer that a scheduler needs, `zict` manages buffers that spill to disk, `tblib` is used for traceback pickling, `psutil` reports on processes and memory, `cloudpickle` sends Python objects to the workers, `click` builds the command line, and `toolz` supplies the functional utilities the codebase is built on. `sortedcontainers` and `pyyaml` cover ordered data structures and configuration. Note the explicit exclusions on `tblib`, which is pinned with two versions barred outright, and the fact that `numpy` and `pandas` are absent, because this package schedules work rather than computing on it. The build itself uses setuptools with setuptools-scm and derives its version dynamically.

## The test report is treated as a shipped artifact

The README is mostly badges, and the badges are unusual in what they point at. There is a test status badge for the main branch, a coverage badge, a documentation status badge for the readthedocs build, a badge for the version published on PyPI, a link to the project discourse forum, and a NumFOCUS badge for the foundation that hosts the project. Two of them are not standard: a longitudinal test report in a full and a short version, both produced by a dedicated workflow and published to a report site. That is a continuous integration setup that tracks how the test suite behaves over time rather than only on the current commit, which is the kind of signal that matters in a package used to schedule other people's long running work.

## A modern Python floor and a reproducible development setup

The manifest declares a Python floor of 3.10 or later and lists classifiers for 3.10, 3.11, 3.12, 3.13 and 3.14, with the development status recorded as Production/Stable and the audience as developers and science and research. The build requirements ask for setuptools 80 or later and setuptools-scm 9 or later, and the version is dynamic, so it is derived from the repository state rather than written by hand. Around that, the repository uses a pixi manifest and lock file for environment management, a pre-commit configuration for formatting and lint checks, a codecov configuration, and a readthedocs configuration for the documentation build. The combination of a pinned toolchain, a locked environment and a version derived from source is the shape of a package where a wrong build is hard to diagnose later.

## Conclusion

Use the distributed package when you are running Dask as a real cluster with separate processes, because it is the part that owns scheduling, worker communication and the command line entry points. Do not add it to a project that only needs Dask's array or dataframe API on a laptop, since the Dask core package already carries that. Before you pin a version, read the dependency on the Dask core, which is pinned to an exact patch range, and plan to upgrade the two together rather than letting pip resolve them apart.

## FAQ

### What is the distributed package for?

It is the distributed scheduler for Dask, the part that runs the scheduler and workers and provides the command line entry points, as opposed to the Dask core library that defines the collections and their computations.

### Which Python versions does distributed support?

Python 3.10 and later, with classifiers listed for 3.10 through 3.14. The manifest records the project as production stable and targets developers and scientific users.

### Which commands does the package add to the Dask CLI?

Four: a scheduler, a worker, an ssh helper and a spec command, each mapping to a module in the package's cli directory.

### How is the distributed version determined?

It is declared as a dynamic version in the packaging metadata and derived from the repository state by setuptools-scm, so the published version follows the source rather than being typed in by hand.

## Sources

- [dask/distributed on GitHub](https://github.com/dask/distributed)
- [License: BSD-3-Clause](https://github.com/dask/distributed/blob/main/LICENSE)
- [Project website](https://distributed.dask.org)
- [README](https://github.com/dask/distributed/blob/main/README.md)
- [Releases](https://github.com/dask/distributed/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/dask-distributed