dit implements a whole family of multivariate information measures, on purpose

Why a package implements four mutual informations on purpose

The introduction of dit explains the problem the package is solving, and it is a good problem statement.

Information theory is a powerful extension to probability and statistics, quantifying dependencies among arbitrary random variables in a way that is consistent and comparable across systems and scales. Information theory was originally developed to quantify how quickly and reliably information could be transmitted across an arbitrary channel.

Then the twist: the demands of modern, data-driven science have been coopting and extending these quantities and methods into unknown, multivariate settings where the interpretation and best practices are not known. And the specific example is at least four reasonable multivariate generalizations of the mutual information, none of which inherit all the interpretations of the standard bivariate case, and which is best to use is context-dependent.

So the package is not trying to pick the right one. It implements a vast range of multivariate information measures in an effort to allow information practitioners to study how these various measures behave and interact in a variety of contexts. The stated hope is that having all these measures and techniques implemented in one place will allow the development of dependable techniques for the automated quantification of dependencies within a system and concrete interpretation of what those dependencies mean.

That is a research position rather than a library position, and it explains a lot about the shape of the codebase. A package that picked one generalization would be smaller. A package that implements all of them, with consistent APIs, is an instrument for comparing them.

The package is BSD 3-Clause licensed, with the last push on 2026-10-21 and releases v2.3 from 2026-09-26, v2.2 from 2026-07-14 and v2.1 from 2026-06-24. The documentation is at docs.dit.io, there is a Slack badge, and the project is on PyPI and conda-forge.

The measure catalogue is grouped by what it does

The implemented measures table is the heart of the documentation, and it is organised into four columns of related quantities.

The entropies column starts with Shannon Entropy and then the variants: Renyi Entropy, Tsallis Entropy, Necessary Conditional Entropy, and Residual Entropy, also called Independent Information or Variation of Information.

The mutual informations column is the widest. It has Co-Information, Interaction Information, Total Correlation or Multi-Information, Dual Total Correlation or Binding Information, CAEKL Multivariate Mutual Information, O-Information, Cohesion, Transmission, Union Information, Logarithmic Decomposition, the Delta^k and Gamma^k family, and DeWeese Mutual Information.

The divergences column lists the distance measures: Variational Distance, Kullback-Leibler Divergence and Relative Entropy, Cross Entropy, Jensen-Shannon Divergence, and Earth Mover's Distance.

And the fourth column holds the remaining things: Common Informations, which include the Gacs-Korner Common Information, Wyner Common Information, Exact Common Information, Functional Common Information and MSS Common Information, plus the other measures of Channel Capacity, Complexity Profile, and Connected Informations.

The note above the table is important for reading it. Most of these are implemented in multivariate and conditional generality, where such generalizations either exist in the literature or are relatively obvious. And it gives one example of something that is neither: though it is not in the literature, the multivariate conditional exact common information is implemented here.

So the package is explicit about provenance. Some of these measures have a published definition and others are extrapolations the authors considered obvious. If you are going to cite a number from this package, knowing which category a measure falls into is the difference between citing a result and citing a guess.

The common information column deserves a second look. Common information is a genuinely contested area, with several different definitions going under similar names, which is precisely why a package implementing Gacs-Korner, Wyner, Exact, Functional and MSS side by side is useful.

Installing it and the dependency set

The easiest way to install is a single pip command:

pip install dit

And within a conda environment:

conda install -c conda-forge dit

For development the project recommends uv:

git clone https://github.com/dit/dit.git
cd dit
uv sync --extra dev

That installs dit in editable mode with all development dependencies, being tests, docs, linting, type checking, and optional backends.

The required dependencies are listed in a table, and they explain the package's scope. Python 3.11 or newer. boltons, which is a set of general purpose utilities. debtcollector, an OpenStack project for managing deprecation warnings. lattices, from the same dit organisation. loguru for logging. networkx for graph structures. numpy. PLTable for pretty tables. scipy. And xarray for labelled arrays.

Those are the choices that define what kind of package this is. scipy and numpy for the numerics, networkx because many information measures are defined over graphs, xarray because entropy estimators work over arrays with labels, and lattices because the common information measures need lattice structures.

The optional dependencies are the more interesting list, because it shows the design of the optimisation layer. There are three backends with autodiff support: jax and jaxlib for a JAX-based backend, pytensor for a PyTensor-based backend, and torch for a PyTorch-based backend with autodiff and GPU support. Then numdifftools for numerical evaluation of gradients and hessians during optimization, which is the fallback when no autodiff backend is in play.

The rest of the optional extras are for the surrounding tasks. cython for faster sampling from distributions, hypothesis for random sampling of distributions, which is property-based testing applied to the measures themselves, scikit-learn for faster nearest-neighbor lookups during entropy and mutual information estimation from samples, pint for adding units to informational values, matplotlib and python-ternary for plotting the various information-theoretic expansions, pycddlib-standalone and pypoman for polytope vertex enumeration used by convex-maximization-based measures, and colorama for colored column heads in the partial information decomposition output indicating failure modes.

That last one is a nice detail. The partial information decomposition examples in the tree are notebooks, and the colorama extra is there to mark which of the four components could not be computed.

Testing and the shape of the repository

The testing section is short and gives two routes:

# Using uv (recommended)
uv run pytest

# Or with pip
pip install -e ".[test]"
pytest

The pip route installs the test extra and runs pytest, and the uv route uses the environment the sync command created.

The repository layout is long and tells you about the project's history and its ambitions. There is the dit/ directory for the package, docs/ for the documentation sources, tests/, examples/ with the notebooks, bench/ for benchmarks, site/ for the website, joss_paper/ for the Journal of Open Source Software paper, and binder/ for Binder launch configurations so the examples can run in a browser.

There are also some files that record an older era. A .travis.yml and an appveyor.yml sit next to the .github/ directory, which means the project has moved from Travis and AppVeyor to GitHub Actions without removing the old configuration files. The .prospector.yaml and .pylintrc are likewise from the older Python tooling. It is a repository that has grown up over years rather than been written once.

The CREDITS.rst and HISTORY.rst files are the same kind of artefact, and CITATION is present in a well-known format. The README also gives a BibTeX entry for the JOSS paper, and the journal, volume, number, pages and DOI are all filled in. There is a Zenodo badge and a conda badge and a codecov badge, so the project is set up for archiving and for coverage tracking.

The examples directory is where the value for a newcomer is. There is MDBSI.ipynb, PID.ipynb for partial information decomposition, a Time Series Analysis notebook, and example_maxent.py, example_source_polar.py and example_symbolic.py. There is hypothesis.ipynb, which applies property-based testing to the measures, and linear_building_blocks_probe.py, which is a probe rather than an example.

The examples are in notebooks for a reason specific to this package. The whole argument of the project is that different measures behave differently in different contexts, and a notebook is where you can put two implementations side by side and look at the numbers.

Where this sits against the alternatives

The comparison worth making is with doing the information theory yourself, and with the narrower tools that exist.

For univariate entropy, numpy and scipy are enough, and a few lines will compute Shannon entropy correctly. The case for dit is the multivariate and conditional cases, where each generalization is a research-level decision rather than a formula lookup, and where the number of plausible definitions is large enough that implementing them all consistently is a project in itself.

The other comparison is with scikit-learn, which has mutual information estimators and is far more widely used. The optional dependency list even says scikit-learn provides faster nearest-neighbor lookups during entropy and mutual information estimation from samples, which is a fair description of where it fits: as the estimator backend underneath a measure rather than as the measure itself.

What dit offers that those do not is breadth with a consistent API and the multivariate and conditional generalizations, and the partial information decomposition in particular, which is a whole area that general-purpose libraries leave alone.

The cost is real and worth stating. The package pulls in scipy, networkx, xarray and a set of lattice machinery, and the optional backends are substantial if you want autodiff. And the design decision to implement many generalizations means you have to make the choice yourself, which is the point of the package and also the thing that makes it less convenient.

The right way to use it is as an instrument rather than a dependency. Pick two or three measures that make sense for your data, compute them side by side, and check that your conclusions do not depend on which one you chose. If they do, that is a finding about your system, and it is exactly the kind of thing the package was built to let you discover.

For a research codebase the JOSS citation is a good sign. A package published in a journal that reviews software papers has been through a review of its documentation and its interface, and that tends to produce a stable API.

Editorial conclusion

Adopt dit if you are doing research in information theory on discrete random variables and want every measure in one place with consistent APIs, since the alternative is implementing each one yourself and comparing them by hand. Skip it if you need continuous or differential entropy, since the package is explicitly for discrete random variables, and if you want a single number for a mutual information question, since it deliberately offers several with different interpretations. Verify first which measures you need are implemented in the version you install, that your Python is 3.11 or newer, and which optimization backend you want, since the optional extras swap between JAX, PyTorch and PyTensor.

Frequently asked questions

Official sources

1. dit/dit on GitHub
2. License: BSD-3-Clause
3. Project website
4. README
5. Releases