krasserm/bayesian-machine-learning: A Notebook Collection, Not a Library

What the notebooks cover and who they are written for

The repository addresses a specific gap: Bayesian machine learning is usually taught either as dense notation or as a library call, with little in between. Each notebook picks one method and works it through. The set covers Bayesian linear regression with basis function models, Gaussian process regression, Gaussian process classification, sparse Gaussian processes via a variational approach, Bayesian optimization with hyper-parameter tuning as the application, variational inference in Bayesian neural networks, noise contrastive priors for uncertainty estimates on out-of-distribution data, Gaussian mixture models with the EM algorithm, stochastic variational inference with a variational autoencoder, a deep feature consistent VAE using a perceptual loss, and conditional generation by running Bayesian optimization in a VAE latent space. The intended reader is someone who already writes Python and has seen the mathematics once, and now wants to see the matrix operations that make it work. The README points readers at nbviewer links "to ensure a proper rendering of formulas", which tells you the author expects people to read these in a browser as much as execute them.

The dual-implementation pattern and why it matters

Most notebooks ship two versions of the same method. The Gaussian process notebook implements regression "with plain NumPy/SciPy as well as with scikit-learn and GPy". The Bayesian optimization notebook does the same with NumPy/SciPy, scikit-optimize and GPyOpt. Bayesian linear regression has a plain NumPy and scikit-learn version plus a separate PyMC3 notebook. The sparse Gaussian process notebook is the outlier: its example implementation uses JAX. This duplication is the actual value of the repository. Reading a scikit-learn GaussianProcessRegressor call tells you nothing about marginal likelihood computation or kernel matrix inversion; the NumPy version in the same directory does. The cost is redundancy. If you only want the library usage, you are reading roughly twice the notebook you need, and the two versions can drift apart in notation and variable naming because nothing enforces consistency between them.

Getting a notebook running: requirements.txt per subdirectory

There is no top-level install step. The README states that "Dependencies are specified in requirements.txt files in subdirectories", so the workflow is to enter the directory for the notebook you want and install from there. For the Gaussian process notebooks that means the gaussian-processes directory; for variational autoencoders, latent-variable-models; for the noise contrastive priors notebook, noise-contrastive-priors. Several notebooks carry an Open in Colab badge, which points at the dev branch path, for example colab.research.google.com/github/krasserm/bayesian-machine-learning/blob/dev/gaussian-processes/gaussian_processes.ipynb. Colab is the lowest-friction route because it sidesteps local dependency resolution entirely, but it pins you to whatever versions Colab provides rather than the ones in the requirements file. That distinction matters for the TensorFlow 2 and TensorFlow Probability notebook and for the JAX sparse Gaussian process notebook, where API surface has moved since the notebooks were written. The default branch is dev, not main, so any raw URL or git clone you script should reference dev explicitly.

Version drift is the main failure mode

The three tagged releases are v-0.1 (August 2020), v-0.2 (September 2020) and v-0.3 (December 2020). The repository is not archived and has received pushes more recently than that, but the release tags stopped in 2020, and the README describes implementations against TensorFlow 2.x, Keras and PyMC3. PyMC3 in particular has since been superseded in the wider ecosystem, and the notebook is referenced as an alternative implementation rather than the primary one. If you clone and run the requirements files as-is, expect to spend time resolving incompatible pins for the deep learning notebooks. The NumPy and SciPy notebooks are far more durable because they depend on a much smaller and more stable surface. A practical consequence: treat the plain NumPy implementations as the long-lived part of this repository and the framework implementations as time-stamped illustrations of an API at a point in 2020.

Where a dedicated library is the better choice

If your goal is to fit a Gaussian process or run Bayesian optimization in production, this repository is the wrong tool. GPy and scikit-learn both give you fitted model objects, kernel composition, and prediction APIs that you can put behind a service. scikit-optimize gives you a minimizer with a defined search space interface. The notebooks here reimplement the inner loops for exposition, and exposition code is not written for input validation, numerical edge cases, or batch prediction. The concrete difference in approach: a library hands you a model abstraction and hides the linear algebra, while these notebooks expose the linear algebra and leave you to build the abstraction. Choose the library when you need to ship; choose the notebook when you need to understand what the library is doing before you trust its output on your data.

Licence and what you may reuse

The repository is Apache-2.0. That permits commercial use, modification and redistribution provided you retain the licence and notices, and it includes a patent grant. It does not mean the notebooks are free of third-party obligations: the code imports scikit-learn, GPy, GPyOpt, scikit-optimize, JAX, Keras and TensorFlow Probability, each under its own licence, and copying a notebook into your own repository does not change those terms. The README also carries a Zenodo DOI badge (10.5281/zenodo.4318528), which gives you a citable identifier for the collection. This is a description of the stated licence, not legal advice; check Apache-2.0 and the dependency licences against your own distribution model.

Maintenance cost if you fork it

Forking means owning dependency resolution for every subdirectory you keep. There is no CI configuration described in the material, no lockfile, and no single environment file, so upgrading one notebook's frameworks does not help the others. The realistic maintenance path is to keep only the subdirectories you actually use and pin them yourself. The NumPy and SciPy notebooks in bayesian-linear-regression, gaussian-processes and latent-variable-models part 1 are the cheapest to keep alive. The TensorFlow Probability, Keras and JAX notebooks are the most expensive, and they are also the ones where the surrounding ecosystem has moved furthest since the last release tag. Budget accordingly before treating this as a base to build on rather than a reference to read.

Editorial conclusion

Adopt this repository if you want to read and run Bayesian methods from first principles rather than call them from an API: the notebooks implement Gaussian processes, variational inference and Bayesian optimization in plain NumPy or SciPy before showing the scikit-learn, GPy, GPyOpt, Keras or TensorFlow Probability equivalents. Do not adopt it if you need a maintained library with a versioned API, since the last tagged release is v-0.3 from December 2020 and dependencies live in per-directory requirements.txt files rather than a single pinned environment. Verify first that the subdirectory you care about still resolves its pinned dependencies, and check the Apache-2.0 licence terms against your own redistribution plans before copying notebook code into a product.

Official sources

1. Issues
2. krasserm/bayesian-machine-learning on GitHub
3. License: Apache-2.0
4. README
5. Releases