Bottleneck speeds up the NumPy reductions that skip NaNs

A small set of functions, with a NumPy shaped API

Bottleneck is a collection of fast NumPy array functions written in C, and the demonstration in the readme is the whole pitch in four lines.

You create a NumPy array that contains a missing value:

>>> import numpy as np
>>> a = np.array([1, 2, np.nan, 4, 5])

And you ask for the mean that ignores it, which in NumPy means np.nanmean, a function that has to take a slow path because it has to mask the NaNs before reducing:

>>> import bottleneck as bn
>>> bn.nanmean(a)
3.0

The API is deliberately a subset of NumPy with the same names, so moving a pipeline to Bottleneck is mostly a change of import. What you get in exchange is a C implementation that does not allocate the mask, and for a frame with a million rows that difference is the difference between a function that fits in a pipeline and one that does not.

The function list in the benchmark output is the inventory. On the reduction side there is nansum, nanmean, nanstd, nanvar, nanmin, nanmax, nanmedian, median, nanargmin, nanargmax, anynan, allnan, plus ss, rankdata, nanrankdata, partition, argpartition, replace and push. On the rolling side there is move_sum, move_mean, move_std, move_var, move_min, move_max, move_argmin, move_argmax, move_median and move_rank.

The audience is anyone who has profiled a numeric pipeline and found that the bottleneck is literally the bottleneck package's namesake. The intended audiences in the project metadata are science and research, and the financial and insurance industry, which is a good description of where nan-aware reductions over large tables actually dominate a runtime.

Four dtypes are accelerated, and the rest fall back

The readme is explicit about the boundary, and it is the single most important sentence for an adopter.

Only arrays with data type int32, int64, float32 and float64 are accelerated. All other dtypes result in calls to slower, unaccelerated functions. And in the rare case of a byte-swapped input array, for example a big-endian array on a little-endian operating system, the function will not be accelerated regardless of dtype.

That second clause is the kind of detail that saves a performance investigation. A frame read from a file with a non native byte order, or a view cast onto a buffer with the wrong order, will silently take the slow path even though the dtype is right, and the result is correct but slow.

The same logic explains why the package does not try to be everything. Accelerating every dtype and every byte order would multiply the size of the C code and the test matrix, and the returns would be small. The design choice is to accelerate the four dtypes that dominate real workloads and let everything else be correct but ordinary.

There is a related detail in the build configuration, which shows the same attention to the NumPy boundary. The extension is compiled with the NPY_NO_DEPRECATED_API macro set to the NPY 2.0 API version, and the project metadata builds against numpy>=2,<2.8, with a comment that the upper bound should be three minor releases ahead of the current runtime version. That is how a C extension against a moving C API stays buildable without pinning everyone to an old NumPy.

So the practical checklist for an adopter is short: are your arrays float64 or float32, and are they in native byte order. If both answers are yes, you are in the accelerated path, and if either is no, the speedup you came for will not happen.

The moving window functions and the min_count argument

The second family of functions is the rolling window family, and the example in the readme shows the shape of the API.

>>> bn.move_mean(a, window=2, min_count=1)
array([ 1. ,  1.5,  2. ,  4. ,  4.5])

The window argument is the number of elements in the rolling window, and min_count is the minimum number of non-NaN values required for a window to produce a value. With min_count set to 1, the window that straddles the NaN, the third position, still produces 2.0, which is the mean of 2 and 4, because one of the two values is present. Set min_count to 2 and that same window would be NaN.

That argument is the one to reach for, and it is the reason to use a moving window function rather than rolling your own with a convolution. The rule for how many observations a window needs is a modelling decision, not a detail, and making it a named parameter means the same value is applied consistently across every window in a data set.

The family covers the usual operations. move_sum, move_mean, move_std, move_var, move_min, move_max, move_argmin and move_argmax, plus move_median and move_rank, which are the expensive ones, because a median over a sliding window cannot be done with a running sum.

The readme notes the memory property that makes a hidden state model a hidden state model, and it is worth repeating because it explains why these functions exist at all. By using a hidden state to represent the past observations, a model can have effectively infinite memory, unlike a standard autoregressive model. A moving window is the observable shadow of that idea, and a library that computes it in C is what makes it usable at scale.

The benchmark is part of the package, so you can check the claim yourself

Bottleneck ships its own benchmark, and the readme shows the output.

>>> bn.bench()
Bottleneck performance benchmark
    Bottleneck 1.6.0.post0.dev32; Numpy 2.4.2
    Speed is NumPy time divided by Bottleneck time
    NaN means approx one-fifth NaNs; float64 used

The framing matters more than the numbers. The speed is NumPy time divided by Bottleneck time, so a higher number means Bottleneck was faster, and the benchmark also varies the NaN density, with the NaN column meaning approximately one fifth NaNs, and it runs over several array shapes and both axis directions, because an axis of reduction changes the memory access pattern completely.

For a single function there is a detailed form:

>>> bn.bench_detailed("move_median", fraction_nan=0.3)

So the answer to the question that matters, how much faster is it on my data, is a function call rather than a documentation lookup. That is unusual and it is the most defensible part of the project, because the numbers in the readme are for a machine that is not yours and for a workload that is not yours.

There is a Makefile target that does the same thing from the shell, bench, which runs bottleneck.bench(), and detail, which runs bench_detailed with the argument all. And a target called readme, which runs tools/update_readme.py, so the table in the readme is generated from a run rather than maintained by hand.

The benchmark suite in the repository is asv_bench, the directory name used by the airspeed velocity project, which is the standard tool for tracking the performance of a library across commits.

Install a wheel, and only build from source if you must

The installation guidance is short and it puts the fast path first.

Bottleneck provides binary wheels on PyPI for all the most common platforms, and binary packages are also available in conda-forge. The recommendation is to install binaries with pip, uv, conda or similar, because it is faster and easier than building from source.

The source route has its own requirements table. Bottleneck needs Python greater than 3.9 and NumPy 1.16.0 or newer, compilation needs gcc, clang, MinGW or MSVC, the unit tests need pytest, and the documentation needs sphinx and numpydoc. Note that those are the requirements as the readme states them, while the current project metadata declares requires-python >=3.11 and a NumPy 2 series dependency, so the readme's requirements table is behind the packaging metadata.

On Linux and macOS and similar systems the install is:

$ pip install .

And on Windows you need to install MinGW and add it to your path first, then install with the compiler flag:

$ python setup.py install --compiler=mingw32

The MinGW requirement is the kind of thing that will stop a Windows contributor cold, and it is why the wheels matter. A C extension with no prebuilt wheel for your platform means a compiler toolchain before you can even import the package.

The project also ships a unit test suite that runs from the installed package:

In [1]: import bottleneck as bn
In [2]: bn.test()

with the option of running py.test directly if you are working in a git checkout. The test run shown in the readme collects 190 items across the test modules, covering input modification, list input, move functions, non reduce axis, non reduce, reduce, and scalar input.

The C sources are generated from templates at build time

The build machinery in setup.py is worth a look, because it explains a lot about the project.

The extension is compiled with a set of define_macros that include the NumPy API version macro, with a comment to keep it in sync with the runtime requirements in pyproject.toml. Then there is a custom config command that runs before the build, and its implementation calls create_config_h from a module called bn_config, which generates a header at bottleneck/src/bn_config.h. The build-time configuration is therefore a generated header rather than a checked in one.

The clean command is the mirror image. It walks the package directory and removes the .pyc and .so files, the build directory, and the generated .c files, and it deletes bn_config.h. The detail is that it skips files whose name ends in .c and contains template, and deletes the file that would have been generated from it. So the C sources in the package are themselves generated from templates at build time, which is how a single implementation of a function family can be emitted for every dtype, every axis and every function variant without a combinatorial explosion of hand written files.

That design has a cost. A contributor who edits a .c file directly will find their change deleted by the next clean, because the file is regenerated. The templates are the source of truth, and the project keeps a .clang-format file and a .git-blame-ignore-revs file, which is what you do when a large number of generated files churn in every commit and you want the history to be readable.

The Makefile coordinates it, with a build target that runs setup.py build_ext --inplace, a clean target that removes the build directory, the distribution, the egg info and the generated .c and .so files, and an all target that chains clean, build, test and lint. The lint target runs ruff check even though the Makefile's help text still mentions flake8, which is a small reminder that the help output is maintained by hand while the targets are not.

A permissive licence, and release tags that stopped being used

The licence is a Simplified BSD, the project metadata records it as BSD-2-Clause, and the LICENSE file and the LICENSES directory hold the details. The LICENSES directory is worth noting because it lists the licences of the projects whose code or data is included, with entries for the NumPy, pandas, SciPy and setuptools licences alongside the project's own. That is a level of attribution care you rarely see outside the scientific Python stack, and it is a fair signal about how the project is run.

The release history is where the repository looks odd, and it is worth explaining rather than repeating. The tagged releases on the repository are v1.4.0.dev0 from 2019-12-13, v1.3.0rc2 and v1.3.0rc1 from 2019-11-13. Those are the last tags. The benchmark output in the readme, however, reports Bottleneck 1.6.0.post0.dev32, and the package metadata declares the version as dynamic, generated by versioneer from the git tags, with a versioneer.py at the repository root and a uv.lock for the development environment.

So the project has moved to a development version scheme, where the published artifacts on PyPI carry the real version numbers and the GitHub release tags stopped being maintained. The practical consequence for a consumer is that you should read the version from the installed package or from PyPI, not from the repository's release list, and the presence of a RELEASE.rst file linked as the changelog in the project metadata is where the actual history now lives.

The repository is not archived and the last push was on 2026-10-06, with the metadata classifying the project as Development Status 5, Production/Stable, and listing support for Python 3.11 through 3.15 and a beta classifier for free threading.

Editorial conclusion

Bottleneck is worth adopting if you are doing repeated nan-aware reductions or rolling window computations on large NumPy arrays, because the drop in Python level overhead is the entire point and the API mirrors NumPy closely enough that it is a small change. It is the wrong choice if your data is not one of the four accelerated dtypes, because then you get the slow path with none of the benefit, and it is the wrong choice if you need something outside the function list. Before you depend on it, run bn.bench() on your own machine and your own shapes, and note that the current development version is 1.6.0.post0.dev32 with the repository pushed to on 2026-10-06.

Frequently asked questions

Official sources

1. Issues
2. License: BSD-2-Clause
3. pydata/bottleneck on GitHub
4. README
5. Releases