Mish: the activation function repository that is mostly a paper and a set of notebooks
Official Repository for "Mish: A Self Regularized Non-Monotonic Neural Activation Function" [BMVC 2020]
At a glance
- What is it?
- Mish is a non-monotonic activation function proposed in a BMVC 2020 paper. The repository of the same name is the paper's official home, not a library you install, and that distinction decides whether it belongs in your stack.
- Who is it for?
- Adopt Mish through a framework that already implements it rather than through this repository. The changelog records the PyTorch merge as landing in PyTorch 1.9, with MXNet, TensorFlow JS, TorchSharp, KotlinDL, MONAI and OneFlow added later, so the first thing to verify is whether your framework version exposes the activation natively.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 58 days ago.
- What is it written in?
- Mainly Jupyter Notebook, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What Mish is, and what this repository is not
Mish is an activation function, described in the repository as a self regularized non-monotonic function, published at BMVC 2020 with an arXiv version at 1908.08681v3. The repository is the official home for that work. Its primary language is Jupyter Notebook, and the README is dominated by a citation block, a changelog, and links out to other people's implementations. There is no install section, no package name to add to a requirements file, and no release was retrieved for this review. If you arrived expecting a library, adjust the expectation now. What you get is the paper, the notebooks behind its experiments, and a running record of where the function was merged into other projects. That is a legitimate artifact for a research contribution. It is not a dependency. The practical consequence is that adoption of Mish happens in whatever framework you already use, and this repository is where you go to understand the function and to find the citation, not to import it.
The mechanism: a non-monotonic curve with a softplus inside it
The paper's framing, as the repository title states it, is a self regularized non-monotonic activation function. Non-monotonic means the output is not a strictly increasing function of the input. It dips, reaches a minimum below zero for negative inputs, and then rises. That shape is the whole argument: the small negative region gives the optimiser a gradient signal on negative activations instead of the flat zero that a rectifier produces. The self regularizing part of the name refers to the paper's claim that the function's behaviour reduces the need for other regularization. The repository does not walk through the derivation in the README; the mathematics lives in the linked paper. What the repository does provide is the empirical side, notebooks for layer accuracy and benchmark tables, plus a PyTorch Benchmarks directory referenced in the changelog entry dated 08/13 for updated benchmarks and pretrained models. Treat the notebooks as the primary evidence and the README as an index. The README does not state the closed form of the function, so read the paper before you decide anything about the curve's behaviour.
You do not install this repository, you find Mish in your framework
The changelog is the most useful part of the README for a practitioner, because it is a list of merge points. It records Mish being added to PyTorch, with the note that it would be added in PyTorch 1.9. It records additions to MXNet, TensorFlow JS, TorchSharp, KotlinDL, MONAI, OneFlow, OpenVino, Sony Nnabla, TensorFlow Swift APIs, TFLearn, plaidml, Axon, GPT-Neo, BetaML.jl, and Google's AutoML. For most readers the integration path is therefore a one-line change in the framework already in use, for example swapping an existing activation for the framework's native Mish. The repository does not give a canonical installation command, so there is no honest command to reproduce here beyond the links it provides. Two of those links matter if you need speed rather than correctness: a CUDA version credited to Thomas Brandon, and faster variants of Mish and H-Mish by Yashas Samaga. A memory efficient experimental version is also linked, hosted inside rwightman's gen-efficientnet-pytorch. Note the word experimental on that last one; it is a pointer, not a supported path.
The cost you pay is in memory and in kernel support
An activation function that is not a single comparison has to be evaluated, and evaluation costs bandwidth. The repository acknowledges this indirectly: the existence of a dedicated CUDA implementation and of faster variants implies the default implementation was not fast enough for everyone. The memory efficient version is explicitly labelled experimental, which suggests the straightforward formulation holds intermediate tensors. This is the trade-off a reader should weigh. If your model is already memory bound, or your inference target is a mobile or embedded runtime, an activation that needs a softplus, a tanh and a multiplication per element is a different proposition from a rectifier. The repository does not publish latency numbers, so you cannot settle this from the README. You settle it by profiling your own model with the framework's native implementation. There is also a portability constraint: if your deployment toolchain has no Mish kernel, you will be exporting a graph that either falls back to a slow decomposition or fails to convert. The OpenVino and Nnabla entries in the changelog show that some runtimes did add support, but the list is finite and you should check yours rather than assume.
The benchmark material is notebooks, and that shapes how you read it
The repository's primary language is Jupyter Notebook, and the changelog points to a Layers_Acc notebook and to a PyTorch Benchmarks directory with pretrained models as of the 08/13 update. This is how research artifacts are usually shipped, and it has consequences. A notebook is a narrative: cells run in order, state accumulates, and reproducing a number means reproducing the environment. There are no release tags retrieved for this repository, so there is no version to pin against. If you want to check a claim from the paper, you are reading code and re-running it, not calling a function from a versioned package. One entry in the changelog also illustrates how quickly such claims age: it states that CSP-p7 with multi-scale Mish was, at the time of writing, state of the art on MS-COCO test-dev, with the single-scale variant third. That is a snapshot dated 23/09 against a leaderboard that moves. It is evidence that the function was competitive in a specific detection setup, not a property of the activation itself. Read the notebooks for method; do not treat the tables as current standings.
How Mish differs from Swish and from a plain rectifier
The obvious comparison is Swish, also called SiLU, which is smooth and non-monotonic in the same family. The repository does not run that comparison in the README, so the honest statement is that the paper is where the argument lives. What can be said from the material is structural. A rectifier is monotonic, piecewise linear, and exactly zero for all negative inputs, which means a unit that saturates on the negative side receives no gradient. Mish is non-monotonic and smooth, so negative inputs still produce a nonzero, differentiable response, and the function's minimum sits below zero rather than at it. Swish shares the smoothness and the negative dip. The difference the paper claims is in the self regularizing behaviour, and that claim is empirical, resting on the benchmark notebooks rather than on an architectural guarantee. A second comparison is with the faster variants linked from the README. Those are not different functions in spirit; they are the same curve computed with fewer operations. Choosing between them is an engineering decision about your kernel budget, not a modelling decision.
Maintenance, licensing, and what the repository commits to
The licence is MIT, stated in the README badge and in the repository metadata, which is permissive and places few obligations beyond preserving the notice. This is not legal advice; read the LICENSE file if the distinction matters to your organisation. On maintenance, the honest observation is that the repository's job is largely done. The function is published, it has been merged into the frameworks listed in the changelog, and the last push recorded for the repository is 2026-07-20. The changelog's most recent entries concern framework integrations rather than changes to the function, which is what you would expect from a settled research artifact. There is no upgrade path to plan for because there is no package to upgrade. The cost you carry is instead the cost of the framework you use: when PyTorch or MXNet changes its activation API, that is the thing you track, not this repository. The one maintenance item worth noting is that the README itself is the index, and links to third-party implementations such as Mish CUDA and the Yashas Samaga variants can rot independently of the paper.
Who should take Mish, and who should leave it
Take it if you are training a vision model and your framework already exposes the activation natively, and you want to test whether it helps on your data. The path is short: swap the activation, run your existing evaluation, compare. Take it also if you are reading the paper and want the notebooks that produced its tables. Leave it if you need a supported, versioned dependency with a changelog you can pin; this repository does not offer that, and the frameworks do. Leave it if your inference target lacks a Mish kernel and you cannot afford a decomposed fallback, and check that before you train, not after. Leave it if you are working outside vision at small scale, where the extra transcendental operations per element buy you little. Before committing, verify three things: that your framework version includes the activation, since the PyTorch merge is documented as landing in 1.9; that your export path to the deployment runtime handles it; and that your own benchmark shows a gain, because the repository's evidence is a set of notebooks and a dated leaderboard snapshot, not a guarantee about your model.
Editorial conclusion
Adopt Mish through a framework that already implements it rather than through this repository. The changelog records the PyTorch merge as landing in PyTorch 1.9, with MXNet, TensorFlow JS, TorchSharp, KotlinDL, MONAI and OneFlow added later, so the first thing to verify is whether your framework version exposes the activation natively. If you are on an older release, or you need a CUDA kernel, the repository itself points elsewhere: Mish CUDA by Thomas Brandon, and the faster variants by Yashas Samaga. Skip this repository if you wanted a pip-installable package with a versioned API. It is a paper artifact with notebooks, and it should be read that way.
Community notes