MorphNet: shrinking a seed network's channels without changing its topology
Fast & Simple Resource-Constrained Learning of Deep Network Structure
At a glance
- What is it?
- MorphNet adds a resource regularizer to the training loss so that convolution output channels can be marked for removal. It rewrites channel counts, not the graph, and the README's recommended path is the newer FiGS LogisticSigmoid regularizer.
- Who is it for?
- Adopt MorphNet if you already have a working convolutional seed network whose layer count and connectivity you are happy with, and the only open question is how many output channels each convolution should keep. Skip it if you need topology changes, since the README states the proposed model has the same number of layers and connectivity pattern as the seed.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 76 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem MorphNet targets: channel count, not architecture
Most architecture search work treats the graph as the search space. MorphNet does not. The README is explicit that it does not change the topology of the network, and that the proposed model will have the same number of layers and connectivity pattern as the seed network. What it changes is the number of output channels in each convolution layer.
That narrows the audience considerably. This is for someone who has a convolutional model that already works, and whose remaining problem is that it is too large or too slow for a deployment target. The README frames the motivation as shrinking a model to satisfy constraints such as memory or latency. It is not a tool for discovering a new block design, and it will not remove a layer for you.
The mechanism is a continuous relaxation of structure learning. A regularizer targets the consumption of a specific resource, such as FLOPs or model size. When that regularization loss is added to the training loss and the sum is minimized by stochastic gradient descent or a similar optimizer, the learning problem becomes a constrained optimization of network structure, with the regularizer acting as the constraint. Filters whose influence has been pushed down far enough get their output channels marked for removal.
How the regularizer crawls the graph and what the threshold decides
The regularizer is not a global penalty applied blindly. According to the README, you initialize it with a threshold and with the output boundary ops and optionally the input boundary ops of your model. The regularizer then crawls the graph starting from the output boundary and applies regularization to some of the ops it encounters. When it reaches any of the input boundary ops, it stops and does not crawl past them, and the ops in the input boundary are not regularized.
That crawl direction matters in practice. If you set the output boundary too low in the graph, layers above it never get regularized. If you omit input boundary ops, the crawl can keep going into parts of the model you did not intend to touch. The boundary arguments are the only structural control the user has over where pruning is allowed to happen.
The threshold is the second control. It determines which output channels can be eliminated. The README lists the alive threshold alongside regularization strength as the key hyperparameters, and notes that the regularizer type is not a hyperparameter because it is uniquely determined by the metric of interest and the presence of BatchNorm. That is a deliberate design constraint: you do not get to pick an algorithm independently of your target cost.
Choosing a regularizer: LogisticSigmoid, Gamma, GroupLasso
The regularizer classes live under the network_regularizers/ directory and are named by algorithm and target cost. The README gives LogisticSigmoidFlopsRegularizer as a Logistic-Sigmoid probabilistic method that regularizes FLOP cost, and GammaModelSizeRegularizer as using the batch norm gamma to regularize model size cost.
Three algorithms are documented. LogisticSigmoid is described as designed to control any model type, but it requires adding simple gating layers. Gamma is designed for models with batch norm and requires that batch norm scale is enabled. GroupLasso is designed for models without batch norm, and the README marks it deprecated.
The decision procedure in the README has two branches. If you can add new layers to your model, add the probabilistic gating operation after any layer you wish to prune and use the LogisticSigmoid regularizers. That is marked as recommended. If you cannot add new layers, pick based on architecture: Gamma if the seed network has BatchNorm, GroupLasso otherwise.
The recommendation is not neutral. The README states that FiGS, the probabilistic approach to channel regularization behind LogisticSigmoid, outperforms the previous regularizers and is the recommended way to apply MorphNet. GroupLasso being labelled deprecated means the no-new-layers, no-BatchNorm path is the least supported corner of the project.
Getting it running: the two-round workflow and the loss you actually add
The README lays out a nine-step procedure, and the ordering is the part people get wrong. You choose a regularizer from morphnet.network_regularizers, initialize it with a threshold and your boundary ops, add the regularization term to your loss, train, then export the proposed structure with StructureExporter. The exported files are in JSON format. After that you modify your model using the exporter output and retrain from scratch without the MorphNet regularizer, using standard hyperparameter values such as the normal learning rate schedule.
The README names these two rounds explicitly: structure learning first, retraining second.
Two API details are easy to conflate, and the README warns about them directly. Do not confuse get_regularization_term(), which is the loss you add to your training, with get_cost(), which is the estimated cost of the network if the proposed structure is applied. Only the former belongs in your optimizer step.
Scaling is the other trap. The regularization loss must be scaled, and the README recommends searching for the scaling hyperparameter along a logarithmic scale spanning a few orders of magnitude around 1/(initial cost). Its own example: if the seed network starts with 1e9 FLOPs, explore regularization strength around 1e-9.
There are two configuration requirements that will silently produce a useless run if missed. With BatchNorm you must enable the scale parameters, the gamma variables, for example by setting scale=True on tf.keras.layers.BatchNormalization. With LogisticSigmoid you must add the probabilistic gating op, and the README repeats this as a note because it is the most common omission.
Two smaller points from the README. MorphNet does not currently add the regularization loss to the tf.GraphKeys.REGULARIZATION_LOSSES collection, and the README says this choice is subject to revision. And a fixed learning rate with no decay is recommended for the structure learning step, though the README says this is not strictly necessary.
Where the workflow is underspecified: stopping, stabilization, expansion
The README is honest that the stopping rule is not solved. The proposed model structure changes as training progresses, and there are no specific guidelines on the stopping time, although you would likely want to wait for the regularization loss, reported via summaries, to stabilize. That is the whole guidance. You are expected to read the TensorBoard summaries and make a judgement call.
This is a real cost, not a documentation nitpick. The exported JSON is a snapshot of a structure that is still moving. Export too early and you retrain a model whose channel counts had not converged. Export too late and you have spent compute on a structure that stopped changing many steps ago. The README offers no metric, no patience parameter, and no automatic trigger.
The optional ninth step has the same character. You may uniformly expand the network to adjust the accuracy versus cost trade-off, and the README notes this can be done before the structure learning step instead. Uniform expansion is a blunt instrument: it scales channel counts across the board rather than selectively, so it undoes some of the selectivity that the regularizer produced. The README presents it as a trade-off knob, not as a refinement, and that framing is accurate.
What MorphNet cannot do, and when a different tool fits better
The hard boundary is topology. Because the proposed model keeps the same number of layers and connectivity pattern as the seed, MorphNet cannot discover that a residual branch should be removed, that two convolutions should be merged, or that a stage should be dropped entirely. If your model's cost is dominated by a design decision at the graph level rather than by channel widths, MorphNet is aimed at the wrong variable.
For that class of problem, a differentiable architecture search method that searches over operations and connections is the different approach. The distinction is in what gets relaxed. MorphNet relaxes channel counts while holding the graph fixed; operation-level search relaxes the choice of operation at each edge while letting the graph itself change. The README positions FiGS as usable as either a pruning algorithm or a full fledged Differentiable Architecture Search method, which suggests the project sees the two as related, but the documented workflow here is the channel-count one: export a JSON structure, rebuild, retrain.
There is a second boundary worth stating. MorphNet is a training-time method, not a post-training compressor. The regularizer has to be present during optimization for channels to be pushed down and marked. You cannot take an already-trained checkpoint and run MorphNet over it to get a smaller model without going through the structure learning round.
The third boundary is the training budget. The procedure is two full training runs, structure learning and retraining, plus a hyperparameter search over regularization strength spanning orders of magnitude around 1/(initial cost). If a single training run is already expensive for you, that multiplier is the real adoption cost.
Maintenance, licence, and what the repository state tells you
The licence is Apache-2.0, which permits commercial use and modification and includes an explicit patent grant. That is the standard permissive arrangement for a Google research release. Nothing in the supplied material describes any additional terms, contributor licence agreement, or model-weight restriction, and I am not giving legal advice; read the LICENSE file in the repository before you depend on it.
The repository is not archived and the last push is dated 2026-07-02. No releases were retrieved, so there is no tagged version to pin against. That has a practical consequence: you are tracking the master branch, and any API change in morphnet.network_regularizers or in the StructureExporter output format arrives without a version boundary. If you vendor the code, record the commit you took.
The README's own notes point at maintenance friction. GroupLasso is marked deprecated. The regularization loss is not added to tf.GraphKeys.REGULARIZATION_LOSSES, and the README says that choice is subject to revision, which means the integration point you write today could change. The recommended path, LogisticSigmoid with gating layers, is the newest and the one the README says outperforms the previous regularizers, so the older paths are the ones more likely to drift.
The upgrade surface is small enough to manage. The regularizer classes, the two accessor methods get_regularization_term() and get_cost(), the StructureExporter JSON schema, and the gating op are the pieces your training code touches. Pin the commit, keep the exporter output under version control alongside the model, and a future pull is a diff you can read rather than a surprise.
Editorial conclusion
Adopt MorphNet if you already have a working convolutional seed network whose layer count and connectivity you are happy with, and the only open question is how many output channels each convolution should keep. Skip it if you need topology changes, since the README states the proposed model has the same number of layers and connectivity pattern as the seed. Before committing, verify three things in your own code: that BatchNormalization layers are built with scale=True, that the probabilistic gating op is inserted after every layer you intend to prune when using LogisticSigmoid, and that your regularization strength sweep is centered on 1/(initial cost), for example 1e-9 for a seed network starting at 1e9 FLOPs.
Community notes