NCFM: dataset distillation by matching neural characteristic functions

What NCFM distills, and who the code is written for

Dataset distillation compresses a large training set into a small synthetic one that trains a model to roughly the same accuracy. NCFM approaches this by matching neural characteristic functions, framed in the paper as a minmax problem. The repository is the official implementation of that paper, accepted at CVPR 2025, and the README points to the arXiv version at arxiv.org/abs/2502.20653.

The intended reader is a researcher who wants to reproduce the numbers, not a product team looking for a data-reduction utility. Everything in the layout assumes that: pretrain/, condense/, evaluation/, condenser/, config/, plus CL/, NCFM/, imagenet_subset/, models/, utils/ and argsprocessor/ at the top level. There is no CLI entry point, no importable high-level API, and no packaging metadata beyond requirements.txt. If you want to call distillation on your own dataset from a script, you are writing that integration yourself.

The three-stage pipeline and how data moves through it

The README describes a linear flow. Stage one trains the models that will later be used to supervise condensation; stage two condenses the dataset; stage three evaluates the distilled result. Each stage is a separate script in its own directory, launched with torchrun for multi-GPU and multi-node execution.

Configs are selected by interpolation path rather than by a single config file: --config_path=../config/{ipc}/{dataset}.yaml, so the directory tree under config/ is organized first by images per class and then by dataset. The condense stage additionally takes --ipc directly on the command line, which means the same number is expressed twice, once in the path and once as a flag. Nothing in the README states that the two are validated against each other, so a mismatch between the flag and the config directory is a plausible way to get a run that looks correct and is not.

The evaluation stage takes --load_path pointing at a distilled_dataset.pt file. That is the handoff format between condense and evaluation: a single PyTorch checkpoint, not a directory of images. Anyone expecting the output to be a browsable synthetic image set will be disappointed; the artifacts live inside the .pt.

The README includes a pipeline figure at asset/figure1.png, which is the only place the internal architecture is depicted. The prose does not restate it.

Installing NCFM and running a first condensation

The README gives five steps. Clone the repository first.

git clone https://github.com/gszfwsb/NCFM.git

Then install the pinned dependencies. The file is short: efficientnet_pytorch, matplotlib, numpy, PyYAML, torch, torchvision and tqdm, with torch==2.5.0 and torchvision==0.20.0 pinned. Because torch is pinned, install this into a fresh virtual environment rather than a shared one, or pip will try to replace whatever CUDA build you already have.

pip install -r requirements.txt

Before condensing, you need pretrained models. The README offers two routes: run the pretrain stage yourself, or download the pretrained_models published at huggingface.co/maomaocun/NCFM. The self-training command takes a GPU list, a node count and a config path.

cd pretrain
torchrun --nproc_per_node={n_gpus} --nnodes=1 pretrain_script.py --gpu={gpu_ids} --config_path=../config/{ipc}/{dataset}.yaml

With checkpoints in place, condense. The README's worked example is CIFAR-10 at 50 images per class across eight GPUs, and it pins the rendezvous port explicitly.

cd condense
torchrun --nproc_per_node=8 --nnodes=1 --master_port=34153 condense_script.py --gpu="0,1,2,3,4,5,6,7" --ipc=50 --config_path=../config/ipc50/cifar10.yaml

Finally, evaluate the distilled checkpoint, or download a condensed dataset from the same Hugging Face repository and skip the condense stage entirely.

cd evaluation
torchrun --nproc_per_node={n_gpus} --nnodes=1 evaluation_script.py --gpu={gpu_ids} --ipc={ipc} --config_path=../config/{ipc}/{dataset}.yaml --load_path={distilled_dataset.pt}

What you should see at the end of the condense run is a distilled_dataset.pt that the evaluation script accepts through --load_path. The README does not state the filename the script writes by default, so check the condense output directory before scripting anything around it.

Where NCFM will waste your time

The eight-GPU example is not decoration. Both documented condensation examples use --nproc_per_node=8, and the pipeline is built on torchrun, so single-GPU use is outside what the README demonstrates. If you have one card, you are in undocumented territory.

The repository also carries an imagenet_subset/ directory at the top level and the config paths are templated by dataset, but the README's only worked examples are CIFAR-10 and CIFAR-100. Nothing in the README describes how ImageNet-scale data should be laid out on disk or how long any stage takes. Storage and wall-clock planning are left to you.

Reproducibility has a softer edge than the CVPR framing suggests. The pretrain stage is optional because checkpoints are published on Hugging Face, which means two people running "NCFM" can be running against different starting models. The README does not say which checkpoints correspond to which reported numbers.

Finally, this is a paper repository. There are no release tags, no changelog and no documented rollback path. If an upstream dependency moves and a stage breaks, you are reading source, not release notes.

NCFM against trajectory-matching distillation such as MTT

The README's acknowledgement section names MTT, DATM, DC/DM, IDC, SRe2L, RDED and DANCE as inspirations. MTT is the clearest point of contrast because the two solve the same problem from opposite directions.

MTT-style trajectory matching distills by aligning the parameter trajectory a student follows when trained on synthetic data with the trajectory it follows on real data. That requires unrolling training inside the distillation loop, which is why trajectory methods are expensive per distillation step and sensitive to the learning-rate schedule used during unrolling. NCFM instead matches neural characteristic functions under a minmax objective, which the paper frames as the distinguishing idea. The practical difference for a user is what you tune and what you pay for: trajectory matching makes you reason about expert trajectories and unroll length, while NCFM makes you reason about the minmax objective and the pretrained models that feed it.

Choosing between them is not a quality judgement. If your group already has trajectory-matching infrastructure and expert trajectories cached, moving to NCFM means rebuilding the pretrain stage from scratch. If you are starting fresh and the minmax formulation is what you want to extend, NCFM is the reference implementation and MTT is not.

Maintenance, licence and the cost of upgrading

The repository is not archived, and the last push was on 2026-06-03. That is roughly three and a half months before today's date, so the project has seen recent activity, but with no releases retrieved there is no version to pin against and no upgrade path to follow. Your only stable reference is a commit hash, and you should record one.

The dependency set is small, which limits the blast radius of an upgrade but does not eliminate it. torch==2.5.0 and torchvision==0.20.0 are pinned together, and efficientnet_pytorch==0.7.1 is pinned separately; moving any of the three means re-validating all three stages, because the pretrain stage produces the checkpoints the condense stage consumes. An upgrade is not a one-file change.

The repository carries no licence file, and the README states no licence. That is the sharpest practical constraint here: without a licence, the default position is that the authors retain their rights, and neither commercial nor academic reuse is spelled out. The README asks that you cite the paper if you find NCFM useful, which is a citation request, not a licence grant. If you need to ship anything derived from this code, resolve the licensing question with the authors before you build on it. This is not legal advice; it is a statement that the file is absent.

Editorial conclusion

Adopt NCFM if you are reproducing or extending the CVPR 2025 result and you have multi-GPU hardware plus ImageNet-scale storage, because the repository assumes both. Do not adopt it if you need a supported library with versioned releases, a licence statement, or a single-command workflow: the README documents no rollback, no test suite and no release tags, and the repository carries no licence file. Before you commit, verify three things: that the pretrained checkpoints on Hugging Face match the config files you intend to run, that your PyTorch build matches the pinned torch==2.5.0 and torchvision==0.20.0 in requirements.txt, and that the condensed output of condense_script.py actually loads under evaluation_script.py with the --load_path flag for your chosen ipc and dataset.

Frequently asked questions

Official sources

1. gszfwsb/NCFM on GitHub
2. Issues
3. Project website
4. README