split-folders: deterministic train/val/test splits for image and annotation folders

The directory-shuffling problem split-folders addresses

Most tutorials assume your images already sit in train, val and test folders. In practice you have one folder per class and no split. Writing that split by hand is where people introduce silent bugs: the same image lands in both train and val, an annotation file gets separated from its image, or the split is different every time the script runs. split-folders exists to make that step one function call with a seed. The README describes the input as either a class-based tree (input/class1/img1.jpg) or a flat directory of files, and the output as train, val and test subfolders that mirror the class structure. The intended user is someone preparing a supervised learning dataset from a folder of files, not someone building a data platform. The package is Python only with no external dependencies, and tqdm is an optional extra for a progress bar when files are moved.

How ratio, fixed and kfold divide the files

There are three entry points, and they answer different questions. splitfolders.ratio takes a tuple such as (.8, .1, .1) and allocates by proportion; a two-element tuple gives train and val only. splitfolders.fixed takes a count per set, for example (100, 100), and the README notes you can pass three values to also cap the training set. There is a third mode, fixed="auto", documented alongside oversampling: it computes the validation size from the smallest class, allocating roughly 20 percent of that class to validation and the rest to training. splitfolders.kfold produces k folds for cross-validation, where each fold directory contains train and val subdirectories, and it uses symlinks by default to avoid k times the disk usage. The README states files are shuffled by default and that shuffle=False exists for time series data, where order carries meaning. A seed parameter makes the split reproducible. The README's own guidance on choosing between the modes is blunt: use ratio when the dataset is balanced, otherwise fixed.

Grouping is the part that matters for paired data

If each sample is one image, splitting is trivial. If each sample is an image plus an annotation, a naive split breaks your dataset. split-folders offers four ways to keep groups together, and they are not equivalent. group_prefix is the legacy path: you pass the number of files per group, such as 2, and files are grouped by filename stem, with the requirement that all stems have exactly that many files. group="stem" does the same job without you declaring the count; it discovers the group size and requires every stem to have the same count. The README notes that if every stem has a single file, group="stem" behaves identically to no grouping at all, which makes it a safe default to leave in place. group="sibling" is the one for parallel directory layouts, where images live in one directory and annotations in another; it splits all subdirectories in lockstep and matches files across them by stem. The stated requirements are that the input has at least two subdirectories, every stem exists in every subdirectory, and it cannot be combined with oversample=True. A callable can be passed as group for cases the built-in modes do not cover. The strictness here is a feature: mismatched stems fail loudly rather than producing a train set with orphaned labels.

Oversampling, and why it only touches train

For imbalanced class distributions, split-folders can randomly oversample. The README is explicit that oversampling is off by default and that it is applied only to the train folder, on the reasoning that duplicates in val or test would be cheating. That is the correct default and worth stating plainly, because pipelines that oversample before splitting leak duplicates across the boundary and inflate validation scores. Two constraints follow from the implementation as described. Oversampling is not available with flat directories, since there are no classes to balance. And group="sibling" cannot be combined with oversample=True, so a parallel-directory dataset with heavy class imbalance cannot use both features at once. If you need both, you would have to oversample the images and annotations yourself before running the split, which means writing the pairing logic the package was going to handle for you.

Getting it running: install and the actual calls

Installation is a single pip command, pip install split-folders, or pip install split-folders[full] to pull in tqdm for the progress bar. The module API is four functions. splitfolders.ratio("input_folder", output="output", seed=1337, ratio=(.8, .1, .1), group_prefix=None, group=None, formats=None, move=False, shuffle=True) shows the documented defaults. splitfolders.fixed("input_folder", output="output", seed=1337, fixed=(100, 100), oversample=False, group_prefix=None, group=None, formats=None, move=False, shuffle=True) is the counted variant. splitfolders.kfold("input_folder", output="output", seed=1337, k=5, group_prefix=None, group=None, formats=None, move="symlink", shuffle=True) is the cross-validation variant, and note that move defaults to symlink there rather than False. The README also states the package works as a CLI, but the extracted material shows only the Python calls, so the exact CLI flags are not something I can quote. The formats parameter is listed in every signature and described as splitting by file format, but the README excerpt does not show a worked example of its values, so check the source before relying on it.

Where split-folders stops being the right tool

The package operates on the filesystem. It copies or moves files, and the output is a second directory tree that duplicates your data. There is no manifest, no hash of what went where, and no record of which seed produced which split beyond what you write down yourself. If your dataset is stored in object storage, a database, or a format like WebDataset or TFRecord shards, this tool does not address your problem; you are splitting archives, not folders. The kfold default of symlinks is a second boundary. Symlinks are cheap, but they break when the output tree is copied to another machine, archived, or read by a loader that resolves paths strictly. The README lists move as a parameter on the other functions too, so the copy-or-move decision is yours to make, and moving is destructive if you get the output path wrong. Finally, group_prefix and group="sibling" both impose an all-or-nothing consistency rule on stems. Real datasets often have one class with a missing annotation. Under these modes that class fails, and the fix is to clean the data first, not to relax the flag.

How it compares with writing the split yourself

The obvious alternative is twenty lines of pathlib and random.shuffle. That approach is fine for a flat folder of images and a single train/val split, and it costs nothing to maintain. The difference appears as soon as any of three conditions hold: samples are multi-file, file types live in parallel directories, or you need the same split reproduced across machines. A hand-rolled script typically shuffles a flat list of paths and slices it, which silently separates image from annotation unless you write the grouping logic yourself, and it usually has no seed discipline. split-folders encodes the grouping rules and the seed as first-class parameters, and it fails loudly when stems do not line up. The trade-off is that you inherit its assumptions about directory structure. If your layout does not match the class-based or flat patterns in the README, the sibling mode or a custom callable is the escape hatch, and beyond that you are back to writing it yourself.

Maintenance, licence and what to check before adopting

The package is MIT licensed, which permits commercial and closed-source use with the usual requirement to retain the copyright and licence notice; that is a summary of the identifier, not legal advice, and you should read the LICENSE file in the repository for the binding terms. Maintenance looks active: the repository is not archived, the last push is dated 2026-04-16, and the release list shows 0.6.1 on 2026-01-28 following 0.6.0 and 0.5.1 earlier the same day. The version numbering still sits below 1.0, so treat the API as subject to change between minor releases, particularly around the newer kfold and grouping options. The package has no external dependencies, which keeps the upgrade surface small; the only optional one is tqdm. Before adopting, run the split on a copy of your data and check two things by inspection: that no stem appears in more than one of train, val and test, and that every file in your input appears exactly once in the output. Those two checks catch the failure modes this tool is designed to prevent, and they are cheap to run.

Editorial conclusion

Adopt split-folders if your dataset is already a directory tree of files and you need a reproducible train/val/test partition, especially when images and annotations must stay paired. Do not adopt it as a data versioning or manifest system: it moves and copies files, it does not record provenance, and the kfold function defaults to symlinks, which many training pipelines and archive tools handle poorly. Before committing, verify three things on your own data: that every stem has the same file count when you pass group="stem" or group="sibling", that your input is not a flat directory if you intended to oversample, and that downstream code can follow symlinks if you use kfold.

Official sources

1. Issues
2. jfilter/split-folders on GitHub
3. License: MIT
4. README
5. Releases