segmenteverygrain pairs a small U-Net with SAM 2.1, and then asks you to clean up the result
A SAM-based model for instance segmentation of images of grains
At a glance
- What is it?
- segmenteverygrain is a Python package that finds grains in photographs of sediment for grain size and shape analysis, using a patch-based U-Net to propose prompts that SAM 2.1 then refines, with an interactive editing class for quality control and a fine-tuning path for new image types. It suits a geomorphologist or sedimentologist analysing thin sections and beach samples, and it assumes you have a GPU and a tolerance for manual correction.
- Who is it for?
- segmenteverygrain fits a researcher who needs grain outlines good enough for size and shape measurements and is prepared to correct them, since the interactive editing and fine-tuning paths assume a person in the loop. It does not fit an unattended batch pipeline over thousands of images, because the first-pass segmentation misses grains and the workflow is built around fixing that by hand.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 70 days ago.
- What is it written in?
- Mainly Jupyter Notebook, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on October 23, 2026, and from our analysis. They are not legal advice.
Editorial analysis
Why a U-Net is in front of SAM at all
The design of segmenteverygrain only makes sense once you know what SAM is bad at, and the README explains the problem in detail.
The package relies on SAM 2.1, the Segment Anything Model 2.1 developed by Meta, for getting high-quality outlines of the grains. But SAM requires prompts for every object detected, and when used in everything mode it tends to be slow and results in many overlapping masks and non-grain, or background, objects.
So the architecture is a two stage pipeline. A Unet-style, patch-based convolutional neural network creates a first-pass segmentation, and that first pass is used to generate prompts for the SAM-based segmentation.
The README is honest about the trade: some of the grains will be missed with this approach, but the segmentations that are created tend to be of high quality.
That is a specific and defensible position. A prompt-based segmentation model is excellent at tracing the boundary of an object you point it at and poor at deciding what the objects are. The U-Net supplies the what and SAM supplies the where exactly, and the cost is that the candidate list is only as good as the network proposing it.
The goal is stated in terms of the science rather than the model: to detect most of the grains in a photo so that it will be useful for determining grain size and grain shape, a common task in geomorphology and sedimentary geology.
Installing the whole thing, including the trained U-Net weights
There are three routes in, and they suit different situations.
The quickest way to try the package without installing anything is to run the Segment_every_grain_colab.ipynb notebook in Google Colab. The README notes that this notebook has been adjusted so the segmentation can be tested in Colab, and also flags that the interactivity there is reduced.
For local use the recommendation is to clone the repository, because it contains the trained U-Net model and the example images, and then create a conda environment from the provided environment files. If you do not have conda, install miniforge first.
git clone --depth 1 https://github.com/zsylvester/segmenteverygrain.gitThe environment file differs by platform.
conda env create -f segmenteverygrain/environment.ymlconda env create -f segmenteverygrain/environment_macos.ymlThe first is for Linux and Windows, the second for Mac on Apple Silicon. Creating the environment also installs the segmenteverygrain package and JupyterLab, which saves a separate install step.
Then you activate it and launch JupyterLab from the repository folder.
conda activate segmenteverygrain
cd segmenteverygrain
jupyter labThe third route is for using the library as a dependency in an environment you already have, without the notebooks and model files.
pip install segmenteverygrainThat last route is the one to be careful with, and the README says why. The trained model lives in the repository, so a pip install gives you the code but not the weights, and the repository files also carry environment.yml, environment_macos.yml, a requirements.txt and a setup.py.
Requirements are Python 3.10 or higher, and the dependencies cover a wide spread: TensorFlow and PyTorch both appear, along with SAM 2, scikit-image, opencv-python, rasterio, shapely, geopandas, rtree, pandas and keras.
Running the notebook and where the 860 MB comes from
The walkthrough lives in notebooks/Segment_every_grain.ipynb, and the README describes what it covers: loading the models, running the segmentation, interactively updating the result using the GrainPlot class, and saving the grain data and the mask.
The first thing to expect is a download. The instructions say to open the notebook and run the cells from the top, and that the first cells download the SAM 2.1 model checkpoint, roughly 860 MB, automatically.
That number is the practical headline for anyone planning this work. The SAM 2.1 weights are a large fixed cost that every user pays, and they are fetched at runtime rather than bundled, so the first run of the notebook on a fresh machine is a download rather than a segmentation.
The example images that ship in the repository are a good indication of the intended input. The examples/ directory contains a thin-section image set from the Digital Rocks Portal, a miocene sandstone example, a Barton Creek set, a Torrey Pines beach set, an interactive_edit folder and an unet_training folder, alongside a single torrey_pines.jpg. The Digital Rocks Portal images are described in the README as a relatively large thin-section image of a sandstone, segmented in two halves.
Segmentation of a full thin section is the case the tool is built for, and the tiling in the patch-based U-Net is what makes a large image tractable.
GrainPlot, and the editing that makes the output usable
The interactive interface is the part of the workflow that determines whether you get publishable numbers, and the README lists its controls in full.
Left-clicking on a grain-free area creates a grain instantly, and left-clicking on an existing grain selects or unselects it. Alt plus Left-click places a foreground prompt for multi-prompt grain creation, and Alt plus Right-click places a background prompt. Holding Shift and dragging draws a scale bar for unit conversion, which is how the segmentations get converted from pixels to millimetres.
The single-key actions are: c to create a grain from the placed prompts, d to delete the selected grains, m to merge the selected grains, z to undo the last created grain, h to toggle the coverage mask which highlights unsegmented areas in red, Esc to clear all selections and prompts, and holding Ctrl to temporarily hide the grain masks.
Two of those deserve emphasis. The coverage mask on h is the tool for finding what the first pass missed, which is exactly the failure mode the README admits to. And Shift-drag for the scale bar is not a display nicety, because grain size analysis is meaningless without a length reference.
Beyond the interactive editing, the package includes functions for deleting and merging objects by clicking, and for adding grains that were not segmented automatically. The QC-d masks can be saved and added to a dataset of grain images, and those images can then be used to improve the Unet model. Many of the images in that dataset come from the sedinet project.
That closes a loop: the corrections you make while quality checking become training data for the next version of the first-pass model.
Fine-tuning is the answer to a new image type
The README addresses the obvious failure mode directly. If the base Unet model does not work well on a specific type of image, it is a good idea to generate some new training data, a few small images are usually enough, and to fine tune the base model so that it works better on the new image type.
That is done by running the cells in the last section of the Segment_every_grain.ipynb notebook, labelled Finetuning the base model. The unet_training/ example folder and the tests/ directory are the other places to look for how the training data and evaluation are set up.
The shape of this is unusual and worth spelling out, because it inverts the usual assumption about machine learning tooling. You are not expected to train a segmentation model from scratch, and you are not expected to have a large labelled dataset. You run a pretrained model, you fix its mistakes by hand on a handful of images, and you use those corrections to adapt the small first-pass network for your specific rock, sediment or sand sample.
The release history shows the project moving at a steady pace. Version 0.5.0 came on 2026-08-13 with area-weighted grain size distributions, v0.4.0 on 2026-04-24 with a U-Net only mode and training fixes, and v0.3.0 on 2026-02-09 with the SAM 2.1 migration. The last push was on 2026-08-14, and the repository is not archived.
The U-Net only mode from v0.4.0 is a sensible escape hatch for anyone who wants the first-pass result without the SAM refinement, at the cost of the outline quality that SAM provides.
Grain extraction and clustering for the classification step
Once the grains are segmented, the grain_utils module handles the next stage, and this is the part that turns outlines into data.
It can extract grain images, cropping and normalizing individual grains from segmented images. It does feature extraction with pre-trained CNNs, namely VGG16, ResNet50 and InceptionV3, or with colour features. It clusters similar grains using K-means, DBSCAN or hierarchical clustering. And it provides interactive labelling through ClusterMontageLabeler, for manually labeling grains by category, and quality control through ClusterMontageSelector to remove unwanted grains or clusters.
The design is sensible in a specific way. Once you have several thousand grain outlines, sorting them by eye is the bottleneck in any morphological study, and the clustering step turns that into a labelling task on a montage of representative examples rather than a task on every grain.
The setup.py shows the geospatial side of the dependency list: rasterio, shapely, geopandas and rtree are there because grain data is spatial data, and pandas and scikit-learn carry the tabular and clustering work. The package also depends on opencv-python, scikit-image, networkx, tqdm, pillow, scipy, numpy and matplotlib.
The package metadata is honest about its audience, with an Intended Audience of Science/Research classifier and the keywords sedimentology, geomorphology, grain size and segment anything model.
Where segmenteverygrain is the wrong choice
Three situations argue against it.
The first is an unattended pipeline. The architecture is explicitly a first pass plus manual correction. Some grains will be missed, and the interactive editor exists to add them. A project that wants to segment ten thousand images without a person in the loop will find that the error rate, not the speed, is the limiting factor, and there is no accuracy claim in the README to plan against.
The second is an image type far from the training distribution, where you cannot spare the iteration. Fine-tuning is described as needing only a few small images, but that still means segmenting, correcting, saving and retraining, and the README says the base model works well on the grain types it was trained on. A completely different sediment type, or an image with a different scale or contrast, means that loop runs before you get usable results.
The third is hardware. The dependency stack includes both TensorFlow and PyTorch plus SAM 2, the checkpoint is around 860 MB, and the project is a research package with a paper in the Journal of Open Source Software. The installation guide referenced from the README has per-platform GPU notes, which tells you the model is expected to run on a GPU.
The alternative in the same direction is to use SAM, or SAM 2, directly with prompts you generate from your own detector, or to use a classical segmentation approach such as watershed on a thresholded image, which needs no model at all. The difference in approach is that segmenteverygrain combines a learned first pass with a promptable segmenter and adds the interactive correction and the clustering tools, so it saves you building the U-Net, the prompt generation and the quality control loop. What you give up is control over the pipeline and the guarantee that a given image type works, and for a sedimentologist measuring grain size for a publication, the correction step is a feature rather than a defect.
Editorial conclusion
segmenteverygrain fits a researcher who needs grain outlines good enough for size and shape measurements and is prepared to correct them, since the interactive editing and fine-tuning paths assume a person in the loop. It does not fit an unattended batch pipeline over thousands of images, because the first-pass segmentation misses grains and the workflow is built around fixing that by hand. Install it from the repository with the conda environment files so you get the trained U-Net weights, run `Segment_every_grain.ipynb` from the top to download the SAM 2.1 checkpoint, and use `pip install segmenteverygrain` only when you have already sorted out the environment.
Frequently asked questions
What does segmenteverygrain do?
It is a Python package that detects grains, or grain-like objects, in images so that grain size and grain shape can be determined, which is a common task in geomorphology and sedimentary geology. It uses a patch-based U-Net to make a first-pass segmentation and SAM 2.1 to produce the high-quality outlines.
How do I install segmenteverygrain?
For local use, clone the repository because it contains the trained U-Net model and example images, then create a conda environment with `conda env create -f segmenteverygrain/environment.yml` on Linux and Windows or environment_macos.yml on Apple Silicon, and launch JupyterLab from the repository folder. For use as a dependency only, `pip install segmenteverygrain` is available on PyPI.
Why does SAM need a U-Net in front of it in segmenteverygrain?
SAM requires prompts for every object detected and in everything mode it tends to be slow and produces many overlapping masks and background objects. The U-Net creates a first-pass segmentation that is used to generate the prompts, so some grains are missed but the outlines that are produced are of high quality.
How do I fix grains that segmenteverygrain missed?
The GrainPlot interface supports left-clicking a grain-free area to create a grain instantly, Alt plus Left-click and Alt plus Right-click to place foreground and background prompts, and c to create a grain from those prompts. Pressing h toggles a coverage mask that highlights unsegmented areas in red, and Shift-drag draws a scale bar for unit conversion.
Can segmenteverygrain be fine-tuned for a new image type?
Yes. If the base U-Net model does not work well on a specific type of image, the README advises generating some new training data, a few small images are usually enough, and fine tuning the base model. This is done in the last section of the Segment_every_grain.ipynb notebook, and the QC-d masks can be saved into a dataset of grain images for that purpose.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/zsylvester-segmenteverygrain)Was this useful?