Croissant: A JSON-LD Format That Makes ML Datasets Loadable Without Custom Glue Code
Croissant is a high-level format for machine learning datasets that brings together four rich layers.
At a glance
- What is it?
- MLCommons Croissant layers metadata, file descriptions, structure and ML semantics into one JSON-LD document built on schema.org. The mlcroissant Python package turns those documents into iterable records, but the format's value depends on dataset owners actually publishing conformant files.
- Who is it for?
- Adopt Croissant if you publish datasets that others must load, or if you consume many small datasets and want one loader instead of one script per source. Do not adopt it if your pipeline already has a stable internal manifest and no external consumers, because you would be maintaining a second description of the same files.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 63 days ago.
- What is it written in?
- Mainly Jupyter Notebook, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem Croissant addresses: every dataset ships its own loading convention
The README states the motivation directly: datasets are the source code of machine learning, but each one has a unique file organization and its own method for translating file contents into data structures, so using a new dataset requires a novel approach every time. That is a tooling problem more than a storage problem. A CSV, a directory of JPEGs and a Parquet shard set can all hold the same logical table, and nothing in the file tells a loader which column is the label, which field is an image reference, or how records are split.
Croissant's answer is to describe all of that in one JSON-LD file that sits next to the data. The README frames it as four layers: metadata (including responsible ML aspects), resources (the files or other sources holding raw data), structure (how raw data becomes data structures), and ML semantics (how the data is typically used). The intended audience is anyone who has to move a dataset between tools: dataset publishers, paper reviewers accepting dataset submissions, and engineers wiring external data into a training loop.
The format builds on schema.org's Dataset vocabulary, which is already how datasets are described on the open web. That choice matters because it means a Croissant file is also a valid schema.org document, so existing crawlers can read it. The README notes that Google's Datasets Search crawls and indexes Croissant JSON-LD files and offers a filter to restrict results to Croissant datasets, and that Kaggle embeds Croissant JSON-LD directly in its dataset pages.
How the four layers map onto actual JSON-LD nodes
The minimal example in the README shows the mechanism concretely. A top-level node typed sc:Dataset carries name, description, license, url and a conformsTo value of http://mlcommons.org/croissant/1.0. That is the metadata layer plus the version declaration.
The resources layer appears as a distribution array of cr:FileObject entries. Each FileObject has an @id, a contentUrl, an encodingFormat such as text/csv, and a sha256 digest. The digest is part of the format, not an optional extra, which means a loader can verify that the bytes it fetched match what the publisher described.
The structure layer is the recordSet array. A cr:RecordSet named examples holds a field list, and each cr:Field declares a name, a description, a dataType drawn from schema.org types such as sc:Text or sc:Integer, and a source object. The source points back at a FileObject by @id and gives an extract instruction, in this case a column name. So the data flow is: FileObject identifies bytes, Field says which slice of those bytes belongs to a logical column, RecordSet groups fields into a table, and the loader joins them at read time.
The ML semantics layer is the least visible in the minimal example, since it appears mainly through typed fields and record set naming rather than a separate block. The README lists it as a distinct layer but does not show a worked example of, say, a label declaration or a split definition in the excerpt provided. That is a real gap for anyone trying to write a conformant file from the README alone; the specification and the datasets folder in the repository are where those details live.
Installing mlcroissant and iterating records
The reference implementation is a Python package, mlcroissant, requiring Python 3.10 or later. Installation is a single command:
pip install mlcroissant
The README's first example loads a remote Croissant file and prints records:
import mlcroissant as mlc ds = mlc.Dataset("https://raw.githubusercontent.com/mlcommons/croissant/main/datasets/1.0/gpt-3/metadata.json") metadata = ds.metadata.to_json() print(f"{metadata['name']}: {metadata['description']}") for x in ds.records(record_set="default"): print(x)
Two things stand out. First, Dataset accepts a URL or a local path, so the same call works for a published file and a file you are drafting. Second, records() takes a record_set argument by name, which is the cr:RecordSet name from the JSON-LD. If the name does not match, you get nothing back rather than an error about the schema, which is a sharp edge worth knowing about when you first point the loader at a hand-written file.
The README also points to notebook recipes under python/mlcroissant/recipes for more examples. The repository is primarily Jupyter Notebook by language, which is consistent with a project whose documentation is largely executable examples rather than a long prose manual.
The TensorFlow Datasets path and what it costs you
The second README example shows the integration that makes the format useful in a real training loop. You point a CroissantBuilder at a Croissant URL, name the record sets you want, and pick an output file format:
import tensorflow_datasets as tfds builder = tfds.core.dataset_builders.CroissantBuilder( jsonld=url, record_set_ids=["fashion_mnist"], file_format='array_record', ) builder.download_and_prepare() train, test = builder.as_data_source(split=['default[:80%]', 'default[80%:]'])
The URL in the example is a Hugging Face endpoint that serves Croissant JSON-LD for a dataset, which shows the intended pattern: a host exposes the metadata at a predictable path and any Croissant-aware consumer can pick it up.
The cost is a materialization step. download_and_prepare() fetches the source files and writes them into the chosen format, here array_record. That is a real conversion, not a zero-copy view, so you pay disk space and preparation time proportional to the dataset. The split syntax is TFDS's own slicing notation, not something Croissant defines, so split semantics live in the consumer rather than in the format. If you need splits to be portable across loaders, the Croissant file is not where that guarantee comes from.
Where Croissant is the wrong tool
The format describes datasets that already exist. It does not restructure them. If your data is a stream, a database behind an API, or a set of files whose contents change between reads, a static JSON-LD description with sha256 digests will go stale the moment the bytes change. The digest that makes the format trustworthy is also what makes it brittle for mutable sources.
There is a second limit. The README describes Croissant as currently under development by the community, and the specification, examples and verifier are listed under the GitHub repo while the requirements document and responsible AI approach sit in a shared drive. That split means the normative material is not all in one place, and a reader who only has the README cannot resolve every question about conformance.
Third, the loader's behaviour on malformed files is not documented in the excerpt. The minimal example is deliberately small; real datasets with multiple FileObjects, joins across record sets and typed fields will exercise paths the README does not cover. If you are generating Croissant files programmatically, budget time for the verifier rather than assuming the loader will tell you precisely what is wrong.
Finally, if you are the only consumer of your own dataset and your pipeline is stable, writing a Croissant file is duplicate work. You would be maintaining a description of files that your code already knows how to read, and every schema change means editing two places.
Alternatives: TFDS builders, Hugging Face datasets, and plain schema.org
The nearest alternative is a hand-written TensorFlow Datasets builder. A TFDS builder is executable Python: it defines splits, downloads, and feature dictionaries in code. Croissant's difference is that the description is declarative JSON-LD, so it can be produced by a publisher who does not ship Python, read by a crawler, and consumed by more than one framework. The trade-off runs the other way too: a TFDS builder can express arbitrary preprocessing, while a Croissant file describes extraction from files and leaves transformation to the consumer.
Hugging Face datasets occupy a similar slot with a different mechanism. Their loading scripts are Python hosted alongside the data, and the README's own example uses a Hugging Face endpoint as the source of a Croissant file, which suggests the two are complementary rather than exclusive: a hub can serve Croissant metadata for datasets it already hosts.
Plain schema.org Dataset markup is the third option, and it is the one Croissant is explicitly extending. A schema.org description tells a search engine what a dataset is about. It does not say which CSV column holds the label or what the sha256 of the file is. Croissant adds the cr:FileObject, cr:RecordSet and cr:Field vocabulary on top of the schema.org base, which is why the README can claim compatibility with existing web indexing while adding machine-readable structure.
Maintenance, releases and licence
The project ships tagged releases, with v1.1.0 dated 2026-04-16, following v1.0.22 in 2025-08-25 and v1.0.21 in 2025-07-29. The version string in a Croissant file is declared explicitly through conformsTo, shown as http://mlcommons.org/croissant/1.0 in the example, so a file states which revision of the format it targets rather than floating on whatever the loader supports. That is the right design for a format, and it means upgrading mlcroissant does not silently change how an existing file is interpreted.
The repository is Apache-2.0 licensed. That is a permissive licence, which matters if you plan to embed the loader or copy the example files into a commercial pipeline. It says nothing about the licence of the datasets you describe with Croissant; each dataset carries its own license field, and the minimal example sets it to a Creative Commons URL. The format separates those two concerns, and you should not read the repository's licence as covering the data it points at. This is a description of what the licence identifier means in the repository, not legal advice.
Contributing requires signing the MLCommons Association CLA, per the README's getting involved section. If your organisation has rules about contributor agreements, that is a gate to check before planning patches. Maintenance cost for a consumer is low: the loader is a pip dependency and the format is versioned. Maintenance cost for a publisher is ongoing, because every change to the underlying files invalidates the sha256 digests and the field extraction rules.
Editorial conclusion
Adopt Croissant if you publish datasets that others must load, or if you consume many small datasets and want one loader instead of one script per source. Do not adopt it if your pipeline already has a stable internal manifest and no external consumers, because you would be maintaining a second description of the same files. Before committing, verify that the dataset you care about has a conformant Croissant file with sha256 checksums on every FileObject, that the recordSet fields map to real columns, and that mlcroissant can iterate your record_set without errors. The format is only as useful as the files the community publishes.
Community notes