llm-datasets: A Field Guide to Post-Training Data, Not a Silver Bullet
Curated list of datasets and tools for post-training.
At a glance
- What is it?
- Maxime Labonne's curated list of datasets and tools for LLM post-training is a useful starting point, but its value depends on how critically you read it.
- Who is it for?
- Adopt llm-datasets if you are building or fine-tuning an LLM and need a quick, human-curated map of recent SFT and reasoning datasets, especially those with permissive licenses. Do not rely on it as a definitive catalog: it omits RLHF and preference data, mixes licenses without consistent flags, and links to datasets that may change or disappear.
- Can I use it commercially?
- Not without permission. GitHub finds no licence file in the repository, and without a licence all rights are reserved by default: you may read the code but not reuse it. Check the README, or ask the authors, before using it.
- Is it still maintained?
- Yes. The repository last received commits 139 days ago.
- What is it written in?
- GitHub does not report a main language for this repository.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What This List Actually Solves
The README addresses a real bottleneck in LLM development: finding good data for supervised fine-tuning. After pre-training, a model needs instruction-following data, and the ecosystem on Hugging Face is vast and noisy. This repository curates a subset of datasets and tools, organized by domain. It is aimed at engineers and researchers who are about to fine-tune a model and want to avoid scanning hundreds of dataset cards manually. The list is not exhaustive, and it is not a benchmark. It is a hand-picked index with notes on size, whether the data includes reasoning traces, and the license when known. For someone new to post-training, it offers a structured entry point. For a veteran, it may surface datasets released in the last few months that they had not seen.
The Three Quality Criteria the Author Pushes
The README opens with a clear stance on what makes a good dataset: accuracy, diversity, and complexity. Accuracy means samples are factually correct and relevant, which the author suggests can be enforced with solvers for math and unit tests for code. Diversity is about covering many use cases to avoid being out of distribution. Complexity covers multi-turn dialogue, multilingual text, and step-by-step reasoning. These three criteria are sensible, but they are also a filter for the curator's own taste. The list favors datasets that include reasoning traces, marked as 'Thinking: Yes' in the tables. That is a bias toward the current fashion for reasoning models. If you are fine-tuning a small model for a narrow task, a simple instruction dataset might serve you better. The README does not discuss that trade-off.
How the Datasets Are Organized and What the Tables Tell You
The repository is structured as a set of markdown tables, each under a category heading. The categories visible in the material are General, Math, Science, and Code. Each row gives the dataset name, a link to its Hugging Face page, the number of samples, a 'Thinking' flag, and notes. The notes often mention the model that generated the data, the license, and related resources. For example, the Nemotron-Cascade-2-SFT-Data entry notes that responses were generated from DeepSeek-V3.2, GPT-OSS-120B, and Qwen3. That kind of provenance is useful when you need to match data to your base model. The sample counts are rounded, and the dates in parentheses indicate when the dataset was released or updated. The list appears to be updated frequently; the last push to the repository was in April 2026, and some entries are dated March 2026. That freshness is a strength, but it also means the list is a moving target.
Getting It Running: There Is No Code to Run
This is not a software project in the usual sense. There is no installation, no command-line tool, no configuration file. The repository is a document, so 'getting it running' means opening the README on GitHub and following the links. Each dataset link goes to a Hugging Face page where you can download the data or load it with the datasets library. For example, to load a dataset you would use a command like `load_dataset('nvidia/Nemotron-Cascade-2-SFT-Data')`, but that command is not in the README. The README itself contains no code snippets or usage examples. That is fine for a curated list, but it means the project does not provide any tooling to filter, mix, or deduplicate the datasets it lists. You have to bring your own pipeline. If you were hoping for a ready-made data preparation workflow, this is not it.
Licenses: A Patchwork That Needs Your Attention
The README states that unless specified otherwise, all datasets listed are under permissive licenses like Apache 2.0, MIT, or CC-BY-4.0. But the tables show exceptions. The Dolci-Instruct-SFT entry is marked CC-BY-NC-4.0, which is non-commercial. The MegaScience entry is CC-BY-NC-SA-4.0, which adds share-alike and non-commercial terms. Nemotron-Math-Proofs-v1 is CC-BY-SA-4.0, which is share-alike but allows commercial use if you share derivatives under the same license. The Nemotron datasets from NVIDIA use the NVIDIA Open Model License, which is not a standard open-source license and has its own restrictions. The README does not explain these differences. It simply flags them in the notes. That is a responsible approach, but it places the burden on you to read each license carefully. If you are building a commercial product, a dataset marked CC-BY-NC is a dealbreaker unless you pay for a different license. The list does not tell you which datasets are safe for commercial use. You have to check each one.
What Is Missing: Preference Data and RLHF Are Absent
The README's focus is explicitly on supervised fine-tuning, which is one stage of post-training. It does not cover reinforcement learning from human feedback or preference optimization, which are also post-training steps. The title says 'post-training,' but the content is narrower. The only mention of RL is in the Nemotron-Cascade-2 entry, which points to a separate RL dataset. That is a genuine limitation. If you are working on RLHF or DPO, this list will not help you find preference pairs or reward model data. The author might have a separate list for that, but it is not linked here. So the scope is SFT and reasoning data, not the full post-training pipeline. That is worth knowing before you bookmark it as a general resource.
The Alternative: Going Directly to Hugging Face and Papers
The obvious alternative is to search Hugging Face yourself, using filters for dataset size, license, and task. That is what the curator did to build this list. The difference is that llm-datasets adds a layer of human judgement: the notes tell you which model generated the data, whether it includes verified reasoning traces, and whether it was used to train a known model. For example, the NuminaMath-CoT entry notes that it was used to win the first progress prize of the AI Math Olympiad. That kind of context is hard to get from a raw search. Another alternative is to follow the papers and blog posts that the datasets come from, such as the SYNTHETIC-2 release post or the MathX-5M page. Those sources give you deeper detail on filtering methodology and ablation studies. The trade-off is time. The curated list saves you from scanning dozens of dataset cards, but it gives you less depth than reading the original paper. For a quick survey, the list wins. For a deep dive, go to the source.
Maintenance and Upgrade Cost: It Is a Living Document, Not a Dependency
Because this is a markdown file, the maintenance cost is low for you. There is no code to update, no API to track. The risk is that the list becomes stale or links break. The repository was pushed to in April 2026, and several entries are dated March 2026, so it appears actively maintained. But you cannot rely on that forever. The author may stop updating it, and some Hugging Face datasets get taken down or replaced. The README itself notes that some entries are 'cleaned' versions, like the orca-agentinstruct-1M-v1-cleaned, which suggests a curation process that may not be reproducible. There is no versioning scheme for the list itself. If you use it as a reference, you should check the commit history or the last push date to see how fresh it is. The license of the repository itself is not stated in the material, so you should assume the text is under the author's copyright unless you check the GitHub repo for a LICENSE file. That is a minor point, but it matters if you plan to copy the list into your own documentation.
Who Should Use This and Who Should Not
Use this list if you are fine-tuning a model and need a broad survey of what is available in math, code, science, and general instruction following. It is especially useful if you want datasets with reasoning traces, because the 'Thinking' column lets you filter for that. Do not use it as your only source of data. The list does not evaluate quality beyond the author's own criteria, and it does not compare datasets on downstream performance. If you are working on a narrow domain like legal or medical text, you will not find specialized datasets here. If you need preference data for RLHF, look elsewhere. Before you commit to any dataset, verify the license, check the actual sample count on the Hugging Face page, and run a small experiment to see if the data matches your model's style. The list is a map, not a destination.
Editorial conclusion
Adopt llm-datasets if you are building or fine-tuning an LLM and need a quick, human-curated map of recent SFT and reasoning datasets, especially those with permissive licenses. Do not rely on it as a definitive catalog: it omits RLHF and preference data, mixes licenses without consistent flags, and links to datasets that may change or disappear. Before using any dataset, verify its license on the Hugging Face page, check the exact sample count and format, and test a small sample against your own quality criteria. The list is a starting point for exploration, not a substitute for your own data diligence.
Community notes