Bespoke Curator: a Python library for building synthetic data pipelines
Synthetic data curation for post-training and structured data extraction
At a glance
- What is it?
- Curator wraps LLM calls, caching, resumption and batch APIs behind a decorator so you can turn a prompt plus a dataset into training or extraction data. It fits teams already comfortable writing Python and paying for hosted inference, and it is a poor fit if you want a no-code pipeline or a self-contained offline generator.
- Who is it for?
- Adopt Curator if your team writes Python, already calls hosted models or runs vLLM, and wants caching and resumption handled for you rather than rebuilt per project. Do not adopt it if you need a no-code interface, if your data cannot leave your infrastructure and you have no vLLM deployment, or if you only need a handful of completions that a plain SDK loop would cover.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 14 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap Curator fills between a prompt and a training set
Most teams do not fail at writing a prompt. They fail at running that prompt over tens of thousands of rows without losing work, without paying twice for the same completion, and without hand-rolling retry logic per provider. Curator targets that middle layer. The README describes it as a way to create synthetic data pipelines for training a model or extracting structured data, and the topic list on the repository names the same territory: instruction tuning, synthetic dataset generation, prompt engineering. The audience is a Python developer who has a dataset, a model endpoint, and a need to produce a larger or cleaner dataset from the two. It is not aimed at someone who wants a hosted labelling UI, and it is not a data versioning system. The library assumes you bring your own prompts and your own model access.
How the pipeline is put together: decorators, caching, and a viewer
The library exposes a Python API rather than a config file. You write a function that takes a row and returns a prompt, decorate it, and run it over a dataset. The stated feature set covers structured outputs, asynchronous execution, caching, and fault recovery, with inference routed through LiteLLM, vLLM, or provider batch APIs. That routing choice matters: LiteLLM is the path to many hosted providers, vLLM is the path to a model you host yourself, and the batch APIs are the path to lower per-token cost at the price of latency. Caching and resumption are the parts that change how you work day to day, because a failed run does not force you to regenerate everything. A separate viewer process lets you inspect rows while generation is still in progress, which is the difference between noticing a bad prompt after 200 rows and noticing it after 20,000. Code execution is handled by a CodeExecutor component with four backends named in the changelog: local (implemented with multiprocessing), Ray, Docker, and e2b. That is a meaningful design decision, since model-generated code is not something you want running in your main process by default.
Installing Curator and pointing it at a model
The README gives a single install line: pip install bespokelabs-curator. The package name on PyPI is bespokelabs-curator, which differs from the import name and from the repository name, so a requirements file that says curator will not resolve. Beyond installation, the repository points at its documentation site for getting started, tutorials, how-to guides and an API reference for the LLM interface, and the examples directory carries the runnable code. The changelog entries name specific integrations worth reading before you start: examples/fireworks for managed supervised fine-tuning with a FireworksTrainer, and examples/poem_finetuning_example.py for a Tinker-based LoRA path. The README also references examples/blocks/raft for a retrieval-augmented fine-tuning workflow. If you need the exact decorator signatures, dataset arguments and configuration keys, they live in the docs rather than in the README, so budget time for that reference page before writing your first pipeline.
Where Curator is the wrong tool
Curator sits on top of an inference provider, so it inherits that provider's cost, rate limits and failure modes. If your prompts are cheap and your dataset is small, the caching and resumption machinery buys you very little, and a short script over an SDK is easier to debug. The batch path is the clearest example of a trade-off rather than a free win: batch APIs reduce token cost but add turnaround time, which is fine for overnight dataset construction and wrong for an interactive loop. The CodeExecutor backends also imply a decision the library cannot make for you. Local multiprocessing is the cheapest and the least isolated; Docker and e2b add setup and, in the e2b case, an external dependency. Running generated code on the local backend in a shared environment is a choice you should make deliberately. Finally, the version history shows a project still on 0.x releases, with the most recent listed release dated 2026-03-15 and the previous one in July 2025. Treat the API as moving and pin your version.
What you would use instead, and how the approach differs
The closest general-purpose alternative for this job is a workflow orchestrator such as Prefect or Dagster, which models a pipeline as a graph of tasks with retries, scheduling and observability built around the graph rather than around LLM calls. With an orchestrator you write the model call yourself and get scheduling, backfills and alerting for free, but you also own prompt templating, structured output parsing, response caching keyed on prompt content, and the viewer. Curator inverts that: it gives you the LLM-specific pieces and leaves orchestration to you or to a scheduler you already run. The other realistic alternative is no library at all, just the provider SDK plus a local cache keyed on a hash of the prompt. That is genuinely sufficient for a few thousand rows and one provider, and it is worth being honest that Curator's value grows with dataset size, provider count and the frequency of failed runs.
Licence, maintenance and the cost of staying current
Curator is Apache-2.0, which permits commercial use and modification and includes a patent grant, with the usual obligations around preserving notices and stating changes. That is a permissive licence, and it is compatible with the way most teams would vendor or wrap the library. This is not legal advice; check the full text if you plan to redistribute a modified version. On maintenance, the material shows a steady release cadence through 2025 into 2026 and a recent push to the main branch, so the project is active. The practical cost is upgrade churn. A 0.x library with integrations for LiteLLM, vLLM, Tinker, Fireworks and several provider batch APIs has many surfaces that can break when an upstream SDK changes. Pin the version in your environment, and re-run one representative pipeline after each bump rather than assuming compatibility.
Who should pick this up
Curator is for a team that has already decided to generate or extract data with a hosted or self-hosted model and now needs that work to survive interruption. The caching and fault recovery described in the README address a real operational cost, and the viewer addresses a real review cost. The structured output support and the LiteLLM, vLLM and batch API routing mean you can start on one provider and move without rewriting the pipeline. If you are building reasoning datasets, the changelog names OpenThoughts-114k, OpenThoughts2-1M, s1K-1.1 and Bespoke-Stratos-17k as datasets the maintainers produced with the tool, which tells you the intended scale. If your scale is smaller, or your constraint is that no prompt leaves your network and you have no vLLM deployment, the library adds a dependency without removing work.
Editorial conclusion
Adopt Curator if your team writes Python, already calls hosted models or runs vLLM, and wants caching and resumption handled for you rather than rebuilt per project. Do not adopt it if you need a no-code interface, if your data cannot leave your infrastructure and you have no vLLM deployment, or if you only need a handful of completions that a plain SDK loop would cover. Before committing, run the pinned install, confirm the viewer renders in your environment, and check that the batch path you intend to use is the one your provider actually supports.
Community notes