DataArc SynData Toolkit: one config file drives synthesis, post-training and evaluation

What the toolkit actually covers, from generation to GRPO

DataArc SynData Toolkit is described as a modular synthetic data generation toolkit supporting multi-source and multi-language data synthesis, developed and open-sourced by DataArcTech and the International Digital Economy Academy. The paper describing it, DataArc-SynData-Toolkit: A Unified Closed-Loop Framework for Multi-Path, Multimodal, and Multilingual Data Synthesis, appeared in the ACL 2026 System Demonstrations track.

What sets it apart from a generator script is where it stops. Generation is one stage of four. The pipeline covers synthetic data generation, then data filtering and rewriting, then an integrated post-training module powered by verl supporting SFT and GRPO, then post-training model evaluation using DeepEval. Most tools in this space end at stage one and hand you JSONL; this one treats training and scoring as part of the same run.

Three generation paths are supported. Local synthesis works from corpora you already have. Hugging Face integration crawls and filters datasets from the hub. Model distillation generates data through a teacher model, which is also the path that connects naturally to the training stage.

Model access is not tied to one vendor: the feature list names local deployment, OpenAI APIs and more, and the dependency list includes both the openai and anthropic packages, so the multi-provider claim has some backing in the imports.

For a reader deciding whether this fits, the honest summary is that it is a platform for people who intend to train, not a library for sampling a few thousand examples.

The configuration file and the sdg entry point

The README describes synthesis with a single command plus a configuration file, and the packaging confirms the shape of that command: pyproject.toml registers a console script named sdg, bound to sdgsystem.cli:cli.

[project.scripts]
sdg = "sdgsystem.cli:cli"

The documentation links to a Quick Start section for the CLI path and a Run with GUI section, and the examples/ directory holds finance, mathematics and medicine, which are the kind of domain configuration a first run would copy. A tip at the end of the changelog says that if you cannot use the latest feature you should pull the latest code, which reads as an admission that the repository moves faster than its releases do.

For customization, the README names two extension points: BaseTaskConfig in sdgsystem/configs/config.py and BaseTaskExecutor in sdgsystem/tasks/base.py. Developers inherit from those to replace a generation task, which is the modularity claim made concrete. The rest of the tree follows the same split, with sdgsystem/ holding the system, configs/ holding configurations, docs/ holding documentation, and a vendored verl/ directory at the root that the wheel also packages.

A GUI is available as well as the CLI, and since the 25/12/25 upgrade the architecture is a separated frontend and backend: a FastAPI backend exposing REST APIs plus SSE streaming for real-time progress, with a standalone React frontend on top. That is the structure to look at if you care about monitoring a long synthesis run rather than only its output.

Operationally expensive by design: torch, vllm and flash-attn are not optional

The dependency list is the clearest statement of what this tool requires, and it is heavy.

Three pins stand out. requires-python is ==3.11.13, an equality constraint rather than a floor, so 3.11.12 or 3.12 will not satisfy it. torch is pinned to 2.8.0 with torchaudio at 2.8.0 and torchvision at 0.23.0, and all three are routed through a dedicated index rather than the default package source:

[[tool.uv.index]]
name = "pytorch-cu128"
url = "https://download.pytorch.org/whl/cu128"
explicit = true

[tool.uv.sources]
torch = { index = "pytorch-cu128" }

The cu128 wheel index is a CUDA 12.8 build index, so the training and generation paths assume an NVIDIA stack. flash-attn is a required dependency and the build config disables build isolation for it and setuptools, which is a strong hint that compiling it locally is expected and slow. vllm is also required, not an extra, which means even a pure synthesis run pulls in a serving stack.

The default index is configured to an Aliyun mirror, so a deployment outside that network should expect to change it.

Judged as a whole: you are installing a training environment, not a data script. That is defensible if you are going to train, and it is the main reason to look elsewhere if you are not.

The pipeline stages that come after generation

Filtering and rewriting is the stage between raw output and training, and it exists because generated data is not uniformly usable. The README states the filtering and rewriting stage as part of the pipeline whose strategies and implementations can be customised step by step.

Two engineering additions from the changelog make that pipeline survivable on a real corpus. On 25/11/27 a parallel processing module was added to accelerate generation, and on 25/11/28 intermediate result saving was added so a run can resume from the last successful stage rather than restarting the pipeline. The changelog calls the latter a token saver, which is the right framing: a synthesis run is paid for in tokens and in wall clock, and restarting from zero is the failure mode users hit most.

Post-training is handled by verl, wired in on 25/12/25, covering SFT and GRPO on the synthesized data. Evaluation came later, on 26/01/26, through DeepEval with three metrics: Answer Correctness comparing model output against ground truth with customizable rubrics, Pairwise Preference comparing the post-trained model against the base model, and Format Compliance checking adherence to output format instructions.

Those three metrics map onto three different questions, which is the reason they are a coherent set rather than a list. Correctness is absolute, preference is relative to where you started, and format compliance is the one that catches a model that learned the content but not the shape of the answer.

Multilingual and image work, and where the coverage stops

Language support is stated as English plus various low-resource languages, and the 25/12/25 upgrade added Arabic datasets by leveraging an Arabic translation model to produce localized synthetic outputs. The README also ships a Chinese version of the documentation at README_zh.md, which is a reasonable signal about the intended user base but is not itself a data capability.

Image support arrived on 26/01/12 with two tasks. The image modality Local Task generates VQA data from local images or PDF-extracted figures using VLMs, and the image modality Web Task searches for and retrieves image-text datasets from the Hugging Face Hub.

The PDF path is worth noting for the domain the examples cover: pymupdf and mineru[core] are both dependencies, so figure extraction from documents is part of the intended workflow rather than an aspiration. That matters for anyone whose training data lives in reports rather than in text files.

The claim of modularity is the one to test before committing, since the parts you would most likely want to replace, the generation tasks, are exactly the parts the README invites you to subclass. The closed loop framing is the project's differentiator; if you only need the first stage, the rest is weight.

The alternative, and the difference in approach

The direct alternative for synthesis alone is to use a model provider's own batch generation plus your own filtering, or an existing dataset library such as the datasets package the project itself depends on. The difference in approach is that those approaches stop at data, and everything after generation becomes your problem: filtering, formatting into a training layout, choosing a trainer, and building an evaluation harness.

For training, the alternative to the integrated verl module is a separate training script using peft or trl on your existing stack. verl is a strong reinforcement and supervised trainer, but adopting it inside this toolkit also means adopting this toolkit's pinned torch and Python versions, which is the coupling that matters.

For evaluation, DeepEval is the tool either way, so the difference is that here it arrives configured for three specific metrics rather than as a blank framework.

So the honest comparison: a component-by-component rebuild is more flexible and keeps your own versions; this toolkit is faster to a first trained checkpoint because the handoffs between stages already exist. Teams with a settled training stack should lean toward the rebuild.

Licence, releases and maintenance signals

The project is Apache-2.0, and the packaging is unusually thorough about attribution: license-files lists LICENSE, NOTICE and THIRD_PARTY_NOTICES.md, and all three are at the repository root. THIRD_PARTY_NOTICES.md matters more than usual here because the dependency set pulls in torch, vllm, verl, DeepEval and FastAPI, each with its own terms. This is a licence note rather than legal advice.

The maintenance signals are mixed and worth reading carefully. The repository is not archived and the last push was on 2026-09-09. There are no GitHub releases at all, and pyproject.toml still reads version 0.1.0, so there is no published artifact to pin and no changelog generated from tags. The changelog is a hand-maintained list of dated entries in the README, ending with the ACL 2026 paper publication on 26/09/09.

A 0.1.0 version with no releases, combined with a README tip telling users to pull the latest code, describes a project that expects to be used from source rather than consumed as a dependency. That is a legitimate choice for a research platform with an ACL demo paper behind it, and it is a poor fit for a production pipeline that needs a version it can reproduce.

Before adopting, decide whether using from main is acceptable in your environment. If it is not, the version field offers you nothing to pin to.

Editorial conclusion

Adopt DataArc SynData Toolkit if you are building training data for a model you also intend to fine-tune and evaluate yourself, because the closed loop from synthesis through verl training to DeepEval scoring is the reason this exists rather than a standalone generator. Do not adopt it to look at generated text without training it, since torch, vllm, flash-attn and ray are required dependencies rather than extras, and does not fit a workstation that cannot run that stack. Verify first that you can provide Python 3.11.13 exactly, since requires-python is an equality pin and not a floor, and that your package index can serve the pytorch-cu128 wheel index the project configures, because torch, torchvision and torchaudio are all pinned to it.

Frequently asked questions

Official sources

1. DataArcTech/DataArc-SynData-Toolkit on GitHub
2. Issues
3. License: Apache-2.0
4. README