MindAct: a reproducibility wrapper for LeRobot policies on LIBERO benchmarks

The gap MindAct targets: experiments that cannot be rerun

Imitation learning work on desktop manipulation tends to accumulate in three disconnected places: a LeRobot dataset, a policy checkpoint, and a LIBERO evaluation script. The dataset has a `repo_id` and a revision, the policy has a name and an optional `pretrained_model`, and the environment has a task suite and a list of task ids. Nothing in that stack forces those three records to travel together, so a result reported from a run six weeks ago is often reproducible only by whoever kept the shell history. MindAct's stated purpose is to close that gap: it describes itself as a reproducible training and evaluation framework for imitation learning policies on desktop manipulation tasks, integrating LeRobot datasets and policies with LIBERO simulation benchmarks. The intended user is a robotics or embodied AI researcher who already works inside the LeRobot and LIBERO ecosystems and wants the experiment definition, not the model code, to be the durable artifact. That is a narrow audience. If you train policies in a custom simulator or store demonstrations outside the LeRobot format, the framework's central abstraction has nothing to bind to.

Protocols, frozen dataclasses, and lazy extras

The architecture is described as protocol-based, with minimal adapter interfaces for datasets, policies and environments. The README also states that the codebase uses frozen dataclasses and runtime protocols throughout, and that torch, lerobot and libero are loaded lazily through extras. Those three choices point in the same direction: the core package is meant to import and validate configuration without a GPU, a simulator, or any of the heavy dependencies installed. The data flow implied by the example config is a single experiment object that names a dataset, a policy and an environment, plus training and evaluation blocks, and a run identity supplied on the command line. The `--run-id` and `--output-dir` flags in the eval example suggest that outputs are keyed by run, and the README refers to a conventional directory structure for checkpoints, logs and results. Provenance is handled through what the README calls manifest-tracked provenance, though the material does not show the manifest schema or say which fields it records. That is a real gap for anyone evaluating the reproducibility claim: the config file is visible, the manifest is not.

Installing MindAct and validating a config

Installation is a set of extras on top of an editable install. The README gives `pip install -e .` for configuration and CLI only, `pip install -e ".[torch]"` for PyTorch, `pip install -e ".[lerobot]"` for LeRobot datasets and policies, `pip install -e ".[libero]"` for LIBERO simulation environments, and `pip install -e ".[torch,lerobot,libero]"` for all integrations, with `pip install -e ".[dev]"` for development tools. The first command worth running is the validator: `mindact config-check configs/experiments/libero-baseline.yaml`. The second is the smoke path, `mindact eval configs/experiments/libero-baseline.yaml --runner fake --run-id smoke-run --output-dir outputs --episodes 2`. The README is explicit about what that second command does and does not do: it exercises the full evaluation lifecycle with built-in test doubles, and the fake runner is a contract smoke test, not a benchmark result. The example YAML carries the keys you would expect: `name`, `seed`, `output_dir`, a `dataset` block with `repo_id`, `revision` and `split`, a `policy` block with `name` and `pretrained_model`, an `environment` block with `name`, `task_suite` and `task_ids`, a `training` block with `steps`, `batch_size`, `learning_rate`, `log_every` and `checkpoint_every`, and an `evaluation` block with `episodes`, `max_steps` and `record_video`.

The fake runner is the whole runnable surface right now

This is the limitation that decides everything else. MindAct v0.1 is described in the README as the initial skeleton release, with core interfaces and configuration system stable, and the roadmap places unified reproducible training and evaluation in the current Phase B while trajectory quality diagnostics sit in a future Phase A. The README states plainly that real LeRobot policies and LIBERO environments arrive with their adapter implementations. So the shipped, working path is config validation plus an evaluation loop against test doubles. There is no evidence in the supplied material of a completed LeRobot dataset adapter, a LIBERO environment adapter, or a training loop that consumes real demonstrations. A second constraint is historical: the repository was originally MindNLP, a MindSpore-based NLP library, and the legacy codebase is preserved in the `legacy` branch. MindAct is described as a complete pivot to embodied AI with PyTorch and Hugging Face as the native stack. Anyone arriving from search results that mention MindSpore, Transformers or Diffusers should read the README before installing, because the current project has nothing to do with running transformer or diffusion models on MindSpore. The name and the topic list are misleading relative to the code.

Where MindAct sits next to LeRobot and Hydra

The honest comparison is with the pieces MindAct wraps. LeRobot already provides datasets and policies, and LIBERO already provides the simulation benchmark; MindAct's contribution is the seam between them plus a config and provenance layer. Against a plain Hydra setup, which is the common way to parameterize Python training runs, the difference is scope rather than mechanism. Hydra composes configuration and handles multirun sweeps but knows nothing about datasets, policies or environments, so a Hydra-based project still writes its own adapter code for LeRobot and LIBERO and its own artifact layout. MindAct fixes the adapter shape in advance with runtime protocols and frozen dataclasses, which means less freedom and more consistency. If your team already has a working Hydra configuration for LeRobot training, moving to MindAct means rewriting the config into its YAML schema and conforming to its protocols in exchange for the manifest tracking and the conventional output directory. That trade only pays off if you intend to publish or hand off runs that other people must reproduce.

Maintenance, releases, and the Apache-2.0 terms

The release cadence visible in the material is uneven. v0.4.1 is dated 2025-04-10, v0.5.0 and v0.5.1 both landed on 2025-11-03 and 2025-11-05 respectively, and the last push to the default branch is 2026-09-01. The README, however, describes v0.1 as the initial skeleton release, which does not line up with a v0.5.1 tag. Either the README is stale relative to the tags or the version numbers track something other than the skeleton status described there. You cannot tell from the supplied material which is true, and that ambiguity matters if you are pinning a version: check the tag contents before assuming v0.5.1 contains the adapters that v0.1 lacks. The project is not archived, and it is licensed under Apache License 2.0, with the README pointing to LICENSE and NOTICE for details. Apache-2.0 permits commercial use and modification and includes an explicit patent grant, but it also requires that you preserve the NOTICE file and state changes you make. Because this repository absorbed the MindNLP codebase, the NOTICE file is worth reading before you copy any code out of the `legacy` branch into your own project. That is a description of the licence terms, not legal advice; consult your own counsel if the provenance of the inherited code affects your distribution.

What to check before you build on it

Clone the repository and read the protocol definitions for datasets, policies and environments before writing any configuration of your own. The README does not reproduce those interfaces, and the entire value of the framework rests on whether they match the LeRobot policy and dataset signatures you actually use. Then run `mindact config-check` against a YAML derived from `configs/experiments/libero-baseline.yaml` with your own `repo_id` and `task_suite` values, and confirm the validator rejects malformed entries rather than passing them through. Inspect what the manifest records for the `--runner fake` smoke run, since manifest-tracked provenance is the feature the README leads with and the one you cannot verify from the documentation alone. Finally, diff the v0.5.1 tag against the README's description of v0.1; if the adapters are present in the tag, the project is further along than its own documentation claims, and if they are not, you are adopting an interface specification rather than a working toolkit.

Editorial conclusion

Adopt MindAct only if you are building the LeRobot and LIBERO adapters yourself and want its protocol interfaces and YAML schema as a starting point; the v0.1 skeleton gives you dataset, policy and environment adapter contracts plus a config validator, and nothing that trains a real policy. Do not adopt it if you need working imitation learning today, because the README states that real LeRobot policies and LIBERO environments arrive with their adapter implementations and that the fake runner is a contract smoke test. Before committing, run `mindact config-check` on your own YAML, read the adapter interfaces in the repository to confirm they match your LeRobot policy signature, and check the `legacy` branch and NOTICE file if any of your code descends from MindNLP.

Official sources

1. candle-org/MindAct on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases