AIDE ML: the AIDE reference build for agentic tree search over ML code
AIDE: an LLM agent for machine learning engineering - the research Weco grew out of. Referenced in OpenAI MLE-bench.
At a glance
- What is it?
- AIDE ML is the open-source reference implementation of the AIDE algorithm, an LLM agent that writes, evaluates and patches machine-learning code along a search tree. It is built for researchers who want to replicate the paper or swap in their own search heuristics, not for teams that want a managed pipeline.
- Who is it for?
- Adopt AIDE ML if you are an agent-architecture researcher who wants to modify search heuristics, evaluators or LLM back ends, or an ML practitioner willing to read the generated code before trusting it. Do not adopt it if you need a managed pipeline with experiment tracking, since the README itself points production users to Weco.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 12 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem AIDE ML addresses: metric-driven code search instead of one-shot generation
Most LLM coding workflows produce a single artefact and stop. AIDE ML treats each generated Python script as a node in a solution tree and keeps going until a user-defined metric stops improving. The README frames the target audience in two lines: agent-architecture researchers who want to swap in new search heuristics, evaluators or LLM back ends, and ML practitioners who want to build a pipeline given a dataset. That split matters. The repository describes itself as the open-source reference build of the AIDE algorithm, a lean implementation for experimentation and extension, while the paper at arXiv 2502.13138 carries the algorithmic description and weco.ai carries the product. The problem is narrow on purpose: you have a dataset, a goal sentence and a metric name, and you want code that scores well on that metric without hand-tuning a pipeline. The repo does not claim to replace a feature store, an orchestration layer or a model registry.
How the tree search works in this repository
The mechanism is described in the README as LLM-guided agentic tree search in the space of code. A Python script becomes a node. LLM-generated patches spawn children. Metric feedback prunes branches and steers the next expansion. The CLI exposes the knobs that shape that tree: agent.search.num_drafts controls how many drafts are produced per step, agent.steps controls how many improvement iterations run, and agent.code.model selects the model that writes code. The default for agent.code.model is listed as gpt-4-turbo, with agent.steps defaulting to 20 and agent.search.num_drafts defaulting to 5. The README also states that model-neutral plumbing is included for OpenAI, Anthropic, Gemini, or any local LLM that speaks the OpenAI API, and the advanced example switches the coding model with agent.code.model="claude-4-sonnet". Two artefacts come out of a run: logs/<id>/best_solution.py holds the best code found, and logs/<id>/tree_plot.html renders the tree for inspection. That HTML visualiser is the part that makes the search auditable rather than opaque, and it is the reason the tree structure is worth understanding before you tune anything.
Getting AIDE ML running: install, keys and the aide command
The quick start is three commands. Install with pip install -U aideml. Export an LLM key, for example export OPENAI_API_KEY=<your-key>. Then run an optimisation: aide data_dir="example_tasks/house_prices" goal="Predict the sales price for each house" eval="RMSE between log-prices". Note that goal and eval are plain English, not a metric identifier the code parses; the README's own example passes "RMSE between log-prices" as the eval string. There is a second entry point for the web UI, and it requires a clone rather than the pip package: git clone the repository, cd aideml, pip install -e . (the README notes this adds streamlit), then cd aide/webui and streamlit run app.py. The sidebar takes an API key, an uploaded dataset, a Goal and a Metric, and the UI shows live logs, the solution tree and the best code. For embedding, the README gives a Python example using aide.Experiment(data_dir=..., goal=..., eval=...) followed by exp.run(steps=2), after which best_solution.valid_metric and best_solution.code are readable attributes. The same example configures the aide logger through the standard logging module. If you want fewer steps than the default 20, pass agent.steps explicitly; the Python example uses steps=2, which is the cheapest way to confirm your data directory and metric string are wired correctly before spending on a full run.
Where AIDE ML is the wrong tool
The first constraint is that the metric has to be computable by code the agent itself writes and runs, and the README's examples are all offline tabular or time-series tasks: house prices with RMSE on log-prices, churn with AUROC, bitcoin close price with RMSLE. Nothing in the supplied material describes a sandbox, a resource cap, or a guarantee that generated code cannot reach the network or the filesystem, so treat execution isolation as your problem, not the project's. The second constraint is cost. Each step expands drafts, and the default is 20 steps with 5 drafts per step, so the number of LLM calls scales with agent.steps multiplied by agent.search.num_drafts. A full default run is not a single prompt. The third constraint is that the repository is explicitly the reference build, not the product. The README's own table separates the AIDE algorithm (the paper), the AIDE ML repo (this code, for experimentation and extension) and the Weco product, which it says generalises AIDE's capabilities to broader code optimisation scenarios with experiment tracking and enhanced user control. If you need tracking, multi-user access, or a supported deployment, the repository is pointing you elsewhere. Fourth, the material shows no built-in evaluator library: eval is a free-text description, so metric correctness depends on how the agent interprets your sentence, and a subtly wrong metric will be optimised just as enthusiastically as a correct one.
MLE-bench as the harness, and what a benchmark harness does differently
The most useful comparison here is not another agent framework but MLE-bench, the OpenAI evaluation suite the README cites. MLE-bench is a benchmark: 75 Kaggle competitions, fixed tasks, fixed scoring, designed to compare agents under identical conditions. AIDE ML is an agent you point at your own data_dir, goal and eval. The difference in approach is the direction of the interface. In a benchmark harness you write an adapter so your agent fits the harness's task format and grading rules. In AIDE ML you supply the task in natural language and the agent generates the pipeline, the training loop and the evaluation code around your metric string. The README reports that MLE-bench found the tree search of AIDE wins 4x more medals than the best linear agent (OpenHands), which is a claim about the AIDE algorithm under benchmark conditions, not about your dataset. The repositories listed as built on or with AIDE (METR's RE-Bench, Sakana AI's AI Scientist-v2, two Meta projects including llm-speedrunner and aira-dojo, and SJTU's ML-Master) are all research artefacts of the same kind. If your goal is a reproducible comparison against published numbers, use the harness and its task definitions. If your goal is a working script for a dataset you already have, use AIDE ML and read the tree plot.
Maintenance, releases and the MIT licence
The repository is not archived and shows a push in September 2026. The release history in the supplied material is sparse and unevenly spaced: v0.1.4 in April 2024, v0.2.0 in January 2025, and v0.2.2 in November 2025. That cadence suggests a research artefact that gets tagged when something meaningful lands rather than a project on a fixed schedule, so pin a version in your environment rather than tracking main. The licence is MIT, which permits commercial use and modification provided the copyright notice and permission notice are retained; the repository README carries an MIT License badge. Nothing in the supplied material states a support commitment, a deprecation policy, or a compatibility guarantee across minor versions, and the README's own framing puts the maintained product surface at weco.ai. This is not legal advice; read the LICENSE file in the repository if the terms matter to your organisation. The practical upgrade cost is the config surface: agent.code.model, agent.steps and agent.search.num_drafts are the keys the README documents, and if upstream renames them your existing commands break silently or loudly depending on how the CLI handles unknown keys, which the material does not say.
What to verify before you commit a dataset to it
Start with a two-step run against a small example directory, exactly as the Python example does with steps=2, and inspect logs/<id>/best_solution.py before reading any metric. The code is the artefact you will have to maintain; the metric is only a number the agent produced. Open logs/<id>/tree_plot.html and check whether the winning branch is a genuine improvement or a sequence of near-duplicate patches, because a tree that keeps expanding one lineage tells you the search is not exploring. Confirm which model agent.code.model resolves to in your environment, since the documented default is gpt-4-turbo and the advanced example overrides it to claude-4-sonnet, and confirm that your eval string describes a metric the generated code can actually compute from the files in data_dir. If any of those three checks fails, the run is not measuring what you think it is, and no amount of additional agent.steps will fix it.
Editorial conclusion
Adopt AIDE ML if you are an agent-architecture researcher who wants to modify search heuristics, evaluators or LLM back ends, or an ML practitioner willing to read the generated code before trusting it. Do not adopt it if you need a managed pipeline with experiment tracking, since the README itself points production users to Weco. Before running anything, verify which model the default agent.code.model resolves to, how many agent.steps you can afford, and whether your evaluation metric can be computed inside the sandbox the agent writes into.
Community notes