SkyDiscover and the two ways it lets a model rewrite your system

Optimize and Synthesize are different products in one repository

SkyDiscover is described as an open-source framework from UC Berkeley that uses AI to discover better algorithms and build complete systems, and the readme splits it into two numbered modes that are worth keeping apart.

The first is Optimize, where the goal is to improve an algorithm or a component of your system. You provide a way to score solutions, and evolutionary search finds programs that score better and better. The second is Synthesize, where the goal is to build an end-to-end system. You describe it in plain text or a formal spec, and coding agents build it specialized Just-in-Time for your workload, hardware and requirements.

The difference in what you have to bring is the difference that matters. Optimize needs a scoring function, which is usually the hardest part of the work and entirely your responsibility. Synthesize needs a description, and the framework does the rest.

The second dimension is which models do the work. SkyDiscover works with Claude Code, Codex, Cursor and Pi, and with any OpenAI-compatible model. So the coding agent is a choice rather than a fixed dependency, and the model behind the Optimize search is likewise pluggable, with the readme naming OpenAI, Anthropic and Gemini as the key options.

The project is on Apache-2.0, its primary language is Python, and it carries a `benchmarks/` directory, a `docs/` tree, a `skydiscover/` package directory, a `tests/` directory and a `uv.lock`. The version in the project metadata is dynamic rather than hard coded, and the repository publishes no GitHub releases.

The two search algorithms that tune their own settings

Optimize evolves programs against your scoring function, and its two headline algorithms, AdaEvolve and EvoX, are described as achieving the strongest open-source results across roughly 200 benchmarks, and as being used across industry including Google and Uber.

What makes them different from a plain genetic or evolutionary search is written into their one line descriptions. AdaEvolve is adaptive search that tunes its own settings as it runs. EvoX is AI that optimizes its own optimization process. The second one is the more unusual idea, an evolutionary search whose search strategy is itself subject to evolution, so a run that finds a poor region can change how it explores rather than continuing on a bad path.

The comparison numbers in the readme are stated per benchmark family. On Frontier-CS with 172 problems, roughly 34 percent higher median score than OpenEvolve, GEPA and ShinkaEvolve under the same budget. On math and systems with 14 tasks, matching or exceeding AlphaEvolve and human state of the art on 6 of 6 systems and 6 of 8 math tasks. On real-world systems, 41 percent lower cross-cloud transfer cost, 14 percent better GPU load balance for MoE serving, and 29 percent lower KV-cache pressure.

The budget qualifier in the Frontier-CS line is the important detail for a reader. A higher median score under the same budget is a claim about sample efficiency, and a search that costs more per candidate could still win on score in a different budget. Anyone reproducing these comparisons needs to hold the budget fixed themselves.

The quick start names only two of the algorithms by flag, `--search adaevolve` and `--search evox`, and the full Optimize guide covers six more algorithms, evaluator formats, the Python API and complete benchmark results.

Evolving a program means writing the evaluator first

The quick start is short enough to read in full, and it shows the shape of the whole workflow.

export OPENAI_API_KEY=<your-key>   # or ANTHROPIC_API_KEY / GEMINI_API_KEY
uv run skydiscover optimize evaluator.py --search adaevolve --model gpt-5 -i 100
# uv run skydiscover optimize evaluator.py --search evox --model gpt-5 -i 100

The evaluator is a file you write, and the framework runs it against candidate programs. The `-i 100` argument is the iteration count, and the commented second line shows the same run with the other search algorithm.

This is the part of the workflow where most of the effort sits, and the readme is not coy about it. The scoring function is the interface between your domain and the search, and the quality of the results is bounded by the quality of that function. A scoring function that can be gamed will be gamed, and the framework knows this, which is why Synthesize has an explicit mechanism against it.

There is a second consequence of the design that is easy to miss. Because the search operates on programs and evaluates them by running your code, the evaluator needs to be safe to run repeatedly, on generated code, without side effects that compound. An evaluator that writes to a shared database, or that depends on state left over from the previous iteration, will produce a search that converges on whatever exploits that weakness.

So a realistic adoption of Optimize looks like this: you already have a measurable objective, you can express the candidate program as text, and you can write a function that runs it and returns a number. If any of those three is missing, the framework has nothing to work with.

The auditor turns a reward hack into a failing test

Synthesize is the newer half, released in September 2026 as SkyDiscover-Synthesize, abbreviated SkySynth, and it works at a different level: rather than evolving one function, it builds a whole system.

The claim is that instead of adopting a one-size-fits-all system, you get one specialized for your own hardware, workload and requirements. The readme lists what has been built this way: key-value stores up to 2.3x faster than Redis and FASTER, including formally verified distributed ones; LLM inference engines with up to 2.2x the throughput of vLLM and SGLang; and model routers at up to 48 percent lower cost. Each of these is a checked-in example you can run, linked from the examples README.

The two approaches differ in what they require from you. The formal-proof-driven approach is for when you have a formal spec, with Rocq given as the example, and it synthesizes the code and its machine-checked proof together, step by step. The test-driven approach is for when you have a plain-text description, where agents iterate on the implementation and an auditor turns every reward hack it finds into a new test.

That last clause is the most interesting piece of design in the whole project. A reward hack is defined in the readme as a change that raises the score while breaking what you meant. The framework does not merely detect them, it converts them into tests. So the auditor's output becomes part of the specification, and the next iteration of the agent has to satisfy it. That is a mechanism for closing the loop between what the optimizer wants and what you meant, and it is the reason the two papers from May 2026 are described as being about systems that are correct by proof or by test.

The synthesis half is driven by a coding agent rather than a search over a single file, and it is installed by a command that wires the agent in.

init wires whichever coding agents it finds on your PATH

The Synthesize quick start is one command:

uv run skydiscover init   # wires every coding agent found on PATH; or --agent claude|cursor|codex|pi

The comment tells you both behaviours. Run it with no argument and it wires every coding agent it finds on your PATH, which is convenient on a workstation with several installed. Run it with `--agent` and a name from `claude`, `cursor`, `codex` or `pi`, and it wires that one.

The mechanism is a skill. The readme says to install the `/skysynth` skill into the coding agent of your choice, and the repository layout shows where that lives: there is a `.claude-plugin/` directory at the top level, and a `.agents/` directory alongside it, which is the conventional location for agent configuration that several tools can read. There is also a `CONTRIBUTING.md` and a `docs/` tree with the installation guide at `docs/content/docs/installation.mdx`.

The practical point is that Synthesize is not a self-contained program. It is a set of instructions installed into an agent you already run, and the agent does the building. That has two consequences. The first is that the quality of the output depends on the model your agent is running, which is outside the framework's control. The second is that the "system" it produces is delivered as changes in your working tree, in the form of the code and the tests the auditor wrote, and reviewing that is ordinary code review of possibly unfamiliar code.

That second consequence is the one to plan for. Synthesizing a key-value store is an interesting demonstration, and adopting one in production is a different decision that involves reading a system you did not write.

Optional extras decide how far you can go

The dependency structure in the project metadata is unusually revealing about what this framework expects to be run.

The base install is small: `openai>=1.0.0`, `pyyaml>=6.0`, `tqdm>=4.64.0`, `numpy>=1.22.0`, `python-dotenv>=1.2.1`, and `tomli` only for Python below 3.11. Everything else is an optional extra, and the names of the extras tell you what the heavy use cases are.

The `math` extra is the largest, pulling in scipy, sympy, jax, optax, torch, scikit-learn, numba, pandas, matplotlib, plotly, networkx, cvxpy, autograd, pymoo and PyWavelets. The `adrs` extra adds pandas, networkx, torch and numpy for the AdaEvolve work. The `external` extra pulls in openevolve, gepa with the full extra, litellm and alpha-evolve, which are the competing optimisers the readme compares against. The `frontier-cs` extra brings anthropic, colorlog, datasets, the Google generative AI clients, numpy 2.x, skypilot and more, and the `prompt-optimization` extra brings dspy, litellm, bm25s, pystemmer, datasets and diskcache.

So the framework covers three distinct workloads: evolving programs in the general case, running the Frontier-CS benchmark suite, and optimising prompts rather than programs. The `external` extra deserves a note, because it is the one that installs the alternatives you would otherwise compare against, which means the comparison is reproducible rather than rhetorical.

The uv configuration also pins things at the resolver level, with override dependencies for httpx and gepa, and the `frontier-cs` extra carries a comment noting that it requires numpy 2.x while the base requires only 1.22. These are the kind of details that turn into a resolution failure at install time rather than a subtle numerical difference later.

Where the framework stops being the right tool

There are three limits, and they are structural rather than fixable with configuration.

The first is the need for an automatic signal. Both halves of SkyDiscover optimise against something measurable, and if the thing you want to improve cannot be scored by running code, the framework has nothing to work with. Interface design, a documentation rewrite and a product decision are all cases where a human judges the result, and there is no evaluator you can write for them that does not simply encode your own prior judgement.

The second is the cost and the loop time. Optimize runs an LLM to produce candidates and evaluates each one, so a hundred iterations is a hundred or more model calls plus a hundred evaluation runs. The `-i 100` in the example is a small budget for a research setting. In a production setting, the loop is not free and the search is not instant, and the question of how long you are willing to let a search run before taking the best candidate is a budget decision the framework does not make for you.

The third is the review burden. Synthesize produces code and tests you did not write, potentially thousands of lines, on a path where the framework's own claim of correctness rests on either a machine-checked proof or on tests that the auditor derived from the same specification the agent was given. Both are stronger than a code review by eye, and neither replaces one. A formally verified component still needs someone to decide whether the specification was the right one.

There is also a plain operational point. The project requires Python 3.10 to 3.13 and uv, and it needs an LLM API key for Optimize and a coding agent for Synthesize. That is a real environment to stand up, and for a task where a well written static algorithm would do, the machinery is not free.

Editorial conclusion

SkyDiscover fits a performance engineer who has a measurable objective and a program that can be expressed as code, or a systems team that wants a component built for their own hardware rather than adapted from a general one. It is the wrong tool when you cannot score a candidate automatically, because both halves of the framework need a signal to optimise against, and it is the wrong tool when the deliverable is a decision rather than a program. Before you start, verify three things: that the LLM provider you want to use is reachable through the OpenAI-compatible interface the project expects, that your Python version falls inside the supported 3.10 to 3.13 range, and that the evaluator you write produces the same result on a second run, because a noisy score makes evolutionary search optimise noise. The code is Apache-2.0, the repository publishes no GitHub releases, and the last push was on 2026-09-14.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. Project website
4. README
5. skydiscover-ai/skydiscover on GitHub