optim-agent: an AgentSampler that lets a coding agent pick your next hyperparameter
LLM agents as your hyperparameter optimizer.
At a glance
- What is it?
- optim-agent wraps a Claude Code, Codex or OpenCode CLI behind an Optuna-shaped API so the next trial comes from an agent reading parameter context and past outcomes. The design is honest about its own limits: the agent proposes, the library validates and falls back to sampling.
- Who is it for?
- Adopt optim-agent when evaluations are expensive, your parameter count is small enough that a handful of trials matters, and your parameters carry meanings a coding agent can read from your code. Skip it when you need hundreds of cheap trials, when no agent CLI is authenticated on the machine, or when every evaluation must run in an isolated environment with no outbound calls.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 32 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap optim-agent targets: expensive evaluations, few trials, parameters that mean something
Classical hyperparameter search assumes you can afford many evaluations. TPE and similar surrogates need a history before their model of the objective says anything useful. When a single evaluation means a multi-hour training run, a simulation sweep, or a paid inference benchmark, you are stuck in the data-starved region where the surrogate is guessing. optim-agent's pitch is that this is exactly where a coding agent has an advantage, because it can read what a parameter means rather than treating the dimension as an anonymous coordinate. The README states the library is for systems that expose configurable parameters and a measurable objective, and lists model training, inference and serving, quantitative research, reinforcement learning, scientific workflows, and black-box bounded configurations. The intended user is an algorithm engineer who already runs a tuning loop and wants the proposal step to carry domain reasoning. It is not a replacement for the policy-learning algorithm in an RL setting; the README is explicit that it tunes the system around the learning loop.
AgentSampler: what the agent sees, what it returns, and what happens when it returns nonsense
The mechanism is a sampler object passed to create_study, the same slot Optuna users fill with a sampler. AgentSampler takes a backend (claude, codex, or opencode), an effort level, a study-wide context string, a history count, and two booleans, explicit_reasoning and qualitative_notes. Parameters are declared through suggest_float and suggest_int calls inside the objective, and each can carry its own context string. The README's example annotates threshold as a decision threshold where higher values trade recall for precision, and budget as compute or operating budget where larger values may improve quality. Those strings are the semantic payload. The sampler combines parameter meaning with trial history and proposes the next configuration. The important boundary is stated plainly: objective evaluations remain authoritative. The library validates proposed values against the declared space, records outcomes, and falls back to safe sampling when an agent reply is invalid. That fallback is the difference between a research prototype and something you can leave running overnight. Studies can be persisted to JSON or SQLite, and the README says a stored study retains configurations, outcomes, states, context, and optional agent rationale, which is what makes the decisions auditable after the fact.
Installation paths: a skill, a plugin, or a Python package, plus one authenticated CLI
There are three entry points and they are not interchangeable. Codex users install the skill with $skill-installer install https://github.com/Optim-Agent/optim-agent. Claude Code users add the marketplace and install the plugin with claude plugin marketplace add Optim-Agent/optim-agent && claude plugin install optim-agent@optim-agent. Python users install the package from PyPI with python -m pip install optim-agent, or from source with python -m pip install "optim-agent @ git+https://github.com/Optim-Agent/optim-agent.git". All three routes require one authenticated agent CLI on PATH: claude, codex, or opencode. That requirement is the first thing to check on a new machine, because the Python package installs cleanly without it and the failure will only surface when a study tries to propose. The quickstart then builds a study with oa.create_study(direction="maximize", sampler=oa.AgentSampler(backend="claude", effort="high", context=..., history=5, explicit_reasoning=True, qualitative_notes=True), storage="study.json", summarize=True) and calls study.optimize(objective, n_trials=20). Two config keys deserve attention: storage="study.json" is what makes a run resumable, and summarize=True enables an agent-written summary after the last trial, readable as study.summary.
The trajectory figure is an illustration, not a benchmark you can generalize from
The repository ships an animated optimization trajectory comparing TPE and GPT-5.5 on a seed-0 Branin trace under the same 10-trial budget, plotting incumbent objective values after each trial. Branin is a two-dimensional synthetic function. A single seed on a smooth analytic surface tells you almost nothing about how the sampler behaves on a noisy validation metric from a real training run, where evaluation variance can exceed the gap between good and bad configurations. The README itself labels the material a trajectory illustration. Treat the figure as evidence that the loop runs end to end, not as evidence that agent proposals beat TPE. If you want to know whether this helps your problem, the honest test is to run the same objective with AgentSampler and with a standard sampler at the same trial count, on several seeds, and compare best_value. The library makes that comparison cheap because the objective function is unchanged; only the sampler argument differs.
Costs the README does not quantify: agent latency, token spend, and version drift
Every proposal is an agent invocation. That means each trial pays a latency cost that a local sampler does not have, and the wall-clock time of a 20-trial study is dominated by model calls rather than by your objective. The README frames this as upside: proposal quality can improve as the underlying coding agents improve, without changing your optimization code. The same sentence is a maintenance liability. Your tuning results are tied to a specific model version behind a CLI you do not pin, so a study run in July and a study run in October are not strictly comparable even with identical code and seeds. The effort="high" setting implies a further trade of latency for proposal quality, though the documentation does not state what the levels are or how they differ. There is also a reproducibility question the material does not answer: whether a stored study records enough about the agent invocation to replay a proposal. The README says agent rationale can be retained, which helps, but rationale is not a seed. Budget for the agent calls as a real line item before you plan a large sweep.
Where it is the wrong tool, and what to use instead
If your evaluations are cheap, optim-agent is the wrong choice. A gradient-boosted surrogate over hundreds of trials will beat a handful of agent proposals on a smooth, low-dimensional space, and it will do so without network calls, without a CLI dependency, and without per-trial cost. Optuna's TPE sampler is the direct alternative, and the difference is not cosmetic: TPE builds an explicit density model over the trial history and samples from it, so it improves mechanically as data accumulates and it is fully deterministic given a seed. optim-agent substitutes a language model that reasons over parameter descriptions, which is more informative when history is thin and less predictable when history is plentiful. The crossover point is not documented, and you should not assume it favors the agent. Two other cases argue against adoption. Air-gapped or regulated environments where outbound calls from a training job are prohibited cannot use it at all, since the whole mechanism depends on a hosted coding agent. And objectives with no meaningful parameter semantics, such as tuning a set of opaque integer indices, remove the one input the agent has that TPE lacks, leaving you with a slower sampler and no compensating advantage.
Licence, versioning, and what a 0.1.x dependency means for your pipeline
The project is MIT licensed, which permits commercial use and modification with the usual requirement to preserve the copyright and licence notice. That is the permissive end of the spectrum and imposes no copyleft obligation on your own code. This is not legal advice; check the LICENSE file and your organisation's policy. The versioning signals are more informative than the licence. The only release listed is 0.1.1 from July 2026, and the repository was last pushed in August 2026. A 0.1.x line means the public API can change between minor versions, and the surface you depend on is broad: create_study arguments, AgentSampler keyword names, the suggest_float and suggest_int signatures including the context parameter, and the storage format. Pinning optim-agent to an exact version in your requirements file is the cheap insurance. The storage format matters most, because a study.json written by 0.1.1 is your accumulated evaluation record; if the schema shifts, you want to know before you overwrite it. Keep the objective function in your own repository rather than inside a study artifact, so that a library upgrade never puts your domain code at risk.
Editorial conclusion
Adopt optim-agent when evaluations are expensive, your parameter count is small enough that a handful of trials matters, and your parameters carry meanings a coding agent can read from your code. Skip it when you need hundreds of cheap trials, when no agent CLI is authenticated on the machine, or when every evaluation must run in an isolated environment with no outbound calls. Before committing, verify three things: that a study round-trips through study.json with storage= set, that an intentionally malformed agent reply actually falls back to sampling rather than crashing, and that the agent's latency per proposal fits inside your wall-clock budget.
Community notes