LLM4AD: a Python platform that treats algorithm design as an LLM search problem
LLM4AD: A Platform for Algorithm Design with Large Language Model
At a glance
- What is it?
- LLM4AD wraps task evaluation, LLM access and search methods behind one interface so you can let a model write and refine algorithms. It is aimed at researchers, and the setup cost sits in your API key and your evaluation function, not in the install.
- Who is it for?
- Adopt LLM4AD if you already have a scored task and want a search method such as EoH driving an LLM against it, and if you accept an HTTP API key as a dependency. Do not adopt it if your evaluation cannot be made fast, deterministic and safe to run in a subprocess, or if you need a language other than Python, which the feature table still lists as coming soon.
- Can I use it commercially?
- Yes. BSD-2-Clause is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 78 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap LLM4AD fills between a prompt and a scored algorithm
Most people using an LLM to write code run a loop by hand: paste a problem, read the answer, test it, paste the failure back. That loop has no shared interface, so every new problem means rewriting the harness. LLM4AD's stated purpose is to be a platform for automatic algorithm design, which in practice means it supplies the harness. It gives you unified interfaces for methods, tasks and LLMs, so an LLM client, a search method and an evaluation function can be swapped independently. The README describes it as originally developed for optimisation tasks, with the framework described as usable in machine learning, science discovery, game theory and engineering design. The audience is therefore researchers and engineers who have a task with a numeric score and want to see whether an LLM can produce a better algorithm for it. If you do not have a scorer, this platform has nothing to optimise against.
How a run is assembled: LLM client, evaluation function, search method
The quick start in the README shows the three pieces meeting in one file. You import a task evaluation class, in the example OBPEvaluation from llm4ad.task.optimization.online_bin_packing, an LLM client, HttpsApi from llm4ad.tools.llm.llm_api_https, and a search method with its profiler, EoH and EoHProfiler from llm4ad.method.eoh. The client is constructed with a host, a key and a model. The evaluation object is what turns a candidate algorithm into a number the search method can compare, and the search method is what decides which candidates get proposed next. That separation is the whole architecture: the method never talks to the network directly, and the task never knows which method is running. The feature table lists multiprocessing evaluation, main process protection and timeout interruption under secure evaluation, which tells you the intended execution model. Candidate algorithms are generated as text, evaluated somewhere isolated from the main process, and results are fed back. The README also lists local logs plus Wandb and Tensorboard support, and a resume run capability, so a long search is not lost if it stops. A GUI is listed as supported, covering method selection, task selection, convergence and best algorithm.
Installing it and pointing it at a model
Python must be at least 3.9 and below 3.13; the README marks this as a hard requirement. Two install paths are given. From a clone: cd LLM4AD then pip install . The PyPI route is a single pip install llm4ad. The README recommends a conda environment in both cases. Optional dependencies are split by feature, which matters because the base install is not enough for everything: numba for accelerated evaluation, tensorboard or wandb for those loggers, gym if you want the GUI or machine learning tasks, and pandas for science discovery tasks. The README warns that the gym version may conflict with your own Python environment and points you at gym's own documentation to pick a compatible one. Configuration happens in code rather than a config file. The quick start sets host to something like api.deepseek.com, key to your API key, and model to a name such as deepseek-chat, all passed to HttpsApi. The README notes you must configure your LLM API before running the script. There is an Open In Colab badge for the online bin packing tutorial, which is the cheapest way to see the loop work before installing anything locally.
Evaluation speed and process isolation are the real constraints
The feature table advertises multiprocessing evaluation and timeout interruption as supported, which is a statement about what the platform expects your code to survive. An LLM search method may propose many candidates per iteration, and each one has to be executed to get a score. If a single evaluation takes minutes, the search becomes impractical regardless of how good the method is. Timeout interruption also implies that a candidate can be killed, so an evaluation function that writes files or holds external resources mid-run can be interrupted in a state you did not plan for. This is the case where LLM4AD is the wrong tool: a task whose correctness cannot be reduced to a number, or whose evaluation is slow, non-deterministic, or unsafe to run in a subprocess. The README also lists support for other programming languages as coming soon, so the platform is Python-only today, and more search methods and more task examples are likewise listed as coming soon rather than present.
Where it sits next to a plain evolutionary or prompt loop
The obvious alternative is to write the loop yourself: call an LLM API, extract code from the response, exec it, score it, and build the next prompt from the result. That is maybe a hundred lines and it has no dependencies beyond your HTTP client. The difference is what you give up. A hand-rolled loop has no resume, no multiprocessing evaluation, no timeout interruption, no Wandb or Tensorboard logging, and no GUI, all of which LLM4AD lists as supported. It also has no shared interface, so moving from online bin packing to another task means rewriting the harness. LLM4AD's value is that the task evaluation class is the only thing you write, and the method and client are pluggable. The cost is a heavier dependency set and a fixed execution model. If your evaluation is trivially fast and you only want to try one prompt pattern once, the hand-rolled loop is less machinery. If you intend to compare search methods across several tasks, the interface is the point.
Maintenance, versioning and the BSD-2-Clause licence
The repository is not archived and the last push is dated 2026-06-30, with v1.0.0 released on 2025-03-20. The README carries a maintained badge and an explicit invitation for pull requests, and the news section records activity through 2026, including a CVRPLib BKS competition result and a survey paper accepted by ACM Computing Surveys. That is evidence of ongoing work, not a guarantee about any particular task module. The practical upgrade cost is the Python ceiling: the README requires Python below 3.13, so a project on a newer interpreter cannot install it without a separate environment. Optional dependencies add their own constraints, and the README itself flags gym version conflicts as a thing you resolve on your side. On licensing, the project is BSD-2-Clause, a permissive licence that generally allows use and redistribution with the copyright notice and disclaimer retained, but the text of the licence in the repository is what governs and this is not legal advice. One thing worth checking before you depend on it: the README does not state what happens to a run in progress when the LLM host changes its API surface, so pin your client configuration deliberately.
Editorial conclusion
Adopt LLM4AD if you already have a scored task and want a search method such as EoH driving an LLM against it, and if you accept an HTTP API key as a dependency. Do not adopt it if your evaluation cannot be made fast, deterministic and safe to run in a subprocess, or if you need a language other than Python, which the feature table still lists as coming soon. Before committing, verify three things in your own environment: that your Python version is 3.9 or higher and below 3.13, that the Open In Colab tutorial for online bin packing runs against your LLM host, and that your evaluation function survives the timeout interruption path rather than being killed mid-write.
Community notes