Adala: a labeling agent framework that learns its prompt from your ground truth

The gap Adala fills: prompts that are guessed versus prompts that are fitted

Most LLM labeling work starts with someone writing an instruction string, running it over a sample, reading the mistakes, and editing the string. That loop is manual, and the artifact it produces is a prompt that lives in a notebook cell or a config file with no recorded relationship to the examples that shaped it. Adala's premise is that the labeled examples you already have should be the thing that produces the instruction, not a person's intuition about what the instruction should say. The README frames this as agents that "can independently acquire one or more skills through iterative learning," with the learning "influenced by their operating environment, observations, and reflections." The environment, in Adala's vocabulary, is your ground truth dataset. So the input to the framework is a DataFrame with labels, and the output is a skill whose instructions have been adjusted against that DataFrame. The audience the README names is specific: AI engineers building agent systems, ML researchers experimenting with problem decomposition, data scientists who want to work "natively through Python notebooks when working with large Dataframes," and educators. The common thread is that all of them already have labels. If you do not have a ground truth set, Adala's central mechanism has nothing to fit against, and you are better served by a plain prompt and a manual eval loop.

Agent, Skill, Environment, Runtime: the four objects in the quickstart

The quickstart imports four things and shows how they fit. Agent is the container. Environment is the ground truth: StaticEnvironment(df=train_df) wraps a pandas DataFrame. Skill is the unit of work: ClassificationSkill takes a name, an instructions string, a labels list, an input_template and an output_template. Runtime is the model connection: OpenAIChatRuntime(model='gpt-4o'). The Agent constructor takes environment, skills, runtimes and teacher_runtimes as separate arguments, which is the detail worth pausing on. Runtimes is a dictionary of the models the skills may use at inference time. Teacher_runtimes is a separate dictionary. The README describes this as a "student/teacher architecture," where a single skill can be deployed across multiple runtimes. The practical reading is that the teacher runtime is what drives the learning step and the student runtime is what runs the learned skill over new data, so you can fit with an expensive model and apply with a cheaper one. The README does not spell out the exact call sequence that connects them, and I have not run it, so treat that division of labor as the documented intent rather than a verified behavior. The input_template and output_template pair is the other mechanism worth noting: input_template="Text: {text}" and output_template="Sentiment: {sentiment}" show that the skill controls both how a row is rendered into the prompt and how the model's answer is parsed back into a column. That is a string-formatting contract, and it means your DataFrame column names have to match the placeholders.

Installation, the API key, and what the environment variable actually gates

Installation is a single pip command, pip install adala, with the README recommending the GitHub form, pip install git+https://github.com/HumanSignal/Adala.git, on the grounds that releases are frequent. That recommendation is a signal about release cadence, and it is also a warning: pulling from master means you are tracking an unreleased state. For development work the README gives a poetry path: git clone the repository, cd Adala/, then poetry install. The only prerequisite listed is OPENAI_API_KEY, set with export OPENAI_API_KEY='your-openai-api-key'. The quickstart comments note that you can instead pass the key per runtime, as OpenAIChatRuntime(..., api_key='your-api-key'), which matters if you want different keys for the student and teacher runtimes. The supported Python versions are given in the README badge as 3.8 through 3.11. One thing to check before committing: the README's commented-out section mentions that Adala supports Label Studio format out of the box and suggests installing Label Studio if you need a human-in-the-loop labeling tool or a way to produce the ground truth set in the first place. That block is commented out in the README, so treat the Label Studio integration as documented-but-de-emphasized rather than a headline feature.

The 0.0.x version number is the most important line in the repository

The releases listed are 0.0.2, 0.0.3 and 0.0.4, dated November 2023, with 0.0.4 described as introducing "a New Code Environment and Enhanced Automated Prompt Engineering." A 0.0.x series means the maintainers have made no compatibility promise, and the release notes themselves show churn in the core surface: 0.0.2 introduced Skill Sets, 0.0.3 extended feedback and control plus parallel execution, 0.0.4 added a code environment. Those are not additive conveniences; each one touches how skills are defined and executed. If you build on Adala, pin the version and expect to read the changelog before every bump. The second limitation is provider coverage. Every runtime named in the README is OpenAI: OpenAIChatRuntime appears in both the runtimes and teacher_runtimes dictionaries in the quickstart, and the prerequisite is an OpenAI key. A commented-out line mentions "providers like OpenAI and VertexAI," but that line is inside an HTML comment, so VertexAI support is not something the README commits to. The README does say the framework "invites the community to extend and tailor runtimes," which is the honest framing: if you need a non-OpenAI model, you are writing the runtime adapter yourself. The third limitation is the cost profile. Iterative learning against a ground truth set means repeated model calls before you label a single production row, and the README gives no token budget, iteration count, or stopping criterion. That is the number you need to measure on your own data before scaling up.

Where Adala is the wrong tool

Adala is a poor fit when the labeling task has no reusable ground truth. If your categories are still being defined, or if every batch needs a different judgment call, there is nothing stable for the learning step to converge on, and you are paying for iterations that will be discarded. It is also the wrong tool when the output needs to be a review queue rather than a column. Adala writes labels into a DataFrame; the README's human-in-the-loop path points at Label Studio as a separate installation, and that block is commented out. A team that needs annotators to accept, reject and correct model output is looking at a labeling platform, not at this library. Third, if your organization requires a specific model provider and the runtime for it does not exist, you are signing up to maintain an adapter against a 0.0.x API. Finally, the quickstart's own example is a three-class sentiment task on six training rows. That is a demonstration, not a scale test. Nothing in the README describes how the learning step behaves as the ground truth set grows into the thousands, and I cannot tell you from the material whether it samples, batches, or uses every row.

How this differs from writing a prompt and calling an LLM directly

The obvious alternative is a thin wrapper: build your prompt string, call the model, parse the response, and keep the prompt in version control. That approach is more work to set up for a single task and far less work to debug, because the prompt is a literal you can read. Adala's difference is that the prompt becomes a derived artifact. You supply instructions as a starting point and labels as a target, and the framework adjusts the instructions between them. The trade is legibility for adaptability: with a hand-written prompt you always know what the model was told, and with Adala you know what it was told initially and what data it was tuned against. If your task is stable and your prompt works, the wrapper wins on simplicity. Adala earns its place when you have several related labeling tasks sharing a runtime, when the instruction text is genuinely hard to write by hand, or when you want to swap the underlying model without rewriting the prompt. A second alternative is a general agent framework with tool-calling, where labeling is one tool among many. Adala's scope is narrower: the README describes agents "specialized in data processing, with an emphasis on diverse data labeling tasks." Narrow scope here means fewer moving parts and a smaller API to learn, at the cost of flexibility if your pipeline later needs retrieval, multi-step tool use, or external API calls inside the skill.

Licence, maintenance and what to check before you depend on it

Adala is Apache-2.0, which permits commercial use and modification and includes a patent grant. That is a permissive licence, and it is the reason a 0.0.x dependency is a survivable bet rather than a trap: if the project stalls, you can fork it. The obligations are the usual ones, and if you redistribute a modified version, read the licence text rather than a summary. On maintenance, the material shows three releases in November 2023 and a last push date of September 2026, with the repository not archived. Those two facts are hard to reconcile without more information, and I am not going to guess at what the gap means. What I can say is that the released versions and the current branch state are not necessarily the same thing, which is exactly why the README suggests installing from GitHub. Before adopting, check three things against the repository rather than against this article: whether the skills you need exist in the current release or only on master, which runtimes ship in the box, and whether the learning step's iteration behavior is documented anywhere outside the quickstart notebook. If the answers hold up, the abstraction is worth the setup. If they do not, a prompt string in a file is still a perfectly good labeling pipeline.

Editorial conclusion

Adala fits teams that already hold a labeled DataFrame and want an LLM labeling prompt derived from it rather than written by hand, and who are comfortable pinning a 0.0.x dependency. It does not fit anyone who needs a stable API surface, a non-OpenAI runtime out of the box, or a labeling tool with a human review interface. Verify first that the skills you need exist in the current release, that your runtime provider is supported, and what the iterative learning step costs in tokens before you point it at a large unlabeled table.

Official sources

1. HumanSignal/Adala on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases