Adala: an agent framework that learns labeling skills from a ground truth dataset

The problem Adala targets: labeling that has to match your existing labels

Most LLM labeling scripts start from a prompt written by hand. You describe the task, run it over a few hundred rows, spot-check the output, and adjust the wording. The prompt is the artifact, and it has no relationship to the labels you already trust.

Adala inverts that. The environment is a ground truth dataset, and the agent iteratively develops a skill against it. The README describes the agents as autonomous, acquiring skills "through iterative learning" influenced by their operating environment, observations, and reflections. In practice that means you supply labeled examples in a pandas DataFrame and the framework works on the instructions rather than asking you to tune them by hand.

The intended audience is stated fairly narrowly in the README: AI engineers building agent systems, ML researchers experimenting with problem decomposition, data scientists who want to preprocess and postprocess data from a notebook, and educators. The common thread is that you already have labels. If you do not, Adala has nothing to learn from, and the framework's central mechanism is inert.

How the pieces fit: environments, skills, runtimes and teachers

Four objects carry the design. An environment holds the training data. In the quickstart it is StaticEnvironment(df=train_df), a DataFrame of examples. A skill declares what the agent produces: a name, natural-language instructions, a set of labels, an input_template and an output_template. A runtime is the model that executes the skill, and the README treats the runtime as synonymous with the LLM, so a single skill can be deployed across multiple runtimes. The runtimes dictionary in the quickstart maps a name to an OpenAIChatRuntime with a model such as gpt-4o.

The teacher_runtimes dictionary is the part worth pausing on. It is separate from runtimes, which is what makes the student/teacher arrangement possible: one model can act as the teacher that shapes the skill while another executes it. The README lists this as a feature of the flexible runtime, and the quickstart shows both dictionaries populated with OpenAIChatRuntime.

Skills are not limited to classification. The examples directory contains notebooks for text generation, summarization, translation, question answering, code generation, ontology creation, and a skillset that runs a sequence of skills. ClassificationSkill is the one shown in the README quickstart, with labels, input_template and output_template.

There is also a server side. The repository ships a docker-compose.yml with kafka, app, worker and redis services, and pyproject.toml pulls in fastapi, celery[redis], uvicorn and redis-om. A Dockerfile.app builds the image. The app service maps port 30001 on the host to 8000 in the container and runs uvicorn app:app; the worker runs celery against stream_inference.app. Kafka is configured with KAFKA_ENABLE_KRAFT=yes and KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE=false, so topics have to exist before the worker consumes them.

Installing Adala and running the sentiment example

The README gives three install paths. The plain one is from PyPI:

pip install adala

The README notes that Adala releases frequently and recommends installing from GitHub to get the most recent version:

pip install git+https://github.com/HumanSignal/Adala.git

For work on the framework itself, the developer install uses poetry:

git clone https://github.com/HumanSignal/Adala.git
cd Adala/
poetry install

Note the Python requirement in pyproject.toml: requires-python is ">=3.10,<4". The README badge still lists 3.8 through 3.11, which no longer matches the package metadata. Trust pyproject.toml.

Before running anything you need a key. The README's prerequisites section exports it:

export OPENAI_API_KEY='your-openai-api-key'

The quickstart then builds an agent from a training DataFrame, a classification skill, and a runtime. The skill declares the labels and the templates that wrap each row:

from adala.agents import Agent
from adala.environments import StaticEnvironment
from adala.skills import ClassificationSkill
from adala.runtimes import OpenAIChatRuntime

agent = Agent(
    environment=StaticEnvironment(df=train_df),
    skills=ClassificationSkill(
        name='sentiment',
        instructions="Label text as positive, negative or neutral.",
        labels=["Positive", "Negative", "Neutral"],
        input_template="Text: {text}",
        output_template="Sentiment: {sentiment}"
    ),
    runtimes={'openai': OpenAIChatRuntime(model='gpt-4o')},
    teacher_runtimes={'default': OpenAIChatRuntime(model='gpt-4o')},
)

What you should see is an agent object wired to your training rows. The README's quickstart stops short of showing the call that runs the agent over test_df, and the notebook at examples/quickstart.ipynb is where the extended version lives. Read that notebook before writing your own loop.

Where Adala gets in the way

The version number is the first honest signal. pyproject.toml declares version 0.0.4dev, and the most recent tagged release, 0.0.4, dates from 2023-11-30. The last push to master was on 2026-09-03, so the repository is not abandoned, but the release cadence and the dependency pins tell you the API is still moving. pandas is pinned to exactly 2.2.3, guidance to 0.0.64, and openai to a 2.x range. Those pins exist for a reason, and they will collide with a project that needs a different pandas.

The dependency list is heavy for what looks like a notebook library. chromadb, datasets, aiokafka, boto3, gspread, celery, redis-om and fastapi all install by default, because the server extras are declared as regular dependencies. The pyproject.toml comment explains why: poetry could not install them as extras with version strings, and dev groups would not ship with a pip install. The consequence is that a data scientist who only wants StaticEnvironment and OpenAIChatRuntime still pulls a Kafka client and a vector store.

Two more boundaries. First, the framework needs ground truth. The README's claim that agents are built on a foundation of ground truth data is also the constraint: no labeled examples, no environment, no learning. Second, the documentation does not describe what happens when a skill fails to converge, and the README does not document rollback for a skill that regresses. If your workflow needs a guaranteed fallback to a previous prompt, that is on you to build.

Finally, the server path is not a small addition. Running docker-compose.yml brings up Kafka, Redis, a Celery worker and the FastAPI app, with topics that must be created manually because auto-creation is disabled. If you only need batch labeling in a notebook, do not start there.

Adala compared with plain prompt engineering and with a labeling platform

The closest alternative is not another agent framework. It is a script that calls the OpenAI API with a hand-written prompt and a loop over a DataFrame. That approach has no environment abstraction, no skill object, and no teacher runtime, but it also has no dependency on chromadb or celery, and the prompt is a string you can diff and review. Adala's difference is that the skill is a declared object with labels and templates, and the training data is a first-class input rather than a validation set you check afterward. If your labels are already stable and your prompt already works, Adala adds machinery without adding accuracy.

The other alternative is a labeling platform rather than a library. The README's own commented-out note points at Label Studio for human-in-the-loop labeling and for producing ground truth datasets, and says Adala supports Label Studio format out of the box. That is a different division of labor: Label Studio is where humans produce and correct labels, Adala is where an agent reproduces them. pyproject.toml pulls label-studio-sdk from a pinned GitHub archive URL, which confirms the integration is real but also that it tracks a specific commit rather than a released version. If your bottleneck is human review throughput, the platform is the answer. If your bottleneck is applying an existing rubric consistently, Adala is the more direct fit.

Maintenance, licence and what an upgrade costs

The repository is not archived, and the last push to master was on 2026-09-03, which is recent enough that the codebase is being touched. That is a statement about activity, not about stability. The version string 0.0.4dev and a release history that stops at 0.0.4 in November 2023 mean you should read the source rather than assume the documented API is frozen.

Upgrading has two cost centers. The first is the pinned dependency set in pyproject.toml. A move to a newer pandas or a different openai major version is not a version bump you can take casually, because the pins were chosen to make the current code work. The second is the server stack: Kafka, Redis, Celery and the FastAPI app each carry their own upgrade path, and the docker-compose.yml pins the Kafka image to bitnamilegacy/kafka:4.0.0-debian-12-r10. Treat the library and the server as two separate upgrade decisions.

The licence is Apache-2.0, declared in pyproject.toml and in the LICENSE file, with the classifier "License :: OSI Approved :: Apache Software License". Apache-2.0 is permissive and includes an explicit patent grant, which matters if you are embedding the framework in a commercial product. It also carries attribution and notice requirements. Read the LICENSE text yourself and involve counsel if the distinction matters to your organization; nothing here is legal advice.

Editorial conclusion

Adala fits teams that already have a labeled dataset and want an LLM agent to reproduce that labeling on new rows, especially when the task is classification, generation or extraction over a DataFrame. It is the wrong tool if you have no ground truth to define the environment, if you need a hosted labeling UI rather than a library, or if you expect a stable API: the package is still at 0.0.4dev. Before committing, check pyproject.toml for the exact pins (pandas 2.2.3, guidance 0.0.64, openai 2.x), confirm your Python is 3.10 or newer, and read the quickstart notebook to see whether the skill you need already exists or has to be written as a custom class.

Frequently asked questions

Official sources

1. HumanSignal/Adala on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases