Agent-E: browser automation built on the AG2 agent framework

What the system actually automates

Agent-E is described as an agent based system that aims to automate actions on the user's computer, with the browser as the current target. The README is specific about the shape of the tasks it claims: filling out web forms (web forms, explicitly, not PDF forms) using information about you or pulled from another site, searching and sorting products on e-commerce sites by criteria such as bestsellers or price, and locating specific details on sites ranging from sports scores to university contact pages.

The list continues into media and project management. Playing YouTube videos and managing playback settings such as full screen and mute, running broad web searches, filtering issues on project management platforms like JIRA, and giving shopping suggestions based on stated needs such as storage for game cards. Whether any particular one of these works well is an empirical question the README does not attempt to answer with numbers, and it does not pretend to: the framing is that the best task is the one you come up with yourself.

Underneath, the README states the system is based on the AG2 agent framework, and the dependency list in `pyproject.toml` confirms it with `autogen~=0.7` alongside the `autogen[anthropic]` and `autogen[groq]` extras. Both extras are declared, which tells you Anthropic and Groq are first class paths rather than afterthoughts, and the README's own note points at `gpt-4-turbo` for optimal performance when setting `AUTOGEN_MODEL_NAME`.

One script installs it, uv manages everything underneath

There is a single command for macOS and Linux, run from the project root:

./install.sh

Passing `-p` adds Playwright without further prompting, which is the variant you want on a machine without a local Chrome install. Windows has its own script, `win_install.ps1`, taking the same `-p` flag in PowerShell.

What sits behind that script is spelled out in the manual setup section, and it is worth reading because it shows the dependency strategy. The project uses uv to manage the virtual environment and package dependencies. The environment is created for Python 3.11, with a note that 3.10 and newer should also work, and `requirements.txt` is generated from `pyproject.toml` rather than maintained by hand:

uv pip compile pyproject.toml -o requirements.txt
uv pip install -r requirements.txt

Development extras are a separate step, `uv pip install -r pyproject.toml --extra dev`, which the pyproject defines as ruff for linting plus sphinx and sphinx-rtd-theme for docs. Generated extras are pinned to exact versions in `requirements.txt`: `pydantic==2.6.2`, `playwright==1.44.0`, `python-dotenv==1.0.0`, `fastapi==0.111.1`, `uvicorn==0.30.3`, `nest-asyncio==1.6.0`.

Playwright or your own Chrome profile, and why it matters

The Playwright path exists for machines without Chrome. If you would rather drive a real browser with your real logged in session, the README gives you the setting:

playwright install

Then set `BROWSER_STORAGE_DIR` to the path of your Chrome profile, found by opening `chrome://version/` in Chrome. The README is explicit that this is for using a local Chrome instance instead of the Playwright browser.

This is the single most consequential configuration decision in the project and it is easy to skim past. A large share of real browser automation tasks involve a site you are already logged into, and an agent driving a fresh Playwright browser has no cookies, no session and no history. Pointing it at your own profile changes what is possible and also what you are risking, since that profile is the one holding your authenticated state. The variable's existence is documented; the security conversation around it is not.

`SAVE_CHAT_LOGS_TO_FILE` defaults to `true` and decides whether the conversation goes to a file or to standard output. `LOG_MESSAGES_FORMAT` accepts `json` or `text` and defaults to `text`. `ADDITIONAL_SKILL_DIRS` takes a comma separated list of directories or `.py` files to load extra skills from, which is the documented extension point if you want to teach it something new.

The configuration you must fill in before it runs

Running the system is one module invocation, and macOS users are pointed at the unbuffered variant so output appears immediately:

python -m ae.main

python -u -m ae.main

Before that works, two files need attention: `.env` and `agents_llm_config.json`. The README walks through creating the first by copying the example, `cp .env-example .env`, and both the script and the manual path assume it exists.

The required pair is `AUTOGEN_MODEL_NAME` and `AUTOGEN_MODEL_API_KEY`. Three optional variables exist for anything that is not OpenAI: `AUTOGEN_MODEL_BASE_URL` for a different endpoint host, `AUTOGEN_MODEL_API_TYPE` for the provider style (`azure` is the worked example), and `AUTOGEN_MODEL_API_VERSION` for a versioned API. The README warns against putting `/completion` in the base URL, which is the sort of detail that saves an hour when a request 404s. Sampling controls are `AUTOGEN_LLM_TEMPERATURE`, defaulting to `0.0` for `gpt-*` models, and `AUTOGEN_LLM_TOP_P`, defaulting to `0.001`.

Those two sampling defaults are worth pausing on. Near zero temperature and near zero top-p together mean the model is being asked for its highest confidence token at nearly every step. For a system that drives a browser, that is the right instinct: you want the agent to do the obvious thing rather than explore. It is also why the project is sensitive to page layout changes.

What lives in the repository beyond the agent

The tree is small, which makes it readable. `ae/` holds the package, `docs/` the documentation, `test/` the tests, `scripts/` helper scripts, and the root holds `install.sh`, `run.sh` and `win_install.ps1`. There is a `.check-env-example` file at the top, which suggests the project ships a way to verify your environment file matches the example rather than leaving you to diff them by eye.

The dependency list says more about the architecture than the README does. Alongside Playwright and AG2 you get `nest-asyncio`, which is how an async agent loop gets driven from synchronous entry points, and FastAPI with uvicorn, which points to an HTTP surface somewhere in the system. `python-json-logger` is declared for structured log output. There are also `pdfplumber`, `nltk` and `tabulate`, and notably no PDF support in the browser task list, so their presence suggests some of the document work is further along than the task descriptions suggest.

There is a research paper, on arXiv at 2407.13032, and the README attaches a caveat to it that is worth reading before you cite the numbers: the WebVoyager validation used a branch named `nested_chat_for_hierarchial_planning` and GPT4-Turbo, not the default branch. The typo in that branch name is in the upstream repository.

Where to look when it does not do what you expected

The honest limitation of this project is that its accuracy lives outside the repository. The README gives you a list of tasks it is equipped for and an invitation to try your own, with a blog post and a Discord linked for support. It does not publish a pass rate for any of those tasks on the current branch, and the one benchmark number it does cite came from a different branch and a specific model.

So the debugging path matters. Three layers can be at fault and they fail differently. If the agent takes the wrong action, the prompt or the model is the suspect, and `AUTOGEN_LLM_TEMPERATURE` is the knob that trades determinism for variety. If the agent cannot see the page or click the button, Playwright and the browser choice are the suspect, and `--headless` behaviour or a missing browser install is where to look. If the agent gets to the right page and still stops, the configuration is the suspect, most often the model name, API key or base URL in `.env`.

The `test/` directory and the dev extras give you somewhere to put your own regression case. Given that the system's value is entirely in the tasks it completes, capturing a failing task as a test before you change anything is the cheapest way to tell a model problem from a page problem. The paper and blog post are linked from the README for background on the design; the actual behaviour you care about is what you observe on the page in front of you.

Editorial conclusion

Agent-E is a research artifact with a real install path, which is an unusual combination and the main reason to read it. The AG2 foundation means you are not choosing the agent loop, only the task description, so the interesting question is whether your task survives contact with a page that has a login form. Two things to verify before anything else: whether the page you care about needs a real Chrome profile with your logged in session, which the README supports through BROWSER_STORAGE_DIR but does not automate, and whether the pinned dependency versions still install on your Python. The pyproject requires Python 3.10 or newer and pins onnx style exact versions elsewhere in the ecosystem, so treat the lockfile as the source of truth rather than the version ranges. If you need a task that runs unattended on a schedule, this repository does not claim that and does not pretend to.

Frequently asked questions

Official sources

1. EmergenceAI/Agent-E on GitHub
2. Issues
3. License: MIT
4. README