# webllama: Llama 3 Web Browsing Agents

> McGill-NLP/webllama provides a Llama 3 model, training data, and evaluation tools for dialogue-guided browser navigation. It serves research teams, while leaving browser execution and action parsing to the adopter.

**McGill-NLP/webllama** — Llama-3 agents that can browse the web by following instructions and talking to you

- Repository: https://github.com/McGill-NLP/webllama
- Website: https://webllama.github.io
- Stars: 1,398 · Forks: 100
- Language: Python
- License: MIT
- Published: 2026-10-16 · Updated: 2026-10-16 · Language: en
- Canonical page: https://hysenlabs.com/projects/mcgill-nlp-webllama

## webllama's Web Browsing Problem

McGill-NLP/webllama addresses a specific form of browser automation: an agent interprets instructions and dialogue, examines a textual web state, and decides what to do next. The README describes the project as a way to build human-centric agents powered by Meta Llama 3. Its stated goal is to assist people while they browse, not to remove them from the process.

The project is aimed mainly at engineers and researchers working on web navigation, conversational agents, browser benchmarks, and model evaluation. The repository contains separate areas for modeling, data, evaluation, applications, tests, and examples. The example tree includes `examples/browsergym/`, `examples/complete/`, and `examples/web_api/`. The README names Playwright, Selenium, and BrowserGym as deployment environments, but it also says that making those integrations work remains a focus.

The first model is `McGill-NLP/Llama-3-8B-Web`, which is based on `Meta-Llama-3-8B-Instruct`. According to the README, it was fine-tuned on a curated 24K-example subset of `McGill-NLP/WebLINX`. The full dataset is described as containing more than 100K navigation and dialogue instances collected and verified by expert annotators. This makes webllama a research-oriented framework for learned browser actions, not a conventional script library for a fixed sequence of clicks and form submissions.

## WebLlama State and Action Pipeline

The documented inference path begins with a dataset record rather than a live screenshot. A validation example is loaded from `McGill-NLP/WebLINX`, the repository downloads the dataset's `templates/*` files, and `templates/llama.txt` formats the record into a textual browser state. That state is passed to a Hugging Face `transformers` pipeline configured with `McGill-NLP/Llama-3-8b-Web`.

The model then generates text. The README describes this output as an action, but the browser-specific interpretation is deliberately left outside the model call. Its example uses `process_pred(out['generated_text'])` and comments that the function should be implemented for the chosen platform. The converted value is passed to `env.step(action)`, where the surrounding environment applies it.

This separation has one clear benefit. The learned policy does not need a direct dependency on a particular browser-control library. The same model output can potentially be translated for Playwright, Selenium, or BrowserGym. The cost falls on the adopter, who must define the action schema, validate generated operations, resolve page elements, and handle execution failures. The README does not document a universal parser or a production-ready safety layer for live websites, so `process_pred` is an integration point, not a finished WebLlama function.

## WebLINX Training and Evaluation Scope

WebLlama's first training target is WebLINX 1.0. The README says the curated training set contains 24K web interaction examples, while the complete dataset contains more than 100K navigation and dialogue instances. Documented interaction types include `click`, `textinput`, `submit`, and dialogue acts. That limited vocabulary explains why state formatting and output parsing are central parts of the integration.

The benchmark covers 150 websites, according to the README. Its examples include booking, shopping, writing, knowledge lookup, and spreadsheet manipulation. The project describes four test splits intended to measure generalization to new websites, new domains, unseen geographic locations, and cases where the user cannot see the screen and must rely on dialogue. Those dimensions are more relevant to web agents than a single scripted task because they test changes in both site context and communication conditions.

For `Llama-3-8B-Web`, the README reports an overall score of 28.8% on the out-of-domain WebLINX test splits, compared with 10.5% for zero-shot GPT-4V. It also reports link selection, relevant element clicking, and dialogue response metrics. These are project-reported WebLINX results. They do not establish reliability on an arbitrary enterprise dashboard or on sites with different controls and page structures.

The documented evaluation starts in `modeling/`. It searches for `results.jsonl` files and writes `aggregated_results.json`. Readers should see that aggregate file after the evaluator completes.

## First Hugging Face Inference Example

The README does not provide a concrete installation command. It does show that the project has `setup.py`, `requirements-basic.txt`, and `requirements-extra.txt`. The package metadata declares `webllama`, requires Python 3.8 or later, and separates basic dependencies from an `extra` group. Because no documented pip command is available, the repository files themselves are the authoritative installation reference rather than an invented shell sequence.

The first real example loads one validation record, downloads the templates, builds a textual state, creates the model pipeline, and prints its generated action. The code also reserves `process_pred` and `env.step` for the user's browser platform. With those two functions and `env` supplied by an integration, this is the shortest documented model inference path.

```python
from datasets import load_dataset
from huggingface_hub import snapshot_download
from transformers import pipeline

valid = load_dataset("McGill-NLP/WebLINX", split="validation")
snapshot_download("McGill-NLP/WebLINX", repo_type="dataset", allow_patterns="templates/*")
template = open('templates/llama.txt').read()
```

This block prepares the validation record and the local template file. A successful template download creates a local `templates` directory containing the dataset's template assets.

```python
state = template.format(**valid[0])
agent = pipeline("McGill-NLP/Llama-3-8b-Web")
out = agent(state, return_full_text=False)[0]
print("Action:", out['generated_text'])
```

This block formats the first state and prints generated text. The output represents the model's proposed action, not a browser operation that has already been executed. The README's following lines describe converting that text for Playwright or BrowserGym and passing it to the environment.

## Running the WebLINX Results App

Evaluation and visual inspection are separate steps in the repository. The evaluator must already have prediction data saved as `results.jsonl`, because it discovers those files and produces a combined results file. The README does not define the prediction schema or explain how to generate `results.jsonl` for an arbitrary model.

The application is then launched from the repository root. The reader should see the Streamlit interface for `app/Results.py` in a browser if the application starts successfully.

```bash
cd modeling/
python -m weblinx.eval
```

The evaluator searches for `results.jsonl` files and generates `aggregated_results.json` in the documented workflow.

```bash
cd ..
streamlit run app/Results.py
```

This starts the results application after evaluation. The commands assume the repository layout, dependencies, prediction files, and current working directory shown by the README.

A deterministic script built directly on Playwright or Selenium is the clearest alternative for a fixed workflow. Such a script can encode known selectors and deterministic steps, but it requires a programmer to anticipate the allowed path. webllama instead learns browser decisions from WebLINX-style demonstrations and dialogue. That flexibility is useful for variable instructions, while introducing model-output parsing, inference resources, nondeterministic decisions, and the need to constrain actions before execution.

## Maintenance, License, and Adoption Boundaries

The repository is not archived, and its last push was on 2026-08-16. Its listed releases are `0.0.1pre1` from 2024-04-20, `0.1.0pre1` from 2024-06-13, and `0.1.0` from 2024-06-25. This release history remains small even though the repository received a later push. The README also describes planned deployment work and planned Mind2Web evaluation, so documented features should be distinguished from roadmap items.

The package metadata identifies the MIT license. That declaration applies to the repository package, but the README separately links a fine-tuned Llama model and a WebLINX dataset. Teams should review each model's and dataset's own terms before redistribution or commercial deployment. License compatibility is a project-specific check rather than something that can be inferred from the package classifier alone.

The main adoption burden is integration. A team needs a Llama-compatible inference environment, access to the model and dataset, a WebLINX-compatible state template, browser execution code, output validation, and an evaluation setup. The README does not state hardware sizing or supported browser versions. For controlled research, this framework provides a useful starting point. For immediate production use, the first concrete checkpoint is verifying that `process_pred` maps WebLlama's generated text into a constrained, inspectable action accepted by `env.step`.

## Conclusion

Research teams should adopt webllama when they need a Llama 3 decision model trained on WebLINX navigation and dialogue, plus an evaluation workflow. Production application teams should first verify dependency installation, model and dataset access, template compatibility, browser-state design, and the conversion of generated text into safe environment actions. The package declares the MIT license, while model and dataset terms require separate review.

## FAQ

### What is McGill-NLP/webllama used for?

It is used to build and evaluate Llama 3 agents that browse the web through instructions and dialogue. The first model is fine-tuned on WebLINX web-navigation data.

### How do I install McGill-NLP/webllama?

The repository provides `setup.py`, `requirements-basic.txt`, and `requirements-extra.txt`, but the README does not document a concrete installation command. Package metadata requires Python 3.8 or later.

### Which model does McGill-NLP/webllama provide?

The README identifies `McGill-NLP/Llama-3-8B-Web` as the first agent model. It is described as a fine-tuned `Meta-Llama-3-8B-Instruct` model for web navigation and dialogue.

### How is webllama evaluated?

The README runs `python -m weblinx.eval` from `modeling/`, where it finds `results.jsonl` files and generates `aggregated_results.json`. It also documents `streamlit run app/Results.py` for visualization.

## Sources

- [License: MIT](https://github.com/McGill-NLP/webllama/blob/main/LICENSE)
- [McGill-NLP/webllama on GitHub](https://github.com/McGill-NLP/webllama)
- [Project website](https://webllama.github.io)
- [README](https://github.com/McGill-NLP/webllama/blob/main/README.md)
- [Releases](https://github.com/McGill-NLP/webllama/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/mcgill-nlp-webllama