budzianowski/multiwoz: the dataset repo and the end-to-end model behind the MultiWOZ paper
Source code for end-to-end dialogue model from the MultiWOZ paper (Budzianowski et al. 2018, EMNLP)
At a glance
- What is it?
- The repository bundles four versions of the MultiWOZ task-oriented dialogue corpus (2.0, 2.1, 2.2, 1.0) with the code for the end-to-end model from Budzianowski et al. 2018. It is a research artefact with known annotation errors and a version split that will decide how your numbers compare to published ones.
- Who is it for?
- Adopt this repository if you need the MultiWOZ dialogues themselves or want to reproduce the 2018 end-to-end model; do not adopt it if you need a maintained dialogue-system framework, because the README points DST work at the TRADE repository for processing and scoring.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 151 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What MultiWOZ is, and who the repository is actually for
MultiWOZ is a Wizard-of-Oz corpus of human-human written conversations. The README describes it as a fully-labeled collection spanning multiple domains and topics, at a size of 10k dialogues, which it says is at least one order of magnitude larger than all previous annotated task-oriented corpora. The repository holds both the dataset files and the source code for the end-to-end dialogue model from the 2018 EMNLP paper by Budzianowski et al. That combination tells you the audience: researchers who want to train or evaluate a dialogue model on this corpus, and people who need the raw dialogues for analysis. It is not a dialogue-system toolkit. There is no runtime, no dialogue manager service, and no deployment story in the material. The topics listed on the repository include dialogue-manager and dialogue-systems, but nothing in the README describes a component you would run in production. Treat the topic tags as discovery metadata, not as a description of what ships.
Four dataset versions, and why the choice changes your numbers
The README lists four downloads. MultiWOZ 2.0 is the version used in the EMNLP publication. MultiWOZ 1.0 is the version used in the ACL publication. MultiWOZ 2.1 is a corrected version credited to the Amazon crew, and MultiWOZ 2.2 is described as the newest, corrected version, credited to the Google crew. The corrections matter because the benchmark table reports results per version: joint accuracy for the same model is not comparable across columns. TRADE, for example, is listed at 48.62 on 2.0, 46.0 on 2.1 and 45.4 on 2.2. A drop of that size between versions is a property of the labels, not of the model. If you pick a version by convenience and compare against a paper that used another one, you will draw the wrong conclusion. The README also states that the datasets can be loaded directly through DialogStudio, which is the path to take if you do not want to handle the archives yourself. The repository has no releases, so there is no versioned artefact to pin; the default branch is master and the last push shown is 2026-04-18.
Dialogue structure: goals, belief states, and the semi/book/booked split
Each dialogue consists of a goal, multiple user and system utterances, and a belief state. The README adds that the natural-language task description shown to the turkers on the visitor side is included as well. The belief state has three sections: semi, book and booked. Semi holds slots from a particular domain. Book holds booking slots for a domain, and booked is a sub-list of the book dictionary carrying information about the booked entity once the booking has been made. Two caveats are stated plainly. The goal was sometimes wrongly followed by the turkers, which can produce a wrong belief state, and the joint accuracy metric includes all slots. The corpus is split into 3,406 single-domain dialogues, including booking where the domain allows it, and 7,032 multi-domain dialogues covering at least two and up to five domains. The split into train, test and development was random to enforce reproducibility, with 1k examples each in test and development. Only fully successful dialogues appear in validation and test, and there are no hospital or police dialogues in those sets. That last restriction is a design decision worth noticing: it makes comparison fairer but it removes unfinished conversations from evaluation entirely.
File naming and dialogue-act annotation: read the FAQ before writing a loader
The FAQ is the part of the README most likely to break a naive data pipeline. File names encode dialogue type. MUL and PMUL refer to strictly multi-domain dialogues, meaning at least two main domains are involved. SNG, SSNG and WOZ refer to single-domain dialogues that may include sub-domains such as booking. The README states explicitly that there is no 1-to-1 mapping between dialogue acts and sentences, so any code that assumes one act per utterance is wrong. Only system utterances are manually annotated with dialogue acts. User-side dialogue acts in MultiWOZ 2.1 were annotated automatically using heuristics developed in ConvLab, which means user acts carry a different provenance from system acts and should not be treated as equivalent gold labels. There are no dialogue state tracking labels for police and hospital, described as very simple domains, and no dialogues from those domains in validation or testing. If your evaluation code iterates over a fixed domain list, these two domains will produce empty or missing entries.
What the benchmark table does and does not tell you
The README includes a dialog state tracking table with joint accuracy and slot accuracy columns for MultiWOZ 2.0, 2.1 and 2.2. It is a curated list of published results with links to papers, and the README invites pull requests to add new models. Read it as a bibliography, not as a leaderboard you can trust blindly. Many cells are empty, so cross-version comparison is only possible for the handful of models evaluated on more than one version. The table also mixes evaluation setups: a note above it instructs that DST experiments should follow the data processing and scoring scripts from the TRADE model, which means the numbers in the table are only directly comparable if they were produced under that pipeline. The repository does not ship a scoring script of its own for these results. If you need to reproduce a row, the linked paper and the TRADE scripts are the source of truth, not this table.
Getting it running: what the README actually gives you
The README does not provide an installation section, a requirements file listing, or a training command. What it provides is access paths. The corrected dataset is at data/MultiWOZ_2.2 in the repository, while 2.1, 2.0 and 1.0 are distributed as archives at data/MultiWOZ_2.1.zip, data/MultiWOZ_2.0.zip and data/MultiWOZ_1.0.zip. The README also states that these datasets can be loaded directly through DialogStudio. For state tracking work, the instruction is to follow the data processing and scoring scripts from the TRADE model rather than anything in this repository. That is the honest summary of the setup cost: you clone the repository, take the data directory, and bring your own training and evaluation stack. Anyone expecting a pip install and a train script from the README alone will be disappointed, and I would rather say that than imply a workflow the documentation does not describe.
Where this repository is the wrong tool
The clearest limitation is maintenance scope. The repository is the source code for a 2018 model plus the data it was built on. The README's own guidance for DST experiments points elsewhere, to the TRADE repository, for both processing and scoring. If your goal is to build a dialogue state tracker, the practical centre of gravity is that other codebase, and this repository is primarily a data source. The second limitation is label quality. The README states that turkers sometimes followed the goal wrongly, which can result in a wrong belief state, and that the joint accuracy metric includes all slots, so a single wrong slot sinks the whole turn. That is why 2.1 and 2.2 exist as corrections. The third is evaluation scope: because validation and test contain only fully successful dialogues, and exclude hospital and police, a model that scores well here has not been measured on unfinished conversations or on those two domains. If your application involves either, this corpus will not tell you how the model behaves. A concrete alternative for the state tracking task is TRADE, which the README names directly. The difference in approach is that TRADE is a specific DST model with its own data processing and scoring scripts, while this repository is the corpus plus the 2018 end-to-end model. Choosing between them is choosing between a defined evaluation pipeline and the raw material.
Licence and the cost of keeping up
The repository is MIT licensed, which covers the code. The README does not state a separate licence for the dataset files themselves, and I cannot confirm one from the material provided, so if you plan to redistribute the dialogues or use them commercially, verify the dataset terms separately rather than assuming MIT covers everything in the repository. On maintenance: there are no releases, so there is nothing to pin and no changelog to follow. The versioning that matters is the dataset version, and the README treats 2.2 as the current corrected one. Upgrading from 2.0 to 2.2 is not a dependency bump; it changes labels and therefore changes reported metrics, as the benchmark table shows for TRADE. Budget for re-running evaluation rather than assuming a drop-in swap. For the model code itself, the README gives no indication of ongoing development beyond the data corrections.
Editorial conclusion
Adopt this repository if you need the MultiWOZ dialogues themselves or want to reproduce the 2018 end-to-end model; do not adopt it if you need a maintained dialogue-system framework, because the README points DST work at the TRADE repository for processing and scoring. Before you start, decide which dataset version you are targeting, since 2.0, 2.1 and 2.2 are separate downloads with separate benchmark tables, and check whether your evaluation plan requires the fully successful dialogues that the validation and test splits are restricted to.
Community notes