# mwparserfromhell: a wikicode parser that gives you the node tree

> mwparserfromhell is a Python library that parses MediaWiki wikicode into a tree of nodes you can query and edit, without talking to a wiki server. It suits bot developers, template maintenance scripts and anyone who needs to reason about wiki source rather than rendered HTML.

**earwig/mwparserfromhell** — A Python parser for MediaWiki wikicode

- Repository: https://github.com/earwig/mwparserfromhell
- Website: https://mwparserfromhell.readthedocs.io/
- Stars: 907 · Forks: 91
- Language: Python
- License: MIT
- Published: 2026-10-20 · Updated: 2026-10-20 · Language: en
- Canonical page: https://hysenlabs.com/projects/earwig-mwparserfromhell

## A source-level parser, not a MediaWiki client

The library's job is to turn wikicode into a structure, and it deliberately stops there. The documentation states that while the MediaWiki parser generates HTML and has access to the contents of templates, mwparserfromhell acts as a direct interface to the source code only. That single sentence explains most of what the library can and cannot do, and it is the right frame to hold while reading the API. There is no network layer, no site configuration, and no rendering. You hand it a string of page source and get back a tree. The integration notes give the intended consumer: the library is used by and was originally developed for EarwigBot, and Page objects there have a parse method that essentially calls mwparserfromhell.parse() on the page text. The alternative route is to use it alongside Pywikibot, which the documentation shows in an example that opens a site, fetches a page and parses the result. Either way the library is a text and tree layer, not a client, and the difference matters when you decide whether a bug is in your code or in a wiki configuration you never loaded.

## Wikicode behaves like a string with extra methods

The entry point returns a Wikicode object, and the design choice that makes the library pleasant is that this object acts like an ordinary str with some extra methods. Nothing forces you to work in a tree-shaped API if you do not want to. A first session looks like this:

```python
  >>> import mwparserfromhell
  >>> wikicode = mwparserfromhell.parse(text)
```

The result prints back to the original text, so a parse and print round trip is lossless by construction. The filter methods return lists of the nodes you asked for, and the printed representation of a list is a list of the source strings, which keeps debugging readable:

```python
  >>> text = "I has a template! {{foo|bar|baz|eggs=spam}} See it?"
  >>> wikicode = mwparserfromhell.parse(text)
  >>> templates = wikicode.filter_templates()
  >>> print(templates)
  ['{{foo|bar|baz|eggs=spam}}']
  >>> template = templates[0]
  >>> print(template.name)
  foo
  >>> print(template.params)
  ['bar', 'baz', 'eggs=spam']
```

The named parameter case matters more than it looks. A template parameter can be positional or named, and a script that assumes positional indexing will break on any template with a named argument, which is why the example shows both lookups side by side, by index for a positional value and by name for a keyword one.

## Nested templates are the reason to use a parser at all

Regular expressions cannot reliably find a template inside a template, because braces nest. The library handles that directly, and the example makes the point with a deliberately tangled input:

```python
  >>> text = "{{foo|{{bar}}={{baz|{{spam}}}}}}"
  >>> mwparserfromhell.parse(text).filter_templates()
  ['{{foo|{{bar}}={{baz|{{spam}}}}}}', '{{bar}}', '{{baz|{{spam}}}}', '{{spam}}']
```

The list is in document order and includes every level, so a script that wants the innermost templates gets them without writing a brace counter. If you want to walk the structure yourself rather than let the filter flatten it, filter_templates() takes a recursive=False argument, and the reason it can is that nodes contain additional Wikicode objects:

```python
  >>> code = mwparserfromhell.parse("{{foo|this {{includes a|template}}}}")
  >>> print(code.filter_templates(recursive=False))
  ['{{foo|this {{includes a|template}}}}']
  >>> foo = code.filter_templates(recursive=False)[0]
  >>> print(foo.get(1).value)
  this {{includes a|template}}
```

The two modes suit different jobs. The recursive form is right for bulk analysis, counting or renaming every template on a page. The non-recursive form is right when the nesting itself is the problem, because it lets you decide how deep to descend and stop when a template looks like it should be treated as opaque.

## Templates are editable lists, and names compare case-insensitively

The library is not read-only. Template parameters can be added, removed or altered, and Wikicode objects support the list operations you would expect, with append, insert, remove and replace among them. There is also a matches method for comparing page or template names that takes care of capitalization and whitespace, which is the detail that makes the library usable on a real wiki, where Cleanup and cleanup and Cleanup are the same template but not the same string. The documented example edits a page in place:

```python
  >>> text = "{{cleanup}} '''Foo''' is a [[bar]]. {{uncategorized}}"
  >>> code = mwparserfromhell.parse(text)
  >>> for template in code.filter_templates():
  ...     if template.name.matches("Cleanup") and not template.has("date"):
  ...         template.add("date", "July 2012")
  ...
  >>> print(code)
  {{cleanup|date=July 2012}} '''Foo''' is a [[bar]]. {{uncategorized}}
```

Two things are worth noticing there. The has check before adding a parameter is the pattern that prevents duplicate arguments, and the printed output shows the added parameter slotted in with the named syntax rather than appended blindly. Converting back to text is done by calling str() on the object, and the documentation is explicit that the result is a regular str, ready to be saved, with an equality check against the code object confirming the round trip. For a maintainer script, that is the whole workflow: parse, decide, mutate, save.

## The parser cannot see inside a transcluded template

The limitations section is the most valuable part of the documentation, because it names three specific cases where the library gives you something other than what MediaWiki would render. The first is transclusion. The documentation uses a hypothetical template page named Template:End-bold containing the text a closing bold tag: MediaWiki would understand that a bold tag opened before the transclusion is closed by it, but mwparserfromhell has no way of examining the contents of a transcluded template, so it treats the bold tag as unfinished and may extend the formatting further down the page. The second is adjacency to external links, where a template placed directly after a URL is considered part of the link, when in reality it would depend on what the template expands to. The third is crossing constructs, where an italic marker that opens before a template and closes after it cannot be represented in an ordinary syntax tree, so the parser treats the first construct as plain text and only the italic tag is parsed properly. The documented workaround for that last case is to pass skip_style_tags=True to the parse call, which treats the two and three quote markers as plain text, on the grounds that this pattern is common with text formatting and formatting is often not what users care about.

## Wiki-specific settings the parser does not know about

There is a second class of limitation that is about configuration rather than syntax, and it comes down to the parser having no awareness of certain wiki-specific settings. Word-ending links are not supported, because the linktrail rules are language-specific. Localized namespace names are not recognised, so a file link such as a File: prefix is treated as an ordinary wikilink. And anything that looks like an XML tag is treated as a tag, even when it is not a recognised tag name, because the list of valid tags depends on which MediaWiki extensions are loaded. The practical reading of these three is that the parser models the MediaWiki language as implemented by a standard installation in English, and it will not model a particular wiki's local customisations. For a maintenance script running on a large multilingual wiki with custom namespaces and extensions, that is a real constraint. The documentation also notes that a future version may include multiple parsing modes to get around the style tag restriction more sensibly, which tells you the current behaviour is considered provisional in one specific place.

## A C tokenizer with a pure Python escape hatch

The implementation detail that matters for deployment is that the tokenizer is written in C, and the library ships prebuilt wheels for most environments, covering Linux x86_64 and arm64, macOS x86_64 and arm64, and Windows x86 and x86_64. If you are building from source and the C tokenizer cannot be built, the documentation describes the fallback: set the environment variable WITH_EXTENSION=0 when installing and you get the slower pure-Python implementation. The setup.py file shows how the switch is structured, with an enum of REQUIRED, OPTIONAL and IGNORED for the extension state and an explicit error if both WITH_EXTENSION and WITHOUT_EXTENSION are set. That fallback is what makes the library installable on an exotic platform where a compiler is not available, at the cost of speed, and it is a deliberate design choice rather than an accident. The pyproject.toml sets requires-python to 3.10 and above, lists no runtime dependencies at all, and the development group carries pytest, pytest-cov, sphinx and psutil. The licence field on the repository says MIT, and the package ships both a pyproject.toml and a setup.py because the extension build still needs setuptools.

## Conclusion

mwparserfromhell is a good fit for a bot developer or a wiki maintainer who needs to find templates, rewrite parameters and reason about source rather than rendered output, and for anyone who wants to test wikicode handling without standing up a MediaWiki instance. It is the wrong tool for reading the rendered article, for a wiki whose templates are the substance of the page, and for localisation-aware link handling. Before adopting it, read the limitations section, because the parser cannot see inside a transcluded template and it does not know about localised namespace names, then check that your target wiki is not one where the transclusion rules carry the meaning. The code is MIT licensed, the last push was on 2026-10-19, and the latest published release is version 0.7.2 from 2025-07-01.

## FAQ

### How do I install mwparserfromhell?

The easiest route is PyPI, with pip install mwparserfromhell, and prebuilt wheels carry a compiled C tokenizer for most Linux, macOS and Windows environments. If you are building from source and the C tokenizer cannot be built, set the environment variable WITH_EXTENSION=0 to fall back to the slower pure-Python implementation.

### How do I find nested templates in a page with mwparserfromhell?

Call filter_templates() on the parsed Wikicode, which returns every template at every nesting level in document order. Pass recursive=False to get only the top level and walk deeper yourself, which works because template parameter values are themselves Wikicode objects that can be parsed again.

### Can I modify a template with mwparserfromhell?

Yes. Wikicode objects behave like lists with append, insert, remove and replace, template parameters can be added with add, and the name.matches method compares page and template names taking capitalization and whitespace into account. Call str() on the result to get a regular string back for saving.

### What are the limitations of mwparserfromhell?

The parser only sees source, not rendered output, so syntax produced by a transcluded template cannot be detected, templates next to external links are treated as part of the link, and crossed syntax elements cannot be represented in a syntax tree. It also does not support word-ending links, does not recognise localized namespace names, and treats anything that looks like an XML tag as a tag.

### Which Python versions does mwparserfromhell support?

The project metadata sets requires-python to 3.10 and above and classifies Python 3.10 through 3.14. The README states the package supports Python 3.9+, and the most recent published release is version 0.7.2 from 2025-07-01, with the last push on 2026-10-19.

## Sources

- [earwig/mwparserfromhell on GitHub](https://github.com/earwig/mwparserfromhell)
- [License: MIT](https://github.com/earwig/mwparserfromhell/blob/main/LICENSE)
- [Project website](https://mwparserfromhell.readthedocs.io/)
- [README](https://github.com/earwig/mwparserfromhell/blob/main/README.md)
- [Releases](https://github.com/earwig/mwparserfromhell/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/earwig-mwparserfromhell