# ESDB replaces the SCOWL word list files with one queryable database

> The English Speller Database is a rebuild of the SCOWL project that stores how common a word is, its dialect spellings, part of speech and inflections in a single SQLite file instead of a set of combinable lists. The command line is still called scowl, the sizes are still 60 and 70, and the documentation calls the tool alpha with a schema that may still change.

**en-wl/wordlist** — English Speller Database (ESDB)

- Repository: https://github.com/en-wl/wordlist
- Website: https://wordlist.aspell.net
- Stars: 558 · Forks: 101
- Language: Perl
- License: NOASSERTION
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/en-wl-wordlist

## What the database actually stores about a word

The English Speller Database is a database of information on English words useful for creating high quality speller dictionaries and word lists for most dialects of English. The content is specific: how common a word is, differences in spelling between the dialects of English, spelling variant information, basic part of speech, and inflection information.

That is the difference from a dictionary. A dictionary tells you what a word means and how it is spelled. This tells you how frequent it is, which dialect prefers which form, whether it is a noun or a verb, and how it inflects. Those are exactly the four things a spell checker needs in order to decide whether to flag something and how to suggest a correction.

The dialect coverage is broad. The original version, SCOWLv1, compiled the information into simple word lists that could be combined to create dictionaries of various sizes and dialects, and the documentation names them as American, British in both the -ise and -ize spellings, Canadian and Australian.

The primary language of the repository is Perl, which is a historical detail worth noting, since the tool that consumes the database is Python and the packaging is make and SQLite.

## One database instead of a set of combinable lists

The architectural change from SCOWL to ESDB is the reason to look at this project at all.

The original version compiled the information in the database into a set of simple word lists that could be combined to create speller dictionaries of various sizes and dialects. The new version instead combines all that information into a single text file and a SQLite3 database.

In order to keep the file size manageable and to avoid noise entries, the minimum ESDB size is now 35, and the 95 size is not included at all. That is a deliberate change to the size scale, and it means the numeric tiers people remember from SCOWL do not map one to one onto the new ones.

The new version also includes content the original excluded. It has the proper spelling of abbreviations including the trailing dot, and it includes words that were excluded before, such as hyphenated and open compound words with a space, and words with special symbols in them.

The data files themselves are worth understanding. The new version is generated from the same sources as the original but through a far more complicated and unreleased process, and the results of that process live in scowl-pre.txt. That file is then combined with other files to create the final scowl.txt and the sqlite3 database scowl.db. So the published artefact is downstream of a processing step you do not get, and that is worth knowing if you were hoping to trace a word back to its source.

## make builds the database, and scowl is the entry point

Using ESDB requires creating the database from the source files in the data directory first. In a Unix-like environment that is one command:

```bash
    make
```

which creates the sqlite3 file scowl.db, and the documentation says that is all you need for most operations. The flat text file can also be produced with make scowl.txt.

The Makefile shows how little machinery this is. The default target depends on combine.py, and the database is built by running it with a create-db argument, while the text file is produced by exporting from the database:

```makefile
scowl.db: ./combine.py
	./combine.py create-db scowl.db

scowl.txt: scowl.db
	./scowl export --db scowl.db > scowl.txt
```

To work with the database you use the scowl script in the root directory, which the documentation describes as a very thin wrapper around the libscowl Python module. That module is not on PyPI, it ships with the database, and the script is meant to be run from the root directory of the distribution.

The requirements are modest. ESDB requires Python 3 and SQLite, currently Python 3.7 and SQLite 3.33.0, with newer versions expected to work and older ones unsupported. A Unix-like environment is not required, but the rest of the documentation assumes one.

## Size, dialect and variant level are the three knobs

Extracting a word list is one command, and its positional arguments are the whole configuration language:

```bash
    ./scowl --db scowl.db word-list 60 A 1 > wl.txt
```

The arguments are the ESDB size, in this case 60, the spellings to include, in this case A for American, and the maximum variant level, in this case 1, which excludes most variants except for special cases such as dox and doxx. The exact meaning of all these values is described in the File Format section of the documentation.

The dialect argument is a list. A single A gives American, B gives British, and B,Z with a variant level of 5 gives a British list that includes -ise, -ize and other variant spellings.

The variant level is the subtle one. A level of 1 keeps the mainstream forms, and raising it lets more spelling variants through. The default list at level 1 includes Roman numerals and slang words only really used by computer programmers such as grepped, and excluding those and any other special categories is done with an empty categories option:

```bash
    ./scowl word-list 60 A 1 --categories= > wl.txt
```

A larger list comes from raising the size:

```bash
    ./scowl word-list 70 A 1 > wl.txt
```

The default size of 60 is described as vetted for errors, and the larger size of 70 as usable as a spellchecker dictionary, with the caveat that the processing of the source data is completely different so the resulting word lists are not the same.

## Filters for accents, dots and parts of speech

Several filters decide what a word list contains, and they are the part you will spend the most time tuning.

The default word filter strips the trailing dot from abbreviations, so the default American list keeps accents but loses the dot. To keep the dot:

```bash
    ./scowl word-list 60 A 1 --dot True > wl.txt
```

To exclude abbreviations altogether, including the unmarked ones:

```bash
    ./scowl word-list 60 A 1 --wo-poses=abbr > wl.txt
```

To remove diacritical marks, which the default list preserves:

```bash
    ./scowl word-list 60 A 1 --deaccent > wl.txt
```

And to disable the word filter entirely and include everything:

```bash
    ./scowl word-list 60 A 1 --no-word-filter > wl.txt
```

The --wo-poses option takes a part of speech name, and the categories option works the other way round, clearing the categories you do not want. Together they let you build a list for a specific domain, which matters when a spell checker is checking identifiers or a controlled vocabulary rather than prose.

For more options there is a help command, and the database can also be queried directly. Most of the database is defined in schema.sql, views.sql and scowl.sql in the libscowl directory, and the main entry point for extracting word lists is the scowl_v0 query.

## Alpha status, an unstable schema and one stable view

The documentation is unusually blunt about the state of the software. ESDB is still a work in progress, and as it is still in an alpha and testing phase the command line utility and the schema are subject to change, with the command line interface expected to stabilize at some point.

The schema may still change, but a new esdb_v1 view will be provided that is guaranteed to always provide the same results. New columns may be added, but not in a way that breaks existing queries, and if it is necessary to break existing queries a new view will be provided.

That is a reasonable stability contract, and it is worth knowing which half of it applies to you. If you query esdb_v1 you are protected. If you depend on scowl_v0 or on the command line flags themselves, you are working against a moving target.

There is also a search interface, which is fuzzy by default and returns the groups containing any of the supplied words:

```bash
    ./scowl search [--db scowl.db] [--by-cluster] [--exact] WORD [WORD ...]
```

With --exact you search for the exact word, and with --by-cluster you get the entire cluster rather than the groups.

The name is worth a note too. The project was originally called SCOWL, Spell Checker Originated Word Lists, and Friends, the and Friends part referring to VarCon, AGID and the various wordlists used to create the final result. The name change is documented in discussion 465, and because the rename was recent all internal commands still use the original name.

## A layered licence, and a slow but real release line

The licensing has two layers and you need both of them. ESDB is derived from many sources under a BSD compatible licence, and the combined work is freely available under an MIT-like licence, with the file Copyright holding the details. The repository carries that file at the top level, and the platform cannot classify the licence as one of the standard ones, so reading it is not optional for a commercial project.

The separation between source terms and the combined work is the part that catches people. The individual sources have their own terms, and the combined artefact is what you ship in a dictionary. Both sets of terms apply to your use, and the documentation is clear that the derived work is the BSD-compatible combination and the whole is MIT-like.

The release picture is thin but structured. There is one release tag, rel-2026.02.25, dated 2026-02-25, and the default branch is v2 rather than main. The last push was on 2026-07-07, and the repository is not archived.

For context on where the project is going, the documentation lists what has changed since the rework, and most of it is corrections rather than additions. The processing of the source data is completely different, so the resulting word lists are not the same as the old ones, and most of the changes are regarded as corrections for improper handling of derived forms or variants in SCOWLv1. The handling of possessive forms has been completely redone, based partly on the noun category assigned by WordNet, and for American English the new changes to non-possessive forms are listed in the file docs/comp-60.txt.

## Conclusion

Adopt ESDB if you build a spell checker or a word list and need dialect-aware data, part of speech and inflection information rather than a flat list of words, since that is the information the database actually holds. Do not adopt it if you only need a dictionary of common English, since the size tiers and the variant levels are a lot of machinery for that. Before you script against it, check three things: that the command line utility and the schema are still subject to change while the project is in its alpha and testing phase, that you use the esdb_v1 view rather than the scowl_v0 query if you need stability, and that you read the Copyright file, since the database is derived from many sources under BSD compatible terms and the combined work carries an MIT-like licence.

## FAQ

### How do I build the ESDB database?

Run make in a Unix-like environment with Python 3 and SQLite available, which creates the sqlite3 file scowl.db from the sources in the data directory. The flat text file can also be produced with make scowl.txt, and the requirements are currently Python 3.7 and SQLite 3.33.0.

### How do I extract a word list from ESDB?

Use the scowl script with the word-list subcommand: ./scowl --db scowl.db word-list 60 A 1 > wl.txt. The arguments are the ESDB size, the spellings to include, and the maximum variant level, and the script is meant to be run from the root directory of the distribution.

### Is the ESDB command line and schema stable?

Not yet. The project is in an alpha and testing phase, so the command line utility and the schema are subject to change. A new esdb_v1 view is guaranteed to always give the same results, while the older scowl_v0 query may change.

### How do I keep abbreviations with their trailing dot in a word list?

The default word filter strips the trailing dot, so pass --dot True. To exclude abbreviations altogether including unmarked ones use --wo-poses=abbr, and --no-word-filter disables the filter entirely. To remove accents instead, use --deaccent.

### What licence applies to the English Speller Database?

The database is derived from many sources under a BSD compatible licence, and the combined work is freely available under an MIT-like licence, with the details in the Copyright file at the top of the repository. The single release tag is rel-2026.02.25 from 2026-02-25.

## Sources

- [en-wl/wordlist on GitHub](https://github.com/en-wl/wordlist)
- [Issues](https://github.com/en-wl/wordlist/issues)
- [Project website](https://wordlist.aspell.net)
- [README](https://github.com/en-wl/wordlist/blob/v2/README.md)
- [Releases](https://github.com/en-wl/wordlist/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/en-wl-wordlist