miso-belica/sumy: extractive summarization for HTML pages and plain text

What sumy actually solves, and for whom

Sumy produces extractive summaries. It selects sentences that already exist in the source document and returns them, rather than generating new prose. The README describes it as a "Simple library and command line utility for extracting summary from HTML pages or plain texts", and the package also contains a small evaluation framework for summaries.

The audience is developers who need a summary step inside a larger program: a bot that posts a short version of a long article, a script that condenses a changelog, a research harness comparing algorithms. The repository lists projects that use it this way, including a Lemmy AutoTL;DR bot and a tool for summarizing large discussions. Because the output is a subset of the input, you can trace every returned sentence back to the source, which matters when the summary is shown next to a link.

It is not a service and not a model you host. There is no summarization endpoint in the repository. You either call the CLI or import the classes. That also means there is no GPU requirement and no model download beyond the NLTK tokenizer data the code needs.

How the summarization pipeline is put together

The data flow has four stages, and the README example makes each one visible.

First, a parser produces a Document. HtmlParser.from_url fetches a URL and runs it through an HTML extractor; PlaintextParser.from_file and PlaintextParser.from_string read local text. Second, a Tokenizer splits the document into sentences and words, and the language string you pass to it decides the rules. Third, a Stemmer reduces words to stems, and get_stop_words supplies the stop word list for that language. Fourth, the summarizer itself scores sentences and returns the top N.

The summarizers are the interesting part. LexRank and LSA are graph and matrix methods: they build a similarity structure over sentences and rank by centrality or by latent semantic decomposition. Luhn and Edmundson are older, frequency-and-position heuristics. The documentation file docs/summarizators.md describes each method. The practical consequence is that LexRank and LSA are slower and need more memory on long documents, while Luhn and Edmundson are cheap and give noticeably different output on the same text.

The length argument is flexible. The README shows both an absolute count (--length=10) and a percentage (--length=3%), so you can ask for ten sentences or for three percent of the document.

Installing sumy and running a first summary

The README requires Python 3.8 or newer. It recommends uv and also documents plain pip. Both lines below install the released package from PyPI.

$ uv pip install sumy
$ pip install sumy

After installation, the sumy command is on your PATH. The README gives this first example, which summarizes a Wikipedia page with LexRank and asks for ten sentences.

$ sumy lex-rank --length=10 --url=https://en.wikipedia.org/wiki/Automatic_summarization

You should see ten sentences printed to stdout, in document order. Nothing is written to a file unless you redirect it. The same command accepts --language, which is the switch you need for non-English sources.

$ sumy lex-rank --language=uk --length=30 --url=https://uk.wikipedia.org/wiki/Україна

If you would rather not install anything, the README also documents a container image with sumy as its entrypoint, so the algorithm name and flags follow the image name directly.

$ docker run --rm misobelica/sumy lex-rank --length=10 --url=https://en.wikipedia.org/wiki/Automatic_summarization

Using sumy as a library instead of a command

The README warns about one trap before you write any code: do not name your script sumy.py, because it shadows the installed package and you will get an import error on sumy.parsers.html. The README example file is called sumy_example.py for that reason.

The library path is explicit. You build a parser, a stemmer, and a summarizer, then iterate over the returned sentences. Swapping LsaSummarizer for another class is the only change needed to try a different algorithm.

from sumy.parsers.html import HtmlParser
from sumy.parsers.plaintext import PlaintextParser
from sumy.nlp.tokenizers import Tokenizer
from sumy.summarizers.lsa import LsaSummarizer as Summarizer
from sumy.nlp.stemmers import Stemmer
from sumy.utils import get_stop_words

LANGUAGE = "english"
SENTENCES_COUNT = 10

parser = HtmlParser.from_url(url, Tokenizer(LANGUAGE))
stemmer = Stemmer(LANGUAGE)
summarizer = Summarizer(stemmer)
summarizer.stop_words = get_stop_words(LANGUAGE)

for sentence in summarizer(parser.document, SENTENCES_COUNT):
    print(sentence)

The parser can also come from a file or a string, which is what you want when the text is already in memory. Note that the summarizer call takes the Document object, not the raw string, so the tokenizer and stemmer must agree on the same language value.

Language coverage and the stop word dependency

The pyproject classifiers list Chinese (Simplified), Czech, English, French, German, Hebrew, Italian, Japanese, Polish, Portuguese, Slovak, Spanish, Swedish, Ukrainian, Greek, Arabic and Thai. The changelog adds Thai, Polish and Swedish in v0.12.0 and Arabic in v0.11.0, which tells you language support arrives incrementally rather than all at once.

Coverage is not uniform across those languages. The Dockerfile excludes Korean from its build, and the comment in the file explains why: konlpy depends on a JPype1 extension that does not load on musl, and konlpy also needs a JVM at runtime that the image does not install. So Korean is supported by the package in principle but is deliberately left out of the container.

Stop words are the quiet dependency. get_stop_words(LANGUAGE) returns a set, and if that set is empty for your language, every common word counts equally in scoring and the summary quality drops. The README points to docs/index.md for the tokenizer list and to docs/how-to-add-new-language.md for adding a language, which is the honest answer for anything not listed.

Measuring a summary with sumy_eval

Sumy ships an evaluation command, which is unusual for a library this size. You supply a reference summary file and the same source the summarizer used, and sumy_eval scores the generated summary against it.

$ sumy_eval lex-rank reference_summary.txt --url=https://en.wikipedia.org/wiki/Automatic_summarization
$ sumy_eval lsa reference_summary.txt --language=czech --url=https://www.zdrojak.cz/clanky/automaticke-zabezpeceni/

The value here is comparative, not absolute. You write one reference summary, then run several methods against it and see which one agrees with your judgement on your documents. The release notes for v0.13.0 mention fixing Rouge evaluation bugs, so treat older evaluation numbers with care if you are comparing against a run from an earlier version.

What the README does not document is a way to evaluate against multiple references or a held-out corpus. This is a single-reference check, and a single reference summary is one person's opinion of what matters. Use it to rank methods on your own text, not as a quality certificate.

Where sumy is the wrong choice

The first limitation is structural: sumy is extractive. If the source sentences are badly written, the summary will be badly written, because the output is a selection of those sentences. There is no rewriting step and no way to add one through configuration. If you need fluent abstractive output, this is the wrong tool and no flag will change that.

The second is that HTML extraction depends on the page. HtmlParser.from_url fetches and parses live markup. Pages behind JavaScript rendering, paywalls or bot checks will not yield the text you expect, and the README does not document a fallback for that case. Fetching the HTML yourself and passing it to PlaintextParser.from_string is the workaround, but then you own the extraction.

The third is evaluation scope. The framework compares one generated summary against one reference file. It cannot tell you whether the summary is faithful to the source, only how much it overlaps with your reference.

Finally, sumy is a library, not a service. There is no queue, no caching layer and no retry logic. If you run it inside a request handler, the fetch and tokenization happen in that request.

How sumy differs from an abstractive summarizer

The real alternative is an abstractive model such as a transformer-based summarizer, which generates new sentences rather than selecting existing ones. The difference in approach is not a matter of tuning. Sumy scores sentences with graph centrality (LexRank), latent semantic analysis (LSA), or frequency heuristics (Luhn, Edmundson), and returns a subset of the input. An abstractive model encodes the document and decodes a new sequence.

That difference decides several practical questions. Extractiveness means the output cannot hallucinate a fact that is not in the source, which is a real advantage when the summary sits next to the original link. It also means the output can read awkwardly when the source does. Abstractive output reads better but can state things the source never said, and it needs a model file, more memory, and usually a GPU for acceptable latency.

Sumy's other distinction is the bundled evaluation command. If your goal is to compare ranking methods on your own corpus rather than to produce the most fluent paragraph, the extractive approach plus sumy_eval is closer to what you want than a generative model with no scoring harness around it.

Maintenance, licence and upgrade cost

The repository is not archived, and the last push was on 2026-09-09. Releases have been uneven: v0.11.0 in 2022, v0.12.0 in 2026, and v0.13.0 in 2026. The v0.13.0 notes mention bumping minimum dependencies and fixing Rouge evaluation bugs; v0.12.0 added Thai, Polish and Swedish. The pattern is long quiet periods followed by a release that adds languages or fixes the evaluation path.

The licence is Apache-2.0, declared both in the pyproject metadata and in LICENSE.md. Apache-2.0 is permissive and includes a patent grant, which matters if you embed the library in a product. This is a description of the licence text, not legal advice; if the patent or notice clauses affect your distribution, read LICENSE.md with your own counsel.

Upgrade cost is modest but not zero. The library depends on NLTK for tokenization, and the Dockerfile downloads the punkt_tab data at build time, so an environment that blocks that download will fail at runtime rather than at install. The v0.13.0 dependency bump means an old pinned environment may need attention. There is no documented API stability guarantee, and the package classifiers still say Development Status 4 - Beta.

Editorial conclusion

Adopt sumy if you need extractive summaries of HTML or plain text in a pipeline you control, especially if you want to pick between LexRank, LSA, Luhn and Edmundson and measure the result with sumy_eval. Do not adopt it if you need abstractive rewriting, or if your language is not in the tokenizer list and you are not prepared to add it. Before committing, run sumy lex-rank and sumy lsa on the same document at the same --length and compare the output, then check that get_stop_words returns a non-empty set for your language, because an empty stop word list changes which sentences score highest.

Frequently asked questions

Official sources

1. License: Apache-2.0
2. miso-belica/sumy on GitHub
3. Project website
4. README
5. Releases