MIND-Lab/OCTIS: Comparing Topic Models in Python

The problem OCTIS solves: picking a topic model without hand-rolling every comparison

Topic modeling has an awkward property. Two models can be run on the same corpus, produce different topics, and there is no single number that says which is better. Perplexity favors one family, coherence favors another, and the choice of hyperparameters moves both. In practice this means a researcher who wants to compare LDA against a neural model ends up writing a preprocessing script, a training wrapper per model, a metric implementation, and a loop over hyperparameters. Nothing about that work is specific to the research question.

OCTIS targets that layer. The README describes it as aiming to train, analyze and compare topic models whose optimal hyperparameters are estimated with Bayesian Optimization. The package ships both classical and neural models, a set of evaluation metrics, and a search procedure. The intended user is a researcher or an applied NLP engineer who has a corpus and a comparison to make, not someone who wants a topic model embedded in a serving path. The repository layout supports that reading: there is a tests directory, a docs directory, two example notebooks, and a preprocessed_datasets directory, which is what a benchmarking tool looks like rather than a library meant to be imported into an application.

How the training, evaluation and optimization loop fits together

The data flow has four stages. A Dataset object holds the corpus. A model object, wrapped by OCTIS, is trained on it. An evaluation module scores the resulting topics. An optimizer proposes the next hyperparameter configuration and the loop repeats.

The dataset stage has two entry points. The README shows fetching one of the preprocessed benchmarks by name, and it notes the name is case-sensitive. Six datasets are listed: 20NewsGroup, BBC_News, DBLP, M10, DBPedia_IT and Europarl_IT, with document counts from 2,225 to 54,595 and vocabularies from 1,513 to 2,949 words. Four are English, two Italian, and Europarl_IT carries no labels. Loading your own corpus goes through the same Dataset class, with preprocessing handled by the package.

The optimization stage is the part that distinguishes OCTIS from a model zoo. The README states that optimal hyperparameters are estimated by means of a Bayesian Optimization approach, and the requirements file lists scikit-optimize, which is the search backend. That means the search is not a grid. It builds a surrogate over the hyperparameter space and proposes configurations, which matters when a single neural topic model training run takes minutes.

The output surface is twofold. The README lists a Python library for advanced usage and a simple web dashboard for starting and controlling optimization experiments. The requirements file includes flask, and setup.py registers a console script named octis pointing at octis.cli:main, so the dashboard is reached through the installed command rather than a separate package.

Installing OCTIS and running a first model

Installation is a single command from PyPI, as the README states. The package requires Python 3.7 or later according to setup.py.

pip install octis

The requirements file is worth reading before you install, because it pins two packages tightly: scikit-learn==1.1.0 and numpy>=1.23.0,<2.0, with scipy<1.13. If your environment already runs a newer scikit-learn, pip will either downgrade it or fail to resolve. Installing into a fresh virtual environment avoids that argument entirely.

Loading a benchmark dataset is two lines. The dataset name must match the table exactly, including capitalization.

from octis.dataset.dataset import Dataset
dataset = Dataset()
dataset.fetch_dataset("20NewsGroup")

After the fetch completes, dataset holds the 20NewsGroup corpus: 16,309 documents, a 1,612-word vocabulary and 20 labels, per the README table. The first call downloads and caches the preprocessed data, so subsequent runs read from disk.

The repository ships two notebooks that cover the rest of the path. examples/OCTIS_LDA_training_only.ipynb builds an LDA model on 20Newsgroups and evaluates it; examples/OCTIS_Optimizing_CTM.ipynb optimizes a neural topic model on M10. Both are linked as Colab badges in the README, so they run without a local install if you want to see the API shape before committing to the dependency pins. For the dashboard route, setup.py registers the octis console script, and the README lists a web dashboard for starting and controlling optimization experiments.

Where OCTIS is the wrong tool

The dependency pins are the first real constraint. scikit-learn==1.1.0 and numpy<2.0 are exact and upper-bounded respectively. In an environment shared with other packages that require numpy 2.x, OCTIS cannot be installed without isolating it. That is not a bug, but it is a decision the maintainers made and it has consequences for anyone adding OCTIS to an existing project rather than a fresh one.

The second constraint is scope. OCTIS is built around comparing models on a fixed corpus with a fixed vocabulary. The preprocessed datasets have vocabularies between 1,513 and 2,949 words, and the evaluation metrics operate on the topics produced from that vocabulary. If your requirement is to infer topics for new documents as they arrive, or to update a model incrementally, nothing in the README describes that path. You would train with OCTIS, extract the model, and build the inference path yourself.

The third is the maturity signal in setup.py, which classifies the package as Development Status :: 2 - Pre-Alpha at version 1.14.0. That classifier is often stale in research code, and the last push to the repository was on 2026-06-21, so the code is not abandoned. But the classifier is what the project publishes about itself, and it tells you the API is not presented as stable. Treat upgrades as something to test rather than assume.

Finally, the evaluation metrics are a fixed menu. If your comparison depends on a metric OCTIS does not implement, you are writing it yourself and feeding it into the loop, at which point you have taken on the integration work the package was meant to remove.

OCTIS against gensim, and what the difference actually is

The obvious alternative is gensim, which is already a dependency of OCTIS and is the standard Python library for LDA and related models. The difference is what each one considers its job.

gensim gives you model implementations and a corpus dictionary, plus a coherence model you can point at a trained model. It does not ship a hyperparameter search, and it does not ship a fixed set of preprocessed benchmark corpora with published document and vocabulary counts. If you use gensim, the comparison harness is yours to write.

OCTIS wraps model implementations, including gensim's, behind a common interface and adds the two things gensim leaves out: a benchmark dataset registry and a Bayesian Optimization loop over hyperparameters. The trade is that you inherit the pins and the interface. A model that does not fit the OCTIS abstraction is harder to plug in than it would be in plain gensim.

If your goal is a single LDA model on your own corpus and you already know the hyperparameters you want, gensim alone is less machinery. The benchmark registry and the search only pay for themselves when you are actually comparing configurations or models, which is the case OCTIS was built for.

Licence and the cost of keeping OCTIS current

OCTIS is released under the MIT licence, per the LICENSE file and the classifier in setup.py. MIT is permissive: it allows commercial use, modification and redistribution provided the copyright notice and licence text are retained. The repository does not document any separate terms for the preprocessed datasets, and the README table attributes each one to an external source (20Newsgroups from scikit-learn, DBLP and M10 from a cited paper, DBPedia_IT from the DBpedia ontology). If you redistribute a model trained on one of those corpora, the corpus attribution is a question for whoever owns that data, not something the MIT licence settles. This is a description of what the repository states, not legal advice.

Upgrade cost is driven by the pins rather than by API churn. The version in setup.py is 1.14.0, and the last push was on 2026-06-21. There is a HISTORY.rst file at the top level, which is where release notes would live, but the README does not document a deprecation policy or a rollback procedure. Before upgrading in a working pipeline, the concrete check is whether the new version still resolves scikit-learn==1.1.0 and numpy<2.0 against your other dependencies; if it moves those bounds, the upgrade is an environment change, not just a package bump.

Editorial conclusion

Adopt OCTIS if you need to compare classical and neural topic models on one dataset with shared metrics and a documented search over hyperparameters. Do not adopt it if you need a production topic inference service, streaming updates, or a maintained dependency set you cannot pin. Before committing, check that scikit-learn==1.1.0 and numpy<2.0 resolve in your environment, confirm the dataset name you need is in the case-sensitive list, and run the CLI help to see which subcommands the installed version exposes.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. MIND-Lab/OCTIS on GitHub
4. README