CMS: a distributed contest management and grading system

What CMS is for, and the event it was built around

CMS stands for Contest Management System, and the project describes it as a distributed system for running and, to some extent, organizing a programming contest.

The word distributed is the load-bearing one. This is not a script that runs a test suite against submissions on one laptop. It is a set of services, with a database, that together deliver a contest to participants while staff watch it and a grader decides the scores. The repository layout supports that reading, with cms as the main package and sibling packages cmscommon, cmscontrib, cmsranking, cmstestsuite and tools, plus config, docs, scripts and a debian directory for packaging.

The project's own framing of its scope is honest in a useful way. CMS has been designed to be general and to handle many different types of contests, tasks and scorings, and it says in the next sentence that it has nonetheless been explicitly built to be used in the 2012 International Olympiad in Informatics, held in September 2012 in Italy.

That sentence tells you what the design is optimized for. IOI-style events mean individual contests, timed sessions, submissions graded against test cases with partial scores, and a scoring model that is not a simple pass or fail. The generality is a claim about the configuration, while the IOI 2012 lineage is a claim about what has actually been exercised.

The licence is AGPL-3.0-or-later, and the classification in the package metadata is Development Status 5, Production/Stable, with the operating system classifier pinned to POSIX Linux.

Two audiences: packaged releases and the source repository

The README separates the people who run a contest from the people who develop the system, and it does it before anything else.

For end users, the documentation says the best option is to download the latest stable version of CMS, which is already packaged on the project homepage at cms-dev.github.io. The git repository, whose main branch holds the development version, is described as intended for developers and for anyone curious about how the code works or wants to hack on it.

The distinction matters for expectations. The packaged versions are the ones a contest organiser should be looking at, and the repository is where the 1.6.dev0 version in pyproject.toml lives, which is a development version by its own name.

Installation itself is not described in the README. The documentation lives at cms.readthedocs.org, and the README's advice to learn how to install and use CMS is to read it. There is an install.py script at the repository root and a debian directory for packaging, and the Dockerfile plus the docker directory cover the containerised route.

The project also says it has been used in several official and unofficial contests, with a list of testimonials on the homepage, and it invites organizers who used it to be added to that list.

The service stack: Tornado, SQLAlchemy, PostgreSQL and gevent

The dependency list in pyproject.toml describes the architecture, and it is specific enough to tell you what a deployment involves.

The web layer is Tornado at 4.5.3, with Werkzeug at 3.1.5 alongside it, and the backports.ssl-match-hostname package is pinned in with the comment that it is required by tornado<5.0. The database layer is SQLAlchemy constrained to the range above 1.3 and below 1.4, with psycopg2 at 2.9.10, which makes PostgreSQL the supported database.

Concurrency comes from gevent at 25.8.1 with greenlet, and the presence of netifaces suggests the code inspects network interfaces directly, which is what a distributed service needs to know its own addresses and to route traffic between the services that make up a contest.

The rest of the list is ordinary infrastructure. requests for outbound calls, psutil for process management, pycryptodomex for cryptography, patool for archive handling, bcrypt for password hashing, chardet for character encoding detection, babel for localisation, and pyxdg for desktop paths.

Two of these deserve a second look. chardet and babel are there because a contest runs in more than one language, and the project runs a Weblate instance so that CMS itself is translated, with a translation status badge in the README header. The internationalisation is not an afterthought bolted on for the logo, it is part of a system that has to present an interface to participants who may not read the organiser's language.

The evaluation image carries a dozen compilers

The Dockerfile is where CMS earns its keep, because the grading environment is the hard part of running a contest and it is assembled here.

The base image is a build argument defaulting to ubuntu:noble, and the comment states that the supported combinations are ubuntu:noble and debian:bookworm.

# syntax=docker/dockerfile:1
# Supported combinations: ubuntu:noble, debian:bookworm.
ARG BASE_IMAGE=ubuntu:noble
FROM ${BASE_IMAGE}

The package list is the interesting part. It installs default-jdk-headless for Java, fp-compiler for Pascal, ghc for Haskell, mono-mcs for C#, php-cli, rustc for Rust, pypy3 and python3 with python3-dev, plus build-essential for C and C++. It also installs postgresql-client, curl, git, zip, wait-for-it, sudo and the shared-mime-info and cppreference-doc-en-html packages.

        default-jdk-headless
        fp-compiler
        ghc
        mono-mcs
        php-cli
        rustc
        pypy3

So a participant can submit code in any of those languages and it will be compiled in the grading environment. That is not a detail, because the alternative for every organiser is to build their own toolchain image and keep it in step with the contest's allowed languages.

The sandboxing is handled with isolate, a cgroup based tool, and the Dockerfile adds the UCW isolate package repository, imports the signing key, installs it and then rewrites the cgroup root in the configuration to point at /sys/fs/cgroup. That line is the difference between isolation working inside a container and isolation silently doing nothing.

Python 3.11 alongside SQLAlchemy 1.3 and Tornado 4.5

The dependency pins in this project tell a story that anyone planning a deployment should read carefully.

The package metadata requires Python 3.11 or later, and the classifier list names Python 3.11, while the version in pyproject.toml is 1.6.dev0 and the latest tagged release listed for the repository is v1.5.1 from 2025-06-07, with v1.5.0 from 2024-12-26 and a v1.4.rc1 from 2018.

Against that modern Python floor, SQLAlchemy is held at above 1.3 and below 1.4 and Tornado is held at 4.5.3, with the ssl-match-hostname backport needed because of it. The last release of Tornado 4 dates from well before Python 3.11 existed, so the project is running a contemporary interpreter on a deliberately conservative web and ORM stack.

There is a reasonable engineering reason for that, which is that contest infrastructure is not something you want to break during an event, and a pinned dependency set is a feature when the whole point is that the same binary behaves identically on every grading machine. It is also a constraint, because anything you build around CMS in the same environment inherits the same pins, and the SQLAlchemy 1.3 API in particular is not what a current application would be written against.

The last push to the repository was on 2026-10-05, so the project is being worked on even though the release tags are infrequent.

Running an event: logs, chat, mailing lists and issues

The support section is written by people who have run events, and it tells you where to look when something goes wrong mid-contest.

For troubleshooting, the relevant log files are in /var/local/log/cms/, and the README asks you to upload them to a pastebin when you ask for help. That path is the most useful single fact in the document for an organiser, because a failure during a contest is usually a configuration or environment problem and the log is the only evidence of it.

Help is offered through a Telegram chat and a support mailing list, [email protected], where the README notes that no registration is required and the archives are public. Development questions go to a separate list, contestms-discuss, and release announcements go to contestms-announce, which is the list to watch if you run instances.

The project also asks people not to file GitHub issues to ask for help, and to use the mailing list or Telegram instead, on the grounds that someone is more likely to answer there.

Bugs themselves do go to the issue tracker, following the instructions in the issue template, and the repository carries a build workflow badge, a Codecov badge and a Weblate translation badge, which together describe a project with automated tests, coverage tracking and community translation.

The limits: an IOI-shaped system, Linux only, and a general claim

Three boundaries are worth stating before you commit.

The first is the shape of the contest. The design goal is general, but the exercised case is an IOI-style event with individual timed sessions, test case based scoring and partial credit. A team match, a hackathon, an open-ended challenge with human judging, or a marathon needs a different system, and no configuration is documented for those.

The second is the platform. The classifiers list POSIX Linux and nothing else, the Dockerfile is a Linux image, the isolation mechanism is cgroup based, and the sandboxing and networking assumptions all sit on top of a Linux kernel. A macOS or Windows organiser is out of scope.

The third is the third-party surface, and it is a real operational cost rather than a nitpick. A grader that judges untrusted code from participants is a security boundary, and this project implements it with isolate, cgroups, and a base image you assemble yourself. The AGPL-3.0-or-later licence also means that if you modify CMS and run a modified version as a service, the licence's terms apply to that modified version, which is a point to take seriously for a hosted or shared instance.

Within those boundaries, for a contest that looks like the ones CMS was built for, the pieces are all present: contestant interface, admin interface, scoring, ranking, sandboxed evaluation across a dozen languages, and a PostgreSQL database.

Editorial conclusion

Adopt CMS if you are running a competitive programming contest that needs a contestant interface, an admin interface, live scoring and a reproducible evaluation sandbox, because those parts are built and tested as one system and the Docker image already contains the compilers. Do not adopt it for a single practice session or a small internal exercise, since the deployment assumes multiple services and a database. Before you plan the install, read the documentation at cms.readthedocs.org and check the dependency pins, because SQLAlchemy is held below 1.4 and Tornado is held at 4.5.3, which constrains what you can add around it.

Frequently asked questions

Official sources

1. cms-dev/cms on GitHub
2. License: AGPL-3.0
3. Project website
4. README
5. Releases