PyCM: a confusion matrix library that computes statistics you would otherwise hand-roll

The gap PyCM fills between sklearn.metrics and a spreadsheet

Most Python evaluation code stops at accuracy, a classification report, and maybe a confusion matrix rendered as an array. That is enough when you have two classes and one number to defend. It stops being enough when a model has several classes and the interesting question is which class is being confused with which, and by how much. PyCM is aimed at that case. The README describes it as a multi-class confusion matrix library that accepts both input data vectors and a direct matrix, and calls it a tool for post-classification model evaluation that supports most classes and overall statistics parameters. The target user is stated plainly: data scientists who need a broad array of metrics for predictive models. The problem it solves is not building the matrix. It is producing a consistent set of derived statistics from that matrix without writing the formulas yourself and getting the edge cases wrong.

Two input paths and one ConfusionMatrix object

The mechanism is a single class. You construct ConfusionMatrix with either actual_vector and predict_vector, or with a matrix you already have. In the README example, twelve actual labels and twelve predicted labels produce cm.classes as [0, 1, 2] and cm.table as a nested dictionary keyed by actual class then predicted class. Everything else hangs off that object. cm.print_matrix() renders the raw counts, cm.print_normalized_matrix() renders row-normalised proportions, and cm.stat(summary=True) returns two blocks: Overall Statistics and Class Statistics. The overall block in the example includes ACC Macro, F1 Macro, FPR Macro, Kappa, Overall ACC, PPV Macro, TPR Macro, Zero-one Loss, and a SOA1(Landis & Koch) interpretation that maps the kappa value onto a verbal label such as Fair. The class block is a table with one column per class and rows for ACC, AUC, AUCI, F1, FN, FP and more. That layout is the design decision worth noting: PyCM does not try to be a metrics namespace with one function per metric. It computes the set and hands you a table.

Installing PyCM and the version pins you inherit

The README gives three install paths. From PyPI, pip install pycm==4.6. From source, download the v4.6 archive or the dev branch and run pip install . . From Conda, conda install -c sepandhaghighi pycm. The version warnings matter more than the commands. PyCM 4.3 is the last version supporting Python 3.6, 3.9 is the last for Python 3.5, and 2.4 is the last for Python 2.7 and Python 3.4. If you are on an older interpreter, the current release is not available to you and you are installing a version that no longer receives changes. Plotting is a separate concern: the README states that plotting capability requires Matplotlib 3.0.0 or newer, or Seaborn 0.9.1 or newer. Neither is a hard dependency for the statistics themselves, so a minimal install can stay free of plotting libraries. There is also a MATLAB route: install MATLAB 8.5 or newer, install Python 3.7 or newer with pip, run pip install pycm, then point MATLAB at the interpreter with pyversion PYTHON_EXECUTABLE_FULL_PATH.

What the class statistics table does not tell you

The output is a summary. It is not a diagnostic. A class with an F1 of 0.4 appears as a row in a table, and nothing in the object explains why. There is no per-sample attribution, no list of the specific indices that were misclassified, no confidence information. If your question is why the model confuses class 1 with class 2, PyCM gives you the count of that confusion and stops. The normalised matrix helps you see the pattern, but the interpretation is yours. A second limitation is the shape of the API. Because cm.stat() returns formatted text rather than only structured data, code that parses PyCM output as strings is fragile across releases. The README shows a printed table, which is designed for a human reading a notebook, not for a pipeline that needs a stable machine-readable contract. If you need to feed metrics into a dashboard or a test assertion, check what structured accessors the documentation exposes before you build on the printed form.

Where scikit-learn is the better fit, and where it is not

scikit-learn's confusion_matrix and classification_report cover the common case, and they live in the same library you probably trained the model with. The difference in approach is scope and coupling. scikit-learn gives you the matrix and a fixed set of per-class precision, recall and F1 values, integrated with the rest of its estimator API. PyCM gives you a larger catalogue of named statistics, including macro aggregates and interpretation bands such as the Landis and Koch kappa scale, in a standalone object that has no relationship to how the model was trained. If you want one call after fit and predict, scikit-learn is shorter. If you want AUC per class, zero-one loss, and a normalised view from the same object, PyCM is doing work you would otherwise write yourself. The honest framing is that these overlap heavily on the basics and diverge on breadth. PyCM is not a replacement for a training or evaluation pipeline; it is a component you call at the end of one.

Maintenance, release cadence and the MIT licence

The repository is not archived and the last push recorded is 2026-08-31. Recent releases are v4.6 in March 2026, v4.5 in October 2025, and v4.4 in August 2025, which suggests a release roughly every few months rather than continuous churn. For a metrics library that is a reasonable cadence: the underlying formulas do not change, so most releases are likely to be compatibility and feature work. The cost you carry is the version pinning described above. Each Python version drop is a breaking boundary for someone, and the README documents those boundaries explicitly rather than hiding them. The licence is MIT, which is permissive and permits use in closed-source products, but this is a description of the licence identifier only and not legal advice; check the LICENSE file in the repository and your own organisation's policy. There is no stated commercial support tier in the material, so maintenance expectations should be set accordingly.

A concrete decision rule for trying PyCM

Install it in a scratch environment with pip install pycm==4.6, construct a ConfusionMatrix from the twelve-element vectors in the README, and compare cm.stat(summary=True) against whatever you currently compute. If the extra rows (AUC per class, Zero-one Loss, the kappa interpretation) are things you already report or want to report, PyCM removes hand-written code. If your evaluation is two-class and your report is precision, recall and F1, the added surface area buys you little and you are taking on a dependency plus a version-pin conversation. The deciding factor is the metric set, not the matrix. The matrix is the easy part.

Editorial conclusion

Adopt PyCM if you are evaluating a multi-class classifier and want a large set of class-level and overall statistics from a single object, including normalised matrices and optional plots. Do not adopt it if you need per-sample explanations, threshold tuning, or integration with a training loop; it is a post-hoc calculator. Before committing, verify that the statistics you actually report are present in the class statistics output for your number of classes, confirm which Python version you are pinned to against the release notes, and check whether Matplotlib or Seaborn is already in your environment if you intend to call plot().

Official sources

1. License: MIT
2. Project website
3. README
4. Releases
5. sepandhaghighi/pycm on GitHub