scorecardpy: traditional credit scorecards in Python

The classic scorecard workflow, ported to Python

scorecardpy is a Python implementation of an R package called scorecard, and the README states its goal plainly: to make the development of traditional credit risk model scorecards easier and more efficient by providing functions for the common tasks.

Those tasks are listed, and the list is the outline of the method.

Data partitioning comes first, with a split function. Then variable selection, through an information value calculation and a variable filter function. Then weight of evidence binning, with the binning function itself, a plotting helper, an adjustment helper for refining the breaks and a function that applies the fitted bins to new data. Then scorecard scaling, with the scaling function and a companion that applies the card. And finally performance evaluation, with a function that produces the standard metrics and another for population stability.

That is the traditional credit scoring approach rather than a general machine learning toolkit, and it is worth being clear about why that approach is still widely used. A scorecard is a logistic regression on binned variables, where each variable is replaced by the weight of evidence of each of its bins. The result is a model whose every input is a human readable band, which is what makes it explainable to a regulator, auditable by hand and stable over time.

The package is MIT licensed, published on PyPI as scorecardpy, with a project page linked from the README.

The whole workflow in one example

The README ships a complete example, and reading it top to bottom is the fastest way to understand the library.

import scorecardpy as sc

dat = sc.germancredit()

dt_s = sc.var_filter(dat, y="creditability")

train, test = sc.split_df(dt_s, 'creditability').values()

The data comes from a built-in German credit dataset, the variables are filtered by missing rate, information value and identical value rate, and the result is split into training and test sets.

The binning stage follows, and it is where the interesting control lives.

bins = sc.woebin(dt_s, y="creditability")
bins_adj = sc.woebin(dt_s, y="creditability", breaks_list=breaks_adj)

The breaks can be adjusted interactively with a helper, or specified manually as a dictionary of variable name to break points, and the example shows both routes with one commented out. The manual route is what a risk analyst would use, since the automatic bins rarely match a lending policy on the first try.

After the bins are fixed, the training and test sets are converted into weight of evidence values, a logistic regression is fitted with an L1 penalty and the saga solver, the performance is evaluated, the scorecard is scaled from the bins and the model, and the population stability index is computed across the training and test scores.

What weight of evidence binning actually gives you

The mechanics are worth spelling out, because they explain every function name in the package.

Binning takes a continuous variable and splits it into bands, and for each band computes the weight of evidence, which compares the distribution of the band among the good outcomes with its distribution among the bad ones. A band where bad outcomes are over represented gets a large negative weight of evidence, and the other end of the scale. The information value function then summarises how much predictive power the variable carries across its bins, which is what the variable filter uses to drop weak variables.

The transformation to weight of evidence values is what the apply function does, and it is what makes the model readable. Instead of a logistic regression on age and income and months of employment, you get a regression on the weight of evidence of each band, so a prediction can be explained as a sum of contributions that a person can check.

The scaling function then turns that model into a score. Points are assigned per band, scaled around a target score at a target odds, which is what turns a probability into the number a credit officer or an automated decisioning system actually uses.

And the population stability index function is the monitoring piece. It compares the distribution of a score between two periods, and a rising value tells you that the population has shifted and the model may need review, which is a routine control in lending.

The evaluation function produces the other standard numbers, the Kolmogorov-Smirnov statistic and the ROC curve from the same predicted probabilities.

What this fork changed

The README has a section on performance optimisations, and it names two changes against the original package.

The first is in the weight of evidence binning functions. Pre-compiled regular expression patterns and reduced DataFrame chaining make the binning faster, which matters because binning is the slowest part of the workflow and it runs on every variable.

The second is in the performance evaluation. The manual AUC calculation was replaced with the roc_auc_score function from scikit-learn, for what the README calls a significant speedup, and the use of eval was eliminated in favour of dictionary based function dispatch.

That second change is the more interesting one, and the README frames it as being about security as well as performance. A call to eval in library code means executing a string built at runtime, and replacing it with a dictionary lookup means the set of callable functions is fixed at import time rather than computed from input. If your use of the library involves configuration values that reach that dispatch, the difference between the two approaches is the difference between a lookup and arbitrary code execution.

So this fork is best read as a maintenance pass on a working implementation rather than a redesign, which is what you would want from a library that people depend on in a lending pipeline.

Installing and the packaging

There are two installation routes, and they differ only in where the code comes from.

pip install scorecardpy

pip install git+git://github.com/shichenxie/scorecardpy.git

The first takes the release from PyPI and the second takes the current code from GitHub.

The packaging is a classic setuptools module, and the setup file shows a detail that is still common in older Python packages: the version is not written in the setup file but read out of the package's init file with a regular expression, and the long description is read from the README at build time.

with open(path.join(here, 'scorecardpy/__init__.py'), encoding='utf-8') as f:
    __version__ = re.search(r'__version__\s*=\s*[\'"]([^\'"]*)[\'"]', f.read()).group(1)

The classifiers describe the project as a beta with an intended audience of developers and an MIT licence. The repository root holds the package directory, the setup file, a MANIFEST file, a NEWS file that serves as the changelog, a build directory and a GitHub directory for the continuous integration configuration.

The topic tags give a good summary of what the library is about: binning, weight of evidence, WOE binning, credit scoring, scorecard and release.

Where it fits, and where it does not

The honest comparison has two axes.

The first is against the R package it was ported from. The R version has the advantage of being the original, with the reference implementation of the method and a longer history of fixes. The Python version wins on integration, because a data science team that works in Python can keep the credit risk work in the same environment as everything else, and this fork adds the performance and safety improvements on top.

The second is against modern machine learning for credit risk. A logistic regression on binned variables is not the model with the highest predictive power, and gradient boosted trees usually beat it. The reason scorecards persist is that they are explainable, stable and acceptable to a regulator, and the reason they are a burden is that they need binning, monitoring and periodic review, which is exactly the workflow this library implements.

So the question is not whether a tree model would score better. It is whether your market, your regulator and your own model risk function require the traditional form. If they do, this library saves you the mechanical work. If they do not, the binning and scaling steps are overhead and you should use a general modelling stack.

Three practical checks. The binning defaults are a starting point rather than a policy, and the README example shows the manual break specification that a risk analyst would actually use. The logistic regression in the example uses an L1 penalty with the saga solver, which is a reasonable sparse choice but is a modelling decision your team owns. And the population stability function is the one to wire into your monitoring before you go live, because a scorecard that drifts silently is the failure mode that gets discovered by an auditor.

Editorial conclusion

scorecardpy suits a credit risk team that wants to build a classic scorecard, with WOE binning, logistic regression and PSI monitoring, in a Python environment rather than the R original, because the whole workflow is available as functions with a German credit dataset included for practice. It is the wrong choice for a team that has moved to tree based models or needs automated feature engineering at scale, since the library implements the traditional approach rather than a general modelling stack, and it is the wrong choice if you need a supported production dependency. Verify first that the binning behaviour matches the regulatory expectations of your market, that the version you install has the performance changes rather than the older code path, and that the logistic regression setup in the example is the one your own model risk team signs off on.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. Project website
4. README
5. ShichenXie/scorecardpy on GitHub