XGBoostLSS: Distributional Regression on Top of XGBoost
An extension of XGBoost to probabilistic modelling
At a glance
- What is it?
- XGBoostLSS extends XGBoost so that every parameter of a chosen response distribution is modelled as a function of covariates, not just the mean. It is Apache-2.0 Python, built on PyTorch and Pyro, and it asks for more setup discipline than a standard booster.
- Who is it for?
- Adopt XGBoostLSS when you need the full conditional distribution of a univariate or multivariate target and you are willing to pick a distributional family and tune its parameters. Do not adopt it if a point forecast plus a fixed-width interval is enough, or if you cannot accept PyTorch and Pyro in your dependency tree.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 33 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What XGBoostLSS adds to a plain XGBoost model
A standard XGBoost regression predicts one number per row. If you want uncertainty around that number, you usually bolt on a quantile objective, train several models, and hope the pieces agree. XGBoostLSS takes a different route: it models and predicts the full conditional distribution of the target as a function of the covariates, as the README puts it. Every parameter of the chosen distribution gets its own booster rather than only the location. For a Gaussian response that means a model for the mean and a second model for the variance, both driven by the same feature matrix. The payoff is that prediction intervals and quantiles of interest fall out of the fitted distribution instead of being estimated separately. The intended audience is the person who already reaches for gradient boosting on tabular data but needs calibrated uncertainty: risk, demand, insurance-style severity, or any setting where the spread matters as much as the centre.
How the distributional parameters get estimated
The mechanism rests on a division of labour. PyTorch supplies automatic differentiation: gradients and Hessians of every distributional parameter are derived rather than hand-coded, which is what makes it practical to support a long list of families. Pyro supplies the distributional vocabulary. XGBoost supplies the boosting machinery, and the README claims full compatibility with all the features and functionality of XGBoost, so the tree-building side is not a reimplementation. Training is therefore a cyclic affair: for each distributional parameter, compute the derivatives of the negative log-likelihood with respect to that parameter, fit or update a booster, and move to the next parameter. The README notes that the framework is initialized with suitable starting values to improve convergence of estimation, which matters because a variance or shape parameter fitted from a cold start can wander. On top of this base the project layers Normalizing Flows for multi-modal shapes, Mixture-Densities for heterogeneous data, Zero-Adjusted and Zero-Inflated variants for excess zeros, multi-target regression for multivariate responses and their dependencies, and Expectile Regression as an alternative route to the predictive distribution. Each of those is a separate modelling choice with its own failure modes.
Installation and the shape of a first run
Two install paths are documented. For the released package: pip install xgboostlss. For the development version: pip install git+https://github.com/StatMixedML/XGBoostLSS.git. Beyond that the README does not spell out an end-to-end script; it points to the example section of the documentation for guidance on how to use the framework, with a Gaussian regression example as the entry point. That is a deliberate choice by the maintainers and it is the first place to look, because the API surface is larger than a single fit call. Expect to make three decisions before training: which distributional family to use, which features feed which parameter, and how the hyperparameter search is configured. The README states that automated hyperparameter search, including pruning, is done via Optuna, so a tuning run is part of the intended workflow rather than an optional extra. Model output is explained with SHapley Additive exPlanations, which the project lists as a supported feature. What the README does not give is a default configuration you can copy, so budget time for reading the distribution pages before writing code.
Where the framework becomes awkward
The cost of flexibility is that you must choose a distributional family before you can fit anything, and that choice is not free. Pick a Gaussian and you have assumed symmetric, light-tailed errors; the variance model will not rescue you from a heavy tail. The README lists Normalizing Flows and Mixture-Densities as answers to complex, multi-modal data, but those are additional machinery with their own tuning burden, not a switch you flip. The second constraint is the dependency stack. The framework is built upon PyTorch and Pyro, so adopting XGBoostLSS is not adopting a booster, it is adopting a deep learning runtime and a probabilistic programming library alongside it. The v0.5.0 release notes mention dependency minimization and Python 3.13 support, which suggests the maintainers are aware of the weight, but the PyTorch and Pyro requirement is structural. Third, the multi-parameter cyclic fitting means training time scales with the number of distributional parameters, and Optuna search multiplies that again. If your problem is a point forecast on a deadline, this is the wrong tool.
How it differs from a quantile-regression setup
The obvious alternative is quantile regression, either XGBoost's own quantile objective or a dedicated library. The difference in approach is fundamental. Quantile regression fits a separate model per quantile and makes no claim about the shape of the distribution between them; you can get crossing quantiles and no density. XGBoostLSS fits one coherent distribution and reads quantiles off it, so intervals are internally consistent by construction. The trade is that you commit to a parametric family, and if the family is wrong the intervals are confidently wrong. A second alternative is a Bayesian additive model or a GAMLSS-style implementation in R, which shares the distributional-regression idea and the parameter-by-parameter fitting loop. Those give you a different inference story and a different ecosystem; XGBoostLSS gives you gradient boosting on tabular data with the XGBoost feature set. If your covariates are few and interpretable, the GAMLSS route may be simpler. If your covariates are many and interactions matter, the boosting approach is the reason to be here.
Maintenance, releases and the licence
The release cadence is uneven. v0.4.0 landed in August 2023, then the project jumped to v0.5.0 in October 2025 and v0.6.0 and v0.6.1 in December 2025. The last push to the repository is dated August 2026. That pattern suggests a project that goes quiet for long stretches and then ships a batch, which is normal for research-adjacent tooling but worth knowing if you need predictable upgrade windows. The v0.5.0 notes flag Python 3.13 support and dependency minimization, so an upgrade from 0.4.x is not a drop-in; read the release notes before moving. The licence is Apache-2.0, which is permissive and includes an explicit patent grant, and it is compatible with commercial use. That is not legal advice; if you redistribute the library or ship it inside a product, have your own counsel confirm the obligations, particularly around NOTICE files and any bundled dependencies. The framework also depends on PyTorch and Pyro, whose licences you inherit separately.
Editorial conclusion
Adopt XGBoostLSS when you need the full conditional distribution of a univariate or multivariate target and you are willing to pick a distributional family and tune its parameters. Do not adopt it if a point forecast plus a fixed-width interval is enough, or if you cannot accept PyTorch and Pyro in your dependency tree. Before committing, verify three things against the documentation: whether your target's distribution is on the supported list, how the multi-target case sets up its dependencies, and whether the automatic hyperparameter search with Optuna fits your compute budget. The framework's value is bounded by that list of distributions.
Community notes