TALENT: a tabular data benchmark that also ships the methods it compares

A benchmark and a toolbox that grew in the same directory

TALENT is described as a benchmark with a machine learning toolbox designed to enhance model performance on tabular data. The scale is stated in the repository description: more than 35 deep methods, more than 10 classical methods, and 300 diverse tabular datasets.

The four advantages the README lists are worth reading as design goals. Diverse methods, covering classical algorithms, tree-based methods and the newer deep learning approaches. An extensive dataset collection of 300 datasets, spread over task types, size distributions and domains. Customizability, meaning it is straightforward to add a dataset or a method of your own. And versatile support, covering different normalisations, encodings and metrics.

That last pair of goals is what makes the repository a research tool rather than a leaderboard. A benchmark that cannot accept your method is a leaderboard, and the ability to plug in new datasets and new algorithms is what lets a paper build on the existing state of the art instead of rebuilding the harness.

The academic framing is explicit. The project asks that you cite two papers if you use any of its content: the JMLR paper Talent: A Tabular Analytics and Learning Toolbox from 2025, and A Closer Look at Deep Learning on Tabular Data, which is the study of the same group of authors behind the earlier arXiv work. The bibtex entries are in the README and there is a CITATION.bib file in the repository.

The project is MIT licensed, written in Python, targets Python 3.10 and PyTorch 2.8 or newer, and its documentation is published on readthedocs. The last push was on 2026-10-13 and there are no GitHub releases, so the practical unit of consumption is a commit.

The June 2026 evaluation defaults change what the numbers mean

One entry in the changelog matters more than the method additions, and it is easy to miss because it is phrased as a feature.

From June 2026, calibration metrics are reported by default for every classifier. The Brier score and the expected calibration error are now in the standard output rather than something you have to request. On its own that is a good change, since calibration is exactly the property that separates a well-ranked classifier from a well-calibrated one, and it is cheap to compute.

The second part changes the numbers themselves. Decision thresholds are now tuned on the validation set for binary classification, and those thresholds are then used for accuracy, F1, precision and recall. The changelog is explicit about the boundary: threshold-independent metrics such as AUC, LogLoss, Brier and ECE are unaffected. That distinction is the one to carry into any comparison, because a table of F1 scores from a version before June 2026 and a table of F1 scores from a version after it are not measuring the same thing. The threshold was previously fixed, and now it is chosen with knowledge of the validation data.

The third part is an interface change. RunResult now exposes a uniform predict_proba and predict_labels interface regardless of whether the underlying method natively returns logits or probabilities. For a toolbox whose whole job is putting a dozen methods side by side, removing a per-method branch from the calling code is a large reduction in the work needed to add the next method.

The last piece is a portability fix. Bundled checkpoints are now resolved through importlib.resources, so the methods work from any working directory, which is the kind of detail that only becomes visible when someone runs the code from a different folder and it breaks.

Foundation models arrive in waves, behind one shared interface

The most recent entry in the changelog is from October 2026 and shows how this project absorbs new work: a wave of seven tabular foundation models added at once, namely TabPFN-3.5, LimiX-2, NVIDIA Kumo Tabular, Mitra-v2, Causilo, Xiaomi-TabLDM and EXAONE-Tabular.

What makes that tractable is the shared interface. All foundation models now download their weights automatically and expose the same two parameters, n_estimators and sample_size, so a benchmark configuration written for one of them works for the others. The prior waves follow the same pattern. TabPFN v3 and TabICL v2 arrived in May 2026, TabPFN v2.5 and TabDPT in June, and the TalenT abbreviation aside, the project now ships the full TabPFN family from v1 through v3 alongside TabDPT, TabICL, Mitra and LimiX.

There is one version trap worth flagging. The changelog notes that tabpfn>=9.0 changed the package default to TabPFN-3.5, and that the tabpfn_v3 method now pins the TabPFN-3 checkpoint. So the version of the upstream package, not just the version of TALENT, decides which weights you run. A reproduction that pins TALENT but not tabpfn can silently end up on a different checkpoint.

The optional-method mechanism is also simple. TabFM, the Google Research zero-shot model, is available as an optional method through an extra:

pip install -U "tabfm[pytorch]"

The older entries show the same steady accumulation: RFM from Science in November 2025, Real-TabPFN, xRFM, Mitra from Amazon, TabAutoPNPNet, TabICL at ICML 2025, and before that TRC in August 2026, a post-hoc representation corrector that trains and freezes an FT-Transformer backbone before learning representation re-estimation and light-space mapping.

Dependency floors are deliberate, and they cut both ways

The packaging is unusual enough to be worth reading closely, because the two dependency files disagree in a way that is intentional.

The pyproject.toml uses flit_core as the build backend, with the project name TALENT and the version and description declared as dynamic. Its dependencies are all lower bounds: torch at 2.8.0 or newer, numpy at 1.26.4, pandas at 2.2.2, scikit_learn at 1.4.2, scipy at 1.13.0, optuna at 3.6.1, einops, category_encoders, delu, qhoptim, tqdm and Requests.

The comment above that list explains the reasoning. The versions are the ones TALENT is tested against, but newer releases are allowed so the package installs cleanly into existing environments. The requirements.txt file repeats the same philosophy with a longer comment, and it adds the packages that the pyproject list leaves out: the gradient boosting trio of xgboost, lightgbm and catboost, the annoy approximate nearest neighbour library, xrfm for the RFM family, and huggingface_hub with safetensors for downloading and loading the bundled foundation model checkpoints.

numpy>=1.26.4
torch>=2.8.0
scikit_learn>=1.4.2

The advantage is obvious. A research toolbox that pins exact versions stops being installable in any environment that already has something else, and the authors state the intent plainly, so that TALENT can coexist with other packages.

The cost is reproducibility. The requirements file says newer releases are allowed and generally work, and that if you hit an incompatibility with a much newer release you should pin the offending package back to the version listed there. That is a reasonable stance for a library, and a weaker one for a benchmark, where the whole point is that two runs are comparable. Anyone publishing numbers should freeze their environment, and the changelog note about tabpfn changing its default checkpoint is exactly the kind of event that will otherwise make two results incomparable.

The datasets, the results and the extension packages are part of the deliverable

The repository tree shows that TALENT is more than a package of model wrappers. There is a TALENT/ directory with the code, a docs/ directory, an example_datasets/ directory, a results/ directory, a test/ directory, a visualization/ directory and a resources/ directory holding the logo and other assets.

The 300 datasets are the point of the benchmark, and the README says they cover a wide range of task types, size distributions and dataset domains. The example_datasets directory suggests the intent that a new user can run something without a download step, and the visualization directory is for the plotting side of analysis.

The results are treated as a published artefact too. The changelog records that in March 2026 the TALENT-extension datasets and results were updated, and the link for them is a file sharing link rather than a repository path, which tells you the extension results are large enough to live outside Git. The same part of the README carries a Hugging Face dataset link for TALENT itself, and the toolkit is also the subject of several other papers from the same group, including MMTU, Tabular-Temporal-Shift and BETA, accepted at ICML 2025, and a survey on representation learning for tabular data that organises existing methods into specialised, transferable and general models.

That last categorisation is worth remembering when reading benchmark tables. If the toolbox can be pointed at a method of any of those three kinds, then a comparison across the categories is a choice about what to include, and a fair reading of any result table requires knowing which category each method falls into. The three papers the group has published in this area are the natural place to start for that framing.

Editorial conclusion

Adopt TALENT if you are comparing methods on tabular data and want one codebase that holds the classical baselines, the deep models and the newer tabular foundation models behind a single interface, and if you are willing to read the evaluation defaults carefully because they changed in June 2026. Do not adopt it as a production training stack, since the repository publishes no releases, pins lower bounds rather than exact versions on purpose, and the extension datasets and results live outside the repository. Before you compare any numbers, check whether the classification thresholds were tuned on the validation set in the version you are running, and match the method versions you cite against the checkpoint pinning behaviour, since the tabpfn package version alone changes which TabPFN checkpoint runs.

Frequently asked questions

Official sources

1. Issues
2. LAMDA-Tabular/TALENT on GitHub
3. License: MIT
4. README