generative-recommenders: HSTU and M-FALCON for trillion-parameter sequential recommendation

DLRMs reformulated as generative modeling: the core argument of the ICML 2024 paper

Classical deep learning recommendation models (DLRMs) treat recommendation as a classification or ranking problem. This project, described in "Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations" (ICML 2024, proceedings.mlr.press/v235/zhai24a.html), argues that reformulating DLRMs as generative modeling removes the compute scaling bottlenecks that prevent those models from reaching trillion-parameter scale.

Two algorithms are the result: HSTU (Hierarchical Sequential Transduction Units) as the core model architecture and M-FALCON as an inference accelerator. The README says the repository also demonstrates, for the first time, a scaling law in a deployed recommendation system at billion-user scale. That claim is a research claim with no reproduction steps given outside the public experiments on MovieLens and Amazon Reviews.

The paper appeared in the ICML 2024 proceedings, and the repository sits under the meta-recsys GitHub organization. Topics tagged on the repository include hstu, generative-recommenders, recsys and recommendations.

HSTU attention kernels in Triton and CUDA; FlashAttention V3 backs the H100 path

HSTU's attention implementation lives in ops/. Two kernel paths exist: ops/triton holds Triton kernels for the efficiency experiments, and ops/cpp holds CUDA kernels. Within ops/cpp, ops/cpp/hstu_attention is the most specific: it is built on FlashAttention V3 and carries state-of-the-art efficiency on H100 GPUs according to the README.

A benchmark script at ops/hstu_ultra.py provides named, model-derived HSTU Ultra self-attention benchmarks. M-FALCON handles inference acceleration. The README claims HSTU and M-FALCON together deliver 10x to 1000x speedups over classical deep recommendation training and inference. That is a wide range, and the README does not break down which configurations produce 10x versus 1000x.

The generative_recommenders Python package is in the generative_recommenders/ subdirectory. Entry points are main.py for training and preprocess_public_data.py for data preparation, both at the repository root.

Install from requirements.txt; torch>=2.6.0, fbgemm_gpu>=1.1.0 and torchrec>=1.1.0 are required

The recommended install path is:

pip3 install -r requirements.txt

This path was tested on Ubuntu 22.04 with CUDA 12.4 and Python 3.10. The requirements.txt pins: torch>=2.6.0, fbgemm_gpu>=1.1.0, torchrec>=1.1.0, gin_config>=0.5.0, pandas>=2.2.0, tensorboard>=2.19.0, and pybind11. setup.py also adds matplotlib and click. fbgemm_gpu and torchrec are Meta's libraries for GPU-accelerated embedding operations and distributed recommendation training, so this codebase is tested against Meta's specific GPU software stack.

For a manual alternative, first install PyTorch following the official instructions, then:

pip3 install gin-config pandas fbgemm_gpu torchrec tensorboard

The package version in setup.py is 0.1.0, the code is under Apache-2.0, and the last push was on 2026-10-02.

Training on MovieLens requires a 24GB GPU, a gin config file and a --master_port flag

Public experiments reproduce Section 4.1.1 of the paper on MovieLens-1M, MovieLens-20M and Amazon Reviews (Books). Data preprocessing runs first:

mkdir -p tmp/ && python3 preprocess_public_data.py

A GPU with 24GB or more HBM is needed for most datasets. Training on ML-1M then starts with:

CUDA_VISIBLE_DEVICES=0 python3 main.py --gin_config_file=configs/ml-1m/hstu-sampled-softmax-n128-large-final.gin --master_port=12345

Gin config files for all three datasets sit under configs/ml-1m, configs/ml-20m and configs/amzn-books, making it straightforward to switch datasets by changing the --gin_config_file path. Logs write to exps/ by default and can be inspected with:

tensorboard --logdir ~/generative-recommenders/exps/ml-1m-l200/ --port 24001 --bind_all

Gin is an injection mechanism: any configuration change requires editing or creating a .gin file rather than passing command-line flags directly. That keeps experiment configurations reproducible but adds a file-editing step that is not obvious from the main.py entry point alone.

HSTU-large outperforms SASRec by 15.5% on ML-1M, but the comparison baseline is not stock SASRec

On MovieLens-1M, HSTU-large achieves HR@10 of 0.3294 against SASRec at 0.2853, a 15.5% gain. NDCG@10 improves from 0.1603 to 0.1893, a gain of 18.1%. On MovieLens-20M, HSTU-large reaches HR@10 of 0.3556 versus SASRec's 0.2889, up 23.1%. On Amazon Reviews Books, HSTU-large scores HR@10 of 0.0478 against SASRec's 0.0306, a 56.7% increase. These figures were verified as of 04/15/2024 per the README.

Two caveats are written into the same table notes. First, the SASRec rows swap the original binary cross entropy loss for sampled softmax loss, making the baseline a modified variant. Second, BERT4Rec and GRU4Rec rows come from an external paper that used full negatives, while the HSTU and SASRec rows used 128 or 512 sampled negatives. The README itself says this "slightly favors" the BERT4Rec and GRU4Rec rows. A reader who skims the table headers without reading the footnote will see larger HSTU margins against those two models than a fully controlled comparison would show.

Fractal expansion converts ML-20M's 20 million ratings into a 3 billion synthetic set

Scale experiments beyond the public datasets use a synthetic dataset called ML-3B, generated from ML-20M by fractal expansion (described in https://arxiv.org/abs/1901.08910). A pre-generated download is available:

pip3 install gdown
mkdir -p tmp/ && cd tmp/
gdown https://drive.google.com/uc?id=1-jZ6k0el7e7PyFnwqMLfqUTRh_Qdumt-
unzip ml-3b.zip && rm ml-3b.zip

For generating the dataset from scratch:

python3 run_fractal_expansion.py --input-csv-file tmp/ml-20m/ratings.csv --write-dataset True --output-prefix tmp/ml-3b/

ML-3B is a synthetic expansion, not real user data. Whether its statistical properties generalize to behavior in a production recommendation system is outside what the README addresses.

fbgemm_gpu and torchrec bind this project to Meta's GPU stack; plain-PyTorch alternatives exist

The dependency on fbgemm_gpu and torchrec is the main infrastructure constraint. Both are Meta's own libraries, and their presence means the project is tested against Meta's specific CUDA configuration. A system where fbgemm_gpu or torchrec fail to install will be blocked before reaching any of the model code. The setup.py entry specifies python_requires=">=3.10", so anything below Python 3.10 is out of scope.

SASRec, the main comparison baseline from the paper, has independent implementations in plain PyTorch. Those alternatives do not depend on fbgemm_gpu or torchrec and can run on a wider range of hardware. What they lack is the HSTU architecture, the M-FALCON inference path, the Triton kernels and the FlashAttention V3 implementation specific to this project. Choosing between them comes down to whether H100 access and Meta's GPU stack are available, since this codebase is documented only for Ubuntu 22.04 with CUDA 12.4.

Editorial conclusion

Use generative-recommenders if you are researching large-scale sequential recommendation and have access to H100 GPUs and Meta's CUDA stack, since the HSTU attention implementation is optimized for that environment. Skip it for production deployments without those dependencies, since fbgemm_gpu and torchrec are not part of the standard PyTorch ecosystem. Before adopting the benchmark numbers, read the table footnotes: the SASRec baseline uses sampled softmax, not the original binary cross entropy loss, and BERT4Rec and GRU4Rec rows used full negatives while HSTU used 128 or 512 sampled negatives.

Frequently asked questions

Official sources

1. Issues
2. License: Apache-2.0
3. meta-recsys/generative-recommenders on GitHub
4. README