xlstm: The Official xLSTM Package, Its 7B Variant, and the Kernels You Have to Install Separately

Two Architectures in One Repository, With Different Dependencies

The repository holds two things that are easy to conflate. The first is the xLSTM architecture from the NeurIPS paper (arXiv 2405.04517), exposed through xLSTMBlockStack for non-language or mixed-architecture use and xLSTMLMModel for token-based language modeling. The second is xLSTM Large, the architecture behind the 7B model described in arXiv 2503.13427, whose code lives in xlstm/xlstm_large and is described in the README as optimized for training throughput and stability. The README states that the xLSTM Large implementation is a standalone single file at xlstm/xlstm_large/model.py and, apart from mlstm_kernels, has no dependency on the NeurIPS implementation. That separation matters: you can read and vendor the large-model file without pulling in the older block stack, and the two code paths carry different kernel requirements. If you are evaluating the project for a language model, the xlstm_large path is the relevant one. If you want an LSTM-style recurrent block to drop into a non-language sequence model, xLSTMBlockStack is the entry point.

What Exponential Gating and Matrix Memory Change, and What They Do Not

The README describes the mechanism at a high level: exponential gating with normalization and stabilization techniques, plus a new matrix memory, presented as overcoming limitations of the original LSTM. It does not, in the supplied material, specify the stabilization math, the normalization scheme, or how the matrix memory is parameterized. Treat the architectural claims as pointers to the papers rather than as something the repository explains. What the repository does give you concretely is the configuration surface that exposes these choices: xLSTMLargeConfig takes embedding_dim, num_heads, num_blocks, vocab_size, return_last_states, mode, and three kernel selectors. The kernel selectors are where the architecture meets the hardware, and they are the part most likely to determine whether your first forward pass succeeds. mode is set to "inference" in the demo, which tells you the config distinguishes inference from another mode, though the README does not enumerate the alternatives.

Installing xlstm and the Separate mlstm_kernels Package

The README gives a minimal installation path and a tested environment file. The tested route is conda: create an environment from environment_pt240cu124.yaml with conda env create -n xlstm -f environment_pt240cu124.yaml, then conda activate xlstm. For the package alone, pip install xlstm, or clone and install editable with git clone https://github.com/NX-AI/xlstm.git, cd xlstm, pip install -e . The package is based on PyTorch and the README says it was tested for versions >=1.8. For the xLSTM Large 7B model there is an extra step the README states plainly: install mlstm_kernels via pip install mlstm_kernels, because that package provides the fast kernels. This is the detail that trips people up. Installing xlstm alone gets you the code but not the optimized kernels the 7B configuration expects. The environment filename itself encodes a version pairing (pt240cu124), which is a reasonable hint that the tested combination is specific rather than a wide matrix.

Kernel Selection: Triton, Native, and the Hardware Boundary

The demo notebook at notebooks/xlstm_large/demo.ipynb shows the fast configuration: chunkwise_kernel="chunkwise--triton_xl_chunk", sequence_kernel="native_sequence__triton", step_kernel="triton". The README annotates xl_chunk as equivalent to the TFLA kernels. For non-NVIDIA hardware the README recommends swapping all three to native implementations: chunkwise_kernel="chunkwise--native_autograd", sequence_kernel="native_sequence__native", step_kernel="native", with the comment that these use no Triton kernels. This is a real constraint, not a footnote. The README says the model was tested mostly on NVIDIA GPUs and that the Triton kernels should also run on AMD GPUs, which is a statement of expectation rather than a tested guarantee. For Apple Metal it recommends the native PyTorch path for now and points to a community MLX port, xLSTM-metal, hosted outside this repository. If you are on Apple Silicon, plan for the native path and expect the kernel-level speedups to be unavailable.

The sLSTM CUDA Kernels Have a Hard Compute Capability Floor

The NeurIPS-path sLSTM CUDA kernels require Compute Capability >= 8.0, per the README's link to NVIDIA's GPU list. That excludes older cards outright. If compilation fails, the README suggests setting export TORCH_CUDA_ARCH_LIST="8.0;8.6;9.0", crediting a contributor for the workaround. For include-path problems there is an environment variable, XLSTM_EXTRA_INCLUDE_PATHS, which can be set in the shell (export XLSTM_EXTRA_INCLUDE_PATHS='/usr/local/include/cuda/:/usr/include/cuda/') or inside Python via os.environ. The README also warns that torch and CUDA versions must match for custom setups. The existence of these escape hatches is informative: the build path is sensitive enough that the maintainers document arch-list pinning and include-path injection rather than assuming a clean compile. For standalone, faster sLSTM kernels the README points to the separate FlashRNN library. None of this applies if you stay on the xLSTM Large path, which is a meaningful reason to prefer it for language modeling work.

Where xlstm Is the Wrong Tool

The repository is model code. The README documents installation, configuration, a forward pass, and kernel selection. It does not document a training loop, a data pipeline, a tokenizer, checkpoint conversion, or a fine-tuning recipe. If your task is to train an xLSTM from scratch on your own corpus, this package gives you the architecture and the kernels but leaves the surrounding machinery to you. The demo notebook's own framing supports this: the README says it initializes a random model and performs a forward pass. Random initialization plus a forward pass is a smoke test, not a workflow. A second boundary is hardware. On Apple Silicon the README routes you to native PyTorch or to an external MLX port, so the optimized kernels are not the story there. A third is scope creep: if you only need a recurrent block inside an existing non-language model, pulling in the xLSTM Large configuration and its mlstm_kernels dependency is unnecessary weight, and xLSTMBlockStack is the smaller surface.

Alternatives and How the Approach Differs

The README itself positions xLSTM against Transformers and State Space Models, and the two comparison points behave differently. A Transformer implementation gives you attention with quadratic cost in sequence length and a large body of optimized kernels and serving infrastructure; xLSTM's pitch is a recurrent formulation, which changes the inference profile rather than the training recipe. State Space Models such as Mamba-style architectures also target long sequences with sub-quadratic cost, but they derive from a different recurrence and a different parameterization than the exponential-gated LSTM lineage here. The practical difference for an adopter is ecosystem depth: attention-based stacks have more tooling for training, quantization, and serving, while xLSTM's fast path routes through mlstm_kernels and Triton, with a native fallback that trades speed for portability. Within the xLSTM family there is also a fork in the road. FlashRNN, referenced by the README, provides standalone sLSTM kernels, and the community xLSTM-metal port targets Apple Silicon through MLX. Choosing among them is a hardware decision before it is an architecture decision.

Maintenance, Versions, and the Licence Split

The package is published on PyPI and the repository is active, with v2.0.6 released 2026-09-07 and v2.0.4 in 2025-05-28. The gap between those two releases is roughly fifteen months, which is worth noting if you depend on prompt fixes: this is not a fast-cadence project, and the version jump from 2.0.4 to 2.0.6 suggests patch-level changes rather than a steady stream. The code in this repository is Apache-2.0. The xLSTM 7B model weights are a separate matter: the README's Hugging Face badge links to a license labelled nxai_community, which is not Apache-2.0 and is not reproduced in the supplied material. That distinction is the one to check before shipping anything built on the 7B weights, and it is a licensing question rather than a technical one, so read the linked terms rather than inferring from the repository licence. The mlstm_kernels package is a second external dependency with its own release cycle, so an upgrade of xlstm may or may not line up with the kernels you have installed.

Editorial conclusion

Adopt xlstm if you want reference implementations of the xLSTM block stack or the xLSTM Large 7B architecture inside a PyTorch project, and you are prepared to install mlstm_kernels separately for the fast Triton path. Do not adopt it expecting a training or fine-tuning pipeline: the README documents model code, configs, and a forward-pass demo, and nothing about a trainer. Before committing, verify three things on your own hardware: that your GPU meets Compute Capability >= 8.0 if you intend to use the sLSTM CUDA kernels, that mlstm_kernels installs cleanly against your torch and CUDA versions, and that the xLSTM 7B weights license on Hugging Face is acceptable for your use, since it is listed as nxai_community and not Apache-2.0.

Official sources

1. License: Apache-2.0
2. NX-AI/xlstm on GitHub
3. Project website
4. README
5. Releases