LIBXSMM 2.0: Tensor Processing Primitives Without Target Flags

The problem LIBXSMM targets: small matrices that BLAS handles badly

Vendor BLAS implementations are tuned for large GEMM calls, where the cost of a function call and argument checking is negligible against the arithmetic. Deep learning and some HPC kernels do the opposite. They call GEMM on matrices with dimensions in the single or low double digits, thousands of times, and at that size the dispatch overhead and the generic kernel loop dominate. LIBXSMM exists for that regime. The README describes it as a high performance library for small dense and sparse linear algebra operations including GEMM and elementwise primitives often seen in deep learning applications. The intended audience is people writing C or Fortran kernels who know their matrix shapes at runtime and want code specialized to those exact shapes, not a library that picks a kernel from a fixed table. Since version 2.0 the project also positions itself as the reference implementation of Tensor Processing Primitives, a programming abstraction for deep learning and HPC workloads. That second role matters for adoption: the TPP frontend is what the project now considers its primary interface, with convolutions and fully-connected layers described as compositions of GEMM, BRGEMM and elementwise primitives rather than as library entry points.

How dispatch works: JIT specialization instead of target flags

The mechanism is Just-In-Time code generation. You describe the operation with a shape descriptor, call a dispatch function, and get back a function pointer. The README's Hello LIBXSMM example shows the pattern for a GEMM: libxsmm_create_gemm_shape takes m, n, k, the leading dimensions, and four datatype arguments, then libxsmm_dispatch_gemm takes that shape plus flags and a prefetch setting and returns a libxsmm_gemmfunction. You then fill a parameter struct with pointers and call through the function pointer. Elementwise work follows the same shape: libxsmm_create_meltw_unary_shape plus libxsmm_dispatch_meltw_unary for a ReLU, the binary variant for a bias add, and the ternary variant for a fused multiply-add. The dispatch is where specialization happens. The library generates machine code for the specific m, n, k and datatypes you asked for, targeting whichever instruction set the running CPU exposes. That is the basis for the README's claim of build once and deploy everywhere: no special target flags are needed to exploit the available performance. The supported targets are listed as Intel SSE, AVX, AVX2, AVX-512 with VNNI and Bfloat16, Intel AMX, AArch64 with NEON, SVE and SME, RISC-V RVV, and PowerPC 64-bit little-endian POWER10 with VSX and MMA. The datatype list is unusually wide: FP64, FP32, FP16, bfloat16, BF8, HF8, MXBF8, MXHF8, MXINT8, int16, int8, MXBF6, MXHF6, MXFP4, int4, int2 and int1, plus what the README calls various non-standard low precision combinations. The cost of this design is the first call. Generating code takes time that a precompiled kernel does not spend, so a workload that calls each shape once pays for specialization it never reuses.

Building LIBXSMM and running the Hello example

The README gives a concrete path. From the repository root, make STATIC=0 produces a shared library. The Hello example is compiled with gcc -I/path/to/libxsmm/include hello.c -L/path/to/libxsmm/lib -lxsmm -lm -o hello, and run with LD_LIBRARY_PATH=/path/to/libxsmm/lib LIBXSMM_VERBOSE=2 ./hello. LIBXSMM_VERBOSE is the environment variable that controls diagnostic output, and setting it to 2 is how the README's own instructions let you see what the JIT is doing. The include line is a single header, libxsmm.h, which is worth noting if you are integrating into an existing build system: there is no multi-header include graph to wire up. The example itself uses libxsmm_blasint for dimensions, which is the library's BLAS-compatible integer type, and column-major layout, matching the Fortran convention that the surrounding ecosystem uses. A Fortran version of the same example exists at samples/hello/hello.f, and more complete drivers are under samples/xgemm. If you want to know whether LIBXSMM is worth the integration cost, the honest first step is to build the shared library, compile the Hello sample, and read the verbose output on your actual target hardware. The README does not publish expected output for that run, so treat whatever it prints as the baseline you are evaluating against, not as a documented result.

What 2.0 removed, and why that changes the install decision

The 2.0 release is a narrowing, not an expansion. The README states that several application-specific pieces previously shipped in this repository have moved to dedicated companion repositories: the deep learning operators and convolution drivers to LIBXSMM-DNN, the PyTorch integration to the TPP PyTorch Extension, and the spectral-element reproducers to separate locations. The core library, the JIT backend and the TPP frontend remain. Anyone whose mental model of LIBXSMM is the 1.17 era, where convolution drivers and deep learning operators were part of the tree, will find a different repository. The version history makes the gap visible: 1.17 landed in December 2021, and 2.0.0 arrived in June 2026, with 2.1.0 following in July 2026. That is a long quiet period followed by two major-version releases in two months, which is consistent with a deliberate restructuring rather than incremental drift. The practical consequence is that adopting LIBXSMM 2.x for a convolutional model means adopting at least two repositories and tracking their compatibility, not one. The README does not state a version-compatibility policy between the core library and LIBXSMM-DNN, so that is something to establish yourself before pinning versions.

Where LIBXSMM is the wrong tool

Two cases stand out. The first is large GEMM. If your matrices are in the hundreds or thousands per dimension and you call GEMM a handful of times, the JIT specialization is overhead you pay without return, and a tuned vendor BLAS is the natural choice. LIBXSMM's own framing is small operations, and nothing in the README suggests it competes at the large end. The second case is workloads with unstable shapes. JIT specialization assumes you reuse a shape often enough to amortize code generation. If every call has different dimensions, you generate a new kernel every time and the dispatch cost never gets paid back. The README does not describe a caching policy, a cache size limit, or what happens when the number of distinct shapes grows, so if your workload has high shape cardinality you should measure that before assuming the design holds up. A third consideration is that LIBXSMM is a C library with a Fortran-facing example, not a framework. It gives you primitives; composing them into a training loop, a data pipeline or a model format is your work. The README's own move of operators into LIBXSMM-DNN is an acknowledgement that the core library is not the place for that composition.

Compared with oneDNN and vendor BLAS

The closest comparison in the Intel ecosystem is oneDNN, which also provides primitives for deep learning and also dispatches to optimized kernels per architecture. The difference in approach is where specialization happens. oneDNN selects from precompiled, hand-tuned kernels and primitives for its supported configurations, with a defined set of supported shapes and datatypes. LIBXSMM generates code at runtime for the exact shape you request, which is why it can cover the long tail of small, odd dimensions that a fixed kernel table would miss, and why it can support the wide low-precision datatype list in the README including MX formats and int1. The trade is predictability: a precompiled kernel library has a known cost per call and a known set of supported configurations, while a JIT has a first-call cost and a code-generation path that you should verify on your target. Against vendor BLAS the difference is simpler. BLAS gives you a stable, widely available interface tuned for large problems. LIBXSMM gives you a dispatch API, a JIT backend, and an explicit TPP abstraction, at the cost of a library-specific interface and a build step. If your code is already written against the BLAS interface, LIBXSMM is a rewrite of the call sites, not a drop-in replacement.

Licence and the cost of tracking two repositories

LIBXSMM is BSD-3-Clause, which is permissive and imposes no copyleft obligation on code that links against it. The README does not discuss the licences of the companion repositories, so if you pull in LIBXSMM-DNN or the TPP PyTorch Extension, check each one separately rather than assuming they match. On maintenance: the core library has a documented build path (make STATIC=0), a single public header, and sample drivers under samples/xgemm, which keeps the integration surface small. The upgrade risk is concentrated in the 1.17 to 2.x transition, where functionality moved out of the tree. If you are starting fresh on 2.x, you inherit the split from the beginning and there is no migration to plan. If you are on 1.17, the move of convolution drivers and deep learning operators means an upgrade is a re-integration effort, not a version bump. The README does not provide a migration guide for that transition, so plan on reading the companion repositories' documentation rather than expecting a compatibility shim. For new work, pinning to 2.1.0 and tracking the core library separately from LIBXSMM-DNN is the arrangement the project's own structure suggests.

Editorial conclusion

Adopt LIBXSMM if you are writing a C or Fortran kernel that repeatedly multiplies small matrices, or if you want a portable TPP layer that emits SSE through AMX code from one build. Do not adopt it expecting a complete deep learning runtime: convolutions, fully-connected layers, normalization and pooling now live in LIBXSMM-DNN, and the PyTorch path lives in a separate extension repository. Before committing, build with STATIC=0, compile the samples/hello example, and run it with LIBXSMM_VERBOSE=2 to see which kernels the JIT actually emits on your target CPU. If the verbose output shows your shapes falling back to a generic kernel rather than a specialized one, the JIT is not buying you anything on that workload.

Official sources

1. libxsmm/libxsmm on GitHub
2. License: BSD-3-Clause
3. Project website
4. README
5. Releases