Lmod: environment modules rebuilt on Lua, still reading Tcl modulefiles

A Lua core that still speaks Tcl

Lmod is a program to manage the user environment under Unix, Linux and Mac OS X among others, and it is described as a new implementation of environment modules.

The interesting technical decision is in the repository description: it is an environment module system based on Lua that reads TCL modules and supports a software hierarchy. The TcL interpreter is embedded rather than dropped, which is what allows an existing module tree to keep working while the implementation underneath is replaced.

That matters because environment module trees on shared clusters are enormous and slow to migrate. A Tcl modulefile in a site collection is usually a set of setenv commands, and a Lua modulefile is a set of calls that do the same thing, and the site is inevitably full of both. Supporting the older format means the migration is incremental.

The 8.0 release notes make the embedding explicit: embed the TCL interpreter in Lmod when a site allows TCL files. The wording about sites is doing work there, since a site that does not want Tcl evaluation of arbitrary files has a reason not to enable it.

The project is hosted by TACC, the Texas Advanced Computing Center, and the README lists several sites rather than one. The documentation is at lmod.readthedocs.org, there is a SourceForge mirror at lmod.sf.net, a TACC project page, a separate Lmod Test Suite repository, and a mailing list at [email protected].

Software hierarchy, module collections and the spider cache

The feature that distinguishes Lmod from a plain environment module system is the software hierarchy, and it is what makes a modern module tree manageable.

The software hierarchy lets a site organise modules into a tree, so the module name encodes the path, as in intel/17 for a compiler version under a vendor directory. The 8.1 release added the Extended Default feature, where module load intel/17 finds the best intel/17.* and so on, so users do not have to know the exact patch version. The 8.4 release improved the documentation for the software hierarchy and for community module collections.

Underneath, the implementation crawls the tree and caches the result. The spider is the process that walks the module directories, and there are hooks around it. The 8.6 release added the spiderPathFilter hook so that sites can control which directories are ignored, which matters when a site has large or slow network filesystems in the module path. The 8.5 release made the spider skip .version* and .modulerc* files.

The 9.0 release list includes a related change: the collection is only written to the ~/.config/lmod directory, so the spider cache lives in the user's own configuration rather than in a system location. That is a change in where state lives, and it follows the general direction of the project, which is to push site-specific behaviour into configuration rather than into code.

The configuration file for the whole system is /etc/lmod/lmod_config.lua, introduced in 8.6, which is where a site sets behaviour that applies to all users. There is also the LMOD_QUARANTINE_VARS environment variable, a colon separated list of names that Lmod will not change, which is a mechanism for protecting variables a site does not want modules to touch.

What the 9.0 release changed, and the performance note

The current major series is 9.0, and its feature list is short enough to read in full.

   1. Irreversible mode: Unloading a module can set env. vars.

   2. Improved optional module tracking.

   3. More powerful hide{} and forbid{} functions to hide and forbid modules.

   4. The function depends_on_any() added.

   5. Collection only written to ~/.config/lmod directory.

   6. Great performance improvements when loading a module with many dependencies.

Two of those deserve attention. The first is irreversible mode, where unloading a module can set environment variables. That is unusual, and it exists because some modulefiles need to leave state behind, for example to undo a variable that the module set as a side effect. The cost is that the unload is not a clean reversal, and a site that enables it has to accept that.

The fourth is depends_on_any(), which sits alongside the existing depends_on and between() functions. The 8.3 release added atleast() and between() support for a less-than comparison as distinct from less-than-or-equal, and the 8.7 release repaired support for depends_on(between()).

The sixth item is the one a site administrator will care about most. Loading a module with many dependencies is the case that hurts on a busy cluster, because each dependency is a separate evaluation and a separate environment change. The 8.7 release also made module unload never fail, with module eval errors becoming warnings during unload, and the 8.2 release added a clearLmod command that does a module purge and removes all Lmod aliases and environment variables.

The hide{} and forbid{} functions are a permissions mechanism. A site can hide a module from the module avail listing without preventing its load, or forbid it outright, and the 9.0 release made both more powerful, which is a natural progression as sites move from listing policy towards enforcement.

What the older releases say about where the design came from

Reading the 8.x feature lists in order shows what the project was trying to fix, and the themes are consistent.

Two releases are about output correctness rather than new capability. The 8.2 release fixed a case where Lmod output only fills when the text is more than one line or it is wider than the current width, and the 8.7 release delayed puts stdout in Tcl modules until the end of the module evaluation to match Tmod 4.1, while puts prestdout is printed at the beginning of the output to match Tmod 5.1. Both of those are compatibility decisions with a specific sibling implementation, which tells you the project is used alongside other module systems and the differences were user visible.

The 8.5 and 8.6 releases are where the user-facing commands arrived. A new module overview command lists names and the number of versions for each, and a new module function source_sh() sources a shell script inside a modulefile, with the sh_to_modulefile script updated to capture shell functions and shell aliases. The 8.5 release added a userInGroups(group1, group2, ...) function to check whether a user is in one of those groups, and a requireFullName() function that generates an error if the full name of a module is not specified. That last one is a policy tool: it lets a site require that users type the complete module name rather than relying on the default extension.

The 8.4 release is the portability work, adding support for Lua 5.4, improved support for ksh and ksh scripts, improved support for the fish shell, and handling of /bin/dash startup. The 8.3 release added support for a less-than comparison in the atleast() and between() functions, and the 8.2 release brought better fish shell support including tab completion, and the clearLmod command that does a module purge and removes all Lmod aliases and environment variables.

One release is about the module search itself. The 8.2 release fixed the removal of the absolute path prompt for generating the spider tables, so it is now only used when building the reverseMapT, and Lmod now requires rx other access when searching for modulefiles, which is a filesystem permission requirement rather than a code change.

Where the source of truth lives, and how the release flow works

The source management policy is stated plainly in the README, and it is a two tier arrangement that explains the release numbers.

The most up-to-date source is at GitHub. All known bugs have been fixed if it is released on GitHub. When there has been sufficient improvement or important bugfixes there is a new release at SourceForge.

So GitHub is where the work happens and SourceForge is where the tagged releases land, which is a SourceForge convention for a project that started there. The consequence for a user is that a bug fix appears on GitHub before it appears in a tagged release, and that the release tag is the point at which the maintainers consider the tree settled.

The recent releases are 9.4.2 on 2026-09-09, 9.4.3 on 2026-10-05 and 9.4.5 on 2026-10-13, and the last push was on 2026-10-13. The numbering is worth understanding because the changelog in the README is organised by major version while the tags are patch versions, so the 9.4.x series picks up the 9.0 features with fixes.

The repository layout shows a project with a long history. There is a src/ directory, a tcl/ directory and an init/ directory for the initialisation scripts, a shells/ directory and an sh_src/ directory, a settarg/ directory, a spec/ directory, a doc/ directory and a my_docs/ directory, a cookbook/ directory, a contrib/ directory, a docker/ directory, a debian/ directory for packaging, an embed/ directory, a messageDir/ for translated messages, a bugReport/ directory, and a proj_mgmt/ directory for project management notes. There are also several README files, README.design.md, README.old, README.new and ML_README.txt, which is what a twenty year old project looks like when the documentation has to serve several audiences.

The build is autotools based, with configure, configure.ac, aclocal.m4 and Makefile.in, and the licence is a custom one recorded in the License file, which the repository metadata flags as something GitHub cannot classify.

Editorial conclusion

Lmod suits an HPC site that already has a large body of Tcl modulefiles and wants the loader itself rewritten in Lua, since the interpreter can read both and the 9.0 performance work targets exactly the many-dependency case that hurts on a busy cluster. It does not suit a site that has decided to standardise on the original Environment Modules implementation and has no Lua expertise on staff. Before you commit, read README.design.md for the design rationale, check the 9.0 change list for the irreversible mode, since unloading a module that sets environment variables cannot be undone, and note that the licence is a custom one recorded in the License file rather than a standard OSI licence.

Frequently asked questions

Official sources

1. Issues
2. Project website
3. README
4. Releases
5. TACC/Lmod on GitHub