Latest News

 

Lightening the load for advanced computing users

Published on October 2, 2019 by Aaron Dubrow



Lmod: An environment module system developed at TACC.

Supercomputers can present complex environments with lots of choices. Researchers hoping to launch a job must pick the right version of a compiler, application and library to get the right collection of tools to compute, and must make sure they do so every time to assure reproducibility.

Environment modules make this process simpler by providing a convenient way to dynamically choose or change a users' environment, and providing easy paths to the right collection of tools needed to run a job.

An example module file layout.

The Lmod system, developed at TACC, provides environment variables that specify where the library and header files can be found. Its convenience and ease of use has made it a favorite of high-performance computing administrators and users around the world, with instances found in such far-flung locations as Belgium and Australia.

The lead developer of Lmod, Robert McLay, is always on the lookout for ways to make the tool simpler to use. He recently noted that, with so many versions available of many common compilers and libraries, Lmod needed a simpler way for users to identify the version they wanted.

"I was thinking about how a site might manage some sort of short cuts and I realized that there was a way to modify Lmod to support this automatically by creating an extended default mechanism," McLay said. "Essentially, Lmod takes the version that you have given and creates a pattern, then picks the best modulefiles that match the pattern."

On April 23, he coded up a change to Lmod that enabled researchers to either automatically choose the latest version of a package by not stipulating the version at all; or to select the latest version of a given year's package by just specifying the year.

He sent out a message to the community of Lmod users:

"Lmod 8.0.7 now allows users to do:

$ module load intel/17 # -> would load intel/17.0.3
$ module load intel/17.0 # -> would load intel/17.0.3
$ module load intel/18 # -> would load intel/18.0.4
$ module load intel # -> would load intel/18.0.4
$ module load intel/1 # fails to load anything!"
The response was immediate and positive and enthusiastic.

Administrators at Stanford University deployed it for their users the very next day. In a blog post, they wrote: "Starting now, you can simply load a module using its major version number, and you'll get the best module that matches your request… We hope this will make things easier..."

Snippet of Lmod code related to recent change in version selection.

Ryan Novosielski, senior technologist at the Rutgers University Office of Advanced Research Computing, wrote McLay to say: "Thank you very much — this is exactly what I was hoping for/hoped maybe Lmod already supported and I just didn't know about it. Seems like a very logical way for this to work."

At TACC, Stampede 2, Lonestar5 and Frontera all now use the latest version of Lmod which include this feature.

The change to Lmod, and its response, show how TACC has become a leader in creating the software tools that make using supercomputers more intuitive, and in building a community that can rapidly apply these tools to the broader community.

"If you asked the 1,000 active users on our system, I would say 90% of them don't even know that we have a particular version of modules, or even that modules are there," McLay said. "It shows up and it just works. And I think that's great."


Contact

Faith Singer-Villalobos

Communications Manager
faith@tacc.utexas.edu | 512-232-5771

Aaron Dubrow

Science And Technology Writer
aarondubrow@tacc.utexas.edu

Jorge Salazar

Technical Writer/Editor
jorge@tacc.utexas.edu | 512-475-9411