Smooth Sailing for Simulations of Roughness

TACC and TeraGrid assist University of Puerto Rico researcher in search for turbulence insights

When it comes to natural beauty and abundance from the sea, Puerto Rico lives up to its name: "Rich Port." For researchers in Puerto Rico who depend on high performance computing (HPC), the National Science Foundation's TeraGrid is their richest port.

One veteran Puerto Rico-based TeraGrid researcher is Stefano Leonardi, assistant professor in the department of mechanical engineering at the University of Puerto Rico at Mayaguez. Originally from Rome, Leonardi has worked and used HPC resources around the world.

"The Teragrid allows my research group to carry out complicated numerical simulations that run for months," Leonardi explained. "We could not perform such simulations on in-house clusters of computers."

The group's fluid dynamics research explores how roughness on the surface of materials affects the drag and transport of air, especially around building structures and turbine blades in aeronautical engines. Their current work is clarifying basic questions in fluid mechanics, like how much of an effect roughness has on flow, and whether distance from the surface changes these effects. The work also addresses some specific, and socially relevant questions, such as how the density of the buildings influences the drag in an atmospheric boundary layer.

"By improving the knowledge of fluid mechanics in urban canopies or atmospheric boundary layers, we can improve models to calculate pollution dispersion or predict how a contaminant can be diffused in case of terroristic attack," he said.

Stream traces in a turbulent channel with V-shaped turbulators on the bottom wall.

These kinds of simulations are challenging, in large part, because of their non-linear, multiscale nature. One needs to solve complex mathematical equations on a very large scale to take into account very large vortices; at the same time, it is necessary to preserve very small resolution, so as to describe the small vortices near the wall. To achieve both simultaneously, in a reasonable timeframe, requires massive supercomputers.

Leonardi uses several TeraGrid systems to accomplish these numerical tasks, specifically , Lonestar, Spur, and Longhorn, at the Texas Advanced Computing Center (TACC). These advanced computing systems perform huge calculations and visualizations that generate vast amounts of data — about 10-15 terabytes per simulation.

This itself leads to a common cyberinfrastructure challenge. To transfer a 10-15 terabyte database from Puerto Rico to Texas and back would take too long. Therefore, Leonardi performs his computational tasks remotely, taking advantage of TACC's connected resources to complete the entire workflow. Simulations on Ranger, TACC's massive parallel computing system, are passed to Longhorn for visualization and data analysis. The associated data is stored locally in Austin, Texas on TACC's Ranch system, a 10-petabyte data repository.

"With our in-house cluster, I often have to spend time changing broken disks or RAM. With TeraGrid, I can rely on specialists who keep their cluster running so I can focus on my research," said Leonardi.

The group's findings have led to several papers in prestigious fluid mechanics journals, including a recent publication in the Journal of Fluid Mechanics. The insights from the research have the potential to improve the efficiency of wind and gas turbine engines, and enhance the quantity of the energy generated, while decreasing pollution.

There are many researchers who live and work in U.S. states and regions with limited access to local HPC systems. For these researchers, the TeraGrid and its resources are a vital link to the world of advanced computational study, a scientific lifeline across the sea.

"TACC is proud to be a member of the NSF TeraGrid team that enables researchers from all over the world access to these unique data, visualization, and compute resources," said Chris Hempel, associate director of resources and services at TACC. "Research teams can now investigate much larger problems than what is possible using the resources at their home institutions."

---

A High-Performance Education

The Leonardi research group. (From Left to right: John Lucena, Ricardo Bonilla, Stefano Leonardi, Luis Tossas, Amabel Reyes, Benjamin Cruz.)

Benjamin Cruz Perez, a graduate student at the University of Puerto Rico at Mayaguez and one of Dr. Leonardi's apprentices, is a big TeraGrid fan too.

"I used TeraGrid resources to complete my master's thesis. Without TeraGrid's tools and training, I would not have been able to obtain results in time for my defense," he said.

Perez has been beneficiary to several NSF and TeraGrid programs, including a course on Parallel Computing taught by staff at TACC, and a summer school in Italy that was co-sponsored by TeraGrid and its European counterpart, DEISA (Distributed European Infrastructure for Supercomputing Applications).

"Thanks to these experiences, I have met others with similar interests who also use supercomputing for their research. I'm now comfortable using the techniques I've learned for the efficient use of HPC, and look forward to participating in multidisciplinary collaborations in the future," he added.

March 23, 2011

---

The Texas Advanced Computing Center (TACC) at The University of Texas at Austin is one of the leading centers of computational excellence in the United States. The center's mission is to enable discoveries that advance science and society through the application of advanced computing technologies. To fulfill this mission, TACC identifies, evaluates, deploys, and supports powerful computing, visualization, and storage systems and software. TACC's staff experts help researchers and educators use these technologies effectively, and conduct research and development to make these technologies more powerful, more reliable, and easier to use. TACC staff also help encourage, educate, and train the next generation of researchers, empowering them to make discoveries that change the world.