Scientists need sophisticated visualization tools to interpret simulations and observational data, and these depend upon the technologies of computer graphics. "We've known for some time that a significant driver of computer graphics technology is, amazingly, a nation of 14-year-old kids," says Dr. Kelly Gaither. "Computer games are today's fastest and most demanding graphics applications."

Gaither is Associate Director for Research and Development at the Texas Advanced Computing Center (TACC) at The University of Texas at Austin. She, Visualization and Data Analysis (VDA) manager Gregory S. Johnson, VDA visualization specialist Christopher A. Burns, and two colleagues in the university's Computer Sciences department are participants in a project funded by Microsoft Research. Their objective is to define the hardware architecture, computer algorithms, programming tools, and system interfaces for visualization systems of the future.

"The result will be better games," says project leader Dr. William R. Mark, assistant professor of computer sciences at the university, "and more importantly for us, using the same techniques, better scientific visualization and simulation." Joining Mark, Gaither, Johnson, and Burns in the $215,000 research project is Trammell Crow Regents Professor Donald S. Fussell of the Computer Sciences Department. In the past year, the group has obtained compelling research results, some of which will be published in October in the Association for Computing Machinery (ACM) journal Transactions on Graphics.

Supercomputer on a chip?

The four researchers have distinct interests, but they share an understanding of the possibilities that are inherent in the latest hardware developed for computer games: graphics processing units, or GPUs. "It is now becoming possible to build a highly parallel computer on a single chip," Mark explains. "Graphics processors are leading the transition to single-chip parallel processing in PCs, because graphics algorithms are highly parallelizable and consumer game software needs the improved performance that this parallelization can provide."

Mark, Johnson, and colleagues see the current GPUs as forerunners of chips that will have multiple processors sharing multiple levels of memory. "Today's GPUs are initial steps in this direction, but they are optimized for only a particular type of 3D graphics," Mark says. Mark led the design of Cg, a programming language for GPUs, at NVIDIA, before he joined the UT Austin faculty.

The GPU chips have some programmable units, the research team notes, but as a whole, they are still highly hard-wired for the classical algorithm used in real-time graphics, the Z-buffer rendering algorithm. This algorithm captures a three-dimensional scene in two dimensions (X and Y), keeping the depth information in a buffer (the Z-buffer) for use in determining visible surfaces and other distance-related effects as the X-Y picture is rendered. "Unfortunately," says Johnson, "the classical Z-buffer algorithm has fundamental limits in its ability to provide high-quality global illumination effects, such as smooth shadows and reflections. This has long been known to developers of games and scientific visualization tools."

To address these limits, the collaborators explore ways to incorporate more flexibility in the algorithms supported by GPUs, ultimately to incorporate the optically optimal technique of ray tracing. "We have worked on modified Z-buffer algorithms and other image-rendering schemes, and we have designed GPU chips optimized for these schemes, but our main objective is to enable the GPU of the future to support real-time ray tracing," Johnson says. "The high-end computational resources at TACC have been instrumental here, as has Don Fussell's expertise."

The ray-tracing method can compute indirect lighting effects accurately and efficiently instead of ignoring them or poorly approximating them as other methods do. "Realistic lighting is important in scientific visualization," Johnson says, "especially for problems like medical imaging or imaging of complex physical phenomena, such as turbulent flows, where rapid, three-dimensional rendering is needed." But ray-tracing algorithms perform too slowly on current GPU chip architectures, so the researchers are trying to evolve both algorithms and chip designs in the direction of greater accuracy without loss in speed or other aspects of high-end performance.

Shadows with substance

The Transactions on Graphics article, by Johnson, Burns, Mark, and computer sciences graduate student Juhyun Lee, explains a new algorithm, which the researchers call the irregular Z-buffer, and gives details of a GPU chip architecture designed to optimize the speed of the algorithm. By simulating the algorithm's performance with the new GPU design, the researchers were able to show that it could perform efficiently at interactive frame rates. The speed is a sine qua non for gaming, and it is indispensable for complex, interactive scientific visualizations, when scientists need to "steer" calculations in real time.

Games and science?

One of the software tools the researchers will continue to work with is Microsoft's DirectX, which is an advanced suite of multimedia application programming interfaces (APIs). "DirectX is the most commonly used API for graphics hardware," Mark says, "and we will investigate how it can be used and extended to support real-time ray-tracing applications."

One purpose of the Microsoft grant is to train students who will be able to enter the growing graphics industry. The grant explicitly supports a number of graduate student assistantships, and three students are now spending part of their time on these project activities.

If you are tempted to laugh at the idea of computer games in the driver's seat for new technology--"Don't!" say the researchers. "Games are increasingly incorporating physics simulations similar to those used in scientific computation, not only for compute-intensive rendering but also for solving problems posed by the game situation or for giving a truly probabilistic cast to the choices players make. Scientific visualization can get a real boost from this convergence," Johnson says.

"TACC is becoming recognized for expertise in visualization," says Todd Needham, Program Manager at Microsoft Research. "Research in GPU-based computing efforts like this, together with next-generation chips and architectures, offers tantalizing possibilities for the future of high-performance visualization and graphics."

The advantage of the academic research approach, Needham notes, is that the participants can attack the future from a number of directions without being driven by the more immediate interests that apply for, say, game developers or commercial chip builders. "What we're investigating, using the high-end resources from TACC to simulate new GPU and CPU architectures, is how real-time three-dimensional graphics should be done several years from now," Mark says. "Realistic gaming will be one beneficiary, and another will be the scientific research community."