Preparing for Many-Core
TACC-Intel Highly Parallel Computing Symposium showcases development and early experiences with Intel MIC Architecture
Leading researchers, hardware and software engineers, and high performance computing specialists from around the country attended the TACC-Intel Highly Parallel Computing Symposium on April 10 and 11 at the Texas Advanced Computing Center (TACC) in Austin, Texas. The meeting showcased the experiences of researchers who had ported their scientific computing codes to Intel's Knights Ferry (KNF) software development platform, the prototype hardware and software development package for Intel's Many Integrated Core (MIC) architecture — as well as researchers working on the single-chip cloud (SCC).
Over 100 participants from all sectors of the science and technology community attended the symposium and more than 30 stayed on to attend a tutorial on "Preparing for Many Core Programming," taught by Kent Milfeld, research associate at TACC. By and large, participants expressed enthusiasm for the forthcoming chips based on their early experiences. The symposium followed a series of Intel-led discussions and training sessions held around the world over the last six months in anticipation of the release of Knights Corner, the first product to use the Intel MIC architecture.
Two long-time Intel engineers, James Reinders and Tim Mattson, anchored day one of the symposium. Both researchers had worked on ASCI Red, the first supercomputer to break a teraflop of sustained performance in 1996. In addition, Reinders had worked on Knights Corner, the first commercial supercomputing chip to achieve one teraflop, which debuted at the Supercomputing 2011 conference.
Leading researchers, hardware and software engineers, and high performance computing specialists attended the symposium, which showcased the experiences of researchers who had ported their scientific computing codes to Intel's Knights Ferry software development platform.
The invited speakers steered clear of discussions about the raw performance or architectural details of the Intel MIC chip itself, the ostensible topic of the workshop, but still being closely held by Intel. Instead, they focused on the goals and research processes that led to the development of Intel's MIC architecture, and the ecosystem of libraries, kernels and programming paradigms that Intel hopes will make its new coprocessors a long-term success in the HPC community.
In his talk, Reinders described the suite of tools and libraries that Intel has developed in anticipation of their manycore chips, as well as their efforts to improve the open standards that enable parallel programming. He expressed the oft-repeated goal, voiced by many Intel staffers, of making the Intel MIC coprocessor usable to researchers right out of the gate.
Reinders spoke at length about the toolkit developed by Intel called Threading Building Blocks, which assists in the implementation of threading processes, while providing capabilities for task scheduling, synchronization and memory allocation. A thread is the smallest unit of processing that can be scheduled by an operating system. On a multicore system, many threads can run at the same time with each core running a particular thread or task.
"If your program can scale, Intel Threading Building Blocks won't get in your way," Reinders said. "There's little penalty to using it."
He also noted Intel's acquisition and subsequent development of Intel Cilk Plus, a sister to Intel Thread Building Blocks for the C and C++ languages. According to Reinders, Intel Cilk Plus offers an easy, reliable way to improve the performance of programs on multicore processors.
Vectorization was a buzzword throughout the symposium. In parallel computing parlance it refers to software programs that are modified to perform the same operation many times simultaneously on a large number of operands. Vectorizing a code allows it to achieve significantly better performance, as much as four times as many operations per clock period on a CPU, and even more on a manycore chip like the future Knights Corner. Reinders was the first, but not the last of the Intel speakers to highlight the importance and value of vectorizing and threading one's code.
Intel researcher, Tim Mattson, the second invited speaker, works on parallel system design, parallel programming environments and application software that are sometimes "far off the roadmap." He pushed the date of the first teraflop chip back a few years to his time working on Intel's 80-core Teraflops Research Chip, a "flop monster" on which the company tested its manycore ideas and architectures.
More than anything related to Intel MIC architecture, Mattson's talk gave a compelling picture of the way Intel operates, and the concern, care and angst shared by its engineers as they create a new product line for the multi-billion dollar company. According to Mattson, the power problem looms largest in his group's research, overshadowing even performance gains.
To minimize energy consumption, Mattson and his colleagues conducted a component-by-component power analysis of the experimental chips on which they worked. On the 80-core chip, they found that 28 percent of the power was going into networking capabilities. After tweaking the design, energy consumption was down to 10 percent on the 48-core, version. These two development chips, along with Larrabee, shaped the theoretical basis for Intel's MIC architecture, and researchers from the three projects comprise the core Intel MIC team.
More than a dozen research groups at Intel are working on the manycore problem from a variety of angles. How many cores can you fit on die? What architecture should many-core chips have? Can such chips be manufactured at reasonable cost? How do you implement a clock infrastructure that can scale?
"It's the small things that will kill you," Mattson said.
He also spoke of future-looking ideas such as fine-grained management of the voltage, frequency and power consumption of compute cores that can be controlled by the programmer. Many challenges exist with such a solution, but Mattson's description of the project provided a glimpse of the kinds of ideas Intel is exploring in their experimental research labs.
"The architecture for many-core is still being determined," he said.
Along the road to debugging and developing software for its manycore offerings, Intel went in an unusual direction. Instead of hiring a large internal software development group, Intel partnered with hundreds of institutions worldwide — among them TACC, many U.S. national laboratories, and top research institutions in the U.S., Europe and Asia — to test-drive the prototype MIC equipment.
"We built the silicon and said, ‘Let's look to the academic community to do the research on it,'" Mattson explained in his talk.
Since late 2010, this elite group, the Many-Core Application Research Community (MARC), has been experimenting with the Single-Chip Cloud Computer (SCC) experimental processor, a 48-core ‘concept vehicle' created by Intel Labs as a platform for manycore software research. More recently, researchers have been experimenting with the Knights Ferry hardware as well. Through these initiatives, researchers have implemented solutions that Intel never would have thought of, according to Mattson. "The smartest people in the world don't necessarily work at your company," he admitted.
The two-day event pointed to the promise and challenge of implementing existing codes on Intel's new co-processor. Scalability plots, comparative studies, and anecdotal experiences provided an overall positive impression of the group's experiences to date. Most impressively, several groups reported independently that their applications ran right away on Knights Ferry with no or very minor code modifications.
This sentiment served as an apt segue to the paper and poster sections of the symposium, where early users presented results from code porting, testing and performance characterization.
The two-day event pointed to the promise and challenge of implementing existing codes on Intel's new coprocessor. No absolute performance results were offered, yet scalability plots, comparative studies, and anecdotal experiences provided an overall positive impression of the group's experiences to date. Most impressively, several groups reported independently that their applications ran right away on Knights Ferry with no or very minor code modifications.
Symposium papers and presentations are available online. A sampling of presentation briefs follows:
Robert Harkness: "Experiences with ENZO on the Intel MIC Co-Processor"
Harness (National Institute for Computational Sciences/Oak Ridge National Laboratory) is one of the creators of ENZO, a large cosmological community code used to study the early ages of the universe. Harkness estimated that it had taken 25 person-years to develop ENZO. The code can't be abandoned, he said, but no one will pay to rewrite it either. The question he posed was: how to move forward with a large, working science code?
Harkness tested the application on three KNF systems at NICS, which cover most of the choices in basic cluster configurations:
- An Intel development workstation with two Intel Xeon 5600 processors and 2 KNF cards.
- A Cray CX-1 system consisting of a single head node with two Intel Xeon 5600 processors
- And two compute nodes each containing two Intel Xeon 5600 processors plus one KNF card.
- An Appro cluster consisting of four compute nodes, each of which contains two Intel Xeon 5600 processors and two KNF cards, for a total of eight KNF cards.
He tested the Knights Ferry in both native and offload modes and presented relative scaling results for both.
Harkness said he was personally able to port his code to KNF in one week. In his paper he wrote: "With all these components in place for offload and native builds, the compilation of ENZO-R itself is straightforward. No ENZO-R source code modifications are required to build a functional ENZO-R binary in native mode although, clearly, some source code changes will be necessary to achieve full optimization on the Intel MIC co-processor."
After the experience, Harkness said he is "bullish on Intel MIC co-processors." However, he warned the chip community to not waste time on vendor-specific interfaces.
Karl W. Schulz: "Early Experiences Porting Scientific Applications to the Many Integrated Core (MIC) Platform"
Schulz is the director of the scientific applications group at TACC and also chief software architect for The Center for Predictive Engineering and COmputational Sciences (PECOS), a DOE-funded Center of Excellence at The University of Texas at Austin. The group was funded four years ago to develop new uncertainty quantification (UQ) methodologies and apply them to multiscale simulations of atmospheric reentry vehicles.
He and his collaborators tried several codes from their portfolio on Maverick, a Knights Ferry test system with eight nodes and one Knights Ferry per Westmere core, and also built 3rd party libraries to facilitate runs. He presented relative scaling results and anecdotal experiences from several application tests.
Included among these applications was the porting of a quantum chemistry application that had 1.4 million lines of Fortran code. They also tested the kernel of a Direct Numerical Simulations (DNS) code for turbulent fluid flow, a multi-threaded FFT kernel that exhibited good strong-scaling properties on the Knights Ferry. In the future, the group is targeting very large turbulent flow simulations (10 billion gridpoints) and proper utilization of the Intel MIC coprocessor will be crucial for achieving that goal.
Schulz, who won best paper at the symposium, gave the most comprehensive insights into the process that MARC members have been experiencing over the past few months. To the assembled application developers in the room he counseled that "moving forward, we must take vectorization more seriously." Overall, he remarked, "It was a positive experience for all of us."
Rane is one of the developers of the tools PerfExpert and MACPO, which are designed to analyze the performance and performance characteristics of multi-threaded applications on multicore chips. In his spirited talk he presented a process by which application developers can reliably determine the parts of a code that are good candidates for execution on Intel MIC co-processors. Guidance in this area is certainly needed. Focusing on suitable candidates for porting (either whole functions or loops nests) has the potential to not only greatly reduce the level of human effort, but also to increase the achievable speed-up.
In Rane's two-step approach, PerfExpert is used to identify the hot spots in an application. MACPO, currently under heavy development by Rane and others, analyzes data access patterns and TLB misses within the hot spots to eliminate those that exhibit irregular access strides, etc. Interestingly, the analysis is not done on the Intel MIC co-processor itself, but on the host, which enables users today, who do not have access to any Intel MIC platform yet, to prepare using this tool.
Three examples were presented as a proof of concept. Those code regions that were recommended by the tools did indeed perform best on the Knights Ferry Software Development Package (SDP).
Sreeram Potluri: "Intra-MIC MPI Communication using MVAPICH2: Early Experience"
Sreeram Potluri is a Ph.D. student at The Ohio State University who has been researching and developing the MVAPICH2 library for MPI on Infiniband and other communications platforms. Potluri describe his group's work implementing MVAPICH2 for communications between cores on the Knights Ferry platform.
Their initial experiments ported MVAPICH2 directly to the Intel MIC card with no changes. Subsequently, they developed two kinds of optimizations of the library. The first tuned its parameters to take maximal advantage of the memory hierarchy and cache structure on the Knights Ferry card. The second set of optimization extended the work done in the first to use Intel MIC-specific communications protocols to further improve the performance.
The results showed impressive performance gains relative to the baseline case for all varieties of communications operations that Potluri tested. These included direct, point-to-point communications between pairs of cores, and collective communications operations that gather information from many cores together or spread information from one core to all the others.
Future work will take advantage of protocols for communication between Intel MIC cards in the same host, between tasks on the Intel MIC card and tasks on the hosts, and between Intel MIC cards running in different host computers. Potluri received the award for best student paper at the symposium.
The day two invited talks focused more explicitly on the Intel MIC coprocessors, the programming models that will drive usage, and the market conditions that inspired Intel to develop the coprocessor.
Scott McMillan, a staff software engineer at Intel, spoke on "Programming Models for Intel Xeon processors and Intel MIC Architecture." His presentation highlighted an important point: that Xeon and Intel MIC co-processor will work together in most installations.
TACC's upcoming Stampede system, which will be the first large-scale deployment of the Intel MIC co-processors, has a two-petaflop base system built on the Intel Xeon E5 2600 processors (formerly codenamed Sandy Bridge). The Knights Corner coprocessors will add eight petaflops of expected peak performance. Many of the discussions about parallelism, threading and vectorization apply equally to Intel's multicore Xeon processor and to the Intel MIC architecture.
The combination of these two architectures lend themselves to a spectrum of programming models and mindsets, which McMillan outlined in his talk, from all multicore, to offload, to all manycore, to symmetric usage of both the Intel Xeon and the Intel MIC architectures. "Which to use depends on the shape and structure of one's problem," he said.
The keys to productive performance on the Intel MIC architecture are a matter of common sense, McMillan insisted, and could begin immediately. He offered a few expert tips:
- Pick the right model for your application.
- Vectorize your application (today).
- Parallelize your application (today).
- Go asynchronous to overlap computation and communication.
"Hybrid programming today prepares you for Intel MIC architecture," he concluded.
Last up was Nash Palaniswamy from Intel's Technical Computing Group, who offered some perspective on the strategy behind Intel's manycore architecture. For more than 15 years, Palaniswamy has been working in the accelerator and coprocessor space. Throughout that time, technologically-driven companies in the oil and gas, financial services and pharmaceutical industries, as well as academic and government researchers, have explored the use of accelerators to find answers quickly, driving strong research in this market.
Intel's development of manycore chips over several years shows a unique level of commitment to enhancing performance, Palaniswamy said. In 2006, Intel began its accelerator and coprocessor effort under the Intel QuickAssist Technology program, and in 2010 after a thorough analysis of 80 to 90 workloads from customers, they announced a roadmap for manycore. According to Palaniswamy's assessment, manycore processors are useful for 10 to15 percent of applications today, and this number will continue to grow as programmers increasingly think parallel.
In the road-map, "little cores" — smaller, low-power cores with reduced single thread performance that rely on a high degree of parallelism — co-exist and supplement "big cores" – higher-power, higher-performance cores. This change, Palaniswamy insisted, could be achieved in a way that maximizes code reuse while requiring minimal developer re-training.
"We want to make sure the software that runs today will run 10 years from now with minimal changes," Palaniswamy said. "Architecture, models and applications evolve. Revolution is unnecessary and unaffordable."
With scientists continuing to port, test and optimize their software for Knights Ferry, and ultimately Knights Corner, conversations about the Intel MIC architecture will be ongoing for many months to come.
May 3, 2012
The Texas Advanced Computing Center (TACC) at The University of Texas at Austin is one of the leading centers of computational excellence in the United States. The center's mission is to enable discoveries that advance science and society through the application of advanced computing technologies. To fulfill this mission, TACC identifies, evaluates, deploys, and supports powerful computing, visualization, and storage systems and software. TACC's staff experts help researchers and educators use these technologies effectively, and conduct research and development to make these technologies more powerful, more reliable, and easier to use. TACC staff also help encourage, educate, and train the next generation of researchers, empowering them to make discoveries that change the world.
- Nearly 100 technologists attended the TACC-Intel Highly Parallel Computing Symposium in Austin, Texas on April 10-11.
- The symposium included invited talks by Intel researchers and paper and poster presentations by computational scientists who had early access to Intel's Knight's Ferry and Single Cloud Chip (SCC) technologies.
- Based on the experience of the presenters, many scientific computing codes can be ported easily and rapidly to the Many Integrated Core (MIC) architecture. However, threading and vectorization will be crucial to achieve maximum code performance.
- "Stampede's" Comprehensive Capabilities to Bolster U.S. Open Science Computational Resources
- Intel's Many Integrated Core (MIC) Architecture
- TACC-Intel Highly Parallel Computing Symposium Program
Science and Technology Writer