Latest News


Cracking Open the Proton

Published on June 5, 2020 by Jorge Salazar

Physicists are unlocking the secrets of the subatomic proton, using instruments on the COMPASS experiment of CERN such as UIUC-built drift chamber DC5 shown here. TACC's Frontera supercomputer will help analyze COMPASS data and guide design for future experimental upgrades.

Physicists around the world are cracking open the proton, within the nucleus of the atom, to see what's inside.

The proton is a fundamental building block of the atomic nucleus, and among other things it's used as a medical probe in magnetic resonance imaging. It also has a rich inner structure made up of subatomic particles called quarks and gluons, which bind the quarks together.

Scientists are running a unique experiment involving the world's largest particle physics laboratory and the world's fastest university supercomputer to see and understand the dynamic world inside the proton.

About 240 physicists in 12 countries and 24 institutions collaborate on the COMPASS experiment — short for Common Muon and Proton Apparatus for Structure and Spectroscopy — at CERN, the European Organization for Nuclear Research. They explore the proton structure there by breaking it apart in particle collisions using particle beams of the CERN North Area's Super Proton Synchrotron and a spin-polarized fixed target.

The smashed insides of the proton are invisible to the naked eye and require large detectors, which record and digitize information about the particle and store it in a special data format. To interpret the data, physicists process it using complex algorithms.

"The spatial pattern and the velocities of the fragmenting particles allow us to create a dynamic picture of the proton and other objects composed of quarks," said Caroline Riedl, a research assistant professor of nuclear physics at the University of Illinois at Urbana-Champaign (UIUC). With her UIUC group, Riedl is involved in the COMPASS polarized Drell-Yan program and was the COMPASS technical coordinator for the 2018 run.

A particle collision event in the COMPASS apparatus displaying the trajectories of charged and neutral particles reconstructed from hits in the detectors. A beam of high-energetic (190 GeV) charged particles (pions) impinges (coming from the lower left corner) on a fixed target of transversely spin-polarized protons. The red dots indicate hits in the various COMPASS tracking detectors registered in coincident mode. The true hit information is determined by looking up information in alignment and calibration data bases. A tracking algorithm tries to find the best possible fit of these about 200 hits and reconstructs the trajectory of each charged particle. From the bending radius in the field of two strong dipole magnets, the momentum of each charged particle is determined, while other detectors allow to identify different particle species.
Her team previously used the Blue Waters supercomputer at the National Center for Supercomputing Applications to process the many petabytes of COMPASS data. She's extending her research onto the Frontera system at the Texas Advanced Computing Center (TACC), the fifth most powerful and the fastest university supercomputer in the world.

Frontera will boost the analysis of the existing COMPASS data taken between 2015 and 2018. Analyzing the COMPASS data collected between 2015 and 2018, her team together with the collaborating COMPASS colleagues was able to confirm for the first time the theoretically expected sign change of the Sivers function in Drell-Yan scattering as compared to Deep-Inelastic Scattering.

This so-called Sivers TMD ("Transverse-Momentum Dependent") distribution arises from correlations between proton spin and quark transverse momentum and thus appears connected to quark orbital motion inside the proton. Observation of the sign change of the Sivers TMD is one of few NSAC (Nuclear Science Advisory Committee) performance milestones for DOE- and NSF-funded research in nuclear physics.

The COMPASS experiment shoots a beam of pions (particles made of quarks) onto a fixed target. The aftermath is chronicled by 240 tracking planes that follow the path of the subatomic particles released. Here's where the computational challenges get heavy.

"The procedure of finding particle tracks emerging from the interaction point and traversing hundreds of COMPASS detector layers is CPU-intensive," Riedl said. The tracking procedure is one of the first steps in the data analysis. An additional highly CPU-expensive task is the sampling of about two percent of the data to determine efficiencies for the detector planes, according to Riedl.

Providing the data in a timely manner for physics analysis presents a hurdle.

"The challenge consists of parallelizing the submissions of the tracking code on the computing grid while respecting the system in terms of I/O and numbers of requested computing nodes. A typical production campaign requires about 50,000, ideally parallel, submissions of the tracking code," Riedl said.

All in all, about three petabytes of COMPASS data has been moved from Blue Waters to TACC's Ranch storage management system, which enables it to be analyzed on Frontera.

Workflow of COMPASS experimental data production. The raw COMPASS data were transferred earlier from CERN to NCSA's Blue Waters and bundled in tar archives. In 2019, the tar archives were transferred from BW to TACC's Ranch. Some portion of the COMPASS data remains to be transferred from CERN to TACC. When there is a production request, the data are retrieved from tape, untarred and submitted to the Frontera grid in individual production campaigns, which in average require 50,000 CPUs for 50,000 tracking jobs. While 56 CPUs are available on one Frontera node, 1 CPU is used to launch the calibration data base and the remaining 55 CPUs are used for tracking. The alignment / geometry and calibration data bases are compiled at CERN and are remotely accessible at Frontera. Riedl uses two production systems: ESCALADE, which requires a higher level of human intervention, and recently also the more automated PanDA, which was developed for the large LHC experiments at CERN. The reconstructed output data for usage by physics analyzers are stored in the format of Data Summary Tables (DSTs) and are transferred back to CERN for analysis by dozens of international COMPASS collaborators.
In addition to analyzing past COMPASS data, her team is using Frontera to design new detectors for the future COMPASS++ /AMBER experiment. This new facility at the M2 beam line of the CERN Super Proton Synchrotron will enable a great variety of measurements to address fundamental issues of quantum chromodynamics.

The proposed program covers measurements of the proton charge radius using beams of muons, elementary particles similar to the electron but with much greater mass; the spectroscopy of mesons and baryons by using dedicated meson beams; the study of meson and baryon structure via the Drell-Yan process; and eventually the fundamental quest on the emergence of hadronic mass.

Riedl is driven by fundamental questions at the heart of the proton. How do the quarks move inside the proton, and what is their orbital motion? How are quarks distributed in the proton? And how do quarks and gluons generate the large observed nuclear masses?

The latter question will be accessed by the future COMPASS++/AMBER experiment at CERN, according to Riedl.

"We run mass productions of COMPASS data on Frontera, determine detector efficiencies, and simulate COMPASS and COMPASS++/AMBER data. The simulated data play a central role in understanding subtle detector effects and complement the experimental data," Riedl said. "Frontera will allow us to analyze the COMPASS data in a timely manner and at the precision required to obtain an absolute normalization of the data with the smallest possible uncertainties."

Riedl hopes that improved analysis on Frontera will allow researchers to reach discoveries inside the proton faster than ever.

Caroline Riedl, Research Assistant Professor of Nuclear Physics, University of Illinois at Urbana-Champaign.
"Only Frontera will allow for the detailed simulations necessary to optimize instrumentation upgrades for the future COMPASS++/AMBER experiment," she added. "Frontera is a leading-edge supercomputing system funded by the National Science Foundation that will allow U.S researchers to compete with international research teams."

Riedl's research fits into the bigger picture of understanding nuclear physics and quantum chromodynamics, the field theory of the strong nuclear force. She investigates questions such as how quarks and gluons form the nuclei of matter, and how protons can be described in terms of Parton Distribution Functions, "partons" referring more generally to quarks and gluons.

"The special feature of our experiments lies in the usage of spin-polarized particle beams on spin-polarized fixed targets," Riedl said. "By introducing transverse quark momenta, spin, and orbital angular momenta into the formalism, proton substructure becomes similarly rich as the substructure of the hydrogen atom, which was first described in the 1930's," she added. "During the early decades of the 21st century, proton hyperfine structure has moved into the focus of spin physicists."

Unrelenting curiosity drives her work.

"Humans are and always have been curious to find out what holds the world together at its core," Riedl said. "We attempt to unravel the origin of the mass of objects in our daily life and to map the dynamic quark structure of the proton."

This research is supported by the NSF Office of Advanced Cyberinfrastructure Award # 1713684 : Mapping Proton Quark Structure using Petabytes of COMPASS Data

Story Highlights

Analysis of CERN COMPASS data reveals the dynamic picture of the subatomic proton.

NSF-funded Frontera system will boost analysis initiated by Blue Waters of NCSA.

Prior COMPASS findings confirmed for the first time theoretically expected sign change of the Sivers function in Drell-Yan scattering as compared to Deep-Inelastic Scattering.

Frontera is helping scientists design new detectors for the future COMPASS++ /AMBER experiment.


Faith Singer-Villalobos

Communications Manager | 512-232-5771

Aaron Dubrow

Science And Technology Writer

Jorge Salazar

Technical Writer/Editor | 512-475-9411