Scientists access ORNL's supercomputer from all over: New York, a plane, even Hardee's

Frontier, the world's fastest supercomputer at Oak Ridge National Laboratory, could not fit into your pocket. It is made of 74 cabinets, each of which weighs 8,000 pounds.

But that doesn't mean it can't be used from a lab thousands of miles away or even from a scientist's favorite fast-food restaurant.

“I logged onto Frontier from an airplane,” said Bronson Messer, an astrophysicist and the director of science at the Oak Ridge Leadership Computing Facility. “I’ve logged onto a lot of world No. 1 supercomputers from random Hardee’s in the middle of nowhere in the Southeast, so doing it from an airplane I find utterly unsurprising.”

One of the wonders of supercomputers, in addition to their lightning-fast speed and cavernous memory, is that they can be accessed from just about anywhere to solve generational scientific problems.

Stephan Priebe, a senior engineer at GE Aerospace Research, is using Frontier to see how a new generation of jet engine might work in flight.

But Priebe has never seen Frontier in person. He's never even visited ORNL.

Priebe accesses Frontier from GE Global Research in upstate New York. Other researchers have logged on from Los Alamos National Laboratory, Stanford University, Germany, France and Australia.

Frontier is housed at the Oak Ridge Leadership Computing Facility, one of nine open-user facilities at ORNL that allow thousands of researchers across the world to use the lab's technology to solve complex scientific problems.

The Department of Energy grants scientists time on the supercomputer if their research is relevant, urgent and requires unprecedented computing power. In exchange for using the system free of charge, researchers must share their results with the public through scientific journals or conferences.

Priebe's research fit the bill. After running aerospace simulations on Summit, Frontier's predecessor at ORNL and currently the fifth-fastest supercomputer in the world, his research on a new kind of jet engine architecture called open fan required a faster computer.

“These particular simulations that we’re doing are only possible on Frontier,” Priebe said. “What it allows us to do is to get this detailed view of the aerodynamics and the acoustics in the design phase, which is new and unique."

When it debuted last year, Frontier broke what's called the exascale barrier, meaning it performed more than one quintillion calculations in a single second. It remains the only system in the world to have done so, though others will soon follow.

Scientists have been logging onto the machine to run giant codes at full scale, which could help unlock discoveries about the human body, climate-friendly power and the origins of the universe.

How do scientists access Frontier?

Most scientists who use Frontier are not based at ORNL, so they must log onto the supercomputer remotely.

The lab provides researchers with a token that changes login credentials every 30 seconds to ensure the system's security. Some users have software that allows them to access the system from within a web browser, and some use a program called a Linux terminal.

Messer, who has worked in high-performance computing since the 1990s, admitted the terminal used to access Frontier looks like something out of an old-school movie.

“It remains the most supple and useful way to quickly do the things you need to do, and that’s why it survives," he said.

Priebe described the series of steps users around the world take to get their code running on Frontier.

"You essentially have a terminal where you can input text-based commands," Priebe said. "You would start by inputting commands to log onto the machine, and then once you're on the machine, you would input commands to get your simulation ready, and then to submit your simulation to run on Frontier."

Frontier has more than 9,400 nodes that contain its processors, and researchers are given time in allotments of node hours, or hours spent on the machine per node. If Priebe spent one hour using 5,000 nodes, for example, he would use 5,000 node hours. His team was awarded 1.4 million node hours by the Department of Energy.

When users submit their simulations to run on Frontier, they are added to a queue with other researchers, and the lab must solve the puzzle of running enough codes at every moment to keep the system busy. Some codes, like Priebe's, may require the entire system at once, while others only require a fraction.

“Scheduling all the jobs on the machine is like doing a glorified version of Tetris,” Messer said. “Our Tetris is frankly a little bit harder than some other big computing centers because we are dedicated to these very, very largest of problems.”

Messer has been at ORNL for the debut of four supercomputers, each of which was the fastest in the world when it came online.

For him, the biggest nightmare would be if any part of Frontier went unused. He said the lab usually does not cut researchers off completely from the machine, even if they are out of node hours, because the mission is to reach scientific goals.

That said, the lab does keep meticulous track of users' progress and requires their home institutions to sign a user agreement and to submit quarterly and closeout reports, said Katie Bethea, user access, outreach and communications group leader at the Oak Ridge Leadership Computing Facility.

“There’s a lot of bureaucratic steps along the way to make sure they’re doing what they say they’re doing and that they’re using the machine,” Bethea said. "They have to make smart choices."

What does Frontier allow researchers to do?

It's no exaggeration to say Frontier has made the impossible possible across scientific fields. A key word for understanding its power is "resolution."

In the digital age, nearly everyone has an intuitive sense of resolution as the level of detail in an image or video. In computational science, resolution also deals with detail. Because of its speed and memory, Frontier gives scientists more detailed simulations than were possible on other systems, down to tiny details and across sometimes galactic scales.

Frontier is not only capable of more than a quintillion calculations a second, but each of its nodes has 32 times the memory of a standard laptop. In total, its system has 9.6 petabytes, or 9.6 million gigabytes, of memory. A file system attached to Frontier provides around 700 petabytes of data storage.

For Priebe, the GE researcher, the supercomputer is able to show how an open fan jet engine would function at full-flight scale. Rather than breaking the simulation up into smaller pieces, his team uses Frontier to get a sense of how the engine would actually work in flight.

As the name suggests, open fan engines look quite different from today's jet engines. In open fan designs, uncovered fans improve the engine's propulsive efficiency.

Open fan engines could improve the fuel efficiency and carbon dioxide emissions of planes by 20%, a revolutionary improvement. The engines are being developed by CFM International, a joint venture of GE Aerospace and Safran Aircraft Engines, and could be in use by the mid-2030s.

For astrophysicists like Messer, Frontier is solving overlapping physics problems about how the universe was formed, including the mechanisms behind the explosion and collapse of stars.

One enduring problem for physicists and engineers to solve is turbulence, which impacts aircraft stability, weather and astrophysical phenomena. Only an exascale system is capable of fully simulating the complex interactions that cause an airplane to dip or tilt in flight, or move pollutants through the atmosphere.

“To really understand turbulence, you need to be able to divide what you’re looking at into very, very small volumes to bring out detail," Messer said. "What happens is energy gets put into a system at some big scale and then it cascades down to smaller and smaller and smaller scales until molecules rub up against one another."

It takes teams of scientists years to develop a code big enough to make full use of Frontier's processors. The Oak Ridge Leadership Computing Facility helped prepare Frontier users through "hackathons" and the Center for Accelerated Application Readiness, or CAAR.

Before Frontier opened to users earlier this year, eight research teams were selected to use the supercomputer with technical support from the lab and the companies that built its processors. Their work, ranging in scale from atoms to an entire galaxy, helped scientists ready the system for future research.

Expanding the next generation of supercomputers

Two more exascale supercomputers are on the way at national labs within the Department of Energy.

Aurora at Argonne National Laboratory in Illinois was fully installed in June and El Capitan at Lawrence Livermore National Laboratory in California is under development. Aurora contains more than 10,000 nodes and has a theoretical peak performance of more than two quintillion calculations a second. It could soon outperform Frontier on the Top500 list, which tracks the world's fastest supercomputers.

For scientists working at ORNL, it's not the bragging rights of first place that matter, but the scientific knowledge the machines unlock. More exascale systems means more knowledge and better solutions to problems like climate change.

Messer thinks there will be no problem finding researchers to use the expanded suite of supercomputers.

“One of the beauties of truly cutting-edge, high-performance computing, especially at the exascale, is that we can attack scientific problems over the entirety of human curiosity," Messer said. "I have anxiety about a lot of things about the future of technology. I don’t have any anxiety about the demand.”

Already, the lab's next supercomputer is in the works, which could continue the trend of its last four systems, all of which debuted at No. 1 worldwide and helped reimagine the possible. Though the lab cannot predict exactly which year the next supercomputer will be delivered, its original estimate is around 2026 or 2027.

Daniel Dassow is a reporting intern focusing on trending and business news. Phone 423-637-0878. Email daniel.dassow@knoxnews.com.

Support strong local journalism by subscribing at knoxnews.com/subscribe

This article originally appeared on Knoxville News Sentinel: Oak Ridge National Laboratory supercomputer Frontier used worldwide