Applications:

U.S. SCIENTIFIC COMPUTING ENTERS NEW ERA WITH Grid3

Scientists of the Open Science Grid Consortium, who met at a workshop at Harvard University on Sept. 9, announced the success of a nine-month trial operation of a U.S. data Grid for particle physics experiments and other scientific applications. The prototype Grid, called Grid3, uses the Internet to combine the computational resources of 26 universities and national laboratories across the United States to serve the computing needs of more than 10 research groups in particle physics, astrophysics, bioinformatics and computer science.

"This is a breakthrough for scientific Grid computing," said Paul Avery, professor of physics from the University of Florida, and director of the National Science Foundation's International Virtual Data Grid Laboratory. "Grid3 is a simple Grid, but it does what a computer Grid is supposed to do for scientific research. It combines computing power from multiple sources and transports data to and from offsite locations so that individual scientists can solve scientific problems at their desktop computers."

Grids are becoming critical to data storage and analysis in a range of data-intensive sciences. For example, when they begin operating in 2007, the world's largest-ever particle physics experiments—the ATLAS and CMS experiments at the Large Hadron Collider, a particle accelerator now under construction at CERN, the European Particle Physics Laboratory in Geneva, Switzerland—will depend on Grid technologies for storage, transport and analysis of unprecedented volumes of data by collaborators at universities and laboratories across the globe. Each LHC experiment numbers over 2,000 collaborators.

To ensure that ATLAS and CMS can meet their computational needs when scientific operations begin, the experiments must meet a series of "data challenges," preoperational exercises of computational and data capacity tested with simulations of successively larger percentages of the actual data production and analysis that will occur at LHC experiments.

"For US-ATLAS and US-CMS, the United States teams in the LHC collaborations, Grid3 has been key to meeting the milestones of our data challenges," said Boston University physicist James Shank, executive program manager for US-ATLAS computing. "Grid3 has run stably and with little effort during its nine months of operation. Grid3 has also provided unique opportunities for collaboration between the experiments through cooperative use of each others' computing resources. Both CMS and ATLAS have altered the way they allocate CPUs, for example, in order to accommodate each others' priorities and schedules."

While Grid3's simple functionality has supported the experiments' data challenges thus far, scientists said it must be significantly enhanced to meet actual analysis needs for LHC data-taking when the experiments begin operations in 2007. Toward that goal, U.S. researchers and computer scientists have formed the Open Science Grid Consortium to develop Grid3 into a production infrastructure that will operate at a larger scale with a broader base of partners and resources from more organizations, offering more sophisticated services.

"We have a lot of work to do to take Grid3 to the scale and capabilities of the Open Science Grid," said Ruth Pordes, associate head of the Computing Division at the Department of Energy's Fermilab and coordinator of the Department of Energy's Particle Physics Data Grid project. "But the success of Grid3 and the enthusiasm for Open Science Grid give us confidence that we can make it work. US-ATLAS and US-CMS have committed their ongoing computing programs to the support of the Open Science Grid, not only to benefit their own experiments but to promote the open use of a shared Grid infrastructure by the broader science community."

The core technologies for Grid3 as well as for the European Grids are based on the Virtual Data Toolkit, which includes the NSF middleware initiative software distributions.

"Achieving a common middleware base across the Grids is the result of a lot of good collaboration and hard work," said computer science professor Miron Livny, leader of the Condor Project at the University of Wisconsin.

The Open Science Grid will "federate" with other Grids now under development around the world to create a truly global data Grid for science. For ATLAS and CMS, the Open Science Grid will "interoperate" with the LHC Computing Grid in Europe, and others, to provide a global Grid for LHC experimenters around the world.

"The demonstrated interoperability of Grid3 and the European LHC Computing Grid for the movement of data and distribution of computation gives us growing confidence in the applicability and usability of our infrastructure for the future of global Grid computing," said computer scientist Ian Baird, head of Grid deployment at CERN, where rapid development of Grid capability is also underway.

University of Chicago physicist Rob Gardner, an ATLAS collaborator, cited the cooperation among Grid3, the LHC Computing Grid and NorduGrid, a Scandinavian-developed research Grid.

"The ATLAS Collaboration has used the combined resources of the international LHC Computing Grid, the NorduGrid Project and Grid3 for its data challenge simulations," said Gardner, who, with Pordes, serves as Grid3 coordinator. "It is especially gratifying that the groups developing the various Grid infrastructures have cooperated so that our global experiment community can run science jobs efficiently across the continents."

Besides the LHC particle physics experiments, Grid3 has also served computational needs of other researchers. Astrophysicists from the Sloan Digital Sky Survey, a project to create the world's most comprehensive map of the sky, used Grid3 to perform data analysis. Collaborators of the proposed BTeV experiment at Fermilab also used Grid3 for particle event simulations. Experimenters of LIGO, the Laser Interferometer Gravitational Wave Observatory, performed an analysis searching for continuous gravitational wave signals. Experimenters performed two biology applications, a biomolecular analysis and a protein sequence analysis, on Grid3. Computer scientists have also used Grid3 for research projects.

Computer scientist Ian Foster, of the Department of Energy's Argonne National Laboratory and the University of Chicago, is leader of the Globus Alliance, an organization devoted to the development of common worldwide standards and open source software for global Grid computing.

"We are especially pleased that scientists from a number of different fields use Grid3 and show that many areas of research can share the same computing resources. The mix of applications from experimental science and computer science research running on Grid3 shows that our underlying Grid protocols are increasingly robust and flexible," Foster said.

To use Grid3, a scientist must belong to one of the member organizations that provide computing capacity. Each user registers through an authentication system to receive a "Grid certificate," which works like a passport to identify users. US-CMS scientist Rob Harris of Fermilab said he had used Grid3 for particle event simulations.

"Grid3 provides us convenient access to the intensive computing required for our complex detector simulations," Harris said.

Each Grid3 member site presents a common interface to users.

"The use of common standards in Grid development is an important factor in determining future compatibility and interoperability among the world's data Grids," said Vicky White, head of Fermilab's Computing Division. "Grid3 shares software and infrastructure with other national Grid infrastructures in the U.S. and Europe. The Open Science Grid will continue to work toward common protocols that will allow truly ubiquitous access to the data as well as the computers."

Some 30 scientists from member universities and laboratories collaborated on the development of Grid3. The National Science Foundation, the Department of Energy's Office of Science and the member universities provided funding for the project.

More information about Grid3 is available at www.ivdgl.org/Grid2003/.

Grid3 Member Organizations include: