Special Features:
THE GRID IS IN THE AIR: AN
INTERVIEW WITH SDSC'S FRAN BERMAN
By Neil Alger, Special Correspondent,
HPCwire
At SC02 this year, the word "grid" was thick in the air. Conversations in
nearly every booth found their way into discussions of the grid in one context
or another. GRIDtoday correspondent Neil Alger caught up with SDSC and NPACI's
Fran Berman at the tail end of the show to discuss grid's role in this year's
conference, and to talk about what we can all expect from the grid in the near
future.
HPCwire: Here at the tail end of SC02, what aspects of the show most excite
you in respect to the future of the grid?
Fran Berman: One of the most exciting things about the show was Rita
Colwell's presentation. It was a tremendous vote of confidence to have the
Director of NSF present a vision of cyberinfrastructure and her plans for the
future. Dr. Colwell expressed very well the state of the art and the need for
long-term persistent infrastructure to enable the next generation of science
and technology advances.
This is a very exciting time because we understand what the problems are,
we understand the linkages that are required, we have 10 years of experience
in grid computing, and we are starting to build infrastructure that will be
persistent, useful and usable for the next generation. We now have a wealth of
experience from which to go forward and really start addressing some of the
most challenging problems in cyberinfrastructure. I'm really excited about
that.
If you look around the exhibition floor, every year it is more exciting
than the last, the word 'grid' is everywhere. The grid has provided an
overarching concept to link together the last two decades of technology
advances in computing hardware, storage, networks, scientific instruments,
visualization, etc.
There's also a recognition this year that building the grid and
cyberinfrastructure does not mean that there shouldn't be an investment in
high-end facilities. It's not an either/or proposition; high-end facilities
are a very important part of the grid.
The grid provides access; it provides ways to link high-end facilities with
data, visualization, and other resources. It provides a way to link small
resources with larger resources. The grid includes a wide spectrum of
resources and this is really clear from the exhibition floor.
HPCwire: What do you see as the big hurdles right now that need to be
overcome in order to propel grid computing to the next level?
FB: The biggest hurdle for the grid right now is that there is a lot of
really hard work to do. There has been considerable underestimation of the
level of difficulty of the problems that one must address in order to deploy
the most sophisticated vision of the grid. For example, in order for us to
deploy a grid that is ubiquitous, adaptive, and promotes application
performance, there are many research problems that must be solved; problems
having to do with policy, performance modeling, adaptive computing, etc.
For example, how do we compile for the grid, how do we schedule for the
GRID, how do we debug grid programs, what's fault tolerance on the grid,
what's security on the grid? At this point in time, it's important to
coordinate a strong research effort focused on some of the most challenging
problems involved in developing the grid with the activities of integrating,
designing and deploying grid infrastructure.
One of the things we're trying to do in NPACI is really to look at enabling
grid applications by developing robust, usable and widely deployed grid
infrastructure. At the end of the day, what the applications folks really want
is for the underlying infrastructure to be completely mundane.
They don't want to have to be heroes to use the software, they want it to
be easy to use, to be reliable, to be robust, and to evolve with the
technology, and I think it's really important for us as a technology community
to provide that. The NPACI partnership is prototyping this kind of robust
software and end-to-end solution that links applications and resources.
The model NPACI is following is the community grid model that has emerged
over the last decade. This is a layered model. The bottom layer consists of
everyone's hardware resources. By definition, these are heterogeneous, because
people buy what they need, they buy the best hardware they can buy at the
time, and not everyone buys the same thing.
Above the resources is a layer of common infrastructure. At this point,
people are really coming together through NSF's middleware initiative NMI,
through the Global Grid Forum, through OGSA, etc., to put in place a set of
evolutionary protocols and interfaces so people can start to build on
something that doesn't change all the time.
At the next layer, over the common infrastructure, NPACI is developing the
NPACKage, which is our ready-for-primetime software. This involves a number of
software efforts that NPACI has been developing over the years including the
Network Weather Service, Globus, APST, DataCutter, GridPort, etc.
All of this software must interoperate, and needs to be tested, documented
and robustified to be of maximal use to the scientists. This year, NPACI is
expending considerable effort on NPACKage and other NPACI software so that it
will be available for the NPACI alpha projects and other applications to
utilize.
The deployment of common infrastructure on NPACI resources and the
utilization of NPACKage and NPACI software by applications will give the
partnership considerable experience with end-to-end grid computing, exactly
what is needed as a foundation for Dr. Colwell's vision of
cyberinfrastructure.
HPCwire: What are the stumbling blocks right now in terms of funding that
you see?
FB: As you know, the NSF blue ribbon panel on cyberinfrastructure has been
meeting for the last year. There is considerable expectation that the report's
recommendations will precipitate a new program at NSF with new funding for
cyberinfrastructure. The focus of that program will be of great interest to
the entire community.
Based on the preliminary report and the common themes that came out of the
testimony of an outstanding array of almost 100 scientific leaders, we have
some notion of the themes underlying the Blue Ribbon Panel's recommendations:
the grid is a fundamental platform; data is a critical focus for the future;
the TeraGrid project and PACI resource sites have had a great impact on
science and technology and should form the foundation of a national
cyberinfrastructure; partnership and coordination is important; applications
must be drivers, etc.
These themes are actually quite consistent with the PACI program mission.
NSF's job will be to determine how they want to act on the recommendations in
the upcoming cyberinfrastructure report. It's clear that there is tremendous
excitement and discussion going on at NSF around the concept of
cyberinfrastructure.
I believe that cyberinfrastructure got a vote of confidence from Rita
Colwell during her SC02 presentation and that there is real commitment at the
highest levels at NSF to expand our leadership in cyberinfrastructure.
HPCwire: One of the things that I've heard a lot, as I've been talking to
people here on the floor, is the perception that there is a need for
applications almost to catch up, now, with all of the hardware and
infrastructure that does currently exist. There's this idea that applications
are the dog that's being tugged by the leash. What do you think that people
can expect in the near future on the applications level?
FB: I disagree with that assessment. I think that what we're experiencing
is a sort of multidisciplinary bootstrapping. Technology provides a platform
that enables application scientists to achieve their next set of results, and
science advances in turn motivate new advances in technology. You always have
a set of pioneers that are out there using the cutting edge of technology to
provide the cutting edge of science.
Mark Ellisman is a great example of this with his work on the BIRN project
and Telescience. The GEON researchers are developing infrastructure to support
a generation of new advances in the Geosciences. Phil Bourne, Helen Berman and
the Protein Data Bank group are another great example of a group that has been
extremely successful using cutting edge database technology to enable an
entire community.
Bourne and other researchers at SDSC are now working on the Encyclopedia of
Life project, an extremely exciting project to provide data-minable and Web-
accessible function and structure information for all publicly available
genomes.
Developing the Encyclopedia of Life is at the same time compute intensive,
data intensive, and grid intensive. All of these application groups have
really pushed the technology.
HPCwire: It's interesting that you mentioned this idea that even as many
people focus on grid computing, there is still this definite need for the high
power, heavy compute, monolithic machine.
FB: Absolutely.
HPCwire: That's come up in a number of conversations that I've had here on
the floor, even with the large vendors, who are really trying to push grid as
part of their portfolios this year.
FB: It's not an "us against them" type of issue, it's more 'use the right
tool for the right job. Big supercomputers are always going to be the right
tool for certain codes. Other codes don't need the big machines. SETI at home
sustains about the same number of TeraFLOPS on two-million-plus computers
connected to the Internet as applications on the Japanese Earth Simulator.
It has a different structure and doesn't need a monolithic supercomputer to
achieve results. However the codes run on the Japanese Earth Simulator are
certainly not appropriate to be run on two million PCs; they need a machine of
the Earth Simulator's size and power.
We expect our infrastructure to support a lot of different activities and
we want to have resources on the grid which are well suited to the things
scientists want to do.
HPCwire: It's interesting that that's taking a turn, at least that's the
perception of a lot of the people who I've been speaking with. That's really
one of the major things they've picked up on this year at SC02, this idea that
people are accepting the fact that there are going to be tremendous points of
convergence as opposed to "this camp and that camp." There really has been an
emphasis on the collaboration that's happening now and that needs to continue
to happen within the grid community, and even more in the external
community.
FB: "Use the right tool for the right job" is really the right approach in
the sense that just because you have a hammer doesn't mean that you don't need
a screwdriver, you just use them for different things.
The high-end resources are critical resources for the grid. The grid should
provide an overarching umbrella for us to get to and use resources in a better
way. But it doesn't replace the need for those resources.
HPCwire: Is there anything that you would like to add?
FB: In many ways it is the best of times for applications researchers,
technologists, and for the broader community. The vision of
cyberinfrastructure is very powerful and fundamental to the way we have begun
to use technology in our society.
These days at SDSC, there is a lot of energy and excitement around the
challenges of building the world's most complex and useful systems. SDSC is
re-inventing the concept of supercomputing to embrace modern technology and to
help define cyberinfrastructure.
There are all kinds of new and exciting initiatives at the center, led by a
great group of energetic and outstanding professionals, partners and
collaborators. This is a wonderful time for SDSC and an important time for
science.
|
