DAILY NEWS AND INFORMATION FOR THE GLOBAL GRID COMMUNITY / SEPTEMBER 1, 2003; VOL. 2 NO. 35    ( Table of Contents )

Special Features:

GRID COMPUTING AND ITS FUTURE IMPACT ON TECHNOLOGY

According to some industry experts, Grid computing is the next evolution of enterprise technology architectures, and will furthermore revolutionize how businesses think about and manage their computing resources. More so than client/server or Internet computing, the technology will be more of a revolution in computing architectures and could spark a revival of the entire technology industry.

While client/server and Internet computing centered on usability and end-user benefits, they hardly reduced the complexity of creating and managing enterprise computing resources -- and often complicated matters further.

On the other hand, Grid computing offers a layer of separation between the end-user and the underlying computing resources, resulting in a more flexible and more efficient architectural layer. The increasingly efficient use of existing resources could spark an information technology revival, simplifying ways to cost-justify individual application projects while maximising investment in IT.

At least that is the idea. At this early adopter stage of Grid deployment, examples of customers achieving these sort of returns are hard to come by, and most Grid projects appear to be limited to educational and research establishments. The ultimate Grid deployment -- the global, or external Grid -- whereby multiple organisations have access to a shared resource of computing technologies around the globe, is still a long way off, but smaller departmental and enterprise Grids have been deployed to good effect.

An example of the potential for Grid computing comes from North America's National Digital Mammography Archive, which runs on the University of Pennsylvania Grid that connects hospitals at the Universities of Pennsylvania, Chicago and North Carolina and the Sunnybrook and Women's College Hospital in Toronto.

Based on IBM Unix, Linux and Windows eServers running the company's DB2 Universal database management system, the University of Pennsylvania Grid enables the capture of healthcare files from any location on the Grid -- including patient medical images, records and clinical history.

The National Digital Mammography Archive has enabled access to current and past patient records in between two and 90 seconds, enabling faster diagnosis and reducing overhead costs associated with transferring paper and film records between hospitals and offices.

The Grid architecture also enables all hospitals to take advantage of shared processing power to run sophisticated algorithms that enable them to identify disease patterns, as well as analytical tools to aid the diagnosis of diseases. The Grid also supports educational tools for radiologist training and computer-aided diagnosis.

Examples such as these indicate the potential for Grid computing, at least in environments where cooperation is advantageous, the systems and software involved are proprietary, and where the upfront funding for a project is forthcoming. Funding for the University of Pennsylvania Grid came in the form of a number of grants from the National Science Foundation, the National Library of Medicine, the National Institute of Health and Next Generation Internet.

Few commercial companies have organisations such as these willing to fund the development of Grid computing architectures, but that does not mean that Grid computing has to be restricted to educational and research establishments. Sometimes there is cross-over between the commercial and research organisations, as with the example of the White Rose Consortium.

A collaboration between the Universities of York, Sheffield and Leeds, the White Rose Consortium has invested over £3m in a Grid infrastructure to support its research and development across university departments.

As well as fostering interest in Grid technology among the universities, one of the five main objectives of the White Rose Grid (WRG) is also to identify, develop and sustain commercial interest in Grid technology, according to the WRG's business development manager, Philip Morris.

"One of the things we're working on, funded by Yorkshire Forward [a project to regenerate the region's economy], is to provide business benefit for the region," he says. "Talking to groups about Grid technology and possibly providing the architecture to support local businesses. There's a vast interest in it from SMEs to vast multinationals."

Helping to foster interest in the White Rose project is Rolls Royce, which is using the Grid as part of its Distributed Aircraft Maintenance Environment (DAME) project that provides real-time diagnostics of the data from 100,000 of its operational aircraft engines. DAME is part of the UK's eScience initiative to advance Grid computing.

"Each time there's a flight 1GB of information is generated from engine sensors," explains Morris. "This data is wasted because there's so much of it they can't handle the sheer amount that they have. They approached the universities to dump the data from each flight on to the Grid to manage it and identify potential problems through pattern matching."

The project collects engine data from flights and runs it through distributed diagnostics applications running on the WRG, which is based on Sun V880 and 6800 servers running in Leeds, York and Sheffield and a Beowulf cluster of Intel servers at Leeds, all running Sun's Grid Engine Enterprise software and the Globus Toolkit.

A total of 424 processors and 18TB of Sun T3 storage capacity are involved over the three sites, of which 75 percent of the resources are reserved for internal academic use, and 25 percent open for external commercial use.

Despite the high level of interest from local businesses in Grid computing, fuelled by Rolls Royce's proof of concept project, Morris admits that there are some technical and cultural issues to be overcome before many commercial organisations will tap into independent compute resources.

"There's a whole range of issues and security is certainly one of them," he says, "but I am confident that the technology will achieve this. The real difficulty is on the business side of things." As well as the obvious security concerns, many organisations, it seems, have to overcome issues about resource ownership, while commercial factors such as billing and charging strategies are still to be worked out.

As Morris suggests, while there are some related technical concerns to be overcome, the focus at the moment is on convincing users of the business benefits. "We are probably focusing more on the business side than the technology side at the moment," he says.

The Ever-Expanding Grid

It is generally agreed that there are three stages of Grid computing, ranging from initial internal deployments to global Grids that link multiple organisations around the globe. At this stage, however, the vendors involved in Grid computing have not agreed exactly what those three stages are. For all intents and purposes, therefore, there are four types of Grid:

• Departmental Grids -- Considered by some to be a sub-sector of enterprise Grids, and indeed by others as not technically Grid computing at all, departmental Grids have also been referred to as cluster Grids and see the clustering of computing resources within an individual department to ensure maximum uptime and utilisation of available resources. This is something that many organisations have achieved with workload management and scheduling technologies without considering that they may be rolling out the essence of a Grid architecture.
• Enterprise Grids -- The first true stage of Grid deployment, enterprise Grids involve distributed applications sharing resources across multiple enterprise departments and locations within a single organisation. Employing many of the same workload management and scheduling technologies of departmental Grids, enterprise Grids add distributed computing functionality that enables the sharing of resources across physical and geographical boundaries.
• Partner Grids -- Not identified as part of the three-stage process of Grid adoption by all vendors, partner Grids are an extension of enterprise Grids that enable collaboration between partner organisations. Typically deployed among research organisations the potential for commercial adoption may be limited by fears over providing partners with access to compute resources and data. However, there are potential advantages for resource sharing among supply chain partners, or for ad hoc partner Grids to facilitate collaboration projects between companies.
• Global Grids -- The global Grid may or may not happen, depending on whose definition of what it means you listen to. Certainly the idea of a single Grid of compute resources spanning the globe, into which users dip via the web to make use of the available processing power, is unlikely to arrive any time soon. But the development of global utility Grids by service providers, from which small business and individuals may rent computing power, is a much more realistic target.

While many research and academic establishments are adopting Grid technology, so far it is not as prevalent in commercial environments. There are commercial early adopters of Grid technology, however, particularly in the financial services arena. Back in January financial services giant JP Morgan Chase announced that it had selected BladeFrame servers from Egenera to form the heart of its new Grid computing project, called Compute Backbone.

The project is designed to combine available processing power to serve the company's trading and risk management applications based on Platform's financial services focused Symphony enterprise Grid solution.

Another financial services firm investing in Grid computing is brokerage house Charles Schwab, which has set up a Grid that lashes together xSeries servers running Linux, a version of the AIX LoadLeveler Grid scheduling program that IBM Research has ported to Linux and a task scheduling program that IBM Research calls Tags that rides on top of Globus.

The Grid has allowed a financial services application that runs on this cluster of machines in Charles Schwab's San Francisco data centre to do its work more efficiently and therefore cut down on the processing time for this application from four minutes to 15 seconds, which, in the financial services world, can be the difference between making some money and making a lot of money.

While it is often the case with new technologies that early adopters are to be found at research establishments and in the financial services sector, there are also some indications that there are other factors holding back the adoption of Grid computing among enterprises.

Evidence of this comes from microprocessor vendor ARM, which has a strategy aim of Grid-enabling its major engineering data centres to increase efficiency of resource utilisation and reduce development costs.

"I'm a bit confused as to whether we've actually done that," admits the company's engineering IT manager, Ibrahim Chadirichi. "We've deployed the cluster and load balancing technologies, and we'd love to do real Grid computing for our engineering work."

While the company has deployed Platform's LSF software to cluster and load balance its mixture of Unix and Linux servers, it has been hampered in its plans to Grid enable the engineering infrastructure. According to Chadirichi, this is due to a number of reasons.

"The first and major barrier for ARM is the cost of the EDA tools, to use them in a global way we would have to pay up to 50 percent more," he says. "The second issue is data management. With an engineering job you have to ensure the data is close to the CPU, so you have a central data repository in a cluster with network attached storage.

"With Grid you cannot guarantee the resource has access to all the data it needs at a high enough rate," he continues. "You can replicate and distribute the data for the tools, but the job information changes all the time." Chadirichi says that there needs to be more auto-replication and data management functionality to enable data to be moved quickly across the Grid.

The third barrier, according to Chadirichi, is more cultural, and is due to departments and individuals fearing that they are losing control of their computing resources. While the third issue can be fixed by ARM with internal education about Grid computing, the other two factors are currently out of its hands.

While Grid computing technologies are in the early stages, and are still maturing, Chadirichi believes that vendors could be doing more to solve the data management problems. "We're not getting much help from the industry," he says. "Companies like [design collaboration and management software vendor] Synchronicity do it, and we do have a thread with working groups to try and provide a way forward."

There is hope, however. "Data management is the area likely to get the most movement," he adds. "There are start-ups emerging, although a lot of them are in the storage area and things like [IBM Rational] ClearCase and software configuration management products. The two need to come together."

While more functional data management technologies are in development, a longer-term problem is the issue of software licensing. "People utilising Grids often have their own software, so they don't face the software licensing problem," says Chadirichi. "I think they [ISVs] need to have a rethink. At the moment they see it as a way of getting more money out of people."

Fundamental to the problem is the way in which ISVs license their software by the number of processors it runs on. In a Grid environment the number of processors the software needs to be licensed for could theoretically run into the hundreds or thousands, for which the licensing costs are obviously prohibitive.

"We're at the behest of the EDA tool vendors," says Chadirichi. "Even adapting for clusters has given us some problems. They look upon this sometimes as a threat to them selling licenses. It's the tools vendors that have to come to the table if Grid technology is to survive in the EDA market."

This is an issue that is affecting not only EDA tool vendors but all enterprise software vendors, who will have to rethink the way in which they license software to ensure that it is practical and cost-effective for users to deploy it on Grid-enabled architectures.

Chadirichi relates the situation to the previous argument between node-locked licenses and floating licenses. "In the end you don't see too many node-locked licenses these days," he comments.

Despite the barriers to adoption, ARM is convinced that Grid computing is the future of its engineering architecture, enabling the most efficient use of existing resources. Unfortunately, for the short-term future at least, the industry is not moving fast enough to enable the company to follow through on its plans, however.

"In the short-term we are really unable to make use of this," says Chadirichi. "People are using the Grid as a marketing term, the Grid is definitely out there, but at the moment the licensing and data management issues are a real barrier to us."

Although it is early days for enterprise Grid computing it is clear that there are both advantages to be exploited and challenges to be faced. The frustration suffered by ARM is not limited to the design automation software space, and software vendors in all markets need to reflect on what changes they need to make to their licensing schemes to enable Grid computing. Issues of data management will depend on the requirements of the end user, but will be fulfilled as Grid technologies mature. The scale of Grid computing projects may be too much for smaller companies, but as projects such as the White Rose Grid evolve there will be opportunities for all companies to take advantage of Grid environments.