 |
|
DAILY NEWS AND INFORMATION
FOR THE GLOBAL GRID COMMUNITY /
|
Special Features:
10 APPLICATIONS POWERED BY GRID
COMPUTING
By Brooklin J. Gore, Senior Fellow, Micron Technology
Inc
This report along with two other new, groundbreaking Grid reports from IDC and
the Economic Strategy Institute will be released at Gt'04 and included with
conference registration. www.gridtoday.com/04/conference/index.html.
This is the second installment of a three-part series derived from the report
"Enterprise Grids For General Purpose Computing" by Brooklin J. Gore.
Web services, utility computing, .NET, CPU harvesting and distributed
computing are just a few of the technologies that fall under the Grid
computing umbrella. Gt04 -- a premiere enterprise Grid computing conference
targeting industrial and commercial users -- will gather experts, and outline
strategies and road maps for Grid deployment. For more information, visit
www.gt04.com.
Grid computing is here!
Ten General Purpose Grid Applications
The first five applications discussed are truly general purpose -- common
functions that virtually every organization performs in some form or another.
The last five may be a little more specific to manufacturing organizations but
have wide applicability in that sector. The most important aspect about all of
these examples is that none of them was designed from the ground up to run on
the Grid, only slight modifications were required. More information about what
it takes to Grid-enable an application is provided in the Challenges section
below. The examples below could be implemented on a Condor Enterprise Grid
with a few hundred machines.
1. Data management. So, you found the backup tapes with that critical
reliability data from six years ago, but now you can't find anyone that knows
anything about the programs to read the data. Why not just convert the data?
Given a requirement to convert over 10 million data files from a proprietary
format to XML and since 10 million of anything takes a long time, the Grid
provides sufficient resources for the conversion. At 0.5 seconds per file,
approximately 5 million seconds or two months of processing is required. By
running 200 conversion jobs (each job just grabs a file and converts it) on
the Grid, one can complete the entire conversion in just under 8 hours. Data
manipulation and conversion is perfectly suited for Grid Computing because
most of the overall wait time is for disks or databases where improved
processor speed doesn't improve run time -- but improved processor count has a
dramatic impact.
2. Trend charting and reporting. A form of data manipulation appears in trend
charting and reporting for management, engineering and/or production decision
support systems. As above, these are perfect jobs for the Grid. A large
organization can easily be faced with generating hundreds of thousands of
trend charts every day. Each job consists of gathering requirements for a
given chart type (trend, scatter, box, etc.), acquiring the data from a
combination of databases and files and rendering the chart using a graphing
tool like GNUplot. While chart rendering can be a CPU intensive process, most
of the time is spent waiting for data and it is this fact that makes charting
and reporting a great Grid application.
3. Network management. One ambitious and innovative Grid applications didn't
come from a software engineer but a network engineer. No slam intended on the
network guys -- this illustrates the point that you don't need a Ph.D. in
parallel or high performance computing to take advantage of the Grid to solve
a wide range of problems. Many organizations have a need to collect network
device configurations for rapid restores in the event of a device failure or
misconfiguration. Collecting thousands of global configurations can easily
take well over a day using traditional methods. By leveraging the Grid,
completion time can be reduced to less than two hours thanks. This application
is a great candidate for platform independence -- it easily runs on Linux,
Solaris and Windows taking maximum advantage of Grid resources.
4. Security. Having a better than a 95 percent share of the operating system
market certainly has its advantages in consistency, etc. but it also has its
dark side as witnessed by the flurry of critical Microsoft security patches
issued of late. Many companies are just wrapping up their second pass of
patching tens of thousands of systems worldwide. Part of a company's ongoing
vulnerability analysis can leverage the Grid to periodically scan their entire
address space for known problems and target identified systems for upgrades.
Knowing your weakness is half the battle of a strong security posture and the
Grid can quickly help identify them before someone else does.
5. Software development, verification and deployment. Large software systems
can take considerable time to build. The Grid enables parallel compilation
followed by the linking step after all compiles succeed. This reduces overall
build time to the longest compile + link time. Source code repositories can
also be quickly checked for consistency using the Grid. The Microsoft System
Installer (MSI) is a standard tool for desktop software distribution and
installation. However, the MSI packaging process can be somewhat time
consuming, especially when building over 1000 software packages in preparation
for new deployments. Using the Grid to run MSI builds in parallel can
dramatically reduce the time to deploy. Some groups additionally leverage the
Grid for regression testing to ensure version compatibility and this practice
is expected to grow.
6. Process control. As product specifications require tighter manufacturing
tolerances, real-time process control is becoming the norm. The Grid is useful
for running complex mathematical models to predict optimal equipment settings
based on a set of equipment performance measurements. What makes this approach
interesting is the use of a commercial third party statistics package to
perform the analysis -- the very same tool that the process engineer and
statistician use to create the process models. One can leverage the licenses
purchased for desktops and labs to run the production process control jobs on
the Grid. In some cases, implementing such a process may also involve checking
with third party software suppliers before utilizing their software on the
Grid to ensure compliance with the relevant licenses.
7. Capacity optimization and scheduling. Most manufacturers are faced with the
problem of determining optimal schedules for product on machines that minimize
time-to-ship. The Grid is useful in optimizing the execution of expensive
scheduling software. This is an application where CPU performance is critical
and where Grid software can find the fastest available machine that also meets
all other jobs requirements. One may wish to dedicate a couple of machines to
this task and the Grid software nicely load balances the work across these
machines. If one or more of the preferred machines is down, however, the Grid
software quickly finds the next best machine for the job. In these cases, the
Grid automates machine performance and availability concerns that
traditionally burden development and support staffs.
8. Design automation. Some applications, such as design automation can be a
tough nut to crack for Grid Computing. One reason for this is the high cost of
commercial design automation tools, with $100,000 licenses not uncommon.
Therefore, while the Grid can offer hundreds of compute resources for a tool,
few companies can afford the licensing costs this implies. Where this
situation exists, Grid computing may not be the preferred solution. However,
where more reasonably priced licenses are available, this barrier to
implementation does not exist. Additionally, in some applications, the
benefits of Grid computing may provide an impetus to develop "homegrown"
design automation tools to realize both direct savings in license fees and
improved computational efficiency. There are opportunities today for realizing
Grid computing advantages with design automation tools that are developed in
house.
9. Engineering analysis. Engineers are the kings of ad hoc computing. They are
continually looking to optimize processes, understand failures and
characterize behavior. To achieve these objectives engineers often run
repetitive, compute intensive processes. There are significant opportunities
to dramatically speed up engineering analysis via the Grid. Many of these
programs take the form of Perl or (Unix) shell programs that are very easily
modified to run on the Grid. So easy, even an engineer can do it.
10. Defect analysis. As with process control, defect analysis plays a growing
role in quality assurance as product complexity grows and manufacturing
tolerances decrease. Defect analysis is a problem begging for Grid Computing.
Why? Because computational requirements are guaranteed to grow with volume and
as product complexity warrants more intensive and therefore longer running
analysis. Experience proves this fact. Before moving to the Grid, defect
analysis software engineers were constantly juggling and purchasing compute
resources. After Grid-enabling core defect analysis routines, these worries
are a thing of the past. More time is now available for software engineers to
focus on higher priority tasks.
About Brooklin J. Gore
Brooklin Gore has been researching and implementing enterprise Grid
technologies for the past three years to create Micron's global Grid
infrastructure which runs over 15 production applications today. Brooklin has
been with Micron for 16 years. In that time he served as a product engineer,
Computer Aided Design group manager, network manager and General Manager of
Micron's Internet Services Division. Brooklin has been issued several US
patents and is a Senior Member of the IEEE. He holds Bachelor of Science
degrees in Computer Science and Electrical Engineering from the University of
Idaho and a Masters of Science in Computer Science from the National
Technological University.
|