Systems/Enterprise:
A TURBOWORX INFORMATICS
REVOLUTION
By Jeff Augen, PhD, President, TurboWorx
An Informatics Revolution from Within TurboWorx integrates disparate
applications and heterogeneous computing resources into an accelerated
pipelining and workflow solution without forcing businesses to change the way
their IT professionals do their jobs. By Jeff Augen, PhD, President,
TurboWorx
Almost every scientific, technical and research-intensive organization
supports its own distinctive culture of IT personnel, practices and policies
that governs how it goes about its business. That culture determines
everything from hardware selection to whether the organization deploys
commercial software or relies on its own resources to program and maintain
Open Source Software (OSS) applications. Typically, the IT culture becomes so
firmly embedded throughout an organization that it cannot be changed without
serious disruption and expense.
This leads to an inherent conflict. Research and operations computing
problems
change and grow all the time. These mission-critical problems are complex-
solutions often involve complex sequences of computational steps with large
quantities of data passing from one step to the next. Further complicating
the process is the heterogeneous nature of the data; most often derived from
numerous different sources. In addition, the individual computational steps
themselves are often compute intensive and very time-consuming. An IT culture
must be able to respond rapidly and efficiently with high performance
solutions to meet the demands posed by this new class of computing problems,
or it will become a barrier to business success.
Current budgetary constraints only worsen the difficulties that businesses
face in seeking to meet the multiplicity of computing demands. The conflict
between an entrenched IT culture and the need for flexibility, responsiveness
and growth in computing capability is one of the sharpest that any
organization faces.
Previously, only revolutionary cultural change-accompanied by costly
replacement of legacy systems-could break these bottlenecks. Today, however,
grid and cluster computing networks present a breakthrough opportunity that
allows virtually any organization or department within an enterprise to build
its own high performance computing environment by evolving the IT culture
instead of destroying it. Through innovative application integration tool sets
and the use of automated workflow software such as that from TurboWorx,
organizations can leverage existing computing resources and capabilities to
achieve levels of computing performance that previously required massive
investment in new hardware and staff. In most cases, the gains can be achieved
while actually easing the burdens on software developers and system
administrators.
Changing Computing without Changing Cultures
Advances in grid and cluster computing have made it possible to construct
"virtual supercomputers" from networks of computing resources These new
capabilities have played an important role in a wide variety of industries,
but especially in the life sciences where compute demands continue to grow at
a rate exceeding Moore's Law. Harnessing computing power into virtual
supercomputers is not for the unschooled. It is difficult to achieve
productive uses of clusters and grids without specialized knowledge. Such
platforms are far too complex for effective use of ad hoc manual and OSS
approaches to platform management or application optimization and tuning. To
use these virtual supercomputers effectively most decisions must be made
automatically by specialized software systems.
TurboWorx has stepped in to harness compute networks into virtual
supercomputers that can be run and modified even by users with minimal
programming skill. With our application integration and workflow pipelining
software, users throughout an enterprise can now have unprecedented compute
resources at their desktop workstation. For the first time it has become
possible to meet all of an organization's IT needs by leveraging existing
resources without extraordinary upheaval in its IT culture.
As a result, any business can make changes in its practices that measurably
impact productivity, speed-to-market and job performance with minimal
investment and, perhaps most importantly, without cultural disruption.
Business organizations can undergo an informatics revolution from within.
Rethinking Open Source
The IT cultures of research-intensive organizations, especially those in
the
life sciences, reflect both the academic backgrounds of many bench scientists
and the complexity of the compute tasks they face. These organizations
encounter significant informatics challenges as the complexity and volume of
experimental data needing to be processed and interpreted grows rapidly. As
research projects change in scope and size, the applications and data formats
required also change. The conventional approach is to write custom open source
Perl programs designed to glue together applications and databases for
individual projects. As soon as any aspect of the project changes, skilled
programmers are required to modify and rewrite code.
I have personally been a longtime supporter of a number of open source
initiatives. As Director of Strategy at IBM Life Sciences, I was an early
advocate for Linux and campaigned aggressively to launch an open source
initiative inside IBM as early as 1998. IBM derived great business benefit
from that move, and the community of Linux users has experienced enormous
growth as a result. Nowhere is the OSS community more active than in
bioinformatics. For academic and government laboratories with unique, highly
specialized application needs, OSS has allowed scientists to develop exactly
the application programs they require for their research. IDC research
estimates that 25 percent of all software used in the life sciences is open
source.
While OSS has opened up the computationally intensive world of
bioscientific
research to laboratories with limited financial resources, it should be well
understood that open source software is not free. Particularly when OSS is
used as the core of the computing infrastructure, the costs of choosing OSS
for harnessing the power of virtual supercomputers are often far greater than
the costs of relying on commercial software. When working with computing grids
and clusters linked into virtual supercomputers, the relative costs of relying
on an OSS platform to automate and integrate applications into effective high
performance workflows can dwarf any initial OSS price advantage. An
organization must be ready to maintain and upgrade software that they did not
develop and to rework it as their workflow needs change. This is a high-cost
way to go about supporting mission-critical operations.
More significantly, the cumbersome and costly process of dealing with OSS
can
tax the IT resources of even the most well-endowed research facilities. The
programming dexterity and efficiency of the IT staff, rather than the talents
and capabilities of bench scientists, becomes the critical step in innovation.
As a result, growing computing bottlenecks slow scientific and drug discovery
or, in other industries, limit the productive use of computing power.
Automated application integration, pipelining and workflow software, such
as
ours, can significantly accelerate drug discovery, scientific advances and
other business computing processes. The beauty of the software is that it
allows users to leverage the power of OSS-with a simple drag-and-click, new
OSS software modules that perform important computations can be added to any
workflow without programming intervention. TurboWorx's platform will quickly
pay for itself while those who rely solely on OSS and internal development for
workflow management are likely to generate additional costs each time they
modify a workflow, thwarting productivity and business-critical operations
Finally, open source design methodologies are best suited to tasks that
benefit from the simultaneous efforts of hundreds, even thousands of
developers. The opposite is true of architectural-design initiatives such as
our TurboBench platform. The development of mechanisms for application
parallelization and integration does not fit well with open source
methodologies because it does not scale well across large numbers of
developers-these tasks are best left to small development teams with very
specialized skills.
Breaking Computing Bottlenecks
TurboWorx is a best-of-breed software company specializing in application
management tools and middleware for high-performance technical computing and
workflows on clusters and computing grids. Designed by a small team of
recognized experts in autoparallelization and application integration, our
technologies and products represent 25 man-years of investment in their
development. They address the key issues of application integration, workflow
automation and parallel acceleration that must be handled in order to solve
the new class of mission-critical technical computing problems found in many
industries-including biotechnology, pharmaceutical, financial, geophysical and
energy, automotive and manufacturing, and aerospace and defense
industries.
To use clusters and computer grids as "virtual supercomputers" requires
middleware and management tools that address three major tasks: resource
management, data management and application management. Our TurboBench
platform deals with all three of these business-critical functions in ways
that require remarkably little training for new users or change in existing
systems. It can be bolted on, embedded or coexist with any legacy system or
set of applications, whether commercially developed or OSS.
The TurboBench platform integrates your chosen applications and data sets
into
computational workflows that exploit your heterogeneous network of PC
workstations, servers, and rack-mounted blades without the need for complex,
source-level programming. TurboBench completely automates workflow execution
by handling all of the details of running distributed workflows on large
quantities of data processed through multiple applications. Proven in leading
life sciences laboratories around the world, TurboBench can optimize
computational performance in any compute-intensive environment.
With its powerful graphical user interface, TurboBench provides users,
including those with little programming skill, with an intuitive tool set so
easy to use that a pilot system can be up-and-running in literally minutes.
TurboBench's Java-based, XML integration can create automated workflows that
take only a simple drag-and-click to modify. The system also includes a
complete, easily mastered parallel programming interface that enables the
creation of sophisticated parallel applications in any compute-intensive
environment.
TurboWorx has partnered with the leading industry IT solution providers,
including Dell, IBM, SUN and Apple, to assure smooth and simple integration
as well as reliable service and superior support, so that any organization's
IT resources can be leveraged to work at optimal performance levels now and in
the future.
TurboBench Features
TurboBench software provides users with:
- Easy-to-use Wizard to create simple building blocks called components.
- Drop-and-drag feature to link components into workflows.
- The capability to share, modify and debug components.
- State-of-the-art scheduling capability for setting job priorities and
running components on specific, particularly well-suited machines.
- Plug-and-play scalability. As jobs require more computing horsepower,
researchers simply add more computers to the network to improve
performance.
- Programs are automatically distributed among computers ("Workers") and
executed in parallel.
- Exceptional fault tolerance. If a component fails while executing a task,
the task is automatically picked up by another Worker and executed.
- Process monitoring to view and list job details and terminate jobs.
- A single, virtual data repository that acts as a pointer for all data
sources.
- Tiered and controlled multi-user access. TurboBench supports different
levels of user access: end-users to execute components and workflows,
designers to design and execute workflows and components, and
administrators to manage system setup and maintenance.
Web site: www.turboworx.com
|
