Although the precise definition of what constitutes a grid application can vary, there is general consensus that grid computing involves coordinating resources that are not subject to centralized control and using standard, open, general-purpose protocols and interfaces to deliver non-trivial qualities of service.

The issues facing the grid applications community today are very different from those faced five years ago. It is generally agreed that the major early barrier to grid-enabling applications was the status of the infrastructure. There has been progress in the infrastructure -- though probably not as much as required -- but it has also become obvious that there are additional bottlenecks to the widespread adoption of grid applications and the effective utilization of these distributed resources. In spite of much worldwide effort, the number of complex scientific applications that can effectively utilize distributed infrastructure remains minuscule.

There remain only a handful of applications that have been designed a priori for distributed systems (e.g., GridSAT). The pressing question is: Why in spite of the resources that have gone into the global grid effort, do the previous two statements remain true? Are these symptomatic of a silent killer?

Looking at the landscape of grid computing research today, it appears that much of what is being undertaken shares strong similarities with work in distributed computing. The precise relationship between grid computing and distributed computing, however, is a matter of open debate, but there is consensus that the evolving infrastructure presents both unique opportunities, as well as challenges for applications and application scientists. Of concern however, is that the majority of scientists retain the same mental model of programming applications for distributed infrastructure as they did for isolated serial and parallel computers. Thanks to many years of experience and research, it is relatively easy to determine which algorithms and data structures should be used such that application codes scale well across a single platform. But how should scientific applications -- both compute-intensive and data-intensive -- be programmed such that they are easily able to utilize a distributed infrastructure? How can computer scientists aid the design and implementation of applications that shields such applications from future changes in computing environments?

Based on these key questions, the Distributed Programming Abstractions (DPA) theme was launched at NeSC/eSI (in Edinburgh, Scotland) as a year-long theme to provide a better understanding of how distributed infrastructure can be used more effectively for developing grid applications. A unique feature of this theme is the strong coupling between distributed programming models, techniques and abstraction on the one hand and real applications on the other. The DPA theme brings together application/computational scientists from different domains, along with computer scientists, to first explore programming models and abstractions in detail and then apply them to specific application classes and domain.

The first workshop in the series took place at the end of May and included participation from a number of key researchers in the area of grid computing. The workshop also had discussion sessions, such as "What does your application need from the Grid" and "What Lessons Can We Learn from Parallel Programming" -- topics aimed at a better understanding of the context of distributed programming. For example, it was generally agreed that the MPI library provided a useful standard for developing parallel applications as the most commonly used and best-understood parallel model was used for standardization. It was argued whether a similar approach could be found for distributed computing. There was also consensus that a new language for distributed computing is not desired in this context, as what is needed are new tools and models that assist composing applications. For instance, many grid applications today involve coupling of legacy codes, managed by different institutions, and located at geographically distributed sites. Each code (or component) within such an application may be parallel and involve access to proprietary data sets. It was the composition of such independent codes that provided a significant challenge to grid application architects. The SCOOP application for Hurricane modeling provides an example of this.

Subsequent discussion focused on work in distributed objects, such as CORBA and related efforts from the Object Management Group (OMG) -- which also had a somewhat similar scope. The CORBA effort also focused on either wrapping existing legacy codes, or integrating newly developed components with such legacy codes. A variety of "container services" were then provided that could allow the wrapped legacy application to make use of common infrastructure services. Much of the early efforts in CORBA involved developing specialist middleware infrastructure to facilitate the creation of distributed applications. A number of "horizontal" and "vertical" services were identified to enable commonly used features (such as a naming service, an event service, etc.) to be standardized and reused. A similar effort was initiated in the Web services community, where commonly used capabilities were captured in WS-* specifications (referred to as WS-Nightmare by Geoffrey Fox during the workshop) that could be reused across different application domains. Perhaps both efforts have had limited impact on the application development community due to the complexity of the solution proposed, and the performance penalties that are often incurred when wrapping an existing legacy application in such approaches. In general, the session on parallel computing and distributed objects provided interesting insight into the important characteristics of successful and usable programming abstractions.

The second part of this workshop focused on listing the various programming models/abstractions and then identifying the common features -- to divide them into high-level categories. It was concluded that the five high-level categories are: Composition, Messaging, Component, Services and Grid Aware (grid libraries).

This categorization was initially undertaken without reference to any particular existing effort, but afterward compared with published work of Lee & Talia¹, Parashar & Brown² and Soh³, and found to generally agree with these papers, with differences that were well-understood. The recent book by "Grid Computing: Software Environments and Tools" also explains the software engineering issues underpinning the development of grid computing applications.

A similar exercise was undertaken in the subsequent sessions to list the range of applications and then categorize them. Arguably, the exercise here was not as successful, as it was difficult to correctly identify the necessary categories. In the second half of this session, a gap analysis of the various programming models/abstractions listed in the previous session was undertaken. Participants of the workshop are working on classifying the range of distributed applications and aligning them with the five programming models and abstractions outlined. It was also agreed in discussions that effort would be spent on understanding "Distributed Patterns" -- in defining and devising an ontology for them.

Workshop participants agreed that there is a critical need for: (1) classification and analysis of the various programming abstractions that are available/possible for distributed applications; (2) a similar taxonomy for distributed applications and subsequent mapping of these application; and (3) a gap analysis of the programming abstractions. The discussion sessions provided a strawman document for 1 to 3. It was felt that an objective of this theme should be to produce a document somewhat analogous to the recent Berkeley landscape of parallel computing research but for distributed computing -- i.e., to survey the status of distributed applications in relation to the programming models, tools and architecture, and, in particular, to identity analogous "dwarfs" for distributed computing as the Berkeley paper did for parallel computing.

The next workshop is scheduled for the end of October in Edinburgh. Anyone interested in contributing or participating in the theme should contact any of the core members listed at http://wiki.esi.ac.uk/Distributed_Programming_Abstractions.

-----

¹ Grid Programming Models: Current Tools, Issues and Directions by Craig Lee and D. Talia www.di.unipi.it/~coppola/GRIDsem/c618Grid2002_LeeTalia.pdf

² Conceptual and Implementation Models for the Grid Parashar & Brown: www.caip.rutgers.edu/TASSL/Papers/proc-ieee-grid-computing-models-05.pdf

³ Grid Programming Models And Environments, Soh, et al http://gridbus.csse.unimelb.edu.au/papers/GridProgrammingModels2006.pdf