Inspired Grid networking

Grid infrastructure can provide a powerful means of resource sharing and computational power. With this in mind, Steven Newhouse describes how The EGI-INSPIRE project is inspiring the Future for Distributed Computing Infrastructures in Europe.

The EGI-InSPIRE project submitted to the FP7 call in November 2009 represents the start of a new era for the provision of distributed computing infrastructures (DCIs) in Europe. Building on the work of the EGEE (Enabling Grids for E-sciencE) series of projects, the EGI-InSPIRE project will support the production deployment of an INtegrated Sustainable Pan-European Infrastructure for Researchers in Europe for the next four years.

Integrated

EGI-InSPIRE will use its network of resource providers throughout geographical Europe (and beyond) to deploy advanced DCI technologies selected from providers from outside the project to form a production infrastructure to support European research and innovation. This production infrastructure, which will provide an enabling resource to researchers across Europe, will evolve to meet the growing needs of its user communities. Starting with grids composed of High Throughput Computing (HTC) resources pioneered within the EGEE project, we expect this to increasingly include High Performance Computing (HPC) resources, which are already becoming more common within EGEE. Eventually the resources integrated into the production infrastructure could include Volunteer Desktop Grids, such as those provided through the EDGES project, and new technologies such as virtualisation and clouds.

Sustainable

The European Commission, through FP6 and FP7, funded both the EDG and EGEE series of projects to prototype, pioneer and finally bring into production a DCI fabric to support high-throughput data science. Moving away from this short-term project based model, the European DCI community has agreed to establish a new organisation, EGI.eu, to coordinate the production infrastructure on its behalf. EGI.eu will be based in Amsterdam but with staff distributed throughout Europe. The stakeholders of EGI.eu are the National Grid Initiatives (NGIs), the European International Research Organisations (eg. CERN, ESA, EMBL), and other organisations that are committed to EGI.eu’s statues. These stakeholders contribute to the running costs of EGI.eu and the services it provides to coordinate activity across Europe. Within each NGI, a team interfaces their infrastructure with EGI.eu’s coordination and technical activities ensuring that software updates are promptly deployed, support issues are promptly handled and that activities taking place within a country are integrated into those taking place across Europe.

Pan-European Infrastructure

The production infrastructure within the EGEE project was augmented by the work undertaken by collaborating infrastructure projects around Europe – notably BalticGrid (for the Baltic States), SEEGrid (for South Eastern Europe) and the Nordic DataGrid Facility (for the Nordic countries). Within EGI-InSPIRE these countries are all integrated into the single proposal. Bringing together 38 separate NGIs provides geographical coverage across the whole of Europe including Russia and Georgia. In addition, strong collaborations will be established with emerging NGIs in the Asia Pacific region and with resource providers in North America.

For Researchers Everywhere

EGEE currently supports in excess of 13,000 researchers in their use of distributed HTC resources across Europe organised into Virtual Organisations supporting research in Life Sciences, Astronomy & Astrophysics, Computational Chemistry and Materials Science, Earth Science, Fusion, Grid Observatory, High Energy Physics and many others. The computational and storage resources are contributed by these communities to support collaborations within these virtual organisations.

The paradigm of data-intensive science is now firmly established as an analysis model within Europe and across the world. Within the portfolio of the ESFRI (European Strategic Forum on Research Infrastructures) projects there are many scientific instruments that will provide Petabytes of data annually to an international research community for subsequent analysis. This form of distributed analysis has been firmly established during the EGEE series of projects and one of the objectives of EGI-InSPIRE is to make this model available beyond the current EGEE user base to the emerging ESFRI projects.

Future Distributed Computing Infrastructure in Europe

The European Grid Initiative is not just a single activity encapsulated within EGI-InSPIRE. The EGI ecosystem will rely on a number of activities beyond the European production infrastructure brought together through the EGI-InSPIRE project:

The production infrastructure will rely on the proposed European Middleware Initiative (EMI) project to provide continued support and new features as its primary technology provider

The production infrastructure will rely on the proposed ROSCOE and SAFE projects to support their user communities and provide an interface between the infrastructure and the user communities

The production infrastructure will use the services from proposed projects such as TAPAS (for application porting), CUE (for training, dissemination and business outreach), SIENA (to support standard development activities within Europe), GRIDTALK-II (for general dissemination activity), amongst many others

By moving the work previously done within a single project into several projects, each project is able to focus on delivering its work to its individual stakeholders. For instance, EMI is able to focus on the delivery on high quality software to its customers – infrastructure providers such as EGI and PRACE or individual user communities. SAFE is able to provide support to its user communities that need to access both high throughput computing resources (ie. those provided by EGI) and high performance resources (ie. those provided by the DEISA project).

Summary

The restructuring that has taken place within the community during the course of the recently completed EGI Design Study project and has now been consolidated through the project proposals submitted into the November 2009 call. Together, these proposed projects will continue and expand the work started by the EGEE series of projects in providing a secure production quality distributed infrastructure for data intensive science. The long-term sustainability of this operating model will be achieved by moving key activities (ie. the production infrastructure, technology development, community support) closer to their main stakeholders – the NGIs, the technology providers and the user communities.

Steven Newhouse is the Director of the proposed EGI-InSPIRE project submitted into the November 2009 FP7 Infrastructure Call. Following a degree (1991) and PhD (1995) in Aeronautical Engineering at Imperial College London his research career moved into High Performance and High Throughput Computing. He has been active in the Open Grid Forum since 2001 and is currently the interim Director of EGI.eu and the Technical Director of the EGEE-III project.

Published: Wednesday, 15th September 2010 by Adelle Kehoe