egi services for structural biology - s brewer egi

www.egi.euEGI-InSPIRE RI-261323

EGI-InSPIRE

www.egi.euEGI-InSPIRE RI-261323CECAM, Oxford - May 22, 2012

Integrated e-infrastructure for integrated software for integrated structural biology:

defining the roadmap for collaboration

1

Steve Brewer [email protected]

Presented by:

David Wallom [email protected]

mailto:[email protected]



Outline

1. The background: integrated software for integrated structural biology

2. The landscape for integrated e-infrastructure

3. Defining the roadmap for collaboration

2CECAM, Oxford - May 22, 2012


Outline






Background

• Structural Biology: the view from outside– A range of in silico techniques to study biological

structures – including NMR:• Nuclear magnetic resonance spectroscopy of proteins

– A major division of Biology • therefore results need to be shared and re-used?

– At Oxford, The Division of Structural Biology - (STRUBI), has a number of centres and projects

– At EU level – INSTRUCT an ESFRI project – Also, WeNMR has been steadily building its own

international infrastructure & collaborating with EGI.



EGI and Structural Biology

– WeNMR VRC – representing the Structural Biology community to date (significant body of users)

– Participation in User Community Board (UCB) (1st VRC)

• Synergy with other communities• Requirements gathering• Contact Alexandre Bonvin for further

info.

– www.wenmr.eu/

– enmr.eu >400 users– Largest VO in Life Sci.


operations-portal.egi.eu/vo/usersMetrics

http://www.wenmr.eu/


Contents






C21: Digital Research


Extracting Knowledge from the Data Deluge


Digital Agenda for Europe


• Borderless Services• Interoperability• Supporting Innovation

Digital Agenda for Europe“Europe should also build its innovative

advantage in key areas through reinforced e-Infrastructures (i.e. GEANT & EGI)”

remove barriers to the free movement

of knowledge


What is EGI?

• European Grid Infrastructure (EGI)– Federation of National Grid Infrastructures– Represented by EGI.eu (Dutch foundation)

• National Grid Infrastructures (NGIs)– Sole national point of contact for ‘Grids‘– Integrates individual resource centres

• Current Status:– 35+ NGIs & 350+ resource centres– 300,000 CPUs & 200PB+ storage



EGI


• European– Over 35 countries

• Grid– Secure sharing

• Infrastructure– Computers– Data– Instruments– …. and beyond!!


EGI Resource Infrastructure Providers – RPs (April 2012)



EGI Resource Centres(April 2012)



VO Statistics (April 2012)


Metrics Value

Registered national and international VOs

226 (3.20% )

Registered users 20883 (14.30% )

Active VOs (High/Medium/Low)

25/23/8

ArcheologyAstronomyAstrophysicsCivil ProtectionComp. ChemistryEarth SciencesFinance

FusionGeophysicsHigh Energy PhysicsLife SciencesMultimediaMaterial Sciences

…

Supported Research Communities


EGI.eu


• Coordination for European Grid resources– Established February 8th 2010– Central policy & services needed to run a grid– Sustainable small coordinating organisation

• Based in Amsterdam– Coordinating core (~20 people) in Amsterdam– Technical services from partners (~20 people)

EGI and EGI.eu: Supported by the EGI-InSPIRE project


EGI-InSPIRE: project objectives

• A sustainable production infrastructure– Resource providers in Europe and worldwide – With new technologies as they mature

• Support structured international research– Sustain current domain specific services – Attract new user communities (e.g. ESFRI)



EGI-InSPIRE Project


Integrated Sustainable Pan-European Infrastructure for Researchers in Europe

A 4 year project with €25M EC contribution

– Project cost €72M– Total Effort ~€330M– Effort: 9261PMs

Project Partners (50)

EGI.eu, 38 NGIs, 2 EIROs

Asia Pacific (9 partners)


Technology Innovation


• Will come from outside EGI– Moving research technologies into production

• Partnership with technology projects– EMI (European Middleware Initiative)– IGE (Initiative for Globus in Europe)– EDGI (European Desktop Grid Initiative)– StratusLab– VenusC– SAGA


VOs, VRCs and partner projects

18

• Virtual Organisations: rules defining people & resources• VRCs are self-organised (virtual) research communities

which give individuals within their community a clear mandate to represent the interests of their research field within the EGI ecosystem.

• Current EGI VRCs: (http://www.egi.eu/community/vrcs/)– WeNMR - Structural biology– LSGC – Life sciences– HMRC – Hydro-meteorology– WLCG – High energy physics NEW!– CLARIN/DARIAH – Humanities (with a Letter of Intent)

CECAM, Oxford - May 22, 2011

http://www.egi.eu/community/vrcs/


European Grid Infrastructure

EGI Ecosystem

EGI.eu (not-for-profit org)• Established February 8th 2010• Central policy & services• Sustainable, ~20 people• Based in Amsterdam, NL• Partners provide services

EGI-InSPIRE project• 4 year project, 25M EC funds• 50 project partners



EGI: strategy for 2020

• Pillars of the strategy:1. European-wide coordination and interaction with research

communities and national resource infrastructure providers

2. Coordination, maintenance, operation and delivery of an open uniform European-wide federated production infrastructure

3. Developing and promoting technologies for federating new resources

4. Supporting the integration and operation of scalable interdisciplinary Virtual Research Environments personalised to each research community

http://go.egi.eu/EGI2020



VRCs – VREs & EGI

EGI Community

Platform

EGI Collaboration

platform

EGI resources(clusters, storages,...)Dedicated or shared resources/platforms

(e.g. Clusters; EC2, PRACE, XSEDE)

Research facilities (e.g. detectors, sensor

networks)

VMVM

VM

SWVM

VMSW EGI Infrastructure platform

DB

DB

Virtual Research

Environment

Created by a project such as CLARIN, ELIXIR,

LifeWatch, ...

Virtual Research Communities



Services(Applications & data)

Anatomy of a VRE

22

Service access (Science gateways - portal, desktop and mobile)

Off-the-shelf and custom VM images

Domain specific and generic data

repositories

Off-the-shelf and custom applications

Service integration(workflows)

EGI support

EGI support

EGI support

EGI suppo

rt



EGI support:Service access

• Catalogue of existing solutions– http://go.egi.eu/sciencegateways

• Gateway components repository– SCI-BUS portlet repository

will be used (see MoU)

• Plan: “How-to” documentation– Develop an EGI science gateway primer

(through a new EGI Virtual Team project and [email protected])

• Portal & gateway workshops– CF & TF– Portal workshops by NGIs– Community workshops on portals


http://go.egi.eu/sciencegateways

https://documents.egi.eu/document/1009

https://wiki.egi.eu/wiki/Virtual_Team_Projects



EGI support: Cross cutting support

• Applications Database (EGI software catalogue)– Stores information about any EGI software– Tags:

• Organise software into clusters

– Subscription:• Join an item/cluster as supporter

(developer, operator, expert)

– Notifications:• To get informed about changes

– Web gadgets:• Embed customised lists of software into your website

– Write API Planned• Itegrate the catalogue with the EGI platform (e.g. Info system)• Integrate the catalogue with other catalogues

• Training Marketplace– Advertise and search for events, materials, online courses– Web gadgets:

• Embed customised lists of software into your website

• Events– Community and Technical Forums (Call for TF2012 co-located events is open!)– Topical workshops (hosted by NGIs)


http://appdb.egi.eu/

http://training.egi.eu/


EGI support: Service development and operation

• Established & evolving processes from various EGI(-InSPIRE) stakeholders:

– EGI Operations– EGI Technology– NGIs, VRCs, EGI.eu– Pojects (e.g. BioVel, Scalalife, ...)

• But the EGI platform is about to change:– Services will run in Virtual Machines

• EGI Strategic Plan; EGI Platform Roadmap

– Developers & operators will need new support mechanisms:• VM image repository• VM information system• VM monitor with helpdesk integration• ...

EGI Federated Cloud Task force EGI Infrastructure platformJoin as user community!





Summary of EGI

• EGI.eu established in Amsterdam– Supported through EGI-InSPIRE project

• EGI has transitioned from a federation of regional to national resource providers

• EGI works with technology providers on an open standards based architecture

• EGI will evolve to support the needs of its current and new research communities



Outline






Roadmap for collaboration

• EGI wants to support Structural Biology– work with INSTRUCT– Continue working with WeNMR VRC– Co-located meetings?– Virtual Teams to achieve short-term goals

• Specific use cases and scenarios – descriptions?• Formally involve specific NGIs – which?• Virtualisation to achieve scalability of solutions?• Identify which software applications are needed?



EGI activities: forums, workshops, VTs and TFs

– Forums• Every 6 months, around 500 people meet to share knowledge and plans• Keynotes, presentations, workshops, training, demonstrations, posters

– Task Forces• Federated Clouds• MAPPER• Identity

– Workshops (face-to-face and also ‘webinars’)• Integration of complex scientific workflow systems with EGI’s services.• Research Infrastructure interested in adopting EGI’s operational tools to

support their own distributed production infrastructure• Science Gateways Workshop

– Virtual Team (VT) projects – 3- 6 months (with NGIs)



Technical Forum TF12 - Prague

– Theme of Technical Forum in September (17th – 21st):• “Developing an open and sustainable EGI ecosystem to support open science in the

digital European Research Area.”

– 5 tracks:1. Operational services

Operational services used by EGI to deliver a federated infrastructure composed of national resource providers

2. Physical Infrastructure servicesWork of technology providers and platform integrators to provide generic infrastructure services that can be integrated into virtual research environments upon which research communities depend

3. Virtualised Resources: challenges and opportunitiesWork of technology providers, platform integrators and platform operators of the grand challenges that relate to the integration of virtualized resource

4. Virtual Research EnvironmentsResearch communities are seeking to establish their own virtual research environments with their own resources and operating their own customized services through the work of platform integrators and platform operators

5. Coordination and CommunicationFocus on reaching out to communities and partners across the ecosystem



VT projects: examples

VT projects of interest to Structural Biology


1. The Federated Cloud Task Force is already providing a cloud infrastructure through a few committed NGIs. This pilot resource is being made available to WeNMR and other research communities participating in the Task.

2. The Message Passing Interface (MPI) group is providing support for application developers to port and scale up parallel applications to the European Grid Infrastructure.

3. A lively portal developer and provider community is being supported by EGI. The group is able to setup and operate custom, application and/or community specific portals for members of the Structural Biology community to interact with EGI.

4. A project to assess the possibilities for federated identity management in EGI and the readiness of EGI member states for federated authentication and authorisation mechanisms will deliver greater flexibility for integrated services.

5. EGI’s operating resources and user support services continue to grow, with new types of services and infrastructures, such as desktop grids, map-reduce frameworks, GPU clusters continuously being developed and made available.

Suggestions for future VT projects may emerge from discussions at CECAMFurther information on VTs: https://wiki.egi.eu/wiki/Virtual_teams

https://wiki.egi.eu/wiki/Virtual_teams

https://wiki.egi.eu/wiki/Virtual_teams


Conclusions

• EGI is more than a production infrastructure:– A framework for sharing and innovation– A forum for communication and collaboration

• Structural Biology research community has strong grid history– Many years experience plus involved from the start of EGI– Many grid-based applications, research and publications– http://www.wenmr.eu

• Structural Biology community key part of future EGI Strategy– EGI Strategy is aligned with EU 2020 strategy inc. Innovation Union – EGI will support and partner various ESFRI projects and researchers– EGI will develop requirements with technology developers & providers



EGI-InSPIRE

www.egi.euEGI-InSPIRE RI-261323CECAM, Oxford - May 22, 2012

Questions?

EGI user community support team:

[email protected]

33

See you at the EGI Technical Forum 2012,Prague, Czech Republic between 17–21 September
