the egi federated cloud
TRANSCRIPT
www.egi.euEGI-InSPIRE RI-261323
EGI-InSPIRE
www.egi.euEGI-InSPIRE RI-261323
The EGI Federated Cloud
David Wallom
University of Oxford
Chair, EGI Federated Cloud
www.egi.euEGI-InSPIRE RI-261323
Why federation?
• User load larger than a single in-house system
• Single public providers introduce SPoF• Multiple service providers give resilience• But…
– Multiple service providers cause management headaches
– Multiple interfaces cause development headaches
www.egi.euEGI-InSPIRE RI-261323
HardwareHardware
HardwareHardware
Hardware
Cloud ManagementCloud Management
Cloud ManagementCloud Management
Cloud Management
User CommunitiesUser Communities
User Communities
Federated interfaces
Federated services
• Heterogeneous implementation: no mandate on the cloud technology.
• Standards and validation: Recommended and common open standards for the interfaces and images – OCCI, CDMI, OVF, GLUE2.
• Provider agnosticism: the only condition to federate resources is to expose the chosen interfaces and services.
• Resource integration: Cloud Computing to be integrated into the existing production infrastructure.
Principles of Federation
www.egi.euEGI-InSPIRE RI-261323
Value proposition
The EGI Federated Cloud is the federation of public and private Clouds, offering Cloud Services to consumers in Europe and the rest of the world
A cloud system able to • Scale to user* needs presenting individually to each community• Integrate multiple different providers to give resilience• Prevent vendor lock-in • Enable resource provision targeted towards the research community
Standards based federation of IaaS cloud: • Exposes a set of independent cloud services accessible to users
utilising a common standards profile• Allows deployment of services across multiple providers and capacity
bursting* Where user is no longer just the end user
www.egi.euEGI-InSPIRE RI-261323
EGI Cloud Infrastructure
5
EGI Core Platform
Federated AAI
ServiceRegistry Monitoring AccountingInformation
Discovery
Help and Support
Security Co-ordination
Training and Outreach
EGI C
olla
bora
tion
Platf
orm
EGI
Appl
icati
on
DB
Imag
e Re
posi
tory
EGI C
loud
Ser
vice
Mar
ketp
lace
Sustainable Business Models
User Communitty
EGI Cloud Infrastructure Platform
Instance Mgmt
Storage Management
Cloud Management Stacks(OpenStack, OpenNebula, Synnefo, …)
Providers Cloud Management Framework(new one we don’t now about and don’t want to care about)
Standards
OCCI CDMI
OVF
www.egi.euEGI-InSPIRE RI-261323
Personalised Federations
EGI Platforms
BioVel Cloud Federation CLARIN Cloud
Federation
SCI-BUS Cloud
Federation
GSI Cloud Federation
ELIXIR Cloud Federation*
CSGw Cloud Federation
DCH
-RP Cloud Federation
www.egi.euEGI-InSPIRE RI-261323
Geographical dispersion
• 12 countries provide 15 certified resources
– Czech Republic, Germany, Greece, Hungary, Italy, Macedonia, Poland, Slovakia, Spain, Sweden, Turkey, United Kingdom
• 2 countries currently integrating– Croatia, Finland
• 5 countries interested– Bulgaria, France, Israel*, The
Netherlands, Switzerland
• Worldwide interest– South Africa* (SAGrid)– South Korea* (KISTI)– United States* (NIST, NSF Centres)
* Not shown on map
www.egi.euEGI-InSPIRE RI-261323
EGI Marketplace
• Rich and diverse set of individual offerings• Application/VM Image repository• User community choice of provider at all levels• Academic and commercial resource & service providers• User community from the public and private sector
• Client tools at either the GUI (Broker), command- line or API level
www.egi.euEGI-InSPIRE RI-261323
The BioVel Portal
04/15/2023 9
www.egi.euEGI-InSPIRE RI-261323
OpenModeller on the Biovel Portal
04/15/2023 10
The Ecological Niche Modelling (ENM) Workflow takes as input a file containing species occurrence points to create a model with the openModeller Web Service.
• The EUBrazilOpenBio ENM service is exposed through an extended openModeller Web Service interface• Multi-staging and multi-parametric oM experiments are implemented through COMPSs that dynamically creates the virtual resources to execute the operations.• An OCCI connector is used for the VMs management while data management supports CDMI endpoints.
ENM Service (OMWS2)
VENUS-C Cloud Middleware
COMPSs Workflow Orchestrator
COMPSs Workflow OrchestratorOCCIOCCI CDMICDMI
EGI Federated Cloud
Service available athttps://portal.biovel.eu/
www.egi.euEGI-InSPIRE RI-261323
Strengthening the underpinning platform
Technology Evolution• Broader support for open standards in Cloud
management frameworks– Configuration of interfaces to commercial cloud frameworks
• New feature additions to foundational tools depending on requests– Accounting, monitoring, service discovery, Application DB
• Innovate and deploy PaaS & SaaS• Large improvements on documentation for all 3
stakeholder groups
11
Valu
e ad
ded
serv
ices
for U
ser
Com
mun
ities
www.egi.euEGI-InSPIRE RI-261323
Conclusions
• Paving the way for a global federated cloud marketplace– Established best practice– Illustrating European leadership– Open standards, open technology– Open membership, open processes– Open competition
• Aiming to become a philosophy on how we create federated e-infrastructure
Launch capability – 5,000 cores, 225 TB storage
Q4 2014 (planned commitments) – 18,000 cores, 6000 TB storage
2020 (Goal) – 1,000,000 cores, 1 EB storage
12
www.egi.euEGI-InSPIRE RI-261323
Thank you!
Questions?
13
Task Force resources
Mailing List: [email protected]
Wiki site: http://go.egi.eu/tf-fedclouds
GitHub: https://github.com/EGI-FCTF
Indico site: https://www.egi.eu/indico/categoryDisplay.py?categId=112