grids, clouds and hpc for research & education dr. ognjen prnjat, grnet
DESCRIPTION
Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET. eAge 2012, Dubai, December 2012. Outline. State of the art in eInfrastructures Europe Worldwide Grid collaborations Towards CHAIN-REDS: Example of HPC collaboration in SEE Towards CHAIN-REDS: Example R&E cloud platform. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/1.jpg)
Co-ordination & Harmonisation of Advanced e-Infrastructures for Research and Education
Data Sharing
Research Infrastructures – Grant Agreement n. 306819
Grids, Clouds and HPC for Research & Education
Dr. Ognjen Prnjat, GRNET
eAge 2012, Dubai, December 2012
![Page 2: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/2.jpg)
2
Outline
State of the art in eInfrastructures Europe Worldwide Grid collaborations Towards CHAIN-REDS: Example of HPC
collaboration in SEE Towards CHAIN-REDS: Example R&E cloud platform
![Page 3: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/3.jpg)
3
Outline
State of the art in eInfrastructures Europe Worldwide Grid collaborations Towards CHAIN-REDS: Example of HPC
collaboration in SEE Towards CHAIN-REDS: Example R&E cloud platform
![Page 4: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/4.jpg)
4
Pan-EU e-Infrastructures vision The Research Network infrastructure
provides fast interconnection and advanced services among Research and Education institutes of different countries’
The Research Distributed Computing Infrastructure provides a distributed environment for sharing computing power, storage, instruments and databases through the appropriate software (middleware) in order to solve complex application problems
This integrated networking & DCI environment is called electronic infrastructure (eInfrastructure) allowing new methods of global collaborative research - often referred to as electronic science (eScience)
The creation of the eInfrastructure is a key objective of the European Research Area
Network Infrastructure
e-Science Collaborations
Distributed Computing
Infrastructure
![Page 5: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/5.jpg)
5
Network: GEANT
![Page 6: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/6.jpg)
6
Grid: European Grid Infrastructure
30/05/2012 Project Presentation – May 2012
![Page 7: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/7.jpg)
7
Metrics Value (Yearly increase)
Installed Capacity
Logical CPUs
EGI-InSPIRE and EGI Council members
270,800 (+30.7% )
Including integrated RPs
399,300
HEP-SPEC 06
EGI-InSPIRE and EGI Council members and integrated RPs
2.96 Million (+49.5% )
Storage Disk (PB) 139 PB (+31.4% )
Tape (PB) 134.3 PB (+50%)
Resource Centres
EGI-InSPIRE and EGI Council members
326
Including integrated RPs 352
Supporting MPI 90
Countries EGI-InSPIRE and Council members 42
Including integrated RPs 54
Performance Monthly Availability/Reliability 94.50%/95.42%
Utilization HEP-SPEC 06 Hours 10.5 Billion (+52.91% )
Jobs 492.5 Million Jobs /year 1.35 Million Jobs/day (+46.42%)
Grid: European Grid Infrastructure
![Page 8: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/8.jpg)
8
HPC: PRACE
Tier-0European centres
Tier-1National centres
Tier-2Regional/University
centres
• 24 countries• 5 Tier0 systems currently
deployed• ~9 Pflops total power (Tier-0)• 15+ Tier 1 systems• PRACE PP + phases of
implementation projects, total EC investment ~ 70MEuro
• Access by European peer review
![Page 9: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/9.jpg)
9
Pan-European Cloud initiatives
No core initiative available yet StartusLab – GRNET leads operations, IaaS EGI cloud activity – providing virtualised
Grid sites on cloud Future possibilities: GRNET and other
clouds to be accessible via CHAIN gateway
![Page 10: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/10.jpg)
10
Outline
State of the art in eInfrastructures Europe Worldwide Grid collaborations Towards CHAIN-REDS: Example of HPC
collaboration in SEE Towards CHAIN-REDS: Example R&E cloud platform
![Page 11: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/11.jpg)
11
Regional Grid Infrastructures
CNGrid
NKN & Garuda
EUAsiaGrid
SAGrid & SANREN
GISELA
![Page 12: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/12.jpg)
12
CHAIN: global coverage
![Page 13: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/13.jpg)
13
Operational Sites
UoMCYNET
TUBITAK ULAKBIM
JUNET
HIAST
ERIEUN
INFN RM3
NARSS
39 sites (30 in production)in 16 countries
CERISTINFN LNS
(UAE)
In progress / Maintenance
IUGAZA
INFN BA
OBSPM
ANKABUT
CCK
INFN TS
CNRST
INFN PI
SNS PISA
IN2P3 LPSC
IN2P3 IRES
Univ.Bordeaux
IN2P3 CPPM
13
ECTP
GARR
INFN CTCOMETA
Mediterranean Grid (EUMEDGRID)
13
![Page 14: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/14.jpg)
14
Africa & Arabia Regional Operation Centre
http://roc.africa-grid.org
14
![Page 15: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/15.jpg)
15
China
CNGrid Environment 14 sites One OP Center Some domain app. Grids CNGrid GOS
Applications Resource & Environment Manufacturing Services
Sites with gLite/EMI (IHEP, PKU) in WLCG
HPC Systems Two 100 Tflops 3 PFlops
CNGrid
![Page 16: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/16.jpg)
16
India National Knowledge Network
A state-of-the-art multi-gigabit pan-India network for providing a unified high speed network backbone for all knowledge related institutions in the country
An ultra-high speed CORE, multiples of 10 Gbps
![Page 17: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/17.jpg)
17
GARUDA &
GARUDA Connects 45 C-DAC
centres and partners Over 6000 CPUs (~ 70
teraflops) and terabytes of mass storage to provide distributed data.
GARUDA Grid is now powered by the NKN
![Page 18: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/18.jpg)
18
Latin America
Red CLARA connects almost all the countries in Latin America and is managed by CLARA
CLARA interested t manage also computing initiatives
Currently ROC-LA managing Grid operations
18
![Page 19: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/19.jpg)
1919
15 Countries (11 in Latin America)19 Partners (14 in Latin America)
12 Third Parties (9 in Latin America)
LA: GISELA Project
![Page 20: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/20.jpg)
20
CHAIN: global coverage
20
![Page 21: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/21.jpg)
21
CHAIN contributions
State of the art in eInfrastructures analysis around the world
Recommendations and guidelines Roadmaps Support to Intercontinental VRC Interoperability through Science Gateway
![Page 22: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/22.jpg)
22
Outline
State of the art in eInfrastructures Europe Worldwide Grid collaborations Towards CHAIN-REDS: Example of HPC
collaboration in SEE Towards CHAIN-REDS: Example R&E cloud platform
![Page 23: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/23.jpg)
23
High-Performance Computing in SEE
120 Tflops aggregate 2 BlueGene machines Bulgaria, Romania, Serbia,
Hungary, FYRoM offering resources Procurements coming - Greece and
Serbia
26 applications in 3 VRCs
Envisaged as bridge to PRACE
Joint operations centre studied and assessed
![Page 24: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/24.jpg)
24
HPC Systems in the region
![Page 25: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/25.jpg)
25
Producing results / scientific publications – 26 apps
![Page 26: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/26.jpg)
26
Flexible Access Mechanisms
Pilot call for access to resources Call closed -> 5th of October Resources to be offered: 4.6M Core hours, 1.8 M GPU hours Allocations for 1 year – starting December 2012 Peer review based Access to the resources from all countries of the region Access to LinkSceem applications
Fast track access mechanism Limited resources provided 2 Month allocation period Suitable for: New user communities – Non experienced
users
![Page 27: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/27.jpg)
27
Outline
State of the art in eInfrastructures Europe Worldwide Grid collaborations Towards CHAIN-REDS: Example of HPC
collaboration in SEE Towards CHAIN-REDS: Example R&E cloud
platform
![Page 28: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/28.jpg)
28
GRNET okeanos service
• ~okeanos is set to deliver Infrastructure– Compute (Virtual Machines)– Network (Virtual Networks, private Ethernets
(L2) or public IPv4/6)– and Storage (Virtual Disks)
as a ServiceAlpha2: from March 2012 - 2000 VMs - 1000 alpha users; Beta version now
Target group: GRNET’s customers direct: IT depts of connected institutions indirect: university students, researchers in
academia
![Page 29: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/29.jpg)
29
Compute
Network
Storage
Security
Virtual Machines
Virtual Ethernets
Virtual Disks
Virtual Firewalls
okeanos service
![Page 30: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/30.jpg)
30
1x
2x5x
8x
okeanos service
![Page 31: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/31.jpg)
31
Compute: Cyclades Files: Pithos+ Images: Plankton Identity: Astakos
Volumes: Archipelago Accounting/Billing: Aquarium
okeanos service
![Page 32: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/32.jpg)
32
Cloud for R&E: the process Vision: flexible, production-quality cloud services for Greek R&E
community Rationale:
– Step beyond Grid in terms of flexibility and availability– Economies of scale for the community; solving understaffing
problems, poor service, low maintenance– Minimizes the investment in equipment and support contracts
Policy background: existing MoU in place for Grid computing, expanding for HPC as well
Strategy: – Technical workshops and requirements capture meetings– Gradual offering of services, starting with storage, moving to VM on
demand, IaaS, and then SaaS– Paving the way to public sector
Funding: in the context of GRNET4 project, 2.2ME DCs; 4.5ME s/w and services)
![Page 33: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/33.jpg)
33
Commercial IaaS vs own IaaS Commercial IaaS
Amazon EC2 not an end-user service Need to develop custom UI, AAI layers Vendor lock-in Unsuitable for IT depts
persistent, long-term servers, custom networking requirements
Gain know-how, build on own IaaS reusefor own services
okeanos: Design
![Page 34: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/34.jpg)
34
user@homeWeb Client CLI Client Web Client 2
GRNETdatacenter
Debia
n
VirtualHardware
OpenStack Compute API v1.1
Direct Out of Band Access
Synnefo cloud management software
Google Ganeti
KVM
GRNET Proprietary
okeanos: Platform design
admin@home
![Page 35: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/35.jpg)
35
Summary
Diverse Grid solutions from around the world
CHAIN provided a platform for integration of Europe and other continents; CHAIN-REDS will continue this support
CHAIN-REDS will expand towards HPC and clouds, some examples given here
![Page 36: Grids, Clouds and HPC for Research & Education Dr. Ognjen Prnjat, GRNET](https://reader030.vdocuments.site/reader030/viewer/2022033100/5681485d550346895db56ba4/html5/thumbnails/36.jpg)
Co-ordination & Harmonisation of Advanced e-Infrastructures for Research and Education
Data Sharing
Research Infrastructures – Grant Agreement n. 306819
Thank you