INCD:➢ Provide computing services to the
academic and scientific communities.
➢ Started in 2004 with EGEE FP6. National Grid Initiative 2006. Became a wider service in 2010.
➢ Approved for integration in the Portuguese Science Foundation (FCT) infrastructures roadmap.
➢ Consortium partners LIP, FCCN and LNEC.
➢ Letters of support from U.Aveiro, U.Porto and U.Minho.
1
Infraestrutura Nacional de Computaçao DistribuidaNational Distributed Computing Infrastructure
INCD: Services➢ High Throughput Computing (HTC)
➢ High Performance Computing (HPC)
➢ Grid computing
➢ Cloud computing
➢ Data services
➢ Housing
➢ Training, consulting, porting, others ...
2
Grid Computing HPC/HTC
Computing - Storage - Data Management
International Federation DCIs
INCD: Core Infrastructure
3
FCCN Private Layer 2
LIP Lisbon
UniversityCoimbra
NCGSala-Grid
GEANT: European research network
U.Porto
FCCNRCTS
U.Aveiro CFP/IST
Other collaborating organizations
INCD: NCG/Sala-Grid @LNEC campus
4
➢ Main INCD resource centre
➢ Located at LNEC in Lisbon
➢ Datacenter managed by FCCN
➢ Advanced computing services by LIP
➢ Fully redundant: power, net, cooling
INCD: Services I
5
➢ High Performance Computing○ MPI batch jobs
○ NCG computing cluster
○ Ethernet and Infiniband interconnects
➢ High Throughput Computing○ Sequential batch jobs
○ NCG computing cluster
○ Multithreading and OpenMP also supported
➢ GPGPU computing○ Pilot activity
○ Evaluation of GPGPU with CUDA and OpenCL under several use cases
INCD: Services II
6
➢ Grid access to computing and storage○ Via grid middleware compatible with European Grid Infrastructure
○ NCG, LIP-Lisbon, U.Coimbra, CFP/IST, U.Porto, U.Aveiro
○ European level, Iberian level, National level
○ Integration of computing resources
➢ Data services○ High Performance Filesystem (Lustre)
○ Object storage (under development)
○ Block devices (under development)
○ Coupled with grid and cloud services
○ Other depending on user requirements
INCD: Services III
7
➢ Cloud Computing○ IaaS pilot service based on Openstack
○ Coupled with object and block storage
○ Pilot being opened to the community for testing
○ Integrated in the EGI federated cloud
➢ Housing○ Equipment housed in adequate conditions
○ RCTS network connectivity
○ Integration with INCD resources if needed
➢ Other○ Training, consulting, porting of applications, user support, etc
○ Open to new user requirements and needs
International e-Infrastructures
➢ Participation in several national and international Distributed Computing Infrastructures (DCI)○ IBERGRID - Iberian Grid Infrastructure
○ EGI - European Grid Infrastructure
○ WLCG - Worldwide LHC Computing Grid (CERN)
➢ R&D projects: H2020, etc ○ Participate: EGI-ENGAGE, INDIGO-DATACLOUD, etc
○ Bridge with others: AARC, infrastructure and community oriented, etc
➢ Collaboration: User Communities, etc ○ ESFRIs, strategic projects, organizations, research groups, etc
8
European Grid Infrastructure I
➢ Publicly funded e-infrastructure
○ 32 member countries
○ EIROs (CERN, EMBL)
○ Portugal member through FCT
➢ Supporting multiple scientific domains
○ Biomed, chemistry, oceanography, physics …
➢ Federates computing and storage resources
○ Grid -> Cloud, Data
➢ INCD participates in EGI through IBERGRID
○ Portuguese and Spanish centres together participate in the EGI infrastructure
9
European Grid Infrastructure II➢ EGI in numbers
○ ~22 000 users
○ ~230 Virtual Organizations (groups of user communities)
○ ~600 kCPUs
○ ~290 PB online storage
○ 355 resource centers (worldwide), in 54 countries
○ ~1.5 million application jobs/day
➢ IBERGRID in numbers
○ 22 resource centres
○ ~ 24 kCPUs
○ ~ 20PB
10
IBERGRID IN 2014
Portuguese Resource Centres
in IBERGRID and EGI
IBERGRID is a large EGI consumer
Open Access
11
➢ INCD infrastructure access classes○ Flagship research projects (strategic interest: roadmap)
○ Meritorious research projects (approved: FCT, peer review)
○ Consortium members (managed by the partners)
○ e-Government
○ Private companies (for R&D: SMEs, startups)
➢ In 2014○ ~3.5 million jobs (Grid)○ ~1.5 million CPU hours (non HEP)
■ General users Grid + HPC (22%)○ ~5.2 million CPU hours (HEP)
■ HEP users (Grid only)
Cloud Service Goals
➢ Cloud pilot infrastructure in the framework of INCD○ The aim is to provide a National Cloud service for the Portuguese
academic and research community
■ Support Infrastructure as a Service (IaaS): compute and storage resources
■ Support users applications integration, portals, community platforms
■ Integration in RCTS AAI
○ Integration in the EGI federated cloud
■ Federation of scientific and academic cloud sites at international level
○ Evaluate hybrid clouds
■ Evaluate the use of public cloud providers
■ Scale from INCD to commercial providers
■ Study network and interface aspects
12
Cloud Service II➢ Cloud pilot infrastructure deployment details
○ Openstack (Havana) on CentOS 6
○ Nova with KVM hypervisors (compute)
○ Neutron (networking)
○ Cinder (block storage) and Swift (object storage)
○ Glance (images upload supported)
○ Compute APIs: OCCI (open standard), EC2 (Amazon), Nova (Openstack)
○ Object Store APIs: S3 (Amazon), Swift (Openstack)
○ Keystone (User Authentication and Authorization)
■ Local users through username/password (PT users)
■ X.509 certificates, Virtual Organizations (EGI FedCloud)
○ Resource integrated in EGI Federated Cloud
○ Accounting, monitoring
13
Cloud Service III➢ Future production infrastructure (ongoing)
○ Openstack (Juno) on CentOS 7
○ High Availability of Openstack services
■ Load balancing, redundancy and failover (HAproxy)
■ Secure public access to Openstack APIs, and dashboard
○ Testing Ceph distributed filesystem
■ Block storagem
■ Object store
■ Image storage
○ Test Federated Authentication and Authorization Infrastructure (AAI)
■ Integration with SAML/Shibboleth, RCTSaai, EDUGAIN
■ Bridge with H2020 AARC: Géant, EGI, IGTF/EUgridPMA, REFEDS, FIM4R, RDA
○ Monitoring
■ Better accounting and monitoring (Ceilometer, Elastic Search, Flume, Kibana, ...)
14
Cloud Service IV➢ Future production infrastructure (ongoing)
○ Network
■ Improve virtual networks (Neutron)
■ IPv6
○ Special services
■ GPGPU
■ MPI / OpenMP
■ Linux Containers
■ Commercial software
○ Other
■ Scheduling, fair-share, priorities, etc
■ Orchestration (Heat)
15
Openstack Architecture
HORIZON dashboard web portal
KEYSTO
NE
Authentication
Authorization
NO
VAM
achine m
anagement
GLA
NC
ESystem
images
CIN
DER
Block storage
NEU
TRO
NN
etworking
SWIFT
Object storage
Users Applications Automation
API API API API API API
CLI Bindings / libsWeb browser
…
OS, EC2, S3, OCCI
TroveSaharaHeatCeilometer
APIs
EGI Federated cloud: Overview
➢ Federation of institutional private Clouds, offering Cloud Services to researchers in Europe and worldwide○ Enable resource provision targeted towards the research community
○ Scale to user needs
○ Resilience through integration of multiple different providers
○ Prevent vendor lock-in
➢ Standards based federation of IaaS cloud:○ Exposes a set of independent cloud services accessible to users utilising a
common standards profile
○ Allows deployment of services across multiple providers and capacity bursting
17
EGI Federated cloud: Architecture
18
EGI C
loud
Ser
vice
Mar
ketp
lace
EFI C
olla
bora
tion
tool
s
Providers Cloud Management Framework(Cloud middleware agnostic:
openstack, opennebula, WNoDeS, Syneffo, etc)
Instance Mgmt
EGI Cloud Infrastructure PlatformStorage Mgmt
Service Registry
Information Discovery
Federated AAI Monitoring Accounting
Helpdesk Support
Security Coordination
Training Outreach
Sustainable Business Models
OVF
EGI Core Platform
OCCI CDMI
GLUE2 GSI SAM UR
19
EGI Conference 2015http://conf2015.egi.eu/
Lisbon, 18-22 May 2015ISCTE
2x Xeon X5650 @2.67GHz 12 GBRAM ScientificLinux 5.5
Cloud - 16 GbRAM
Nº de processadores
Exemplo de aplicação: Hidrodinâmica e contaminação fecal no estuário do Tejo
Tem
po (m
inut
os)
- Modelo 3D acoplado circulação e evolução de traçador
- 20 níveis na vertical e a resolução espacial: entre 2 km e 30 m
Adiram ao piloto !http://www.incd.pt
Openstack: pros and cons➢ Advantages
○ Free, opensource, multi-tenant solution
○ Widely adopted in science and industry
○ Good interoperability and federation capabilities
○ Well supported, good documentation, active and dynamic community
○ Platinum members: Canonical, HP, AT&T, IBM, Intel, Rackspace, Red Hat, SUSE
○ Modular and flexible architecture with good scalability
○ Writen in Python can be easily adapted to specific requirements
○ Adopted by several NRENs for similar purposes
○ Supported in the European Grid Infrastructure FederatedCloud
○ CERN is adding 100 hypervisors p/ week aim is 15000 in Dec 2015
➢ Disavantages○ Configuration can be complex
21