INCD:➢ Provide computing services to the

academic and scientific communities.

➢ Started in 2004 with EGEE FP6. National Grid Initiative 2006. Became a wider service in 2010.

➢ Approved for integration in the Portuguese Science Foundation (FCT) infrastructures roadmap.

➢ Consortium partners LIP, FCCN and LNEC.

➢ Letters of support from U.Aveiro, U.Porto and U.Minho.

1

Infraestrutura Nacional de Computaçao DistribuidaNational Distributed Computing Infrastructure

INCD: Services➢ High Throughput Computing (HTC)

➢ High Performance Computing (HPC)

➢ Grid computing

➢ Cloud computing

➢ Data services

➢ Housing

➢ Training, consulting, porting, others ...

2

Grid Computing HPC/HTC

Computing - Storage - Data Management

International Federation DCIs

INCD: Core Infrastructure

3

FCCN Private Layer 2

LIP Lisbon

UniversityCoimbra

NCGSala-Grid

GEANT: European research network

U.Porto

FCCNRCTS

U.Aveiro CFP/IST

Other collaborating organizations

INCD: NCG/Sala-Grid @LNEC campus

4

➢ Main INCD resource centre

➢ Located at LNEC in Lisbon

➢ Datacenter managed by FCCN

➢ Advanced computing services by LIP

➢ Fully redundant: power, net, cooling

INCD: Services I

5

➢ High Performance Computing○ MPI batch jobs

○ NCG computing cluster

○ Ethernet and Infiniband interconnects

➢ High Throughput Computing○ Sequential batch jobs

○ NCG computing cluster

○ Multithreading and OpenMP also supported

➢ GPGPU computing○ Pilot activity

○ Evaluation of GPGPU with CUDA and OpenCL under several use cases

INCD: Services II

6

➢ Grid access to computing and storage○ Via grid middleware compatible with European Grid Infrastructure

○ NCG, LIP-Lisbon, U.Coimbra, CFP/IST, U.Porto, U.Aveiro

○ European level, Iberian level, National level

○ Integration of computing resources

➢ Data services○ High Performance Filesystem (Lustre)

○ Object storage (under development)

○ Block devices (under development)

○ Coupled with grid and cloud services

○ Other depending on user requirements

INCD: Services III

7

➢ Cloud Computing○ IaaS pilot service based on Openstack

○ Coupled with object and block storage

○ Pilot being opened to the community for testing

○ Integrated in the EGI federated cloud

➢ Housing○ Equipment housed in adequate conditions

○ RCTS network connectivity

○ Integration with INCD resources if needed

➢ Other○ Training, consulting, porting of applications, user support, etc

○ Open to new user requirements and needs

International e-Infrastructures

➢ Participation in several national and international Distributed Computing Infrastructures (DCI)○ IBERGRID - Iberian Grid Infrastructure

○ EGI - European Grid Infrastructure

○ WLCG - Worldwide LHC Computing Grid (CERN)

➢ R&D projects: H2020, etc ○ Participate: EGI-ENGAGE, INDIGO-DATACLOUD, etc

○ Bridge with others: AARC, infrastructure and community oriented, etc

➢ Collaboration: User Communities, etc ○ ESFRIs, strategic projects, organizations, research groups, etc

8

European Grid Infrastructure I

➢ Publicly funded e-infrastructure

○ 32 member countries

○ EIROs (CERN, EMBL)

○ Portugal member through FCT

➢ Supporting multiple scientific domains

○ Biomed, chemistry, oceanography, physics …

➢ Federates computing and storage resources

○ Grid -> Cloud, Data

➢ INCD participates in EGI through IBERGRID

○ Portuguese and Spanish centres together participate in the EGI infrastructure

9

European Grid Infrastructure II➢ EGI in numbers

○ ~22 000 users

○ ~230 Virtual Organizations (groups of user communities)

○ ~600 kCPUs

○ ~290 PB online storage

○ 355 resource centers (worldwide), in 54 countries

○ ~1.5 million application jobs/day

➢ IBERGRID in numbers

○ 22 resource centres

○ ~ 24 kCPUs

○ ~ 20PB

10

IBERGRID IN 2014

Portuguese Resource Centres

in IBERGRID and EGI

IBERGRID is a large EGI consumer

Open Access

11

➢ INCD infrastructure access classes○ Flagship research projects (strategic interest: roadmap)

○ Meritorious research projects (approved: FCT, peer review)

○ Consortium members (managed by the partners)

○ e-Government

○ Private companies (for R&D: SMEs, startups)

➢ In 2014○ ~3.5 million jobs (Grid)○ ~1.5 million CPU hours (non HEP)

■ General users Grid + HPC (22%)○ ~5.2 million CPU hours (HEP)

■ HEP users (Grid only)

Cloud Service Goals

➢ Cloud pilot infrastructure in the framework of INCD○ The aim is to provide a National Cloud service for the Portuguese

academic and research community

■ Support Infrastructure as a Service (IaaS): compute and storage resources

■ Support users applications integration, portals, community platforms

■ Integration in RCTS AAI

○ Integration in the EGI federated cloud

■ Federation of scientific and academic cloud sites at international level

○ Evaluate hybrid clouds

■ Evaluate the use of public cloud providers

■ Scale from INCD to commercial providers

■ Study network and interface aspects

12

Cloud Service II➢ Cloud pilot infrastructure deployment details

○ Openstack (Havana) on CentOS 6

○ Nova with KVM hypervisors (compute)

○ Neutron (networking)

○ Cinder (block storage) and Swift (object storage)

○ Glance (images upload supported)

○ Compute APIs: OCCI (open standard), EC2 (Amazon), Nova (Openstack)

○ Object Store APIs: S3 (Amazon), Swift (Openstack)

○ Keystone (User Authentication and Authorization)

■ Local users through username/password (PT users)

■ X.509 certificates, Virtual Organizations (EGI FedCloud)

○ Resource integrated in EGI Federated Cloud

○ Accounting, monitoring

13

Cloud Service III➢ Future production infrastructure (ongoing)

○ Openstack (Juno) on CentOS 7

○ High Availability of Openstack services

■ Load balancing, redundancy and failover (HAproxy)

■ Secure public access to Openstack APIs, and dashboard

○ Testing Ceph distributed filesystem

■ Block storagem

■ Object store

■ Image storage

○ Test Federated Authentication and Authorization Infrastructure (AAI)

■ Integration with SAML/Shibboleth, RCTSaai, EDUGAIN

■ Bridge with H2020 AARC: Géant, EGI, IGTF/EUgridPMA, REFEDS, FIM4R, RDA

○ Monitoring

■ Better accounting and monitoring (Ceilometer, Elastic Search, Flume, Kibana, ...)

14

Cloud Service IV➢ Future production infrastructure (ongoing)

○ Network

■ Improve virtual networks (Neutron)

■ IPv6

○ Special services

■ GPGPU

■ MPI / OpenMP

■ Linux Containers

■ Commercial software

○ Other

■ Scheduling, fair-share, priorities, etc

■ Orchestration (Heat)

15

Openstack Architecture

HORIZON dashboard web portal

KEYSTO

NE

Authentication

Authorization

NO

VAM

achine m

anagement

GLA

NC

ESystem

images

CIN

DER

Block storage

NEU

TRO

NN

etworking

SWIFT

Object storage

Users Applications Automation

API API API API API API

CLI Bindings / libsWeb browser

…

OS, EC2, S3, OCCI

TroveSaharaHeatCeilometer

APIs

EGI Federated cloud: Overview

➢ Federation of institutional private Clouds, offering Cloud Services to researchers in Europe and worldwide○ Enable resource provision targeted towards the research community

○ Scale to user needs

○ Resilience through integration of multiple different providers

○ Prevent vendor lock-in

➢ Standards based federation of IaaS cloud:○ Exposes a set of independent cloud services accessible to users utilising a

common standards profile

○ Allows deployment of services across multiple providers and capacity bursting

17

EGI Federated cloud: Architecture

18

EGI C

loud

Ser

vice

Mar

ketp

lace

EFI C

olla

bora

tion

tool

s

Providers Cloud Management Framework(Cloud middleware agnostic:

openstack, opennebula, WNoDeS, Syneffo, etc)

Instance Mgmt

EGI Cloud Infrastructure PlatformStorage Mgmt

Service Registry

Information Discovery

Federated AAI Monitoring Accounting

Helpdesk Support

Security Coordination

Training Outreach

Sustainable Business Models

OVF

EGI Core Platform

OCCI CDMI

GLUE2 GSI SAM UR

19

EGI Conference 2015http://conf2015.egi.eu/

Lisbon, 18-22 May 2015ISCTE

2x Xeon X5650 @2.67GHz 12 GBRAM ScientificLinux 5.5

Cloud - 16 GbRAM

Nº de processadores

Exemplo de aplicação: Hidrodinâmica e contaminação fecal no estuário do Tejo

Tem

po (m

inut

os)

- Modelo 3D acoplado circulação e evolução de traçador

- 20 níveis na vertical e a resolução espacial: entre 2 km e 30 m

Adiram ao piloto !http://www.incd.pt

Openstack: pros and cons➢ Advantages

○ Free, opensource, multi-tenant solution

○ Widely adopted in science and industry

○ Good interoperability and federation capabilities

○ Well supported, good documentation, active and dynamic community

○ Platinum members: Canonical, HP, AT&T, IBM, Intel, Rackspace, Red Hat, SUSE

○ Modular and flexible architecture with good scalability

○ Writen in Python can be easily adapted to specific requirements

○ Adopted by several NRENs for similar purposes

○ Supported in the European Grid Infrastructure FederatedCloud

○ CERN is adding 100 hypervisors p/ week aim is 15000 in Dec 2015

➢ Disavantages○ Configuration can be complex

21