hpc over cloud - 광주과학기술원 슈퍼컴퓨팅센터 · pdf filehpc over cloud july...
TRANSCRIPT
SCENT
HPC over Cloud
July 16th, 2015
HPC Summer School @ GIST
SCENT (Super Computing CENTer)
GIST (Gwangju Institute of Science & Technology)
Dr. JongWon Kim [email protected]
SCENT
Interplay between Theory, Simulation,
and Experiment
SCENT- 3 -
Cloud & Software-Defined
Infrastructure
SCENT
• T-Shaped vs I-Shaped Development
• Software-defined X (Infrastructure, Interface, Economy, ..)
Open Innovation Platform
Open Networking
Fast, Cheap,
Performance Fast,
Fast, Fast
SCENT
Cloud &
Software-Defined
Infrastructure (SDN/NFV/Cloud Integration)
• Amazon AWS
• Microsoft Azure
• Google Cloud Engine
• VMware Hybrid
• OpenStack (IBM, HP, Cisco,
RackSpace, …) + Network
Operator Cloud (AT&T.
Verizon, NTT, …)
Cloud DC Traffic
SCENT
Human-Defined Services over
Software-Defined Infrastructure
Open Platforms
(Tools)
Open
Software
(Services)
Open Infrastructure
(Resources)
Big
Data
Industrial
Internet
of Things
Open Data
Cloud
Open API
Software-Defined Infrastructure (SDN/NFV integrated Mobile+Cloud)
6
SCENT
Converged
Software-Defined
Infrastructure (SDN/NFV/Cloud
Integrated)
7
SCENT
• Nutanix, Simplivity, …
• Pluribus Freedom Server-Switch
• VMware Project Mystic Evo:
Rails, Evo: Racks
Hyper-Convergence
Boxes
SCENT
SmartX Box: Inter-Connected
Functions inside Boxes/Sites
Box Functions
Inter-Connect
Site
Resources – Workloads - Services
SCENT
SmartX Box: Inter-Connected Functions inside Boxes/Sites
Secured
Boxes
Functions
Inter-Connect
Site
P+M, C, D ports
Normal Secured Tag/Steer/Mapped
Discover & Connect
Secured VMs, Containers, VNFs
Configuration Visibility Control
SCENT11
SDN
IoT
Cloud
NFV
SDI & Open-Source
SW/HW
SCENT
SmartX Provisioning & Orchestration: Converged Resources - Workloads – Diversified Functions
Zero-touch Configuration
Flexible Control
Instant
Visibility
Auto Scaling
Continuous
Integration
Service
Chaining
SCENT- 13 -
Realize Smart Services with
Open APIs enabled by
SmartX Boxes
Architecture Your Smart Things with API Tools
Prepare Programma
ble & Virtualized Resources
with Hyper-converged
SmartX Box
Build Open APIs with Inter-connected Functions
SCENT
Cloud & PaaS
(Platform as a
Service)
• Model-driven PaaS: domain
specific languages
• PaaS (Products; Frameworks): aPaaS (App Server aaS) and
language runtimes / DBaaS
• Mini-PaaS (Foundational PaaS,
IaaS+): App. Containers + α
15 Dzone Cloud Report 2014
SCENT
Super
Computing
Computing
Cloud
HPC over
Cloud
Higher
efficiency
Workload
bursting
Pay-per-use
Scalability Simplified
self-service
HPC over
Cloud
Benefits
Sophisticated
Migration
policies
Interfacing
With
External
components
Accelerated
Collaboration
HPC over Cloud: Target & Benefits
Economic (Easy & Improved Sharing), Green (Energy-efficient)
SCENT
HPC over Cloud: HPC, HSC, …
http://www.cloudscaling.com/blog/cloud-computing/grid-cloud-hpc-whats-the-diff/
• High Performance Computing: Single program
• High Scalability Computing: Throughput focus; Can be distributed; e.g. Physics Simulation
SCENT
Technical Computing
Infrastructure (Resource)
Software (Service)
Academic
Computing
Shared HPC Resources
Technical
Computing
Me
ta-O
pe
rati
on
HPC Services
F3 Racing Team = SC Center
Dedicated
computing
Dedicated
Resources
HPC over Cloud Platform (Tool)
19
SCENT
Institute HPC Centers
National HPC Cloud Centers
(Shared Infra/Platform)
HPC over Cloud
HPC SW Services
(Licenses) CPU/GPU-intensive
Memory-intensive
Data-intensive
Virtualized HPC Clusters
Dedicated HPC Centers
SCENT
Building Shared National HPC
Resources (every 6 years)
HPC Cloud Center #1
HPC Cloud Center #2
HPC
Cloud Center #3
HPC Cloud
Center #4
#5 #6
국가
지방
Coordinated and Operated by National HPC Center
기업
SCENT
Vision for National HPC Services (Human Resources instead of Hardware Resources)
Software
Platform
Infrastructure (Hardware)
Software
Platform
Infrastructure (Hardware)
22
SCENT
Diversified HPC Services over Integrated &
Reconfigurable Super Computers
Reconfigurable
(Software-Defined)
HPC Platforms
Diversified &
Portable
HPC Software
Open & Convergent
HPC Resources
Open & Common API
Data-Intensive
Compute-Intensive Hybrid
Cloud-Ready HPC
23
SCENT- 24 -
DevOps
Automation
SCENT
Development + Operation
OpenStack (Puppet, Chef, …)
DevOps for Software-Defined
Infrastructure
SCENT
Computers in
Data Centers
- 26 -
SDI & Rack Scale Computing
SCENT
Cloud Datacenter as a BIG Computer
- 27 -
• Luiz André Barroso, Urs Hölzle, “The Datacenter as a
Computer: An Introduction to the Design of Warehouse-
Scale Machines”
• Google Omega & Apache Mesos
SCENT
Cloud Datacenter as a BIG Computer
Distributed Resource Scheduling
& Containers
28
SCENT
HPC over Cloud
Examples Cloud-
Ready HPC
M&S Service Market Place
30
SCENT
Data-
Intensive
Compute-
Intensive Hybrid
Open & Convergent
Shared HPC Resources
Diversified & Portable
HPC M&S Services
Open & Common API Reconfigurable
HPC over Cloud Platforms
Materials/Biomedical Deep Learning (GPU) CAE
HPC over Cloud 모델에 따른 Technical Computing Services
31
SCENT
HPC over Cloud Examples: CERN /
Rackspace OpenLab
• OpenLab: Hybrid Cloud Model based on OpenStack (open-source cloud OS)
COMPUTE NETWORKING
STORAGE
Rackspace Public Cloud
CERN Private Cloud
Rackspace Private Cloud
@ CERN
Demonstration of federated identity and aggregated services between a Rackspace Private Cloud at CERN and at least one other cloud.
SCENT
HPC over Cloud Examples: MegaRun
• Cycle Computing MegaRun: Schrödinger’s quantum chemistry software on 156,000 cores across 8 AWS regions (to sort through 205,000 compounds for solar panel). (Nov. 2013)
• $33,000 / 18 hours vs $68 million / 18 hours; $132,000 / 10 months (existing internal 300-core cluster)
SCENT
HPC over Cloud Examples: CloudFlow
Integrates
computational
services in the
Cloud into the
engineering
workflows of
manufacturing
SMBs
SCENT
HPC over Cloud Examples: CloudFlow
SCENThttps://www.theubercloud.com
workstation
HPC server
HPC over Cloud End-User Portable Units are like containers
Simulation, CAE
HPC over Cloud Examples: UberCloud
36
SCENT
HPC over Cloud Examples: Academic
Community Cloud & Overlay Cloud (Japan)
SCENT
Public Cloud Services (EU)
SCENT- 39 -
SCENT
Control Tower & Access Center
Resource
Provisioning
for
S&T R&D Automated
Experiments S&T R&D
Portal
Multi-user Support
Configuration,
Control, Visibility
A Virtual
Playground
for
Virtual HPC
Ops
Dev
Virtual BigData
Virtual Presence
DevOps
Science and Technology R&D Cyber-Lab with
SDN-Cloud & HPC/DataFarm Playground
HPC & DataFarm
SDN-Cloud
SCENT
SmartX Software Framework
Zero-touch Configuration
Flexible Control
Multi-level Visibility
Provisioning
Orchestration
BigData / FastData
Intelligence
Automated Scaling / Chaining
Policy- driven
Orchestration
Intelligence
Visibility
SmartX Automation
