from next gen cloud research to future campus …...2016/10/04 · cloudlab updated: 9/28/16...

updated: 9/28/16 CloudLab

Kuang-Ching (KC) Wang

Associate Professor, Networking CTO

Director, Center of Excellence for Next Gen Computing & Creativity

From Next Gen Cloud Research

To Future Campus Computing Infrastructure


The CloudLab Team

Robert Ricci (PI)Eric EideSteve CorbatóKobus Van der Merwe

Aditya Akella (co-PI)Remzi Arpaci-DusseauMiron Livny

KC Wang (co-PI)Jim BoKumJim Pepin

Chip EllioK (co-PI)Larry Landweber

Mike Zink (co-PI)David Irwin

Glenn Ricart (co-PI)

2

WINLAB ORBIT

CloudLab Now Speaks Wireless!!

Suman Banerjee

updated: 9/28/16 CloudLab 3

Clouds are great but have “well known” constraints

The line between can’s and cannot’s …?

Clouds are around and gaining importance by day

Lots of opportunities for research


CloudLab enables research on the future of cloud computing architectures and the new applications they enable


• A “meta-cloud” for building clouds•  Build your own cloud on our hardware resources• Agnostic to specific cloud software

•  Run existing cloud software stacks (like OpenStack, Hadoop, etc.)•  … or new ones built from the ground up

• Control and visibility all the way to the bare metal•  “Sliceable” for multiple, isolated experiments at once

The CloudLab Vision

With CloudLab, it’s as easy to get an entire cloud as it is to get a VM in a cloud

5


DIBBs,Wireless,

OtherInstruments

Focus Today – What CloudLab Means to … ?

7

CloudResearchers

Industry

OtherClouds

Synergy Value Ecosystem Evolution

DomainResearchers

UniversityIT


What Is CloudLab?

Utah Wisconsin Clemson Federated Facilities

Slice B

Stock OpenStack

CC-NIE, Internet2 AL2S, Regionals

Slice A

Geo-Distributed Storage Research

Slice D

Allocation and Scheduling Research for Cyber-Physical Systems

Slice C

Virtualization and Isolation Research

8

A place to build your own cloud


•  Built on Emulab and GENI (“ProtoGENI”)•  In active development at Utah since 1999•  Several thousand users (incl. GENI users)•  Provisions, then gets out of the way

•  “Run-time” services are optional

• Controllable through a web interface and GENI APIs

•  Scientific instrument for repeatable research•  Physical isolation for most resources•  Profiles capture everything needed for experiments

•  Software, data, and hardware details•  Can be shared and published (eg. in papers)

Technology Foundations

9


Many Sites, One Facility

10

= GENI racks


CloudLab’s Hardware One facility, one account, three locations (+ more!)

Wisconsin Clemson Utah

•  About 5,000 cores each (15,000 total)•  8-16 cores per node•  Baseline: 4GB RAM / core•  Latest virtualization hardware

•  TOR / Core switching design•  10 Gb to nodes, SDN•  100 Gb to Internet2 AL2S•  Partnerships with multiple vendors

•  Storage and net.•  Per node:

•  128 GB RAM•  2x1TB Disk•  400 GB SSD

•  Clos topology•  Cisco and HP

•  High-memory •  16 GB RAM / core•  16 cores / node•  Bulk block store•  Net. up to 40Gb•  High capacity•  Dell

•  Power-efficient•  ARM64 / x86•  Power monitors•  Flash on ARMs•  Disk on x86•  Very dense•  HP

11


• At Rutgers, CloudLab will integrate with ORBIT• At Utah, CloudLab will integrate with PhantomNet, a

mobile network testbed• At Wisconsin, CloudLab will integrate with outdoor

wireless from LTE to SDRs• Once in place, CloudLab can support research such as:

•  Repeatable, combined mobile/cloud experimentation•  Cloud-RAN•  mobile cloud•  cloud offloading•  5G•  …

And Wireless

12


Profiles: Packaged environments

13

Your Software,

Data, Compute, & Network


Creating Profile via GUI

14

GENI RSpec


Creating Profile via Code

15

#!/usr/bin/env python """An example of constructing a profile with a single Xen VM. Instructions: Wait for the profile instance to start, and then log in to the VM via the ssh port specified below. (Note that in this case, you will need to access the VM through a high port on the physical host, since we have not requested a public IP address for the VM itself.) """ # Import the Portal object. import geni.portal as portal # Import the ProtoGENI library. import geni.rspec.pg as pg # Create the Portal context. pc = portal.Context() # Create a Request object to start building the RSpec. rspec = pg.Request() # Create a XenVM and add it to the RSpec. node = pg.XenVM("node") rspec.addResource(node) # Print the RSpec to the enclosing page. pc.printRequestRSpec(rspec)


• CloudLab can be used with a GENI account, and vice-versa• GENI Racks: ~ 50 small clusters around the country•  Programmable wide-area network

•  Openflow at dozens of sites•  Connected in one layer 2 domain

•  Large clusters (100s of nodes) at several sites• Wireless and mobile

•  WiMax at 8 institutions•  LTE / EPC testbed (“PhantomNet”) at Utah

•  International partners•  Europe (FIRE), Brazil, Japan

Federated with GENI

16


CloudLab Users So Far

17

May 2016: 300 projects 1,250 users 21,000 experiments


•  Data center networking•  Intra- and inter-data center

•  Genomics, near-real-time GPU-based remote surgery, …

•  Network function virtualization•  Virtual machines, containers

•  Database

•  Parallel computing

•  Storage

Reported Research Use of CloudLab

18

•  Edge cloud

•  Cybersecurity

•  Resource allocation

•  Web retrieval

•  Cyber physical systems

•  Robotics

•  Smart grid

•  …


•  Enables cloud architecture & application research by•  providing the latest, bleeding edge hardware feature•  providing deep programmability on network, compute, storage•  plugging in researcher provided specialized compute devices (e.g.,

FPGA/GPUs, wireless systems acquired from other grants)•  All these come with beKer isolation (slices), repeatability (profiles)

•  Enables novel computing methods research for domain sciences•  Exploring benefits of novel compute methods with CloudLab

features, to inform creation of next gen production infrastructure.•  Focus is on new architecture at modest scale, not compute cycles

CloudLab To Researchers

19


•  (Level 1) For CloudLab sites, IT is heroic when it comes to hosting CloudLab, being•  Innovative, flexible, and agile to changes•  A partner in both design and operation

•  (Level 2) Bootstraps IT innovations•  Seeing CloudLab as a sandbox, for exploring “cloud strategies”,

“new features” (in collaboration with researchers*)•  Seeing CloudLab as a model for campus computing infrastructure

•  (Level 3) Works with ACI-REF to reach out to researchers

CloudLab To IT

20


•  For vendors, is a place to introduce novel capabilities by partnering with CloudLab•  HP, Dell, Cisco were the core partners in phase I•  Startups offering/developing new solutions for CloudLab•  SBIR partnership

•  For industry research labs, is a place for innovative research•  E.g., Dell Research partnered with Clemson researchers to research

scalable OpenStack architecture for Smart Grid

•  Is a bridge between vendors and broad research communities•  Communicating new vendor features that have the potential for novel

research use•  Communicate researcher needs to vendors to drive new features

CloudLab To Industry

21


Other clouds can include national and institutional research clouds, commercial public clouds, ... •  Is created for a DIFFERENT, complementary purpose

•  To enable research on new cloud architecture and applications•  NSF CloudLab is a TESTBED for research•  While other clouds serve production computing needs

• Can shed light on future production cloud design choices

• Can be used to build sandboxes for pre-production dev

CloudLab To Other Clouds

22


• Across domain sciences, increasing focus on novel integration of data and compute, as DIBBs (Data Infrastructure Building Blocks)•  CloudLab has worked with NSF DIBBS project to explore

innovative approaches for new science workflows

•  Future Internet foresees increasing integration between edge infrastructure and computing backend•  Smart grid, wireless, …•  CloudLab is a perfect enabler for prototyping novel approaches for

feasibility studies and architectural experiments

CloudLab To Other DIBBS & instruments

23


• CloudLab mission is to enable researchers’ research

• CloudLab is NOT about speed and feed, but it IS about newest and coolest technology components from Industry

• CloudLab is designed and run by a partnership among researchers, IT, and industry. All parties are learning and contributing knowledge and ideas.

To CloudLab, the Ecosystem is the Core

24


To Go From Here … 25

OtherDIBBs,

Instruments

CloudResearchers

Industry

OtherClouds

Synergy Value Ecosystem Evolution

DomainResearchers

IT


Learn more, sign up, share your research:

www.CloudLab.us

This material is based upon work supported by the National Science Foundation under Grant No. 1419199. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

26

from next gen cloud research to future campus …...2016/10/04 · cloudlab updated: 9/28/16...

Documents