“The Pacific Research PlatformTwo Years In”

Welcome and Overview Talk to the Pacific Research Platform “PRPv2” Workshop 2017

University of California, San DiegoFebruary 21, 2017

Dr. Larry SmarrDirector, California Institute for Telecommunications and Information Technology

Harry E. Gruber Professor, Dept. of Computer Science and Engineering

Jacobs School of Engineering, UCSDhttp://lsmarr.calit2.net

1

DOE ESnet’s Science DMZ: A Scalable Network Design Model for Optimizing Science Data Transfers

• A Science DMZ integrates 4 key concepts into a unified whole:– A network architecture designed for high-performance applications,

with the science network distinct from the general-purpose network

– The use of dedicated systems as data transfer nodes (DTNs)

– Performance measurement and network testing systems that are regularly used to characterize and troubleshoot the network

– Security policies and enforcement mechanisms that are tailored for high performance science environments

http://fasterdata.es.net/science-dmz/Science DMZCoined 2010

The DOE ESnet Science DMZ and the NSF “Campus Bridging” Taskforce Report Formed the Basis for the NSF Campus Cyberinfrastructure Network Infrastructure and Engineering (CC-NIE) Program

FIONAs and FIONettes – Flash I/O Network Appliances – Are DTNs:Linux PCs Optimized for DMZs over Distance

FIONAs Are Science DMZ Data Transfer Nodes (DTNs) &

Also Compute/Visualization/ML Nodes

Phil Papadopoulos & Tom DeFantiJoe Keefe & John Graham

FIONAS—40G, $8,000FIONette—1G, $1,000

The Pacific Research Platform’s Second Year:a Working End-to-End Science-Driven DMZ-Connector

FIONAs as Uniform DTN End Points

NSF CC*DNI Grant$5M 10/2015-10/2020

PI: Larry Smarr, UC San Diego Calit2Co-Pis:• Camille Crittenden, UC Berkeley CITRIS, • Tom DeFanti, UC San Diego Calit2, • Philip Papadopoulos, UCSD SDSC, • Frank Wuerthwein, UCSD Physics and SDSC

PRP Continues to Expand Rapidly While Increasing Connectivity:One Year of Progress – 12 Sites to 20 Sites

January 29, 2016 December 15, 2016

Connected 20 DMZs at 10G and 40G, demonstrating disk-to-disk GridFTP

at ~7.5G and 12.5G respectively, and 900Mb at 1G

Many PRP Science Engagement Workshops Were Held in 2016

• PRP Workshop Held in Collaboration with UC-Wide Research IT – May 1, 2016– 45 Attendees– Ten UC Campuses

See Talk by Camille Crittenden and Tom DeFanti

PRP’s First 1.5 Years: Connecting Campus Application Teams and Devices

100 Gbps FIONA at UCSC Connects the UCSC Hyades Cluster to the NERSC Supercomputer at LBNL

Supporting UCSC Remote Access to Large Data Subsets

of the Dark Energy Spectroscopic Instrument (DESI) and AGORA Galaxy Simulation Data

Produced at NERSC.

250 images per night

800GB per night

See Talk by Shawfeng Dong

UCSC Feb 7, 2017

40G FIONAs

20x40G PRP-connected

WAVE@UC San Diego

PRP Will Enable Distributed Virtual Reality

PRP

MerWAVE @UC Merced

See Talk by Jeff Weekley

PRP Will Link the Laboratories of the Pacific Earthquake Engineering Research Center

http://peer.berkeley.edu/

The Second FIONette was Deployed at the PEER Facility at UC Berkeley, and its Performance is Being Monitored

John Graham Installing FIONette at PEER Feb 10, 2017

SIO Researchers Beginning to UseBig Data Networks (Prism, IDI, PRP)

Jules Jaffe - Microscope Off Scripps Pier Frank Vernon - Expansion of HPWREN

Dan Cayan, Mike Dettinger Regional Downscaling of Climate Models

Scott Sellars, Marty RalphCenter for Western Weather and Water Extremes

PRP Backbone Sets Stage for 2017 Expansion of HPWREN, Connected to CENIC, into Orange and Riverside Counties

• PRP CENIC 100G Link UCSD to SDSU– DTN FIONAs Endpoints– Data Redundancy – Disaster Recovery – High Availability – Network Redundancy

• Anchor to CENIC at UCI– PRP FIONA Connects to

CalREN-HPR Network– Data Replication Site

• Potential Future UCR CENIC Anchor

UCR

UCI

UCSD

SDSU

Source: Frank Vernon, Greg Hidley, UCSD

PacificCity

Neptune Canada

45°N

47°30’N

130°W 127°30’W

N

SeattleGigaPOP

Portland

Possible PRP 2017 Expansion to Include NSF’s Ocean Observatory Initiative Fiber Optic SensorNets on Seafloor Off Washington

To PRP viaPacific Wave

Sea BottomElectro-optical Cable:

8,000 Volts10 Gbps Optics

Slide Courtesy, John Delaney, UWash John Delaney Visiting UCSD’s SIO For Three Months in 2017

Axial Volcano

140 Scientific Instruments

Being There - Remote Live High Definition Videoof Deep Sea Hydrothermal Vents

http://novae.ocean.washington.edu/story/Ashes_CAMHD_Live

Mushroom Hydrothermal Vent on Axial Seamount

1 Mile Below Sea Level

Picture Created From 40 HD Frames

14 Minutes Live HD VideoOn-Line Every 3 Hours

15 feet

Slide Courtesy, John Delaney, UWash

The Future of Supercomputing Will Blend Traditional HPC and Data Analytics Integrating Non-von Neumann Architectures

“High Performance Computing Will Evolve Towards a Hybrid Model,

Integrating Emerging Non-von Neumann Architectures, with Huge Potential in Pattern Recognition,

Streaming Data Analysis, and Unpredictable New Applications.”

Horst Simon, Deputy Director, U.S. Department of Energy’s

Lawrence Berkeley National Laboratory

Calit2’s Qualcomm Institute Has Established a Pattern Recognition Lab For Machine Learning on non-von Neumann Processors

Source: Dr. Dharmendra ModhaFounding Director, IBM Cognitive Computing Group

August 8, 2014

UCSD ECE Professor Ken Kreutz-Delgado Brings the IBM TrueNorth Chip

to Start Calit2’s Qualcomm Institute Pattern Recognition Laboratory

September 16, 2015

New Brain-Inspired Non-von Neumann Processors Are Emerging:KnuEdge Has Provided Processor to Calit2’s PRL

www.tomshardware.com/news/knuedge-announces-knuverse-and-knupath,31981.html

www.calit2.net/newsroom/release.php?id=2704

“KnuEdge and Calit2 have worked together since the early days of

the KnuEdge LambdaFabric processor, when key

personnel and technology from UC San Diego

provided the genesis for the first processor design.”

www.calit2.net/newsroom/release.php?id=2726

June 6, 2016

Proposed Cognitive Hardware and Software EcosystemOn the Pacific Research Platform

• Working With 30 CSE Machine Learning Researchers– Goal is 320 Game GPUs in 32-40 FIONAs at 10 PRP Campuses– PRP Couples FIONAs with GPUs into a Condor-Managed Cloud

• PRP Access to Emerging Processors– IBM TrueNorth, KnuEdge, FPGA, and Qualcomm Snapdragon

• Software Including a Wide Range of Open ML Algorithms • Metrics for Performance of Processors and Algorithms

Source: Tom DeFanti, Calit2

Multiple Proposals Under Review

FIONA with 8-Game GPUs

Expanding to National Research Platform and Global Research PlatformVia CENIC/Pacific Wave, Internet2, and International Links

PRP’s Current International

Partners

PRP Timeline

• PRPv1– A Routed Layer 3 Architecture – Tested, Measured, Optimized, With Multi-Domain Science Data– Bring Many Of Our Science Teams Up – Each Community Thus Will Have Its Own Certificate-Based Access

To its Specific Federated Data Infrastructure• PRPv2

– Incorporating SDN/SDX, AutoGOLE / NSI– Advanced IPv6-Only Version with Robust Security Features

– e.g. Trusted Platform Module Hardware and SDN/SDX Software– Support Rates up to 100Gb/s in Bursts and Streams– Develop Means to Operate a Shared Federation of Caches– Cooperating Research Groups

Our Support:

• US National Science Foundation (NSF) awards CNS 0821155 and CNS-1338192, CNS-1456638, ACI-1540112, and ACI-1541349

• University of California Office of the President CIO• UCSD Chancellor’s Integrated Digital Infrastructure Program • UCSD Next Generation Networking initiative• Calit2 and Calit2 Qualcomm Institute• CENIC, PacificWave and StarLight• DOE ESnet