artificial intelligence at the edge meets big data...

1

Artificial Intelligence at the Edge meets Big Data and HPC in the Cloud

Summer 2018

Pete Beckman, Nicola Ferrier, Charlie Catlett, Rajesh SankaranCo-Director Northwestern University / Argonne Institute for Science and Engineering (NAISE)

Argonne National Laboratory, Northwestern University, University of Chicago

2Pete Beckman [email protected]

2013


Example: SPECIM Camera:PFD VNIR with 768 bands(2734 x 1312) x 768 x 2bytes = 5.1GB image

1 sample every 5 minTwilight to twilight on June 21 = 1TB

We need a parallel computer with each sensor!


“Out on the edge you see all kinds of things you can't see from the center. [...] Big, undreamed-of things - the people on the edge see them first.”

- Kurt Vonnegut, Player Piano

5

2013 2014 2015

2016 2017

6

Sensors

PowerfulParallel Edge

Computing

Deep LearningTraining

Reduced, Compressed data

New inference (program code)

The Edge + Machine LearningA Revolution

HPC/Cloud

Edge computing and deep learning with feedback for continuous improvement

LIDAR

SoftwareDefinedRadios

Cameras

HyperspectralImaging

MicrofluidicSensors

Artificial IntelligenceDeep Learning Inference

SemanticOutput

Pete Beckman [email protected]

Servos

Actuators

Dynamicadaptation

7

PowerfulParallel Edge

Computing

Deep LearningTraining

Reduced, Compressed data

New inference (program code)

The Edge + Machine LearningA Revolution

HPC/Cloud

Edge computing and deep learning with feedback for continuous improvement

Artificial IntelligenceDeep Learning Inference

SemanticOutput


Facility

Servos

Actuators

Dynamicadaptation

8

Why Live on the Edge?

§ More data than bandwidth– Spallation neutron source, light source, HD Cameras, LIDAR, radar,

hyperspectral imaging, grid micro-synchrophasors, etc.§ Latency is important

– Quick local decision & actuation; adaptive sensing & control systems§ Privacy/Security requires short-lived data: process and discard

– Compromised devices have no sensitive data to be revealed§ Resilience requires distributed processing, analysis, and control

– Predictable service degradation, autonomy requires local (resilient) decision§ Quiet observation and energy efficiency

– Vigilant sensors, transmit only essential observations, not big data streams


9

• Borrowed BG/Q control system ideas• Designed mini “rack controller”

• Devices can be disconnected• Devices can be power cycled

• “Deep Space Probe” design• Heart beat signals to each device• Alternative boot image / safe mode• Current and voltage monitoring• Environmental monitoring

• Strict cybersecurity design

The Waggle Manager (WagMan)

Smoothing Out The Rough Edges

A Pocket-Sized Controller for Edge Computing


10

Waggle: Argonne’s Edge Computing Platform

§ Supports powerful, parallel computation at the edge– Computer vision and deep learning frameworks (Caffé, TensorFlow, OpenCV)– Supports edge-optimized & experimental computing

• ML hardware, GPUs, neuromorphic, FPGAs, etc.§ Open Source, open interfaces§ Integrating advanced sensors easy, with plug-in architecture§ Robust remote system management subsystem§ Manufactured at local electronics company

Bring Parallel Computing to the Edge

§ ~5 years of development by team at Argonne National Laboratory


13

The Array of Things Project is Deploying Hundreds of Waggle-based Nodes in Cities

§ 500 nodes will be deployed in Chicago§ Pilot Cities: Denver (Panasonic), Seattle, Portland, Palo Alto,

Detroit, Syracuse, Tokyo, Chapel Hill.§ 20+ other cities preparing for pilot projects§ Nodes have 2 cameras, one up, one down§ An instrument to understand urban issues

13

Rack of AoT nodes headed to Detroit

UChicago / National Science Foundation


PI: Charlie Catlett

14

Array of ThingsTeardown

Edge Computing

Sensor Pod

15

Initial 105 AoT node locations, showing that locations are selected in groups as part of specific science investigations

Current (red) and 60 of 100 additional planned (blue) AoT nodes. Both 1km and 2km buffers are shown, illustrating that even with 200 nodes over 95% of Chicago’s residents will live within 2km of a node and over 75% will live within 1km.

GIS map created by A. Laha, Center for Spatial Data Science, University of Chicago

•HPC: Training, Forecast, Optimization, Observation

•Edge: Inference, Actuation,Lightweight Learning

16

Size Nano Micro Milli Server Fog Campus FacilityExample Adafruit

TrinketParticle.io

BoronArray of Things

Linux Box Co-locatedBlades

1000-nodecluster

Datacenter

Memory 0.5K 256K 8GB 32GB 256G 32TB 16PBNetwork BLE WiFi/LTE WiFi/LTE 1 GigE 10GigE 40GigE N*100GigE

Cost $5 $30 $600 $3K $50K $2M $1000M

IoT/Edge HPC/CloudFogThe Computing Continuum


17AoT Image from Lake Shore Drive

Advanced computer vision to understand pedestrian movement, eventually to predict dangerous interactions with vehicles

Research Credits:Zeeshan Nadir (Purdue PhD Student @ ANL, 2017)Nicola Ferrier (ANL Scientist)

Mask r-CNN [He, 2017]trained on COCO dataset

Research Credits: Yongho Kim, Seongha Park (Purdue PhD Students @ ANL, 2018)Pete Beckman, Nicola Ferrier (ANL Scientists)


Transportation


Transportation

Funding: Illinois DOTPartners: Argonne, UChicago, Chicago Metropolitan Agency for Planning, Chicago DOT

Science: Prototype model of at-grade crossing with impact analysis (interrupt duration and impact; emergency vehicles delayed)

Objective: Prioritize among hundreds of at-grade crossings in context of $1B planned investments to improve rail throughput by eliminating key at-grade crossings.

Science: Predictive model for multi-modal optimization and control, integrating edge-AI capabilities with traditional transportation data, coupled with HPC models and control systems.

Deployment: Integrate transportation measurements from AoT/Waggle (density, flow, vehicle mix, parking) with live traffic data and traffic model around O’Hare International Airport.

Funding: EERE VTOPartners: Argonne, Chicago DOT, Chicago Dept. of Aviation, Chicago Dept. of Innovation and Technology, Arity

$3.2m

19

Live HPC Flood Modelingand Prediction?

Work with Aaron Packman and William Miller (Northwestern University)Cristina Negri, Rajesh Sankaran, Nicola Ferrier (Argonne)

50 consecutive frames to flood water and segment image

water

not-water

Using advanced computer vision to detect surface flooding

Research Credits:Ethan Trokie (Northwestern Undergrad Student @ ANL, 2017)Vivien Rivera (Northwestern PhD Student, SCGSR 2018)Nicola Ferrier (ANL Scientist)Rajesh Sankaran (ANL Scientist)

Hydrology: Flooding

Waggle Nodes in Tuley Park, Chicago

20

Live HPC Modeling

and Prediction…

Disaster: Flood FirePartnership with CSIRO Australia (MOU, visiting postdoc)

Impact analysis from ensemble wildfire simulations

Integration with PostGIS database

GIS visualisation

•Basis of D61 natural hazard applications:• Wildfire and wildfire impact (Spark) • Flood and coastal inundation (Swift)

Nikhil Garg

21

Original Auto SegmentationUsing advanced computer vision to monitor plantsResearch Credits:Renee Zha (Northwestern Undergrad Student @ ANL, 2017)Zeeshan Nadir (Purdue PhD Student @ ANL, 2017)Nicola Ferrier (ANL Scientist)

Undergrads Caeley and Jordan developed soil moisture sensor now deployed in Chicago

Research Credits:Vivien Rivera (Northwestern Univ. PhD Student, 2018)Aaron Packman, Bill Miller (Northwestern Univ. Professors)Pete Beckman (ANL Scientist)

Advanced sensors and computer vision to monitor pristine prairie

Earth Modeling: Ecosystem Response

Chicago Botanic GardenConservation Science Center

22

It’s a Bird! It’s a Plane! No… It’s a Drone!

Object Classification

Advanced computer vision and machine learning to identify drones, birds, or fixed-wing aircraft.

Research Credits:Sean Richardson (USAF visiting ANL)

Adam Szymanski (ANL Scientist)

National Security

23

not-water Benchtop integration of DARPA-funded radiation sensor (Kromek-D3S)

National Security

Testing Edge / WaggleDARPA SIGMA+


24

DOEARPA-E

Energy: Power Grid

• Load Forecasting• Grid Stress• Air Quality

CRADA Approved!!

25

Computational Forecasting

time

data assimilation

Forecast at 6h

data assimilation Forecast

at 12h

0h 6h 12h 18h 24h

Forecast at 0h

Domain setup for HPC forecast

Earth Systems: Wind & Solar

Research Credits:Emil Constantinescu (ANL Scientist)

Solar forecasts and measurements

Wind forecasts and measurements

DC Forecast

26

8 10 12 14 160

50

100

150

200

250

300

350

400

450

Time [hour of the day]

Irrad

iatio

n [w

/m2 ]

ClimatologyObs for calibrationNWPObservationsForecast

Edge to HPC: Live & Historical Data for Forecasting

Conceptual Edge solar forecast

Validation observations not used in inference

Numerical forecast

Edge observations used in inferenceHistorical data

Edge forecast

y⇥|X,X⇥,y = m(X⇥) + K21 (K11 + )−1 (y m(X))

GP & DNNs

Earth Systems: Wind & Solar

Research Credits:Emil Constantinescu (ANL Scientist)

27LLZO transition

Research Credits:N. Ferrier, J.Libera & S. ChaudhuriMaterials Engineering Research Facility, ANL

• Use data collected to date to develop ML/DL models • Relate process parameters to output measures• Optimize

Flame Spray Pyrolysis

Manufacturing

28

Adaptive sampling of the atmosphere• Atmosphere sensing radars have a

wide range of configurations. • Ideal configuration depends on

• Atmospheric scene:• hurricane, supercell, etc.

• Phenomena of interest:• clouds, tornadoes, birds, bugs

• AI@Edge needed to identify scene• Automated slicing and dicing to

reconstruct spatial structure using machine learning.

Operational NOAA Radar

DoE Research Radar

Research radar slice

Atmospheric Science

Research Credits:Scott Collis, EVS Division, ANL

New NSF Funding

29

Real-time reconstruction of a shale sample. The scanning pattern and voxel coverage affect the reconstruction quality.

Imaging aluminum foam (dynamic features) sample. Data acquisition and analysis parameters have significant affect on quality of reconstructed feature.

Data rates for APS-U will increase several orders of magnitude

Current Experiments:Times vary: (from seconds to days/weeks)

• Many experimental parameters need to be optimizes

• Data analysis happens after experiment is finalized

Data collection is in the dark

• Parameters are guessed (experience) and then optimized

(repeated experiments)

Edge Computing and Experimental Steering• Improving the science and the efficiency of the experiments

• Real-time data analysis and feedback, data verification,

correction, normalization, and configuration parameter

optimizations

Facilities: Light Source

Research Credits:

Tekin Bicer, ANL

30

Investments in AI Hardware are Accelerating ChangeAt BOTH ends of continuum….

Intel Myriad NVIDIA TX2

Missing: The programming framework for Edge-HPC Science July

2018“Edge TPUs are designed to complement our Cloud TPU offering, so you can accelerate ML training in the cloud, then have lightning-fast ML inference at the edge. Your sensors become more than data collectors — they make local, real-time, intelligent decisions.”

GoogleEdge TPU

FPGA


Are We Building a “Software Defined Instrument”?

32

Harnessing The Computing Continuum

ExistingResources& Services

ContinuumAbstract Model & Runtime

Goal-orientedAnnotations

NotionalExample:

Science-drivenProblems

e.g.: “Predict urban response to rainfall, trigger intelligent reaction…”

trigger {flood_actuation, resident_warning} when {wx_prediction, sewer_model} implies

(traffic_capacity < 70%) or (home_flooding > 5%)

33

Edgy Research: Edge-HPC


§ Continually improving Edge-HPC Systems– Deep learning + lightweight training + continual improvement– Incremental model updates– Is Edge really a layer in the model?

§ How will the OS/R and system software evolve for Edge-HPC?– Scheduling, security, resource management, streaming data

§ Programming model & framework for Continuum Computing§ Optimized ML hardware for both Edge & HPC§ Theoretical foundations for failures and correctness of edge/training§ Dynamic resource management and adaptive inference priority

– AI at the Edge is limited by power and computation – just like HPC§ Fluid HPC to support complex and on-demand workflows on future exascale

34

Questions?

http://www.wa8.gl

http://arrayofthings.github.io

artificial intelligence at the edge meets big data...

Documents