ubiquitous machine learning - o'reilly media...module (gmm) efm extracts data from sensors,...

Han Yang, PhD, @hanyang1234 Senior Product ManagerSeptember, 2018

Ubiquitous Machine Learning

© 2 0 1 7 C i s c o a n d / o r i t s a f f i l i a t e s . A l l r i g h t s r e s e r v e d . C i s c o C o n f i d e n t i a l

How Important is AI & ML?

By 2035, AI technologies are

projected to increase business productivity

by up to

40%

By 2020, insights-driven businesses will steal

$1.2Tper annum from their less-informed peers

8 out of 10businesses have already

implemented or are planning to adopt AI as

a customer service solution by 2020

• $1.2T https://go.forrester.com /w p-content/up loads/Forrester_Predic tions_2017_-Artific ia l_ Inte lligence_W ill_Drive_The_Insights_Revo lution.pdf

• 8 out o f 10 - O rac le - https://w w w .orac le .com /w ebfo lder/s/de livery_production/docs/FY16h1/doc35/C XResearchV irtualExperiences.pdf• Accenture https://w w w .accenture .com /us-en/ins ight-artific ia l- inte lligence-future-grow th

https://go.forrester.com/wp-content/uploads/Forrester_Predictions_2017_-Artificial_Intelligence_Will_Drive_The_Insights_Revolution.pdf

© 2 0 1 7 C i s c o a n d / o r i t s a f f i l i a t e s . A l l r i g h t s r e s e r v e d . C i s c o C o n f i d e n t i a l© 2 0 1 7 C i s c o a n d / o r i t s a f f i l i a t e s . A l l r i g h t s r e s e r v e d . C i s c o C o n f i d e n t i a l

Relative Change in Cash Flow by AI Adoption Cohort

40 McKinsey Global Institute Notes from the frontier: Modeling the impact of AI on the world economy

The simulation finds that front-runners could increase economic value (economic output minus AI-related investments and transition costs) by 122 percent by 2030, or an implicit additional growth rate of cash generation of about 6 percent a year for the next 12 years. Achieving and sustaining this rate of growth over such a long period would be remarkable, as it conflicts with Gibrat’s Law that a firm’s growth rate is independent of its size.93 The analysis suggests that cash generation is likely to accelerate after a five-year period in which front-runners could experience cash outflows as they invest in, and scale up, AI. Front-runners tend to slowly concentrate the profit pool of their industry in a winner-takes-all phenomenon. This may lead to the phenomenon of increasing concentration and the rise of “superstar” firms. Some researchers argue that technology could enable superstars to pull away from the pack, but also that a slowdown in the diffusion of technology could

93 Yoshi Fujiwara et al., “Do Pareto-Zipf and Gibrat laws hold true? An analysis with European firms,” Physica A: Statistical Mechanics and Its Applications, 2004, Volume 335, Issue 1.

Exhibit 13

Faster adoption and absorption by front-runners can create larger economic gains for these companies.

SOURCE: McKinsey Global Institute analysis

SIMULATION

122

10

-23

30

-30

10

-20

-10

0

20

40

50

130

60

70

80

90

100

110

120

Laggard(do notabsorbby 2030)

252017 20 2030

Front-runners(absorbingwithin first5 to 7 years)

Follower(absorbingby 2030)

Relative changes in cash flow by AI adoption cohort% change per cohort, cumulative

Front-runner breakdown % change per cohort

82Economy-wideoutput gains

Output gain/lossfrom/to peers 135

Transition costs -18

-77Capitalexpenditure

122Total

Total

Economy-wideoutput gains 11

49Output gain/lossfrom/to peers

4

Capitalexpenditure 19

Transitioncosts

23

Laggard breakdown % change per cohort

Global averagenet impact = 16

NOTE: Numbers are simulated figures to provide directional perspectives rather than forecasts.

https://w w w .m ckinsey.com /featured- insights/artific ia l- inte lligence/notes-from -the-frontie r-m ode ling-the- im pact-of-ai-on-the-

w orld-econom y?c id=other-em l-alt-m gi-m ck-oth-1809&hlk id=5ebe957bb3594f96bedda5695e4664fd&hctky=10366723&hdpid=677435cb-04b0-445e-afba-4588aa47d2fe


Data is Exploding Everywhere

1.3 Zettabytes of Data in Data

Centers5.9 Zettabytes of Data Stored

in Devices

847 Zettabytes

of Data Created

Everywhere

https://w w w .c isco.com /c/en/us/so lutions/co llateral/serv ice-provider/g lobal-c loud- index-gc i/w hite-paper-c11-738085.htm l

By 2021


5181

124

179

272

403

0

50

100

150

200

250

300

350

400

450

2016 2017 2018 2019 2020 2021

Big Data ForecastBig Data Volume Grows Nearly 8-FoldBig Data Will Represent 30% of All Data Stored in Data Center by 2021

51% CAGR 2016–2021

Exab

ytes

Source: Cisco Global Cloud Index, 2016–2021



2231

40

52

67

85

7 9 12 14 17 21

0102030405060708090

2016 2017 2018 2019 2020 2021

Us eable Data Created per YearData Center Traffic per Year

Data Created vs. Data Center TrafficData Created Outpaced

Zettabytes per Year

Opportunity for Edge or Fog Computing

Source: Cisco Global Cloud Index, 2016–2021



US Download Speeds Distribution—2017

Text and IM A ugm ented R eality G am ing Virtual R eality S tream ing Three C oncurrent Sam ple A pps

D ow nload (D L) Speeds in M bps

10th 20th 30th 40th 50th 70th 80th 90th60th

No

. of

Sp

eed

Tes

ts

0

0. 5

1

1. 5

2

2. 5

0 10 20 30 40 50 60 70 80 90 10 0 11 0 12 0 13 0 14 0 15 0 16 0

Mill

ion

s Median 33.3 Mbps

Mean 46.2 Mbps

Percentiles

8 Min to Load 2GB Model



0

10

20

30

40

50

60

70

80

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

Th

ou

san

ds

S. Africa Download Speeds Distribution—2017

Text and IM A ugm ented R eality G am ing Virtual R eality S tream ing Three C oncurrent Sam ple A pps

D ow nload (D L) Speeds in M bps

No

. of

Sp

eed

Tes

ts

Percentiles

10th 20th 30th 40th 50th 70th 80th 90th60th

Median 4.6

Mbps

Mean 7.7

Mbps

58 Min to Load 2GB Model



1.3 Zettabytes of Data in Data

Centers5.9 Zettabytes of Data Stored

in Devices

847 Zettabytes

of Data Created

EverywhereAnalytics Need to Follow Data

Data is Exploding Everywhere

By 2021



Analytics at the Edge


Edge Analytics in Industrial IoT

of large enterprises will be integrating edge computing into their 2021 projects

40%

of workload deployment has latency & BW requirements

of large enterprises will use edge locations by 2021 driven by interactive UI

30%Oil and Gas

Many drilling platformsLow BW uplink (satellite)

Need tiered data evaluation

RoadwayThousands of road signs

Widely distributedNeed local data evaluation

FactoriesLegacy systems and protocols

High data volume collected from machinesSecurity and data privacy

OT data analysis & policy at factory levelNeed local data normalization, storage, & processing

30%

* Source Gartner


Edge Analytic Use Cases

There may not be enough network bandwidth

The use of data may be at the edge

Most of the data is not interesting

Computation can be optimized for some purposes

Data normalization

Data redirection based on the content of the data

Data time stamping for later forensic analytics


IoT World Forum Reference Model

Credit to: Jim Green, CTO Cisco Kinetic & Chairman IoT World Forum


Cisco Kinetic – the IoT data management platform

Move dataprogrammatically to the desired destination with the user defined policy

Analyze dataTransform data, apply rules, perform distributed micro-processing

Connect devices and extract data

Clouds andon prem apps

Cisco Kinetic

Cisconetwork

IoT devices


How the different Kinetic components interactSensors & Endpoints Applications

Edge Devices (IE4k, GW, Compute)

EnterpriseData Centers

Fog Compute

Cloud Apps

Edge Fog Manager

Fog & DC EFM adds additional compute capabilities, including normalization, aggregation, storage and visualization

2

DCM adds secure cloud connection capabilities

3GMM enables mass deployment and management of gateways

4

Edge Fog Manager

Gateway Management Module (GMM)

EFM extracts data from sensors, transforms it and sends it either to another IoT device,

1

or to a hosted app

Edge Fog Manager

Edge Fog Manager

Edge Fog Manager

Note: these components are independent and rarely all used in one deployment

On-prem or private cloud apps

Edge Fog M anager

Data Control Module (DCM)

Edge Fog Module


Secure connection

• Secure connection of IoT Devices • Simplified GW management at scale

• Secure IoT app lifecycle management

Cisco Kinetic

APPconfig


How does it work

1 Gateway secure boot

2 Secure onboarding

3 Secure gateway Provisioning

4Fog App (Cisco or 3rd party) download & life cycle management

5

5 Secure connectivity ofsensors

6Data gathering, processing, secure delivery to apps

Gateway1

23

4

010101010101010101

010101010101010 0101010101010101010101010101010100101010

10101010101010

10 0101010101010101010101001010101

0101010 101010101010101

101010101010101010

6

Gateway Provisioning

Partner

Fog

Fog App

DC/Cloud

Cisco / 3rd Party Applications

LoRa™ Network Features

Low Cost � Minimal infrastructure � Low cost end-node � Open SW

Long Range � Greater than cellular � Deep indoor coverage � Star topology

Max Lifetime � Low power optimized � 10-20yr lifetime � >10x vs cellular M2M

Multi-Usage � High capacity � Multi-tenant � Public network

Semtech Proprietary and Confidential 12

Field Area Network (Wi-SUN)

AM I sm art m eteringD istribution autom ationStreet lightingO &G w ellhead m onitoringW ater/w astew ater

Low Power Long Range Wireless (LPWA – LoRA)

SP IoT InfrastructuresBattery pow ered sensorsEnvironm ental m onitoringSm art C ities, parking, and AgricultureSP cell tow er m onitoring

Remote Asset Monitoring

Pipeline m onitoringR oadside infrastructureD istribution autom ationATM sD igita l S ignage

Fleet VehiclesMass Transit

Autom ated Vehic le Location tracking, D ata U ploaded in Seconds w ith 4G /LTEH andles M ultip le W ire less Laptops, Sm artphones, Tablets S im ultaneously

Premium Mobile Broadband (PMB)

Public safety and security C PE

CGR1000SD K IR500 IR910

IR8x9 + LoRA

Modem(future)

819H IR829IR809

IOT Field Network Director/Industrial Operations Kit – Zero Touch Provisioning, Firmware upgrade,… IOT Software Platform – Fog Computing, BYOI,…

819HIR829

IR809819H

Investment with Multiple Options


• Extract data from diverse, distributed devices

• Tiered microservices from edge to DC, enabled by flexible, distributed service architecture

• High performance data historian

• Support real—time data visualization

• Intuitive data policy editor

Extract and compute the data

ParStream(High-speed Historian)

Low footprint, high performance

Message BrokerDistributed services

architecture (DSA & Links)

Intuitive visual Data Policy Editor

System monitoring & Configuration

Protocol connectors

Data CenterFog Nodes Fog Nodes

Edge NodesDevices


Machine Learning the Data Center


Key ChallengesMassive & Active

Data Sets

Volume, Velocity, and Variability of AI Workloads at Scale Demand New Data Center Architectures

Uncharted Territory

Rapidly Evolving AI/ML Ecosystem and

Requirements; Skill Shortages in Data Science and IT

The Data Center Follows the Data

Distributed Data Sources and Technologies Risk Operational

Silos and Complexity


Eliminating Operational Silos Demystifying AI/ML StacksCurating top-to-bottom SW and HW stacks with leading ecosystem partners to ensure a faster and more predictable deployment

Full array of accelerated computing options for test/dev, training and inference, all unified by cloud-based management

Integrating changing data sources as part of a dynamic data pipeline

Powering the Full AI Data Lifecycle

Cisco Computing Solutions for Machine Learning


Inference in Regional and Micro Data Centers

Cisco AI/ML Computing Platforms, Partners, and Solutions

UCS C220 Servers

UCS C240 ServersHyperFlex GPU Nodes

Test/Dev and ML in Private Cloud

Deep Learning in DC Core

Accelerated Computing ISV Partners Solution Partners

NEW !

UCS C480 ML

UCS C480 ML


Cisco UCS C480 ML Rack ServerNo-compromise balance of performance and capacity to power AI workloads at scale

NEWPrevents Operation Silos: Extends Existing UCS Environments with Consistent, Cloud-Based Management

Validated with Popular Machine Learning Software to Accelerate and Simplify AI/ML Projects on Premise

Fully Integrated Platform Designed to Accelerate Deep Learning• Eight NVIDIA Tesla V100s with NVIDIA NVLink Interconnect• Up to 24 Drives; 182TB• Up to 6 NVMe Drives• Network: Up to 4x100GB• High Availability Design


Cisco UCS C480 ML M5 Rack Server for Deep LearningA No-Compromise Purpose Built Server for Deep Learning

Raid Controller

NetworkChoice of 10/25 or 40/100G Four PCIe Slots

GPUs8 X V100 32GB: 1st GPU to break 100 teraflops NVLink Interconnect: >300GB/s bandwidth

Redundant Fans

StorageUp to 24 SAS/SATA SSD/HDDUp to 6 NVMe Drives

CPUs2 * Intel® Xeon® Processor Scalable Family(Up to 28 cores per socket)24 DDR4 DIMMS—up to 3 TB Memory


Focus on Full Data Life Cycle, Simplicity, and Manageability

Deployment/Inference

Data

Source & AggregateCleanupTransformation

Training

Model DevModel ValidationModel Execution

Full portfolio for all AI/ML computing needs

Validated solutions with technology partners

Natural extension of existing computing

environment

Application runtime at source of demand


Data Centric Approach: Expanding Big Data to AI/ML SolutionsCisco Validated Designs

Cloudera Data Science Work Bench Kubeflow

Hortonworks Hadoop 3.1 Data Lake

Hadoop Coupled with GPU Nodes for Deep Learning with

Jupyter Notebook

Portable, Scalable ML Stack Enabling Rapid Development

and Deployment

Integrate Hadoop and AI/ML: YARN Scheduling CPU and GPU with Docker Application Support

YARN Scheduler

HDFS Hot Tier

HDFS Cold TierHDFS


UCS: One System for All Workloads

VDI, VSI, Distributed Databases, CI, Enterprise Applications: Oracle and SAP HANA, Big Data and Analytics, AI/ML with GPUs

Mainstream Workloads B200 M5 C220 M5 and C240 M5 HyperFlex

Cloud Computing, Microprocessor Design and Simulation, Computational Fluid Dynamics (CFD), High Frequency Trading (HFT,) Fraud Detection, Online Gaming

Scale Out and Compute Intensive Applications C4200 and C125

6300 Series Fabric Interconnects

AI/ML W ith Dense GPUs, and Memory-Intensive Mission-Critical Enterprise Applications: In-Memory Databases

AI/ML and Scale up Workloads C480 ML M5 and B480 M5

Data Protection, SDS, Scale-Out Unstructured Data Repositories, Media Streaming, and Content Distribution

Data Intensive WorkloadsS3260 M5

NEW


SaaS Simplicity

ActionableIntelligence

SaaS Delivered

Intuitive Experience

Enhanced Support

Proactive Guidance

Secure and ExtensibleCentralized Management

Global Policies

Comprehensive Automation

Single Pane of Glass

Cloud-Powered Systems Management


Customer-Centric MarketingBanking Customer with Cisco UCS for On-Premise AI Farm

Existing UCS Big Data Cluster

AI Farm: UCS C240 with V100 GPUs

Product recommendation

Experience personalization

Attrition prediction


AI Platform (for Develop / Train)

Cisco: Improving Customer Experience

Component Failure Prediction

Training Farm InferencingUCS UCS with GPUs UCS

Communicate/Action

Dashboard

Visualization

Integrating data lake with ML

Predicting Failures

Automatically Position Spares at Depots

Optimizing Supply Chain and Customer Experience

A c c e le r a te d C o m p u te

Data Sources

Data

Lak

e

P r o g r a m m in g L a n g u a g e s

D a ta E x p lo r a t io n

M a c h in e L e a r n in g L ib r a r ie s

Data


Activating Data with the Power of UCSNew Cisco Computing Solutions for AI/ML

Powering the Full AI Data Lifecycle

Accelerating Insight and Action

Unified Architecture

Adaptable Cloud-Managed Systems for Distributed IT

Demystifying AI/ML Stacks

Validated Solutions with Industry Leaders


Putting Everything Together


Data Mgmt. & Pattern Detection using ESP

Data: power frequency,

voltage, current, phasor

angle, …

Cisco UCS

Pattern Detection using ESP

Visualization of streaming data

using SAS ESP Streamviewer

Application Life Cycle Management (Cisco FogDirector)

Update Streaming

models

Connect to external apps

with ESP connectorsCisco UCS

Cisco UCS

Pattern discovery using SAS Visual Analytics and SAS Visual

Statistics

Cisco ISA

KA FKA

Edge Enterprise

Visualization of streaming data using SAS ESP Streamviewer

Edge to Enterprise IoT Analytics





angle, …

Cisco UCS





Update Streaming

models



Cisco UCS


Statistics

Cisco ISA

KA FKA

Substation Operations Center


Edge to Enterprise IoT Analytics: Electric Utility


Local and Global Analytics

Global AnalyticsLocal Analytics

Frequency


IR 8x9

• Ruggedized Integrated Services Router

• Runs SAS analytics as data is being gathered

• Securely send relevant summary back to data center

Integrated Infrastructure for Big Data

• Industry leading Cisco UCS

• Tested and validated configuration

• High performance & availability

Cisco Infrastructure

IR 829IR 809





angle, …

Cisco UCS





Update Streaming

models



Cisco UCS


Statistics

Cisco ISA

KA FKA


Remote Data and Raised Abstraction

Part of Data Pipeline

Reduce Bandwidth

Raise Abstraction






angle, …

Cisco UCS







Cisco UCS


Statistics

Cisco ISA

KA FKA



Discovery and Edge Analytics

Update Streaming

models

Discovery -> Updated Edge Analytics Model


• By 2020: Insight driven will steal $1.2T / year from less-informed peers

• By 2021, there will be 847 Zettabytes of Data• Only 1.3 ZB is in the data center

• Analytics needs to follow data to the edge: Ubiquitous Machine Learning

• Cisco provides holistic analytic tools for edge, data center, and even hybrid cloud with data lifecycle approach

Summary

ubiquitous machine learning - o'reilly media...module (gmm) efm extracts data from sensors,...

Documents