the evolution of ai in the network edge...the evolution of ai in the network edge remi el-ouazzane,...

1

The Evolution of AIin the Network Edge

Remi El-Ouazzane, COO/VP, AI Products Group, Intel @relouazzaneGSA Executive Forum, Sept 18, 2018

2

Computer Vision Then and Now

NOWthen

o Heuristic Algorithms

o Human Engineered

o Server Based

o AI / Deep Learning

o Real-time, High Accuracy

o Edge Based

1960

3

Today we’ll see how semiconductor industry iscritical in the evolution of AI at the Edge

DEFINITION TRENDS

MARKETS AND EXAMPLES CHALLENGES AHEAD

4

Devices • Things

EDGE-TO-CLOUD: DATA IS KEY DRIVER

LatencyBandwidth

Securityconnectivity

Core Network

CloudData Center

NETWORK HUBOR REGIONALDATA CENTER

EdgeCompute Node

By 2019, 45% of datawill be stored, analyzed, and acted on at the edge

Video • Healthcare • Manufacturing

Smart Buildings • Energy

Transportation • Retail

Public Sector • Logistics • Smart Cities Drivers for edge

5

There is a growing opportunityfor ai at the edge

Data CenterTraining

Endpoint &Edge Inference

Data CenterInference

• 25%-35% CAGRin AI for silicon industry

• Growth largerat the Edge,compared toData center

2018 2023

TAM

Note: Endpoint and Edge excludes smartphones and client PCsSource: Intel, IDC, Gartner

6

THE VIRTUOUS CYCLE DRIVING INNOVATION INAI AT THE EDGE

• Huge Compute

Requirements

• Low Latency

• Power Efficiency

• Local Data / Privacy

• Efficient Data Flow

• Better Algorithm Processing

• Efficient Memory Use

• Silicon Process

Technology

EDGE PROCESSINGREQUIREMENTS

CHIP TECHNOLOGYIMPROVEMENTS

77

CITIES • STATE • FEDERAL FINANCE • BANKING INDUSTRIAL CASINO GAMING

TRANSPORTATION ROBOTICS DRONES

Public Safety & SurveilanceTraffic, Parking and LPR

Emergency Response

People Counting Customer(i.e. Gender, Wait Time)ATM Facial Recognition

Autonomous VehiclesPublic Safety (i.e. Bus/Rail)Traffic & People Counting

Emergency ResponseAsset Inspection (i.e. Windmill)

Security & SurveillanceResponsive Retail Advertising

Digital Home Assitant

Manufacturing AutomationIndustrial (i.e. Pipeline Welding)

Public Safety & SurveilanceFacial Recognition

Machine Vision Asset Inspection(i.e. Pipeline)

Augmented Reality

Edge AI capabilities enable AUTONOMOUS FEATURES IN MANY APPLICATIONS

HOME • RETAIL • SURVEILLANCE

8

INDUSTRIAL EXAMPLE: SMART FACTORY

WAREHOUSE

CARRYING

PARKING SORTING

Smart Factory:

Generates an Estimate of 1PB data/day by 2020

Things & Edge Node Edge Network & Core Network Cloud

WAREHOUSE MANAGEMENT SYSTEM

9

• Item Tracking

• Mobile Payment

• Store Analytics

• Inventory Management

• Customer Tracking

RETAIL EXAMPLE: SMART STORESRetail Store:

Huge Data Processing Requirements

Things & Edge Node Edge Network & Core Network Cloud

10Intel® Movidius™

DRONE EXAMPLE: LIFE SAVING WITH AI

https://www.youtube.com/watch?v=QJOMfDyhUyo

The Intel® Movidius™ Neural

Compute Stick was used with

Australia’s Little Ripper Lifesaver

UAV to monitor the New South

Wales coastline for sharks.

The NCS is a proof-of-concept

system, showing how modern AI

processing can take place ‘on

device’ (rather than in the cloud),

leading to faster danger detection

and shorter response times

11

SILICON INDUSTRY HAS CHALLENGES TO OVERCOME TO MAKEAUTONOMY AT THE EDGE POSSIBLE

’18 ’19 ’20 ’21 ’22 ’23

DEEP NEURALNETWORKPERFORMANCE*InferencesPer Secondfor ResNet50,Batch Size = 1

AI PERFORMANCE*FOR ISO POWER AT 5~10 W

10

100

1000

10,000

SEMI INDUSTRY PROGRESS GOALS:REDUCED PRECISION NEURAL NETWORKSImproved performance moving from 16b tobinary, for example

NEURAL NETWORK COMPRESSION / SPARSITYReducing compute requirement for NN structure, and taking advantage of zeros in matrix computation

EFFICIENT MEMORYPower efficient memory access with improved bandwidth

ACCELERATORSBalancing fixed function performance with flexibility/programmability

COMPILER INNOVATIONTools making best use of new hardware

12

*I360

The Future is in Our Hands