1
The Evolution of AIin the Network Edge
Remi El-Ouazzane, COO/VP, AI Products Group, Intel @relouazzaneGSA Executive Forum, Sept 18, 2018
2
Computer Vision Then and Now
NOWthen
o Heuristic Algorithms
o Human Engineered
o Server Based
o AI / Deep Learning
o Real-time, High Accuracy
o Edge Based
1960
3
Today we’ll see how semiconductor industry iscritical in the evolution of AI at the Edge
DEFINITION TRENDS
MARKETS AND EXAMPLES CHALLENGES AHEAD
4
Devices • Things
EDGE-TO-CLOUD: DATA IS KEY DRIVER
LatencyBandwidth
Securityconnectivity
Core Network
CloudData Center
NETWORK HUBOR REGIONALDATA CENTER
EdgeCompute Node
By 2019, 45% of datawill be stored, analyzed, and acted on at the edge
Video • Healthcare • Manufacturing
Smart Buildings • Energy
Transportation • Retail
Public Sector • Logistics • Smart Cities Drivers for edge
5
There is a growing opportunityfor ai at the edge
Data CenterTraining
Endpoint &Edge Inference
Data CenterInference
• 25%-35% CAGRin AI for silicon industry
• Growth largerat the Edge,compared toData center
2018 2023
TAM
Note: Endpoint and Edge excludes smartphones and client PCsSource: Intel, IDC, Gartner
6
THE VIRTUOUS CYCLE DRIVING INNOVATION INAI AT THE EDGE
• Huge Compute
Requirements
• Low Latency
• Power Efficiency
• Local Data / Privacy
• Efficient Data Flow
• Better Algorithm Processing
• Efficient Memory Use
• Silicon Process
Technology
EDGE PROCESSINGREQUIREMENTS
CHIP TECHNOLOGYIMPROVEMENTS
77
CITIES • STATE • FEDERAL FINANCE • BANKING INDUSTRIAL CASINO GAMING
TRANSPORTATION ROBOTICS DRONES
Public Safety & SurveilanceTraffic, Parking and LPR
Emergency Response
People Counting Customer(i.e. Gender, Wait Time)ATM Facial Recognition
Autonomous VehiclesPublic Safety (i.e. Bus/Rail)Traffic & People Counting
Emergency ResponseAsset Inspection (i.e. Windmill)
Security & SurveillanceResponsive Retail Advertising
Digital Home Assitant
Manufacturing AutomationIndustrial (i.e. Pipeline Welding)
Public Safety & SurveilanceFacial Recognition
Machine Vision Asset Inspection(i.e. Pipeline)
Augmented Reality
Edge AI capabilities enable AUTONOMOUS FEATURES IN MANY APPLICATIONS
HOME • RETAIL • SURVEILLANCE
8
INDUSTRIAL EXAMPLE: SMART FACTORY
WAREHOUSE
CARRYING
PARKING SORTING
Smart Factory:
Generates an Estimate of 1PB data/day by 2020
Things & Edge Node Edge Network & Core Network Cloud
WAREHOUSE MANAGEMENT SYSTEM
9
• Item Tracking
• Mobile Payment
• Store Analytics
• Inventory Management
• Customer Tracking
RETAIL EXAMPLE: SMART STORESRetail Store:
Huge Data Processing Requirements
Things & Edge Node Edge Network & Core Network Cloud
10Intel® Movidius™
DRONE EXAMPLE: LIFE SAVING WITH AI
https://www.youtube.com/watch?v=QJOMfDyhUyo
The Intel® Movidius™ Neural
Compute Stick was used with
Australia’s Little Ripper Lifesaver
UAV to monitor the New South
Wales coastline for sharks.
The NCS is a proof-of-concept
system, showing how modern AI
processing can take place ‘on
device’ (rather than in the cloud),
leading to faster danger detection
and shorter response times
11
SILICON INDUSTRY HAS CHALLENGES TO OVERCOME TO MAKEAUTONOMY AT THE EDGE POSSIBLE
’18 ’19 ’20 ’21 ’22 ’23
DEEP NEURALNETWORKPERFORMANCE*InferencesPer Secondfor ResNet50,Batch Size = 1
AI PERFORMANCE*FOR ISO POWER AT 5~10 W
10
100
1000
10,000
SEMI INDUSTRY PROGRESS GOALS:REDUCED PRECISION NEURAL NETWORKSImproved performance moving from 16b tobinary, for example
NEURAL NETWORK COMPRESSION / SPARSITYReducing compute requirement for NN structure, and taking advantage of zeros in matrix computation
EFFICIENT MEMORYPower efficient memory access with improved bandwidth
ACCELERATORSBalancing fixed function performance with flexibility/programmability
COMPILER INNOVATIONTools making best use of new hardware
12
*I360
The Future is in Our Hands