an introduction to machine learning silicon
TRANSCRIPT
Title 44pt sentence case
Affiliations 24pt sentence case
20pt sentence case
Insight forTechnologyInvestors
© 2017 Arm Limited 3
AI/ML terminology
Additional terms
• Location
• Cloud – processing done in data farms
• Edge – processing done in local devices
• Types of machine learning
• Model – a mathematical approximation of a collection of input data
• Training – in deep learning, data-sets are used to create a ‘model’
• Inference – using a ‘model’ to check against new data
Artificial Intelligence
Machine Learning
Deep Learning
Algorithms: CNNs,
RNNs, etc.
© 2017 Arm Limited 4
Neural Networks (NNs) outperform humans
28%
26%
16%
12%
7.3% 6.7%
3.6% 3%
2010 2011 2012 2013 2014 2015 2016
shallow
AlexNet, 8 layers
Human error
deep
ZF, 8 layers
VGG, 19 layers
GoogleNet, 22 layers
ResNet, 152 layers
CUImage
Data for ImageNet Large Scale Visual Recognition Challenge
Deep networks, introduced in 2012, resulted in big improvements
Error rates have now stabilized at ~3%
Cla
ssif
icat
ion
err
or
(Image source: Synopsys)
© 2017 Arm Limited 5
Machine Learning training
Training data Model
For each piece of data used to train the model, millions of model parameters are adjusted.
The process is repeated many times until the model delivers satisfactory performance.
© 2017 Arm Limited 6
Machine Learning inference
✓
Input Model Output
96.4% confidence
97.4% confidence
When new data is presented to the trained model, large numbers of multiply-add operations
are performed using the new data and the model parameters. The process is performed once.
© 2017 Arm Limited 8
Inference everywhere
Robotics
Surveillance IoT Augmented reality
Mobile DronesAutomotive
Shipping & logistics
© 2017 Arm Limited 10
A System-on-Chip contains multiple compute engines
Main processor (CPU)A versatile compute engine for running rich software. The main CPU runs device’s operating system, applications and user interface. It also manages the flow of data to specialist processors in the device.
Graphics processor (GPU)Used for generating 2D/3D images and executing highly-parallelised workloads such as neural network arithmetic
Digital signal processors (DSPs)A specialist form of CPU, optimised for analysing waveforms.Useful for radio control, sensor readings, audio and image processing
Accelerators Heavily-optimised data processors for frequently-used tasks,e.g. encryption, video, computer vision
© 2017 Arm Limited 11
Comparing processor options for Machine Learning
Training Inference Usability
Hardware cost Power efficiency Hardware cost Power efficiency Flexibility Programmability
CPU
DSP
GPU 1 2 3 1 2
Accelerator
FPGA
Weak, relative to alternatives
Good, relativeto alternatives
1 = High volume, evolving workload2 = High volume, stable workload3 = Low volume, evolving workload
1 = A client device that requires a GPU for graphics2 = A device that uses a GPU for ML work only
© 2017 Arm Limited 12
Processor options for various sizes of chip
Machine Learning demands (accuracy, response time) vary by use case
All use cases can default to a CPU
A GPU is often a good ‘all-rounder’ solution
Accelerators are useful when it is essential to either maximize response speed or minimize power consumption
Silicon area / power consumption
Perf
orm
ance
Keyword detection
Speech recognition
Visual object detection
Visual object recognition
Accelerator
Accelerator
Cortex-M
Cortex-A
(little CPU)
Cortex-A
(big CPU)
GPU
© 2017 Arm Limited 13
Arm’s ML computing platform
Arm DS-5 / Keil tools / compilers / drivers
AI Applications: ML, CV, speech recognition etc. Applications
Edge devices
Stable SW interfaces
Neural network frameworks(e.g. Tensorflow, Caffe, AndroidNN)
SpiritComputer Vision
Compute librarySpirit metadata
library
Optional Spirit libraries
& model sets
SVE
CPU CPU GPU Partner IP: DSPs, FPGAs, accelerators Provided
by Arm
Provided by third-party
© 2017 Arm Limited 14
Machine Learning is driving all of Arm’s technology roadmap
Processor design Software support Computer vision
1515 © 2017 Arm Limited
The Arm trademarks featured in this presentation are registered trademarks or trademarks of Arm Limited (or its subsidiaries) in the US and/or elsewhere. All rights reserved. All other marks featured may be trademarks of their respective owners.
www.arm.com/company/policies/trademarks