arm and machine learning
TRANSCRIPT
© ARM 2016
ARM and Machine Learning
Jem DaviesARM Fellow & VP TechnologyMedia Processing Group, Imaging & Vision Group
December 12 2016
© ARM 2016 2
Jem Davies - ARM Fellow and VP of Technology - Media Processing and Imaging & Vision Groups Working on multimedia, computer vision and machine learning
13 years at ARM
Mathematician turned chemist, turned software engineer… … turned architect … turned tech future predictor
Glider pilot instructor, fireworks lighterand scuba diver…
… what next?
Who am I?
© ARM 2016 3
Machine Learning overview
Speech recognitionRobotics Home security
Machine Learning makes smart connections to previously encountered concepts
It’s useful when:• We don’t have algorithms…• but we do have a lot of data
Image recognition
© ARM 2016 4
We use Machine Learning technology every day
Speech recognition
Predictive text
Face tracking camera
Fingerprint ID
Computational photography
Digital assistant
5 © ARM 2016
Definitions and recent developments
© ARM 2016 6
The AI landscape
AI
Machine Learning
Computer Vision
Natural language
processing
Reasoning
Search and Optimization
Constraints and control
theory
Artificial Intelligence
Machine Learning
Deep Learning
Algorithms: CNNs,
RNNs, etc.
© ARM 2016 7
Key terms and definitionsArtificial Intelligence• The broadest term - applying to any technique enabling computers to mimic
human intelligence
Machine Learning• A subset of AI including techniques enabling computers to improve at tasks
with experience. Includes deep learning
Deep Learning/Neural Networks• A branch of machine learning that attempts to model real life ideas in data by
using a deep graph with multiple processing layers
Algorithms• DNNs, CNNs, RNNs, SGEMM etc.
Computer Vision• An interdisciplinary field that allows computers to gain understanding from
digital images or videos. Many computer vision applications use ML algorithms.
© ARM 2016 8
Recent developments in deep learning
The image detection benchmark ImageNet has reached nearly 100% accuracy Largely due to Neural Networks
Research groups feel it’s not useful to work on ImageNet anymore (it’s a solved problem) GoogLeNet, Inception, etc.
The successful approach used for ImageNet has spread to other domains, yielding great improvements
© ARM 2016 9
Why did this happen ?The Machine Learning learning loop
© ARM 2016 10
Basic neural network
[Inputs] [Output]
w2
w1
w3
wn
wn-1
y)(
1
zHy
xwzn
iii
=
=∑=
x1
x2
x3
xn-1
xn
Squashing function
Sigmoid: y=1/(1+e^(-z))
11 © ARM 2016
Machine Learning computation in the cloud and on-device
© ARM 2016 12
Dissecting the Machine Learning process
Generate inference engines
Distribute & optimize
Training engine
Inference engine
Platform acceleration libraries
ARM Cortex + Mali + Others
Heterogeneous tools
Cloud side
Device side
© ARM 2016 13
Machine Learning process on ARM
Middleware
Cloud services / applications Cloud applications
Applications
CPU GPU
(Cache-coherent interconnect)
HW CoresCV/ML IP
HW Interconnect
HW
SW
Metadata streams, thumbnails etc.
Device
Cloud
Cloud service SDK+ analytics
Cloud APIs
Training engine
Inference engine
Platform acceleration libraries
ARM Cortex + Mali + Others
Heterogeneous tools
Cloud side
Device side
14 © ARM 2016
It is easy to recognise speechIt is easy to wreck a nice beach
© ARM 2016 15
Automatic speech recognition is not easy
Active research area for over 50 years! Significant attention recently due to large amount of
cloud computing Neural Networks has improved performance Siri, Google Now, Cortana, Amazon Echo now creating
usable applications
Applications Safety and security Interactions with smaller or hands free devices (e.g. wearables) Improved accessibility for hearing-impaired Allows indexing of spoken words
© ARM 2016 16
ASR use cases and Machine Learning
Large Vocabulary Continuous Speech Recognition (LVCSR) Dictation/transcription, virtual assistant Requires dictionary, knowledge of grammar
Keyword spotting of simple commands “OK Google”, “Set alarm for 7”
Sound monitoring Early/automatic anomaly detection
© ARM 2016 17
Keyword spotting
Listen for certain words/phrases Only need to learn certain words “Okay Google” “Play music”
Simpler algorithm No knowledge of grammar Only needs acoustic model
Algorithm must run locally on device Potentially always listening Power consumption vitally important
Speech Signal
Acoustic model
Features
Words Play music
Feature extraction
18 © ARM 2016
Computer Vision
© ARM 2016 19
Computer vision pipeline
Pre-processingFeature
extraction
Detection and segmentation
High-level processing
Image acquisition
Output
Application
ISP/GPU GPU/CPU/DSP/Spirit CV CPU
20 © ARM 2016
ARM and Machine Learning
© ARM 2016 21
Machine Learning runs on ARM-powered client devices today
© ARM 2016 22
Caffe framework deep learning on ARM
Video hosted on youtube
https://www.youtube.com/watch?v=k4ovpelG9vs&t=13s
© ARM 2016 23
Deep learning frameworks on ARM
DNN used for real-time detection of objects
Based on the Caffe deep learning framework
Detection entirely on the mobile device – not cloud based
Optimized for ARM Cortex CPU or Mali GPU
Running Machine Learning on the GPU freesup the CPU for other tasks
Optimised libraries also being developed for TensorFlow and OpenVX
2.7 5.564% 20%
© ARM 2016 24
Growth MaturityInnovationMachine Learning
Platforms & Tools
Conclusions
The rapid uptake in Machine Learning isgoing mainstream, affecting computeeverywhere
Machine Learning dramatically increasescompute demands
As much as possible, Machine Learningworkloads should run locally on device, not on remote servers
Machine Learning is driving demand for advanced ARM processors and accelerator IP
Machine Learning is having a significant impact on ARM’s roadmap for future processors and architectures
The trademarks featured in this presentation are registered and/or unregistered trademarks of ARM Limited (or its subsidiaries) in the EU and/or elsewhere. All rights reserved. All other marks featured may be trademarks of their respective owners.Copyright © 2016 ARM Limited
© ARM 2016
Thank you for listening