arm and machine learning

© ARM 2016

ARM and Machine Learning

Jem DaviesARM Fellow & VP TechnologyMedia Processing Group, Imaging & Vision Group

December 12 2016

© ARM 2016 2

Jem Davies - ARM Fellow and VP of Technology - Media Processing and Imaging & Vision Groups Working on multimedia, computer vision and machine learning

13 years at ARM

Mathematician turned chemist, turned software engineer… … turned architect … turned tech future predictor

Glider pilot instructor, fireworks lighterand scuba diver…

… what next?

Who am I?

© ARM 2016 3

Machine Learning overview

Speech recognitionRobotics Home security

Machine Learning makes smart connections to previously encountered concepts

It’s useful when:• We don’t have algorithms…• but we do have a lot of data

Image recognition

© ARM 2016 4

We use Machine Learning technology every day

Speech recognition

Predictive text

Face tracking camera

Fingerprint ID

Computational photography

Digital assistant

5 © ARM 2016

Definitions and recent developments

© ARM 2016 6

The AI landscape

AI

Machine Learning

Computer Vision

Natural language

processing

Reasoning

Search and Optimization

Constraints and control

theory

Artificial Intelligence

Machine Learning

Deep Learning

Algorithms: CNNs,

RNNs, etc.

© ARM 2016 7

Key terms and definitionsArtificial Intelligence• The broadest term - applying to any technique enabling computers to mimic

human intelligence

Machine Learning• A subset of AI including techniques enabling computers to improve at tasks

with experience. Includes deep learning

Deep Learning/Neural Networks• A branch of machine learning that attempts to model real life ideas in data by

using a deep graph with multiple processing layers

Algorithms• DNNs, CNNs, RNNs, SGEMM etc.

Computer Vision• An interdisciplinary field that allows computers to gain understanding from

digital images or videos. Many computer vision applications use ML algorithms.

© ARM 2016 8

Recent developments in deep learning

The image detection benchmark ImageNet has reached nearly 100% accuracy Largely due to Neural Networks

Research groups feel it’s not useful to work on ImageNet anymore (it’s a solved problem) GoogLeNet, Inception, etc.

The successful approach used for ImageNet has spread to other domains, yielding great improvements

© ARM 2016 9

Why did this happen ?The Machine Learning learning loop

© ARM 2016 10

Basic neural network

[Inputs] [Output]

w2

w1

w3

wn

wn-1

y)(

1

zHy

xwzn

iii

=

=∑=

x1

x2

x3

xn-1

xn

Squashing function

Sigmoid: y=1/(1+e^(-z))

11 © ARM 2016

Machine Learning computation in the cloud and on-device

© ARM 2016 12

Dissecting the Machine Learning process

Generate inference engines

Distribute & optimize

Training engine

Inference engine

Platform acceleration libraries

ARM Cortex + Mali + Others

Heterogeneous tools

Cloud side

Device side

© ARM 2016 13

Machine Learning process on ARM

Middleware

Cloud services / applications Cloud applications

Applications

CPU GPU

(Cache-coherent interconnect)

HW CoresCV/ML IP

HW Interconnect

HW

SW

Metadata streams, thumbnails etc.

Device

Cloud

Cloud service SDK+ analytics

Cloud APIs

Training engine

Inference engine

Platform acceleration libraries

ARM Cortex + Mali + Others

Heterogeneous tools

Cloud side

Device side

14 © ARM 2016

It is easy to recognise speechIt is easy to wreck a nice beach

© ARM 2016 15

Automatic speech recognition is not easy

Active research area for over 50 years! Significant attention recently due to large amount of

cloud computing Neural Networks has improved performance Siri, Google Now, Cortana, Amazon Echo now creating

usable applications

Applications Safety and security Interactions with smaller or hands free devices (e.g. wearables) Improved accessibility for hearing-impaired Allows indexing of spoken words

© ARM 2016 16

ASR use cases and Machine Learning

Large Vocabulary Continuous Speech Recognition (LVCSR) Dictation/transcription, virtual assistant Requires dictionary, knowledge of grammar

Keyword spotting of simple commands “OK Google”, “Set alarm for 7”

Sound monitoring Early/automatic anomaly detection

© ARM 2016 17

Keyword spotting

Listen for certain words/phrases Only need to learn certain words “Okay Google” “Play music”

Simpler algorithm No knowledge of grammar Only needs acoustic model

Algorithm must run locally on device Potentially always listening Power consumption vitally important

Speech Signal

Acoustic model

Features

Words Play music

Feature extraction

18 © ARM 2016

Computer Vision

© ARM 2016 19

Computer vision pipeline

Pre-processingFeature

extraction

Detection and segmentation

High-level processing

Image acquisition

Output

Application

ISP/GPU GPU/CPU/DSP/Spirit CV CPU

20 © ARM 2016

ARM and Machine Learning

© ARM 2016 21

Machine Learning runs on ARM-powered client devices today

© ARM 2016 22

Caffe framework deep learning on ARM

Video hosted on youtube

https://www.youtube.com/watch?v=k4ovpelG9vs&t=13s

© ARM 2016 23

Deep learning frameworks on ARM

DNN used for real-time detection of objects

Based on the Caffe deep learning framework

Detection entirely on the mobile device – not cloud based

Optimized for ARM Cortex CPU or Mali GPU

Running Machine Learning on the GPU freesup the CPU for other tasks

Optimised libraries also being developed for TensorFlow and OpenVX

2.7 5.564% 20%

© ARM 2016 24

Growth MaturityInnovationMachine Learning

Platforms & Tools

Conclusions

The rapid uptake in Machine Learning isgoing mainstream, affecting computeeverywhere

Machine Learning dramatically increasescompute demands

As much as possible, Machine Learningworkloads should run locally on device, not on remote servers

Machine Learning is driving demand for advanced ARM processors and accelerator IP

Machine Learning is having a significant impact on ARM’s roadmap for future processors and architectures

The trademarks featured in this presentation are registered and/or unregistered trademarks of ARM Limited (or its subsidiaries) in the EU and/or elsewhere. All rights reserved. All other marks featured may be trademarks of their respective owners.Copyright © 2016 ARM Limited

© ARM 2016

Thank you for listening