introduction to deep learning - heriot-watt university · introduction to deep learning musab...
TRANSCRIPT
1© 2018 The MathWorks, Inc.
Introduction to Deep Learning
Musab Khawaja
Application Engineer
2
What is deep learning?
3
Deep learning is a type of machine learning in which a model learns to
perform automated predictions directly from images or sequences such as
text, sound.
Deep learning is usually implemented using a neural network architecture.
What is Deep Learning?
4
Diverse Applications of Deep Learning
Automated iris recognition – 99.4% accuracy1
1. “An experimental study of deep convolutional features for iris recognition” Minaee et al
2. “Learning to See in the Dark” Chen et al
3. “Cardiologist-Level Arrhythmia Detection with Convolutional Neural Networks” Rajpurkar et al
Cardiologist-level arrhythmia detection3
Seeing in the dark – boosting the contrast of an image2
ISO 8,000 ISO 400,000 Deep Learning
5
Deep Learning vs Other TechniquesClassical Image Processing
▪ Classical image processing to create a set of rules, heuristics
▪ What happens when the image changes slightly?
6
Deep Learning vs Other TechniquesMachine Learning
▪ Classical image processing to extract summary statistics – “features”
▪ Machine learning to find patterns amongst the features
▪ What happens when the image changes?
Machine
learning
7
Deep Learning vs Other TechniquesDeep Learning
▪ Automated feature extraction, using whole images
Rabbit
Kangaroo
Bus
8
Why Now?
▪ Publicly available labelled data
▪ Increased GPU acceleration
▪ Race for better models through challenges
– e.g. ImageNet image recognition
– Models are then made publicly available
AlexNetPRETRAINED MODEL
ResNet PRETRAINED MODEL
VGG-16PRETRAINED MODEL
GoogLeNetPRETRAINED MODEL
And many more…
9
Human
Accuracy
Why now?
ImageNet Challenge
Source: ILSVRC Top-5 Error on ImageNet
10
How does deep learning work?Network architecture
Input OutputHidden (n)
• Pass data in
• Perform operations
• Get output prediction
• Training: the process of tuning
the weights in the hidden
layers so that the predicted
output matches the expected
output
• Inference/prediction: pass new
input through the network,
obtain prediction
11
Deep learning in MATLAB
12
Demo: Deep Learning in 12 Lines of Code
14
Deep Learning Made Easy in MATLAB
ACCESS AND EXPLORE
DATA
Files Databases Sensors
▪ Import data from multiple sources
Cloud
15
Deep Learning Made Easy in MATLAB
ACCESS AND EXPLORE
DATA
LABELLING AND
PREPROCESSING
▪ Apps for labelling data
Ground Truth Labeler App
16
Deep Learning Made Easy in MATLAB
ACCESS AND EXPLORE
DATA
LABELLING AND
PREPROCESSING
▪ Apps for labelling data
▪ Load reference models
AlexNetPRETRAINED MODEL
ResNet PRETRAINED MODEL
VGG-16PRETRAINED MODEL
GoogLeNetPRETRAINED MODEL
And many more…
17
Deep Learning Made Easy in MATLAB
ACCESS AND EXPLORE
DATA
LABELLING AND
PREPROCESSING
▪ Apps for labelling data
▪ Load reference models
▪ Import models from other frameworks
– Caffe
– Keras-Tensorflow
– ONNX
18
Deep Learning Made Easy in MATLAB
ACCESS AND EXPLORE
DATA
DEVELOP PREDICTIVE
MODELS
LABELLING AND
PREPROCESSING
▪ Train from scratch
▪ Transfer learning
19
Deep Learning Made Easy in MATLAB
ACCESS AND EXPLORE
DATA
DEVELOP PREDICTIVE
MODELS
LABELLING AND
PREPROCESSING
▪ Train from scratch
▪ Transfer learning
▪ Visualise networks
Deep Network Designer App
20
Deep Learning Made Easy in MATLAB
ACCESS AND EXPLORE
DATA
DEVELOP PREDICTIVE
MODELS
LABELLING AND
PREPROCESSINGDEPLOY AND SCALE
▪ Code generation
MATLAB
Coder
Products
Deep Learning
Networks
NVIDIA
TensorRT &
cuDNN
Libraries
ARM
Compute
Library
Intel
MKL-DNN
Library
21
Deep Learning Made Easy in MATLAB
ACCESS AND EXPLORE
DATA
DEVELOP PREDICTIVE
MODELS
LABELLING AND
PREPROCESSINGDEPLOY AND SCALE
▪ Code generation
▪ Web deployment with MATLAB Production Server
MATLAB Production Server
HTML
XML
Java Script
Web Server(s)
22
Deep Learning Made Easy in MATLAB
ACCESS AND EXPLORE
DATA
DEVELOP PREDICTIVE
MODELS
LABELLING AND
PREPROCESSINGDEPLOY AND SCALE
▪ Code generation
▪ Web deployment with MATLAB Production Server
▪ Scale up your processing
– MATLAB Distributed Computing Server (MDCS)
– 1 click deployment of virtual machines to the cloud
23
Deep Learning Made Easy in MATLABPlenty of documentation Examples
24
Deep Learning Made Easy in MATLABPlenty of documentation examples
25
Deep Learning for Machine Vision at Shell
James Martin & Amjad Chaudry
Advanced Analytics COE
26
Terrain recognition in hyper-spectral satellite data
• $10m’s spent on land seismic acquisition each year.
• Terrain type very important to daily shot target.
• Currently manually drawn polygons on satellite/drone
images + direct site visits - weeks.
• Automate whole workflow with DL semantic segmentation
approach (segnet).
Radar image with rough polygons overlaid
27
Data prep – colour composite images
27
◼Aerial photo (RGB)
◼Radar
◼DSM
RGB
BW
BW
BW
28
Network (SegNet)
28
Colour
composite
Classified image
• Currently trained with 1000 examples.
• 3 encoder and decoder sections.
• 8 hours to train on 4Gb GPU.
29
Prediction (Semantic segmentation)
29
30
Delivery via WebApp
• Customers interaction via WebApp tool.
• Can upload new data and regions of interest.
30
31
Terrain recognition – next steps
• Optimise network and training parameters.
• Increase training data.
• Add more modalities and classes.
• Tailor app to business demands.
• Business wants technology integration + upskilling staff.
• Potential for further work with Middle East and Asian business
units.
Facilities class
Additional drone data
32
Tag recognition:
Identifying Machinery at Location
3
2
33
Regional Convolutional Neural Networks
Region proposal:
Use a standard
method
Use pre-trained Neural Network,
and adapt to our specific task using
“transfer learning”
34
Region proposals
Pdollar’s EdgeBoxes
Fish-eye correction
35
Training labels and ‘attention’ areas
Training dataset
36
Identified tags ready for OCR and integration into SAP
37
Tag recognition – next steps
• Transfer learning used to identify tags in industrial images.
• OCR to extract SAP codes.
• Currently 3-4 minutes detection, considering Fast RCNN for 100x
improvement.
• Use larger GPU to increase image size of network (VGG16).
• Ready to experiment with augmented reality overlay of information.
• Further interest from European and Asian business units.
38
Wrap Up and Next Steps
39
Wrap Up
▪ Deep learning is well-suited to problems where:
– You have plenty of (labelled) data
– It is hard to find a good set of features to use
– You need your algorithm to generalise well to different situations
– You don’t mind your model being very complex
– Systems that will run autonomously, need to scale and adapt
▪ Not a substitute for physical insights, domain knowledge – it’s about fitting
models to data
▪ BUT very effective for image-based tasks
– In many cases easier than alternatives
– Easy to try in MATLAB: lots of universities have campus-wide licences to all of our tools
40
Hands on deep learning workshop
Go to MATLAB Academy https://matlabacademy.mathworks.com/ and launch
the Deep Learning Onramp