intro to convolutional neural networks

Intro To Convolu,onal Neural Networks

Mark Scully datapraxis.com

Why CNNs?

h@ps://papers.nips.cc/paper/4824-‐imagenet-‐classifica,on-‐with-‐deep-‐convolu,onal-‐neural-‐networks

Image Classifica,on

Object Recogni,on

h@ps://research.googleblog.com/2014/09/building-‐deeper-‐understanding-‐of-‐images.html

h@p://cs.stanford.edu/people/karpathy/deepimagesent/

Automa,c Cap,oning

h@ps://research.googleblog.com/2014/11/a-‐picture-‐is-‐worth-‐thousand-‐coherent.html

Facial Recogni,on

Y. Taigman, M. Yang, M. Ranzato, L. Wolf, DeepFace: Closing the Gap to Human-‐Level Performance in Face Verifica,on, CVPR 2014

Terminator Vision

Colorize Black & White Images

h@p://richzhang.github.io/coloriza,on/

Style Transfer

h@p://genekogan.com/works/style-‐transfer/

Mona Lisa restyled by Picasso, van Gough, and Monet

Generate An Image From A Sketch

h@ps://affinelayer.com/pixsrv/

ImageNet Challenge

Alexnet Li Fei-‐Fei: ImageNet Large Scale Visual Recogni,on Challenge, 2014

ImageNet Challenge ILSVRC+

ImageNet Classification error throughout years and groups

Li Fei-‐Fei: ImageNet Large Scale Visual Recogni,on Challenge, 2014

Alexnet Architecture -‐ 2012

Input Conv

Relu Pool

Conv Relu

Dropout

ImageNet Classifica,on with Deep Convolu,onal Neural Networks Alex Krizhevsky, Ilya Sutskever and Geoffrey E. Hinton Advances in Neural Informa,on Processing Systems 25 eds.F. Pereira, C.J.C. Burges, L. Bo@ou and K.Q. Weinberger pp.

1097-‐1105, 2012

FC 1000

ImageNet Classifica,on with Deep Convolu,onal Neural Networks Alex Krizhevsky, Ilya Sutskever and Geoffrey E. Hinton Advances in Neural Informa,on Processing Systems 25 eds.F. Pereira, C.J.C. Burges, L. Bo@ou and K.Q. Weinberger pp. 1097-‐1105,

ImageNet Challenge

Alexnet Li Fei-‐Fei: ImageNet Large Scale Visual Recogni,on Challenge, 2014

Tradi,onal Approach To Image Classifica,on

Input Image Hand

Extracted Features

Classifier Object Label

Issues

•  Who makes the features? – Need an expert for each problem domain

•  Which features? – Are they the same for every problem type?

•  How robust are these features to real images? – Transla,on, Rota,on, contrast changes, etc.

Are these pictures of the same thing?

Features Are Hierarchical

•  A squirrel is a combina,on of fur, arms, legs, & a tail in specific propor,ons.

•  A tail is made of texture, color, and spa,al rela,onships

•  A texture is made of oriented edges, gradients, and colors

Image Features

•  A feature is something in the image or derived from it that’s relevant to the task

•  Edges •  Lines at different angles, curves, etc. •  Colors, or pa@erns of colors •  SIFT, SURF, HOG, GIST, ORB, etc

Ideally We’d Learn Features

Input Image

Output Label

Ideally We’d Learn Features

Input Image

Output Label

What is a Neural Network?

•  Perceptron is biologically inspired •  A mental model for interpre,ng the math

h@p://cs231n.stanford.edu/index.html

Perceptron

Σ Output

Ac,va,on Func,on Sum

Weights Inputs

Perceptron

Σ Output

Weights Inputs

wixii=0

∑ = w0x0 +w1x1 +w2x2 +...+wmxm

Ac,va,on Func,ons

Training: Upda,ng Weights

Σ Output

Weights Inputs

Error = Output -‐ Target

Perceptron Decision Boundary

Deep (Mul,-‐Layer) Neural Network

Backpropaga,on

•  Error propagates backward and it all works via (normally stochas,c) gradient descent.

•  (wave hands)

CNN Layer Architecture

Convolu,on

Nonlinearity

Pooling (op,onal)

Dropout (op,onal)

Convolu,on

Nonlinearity

Pooling (op,onal)

Dropout (op,onal)

Input: Pixels Are Just Numbers

h@ps://medium.com/@ageitgey/machine-‐learning-‐is-‐fun-‐part-‐3-‐deep-‐learning-‐and-‐convolu,onal-‐neural-‐networks-‐f40359318721

Convolu,on

Nonlinearity

Pooling (op,onal)

Dropout (op,onal)

•  Need to detect the same feature anywhere in an image

•  Reuse the same weights over and over •  What we really want is one neuron that detects a feature that we slide over the image

Neuron = Filter

•  Act as detectors for some specific image feature

•  Take images as inputs and produce image like feature maps as outputs

Convolu,on

•  Like sliding a matrix over the input and performing dot products

•  It’s all just matrix mul,plica,on

Convolu,on

Filters (or Kernels)

Sharpen

Box Blur

Edge Detec,on

Feature Map

Alexnet Architecture

Convolu,ons

Convolu,on

Nonlinearity

Pooling (op,onal)

Dropout (op,onal)

Nonlinearity

Convolu,on

Nonlinearity

Pooling (op,onal)

Dropout (op,onal)

Max Pooling Example

3x3 stride 2 Max Pooling

Pooling

•  Allows us to look at more of the image •  Max, sum, and L2 pooling •  A type of downsampling

Convolu,on

Nonlinearity

Pooling (op,onal)

Dropout (op,onal)

Input Conv

Relu Pool

Conv Relu

Dropout

FC 1000

1097-‐1105, 2012

Dropout

h@p://cs231n.github.io/neural-‐networks-‐2/

•  Randomly disable some neurons on the forward pass

•  Prevents overfiong

Let’s Predict Something!

•  We have all these features, how do we learn to label something based on them?

Input Conv

Relu Pool

Conv Relu

Dropout

FC 1000

1097-‐1105, 2012

Fully Connected

Fully Connected Layers

•  Each neuron is connected to all inputs •  Standard mul,layer neural net •  Learns non-‐linear combina,ons of the feature maps to make predic,ons

Input Conv

Relu Pool

Conv Relu

Dropout

1097-‐1105, 2012

FC 1000

Which Class Is It Again?

•  FC-‐1000 gives us 1000 numbers, one per class, how do we compare them?

Soqmax

•  Mul,-‐class version of logis,c func,on •  Outputs normalized class “probabili,es” •  Takes m inputs and produces m outputs between zero and one, that sum to one

•  Cross-‐entropy loss •  Differen,able

h@ps://papers.nips.cc/paper/4824-‐imagenet-‐classifica,on-‐with-‐deep-‐convolu,onal-‐neural-‐networks

Image Classifica,on

Layer 1

Learned Filters – Layer1

Layer 2

Learned Filters – Layer2

Visualizing and Understanding Convolu,onal Networks -‐ Zeiler & Fergus, ECCV 2014

Layer 3

Learned Filters -‐ Layer3

Visualizing and Understanding Convolu,onal Networks -‐ Zeiler & Fergus, ECCV 2014

Layer 4 Layer 5

Learned Features – Layers 4 & 5

Input Conv

Relu Pool

Conv Relu

Dropout

1097-‐1105, 2012

FC 1000

h@ps://blog.heuritech.com/2016/02/29/a-‐brief-‐report-‐of-‐the-‐heuritech-‐deep-‐learning-‐meetup-‐5/

Google’s Incep,on Module

To Learn More

•  h@p://colah.github.io/posts/2014-‐07-‐Understanding-‐Convolu,ons/

•  h@ps://adeshpande3.github.io/adeshpande3.github.io/The-‐9-‐Deep-‐Learning-‐Papers-‐You-‐Need-‐To-‐Know-‐About.html

•  h@p://cs231n.github.io/ •  h@p://course.fast.ai/

Ques,ons?

intro to convolutional neural networks

Data & Analytics

convolutional neural networks for biomedical image ...deep...

convolutional neural networks: motivation, …

convolutional neural networks for no-reference image quality...

o-cnn: octree-based convolutional neural networks for...

convolutional neural networks for object...

interpretable convolutional neural...

shepard convolutional neural networks - neural information...

neural predictive coding using convolutional neural...

convolutional neural networks (cnn)

convolutional neural networks

very deep convolutional neural networks for noise...

convolutional neural networks deepa

hyperbolic graph convolutional neural networkspapers.nips.cc...

convolutional neural networks arise from ising models and...

convolutional neural networks analyzed via convolutional ......

convolutional neural networks for deep learning: an...

neural network part 3: convolutional neural...

using support vector machines, convolutional neural ... ·...

convolutional neural network

visualizing and understanding convolutional neural ... -...