cs 4803 / 7643: deep learning...(c) dhruv batra 2. recap from last time (c) dhruv batra 3....
TRANSCRIPT
![Page 1: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/1.jpg)
CS 4803 / 7643: Deep Learning
Dhruv Batra
Georgia Tech
Topics: – Convolutional Neural Networks
– Stride, padding – Pooling layers– Fully-connected layers as convolutions
![Page 2: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/2.jpg)
Administrativia• HW2 Reminder
– Due: 09/23, 11:59pm– https://evalai.cloudcv.org/web/challenges/challenge-page/6
84/leaderboard/1853
• Project Teams– https://gtvault-my.sharepoint.com/:x:/g/personal/dba
tra8_gatech_edu/EY4_65XOzWtOkXSSz2WgpoUBY8ux2gY9PsRzR6KnglIFEQ?e=4tnKWI
– Project Title– 1-3 sentence project summary TL;DR– Team member names
(C) Dhruv Batra 2
![Page 3: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/3.jpg)
Recap from last time
(C) Dhruv Batra 3
![Page 4: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/4.jpg)
Convolutional Neural Networks(without the brain stuff)
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 5: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/5.jpg)
5
Example: 200x200 image 40K hidden units
Fully Connected Layer
Slide Credit: Marc'Aurelio Ranzato
Q: what is the number of parameters in this FC layer?A: 1.6B
![Page 6: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/6.jpg)
6
Example: 200x200 image 40K hidden units Connection size: 10x10
4M parameters
Note: This parameterization is good when input image is registered (e.g., face recognition).
Assumption 1: Locally Connected Layer
Slide Credit: Marc'Aurelio Ranzato
![Page 7: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/7.jpg)
7
STATIONARITY? Statistics similar at all locations
Slide Credit: Marc'Aurelio Ranzato
Assumption 2: Stationarity / Parameter Sharing
![Page 8: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/8.jpg)
8
Share the same parameters across different locations (assuming input is stationary):Convolutions with learned kernels
Convolutional Layer
Slide Credit: Marc'Aurelio Ranzato
![Page 9: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/9.jpg)
Convolutions!
(C) Dhruv Batra 9
![Page 10: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/10.jpg)
Convolutions for programmers
(C) Dhruv Batra 10
![Page 11: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/11.jpg)
Convolution
Slide Credit: Marc'Aurelio Ranzato(C) Dhruv Batra 11
![Page 12: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/12.jpg)
Convolutional Layer
Slide Credit: Marc'Aurelio Ranzato(C) Dhruv Batra 12
![Page 13: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/13.jpg)
Convolution
Slide Credit: Marc'Aurelio Ranzato(C) Dhruv Batra 13
![Page 14: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/14.jpg)
Convolution
Slide Credit: Marc'Aurelio Ranzato(C) Dhruv Batra 14
![Page 15: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/15.jpg)
Convolution
Slide Credit: Marc'Aurelio Ranzato(C) Dhruv Batra 15
![Page 16: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/16.jpg)
Convolution
Slide Credit: Marc'Aurelio Ranzato(C) Dhruv Batra 16
![Page 17: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/17.jpg)
Convolution
Slide Credit: Marc'Aurelio Ranzato(C) Dhruv Batra 17
![Page 18: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/18.jpg)
Convolution
Slide Credit: Marc'Aurelio Ranzato(C) Dhruv Batra 18
![Page 19: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/19.jpg)
Convolution
Slide Credit: Marc'Aurelio Ranzato(C) Dhruv Batra 19
![Page 20: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/20.jpg)
Convolution
Slide Credit: Marc'Aurelio Ranzato(C) Dhruv Batra 20
![Page 21: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/21.jpg)
Convolution
Slide Credit: Marc'Aurelio Ranzato(C) Dhruv Batra 21
![Page 22: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/22.jpg)
Convolution
Slide Credit: Marc'Aurelio Ranzato(C) Dhruv Batra 22
![Page 23: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/23.jpg)
Convolution
Slide Credit: Marc'Aurelio Ranzato(C) Dhruv Batra 23
![Page 24: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/24.jpg)
Convolution
Slide Credit: Marc'Aurelio Ranzato(C) Dhruv Batra 24
![Page 25: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/25.jpg)
Convolution
Slide Credit: Marc'Aurelio Ranzato(C) Dhruv Batra 25
![Page 26: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/26.jpg)
Plan for Today• Convolutional Neural Networks
– Features learned by CNN layers– Stride, padding – 1x1 convolutions– Pooling layers– Fully-connected layers as convolutions
(C) Dhruv Batra 26
![Page 27: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/27.jpg)
27
FC vs Conv Layer
![Page 28: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/28.jpg)
28
FC vs Conv Layer
![Page 29: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/29.jpg)
32
32
3
Convolution Layer
32x32x3 image
width
height
depth
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 30: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/30.jpg)
32
32
3
Convolution Layer
5x5x3 filter
32x32x3 image
Convolve the filter with the imagei.e. “slide over the image spatially, computing dot products”
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 31: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/31.jpg)
32
32
3
Convolution Layer
5x5x3 filter
32x32x3 image
Convolve the filter with the imagei.e. “slide over the image spatially, computing dot products”
Filters always extend the full depth of the input volume
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 32: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/32.jpg)
32
32
3
32x32x3 image5x5x3 filter
1 number: the result of taking a dot product between the filter and a small 5x5x3 chunk of the image(i.e. 5*5*3 = 75-dimensional dot product + bias)
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
Convolution Layer
![Page 33: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/33.jpg)
32
32
3
32x32x3 image5x5x3 filter
convolve (slide) over all spatial locations
activation map
1
28
28
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
Convolution Layer
![Page 34: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/34.jpg)
32
32
3
32x32x3 image5x5x3 filter
convolve (slide) over all spatial locations
activation maps
1
28
28
consider a second, green filter
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
Convolution Layer
![Page 35: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/35.jpg)
32
32
3
Convolution Layer
activation maps
6
28
28
For example, if we had 6 5x5 filters, we’ll get 6 separate activation maps:
We stack these up to get a “new image” of size 28x28x6!
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 36: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/36.jpg)
Im2Col
(C) Dhruv Batra 36Figure Credit: https://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/
![Page 37: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/37.jpg)
GEMM
(C) Dhruv Batra 37Figure Credit: https://petewarden.com/2015/04/20/why-gemm-is-at-the-heart-of-deep-learning/
![Page 38: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/38.jpg)
Time Distribution of AlexNet
(C) Dhruv Batra 38Figure Credit: Yangqing Jia, PhD Thesis
![Page 39: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/39.jpg)
Preview: ConvNet is a sequence of Convolution Layers, interspersed with activation functions
32
32
3
28
28
6
CONV,ReLUe.g. 6 5x5x3 filters
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 40: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/40.jpg)
Preview: ConvNet is a sequence of Convolutional Layers, interspersed with activation functions
32
32
3
CONV,ReLUe.g. 6 5x5x3 filters 28
28
6
CONV,ReLUe.g. 10 5x5x6 filters
CONV,ReLU
….
10
24
24
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 41: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/41.jpg)
Convolutional Neural Networks
(C) Dhruv Batra 41Image Credit: Yann LeCun, Kevin Murphy
![Page 42: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/42.jpg)
preview:
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 43: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/43.jpg)
example 5x5 filters(32 total)
one filter => one activation map
Figure copyright Andrej Karpathy.
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 44: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/44.jpg)
Visualizing Learned Filters
(C) Dhruv Batra 45Figure Credit: [Zeiler & Fergus ECCV14]
![Page 45: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/45.jpg)
Visualizing Learned Filters
(C) Dhruv Batra 46Figure Credit: [Zeiler & Fergus ECCV14]
![Page 46: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/46.jpg)
Visualizing Learned Filters
(C) Dhruv Batra 47Figure Credit: [Zeiler & Fergus ECCV14]
![Page 47: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/47.jpg)
Visualizing Learned Filters
(C) Dhruv Batra 48Figure Credit: [Zeiler & Fergus ECCV14]
![Page 48: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/48.jpg)
Linear Classifier
Low-LevelFeature
Mid-LevelFeature
High-Level
Feature
Feature visualization of convolutional net trained on ImageNet from [Zeiler & Fergus 2013]
“car”
Slide Credit: Marc'Aurelio Ranzato, Yann LeCun
We can learn image features now!
![Page 49: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/49.jpg)
Plan for Today• Convolutional Neural Networks
– Features learned by CNN layers– Stride, padding – 1x1 convolutions– Pooling layers– Fully-connected layers as convolutions
(C) Dhruv Batra 50
![Page 50: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/50.jpg)
A closer look at spatial dimensions:
32
32
3
32x32x3 image5x5x3 filter
convolve (slide) over all spatial locations
activation map
1
28
28
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 51: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/51.jpg)
7x7 input (spatially)assume 3x3 filter
7
7
A closer look at spatial dimensions:
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 52: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/52.jpg)
7x7 input (spatially)assume 3x3 filter
7
7
A closer look at spatial dimensions:
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 53: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/53.jpg)
7x7 input (spatially)assume 3x3 filter
7
7
A closer look at spatial dimensions:
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 54: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/54.jpg)
7x7 input (spatially)assume 3x3 filter
7
7
A closer look at spatial dimensions:
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 55: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/55.jpg)
7x7 input (spatially)assume 3x3 filter
=> 5x5 output
7
7
A closer look at spatial dimensions:
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 56: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/56.jpg)
7x7 input (spatially)assume 3x3 filterapplied with stride 2
7
7
A closer look at spatial dimensions:
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 57: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/57.jpg)
7x7 input (spatially)assume 3x3 filterapplied with stride 2
7
7
A closer look at spatial dimensions:
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 58: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/58.jpg)
7x7 input (spatially)assume 3x3 filterapplied with stride 2=> 3x3 output!
7
7
A closer look at spatial dimensions:
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 59: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/59.jpg)
7x7 input (spatially)assume 3x3 filterapplied with stride 3?
7
7
A closer look at spatial dimensions:
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 60: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/60.jpg)
7x7 input (spatially)assume 3x3 filterapplied with stride 3?
7
7
A closer look at spatial dimensions:
doesn’t fit! cannot apply 3x3 filter on 7x7 input with stride 3.
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 61: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/61.jpg)
N
NF
F
Output size:(N - F) / stride + 1
e.g. N = 7, F = 3:stride 1 => (7 - 3)/1 + 1 = 5stride 2 => (7 - 3)/2 + 1 = 3stride 3 => (7 - 3)/3 + 1 = 2.33 :\
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 62: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/62.jpg)
Remember back to… E.g. 32x32 input convolved repeatedly with 5x5 filters shrinks volumes spatially!(32 -> 28 -> 24 ...). Shrinking too fast is not good, doesn’t work well.
32
32
3
CONV,ReLUe.g. 6 5x5x3 filters 28
28
6
CONV,ReLUe.g. 10 5x5x6 filters
CONV,ReLU
….
10
24
24
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 63: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/63.jpg)
In practice: Common to zero pad the border
0 0 0 0 0 0
0
0
0
0
e.g. input 7x73x3 filter, applied with stride 1 pad with 1 pixel border => what is the output?
(recall:)(N - F) / stride + 1
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 64: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/64.jpg)
In practice: Common to zero pad the border
e.g. input 7x73x3 filter, applied with stride 1 pad with 1 pixel border => what is the output?
7x7 output!
0 0 0 0 0 0
0
0
0
0
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 65: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/65.jpg)
In practice: Common to zero pad the border
e.g. input 7x73x3 filter, applied with stride 1 pad with 1 pixel border => what is the output?
7x7 output!in general, common to see CONV layers with stride 1, filters of size FxF, and zero-padding with (F-1)/2. (will preserve size spatially)e.g. F = 3 => zero pad with 1 F = 5 => zero pad with 2 F = 7 => zero pad with 3
0 0 0 0 0 0
0
0
0
0
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 66: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/66.jpg)
Examples time:
Input volume: 32x32x310 5x5 filters with stride 1, pad 2
Output volume size: ?
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 67: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/67.jpg)
Examples time:
Input volume: 32x32x310 5x5 filters with stride 1, pad 2
Output volume size: (32+2*2-5)/1+1 = 32 spatially, so32x32x10
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 68: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/68.jpg)
Examples time:
Input volume: 32x32x310 5x5 filters with stride 1, pad 2
Number of parameters in this layer?
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 69: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/69.jpg)
Examples time:
Input volume: 32x32x310 5x5 filters with stride 1, pad 2
Number of parameters in this layer?each filter has 5*5*3 + 1 = 76 params (+1 for bias)
=> 76*10 = 760
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 70: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/70.jpg)
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 71: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/71.jpg)
Common settings:
K = (powers of 2, e.g. 32, 64, 128, 512)- F = 3, S = 1, P = 1- F = 5, S = 1, P = 2- F = 5, S = 2, P = ? (whatever fits)- F = 1, S = 1, P = 0
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 72: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/72.jpg)
Plan for Today• Convolutional Neural Networks
– Features learned by CNN layers– Stride, padding – 1x1 convolutions– Pooling layers– Fully-connected layers as convolutions– Backprop in conv layers
(C) Dhruv Batra 74
![Page 73: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/73.jpg)
Can we have 1x1 filters?
(C) Dhruv Batra 75
![Page 74: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/74.jpg)
1x1 convolution layers make perfect sense
64
56
561x1 CONVwith 32 filters
3256
56
(each filter has size 1x1x64, and performs a 64-dimensional dot product)
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n
![Page 75: CS 4803 / 7643: Deep Learning...(C) Dhruv Batra 2. Recap from last time (C) Dhruv Batra 3. Convolutional Neural Networks (without the brain stuff) Slide Credit: Fei-Fei Li, Justin](https://reader033.vdocuments.site/reader033/viewer/2022051922/60103a7e2cec1f156d28e572/html5/thumbnails/75.jpg)
30721
Fully Connected Layer as 1x1 Conv
32x32x3 image -> stretch to 3072 x 1
10 x 3072 weights
activationinput
1 number: the result of taking a dot product between a row of W and the input (a 3072-dimensional dot product)
110
Slide Credit: Fei-Fei Li, Justin Johnson, Serena Yeung, CS 231n