high level computer vision deep learning for computer vision … · 2017. 6. 14. · high level...
TRANSCRIPT
![Page 1: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/1.jpg)
High Level Computer Vision
Deep Learning for Computer Vision Part 4
Bernt Schiele - [email protected] Mario Fritz - [email protected]
https://www.mpi-inf.mpg.de/hlcv
![Page 2: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/2.jpg)
High Level Computer Vision - June 14, 2o17
Overview
• Recurrent Neural Networks ‣ motivation for recurrent neural networks
‣ a particularly successful RNN: Long Short Term Memory (LSTM)
‣ slide credit to Andrej Karpathy, Jeff Donahue and Marcus Rohrbach
• Yann LeCun… ‣ What’s Wrong With Deep Learning (keynote June 2015)
‣ slide credit to Yann LeCun (and Xiaogang Wang)
2
![Page 3: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/3.jpg)
Recurrent Networks offer a lot of flexibility:
slidecredit:AndrejKarpathy
![Page 4: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/4.jpg)
Sequences in VisionSequences in the input… (many-to-one)
JumpingDancingFightingEating
Running
slidecredit:JeffDonahue
![Page 5: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/5.jpg)
Sequences in VisionSequences in the output…(one-to-many)
A happy brown dog.
slidecredit:JeffDonahue
![Page 6: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/6.jpg)
Sequences in VisionSequences everywhere!(many-to-many)
A dog jumps over a hurdle.
slidecredit:JeffDonahue
![Page 7: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/7.jpg)
ConvNets
Krizhevsky et al., NIPS 2012
slidecredit:JeffDonahue
![Page 8: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/8.jpg)
Problem #1
fixed-size, static input
224
224
slidecredit:JeffDonahue
![Page 9: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/9.jpg)
Problem #1
fixed-size, static input
224
224
???
slidecredit:JeffDonahue
![Page 10: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/10.jpg)
Problem #2
Krizhevsky et al., NIPS 2012
slidecredit:JeffDonahue
![Page 11: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/11.jpg)
output is a single choice from a fixed list of options
doghorsefishsnake
cat
Problem #2slidecredit:JeffDonahue
![Page 12: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/12.jpg)
output is a single choice from a fixed list of options
a happy brown doga big brown doga happy red doga big red dog
Problem #2
…
slidecredit:JeffDonahue
![Page 13: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/13.jpg)
Recurrent Networks offer a lot of flexibility:
slidecredit:AndrejKarpathy
![Page 14: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/14.jpg)
Language Models
Recurrent Neural Network Based Language Model [Tomas Mikolov, 2010]
Word-level language model. Similar to:
slidecredit:AndrejKarpathy
![Page 15: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/15.jpg)
Suppose we had the training sentence “cat sat on mat”
We want to train a language model: P(next word | previous words)
i.e. want these to be high: P(cat | [<S>]) P(sat | [<S>, cat]) P(on | [<S>, cat, sat]) P(mat | [<S>, cat, sat, on]) P(<E>| [<S>, cat, sat, on, mat])
slidecredit:AndrejKarpathy
![Page 16: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/16.jpg)
Suppose we had the training sentence “cat sat on mat”
We want to train a language model: P(next word | previous words)
First, suppose we had only a finite, 1-word history: i.e. want these to be high: P(cat | <S>) P(sat | cat) P(on | sat) P(mat | on) P(<E>| mat)
slidecredit:AndrejKarpathy
![Page 17: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/17.jpg)
h0
x0 <START>
y0
x1 “cat”
h1
y1
“cat sat on mat”
x2 “sat”
h2
y2
x3 “on”
h3
y3
x4 “mat”
h4
y4
300 (learnable) numbers associated with each word in vocabulary
slidecredit:AndrejKarpathy
![Page 18: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/18.jpg)
h0
x0 <START>
y0
x1 “cat”
h1
y1
“cat sat on mat”
x2 “sat”
h2
y2
x3 “on”
h3
y3
x4 “mat”
h4
y4
300 (learnable) numbers associated with each word in vocabulary
hidden layer (e.g. 500-D vectors) h4 = tanh(0, Wxh * x4)
slidecredit:AndrejKarpathy
![Page 19: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/19.jpg)
h0
x0 <START>
y0
x1 “cat”
h1
y1
“cat sat on mat”
x2 “sat”
h2
y2
x3 “on”
h3
y3
x4 “mat”
h4
y4
300 (learnable) numbers associated with each word in vocabulary
10,001-D class scores: Softmax over 10,000 words and a special <END> token. y4 = Why * h4
hidden layer (e.g. 500-D vectors) h4 = tanh(0, Wxh * x4)
slidecredit:AndrejKarpathy
![Page 20: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/20.jpg)
h0
x0 <START>
y0
x1 “cat”
h1
y1
“cat sat on mat”
x2 “sat”
h2
y2
x3 “on”
h3
y3
x4 “mat”
h4
y4
300 (learnable) numbers associated with each word in vocabulary
10,001-D class scores: Softmax over 10,000 words and a special <END> token. y4 = Why * h4
hidden layer (e.g. 500-D vectors) h4 = tanh(0, Wxh * x4 + Whh * h3)
Recurrent Neural Network:
slidecredit:AndrejKarpathy
![Page 21: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/21.jpg)
Training this on a lot of sentences would give us a language model. A way to predict
P(next word | previous words)
x0 <START>
slidecredit:AndrejKarpathy
![Page 22: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/22.jpg)
Training this on a lot of sentences would give us a language model. A way to predict
P(next word | previous words) h0
x0 <START>
y0
slidecredit:AndrejKarpathy
![Page 23: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/23.jpg)
Training this on a lot of sentences would give us a language model. A way to predict
P(next word | previous words) h0
x0 <START>
y0
x1 “cat”
sample!
slidecredit:AndrejKarpathy
![Page 24: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/24.jpg)
Training this on a lot of sentences would give us a language model. A way to predict
P(next word | previous words) h0
x0 <START>
y0
x1 “cat”
h1
y1
slidecredit:AndrejKarpathy
![Page 25: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/25.jpg)
Training this on a lot of sentences would give us a language model. A way to predict
P(next word | previous words) h0
x0 <START>
y0
x1 “cat”
h1
y1
x2 “sat”
sample!
slidecredit:AndrejKarpathy
![Page 26: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/26.jpg)
Training this on a lot of sentences would give us a language model. A way to predict
P(next word | previous words) h0
x0 <START>
y0
x1 “cat”
h1
y1
x2 “sat”
h2
y2
slidecredit:AndrejKarpathy
![Page 27: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/27.jpg)
Training this on a lot of sentences would give us a language model. A way to predict
P(next word | previous words) h0
x0 <START>
y0
x1 “cat”
h1
y1
x2 “sat”
h2
y2
x3 “on”
sample!
slidecredit:AndrejKarpathy
![Page 28: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/28.jpg)
Training this on a lot of sentences would give us a language model. A way to predict
P(next word | previous words) h0
x0 <START>
y0
x1 “cat”
h1
y1
x2 “sat”
h2
y2
x3 “on”
h3
y3
slidecredit:AndrejKarpathy
![Page 29: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/29.jpg)
Training this on a lot of sentences would give us a language model. A way to predict
P(next word | previous words) h0
x0 <START>
y0
x1 “cat”
h1
y1
x2 “sat”
h2
y2
x3 “on”
h3
y3
x4 “mat”
sample!
slidecredit:AndrejKarpathy
![Page 30: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/30.jpg)
Training this on a lot of sentences would give us a language model. A way to predict
P(next word | previous words) h0
x0 <START>
y0
x1 “cat”
h1
y1
x2 “sat”
h2
y2
x3 “on”
h3
y3
x4 “mat”
h4
y4
slidecredit:AndrejKarpathy
![Page 31: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/31.jpg)
Training this on a lot of sentences would give us a language model. A way to predict
P(next word | previous words) h0
x0 <START>
y0
x1 “cat”
h1
y1
x2 “sat”
h2
y2
x3 “on”
h3
y3
x4 “mat”
h4
y4
samples <END>? done.
slidecredit:AndrejKarpathy
![Page 32: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/32.jpg)
“straw hat”
training example
slidecredit:AndrejKarpathy
![Page 33: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/33.jpg)
“straw hat”
training example
slidecredit:AndrejKarpathy
![Page 34: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/34.jpg)
“straw hat”
training example
X
slidecredit:AndrejKarpathy
![Page 35: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/35.jpg)
“straw hat”
training example
X
h0
x0 <START>
y0
x1 “straw”
h1
y1
x2 “hat”
h2
y2
<START> straw hat
slidecredit:AndrejKarpathy
![Page 36: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/36.jpg)
“straw hat”
training example
X
h0
x0 <START>
y0
x1 “straw”
h1
y1
x2 “hat”
h2
y2
<START> straw hat
before: h0 = tanh(0, Wxh * x0)
now: h0 = tanh(0, Wxh * x0 + Wih * v)
slidecredit:AndrejKarpathy
![Page 37: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/37.jpg)
test image
slidecredit:AndrejKarpathy
![Page 38: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/38.jpg)
test image
x0 <START>
<START>
slidecredit:AndrejKarpathy
![Page 39: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/39.jpg)
h0
x0 <START>
y0
<START>
test image
slidecredit:AndrejKarpathy
![Page 40: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/40.jpg)
h0
x0 <START>
y0
<START>
test image
straw
sample!
slidecredit:AndrejKarpathy
![Page 41: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/41.jpg)
h0
x0 <START>
y0
<START>
test image
straw
h1
y1
slidecredit:AndrejKarpathy
![Page 42: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/42.jpg)
h0
x0 <START>
y0
<START>
test image
straw
h1
y1
hat
sample!
slidecredit:AndrejKarpathy
![Page 43: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/43.jpg)
h0
x0 <START>
y0
<START>
test image
straw
h1
y1
hat
h2
y2
slidecredit:AndrejKarpathy
![Page 44: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/44.jpg)
h0
x0 <START>
y0
<START>
test image
straw
h1
y1
hat
h2
y2
sample! <END> token => finish.
slidecredit:AndrejKarpathy
![Page 45: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/45.jpg)
Sequence Learning• Instances of the form x = <x1, x2, x3, …, xT>
• Variable sequence length T
• Learn a transition function f with parameters W:
• f should update hidden state ht and output yt
h0 := 0
for t = 1, 2, 3, …, T:
<yt, ht> = fW(xt, ht-1)
ht-1
xt
ht
yt
f
slidecredit:JeffDonahue
![Page 46: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/46.jpg)
Sequence LearningEquivalent to a T-layer deep network, unrolled in time
0
x1
h1
z1
f
x2
h2
z2
f
xT
hT
zT
fhT-1…
slidecredit:JeffDonahue
![Page 47: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/47.jpg)
Sequence Learning• What should the transition function f be?
• At a minimum, we want something non-linear and differentiable
ht-1
xt
ht
zt
f
slidecredit:JeffDonahue
![Page 48: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/48.jpg)
Sequence Learning• First attempt — a “vanilla” RNN:
ht = σ(Whxxt + Whhht-1 + bh)
zt = σ(Whzht + bz)
• Problems
• Difficult to train — vanishing/exploding gradients
• Unable to “select” inputs, hidden state, outputs
slidecredit:JeffDonahue
![Page 49: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/49.jpg)
SequenceLearning
• LSTM - Long Short Term Memory[Hochreiter & Schmidhuber, 1997]
• Selectively propagate or forget hidden state • Allows long-term dependencies to be learned
• Effective for • speech recognition • handwriting recognition • translation • parsing
slidecredit:MarcusRohrbach
![Page 50: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/50.jpg)
MarcusRohrbach|LRCN–anArchitectureforVisualRecognition,Description,andQuestionAnswering|
LSTMforsequencemodeling
50
slidecredit:MarcusRohrbach
![Page 51: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/51.jpg)
MarcusRohrbach|LRCN–anArchitectureforVisualRecognition,Description,andQuestionAnswering|
LSTMforsequencemodeling
51
slidecredit:MarcusRohrbach
![Page 52: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/52.jpg)
Sequence Learning
LSTM (Hochreiter &
Schmidhuber, 1997)
ct-1
Exactly remember previous
cell state — discard input
0
ct-11
ct-1
slidecredit:JeffDonahue
![Page 53: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/53.jpg)
Sequence Learning
LSTM (Hochreiter &
Schmidhuber, 1997)
Forget previous
cell state — only use
modulated input Wxt-1
Wxt-1
00
1Wxt-1
slidecredit:JeffDonahue
![Page 54: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/54.jpg)
LRCN• Long-term Recurrent Convolutional Networks
• End-to-end trainable framework for sequence problems in vision
CNN LSTM
slidecredit:JeffDonahue
![Page 55: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/55.jpg)
Image Description
CNN
LSTM LSTM LSTM LSTM LSTMsequential output
a<BOS> dog is jumping <EOS>
slidecredit:JeffDonahue
![Page 56: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/56.jpg)
Image Description
CNN
LSTM LSTM LSTM LSTM LSTMsequential output
a<BOS> dog is jumping <EOS>
= embed a one-hot vector
slidecredit:JeffDonahue
![Page 57: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/57.jpg)
Image Description
CNN
LSTM LSTM LSTM LSTM LSTMsingle LSTM layer
a<BOS> dog is jumping <EOS>
= embed a one-hot vector
slidecredit:JeffDonahue
![Page 58: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/58.jpg)
Image Description
two LSTM layers
LSTM
LSTM
LSTM
LSTM
LSTM
LSTM
LSTM
LSTM
LSTM
LSTM
<BOS> <EOS>
CNN
a dog is jumping
slidecredit:JeffDonahue
![Page 59: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/59.jpg)
Image Description
two LSTM layers, factored
CNN
LSTM
LSTM
LSTM
LSTM
LSTM
LSTM
LSTM
LSTM
LSTM
LSTM
dog is jumping <EOS>a <BOS>
slidecredit:JeffDonahue
![Page 60: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/60.jpg)
Image Description
ArchitectureFlickr30k [1]
Caption-to-Image Recall@1
Single Layer 14.1%
Two Layer 3.8%
Two Layer, Factored 17.5%
Four Layer, Factored 15.8%
[1] P. Hodosh, A. Young, M. Lai, and J. Hockenmaier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions.
slidecredit:JeffDonahue
![Page 61: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/61.jpg)
Image Description
COCO [1] CIDEr-D c5
ScoresCaffeNet VGGNet
[2]
Raw 68.8% 77.3%
Finetuned 75.8% 83.9%
[1] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick. Microsoft coco: Com- mon objects in context. arXiv preprint arXiv:1405.0312, 2014. [2] K. Simonyan & A. Zisserman. “Very Deep Convolutional Networks for Large-Scale Image Recognition”. ICLR 2015.
slidecredit:JeffDonahue
![Page 62: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/62.jpg)
Image Descriptionslidecredit:JeffDonahue
![Page 63: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/63.jpg)
Image Descriptionslidecredit:JeffDonahue
![Page 64: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/64.jpg)
Activity Recognition
LSTM
CNN
LSTM
CNN
LSTM
CNN
LSTM
CNN
studying running jumping jumping
Average
jumping
sequential input
slidecredit:JeffDonahue
![Page 65: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/65.jpg)
Activity Recognition
UCF101Class.Acc.
RGB Optical Flow
RGB+Flow
Single-Frame CNN
67.7% 72.2% 78.8%
LRCN 68.2% 77.5% 82.7%
Khurram Soomro, Amir Roshan Zamir and Mubarak Shah, UCF101: A Dataset of 101 Human Action Classes From Videos in The Wild., CRCV-TR-12-01, November, 2012.
slidecredit:JeffDonahue
![Page 66: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/66.jpg)
Video Description
LSTM
CNN
LSTM
LSTM
LSTM
LSTM LSTM LSTM LSTMLSTM
CNN
LSTM
LSTM
CNN
LSTM
LSTM
CNN
LSTM
sequential input & output
<BOS>
LSTM LSTMLSTMLSTM
<EOS>jumpingisdoga
slidecredit:JeffDonahue
![Page 67: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/67.jpg)
Video Description
Coherent Multi-Sentence Video Description with Variable Level of Detail. A. Rohrbach, M. Rohrbach, W. Qiu, A. Friedrich and M. Pinkal and B. Schiele. GCPR, 2014.
MPII TACoS Multi-Level Dataset
slidecredit:JeffDonahue
![Page 68: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/68.jpg)
Video Description
LSTM
LSTM
LSTM LSTM LSTM LSTM
LSTM LSTMLSTMLSTM
Pre-trained Detector Predictions
<BOS> <EOS>vegetablescutsmana
slidecredit:JeffDonahue
![Page 69: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/69.jpg)
Video Description
ApproachGeneration Accuracy(BLEU)
SMT 26.9%
LRCN 28.8%
slidecredit:JeffDonahue
![Page 70: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/70.jpg)
Video Description
CNNCNNCNNCNN
Average
LSTM LSTM LSTM LSTM LSTM
a dog is jumping <EOS><BOS>
Subhashini Venugopalan, Huijuan Xu, Jeff Donahue, Marcus Rohrbach, Raymond Mooney, Kate Saenko. “Translating Videos to Natural Language Using Deep Recurrent Neural Networks,” NAACL 2015 (oral). http://arxiv.org/abs/1412.4729
slidecredit:JeffDonahue
![Page 71: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/71.jpg)
Wow I can’t believe that worked
![Page 72: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/72.jpg)
Wow I can’t believe that worked
![Page 73: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/73.jpg)
Well, I can kind of see it
![Page 74: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/74.jpg)
Well, I can kind of see it
![Page 75: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/75.jpg)
Not sure what happened there...
![Page 76: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/76.jpg)
“The Unreasonable Effectiveness of Recurrent Neural Networks”
karpathy.github.io
slidecredit:AndrejKarpathy
![Page 77: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/77.jpg)
Character-level language model example
Vocabulary: [h,e,l,o]
Example training sequence: “hello”
slidecredit:AndrejKarpathy
![Page 78: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/78.jpg)
slidecredit:AndrejKarpathy
![Page 79: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/79.jpg)
slidecredit:AndrejKarpathy
![Page 80: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/80.jpg)
slidecredit:AndrejKarpathy
![Page 81: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/81.jpg)
slidecredit:AndrejKarpathy
![Page 82: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/82.jpg)
Try it yourself: char-rnn on Github (uses Torch7)
slidecredit:AndrejKarpathy
![Page 83: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/83.jpg)
Cooking Recipes
slidecredit:AndrejKarpathy
![Page 84: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/84.jpg)
Obama Speeches
slidecredit:AndrejKarpathy
![Page 85: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/85.jpg)
slidecredit:AndrejKarpathy
![Page 86: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/86.jpg)
slidecredit:AndrejKarpathy
![Page 87: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/87.jpg)
Learning from Linux Source Code
slidecredit:AndrejKarpathy
![Page 88: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/88.jpg)
slidecredit:AndrejKarpathy
![Page 89: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/89.jpg)
slidecredit:AndrejKarpathy
![Page 90: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/90.jpg)
slidecredit:AndrejKarpathy
![Page 91: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/91.jpg)
Yoav Goldberg n-gram experiments
Order 10 ngram model on Shakespeare:
slidecredit:AndrejKarpathy
![Page 92: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/92.jpg)
But on Linux:
slidecredit:AndrejKarpathy
![Page 93: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/93.jpg)
Visualizing and Understanding Recurrent Networks Andrej Karpathy*, Justin Johnson*, Li Fei-Fei (on arXiv.org as of June 2015)
slidecredit:AndrejKarpathy
![Page 94: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/94.jpg)
Hunting interpretable cells
![Page 95: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/95.jpg)
Hunting interpretable cells
slidecredit:AndrejKarpathy
![Page 96: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/96.jpg)
Hunting interpretable cells
quote detection cell
slidecredit:AndrejKarpathy
![Page 97: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/97.jpg)
Hunting interpretable cells
line length tracking cell
slidecredit:AndrejKarpathy
![Page 98: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/98.jpg)
Hunting interpretable cells
slidecredit:AndrejKarpathy
![Page 99: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/99.jpg)
Hunting interpretable cells
if statement cell
slidecredit:AndrejKarpathy
![Page 100: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/100.jpg)
Hunting interpretable cells
quote/comment cell
slidecredit:AndrejKarpathy
![Page 101: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/101.jpg)
Hunting interpretable cells
code depth cell
slidecredit:AndrejKarpathy
![Page 102: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/102.jpg)
Hunting interpretable cells
something interesting cell (not quite sure what)
slidecredit:AndrejKarpathy
![Page 103: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/103.jpg)
103
![Page 104: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/104.jpg)
104
slidecredit:YanLeCun
![Page 105: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/105.jpg)
105
![Page 106: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/106.jpg)
106
slidecredit:YannLeCun
![Page 107: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/107.jpg)
High Level Computer Vision - June 14, 2o17 107
![Page 108: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/108.jpg)
108
![Page 109: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/109.jpg)
109
slidecredit:YannLeCun
![Page 110: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/110.jpg)
110
slidecredit:XiaogangWang
![Page 111: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/111.jpg)
111
slidecredit:XiaogangWang
![Page 112: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/112.jpg)
112
slidecredit:XiaogangWang
![Page 113: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/113.jpg)
113
slidecredit:XiaogangWang
![Page 114: High Level Computer Vision Deep Learning for Computer Vision … · 2017. 6. 14. · High Level Computer Vision - June 14, 2o17 Overview • Recurrent Neural Networks ‣ motivation](https://reader033.vdocuments.site/reader033/viewer/2022053023/6053b9717bab064ba721c414/html5/thumbnails/114.jpg)
b
114
slidecredit:XiaogangWang