deep learning - with artificial neural networks · ... a history of biological & artificial...
TRANSCRIPT
![Page 1: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/1.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Deep Learningwith Artificial Neural Networks
Chief Data Scientist at untapt
Metis — January 17th, 2017(slides available at jonkrohn.com/talks)
![Page 2: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/2.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Outline1 Antecedents
The Velocity of Technological ProgressCase Study: A History of Biological & Artificial VisionMachine Intelligence
2 TheoryBiological & Artificial NeuronsNeural NetworksDeep Neural Networks
3 Contemporary ApplicationsConvolutional Neural NetworksLong Short-Term Memory Recurrent Neural NetworksDeep Learning at untaptDeep Reinforcement LearningBuilding Blocks
4 The Future
![Page 3: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/3.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Outline1 Antecedents
The Velocity of Technological ProgressCase Study: A History of Biological & Artificial VisionMachine Intelligence
2 TheoryBiological & Artificial NeuronsNeural NetworksDeep Neural Networks
3 Contemporary ApplicationsConvolutional Neural NetworksLong Short-Term Memory Recurrent Neural NetworksDeep Learning at untaptDeep Reinforcement LearningBuilding Blocks
4 The Future
![Page 4: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/4.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Outline1 Antecedents
The Velocity of Technological ProgressCase Study: A History of Biological & Artificial VisionMachine Intelligence
2 TheoryBiological & Artificial NeuronsNeural NetworksDeep Neural Networks
3 Contemporary ApplicationsConvolutional Neural NetworksLong Short-Term Memory Recurrent Neural NetworksDeep Learning at untaptDeep Reinforcement LearningBuilding Blocks
4 The Future
![Page 5: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/5.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Outline1 Antecedents
The Velocity of Technological ProgressCase Study: A History of Biological & Artificial VisionMachine Intelligence
2 TheoryBiological & Artificial NeuronsNeural NetworksDeep Neural Networks
3 Contemporary ApplicationsConvolutional Neural NetworksLong Short-Term Memory Recurrent Neural NetworksDeep Learning at untaptDeep Reinforcement LearningBuilding Blocks
4 The Future
![Page 6: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/6.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Outline1 Antecedents
The Velocity of Technological ProgressCase Study: A History of Biological & Artificial VisionMachine Intelligence
2 TheoryBiological & Artificial NeuronsNeural NetworksDeep Neural Networks
3 Contemporary ApplicationsConvolutional Neural NetworksLong Short-Term Memory Recurrent Neural NetworksDeep Learning at untaptDeep Reinforcement LearningBuilding Blocks
4 The Future
![Page 7: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/7.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Jeanne Calment
![Page 8: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/8.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Jeanne Calment(1875-1997 — i.e., 122 years)
![Page 9: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/9.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Life in the Year 2138
![Page 10: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/10.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 11: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/11.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 12: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/12.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Outline1 Antecedents
The Velocity of Technological ProgressCase Study: A History of Biological & Artificial VisionMachine Intelligence
2 TheoryBiological & Artificial NeuronsNeural NetworksDeep Neural Networks
3 Contemporary ApplicationsConvolutional Neural NetworksLong Short-Term Memory Recurrent Neural NetworksDeep Learning at untaptDeep Reinforcement LearningBuilding Blocks
4 The Future
![Page 13: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/13.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 14: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/14.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 15: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/15.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Hubel & Wiesel (1959)
![Page 16: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/16.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 17: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/17.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 18: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/18.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 19: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/19.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Camera Obscurada Vinci (15th Century)
![Page 20: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/20.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Block WorldLarry Roberts (1965)
![Page 21: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/21.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Viola & Jones (2001)
![Page 22: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/22.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
NeurocognitronFukushima (1980)
![Page 23: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/23.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
MNIST & LeNet-5LeCun et al. (1998)
![Page 24: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/24.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
LeCun, Boutou, Bengio &Haffner (1998)
![Page 25: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/25.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 26: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/26.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
ImageNet
![Page 27: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/27.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
ImageNet Classification ErrorKrizhevsky, Sutskever & Hinton (2012)
![Page 28: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/28.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Krizhevsky et al. (2012)
![Page 29: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/29.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Outline1 Antecedents
The Velocity of Technological ProgressCase Study: A History of Biological & Artificial VisionMachine Intelligence
2 TheoryBiological & Artificial NeuronsNeural NetworksDeep Neural Networks
3 Contemporary ApplicationsConvolutional Neural NetworksLong Short-Term Memory Recurrent Neural NetworksDeep Learning at untaptDeep Reinforcement LearningBuilding Blocks
4 The Future
![Page 30: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/30.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 31: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/31.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 32: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/32.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 33: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/33.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Deep Learning in Late 2016
[Image to Lyrics and Music]
[Daddy’s Car]
[Sunspring]
![Page 34: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/34.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 35: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/35.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Outline1 Antecedents
The Velocity of Technological ProgressCase Study: A History of Biological & Artificial VisionMachine Intelligence
2 TheoryBiological & Artificial NeuronsNeural NetworksDeep Neural Networks
3 Contemporary ApplicationsConvolutional Neural NetworksLong Short-Term Memory Recurrent Neural NetworksDeep Learning at untaptDeep Reinforcement LearningBuilding Blocks
4 The Future
![Page 36: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/36.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Biological Neuron Morphology
![Page 37: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/37.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
PerceptronRosenblatt (1957)
![Page 38: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/38.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Biological Neuron PhysiologyThe Binary Action Potential
![Page 39: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/39.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
PerceptronRosenblatt (1957)
![Page 40: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/40.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Multi-Layer Perceptron
![Page 41: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/41.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Multi-Layer Perceptron
![Page 42: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/42.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Sigmoid Neuron
![Page 43: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/43.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
tanh Neuron
![Page 44: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/44.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
ReLU: Rectified Linear UnitsNair & Hinton (2010); Maas, Hannun & Ng (2014)
![Page 45: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/45.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Outline1 Antecedents
The Velocity of Technological ProgressCase Study: A History of Biological & Artificial VisionMachine Intelligence
2 TheoryBiological & Artificial NeuronsNeural NetworksDeep Neural Networks
3 Contemporary ApplicationsConvolutional Neural NetworksLong Short-Term Memory Recurrent Neural NetworksDeep Learning at untaptDeep Reinforcement LearningBuilding Blocks
4 The Future
![Page 46: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/46.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
MNISTHandwritten Digits
![Page 47: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/47.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Fully-Connected Neural NetSingle Hidden Layer
![Page 48: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/48.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Popular LibrariesNever pay for software
• Theano• Torch• Caffe• TensorFlow [demo]
Higher-Level APIs:• TFLearn• Keras
![Page 49: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/49.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Popular LibrariesNever pay for software
• Theano• Torch• Caffe• TensorFlow [demo]
Higher-Level APIs:• TFLearn• Keras
![Page 50: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/50.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Popular LibrariesNever pay for software
• Theano• Torch• Caffe• TensorFlow [demo]
Higher-Level APIs:• TFLearn• Keras
![Page 51: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/51.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Popular LibrariesNever pay for software
• Theano• Torch• Caffe• TensorFlow [demo]
Higher-Level APIs:• TFLearn• Keras
![Page 52: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/52.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Popular LibrariesNever pay for software
• Theano• Torch• Caffe• TensorFlow [demo]
Higher-Level APIs:• TFLearn• Keras
![Page 53: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/53.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Popular LibrariesNever pay for software
• Theano• Torch• Caffe• TensorFlow [demo]
Higher-Level APIs:• TFLearn• Keras
![Page 54: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/54.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Outline1 Antecedents
The Velocity of Technological ProgressCase Study: A History of Biological & Artificial VisionMachine Intelligence
2 TheoryBiological & Artificial NeuronsNeural NetworksDeep Neural Networks
3 Contemporary ApplicationsConvolutional Neural NetworksLong Short-Term Memory Recurrent Neural NetworksDeep Learning at untaptDeep Reinforcement LearningBuilding Blocks
4 The Future
![Page 55: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/55.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Deep Fully-Connected Net3 (or more) Hidden Layers
![Page 56: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/56.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Synaptic Pruning
![Page 57: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/57.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
(Stochastic) Gradient DescentAdam = AdaGrad + RMSprop
![Page 58: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/58.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Backpropagationcomputes error & gradient of cost function
![Page 59: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/59.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Overfitting...and avoiding it
• L1/L2 regularization• dropout• artificial data set expansion
![Page 60: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/60.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Overfitting...and avoiding it
• L1/L2 regularization• dropout• artificial data set expansion
![Page 61: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/61.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Overfitting...and avoiding it
• L1/L2 regularization• dropout• artificial data set expansion
![Page 62: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/62.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Improving Neural NetworksAttribute & Hyperparameter Tuning
• problem simplification• number and width of layers• cost fxn: quadratic, cross-entropy, log-likelihood, &c.• more epochs, early stopping• clever initialization of weights and biases• learning rate η, variable schedule• regularization parameter λ• mini-batch size• automation, e.g., with Spearmint
![Page 63: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/63.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Improving Neural NetworksAttribute & Hyperparameter Tuning
• problem simplification• number and width of layers• cost fxn: quadratic, cross-entropy, log-likelihood, &c.• more epochs, early stopping• clever initialization of weights and biases• learning rate η, variable schedule• regularization parameter λ• mini-batch size• automation, e.g., with Spearmint
![Page 64: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/64.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Improving Neural NetworksAttribute & Hyperparameter Tuning
• problem simplification• number and width of layers• cost fxn: quadratic, cross-entropy, log-likelihood, &c.• more epochs, early stopping• clever initialization of weights and biases• learning rate η, variable schedule• regularization parameter λ• mini-batch size• automation, e.g., with Spearmint
![Page 65: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/65.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Improving Neural NetworksAttribute & Hyperparameter Tuning
• problem simplification• number and width of layers• cost fxn: quadratic, cross-entropy, log-likelihood, &c.• more epochs, early stopping• clever initialization of weights and biases• learning rate η, variable schedule• regularization parameter λ• mini-batch size• automation, e.g., with Spearmint
![Page 66: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/66.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Improving Neural NetworksAttribute & Hyperparameter Tuning
• problem simplification• number and width of layers• cost fxn: quadratic, cross-entropy, log-likelihood, &c.• more epochs, early stopping• clever initialization of weights and biases• learning rate η, variable schedule• regularization parameter λ• mini-batch size• automation, e.g., with Spearmint
![Page 67: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/67.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Improving Neural NetworksAttribute & Hyperparameter Tuning
• problem simplification• number and width of layers• cost fxn: quadratic, cross-entropy, log-likelihood, &c.• more epochs, early stopping• clever initialization of weights and biases• learning rate η, variable schedule• regularization parameter λ• mini-batch size• automation, e.g., with Spearmint
![Page 68: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/68.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Improving Neural NetworksAttribute & Hyperparameter Tuning
• problem simplification• number and width of layers• cost fxn: quadratic, cross-entropy, log-likelihood, &c.• more epochs, early stopping• clever initialization of weights and biases• learning rate η, variable schedule• regularization parameter λ• mini-batch size• automation, e.g., with Spearmint
![Page 69: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/69.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Improving Neural NetworksAttribute & Hyperparameter Tuning
• problem simplification• number and width of layers• cost fxn: quadratic, cross-entropy, log-likelihood, &c.• more epochs, early stopping• clever initialization of weights and biases• learning rate η, variable schedule• regularization parameter λ• mini-batch size• automation, e.g., with Spearmint
![Page 70: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/70.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Improving Neural NetworksAttribute & Hyperparameter Tuning
• problem simplification• number and width of layers• cost fxn: quadratic, cross-entropy, log-likelihood, &c.• more epochs, early stopping• clever initialization of weights and biases• learning rate η, variable schedule• regularization parameter λ• mini-batch size• automation, e.g., with Spearmint
![Page 71: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/71.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
UniversalitySolve Any Continuous Function (Nielsen, 2015)
![Page 72: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/72.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Unstable GradientTypically Vanishes (but can Explode)
![Page 73: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/73.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 74: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/74.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 75: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/75.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Outline1 Antecedents
The Velocity of Technological ProgressCase Study: A History of Biological & Artificial VisionMachine Intelligence
2 TheoryBiological & Artificial NeuronsNeural NetworksDeep Neural Networks
3 Contemporary ApplicationsConvolutional Neural NetworksLong Short-Term Memory Recurrent Neural NetworksDeep Learning at untaptDeep Reinforcement LearningBuilding Blocks
4 The Future
![Page 76: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/76.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Hubel & Wiesel (1959)
![Page 77: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/77.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 78: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/78.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 79: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/79.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
DeConvNetYosinski et al. (2015)
[Deep Visualization Toolbox]
![Page 80: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/80.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
“2.5-dimension” CT ScansRoth et al. (2015)
![Page 81: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/81.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Computer-Aided DetectionShin et al. (2016); Roth et al. (2016)
![Page 82: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/82.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 83: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/83.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 84: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/84.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Outline1 Antecedents
The Velocity of Technological ProgressCase Study: A History of Biological & Artificial VisionMachine Intelligence
2 TheoryBiological & Artificial NeuronsNeural NetworksDeep Neural Networks
3 Contemporary ApplicationsConvolutional Neural NetworksLong Short-Term Memory Recurrent Neural NetworksDeep Learning at untaptDeep Reinforcement LearningBuilding Blocks
4 The Future
![Page 85: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/85.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Long Short-Term MemoryHochreiter & Schmidhuber (1997)
![Page 86: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/86.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Word2VecMikolov, Sutskever, Chen, Corrado & Dean (2013)
![Page 87: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/87.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
t-SNEHinton & van der Maaten (2008)
![Page 88: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/88.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Word2Vec + t-SNE
![Page 89: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/89.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Word2Vec + t-SNE
![Page 90: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/90.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Word2Vec + t-SNE
![Page 91: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/91.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Outline1 Antecedents
The Velocity of Technological ProgressCase Study: A History of Biological & Artificial VisionMachine Intelligence
2 TheoryBiological & Artificial NeuronsNeural NetworksDeep Neural Networks
3 Contemporary ApplicationsConvolutional Neural NetworksLong Short-Term Memory Recurrent Neural NetworksDeep Learning at untaptDeep Reinforcement LearningBuilding Blocks
4 The Future
![Page 92: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/92.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
untaptDigital Recruitment Platform
![Page 93: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/93.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
untapt
![Page 94: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/94.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
untaptCandidate-Side Feedback
![Page 95: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/95.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
untaptClient-Side Feedback
![Page 96: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/96.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
untaptMulti-Stage Bayesian Regression
Krohn, Rives-Corbett & Donner (2016)
![Page 97: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/97.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
untapt
Krohn, Rives-Corbett & Donner (2016)
![Page 98: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/98.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
untaptEnsemble with Deep Neural Net
deep-orange.untapt.com
![Page 99: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/99.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 100: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/100.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Outline1 Antecedents
The Velocity of Technological ProgressCase Study: A History of Biological & Artificial VisionMachine Intelligence
2 TheoryBiological & Artificial NeuronsNeural NetworksDeep Neural Networks
3 Contemporary ApplicationsConvolutional Neural NetworksLong Short-Term Memory Recurrent Neural NetworksDeep Learning at untaptDeep Reinforcement LearningBuilding Blocks
4 The Future
![Page 101: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/101.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
AlphaGOSilver et al. (2016)
![Page 102: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/102.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Deep Q-LearningMnih et al. (2015)
[Atari Games]
![Page 103: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/103.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Deep Q-LearningMnih et al. (2015)
[Atari Games]
![Page 104: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/104.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Deep Q-LearningMnih et al. (2015)
[Atari Games]
![Page 105: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/105.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
[OpenAI Universe]
[Google DeepMind Lab]
![Page 106: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/106.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Outline1 Antecedents
The Velocity of Technological ProgressCase Study: A History of Biological & Artificial VisionMachine Intelligence
2 TheoryBiological & Artificial NeuronsNeural NetworksDeep Neural Networks
3 Contemporary ApplicationsConvolutional Neural NetworksLong Short-Term Memory Recurrent Neural NetworksDeep Learning at untaptDeep Reinforcement LearningBuilding Blocks
4 The Future
![Page 107: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/107.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Hardware
• local machine• build your own cluster• AWS• GPU(s) / TPU(s)
![Page 108: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/108.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Hardware
• local machine• build your own cluster• AWS• GPU(s) / TPU(s)
![Page 109: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/109.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Hardware
• local machine• build your own cluster• AWS• GPU(s) / TPU(s)
![Page 110: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/110.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Hardware
• local machine• build your own cluster• AWS• GPU(s) / TPU(s)
![Page 111: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/111.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Resources for Human Learning
![Page 112: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/112.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Transfer Learning
![Page 113: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/113.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Jeanne Calment
![Page 114: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/114.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Jeanne Calment(1875-1997 — i.e., 122 years)
![Page 115: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/115.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Life in the Year 2138
![Page 116: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/116.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Thiel & Masters (2014)
![Page 117: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/117.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Thiel & Masters (2014)
![Page 118: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/118.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Thiel & Masters (2014)
![Page 119: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/119.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 120: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/120.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 121: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/121.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 122: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/122.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Pinker & Mack (2014)
![Page 123: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/123.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Pinker & Mack (2014)
![Page 124: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/124.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
Pinker & Mack (2014)
![Page 125: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/125.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 126: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/126.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 127: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/127.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 128: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/128.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 129: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/129.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 130: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/130.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 131: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/131.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 132: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/132.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 133: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/133.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 134: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/134.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 135: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/135.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 136: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/136.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future
![Page 137: Deep Learning - with Artificial Neural Networks · ... A History of Biological & Artificial Vision Machine Intelligence ... Deep Learning in Late 2016 [Image to Lyrics and Music]](https://reader033.vdocuments.site/reader033/viewer/2022052516/5ac1231e7f8b9aca388cb55e/html5/thumbnails/137.jpg)
DeepLearning
AntecedentsTech Velocity
Vision Case Study
Machine Intelligence
TheoryNeural Units
Neural Nets
Deep Neural Nets
ApplicationConvNets
LSTMs
untapt
Reinforcement
Building Blocks
The Future