cutting edge of machine learning
DESCRIPTION
by Sergii ShelpukTRANSCRIPT
![Page 2: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/2.jpg)
Classification Problem
Recognize what is a bike and what is a moon
![Page 3: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/3.jpg)
Classification Problem
Classifier
©A. Ng
![Page 4: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/4.jpg)
Classification Problem
pixelintensity
![Page 5: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/5.jpg)
Classification Problem
Raw data does not represent the picture well. You need some smart features
contains wheels
contains seas
![Page 6: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/6.jpg)
Feature Extraction
Classifier
Feat
ure
extr
acto
r
©A. Ng
![Page 7: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/7.jpg)
Feature Extraction
Can we do better?
![Page 8: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/8.jpg)
Neural Networks
a
a
a
a
a
a
a
a
a
a
a
a
a
afeat
ures bike
moon
![Page 9: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/9.jpg)
Neural Networks
![Page 10: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/10.jpg)
Neural Networks
aX
a0
a1
a2
w0
w1
w2
Activation function:aX = f(a0, a1, a2, w0, w1, w2)
Example (logistic):aX = 1 / (1 + e-(a0*w0+a1*w1+a2*w2))
![Page 11: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/11.jpg)
Autoencoder
![Page 12: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/12.jpg)
Autoencoder
© H. Lee et al.
![Page 13: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/13.jpg)
Autoencoder
© Q Le et al.
![Page 14: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/14.jpg)
Deep Learning Neural Network
Pre-trained as AutoencoderTypical classification
neural network
Moon
![Page 15: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/15.jpg)
Deep Learning Neural Network
Vide
oText/N
LPIm
ages
©A. Ng
![Page 16: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/16.jpg)
Deep Learning Neural Network
Hints and Tips Using unlabeled data Avoiding overfitting Computational efficiency
![Page 17: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/17.jpg)
Using Unlabeled Data
wheels
handlebar
![Page 18: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/18.jpg)
Avoiding Overfitting
Sparsity constraint limits variance of autoencoder
![Page 19: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/19.jpg)
Avoiding Overfitting
Dropout ensures generalization of the neural network
![Page 20: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/20.jpg)
Computational Efficiency
Thousands of cores Base Clock: 300-900 MHz Memory: 2-6 Gb Performance: up to 3.5 Tflops Instruction-level parallelism Shared memory Up to 4 devices in cluster
GPU computing provides cheapest computational power
![Page 21: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/21.jpg)
Feature Learning: MNIST
Data:
Features:
![Page 22: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/22.jpg)
Feature Learning: Galaxy Zoo
Data: Features:
![Page 23: Cutting edge of Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022062312/554a06d4b4c905557a8b55a6/html5/thumbnails/23.jpg)
Thank you!
?