building a neural network from scratch in r
TRANSCRIPT
Building a Neural Network from scratch
in ROwen Jones ∙ Bath Machine Learning Meetup 1∙ st March 2017
What is the most important network we’ll be using tonight?
Building a Social Network from scratch
in BMLMOwen Jones ∙ Bath Machine Learning Meetup 1∙ st March 2017
What’s a neural network?
What’s a neural network?
This is a dog
This is a dog
This is a cat
This is a dog
This is a cat
This is a cat
This is a dog
OK.
What’s a neural network?
Dog!
What is this?
Data
Kaggle competition page• https://kaggle.com/c/digit-recognizer
Data• https://kaggle.com/c/digit-recognizer/data
What are these numbers?
What are these numbers?
0 60 140 200 255
0 60 140 200 255
0 60 140 200 255
0 60 140 200 255
0 60 140 200 255
What are these numbers?
0 60 140 200 255
0 60 140 200 255
0 60 140 200 255
0 60 140 200 255
0 60 140 200 255
What are these numbers?
0 60 140 200 255 0 60 140 200 255 0 60 140 200 255 0 60 140 200 255 0 60 140 200 255
Why relabel 0?
0 1 2 3 4 5 6 7 8 9
1 2 3 4 5 6 7 8 9 10
Zero-indexed
One-indexed
1st 2nd 3rd
Why relabel 0?
0 1 2 3 4 5 6 7 8 9
1 2 3 4 5 6 7 8 9 10
Zero-indexed
One-indexed
1st 2nd 3rd
Why relabel 0?
0 1 2 3 4 5 6 7 8 9
1 2 3 4 5 6 7 8 9 10
Zero-indexed
One-indexed
1st 2nd 3rd
0 0 1 0 0 0 0 0 0 03
Making sets
Everything!
100%
Making sets
Cross-validation set
20%
Test set
20%
Training set
60%
How does the net make predictions?
How does the net make predictions?
How does the net make predictions?
How does the net make predictions?
How does the net make predictions?
Sum = 1000 Sum = 300 Sum = 800
Network structure
Network structure
Network structure
Network structure
Matrices
Matrices
Biological neuron
Biological neuron
Biological neuron
Activation functions
Activation functions
Network structure
How wrong were we?Prediction (A3)
Actual 0 0 0 0 0 0 0 0 1 0
0.08 0.31 0.85 0.37 0.43 0.51 0.12 0.54 0.71 0.42
-log(A3) * Actual 0 0 0 0 0 0 0 0 0.34 0
How wrong were we?Prediction (A3)
Actual 0 0 0 0 0 0 0 0 1 0
0.08 0.31 0.85 0.37 0.43 0.51 0.12 0.54 0.71 0.42
1 1 1 1 1 1 1 1 0 11 - Actual
-log(A3) * Actual 0 0 0 0 0 0 0 0 0.34 0
0.08 0.37 1.90 0.46 0.56 0.71 0.13 0.78 0 0.54-log(1 - A3) * (1 - Actual)
0.92 0.69 0.15 0.63 0.57 0.49 0.88 0.46 0.29 0.581 - A3
How wrong were we?Prediction (A3)
Actual 0 0 0 0 0 0 0 0 1 0
0.08 0.31 0.85 0.37 0.43 0.51 0.12 0.54 0.71 0.42
1 1 1 1 1 1 1 1 0 11 - Actual
-log(A3) * Actual 0 0 0 0 0 0 0 0 0.34 0
0.08 0.37 1.90 0.46 0.56 0.71 0.13 0.78 0 0.54-log(1 - A3) * (1 - Actual)
0.92 0.69 0.15 0.63 0.57 0.49 0.88 0.46 0.29 0.581 - A3
Cost (J) = 0.34 + 0.08 + 0.37 + 1.90 + 0.46 + 0.56 + 0.71 + 0.13 + 0.78 + 0.54 = 5.87
Multilayer perceptron
Convolutional neural networks
Recurrent neural networks
Recurrent neural networks
Learning from experience
This is a dog
This is a dog
This is a cat
This is a dog
This is a cat
This is a cat
This is a dog
OK.
Learning from experience
Dog!
What is this?
Learning from experience
Cat!
What is this?
Learning from experience
Dog?
What is this?
Learning from experience
Cat???
What is this?
Learning from experienceWhat even
is that?
What comes next?
Once upon a
General intelligence
Backpropagation
Backpropagation
Backpropagation
Backpropagation
Backpropagation
Backpropagation
Backpropagation
Backpropagation
Contact me
Website: https://owenjonesuob.github.io
Email: [email protected]
Twitter: @owenjonesuob
GitHub: https://github.com/owenjonesuob
LinkedIn: https://uk.linkedin.com/in/owenjonesuob
• https://uk.linkedin.com/in/owenjonesuob