introduction to neural networks - in.tum.de · introduction to neural networks freek stulp. 2...
TRANSCRIPT
![Page 1: Introduction to Neural Networks - in.tum.de · Introduction to Neural Networks Freek Stulp. 2 Overview Biological Background Artificial Neuron Classes of Neural Networks 1. Perceptrons](https://reader030.vdocuments.site/reader030/viewer/2022040112/5ec4e729d39111156f60e3a3/html5/thumbnails/1.jpg)
Introduction to Neural Networks
Freek Stulp
![Page 2: Introduction to Neural Networks - in.tum.de · Introduction to Neural Networks Freek Stulp. 2 Overview Biological Background Artificial Neuron Classes of Neural Networks 1. Perceptrons](https://reader030.vdocuments.site/reader030/viewer/2022040112/5ec4e729d39111156f60e3a3/html5/thumbnails/2.jpg)
2
Overview
Biological BackgroundArtificial NeuronClasses of Neural Networks
1. Perceptrons2. Multi-Layered Feed-Forward Networks3. Recurrent Networks
Conclusion
![Page 3: Introduction to Neural Networks - in.tum.de · Introduction to Neural Networks Freek Stulp. 2 Overview Biological Background Artificial Neuron Classes of Neural Networks 1. Perceptrons](https://reader030.vdocuments.site/reader030/viewer/2022040112/5ec4e729d39111156f60e3a3/html5/thumbnails/3.jpg)
3
Biological BackgroundNeuron consists of:? Cell body? Dendrites? Axon? Synapses
Neural activation :? Throught dendrites/axon? Synapses have different
strengths
![Page 4: Introduction to Neural Networks - in.tum.de · Introduction to Neural Networks Freek Stulp. 2 Overview Biological Background Artificial Neuron Classes of Neural Networks 1. Perceptrons](https://reader030.vdocuments.site/reader030/viewer/2022040112/5ec4e729d39111156f60e3a3/html5/thumbnails/4.jpg)
4
Artificial Neuron
aj Wji
Input links(dendrites)
Unit(cell body)
Output links(axon)
ai ai =
g(ini)ini =
? ajWji
![Page 5: Introduction to Neural Networks - in.tum.de · Introduction to Neural Networks Freek Stulp. 2 Overview Biological Background Artificial Neuron Classes of Neural Networks 1. Perceptrons](https://reader030.vdocuments.site/reader030/viewer/2022040112/5ec4e729d39111156f60e3a3/html5/thumbnails/5.jpg)
5
Class I: Perceptron
a = g(in)
in = ? ajWj
a = g(-W0 + W1a1 + W2a2) g(in) = 0, in<01, in>0{
a1
a2
-1
Ij
a
O
W0
W1
W2
Wj
![Page 6: Introduction to Neural Networks - in.tum.de · Introduction to Neural Networks Freek Stulp. 2 Overview Biological Background Artificial Neuron Classes of Neural Networks 1. Perceptrons](https://reader030.vdocuments.site/reader030/viewer/2022040112/5ec4e729d39111156f60e3a3/html5/thumbnails/6.jpg)
6
![Page 7: Introduction to Neural Networks - in.tum.de · Introduction to Neural Networks Freek Stulp. 2 Overview Biological Background Artificial Neuron Classes of Neural Networks 1. Perceptrons](https://reader030.vdocuments.site/reader030/viewer/2022040112/5ec4e729d39111156f60e3a3/html5/thumbnails/7.jpg)
7
Learning in PerceptronsError often defined as:
E(W) = 1/2? d? D(td-od)2
Go towards the minimum error!
Update rules:? wi = wi ??? wi
? ? wi = -? ?E/?wi
? ?E/?wi = ?/?wi 1/2? d? D(td-od)2
= ? d? D(td-od)iid
This is called gradient descent
i
![Page 8: Introduction to Neural Networks - in.tum.de · Introduction to Neural Networks Freek Stulp. 2 Overview Biological Background Artificial Neuron Classes of Neural Networks 1. Perceptrons](https://reader030.vdocuments.site/reader030/viewer/2022040112/5ec4e729d39111156f60e3a3/html5/thumbnails/8.jpg)
8
Class II: Multi-layer Feed-forward Networks
Feed-forward:? Output links only connected
to input links in the next layer
Input Hidden OutputMultiple layers:? hidden layer(s)
Complex non-linear functions can be represented
![Page 9: Introduction to Neural Networks - in.tum.de · Introduction to Neural Networks Freek Stulp. 2 Overview Biological Background Artificial Neuron Classes of Neural Networks 1. Perceptrons](https://reader030.vdocuments.site/reader030/viewer/2022040112/5ec4e729d39111156f60e3a3/html5/thumbnails/9.jpg)
9
Learning in MLFF NetworksFor output layer, weight updating similar to perceptrons.Problem: What are the errors in the hidden layer?Backpropagation Algorithm? For each hidden layer (from output to input):
?For each unit in the layer determine how much it contributed to the errors in the previous layer.
?Adapt the weight according to this contribution
This is also gradient descent
![Page 10: Introduction to Neural Networks - in.tum.de · Introduction to Neural Networks Freek Stulp. 2 Overview Biological Background Artificial Neuron Classes of Neural Networks 1. Perceptrons](https://reader030.vdocuments.site/reader030/viewer/2022040112/5ec4e729d39111156f60e3a3/html5/thumbnails/10.jpg)
10
Class III: Recurrent Networks
Input Hidden OutputNo restrictions on connections
Behaviour more difficult to predict/ understand
![Page 11: Introduction to Neural Networks - in.tum.de · Introduction to Neural Networks Freek Stulp. 2 Overview Biological Background Artificial Neuron Classes of Neural Networks 1. Perceptrons](https://reader030.vdocuments.site/reader030/viewer/2022040112/5ec4e729d39111156f60e3a3/html5/thumbnails/11.jpg)
11
Conclusion
Inspiration from biology, though artificial brainsare still very far away.
Perceptrons too simple for most problems.
MLFF Networks good as function approximators.? Many of your articles use these networks!
Recurrent networks complex but useful too.
![Page 12: Introduction to Neural Networks - in.tum.de · Introduction to Neural Networks Freek Stulp. 2 Overview Biological Background Artificial Neuron Classes of Neural Networks 1. Perceptrons](https://reader030.vdocuments.site/reader030/viewer/2022040112/5ec4e729d39111156f60e3a3/html5/thumbnails/12.jpg)
12
Literature
Artificial Intelligence: A Modern Approach? Stuart Russel and Peter Norvig
Machine Learning? Tom M. Mitchell