neural network models in vision
DESCRIPTION
Neural Network Models in Vision. Peter Andras [email protected]. The Visual System. R. LGN. V1. V3. V2. Lower. V5. V4. Higher. Neurons. Rod. Horizontal. Bipolar. Amacrine. Ganglion. Neuron Models. The McCullogh-Pitts model. x 1 x 2 x 3 … x n-1 x n. w 1. Output. - PowerPoint PPT PresentationTRANSCRIPT
![Page 2: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/2.jpg)
The Visual System
R LGN V1
V2
V4
V3
V5
Higher
Lower
![Page 3: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/3.jpg)
Neurons
Rod
Horizontal
Bipolar
Amacrine
Ganglion
![Page 4: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/4.jpg)
Neuron Models
The McCullogh-Pitts model
InputsOutputw2
w1
w3
wn
wn-1
. . .
x1
x2
x3
…
xn-1
xn
y)(;
1
zHyxwzn
iii
![Page 5: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/5.jpg)
Neuron ModelsThe Hodgkin-Huxley Model
Na+K+
Na+Na+
Na+
Na+Na+
Na+
Na+
Na+
K+
K+
K+
K+K+
K+
K+
3,2,1;)1(
)()()( 432
310
ixxdt
dx
EVcEVxcEVxxcIdt
dVC
iiiii
LLKKNaNa
![Page 6: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/6.jpg)
Modelling Methodology
Physiological measurements
StimulusResponse
Electrode
Other methods: EEG, MRI, PET, MEG, optical recording, metabolic recording
![Page 7: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/7.jpg)
Modelling MethodologyResponse characterisation in terms of stimulus properties
02468
101214161820
0 20 40 60 80 100
120
140
160
180
Degrees
Spike count
02468
101214161820
Spatial frequency
Spike count
4
02468
101214161820
Temporal frequency
Spike count
Stimulus
![Page 8: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/8.jpg)
Modelling Methodology
Models:
A. Statistical models: large number of neurons, with a few well-defined properties, the response is analysed at the population level;
![Page 9: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/9.jpg)
B. Macro-neural models: simplified model neurons organised in relatively simple networks, the overall input-output relationship of the full network is analysed;
Modelling Methodology
Models:
![Page 10: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/10.jpg)
C. Micro-neural models: the neurons are modelled with many details and models of individual neurons or networks of few detailed neurons are analysed.
Modelling Methodology
Models:
![Page 11: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/11.jpg)
Modelling Methodology
Physiological measurements
Response characterisation
Model selection
OBJECTIVE 1: match the measured response properties by the response properties of the model.
OBJECTIVE 2: test the theories, generate predictions.
![Page 12: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/12.jpg)
Neural Network Models
Retina: ON and OFF centre ganglion cells
Bipolar cells
+1
-1
ON OFF
Preferred stimulus
![Page 13: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/13.jpg)
Neural Network Models
Retina: ON and OFF centre ganglion cells
0
2
4
6
8
10
12
14
16
18
20
Spike rate
0
2
4
6
8
10
12
14
16
18
20
Spike rate
Measured response of an ON cell The response of a model ON cell
![Page 14: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/14.jpg)
Neural Network Models
V1: Orientation selective cells
Preferred stimulus
LGN cells
![Page 15: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/15.jpg)
02468
101214161820
Degrees
Spike count
Neural Network Models
V1: Orientation selective cells
02468
101214161820
Degrees
Spike count
Measurement Model
![Page 16: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/16.jpg)
Neural Network Models
V1: Ocular dominance patterns and orientation maps
![Page 17: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/17.jpg)
Neural Network Models
V1: Ocular dominance patterns and orientation maps
Neuron = Feature vector:
• orientation preference;
• spatial frequency;
• eye preference;
• temporal frequency;
Training principles:
• the neuron fires maximally when the stimulus matches its preferences set by the feature vector;
• the neuron fires if its neighbours fire;
• when the neuron fires it adapts its feature vector to the received stimulus.
![Page 18: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/18.jpg)
Neural Network Models
V1: Ocular dominance patterns and orientation maps
Mathematically:
Neurons: (wi , ci); wi – feature vector; ci – position vector;
Training set: xt , training vectors, they have the same dimensionality as the feature vectors;
Training:
i* = index of the neuron for which d(wi*, xt) < d(wi, xt), for every i i*;
wi = (1-) wi + xt , for all neurons with index i, for which d(ci, ci*) < .
![Page 19: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/19.jpg)
Neural Network Models
V1: Contour detection
Stimulus
![Page 20: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/20.jpg)
Neural Network Models
V1: Contour detection
Neural interactions: specified by interconnection weights.
Mechanism: constraint satisfaction by mutual modification of the firing rates.
Result: the neurons corresponding to the contour position remain active and the rest of the neurons become silent.
![Page 21: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/21.jpg)
Neural Network ModelsV5: Motion direction selective cells
Orientation selective cells
delay effect-1
+1Preferred stimulus
![Page 22: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/22.jpg)
Neural Network Models
Visual object detection
Object Features:
• colour;
• texture;
• edge distribution;
• contrast distribution;
• etc.
Invariant combination of features
Object detection
![Page 23: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/23.jpg)
Neural Network Models
Visual object detection
Method 1: Hierarchical binary binding of features
Colour
Texture
Edges
Contrast
This method leads to combinatorial explosion.
![Page 24: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/24.jpg)
Neural Network Models
Visual object detection
Method 2: Non-linear segmentation of the feature space.
Colour
Texture
Edges
Contrast
Learning by back-propagation of the error signal and modification of connection weights.
![Page 25: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/25.jpg)
Neural Network Models
Visual object detection
Method 3: Feature binding by synchronization.
![Page 26: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/26.jpg)
Critical Evaluation• Neural network models typically explain certain selected behavioural features of the modelled neural system, and they ignore most of the other aspects of neural activity.
• These models can be used to test theoretical assumptions about the functional organization of the neurons and of the nervous system. They provide predictions with which we can determine the extent of the validity of the model assumptions.
• One common error related to such models is to invert the causal relationship between the assumptions and consequences: i.e., the fact that a model produces the same behavior as the modelled, does not necessarily mean that the modelled has exactly the same structure as the model.
![Page 27: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/27.jpg)
Revised View of the Neural Network Models
Revised interpretation:
• neurons = anatomical / functional modules (e.g., cortical columns or cortical areas);
• connections = causal relationships (e.g., activation of bits of LGN causes activation of bits of V1);
• activity function of a neuron = conditional distribution of module responses, conditioned by the incoming stimuli;
![Page 28: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/28.jpg)
Revised View of the Neural Network Models
Neural network model Bayesian network model
x1
x2
x3
x4
f1(x1)
f2(x2)
f3(x3)
f4(x4)
y1
y2
y3
y4
f(y1, y2, y3, y4)y
yi = fi(xi)
y = f(y1, y2, y3, y4)
x1
x2
x3
x4
y1
y2
y3
y4
y
P(y1|x1)
P(y2|x2)
P(y3|x3)
P(y4|x4)
P(x1, x2, x3, x4) P(y | y1, y2, y3, y4)
P(x1, x2, x3, x4)
P(yi | xi)
P(y | y1, y2, y3, y4)
![Page 29: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/29.jpg)
Revised View of the Neural Network Models
Advantages of the Bayesian interpretation:
• relaxes structural restrictions;
• makes the models conceptually open-ended;
• allows easy upgrade of the model;
• allows relaxed analytical search for minimal complexity models on the basis of data;
• allows statistically sound testing;
![Page 30: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/30.jpg)
Conclusions• Neuron and neural network models can capture important aspects of the functioning of the nervous system. They allow us to test the extent of validity of the assumptions on which the models are based.
• A common mistake related to neural network models is to invert the causal relationship between assumptions and consequences. This can lead to far reaching conclusions about the organization of the nervous system on the basis of natural-like functioning of the neural network models that are invalid.
• The Bayesian reinterpretation of neural network models relaxes many constraints of such models, makes their upgrade and evaluation easier , and prevents to some extent incorrect interpretations.
![Page 31: Neural Network Models in Vision](https://reader035.vdocuments.site/reader035/viewer/2022062422/56813a9d550346895da2981d/html5/thumbnails/31.jpg)
Seminar Papers
1. PNAS, 93, 623-627, Jan. 1996
2. PNAS, 96, 10530-10535, Aug. 1999