nonlinear models for natural images
DESCRIPTION
Nonlinear models for Natural Images. Urs Köster & Aapo Hyvärinen University of Helsinki. 1. Overview. Limitations of linear models A hierarchical model learns Complex Cell pooling A Horizonal product model for Contrast Gain Control A Markov Random Field generalizes ICA to large images. 2. - PowerPoint PPT PresentationTRANSCRIPT
1
Nonlinear models for Natural Images
Urs Köster & Aapo HyvärinenUniversity of Helsinki
2
Overview
1.Limitations of linear models
2.A hierarchical model learns Complex Cell pooling
3.A Horizonal product model for Contrast Gain Control
4.A Markov Random Field generalizes ICA to large images
2
1. Limitations of ICA image models
Natural images have complex structure, cannot be modeled as superpositions of basis functions
Linear models ignore much of the rich interactions between units
Modeling the dependencies leads to more abstract representations
Variance dependencies are particularly obvious structure not captured by ICA
Model with (complex cell) pooling of filter outputs - hierarchical models
Alternative: Model dependencies by gain control on the pixel level
Schwartz & Simoncelli 2001
5
A hierarchical model estimated with Score Matching learns Complex Cell
receptive fields
6
2. Two Layer Model estimated with Score Matching
Define an energy based model of the form
Squaring the outputs of linear filters
Second layer linear transform v
Nonlinearity that leads to a super-gaussian pdf.
Cannot be normalized in closed form. Estimation with Score Matching makes learning possible without need for Monte Carlo methods or approximations
7
Results
Some pooling patterns
The second layer learns to pool over units with similar location and orientation, but different spatial phase
Following the energy model of Complex Cells without any assumptions on the pooling
Estimating W and V simultaneously leads to a better optimum and more phase invariance of the higher order units
8
Learning to perform Gain Control with a Horizontal Product model
9
Multiplicative interactions Data is described by element-wise
multiplying outputs of sub-models
Can implement highly nonlinear (discontinuous) functions
Combine aspects of a stimulus generated by separate mechanisms
Horizontal layers Two parallel streams or layers on
one level of the hierarchy
Unrelated aspects of the stimulus are generated separately
Observed data is generated by combining all the sub-models
A horizontal network model:
10
The model
Definition of the model:
Likelihood:
Constraints: B and t are non-negative, W invertible g(.) is a log-cosh nonlinearity (logistic distribution) t has a Laplacian sparseness prior
10
Results
First layer W 4 units in B
First layer W 16 units in B
Second Layer: Contrast Gain Control
Reconstruction from As only
True image patches
Modulation from Bt
Emergence of a contrast map in the second layer
It performs Contrast Gain Control on the LGN level (rather than on filter outputs)
Similar effect to performing divisive normalization as preprocessing
The model can be written as
Something impossible to do with hierarchical models
13
The “big” picture: A Markov Random Field generalizes ICA to
arbitrary size images
14
4. Markov Random Field
Goal: Define probabilities for whole images rather than small patches
A MRF uses a convolution to analyze large images with small filters
Estimating the optimal filters in an ICA framework is difficult, the model cannot be normalized
Energy based optimization using Score Matching
15
Model estimation
The energy (neg. log pdf) is
We can rewrite the convolution
where xi are all possible patches from the image, wk are the different filters
We can use score matching just like in an overcomplete ICA model
The MRF is equivalent to overcomplete ICA with filters that are smaller than the patch and copied in all possible locations.
16
Results We can estimate MRF filters of size 12x12 pixels
(much larger than previous work, e.g. 5x5)
This is possible from 23x23 pixel ‘images’, but the filters generalize to images of arbitrary size
This is possible because all possible overlaps are accounted for in the (2 n -1) size image
Filters similar to ICA, but less localized (since they need to explain more of the surrounding patch)
Possible applications in denoising and filling-in
16