nonlinear models for natural images
DESCRIPTION
Nonlinear models for Natural Images. Urs Köster & Aapo Hyvärinen University of Helsinki. 1. Overview. Limitations of linear models A hierarchical model learns Complex Cell pooling A Horizonal product model for Contrast Gain Control A Markov Random Field generalizes ICA to large images. 2. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Nonlinear models for Natural Images](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813600550346895d9d73f6/html5/thumbnails/1.jpg)
1
Nonlinear models for Natural Images
Urs Köster & Aapo HyvärinenUniversity of Helsinki
![Page 2: Nonlinear models for Natural Images](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813600550346895d9d73f6/html5/thumbnails/2.jpg)
2
Overview
1.Limitations of linear models
2.A hierarchical model learns Complex Cell pooling
3.A Horizonal product model for Contrast Gain Control
4.A Markov Random Field generalizes ICA to large images
2
![Page 3: Nonlinear models for Natural Images](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813600550346895d9d73f6/html5/thumbnails/3.jpg)
1. Limitations of ICA image models
Natural images have complex structure, cannot be modeled as superpositions of basis functions
Linear models ignore much of the rich interactions between units
Modeling the dependencies leads to more abstract representations
Variance dependencies are particularly obvious structure not captured by ICA
Model with (complex cell) pooling of filter outputs - hierarchical models
Alternative: Model dependencies by gain control on the pixel level
Schwartz & Simoncelli 2001
![Page 4: Nonlinear models for Natural Images](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813600550346895d9d73f6/html5/thumbnails/4.jpg)
5
A hierarchical model estimated with Score Matching learns Complex Cell
receptive fields
![Page 5: Nonlinear models for Natural Images](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813600550346895d9d73f6/html5/thumbnails/5.jpg)
6
2. Two Layer Model estimated with Score Matching
Define an energy based model of the form
Squaring the outputs of linear filters
Second layer linear transform v
Nonlinearity that leads to a super-gaussian pdf.
Cannot be normalized in closed form. Estimation with Score Matching makes learning possible without need for Monte Carlo methods or approximations
![Page 6: Nonlinear models for Natural Images](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813600550346895d9d73f6/html5/thumbnails/6.jpg)
7
Results
Some pooling patterns
The second layer learns to pool over units with similar location and orientation, but different spatial phase
Following the energy model of Complex Cells without any assumptions on the pooling
Estimating W and V simultaneously leads to a better optimum and more phase invariance of the higher order units
![Page 7: Nonlinear models for Natural Images](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813600550346895d9d73f6/html5/thumbnails/7.jpg)
8
Learning to perform Gain Control with a Horizontal Product model
![Page 8: Nonlinear models for Natural Images](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813600550346895d9d73f6/html5/thumbnails/8.jpg)
9
Multiplicative interactions Data is described by element-wise
multiplying outputs of sub-models
Can implement highly nonlinear (discontinuous) functions
Combine aspects of a stimulus generated by separate mechanisms
Horizontal layers Two parallel streams or layers on
one level of the hierarchy
Unrelated aspects of the stimulus are generated separately
Observed data is generated by combining all the sub-models
A horizontal network model:
![Page 9: Nonlinear models for Natural Images](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813600550346895d9d73f6/html5/thumbnails/9.jpg)
10
The model
Definition of the model:
Likelihood:
Constraints: B and t are non-negative, W invertible g(.) is a log-cosh nonlinearity (logistic distribution) t has a Laplacian sparseness prior
10
![Page 10: Nonlinear models for Natural Images](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813600550346895d9d73f6/html5/thumbnails/10.jpg)
Results
First layer W 4 units in B
First layer W 16 units in B
![Page 11: Nonlinear models for Natural Images](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813600550346895d9d73f6/html5/thumbnails/11.jpg)
Second Layer: Contrast Gain Control
Reconstruction from As only
True image patches
Modulation from Bt
Emergence of a contrast map in the second layer
It performs Contrast Gain Control on the LGN level (rather than on filter outputs)
Similar effect to performing divisive normalization as preprocessing
The model can be written as
Something impossible to do with hierarchical models
![Page 12: Nonlinear models for Natural Images](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813600550346895d9d73f6/html5/thumbnails/12.jpg)
13
The “big” picture: A Markov Random Field generalizes ICA to
arbitrary size images
![Page 13: Nonlinear models for Natural Images](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813600550346895d9d73f6/html5/thumbnails/13.jpg)
14
4. Markov Random Field
Goal: Define probabilities for whole images rather than small patches
A MRF uses a convolution to analyze large images with small filters
Estimating the optimal filters in an ICA framework is difficult, the model cannot be normalized
Energy based optimization using Score Matching
![Page 14: Nonlinear models for Natural Images](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813600550346895d9d73f6/html5/thumbnails/14.jpg)
15
Model estimation
The energy (neg. log pdf) is
We can rewrite the convolution
where xi are all possible patches from the image, wk are the different filters
We can use score matching just like in an overcomplete ICA model
The MRF is equivalent to overcomplete ICA with filters that are smaller than the patch and copied in all possible locations.
![Page 15: Nonlinear models for Natural Images](https://reader036.vdocuments.site/reader036/viewer/2022062517/56813600550346895d9d73f6/html5/thumbnails/15.jpg)
16
Results We can estimate MRF filters of size 12x12 pixels
(much larger than previous work, e.g. 5x5)
This is possible from 23x23 pixel ‘images’, but the filters generalize to images of arbitrary size
This is possible because all possible overlaps are accounted for in the (2 n -1) size image
Filters similar to ICA, but less localized (since they need to explain more of the surrounding patch)
Possible applications in denoising and filling-in
16