cvpr2010: advanced itincvpr in a nutshell: part 4: isocontours, registration
TRANSCRIPT
Tutorial
Advanced Information Theory in CVPR “in a Nutshell”
CVPRJune 13-18 2010
San Francisco,CAIsocontours and Image Registration
Anand Rangarajan
Image Registration
The need for information-theoretic measuresWhen there is no clearly established analytic relationship betweentwo or more images, it is often more convenient to minimize aninformation-theoretic distance measure such as the negative of themutual information (MI).
Figure: Left: MR-PD slice. Right: Warped, noisy MR-T2 slice.
2/20
The joint space of two images
3/20
Density and Entropy estimation
Density estimation
I HistogrammingI Parzen windowsI Mixture models, wavelet densities (and other parametrizations)
Entropy estimation
I Entropy estimation from the joint density (or distribution)I Direct entropy estimation (kNN, MST, Voronoi etc.)I Entropy estimation from the cumulative distribution (cdf)
4/20
Moving away from samples
The underlying commonality in all of the previous approachesAll previous approaches are sample-based. Our new approach doesnot begin with the idea of individual samples.
Obtain approx. todensity and entropy
Obtain improved approximation
Take samples
Take more samples
5/20
Image-based density estimation
Assume uniform distribution on location
Transformation Location
Intensity
Distribution on intensity
Uncountable infinity of samples taken
Each point in the continuum contributes
to intensity distribution
Image-Based
6/20
Isocontours
7/20
Isocontour area-based density
Isocontour density estimationArea trapped between level sets α and α+ ∆α is proportional to theprobability Pr(α ≤ I ≤ α + ∆α). The density function is
p(α) =1A
ˆI (x ,y)=α
1|∇I (x , y)|
du
Level sets at I (x , y) = α
8/20
Isocontour area-based density
Isocontour density estimationArea trapped between level sets α and α+ ∆α is proportional to theprobability Pr(α ≤ I ≤ α + ∆α). The density function is
p(α) =1A
ˆI (x ,y)=α
1|∇I (x , y)|
du
Level sets at I (x , y) = α and I (x , y) = α + ∆α
8/20
Isocontour area-based density
Isocontour density estimationArea trapped between level sets α and α+ ∆α is proportional to theprobability Pr(α ≤ I ≤ α + ∆α). The density function is
p(α) =1A
ˆI (x ,y)=α
1|∇I (x , y)|
du
Area in between I (x , y) = α and I (x , y) = α + ∆α
8/20
Joint Probability
Figure: Two synthetic images
9/20
Joint Probability
Figure: Level sets of the two synthetic images
10/20
Joint Probability
Isocontour overlay exhibits area overlap
Figure: Overlay of the two sets of isocontours
11/20
Joint Probability
Level sets at I1(x , y) = α1 and I2(x , y) = α2
The cumulative area of the black regions is proportional toPr(α1 ≤ I1 ≤ α1 + ∆α1, α2 ≤ I2 ≤ α2 + ∆α2).
12/20
Joint Probability
Level sets at I1 = α1, α1 + ∆α1 and I2 = α2 and α2 + ∆α2
The cumulative area of the black regions is proportional toPr(α1 ≤ I1 ≤ α1 + ∆α1, α2 ≤ I2 ≤ α2 + ∆α2).
12/20
Joint Probability
Areas: α1 ≤ I1 ≤ α1 + ∆α1 and α2 ≤ I2 ≤ α2 + ∆α2
The cumulative area of the black regions is proportional toPr(α1 ≤ I1 ≤ α1 + ∆α1, α2 ≤ I2 ≤ α2 + ∆α2).
12/20
Joint Probability Expression
I The joint density of images I1(x , y) and I2(x , y) with area ofoverlap A is related to the area of intersection of regionsbetween level curves at α1 and α1 + ∆α1 of I1 and at α2 andα2 + ∆α2 of I2 as ∆α1 → 0, ∆α2 → 0.
I The joint density
p(α1, α2) =1A
ˆ ˆI1(x ,y)=α1,I2(x ,y)=α2
du1du2
|∇I1(x , y)∇I2(x , y) sin(θ)|
where u1 and u2 are the level curve tangent vectors in I1 and I2respectively and θ the angle between the image gradients.
13/20
When there’s no joint density
Pathological cases
Examine 1|∇I1(x ,y)∇I2(x ,y) sin(θ)| :
Region in Image 1of constant intensityα1
Region in Image 2of constant intensityα2
Area of intersectionof the two regions[contribution to P(α1,α2)]
Region in Image 1with constant intensityα1
Level curves of Image 2at intensities α2 andα2+∆α
Area of intersection(contribution toP(α1,α2)
Level curves of Image 1at intensities α1 andα1+∆α
Level curves of Image 2at intensities α2 andα2+∆αArea where level curves
from images 1 and 2are parallel
Figure: Left: Both images flat. Middle: One image flat. Right: Gradientsrun locally parallel.
14/20
Binning without the binning problem
Choose as many bins as desired
15/20
Binning without the binning problem
Choose as many bins as desired
15/20
Binning without the binning problem
Choose as many bins as desired
15/20
Binning without the binning problem
Choose as many bins as desired
15/20
Information-theoretic formulation
Mutual Information-based registrationGiven two images I1 and I2, a now standard approach to imageregistration minimizes
E (T ) = −MI (I1, I2(T )) = H(I1, I2(T ))− H(I1)− H(I2(T ))
where the mutual information (MI) is unpacked as the sum of themarginal entropies minus the joint entropy. The entropies (Shannon)can be easily estimated from the iscontour density estimators (as wellas other estimators such as histogramming and Parzen windows).The transformation T (usually rigid or affine) is applied to only I2 inthis formulation.
16/20
Comparison with std. histograms
32 bins
Left: Standard histogramming. Right: Isocontours17/20
Comparison with std. histograms
64 bins
Left: Standard histogramming. Right: Isocontours17/20
Comparison with std. histograms
128 bins
Left: Standard histogramming. Right: Isocontours17/20
Comparison with std. histograms
256 bins
Left: Standard histogramming. Right: Isocontours17/20
Comparison with std. histograms
512 bins
Left: Standard histogramming. Right: Isocontours17/20
Comparison with std. histograms
1024 bins
Left: Standard histogramming. Right: Isocontours17/20
Joint density comparisons
16 bins
05
1015
20
0
5
10
15
200
0.01
0.02
0.03
0.04
0.05
Joint density histograms: 16 bins
05
1015
20
0
5
10
15
200
0.01
0.02
0.03
0.04
0.05
0.06
Joint density isocontours: 16 bins
Left: Standard histogramming. Right: Isocontours
18/20
Joint density comparisons
32 bins
010
2030
40
0
10
20
30
400
0.002
0.004
0.006
0.008
0.01
0.012
Joint density histograms: 32 bins
010
2030
40
0
10
20
30
400
0.002
0.004
0.006
0.008
0.01
0.012
0.014
Joint density isocontours: 32 bins
Left: Standard histogramming. Right: Isocontours
18/20
Joint density comparisons
64 bins
020
4060
80
0
20
40
60
800
1
2
3
4
x 10−3
Joint density histograms: 64 bins
020
4060
80
0
20
40
60
800
0.5
1
1.5
2
2.5
3
3.5
x 10−3
Joint density isocontours: 64 bins
Left: Standard histogramming. Right: Isocontours
18/20
Joint density comparisons
128 bins
0
50
100
150
0
50
100
1500
0.5
1
1.5
2
x 10−3
Joint density histograms: 128 bins
0
50
100
150
0
50
100
1500
0.2
0.4
0.6
0.8
1
x 10−3
Joint density isocontours: 128 bins
Left: Standard histogramming. Right: Isocontours
18/20
Mutual Information comparisons
Single rotation parameter in 2D
Noise standard deviation 0.05
Left: 32 bins, Right: 128 bins
0 10 20 30 40 500
0.1
0.2
0.3
0.4
ISOCONTOURSHIST BILINEARPVIHIST CUBIC2DPointProb
0 10 20 30 40 500
0.2
0.4
0.6
0.8
ISOCONTOURSHIST BILINEARPVIHIST CUBIC2DPointProb
19/20
Mutual Information comparisons
Single rotation parameter in 2D
Noise standard deviation 0.2
Left: 32 bins, Right: 128 bins
0 10 20 30 40 500
0.05
0.1
0.15
0.2
ISOCONTOURSHIST BILINEARPVIHIST CUBIC2DPointProb
0 10 20 30 40 500
0.2
0.4
0.6
0.8
ISOCONTOURSHIST BILINEARPVIHIST CUBIC2DPointProb
19/20
Mutual Information comparisons
Single rotation parameter in 2D
Noise standard deviation 1.0
Left: 32 bins, Right: 128 bins
0 10 20 30 40 500
0.02
0.04
0.06
0.08
ISOCONTOURSHIST BILINEARPVIHIST CUBIC2DPointProb
0 10 20 30 40 500
0.1
0.2
0.3
0.4
0.5
ISOCONTOURSHIST BILINEARPVIHIST CUBIC2DPointProb
19/20
Discussion
I With piecewise linear interpolation, much faster than upsampledhistogramming
I Extended to multiple image registration and 3DI Statistical significance (Kolmogorov-Smirnov) tests runI Other groups (Oxford etc.) involved - analytic studiesI Applied to mean shift filtering and unit vector density estimationI Drawbacks: Non differentiable, no clean extension to higher
dimensions
20/20