image pyramids - idc.ac.il · • gaussian pyramid • laplacian pyramid • wavelet/qmf. what is a...

• Gaussian pyramids

• Laplacian Pyramids

• Wavelet Pyramids

• Applications

Image Processing - IDC

Image Representation

Image PyramidsImage features at different resolutions require filters at different scales.

f(x)

f (x)

Edges (derivatives):

Image Pyramid = Hierarchical representation of an image

A collection of images at different resolutions.

Low Resolution

High Resolution

Details in image -low+high frequencies

No details in image -(blurred image)low frequencies

Image PyramidsImage pyramids

• Gaussian Pyramids• Laplacian Pyramids• Wavelet/QMF

Image pyramids

• Gaussian Pyramids• Laplacian Pyramids• Wavelet/QMF

Image Pyramid

High resolution

Low resolution

Image Pyramid Frequency Domain

High resolution

Low resolution

*

= =Image Blurring = low pass filtering

~

*

*

*

=

=

=

Gaussian Pyramid

High resolution

Low resolution

Gaussian Pyramid

Level 02n X 2n

Level 12n-1 X 2n-1

Level M1 X 1

The Gaussian weights

w0 w-1w-1w-2 w-2

w0 w-1w-1w-2 w-2

a bbc c

a bbc c

Burt & Adelson (1981)

Normalized: Σwi = 1

Symmetry: wi = w-i

Unimodal: wi ≥ wj for 0 < i < j

Equal Contribution: for all j Σwj+2i = constant

a + 2b + 2c = 1a + 2c = 2b

a > 0.25b = 0.25c = 0.25 - a/2

The Gaussian weightsFor a = 0.4 most similar to a Gauusian filter

g = [0.05 0.25 0.4 0.25 0.05]

low_pass_filter = g’ * g =

0.0025 0.0125 0.0200 0.0125 0.00250.0125 0.0625 0.1000 0.0625 0.01250.0200 0.1000 0.1600 0.1000 0.02000.0125 0.0625 0.1000 0.0625 0.01250.0025 0.0125 0.0200 0.0125 0.0025

12

34

5

12

34

50

0.05

0.1

0.15

0.2

Gaussian Pyramid -Computational Aspects

Memory:

2NX2N (1 + 1/4 + 1/16 + ... ) = 2NX2N * 4/3

Computation:

Level i can be computed with a single convolutionwith filter: hi = g * g * g * .....}

i times

g g

h2 = * =

Example:

Basis vecrors (Fourier, Wavelet, etc)

fUFrr

=

Vectorized imagetransformed image

Convolution and subsampling in matrix notation (1D case)

U1 =

1 4 6 4 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 1 4 6 4 1 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 1 4 6 4 1 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 1 4 6 4 1 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 1 4 6 4 1 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 1 4 6 4 1 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 1 4 6 4 1 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 4 6 4 1 0

from: B.Freeman

- Next pyramid levelU2 =

1 4 6 4 1 0 0 0

0 0 1 4 6 4 1 0

0 0 0 0 1 4 6 4

0 0 0 0 0 0 1 4

U2 * U1 =

1 4 10 20 31 40 44 40 31 20 10 4 1 0 0 0 0 0 0 0

0 0 0 0 1 4 10 20 31 40 44 40 31 20 10 4 1 0 0 0

0 0 0 0 0 0 0 0 1 4 10 20 31 40 44 40 30 16 4 0

0 0 0 0 0 0 0 0 0 0 0 0 1 4 10 20 25 16 4 0

- The combined effect of the two pyramid levels

Image pyramids

• Gaussian Pyramid• Laplacian Pyramid• Wavelet/QMF

Laplacian Pyramid

G0, G1, .... = the levels of a Gaussian Pyramid.

Predict level Gl from level Gl+1 by Expanding Gl+1 to obtain G’l

Gl

Gl+1

Reduce

G’l

Expand

Denote by Ll the error in prediction:

Ll = Gl - G’l

L0, L1, .... = the levels of a Laplacian Pyramid.

What does blurring take away?

original


smoothed (5x5 Gaussian)


smoothed – original

expand

-

expand

expand

-

- =

=

=

Gaussian Pyramid

LaplacianPyramid

Laplacian PyramidGaussian Pyramid

LaplacianPyramid

FrequencyDomain

-

-

-

=

=

=

from: B.Freeman

LaplacianPyramid

Reconstruction of the original image from the Laplacian Pyramid

+ =

+ =

+ =

= Original Image

Gl = Ll + G’l

expand

expand

expand

Laplacian Pyramid -Computational Aspects

Memory:

2NX2N (1 + 1/4 + 1/16 + ... ) = 2NX2N * 4/3

Computation:

However coefficients are highly compressable.

Li can be computed from G0 with a single convolutionwith filter: ki = hi-1 - hi

hi-1 hi

- =

ki

k1 k2 k3

Image pyramids

• Gaussian Pyramid• Laplacian Pyramid• Wavelet/QMF

What is a good representation for image analysis?

• Fourier transform domain tells you “what” (textural properties), but not “where”. – In space, this representation is too spread out.

• Pixel domain representation tells you “where” (pixel location), but not “what”. – In space, this representation is too localized

• Want an image representation that gives you a local description of image events—what is happening where. – That representation might be “just right”.

Spatial

Spatial

Spatial

Freq.

Freq.

Freq.

Standard basis

Fourier basis

Waveletb basis

Space-Frequency Tiling

Wavelets/QMF’s

fUFrr

=

• The simplest Wavelet transform: The Haar transform:

• The inverse transform for Haar:

U =

1 1

1 -1

inv(U) =

0.5000 0.5000

0.5000 -0.5000

Apply this over multiple spatial positions

U =

1 1 0 0 0 0 0 0

0 0 1 1 0 0 0 0

0 0 0 0 1 1 0 0

0 0 0 0 0 0 1 1

1 -1 0 0 0 0 0 0

0 0 1 -1 0 0 0 0

0 0 0 0 1 -1 0 0

0 0 0 0 0 0 1 -1

Note: the transform is orthogonal !

The High Frequencies

U =

1 1 0 0 0 0 0 0

0 0 1 1 0 0 0 0

0 0 0 0 1 1 0 0

0 0 0 0 0 0 1 1

1 -1 0 0 0 0 0 0

0 0 1 -1 0 0 0 0

0 0 0 0 1 -1 0 0

0 0 0 0 0 0 1 -1

The Low Frequencies

U =

1 1 0 0 0 0 0 0

0 0 1 1 0 0 0 0

0 0 0 0 1 1 0 0

0 0 0 0 0 0 1 1

1 -1 0 0 0 0 0 0

0 0 1 -1 0 0 0 0

0 0 0 0 1 -1 0 0

0 0 0 0 0 0 1 -1

The Inverse Transform

U =

0.5000 0.5000 0 0 0 0 0 00 0 0.5000 0.5000 0 0 0 00 0 0 0 0.5000 0.5000 0 00 0 0 0 0 0 0.5000 0.5000

0.5000 -0.5000 0 0 0 0 0 00 0 0.5000 -0.5000 0 0 0 00 0 0 0 0.5000 -0.5000 0 00 0 0 0 0 0 0.5000 -0.5000

Wavelet bands are split recursively

L

L

L

H

H

H

Wavelet Decomposition in 2D

Frequency domain

Horizontal high pass

Horizontal low pass

Apply the wavelet transform separable in both dimensions

Horizontal high pass, vertical high pass

Horizontal high pass, vertical low-pass

Horizontal low pass, vertical high-pass

Horizontal low pass,Vertical low-pass

• Splitting can be applied recursively: The Wavelet coefficients

Fourier transform

= *

pixel domain image

Fourier bases are global: each transform coefficient depends on all pixel locations.

Fourier transform

From: B. Freeman

Gaussian pyramid

= *pixel image

Overcomplete representation. Low-pass filters, sampled appropriately for their blur.

Gaussian pyramid

From: B. Freeman

Laplacian pyramid

= *pixel image

Overcomplete representation. Transformed pixels represent bandpassed image information.

Laplacianpyramid

From: B. Freeman

Wavelet (QMF) transform

= *pixel imageOrtho-normal

transform (like Fourier transform), but with localized basis functions.

Wavelet pyramid

From: B. Freeman

Image Pyramids - Comparison

Image pyramid levels = Filter then sample.

Filters:

Gaussian Pyramid

Laplacian Pyramid

Wavelet Pyramid

Image Pyramids as Linear Transforms

Fourier Sines+Cosines Not localized in spaceLocalized in Frequency

Wavelet Pyramid Wavelet Filters Localized in space

Localized in Frequency

Transform Basis Characteristics

Delta StandardLocalized in spaceNot localized in Frequency

space spacespace

Delta Fourier Wavelet

Applications

• Compression• Noise removal• Multi-scale Pattern Matrching• Image Stitching

Very useful property: Low entropy of Wavelet bands

Applications:- Compression- Noise removal

Wavelet Shrinkage Denoising

Image statistics (or, mathematically, how can you tell image from noise?)

Noisy image

From: B. Freeman

nxy += ( )σ,0~ Nn

Clean image

From: B. Freeman

Image domain histogram

From: B. Freeman

Wavelet domain image histogram

From: B. Freeman

Image domain noise histogram

From: B. Freeman

Wavelet domain noise histogram

Noise-corrupted full-freq and bandpass images

From: B. Freeman

But want the bandpass image histogram to look like this

argmaxx P(x|y) = P(y|x) P(x) / P(y)

Using Bayes theorem

The parameters you want to estimate

What you observe Prior probability

Likelihood function

Constant w.r.t. parameters x.

From: B. Freeman

P(x)

Bayesian MAP estimator for clean bandpass coefficient values

Let x = bandpassed image value before adding noise.Let y = noise-corrupted observation.

By Bayes theorem P(x|y) = k P(y|x) P(x)

P(y|x)

y

P(y|x)

P(x|y)

P(x|y)

From: B. Freeman

maximum

y

P(y|x)

P(x|y)

maximum


From: B. Freeman

y

P(y|x)

P(x|y)

maximum


From: B. Freeman

MAP estimate, , as function of observed coefficient value, y

y

x̂

x̂

From: B. Freeman

Wavelet Shrinkage PipeWavelet Shrinkage Pipe--lineline

y TransformW

TransformW

Mapping functionsMapping functions

InverseTransform

WT

InverseTransform

WT

x)

yiw

xiw

Noise removal results

Simoncelli and Adelson, Noise Removal via Bayesian Wavelet Coring

From: B. Freeman

More results

More results Hierarchical Pattern Matching

• Two methods for searching a target pattern over many scales.

Image Mosaicing

Transformation

Image Blending

Blending Alpha Blending

High Frequencies

Middle Frequencies

Low Frequencies

When splining two images, transition from one image tothe other should behave:

Multiresolution Spline - Example

Left Image Right Image

Left + Right Narrow Transition

Wide Transition

(Burt & Adelson)

Multiresolustion Spline - Using Laplacian Pyramid


Left Image Right Image

Left + Right Narrow Transition

Wide Transition Multiresolution Spline

(Burt & Adelson)


Original - Left Original - Right

Glued Splined


The colored version

© prof. dmartin

image pyramids - idc.ac.il · • gaussian pyramid • laplacian pyramid • wavelet/qmf. what is a...

Documents