digital image processing - california institute of...

Shuozhong Wang, SCIE, Shanghai UniversityShuozhong Wang, SCIE, Shanghai University

Digital Image Processing

Kenneth R. Castleman

2008/2/26

Presentation by S. Wang

Kenneth R. Kenneth R. CastlemanCastleman

2008/2/26

Presentation by S. WangPresentation by S. Wang


Part Three Part Three -- 11

Image RestorationImage RestorationImage Restoration


Chapter 16Chapter 16

Image RestorationImage RestorationImage Restoration

3Shuozhong Wang, SCIE, Shanghai UniversityShuozhong Wang, SCIE, Shanghai University

IntroductionIntroduction

Definition of image restoration:

Removal or reduction of degradations incurred while Removal or reduction of degradations incurred while

the digital image was being obtained.the digital image was being obtained.

Image degradation:

– Blurring introduced by optical systems or image motion

– Noise interference

Typical application: restoration of space pictures.


Approaches of Image RestorationApproaches of Image Restoration

Estimation: When little is known about the image, –– ModelModel and characterizecharacterize the sources of degradation

(blurring and noise), and–– RemoveRemove or reduce reduce their effects.

Detection: When a great deal of prior knowledge about the image is available,– Develop a math model of the original image, and– Fit the model to the observed image.

• Example: The image is known to contain circles. The objective is to determine their size, position, numbers, etc.

We will concentrate on the first approach.


Restoration vs. EnhancementRestoration vs. Enhancement

Restoration: to estimate an ideal image that would be

observed if no degradation were present.

– Least error with respect to an “ideal image” is sought.

Enhancement: to modify, or improve, an image to better

suit human viewing or computer processing.

– No “ideal image” is sought. Deviation from the ideal is

allowed, or even desired.


Sources of Image DegradationSources of Image Degradation

Diffraction in optical system

Sensor nonlinearilties

Optical aberrations

Film or CCD/CMOS nonlinearities

Atmospheric turbulence

Image motion blur

Geometric distortion

Electronics and film grain noise


Image BlurringImage Blurring


OutlineOutline

Classic, continuous frequency domain approaches

Discrete spatial domain approaches

Estimation of blurring characteristics ⎯ system

identification

Noise modeling

Implementation


Continuous Frequency Continuous Frequency Domain ApproachDomain Approach


Classical Restoration FiltersClassical Restoration Filters

Model: continuous image restoration:

The goal: to find a restoration filter g(x,y) that gives the best estimate of the ideal image f(x,y).

hh((xx,,yy)) gg((xx,,yy))+f (x,y) w(x,y)

n(x,y)

),(ˆ yxf


DeconvolutionDeconvolution: Inverse Filtering: Inverse Filtering

Without noise and if point spread function is well behaved, perfect restoration is possible:

where

Example: Fig.16-3 on p.390.ExperimentDetermination of PSF: analytical or experimental.

hh((xx,,yy)) gg((xx,,yy))f (x,y) f (x,y)

),(1),(

vuHvuG =

Demo_InvFilterDemo_InvFilter


Inverse Filtering (cont.)Inverse Filtering (cont.)

Simple inversion of PSF is rarely practical because:

– At frequencies where H(u,v) is small, error is

intolerable due to noise amplification.

– Small changes in N(x,y) cause radical changes in the

reconstructed image.

– At zeros of H(u,v), the inverse filter is not physically

realizable.


Inverse Filtering (cont.)Inverse Filtering (cont.)

To combat the ill-conditioning problem, the inverse filter transfer function may be modified.Solution: undo blurring with inverse filtering, and limit high frequency response to combat noise.

Theoretical response Actual

response

Nathan’s inverse

responseCorrected response


Wiener Wiener DeconvolutionDeconvolution: A Review: A Review

Statistical properties of image:– Adjacent pixels are closely correlated;– Widely separated pixels are loosely correlated; and– Power spectrum decreases toward high frequencies.

Statistical properties of additive noise:– Zero mean;– Statistically independent of the image; and– The power spectral density is flat (or drops

considerably slower than the image).When the PSF H(u,v) and both Pf(u,v) and Pn(u,v) are known, one has the Wiener filter.


Wiener Wiener DeconvolutionDeconvolution (cont.)(cont.)

The two-dimensional Wiener filter: Compensating for blurring and suppression of noise.

),(),(),(),(),( 2 vuPvuPvuH

vuHvuGfn+

=∗ Pf(s)

Pn(s)

H(s)

1

Compensate for blurring

Suppress noise

Blurring

Image

Noise

G(s)


Wiener Wiener DeconvolutionDeconvolution : Discussion: Discussion

Pn(u,v)/Pf(u,v) in the denominator suppresses noise in an optimized fashion (MSE criterion).

It is conservative in terms of de-blurring ⎯ too much emphasis on noise suppression.

Cannot handle spatially variant degradations.

Cannot handle non-stationary signal and noise ⎯ PSF is only meaningful for stationary processes.

),(),(),(),(),( 2 vuPvuPvuH

vuHvuGfn+

=∗


Wiener Filter Variation: PSE FilterWiener Filter Variation: PSE Filter

Power spectrum equalizationPower spectrum equalization: restore power spectrum of image to its original amplitude:

Like the Wiener filter, – The PSE filter is phaseless. – Reduces to deconvolution in the absence of noise.– Completely cuts off when signal does not exist.

However, it does not cut off at zeros of H(u,v).Also referred to as homomorphic filter.

21

2 ),(),(),(

),(),(

⎥⎥⎦

⎤

⎢⎢⎣

⎡

+=

vuPvuPvuH

vuPvuG

nf

f


Parametric Wiener FilteringParametric Wiener Filtering

The geometric mean filter:

where α and γ are positive parameters.

α=1: reduces to simple deconvolution.

α=1/2, and γ =1: power spectrum equalization filter.

α=0, and γ =1: Wiener filter.

α=0, and γ =0: simple deconvolution.

Degree of smoothing may be adjusted by γ.

αα

γ

−∗∗

⎥⎥⎦

⎤

⎢⎢⎣

⎡

+⎥⎥⎦

⎤

⎢⎢⎣

⎡=

1

22 ),(),(),(),(

),(),(),(

vuPvuPvuHvuH

vuHvuHvuG

fn


Summary of Wiener FilteringSummary of Wiener Filtering

Simple deconvolution: ill-conditioned:

– instability at zeros of the PSF, and

– magnification of noise at high frequencies.

Wiener deconvolution suppresses noise and restores

sharpness. But de-burring is somewhat unsatisfactory.

The parametric Wiener filter with α ≈ 0 and γ < 1

generally produces more pleasing results.


Linear Algebraic ApproachLinear Algebraic Approach


Linear Algebraic RestorationLinear Algebraic Restoration

It is desirable to use matrix algebramatrix algebra instead of integral calculus, and discrete mathematicsdiscrete mathematics instead of continuous functions.

Linear algebraic approach offers insight into the numerical aspects of image restoration.

In implementation, linear algebra may not provide economic solutions due to large matrix sizes.

Therefore, techniques developed here may be implemented by other means.


Discrete Restoration ModelDiscrete Restoration Model

Ideal Ideal digitizerdigitizer

f

RestorationRestorationIdeal Ideal digitizerdigitizer

Linear Linear blurblur ++

f(x,y)h(x,y) g(x,y)

n(x,y)

g f

LinearLinearblurblur

Ideal Ideal digitizerdigitizer ++ RestorationRestoration

fH n

g f

The ideal situation

The actual situation

The discrete model23


Discrete Restoration Model (cont.)Discrete Restoration Model (cont.)

The discrete model of degradation:

– The digital image is N-by-N after zero padding.

– g, f, and n are N2-by-1 column vectors.

– H is an N2-by-N2 matrix.

– If blur is shift-invariant, H is a block-circulant matrix.

nHfg +=

LinearLinearblurblur

Ideal Ideal digitizerdigitizer ++ RestorationRestoration

fH n

g ff(x,y)


Implications of the Discrete ModelImplications of the Discrete Model

Modeling the continuous degradation with discrete Modeling the continuous degradation with discrete

operations has two implications:operations has two implications:

Effectiveness of image restoration largely depends on the

accuracy of modeling continuous degradation with

discrete operations.

Images restoration becomes a numerical exercise:

accuracy may be achieved within round-off errors.


Unconstrained RestorationUnconstrained Restoration

The degradation model:

If knowledge about noise is unavailable, a least square minimization problem is set up as follows.

Select an estimate of the image:

Assume is blurred by H:

The error in as compared to the observation g:

nHfg +=

ff

fHg ˆˆ =

g

fHgggfe ˆˆ)ˆ( −=−=


Unconstrained Restoration (cont.) Unconstrained Restoration (cont.)

Define Euclidean norm as an objective function:

Minimizing with respect to

Frequency domain representation:

Note that if H(u,v) has zeros, H is singular, neither H−1 nor (H'H)−1 exists.

)ˆ()'ˆ(||)ˆ(||)ˆ( fHgfHgfef −−==W

)ˆ(fW f

0)ˆ('2ˆ)ˆ(

=−−=∂

∂ fHgHffW gHgHHHf 11 ')'(ˆ −− ==

),(),(),(ˆ

vuHvuGvuF =

Comparison with Wiener filter27


Constrained Least Square RestorationConstrained Least Square Restoration

Re-write the degradation model:

Let norm of both sides be the same:

Introduce a Lagrange multiplier λ, and define the following object function:

where the matrix Q is a linear operator. It will be specified

to achieve different goals.

nHfg += nHfg =−

22 ||||||ˆ|| nfHg =−

)||||||ˆ(||||ˆ||)ˆ( 222 nfHgfQf −−+= λW


Constrained LS Restoration (cont.)Constrained LS Restoration (cont.)

Find that minimizes W:

This is a general solution to constrained LS restoration. Different filters can be obtained by selecting the form of Q.

)||||||ˆ(||||ˆ||)ˆ( 222 nfHgfQf −−+= λW

f

0)ˆ('2ˆ'2ˆ)ˆ(

=−−= fHgHfQQff λ

ddW

gHQQHHf ')''(ˆ 1−+= γ where λγ 1=


PseudoinversePseudoinverse FilterFilter

Let Q=I,

If H is not invertible (non-square, or singular):

where H+ is pseudoinverse (Moore-Penrose) of H:

gHQQHHf ')''(ˆ 1−+= γ

gHIHHf ')'(ˆ 1−+= γ gHf 1ˆ −=γ =0

gHf +=ˆ

⎪⎪⎩

⎪⎪⎨

⎧

==

==

+++

+

++

++

HHHHHHHH

HHHHHHHH

)'()'(


Parametric Wiener FilterParametric Wiener Filter

Treat f and n as random vectors, and select Q to be the noise-to-signal ratio:

where Rf=E{f f '} and Rn=E{nn'} are covariance matrices. The solution becomes

Frequency domain representation (γ =1→Wiener):

2/12/1nf RRQ −=


gHRRHHf ')'(ˆ 11 −−+= nfγ

),(),(),(),(

),(),(ˆ2 vuG

vuPvuPvuHvuHvuF

fn ⎥⎥⎦

⎤

⎢⎢⎣

⎡

+=

∗

γ31


Smoothness ConstraintsSmoothness Constraints

Frequently, a blur matrix suffers from ill-conditioning, even singularity. Restored image may suffer from artifacts of oscillations.

To combat this, select Q to enforce smoothnesssmoothness in the restored image.

This is achieved by letting Q correspond to some high-pass operation such as Laplacian:

)||||||ˆ(||||ˆ||)ˆ( 222 nfHgfQf −−+= λW



Smoothness Constraints (cont.)Smoothness Constraints (cont.)

Thus the block-circulant matrix Q may be constructed using the following kernel:

Restored image with a smoothness constraint is

where

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−

−

010141

010

),(),(),(

),(),(ˆ2 vuG

vuPvuHvuHvuF

⎥⎥⎦

⎤

⎢⎢⎣

⎡

+=

∗

γ

)(4),( 222 vuvuP +−= π

⎥⎦

⎤⎢⎣

⎡∂∂

+∂∂

=∇ 2

2

2

22

yx


§§16.4 16.4 and and §§16.5: 16.5: skippedskipped

Restoration of less restricted degradationSuper-resolution


System Identification and System Identification and Noise ModelingNoise Modeling


System IdentificationSystem Identification

PSF must be known before image restoration.

PSF may be known a priori in some problems from, for

example, physical considerations.

More frequently, the PSF is not known in advance. It must

be determined from the degraded image.

Transfer function of a system may be determined from a

suitable test image and the blurred output:

),(),(),(

vuFvuGvuH = hh((xx,,yy))

f(x,y) g(x,y)


System Identification: Point SourceSystem Identification: Point Source

With a point source, the blurred output is the PSF h(x,y).

Example: Stars can be used to measure the PSF of a

telescope.

The problem: Turbulence of atmosphere may cause

exaggerated blurring effects.

In general, direct measurement of the PSF from point

objects is impractical.


System ID Using Sine Wave TargetsSystem ID Using Sine Wave Targets

With a sinusoidal input (vertical bars with a sinusoidal profile):

Since the image only has a frequency component at u=s0

without v components, output from a system is:

Therefore

)2cos(),( 0xsyxf π=

)2cos()0,(),( 00 xssHyxg π=


System ID Using Sine Targets (cont.)System ID Using Sine Targets (cont.)

In this way, H(u,v) may be identified point-by-point in the frequency domain with sinusoidal signals of different frequencies, and in different directions.

[ ] )()()()0,(),( 000 vsususHvuG δδδ ++−=

v

us0−s0 0


System ID Using Line TargetsSystem ID Using Line Targets

A line parallel to the y-axis may be considered the product of a delta function in the x-direction and a constant in the y-direction:

The output can be obtained by convolution:

Or, in the frequency domain,

)(),( xyxf δ=

∫ ∫ ∫∞

∞−

∞

∞−

∞

∞−=−= dyyxhdpdqpxqphyxg ),()(),(),( δ

)()0,()(),(),( vuHvvuHvuG δδ ==


System ID Using Line Targets (cont.)System ID Using Line Targets (cont.)

G(u,v) is the value of H(u,v) at v = 0.

If h(x,y) has a circulant property, H(u,v) can thus be determined completely.

If h(x,y) is separable, the horizontal and vertical line-spread functions are enough to specify the system characteristics.

v

u

H(u,v)y

x


System ID Using Edge TargetsSystem ID Using Edge Targets

The edge target ⎯ integral of a line:

Integration merely introduce a coefficient 1/j2πs into the Fourier transform, therefore, from the result of a line target,

)(),( xuyxf =

ujvuHvuG

πδ

2)()0,(),( =


Frequency Sweep TargetsFrequency Sweep Targets

It can be shown that, when the input is a linear chirp,

the output signal is the input in an envelop that is the transfer function.

-10 -5 0 5 10-1

0

1

-10 -5 0 5 10-1

0

1

)2exp()( 2axjxf π=

)2()()( axHxfxg =


System ID by CrossSystem ID by Cross--CorrelationCorrelation

Cross-correlate the input with a system output:

The output spectrum of the cross-correlator:

Pf(s) is the power spectrum of the input. If Pf(s) is constant (white noise), H(s)=Z(s).

hh((xx))Rfg

f(x) g(x)

z(x)

)()()()()()()()( sPsHsFsFsHsFsGsZ f=== ∗∗


Identify the System from ImageIdentify the System from Image

It is not always possible to calibrate the imaging system using the above methods. In this case, the PSF can only be estimated from the degraded image.Point sources such as stars: vulnerable to corruption by system noise.Lines: can be integrated along the line to combat noise. The lines must be thin and bright enough.Edges: – Can be integrated along the edge to combat noise.– Curvature may cause blurring.– Skewed edge may be utilized by rotation.


Noise ModelingNoise Modeling

Noise is categorized into:

– Electronic noise: Gaussian, characterized by σ2,

– Photoelectronic noise: at low intensity, Poisson, and at

high light level, Gaussian, and

– Film grain noise: Gaussian.

Noise may be measured from the observed image, in flat

areas.


Implementation of Image Implementation of Image RestorationRestoration


Implementation of Image RestorationImplementation of Image Restoration

Transform domain filtering

Large-kernel convolution

Small-kernel convolution

Decomposition of convolution kernel


Transform Domain FilteringTransform Domain Filtering

A linear and shift invariant restoration operation can be done by Fourier transform domain multiplications.

– Fourier transform the degraded image.

– Point-by-point multiply the image spectrum with the restoration transfer function.

– Inverse Fourier transform to produce a restored image.

With FFT, the number of operations ∝ Nlog2N. Number of point-by-point multiplications is N2.

Using other 2D linear transforms, complexity of filtering depends on the specific transform.


LargeLarge--Kernel ConvolutionKernel Convolution

A linear and shift-invariant restoration can be done

directly in the space domain using convolution.

It is equivalent to the Fourier transform approach when

both the image and the convolution kernel are N-by-N in

size.

The number of required operations is N4, significantly

greater than the FFT method.

It is impractical in most cases.


SmallSmall--Kernel ConvolutionKernel Convolution

Usually the transfer function of a restoration filter spans a broad band in the frequency domain.

A broad transfer function corresponds to a narrow impulse response in the space domain.

A large convolution kernel is often unnecessary as only a few elements around the center are significant.

A properly designed small kernel, M-by-M (M << N), may provide satisfactory restoration results.

Saving in computations: N4 → N2M2.A small kernel can be obtained by subsampling the transfer function or truncating the large kernel.


SubsamplingSubsampling Transfer FunctionTransfer Function

By IDFT, an N-by-N transfer function leads to a

convolution kernel of the same size.

Subsampling the transfer function to M-by-M, the

resulting kernel will be M-by-M.

Before subsampling, the original high-resolution transfer

function should be smoothed to avoid aliasing.

This way, main features in the frequency response of the

filter are preserved.


SubsamplingSubsampling Transfer Function (cont.)Transfer Function (cont.)

A 28-element transfer function is subsampled to 7, and

then take IDFT to yield a 7-point convolution kernel:

]001.04.1.23.44.69.91.191.69.44.23.1.04.01.0[=F

]0001.02.06.12.18.21.18.12.06.02.01.00[=f

]0023.123.00[ˆ =F

]08.13.18.21.18.13.08[.ˆ =f


Truncating Large KernelTruncating Large Kernel

Simply truncating a large kernel may alter the transfer

function significantly.

It is necessary to compare the modified transfer function

with the original to see if it is acceptable.

Nonetheless a truncated kernel is generally the best

approximation to a large kernel, in the mean square

sense.


Computing the KernelComputing the Kernel

An N-point kernel and its N-point transfer function form a Fourier transform pair:

Or, in the matrix form:

where W is an N-by-N unitary matrix with elements

Note that the DC component in F is not centered.

1,,1,0,2exp1 1

0−=⎟

⎠⎞

⎜⎝⎛−= ∑

−

=

NkNikjf

NF

N

iik …π

FWffWF 1−==

⎟⎠⎞

⎜⎝⎛−=

Nikj

Nw ki π2exp1

,


Computing the Kernel (cont.)Computing the Kernel (cont.)

To shift the DC component to the center:

Note: No fast algorithm exists for the shifted DFT.

Now, consider an M-by-1 kernel g, and compute from it an N-by-1 transfer function, where N>M, with the same frequency range as an M-point spectrum:

⎩⎨⎧

−=⎥⎦

⎤⎢⎣⎡ −

−−= ∑−

= odd 2/)1(even 2/

,)()(2exp1 1

0 NNNN

aN

aiakjfN

FN

iik π

1010

,)()(2exp1 1

0 −≤≤−≤≤

⎥⎦⎤

⎢⎣⎡ −

−−= ∑−

= MmNk

Nbmakjg

MF

M

mmk π

⎩⎨⎧

−=

⎩⎨⎧

−=

odd 2/)1(even 2/

odd 2/)1(even 2/

MMMM

bNNNN

a


Computing the Kernel (cont.)Computing the Kernel (cont.)

Matrix formula for computing an N-point spectrum from an M-point signal:

where W is an N-by-M matrix with elements

Suppose F is an N-point transfer function. Try to find an M-point convolution kernel g from F.

A minimum mean square error solution to the over-determined problem (N equations and M unknowns):

gWF =

⎥⎦⎤

⎢⎣⎡ −

−−= )()(2exp1

, bmN

akjM

w mk π

FWFWWWg +∗−∗ == 1][57


Kernel DecompositionKernel Decomposition

Hardware implementation of 2D convolution usually

applies to 3-by-3 kernels.

It is useful to decompose a larger kernel into several

small ones.

– SVD method: singular value decomposition

– SGK (small generating kernel) decomposition


SVD ConvolutionSVD Convolution

Review of SVD.

An M-by-M matrix of rank R can be expressed as a sum of R M-by-M matrices of rank 1, each of which is an outer product of two M-by-1 eigenvectors, and weighted by a singular value:

By using the commutative law of convolution:

∑=

=Λ=R

iiii

1'' vuVUA λ

( )∑∑==

∗=∗=∗R

iiii

R

iiii

11'' vuFvuFAF λλ


SVD Convolution: an ExampleSVD Convolution: an Example

Consider

The eigenvalues and eigenvectors:

Thus, the singular values and SVD are given by

Convolution with Convolution with FF may be implemented as convolutions may be implemented as convolutions with with uu11 (row) and (row) and vv11 (column) sequentially, or vice versa.(column) sequentially, or vice versa.

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

121232121

F⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡==

61061017106106

'' FFFF

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−

−====

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

707.0542.0454.00643.0766.0707.0542.0454.0

][][,014.086.28

321321

3

2

1

vvvVuuuUλλλ

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−==

0000372.000037.5

FVU'Λ

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=Λ≈Λ= ∑

= 11.187.111.187.115.387.111.187.111.1

'' 111,1

2

1, vuvuF

iiiii

Note


SVD Convolution Example 2SVD Convolution Example 2

The following example makes the point more strongly:

The singular values are: [36, 0, 0]. Only one term in the SVD summation is needed for exact decomposition:

The number of operations becomes 6, instead of 9 for the straight convolution with F.

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

12122121

4F⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−−−

==⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

796.0691.0408.0100.0569.0816.0596.0446.0408.0

,00

36

3

2

1

VUλλλ

[ ]121121

'111,1

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=Λ= vuF


SGK DecompositionSGK Decomposition

The small generating kernel (SGK) decomposition can be used in combination with SVD to decompose any M-by-Mkernel into (M−1)/2 kernels sized 3-by-3.As already shown,

M-by-1 vectors ui and vi can then decomposed by SGK as a sequential convolution of 3-by-1 kernels.

With this approach, R(M −1) convolutions with 3-by-1 kernels are required to implement convolution with an M-by-M kernel of rank R.

∑=

=Λ=R

iiii

1'' vuVUA λ


SGK Decomposition: an ExampleSGK Decomposition: an Example

Consider a vector h=[1, 3, 4, 3, 1]', and try to decompose it into two 3-by-1 kernels, f and g.

In the frequency domain:

The job becomes factoring H(s) into two transfer functions suitable for 3-by-3 convolution sequences.

Express H in z-transformation:

gfhxgfxhy ∗=⇒∗∗=∗= )(

)()()( sGsFsH =

⎥⎦

⎤⎢⎣

⎡++++=

++++==

−

−−−−−

=

−∑

0

3

0

32

0

23

0

1440

44

33

22

110

1

0

1)(

hhz

hhz

hhz

hhzzh

zhzhzhzhhzhzHM

ii


SGK Decomposition Example (cont.)SGK Decomposition Example (cont.)

Similarly, for F and G:

Let H(z)=F(z)G(z), and substitute h=[1, 3, 4, 3, 1]' into the equation:

Factor the left polynonial:

⎥⎦

⎤⎢⎣

⎡++= −

0

2

0

1220)(

ffz

ffzzfzF ⎥

⎦

⎤⎢⎣

⎡++= −

0

2

0

1220)(

ggz

ggzzgzG

[ ] ⎥⎦

⎤⎢⎣

⎡++⎥

⎦

⎤⎢⎣

⎡++=++++

0

2

0

12

0

2

0

1200

234 1343gg

ggz

ff

ffzgfzzzz

( )( )1211343 22234 ++++=++++ zzzzzzzz


SGK Decomposition Example (cont.)SGK Decomposition Example (cont.)

Therefore:

Thus the 5-by-5 kernel hh' can be decomposed into convolution of two 3-by-3 kernels, ff' and gg':

[ ][ ] ⎥⎦

⎤⎢⎣

⎡++⎥

⎦

⎤⎢⎣

⎡++=++++

0

2

0

12

0

2

0

1200

22 121gg

ggz

ff

ffzgfzzzz

[ ] [ ]'121'111 == gf

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡∗

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

121242121

111111111

134313912934121612439129313431


A Restoration ExampleA Restoration Example

Least Square Based Convolution Mask for Least Square Based Convolution Mask for Image RestorationImage Restoration

Image restoration using truncated SVD filter Image restoration using truncated SVD filter bank based on an energy criterionbank based on an energy criterion

A Brief Introduction


Matrix Formulation: ReviewMatrix Formulation: Review

Each row of GP forms an N×N circulant matrix as in 1D case:

where

Form a block block circulantcirculant matrixmatrix, sized N2×N2:

Ni GGGG ,,,,, 21 ……

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−

=

)1()1()(

)3()1()2()2()()1(

iii

iii

iii

i

gNgNg

ggggNgg

G

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

− 11

312

21

GGG

GGGGGG

GB

NN

N

PBP fGh =

Remarks:– Using matrix algebra, computation load is huge.– Useful in filter design such as in image restoration.


Discrete Fourier TransformDiscrete Fourier Transform

The spectrum vector:

The two-dimensional DFT:

0 1 N/2 N−1

0 2fN/N fNi2fN/N

i

−2fN/N

f00

f00


The SVD TransformThe SVD Transform

Singular value decomposition of a matrix: Any N-by-Nmatrix A can be expressed as

The columns of U and V are the eigenvectors of AA' and A'A respectively. They are orthogonal.Λ is an N-by-N diagonal matrix containing the singular values of A along its diagonal.Since U and V are orthogonal,

If A is symmetric, U=V.Note, the transform matrices U and V depend on the image A.

'VUA Λ=

AVU'=Λ (The forward SVD transformThe forward SVD transform)

(The inverse SVD transformThe inverse SVD transform)


The SVD Transform (cont.)The SVD Transform (cont.)

Expand the inverse SVD transform (image reconstruction) into a weighted sum of the basis images uivi':

It can be shown that rank of the basis matrix uivi' = 1.We can drop terms in the sum corresponding to small λksto get an approximate reconstruction with little error:

Kernel matrices U and V are needed in reconstruction.

∑=

=Λ=N

iiii

1'' vuVUA λ

∑=

≈R

iiii

1'ˆ vuA λ


Transfer Function of Wiener FilterTransfer Function of Wiener Filter

),(),(),(),(),( 2 vuPvuPvuH

vuHvuGfn+

=∗

Pf(s)

Pn(s)

H(s)

1

Compensate for blurring

Suppress noise

Blurring

Image

Noise

G(s)


Algebraic Restoration & Wiener FilterAlgebraic Restoration & Wiener Filter

Minimizing is to find an estimate of f so that if it is blurred by H, the result will be as close to the observed gas possible.

In other words, here one seeks to minimize the difference between the blurred originalblurred original and a similarly blurred similarly blurred estimate of the originalestimate of the original.

In Wiener filtering, one seeks to minimize the difference between the restored imagerestored image and the originaloriginal.

)ˆ()'ˆ(||)ˆ(||)ˆ( fHgfHgfef −−==W

)ˆ(fW


NoteNote

Error in Eq.(84) of p.421: U and V are not FF' and F'F,

but matrices whose columns are eigenvectors of FF' and

F'F.

digital image processing - california institute of...

Documents