Chapter 5

Image Restoration

Preview Goal: improve an image in some predefined sense.Image enhancement: subjective processImage restoration: objective processRestoration attempts to reconstruct an image that has been degraded by using a priori knowledge of the degradation process.Modeling the degradation and applying the inverse process to recover the original image.When degradation model is unknown blind deconvolution (ICA)

A Model of Degradation orGiven g(x,y), some knowledge about H, and some knowledge about the noise term, obtain an estimate of the original image.

),(),(*),(),( yxyxfyxhyxg

),(),(),(),( vuNvuFvuHvuG

Noise ModelsGaussian noise: electronic circuit sensor noiseRayleigh noise: range imagingErlang (Gamma noise): laser imagingExponential noise: laser imagingUniform noiseImpulse (salt-and-pepper noise): faulty switchingPeriodic noise

Gaussian NoiseThe PDF of a Gaussian random variable, z, is given by:

22 2/)(

2

1)(

zezp

Rayleigh Noise• The PDF of Rayleigh noise is given by:

• Mean and variance are given by:

• Useful for approximating skewed histograms.

azaz

eazbzp

baz

for for 0

)(2

)(/)( 2

4/ba 4

)4(2

b

Erlang (Gamma) Noise• The PDF of Erlang noise is given by:

• Mean and variance:

0for 0

0for )!1()(

1

z

zeb

zazp

azbb

a

b 2

2

a

b

Exponential NoiseThe PDF of exponential noise is given by:

where a >0Mean and variance:

0for 0

0for )(

z

zaezp

az

a

1

22 1

a

Uniform NoiseThe PDF of uniform noise is given by:

Mean and variance:

otherwise 0

if 1

)( bzaabzp

2

ba

12

)( 22 ab

Impulse (Salt-and-Pepper) Noise

The PDF of (bipolar) impulse noise is given by:

otherwise 0

for

for

)( bzP

azP

zp b

a

Periodic NoiseArises typically from electrical or electromechanical interference during image acquisition.The only type of spatially dependent noise considered in this chapter.

Illustration (I)

Illustration (II)

Estimation of Noise Parameters

Periodic noises: from Fourier spectrumOthers: try to compute the mean and variance of a subimage S (containing only constant gray levels).

Restoration in the Presence of Noise Only – Spatial

FilteringMean filters:

Arithmetic mean filtersGeometric mean filterHarmonic mean filter:Contraharmonic mean filter:

Q: the order of the filter. Q>0 eliminates pepper noise, Q <0 eliminates salt noise.

xySts tsg

mnyxf

),( ),(1

),(ˆ

xy

xy

Sts

Q

Sts

Q

tsg

tsg

yxf

),(

),(

1

),(

),(

),(ˆ

Illustration (I)

Illustration (II)

Illustration (III)

Order-Statistics FiltersMedian filtersMax and min filtersMidpoint filter: Alpha-trimmed mean filter: delete the d/2 lowest and d/2 highest gray-level values of g(s,t) in the neighborhood of Sxy , the average

)},({min)},({max

2

1),(ˆ

),(),(tsgtsgyxf

xyxy StsSts

xySts

r tsgdmn

yxf),(

),(1

),(ˆ

Illustration (I)

Illustration (II)

Illustration (III)

Adaptive FiltersFilter’s behavior changes based on statistical characteristics of the image inside the filter region defined by the mxn window.Adaptive, local noise reduction filterAdaptive median filter

Adaptive, local noise reduction filter

(a) g(x,y): the value of the noisy image at (x,y)(b) The variance of the noise(c) The mean of the pixels in Sxy

(d) Local variance of the pixels in Sxy

If (b) is zero, return g(x,y)If (d) is high relative to (b), the filter should return a value close to g(x,y)If the two variances are equal, return the arithmetic mean of the pixels in Sxy

]),([),(),(ˆ2

2

LL

myxgyxgyxf

Illustration

Adaptive Median FilterNotation:

zmin: minimum gray level value in Sxy

zmax: maximum gray level value in Sxy

zmed: median of gray levels in Sxy

zxy: gray level value at (x,y)Smax: maximum allowed size of Sxy

Level A: A1= zmed – zmin, A2= zmed – zmaxif A1> 0 and A2 <0, go to level BElse increase the window sizeIf window size <= Smax repeat level Aelse output zxy

Level B: B1= zxy – zmin, B2= zxy – zmax

if B1> 0 and B2 <0, output zxyElse output zmed

Illustration

Periodic Noise ReductionBy Fourier domain filtering:

Bandreject filtersBandpass filtersNotch filters

Illustration

Ideal Notch Reject FilterIdeal notch reject filter:

where

otherwise 1

),(or ),( if 0v)H(u, 0201 DvuDDvuD

2/120

202

2/120

201

])2/()2/[(v)(u,D

])2/()2/[(v)(u,D

vMvuMu

vMvuMu

Butterworth Notch Reject Filter

n

vuDvuDD

),(),(1

1v)H(u,

21

20

Gaussian Notch Reject Filter

20

21 ),(),(

2

1exp1)vH(u,

D

vuDvuD

Notch Filters

Linear, Position-Invariant Degradations

Estimating the degradation function

By image observationBy experimentationBy modeling

Estimation by Image Observation

In the strong signal area, using sample gray levels of the object and background to construct an unblurred image

Then,

Use Hs(u,v) to estimate H(u,v)

),(ˆ yxf s

),(ˆ),(

),(HvuF

vuGvu

s

ss

Estimation by Experimentation

Simulate an impulse by a (very) bright dot of light, the response G(u,v) is related to H(u,v) by:

A

),(),(H

vuGvu

Figure 5.24

Estimation by ModelingModeling atmospheric turbulence

])(exp[v)H(u, 6/522 vuk

Atmospheric Turbulence

Estimation by Modeling (cont’d)

Modeling effect of planar motion x0(t),y0(t):

If T is the duration of the exposure, then

It can be shown that:

dttyytxxfg(x,y)T

)](),([ 000

dttvytuxjvuFv)G(uT

0 00 )]()([2exp),(,

Motion BlurIf x0(t)=at/T and y0(t)=0, then

]exp[)sin(

)](2exp[),(0 0

uajuaua

T

dttuxjvuHT

Motion Blur Example

DeconvolutionInverse filteringMinimum mean square error (Wiener) filteringConstrained least squares filteringGeometric mean filterhttp://vision.cs.nccu.edu.tw/publications/CVPRIP2003_A.pdf

Results (Inverse Filter)

Results (Inverse and Wiener)

Results (Motion Blurs)

Results (Constrained LS Filter)

Geometric Transformations

Image warpingSpatial transformationsGray-level interpolation