1

ECE 472/572 - Digital Image Processing

Lecture 5 - Image Enhancement - Frequency Domain Filters 09/13/11

2

Roadmap

¬  Introduction –  Image format (vector vs. bitmap) –  IP vs. CV vs. CG –  HLIP vs. LLIP –  Image acquisition

¬  Perception –  Structure of human eye

•  rods vs. conss (Scotopic vision vs. photopic vision)

•  Fovea and blind spot •  Flexible lens (near-sighted vs. far-

sighted) –  Brightness adaptation and

Discrimination •  Weber ratio •  Dynamic range

–  Image resolution •  Sampling vs. quantization

¬  Image enhancement –  Enhancement vs. restoration –  Spatial domain methods

•  Point-based methods –  Log trans. vs. Power-law

•  Gamma correction •  Dynamic range compression

–  Contrast stretching vs. HE •  What is HE? •  Derivation of tran. func.

–  Gray-level vs. Bit plane slicing –  Image averaging (principle)

•  Mask-based (neighborhood-based) methods - spatial filter

–  Smoothing vs. Sharpening filter –  Linear vs. Non-linear filter –  Smoothing

•  Average vs. weighted average •  Average vs. Median

–  Sharpening •  UM vs. High boosting •  1st vs. 2nd derivatives

–  Frequency domain methods

3

Questions

¬  In-depth understanding –  Why do we need to conduct image processing in the frequency

domain? –  What does Fourier series do? –  What does the Fourier spectrum of an image tell you? –  How to calculate the fundamental frequency? –  Why is padding necessary?

¬ Properties –  Is FT a linear or nonlinear process? –  What would the FT of a rotated image look like? –  When implementing FFT, what kind of properties are used? –  What does the autocorrelation of an image tell you? –  What is F(0,0)? Or Why is the center of the FT extremely bright?

2

4

Why FT? – 1

5

Why FT? – 2

( ) ( ) ( ) ( ) ( ) ( )( ) ( ) ( ){ }vuFvuHFyxg

vuFvuHvuGyxfyxhyxg

,,,,,,,,,

1−=

=⇔∗=

6

Another example

3

7

Fourier series

¬ Fourier series (SF) can represent any function over a finite interval TF

¬ Outside TF, SF repeats periodically with period TF.

€

complex form: s t( ) = cnej2nπfF t

n=−∞

∞

∑

cn =1TF

s t( )e− j 2nπfF tdt−TF / 2

TF / 2∫

8

Fourier series (cont’)

¬ TF is the interval of signal s(t) over which the Fourier series represents

¬  fF = 1/TF is the fundamental frequency of the Fourier series representation

¬ n is called the “harmonic number” –  E.g., 2fF is the second harmonic of the fundamental frequency fF.

¬ The Fourier series representation is always periodic and is linear combinations of sinusoids at fF and its harmonics.

9

Fourier transform

¬ Describe the frequency distribution

nfF

cn

u

v

(0,0)

4

10

11

1-D Fourier transform

¬  Fourier transform:

¬  Inverse FT:

¬  Complex form

¬  Fourier spectrum ¬  Power spectrum

(spectral density) ¬  Phase angle

€

F u( ) = f t( )−∞

∞

∫ exp − j2πut( )dt

f t( ) = F u( )−∞

∞

∫ exp j2πut( )du

F u( ) = R u( ) + jI u( )F u( ) = F u( )e jφ u( )

F u( ) = R2 u( ) + I2 u( )

P u( ) = F u( )2

φ u( ) = tan−1I u( )R u( )

'

( )

*

+ ,

€

F u( ) =1N

f x( )x= 0

N−1

∑ exp − j2πux /N( )

f x( ) = F u( )u= 0

N−1

∑ exp j2πux /N( )

DFT

12

2-D Fourier transform

¬ CFT

¬ DFT

( ) ( ) ( )[ ]

( ) ( ) ( )[ ]∫ ∫

∫ ∫∞

∞−

∞

∞−

∞

∞−

∞

∞−

+=

+−=

dudvvyuxjvuFyxf

dxdyvyuxjyxfvuF

π

π

2exp,,

2exp,,

( ) ( ) ( )[ ]

( ) ( ) ( )[ ]∑∑

∑∑−

=

−

=

−

=

−

=

+=

+−=

1

0

1

0

1

0

1

0

//2exp,,

//2exp,1

,

M

u

N

v

M

x

N

y

NvyMuxjvuFyxf

NvyMuxjyxfMN

vuF

π

π

5

13

Understanding and implementing Fourier transform

( ) ( ) ( )[ ]

( ) ( ) ( )[ ]∑∑

∑∑−

=

−

=

−

=

−

=

+=

+−=

1

0

1

0

1

0

1

0

//2exp,,

//2exp,1

,

M

u

N

v

M

x

N

y

NvyMuxjvuFyxf

NvyMuxjyxfMN

vuF

π

π

(0,0)

f(x,y)

x

y (0,0)

|F(u,v)| u

v yN

vxM

uΔ

=ΔΔ

=Δ1,1

14

0)*)1,1(*)0,1(

*)1,0(*)0,0((2*2

1)1,1(

0)*)1,1(*)0,1(

*)1,0(*)0,0((2*2

1)0,1(

5.127)*)1,1(*)0,1(

*)1,0(*)0,0((2*2

1)1,0(

5.127))1,1()0,1()1,0()0,0((2*2

1)0,0(

)2/1*12/1*1(2)2/0*12/1*1(2

)2/1*12/0*1(2)2/0*12/0*1(2

)2/1*02/1*1(2)2/0*02/1*1(2

)2/1*02/0*1(2)2/0*02/0*1(2

)2/1*12/1*0(2)2/0*12/1*0(2

)2/1*12/0*0(2)2/0*12/0*0(2

=++

+=

=++

+=

−=++

+=

=+++=

+−+−

+−+−

+−+−

+−+−

+−+−

+−+−

ππ

ππ

ππ

ππ

ππ

ππ

jj

jj

jj

jj

jj

jj

efef

efefF

efef

efefF

efef

efefF

ffffF

15

Understanding and implementing Fourier transform

¬ According to “translation”

( )( ) ( )2/,2/1,

period, complete oneFor

NvMuFyxf yx −−⇔− +

(0,0)

f(x,y) x

y

-255

255

(0,0)

f(x,y)(-1)x+y

x

y

-0

0

(0,0)

|F(u-M/2,v-N/2)| u

v

6

16

Understanding and implementing Fourier transform

17

Block diagram of FTIFT

f(x,y)(-1)x+y g(x,y)(-1)x+y

18

Impulse transforms

7

19

Typical transforms

¬ Gaussian hump ßà Gaussian hump

¬ Rectangular (square aperture) ßàsinc

¬ Pillbox (circular aperture) ßàjinc

20

Typical transforms

¬ Gaussian ridge

¬ Line impulse

( )2exp xπ−

( )0when

/exp 221

→

−−

τ

τπτ x

21

Reference

¬ All figures scanned from R. N. Bracewell’s “Two-Dimensional Imaging,” Prentice Hall, 1995.

8

22

2D FT pairs

( )

( )( )

( )( )( )( )[ ]22

00

exp

,,,,,

,

yx

yxcombyxtriyxrectyyxx

yx

yxf

+−

±±

π

δ

δ

( )

( ) ( )( )( )( )( )[ ]22

2

00

exp

,,sin

,sin2exp2exp

1

,

vu

vucombvuc

vucvyjuxj

vuF

+−

±±

π

ππ

23

Important properties of FT

¬ Linearity (distributivity && scaling) ¬ Separability ¬ Translation ¬ Periodicity ¬ Conjugate symmetry ¬ Rotation ¬ Convolution ¬ Correlation ¬ Sampling

24

Linearity

¬ FT is a linear image processing method

Linear System

x1(t) y1(t) x2(t) y2(t)

a*x1(t) + b*x2(t) a*y1(t) + b*y2(t)

9

25

Separability

( ) ( ) ( )[ ]∑∑−

=

−

=

+−=1

0

1

0

//2exp,1

,M

x

N

y

NvyMuxjyxfMN

vuF π

f(x, y) F(x, v) Row

transform F(u, v) Column

transform

26

Translation

( ) ( )[ ] ( )( ) ( ) ( )[ ]NvyuxjvuFyyxxf

vvuuFNyvxujyxf/2exp,,

,/2exp,

0000

0000

+−⇔−−

−−⇔+

π

π

€

To show one complete period,

f x,y( ) −1( )x+y⇔ F u − N /2,v − N /2( )

27

Periodicity and Conjugate symmetry

¬ Periodicity

¬ Conjugate symmetry ( ) ( ) ( ) ( )NvNuFNvuFvNuFvuF ++=+=+= ,,,,

( ) ( )( ) ( )vuFvuF

vuFvuF−−=

−−= ∗

,,,,

10

28

Example

29

Rotation

( ) ( )00 ,, θφωθθ +=+ Frf

30

Averaging

( ) ( )0,0, Fyxf =

( ) ( ) ( )[ ]

( ) ( ) ( )[ ]∑∑

∑∑−

=

−

=

−

=

−

=

+=

+−=

1

0

1

0

1

0

1

0

//2exp,,

//2exp,1

,

M

u

N

v

M

x

N

y

NvyMuxjvuFyxf

NvyMuxjyxfMN

vuF

π

π

11

31

Convolution

¬  Continuous and discrete convolution

¬  The convolution theorem

¬  Practically, computing the discrete convolution in the frequency domain often is more efficient than doing it in the spatial domain directly

( ) ( ) ( ) ( )( ) ( ) ( ) ( )vuGvuFyxgyxf

vuGvuFyxgyxf,,,,,,,,

∗⇔

⇔∗

( ) ( ) ( ) ( )

( ) ( ) ( ) ( )∑∑

∫ ∫−

=

−

=

∞

∞−

−−=∗

−−=∗

1

0

1

0,,1,,

,,,,

M

m

N

neeee nymxgnmf

MNyxgyxf

ddyxgfyxgyxf βαβαβα

32

Correlation

¬ Continuous and discrete correlation

¬ The correlation theorem

( ) ( ) ( ) ( )

( ) ( ) ( ) ( )∑∑

∫ ∫−

=

−

=

∞

∞−

++=

++=

1

0

1

0

*

,,1,,

,,,,

M

m

N

neeee nymxgnmf

MNyxgyxf

ddyxgfyxgyxf

βαβαβα

( ) ( ) ( ) ( )( ) ( ) ( ) ( )vuGvuFyxgyxf

vuGvuFyxgyxf

,,,,,,,,

*

*

⇔

⇔

33

Correlation (cont’)

¬ Autocorrelation vs. cross correlation ¬ Autocorrelation theorem

¬ Application: template or prototype matching

( ) ( ){ } ( )2,,, vuFyxfyxfF =

12

34

Practical issues – Implement convolution in frequency domain

¬ In spatial domain

¬ In frequency domain –  f*g çè F(f)G(g) –  Phase? Mag? –  How to pad?

35

36

13

37

38

39

Difference image from convolution in the spatial domain

Convolution in the frequency domain

Conv. spatially

No padding With padding

14

40

Different enhancement approaches

¬ Lowpass filter ¬ Highpass filter ¬ Homomorphic filter

41

Lowpass filtering

¬ Ideal filter –  D(u, v): distance from point (u, v) to the origin –  cutoff frequency (D0) –  nonphysical –  radially symmetric about the origin

¬ Butterworth filter

¬ Gaussian lowpass filter

( )( )( )!

"#

>

≤=

0

0

, if 0, if 1

,DvuDDvuD

vuH

( )( )[ ] nDvuD

vuH 20/,1

1,+

=

( ) ( ) 20

2 2/,, DvuDevuH −=

42

15

43

Power ratio 99.96à99.65à99.04à97.84

( ) ( )!"

#$%

&= ∑∑

u vT vuPvuP ,/,100β

44

Highpass filter

¬ Ideal filter

¬ Butterworth filter

¬ Gaussian highpass filter

( )( )( )!

"#

>

≤=

0

0

, if 1, if 0

,DvuDDvuD

vuH

( )( )[ ] nvuDD

vuH 20 ,/11,

+=

( ) ( ) 20

2 2/,1, DvuDevuH −−=

45

16

46

47

Example

48

The Laplacian in the frequency domain

€

∇2 f x,y( ) =∂2 f∂x 2

+∂2 f∂y 2

€

H(u,v) = −4π 2 u2 + v 2( )

Pay attention to the scaling factor

17

49

UM in the frequency domain

€

g(x,y) = f (x,y) + k ∗ gmask (x,y) = f (x,y) + k ∗ ( f (x,y) − fLP (x,y))g(x,y) =ℑ−1{[1+ k ∗HHP (u,v)]F(u,v)}

50

Homomorphic filtering

¬ A simple image model –  f(x,y): the intensity is called the gray level for

monochrome image –  f(x, y) = i(x, y).r(x, y) –  0 < i(x, y) < inf, the illumination –  0< r(x, y) < 1, the reflectance

51

Homomorphic filter (cont’) ( ) ( ) ( )( ) ( ) ( ) ( )( ){ } ( ){ } ( ){ }

( ) ( ) ( ) ( ) ( )

( ) ( ) ( )

( ) ( )[ ] ( )[ ] ( )[ ]yxryxiyxsyxg

yxryxiyxs

vuFvuHvuFvuHvuS

vuFvuFvuZyxrFyxiFyxzFyxryxiyxfyxz

yxryxiyxf

ri

ri

,exp,exp,exp,

,,,

,,,,,

),(),(),(,ln,ln,,ln,ln,ln,

,,,

!!==

!+!=

+=

+=

+=

+==

⋅=

18

52

Homomorphic filter (cont’)

¬ The illumination component –  Slow spatial variations –  Low frequency

¬ The reflectance component –  Vary abruptly, particularly at the junctions of

dissimilar objects –  High frequency

¬ Homomorphic filters –  Affect low and high frequencies differently –  Compress the low frequency dynamic range –  Enhance the contrast in high frequency

53

Homomorphic Filter (cont’)

11

<

>

L

H

γ

γL

DvuDcLH evuH γγγ +−−= − ]1)[(),( )/),(( 2

02

54

19

55

Homomorphic filter - example

56

¬ Point processing

¬  Simple gray level transformations –  Image negatives –  Log transformations –  Power-law

transformations –  Contrast stretching –  Gray-level slicing –  Bit-plane slicing

¬  Histogram processing –  Histogram

equalization –  *Histogram matching

(specification) ¬  Arithmetic/logic

operations –  Image averaging

¬ Mask processing (spatial filters)

¬  Smoothing filters (blur details) –  Average, weighted

average –  Order statistics (e.g.

median)

¬  Sharpening filters (highlight details) –  Unsharp masking –  High-boost filters –  Derivative filters

•  The Laplacian •  The Gradient

•  Frequency domain filters

•  Smoothing filters (blur details)

•  Ideal lowpass filter •  Butterworth lowpass •  Gaussian lowpass

•  Sharpening filters (highlight details)

–  Unsharp masking –  High-boost filters –  Derivative filters - The

Laplacian –  Ideal highpass filter –  Butterworth highpass filter –  Gaussian highpass filter

•  Homomorphic filtering

57

FFT and IFFT