practising fourier analysis with digital images

PRACTICING FOURIERANALYSIS WITH DIGITAL

IMAGESMASTERS IN COMPUTER VISION

Frédéric Morain-Nicolier

[email protected]

2014

1. INTRODUCTION 1.1. WHO IS IT ?

CONTENTS

1. INTRODUCTION1.1 Who is it ?1.2 Outline1.3 References

2. FOURIER AND ITSREPRESENTATIONS

3. UNDERSTANDING THEFOURIER ANALYSIS

4. FOURIER ANALYSISAPPLICATIONS

2 / 139


WHO IS IT ?

3 / 139


CONTACT INFORMATIONS

Frédéric Morain-Nicolier

I http://pixel-shaker.frI [email protected] Dept Geii, IUT Troyes, 9 rue de Québec, 10026 Troyes

CedexI Phone : 03 25 42 71 68

4 / 139

1. INTRODUCTION 1.2. OUTLINE

CONTENTS





5 / 139

1. INTRODUCTION 1.2. OUTLINE

OUTLINE

I Fourier and its representationsI Understanding the Fourier AnalysisI Fourier Analysis Applications

(Focusing on Magnitude and Phase)

6 / 139

1. INTRODUCTION 1.3. REFERENCES

CONTENTS





7 / 139


WEB

I Earl F. Glynn - Research Noteshttp://research.stowers-institute.org/efg

I Nicolas Thome - Introduction to Image Processinghttp://webia.lip6.fr/~thomen/Teaching/BIMA.html

8 / 139

http://research.stowers-institute.org/efg

http://webia.lip6.fr/~thomen/Teaching/BIMA.html


BOOKS

I D.W. Kammler, A first course in Fourier analysis,Cambridge University Press, 2008.

I Jean Dhombres et Jean-Bernard Robert, Fourier, créateur dela physique mathématique, collection “Un savant, uneépoque”, Belin, 1998.

9 / 139


ARTICLES

I Q. Chen, M. Defrise and F. Deconinck, "Symmetric phase-onlymatched filtering of Fourier-Mellin transforms for imageregistration and recognition", IEEE pattern analysis and machineintelligence, vol. 16, 1994, p. 1156-1168.

I Van des Schaaf A., Van Hateren J., "Modelling the Power Spectraof Natural Images : Statistics and Information", Vision Research,vol. 36, n°17, p. 2759-2770, 1996

I Y. Shapiro, and M. Porat, “Image Representation andReconstruction from Spectral Amplitude or Phase,” in IEEEInternational Conference on Electronics, Circuits and Systems1998, Lisboa, Portugal, 1998, pp. 461-464

I F. Nicolier, O. Laligant, F. Truchetet. Discrete wavelet transformimplementation in fourier domain. Journal of ElectronicImaging, 11(3) :338–346, jul 2002

I N. Skarbnik, The Importance of Phase in Image Processing,CCIT Report, 2010

10 / 139

2. FOURIER AND ITS REPRESENTATIONS 2.1. FOURIER

CONTENTS

1. INTRODUCTION

2. FOURIER AND ITSREPRESENTATIONS2.1 Fourier2.2 Fourier Series and Transform2.3 Discrete Fourier Transform2.4 Fast Fourier Transform2.5 2D DFT



11 / 139


WHO IS FOURIER ?• 1768 (né à Auxerre) - 1830• Participe à la révolution• 1798 : campagne d’Égypte• 1802 : préfet de l’Isère (destitué à la restauration)• 1817 : élu membre de l’Académie des Sciences• 1822 : secrétaire perpétuel de l’AS• 1826 : membre de l’Académie française

• 1822 : publication de la «théorie analytique de la chaleur»

• Jean Dhombres et Jean-Bernard Robert, Fourier, créateur de la physique mathématique, collection « Un savant, une époque », Belin (1998), ISBN 2-7011-1213-3.

Joseph Fourier 12

lundi 6 septembre 2010

Jean Baptiste Joseph Fourier (1768-1830)

I 1768 (born in Auxerre) - 1830I Active in French revolutionI 1798 : Napoleon’s Egypt Campaign

12 / 139


WHO IS FOURIER ?

Where is Auxerre ?

13 / 139


WHO IS FOURIER ?

I 1802 : préfet de l’Isère (dismissed during restauration)I On the Propagation of Heat in Solid Bodies, was read to Paris

Institute on 21 dec 1807. Laplace and Lagrange objected towhat is now Fourier series : “... his analysis ... leavessomething to be desired on the score of generality and evenrigour...” (from report awarding Fourier math prize in 1811)

14 / 139


WHO IS FOURIER ?

I 1802 : préfet de l’Isère (dismissed during restauration)I 1817 : member of Académie des SciencesI 1822 : perpetual secretary of A.S.I 1826 : membre de Académie FrançaiseI 1822 : publication of La théorie analytique de la chaleur

(Analytic Theory of Heat)

15 / 139


WHO IS FOURIER ?

I In La Théorie Analytique de la Chaleur (Analytic Theory ofHeat) (1822), FourierI developed the theory of the series known by his name,I and applied it to the solution of boundary-value problems

in partial differential equations.

Good Book (in french !) : Jean Dhombres et Jean-BernardRobert, Fourier, créateur de la physique mathématique, collection“Un savant, une époque”, Belin, 1998.

16 / 139

2. FOURIER AND ITS REPRESENTATIONS 2.2. FOURIER SERIES AND TRANSFORM

CONTENTS

1. INTRODUCTION




17 / 139


FOURIER TRANSFORM : WHY ? 403c) Propagation de la chaleur

lundi 6 septembre 2010

18 / 139


HEAT PROPAGATION

I The temperature u(x, t) at time t ≥ 0 and coordinate x is asolution of the partial differential equation (PDE) :

∂u∂t

(x, t) = a2 ∂2u∂x2 (x, y) (2.1)

(a2 is the thermal diffusivity of the material).

I Fourier observed that

e2πisx.e−4π2a2s2t (2.2)

satisfies the PDE for every choice of s. Its idea was to combinesuch elementary solutions.

19 / 139


HEAT PROPAGATION : SOLUTION

I Fourier wrote the solution as

u(x, t) =∫ ∞

−∞A(s)e2πisxe−4π2a2s2tds. (2.3)

The function A(s) is needed : as the initial temperature isknown,

u(x, 0) =∫ ∞

−∞A(s)e2πisxds (we recognize the synthesis equation).

(2.4)

I A(s) can be computed from

A(s) =∫ ∞

−∞u(x, 0)e−2πisxdx. (2.5)

20 / 139


FOURIER SERIESI Is only defined on periodic signals :

f (t + T) = f (t). (2.6)

6

Periodic Signals

Biological time series can be quite complex, and will contain noise.

x(t + T) = x(t)

-2-1

01

2

t

x(t)

0 π 2 π 3π 2 2π 3π 4π

“Biological” Time Series

T0

(Source : Earl F. Glynn - Research Notes)

I The fundamental period T0 is the smallest T satisfying (2.6).Fundamental frequency f0 and angular frequency ω0 are :

ω0 =2π

T0= 2πf0. (2.7)

21 / 139


FOURIER SERIES : REAL COEFFICIENTSExpansion of continuous function into weighted sum of sinesand cosines.

I A P0-periodic function f , defined on R can be written as

f (t) = a0 +∞

∑k=1

(ak cos(kω0t) + bk sin(kω0t)) (2.8)

with

a0 =1

P0

∫

P0

f (t)dt, (2.9)

ak =2

P0

∫

P0

f (t) cos(kωt)dt, (2.10)

bk =2

P0

∫

P0

f (t) sin(kωt)dt. (2.11)

22 / 139


FOURIER SERIES : AN EXAMPLE

(Source : http://www.science.org.au/nova/029/029img/wave1.gif)

23 / 139

http://www.science.org.au/nova/029/029img/wave1.gif


FOURIER SERIES : COMPLEX COEFFICIENTS

IWith complex coefficients :

f (t) =∞

∑k=−∞

ckeikω0t (2.12)

where

ck =1

P0

∫

P0

f (t)e−ikω0tdt. (2.13)

I If f (t) is real, c−k = c∗k .I For k = 0, ck = average value of f (t) over one period.I a0/2 = c0 ; ak = ck + c−k ; bk = i(ck − c−k)

24 / 139


FOURIER SERIES : COMPLEX COEFFICIENTS

f (t) =∞

∑k=−∞

ckeikω0t

I Coefficients can be written as

ck = |ck|eiφk (keep this in mind). (2.14)

I ck are the spectral coefficients of f .I Plot of |ck| vs angular frequency ω is the Magnitude

spectrum.I Plot of φk vs ω is the phase spectrum.I With discrete Fourier frequencies (kω0), both are discrete

spectra.25 / 139


FOURIER SERIES : OTHER EXAMPLESGiven a π-periodic x(t) = t, its Fourier serie is

x(t) = 2(

sin t− sin 2t2

+sin 3t

3− . . .

). (2.15)

14

Fourier Series

selected

Approximate any function as truncated Fourier series

Given: x(t) = t Fourier Series:

−+−= ...33sin

22sinsin2)( ttttx

−+−= ...33sin

22sinsin2)( ttttx

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

Fourier Terms in Expansion of x(t) = t

t

Four

ier T

erm

s

0 π 4 π 2 3π 4 π

1

2

3

4

5 6

Fourier Series Approximation

t

x(t)

First Six Series Terms

0 π 4 π 2 3π 4 π

01

23

14

Fourier Series

selected



−+−= ...33sin

22sinsin2)( ttttx

−+−= ...33sin

22sinsin2)( ttttx

-1.0

-0.5

0.0

0.5

1.0

1.5

2.0

Fourier Terms in Expansion of x(t) = t

t

Four

ier T

erm

s

0 π 4 π 2 3π 4 π

1

2

3

4

5 6


t

x(t)

First Six Series Terms

0 π 4 π 2 3π 4 π

01

23

15

Fourier Series

selected



−+−= ...33sin

22sinsin2)( ttttx

−+−= ...33sin

22sinsin2)( ttttx


t

x(t)

0 π 4 π 2 3π 4 π

01

23 100 terms

200 terms


(show animations)

26 / 139


(CONTINUOUS) FOURIER TRANSFORM

I A non-periodic function, defined on R, can be synthesizedwith

f (t) =1

2π

∫ +∞

−∞F(ω)eiωtdω. (2.16)

I The analysis equation being

F(ω) =∫ +∞

−∞f (t)e−iωtdt. (2.17)

(Beware to convergence conditions - Gibbs - see Dirichlet theorem)

27 / 139


(CONTINUOUS) FOURIER TRANSFORM : MAIN

PROPERTIES

18

Fourier TransformProperties of the Fourier Transform

From http://en.wikipedia.org/wiki/Continuous_Fourier_transformAlso see Schaum’s Theory and Problems: Signals and Systems, Hwei P. Hsu, 1995, pp. 219-223

(Source : Wikipedia)

28 / 139


(CONTINUOUS) FOURIER TRANSFORM : MAIN

PROPERTIES

Important properties (for this course) :

I Shift theorem :

g(t− a) e−iaωG(ω). (2.18)

I Scaling :

g(at) 1|a|G(

ω

a). (2.19)

I Convolution theorem :

(g ∗ h)(t) G(ω)H(ω). (2.20)

29 / 139

2. FOURIER AND ITS REPRESENTATIONS 2.3. DISCRETE FOURIER TRANSFORM

CONTENTS

1. INTRODUCTION




30 / 139


DISCRETE FOURIER TRANSFORM

As a digital image is a discrete 2D signal, a discrete version ofFT is needed. The previous definitions are adapted.

I A discrete signal s[n] is N-periodic if s[n + N] = s[n].I Fundamental period N0 is the smallest N satisfying above

equation.I Fundamental angular frequency is Ω0 = 2π

N0.

31 / 139


DISCRETE FOURIER TRANSFORM (DFT) : DEFINITION

I A discrete signal s[n], n = 0, 1, . . . , N− 1, can be analyzedwith

S[k] = DFTs[n] =N−1

∑n=0

s[n]e−i2πkn/N. (2.21)

with k = 0, 1, . . . , N− 1I The inverse DFT (IDFT = DFT−1 = synthesis equation) is

s[n] = IDFTS[k] = 1N

N−1

∑k=0

S[k]ei2πkn/N. (2.22)

32 / 139


DISCRETE FOURIER TRANSFORM (DFT)

I One-to-one correspondence between s[n] and S[k]I DFT closely related to discrete Fourier series and the

Fourier TransformI DFT is ideal for computer manipulationI Share many of the same properties as Fourier TransformI Multiplier 1

N can be used in DFT or IDFT. Sometimes 1√N

used in both.I Remember that FT (and therefore DFT) is defined on

periodic signals.

33 / 139

2. FOURIER AND ITS REPRESENTATIONS 2.4. FAST FOURIER TRANSFORM

CONTENTS

1. INTRODUCTION




34 / 139


FAST FOURIER TRANSFORM

X[k] =N−1

∑n=0

x[n]e−i2πkn/N. (2.23)

I The FFT is a computationally efficient algorithm tocompute the Discrete Fourier Transform and its inverse.

I Evaluating the sum above directly would take O(N2)arithmetic operations.

35 / 139



X[k] =N−1

∑n=0

x[n]e−i2πkn/N, (2.24)

WknN = e−i kωn

N ⇒ X[k] =N−1

∑n=0

x[n]WknN . (2.25)

Butterfly algorithm36 / 139



I The FFT algorithm reduces the computational burden toO(N log N) arithmetic operations.

I FFT requires the number of data points to be a power of 2(usually 0 padding is used to make this true)

I FFT requires evenly-spaced time seriesI Even faster FFT with sparse signals (SFFT : Sparse FFT)

37 / 139

2. FOURIER AND ITS REPRESENTATIONS 2.5. 2D DFT

CONTENTS

1. INTRODUCTION




38 / 139


SPATIAL FREQUENCIES

33

1 Cycle

Frequency = 1 Frequency = 2

2 Cycles

Spatial Frequency in Images


39 / 139


DFT ON IMAGE

F[u, v] =1

MN

M−1

∑m=0

N−1

∑n=0

I[m, n]e−i2π(u mM+v n

N ) (2.26)

34

2D Discrete Fourier Transform

Source: Seul et al, Practical Algorithms for Image Analysis, 2000, p. 249, 262.

2D FFT can be computed as two discrete Fourier transforms in 1 dimension

I[m,n] F[u,v]

FourierTransform

Spatial Domain Frequency Domain

(0,0)

(0,0)

(0,N/2)

(0,-N/2)

(-M/2,0) (M/2,0)

(M,N)

∑ ∑−

=

+−−

=

⋅=1

0

21

0],[1],[

M

m

Nvn

Mumi

N

nenmI

MNvuF π

M pixelsSM units

N p

ixel

sS N

units


40 / 139


DFT ON IMAGE

I Separable implementation :

2D DFT (or FFT) is computed as two stages of 1D discreteFourier transforms (matlab : fft2).

I Ix Ixy

(process on columns) (process on lines)

41 / 139

fft2

3. UNDERSTANDING THE FOURIER ANALYSIS 3.1. READING THE 2D-DFT

CONTENTS

1. INTRODUCTION


3. UNDERSTANDING THEFOURIER ANALYSIS3.1 Reading the 2D-DFT3.2 Magnitude and Phase Spectra3.3 Translation and Rotation in

Fourier Domain3.4 Magnitude and Phase

Information3.5 Magnitude and Phase

Reconstruction


42 / 139


WHERE IS THE INFORMATION ?

35

2D Discrete Fourier Transform

I[m,n] F[u,v]

FourierTransform


(0,0)

(0,0)

(0,N/2)

(0,-N/2)

(-M/2,0) (M/2,0)

(M,N)

M pixelsSM units

N p

ixel

sS N

units

Edge represents highest frequency,smallest resolvable length (2 pixels)

Center represents lowest frequency,which represents average pixel value


43 / 139


EXAMPLE 1

36

2D FFT ExampleFFTs Using ImageJ


ImageJ Steps: (1) File | Open, (2) Process | FFT | FFT

(0,0) Origin (0,0) Origin


Image 2D-DFT (Magnitude)

44 / 139


EXAMPLE - SWAPPING THE QUADRANTS

37



ImageJ Steps: Process | FFT | Swap Quadrants

(0,0) Origin

(0,0) Origin

Regularity in image manifests itself in the degree of order or randomness in FFT pattern.

Default display is to swap quadrants(Source : Earl F. Glynn - Research Notes)

Image 2D-DFT (Magnitude)(matlab : fftshift)

45 / 139

fftshift


TOY EXAMPLES170 The Fourier transform

Figure 7.1 Examples of Fourier magnitude images (right column) of images containing onlysinusoids (left and middle column). Axes have been added for clarity. See text for details.

(Source : Introduction to Image Processing - Univ. Utrecht)

46 / 139


REAL EXAMPLE

38


Overland Park Arboretum and Botanical Gardens, June 2006






47 / 139


REAL EXAMPLES7.1 The relation between digital images and sinusoids 171

Figure 7.2 Examples of Fourier magnitude images (right column) of real images (left column).The top example is of a binary image, the other images are grey-valued. See text for details.

(Source : Introduction to Image Processing - Univ. Utrecht)

48 / 139

3. UNDERSTANDING THE FOURIER ANALYSIS 3.2. MAGNITUDE AND PHASE SPECTRA

CONTENTS

1. INTRODUCTION





Reconstruction


49 / 139


MAGNITUDE AND PHASE SPECTRA

The DFT coefficients are complex numbers :

I Magnitude spectrum is generally considered the mostreadable

I Phase spectrum is intricated

50 / 139


AN EXAMPLE

0

0.2

0.4

0.6

0.8

1

f

50 100 150 200 250

50

100

150

200

250

0.5

1

1.5

2

2.5

3x 104

50 100 150 200 250

50

100

150

200

250 −3

−2

−1

0

1

2

3

Magnitude : ‖F‖ Phase : φ(F)

51 / 139


ANOTHER EXAMPLE

0

0.2

0.4

0.6

0.8

1

f

50 100 150 200 250

50

100

150

200

250

0.5

1

1.5

2

2.5

3

3.5x 104

50 100 150 200 250

50

100

150

200

250 −3

−2

−1

0

1

2

3

Magnitude : ‖F‖ Phase : φ(F)

52 / 139


MAGNITUDE SPECTRUM

0

0.2

0.4

0.6

0.8

1

50 100 150 200 250

50

100

150

200

250

0.5

1

1.5

2

2.5

3

3.5x 104

I F[0, 0] (at the center) contains the mean value of the image

F[u, v] =1

MN

M−1

∑m=0

N−1

∑n=0

I[m, n]e−i2π(u mM+v n

N ) (3.1)

⇒ F[0, 0] =1

MN

M−1

∑m=0

N−1

∑n=0

I[m, n] (3.2)

53 / 139


MAGNITUDE SPECTRUM

0

0.2

0.4

0.6

0.8

1

50 100 150 200 250

50

100

150

200

250

0.5

1

1.5

2

2.5

3

3.5x 104

f ‖F‖

I Very high dynamicsI Low frequencies have a greater magnitude than high

frequenciesI It is common to represent log(1 + ‖F‖)

54 / 139


LOG MAGNITUDE SPECTRUM

0

0.2

0.4

0.6

0.8

1

50 100 150 200 250

50

100

150

200

250

0.5

1

1.5

2

2.5

3

3.5x 104

50 100 150 200 250

50

100

150

200

2501

2

3

4

5

6

7

8

9

10

‖F‖ log(1 + ‖F‖)55 / 139


LOG MAGNITUDE SPECTRUM #2

0

0.2

0.4

0.6

0.8

1

50 100 150 200 250

50

100

150

200

250

0.5

1

1.5

2

2.5

3x 104

50 100 150 200 250

50

100

150

200

2501

2

3

4

5

6

7

8

9

10

‖F‖ log(1 + ‖F‖)56 / 139


STRUCTURES IN MAGNITUDE SPECTRUM

0

0.2

0.4

0.6

0.8

1

50 100 150 200 250

50

100

150

200

2501

2

3

4

5

6

7

8

9

10

f log(1 + ‖F‖)

I Edge in spatial domain⇔ line in Fourier Domain(orthogonal to the edge)

57 / 139



38








I Where are the vertical and horizontal edges ?

⇒ Remember the implicit periodicity !

58 / 139



38








I Where are the vertical and horizontal edges ?⇒ Remember the implicit periodicity !

59 / 139


STRUCTURES IN MAGNITUDE SPECTRUMI Strong main lines in image are emphasized in Fourier

40

Application of FFTPattern/Texture Recognition

Source: Lee and Chen, A New Method for Coarse Classification of Textures and Class Weight Estimation for Texture Retrieval, Pattern Recognition and Image Analysis, Vol. 12, No. 4, 2002, pp. 400–410.


60 / 139



I Strong main lines in image are emphasized in Fourier

I

(Source : N. Thome)

61 / 139

3. UNDERSTANDING THE FOURIER ANALYSIS 3.3. TRANSLATION AND ROTATION IN FOURIER DOMAIN

CONTENTS

1. INTRODUCTION





Reconstruction


62 / 139


TRANSLATION EXAMPLE

50 100 150 200 250

50

100

150

200

2500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

50 100 150 200 250

50

100

150

200

2500

0.5

1

1.5

2

2.5

3

3.5

4

f log(1 + ‖F‖)

50 100 150 200 250

50

100

150

200

2500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

50 100 150 200 250

50

100

150

200

2500

0.5

1

1.5

2

2.5

3

3.5

4

fT log(1 + ‖FT‖)63 / 139


TRANSLATION

I Shift in spatial domain :

f [k]⇒ F[u] ⇒ ‖F[u]‖f [k− a]⇒ e−i2πa u

N F[u] ⇒ ‖F[u]‖. (3.3)

I Magnitude spectrum is invariant to spatial translation.I Localization information is in phase.I Remember this for later !

64 / 139


ROTATION EXAMPLE

(Source : N. Thome)65 / 139


ROTATION

I One can easily shows that

FT [f [x cos θ + y sin θ,−x sin θ + y cos θ]] =

F[u cos θ + v sin θ,−u sin θ + v cos θ]. (3.4)

I θ-rotation in spatial domain⇔ θ-rotation in Fourierdomain (nice !)

66 / 139


ROTATION : ANOTHER EXAMPLE

(Source : N. Thome)

I Pay attention to padding when rotating.67 / 139

3. UNDERSTANDING THE FOURIER ANALYSIS 3.4. MAGNITUDE AND PHASE INFORMATION

CONTENTS

1. INTRODUCTION





Reconstruction


68 / 139


MAGNITUDE AND PHASE

0

0.2

0.4

0.6

0.8

1

50 100 150 200 250

50

100

150

200

2501

2

3

4

5

6

7

8

9

10

50 100 150 200 250

50

100

150

200

250 −3

−2

−1

0

1

2

3

I Localization is in Phase, hard to readI Frequential content is in Magnitude, easy to read

I But, what "spatial domain" information is in magnitude (andphase) ?

69 / 139


MAGNITUDE AND PHASE

f

log(1 + ‖F‖) φ(F)

I Take the IFT with only magnitude or phase.70 / 139


IFT WITH MAGNITUDE AND PHASE ONLY

I Starting from

F = FT[f ] = ‖F‖.eiφ(F) (3.5)

I two images can be obtained :

fM = FT−1[‖F‖], (3.6)

fP = FT−1[eiφ(F)] = FT−1[F‖F‖ ]. (3.7)

71 / 139


IFT WITH MAGNITUDE AND PHASE ONLY

fM fP

I Magnitude contains almost no useful spatial informationI Main structures can be retrieved from phaseI Let’s play to mix images !

72 / 139


MAGNITUDE AND PHASE MIXINGI Take two images :

f g

I and their Fourier transforms :

F = ‖F‖.eiφ(F), (3.8)

G = ‖G‖.eiφ(G) (3.9)

73 / 139


MAGNITUDE AND PHASE MIXING

I Two new images by swapping magnitude and phase :

I1 = ‖F‖.eiφ(G), (3.10)

I2 = ‖G‖.eiφ(F). (3.11)

I Guess the result !

74 / 139


MAGNITUDE AND PHASE MIXING

f g

‖F‖.eiφ(G) ‖G‖.eiφ(F)

75 / 139


MAGNITUDE AND PHASE MIXING : ANOTHER

EXAMPLE

2

The report is organized as follows. We begin with discussion of the importance of phase demonstrating it by means of several examples. We then proceed with two primal applications in image processing tasks of segmentation and edge detection, where novel phase-based methods show good performance compared to classical methods. Subsequently we introduce our own edge detector- the Local Phase Quantization error (LPQe), and address its performance and possible applications [15]. Finally we describe the Rotated Local Phase Quantization (RLPQ) which achieves controlled image primitives deprival (presented in this report for the first time). Several RLPQ applications are also discussed.

Global and Local Phase

Global Phase

We first wish to examine qualitatively- which of the two, magnitude or phase, carries more visual information. This can be most vividly demonstrated by the following experiment: the Fourier components- (phase and magnitude) are generated for the two images of same dimensions, and then swapped (see figure 1), whereby the reconstructed images appears to be more similar to the one whose Fourier phase was used in the reconstruction. This experiment was previously suggested by Oppenheim in [7] and elsewhere.

Figure 1: Swapping the Fourier phase and magnitude in images. Top left - original Lena image. Top right- original monkey image. Bottom left- IFT of Lena phase and monkey magnitude. Bottom right- IFT of monkey phase and Lena magnitude.

(Source : N. Skarbnik - CCIT Report)

76 / 139


MAGNITUDE AND PHASE MIXING : EVEN FOR

SIGNALS !

3

Figure 3: Global Fourier phase and magnitude of Lena image. Left- global Fourier phase. Right- global Fourier magnitude. It can be seen that the magnitude decays, and most of it energy is concentrated in the middle, while the phase is distributed through all frequencies.

We have repeated the same experiment for a 1D signal- voice in this case: two different sentences, pronounced by individuals of different gender were recorded. The signals' phase and magnitude were swapped. The resulting sentences were played to human listeners- which were able to understand the meaning of the sentence, as well as to identify the gender of the speaker. Thus, it appears that most of the signal's information is carried by its phase in 1D case as well. The effect of using a swapped magnitude resulted in appearance of noise, in a manner similar to the 2D case.

The reader is encouraged to review figure 2 and to examine the spectrograms similarity, or to use this link for the audio files (click images to download the file) in order to evaluate the importance of phase in human voice signals.

Next, let us compare the global phase and magnitude by reviewing their distribution in a realistic image (Lena image in this case).

Figure 2: Exchanging the Fourier phase and magnitude in voice. Top left - woman voice spectrogram. Top right- man voice spectrogram. Bottom left- spectrogram of woman voice phase and man voice magnitude. Bottom right- spectrogram of man voice phase and woman voice magnitude. Both reconstructions are primarily dominated by Fourier phase, and not the magnitude.

(Source : N. Skarbnik - CCIT Report)

I Listen the results

77 / 139


MAGNITUDE MODELISATION

I Phase is more informative than magnitudeI The Magnitude spectra is predictable. For natural images

(and signals) :

‖F[u, b]‖ decreases when√

u2 + v2 increases.

I Some models exists, see [SCH96] 1

1. Van des Schaaf A., Van Hateren J., "Modelling the Power Spectra of Na-tural Images : Statistics and Information", Vision Research, vol. 36, n°17, p.2759-2770, 1996

78 / 139


POWER SPECTRA OF NATURAL IMAGES

From [SCH96] 2 :

I It has been found that power spectra of natural (ie notartificial) images tends to depend as 1/f 2.

I From a set of 276 images, taken from a CCD cameraI Different outdoor environments (woods, fields, parks,

residential areas), at various times of the day, in variousseasons, and in various types of weather (sunny, overcast,foggy, rainy)

I Power Spectra : S = 1Npixels‖F‖2

I Take the average Power Spectra over the 276 images

2. Van des Schaaf A., Van Hateren J., "Modelling the Power Spectra of Na-tural Images : Statistics and Information", Vision Research, vol. 36, n°17, p.2759-2770, 1996

79 / 139



22

because it is absent in a set of images without dominant orientations (mostly imagestaken from soil covered with leaves and twigs with the camera pointed vertically atrandom orientations). The small dots in Fig. 2.1A,B show the standard deviation ofthe corresponding plots of individual images in the set.

Although the average power spectrum is quite smooth, spectra of individual imagesare much more irregular and different in shape, as shown by the large standarddeviations in Fig. 2.1A and the example traces in Fig. 2.2A. The individual powerspectra differ not only in shape, but also in the total power of the spectra. This totalpower is related to the root-mean-square(r.m.s.)-contrast of individual images. Ther.m.s.-contrast equals the standard deviation of the intensity values of all pixels in theimage divided by the mean intensity. The square of the r.m.s.-contrast is proportionalto the total power in the spectral domain under consideration (see Appendix A). Thusthe r.m.s.-contrast has a simple and intuitively clear interpretation in both the spatialand the spectral domains, and we will therefore use it here to illustrate the variabilityof contrast. Different definitions of contrast that we will introduce below lead tosimilar distributions. Figure 2.2B shows the distribution of contrast values for allimages. The r.m.s.-contrast varies substantially, with a mean of 0.92 and a standarddeviation of 0.44.

FIGURE 2.1: (A) The average power as a function of spatial frequency. Fat dots show the average over thecomplete set of the logarithm of the circularly averaged individual power spectra. Small dots give thestandard deviation from the average of corresponding plots of individual images. (B) The average power asa function of orientation. Here the power spectra are first averaged over spatial frequency, then thelogarithm is taken, and finally the plots are averaged over the complete set (fat dots). Small dots as in (A).

(Source : Van des Schaaf A., Van Hateren J., "Modelling the Power Spectra of Natural Images : Statistics and

Information", Vision Research, vol. 36, n°17, p. 2759-2770, 1996)

80 / 139



23

Not only the r.m.s.-contrast, and consequently the total power, vary for individualimages, but also the shape of the power spectrum. As shown in previous studies andby the almost straight line in Fig. 2.1A, the spectral power, averaged over manyimages, varies approximately as 1/f α as a function of spatial frequency, with the 1/f-exponent, α, close to 2. If we instead inspect the spectra of individual images,examples of which are shown in Fig. 2.2A, we still find that they are roughlyproportional to 1/f α. The root mean square error of the fit of the 1/f α model to thecircularly averaged power spectra (see Fig. 2.2A) is on average 0.13 log-units overthe complete set. For the individual spectra, α varies considerably from image toimage (Tolhurst et al., 1992; van Hateren 1992a; Field, 1993). Figure 2.2C showshow α is distributed for our set of images. We found an average α of 1.88 and astandard deviation of 0.43. There is a correlation of +0.35 between the r.m.s.-contrast and 1/f-exponent values. This means that steeper spectra tend to have morepower. This relationship, however, is very weak and unlikely to be useful as priorknowledge.

The 1/f α-model of the spectral power distribution is isotropic, i.e., the statistics arethe same for every orientation. Figure 2.1B showed, however, that the averagespectral power is anisotropically distributed over orientation. This is because there

FIGURE 2.2: (A) Example traces of the power spectra of five individual natural images. Dots show thelogarithm of the circularly averaged power spectrum as a function of spatial frequency. The lines show thefits of the 1/f α model. The scaling of the vertical axis belongs to the top trace. For clarity, the lower tracesare shifted -2, -6, -8, and -10 log-units, respectively. (B) Distribution of r.m.s.-contrasts for the entire set of276 natural images. (C) Distribution of 1/f-exponents (α) for the entire set.

(Source : Van des Schaaf A., Van Hateren J., "Modelling the Power Spectra of Natural Images : Statistics and

Information", Vision Research, vol. 36, n°17, p. 2759-2770, 1996)

81 / 139

3. UNDERSTANDING THE FOURIER ANALYSIS 3.5. MAGNITUDE AND PHASE RECONSTRUCTION

CONTENTS

1. INTRODUCTION





Reconstruction


82 / 139


PARTIAL MAGNITUDE RECONSTRUCTION

I How much magnitude is itpossible to reconstruct ?

I Iterative scheme proposedin [Sha98]

I Allow the reconstructionof 25% of the image

I The authors report adecent signalreconstruction after 50iterations

5

localized (Gabor)-phase is sufficient for image reconstruction and that the row data of Gabor phase depicts

the contour information starting from the first iteration of the reconstruction process [4]. This fact alone

implies that localized phase is substantial for signal analysis, as it carries all necessary signals information.

We will compare the difference signal reconstruction schemes, based on partial Fourier Transform

information, both local and global. A wide range of papers [2-4, 16, 17] are devoted to signal reconstruction

using partial spatial and frequency data divided to Fourier phase or magnitude. These can be handy in cases

where not all FT information is available (like with SAR images and X-ray crystallography) or when it is

degraded. We wish to demonstrate that the use of local features allows better algorithm performance:

faster convergence, or usage of less a priory known data. We also intend to demonstrate that phase based

algorithms sometimes result in a superior outcome compared to magnitude based ones.

We will address iterative schemes, as the closed form

solutions demand solving a large set of linear equations,

which in turn involves inversion of appropriate matrices.

Those matrices inversion is impractical for images of

dimensions above 16X16 pixels.

A Global Magnitude reconstruction scheme presented in

[2] can be seen in the following figure 5. The proposed

methods allow the reconstruction of a signal using at least

25% of the image (half of the signal in each dimension) and

its Fourier magnitude. As the reader can see, the

reconstruction is achieved by an iterative detection of the

unknown part of the signal. The authors of [2] report a

decent signal reconstruction after 50 iterations. In their

next paper [3] the authors propose a Local magnitude-

based image reconstruction method. While the

reconstruction process converges faster (fever stages for

same quality- see figure 7), it demands more computations.

Out of those stages, the last one, for example, will demand

the same number of iterations as the whole Global

Magnitude based method. On the other hand the number

of spatial points to be known in advance drops to 1 as

opposed to the ~25% needed by the Global Magnitude

based method. The proposed method consists of

application of the Global Magnitude based method to an

increasing part of the original signal, until whole signal

reconstruction is achieved. A graphical description can be

seen in figure 7.

As can be seen from the following figure, the scheme is applied to a sub signal of the dimensions of [2k, 2k],

where k is the iteration number (Xk is the appropriate label on the figure). An N by M image will demand

2log (max[ , ])M N applications of the Global magnitude scheme (which is iterative too) to a sub image of

[2k, 2k] dimensions.

Figure 5: A flowchart of image reconstruction from global magnitude. Diagram adopted from [2].

[SHA98] Y. Shapiro, and M. Porat, “Image Representation and Reconstructionfrom Spectral Amplitude or Phase,” in IEEE International Conference onElectronics, Circuits and Systems 1998, Lisboa, Portugal, 1998, pp. 461-464.

83 / 139


PARTIAL MAGNITUDE RECONSTRUCTION -ALGORITHM

FFT

x

X Phase eiφX

||Y ||IFFT

Mag.

Y

FFT

y

insertxc

x cancelborder xc

84 / 139


PARTIAL MAGNITUDE RECONSTRUCTION - EXAMPLE

Initial After 50 iterations

85 / 139


PHASE ONLY RECONSTRUCTION

I Is it possible to reconstructan image from phase only ?

I Iterative scheme proposedin [HAY82]

I Needs an M-point FFT(with M > 2N).

616 IEEE TRANSACTIONS ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOL. ASSP-28, NO. 6, DECEMBER 1980

v. NUMERICAL ALGORITHMS FOR RECONSTRUCTION FROM SAMPLES OF A PHASE FUNCTION

In Section 11, we presented two sets of conditions, embodied in Theorems 1 and 2 , under which a sequence is uniquely specified to within a positive scale factor by the phase of its Fourier transform. In this section, we describe two numerical algorithms which can be used to reconstruct a sequence satisfying the requirements of Theorem 1 from samples of its phase function when the location of the first nonzero point of x [ n ] and the interval outside of which x [ n ] is zero are known. Although these algorithms will only be discussed in terms of reconstructing sequences satisfying the conditions of Theorem 1, the reconstruction of sequences meeting the requirements of Theorem 2 may be accomplished by simply reconstructing the finite length sequence %In] defined in ( 5 ) using the nega- tive of the specified phase samples and then computing the convolutional inverse sequence.

The first algorithm presented below is an iterative technique in which the estimate of X [ . ] is improved in each iteration. This algorithm is similar to the iterative algorithms developed by Gerchberg and Saxton [ 6 ] and Fienup [ 7 ] for reconstructing a signal from magnitude information and to the iterative algorithm developed by Quatieri [ 8 ] for reconstructing a signal from its phase under the assumption that the signal is minimum phase. The second algorithm is a closed form solution which is obtained by solving a set of linear equations. Under the conditions specified in Theorem 1, this algorithm provides the desired sequence x [ n ] to within a scale factor when the location of the first nonzero point of x [ n ] and the interval outside of which x [n] is zero are known.

In the discussions which follow, x [ n ] is used to denote a sequence which satisfies the conditions of Theorem 1 and is zero outside the interval 0 d n d N - 1 with x [ O ] # 0. In the more general case (see footnote 3) , a linear phase term may be added to the given phase to accomplish this.

A. Iterative Algorithm The M-point discrete Fourier transform (DFT) of x [ n ] will

be denoted as

= I X(k>I e i o x (k) (2 1)

where it is assumed that M 2 2N. Then, an iterative technique to reconstruct the sequence x [ n ] from the M samples of its phase e,@), k = 0, 1, - - , M - 1, as illustrated in Fig. 1 and may be described as follows.

Step 1: We begin with I Xo(k) l , an initial guess of the unknown DFT magnitude and form the first estimate, X,@), of X(k) using the specified phase function, i.e.,

Xl(k ) = IXO(k)l e ie,(k) (22) Computing the inverse DFT of X,@) provides the first estimate, x1 [ n ] , of x [ n ] . Since an M-point DFT is used, x1 [ n ] is an M-point sequence which is, in general, nonzero for N < n d

Step 2: From x1 [ n ] , another sequence, y1 [ n ] , is defined by M - 1.

Step 3: The magnitude I Yl(k)l of the M-point DFT of y1 [n] is then considered as a new estimate of IX(k)l and a

I I I I I I

t r p

I I I I r - l M-POINT DFT I I I I A I I

I I I I

M-POINT IDFT

I I

Fig. 1. Block diagram of the iterative algorithm for reconstructing a signal from its phase.

From this, a new estimate x z [ n ] is obtained from the inverse DFT of X , (k). Repetitive application of Steps 2 and 3 defines the iteration.

In this iterative procedure, the total squared error between x [ n ] and its estimate is nonincreasing with each iteration. To see this, let x p [ n ] denote the estimate after the pth iteration and define the error E p as

From Parseval's theorem,

Since X(k) and X p ( k ) have the same phase, then

[HAY82] Hayes, M. "The Reconstruction of a Multidimensional SequenceFrom the Phase or Magnitude of Its Fourier Transform." Acoustics, Speechand Signal Processing, IEEE Transactions on 30, no. 2 (1982) : 140-154

86 / 139


PHASE ONLY RECONSTRUCTION - ALGORITHM

FFTM

x

X Phase eiφX

||Y ||IFFT

Mag.

Y

y

N

0

M=3N-1

FFT

initialize with||Y ||=1

N

87 / 139


PHASE ONLY RECONSTRUCTION - EXAMPLE

Initial After 10 iterations andM = 2N− 1

88 / 139

4. FOURIER ANALYSIS APPLICATIONS 4.1. CLASSIC APPLICATIONS

CONTENTS

1. INTRODUCTION



4. FOURIER ANALYSISAPPLICATIONS4.1 Classic Applications4.2 Fourier Shape Descriptors4.3 Filter Banks in Fourier Domain4.4 FMI-SPOMF Image Matching4.5 Some final words

89 / 139


NOISE REMOVAL

39

Application of FFT in Image ProcessingNoise Removal

Source: www.mediacy.com/apps/fft.htm, Image Pro Plus FFT Example. Last seen online in 2004.

FFT InverseFFT

Edit FFT

Four NoiseSpikes Removed

Noise PatternStands Out as Four Spikes


90 / 139


TEXTURE RECOGNITION

40

Application of FFTPattern/Texture Recognition

Source: Lee and Chen, A New Method for Coarse Classification of Textures and Class Weight Estimation for Texture Retrieval, Pattern Recognition and Image Analysis, Vol. 12, No. 4, 2002, pp. 400–410.


91 / 139


TEXTURE RECOGNITION

41

Application of FFT

Source: http://www.rpgroup.caltech.edu/courses/PBL/size.htm

Could FFT of Drosophila eye be used to identify/quantify subtle phenotypes?

The Drosophila eye is a great example a cellular crystal withits hexagonally closed-packed structure. The absolute value of the Fourier transform (right) shows its hexagonal structure.

Pattern/Texture Recognition


The Drosophila eye is a great example a cellular crystal with itshexagonally closed-packed structure. The absolute value of theFourier transform (right) shows its hexagonal structure.

92 / 139


DEBLURRING - DECONVOLUTION

45

The Point Spread Function (PSF) is the Fourier transform of a filter.(the PSP says how much blurring there will be in trying to image a point).

Source: http://www.reindeergraphics.com/index.php?option=com_content&task=view&id=179&Itemid=127

Hubble image and measured PSFDividing the Fourier transform of the PSF into the transform of the blurred image, and performing an inverse FFT, recovers the unblurred image.

Application of FFTDeblurring: Deconvolution

FFT(Unblurred Image) * FFT(Point Spread Function) = FFT(Blurred Image)

Unblurred Image = FFT-1[ FFT(Blurred Image) / FFT(Point Spread Function) ](Source : Earl F. Glynn - Research Notes)

93 / 139


DEBLURRING - DECONVOLUTION

46

The Point Spread Function (PSF) is the Fourier transform of a filter.(the PSP says how much blurring there will be in trying to image a point).

Source: http://www.reindeergraphics.com/index.php?option=com_content&task=view&id=179&Itemid=127

Hubble image and measured PSF

Deblurred image

Dividing the Fourier transform of the PSF into the transform of the blurred image, and performing an inverse FFT, recovers the unblurred image.

Application of FFTDeblurring: Deconvolution


94 / 139

4. FOURIER ANALYSIS APPLICATIONS 4.2. FOURIER SHAPE DESCRIPTORS

CONTENTS

1. INTRODUCTION




95 / 139


OVERVIEWFourier Descriptors: OverviewFourier Descriptors: Overview

...

...

Concise and description of (object) contours Contours are represented by vectors

Numerous application Contour Processing (filtering, interpolation, morphing) Image analysis: Characterising and recognising the shapes of object

(Source : Computer Vision & Remote Sensing - Univ. Berlin)

I Description of (object) contours represented as vectors.I Applications :

I Contour Processing (filtering, interpolation, morphing)I Image analysis : characterizing and recognizing the shapes

of objectI Shape = closed contour !

96 / 139


REPRESENTING A CONTOUR WITH DFTRepresenting a Contour using the DFTRepresenting a Contour using the DFT

...

...

(xN,y

N): Coordinates of

the Nth point along thecircumference

Pixels on the contour are assumed to be ordered (e.g.

clockwise)!

Define a complex vector using coordinates (x,y).

1st Step

2nd Step

Apply the 1D DFT


97 / 139


EXAMPLEApplicaApplication: Recognising and classifying tion: Recognising and classifying

leavesleavesDatabase

Two types of leaves are to be recognised and classified


98 / 139



leavesleaves

Image with unclassified objects(Source : Computer Vision & Remote Sensing - Univ. Berlin)

99 / 139


EXAMPLE

ApplicaApplication: Recognising and classifying tion: Recognising and classifying leavesleaves

Segmented Objects(Thresholding)

(Source : Computer Vision & Remote Sensing - Univ. Berlin)100 / 139



leavesleaves

Leaves detected and classified(Source : Computer Vision & Remote Sensing - Univ. Berlin)

101 / 139


TRANSLATIONTranslationTranslation

t

t

Translating U by t:


I Only on F[0].102 / 139


SCALINGChanges in ScaleChanges in Scale

Magnification by factor s:


103 / 139


ROTATIONRotationRotation

Rotation by an angle θ:

(Derivation identical to scale change: Multiplication by constant)(Source : Computer Vision & Remote Sensing - Univ. Berlin)

I in Phase104 / 139

4. FOURIER ANALYSIS APPLICATIONS 4.3. FILTER BANKS IN FOURIER DOMAIN

CONTENTS

1. INTRODUCTION




105 / 139


LINEAR TRANSLATION INVARIANT SYSTEMSExamples of Fourier’s representation 17

fi

Input! System

A

fo

Output!

Figure 1.14. Schematic representation of a system A.

In practice we often deal with systems that are homogeneous and additive, i.e.,

A(cf) = c(Af)

A(f + g) = (Af) + (Ag)

when f, g are arbitrary inputs and c is an arbitrary scalar. Such systems are said tobe linear. Many common systems also have the property of translation invariance.We say that a system is translation invariant if the output

go = Agi

of an arbitrary ! -translate

gi(t) := fi(t + !), "# < t < #,

of an arbitrary input function fi is the corresponding ! -translate

go(t) = fo(t + !), "# < t < #

of the outputfo = Afi

to fi, i.e., when we translate fi by ! the system responds by shifting fo by ! ,"# < ! < #. Systems that are both linear and translation invariant are said tobe LTI.

A variety of signal processing devices can be modeled by using LTI systems. Forexample, the speaker for an audio system maps an electrical input signal from anamplifier to an acoustical output signal, with time being the independent variable.A well-designed speaker is more-or-less linear. If we simultaneously input signalsfrom two amplifiers, the speaker responds with the sum of the corresponding out-puts, and if we scale the input signal, e.g., by adjusting the volume control, theacoustical response is scaled in a corresponding manner (provided that we do notexceed the power limitations of the speaker!) Of course, when we play a familiar CDor tape on di!erent occasions, i.e., when we time shift the input signal, we expectto hear an acoustical response that is time shifted in exactly the same fashion (pro-vided that the time shift amounts to a few hours or days and not to a few millionyears!)

A major reason for the importance of Fourier analysis in electrical engineering isthat every complex exponential

es(t) := e2!ist, "# < t < #

I Linear System :

A(cf ) = c(Af ) and (4.1)A(f + g) = (Af ) + (Af ). (4.2)

I Shift (or translation) invariance :

gi(t) = fi(t + τ) (4.3)⇒ gi(t) = fo(t + τ). (4.4)

(the output is shifted by the same amount than the input).106 / 139


LINEAR TRANSLATION INVARIANT SYSTEMS

I An LTI system responds sinusoidally when it is shakensinusoidally

I The output can be obtainedI from the impulse response g[n]I by a convolution product :

fo[n] = (fi ∗ g)[n] =∞

∑k=−∞

f [k]g[n− k] (4.5)

I or in Fourier Domain

Fo[u] = Fi[u]G[u] (Convolution Theorem) (4.6)

107 / 139


CONVOLUTION IN FOURIER DOMAIN

42

Application of FFTFiltering in the Frequency Domain: Convolution

Source: Gonzalez and Woods, Digital Image Processing (2nd ed), 2002, p. 159

I[m,n]Raw Image

I’[m,n]Enhanced Image

Fourier TransformF[u,v]

Filter Function H[u,v]

InverseFourier Transform

Pre-processing

Post-processing

F[u,v] H[u,v] · F[u,v]

FFT I[u,v] FFT-1 H[u,v] · F[u,v]


108 / 139


FILTER BANKS

I A filter bank is an array of band-pass filters that spans theentire frequency spectrum.

I The bank serves to isolate different frequency componentsin a signal

(Source : http://www.aamusings.com/project-documentation/wavs/filterBank.html)

109 / 139

http://www.aamusings.com/project-documentation/wavs/filterBank.html


FILTER BANKS

(Source : http://www.aamusings.com/project-documentation/wavs/filterBank.html)

Frequent scheme :

I DCT : Discrete Cosine Transform (a special case of DFT),I DWT : Discret Wavelet Transform.

110 / 139

http://www.aamusings.com/project-documentation/wavs/filterBank.html


FILTER BANKS - FWT

A common implementation of the Discrete Wavelet Transformis the Mallat Algorithm (the Fast WT) :

I h is a low-pass filterI g is a high-pass filter

111 / 139


FWT - B-SPLINE WAVELETS

ImpulseResponses

FrequentialResponses

Obtained from the autoconvolution of the box function :

112 / 139


B-SPLINE WAVELETS - EXAMPLE

113 / 139


FILTER BANK IN FOURIER DOMAINThe high-pass branch :

I g is a linear filter (ok in Fourier Domain)I the sub-sampling can also be obtained in Fourier Domain :

(u : zeroing the even samples)

114 / 139


FILTER BANK IN FOURIER DOMAIN

115 / 139


FWT IN FOURIER DOMAIN

Figure 5.7 – Comparaison de la complexité d’algorithmes d’analyse multirésolution : rapportentre le nombre d’opérations nécessaires dans le cas classique par rapport aux calculs intégra-lement réalisés dans Fourier. L est la profondeur de l’analyse multirésolution. Ces rapportssont calculés en fonction de la taille M des images. A gauche : cas séparable ; et à droite : casquinconce.

Nous avons ainsi proposé un algorithme de calcul des coecients d’une analyse multiréso-lution où toutes les opérations (filtrage et échantillonnages) sont eectuées dans le domainede Fourier discret. Cet algorithme, présenté en figure 5.6 nécessite une FFT en entrée et uneFFT par signal de détails (coecients de l’analyse multirésolution).

Figure 1.7 – Comparaison de la complexite d’algorithmes d’analyse multiresolution. Rapport entre lenombre d’operations necessaires dans le cas classique par rapport aux calculs integralement realises dansFourier. L est la profondeur de l’analyse multiresolution. Ces rapports sont calcules en fonction de la tailleM des images. A gauche : cas separable ; et a droite : cas quinconce.

FFTAs

filtrage parmultiplications

complexes

sous-echantillonnage

Ds+1

filtrage oarmultiplications

complexes

sous-echantillonnage

Ds+2

...

FFT1 FFT1

Figure 1.6 – algorithme de calcul des coecients d’une analyse multiresolution dans le domaine deFourier discret.

Par rapport a une implementation classique, ou seul les filtrages seraient eectues dans le domainede Fourier, notre algorithme necessite un nombre plus faible d’operations. Les graphiques de la figure1.7 indiquent que pour une profondeur d’analyse sur quatre echelles (un cas frequent), notre algorithmenecessite 1,8 fois moins d’operations. De plus, nous avons montre que la complexite de cet algorithme estO(N log N) alors que la complexite de l’algorithme issu de la factorisation polyphase est O(N2).

1.3.3 Citations

La methode propose peut permettre un usage plus frequent de l’analyse multiresolution. Par exempledans [?] des defauts dans un systeme de production de papeterie sont caracterises a l’aide de descripteursde forme tiree d’une AMR. Par rapport des descripteurs de Fourier plus classique, les auteurs indiquentque le cout de calcul est plus important. Mais en conclusion, ils mentionnent egalement que ce cout peutetre amoindri en utilisant l’implementation de l’AMR dans Fourier que nous avons propose.

Dans un autre article, une decomposition en ondelettes 2D multi-bandes est proposee [?]. Les auteursindiquent que si les implementations dans le domaine temporel sont populaires, une implementation telleque celle que nous avons propose peut-etre ecace. Un telle implementation peut-etre meme preferablesi les ondelettes sont de supports infinis, ce qui est le cas de leur decomposition.

D’autres auteurs ont repris et ameliore, voir simplifie dans certains cas particuliers, notre methode.

11

Figure 5.6 – Algorithme de calcul des coecients d’une analyse multirésolution dans le do-maine de Fourier discret.

Par rapport à une implémentation classique, où seul les filtrages seraient eectués dans ledomaine de Fourier, notre algorithme nécessite un nombre plus faible d’opérations. Les gra-phiques de la figure 5.7 indiquent que pour une profondeur d’analyse sur quatre échelles (uncas fréquent), notre algorithme nécessite 1,8 fois moins d’opérations. De plus, nous avons mon-tré que la complexité de cet algorithme est O(N log N) alors que la complexité de l’algorithmeissu de la factorisation polyphase est O(N2) [Kovacevic et Vetterli, 1993].

51

116 / 139


2D-FWT IN FOURIER DOMAIN

Even for images :

Figure 5.5 – Sur-échantillonnage d’une image : une rotation et une dilatation sont impliquéedans la transformation.

5.3.2 Notre contribution

Ces deux équations (5.34 et 5.36) permettent d’exprimer les échantillonnages dans le do-maine de Fourier. Cependant les signaux manipulés sont continus ce qui implique que lors del’implémentation, une discrétisation du domaine de Fourier sera nécessaire. Ce passage peutêtre évité en écrivant les équations directement dans le domaine de Fourier discret. Les indicesdes signaux doivent donc être des entiers. Une image X[k] de dimension NN est transforméeen une image Y [k] de dimension M M (figure 5.5).

L’équation (5.34) est donc réécrite de la façon suivante :

Y () = X() avec

= 2

M m

= 2M m , (5.37)

où m = NM J tm. L’équation peut donc être transformée pour exprimer le sur-échantillonnage

de signaux multi-dimensionnels dans le domaine de Fourier discret :

Y [m] = X

N

MJ tm

modN

. (5.38)

L’opérateur modulo (mod) est nécessaire pour tenir compte de la périodicité de transfor-mées de Fourier discrètes rapides (FFT) des signaux. Selon une méthode analogue, le sous-échantillonnage est :

Y [m] =1

|detJ |

|detJ |1

l=0

X

N

MJtm NJtvl

modN

. (5.39)

Dans certains cas particuliers les équations se simplifient fortement, permettant fréquem-ment d’exprimer les échantillonnages comme des duplications (sur-échantillonnage) ou dessommes de sous-parties des images (sous-échantillonnage). Par exemple dans le cas séparable,les sous-échantillonnages bi-dimensionnels s’expriment comme un enchaînement de deux sous-échantillonnages mono-dimensionnels. Ou encore, dans le cas quinconce le sous-échantillonnagepeut s’écrire comme la somme de quatre sous-images.

50

Figure 5.5 – Sur-échantillonnage d’une image : une rotation et une dilatation sont impliquéedans la transformation.

5.3.2 Notre contribution

Ces deux équations (5.34 et 5.36) permettent d’exprimer les échantillonnages dans le do-maine de Fourier. Cependant les signaux manipulés sont continus ce qui implique que lors del’implémentation, une discrétisation du domaine de Fourier sera nécessaire. Ce passage peutêtre évité en écrivant les équations directement dans le domaine de Fourier discret. Les indicesdes signaux doivent donc être des entiers. Une image X[k] de dimension NN est transforméeen une image Y [k] de dimension M M (figure 5.5).

L’équation (5.34) est donc réécrite de la façon suivante :

Y () = X() avec

= 2

M m

= 2M m , (5.37)

où m = NM J tm. L’équation peut donc être transformée pour exprimer le sur-échantillonnage

de signaux multi-dimensionnels dans le domaine de Fourier discret :

Y [m] = X

N

MJ tm

modN

. (5.38)

L’opérateur modulo (mod) est nécessaire pour tenir compte de la périodicité de transfor-mées de Fourier discrètes rapides (FFT) des signaux. Selon une méthode analogue, le sous-échantillonnage est :

Y [m] =1

|detJ |

|detJ |1

l=0

X

N

MJtm NJtvl

modN

. (5.39)

Dans certains cas particuliers les équations se simplifient fortement, permettant fréquem-ment d’exprimer les échantillonnages comme des duplications (sur-échantillonnage) ou dessommes de sous-parties des images (sous-échantillonnage). Par exemple dans le cas séparable,les sous-échantillonnages bi-dimensionnels s’expriment comme un enchaînement de deux sous-échantillonnages mono-dimensionnels. Ou encore, dans le cas quinconce le sous-échantillonnagepeut s’écrire comme la somme de quatre sous-images.

50

IMore details in [NIC02] 3.

3. F. Nicolier, O. Laligant, F. Truchetet. Discrete wavelet transform imple-mentation in fourier domain. Journal of Electronic Imaging, 11(3) :338–346, jul2002

117 / 139

4. FOURIER ANALYSIS APPLICATIONS 4.4. FMI-SPOMF IMAGE MATCHING

CONTENTS

1. INTRODUCTION




118 / 139


FMI-SPOMF ?

I SPOMF = Symmetric Phase Only Matched FiltersI FMI = Fourier-Mellin Invariant

I FT for matching imagesI Translation, rotation, scaling invariant registeringI Described in [Chen94] 4

4. Q. Chen, M. Defrise and F. Deconinck, "Symmetric phase-only matchedfiltering of Fourier-Mellin transforms for image registration and recognition",IEEE pattern analysis and machine intelligence, vol. 16, 1994, p. 1156-1168.

119 / 139


IMAGE MATCHING

IWhere is the small bear ?

120 / 139


IMAGE MATCHINGFrom the image and a pattern (reference) :

The idea is to

I slide the pattern on the image,I compute sum of the product pixel-to-pixel.I This is a cross-correlation.

121 / 139


CC AND SSD RELATION

I A classical template matching solution is the Sum of SquaredDifferences (SSD) - here with continuous functions :

SSDW =∫

W(f (t)− g(t))2 dt (4.7)

I SSD is related to CC :∫

W(f (t)− g(t))2 dt =

∫

W

(f (t)2 + g(t)2 − 2f (t)g(t)

)dt (4.8)

=∫

Wf (t)2dt +

∫

Wg(t)2dt− 2

∫

Wf (t)g(t)dt.

(4.9)

I If f is the pattern, the first term is constant !

122 / 139


CC AND SSD RELATION

∫

W(f (t)− g(t))2 dt = Cte +

∫

Wg(t)2 − 2

∫

Wf (t)g(t)dt. (4.10)

I If the local energy of the image (g) is constant :∫

W(f (t)− g(t))2 = Cte− 2

∫

Wf (t)g(t)dt. (4.11)

I In this case, the SSD is the same as the CCI But the local energy of the image must be constant !

123 / 139


IMAGE MATCHING - CROSS CORRELATIONI The Cross-Correlation (CC) is defined as (for real signals) :

(f ? g)[n] =∞

∑k=−∞

f [k]g[n + k] (4.12)

=∞

∑k=−∞

f [n− k]g[n] (4.13)

I CC is strongly related to convolution :

(f ? g)[n] = f [−n] ∗ g[n] (4.14)

I CC can also be easily expressed in Fourier Domain (fastcomputations) :

FT[(f ? g)][u] = F∗[u]G[u] (4.15)

so (f ? g) = FT−1[FT∗[f ]FT[g]]. (4.16)

124 / 139


CROSS-CORRELATION IN FOURIER DOMAINI Remember that the DFT implies the function is periodicI The product F∗[u, v]G[u, v] implies F and G have the same

size

The pattern P is thus modified :

I its size must be the same as I,I the origin of the image must corresponds to the center of

the pattern.

125 / 139


CROSS-CORRELATION - RESULT

I Local max at the center (good) but not the absolute oneI The second bear is not detected (rotation)I CC is very sensitive to luminance

126 / 139


NORMALIZED CROSS-CORRELATION

I One solution (to luminance sensitivity) is to normalize theimages before the comparison :

NCCI,P =1N ∑

x,y

(I[x, y]− I)(P[x, y]− P)σIσP

(4.17)

I is the average of I, σI is the standard deviation of I

I Very classic solution to template matchingI same as Pearson Correlation Coefficient

127 / 139


SPOMF

I Another solution is to only compare the Phase information(structures !)

I The dectector is thus modified :

DI,P = FT−1[FT[I]‖FT[I]‖ ·

FT[P]‖FT[P]‖ ] (4.18)

I SPOMF : Symmetric Phase Only Matched Filters

128 / 139


SPOMF - RESULT

CC SPOMF

129 / 139


FMI-SPOMF IMAGE REGISTRATION

I CC based comparison (NCC and SPOMF) are rotation andscaling sensitive

I Fourier-Mellin Transform is a solutionI The key point is to

I reduce rotation and scaling to translationsI and reduce the dimension of the parameter size.

I Polar coordinates : Rotation→ TranslationI Logarithmic scale : Scaling→ Translation

(log(αx) = log(α) + log(x))

130 / 139


FMI-SPOMF - PRINCIPLE

I 4 parameters : Translation (x and y), Rotation, ScalingI Facts :

I Phase contains localisationI Magnitude is insensitive to translationI A rotation of the image rotates the spectral magnitude by

the same angleI A scaling by σ scales the spectral magnitude by σ−1

I Keeping magnitude only allows to isolate rotation andscaling

I Rotation and scaling are transformed into translations ...detected by SPOMF.

I Fourier-Mellin Transform = FT of polar-log magnitudespectral image.

131 / 139


FMI - SPOMF - ALGORITHM

Given I and P :

I Compute the log-polar magnitude spectral images of I andP

I Detect the max of the SPOMF image between I and PI Identify σ and θ

I Re-scale and re-rotate P by (σ−1, θ)

I Compute the SPOMF between I and the rectified PI Locate the max, and identify (x, y) the translation vector.

132 / 139


LOG-POLAR REPRESENTATION

I Assuming I is a N×N image, its log-polar representationIlp is :

Ip(ρ, θ) = I(ρ cos θ +N2

, ρ sin θ +N2

(4.19)

Ilp(m, k) = Ip(12

NmN ,

2πkN− π). (4.20)

I m ∈ [1, N] is the discrete radial coordinateI k ∈ [1, N] is the discrete angular coordinateI (an interpolation is needed : nearest-neighbor for example)

133 / 139



More general equations are given in [Chen94] 5 :

I A N×N image I can be resampled onto a M× K polar-loggrid in one step :

um,k =

N/2−1M−1 (M− 1)

mM−1 cos(πk

K ) + N2

vm,k =N/2−1M−1 (M− 1)

mM−1 sin(πk

K ) + N2

, (4.21)

m ∈ [0, M− 1], k ∈ [0, K− 1].

5. Q. Chen, M. Defrise and F. Deconinck, "Symmetric phase-only matchedfiltering of Fourier-Mellin transforms for image registration and recognition",IEEE pattern analysis and machine intelligence, vol. 16, 1994, p. 1156-1168.

134 / 139



I A detection of the max provides two integers m and k.I Rotation and scaling :

θ =m−N/2

M− 1 (4.22)

σ = (M− 1)k

M−1 for 0 ≤ k < M/2 (enlargement) (4.23)

σ¯1 = (M− 1)M−kM−1 for M/2 < k < M (shrinkage) (4.24)

135 / 139


SOME EXAMPLES

original distorted

(σ = 0.7, θ = 30 deg)

136 / 139


SOME EXAMPLES

137 / 139

4. FOURIER ANALYSIS APPLICATIONS 4.5. SOME FINAL WORDS

CONTENTS

1. INTRODUCTION




138 / 139

4. FOURIER ANALYSIS APPLICATIONS 4.5. SOME FINAL WORDS

SOME FINAL WORDS

I Fourier Transform still very popular in scienceI Phase congruency - Local Phase. See Peter Kovesi

webpage 6

I Practice with :I Playing with phase and magntiudeI Image ReconstructionI Sampling in Fourier domainI Template matching with phase spectrumI Image registration with FMI-SPOMF

6. http://www.csse.uwa.edu.au/~pk/research139 / 139

http://www.csse.uwa.edu.au/~pk/research

practising fourier analysis with digital images

Education

fourier series

fourier domain

outlineoutlinei fourier

fourier analysisapplications2

fourier analysisapplications5

fourier analysisapplications7

fourier analysisapplications11

discrete fourier transform2