wavelets & wavelet algorithms: 1d discrete fourier transform & inverse discrete fourier...

102
Wavelets & Wavelet Algorithms Vladimir Kulyukin www.vkedco.blogspot.com www.vkedco.blogspot.com 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Upload: vladimir-kulyukin

Post on 29-Jul-2015

112 views

Category:

Science


6 download

TRANSCRIPT

Page 1: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Wavelets & Wavelet Algorithms

Vladimir Kulyukin

www.vkedco.blogspot.comwww.vkedco.blogspot.com

1D Discrete Fourier Transform&

Inverse Discrete Fourier Transform

Page 2: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Outline

● Review● Complex Numbers & Euler's Identity● Nth Roots of Unity● Roots of Unity & Complex Sinusoids● Discrete Fourier Transform (DFT) & Inverse DFT● DFT & Inverse DFT as Matrix Multiplication

Page 3: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Review

Page 4: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Sinusoids

(radians). phase ousinstantane theis

(radians); phase initial theis

seconds; 0.01 1/100 is sampleeach ofduration thethus

cond,samples/se 100 means 100Hz e.g. (Hz);frequency theis

(sec); timeis

(rad/sec);frequency radian theis

amplitude;peak negative-non theis

constants. are ,, variable;realt independenan is

.sin:form theoffunction a is sinusoidA

t

f

t

A

At

tAtx

Page 5: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Harmonic Function Forms

b

abaA

tAtxtbtatx

122 tan,

where,sinsincos

Page 6: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Definition of Function Orthogonality

b

a

dxxgxfxgxf 0 if orthogonal are , Functions

Page 7: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Orthogonality of Basic Trigonometric System

.2, interval over the

0 is system tric trigonomebasic theof functionsdifferent

any two of integral that theshow formulasn integratio The

,...sin,cos,...,2sin,2cos,sin,cos,1

functions ofset infinite theis system tric trigonomebasic The

aa

nxnxxxxx

Page 8: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Fourier Coefficients

,...3,2,1,sin

1ndxnxxfbn

,...3,2,1,cos

1ndxnxxfan

Page 9: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Fourier Series

. of seriesFourier the

called is sincos2

series tric trigonomeThe

,...3,2,1,sin1

and cos1

where,sincos2

:expansion series

tric trigonomefollowing thehas and 2 period offunction a is If

1

0

1

0

xf

kxbkxaa

nnxxfbnxxfa

kxbkxaa

xf

xf

kkk

nn

kkk

Page 10: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

1D Fourier Analysis Algorithm

Page 11: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

1D Fourier Analysis

● Given a 1D data array, determine the frequency range [W

lower, W

upper]

● Compute the cosine and sine coefficients for each frequency value in the frequency range

● Once the sine and cosine coefficients are computed, determine the amplitude and phase for every constituent harmonic

● Optional: If the signal function is known, recombine the reconstructed harmonics and compute how closely their sum approximates the signal function

Page 12: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

1D Fourier Analysis: Harmonic Recovery

.sin

harmonicth - therepresents ,, tuple-3 The

}

;under indexed

isit that socontainer map ain ,, Save

;tan

;sin1

;cos1

{in every For

., e.g., range,frequency a be Let

.:001.0: e.g., interval, timea be Let

function. signal some from valuesofarray 1D a be Let

22

1

kkkk

kkk

k

kkk

k

kk

kkkk

k

upperlower

atabaa

kaabaa

aabaa

ab

aaa

tdataabtdataaa

W

WWWW

tt

data

Page 13: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

1D Fourier Analysis: Harmonic Recovery

0l

1l

2l

u

002

02

00000 sinsincos llllllll batbta

112

12

11111 sinsincos llllllll batbta

222

22

22222 sinsincos llllllll batbta

uuuuuuuu batbta sinsincos 22

upperlower WW ,

Frequency Reconstructed Harmonics

Page 14: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Example

0.001. of incrementsin dconstructe ,on of analysisFourier 1D Do

.57.30,98.7,73.1,32.12,5.0,5.4,42

where

,20sin573030cos987

20sin73120cos3212

10sin5010cos544

Let

3322110

tf

bababaa

t.t.

t.t.

t.t.π

tf

Page 15: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Solution Steps

● Generate 1D data (this step is unnecessary if the data array is given)

● Approximate sine & cosine coefficients● Reconstruct 10th, 20th, and 30th harmonics● Combine reconstructed harmonics to reconstruct

the original sinusoid● Compute approximate error b/w reconstructed &

original sinusoids

Page 16: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Plotting Computed Cosine Coeffs

>> cosine_coeff_map(0)=1.5886

>> cosine_coeff_map(10)=4.5178

>> cosine_coeff_map(20)=12.3378

>> cosine_coeff_map(30) = 38.0054

>> cosine_coeff_map(40)=5.9178

>> cosine_coeff_map(50)=7.3578

Page 17: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Plotting Computed Sine Coeffs

>> sine_coeff_map(10)=0.5000

>> sine_coeff_map(20)=1.7300

>> sine_coeff_map(30)=2.4350

>> sine_coeff_map(40)=10.7799

>> sine_coeff_map(50)=3.4779

Page 18: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Complex Numbers &

Euler's Identity

Page 19: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Square Roots of Negative Numbers

.551515 :Example

.11

,0number negativeany for Then .1Let

j

cjccc

cj

Page 20: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Exponents of J

,...3,2,1,0for ,

...

;

;1

;

;111

;

;111

;1

;1

.1 of powers some through cycle usLet .1Let

44

67

56

45

224

23

22/122

1

0

njjjj

jjjj

jjjjj

jjjj

jjj

jjjj

j

j

j

jj

nnn

Page 21: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Complex Numbers: Imaginary & Real Parts

.0 becausenumber complex a is number realEvery

.; Formally,

. ofpart imaginary theis and ofpart real theis and reals are

, where, as written becan number complex Every

jrrr

yzimxzre

zyzx

yxjyxzz

Page 22: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Complex Plane

sin

;cos

;,tan

;,

;

1

22

ry

rx

xy

yxr

Page 23: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Derivative of f(x) = ax & Number e

function. continuous a is .0,1

Consider

.1

lim1

limlimlim

have welimit, theof definitionBy real. is and real positive a is where,Let

000

00

00

' 00

00

aga

ag

aa

aa

aaxfxfxf

xaaxf

xxxx

x

Page 24: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Derivative of f(x) = ax & Number e

.11

lims,other wordIn .1limsuch that

32number someexist must there,continuous is Since

.113

lim3lim )2

and 112

lim2lim )1

thatlly verifycomputionacan We

00

00

00

eeg

eag

g

g

Page 25: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

What is e?

.1 lim

Therefore, .1

1 1 1

,0 asThen

.11

lim shown thatjust have We

/1

0

/1

/1/1

0

e

e

eee

e

Page 26: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

What Is e Equal To?

smaller. andsmaller makingby

2after digits decimal ofnumber any computecan We

....02874759045235367182818284.21 lim /1

0

e

Page 27: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Derivative of ex

. s,other wordIn

.11

lim because ,1

lim1

lim

limlim

any for Then .let and Let

000

0

00

00

'

0

000

00

xx

xxx

xx

x

eedx

d

ee

ee

ee

eexfxfxf

xeaaxf

Page 28: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Euler's Identity

.01

.1sincos then , If

.sincos:Identity sEuler'

.sincosThen .Let

j

j

j

e

je

je

jrrzjyxz

Page 29: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Rectangular & Polar Form of Complex Numbers

form.polar theis

form.r rectangula theis

.sincossincosThen

.Let

.sincos:identity sEuler'

j

j

j

rez

jyxz

rejrjrrz

jyxz

je

Page 30: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Complex Number Multiplication

.

Then . and Let 212121

21

21212121

2211

jjjjj

jj

erreerrererzz

erzerz

Page 31: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Euler's Formula & Conjugate Multiplication

.

Then . and

. and Let

22 rerrerezz

rezrez

jyxzjyxz

jjjj

jj

Page 32: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Expressing Conjugates with Euler's Identity

.sincos

sincossincos

Then .Let

.sinsin i.e.,odd, is sin

while,coscos i.e.,even, is cos that Recall

j

j

j

rejrr

jrrjrrrez

rez

x

x

Page 33: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Sine & Cosine with Euler's Formula

.2

sin

So, .sincos

sincossincossin2

.22

cos

So, .sincossincoscos2

j

ee

eeje

jjj

eeee

eejj

jj

jjj

jjjj

jj

Page 34: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Complex Sinusoid

.sincos then ,1 and 0 If

.sincos

as defined is sinusoidcomplex then offset, the

is and seconds,in timeis (rad/sec),frequency

radian theis where, and 0Let

.sincos :identity sEuler' Recall

tjtetsA

tjAtAAets

t

tA

je

tj

tj

j

Page 35: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Real & Imaginary Parts of Complex Sinusoid

. sinusoidcomplex theofpart imaginary theis sin

; sinusoidcomplex theofpart real theis cos

.sincosLet

tsteim

tstere

tjtets

tj

tj

tj

Page 36: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Circular Motion in 3D

Let us assume that a point is moving in circles in 3D, as shown on the right. If we agree that time can be measured along the three axes (x, y, & z), then we can project the point's circular motion along the three axes (lines): x, y, and z vs. time

Page 37: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Projection of 3D Circular Motion on X-Axis

The X-axis projection is obtained from an observer who looks at the trace of the 3D circular motion from the x-y plane.

.cos i.e., sinusoid,complex the

ofpart real theis axis-X on the projection The

tere tj

Page 38: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Projection of 3D Circular Motion on Y-Axis

The Y Axis projection is obtained from an observer who looks at the trace of the 3D circular motion from the y-z plane.

.sin i.e.,

sinusoid,complex theofpart imaginary

theis axis-Y on the projection The

teim tj

Page 39: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Projection of 3D Circular Motion on Z

The Z Axis projection is obtained from the observer who looks at the trace of the 3D circular motion from the x-z plane.

.by defined is circle This tje

Page 40: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Circular Motion

increases. as

planecomplex in the circleunit thealongmotion

circular clockwise a of trace thedefines

increases. as plane

complex in the circleunit thealongmotion circular

clockwise-counter a of trace thedefines

plane.complex in the circle aon liesit

constant, is sinusoidcomplex a of modulus theSince

t

ets

t

ets

tj

tj

Page 41: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Positive & Negative Frequency Sinusoids

sinusoidfrequency -negative a is

sinusoidfrequency -positive a is

tj

tj

ets

ets

Page 42: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Positive & Negative Frequencies

s.frequencie negative and

positive of amounts equal contains signal realEvery 2)

.componentsfrequency negative and positive of

on contributi equalan of consists sinusoid realEvery 1)

:nsobservatio Two

2sin

2cos

thatshow oidentity t sEuler' used We

j

ee

ee

jj

jj

Page 43: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Modulus of Complex Sinusoid

.1sincos

sincos

:follows as is

sinusoidcomplex a of modulus the,1cossin Since

2222

222222

22

AAttA

tAtAtsimtsrets

ts

Page 44: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Nth Roots of Unity

Page 45: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Finding Nth Roots

.1210for results are There

.,

isroot th The .

Therefore, .for ,12sin2cos

., and 0 where, identity, sEuler'By

212

112

1

2

2

,...,N-,,kN

Zkereererez

Nerez

Zkkjke

rrez

N

kj

NN

kj

Nj

NNkjjN

kjj

kj

j

Page 46: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Roots of Unity

.1,...,3,2,1,0, :it from

rootsother allobtain can webecause unity, ofroot th primitive a

called is 1for case special The unity. of rootsth theare These

.1,...,3,2,1,0,1

Thus, .integer every for ,1

/2/2

/2/

2

Nkee

N

kN

Nke

ke

NkjkNj

NkjNk

kj

Page 47: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Commonly Used Formulas: Cheat Sheet

.

.1

.2122

./2/2

nTtT

f

NT

k

TNkf

Nk

eee

n

s

sk

tjnTNfkjNnkj nks

Page 48: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

A Different Notation for Roots of Unity

.1

.1,...,3,2,1,0,

:used isnotation following the,processing signalIn

2/2

/2/2

kjNNkjNTjNkN

NkjTNfjkTjkN

eeeW

NkeeeW

k

sk

Page 49: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Computing Nth Roots of Unity: Example

.,,

,,,,, So,

parts. equal 8 into circleunit thedividecan wey,Graphicall

.,,,,,,, compute usLet 8.Let

765

43210

78

68

58

48

38

28

18

08

78

68

58

48

38

28

18

08

TjTjTj

TjTjTjTjTj

eWeWeW

eWeWeWeWeW

WWWWWWWWN

Page 50: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Computing Nth Roots of Unity

.4

7

8

27,

2

3

8

26,

4

5

8

25,

18

24

,4

3

8

23,

28

22,

48

21,0

8

20

case,our In .22

that Recall 1sec. set We.,,

,,,,, So,

7654

3210

78

68

58

48

38

28

18

08

765

43210

NTk

N

fk

TeWeWeW

eWeWeWeWeW

sk

TjTjTj

TjTjTjTjTj

Page 51: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Computing Nth Roots of Unity

.2

2

2

2

4sin

4cos

.010sin0cos

418

008

1

0

jjeeW

jjeeW

jj

j

08W

18W

Re

Im

Page 52: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Computing Nth Roots of Unity

.102

sin2

cos228

2 jjeeWjj

08W

18W

28W

Re

Im

Page 53: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Computing Nth Roots of Unity

.2

2

2

2

4

3sin

4

3cos4

33

83

j

jeeWjj

08W

18W

28W

38W

Re

Im

Page 54: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Computing Nth Roots of Unity

.001sincos

sincos

.01sincos4

4

48

48

jj

jeeW

jjeeWjj

jj

08W

18W

28W

38W

48W

Re

Im

Page 55: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Computing Nth Roots of Unity

.2

2

2

2

4

3sin

4

3cos

4

3sin

4

3cos

.2

2

2

2

4

5sin

4

5cos

4

38

63

8

4

55

8

3

5

jj

j

eeeW

j

jeeW

jjj

jj

08W

18W

28W

38W

48W

38

58

WW

Re

Im

Page 56: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Computing Nth Roots of Unity

.10

2sin

2cos

.10

2

3sin

2

3cos

228

4

56

8

2

6

j

jeeW

j

jeeW

jj

jj

08W

18W

28W

38W

48W

38

58

WW

28

68

WW

Re

Im

Page 57: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Computing Nth Roots of Unity

.2

2

2

2

4sin

4cos

.2

2

2

2

4

7sin

4

7cos

418

4

77

8

1

6

j

jeeW

j

jeeW

jj

jj

08W

18W

28W

38W

48W

38

58

WW

28

68

WW

18

78

WW

Re

Im

Page 58: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Roots of Unity &

Complex Sinusoids

Page 59: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Roots of Unity & Sinusoids

unity. ofroot th -

th - theis where,1,...,2,1,0

,by defined sinusoids

complex generateunity of rootsth - The/2

N

kWNn

eeW

NN

kN

nTjNknjnkN

k

08W

18W

28W

38W

48W

58W

68W

78W

Re

Im

Page 60: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Sinusoid at k=0

7,...,1,0,0sin0cos

.010sin0cos

0

0

08

008

nnjneW

jjeeW

njn

j

08W

Re

Im

Real Sinusoid at k = 0

Imaginary Sinusoid at k = 0

Page 61: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Sinusoid at k=1: Real Sinusoid

08W

18W

Re

Im

Real Sinusoid at k = 1

.7,...,1,0,4

sin4

cos

.2

2

2

2

4sin

4cos

418

418

1

1

nnjneeW

jjeeW

njnjn

jj

7,...,0,4

cos

nn

Page 62: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Sinusoid at k=1: Imaginary Sinusoid

08W

18W

Re

Im

Imaginary Sinusoid at k = 1

.7,...,1,0,4

sin4

cos

.2

2

2

2

4sin

4cos

418

418

1

1

nnjneeW

jjeeW

njnjn

jj

7,...,0,4

sin

nn

Page 63: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Sinusoid at k=1: Real & Imaginary Sinusoids

Imaginary Sinusoid at k = 1

.7,...,1,0,4

sin4

cos :Sinusoid 1

.2

2

2

2

4sin

4cos :Root 1

418

st

418

1

1

nnjneeW

jjeeW

njnjn

jjst

7,...,0,4

sin

nn

Real Sinusoid at k = 1 7,...,0,4

cos

nn

t=0:.1:8;y = cos(pi/4*t);plot(t, y);

t=0:.1:8;y = sin(pi/4*t);plot(t, y);

Page 64: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Sinusoid at k=3: Real & Imaginary Sinusoids

7,...,0,4

3sin

4

3cos :Sinusoid 3

.2

2

2

2

4

3sin

4

3cos :Root 3

4

33

8rd

4

33

8rd

3

3

nnjneeW

jjeeW

njnjn

jj

Real Sinusoid at k = 3 7,...,0,4

3cos

nn

Imaginary Sinusoid at k = 3

t=0:.1:8;y = cos(3*pi/4*t);plot(t, y);

t=0:.1:8;y = sin(3*pi/4*t);plot(t, y);

7,...,0,4

3sin

nn

Page 65: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Complex Sinusoid Basis Set

Page 66: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Complex Sinusoid Basis Set

.1,...,2,1,0for

,,...,,,

,..., , ,

isset basis sinusoidcomplex Then the .1,2Let

/121

/222

/121

/020

1210

Nn

ensensensens

WWWW

iN

NnNjN

NnjNnjNnj

nNN

n

N

n

N

n

N

i

Page 67: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Inner Product of 2 Sinusoids

1

0

/2/21

0

/2

/2

.,

as defined is and ofproduct inner The

.1,0for ,

and Let

N

n

NmnjNknjN

nmkmk

mk

nmN

NmnjnTjm

nkN

NknjnTjk

eessss

nsns

NnWeens

Weens

m

k

Page 68: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Orthogonality of Complex Sinusoids

., Thus,

.01

1

1

1,

.1,...,0, , that Assume

.1,0, is sinusoidcomplex th - The

/2

2

/2

/21

0

21

0

1

0

/2/2

/2

mkss

e

e

e

eeeensnsss

Nmkmk

NnWeensk

mk

Nmkj

mkj

Nmkj

NNmkjN

n

N

nmkjN

n

N

n

NmnjNknjmkmk

nkN

NknjnTjk

k

Page 69: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Periodicity of Complex Sinusoids

.for ,

. divide that periods with periodic all are sinusoids These

.1,...,0,

as defined is sinusoidcomplex th - The

/2

ZmnsmNns

N

NnWeens

k

kk

nkN

NknjnTjk

k

Page 70: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Norm of Complex Sinusoids

. Thus, .,1

0

/2 NsNess k

N

n

Nnkkjkk

Page 71: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

An Orthonormal Sinusoidal Set

lk

lkss

N

e

N

nsns

lk

Nknjk

k

,0

,1~,~

:condition following esatisfy th sinusoids lorthonorma The

.~

set. lorthonormaan obtain tonormalized are sinusoidsComplex

/2

Page 72: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Signal Projection

.in present is signal much the how

i.e., ,projection oft coefficien thecomputesproduct inner The

.,product inner theas computed is onto of projection The

signals. twobe ,Let

yx

xyxy

xy

Page 73: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Discrete Fourier Transform

Page 74: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Human Ear & Sinusoids

● The human ear is a spectrum analyzer● The cochlea of the inner ear splits sound into its

sinusoidal components● A spectrum that displays the amount of each

sinusoid frequency present in sound is likely to be similar to the representation that the brain receives from the ear

● The human ear can be construed as a Fourier spectrum analyzer of sorts

Page 75: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

DFT Definition

1

0

21

0

2

1

0

.1,...,1,0,,

is definition equivalent The

.1

,2,, where

,1,...,1,0,,

:follows as defined is of (DFT) Transform

Fourier Discrete the, spans that sinusoidscomplex of basis

orthogonal thebe and signalcomplex a be Let

N

n

N

knjN

n

tfN

kj

kk

sskntj

k

N

nkkk

N

kN

NkenxenxsxX

Tff

N

knTtens

NknsnxsxX

x

C

sCx

ns

nk

Page 76: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Another Definition of DFT

1

0

/21

0

21

0

12

1

0

1

0

21

0

.DFT

,

:follows as computed is DFT the

Then signal.input thebe Let ./2 as computed

are sfrequencie that theAssume sinusoid.complex amplitude

unit phase,-zero a be 1,-..., 0,1,2,3,,Let

N

nk

NknjN

n

nN

kjN

n

nTNT

kj

N

n

N

n

nTN

kfjN

n

nTjkk

sk

k

nTjk

yenyenyeny

enyenynsnysy

yNkf

Nnens

s

k

k

Page 77: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Continuous Time Fourier Transform

.,

as defined is onto of projection The .

sinusoid timecontinuous on the projected is signal The

0

Ydtetysy

syets

y

tj

tj

Page 78: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Spectrum

. sinusoidcomplex

th - with the ofproduct inner theas defined is of

spectrum theof , sample,th - thes,other wordIn

.frequency at spectrum theof sampleth - theis

. of spectrum thecalled is ,...,,

, signalinput theand 1,...,1,0

,,...,,set basis sinusoidcomplex Given the

110

110

k

k

kk

N

N

s

kxx

Xk

kX

xXXXX

xNn

nsnsns

Page 79: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Computing Spectrum Elements

., If

. if ,1

1

.,1 where,1

1 series geometric theof because

,1

1

: computecan wehow is Here

. andarray signalinput thebe Let

1

0

1

0

1

0

1

0

NXe

eX

erar

raar

e

eeeensnxX

X

ensx

kkx

kxTj

NTj

k

TjN

k

kk

Tj

NTjN

n

nTjN

n

N

n

nTjnTjkk

k

nTjk

kx

kx

kx

kx

kx

kxkx

k

Page 80: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

1D Fourier Analysis: Harmonic Recovery

0l

1l

2l

u

002

02

00000 sinsincos llllllll batbta

112

12

11111 sinsincos llllllll batbta

222

22

22222 sinsincos llllllll batbta

uuuuuuuu batbta sinsincos 22

upperlower WW ,

Frequency Reconstructed Harmonics

Page 81: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

1D Discrete Fourier Transform: Spectral Analysis

0

1

2

1N

],...,[ 10 N

Frequency Presence of complex basic sinusoid in signal x

1

000,

N

n

nsnxsx

1

011,

N

n

nsnxsx

1

022,

N

n

nsnxsx

1

011,

N

nNN nsnxsx

Page 82: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Projection of Signals on Sinusoids

k

kk

kk

ks s

N

Xs

ss

sxx

k

,

,P

xsk in present is

much how indicates projection oft coefficien The

Page 83: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Proportionality of DFT to Projection Coefficients

. onto signal of projection

theoft coefficien theis ,

t coefficien The

set. basis sinusoidcomplex theonto projection of

tscoefficien the toalproportion is DFT Thus, .1,...,1,0

,,

Then . because ,,

2

22

k

k

k

k

k

k

kk

k

k

k

sx

N

X

s

sx

Nk

Xs

sxNNs

N

X

s

sx

Page 84: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Inverse DFT

1

0

2

1

0

.1,...,1,0,1

ly,equivalent or, ,1,...,1,0,

sprojection thefrom signal t thereconstruc nowcan We

N

k

N

knj

k

N

kk

k

NneXN

nx

NksN

Xx

Page 85: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Normalized DFT

.1~,

~

as define is (NDFT) DFT Normalized The

.~

sinusoids. DFT normalized theonto signal theprojectingby obtained

isIt clean.ally theoreticmore be toconsidered is DFT Normalized

1

0

/2

/2

N

n

Nknjkk

Nknj

k

enxN

sxX

N

ens

Page 86: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Normalized Inverse DFT

pure.ally mathematic more isit However,

n.computatio more require they because IDFT and DFT

as frequently as practicein usednot are NIDFT and NDFT

~1~~

:follows as define is (NIDFT) DFT Inverse Normalized

1

0

1

0

/2

N

k

N

k

Nknjkkk eX

NnsXnx

Page 87: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

DFT as a Digital Filter

2/sin

2/sin

as computed isfilter thisof response

frequency theof magnitude The . isoutput whoseand

isinput whose,each for filter digital a isoperation DFT The

T

NTX

Xx

k

kx

kxk

k

Page 88: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

DFT as a Digital Filter

.4/for belowshown is

2/sin

2/sin ofPlot Nk

T

NTX

kx

kxk

Source: J. O. Smith III, Mathematics of the DFT with Audio Applications, 2nd Edition

Page 89: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

DFT Example for N=2

Page 90: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

DFT Example for N=2

.11cos1sin1cos1 and 10 because

,1,11,0 Thus, .1,0for ,

.1,11,0 Thus, .1,0for ,1

., compute usLet

.1,...,0, is sinusoid basis for the formula the

Recall ., :sinusoids basis twohave then we2, have weIf

11

1112/12

1

0002/02

0

10

/2

10

jess

sssneens

sssnens

ss

Nnens

ssN

nj

njnj

nj

Nknjk

Page 91: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

DFT Example for N=2

.41216,

and

81216,

.components two

has spectrum The .2,6 signal theof DFT thecompute usLet

11

00

sxX

sxX

Page 92: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Review: Projection of Signals on Sinusoids

k

kk

kk

ks s

N

Xs

ss

sxx

k

,

,P

xsk in present is

much how indicates projection oft coefficien The

Page 93: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

DFT Example: Projection Coefficients

2,21,1222

4

,P

and

4,41,14 42

8

,P

:sinusoids basis two theonto sprojection thecompute usLet

11111

0

00000

0

1

0

sssss

Xx

sssss

Xx

s

s

Page 94: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

DFT Example:Projections on Two Sinusoids

Page 95: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

DFT & Inverse DFT as

Matrix Multiplication

Page 96: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

DFT Matrix & Inverse DFT Matrix

1 ... 1 0

...

1 ... 1 0

1 ... 1 0

1 ... 1 0

1 1 1 1

...

2 2 2 2

1 1 1 1

0 0 0 0

111

222

111

000

*

1210

1210

1210

1210

Nsss

Nsss

Nsss

Nsss

S

NsNsNsNs

ssss

ssss

ssss

S

NNN

N

N

N

N

N

N

matrix. DFT inverse thecalled is

.matrix DFT thecalled is

,

.,

*

/2**

/2

N

S

S

eWnkSS

eWknSS

N

N

NknjknNNN

NknjknNNN

Page 97: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

DFT as Matrix Manipulation

xS

NNN

X

N Nx

x

x

x

Nsss

Nsss

Nsss

Nsss

X

X

X

X

N

1

...

2

1

0

1 ... 1 0

...

1 ... 1 0

1 ... 1 0

1 ... 1 0

...

*

111

222

111

000

1

2

1

0

1

2

1

0

1

0

/12

1

0

/22

1

0

/12

1

0

/02

*1

*2

*1

*0

*

,

...

,

,

,

......

NN

n

NNnj

N

n

Nnj

N

n

Nnj

N

n

Nnj

N

N

sx

sx

sx

sx

enx

enx

enx

enx

x

s

s

s

s

xSX

., /2** NknjknNNN eWnkSS

Page 98: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

DFT as Matrix Manipulation: DFT Matrix

... 1

...

... 1

... 1

1 ... 1 1 1

1 ... 1 0

...

1 ... 1 0

1 ... 1 0

1 ... 1 0

/112/122/12

/122/8/4

/12/4/2

111

222

111

000

*

*

NNNjNNjNNj

NNjNjNj

NNjNjNj

S

NNN

N

eee

eee

eee

Nsss

Nsss

Nsss

Nsss

S

N

., /2** NknjknNNN eWnkSS

Page 99: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Inverse DFT as Matrix Manipulation

1

0

1

0

1

0

1

0

1

2

1

0

110

110

110

110

1/11

...

2/12

1/11

0/10

...

1 ... 1 1

...

2 2 2

1 1 1

0 0 0

11

well.astion multiplicamatrix as DFT inverse theformulatecan We

N

k kk

N

k kk

N

k kk

N

k kk

NN

N

N

N

N

NsXNNx

sXNx

sXNx

sXNx

X

X

X

X

NsNsNs

sss

sss

sss

NXS

Nx

Page 100: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Back to DFT Example for N=2

.41216,

and

81216,

.components twohasIt .2,6 signal theof spectrum thecompute usLet

11

00

sxX

sxX

XXxSxSx

SxSX

NNNN

x

x

ss

ss

X

X

4

8

21 61

21 61

2

6

1 1

1 1

2

6

1 1

1 1

2

6

1 1

1 1

1

0

1 0

1 0

tion.multiplicamatrix as computed DFT above The

****

11

00

1

0

Page 101: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

Inverse DFT Example for N=2

2

6

2

41 812

4181

4

8

1 1

1 1

2

1

4

8

1 1

0 0

2

11

.4,8 spectrum thefrom signal original erecover th usLet

10

10

ss

ssXS

Nx

Xx

N

Page 102: Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier Transform

References● J. O. Smith III, Mathematics of the Discrete Fourier Transform with

Audio Applications, 2nd Edition.

● G. P. Tolstov. Fourier Series.