unit-i discrete fourier transform. syllabus discrete signals and systems- a review introduction to...

UNIT-I

DISCRETE FOURIER TRANSFORM

Syllabus

• Discrete Signals and Systems- A Review• Introduction to DFT • Properties of DFT• Circular Convolution• Filtering methods based on DFT• FFT Algorithms • Decimation in time Algorithms,• Decimation in frequency Algorithms • Use of FFT in Linear Filtering.

Signal Processing• Humans are the most advanced signal processors

– speech and pattern recognition, speech synthesis,…• We encounter many types of signals in various applications

– Electrical signals: voltage, current, magnetic and electric fields,…– Mechanical signals: velocity, force, displacement,…– Acoustic signals: sound, vibration,…– Other signals: pressure, temperature,…

• Most real-world signals are analog– They are continuous in time and amplitude– Convert to voltage or currents using sensors and transducers

• Analog circuits process these signals using– Resistors, Capacitors, Inductors, Amplifiers,…

• Analog signal processing examples– Audio processing in FM radios– Video processing in traditional TV sets

Limitations of Analog Signal Processing• Accuracy limitations due to

– Component tolerances– Undesired nonlinearities

• Limited repeatability due to– Tolerances– Changes in environmental conditions

• Temperature• Vibration

• Sensitivity to electrical noise• Limited dynamic range for voltage and currents• Inflexibility to changes • Difficulty of implementing certain operations

– Nonlinear operations– Time-varying operations

• Difficulty of storing information

Digital Signal Processing• Represent signals by a sequence of numbers

– Sampling or analog-to-digital conversions

• Perform processing on these numbers with a digital processor

– Digital signal processing

• Reconstruct analog signal from processed numbers

– Reconstruction or digital-to-analog conversion

A/D DSP D/Aanalogsignal

analogsignal

digital signal

digital signal

• Analog input – analog output – Digital recording of music

• Analog input – digital output– Touch tone phone dialing

• Digital input – analog output– Text to speech

• Digital input – digital output– Compression of a file on computer

Pros and Cons of Digital Signal Processing

• Pros– Accuracy can be controlled by choosing word length– Repeatable– Sensitivity to electrical noise is minimal– Dynamic range can be controlled using floating point numbers– Flexibility can be achieved with software implementations– Non-linear and time-varying operations are easier to implement – Digital storage is cheap– Digital information can be encrypted for security– Price/performance and reduced time-to-market

• Cons– Sampling causes loss of information– A/D and D/A requires mixed-signal hardware– Limited speed of processors– Quantization and round-off errors

Discrete Signals and Systems

Discrete-Time Signals: Sequences• Discrete-time signals are represented by sequence of numbers

– The nth number in the sequence is represented with x[n]

• Often times sequences are obtained by sampling of continuous-time signals

– In this case x[n] is value of the analog signal at xc(nT)

– Where T is the sampling period

0 20 40 60 80 100-10

0

10

t (ms)

0 10 20 30 40 50-10

0

10

n (samples)

Basic Sequences and Operations• Delaying (Shifting) a sequence

• Unit sample (impulse) sequence

• Unit step sequence

• Exponential sequences

]nn[x]n[y o

0n1

0n0]n[

0n1

0n0]n[u

nA]n[x

-10 -5 0 5 100

0.5

1

1.5

-10 -5 0 5 100

0.5

1

1.5

-10 -5 0 5 100

0.5

1

Sinusoidal Sequences• Important class of sequences

• An exponential sequence with complex

• x[n] is a sum of weighted sinusoids• Different from continuous-time, discrete-time sinusoids

– Have ambiguity of 2k in frequency

– Are not necessary periodic with 2/o

ncosnx o

jj eAA and e o

nsinAjncosAnx

eAeeAAnx

o

n

o

n

njnnjnjn oo

ncosnk2cos oo

integer an is k2

N if only Nncosncoso

ooo

Discrete-Time Systems• Discrete-Time Sequence is a mathematical operation that maps a given

input sequence x[n] into an output sequence y[n]

• Example Discrete-Time Systems– Moving (Running) Average

– Maximum

– Ideal Delay System

]}n[x{T]n[y T{.}x[n] y[n]

]3n[x]2n[x]1n[x]n[x]n[y

]2n[x],1n[x],n[xmax]n[y

]nn[x]n[y o

Memoryless System• Memoryless System

– A system is memoryless if the output y[n] at every value of n depends only on the input x[n] at the same value of n

• Example Memoryless Systems– Square

– Sign

• Counter Example– Ideal Delay System

2]n[x]n[y

]n[xsign]n[y

]nn[x]n[y o

Linear Systems• Linear System: A system is linear if and only if

• Examples– Ideal Delay System

(scaling) ]n[xaT]n[axT

and

y)(additivit ]n[xT]n[xT]}n[x]n[x{T 2121

]nn[x]n[y o

]nn[ax]n[xaT

]nn[ax]n[axT

]nn[x]nn[x]n[xT]}n[x{T

]nn[x]nn[x]}n[x]n[x{T

o1

o1

o2o112

o2o121

Time-Invariant Systems• Time-Invariant (shift-invariant) Systems

– A time shift at the input causes corresponding time-shift at output

• Example– Square

• Counter Example– Compressor System

]nn[xT]nn[y]}n[x{T]n[y oo

2]n[x]n[y

2oo

2o1

]nn[xn-ny gives output the Delay

]nn[xny is output the input the Delay

]Mn[x]n[y

oo

o1

nnMxn-ny gives output the Delay

]nMn[xny is output the input the Delay

Causal System• Causality

– A system is causal it’s output is a function of only the current and previous samples

• Examples– Backward Difference

• Counter Example– Forward Difference

]n[x]1n[x]n[y

]1n[x]n[x]n[y

Stable System• Stability (in the sense of bounded-input bounded-output BIBO)

– A system is stable if and only if every bounded input produces a bounded output

• Example– Square

• Counter Example– Log

yx B]n[y B]n[x

2]n[x]n[y

2x

x

B]n[y by bounded is output

B]n[x by bounded is input if

]n[xlog]n[y 10

nxlog0y 0nxfor bounded not output

B]n[x by bounded is input if even

10

x

DFS,DTFT & DFT

Fourier Transform of continuous time signals

dtetxtxFTfX ftj 2)()()(

dfefXfXIFTtx ftj 2)()()(

Linear Time Invariant (LTI) Systems and z-Transform

][nx ][ny][nh

If the system is LTI we compute the output with the convolution:

m

mnxmhnxnhny ][][][*][][

If the impulse response has a finite duration, the system is called FIR (Finite Impulse Response):

][][...]1[]1[][]0[][ NnxNhnxhnxhny

n

nznxnxZzX ][][)(

Z-Transform

Facts:

)()()( zXzHzY

][nx ][ny)(zH

Frequency Response of a filter:

fjezzHfH 2)()(

Frequency-Domain Properties

Time

Continuous

Discrete

PeriodicityPeriodic Aperiodic

Fourier SeriesFourier Series Continuous-TimeFourier Transform

Continuous-TimeFourier Transform

DFTDFT

DurationFinite Infinite

Discrete-TimeFourier Transformand z-Transform

Discrete-TimeFourier Transformand z-Transform

Discrete &

Aperiodic

Discrete &

Aperiodic

Continuous &

Aperiodic

Continuous &

Aperiodic

Continuous &

Periodic (2)

Continuous &

Periodic (2)

Discrete &

Periodic

Discrete &

Periodic

Signal Processing Methods

Time

Continuous

Discrete

PeriodicityPeriodic Aperiodic

Fourier SeriesFourier Series Continuous-TimeFourier Transform

Continuous-TimeFourier Transform

DFSDFS

DurationFinite Infinite

Discrete-TimeFourier Transformand z-Transform

Discrete-TimeFourier Transformand z-Transform

The Four Fourier Methods

28

Relations Among Fourier Methods

29

The Family of Fourier Transform

Aperiodic-Continuous － Fourier Transform

dejXtx

dtetxjX

tj

tj

)(2

1)(

)()(

The Family of Fourier Transform

Periodic-Continuous － Fourier Series

k

tjk

T

T

tjk

ejkXtx

dtetxT

jkX

0

0

0

0

)()(

)(1

)(

0

2

20

0

00

22

TF

The Family of Fourier Transform

Periodic-Discrete － DFS (DFT)

1

0

2

1

0

2

)(1

)(

)()(

N

k

nkN

j

N

n

nkN

j

ekXN

nx

enxkX

Introduction

The Discrete Fourier Transform (DFT)

The DFS provided us a mechanism for numerically computing the discrete-time Fourier transform. But most of the signals in practice are not periodic. They are likely to be of finite length.

Theoretically, we can take care of this problem by defining a periodic signal whose primary shape is that of the finite length signal and then using the DFS on this periodic signal.

Practically, we define a new transform called the Discrete Fourier Transform, which is the primary period of the DFS.

This DFT is the ultimate numerically computable Fourier transform for arbitrary finite length sequences.

Finite-length sequence & periodic sequence

The Discrete Fourier Transform (DFT)

)(nx Finite-length sequence that has N samples

)(~ nx periodic sequence with the period of N

)()(~)(

,0

10 ),(~)(

nRnxnx

elsewhere

Nnnxnx

N

))(()(~

)()(~

N

r

nxnx

rNnxnx

Window operation

Periodic extension

The Family of Fourier Transform

Aperiodic-Discrete － DTFT

deeXnx

enxeX

njj

n

njj

)(2

1)(

)()(

DFT and IDFT

IDFT

The definition of DFT

The Discrete Fourier Transform (DFT)

10 ,)(1

)]([IDFT)(

10 ,)()]([DFT)(

1

0

1

0

NnWkXN

kXnx

NkWnxnxkX

N

n

nkN

N

n

nkN

)()(~)()(1

)(

)()(~

)()()(

1

0

1

0

nRnxnRWkXN

nx

kRkXkRWnxkX

N

N

nN

nkN

N

N

nN

nkN

Twiddle factor

Relationship of DFT to other transforms

Problems on DFT

Example of DFT

• Find X[k]

– We know k=1,.., 7; N=8

Example of DFT

Example of DFT

Time shift Property of DFT

Polar plot for

Example of DFT

Example of DFT

Example of DFT

Summation for X[k]

Using the shift property!

Example of DFT

Summation for X[k]

Using the shift property!

Example of IDFT

Remember:

Example of IDFT

Remember:

Problems on DFT

Problems on DFT

Circular shift of a sequence

The Properties of DFT

Linearity

)()()]()([DFT 2121 kbXkaXnbxnax N3-point DFT, N3=max(N1,N2)

Circular shift of a sequence

)()]())(([DFT kXWnRmnx kmNNN

)())(()]([DFT kRlkXnxW NNnl

N Circular shift in the frequency domain

The Properties of DFT

The sum of a sequence

1

00

1

00

)()()(N

nk

N

n

nkNk

nxWnxkX

The first sample of sequence

1

0

)(1

)0(N

k

kXN

x

)())(()]([

)()]([

kRkNNxnXDFT

kXnxDFT

NN

The Properties of DFT

Circular convolution

)( )()())(()(

)())(()()( )(

12

1

012

1

02121

nxnxnRmnxmx

nRmnxmxnxnx

N

N

mN

N

N

mN

N

N

)()()]( )([DFT 2121 kXkXnxnx N

)( )(1

)]()([DFT 2121 kXkXN

nxnx N

Multiplication

The Properties of DFT

Circular convolution

)( )()())(()(

)())(()()( )(

12

1

012

1

02121

nxnxnRmnxmx

nRmnxmxnxnx

N

N

mN

N

N

mN

N

N

)()()]( )([DFT 2121 kXkXnxnx N

)( )(1

)]()([DFT 2121 kXkXN

nxnx N

Multiplication

The Properties of DFT

Circular correlation

nn

xy nymnxmnynxmr )(*)()(*)()(

Circular correlation

)()(*))((

)())((*)()(

1

0

1

0

mRnymnx

mRmnynxmr

N

N

nN

N

N

nNxy

Linear correlation

The Properties of DFT

)()(*))((

)())((*)(

)]([IDFT)(

)()()(

1

0

1

0

*

mRnymnx

mRmnynx

kRmrthen

kYkXkRif

N

N

nN

N

N

nN

xyxy

xy

The Properties of DFT

Parseval’s theorem

1

0

*1

0

* )()(1

)()(N

k

N

n

kYkXN

nynx

1

0

21

0

2

1

0

*1

0

*

)(1

)(

)()(1

)()(then

)()( let

N

k

N

n

N

k

N

n

kXN

nx

kXkXN

nxnx

nynx

The Properties of DFT

Conjugate symmetry properties of DFT

and)(nxep )(nxopLet be a N-point sequence)(nx Nnxnx ))(()(~

]))(())(([2

1)](~)(~[

2

1)(~

]))(())(([2

1)](~)(~[

2

1)(~

NNo

NNe

nNxnxnxnxnx

nNxnxnxnxnx

It can be proved that

)(~)(~)(~)(~

*

*

nxnx

nxnx

oo

ee

The Properties of DFT

)())(())((2

1

)()(~)(

)())(())((2

1

)()(~)(

nRnNxnx

nRnxnx

nRnNxnx

nRnxnx

NNN

Noop

NNN

Neep

Circular conjugate symmetriccomponent

Circular conjugate antisymmetriccomponent

The Properties of DFT

)()()( nxnxnx opep

)())(()(

)())(()(*

*

nRnNxnx

nRnNxnx

NNopop

NNepep

The Properties of DFT

)()()( kXkXkX opep

)())(()(

)())(()(*

*

kRkNXkX

kRkNXkX

NNopop

NNepep

and)(kX ep )(kX op

The Properties of DFT

)]())((Im[)](Im[

)]())((Re[)](Re[

kRkNXkX

kRkNXkX

NNepep

NNepep

)]())((Im[)](Im[

)]())((Re[)](Re[

kRkNXkX

kRkNXkX

NNopop

NNopop

The Properties of DFT

)())(()(then

)())(()( if

kRkNXkX

nRnNxnx

NN

NN

Circular even sequences

Circular odd sequences

)())(()(then

)())(()( if

kRkNXkX

nRnNxnx

NN

NN

Circular convolution in Time domain

Circular convolution

Circular convolution-Example

85

x1(n)=(1,2,3) and x2(n)=(4,5,6).

One sequence is distributed clockwise and

the other counterclockwise and the shift of the inner circle is clockwise.

1

23

6

54

*

+

* *

++

=6+10+12=28

1st term

1

23

4

65

*

+

* *

++

=4+12+15=31

2st term

86

1

23

5

46

*

+

* *

++

=5+8+18=31

3rd term

x1(n)* x2(n)=(1,2,3)*(4,5,6)=(18,31,31).

We have the same symmetry as before

˜ y (n) ˜ x 1 (k) ˜ x 2 (n k)k0

N 1

˜ y (n) ˜ x 2(k) ˜ x 1(n k)k0

N 1

Cirular convolution-Matrix method

Circular convolution property

Use of DFT in linear filtering

1

0

][][][L

k

knhkxny

DFT- based filtering always performs circular convolution.But by zero padding adequetly,Circular convoltion can yield the same result.

where n=0.....(L+M-1)-1

Linear convolution using DFT

1][][)( 21 LLNoflen g thtonhan dnxpa dd in gz er oa

][*][][)( nhnxnya

(2) calculate linear convolution by circular convolution

1][][)( 21 LLNoflen g thtonhan dnxpa dd in gz er oa

1][][)( 21 LLNoflen g thtonhan dnxpa dd in gz er oa

(1)calculate N point circular convolution by linear convolution

(3) calculate linear convolution by DFT

][][][)]([)( nRrNnynhNnxb N

r

][)] ([][][)( nhNnxnhnxb

][][in t)( nhandnxo fD FTspoNb

][)]([][*][

]}[][{][)]([)(

nhNnxnhnx

kHkXIDFTnhNnxc

LINEAR & CIRCULAR CONVOLUTION

Example:Cir.conv

Circular conv

FAST FOURIER TRANSFORM

Discrete Fourier Transform

• The DFT pair was given as

• Baseline for computational complexity:

– Each DFT coefficient requires

• N complex multiplications

• N-1 complex additions

– All N DFT coefficients require

• N2 complex multiplications

• N(N-1) complex additions

• Complexity in terms of real operations

• 4N2 real multiplications

• 2N(N-1) real additions

• Most fast methods are based on symmetry properties

– Conjugate symmetry

– Periodicity in n and k

1N

0k

knN/2jekXN1

]n[x

1N

0n

knN/2je]n[xkX

knNjnkNjkNNjnNkNj eeee /2/2/2/2

nNkNjNnkNjknNj eee /2/2/2

Properties-Example

Decimation-In-Time FFT Algorithms

• Makes use of both symmetry and periodicity• Consider special case of N an integer power of 2• Separate x[n] into two sequence of length N/2

– Even indexed samples in the first sequence– Odd indexed samples in the other sequence

• Substitute variables n=2r for n even and n=2r+1 for odd

• G[k] and H[k] are the N/2-point DFT’s of each subsequence

1

oddn

/21

evenn

/21

0

/2 ][][][N

knNjN

knNjN

n

knNj enxenxenxkX

kHWkG

WrxWWrx

WrxWrxkX

kN

Nrk

Nk

N

Nrk

N

Nkr

N

Nrk

N

]12[]2[

]12[]2[

12/

0r2/

12/

0r2/

12/

0r

1212/

0r

2

Decimation In Time• 8-point DFT example using

decimation-in-time

• Two N/2-point DFTs

– 2(N/2)2 complex multiplications

– 2(N/2)2 complex additions

• Combining the DFT outputs

– N complex multiplications

– N complex additions

• Total complexity

– N2/2+N complex multiplications

– N2/2+N complex additions

– More efficient than direct DFT

• Repeat same process

– Divide N/2-point DFTs into

– Two N/4-point DFTs

– Combine outputs

DIT-FFT

Decimation In Time Cont’d

• After two steps of decimation in time

• Repeat until we’re left with two-point DFT’s

Decimation-In-Time FFT Algorithm

• Final flow graph for 8-point decimation in time

• Complexity:

– Nlog2N complex multiplications and additions

Butterfly Computation

• Flow graph constitutes of butterflies

• We can implement each butterfly with one multiplication

• Final complexity for decimation-in-time FFT– (N/2)log2N complex multiplications and additions

In-Place Computation

• Decimation-in-time flow graphs require two sets of registers– Input and output for each stage

• Note the arrangement of the input indices– Bit reversed indexing

111x111X7x7X

011x110X3x6X

101x101X5x5X

001x100X1x4X

110x011X6x3X

010x010X2x2X

100x001X4x1X

000x000X0x0X

00

00

00

00

00

00

00

00

Example: 8 point FFT

(1)Number of stages:

– Nstages = 3

(2)Blocks/stage:

– Stage 1: Nblocks = 4

– Stage 2: Nblocks = 2

– Stage 3: Nblocks = 1

(3)B’flies/block:

– Stage 1: Nbtf = 1

– Stage 2: Nbtf = 2

– Stage 3: Nbtf = 2

FFT Implementation

WW00-1-1

WW00-1-1

WW00-1-1

WW00-1-1

WW22-1-1

WW00

-1-1WW00

WW22-1-1

-1-1WW00

WW11-1-1

WW00

WW33-1-1

-1-1WW22

-1-1

Stage 2Stage 2 Stage 3Stage 3Stage 1Stage 1

• Decimation in time FFT:– Number of stages = log2N

– Number of blocks/stage = N/2stage

– Number of butterflies/block = 2stage-1

FFT Implementation

Start IndexStart Index 00

Input IndexInput Index 11

Twiddle Factor IndexTwiddle Factor Index N/2 = 4N/2 = 4 44/2 = 2/2 = 2 22/2 = 1/2 = 1

Indicies UsedIndicies Used WW00 WW00

WW22

WW00

WW11

WW22

WW33

0022

0044

WW00-1-1

WW00-1-1

WW00-1-1

WW00-1-1

WW22-1-1

WW00

-1-1WW00

WW22-1-1

-1-1WW00

WW11-1-1

WW00

WW33-1-1

-1-1WW22

-1-1

Stage 2Stage 2 Stage 3Stage 3Stage 1Stage 1

Decimation-in-time FFT algorithm

Most conveniently illustrated by considering the special case of N an integer power of 2, i.e, N=2v.

Since N is an even integer, we can consider computing X[k] by separating x[n] into two (N/2)-point sequence consisting of the even numbered point in x[n] and the odd-numbered points in x[n].

or, with the substitution of variable n=2r for n even and n=2r+1 for n odd

dd

][][][on

nkN

evenn

nkN WnxWnxkX

1)2/(

0

)12(1)2/(

0

2 ]12[]2[][N

r

krN

N

r

rkN WrxWrxkX

1)2/(

0

21)2/(

0

2 )](12[)](2[N

r

rkN

kN

N

r

rkN WrxWWrx

Since

That is, WN2 is the root of the equation WN/2=1

Consequently,

Both G[k] and H[k] can be computed by (N/2)-point DFT, where G[k] is the (N/2)-point DFT of the even numbered points of the original sequence and the second being the (N/2)-point DFT of the odd-numbered point of the original sequence.Although the index ranges over N values, k = 0, 1, …, N-1, each of the sums must be computed only for k between 0 and (N/2)-1, since G[k] and H[k] are each periodic in k with period N/2.

2/)2//(2)/2(22

NNjNj

N WeeW

1,...,1,0 ],[][ NkkHWkG kN

1)2/(

0

21)2/(

0

2 )](12[)](2[][N

r

rkN

kN

N

r

rkN WrxWWrxkX

Decomposing N-point DFT into two (N/2)-point DFT for the case of N=8

We can further decompose the (N/2)-point DFT into two (N/4)-point DFTs. For example, the upper half of the previous diagram can be decomposed as

Hence, the 8-point DFT can be obtained by the following diagram with four 2-point DFTs.

Flow graph of a 2-point DFT

Finally, each 2-point DFT can be implemented by the following signal-flow graph, where no multiplications are needed.

Flow graph of complete decimation-in-time decomposition of an 8-point DFT.

In each stage of the decimation-in-time FFT algorithm, there are a basic structure called the butterfly computation:

The butterfly computation can be simplified as follows:

Flow graph of a basic butterfly computation in FFT.

Simplified butterfly computation.

][][][

][][][

11

11

qXWpXqX

qXWpXpX

mr

Nmm

mr

Nmm

Flow graph of 8-point FFT using the simplified butterfly computation

In the above, we have introduced the decimation-in-time algorithm of FFT.

Here, we assume that N is the power of 2. For N=2v, it requires v=log2N stages of computation.

The number of complex multiplications and additions required was N+N+…N = Nv = N log2N.

When N is not the power of 2, we can apply the same principle that were applied in the power-of-2 case when N is a composite integer. For example, if N=RQ, it is possible to express an N-point DFT as either the sum of R Q-point DFTs or as the sum of Q R-point DFTs.

In practice, by zero-padding a sequence into an N-point sequence with N=2v, we can choose the nearest power-of-two FFT algorithm for implementing a DFT.

The FFT algorithm of power-of-two is also called the Cooley-Tukey algorithm since it was first proposed by them.

For short-length sequence, Goertzel algorithm might be more efficient.

Decimation-In-Frequency FFT Algorithm• Final flow graph for 8-point decimation in

frequency

Decimation-In-Frequency FFT Algorithm

• The DFT equation

• Split the DFT equation into even and odd frequency indexes

• Substitute variables to get

• Similarly for odd-numbered frequencies

1N

0n

nkNW]n[xkX

1

2/

212/

0

21

0

2 ][][][2N

Nn

rnN

N

n

rnN

N

n

rnN WnxWnxWnxrX

12/

02/

12/

0

22/12/

0

2 ]2/[][]2/[][2N

n

nrN

N

n

rNnN

N

n

rnN WNnxnxWNnxWnxrX

12/

0

122/]2/[][12

N

n

rnNWNnxnxrX

Decimation in Frequency algorithm

Decimation in Frequency

DIF-FFT ALGORITHM

DIF-FFT ALGORITHM

Example

• Using decimation-in-time FFT algorithm compute DFT of the sequence

{-1 –1 –1 –1 1 1 1 1}• Solution: Twiddle factors are

Solution and signal flow graph of the example

IDFT Using DIF-FFT