1 lecture 1 sampling of signals by graham c. goodwin university of newcastle australia lecture 1...

1

Lecture 1Sampling of Signals

by

Graham C. GoodwinUniversity of Newcastle

Australia

Lecture 1Presented at the “Zaborszky Distinguished Lecture Series”

December 3rd, 4th and 5th, 2007

2

Recall Basic Idea of Samplingand Quantization

Quantization

Sampling

t1 t3t2 t4

t0

123456

3

In this lecture we will ignore quantization issues and focus on the impact of different sampling patterns for scalar and multidimensional signals

4

Outline1. One Dimensional Sampling2. Multidimensional Sampling3. Sampling and Reciprocal Lattices4. Undersampled Signals5. Filter Banks6. Generalized Sampling Expansion (GSE)7. Recurrent Sampling8. Application: Video Compression at Source9. Conclusions

5

Sampling: Assume amplitude quantization sufficiently fine to be negligible.

Question: Say we are given

Under what conditions can we recover

from the samples?

( );f t t Î ¡

( ) ;if t i ZÎ

6

A Well Known Result (Shannon’s Reconstruction Theorem for Uniform Sampling)

Consider a scalar signal f(t) consisting of

frequency components in the range . If

this signal is sampled at period , then the

signal can be perfectly reconstructed from the

samples using:

[ ]( )

( )

sin2

( )

2

s

sk

t k

y t y kt k

w

w

¥

=- ¥

é ùæ ö÷çê ú- D÷ç ÷çê úè øë û=æ ö÷ç - D÷ç ÷çè ø

å

,2 2s sw wæ ö- ÷ç ÷÷çè ø

2s

pwD <

7

Low pass filter recovers original spectrum

Hence

or

( )sY w

2sw-

2sw

sw

( ) ( ) ( )

( ) 12 2

0 otherwise

s s

s ss

Y H Y

H

w w w

w ww w

=

æ ö- ÷ç= £ £ ÷ç ÷çè ø

=( ) ( ) ( )

( ) [ ] ( )

[ ] ( )

ss

sk

sk

y t h y t d

h y k t k d

y k h t k

s s s

s d s s

¥

- ¥

¥¥

- ¥=- ¥

¥

=- ¥

= -

= - - D

= - D

ò

åò

å

Proof: Sampling produces folding

8

A Simple (but surprising) Extension

where

[Recurrent Sampling]

is a periodic sequence of integers; i.e.,

Let

Note that the average sampling period is

e.g.

average 5

k kMD = D

{ }kM k N kM M+ =

1

N

kk

M K=

=å

T K= DK

N

D=D

1

2

3

4

9

1

9

1

D =

D =

D =

D =

9

Non-uniform

Uniform

0 9-1 10 19 20

x x x xxx

0 5 10 15 20

x x x xx

10

Claim:

Provided the signal is bandlimited to

where , then the signal can be

perfectly reconstructed from the periodic

sampling pattern.

where = average sampling period

Proof:We will defer the proof to later when we will use it as an illustration of Generalized Sampling Expansion (GSE) Theorem.

,2 2s sw wæ ö- ÷ç ÷÷çè ø

2s

pw =D

D

11


12

Multidimensional SignalsDigital Photography

Digital Video

x1

x2

x1

x2

x3 (time)

13


14

How should we define sampling for multi-dimensional signals?

Utilize idea of Sampling Lattice

Sampling Lattice

nonsingular matrix D DT Î ´¡ ¡

( ) { }: DLat T Tn n ZL = = Î

15

Also, need multivariable frequency domainconcepts.

These are captured by two ideas

i. Reciprocal Latticeii. Unit Cell

16

Unit Cell (Non-unique)

i.

ii.

Reciprocal Lattice

( ){ } ( ){ }1 1* 2 2 :T T DLat T T n n Zp p- -

L = = Î

( ) ( ){ } ( ) ( ){ }1 1* *1 2

1 2 1 2

2 2

,

T T

D

UC T n UC T n

n n Z n n

p p- -

L + Ç L + =Æ

Î ¹

( ) ( ){ }12

D

T D

n Z

UC T n Rp-

Î

L + =U

( )*UC L

17

One Dimensional Example

Sampling Lattice

0-20 10 20

x x xx

D

{ }. :n n ZL = D Î

-10

x

18

Reciprocal Lattice and Unit Cell

Unit Cell

12

wp0 1

102

103

10

19

Multidimensional Example

x1

x2

1 2 3 4 5-4 -3 -2 -1-1

-2

-3

-4

5

4

3

2

1

2 1

0 2T

é ùê ú=ê úë û

20

Reciprocal Lattice and Unit Cell for Example

1/4 1/2 3/4 1-1/4

-1/2

-3/4

-1

1/2

1/4

( )1

10

21 1

4 2

TT-

é ùê úê ú= ê úê ú-ê úë û

( )( )12 TUC Tp

-

1

2

wp

2

2

wp

21


22

We will be interested here in the situation where the Sampling Lattice is not a Nyquist Lattice for the signal (i.e., the signal cannot be perfectly reconstructed from the original pattern!)

Strategy: We will generate other samples by ‘filtering’or ‘shifting’ operations on the original pattern.

23

Consider a bandlimited signal .Assume the D-dimension Fourier transform has finite support, S.

Then for given D-dimensional lattice T, there always exists a finite set , such that support

( ), Df x x Î ¡

{ } *

1

P

iw Î L

( )( ) ( )( )*

1

ˆ .P

ii

f S UCw w=

Í = L +U

Heuristically: The idea of “Tiling” the area of interest in the frequency domain

24

One Dimensional ExampleOur one dimensional example continued.Sampling Lattice { };k k ZL = D Î

Unit Cell

12

wp0 1

102

103

10( )f w

Bandlimited spectrum

Use1

2

0

2

10

w

pw

=

æ ö÷ç=- ÷ç ÷çè ø

( )( ) ( ) ( )* *2f UC UCw wé ù é ù= L L +ê ú ê úë û ë ûUSupport

112

112

- 2w

p

25


26

Generation of Extra SamplesSuppose now we generate a data set as shown in below( ){ }{ }1 D

Q

q q n Zg Tn

= Î

( )Q P³

Q – Channel Filter Bank

( )f x

( )1h w

( )ˆqh w

( )ˆQh w

L

L

L

( )1g x

( )qg x

( )Qg x

M

M

27


28

Define

Let

be the solution (if it exists) of

for

( ) ( ) ( )

( )

( ) ( )

1 1 2 1 1

1 2

1

ˆ ˆ ˆ

ˆ( )

ˆ ˆ

Q

P Q P

h h h

hH

h h

w w w w w w

w ww

w w w w

é ù+ + +ê úê ú

+ê ú= ê úê úê úê ú+ +ë û

L

M

( )( )

( )

1 ,

,

,Q

x

x

x

w

w

w

é ùFê úê úF = ê úê úFê úë û

M

( )*UCwÎ L

( ) ( )

1

,

T

TP

j x

j x

e

H x

e

w

w

w w

é ùê úê úF = ê úê úê úë û

M

29

Conditions for Perfect Reconstruction

can be reconstructed from

if and only if has full row rank for all in the Unit Cell

where

( )H w

( )f x

( ) ( ) ( )1 D

Q

q qq k Z

f x g Tk x Tkf= Î

= -å å

( ) ( ),Tj x

q q

UC

x x e dwf w w= Fò

GSE Theorem:

w

30

Proof:

Multiply both sides by where (theReciprocal Lattice). Then sum over ‘q’

Note that “tiles” the entiresupport S

Thus,

( ) ( ) ( )*, ;T T

D

j x j Tkq q

k Z

x e x Tk e UCw ww f w-

Î

F = - Î Lå

( )ˆq ih w w+

( ) ( ) ( ) ( )

( )

1 1

ˆ ˆ ,T T

D

Ti

Q Qj Tk j x

q i q q i qq qk Z

j x

h x Tk e h x e

e

w w

w w

w w f w w w-

= =Î

+

+ - = + F

=

å å å

from the Matrix identity that defines

( ) ( ) ( ) ( )

( )

( ) ( ) ( )( )

*

*

1

1 1

ˆ ˆ

ˆ ˆ

TTi

T

D

Pj xj x

iis UC

QPj Tk

i q i qi q k ZUC

f x f e d f e d

f h x Tk e d

w ww

w

w w w w w

w w w w f w

- +

= L

-

= = ÎL

= = +

= + + -

åò ò

å å åò

*iw Î L

( ), xwF

{ }*; and 1, ,i i Pw w w+ Î L = K

31

where we have used the fact that

Since is the output of f(x) passing through ,then

Hence, we finally have

( ) ( ) ( ) ( )

( )( )

*11

ˆ ˆ Ti

D

Pj Tk

i q ii UC

Q

qq k Z

f x x Tkf h e dw ww w w fw w- +

== Î L

é ùê ú

= -ê úê úê úë û

+ +å å òå

( )1

2 for .T Di T Zw p

-= Îl l

( )qg x

[ ] ( )qg Tk=

( ) ( ) ( )1 D

Q

q qq k Z

f x g Tk x Tkf= Î

= -å å

( )ˆqh w

ÑÑÑ

32


33

Special Case: Recurrent Sampling

(where is implemented by a “spatial” shift )This amounts to the sampling pattern:

where w.l.o.g.

Now, given the samples , our goal is to

perfectly reconstruct

( ){ }1

Q

qq

Lat T x=

Y = +I

{ } ( )qx UC TÎ

( ){ }x

f xÎ Y%

%

( ).f x

qh qx

34

Here , and

Thus

To apply the GSE Theorem we require

( )ˆ Tqj x

qh e ww = ( ) ( )q qg x f x x= +

( ){ } ( ){ }, 1,Pq xn Z q Q

g Tn f xÎ YÎ =

=%K

%

Nonsingular

( )( ) ( )

( ) ( )

11 1

1

11 1 1

1

0

0

TTQ

TTP QP

TT TQ

TTT QP QP

j xj x

j xj x

j xj x j x

j xj xj x

e eH

e e

e e e

ee e

w ww w

w ww w

ww w

www

w++

++

é ùê ú= ê úê úë ûé ùé ùê úê ú= ê úê úê úê úë ûë û

L

L

LO

L

35

Something to think aboutThe GSE result depends on inversion of a

particular matrix, H(w). Of course we have assumed here perfect representation of all coefficients. An interesting question is what happens when the representation is imperfect i.e. coefficients are represented with finite wordlength (i.e. they are quantized)

We will not address this here but it is something to keep in mind.

36

Return to our one-dimensional exampleRecall that we had

so that

support

Say we use recurrent sampling with

1

2

0

2

10

w

pw

=

=-

1

2

0

0.9 ; 10

x

x

=

= D D =

( )( ) ( ) ( )* *2f UC UCw wé ù é ù= L È L +ê ú ê úë û ë û

37

0 10 20

x xx

0 9 19

x xx

0

x xx

1 0x =

2 0.9x = D

-1

-1

x x

19 20

x

9 10

38

Condition for Perfect Reconstruction is

nonsingular

( )( )

1 1 1 2

2 1 2 2

0.9 2

1 1

1

j x j x

j x j x

j

e e

e e

e

w w

w w

p-

é ùê úê úë ûé ùê ú= ê úë û

Hence, the original signal can be recovered from the sampling pattern given in the previous slide.

39

Summary We have seen that the well known

Shannon reconstruction theorem can be extended in several directions; e.g. Multidimensional signals

Sampling on a lattice

Recurrent sampling

Given specific frequency domain distributions, these can be matched to appropriate sampling patterns.

40


41

Application: Video Compression SourceIntroduction to video cameras Instead of tape, digital cameras use 2D sensor

array (CCD or CMOS)

ImageProcessor

ImageProcessor

Memory

ImageProcessor

ImageProcessor

ImageProcessor

ImageProcessing Display

( TV or LCD )Pipeline

DVCDcontroller

42

Image Sensor

A 2D array of sensors replaces the traditional tape

Each sensor records a 'point' of the continuous image

The whole array records the continuous image at a particular time instant

43

2D Colours Sensor Array

Data transfer from array is sequentialand has a maximal rate of Q.

* Based on http://www.dpreview.com/learn/

44

Uniform 3D sampling

a sequence of identical frames equally spaced in time

Current Technology

45

The volume of ‘box’ depends on the capacity: pixel rate = (frame rate) x (spatial resolution)

xx

Video Bandwidth

depends on spatial resolutionof the frames

depends on the frame rate

46

1. Data recording on sensor:

• Sensor array density - for spatial resolution

pixels

frame R

• Sensor exposure time - for frame rate

frames

sec. F

2. Data reading from sensor:

• Data readout time - for pixel rate

pixels

sec. Q

Hard Constraints

47

Generally Q << RFNeed: R1< RF1 < F

s.t. R1F1 = Q

Compromise: spatial resolution R1< R

temporal resolution F1 < F

BUT...

48

x

yt

1R 1F

volume determined by 1 1Q R F

Actual Capacity (Data Readout)

49

Observation

Most energy of typical video scene is concentrated around the plane and the axis.

,x y

t

t x

50

t x

uniform sampling - compromisein frame rate

uniform sampling - compromisein spatial resolution

uniform sampling- no compromise

The Spectrum of this Video Clip

52

y

x

2 1N y

21

Lx

frame type A frame type B

t

t 22 1M t

12 1M t

Recurrent Non-Uniform Sampling

53

yt

x

What Does it Buy?

54

Schematic Implementation

t

Filter bank t t

non-uniform data from the sensor

uniform high def. video

'compression at the source'

55

Recurrent Non-Uniform Sampling

A special case of

Generalized Sampling Expansion Theorem

56

Sampling Pattern

tmULAT

xlULATULAT

M

Mm

L

l

0)(

0)()(

1

2

2

12

2

1

sML

s

xULAT

)(1)(2

1

The resulting sampling pattern is given by:

57

Frequency Domain

rTML

r

UUCS

)2(

1)(2

1

where:

tMxLUUC tx

t

xT

)12(,

)12(:)2(

1

is the unit cell of the reciprocal lattice

2

1

:

)12(0

0)12(

)2( Znn

tM

xLULAT T

58

Reciprocal Lattice

x

x

tM

M

)12(

12

1

tM

M

)12(

12

1

x

t

tM )12( 1

xL )12(

Unit cell

59

Apply the GSE Theorem

)()(

1)(2

2

1

xH

ML

where: is uniquely defined by H1…H2(…) is a set of 2(L+M)+1 constraints

)()(),( 1 xHx If exists, we can find the reconstruction function 1H

( )H w

60

Reconstruction Scheme

H1 1

H2L+1 2L+1

H2(L+M)+1 2(L+M)+1

I(x,t) Î(x,t)

1M)2(L

frequencyNyquist

r

rTr t

xj

sr eH

.

The sub-sampled frequency of each filter H is:

61

Reconstruction functions

t

t1)(2Msin

))1((

x)1)-(r(sin)12(),(

xrx

xx

txLtxr

))12((

t)1)-2L-(r(sin

x

x1)(2Lsin

)12(),(tLrt

tt

txMtxr

for r = 2,3,…,2L+1

for r = 2(L+1),…,2(L+M)+1

Multidimensional ‘sinc like’ functions

62

Demo

Full resolutionsequence

Reconstructedsequence

Temporaldecimation

Spacialdecimation

63


64

Conclusions Nonuniform sampling of scalar signals

Nonuniform sampling of multidimensional signals

Generalized sampling expansion

Application to video compression

A remaining problem is that of joint design of sampling schemes and quantization strategies to minimize error for a given bit rate

65

References One Dimensional Sampling

A. Feuer and G.C. Goodwin, Sampling in Digital Signal Processing and Control. Birkhäuser, 1996. R.J. Marks II, Ed., Advanced Topics in Shannon Sampling and Interpolation Theory. New Your: Springer-Verlag, 1993.

Multidimensional Sampling W.K. Pratt, Digital Image Processing, 3rd ed: John Wiley & Sons, 2001. B.L. Evans, “Designing commutative cascades of multidimensional upsamplers and downsamplers,” IEEE Signal Process Letters, Vol4, No.11, pp.313-316, 1997.

Sampling and Reciprocal Lattices, Undersampled Signals A.Feuer, G.C. Goodwin, ‘Reconstruction of Multidimensional Bandlimited Signals for Uniform and Generalized Samples,’ IEEE Transactions on Signal Processing, Vol.53, No.11, 2005. A.K. Jain, Fundamentals of Digital Image Processing, Englewood Cliffs, NJ: Prentice-Hall, 1989.

66

References Filter Banks

Y.C. Eldar and A.V. Oppenheim, ‘Filterbank reconstruction of bandlimited signals from nonuniform and generalized samples,’ IEEE Transactions on Signal Processing, Vol.48, No.10, pp.2864-2875, 2000. P.P. Vaidyanathan, Multirate Systems and Filter Banks. Englewood Cliffs, NJ: Prentice-Hall, 1993. H. Bölceskei, F. Hlawatsch and H.G. Feichtinger, ‘Frame-theoretic analysis of oversampled filter banks,’ IEEE Transactions on Signal Processing, Vol.46, No.12, pp.3256-3268, 1998. M. Vetterli and J. Kovaĉević, Wavelets and Subband Coding, Englewood Cliffs, NJ: Prentice Hall, 1995.

67

References Generalized Sampling Expansions, Recurrent Sampling

A. Papoulis, ‘Generalized sampling expansion,’ IEEE Transaction on Circuits and Systems, Vol.CAS-24, No.11, pp.652-654, 1977. A. Feuer, ‘On the necessity of Papoulis result for multidimensional (GSE),’ IEEE Signal Processing Letters, Vol.11, No.4, pp.420-422, 2004. K.F.Cheung, ‘A multidimensional extension of Papoulis’ generalized sampling expansion with application in minimum density sampling,’ in Advanced Topics in Shannon Sampling and Interpolation Theory, R.J. Marks II. Ed., New York: Springer-Verlag, pp.86-119, 1993.

Video Compression at Source E. Shechtman, Y. Caspi and M. Irani, ‘Increasing space-time resolution in video’, European Conference on Computer Vision (ECCV), 2002. N. Maor, A. Feuer and G.C. Goodwin, ‘Compression at the source of digital video,’ To appear EURASIP Journal on Applied Signal Processing.

68

Lecture 1Sampling of Signals

by

Graham C. GoodwinUniversity of Newcastle

Australia

Lecture 1Presented at the “Zaborszky Distinguished Lecture Series”

December 3rd, 4th and 5th, 2007

1 lecture 1 sampling of signals by graham c. goodwin university of newcastle australia lecture 1...

Documents