discrete sampling in continuous time, a dissertationch771fw4323/... · 2013. 6. 17. · john pauly,...

DISCRETE SAMPLING IN CONTINUOUS TIME,

BAND-LIMITED SIGNALS

A DISSERTATION

SUBMITTED TO THE DEPARTMENT OF ELECTRICAL

ENGINEERING

AND THE COMMITTEE ON GRADUATE STUDIES

OF STANFORD UNIVERSITY

IN PARTIAL FULFILLMENT OF THE REQUIREMENTS

FOR THE DEGREE OF

DOCTOR OF PHILOSOPHY

Fernando Gomez-Pancorbo

March 2012

http://creativecommons.org/licenses/by-nc/3.0/us/

This dissertation is online at: http://purl.stanford.edu/ch771fw4323

Includes supplemental files:

1. Matlab code that reproduces the numerical examples mentioned in the dissertation

(matlab_files.zip)

© 2012 by Fernando Gomez Pancorbo. All Rights Reserved.

Re-distributed by Stanford University under license with the author.

This work is licensed under a Creative Commons Attribution-Noncommercial 3.0 United States License.

ii



http://purl.stanford.edu/ch771fw4323

I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.

Brad Osgood, Primary Adviser


John Pauly, Co-Adviser


Julius Smith, III

Approved for the Stanford University Committee on Graduate Studies.

Patricia J. Gumport, Vice Provost Graduate Education

This signature page was generated electronically upon submission of this dissertation in electronic format. An original signed hard copy of the signature page is on file inUniversity Archives.

iii

To my parents Pedro and Marıa Dolores,

and the loving memory of my friend May Zhou

v

Abstract

During more than 60 years, the paradigm implied by the celebrated Shannon sampling

theorem has been the basis of most ADC and DAC architectures, with the exception

of the so called oversampling converters. The improvements in semiconductor tech-

nology allowed for little change in these design architectures while faster, lower power

converters and new generations of manufacturing processes were developed. Over the

last 5 to 10 years, the point of diminishing returns with respect to scaling seems to

have been reached. As a result, there is a renewed interest in researching alternative

sampling paradigms to Shannon’s.

Discrete Sampling is one of the numerous sampling alternatives proposed recently.

In this research we show how Discrete Sampling can be extended to work with the

class of continuous time band-limited signals. In addition to several mathematical

results on the matter, the actual implementation of the paradigm is also considered.

It will be shown that the proposed scheme results in a particular type of Hybrid Filter

Bank. Several examples of such banks are presented together with their performance

when sampling and reconstructing real life audio signals.

vii

Acknowledgments

I would not have completed successfully this dissertation without the help, guidance

and mentorship of numerous people whom I want to sincerely thank for their support.

First and foremost I am deeply thankful to my adviser, Professor Brad Osgood, for

giving me the opportunity to work with him. Incidentally, his famous Fourier course

EE261 is the first class that I took as a graduate student in Electrical Engineering.

He set his teaching standard so high that no other professor has come close to it

in any of the classes that I have taken to complete the coursework requirements of

the Master of Science and PhD degrees. In fact, the only reason I took the Linear

Dynamical Systems course EE263 is that it so happened that he taught it during the

Spring Quarter of the 2006-2007 academic year. It goes without saying that I have

nothing but great admiration for the fantastic professors that I have had in my other

classes. In addition, he has been a great research adviser, mentor, and an even better

human being who has always been concerned about my well being and my graduate

student experience. It has been a great honor to work with him during the last three

years.

I want to thank Professor Boris Murmann for advising me during my first two years

as PhD student. Under his guidance, I learned a great deal about what makes data

converters work in real life. The lessons I learned with him inspired a great deal of my

work in extending Discrete Sampling to work with infinitely long, square summable

discrete sequences. During that time, I was co-advised by Professor Tsachy Weiss-

man whom I also want to thank for his critical support and for the many stimulating

discussions on modeling noise. My deep gratitude also goes to Professor John Pauly,

ix

who graciously accepted to serve as my research co-advisor and a reader for this dis-

sertation, Professor Julius Smith, for the wonderful learning experience in his course

Music 421 on Audio Applications of the FFT and for agreeing to be a reader of this

thesis, Professor Jonathan Abel, for accepting to be part of my PhD Orals committee

in a very short notice, and Professor John Eaton, for kindly chairing my PhD Orals.

I am highly indebted to the various people and organizations that have provided me

with financial assistance during my graduate student years, allowing me to gradu-

ate without significant financial burden. Professor Butrus Khuri-Yakub’s rotating

research assistantship funded my first year which I spent doing research at the labo-

ratories of Professors Robert Byer, Philip Wong and Boris Murmann. My next three

years were funded by Professor Boris Murmann with grants from the Robert Bosch

RTC, Research Technology Center. Christoph Lang, manager at RTC, was instru-

mental in setting up this collaboration which not only allowed me to learn a great

deal about automotive electronics but also resulted in United States Patent 7,424,407.

Finally, the funding for my final years was provided through a consultant position

at the Information Surfaces Laboratory at Hewlett Packard Laboratories, that was

offered to me by managers Carl Taussig and Rich Elder. The last three years working

with their team have been exhilarating; they have put together an exceptional group

of researchers who are also exceptional human beings.

Life with the administrative side of the Electrical Engineering Department was made

easy thanks to several of its staff members whose assistance I want to acknowledge:

Denise Murphy, Kelly Yilmaz, Ann Guerra, Natasha Newson, Diane Shankle, Vicky

Carrillo, Amy Duncan and Rafael Ulate. Over the years, Rafael and I became great

friends; I am still in touch with him even after he moved to Santa Clara University.

I have made many friends during my years as graduate student, I want to thank

them for their continuing support and for making my graduate student life richer

on the human side. I want to name a few special ones, although that doesn’t mean

that I have forgotten about the rest: Oren Feinstein, Mahdieh Bagher Shemirani,

Babak Javid, Shen Ren, Pedram Lajevardi, John Cunningham, Grace Gao, Henrique

Miranda, Daniel Fernandes, Isaac Martinez, Deji Akinwande, Mehdi Jahanbakht,

Maryam Fathi, Mohammad Hekmat, Gunhan Ertosun, Kunal Ghosh, Guangyu Shi,

x

Luis Adarve, Tom Lee, Kamil Dada, Xiaowei Ding, James Hohmann, Paul Gurney,

Kivanc Ozonat and Borja Manuel Peleato-Inarrea.

Of course I couldn’t leave out of the acknowledgments section the colleagues that

have made my experience at the different labs enjoyable and fun: for Professor’s

Murmann Group I want to thank Drew Hall, Yoonyoung Chung, Parastoo Nikaeen,

Ross Walker, Alex Guo, Ray Nguyen, Noam Dolev, Clay Daigle, Jim Salvia, Jason

Hu, Echere Iroaga, Alireza Dastgheib, Manar El-Chammas, Yangjin Oh, Christo-

pher Anderson, Justin Kim, Wei Xiong, Donghyun Kim; for Bosch Research and

Technology Center I want to thank Vladimir Petkov, Xinyu Xing, Johan Vanderhae-

gen, Chinwuba Ezekwe and Thomas Rocznik; for HP Labs I want to thank Marcia

Almanza-Workman, Warren Jackson, Hao Luo, Lihua Zhao, Steven Trovinger, Mark

Smith, Craig Perlov, Ping Mei, John Maltabes, Alejandro de la Fuente Vornbrock,

Ohseung Kwon, Han-Jun Kim, Albert Jeans, Mehrban Jam, Edward Holland, Robert

Garcia, Robert Cobene, James Brug, Alesha Cater and Lois Masciola; and last, but

certainly not least, for Professors Brad Osgood and John Gill groups I want to thank

William Wu, Joseph Koo, Aditya Siripuram and Ulrich Barnhoefer.

Amongst the most significant events that happened during my graduate career, going

back to God, after spending many years as an atheist, outshines the rest. In fact,

it’s more accurate to say that He found me and welcomed me back with open arms.

My Bible study brothers have helped my faith grow and have been very supportive

all these years. I am particularly thankful to Kary Eldred for inviting me to attend

the group meetings. My Bible study group members are Alan Hsai, John Lovewell,

Bob Larson, Brian Keating, Derek Blazensky, Erik Rannala, Gabriel Yu, Gibs Sung-

Ku Song, Harry Bims, John Shenk, John Mumford, Ken Eldred, Ken Korea, Kevon

Saber, Lee Kenna, Paul Kim, Paul Chau, Phil Larson, Phil Taylor, Philip Larson,

Preston Butcher, Robert Larson and Scott Feamster.

Of course, I would not have been able to come to Stanford University without the

great education that I received during my K-12 schooling and college years. I am very

thankful to the continuous support and encouragement from my teachers at Primary

School “Virgen de La Oliva” in Carcastillo, Spain, and at High School “Marques de

Villena” in Marcilla, Spain as well as from Professors Rafael Cabeza, Sonia Porta

xi

and Antonio Lumbreras at UPNA, Universidad Publica de Navarra in Pamplona,

Spain. Professor Antonio Lumbreras greatly assisted me in continuing my under-

graduate education at ESIEE-Paris, Ecole Superieure d’Ingenieurs en Electronique et

Electrotechnique de Paris, where I spent two amazing years. I want to thank Profes-

sors Jean Michel Le Notre and Martine Villegas for their support during my time at

ESIEE-Paris.

My parents Pedro and Marıa Dolores worked very hard so that my siblings, Marıa

Guadalupe, Pedro, Laura, and I could have the opportunity to go to college. My

older sister Marıa Guadalupe was the first member of the family to go to college. I

was the first member of the family to get a scientific degree. What my parents could

not envision is that one day I would be getting a doctoral degree from one of the

greatest research universities in the world. My PhD degree is a testament to their

dedication and sacrifice.

During my Stanford years, I also had the bad fortune of losing two good friends in

tragic circumstances. I want to honor the memory of Archie Li, and of my fellow

graduate student and classmate May Zhou. May’s death marked a before and after

moment in my life; her loss made me value the many things that I had taken for

granted until then. It made me understand what truly matters in life and to value

the precious gift of being alive. I just wish that I could have learned these lessons in

a different way.

xii

Preface

Sampling is one of the most fascinating areas in the field of signal processing. The

idea that one can put information in the form of a finite length string of zeros and ones

that can be used at a later time to reconstruct a continuous time signal is mind bog-

gling. When Shannon presented in 1949 his famous sampling theorem that explained

how that feat can be accomplished, provided that the continuous time signal is band-

limited, he wasn’t probably aware of the revolution that he was about to unleash.

Today we take high quality digital cameras, audio CDs, DVDs and so many other

technologies that rely on analog to digital conversion, such as wireless technology, for

granted; in fact, for the people of my age or younger it’s hard to imagine a world

in which none of these existed. The vast majority of Analog to Digital Converters,

ADCs, designed over the years are based on Shanon’s paradigm that one can recover

a continuous time band-limited signal, with frequency components up to BHz, from

samples of the signal taken at regularly spaced time points with the distance between

points equal to 12B

. During the decades following Shannon’s paper most research

about solving the sampling problem differently was motivated by mere intellectual

curiosity since for most ADCs the Shannon paradigm, combined with the benefits of

increasingly small transistors allowed by semiconductor technology advances, made it

possible to build faster and more accurate converters. Today, technology scaling has

reached the point where further miniaturization only provides diminishing returns if

we insist in using the Shannon paradigm. As a result there is a new push for research

in sampling theory this time motivated by applications in the real world rather than

by academic reasons. In this thesis I present the results of my doctoral research,

xiii

aimed at extending one of the several alternative sampling approaches developed re-

cently, Discrete Sampling, to the class of continuous time band-limited signals. The

question explored was whether Discrete Sampling could produce a conversion scheme

that would allow the recovery of such signals when sampled at rates below Shannon’s

rate, also known as Nyquist’s rate. As we shall see, the answer in some cases is posi-

tive, provided that we make extra assumptions about the continuous time signal, in

addition to its band-limitedness. In addition, we showed that any continuous time,

band-limited signal can be fully recovered with a scheme that while needing as many

samples as in the case of the Shannon’s paradigm, it consists of several converters

working in parallel, each at a sampling rate below Shannon’s.

Chapter 1 of the thesis gives details about how the Shannon paradigm is implemented

with electrical circuits and explores in detail the reasons that drive the current interest

in sampling theory. This chapter also gives an introduction to Discrete Sampling, a

theory that allows the reconstruction of finite length discrete signals through discrete

interpolation from a set of samples whose size is strictly smaller than the length of the

original signal. Chapter 2 gives an introduction to the mathematical theory of frames,

which will be later used in Chapter 3 to extend Discrete Sampling to infinitely long

discrete signals that belong to `2(R). Chapter 4 explores the circuit implementation

of the new sampling scheme implied by the mathematical results developed in Chap-

ter 3. These two Chapters, 3 and 4, contain my research contributions. Chapters 1

and 2 summarize existing knowledge which is a requisite to develop the new results

contained the the later chapters. Finally in Chapter 5, I reflect on my research work

during the last three years as well as propose research areas that could be explored

in the future that would facilitate the adoption of the new sampling scheme by the

community of ADC designers.

xiv

Contents

v

Abstract vii

Acknowledgments ix

Preface xiii

1 Challenges in Practical Sampling 1

1.1 Real Life Analog to Digital Conversion . . . . . . . . . . . . . . . . . 1

1.1.1 Amplitude Quantization . . . . . . . . . . . . . . . . . . . . . 1

1.1.2 SNDR and ENOB . . . . . . . . . . . . . . . . . . . . . . . . 3

1.1.3 Practical Anti-Aliasing Filters . . . . . . . . . . . . . . . . . . 4

1.1.4 Practical Sampling . . . . . . . . . . . . . . . . . . . . . . . . 5

1.1.5 Quantization, ADC Architectures, Energy Trade-offs . . . . . 8

1.2 Real Life Digital to Analog Conversion . . . . . . . . . . . . . . . . . 9

1.3 Introduction to Discrete Sampling . . . . . . . . . . . . . . . . . . . . 11

1.4 Other Research Approaches . . . . . . . . . . . . . . . . . . . . . . . 13

2 Introduction to Frame Theory 17

2.1 Bases in Infinite-Dimensional Hilbert Spaces . . . . . . . . . . . . . . 17

2.1.1 Orthonormal Bases . . . . . . . . . . . . . . . . . . . . . . . . 17

2.1.2 Riesz Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.2 Frames in Infinite-dimensional Hilbert Spaces . . . . . . . . . . . . . 21

xv

2.2.1 Definition and Properties of Frames . . . . . . . . . . . . . . . 21

2.2.2 Frames and Riesz Bases . . . . . . . . . . . . . . . . . . . . . 24

2.2.3 Frames and Orthogonal Projections . . . . . . . . . . . . . . . 24

2.2.4 An Illustrating Example, the PWTs Space . . . . . . . . . . . 25

3 Discrete Sampling for Signals in PWTs 27

3.1 Infinitely Long Discrete Signals in `2(Z) . . . . . . . . . . . . . . . . 28

3.1.1 Equivalence of Continuous Time Inner Products With Discrete

Time Inner Products . . . . . . . . . . . . . . . . . . . . . . . 30

3.2 Interpolation of Signals in PWTs . . . . . . . . . . . . . . . . . . . . . 31

3.2.1 Interpolation in Continuous Time, Band-Limited, Shift Invari-

ant Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.3 Frames of Shift Invariant Subspaces of PWTs . . . . . . . . . . . . . 42

3.3.1 Orthogonal Interpolating Atoms . . . . . . . . . . . . . . . . . 46

3.3.2 Computing the Dual Riesz Basis For Compactly Supported c[k] 46

3.3.3 Numerical Example . . . . . . . . . . . . . . . . . . . . . . . . 48

3.3.4 Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

4 Discrete Sampling and HBF’s 53

4.1 A Bit on History of HBF’s . . . . . . . . . . . . . . . . . . . . . . . . 54

4.2 Union of Orthogonal SI Subspaces With ON Generators . . . . . . . 55

4.3 Very Brief Overview of Wavelet Theory . . . . . . . . . . . . . . . . . 56

4.3.1 Multiresolution Representations . . . . . . . . . . . . . . . . . 57

4.3.2 A Multiresolution Approximation is a Collection of Shift In-

variant Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . 58

4.3.3 Conjugate Mirror Filters . . . . . . . . . . . . . . . . . . . . . 59

4.3.4 Wavelet Decomposition . . . . . . . . . . . . . . . . . . . . . . 60

4.3.5 Fast Orthogonal Wavelet Transform . . . . . . . . . . . . . . . 61

4.4 Discrete Sampling for PWTs Meets Wavelet Theory . . . . . . . . . . 64

4.4.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

4.5 Noise Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

4.5.1 A More Detailed Look at the Discrete Interpolation Process . 71

xvi

4.5.2 Linear Filtering of a Zero Mean Gaussian iid Process . . . . . 73

4.5.3 Noise Analysis in an HBF Designed with Discrete Sampling . 74

4.6 Practical Imprementation of Continuous Time Filters . . . . . . . . . 78

4.6.1 Computing H(s) for Impulse Response u(−t) . . . . . . . . . 80

4.6.2 Design Example . . . . . . . . . . . . . . . . . . . . . . . . . . 84

5 Conclusions and Future Research 91

5.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

5.2 Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Bibliography 95

xvii

List of Tables

3.1 u[n] Associated with φ12, Wavelet Decomposition Symmlet 8 J=12,

L=10; see figure 4.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.2 c[k] Associated with u[n] Shown in Table 3.1 . . . . . . . . . . . . . . 49

3.3 Coefficients u[nM ] computed from c[k] shown in Table 3.2 . . . . . . 49

3.4 u[n] Associated With u[n] Shown in Table 3.1 . . . . . . . . . . . . . 50

3.5 d[k] Associated with c[k] Shown in Table 3.2 . . . . . . . . . . . . . . 50

4.1 Numerical Results for 32s Segment of Pageant . . . . . . . . . . . . . 69

4.2 ||uqk||2 , Example of Subsection 4.4.1 . . . . . . . . . . . . . . . . . . 78

4.3 H1(s) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

4.4 H2(s) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

xix

List of Figures

1.1 a) Sampling Grid b)Block Diagrams of Practical ADCs and DACs [1] 2

1.2 a) Practical Anti-Aliasing Filter b)Design Rules [1] . . . . . . . . . . 4

1.3 a) Ideal Track & Hold Stage b)Waveforms [1] . . . . . . . . . . . . . 5

1.4 Maximum Input Signal Frequency vs SNRaperture for Different Jitter

Scenarios [1] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.5 a) Zero Order Hold b) Effect of Zero Order Hold in the Spectrum of a

Signal [1] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.6 State of the Art in ADC design, 2011 [5] . . . . . . . . . . . . . . . . 14

3.1 Sampling Scheme With Interpolation for Continuous Time, Band-Limited

Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3.2 Magnitude of the Spectrum of cos(2π 132n) . . . . . . . . . . . . . . . 33

3.3 Magnitude of the Spectrum of ↑↓ cos(2π 132n) . . . . . . . . . . . . . . 33

3.4 Magnitude of the Spectrum of cos(2π 132n).w[n] . . . . . . . . . . . . . 34

3.5 Magnitude of the Spectrum of ↑↓ {cos(2π 132n).w[n]} . . . . . . . . . . 34

3.6 Interpolating Function u[n] for cos(2π 132n).w[n] . . . . . . . . . . . . 35

3.7 cos(2π 132n).w[n] (red) and its Reconstruction (blue) via Interpolation

With u[n] Shown in Figure 3.6 . . . . . . . . . . . . . . . . . . . . . 36

3.8 Example of a Signal that Doesn’t Satisfy Equation (3.4) with M=8 . 37

3.9 Spectrum of ↑↓ {x[n].w[n]} Associated With the Signal Shown in Fig-

ure 3.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.10 Illustration of Shift Invariant Subspace Sampling When {u[n−iM ]}k∈Zis a Riesz Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

xxi

4.1 Hybrid Filter Bank Architecture, Adapted from [34] . . . . . . . . . . 54

4.2 DWT Decomposition (top), Inverse (bottom) [28] . . . . . . . . . . . 61

4.3 WPT Decomposition (top), Inverse (bottom) [28] . . . . . . . . . . . 62

4.4 HFB Associated with Decomposition Symmlet 8 J=12, L=10 . . . . . 64

4.5 Discrete Equivalent of Figure 4.4 . . . . . . . . . . . . . . . . . . . . 65

4.6 Symmlet 8 Generators With J=12, L=10 . . . . . . . . . . . . . . . . 66

4.7 Symmlet 8, J=12, L=10, Interpolating Atom, Dual Associated With φ12 67

4.8 Symmlet 8, J=12, L=10, Interpolating Atom, Dual Associated With ψ12 67

4.9 Symmlet 8, J=12, L=10, Interpolating Atom, Dual Associated With ψ11 68

4.10 Smoothed Magnitude Spectrum of Signals x[n][blue], xφ12 [n] [red] and

xφ12ψ12 [n] [green] for 32s Segment of Theme Pageant . . . . . . . . . . 70

4.11 Polyphase Decomposition of uφ12 [n], from Wavelet Decomposition Symm-

let 8 With J=12, L=10 . . . . . . . . . . . . . . . . . . . . . . . . . . 72

4.12 Interpolation From The Polyphase Decomposition Point of View . . . 74

4.13 Noise Propagation in a Discrete Sampling HBF . . . . . . . . . . . . 76

4.14 Idealized Filter Frequency Responses [43] . . . . . . . . . . . . . . . . 80

4.15 Analog Filter Design Parameters [43] . . . . . . . . . . . . . . . . . . 81

4.16 Magnitude of L{uφ12(−t)}(jω), Example Subsection 4.4.1 . . . . . . . 82

4.17 uψ[n] and uψ[n] Associated with ψ[n], Beylkin Wavelet Decomposition

2 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

4.18 uφ[n] and uφ[n] Associated with φ[n], Beylkin Wavelet Decomposition

2 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

4.19 Tuncated uψ[−n] and uφ[−n] for Beylkin Wavelet Decomposition 2

Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

4.20 DuTψ(j2πfn) and DuTφ(j2πfn) for Beylkin Wavelet Decomposition 2

Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

4.21 Block Diagram of HBF for Beylkin Wavelet Decomposition 2 Subspaces 89

4.22 Discrete Sinc Based Interpolating Filter Used to Resample x(t) at Rate

0.25Hz . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

xxii

Chapter 1

Challenges in Practical Sampling

A research work about sampling must consider practical implementation if it hopes

to be adopted by the community of mixed signal circuit designers, the very people

who design and implement real life data converters. In this chapter I will give an

overview of how sampling is implemented today as well as the numerous challenges

that must be dealt with when designing a real life Analog to Digital Converter, ADC.

We will see that some of these challenges can be alleviated with Discrete Sampling

at the expense of a more complicated analog front end. I will also talk briefly about

other approaches currently being investigated by other researchers to address these

difficulties.

1.1 Real Life Analog to Digital Conversion

The observations of this section are based on [1]. In addition to taking this class

during my PhD studies, I also had the opportunity to serve as the lead TA when the

class was taught as EE315 in Spring 2008.

1.1.1 Amplitude Quantization

When the problem of sampling is presented in a signal processing class, it is usually

the case that the exposition begins by assuming that there is some continuous time

1

2 CHAPTER 1. CHALLENGES IN PRACTICAL SAMPLING

signal x(t) which we want to fully recover from its samples taken at regular time

instances nTs, where n is an integer that goes from −∞ to +∞. Thus the problem is

how we get x(t) back, assuming that one has measured x[n] = x(nTs). The celebrated

Shannon-Nyquist sampling theorem [2] says that if the Fourier Transform of x(t) is

contained in [ −12Ts, 1

2Ts] then x(t) can be recovered exactly by applying the following

reconstruction formula, which is known in the literature as the sinc interpolation

formula,

x(t) =+∞∑

n=−∞

x[n]sinc

(t− nTsTs

)(1.1)

Figure 1.1: a) Sampling Grid b)Block Diagrams of Practical ADCs and DACs [1]

In practical analog to digital conversion, the Shannon sampling theorem only ad-

dresses part of the problem. Since computers must work with finite precision, all

samples x[n] must be discretized in amplitude before they can be passed to the next

processing stage. As shown in figure 1.1a), the sampling process must fit a signal,

which is continuous in time and amplitude, into a grid that results from discretiz-

ing time and amplitude. The discretization in amplitude is usually uniform with

the quantization step equal to ∆. If we have an nb bits converter, the error in am-

plitude incurred by considering a discrete point in the grid, instead of x(nTs), will

be bound by ±∆2

as long as the amplitude of x(t) stays within the so called full

1.1. REAL LIFE ANALOG TO DIGITAL CONVERSION 3

scale range, FSR = 2nb∆. For this analysis to hold, it doesn’t matter whether

the amplitude of x(t) stays within [0, FSR], [−FSR2, FSR

2] or other arrangements as

long as |max(x(t)) −min(x(t))| = FSR. With this setup, the effect of quantization

in amplitude can be modeled via a zero mean uniform random i.i.d. process, eq[n]

∼ U(−∆2, ∆

2),[3],[1], that is added to x(nTs). Thus, instead of measuring x[n] = x(nTs)

we will have

x[n] = x(nTs) + eq[n] (1.2)

This model works well in practice for signals whose amplitude varies “sufficiently”

from one sample to the next, such as sinusoids of frequency less than 12Ts

, whose

amplitude covers the full scale range. The average energy error due to quantization

can be computed with this model as the variance of the process, e2q[n] = ∆2

12.

1.1.2 SNDR and ENOB

We introduce the metric known in the ADC community as SNDR, which stands for

Signal to Noise and Distortion Ratio, and its companion ENOB, which stands for

Equivalent Number Of Bits. Suppose we have a test signal, typically a full scale

range sinusoid of frequency below fs2

, with fs the Nyquist frequency of the ADC, that

we inject to the ADC in order to measure its non idealities. Then,

SNDR = 10 log10

(Psignal

Pquantization error + Pnoise + Pdistortion

)(1.3)

Where Psignal is the total power of the signal used in the test of the ADC, Pquantization error

is the quantization error described in the previous section, Pnoise is the total power of

electrical noise present at the output, and Pdistortion, is the total power of nonlinear

distortion at the output. The metric ENOB defined as SNDR[dB]−1.766.03

gives us the

resolution of a perfect ADC whose SNDR is affected only by quantization noise that

has the same SNDR as the ADC under consideration. For example, say that we have

a 14 bit ADC whose SNDR is 74.12 dB. The resulting ENOB is 12 bit which means

that this 14 bit ADC is behaving in terms of accuracy as a perfect 12 bit ADC that


has no noise or nonlinear distortion, only quantization error.

1.1.3 Practical Anti-Aliasing Filters

Figure 1.2: a) Practical Anti-Aliasing Filter b)Design Rules [1]

As shown in figure 1.1b), it is a common practice to place a so called anti-aliasing

filter prior to sampling x(t), in order to prevent information contained in frequencies

|f | ≥ 12Ts

from appearing in equation (1.1) during reconstruction. An ideal anti-

aliasing filter has a brick wall frequency response. The real situation is more like the

one shown in figure 1.2 a) where a practical low pass filter, with a smooth transition

between the pass band and the stop band, is used as antialiasing filter. The higher

the order of the low pass filter the steeper the transition but the more complicated is

the design as well. Also the bigger the sampling frequency is than twice the maximum

frequency of the pass band, the more attenuated are the parasitic tones that could

get into the pass band. Figure 1.2b) shows the design trade off between filter order

and the ratio of the sampling frequency to the maximum pass band frequency. In

high speed ADCs, a high sampling frequency might not be an option, leaving no

alternative other than a high order for the low pass filter in such cases.


Figure 1.3: a) Ideal Track & Hold Stage b)Waveforms [1]

1.1.4 Practical Sampling

The block identified as “sampling” in figure 1.1b), is usually implemented via a so

called Track and Hold, T&H, stage. Figure 1.3 shows an ideal T&H stage: during the

track phase the switch is closed so that Vout tracks Vin. When the switch is open the

voltage of the capacitor is held constant. During the hold phase quantization takes

place. Ideally, Ts = Ttrack + Thold with Ttrack = Thold = 0.5Ts. The ideal model is

never encountered in practice. All track and hold circuits suffer from a list of non

idealities:

• Finite acquisition time: this refers to the fact that, in practice, the switch shown

in figure 1.3a) is implemented via a transistor. When the transistor is on, it

has a resistance R greater than zero. Thus, during the track phase, the stage

becomes an RC circuit. In the worst case, the capacitor is initially discharged

and a voltage equal to the FSR is applied in Vin, thus Vout = VFSR(1 − e−tτ )

where τ = RC. Since VFSR = 2nb∆, the time it takes for Vout to be within 0.5∆

of Vin depends on the ADC’s nb. A quick calculation shows that settling Vout

with a 10 bit resolution takes 7.6 time constants τ ; settling Vout with a 14 bit

resolution takes 10.4 time constants τ , and so on. To alleviate this problem, we

would need to make τ = RC small. There is little that can be done to make R

smaller, since it depends on the technology. Making C smaller might or might

not be an option since a smaller C also means a higher electrical noise, as we


will see in the next item.

• KTC

noise: the effect of R being greater than zero also means that it adds a noisy

thermal voltage, generated by the resistor being at a temperature greater than

the absolute zero, to the capacitor during the track phase. It can be shown that

if the number of time constants that we wait to settle is high, such as what is

required for achieving a resolution of 6 bits and higher, the effect on the discrete

time signal that results from measuring the capacitor voltage during the hold

phase is that of adding a discrete zero mean white Gaussian noise of varianceKTC

, where T is the temperature in Kelvin and K the Boltzmann constant. As

pointed out in the previous item, the value of the capacitor C results from a

trade off resolution, speed. A fast T&H stage will require a small capacitor

which in turn will mean higher KTC

noise. In such case the maximum ENOB

achievable in the absence of distortion, dictated by when KTC

becomes as big as

the quantization error, will be limited by thermal noise. On the other hand a

slow ADC allows for a bigger C, which in turns results in a smaller KTC

noise,

allowing for a higher resolution.

• Aperture uncertainty: this is due to the jitter present in the clock that feeds

the switch, which means that the actual moment at which the switch is opened

varies with each sample. A simple way to characterize its effect is to consider

the sampling of a sinusoid A cos(2πfint). If we represent the jitter by the zero

mean random variable τ , and it is small with respect to the sampling period,

we can say that the ∆Vin ≈ dVindtτ . Thus, the energy of the error introduced by

the clock jitter can be approximated by

E[∆V 2

in

]≈ E

[(dVindt

)2

τ 2

]= E

[(dA cos(2πfint)

dt

)2]E[τ 2] ≈ 1

2(2πAfin)2σ2

τ

(1.4)

since the jitter and the signal are statistically independent . We can then define


SNRaperture = 10 log10

(A2

212(2πAfin)2σ2

τ

)= 10 log10

(1

(2πfin)2σ2τ

)(1.5)

Figure 1.4 shows the input frequency fin associated with a level of SNRaperture.

A plot is provided for three different jitter scenarios. If we asume that there is

no source of error other than clock jitter then SNDR = SNRaperture. Under

that assumption the maximum sampling frequency, for a given level of SNDR

or ENOB, is twice fin. The plot highlights the fin associated with an SNDR

of 60 dB, which corresponds to 9.7 ENOB. The maximum sampling frequencies

in this scenario are 31.8 Mhz, 318 Mhz and 3.18 Ghz with 10ps, 1ps and 0.1ps

jitter rms, στ , respectively. Jitter is the non ideality that limits most severely

the ability of designing fast ADCs. Thus, if we are limited by the clock jitter

we will be interested in sampling approaches that use slow ADCs.

Figure 1.4: Maximum Input Signal Frequency vs SNRaperture for Different JitterScenarios [1]

• Signal dependent sampling instant: since the switch is usually implemented


via a transistor, we have to deal with the fact that real transistors become

conductive only when the gate voltage, to which the clock is connected, minus

Vin is greater than a technology dependent threshold voltage. Therefore, the

sampling moment will depend on Vin. This problem can be alleviated if the

clock fall time is much bigger than the maximum dVindt

.

• Track mode nonlinearity: when the transistor becomes conductive it behaves as

a resistor whose resistance depends on Vin. Thus, there is a nonlinear response

added to Vout during track mode. It can be shown that if Vin is a sinusoid

of frequency fin, the nonlinearity caused by this effect is proportional to finfs

,

where fs is the sampling frequency. This non-ideality is usually dealt with using

circuit design techniques.

• Charge injection nonlinearity: when the switch is turned off, the charge stored

in the channel of the transistor doesn’t disappear instantaneously. Part of the

charge ends up in the output capacitor, which adds another nonlinear effect to

Vout. This effect can be mitigated by circuit design techniques as well.

The list of non-idealities presented above is non exhaustive but should give the reader

an idea of the challenges involved in the design of actual ADCs.

1.1.5 Quantization, ADC Architectures, Energy Trade-offs

Once we have designed a Track & Hold stage that meets our specifications, the prob-

lem remains of performing the quantization of the signal in amplitude. The direct

method, implemented by the so called flash ADCs, consists of comparing Vout with

a set of predefined voltage levels through an equal number of voltage comparators.

It is practical only for converters up to 6 bits. For a higher resolution, quantization

is performed in stages: a coarse quantization is performed in the first stage while

later stages perform finer and finer quantizations until the desired resolution level is

reached. The different ADC architectures, pipeline, SAR and cyclic, differ on how

this procedure is implemented.

1.2. REAL LIFE DIGITAL TO ANALOG CONVERSION 9

All architectures mentioned up to this point fall in the category of so called Nyquist

ADCs since they implement the Shannon-Nyquist sampling paradigm. There exists

a different architecture called sigma-delta which works with a different paradigm.

Sigma-delta ADCs are also known as oversampling ADCs because they sample the

signal at rates several times higher, 64 to 128 are typical, than the Nyquist frequency,

although they still output samples at the Nyquist rate. In exchange, they are capable

of very high resolutions, like 16 bits and beyond, with Nyquist sample rates of several

tenths of Mhz possible. A detailed description of each of these architectures is beyond

the scope of this discussion; the interested reader is referred to [1].

Before moving to the next topic the subject of energy dissipation needs to be ad-

dressed since it’s the final item that motivates the research of alternative sampling

paradigms. Generally speaking, the power consumption of an ADC is proportional to

its Nyquist sampling frequency, which means that doubling the sampling frequency

requires doubling the energy dissipated. For resolution, the situation is even worse.

A bit of extra resolution involves doubling, quadrupling or, in some cases such a flash

ADC limited by thermal noise, multiply by eight the power dissipation.

1.2 Real Life Digital to Analog Conversion

Figure 1.5: a) Zero Order Hold b) Effect of Zero Order Hold in the Spectrum of aSignal [1]


Much of the effort of researchers in the field of mixed circuit design focuses on solving

the problems present in practical ADCs because the errors introduced by the sampling

circuit will degrade the signal quality irreparably. On the other hand, in a DAC, the

circuit designer has control over the signal that is generated. That is not to say that

the design of DACs is without challenges but generally speaking ADCs are harder to

design than DACs. I will only discuss here how an actual DAC differs from the ideal

DAC that implements equation (1.1). Figure 1.1b) shows the block diagram of a real

life DAC. As we can see, instead of a train of Dirac deltas, low pass filtered, we have

a three stage process that involves some mechanism to convert a digital sequence to

an electric analog signal, a signal hold, and a reconstruction filter. The typical DAC

converts binary samples to an electrical signal by generating a current, or voltage,

whose value is the analog representation of the sample. If we had the ability to

generate a train of Deltas with the samples, the spectrum of that signal would be,

1

Ts

∞∑n=−∞

X

(f − n

Ts

)

where Ts is the sampling period. Instead, we generate the signal shown in figure 1.5

a), a so called zero order hold approximation in which each sample is held for a time

Tp. The spectrum of the zero order hold approximation is,

TpTs

sinc(fTp)e−jπfTp︸︷︷︸

Amplitude Envelope

∞∑n=−∞

X

(f − n

Ts

)(1.6)

The spectrum that would be obtained by the train of Deltas is here multiplied by a

sinc function in the frequency domain. Figure 1.5b) shows an example of the effect of

this envelope on the overall spectrum of a signal for Tp = Ts. To finalize the digital to

analog conversion process we need, just as in the ideal case involving a train of deltas,

to filter the zero order hold approximation with a low pass filter with pass band equal

to 12Ts

. Unlike the ideal case, this filter not only cannot have a brick wall transfer

function but in addition it needs to correct for the sinc distortion due to the zero

order hold approximation. In terms of practical implementation, the design problem

1.3. INTRODUCTION TO DISCRETE SAMPLING 11

of the reconstruction filter is similar to the design problem of a practical anti-aliasing

filter with the extra complication of requiring a transfer function in the pass band

that compensates for the sinc distortion introduced by the zero order hold.

1.3 Introduction to Discrete Sampling

From what has been presented so far in this chapter, it is clear that a new sampling

paradigm that would involve samplers working at frequencies below the Nyquist rate

in the analog domain would have at its disposal a rich ammunition of circuitry that has

been already developed to work at the Nyquist rate. Discrete sampling fits perfectly

the existing technology for analog to digital conversion. Say that we have a signal

whose bandwidth is 12Ts

; with discrete sampling we could sample it at a rate 1MTs

,

where M is a positive integer, and then digitally interpolate the missing samples.

This would allow the design of an ADC working at a sampling frequency lower than

Nyquist’s but whose output is the signal sampled at the Nyquist rate. Immediate

benefits are the mitigation of the sampling error due to jitter and probably energy

savings due to the ADC working at a lower rate. The final energy balance however will

depend on the complication added in the analog front end necessary to implement

Discrete Sampling. Of course there are many questions that can be raised about

how to accomplish that goal. This thesis is precisely about providing answers to

this proposition. For the purpose of this introductory chapter, I will just give a

summary of the most basic results of Discrete Sampling in a finite dimensional setting.

Understanding how the theory works in this setting will allow the reader to grasp the

mathematical ideas on which Discrete Sampling is based. The following chapters

of the thesis deal with the extension of Discrete Sampling to infinitely long discrete

signals. The reader interested in a detailed exposition of Discrete Sampling in a finite

dimensional setting is referred to the work by William Wu [4].

In the typical Discrete Sampling setting, one is given a finite dimensional space X

of dimension N , typically RN or CN , and a subspace Y ⊂ X, of dimension k, whose

vectors one wants to reconstruct. Discrete Sampling’s goal is to recover a vector v ∈ Ya la Shannon , ie, by measuring v in a number of places of its vector representation,


then interpolating v in those places where v is not measured. Indeed, Shannon’s

sampling theorem tells us that a continuous time, band limited signal of bandwidth 12Ts

can be reconstructed in its entirety by sampling it at regularly spaced time instances.

From the samples taken at times nTs, we can generate the signal at every time instance

through continuous time interpolation with a set of translated sinc functions. Discrete

Sampling takes its name from its interest in reconstructing discrete signals through

discrete interpolation.

Definition 1.1. Given a vector space X of dimension N , a subspace Y ⊂ X of

dimension k < N , and an index set I consisting of k distinct positive integers taken

from the set {1, 2, ..., N}, an Interpolation System is the pair (I,U) where

1. I is the sampling set

2. U is a k-dimensional basis {ui}i∈I, represented in matrix form by a N×k matrix

U, such that ∀ f ∈ Yf =

∑i∈I

f [i]ui. (1.7)

The definition requires that the coordinate of vector f for element ui in basis U be

precisely f [i]. Thus, if we have an interpolation system, we can just measure f [i]i∈I

then recover the full vector f via equation (1.7). Equation (1.7) is an interpolating

equation that imposes on matrix U a very particular structure: the k-dimensional

submatrix made of the rows of U indexed by I is Ik, the k × k identity matrix. We

have the following theorem regarding existence and characterization of interpolation

systems in a finite dimensional space.

Theorem 1.1. Let X, N , Y , k, U and U be as in definition 1.1. Let R be a N × kmatrix whose columns are a basis for Y and ET

I a matrix of 0′s and 1′s that selects

out rows with indexes in I , then there exists U , a matrix whose columns form an

interpolating basis for subspace Y for some sampling set I.

However, matrix U doesn’t necessarily exist for every sampling set. Furthermore,

(I,U) is an interpolation system for Y if and only if

1. ETIR is invertible

1.4. OTHER RESEARCH APPROACHES 13

2.

U = R(ETIR)−1 (1.8)

Equation (1.8) plays a fundamental role in the theory of Discrete Sampling. In fact,

the remaining of the theory of Discrete Sampling for finite dimensional spaces deals

with how equation (1.8) can be applied for particular choices of R. In [4] the cases

where R is a submatrix of the Discrete Fourier Transform matrix and where R is

a submatrix of the Haar wavelet matrix are studied throughly. Another topic of

interest for Discrete Sampling in the finite dimensional setting is finding orthogonal

interpolation systems, i.e., interpolation systems for which the corresponding matrix

U has orthogonal columns. Orthogonality is a desired property for it can be shown

that orthogonal U ’s can be computed in a manner that does not require matrix

inversion as equation (1.8) does for a generic interpolating matrix. Orthogonality also

helps with situations where we have noisy measurements, since the impact of the noise

in the reconstruction can be minimized with orthogonal interpolation matrices. Unlike

non orthogonal interpolation basis, it is not guaranteed that a particular subspace

has an orthogonal interpolation basis. Algorithms to find which subspaces have an

orthogonal interpolation basis for the Discrete Fourier spaces and Haar wavelet spaces

is another contribution of [4].

1.4 Other Research Approaches

The topic of sampling with methods different than the Shannon-Nyquist paradigm has

been the focus of a great deal of research work. During much part of the decades after

Shannon the driver behind research on sampling was mostly intellectual curiosity since

the Shannon paradigm worked very well in practice and existing technology allowed

the Nyquist ADCs to meet the engineering specs. Today however, there is the extra

motivation that existing ADC architectures seem to have reached the point where

miniaturization only provides diminishing returns. As recently as five to ten years

ago, if an engineer wanted to design a lower power, faster and more accurate ADC,

he or she just needed to wait for the technology of the next generation. Scaling made


Figure 1.6: State of the Art in ADC design, 2011 [5]

existing architectures meet the new, more stringent specs. Today the approach of

waiting for the next technology improvement no longer works. Figure 1.6 shows the

performance in terms of bandwidth, defined as fs2

, and resolution for ADCs presented

at the two most prestigious circuits conferences, the International Solid-State Circuits

Conference, ISSCC, and the VLSI Symposium, during the last 15 years. Each point

corresponds to a design presented at the corresponding conference, as indicated in

the legend. Shown also are the theoretical limits imposed by the clock jitter. We see

that the limit has been already reached for a jitter of 1ps rms and that there is not

much room even if jitter can be reduced to 0.1ps rms. Reducing the clock jitter is far

from easy.

Discrete Sampling is just one of the numerous research initiatives in the subject of

sampling. Others include,

• The work of P. P. Vaidyanathan and his coauthors, [6], [7], [8], [9], which relies

heavily in the concepts of multi rate signal processing [10].

• The work of Yonina Eldar and her coauthors, [11], [12], [13], [14] which is

1.4. OTHER RESEARCH APPROACHES 15

based on different ways of modeling the analog signal which results in sampling

architectures that are able to sample data at rates below Nyquist for signals

that meet the analog signal model. Their different proposals fall under the

umbrella of the so called Xampling framework. The most limiting factor of these

approaches is that they require mixing the analog signal with binary patterns

whose clock is higher than the Nyquist rate, thus jitter becomes a problem.

• The approach known as Compressed Sensing, initiated by Candes, Tao and

Donoho [15], [16] but with a very rich set of results by other researchers that

followed such as [17]. In Compressed Sensing the signal is assumed to fit the

so called “sparse model”. This means that there exist some basis in which the

signal representation consists of coefficients that are mostly zero, with a few k

non zero ones. To find those coefficients, Compressed Sensing proposes doing

a set of linear measurements whose number is close to k. Reconstruction is

performed by solving a `1 minimization problem that recovers the k coefficients

from the measured data. Although there is abundant literature in both theory

and algorithms for recovery there are few practical implementations of Com-

pressed Sensing ADCs. The limits in the current chip implementations relate

to jitter, just as in the case of Xampling. The Compressed Sensing ADCs mea-

sure samples at rates below Nyquist but they need to mix the analog signal

with a pseudo random binary sequence whose clock needs to be precise in time.

Such clock works at the Nyquist rate or higher.

• The work by Unser and Aldroubi, [18], [19] which relays in precise models of the

analog signals. The analog signal is sampled after being generated as output

of a custom analog filter, not necessarily band-limited. Part of the recovery is

done in the digital domain via correction with a digital filter. Since the sampled

signals are not necessarily band-limited either, a comparison with the Nyquist

rate is not appropriate. The difficulty of their work is in coming up with signal

models that are good for real signals and whose analog filters models are easy

to invert in the digital domain.

• Theoretical work on signal models that consist of a union of subspaces, such as


[20] by Lu and Do, and [21] by Blumensath and Davies.

• Theoretical work by Walter on sampling wavelet subspaces [22].

• The work of Serdijn and his coauthors on practical implementation of wavelet

samplers for biomedical applications [23],[24].

Chapter 2

Infinite-Dimensional Vector

Spaces, Bases and Frames

In this chapter I give a summary of frame theory, which I will use to extend the

theory of Discrete Sampling to an infinite dimensional space. I assume that the

reader is familiar with the following concepts, which are taught in an undergraduate

level course in analysis: the notion of basis of an infinite dimensional space, operators,

norms, inner products, convergence, Cauchy completeness and Hilbert spaces. A good

reference that describes these topics in detail is [25]. Although frames can be defined

in finite dimensional spaces as well, they become particularly useful when they are

used to analyze infinite dimensional spaces. My exposition is mainly inspired by the

book “Frames and Bases, an Introductory Course” from author Ole Christensen [26],

to which I refer the reader interested in an expanded treatment of the subject.

2.1 Bases in Infinite-Dimensional Hilbert Spaces

2.1.1 Orthonormal Bases

Definition 2.1. Consider a sequence {ek}∞k=1 of vectors in the Hilbert space H.

1. The sequence {ek}∞k=1 is a basis for H if for each f ∈ H, ∃ unique scalar

coefficients {ck(f)}∞k=1 such that f =∑k=∞

k=1 ck(f)ek.

17

18 CHAPTER 2. INTRODUCTION TO FRAME THEORY

2. A basis {ek}∞k=1 is an unconditional basis if∑k=∞

k=1 ck(f)ek converges uncon-

ditionally for every f ∈ H.

3. A basis {ek}∞k=1 is an orthonormal basis if ∀ek, ej ∈ {ek}∞k=1 〈ek, ej〉 = δk,j

with δk,j = 1 if k = j and δk,j = 0 if k 6= j .

We note that the definition is very similar to the definition of basis in a finite dimen-

sional space except that here convergence plays a prominent role. Obviously, we are

primarily interested in unconditional bases. The following three theorems also show

why orthonormal basis are preferred to other unconditional bases.

Theorem 2.1. Consider an orthonormal sequence {ek}∞k=1 of vectors of a Hilbert

space H, the following 6 statements are equivalent:

1. {ek}∞k=1 is an orthonormal basis.

2. f =∑k=∞

k=1 〈f, ek〉ek ∀f ∈ H.

3. 〈f, g〉 =∑k=∞

k=1 〈f, ek〉〈ek, g〉 ∀f, g ∈ H.

4.∑k=∞

k=1 |〈f, ek〉|2 = ||f ||2 ∀f ∈ H. This is called Parseval’s identity.

5. span{ek}∞k=1 = H.

6. if 〈f, ek〉 = 0 ∀k ∈ N, then f = 0.

Theorem 2.2. Consider {ek}∞k=1 an orthonormal basis of Hilbert space H; then, each

f ∈ H has an unconditionally convergent expansion f =∑k=∞

k=1 〈f, ek〉ek.

Theorem 2.3. Every separable Hilbert space H has an orthonormal basis. Separable,

in the context of Hilbert spaces, means that there exists a countable sequence of vectors

{fk}∞k=1 such that span{fk}∞k=1 = H.

To prove Theorem 2.3 we just apply the Gram-Schmidt orthogonalization process to

finite sequences of vectors taken from {fk}∞k=1. Orthonormal bases have the inter-

esting, and useful from an engineering point of view, property that the coordinates

of any vector in f ∈ H can be computed as the inner products of f with elements

2.1. BASES IN INFINITE-DIMENSIONAL HILBERT SPACES 19

of the basis. Parseval’s identity is also very useful since it shows how to compute

the norm of a vector, which in many engineering applications equals the energy of

some signal, from the same coordinates. Inner products can be implemented by linear

filters so the result of this formalism is that the decomposition of infinite duration

signals, or of very long finite duration signals modeled as infinite length signals, can

be performed through filtering followed by sampling. The following theorems give

three other interesting results for orthonormal bases.

Theorem 2.4. Every separable infinite-dimensional Hilbert space H is isometrically

isomorphic to `2(N).

Theorem 2.5. For k ∈ N, let ek be the sequence in `2(N) whose k-th element equals

1 while the rest elements in the sequence are equal to zero.Then {ek}∞k=1 is an or-

thonormal basis of `2(N). It is known as the canonical orthonormal basis of

`2(N).

Theorem 2.6. Let {ek}∞k=1 be an orthonormal basis of a Hilbert space H. All other

orthonormal bases of H are sequences of vectors in the form {Uek}∞k=1 with U : H →H a unitary operator.

Theorem 2.4 will play a very important role in the extension of Discrete Sampling to

infinite dimensional spaces while the last theorem takes us directly to the next topic,

Riesz bases.

2.1.2 Riesz Bases

If orthonormal bases are so great, why bother studying other types of bases? The

main reason is that for a basis to be orthonormal, the requirements are very stringent.

We would like to have bases with characteristics similar to Theorem 2.1.2 and 2.1.4

but easier to build. We begin with a characterization of Riesz bases similar, but

weaker, to Theorem 2.6 for orthonormal bases.

Definition 2.2. Let H be a Hilbert space with an orthonormal basis {ek}∞k=1. A Riesz

basis is a sequence {Uek}∞k=1 where U : H → H is a bounded bijective operator.


We have the following theorem which, among other things, states that a Riesz basis

is actually a basis,

Theorem 2.7. Let {fk}∞k=1 = {Uek}∞k=1 be a Riesz basis in Hilbert space H, then

{fk}∞k=1 is a basis for H and there exists a unique sequence {gk}∞k=1 in H such that

f =∑∞

k=1〈f, gk〉fk unconditionally ∀f ∈ H. Furthermore, the sequence {gk}∞k=1,

gk = (U−1)∗ek, is also a Riesz basis, called the dual basis of {fk}∞k=1.

Since the sequence {gk}∞k=1 in Theorem 2.7 is also a Riesz basis, it makes sense to ask

which is the dual of {gk}∞k=1. It follows immediately that the answer is {fk}∞k=1. For

this reason, the couple {gk}∞k=1,{fk}∞k=1 is called a pair of dual Riesz bases. We have

the following theorem that relates the two.

Theorem 2.8. Let’s {fk}∞k=1 and {gk}∞k=1 be a pair of dual Riesz bases for Hilbert

space H, then

1. {fk}∞k=1 and {gk}∞k=1 are biorthogonal. This means that 〈fk, gj〉 = δk,j.

2. given a Riesz basis {fk}∞k=1 of H, there is a unique set of vectors biorthogonal

to {fk}∞k=1, namely, its dual basis {gk}∞k=1 [27].

3. ∀f ∈ H we have that f =∑∞

k=1〈f, gk〉fk =∑∞

k=1〈f, fk〉gk.

Thus, if we work with a pair of dual Riesz bases we have many of the good things

of orthonormal bases. In general, however, we don’t have an equivalent to Parseval’s

identity. Instead we have the following theorem, which is used in many references

[28] as the definition of a Riesz basis.

Theorem 2.9. Let {fk}∞k=1 = {Uek}∞k=1 be a Riesz basis for Hilbert space H then

there exist constants A,B > 0 such that ∀f ∈ H,

A||f ||2 ≤∞∑k=1

|〈f, fk〉|2 ≤ B||f ||2 (2.1)

The largest possible value for constant A is called the optimal lower bound and equals1

||U−1||2 . The smallest possible value for B is called the optimal upper bound and is

2.2. FRAMES IN INFINITE-DIMENSIONAL HILBERT SPACES 21

equal to ||U ||2. For the dual Riesz basis {gk}∞k=1 = {(U−1)∗ek}∞k=1, if we consider the

optimal bounds A and B, we have,

1

B||f ||2 ≤

∞∑k=1

|〈f, gk〉|2 ≤1

A||f ||2 (2.2)

Theorem 2.9 leads us to the study of frames.

2.2 Frames in Infinite-dimensional Hilbert Spaces

The main feature of a basis in a Hilbert spaceH is that it allows us to uniquely express

every element of f ∈ H as an infinite linear combination of the elements in the basis.

There are situations however, in which we would gladly give up the uniqueness of

the representation if in exchange we get back flexibility about properties of the basis

elements. Frames are generalizations of the concept of basis that provide for that

flexibility. In fact, we will see that just as orthonormal bases can be seen as particular

cases of Riesz bases for which both bounds in equation 2.1 are equal to one, Riesz

bases are particular cases of frames for which we impose that the elements of the frame

be linearly independent. Frames were introduced in 1952 by Duffin and Schaeffer as

a tool to study non uniform sampling. They remained a somehow obscure tool until

the mid 1980s when they started to be used by Daubechies, Mallat and those that

ignited the wavelet revolution [28]. We note that frames are not required to work

with an infinite dimensional space if we have orthonormal or Riesz bases that meet

our needs.

2.2.1 Definition and Properties of Frames

Definition 2.3. Let H 6= {0} be a Hilbert space. A sequence of elements {fk}∞k=1 ∈ His a frame of H if there exists two constants A,B > 0 such that ∀f ∈ H,

A||f ||2 ≤∞∑k=1

|〈f, fk〉|2 ≤ B||f ||2 (2.3)


Equation (2.3) is commonly known as the frame condition. It can be shown that the

frame condition is satisfied for H if it is satisfied for a subspace V ⊂ H which is dense

in H. Equation (2.3) looks very similar to equation (2.1), the main difference is that

the elements of a frame don’t have to be linearly independent. If we consider the

frame operator S, defined below, the infimum over all upper bounds B is the optimal

upper frame bound and equals ||S||. Similarly, the supremum among all lower bounds

A is the optimal lower frame bound and equals ||S−1||−1. If A = B in equation (2.3),

the frame is said to be tight. In a tight frame, equation (2.3) becomes a Parseval-like

identity. From the definition of frame it follows that if {fk}∞k=1 ∈ H is a frame for Hthen span{fk}∞k=1 = H. To see why, compute the orthogonal complement of {fk}∞k=1;

the frame condition imposes that if f is perpendicular to all vectors in the frame,

then f = 0 which is equivalent to say that span{fk}∞k=1 = H. We might be interested

in sequences that are frames of its closed linear span but not necessarily frames of H.

Definition 2.4. Consider a sequence {fk}∞k=1 in a Hilbert space H. We say that this

sequence is a frame sequence if it is a frame for span{fk}∞k=1.

Definition 2.5. Consider {fk}∞k=1, a frame of Hilbert space H; we define the pre-

frame operator or synthesis operator the operator T : `2(N) → H such that

T{ck}k=∞k=1 =

∑k=∞k=1 ckfk. It can be shown that the frame condition makes T bounded.

It can also be shown that the adjoint of T is given by T ∗ : H → `2(N) such that

T ∗f = {〈f, fk〉}∞k=1. T ∗ is called the analysis operator. Composing T and T ∗, we

get the frame operator S : H → H as Sf = TT ∗f =∑∞

k=−1〈f, fk〉fk.

The following theorem lists important properties of S.

Theorem 2.10. Let {fk}∞k=1 be a frame of a Hilbert space H, with frame bounds

A,B, and S its associated frame operator. Then:

1. S converges unconditionally for all f ∈ H.

2. S is bounded, invertible, self-adjoint, and positive.

3. {S−1fk}∞k=1, called the canonical dual frame of {fk}∞k=1, is a frame with

frame operator S−1 and frame bounds 1B, 1A

.


4. If A,B are the optimal frame bounds for frame {fk}∞k=1, then 1B, 1A

are the

optimal frame bounds for {S−1fk}∞k=1.

The reason {S−1fk}∞k=1 is called the canonical dual frame becomes clear in the follow-

ing theorem, which is the most important frame result because it shows how every

element in H admits a representation as an infinite linear combination of the frame

elements.

Theorem 2.11. Let {fk}∞k=1 be a frame of a Hilbert space H with frame operator S.

Then ∀f ∈ H

f =k=∞∑k=1

〈f, S−1fk〉fk (2.4)

and

f =k=∞∑k=1

〈f, fk〉S−1fk (2.5)

Thus {fk}∞k=1 and {S−1fk}∞k=1 give us a way to represent elements of H akin to

the situation with Riesz bases where {fk}∞k=1 plays the role of a Riesz basis and

{S−1fk}∞k=1 plays the role of its dual. For this reason we call {fk}∞k=1 and {S−1fk}∞k=1

a dual frame pair. Unlike the situation with Riesz bases though, here a frame can

have more than one dual frame. However there is an incentive in using the canonical

dual frame. It can be shown that the frame coefficients 〈f, S−1fk〉 have minimal `2

norm among all sequences representing f . The main difficulty of frame theory is that

it requires the computation of the inverse frame operator or at least the effect it has on

all the frame elements fk . A tight frame facilitates things because the canonical dual

frame of a tight frame is {A−1fk}∞k=1. We can even normalize the frame vectors such

that A = 1, in which case the representations of Theorem 2.11 adopts the same form

as the representation with an orthonormal basis. Before we move on, we must notice

that given a sequence of vectors {fk}∞k=1 in an infinite-dimensional Hilbert spaceH the

frame condition is truly necessary to represent every f ∈ H as a linear combination

of vectors {fk}∞k=1. For certain infinite dimensional Hilbert spaces H, it is possible to

have span{fk}∞k=1 = H and find vectors in H that cannot be represented as a linear

combination of vectors {fk}∞k=1 because such vectors don’t satisfy the frame condition.


In contrast, finite-dimensional spaces do have the property that if span{fk}Nk=1 = Hfor some natural number N, then every vector in H can be represented as a linear

combination of vectors in {fk}Nk=1.

2.2.2 Frames and Riesz Bases

From the exposition so far, it is clear that every Riesz basis is a frame. Furthermore,

the Riesz basis bounds are equal to the frame bounds and the dual Riesz basis equals

the canonical dual frame. The frames that are not Riesz bases are called overcomplete

or redundant as shown in the following theorem.

Theorem 2.12. Let {fk}∞k=1 be a frame for Hilbert space H then these two statements

are equivalent:

1. {fk}∞k=1 is a Riesz basis for H

2. If∑∞

k=1 ckfk = 0 for some {ck}∞k=1, then ck = 0 ∀k ∈ N.

Another way of stating this is that if {fk}∞k=1, a frame for H, is not a basis, there

exist {ck}∞k=1 such that∑∞

k=1 ckfk = 0 and not all ck equal zero. Thus, such frame is

composed of vectors that are linearly dependent. The linear dependence of the frame

vectors has several effects in the representation. As we already said above, frames

which are not bases have more than one dual frame. It can also be shown that if

{fk}∞k=1 is a redundant frame for H, then every f ∈ H has several representations

with vectors {fk}∞k=1. With the right normalization of the frame vectors, the frame

bounds tell whether a frame is redundant. If we scale the vectors such that ||fk|| = 1

∀k, then A ≤ 1 ≤ B if the frame is a basis, with A = B = 1 if and only if the frame

is an orthonormal basis. Under such normalization A is greater than one if the frame

is redundant; thus A can be interpreted as the redundancy factor in the normalized

frame.

2.2.3 Frames and Orthogonal Projections

Orthogonal projections play a fundamental role in approximation theory. The orthog-

onality principle states that when we approximate a vector v ∈ H, with an element


v in subspace V ⊂ H such that v /∈ V , the approximation error ||v− v|| is minimized

by picking v ∈ V such that 〈v, v − v〉 = 0. Such v is known as the orthogonal pro-

jection of v onto V because it is orthogonal to the approximation error v − v. When

this problem arises in a finite dimensional setting, the matrix whose columns span

V is full rank, tall and skinny. The orthogonal projection in that case is given by

the Moore-Penrose pseudoinverse of that matrix applied to vector v. In the infinite

dimensional setting, if the subspace V is spanned by a frame we can easily compute

the orthogonal projection, provided that its canonical dual frame is known.

Theorem 2.13. Let {fk}∞k=1 be a sequence of vectors in a Hilbert space H, and P

the orthogonal projection of H onto a closed subspace V ⊂ H, then

1. If {fk}∞k=1 is a frame for H with frame bounds A,B, then {Pfk}∞k=1 is a frame

for V with frame bounds A,B.

2. If {fk}∞k=1 is a frame for V with frame operator S, then the orthogonal projection

of H onto V is given by

Pf =∞∑k=1

〈f, S−1fk〉fk. (2.6)

∀f ∈ H

Theorem 2.13 is valid for all frames, in particular it is true for Riesz bases. Equation

(2.6) is at the heart of the extension of Discrete Sampling to `2(Z).

2.2.4 An Illustrating Example, the PWTsSpace

Consider the Hilbert space composed of real or complex valued square integrable

continuous time signals H = L2(R). With the inner product of f(t), g(t) defined as

〈f(t), g(t)〉 =

∫ +∞

−∞f(t)g(t) dt

we say that f(t) ∈ L2(R) if


〈x(t), x(t)〉 =

∫ +∞

−∞|f(t)|2 dt <∞

We define PWTs ⊂ L2(R) as the space of square integrable, continuous time, band-

limited signals. More precisely, f(t) ∈ PWTs if the support of its Fourier transform

is included in [ −12Ts, 1

2Ts]. PWTs is known as the Paley-Wiener space associated with

bandwidth 12Ts

. PWTs is also a Hilbert space with the same inner product. The

previous results, together with the Shannon sampling theorem, boil down to saying

that { 1√Ts

sinc(t−nTsTs

)}∞−∞ is frame of PWTs with A = B = 1. Not only this frame and

its dual are equal but, as a consequence of having A = 1 and since the vectors of the

frame are normalized to have their norm equal to one, this frame is non redundant; ie,

it’s a Riesz basis of PWTs . It’s also an orthonormal basis of PWTs . Theorem 2.13 tells

us that if we have a signal f(t) ∈ L2(R) which is not band limited, if we compute the

inner products of f(t) with the frame vectors, then use these as coordinates of vectors

{ 1√Ts

sinc(t−nTsTs

)}∞−∞, we get the orthogonal projection of f(t) onto PWTs . This is

equivalent to the standard practice of applying an antialiasing filter of bandwidth 12Ts

prior to sampling a non band limited signal f(t) at rate 1Ts

. The signal reconstructed

through sinc interpolation with these samples is the orthogonal projection of f(t)

onto PWTs .

Chapter 3

Discrete Sampling for Continous

Time, Band-Limited Signals

Discrete Sampling has a great deal of results in the finite dimensional setting, and

all signals we work with in real life are finite dimensional, therefore the question of

why bother with infinite dimensional signals is justified. The answer is that there

are situations in which pretending that our signal is infinitely long allows us to build

simpler models for the signals under consideration. Take for instance an application

that deals with CD quality audio. The application will get 44100 samples per each

second -88200 in case of stereo signals- of sampled signal. Using the finite dimensional

setting for such signals is impractical since we would need to work with very big

matrices, on the order of several million rows, to process signals that are just a few

minutes long. Although one approach could be to break up the signal into smaller

pieces, so that each of which would then be processed independently, there are still

problems. Say, for instance, that you take pieces of 1024 samples; that’s over 40

manipulations of matrices with 1024 rows per second of signal. Even if we manage to

solve the matrix operation problem efficiently with numerical methods adapted to the

matrices in question, there is the additional problem of putting the pieces together

in a meaningful way. It is very unlikely that each of the 1024 long pieces will belong

to the same subspace of R1024; thus, the application would need to work with several

interpolation matrices, have a way to know which one works with a particular block

27

28 CHAPTER 3. DISCRETE SAMPLING FOR SIGNALS IN PWTS

of signal, then put all processed pieces together to get the original long signal. In

digital signal processing a similar situation arises when one tries to use the FFT as

a tool to do filtering of long signals. In the case of the DFT it is possible to process

a very long signal by cleverly overlapping the result of applying the DFT to small

blocks of the long signal; it’s the so called overlap-add method [29]. It is not clear that

a similar approach can be used to extend the finite dimensional methods of Discrete

Sampling to very long signals. This chapter is about using the ideas presented in

Chapter 2 to extend Discrete Sampling to work with infinitely long signals.

3.1 Infinitely Long Discrete Signals in `2(Z)

“the 2WT evenly spaced samples of a signal can be thought of as coordi-

nates of a point in a space of 2WT dimensions. Each particular selection

of these numbers corresponds to a particular point in this space. Thus

there is exactly one point corresponding to each signal in the band W and

with duration T . The number of 2WT dimensions will be, in general, very

high. A 5-Mc television signal lasting for an hour would be represented

by a point in a space with 2 × 5 × 106 × 602 = 3.6 × 1010 dimensions ...

The advantage of this geometrical representation of the signals is that we

can use the vocabulary and the results of geometry in the communication

problem... Passing a signal through an ideal filter corresponds to project-

ing the corresponding point onto a certain region in the space. In fact, in

the frequency-coordinate system those components lying in the pass band

of the filter are retained and those outside are eliminated, so that the

projection is on one of the coordinate lines, planes, or hyperplanes. Any

filter performs a linear operation on the vectors of the space, producing a

new vector linearly related to the old one.”

The above quote is from Shannon’s 1949 paper [2] “Communication in the Presence

of Noise”. The geometric point of view is already used to some extent in the theory

of Discrete Sampling for finite length signals. To deal with the signals like the one

3.1. INFINITELY LONG DISCRETE SIGNALS IN `2(Z) 29

described by Shannon, we can take the idea further by considering that our signals,

while discrete, are infinitely long. From a practical point of view it also makes sense

to focus on signals of finite energy. Mathematically, the Hilbert space H = `2(Z) of

square summable sequences gives us the appropriate ambient space for our signals.

Note that since there is a one to one correspondence between Z and N, we could

consider instead signals in `2(N). With the inner product of two infinite signals

x[n], y[n] defined as,

〈x[n], y[n]〉 =∞∑

n=−∞

x[n]y[n]

we say x[n] ∈ `2(Z) if

〈x[n], x[n]〉 =∞∑

n=−∞

|x[n]|2 <∞

The following theorem, referred to in [27] as the Riesz-Fisher Theorem, allows us to

make a one to one correspondence between `2(Z) and PWTs

Theorem 3.1. Let H be a Hilbert space, {fn}∞n=−∞ an orthonormal basis for H and

{cn}∞n=−∞ a sequence of scalars. Then there exists an f ∈ H such that∑∞

n=−∞ cnfn

converges to f if and only if∑∞

n=−∞ |cn|2 < ∞. Under either condition cn = 〈f, fn〉and there cannot exist g ∈ H, distinct from f , for which cn = 〈g, fn〉.

To see how we can apply the theorem, we remember that the set { 1√Ts

sinc(t−nTsTs

)}∞k=−∞

is an orthonormal basis of PWTs and that the Shannon sampling theorem says that

we can reconstruct any x(t) ∈ PWTs via the sinc interpolation formula

x(t) =∞∑

n=−∞

√Tsx[n]√Ts

sinc

(t− nTsTs

)(3.1)

therefore

• Suppose we have x(t) ∈ PWTs , then by equation (3.1) 〈x(t), 1√Ts

sinc(t−nTsTs

)〉 =

√Tsx[n] with x[n] = x(nTs). By Theorem 3.1, {

√Tsx[n]}∞n=−∞ ∈ `2(Z), thus

{x[n]}∞n=−∞ ∈ `2(Z).


• Conversely, suppose we have {c[n]}∞n=−∞ ∈ `2(Z), thus {√Tsc[n]}∞n=−∞ ∈ `2(Z),

then by Theorem 3.1 there exists a unique x(t) ∈ PWTs such that√Tsc[n] =

〈x(t), 1√T

sinc(t−nTT

)〉 and

x(t) =∞∑

n=−∞

√Tsc[n]√Ts

sinc

(t− nTsTs

)

Since the representation of x(t) in basis { 1√T

sinc(t−nTT

)}∞k=−∞ is unique c[n] =

x[n] = x(nTs).

As a corollary of the above reasoning we find a very important, if surprising, fact

about PWTs, which is that despite being composed of continuous time signals, PWTs

has a discrete nature [30].

Extending Discrete Sampling to work with signals x[n] ∈ `2(Z) will therefore make

the theory of Discrete Sampling work with the class of continuous time band-limited

signals as well. The objective then is to come up with an equation analog to equation

(1.8) so that we can interpolate infinitely long signals out of samples indexed by

some sampling set “smaller” than Z, provided that we know that x[n] belongs to a

subspace of `2(Z). I use the word smaller in between quotes because such sampling

set will likely be, although not necessarily, a countable set whose cardinal is the

same as the cardinal of the integer numbers. This difficulty didn’t exist in the finite

dimensional case. We will see in the next sections that we can greatly simplify the task

by focusing on regular subsampling sets (or schemes) and shift invariant subspaces

of `2(Z). Furthermore, these simplifications will allow us to work with any signal

x[n] ∈ `2(Z).

3.1.1 Equivalence of Continuous Time Inner Products With

Discrete Time Inner Products

The isomorphism between PWTs and `2(Z) implies that inner products of band-

limited continuous time signals in PWTs can be computed from inner products of

discrete time signals. If we take any two band-limited signals x(t) and y(t) we have,

3.2. INTERPOLATION OF SIGNALS IN PWTS 31

〈x(t), y(t)〉 =

〈+−∞∑n1=−∞

x[n1]sinc

(t− n1Ts

Ts

),

+−∞∑n2=−∞

y[n2]sinc

(t− n2Ts

Ts

)〉 =

∫ +∞

−∞{

+−∞∑n1=−∞

x[n1]sinc

(t− n1Ts

Ts

).

+−∞∑n2=−∞

y[n2]sinc

(t− n2Ts

Ts

)} dt =

+∞∑n1=−∞

Tsx[n1]y[n1] = Ts〈x[n], y[n]〉. (3.2)

The penultimate line follows from the orthogonality of the translated sinc functions,

which makes all terms of the double sum of continuous time integrals vanish except

when n1 = n2. When n1 = n2, the continuous time integrals equal Ts, and what

remains is an inner product of discrete sequences.

3.2 Interpolation of Continuous Time, Band-Limited

Signals

Extending Discrete Sampling to `2(Z) can be difficult if we insist in keeping all the

features present in the finite dimensional case, like non regular sampling. By making

the choice of working only with regular sampling schemes we can simplify the task.

This choice is practical in an engineering context.

Consider then the following setup,

• x(t) is a continuous time signal, that belongs to the space PWTs . This is

equivalent to saying that the Fourier transform of x(t), X(f), equals to zero for

|f | > 12Ts

. As an additional technical condition, we require that X(f) 6= 0 for

|f | ≤ 12Ts

. We let B = 1/2Ts and equivalently say that x(t) is a continuous time

band-limited signal whose frequency support is included in [−B,+B].

• x[n] is a discrete sequence such that x[n] = x(nTs); by Theorem 3.1 we have

x[n] ∈ `2(Z). Thus, x[n] is the discrete sequence that results from sampling


x(t) at the Shannon-Nyquist rate.

We have a theorem that tells us when it is possible to recover {x[n]}n∈Z from samples

{x[nM ]}n∈Z

Theorem 3.2. Interpolation of Undersampled Continuous Time Band-Limited Sig-

nals

The following three statements are equivalent

i) There exists a discrete sequence u[n] ∈ `2(Z) and a positive integer M > 1 such

that x[n] can be expressed as

x[n] =+∞∑i=−∞

x[iM ]u[n− iM ] (3.3)

u[n] has the property known in the literature as Nyquist(M) [9], [10], ie, u[0] = 1

and u[n] = 0 for n = ±M,±2M,±3M, . . .

ii) X(z), the z-transform of x[n] satisfies that

|M−1∑k=0

X(ze−j2πMk)| > 0, ∀z = ej2πf (3.4)

iii) There exists a signal u(t) ∈ PWTs such that the function x(t) can be written as

x(t) =+∞∑i=−∞

x[iM ]u(t− iMTs) (3.5)

A key aspect of the theorem is that we are interested in signals that satisfy equation

(3.4). This property guarantees that the DTFT of the discrete signal that results from

downsampling x(t) by a factor M , then upsampling again by a factor M , is non zero

in the range of frequency [−B,+B]. Theorem 3.2 says that such band-limited signals

x(t) can be sampled at rate 1MTs

without loss of information. The reconstruction of

x(t) is performed by first interpolating the samples x[n], out of samples x[nM ], then

applying continuous time sinc interpolation with a low pass filter of bandwidth B; in


fact we will apply its practical version, a zero-order hold followed by the appropriate

low pass / reconstruction filter of bandwidth B. The setup is shown in figure 3.1.

Figure 3.1: Sampling Scheme With Interpolation for Continuous Time, Band-LimitedSignals

Figure 3.2: Magnitude of the Spectrum of cos(2π 132n)

Figure 3.3: Magnitude of the Spectrum of ↑↓ cos(2π 132n)

Proof.

i)⇒ ii) Because x(t) ∈ PWTs , we can write it as,


Figure 3.4: Magnitude of the Spectrum of cos(2π 132n).w[n]

Figure 3.5: Magnitude of the Spectrum of ↑↓ {cos(2π 132n).w[n]}

x(t) =+∞∑

n=−∞

x[n]sinc

(t− nTsTs

)(3.6)

Putting equations (3.3) and (3.6) together we have,

x(t) =+∞∑

n=−∞

+∞∑i=−∞

x[iM ]u[n− iM ]sinc

(t− nTsTs

)=

+∞∑i=−∞

x[iM ]+∞∑

n=−∞

u[n− iM ]sinc

(t− nTsTs

)(3.7)

if we call

u(t) =+∞∑

n=−∞

u[n]sinc

(t− nTsTs

)(3.8)

by Theorem 3.1 u(t) ∈ PWTs , and we can rewrite equation (3.7) as


Figure 3.6: Interpolating Function u[n] for cos(2π 132n).w[n]

x(t) =+∞∑i=−∞

x[iM ]u(t− iMTs) (3.9)

[Note that equation (3.9) is identical to equation (3.5)]. If we further develop

equation (3.9) we get

x(t) =+∞∑i=−∞

x[iM ]u(t− iMTs) = u(t) ∗+∞∑i=−∞

x[iM ]δ(t− iMTs) (3.10)

Since x(t), u(t) ∈ PWTs , their continuous time Fourier transforms are 0 for

|f | > 12Ts

, and we can equivalently work with equation (3.10) in the z domain.

Thus, the right hand side of equation (3.10) is the convolution of u[n] with

a downsampled version, by factor M, then upsampled, again by factor M, of

x[n]. For the purpose of explanation, we can call the latter signal ↑↓ x[n].

In the z domain the convolution becomes a product. Defining X(z) as the z

transform of x[n], downsampling x[n] gives a signal whose z transform is a sum

of delayed, and aliased, versions of X(z). Upsampling this latter signal results

in stretching it in the z domain. Using the z transform operator Z{} we can

show mathematically [29] that Z{↑↓ x[n]} = 1M

∑M−1k=0 X(ze−j

2πMk). Thus, we

have


Figure 3.7: cos(2π 132n).w[n] (red) and its Reconstruction (blue) via Interpolation

With u[n] Shown in Figure 3.6

X(z) = U(z)Z{↑↓ x[n]} = U(z)1

M

M−1∑k=0

X(ze−j2πMk) (3.11)

this gives us

U(z) =X(z)

1M

∑M−1k=0 X(ze−j

2πMk)

(3.12)

which is well defined provided that |∑M−1

k=0 X(ze−j2πMk)| > 0 ∀z = ej2πf . This

condition is precisely equation (3.4). The fact that we required X(f) 6= 0

for |f | ≤ 12Ts

allows us to not consider the case where both numerator and

denominator are zero.

ii)⇒ iii) Suppose that equation (3.4) is true, we can build U(z) as indicated in (3.12).

By construction of U(z), u(t) shown in equation (3.8) is the signal in PWTs

whose samples taken at rate 1Ts

, are u[n]. Going back to the continuous time

domain, u(t) allows us to reconstruct x(t) with equation (3.5).

iii)⇒ i) Suppose (3.5) is true. By assumption, u(t) ∈ PWTs, so it admits an expansion

as shown in equation (3.8). Plugging (3.8) into (3.5) we get (3.7). Since (3.7)


Figure 3.8: Example of a Signal that Doesn’t Satisfy Equation (3.4) with M=8

must be equal to equation (3.6),

+∞∑n=−∞

x[n]sinc

(t− nTsTs

)=

+∞∑n=−∞

{+∞∑i=−∞

x[iM ]u[n− iM ]}sinc

(t− nTsTs

)(3.13)

the expansion of x(t) in basis {sinc(t−kTsTs

)}∞k=−∞ is unique so we have,

x[n] =+∞∑i=−∞

x[iM ]u[n− iM ] (3.14)

which is identical to equation (3.3). This completes the proof of the theorem.

Note that equation (3.12) resembles equation (1.8) in the finite dimensional Discrete


Figure 3.9: Spectrum of ↑↓ {x[n].w[n]} Associated With the Signal Shown in Figure3.8

Sampling case. Intuitively it makes sense that both equations share a similar spirit.

In the finite dimensional setting, the effect of ETI is to keep only certain rows of R

which is analogous to downsampling in the infinite dimensional case. In both cases

the “downsampled” signal is then applied as an inverse to the signal under consider-

ation.

As to how stringent is the requirement imposed by equation (3.4) on real life signals,

the answer is not much. One would expect that the signal x[n] = cos(2π 132n), whose

DTFT magnitude is shown in figure 3.2, would not meet this requirement since its

spectrum is mostly zero. Moreover, figure 3.3 shows the magnitude of the spectrum

of ↑↓ x[n], the signal that results from downsampling, then upsampling, x[n], both by

a factor of M=4. The z transform of ↑↓ x[n] goes into the denominator of equation

(3.12), how can we expect such a signal to give us a valid interpolating sequence?

The answer is that in real life, one never considers signals such as this x[n]. In

practice there is always a window w[n], by default the rectangular window, multiplying

x[n]. We can improve our ability to reconstruct signals such as this cosine by using

other windows. Multiplying x[n] by a 1024 long Hann window [31] results in signals

x[n]w[n] and ↑↓ {x[n].w[n]} whose magnitude spectra are shown in figures 3.4 and 3.5

respectively. Since the Fourier transform of the Hann window is a sinc-like function,

but with a main lobe twice as wide than the sinc function’s, and the multiplication

in the time domain results in convolution in the frequency domain, we clearly see


such functions, in lieu of Delta functions, in these figures. Figure 3.6 shows the

interpolating function associated with x[n]w[n] via equation (3.12) while figure 3.7

compares cos(2π 132n).w[n] with its reconstruction via interpolation with the function

shown in figure 3.6 . We can define

PSNR = 10 log10

(N

max(|x[n]|)2

||x[n]− xrec[n]||2

)(3.15)

where xrec[n] is the reconstructed signal and N the length of x[n] and xrec[n]. In

this example max(|x[n]|)2 = 1 and N = 1024. The reconstruction gives a PSNR

of 74.58dB. This means that the mean squared error is 3.4832e-008 or 1.8663e-004

rms. We notice that cos(2π 132n).w[n] is indistinguishable from its reconstruction in

figure 3.7. In Chapter 4, we will consider examples, derived from wavelet theory, that

behave even better numerically. This same method will give interpolating functions

that achieve an even higher PSNR, showing that perfect reconstruction is not only

achievable in theory but also in practice. These good news don’t mean that (3.4) is

satisfied for all real life signals and all M . Figure 3.8 shows an example of a signal,

obtained from a wavelet decomposition, that doesn’t satisfy equation (3.4) for M

= 8 as the spectrum associated with ↑↓ {x[n].w[n]}, where w[n] is the rectangular

window, shows in figure 3.9. The effect of windowing however means that we will be

able to work with many real life signals that satisfy equation (3.4) even though their

theoretical model might indicate otherwise.

Another simplification in the infinite dimensional setting is to study the case of shift

invariant subspaces. We’ll see in Chapter 4 that we can decompose `2(Z) into a finite

direct sum of discrete shift invariant subspaces, each of which can be analyzed with

the framework presented in subsection 3.2.1 and section 3.3.

3.2.1 Interpolation in Continuous Time, Band-Limited, Shift

Invariant Subspaces

Suppose that x(t) belongs to a shift invariant subspace S ⊂ PWTs, meaning that x(t)

can be expressed in the following way, with s(t) ∈ S,


x(t) =+∞∑

k=−∞

c[k]s(t− kMTs) (3.16)

where M is a positive integer. Note that c[k] are not necessarily samples of x(t). We

are interested in representing x(t) as

x(t) =+∞∑

k=−∞

x(kMTs)u(t− kMTs) (3.17)

where u(t) ∈ S is an interpolating function. There are a few generic results about

this problem in [28], [32]. If s(t) ∈ PWTs, we can get a very simple result when we

apply Theorem 3.2 to s(t).

Theorem 3.3. Suppose x(t) ∈ PWTs admits a representation as in equation (3.16).

Suppose also that S(z), the z transform of the discrete signal that results from sam-

pling s(t) at rate 1Ts

, satisfies equation (3.4). Then, x(t) admits a representation as

in equation (3.17) where u(t) depends on s(t), and is independent of c[k].

Proof. Since s(t) satisfies theorem 3.2, we can define U(z), as well as u(t) ∈ PWTs ,

from S(z),

U(z) =S(z)

1M

∑M−1k=0 S(ze−j

2πMk)

u(t) =+∞∑

n=−∞

u[n]sinc

(t− nTsTs

)s(t) can be then expressed as,

s(t) =+∞∑j=−∞

s[jM ]u(t− jMTs) (3.18)

plugging this expression for s(t) in equation (3.16) we get,


x(t) =+∞∑

k=−∞

c[k]s(t− kMTs) =+∞∑

k=−∞

c[k]+∞∑j=−∞

s[jM ]u(t− kMTs − jMTs) =

+∞∑j=−∞

+∞∑k=−∞

c[k]s[jM ]u(t− kMTs − jMTs) =+∞∑j=−∞

+∞∑i=−∞

c[i− j]s[jM ]u(t− iMTs) =

+∞∑i=−∞

+∞∑j=−∞

c[i− j]s[jM ]u(t− iMTs) =+∞∑i=−∞

b[i]u(t− iMTs) (3.19)

On the right hand side, second line, of equation (3.19), we have made the change of

variable i = k + j; b[i] is defined as

b[i] =+∞∑j=−∞

s[jM ]c[i− j] = s[iM ] ∗ c[i] (3.20)

we can conclude that b[i] = x[iM ] by taking inner products with 1Ts

sinc(t−mMTs

Ts

)on

both sides of equation (3.19),

〈 1

Tssinc

(t−mMTs

Ts

), x(t)〉 = 〈 1

Tssinc

(t−mMTs

sT

),

+∞∑i=−∞

b[i]u(t− iMTs)〉 =

〈 1

Tssinc

(t−mMTs

Ts

),

+∞∑i=−∞

b[i]+∞∑

n=−∞

u[n]sinc

(t− (iM + n)Ts

Ts

)〉 =

+∞∑i=−∞

b[i]+∞∑

n=−∞

u[n]〈 1

Tssinc

(t−mMTs

Ts

), sinc

(t− (iM + n)Ts

Ts

)〉

=+∞∑i=−∞

b[i]u[(m− i)M ] = b[m] (3.21)


In the last line of equation (3.21), we use two properties. First, the orthogonality

of the delayed sinc functions implies that 〈sinc(t−mMTs

Ts

), sinc

(t−(iM+n)Ts

Ts

)〉 = 0

except when iM + n = mM , ie, n = (m − i)M , in which case the inner product is

equal to Ts. Second, since u[n] is Nyquist(M), u[(m − i)M ] = 0 for all (m − i)M

except for (m − i)M = 0 ⇒ m = i, in which case we have u[0] = 1 . Now, since1Ts〈sinc

(t−mMTs

Ts

), x(t)〉 = x[mM ] , we have b[m] = x[mM ].

If we suspect that our signal fits the model (3.16), we just need to sample the output

of the filter whose impulse response is s(t) at rate 1/Ts to get s[n]. If s[n] satisfies

the conditions of theorem 3.2, we build u[n]. The signal x(t), can then be sampled

at rate 1MTs

without loss of information. The sampling of s(t) at rate 1/Ts need not

be physical; it can just be that we have a good theoretical model for our signal from

which we obtain s[n]. From now on, we will call s[n] the atom that generates the

shift invariant subspace while u[n], if it can be defined, will be the interpolation atom

associated with the shift invariant subspace.

3.3 Frame Theory in the Context of Discrete Sam-

pling in Continuous Time Band-Limited Spaces

We now shift gears to study how frame theory can be used to analyze Discrete Sam-

pling in continuous time, band-limited, shift invariant subspaces.

Theorem 3.4. Suppose that we have the setting explained in subsection 3.2.1, that is,

we have signal x(t) ∈ S = span{s(t− kMTs)}k∈Z ⊂ PWTs. In addition assume that

〈s(t−iMTs), s(t−jMTs)〉 = δij, that is, we assume that the vectors {s(t−kMTs)}k∈Zform and orthonormal basis for S. We also assume that s(t) satisfies the conditions

of Theorem 3.3, thus we know that there exists an interpolating function u(t) ∈ S

such that x(t) =∑+∞

i=−∞ x[iM ]u(t− iMT ).

Then {u(t− kMT )}k∈Z is a Riesz basis for S. Furthermore, define c[i− j] = 〈u[n−iM ], u[n− jM ]〉. If c[i− j] is compactly supported, then we can compute explicitly the

optimal frame bounds as well as the dual Riesz basis associated with {u(t−kMTs)}k∈Z.

3.3. FRAMES OF SHIFT INVARIANT SUBSPACES OF PWTS 43

Proof. Consider u[n] = u(nTs) and s[n] = s(nTs). From theorem 3.3, we know that

the following U(z), S(z) and Uf (z) are well defined,

U(z) = S(z)

Uf (z)︷︸︸︷1

1M

∑M−1k=0 S(ze−j

2πMk)

(3.22)

Equation (3.22) can equivalently be expressed in the discrete time domain as u[n] =

s[n] ∗ uf [n], where uf [n] is the sequence whose z transform is Uf (z). By the assump-

tions of this theorem, the operator ∗uf [n] : `2(Z) → `2(Z), such that ∗uf [n]{y[n]} =

y[n] ∗ uf [n] ∀y[n] ∈ `2(Z), is a linear, bounded and bijective operator. Further-

more, one of the properties of the z transform is that if Z{x[n]} = X(z) then

Z{x[n − kM ]} = z−kMX(z); thus, since u[n] = s[n] ∗ uf [n], it follows that Z{u[n −kM ]} = z−kMU(z) = z−kMS(z)Uf (z) = Z{s[n− kM ]}Uf (z). Therefore u[n− kM ] =

s[n− kM ] ∗ uf [n].

By definition 2.2, {u[n − kM ]}k∈Z = {s[n − kM ] ∗ uf [n]}k∈Z is a Riesz basis of

span{s[n− kM ]}k∈Z and therefore {u(t− kMTs)}k∈Z is a Riesz basis of S.

Now working with u[n], define the function c[i − j] = 〈u[n − iM ], u[n − jM ]〉 =

〈u[n], u[n− (i−j)M ]〉 . Since this function depends only on i−j we can call k = i−jand say that c[i, j] = c[i−j] = c[k]. In fact, it follows very quickly that c[k] = ruu[kM ],

ie, c[k] is the downsampled version, by a factor M, of the autocorrelation function of

u[n], ruu[k]. As an immediate consequence, c[k] is symmetric while c[0] = ||u[n]||2.

There are a few steps ahead before we can compute the dual basis of {u(t−kMTs)}k∈Z.

Let’s start by computing the norm of x(t) =∑+∞

i=−∞ x[iM ]u(t− iMTs); we have

||x(t)||2 = 〈x(t), x(t)〉 =+∞∑i=−∞

x[iM ]〈x(t), u(t− iMTs)〉 =

+∞∑i=−∞

x[iM ]+∞∑j=−∞

Tsx[jM ]c[i− j] = Ts

+∞∑i=−∞

+∞∑j=−∞

x[iM ]x[jM ]c[i− j] > 0 ∀x(t) 6= 0

(3.23)


Given that,

〈x(t), u(t− jMTs)〉 = 〈+∞∑i=−∞

x[iM ]u(t− jMTs), u(t− iMTs)〉 =

+∞∑i=−∞

x[iM ]〈u(t− jMTs), u(t− iMTs)〉 =+∞∑i=−∞

x[iM ]Tsc[i− j]

An interesting consequence of equation (3.23) is that ||x(t)|| is entirely determined

by samples x[mM ]. Since x[n] ∈ `2(Z) ⇒ x[nM ] ∈ `2(Z). In addition, since ||x(t)||is well defined, given that x(t) is square integrable, equation (3.23) means that the

sequence c[i− j] is positive definite. If c[i− j] is compactly supported, meaning that

c[k] = 0 for |k| > N , equation (3.23) has two important consequences. First, we can

define a (2N + 1) × (2N + 1) Toeplitz matrix C, with Ci,j = c[i − j], which is also

positive definite, that will play an important role when we compute the dual basis.

The second important implication is that the DTFT of c[k], C(ejω), gives us the upper

and lower bounds for the eigenvalues of the banded matrix C with Ci,j = c[i− j]i,j∈Z.

If we call λC,min > 0 the minimum over ω of |C(ejω)| and λC,max > 0 the maximum

over ω of |C(ejω)| we get that the positive definiteness of c[k] gives [33],

λC,minTs||x[nM ]||2 ≤ ||x(t)||2 ≤ λC,maxTs||x[nM ]||2 (3.24)

We can then derive the frame condition in the following way,

|〈x(t), u(t− lMTs)〉|2 = T 2s

+∞∑i=−∞

+∞∑j=−∞

x[iM ]x[jM ]c[j − l]c[i− l] > 0⇒


+∞∑l=−∞

|〈x(t), u(t− lMTs)〉|2 = T 2s

+∞∑i=−∞

+∞∑j=−∞

x[iM ]x[jM ]+∞∑l=−∞

c[j − l]c[i− l] > 0

(3.25)

Let’s define d[i− j] as

d[i− j] = d[i, j] =+∞∑l=−∞

c[l − i]c[l − j] (3.26)

Since d[i − j] depends only on k = i − j, we can say d[i − j] = d[k]. It follows that

d[k] is the autocorrelation sequence of c[k]. The frame condition, equation (3.25),

implies that d[k] is a positive definite sequence. If c[k] is compactly supported, then

d[i − j] is also compactly supported. Consider the DTFT of d[k], D(ejω) and the

banded matrix D with Di,j = d[i − j]i,j∈Z. If we call λD,min > 0 the minimum over

ω of |D(ejω)| and λD,max > 0 the maximum over ω of |D(ejω)| we get that the frame

condition is equivalent to,

λD,minT2s ||x[nM ]||2 ≤

+∞∑k=−∞

|〈x(t), u(t− kMT )〉|2 ≤ λD,maxT2s ||x[nM ]||2

combined with the inequality shown in equation (3.24) gives,

TsλD,minλC,max

||x(t)||2 ≤+∞∑

k=−∞

|〈x(t), u(t− kMT )〉|2 ≤ TsλD,maxλC,min

||x(t)||2 (3.27)

Thus, the optimal frame bounds are A = TsλD,minλC,max

, B = TsλD,maxλC,min

.


3.3.1 Orthogonal Interpolating Atoms

With this framework, the case of orthogonal interpolating atoms boils down to c[i−j] = c[0]δ[i − j] with c[0] = ||u[n]||2. It follows very quickly that in this case, A =

B = Tsc[0], since λD,min = λD,max = c[0]2 and λC,min = λC,max = c[0]. In turn, this

means that ||x(t)||2 is Tsc[0] = Ts||u[n]|2 multiplied by ||x[nM ]||2, the downsampled

version of x[n]. In addition, the dual basis of {u[n−kM ]}k∈Z is {u[n−kM ]}k∈Z with

u[n− iM ] = u[n− iM ]/||u[n]||2 = u[n− iM ]/c[0] .

3.3.2 Computing the Dual Riesz Basis For Compactly Sup-

ported c[k]

Frame theory provides for the existence of the dual frame but it is less generous with

algorithms to compute the dual frame in the general setting. Fortunately, because of

the particularities of interpolating frames in `2(Z), if we make the extra assumptions

that c[k] is compactly supported, and that {u[n − iM ]}k∈Z is a Riesz basis, there is

a very straightforward algorithm that gives us the dual frame associated with {u[n−iM ]}k∈Z. These two assumptions are by no means restrictive since the most widely

studied and understood shift invariant subspaces come from Wavelet Theory, which

provides several families of shift invariant subspaces whose generators are compactly

supported and satisfy the requisites of Theorem 3.4. To facilitate the manipulation,

let’s assume that c[k] is 2N + 1 long. Suppose that the dual frame is in the form

{u[n−iM ]}k∈Z, which is a reasonable assumption to make that will be later confirmed

by the result. We know that u[n] ∈ span{u[n− iM ]}, thus we can write

u[n] =+∞∑i=−∞

u[iM ]u[n− iM ]

But we also have that a Riesz basis and its dual must satisfy the biorthogonality

condition, thus

〈u[n], u[n−mM ]〉 = 〈+∞∑i=−∞

u[iM ]u[n− iM ], u[n−mM ]〉 = δ(m)


−∞∑i=−∞

u[iM ]〈u[n− iM ], u[n−mM ]〉 =m+N∑i=m−N

u[iM ]c[i−m] = δ(m)

These equations give us the perfect way to estimate the 2N+1 coefficients u[iM ]Nk=−N .

If we rescue the (2N + 1) × (2N + 1) Toeplitz matrix C, such that Ci,j = c[i − j],then we can create the following system of equations,

Cu = b

Where b is a (2N + 1)x1 vector for which bN+1,1 = 1 but bi,1 = 0 for any other i. b

is basically a discrete Delta function. Finally, u is a (2N + 1)x1 vector for which we

want ui,1 = u[(i− (N + 1))M ] . Since C is positive definite, it is also full rank, thus

u = (C)−1b

And with that we get the generator of the dual frame, u[n], which in general is not

an interpolating atom unless {u[n− iM ]}k∈Z are orthogonal,

u[n] =N∑

i=−N

u[iM ]u[n− iM ] (3.28)

From u[n] we get the dual frame as {u[n − iM ]}k∈Z. By construction, the vectors

{u[n − iM ]}k∈Z satisfy the biorthogonality condition 〈u[n], u[n − mM ]〉 = 〈u[n +

mM ], u[n]〉 = δ(m). Thus, since Theorem 2.8 says that the dual of a Riesz basis is

the unique set of vectors whose vectors are biorthogonal to the vectors of the original

Riesz basis, {u[n − iM ]}k∈Z is the dual Riesz basis of {u[n − iM ]}k∈Z. Finally, we

can build u(t) as well,

u(t) =+∞∑

n=−∞

u[n]sinc

(t− nTsTs

)


Mallat’s book [28] reports a general version of this result in its Theorem 7.20 by

starting with a general setup in which φ(t) is defined as an interpolating continuous

time function, not necessarily band-limited, with the interpolation interval normalized

to 1. That result, attributed to Aldroubi and Unser, states that if Φ(ω) is the Fourier

transform of φ(t) and Φ(ω) is the Fourier transform of its dual φ(t), the continuous

time interpolating function and its dual are related by the expression,

Φ(ω) =Φ(ω)∑+∞

k=−∞ |Φ(ω + 2kπ)|2

which is more complicated than the method showed here for band-limited shift in-

variant spaces.

3.3.3 Numerical Example

To illustrate the concepts described in this chapter, let’s consider the shift invariant

subspace spanned by φ12, taken from the wavelet decomposition Symmlet 8 J=12,

L=10, with M = 4. Chapter 4, subsection 4.4.1, will describe in more detail what

this means; for the purpose of this example it is enough to think of φ12 as a subspace

generator that satisfies the requisites of Theorem 3.4 with M = 4, including that

the set {φ12[n − 4k]}k∈Z is an orthonormal basis for the subspace. Its associated

interpolating atom is shown in table 3.1. In this example, u[n] is truncated to have

length equal to 33.

u[−16] = −7.869 · 10−18 u[−15] = 0.005092 u[−14] = 0.008966 u[−13] = 0.006037u[−12] = 3.572 · 10−17 u[−11] = −0.01098 u[−10] = −0.01841 u[−9] = −0.005762u[−8] = −3.954 · 10−17 u[−7] = −0.01938 u[−6] = −0.04299 u[−5] = −0.06309u[−4] = 2.223 · 10−16 u[−3] = 0.2185 u[−2] = 0.5197 u[−1] = 0.8244

u[0] = 1.0 u[1] = 0.9127 u[2] = 0.6399 u[3] = 0.2976u[4] = −3.539 · 10−16 u[5] = −0.1278 u[6] = −0.1232 u[7] = −0.06625u[8] = 7.408 · 10−17 u[9] = 0.02373 u[10] = 0.01503 u[11] = 0.006445u[12] = 4.235 · 10−17 u[13] = −0.0007221 u[14] = 0.00238 u[15] = 0.001992u[16] = 6.069 · 10−17

Table 3.1: u[n] Associated with φ12, Wavelet Decomposition Symmlet 8 J=12, L=10;see figure 4.6

We can then compute the sequence c[k] associated with u[n]. It turns out that not


only c[k] is compactly supported, as expected, in addition, we can get a very good

numerical result by assuming that all coefficients c[k] such that |k| ≥ 5 are zero.

Thus, we can work with the c[k] shown in table 3.2. Inspecting the table, we also

verify that c[k] is symmetric and that c[0] = ||u[n]||2.

c[−4] = 0.01744 c[−3] = 0.01895 c[−2] = −0.1952 c[−1] = 0.4793c[0] = 3.372 c[1] = 0.4793 c[2] = −0.1952 c[3] = 0.01895c[4] = 0.01744

Table 3.2: c[k] Associated with u[n] Shown in Table 3.1

We can easily compute matrix C, 3.372 0.4793 −0.1952 0.01895 0.01744 0 0 0 00.4793 3.372 0.4793 −0.1952 0.01895 0.01744 0 0 0

−0.1952 0.4793 3.372 0.4793 −0.1952 0.01895 0.01744 0 00.01895 −0.1952 0.4793 3.372 0.4793 −0.1952 0.01895 0.01744 00.01744 0.01895 −0.1952 0.4793 3.372 0.4793 −0.1952 0.01895 0.01744

0 0.01744 0.01895 −0.1952 0.4793 3.372 0.4793 −0.1952 0.018950 0 0.01744 0.01895 −0.1952 0.4793 3.372 0.4793 −0.19520 0 0 0.01744 0.01895 −0.1952 0.4793 3.372 0.47930 0 0 0 0.01744 0.01895 −0.1952 0.4793 3.372

as well as it inverse C−1,

0.3048 −0.04747 0.02567 −0.008352 0.001454 −0.000602 8.583 · 10−5 −1.239 · 10−5 2.592 · 10−6

−0.04747 0.3122 −0.05147 0.02697 −0.008579 0.001548 −0.0006153 8.754 · 10−5 −1.239 · 10−5

0.02567 −0.05147 0.3144 −0.05217 0.02709 −0.00863 0.001555 −0.0006153 8.583 · 10−5

−0.008352 0.02697 −0.05217 0.3146 −0.05221 0.02711 −0.00863 0.001548 −0.0006020.001454 −0.008579 0.02709 −0.05221 0.3146 −0.05221 0.02709 −0.008579 0.001454

−0.000602 0.001548 −0.00863 0.02711 −0.05221 0.3146 −0.05217 0.02697 −0.008352

8.583 · 10−5 −0.0006153 0.001555 −0.00863 0.02709 −0.05217 0.3144 −0.05147 0.02567

−1.239 · 10−5 8.754 · 10−5 −0.0006153 0.001548 −0.008579 0.02697 −0.05147 0.3122 −0.04747

2.592 · 10−6 −1.239 · 10−5 8.583 · 10−5 −0.000602 0.001454 −0.008352 0.02567 −0.04747 0.3048

The middle column of C−1 are the coefficients u[nM ] which are shown in table 3.3.

u[−16] = 0.001454 u[−12] = −0.008579 u[−8] = 0.02709 u[−4] = −0.05221u[0] = 0.3146 u[4] = −0.05221 u[8] = 0.02709 u[12] = −0.008579

u[16] = 0.001454

Table 3.3: Coefficients u[nM ] computed from c[k] shown in Table 3.2

With these u[nM ] we can construct u[n] via interpolation with atom u[n] as shown in

equation (3.28); the result is in table 3.4. The plots of both u[n] and u[n] are shown

in figure 4.7.

Finally, we can compute the d[k] sequence associated with c[k], which is shown in

table 3.5; d[k] = 0 for |k| ≥ 3.

Using Theorem 3.4, via the DTFTs of sequences c[k] and d[k], we can compute

λC,max = 4.0129, λC,min = 2.0199, λD,max = 12.2556 and λD,min = 11.5600. Therefore,

the optimal frame bounds for the continuous time frame {u(t− k4Ts)}k∈Z associated


u[−16] = 0.001454 u[−15] = 0.001103 u[−14] = −0.0009104 u[−13] = −0.006149u[−12] = −0.008579 u[−11] = −0.004804 u[−10] = 0.004396 u[−9] = 0.02085u[−8] = 0.02709 u[−7] = 0.009068 u[−6] = −0.02104 u[−5] = −0.05379u[−4] = −0.05221 u[−3] = 0.01809 u[−2] = 0.1283 u[−1] = 0.2451u[0] = 0.3146 u[1] = 0.2826 u[2] = 0.18 u[3] = 0.05255u[4] = −0.05221 u[5] = −0.08306 u[6] = −0.05845 u[7] = −0.01379u[8] = 0.02709 u[9] = 0.037 u[10] = 0.02385 u[11] = 0.006281

u[12] = −0.008579 u[13] = −0.01244 u[14] = −0.008106 u[15] = −0.002859u[16] = 0.001454

Table 3.4: u[n] Associated With u[n] Shown in Table 3.1

d[−2] = 0.0003042 d[−1] = 0.1739 d[0] = 11.91 d[1] = 0.1739 d[2] = 0.0003042

Table 3.5: d[k] Associated with c[k] Shown in Table 3.2

with discrete frame {u[n− 4k]}k∈Z are,

A = TsλD,minλC,max

= 2.8807Ts

B = TsλD,maxλC,min

= 6.0674Ts

3.3.4 Remarks

Figure 3.10: Illustration of Shift Invariant Subspace Sampling When {u[n− iM ]}k∈Zis a Riesz Basis

Frame theory brings a new perspective to Discrete Sampling in an infinite dimensional

setting. If the signal x[n] that we are interested in recovering lies in S, a shift-invariant


subspace of `2(Z) generated by shifts of s[n] ∈ S, for which {s[n− kM ]}k∈Z form an

orthonormal basis of S, and s[n] satisfies equation (3.4), we can explicitly find an

interpolating basis for the subspace which is also shift invariant, {u[n − iM ]}k∈Z;

this interpolation basis is a Riesz basis for S. In such a basis, the coordinates of

x[n] are given by the samples x[nM ]. Thus, we just need to measure the signal at

times x[nM ], then build x[n] via interpolation. The samples x[nM ] happen to be as

well the inner products of x[n] with the dual Riesz basis {u[n − iM ]}k∈Z. The dual

Riesz basis plays an even more important role if x[n] doesn’t lie in S. In that case

taking the samples x[nM ] and interpolating with basis {u[n − iM ]}k∈Z will give us

a signal in S which might or might not be close to x[n]. Theorem 2.13 gives us a

better alternative. By considering the inner products 〈x[n], u[n− iM ]〉k∈Z, instead of

x[nM ], as the coordinates in basis {u[n− iM ]}k∈Z, we get the orthogonal projection

of x[n] onto S, x[n]. By the orthogonality principle, the signal reconstructed, x[n], is

the signal in S which is closest to x[n]. These concepts are illustrated in figure 3.10.

Chapter 4

A New Method to Design Hybrid

Filter Banks

The framework presented in Chapter 3 has a very direct application to the problem

of designing Hybrid Filter Banks, also known as HBF’s. Figure 4.1 shows the block

diagram of an HFB. The basic idea is to use M analog filters, instead of a single an-

tialiasing filter, to process an analog signal of bandwidth 12Ts

prior to taking samples.

Sampling is performed through M Analog to Digital Converters, ADC, working at a

rate of 1TsM

, ie, M times below the Nyquist rate. Each ADC samples one of the filter

outputs. The signal is then recovered via digital filters that process the respective

ADC outputs, upsampled by factor M. The output of the HBF is the sum of the out-

puts of the digital filters; the objective in HBF design is that the output is a delayed

version of the input analog signal sampled at rate 1Ts

. Even though perfect recovery

in a HBF setting requires to collect as many data points as in Shannon sampling,

the fact that the ADC’s work at a lower rate has important practical consequences,

as explained in Chapter 1. In the same chapter we mentioned that except for the

so called sigma delta ADCs, which achieve high resolution at the expense of reduced

sampling rates and a complicated analog design, the architecture, from a paradigm

point of view, of most ADCs has remained unchanged over the last 60 years. HBFs

present an alternative paradigm although nonidealities have prevented the concept of

HBF ADCs from becoming mainstream.

53

54 CHAPTER 4. DISCRETE SAMPLING AND HBF’S

Figure 4.1: Hybrid Filter Bank Architecture, Adapted from [34]

This chapter is about designing HBFs with the tools developed in Chapter 3. In

addition to showing that Discrete Sampling for continuous time, band-limited signals

fits perfectly the problem of designing HBFs, although not every HBF might be

designed with Discrete Sampling, we will study the restrictions imposed on a practical

implementation of an HBF designed with this technique as well as how the noise

present at the output of each analog filter propagates to the HBF output. Examples

will be derived along the way to illustrate each of these items.

4.1 A Bit on History of HBF’s

That one could use a multichannel analog front end to sample signals with converters

working below the Nyquist rate was already suggested by Papoulis in 1977 [35][36].

Papoulis proposed that the reconstruction be analog as well. The work of Velazquez et

al.[37] is considered the first to suggest a hybrid approach. More recent work on both

design and implementation of HBFs can be found here [34][38] and here [39]. Going

4.2. UNION OF ORTHOGONAL SI SUBSPACES WITH ON GENERATORS 55

back to figure 4.1, these works see the problem of designing an HBF as one in which

one has to compute the coefficients of H1(s) to HM(s), a family of M continuous time

filters, and the coefficients of F1(z) to FM(z), a family of M discrete time filters. The

filters H1(s) to HM(s) are known as the analysis filters. The filters F1(z) to FM(z) are

known as the synthesis filters. The objective of the design problem is to pick the filter

coefficients so that for any signal x(t) with bandwidth 12Ts

, y[n] is a delayed version

of x[n] = x(nTs), property known as perfect reconstruction in the literature. The

problem is usually formulated in terms of a numerical optimization. Theoretical

analysis on the filter bank is performed using classic Fourier theory applied to the

output of the synthesis filters. In particular, aliasing terms, due to the samplers

working at 1MTs

instead of the Nyquist rate, are analytically identified at the output

of the HBF y[n]; the designer attempts to drive to zero these components. Once

the aliasing terms are minimized, the remaining constraint is that the bank achieves

perfect reconstruction, or quasi perfect recontruction if some error margin is allowed.

With Discrete Sampling extended to `2(Z), we introduce a novel design method that

might help HBFs become more widely used. It is important to note that the method

we are about to present doesn’t work for every HBF, but given its generality, it’s a

step in the direction of making HBF’s more practical.

4.2 Union of Mutually Orthogonal Shift Invariant

Subspaces With Orthonormal Generators

Let’s use the ideas of Discrete Sampling to introduce a new point of view to the

problem of HBF design. Suppose that it is possible to decompose `2(Z) as `2(Z) =⊕Q−1q=0 span{sq[n − lMq]}l∈Z, with Q a positive integer and {sq[n − lMq]}l∈Z an or-

thonormal basis for the subspace spanned by these vectors; equivalently, suppose that

for any x[n] ∈ l2(Z) we have

x[n] =

Q−1∑q=0

+∞∑l=−∞

〈x[n], sq[n− lMq]〉sq[n− lMq] (4.1)


with 〈sq1[n − l1Mq1], sq2[n − l2Mq2]〉 = 0 ∀q1, q2, l1, l2 ∈ Z, except when q1 = q2 and

l1 = l2, in which case it is equal to 1. Suppose further that the z transform of sq[n]

satisfies equation (3.4), thus we can define uq[n] as the sequence whose z transform is

Uq(z) =Sq(z)

1Mq

∑Mq−1k=0 Sq(ze

−j 2πMk)

then, from the framework developed in Chapter 3 it follows that we can write x[n] as

x[n] =

Q∑q=0

+∞∑i=−∞

〈x[n], uq[n−Mqi]〉uq[n−Mqi]

where uq[n] is the dual of interpolating atom uq[n]. We also know that for any

x(t) ∈ PWTs so that x(nTs) = x[n], 〈x[n], uq[n − Mqi]〉 = 1Ts〈x(t), uq(t − MqTsi)〉

where uq(t) =∑∞

n=−∞ uq[n]sinc( t−nTsTs

). Now, 〈x(t), uq(t−MqTsi)〉 =∫ +∞−∞ x(t)uq(t−

MqTsi)dt =∫ +∞−∞ x(t)uq(−(MqTsi − t))dt = x(t) ∗ uq(−t)(MqTsi). Thus, to get

〈x(t), uq(t−MqTsi)〉 we need to sample an analog filter that implements uq(−t) at rate1

MqTswhen the input of the filter is x(t). If all Mq are equal to M , we have an HFB as

shown in figure 4.1. Discrete Sampling gives us a method to design a particular class

of HBFs, provided that there exists a decomposition of x[n] ∈ l2(Z) as dictated by

equation (4.1). Wavelet Theory gives us a way to do precisely that. In fact, Wavelet

Theory adds the flexibility that all Mq don’t have to be necessarily equal although

they have to be powers of 2. With Discrete Sampling the only difficulty that remains

for having an HFB is the implementation of a set of analog filters, a topic that will

be dealt with in section 4.6.

4.3 Very Brief Overview of Wavelet Theory

I am going to state without proof the main results of Wavelet Theory relevant to

Discrete Sampling. There are several good references on this topic. What I state here

is mostly taken from Stephane Mallat’s book “A Wavelet Tour of Signal Processing”,

Second Edition [28]. Wavelet Theory came about as a tool to represent signals in

4.3. VERY BRIEF OVERVIEW OF WAVELET THEORY 57

L2(R) and l2(Z) in ways that overcame the limitations of the continuous and dis-

crete Fourier Transforms. The Fourier framework is particularly bad for representing

transient signals since it doesn’t represent time location well. Intuitively, the Fourier

framework decomposes signals as a sum of sinusoids. A sinusoid goes from t = −∞ to

t = +∞, or from n = −∞ to n = +∞, oscillating between +1 and −1. A transient

signal, of limited duration, needs a large number of Fourier coefficients because a few

of them would most likely result in infinite duration signals: if you have one Fourier

coefficient you have a single sinusoid, which is of infinite duration; two coefficients

give you a signal which is also of infinite duration with more room to cancel parts of

the signal. Adding coefficients gives you extra degrees of freedom until one can finally

pull off the limited duration signals. There are other ways to think about this, of

course, for instance, that the Fourier transform of a Delta function is a flat spectrum

and that the inverse Fourier transform of a Delta function is a constant signal, but

the conclusion is the same: Fourier analysis doesn’t result in sparse representations

for transient signals. Since the beginning of the XX-th century, mathematicians,

engineers and scientists have tried to come up with alternative representations that

fit particular needs. During the 1980s, Mallat, Daubechies and others unified the

different approaches giving birth to what is known as Wavelet Theory.

4.3.1 Multiresolution Representations

Wavelet bases result from multiresolution decompositions of L2(R). These decompo-

sitions can be discretized to have decompositions of `2(Z). A multiresolution approx-

imation, or decomposition, is a sequence of closed subspaces {Vj}j∈Z of L2(R) that

satisfies the following 6 properties

1. ∀(j, k) ∈ Z2, f(t) ∈ Vj ⇔ f(t− 2jk) ∈ V j

2. ∀j ∈ Z, Vj+1 ⊂ Vj

3. ∀j ∈ Z, f(t) ∈ Vj ⇔ f( t2) ∈ Vj+1

4. limj→+∞ Vj = ∩+∞j=−∞Vj = {0}


5. limj→−∞ Vj = ∪+∞j=−∞Vj = L2(R)

6. There exists θ such that {θ(t− n)}n∈Z is a Riesz basis of V0

If one thinks about it, and considers point 6 together with points 1, 2 and 3, it’s not

difficult to realize that these multiresolution approximations are sequences of shift

invariant subspaces so that any signal f(t) ∈ L2(R) can be approximated to the

required resolution by projecting it onto the appropriate subspace Vj. The nomencla-

ture used calls 2j the scale parameter while it calls its inverse 2−j the resolution. So,

high resolution means low scale parameter while coarse resolution means high scale

parameter. Perhaps, the most intuitive example of a multiresolution approximation

is the sequence of band-limited subspaces. If one defines Vj as the set of functions

whose Fourier transform has a frequency support included in [− 12j+1 ,

12j+1 ], one gets

a multiresolution approximation. In fact, we see that in this case Vj = PW2j . For

any signal f(t) ∈ L2(R), its orthogonal projection on Vj is the signal whose Fourier

transform in [− 12j+1 ,

12j+1 ] equals the Fourier transform of f(t) but it’s zero elsewhere,

ie, it’s f(t) lowpass filtered in the band [− 12j+1 ,

12j+1 ]. Finally, in this approximation,

θ(t) = sinc(t), so V0 = span{sinc(t−n)}n∈Z. In other words, the space of band-limited

signals of support [−0.5,+0.5], PW1, is V0.

4.3.2 A Multiresolution Approximation is a Collection of

Shift Invariant Subspaces

The following result formalizes what was stated previously.

Theorem 4.1. Suppose we are given a multiresolution approximation {Vj}j∈Z. With

Θ(ω), the Fourier transform of θ(t), we can build the function φ(t) whose Fourier

transform is

Φ(ω) =Θ(ω)√∑+∞

k=−∞ |Θ(ω + 2kπ)|2

Define the set of functions


φj,n(t) =1√2jφ

(t− 2jn

2j

)(n,j)∈Z2

Then, the family φj,n(t) is an orthonormal basis of Vj for all j ∈ Z.

The bottom line is that with this framework we can approximate any f(t) ∈ L2(R) to

any resolution we want by computing the orthogonal projection of f(t) onto Vj; since

we have an orthonormal, shift invariant basis for each subspace Vj, the approxima-

tion of f(t) in Vj is fj(t) =∑∞

n=−∞〈f(t), 1√2jφ(t−2jn

2j

)〉 1√

2jφ(t−2jn

2j

). The function

so defined φ(t) is called the scaling function. In the case of the band-limited mul-

tiresolution approximation, φ(t) = θ(t).

4.3.3 Conjugate Mirror Filters

The multiresolution approximations property 2 implies interesting things about the

scaling function, in particular that 2−1/2φ(t/2) ∈ V1 ⊂ V0, which translates into

1√2φ

(t

2

)=

+∞∑n=−∞

h[n]φ(t− n)

With

h[n] = 〈 1√2φ

(t

2

), φ(t− n)〉

h[n] can be thought of as a discrete time filter. It turns out that this filter plays

a very important role in the whole process. Mallat, in addition to introducing the

concept of multiresolution approximation proved the following theorem.

Theorem 4.2. Suppose we are given an integrable scaling function φ(t) ∈ L2(R),

then the Discrete Time Fourier Transform of h[n], H(ω) satisfies that

∀ω ∈ R, |H(ω)|2 + |H(ω + π)|2 = 2


with H(0) =√

2. The filters that satisfy these two conditions are called conjugate

mirror filters and were known independently of wavelet theorists. Conversely, sup-

pose we are given a function H(ω), 2π periodic, and continuously differentiable in

a neighborhood of ω = 0, if ∀ω ∈ R, |H(ω)|2 + |H(ω + π)|2 = 2, H(0) =√

2 and

infω∈[−π2,π2

] |H(ω)| > 0, then we can build the Fourier transform Φ(ω) of a scaling

function φ(t) as

Φ(ω) =+∞∏p=1

H(2−pω)√2

4.3.4 Wavelet Decomposition

Since Vj ⊂ Vj−1, let’s define Wj as the orthogonal complement of Vj in Vj−1. Thus

Vj−1 = V j⊕

Wj. Wj is the so called “wavelet space” at resolution 2−j. Wj

contains the information that is lost when a signal in subspace j− 1 is approximated

by its orthogonal projection in subspace j. We have the following result.

Theorem 4.3. Suppose we are given a scaling function φ(t) and its associated con-

jugate mirror filter, h[n]. Let ψ(t) be a function whose Fourier transform, Ψ(ω), is

defined as,

Ψ(ω) =1√2G(ω

2

)Φ(ω

2

)

With,

G(ω) = e−iωH∗(ω + π)

If we define for any scale 2j

ψj,n(t) =1√2jψ

(t− 2jn

2j

)


Then ψj,n(t)n∈Z is an orthonormal basis of Wj. Further, ψj,n(t)(j,n)∈Z2 is an or-

thonormal basis of L2(R). Any f(t) ∈ L2(R) can be written as,

f(t) =∞∑

n=−∞

∞∑j=−∞

〈f(t),1√2jψ

(t− 2jn

2j

)〉 1√

2jψ

(t− 2jn

2j

)

4.3.5 Fast Orthogonal Wavelet Transform

Figure 4.2: DWT Decomposition (top), Inverse (bottom) [28]

Consider the subspace Vj−1 = Vj⊕

Wj. We can define aj[n] = 〈f(t), φj,n(t)〉 and

dj[n] = 〈f(t), ψj,n(t)〉. Mallat showed that the following analysis and reconstruction

formulas hold, using the filters h[n] and g[n] constructed previously,

aj+1[p] =∞∑

n=−∞

h[n− 2p]aj[n]

dj+1[p] =∞∑

n=−∞

g[n− 2p]aj[n]

aj[p] =∞∑

n=−∞

h[p− 2n]aj+1[n] +∞∑

n=−∞

g[p− 2n]dj+1[n]

The first two formulas correspond to the filter bank implementation of the Discrete


Figure 4.3: WPT Decomposition (top), Inverse (bottom) [28]

Wavelet Transform, DWT, while the third formula is the filter bank implementation

of the Inverse Discrete Wavelet Transform, IDWT.

Introducing the filters g[n] = g[−n] and h[n] = h[−n], figure 4.2 shows the block

diagram of the filter bank implementation of the DWT. In decomposition mode, the

coefficients aj[n] are low pass filtered with h[n] and high pass filtered with g[n]. Then

the output of both filters is downsampled by 2. The downsampled output of the high

pass filter, dj+1[n], is kept, as the wavelet (detail) coefficients of that level, while the

downsampled output of the low pass filter is decomposed anew in the same manner.

When the decomposition is stopped, the downsampled output of both filters is kept:

the downsampled output of g[n], dJ [n], as the wavelet (detail) coefficients at scale 2J ,

the downsampled output of h[n], aJ [n], as the coefficients of the approximation of the

signal with scale 2J . Figure 4.2 also shows how the reverse process, reconstruction or

IDWT, is implemented. It can be shown that a discrete wavelet decomposition with

a tree of depth J − L ≥ 0, gives a decomposition of `2(Z) via an orthonormal basis.


More precisely, given one such decomposition, any x[n] ∈ `2(Z) can be written as,

x[n] =+∞∑l=−∞

aJ [l]φJ [n− 2J−Ll] +J∑

j=L+1

+∞∑l=−∞

dj[l]ψj[n− 2j−Ll] (4.2)

Where the vectors

{φJ [n− 2J−Ll]}l∈Z, {ψj[n− 2j−Ll]}L<j≤J, l∈Z

form an orthonormal basis of `2(Z). It follows that aJ [l] = 〈x[n], φJ [n− 2J−Ll]〉 and

that dj[l] = 〈x[n], ψj[n−2j−Ll]〉. As a corollary the J−L+1 shift invariant subspaces

span{φJ [n−2J−Ll]}l∈Z, span{ψL+1[n−2l]}L<L+1, l∈Z, ... span{ψJ [n−2J−Ll]}J, l∈Z are

mutually orthogonal and the vectors that span each shift invariant subspace form an

orthonormal basis for the corresponding subspace. With the appropriate normaliza-

tion, what really matters is J −L since the above decomposition of `2(Z) starts with

the assumption that aL[n] = x[n].

As an example, say that we take J = 12 and L = 10, equation (4.2) becomes,

x[n] =+∞∑l=−∞

a12[l]φ12[n− 4l] ++∞∑l=−∞

d11[l]ψ11[n− 2l] ++∞∑l=−∞

d12[l]ψ12[n− 4l] (4.3)

which means that l2(Z) = span{φ12[n−4l]}l∈Z⊕

span{ψ12[n−4l]}l∈Z⊕

span{ψ11[n−2l]}l∈Z . Indeed, given a signal x[n] ∈ `2(Z) if we set x[n] = aL[n] = a10[n], we get

through the DWT decomposition recursion a11[n] and d11[n]. We keep d11[n] and

repeat the process with a11[n] to get a12[n] and d12[n], at which point we stop the

recursion. Plugging these coefficients into equation (4.2), we get equation (4.3).

Thus, equation (4.2) implies a decomposition of `2(Z) in a union of mutually orthog-

onal shift invariant subspaces, each of which has an orthonormal generator.

Before we move forward, it must be pointed out that there is also the possibility of

decomposing `2(Z) in a way that both downsampled outputs of ˜h[n] and g[n] are

decomposed recursively, instead of applying the recursion only to the downsampled

output of filter h[n], as the standard DWT does. This alternative way, for both


decomposition and reconstruction, is shown in figure 4.3. It can be proved that this

decomposition also results in a decomposition of `2(Z) via an orthonormal basis. This

is the so called Wavelet Packet Decomposition, WPD. The WPD is not regularly used

in compression applications because it doesn’t result in significant compression gains

when compared to the DWT. In our case however, it is very interesting because it

matches the canonical Hybrid Filter Bank model with M = 2J−L.

We can now apply the results developed in Chapter 3 for Discrete Sampling to a

wavelet decomposition of l2(Z).

4.4 Discrete Sampling for Continuous Time, Band-

Limited Signals Meets Wavelet Theory

Figure 4.4: HFB Associated with Decomposition Symmlet 8 J=12, L=10

In the previous section about wavelets, it was stated without proof that given a

wavelet multiresolution approximation, we can decompse `2(Z) as

`2(Z) = span{φJ [n− 2J−Ll]}l∈Z⊕ J⊕

j=L+1

span{ψj[n− 2l−Ll]}l∈Z

4.4. DISCRETE SAMPLING FOR PWTS MEETS WAVELET THEORY 65

Figure 4.5: Discrete Equivalent of Figure 4.4

Now, suppose that φJ [n] as well as {ψj[n]}L+1≤j≤J , a total of J − L + 1 discrete

waveforms, are compactly supported and that for each of them the interpolation

condition (3.4) for its z transform is satisfied. Because this decomposition of `2(Z)

fits the model described in section 4.2 as a union of mutually orthogonal shift invariant

subspaces with orthonormal generators, any x[n] ∈ `2(Z) can be written as

x[n] =+∞∑i=−∞

〈x[n], uφJ [n− 2J−Li]〉uφJ [n− 2J−Li]+

J∑j=L+1

+∞∑i=−∞

〈x[n], uψj [n− 2j−Li]〉uψJ [n− 2j−Li]

with uφJ [n] and uφJ [n] the interpolation atom and its respective dual, associated with

φJ [n]. Similarly, {uψj [n]}L+1≤j≤J and {uψj [n]}L+1≤j≤J are the interpolation atoms

and their respective duals, associated with {ψj[n]}L+1≤j≤J .

4.4.1 Example

Let’s illustrate what we have developed so far with an example. The so called Symmlet

8 decomposition with parameters J = 12, L = 10, results in `2(Z) = span{φ12[n −


Figure 4.6: Symmlet 8 Generators With J=12, L=10

4l]}l∈Z⊕

span{ψ12[n − 4l]}l∈Z⊕

span{ψ11[n − 2l]}l∈Z with the atoms φ12, ψ12 and

ψ11 compactly supported. These atoms are shown in figure 4.6. The corresponding

interpolating atoms, with their respective duals, are shown in figures 4.7, 4.8 and 4.9.

Figure 4.4 shows the block diagram of this example for decomposition and reconstruc-

tion. To verify numerically that this particular setup gives perfect reconstruction, we

used a 32 seconds long segment of the theme “Pageant” from the show KA by Cirque

du Soleil [40]. Our x[n] is obtained by sampling the first channel of the stereo signal

at fs = 44.1Khz with 16 bit amplitude resolution; x[n] has CD audio signal quality.

For simulation purposes, we used the discrete time equivalent of figure 4.4 shown


Figure 4.7: Symmlet 8, J=12, L=10, Interpolating Atom, Dual Associated With φ12

Figure 4.8: Symmlet 8, J=12, L=10, Interpolating Atom, Dual Associated With ψ12


Figure 4.9: Symmlet 8, J=12, L=10, Interpolating Atom, Dual Associated With ψ11

in figure 4.5: digital filters with impulse responses uφ12 [−n], uψ12 [−n] and uψ11 [−n]

are used in lieu of their respective continuous time filters as well as discrete down-

samplers instead of continuous time samplers. x[n] is 1456128 samples long; thus,

the model x[n] ∈ `2(Z) fits this experiment perfectly. To measure the quality of the

reconstructed signal, we use the metric PSNR defined in equation (3.15), which is

reproduced in equation (4.4) for convenience,

PSNR = 10 log10

(N

max(|x[n]|)2

||x[n]− xrec[n]||2

)(4.4)

As we explained N is the signal’s length. In the case of audio signals in Matlab,

max(|x[n]|) = 1. To gauge the effect of discarding some of the filter outputs, which

would allow us to effectively sample x(t) at a rate lower than Nyquist’s, we define the

discrete signals xφ12 [n] and xφ12ψ12 [n] as,

• xφ12 [n] is the output of the filter whose impulse response is uφ12 [n]. This signal

requires only one forth of the information present in x[n] and one sampler

working at one forth of the Nyquist rate;


• xφ12ψ12 [n] is the signal which results from adding the outputs of the filters whose

impulse responses are uφ12 and uψ12 [n]. This signal requires half the information

present in x[n] but still uses samplers, two of them, working at one fourth of

the Nyquist rate.

The numerical results are shown in table 4.1.

Signal Measured PSNR [dB]xφ12

[n] 40.5xφ12ψ12

[n] 50.7x[n] 136

Table 4.1: Numerical Results for 32s Segment of Pageant

Note that the 136 dB PSNR obtained for x[n] in this case means perfect recon-

struction since x[n]’s amplitude resolution is 16 bits. Indeed one can verify that if

x[n], which was computed using double precision in matlab, is quantized to 16 bits

resolution to obtain x16bit[n], we get ||x16bit[n]− x[n]|| exactly equal to zero. This ex-

ample shows that perfect reconstruction can be obtained not only in theory but also

in practice if one keeps all filter outputs during reconstruction; in other words, if we

keep as many samples as the length of x[n]. But even in such case, all samplers work

at a rate below Nyquist’s. However, table 4.1 also says that we can get an acceptable

reconstruction, both numerically and subjectively, by generating xφ12ψ12 [n] instead of

x[n]. With xφ12 [n] we get artifacts that make it noticeably different from x[n]. To

get a feeling of how these signals differ in the frequency domain, figure 4.10 shows

their magnitude spectra in dB. These spectra have been computed with a 2048 FFT

and smoothed with a moving average filter of length 50 in order to improve their

visualization. The first important thing to note is that all the signals have frequency

components up to fs2

, which means that xφ12 [n] cannot be obtained by just replacing

the filter with impulse response uφ12 [−n] with a low pass filter whose cutoff frequency

is fs4

-in that case the Nyquist frequency would still be fs2

while we can get xφ12 [n]

by sampling at rate fs4

. The second observation is that x[n] and xφ12ψ12 [n] differ in

magnitude mostly in the frequencies close to fs2

, which explains why an untrained

listener cannot differentiate the two.


Figu

re4.10:

Sm

ooth

edM

agnitu

de

Sp

ectrum

ofSign

alsx

[n][b

lue],xφ12 [n

][red

]an

dxφ12ψ12 [n

][green

]for

32sSegm

ent

ofT

hem

eP

ageant

4.5. NOISE ANALYSIS 71

4.5 Noise Analysis

Characterizing how electrical noise affects the reconstruction of signals is vital in

practical ADC design. Electrical noise, and other sources of noise that can be modeled

in the same fashion, add in the output of the ADC with the net result of decreasing

the effective resolution of the ADC. A sound understanding of how, and how much,

electrical noise propagates to the output of the ADC is essential in a practical design.

Assuming that we know that we have the noise at the ADCs output under control,

and that such noise can be modeled as additive zero mean Gaussian iid, we can then

compute how this noise corrupts the output of the HBF.

4.5.1 A More Detailed Look at the Discrete Interpolation

Process

Consider the interpolation equation,

x[n] =+∞∑i=−∞

x[iM ]u[n− iM ]

u[n] can be written equivalently, using multirate discrete signal processing as [29][10],

u[n] =M−1∑k=0

uk[n− k]

with uk[n] = u[n+ k] if n is multiple of M and uk[n] = 0 otherwise. The decomposi-

tion of a filter in this manner is called polyphase decomposition. In the context

of interpolating atoms, u0 = δ[n]. In fact, an equivalent way of characterizing an

interpolating atom is that u[n] is an interpolating atom if and only if its polyphase

decomposition results in u0 = δ[n]. Figure 4.12 shows the polyphase decomposition of

uφ12 [n]. Intuitively this means that the actual information of an interpolating signal

is contained in the samples x[nM ] while the other samples x[n] where n 6= nM are

reconstructed from ↑ x[nM ]- upsampling factor equal to M- through the filters uk[n],


Figure 4.11: Polyphase Decomposition of uφ12 [n], from Wavelet DecompositionSymmlet 8 With J=12, L=10

x[n] =+∞∑i=−∞

x[iM ]u[n− iM ] =+∞∑i=−∞

x[iM ]M−1∑k=0

uk[n− iM − k] =

M−1∑k=0

+∞∑i=−∞

x[iM ]uk[n− iM − k] =

+∞∑i=−∞

x[iM ]δ[n− iM ]+


+∞∑i=−∞

x[iM ]u1[n− iM − 1] + ...+

+∞∑i=−∞

x[iM ]uM−1[n− iM − (M − 1)]

This way of looking at the interpolating process is illustrated in figure 4.12. Let’s call

x0[n] =↑ x[nM ] =∑+∞

i=−∞ x[iM ]δ[n−iM ], x1[n] =↑ x[nM+1] =∑+∞

i=−∞ x[iM ]u1[n−iM − 1], x2[n] =↑ x[nM + 2] =

∑+∞i=−∞ x[iM ]u2[n − iM − 2] and so on. Note that

each of these sequences is the result of a three step process:

i) filtering x[nM ] with filter uk[nM ] (since uk[n] is 0 for n 6= nM).

ii) upsample the sequence obtained in step i) by factor M

iii) delay the signal obtained in step ii) by k to get xk[n]

Therefore x[n] is the sum of M sequences, x[n] = x0[n] + x1[n] + x2[n] + ...+ xM−1[n],

each of which is generaged from ↑ x[nM ] via linear filtering.

Note that in practice it could happen that u[n] is a shifted version of an interpolation

atom that follows this model. In that case uk[nM ] = δ[n] for some k 6= 0 but the

analysis remains valid.

4.5.2 Linear Filtering of a Zero Mean Gaussian iid Process

Before we can analyze how noise affects the fidelity of the reconstructed signal, we

need to consider the filtering of a discrete zero mean Gaussian process since, for

reasons that will be explained later, it is the process that models best the sources of

noise encountered in practice. Suppose that one has a Gaussian iid process w[n] such

that w[n] is distributed as N (0, σ2). Suppose also that we filter this process through

a stable, linear, time invariant filter h[n]. The output of the filter y[n] = w[n] ∗ h[n]

is also a Gaussian process. Although not necessarily independent, y[n] are identically


Figure 4.12: Interpolation From The Polyphase Decomposition Point of View

distributed as N (0, ||h[n]||2σ2) . In general y[n] is correlated with y[n−i] for i 6= 0. In

words, if one filters a discrete zero mean white Gaussian process with a stable linear

filter, the output is a discrete zero mean Gaussian process whose variance is related

to the variance of the input process through the norm of the filter coefficients. The

output process is not necessarily white. Note that if ||h[n]||2 ≤ 1, the filter does not

amplify the noise, which is a good thing.

4.5.3 Noise Analysis in an HBF Designed with Discrete Sam-

pling

In section 4.4 we showed that there exist wavelet decompositions that allow us to

represent any signal in x[n] ∈ l2(Z) as

x[n] =

Q−1∑q=0

+∞∑i=−∞

〈x[n], uq[n−Mqi]〉uq[n−Mqi]

where {uq[n]}Q−1q=0 are interpolating atoms that span mutually orthogonal shift invari-

ant subspaces and {uq[n]}Q−1q=0 are their respective duals. One such representation was

presented in subsection 4.4.1.〈x[n], uq[n−Mqi]〉 is computed in practice by sampling

at rate 1TMq

the output of an appropriately scaled continuous time filter. Let’s call

oq[i] = 〈x[n], uq[n −Mqi]〉. In a typical HBF setting, the most significant source of


noise is the continuous time thermal noise introduced by the filters, which is additive,

zero mean and Gaussian. As we said in Chapter 1, it can be shown that in track

and hold circuits, as those used by ADCs, the effect of this Gaussian continuous time

noise can be equivalently modeled as a discrete time, zero mean Gaussian iid noise

that is added to oq[i] provided that when we acquire a sample, we wait long enough

to settle the measured sample to a resolution of 6 bit or better. The vast majority

of ADCs work at such resolutions. The variance of the discrete Gaussian iid noise

can then be estimated to be KTCq

in a first order approximation, with Cq the capacitor

of the track and hold circuit, T the temperature in degrees Kelvin and K the Boltz-

mann constant. Note that this allows the designer of the ADC to control the energy

of the noise present at the output of the ADC, although, as explained in Chapter

1, since increasing the size of Cq will make the ADC slower, the value of Cq will be

influenced by a trade off accuracy vs speed. Thus, in practice, instead of measuring

oq[i], we will be measuring oq[i]+wq[i] where we model the noise wq[i] as a discrete iid

process distributed as N (0, σ2q ) with σ2

q = KTCq

. This model means that not only wq[i]

is independent of wq[j] ∀i 6= j, but also that wq1 [i] is independent of wq2 [i] ∀q1 6= q2.

Referring to figure 4.13, which shows the different components of the total noise at

the HBF output w[n], what we are saying is that we assume both “horizontal” and

“vertical” independence. Such assumption is realistic because in practice the noise

present at the output of an analog filter will be statistically independent of the noise

present at output of a different analog filter.

So instead of reconstructing x[n], we’ll reconstruct a noisy version, xw[n],

xw[n] =

Q−1∑q=0

+∞∑i=−∞

{oq[i] + wq[i]}uq[n−Mqi] =

Q−1∑q=0

+∞∑i=−∞

oq[i]uq[n−Mqi] +

Q−1∑q=0

+∞∑i=−∞

wq[i]uq[n−Mqi] =

x[n] +

Q−1∑q=0

+∞∑i=−∞

wq[i]uq[n−Mqi] =


Figure 4.13: Noise Propagation in a Discrete Sampling HBF

x[n] +

Q−1∑q=0

fq[n] = x[n] + w[n]

Where w[n] =∑Q−1

q=0 fq[n] is a discrete, zero mean Gaussian random process since ∀n,

w[n] is the sum of zero mean Gaussian random variables as shown in figure 4.13. To

compute σ2w[n], let’s consider fq[n] =

∑+∞i=−∞wq[i]uq[n−Mqi]. We see that fq[n] is the

result of filtering ↑ wq[n]- upsampling factor equal to Mq- with the interpolation filter

associated with the shift invariant subspace q. From the polyphase decomposition of

interpolating atoms we know that fq[n] is in fact the sum of Mq sequences, each of

which is obtained by filtering ↑ wq[n] with filter uqk with k = 0, 1, ..,Mq − 1, which

means that fq[n1] is distributed identically as fq[n2] only if n2 = n1 + jMq with

j ∈ Z. From the properties of filtered Gaussian noise, we conclude that the marginal

distribution of fq[n] is N (0, ||uqk||2σ2q ) where k ≡ n mod Mq, thus σ2

fq [n] = ||uqk||2σ2q ,

with k ≡ n mod Mq. We note that σ2fq [n] is Mq periodic.

Let’s now consider the vertical sum. By assumption, for any n1 and any n2, fq1 [n1]


is independent of fq2 [n2] as long as q1 6= q2 even if n1 = n2. Since w[n] =∑Q−1

q=0 fq[n],

σ2w[n] =

∑Q−1q=0 σ

2fq [n]. Given that σ2

fq [n] is Mq periodic, σ2w[n] is Mlcm periodic, where

Mlcm = lcm(M0,M1, ...,MQ−1). Another way to reach this conclusion is to realize

that w[n] is made of Mlcm different processes, wj[n] for 0 ≤ j < Mlcm, that have the

following properties,

• wj[n] = w[n+ j] with σ2wj [n] = σ2

w[j] if n = lMlcm with l ∈ Z

• wj[n] = 0 if n 6= lMlcm with l ∈ Z

• w[n] =∑Mlcm−1

j=0 wj[n− j]

This polyphase decomposition of w[n] in Mlcm sequences makes it easy to see that

σ2w[n] =

∑Q−1q=0 ||uqk||2σ2

q with k ≡ n mod Mq and therefore σ2w[n] is Mlcm periodic.

If ||uqk||2 ≤ 1 ∀k and the uq[n] are aligned such that uq[0] = 1 ∀q, then for a fixed n,∑Q−1q=0 fq[n] is a Gaussian random variable whose variance is bounded by

∑Q−1q=0 σ

2q . If

we still have ||uqk[n]||2 ≤ 1 ∀k but the uq[n] are not aligned, we might be able to get a

better upper bound. In any case, this analysis allows us to bound the variance, thus

the energy, of the noise that corrupts the reconstructed signal. It also gives us an

interesting property to look for in our interpolating atoms, ||uqk[n]||2 ≤ 1 ∀k, which

in turn would imply supq,n uq[n] = 1 given that uqk[n] is δ[n] for some k -most likely

for k = 0.

With our approach we can then estimate the energy of the overall error introduced in

in xw[n]. Consider a signal x[n] of length N such that N is large and N divides Mlcm.

If Mlcm << N this assumption is reasonable. Since the model implicitly implies that

all sources of noise are ergodic, we can estimate the total squared error introduced

by one of the Mlcm sequences that compose w[n], wj[n] as

length(wj[nMlcm])σ2wj [0] =

length(x[n])

Mlcm

σ2wj [0] =

N

Mlcm

σ2w[j]

=N

Mlcm

Q−1∑q=0

||uqk||2σ2q

with k ≡ j mod Mq. By taking j from 0 to Mlcm − 1, the total squared error

introduced in xw[n] can be estimated as


Mlcm−1∑j=0

N

Mlcm

σ2wj [0] =

N

Mlcm

Mlcm−1∑j=0

Q−1∑q=0

||uqk||2σ2q

where k ≡ j mod Mq. Dividing the above expression byN we obtain as the estimated

mean squared error, MSE

MSE =1

Mlcm

Mlcm−1∑n=0

Q−1∑q=0

||uqk||2σ2q (4.5)

where k ≡ j mod Mq. We see that the expression for the MSE estimate is not de-

pendent on N , the length of the signal. The actual MSE is computed as ||x[n]−xw[n]||2N

.

Let’s apply this formula to the example presented in subsection 4.4.1. In this case

N = 1456128, Q = 3, M0 = 2, M1 = 4 and M2 = 4; thus Mlcm = 4. Table 4.2

shows the different ||uqk||2 , where k ≡ j mod Mq, for 0 ≤ j < Mlcm = 4. We

see in the table that we are in a situation where the uq[n] are not aligned. For the

simulation we set σ20 = 10−6, σ2

1 = 2.10−6, σ22 = 3.10−6. Equation (4.5) gives us an

MSE = 4.578.10−6. The measured MSE, obtained as an average of 10 different

runs, was 4.5766.10−6, which is in agreement with the predicted value.

j = 0 j = 1 j = 2 j = 3q = 0 ||u00||2 = 0.7126 ||u01||2 = 1.0000 ||u00||2 = 0.7126 ||u01||2 = 1.0000q = 1 ||u10||2 = 0.7191 ||u11||2 = 0.2760 ||u12||2 = 1.0000 ||u13||2 = 0.3903q = 2 ||u20||2 = 1.0000 ||u21||2 = 0.8981 ||u22||2 = 0.6972 ||u23||2 = 0.7767

Table 4.2: ||uqk||2 , Example of Subsection 4.4.1

4.6 Practical Imprementation of Continuous Time

Filters

The most critical components in an HBF, from a circuit implementation point of view,

are its analog filters. The HBFs designed with Discrete Sampling are no exception.

Each analog filter has to implement a linear time invariant system whose impulse

response is some function u(−t) =∑+∞

n=−∞ u[n]sinc(t−nTsTs

). By construction, this

4.6. PRACTICAL IMPREMENTATION OF CONTINUOUS TIME FILTERS 79

function is non causal, therefore it is impossible to implement it exactly with real

circuits since we only know how to implement causal filters. The usual workaround

when a problem like this appears in engineering is to try to implement a filter whose

impulse response is a delayed and windowed version of the desired impulse response.

This is the approach that we will use here. Once we have overcome this problem we

also need to face the fact that the only causal filters that we know how to implement

are those whose impulse response has a rational Laplace Transform in the form [41],

H(s) =P (s)

Q(s)=b0s

p + b1sp−1 + · · ·+ bp

a0sq + a1sq−1 + · · ·+ aq(4.6)

With the extra requirements that

• q ≥ p, which ensures causality,

• the poles of H(s), ie, the zeros of Q(s), lie on the left hand side of complex

plane, which ensures stability;

once the H(s) function to implement has been settled, circuit designers put it in a cir-

cuit. The thing to keep in mind is that building an analog filter involves two different

stages which are generally decoupled, and therefore, can be addressed independently,

1. Computing the H(s) that implements as closely as possible the filter require-

ments, which are usually expressed in the frequency domain through the transfer

function or in time domain through the impulse response. This is a problem

mathematical in nature.

2. Designing the circuits that implement the approximation to H(s) computed

in the previous item. This is a problem of circuit design. Over the years a

great deal of circuit architectures have been developed to solve this problem;

the actual architecture used will depend on many factors such as the band of

operation, power specifications, tolerance requirements, volume of units, active

vs passive, etc. These requirements will also determine the technology in which

the filter is implemented: printed circuit, integrated circuit, waveguide, etc.

Everything else being equal, the most limiting factor that prevents a given


H(s) from being implemented accurately is the quality factor Q associated to

each pole-zero of Q(s). For a given point in the complex plane, Q is defined

as the ratio between the absolute value of its imaginary part and the absolute

value of its real part. With the current state of technology, the rule of thumb

is that each Q should be at most 10.

Readers interested in knowing more about the general practice of analog filter design,

which includes how these two problems are solved in most practical situations, are

referred to any good reference on the topic such as [42]. In the rest of the section, I

will focus on how we can accomplish the first task for the analog filters that show up

in an HBF designed with Discrete Sampling, keeping in mind the rule about quality

factors being lower than 10. Once we have the transfer functions that approximate

the analog filters, they can be given to an analog filter engineer for its implementation.

4.6.1 Computing H(s) for Impulse Response u(−t)

Figure 4.14: Idealized Filter Frequency Responses [43]

In classic analog filter design, the specifications for H(s) are given in terms of the

desired frequency response. The ideal frequency responses of the common types of

analog filters used in circuits, shown in figure 4.14, are brick wall and therefore cannot


Figure 4.15: Analog Filter Design Parameters [43]

be implemented. During the first half of the XX-th century [44] the problem of how

to approximate these responses with functions that fit the model of equation (4.6) not

only was studied throughly, but, by 1950 several approximations had been developed

analytically: Butterworth, Chebyshev, Bessel and Cauer-also known as elliptical.

These approximations, used to this day together with a mechanical design process

usually implemented in a computer program, allow the circuit designer to obtain the

transfer function H(s) of practically any low pass, band pass, high pass and band

reject filter encountered in practice. The process begins by specifying the magnitude

of the desired frequency response in terms of several filter parameters, such as Amin,

Amax, fc, fstop, shown in figure 4.15 for the case of a low pass filter design. Then, by

following a step by step algorithm, the coefficients for the numerator and numerator

of H(s), as well as the minimum value required for q- called the filter order- to meet

the specifications, are computed. In fact, the design of all these filters, regardless of

their frequency response, is started by providing the filter parameters for a prototype

low pass filter with fc normalized to 1, out of which all other filters are synthesized for


any response type and arbitrary values of fc and fstop. With this framework, it is not

possible to specify phase requirements, rather, each type of approximation provides

a particular type of phase response. In order to compute the H(s) associated with a

given u(−t) this method is insufficient for two reasons: first, as shown in figure 4.16

for uφ12(−t) from example of subsection 4.4.1, the magnitude spectrum associated

with the Laplace Transform of a typical u(−t), L{uφ12(−t)}(jω), doesn’t fit any of

the standard filter prototypes; second, in our case, we need to be able to specify phase

requirements since we are using the filters with frequency response u(−t) to compute

inner products.

Figure 4.16: Magnitude of L{uφ12(−t)}(jω), Example Subsection 4.4.1

Luckily, there are numerical methods that allow us to find a transfer function in the

form of equation (4.6) that fits a given impulse response, and therefore its frequency

response [45]. These methods can be roughly classified according to whether they try

to approximate h(t), the impulse response of H(s), to the desired impulse response,

in our case a truncated version of u(−t), or whether they try to approximate H(jω),

the frequency response of H(s), to the desired frequency response, in our case U(−f).

These methods are described in great detail in Chapter 5 of [45]. After experimenting


with several approaches, which fall in both categories, the one that produced the

best approximation results in our case was the least squares approximation to the

desired frequency response. This method has the inconvenience that, because of the

geometry of the objective function, there are many local minima, so one has to try

several starting points to get a decent result. This proliferation of local minima also

translates in diminishing returns for the accuracy of the approximation with respect

to increases of q.

Least Squares Approximation of U(−f)

Since u(−t) ∈ PWTs, we can compute its Fourier Transform, U(−f), by computing

the Discrete Time Fourier Transform of u[−n] = u(−nTs) , U(−fn), which is a

function of the continuous normalized frequency fn = ffs

, and noticing that U(− ffs

) =

U(−fn) for −12≤ f

fs≤ 1

2and zero otherwise. To simplify things, we will work with

the assumption Ts = 1, which results in u[−n] = uTs=1(−n). Once we have found the

HTs=1(s) whose associated frequency response, HTs=1(jω), approximates UTs=1(−f),

we will scale it, using the scaling property of the Laplace transform, to find the H(s)

whose frequency response, H(jω), approximates U(−f). To perform the least squares

approximation, we use Matlab’s function invfreqs [46]. This function takes as its

most important inputs,

• vector W of length N which contains the normalized angular frequencies in

which the desired frequency response, D(jω) is evaluated. In our caseD(j2πfn) =

U(−fn).

• vector H of length N such that H(k) = D(jW (k)) for 1 ≤ i ≤ N

• the desired degree of the numerator n = p

• the desired degree of the numerator m = q .

Other inputs not detailed above include options about the desired accuracy and num-

ber of iterations. invfreqs then solves the problem


minb,a

N∑k=1

∣∣∣∣H(k)− P (jW (k))

Q(jW (k))

∣∣∣∣2 (4.7)

Where b is the p + 1 vector such that bi−1 = b(i) for 1 ≤ i ≤ p + 1, a is the q + 1

vector such that ai−1 = a(i) for 1 ≤ i ≤ q + 1 and P (jω), Q(jω) are the frequency

responses associated with P (s) and Q(s) respectively. This problem is solved via the

Gauss-Newton iterative method [47] since the objective function is non linear. The

initial value for the iteration is obtained as the solution of the least squares problem,

minb,a

N∑k=1

|H(k)P (jW (k))−Q(jW (k))|2 (4.8)

which can be solved using linear least squares methods. As an additional feature

invfreqs only produces solutions whose poles lie on the left hand side of the complex

plane, ie, stable solutions.

4.6.2 Design Example

To illustrate the concepts developed in this section, we will present an HBF in which

the analog filters are designed to have impulse responses whose Laplace Transforms

can be expressed in the form of equation (4.6). This example is different from the

example presented in subsection 4.4.1.

The wavelet decomposition used to decompose `2(Z) is Beylkin with a one level

decomposition tree; thus, we have a detail subspace, spanned by ψ[n], and a coarse

approximation subspace, spanned by φ[n]. Their respective atoms uψ[n]-uψ[n] and

uφ[n]-uφ[n] with M = 2 are shown in figures 4.17 and 4.18.

The first thing to decide is how to truncate uψ[−n] and uφ[−n] to facilitate the task of

approximating uψ(−t) and uφ(−n) with filters whose Laplace transform is in the form

of equation (4.6). Remember that at this point we are working with Ts = 1, so the

question can be equivalently formulated for the discrete versions. There is no simple

answer. It’s a matter of trying several different truncated versions to find which one

gives a better approximation. In this example, we decided to make uψ[n] and uφ[n]


Figure 4.17: uψ[n] and uψ[n] Associated with ψ[n], Beylkin Wavelet Decomposition2 Subspaces

Figure 4.18: uφ[n] and uφ[n] Associated with φ[n], Beylkin Wavelet Decomposition 2Subspaces


Figure 4.19: Tuncated uψ[−n] and uφ[−n] for Beylkin Wavelet Decomposition 2 Sub-spaces

Figure 4.20: DuTψ(j2πfn) and DuTφ(j2πfn) for Beylkin Wavelet Decomposition 2Subspaces


0.5254s16 − 2.809s15 + 22.55s14 − 79.61s13

+332.5s12 − 815.5s11 + 2205s10 − 3777s9

+7015s8 − 8194s7 + 1.06e004s6 − 8128s5

+7386s4 − 3287s3 + 2040s2 − 334.4s+ 147.9

s18 + 4.558s17 + 53.05s16 + 186.2s15

+1122s14 + 3088s13 + 1.244e004s12 + 2.691e004s11

+7.923e004s10 + 1.336e005s9 + 2.986e005s8 + 3.828e005s7

+6.585e005s6 + 6.125e005s5 + 8.105e005s4 + 4.962e005s3

+4.991e005s2 + 1.55e005s+ 1.156e005

Table 4.3: H1(s)

−0.01146s14 − 0.0369s13 − 0.05885s12 − 1.192s11

+3.053s10 − 15.56s9 + 39.32s8 − 95.45s7

+174s6 − 263.1s5 + 303.3s4 − 263.5s3 + 159.9s2

−61.1s+ 11.04

s16 + 5.231s15 + 29.02s14 + 95.85s13

+289.8s12 + 666.7s11 + 1351s10 + 2236s9

+3195s8 + 3789s7 + 3790s6 + 3096s5

+2043s4 + 1040s3 + 387.3s2 + 93.51s+ 11.04

Table 4.4: H2(s)


equal to zero for n ≥ 26. Let’s call uTψ[n] and uTφ[n] the truncated versions of uψ[−n]

and uφ[−n] respectively; they are shown in figure 4.19. Once we have these, we need to

compute their frequency response. For that, we compute their Discrete Time Fourier

Transforms, DuTψ(j2πfn) and DuTφ(j2πfn), whose magnitudes are figure 4.20.

These two spectra, both phase and magnitude, are then sampled at 4096 regularly

spaced positions - positive normalized frequencies only. At this point we have W

and H(k) that we can input to invfreqs. As explained earlier, the method of least

squares approximation to a desired frequency response requires to perform several

algorithm executions, each with a different starting point, to find the most accurate

approximation because the objective function has several local minima. In our case

we can do that by trying different values of p and q. In our simulation we obtained

the transfer functions H1(s) and H2(s), that approximate DuTψ(s) and DuTφ(s) re-

spectively, shown in tables 4.3 and 4.4. As we see, p = 16 and q = 18 for H1(s) while

p = 14 and q = 16 for H2(s).

The poles of H1(s) are: −0.1863 + 3.2226j, −0.1863 − 3.2226j, −0.2996 + 2.9563j,

−0.2996−2.9563j, −0.3436+2.6662j, −0.3436−2.6662j, −0.3547+2.3657j, −0.3547−2.3657j, −0.3454+2.0679j, −0.3454−2.0679j, −0.2899+1.7964j, −0.2899−1.7964j,

−0.2083+1.3577j, −0.2083−1.3577j, −0.1550+1.1480j, −0.1550−1.1480j, −0.0962+

0.9031j, −0.0962 − 0.9031j, all of which lie on the left hand side of the complex

plane. We verify that the poles of H2(s) also have this property: −0.1841 + 2.2861j,

−0.1841−2.2861j, −0.2783+2.0259j, −0.2783−2.0259j, −0.2748+1.6794j, −0.2748−1.6794j, −0.2818+1.4638j, −0.2818−1.4638j, −0.3252+1.0224j, −0.3252−1.0224j,

−0.4505+0.7969j, −0.4505−0.7969j, −0.4058+0.4935j, −0.4058−0.4935j, −0.4153+

0.1603j, −0.4153 − 0.1603j. From these poles we compute that the quality factors

required for the implementation of H1(s) are 17.3020, 9.8690,7.7586, 6.6701, 5.9866,

6.1962, 6.5180, 7.4067 and 9.3847. Note that there a total of nine different quality

factors since the eighteen poles come in conjugate pairs. For H1(s) all quality factors

are below 10 except for the quality factor associated with −0.1863 + 3.2226j and

−0.1863− 3.2226j. This means that we might encounter some problems if we decide

to implement this H1(s). The quality factors for H2(s) are 12.4206, 7.2803, 6.1116,

5.1945, 3.1443, 1.7689, 1.2160 and 0.3860. Based on the quality factor alone, H2(s)


should be implementable. We will now test, working with normalized Ts = 1, that

the signal obtained from Pageant, introduced in subsection 4.4.1, can be processed

with this HBF.

Figure 4.21: Block Diagram of HBF for Beylkin Wavelet Decomposition 2 Subspaces

Figure 4.22: Discrete Sinc Based Interpolating Filter Used to Resample x(t) at Rate0.25Hz

As shown in figure 4.21, the main difference with the simulations performed in sub-

section 4.4.1 is that instead of simulating analog filters with ideal impulse responses,

we will simulate the response of H1(s) and H2(s) to the continuous time input signal

x(t). To that aim, we will use Matlab’s continuous time linear simulator lsim, which

takes as inputs,

• the system to be simulated. We just need to put in system format filters H1(s)

and H1(s). Since we have their coefficients, Matlab’s tf function does the job,


• the regularly sampled signal to be simulated. Note that to perform a continuous

time simulation, the input signal needs to be sampled at a frequency several

times higher than fs = 1. A sampling frequency of fs4

= 14Hz is sufficient

in our case. Since x[n] are samples of a function of bandwidth 12Hz, we can

resample x(t) at a rate of 14Hz through the continuous time sinc interpolation

equation, (1.1). We can equivalently filter x[n] = x(nTs) = x(n) with the

discrete filter r[n] such that r[n] = sinc(n4). Since r[n] has infinite duration,

a good approximation can be obtained by using a filter of 161 coefficients, ie,

rt[n] = sinc(n4) for −80 ≤ n ≤ 80, shown in figure 4.22. Thus, we input

x 14[n] = x(n

4) = {↑ x[n]} ∗ rt[n], where the upsampling of x[n] is by a factor

equal to 4,

• the times for which we want the simulation performed. In our case, the times

are 0 + 14i with 0 ≤ i ≤ N − 1, where N is the length of x[n].

The output signal has four times more samples than the length of x[n], thus the sig-

nals provided by lsim have to be downsampled by a factor of 8: factor 4, required

to bring the output to the original sampling rate, multiplied by factor 2, since in our

case M = 2.

Inputting x(t), the signal obtained from Pageant introduced in subsection 4.4.1, we

obtain a PSNR of 56.4dB for x[n] = x(n). If we throw away the information con-

tained in the detail space, thus we keep only the output of H2(s) which is half the

information contained in x[n], we still get a PSNR of 50.3dB for x2[n]. Note that

in terms of quality factors, H2(s) had a better chance to be practical than H1(s), so

depending on the application reconstructing x2[n] might be good enough.

Chapter 5

Conclusions and Future Research

This is a short chapter in which I will discuss my conclusions of three years working

on this research as well as suggestions on how future research works can pick up where

this leaves off.

5.1 Conclusions

I started to work on this research motivated about how the theory of Discrete Sam-

pling, that had been developed by my adviser Brad Osgood and fellow graduate

student William Wu for finite length discrete signals, could be put to practice to

process very long signals, music in particular. Coming from a 2 year background re-

searching on the design of data converters, ADCs and DACs, I wanted to apply it to

sampling generic signals. Music offered the challenge to see whether the theory could

be applied to signals sampled at CD quality rate, 44.1Khz, thus of relatively low

bandwidth up to 22Khz. At that point, I was aware of other techniques that tried

to sample signals at rates below Nyquist, most notably Compressed Sensing. These

techniques however, had important challenges related mostly to practical implemen-

tation. In particular, ADCs based on Compressed Sensing, although they take the

signal samples at rates below the Nyquist rate, require that the analog signal, prior

to taking signal samples, be mixed with, or multiplied by, a digital signal whose rate

is that of Nyquist or higher and whose jitter has great impact in the quality of the

91

92 CHAPTER 5. CONCLUSIONS AND FUTURE RESEARCH

reconstructed signal. Thus, although they effectively sample signals at a lower rate,

the jitter requirements placed on this digital signal made it more difficult to design

general purpose ADCs with Compressed Sensing than with existing techniques. To

me, the fact that Discrete Sampling requires sampling signals at rates strictly below

Nyquist’s was a big motivator. I wanted that any conversion technique that could

come out of Discrete Sampling had in mind the technical limitations of clock jitter and

other effects, such as thermal noise and others described in Chapter 1, that limited

practical data converters.

The first obstacle that I encountered was that the dimension of a normal audio signal

is much higher than the dimension of the matrices that can be processed realistically.

This didn’t prevent me from trying to apply the theory as it was developed at the time

I started. I made many simulations of equation (1.8) with matrices of dimension as

big as 32,768 derived from Wavelet Theory. I wanted to compute several interpolating

matrices that I could use to process long signals. These simulations, run in Matlab,

usually took a whole night in multi CPU, 32 Gb RAM Linux machines. Originally I

thought that I could break the signal into pieces that could be processed independently

by somehow choosing on the fly which interpolating matrix to use for a given segment.

It turned out that this wasn’t a good idea, for the reasons detailed in the introduction

of Chapter 3. One day, while examining the structure of the interpolating matrices

obtained with this method, I realized that their columns where shift invariant, just

as the wavelet matrices were. That’s how I got into thinking about the problem

described in section 3.2. Since it became obvious that interpolating from the signal

samples alone didn’t produce reconstructed signals of good fidelity, I began to study

the problem of orthogonal projections and how that could be done in long signals with

the Discrete Sampling framework. This took me into a deeper study of frames, that I

knew from a class in wavelets offered by the Mathematics Department, Math266, that

produced the results described in section 3.3. A very brief introduction of frame theory

was included in Chapter 2 for completeness purposes. Finally, while investigating how

these results could be put into a working circuit, I found out about HFB’s and realized

that the theory I had developed could be used to obtain a novel way, and probably

more practical than existing methods, of designing them. That’s what Chapter 4 is

5.2. FUTURE RESEARCH 93

about.

My concluding remark is that this has been a fascinating journey in which I have

been able to apply knowledge from different areas of Electrical Engineering and

Mathematics- from ADC design to frames and wavelets- that I have acquired through

all my Stanford years.

5.2 Future Research

One of the many good things about doing interesting research, is that once one

masters the topic at hand, all sorts of new problems pop up in one’s mind that could

extend the research work for many more years. At some point one has to present and

defend one’s work in front of a jury of professors to graduate with a PhD so there are

usually many ways in which the research developed could be extended. In my case, I

see the following three areas that could be explorer further, although I am sure, since

sampling is such an interesting field, that other researchers could find more:

1. Finding decompositions of `2(Z) more adapted to a given particular problem.

What this means is that through this work I have kind of “randomly” chosen

decompositions derived from Wavelet Theory with the only criterion that the

orthonormal atoms that spanned the shift invariant subspaces satisfy the con-

ditions of theorem 3.4. It will be interesting to investigate whether there exists

a more mechanical method of choosing the decomposition so that the signals

of interest for a particular problem require fewer subspaces than those present

in the decomposition. In other words, figuring out a way to design decompo-

sitions for which some subspaces contain negligible information for the signals

of interest. This way, not only the design of the resulting HBF is simplified,

but we can decrease the amount of information sampled to below the Nyquist

rate. In addition, as it happened in the example presented in subsection 4.4.1,

it’s likely that the remaining subspaces could be sampled at rates significantly

below Nyquist. In that example it might be sufficient to recover only signal

xφ12ψ12 [n] which requires half the information required by the Nyquist rate and

where each sampler works at one fourth of the Nyquist rate. The idea would

94 CHAPTER 5. CONCLUSIONS AND FUTURE RESEARCH

be to find a way to do this systematically and not just as a result of trial and

error experimentation.

2. Finding wavelet decompositions that result in simpler practical designs. It is

related to point 1., but different. Using the vocabulary of Compressed Sensing,

point 1. is about finding decompositions of `2(Z) which result in a sparse repre-

sentation in terms of the number of subspaces required to represent the signals

of interest. Here we are concerned with the practicality of the decompositions.

As shown in subsection 4.6.2, finding actual analog filters that approximate the

ones present in an HBF designed with Discrete Sampling is challenging. Maybe

there is a way to impose constraints on the wavelet decomposition such as the

order of the filter required to approximate the frequency response of the reversed

dual, u(−t), or the quality factor of such filter. It would be also interesting to

know if this problem and the problem described in 1. can be solved at the same

time or there is any kind of trade off relating the two.

3. Circuit Implementation. Finally, exploring whether the filters that result from

designing an HBF with Discrete Sampling can be easily implemented with ex-

isting circuit technology and methods. A positive answer to this proposition

could make these HBFs mainstream in the world of ADCs. A good starting

point would be to try the implementation of H2(s) obtained in subsection 4.6.2,

since its order is reasonable, the quality factors of its poles are near or well below

the rule of thumb of 10, and, at least for the audio signal under consideration,

by keeping only its output we can obtain a reasonable reconstruction. It could

be implemented first with discrete components, then, if there is success with

this stage, in an integrated circuit.

Note that I do not mention particular applications amongst these topics because the

spirit of my research has been to find a general framework for designing ADCs that

could be applied widely.

Bibliography

[1] B. Murmann, “EE315B VLSI Data Conversion Circuits Lecture Notes,” Stanford

University, Autumn 2010.

[2] C. Shannon, “Communication in the Presence of Noise,” Proceedings of the IRE,

vol. 37, no. 1, pp. 10 – 21, Jan. 1949.

[3] B. Widrow and I. Kollar, Quantization Noise: Roundoff Error in Digital Com-

putation, Signal Processing, Control, and Communications. Cambridge, UK:

Cambridge University Press, 2008.

[4] W. Wu, “Discrete Sampling: Discrete Generalizations of the Nyquist-Shannon

Sampling Theorem,” Ph.D. dissertation, Stanford University, June 2010.

[5] B. Murmann, “ADC Performance Survey 1997-2011.” [Online]. Available:

http://www.stanford.edu/∼murmann/adcsurvey.html

[6] I. Djbkovic and P. Vaidyanathan, “Generalized Sampling Theorems in Multires-

olution Subspaces,” IEEE Transactions on Signal Processing, vol. 45, pp. 583

–599, March 1997.

[7] P. P. Vaidyanathan and B. Vrcelj, “Biorthogonal Partners and Applications,”

IEEE Transactions on Signal Processing, vol. 49, pp. 1013–1027, Nov. 2001.

[8] B. Vrcelj and P. P. Vaidyanathan, “Fractional Biorthogonal Partners in Channel

Equalization and Signal Interpolation,” IEEE Transactions on Signal Processing,

vol. 51, pp. 1928–1940, July 2003.

95

96 BIBLIOGRAPHY

[9] P. P. Vaidyanathan, “Generalizations of the Sampling Theorem: Seven Decades

After Nyquist,” IEEE Transactions on Circuits and Systems I, vol. 48, pp. 1094–

1109, Sep. 2001.

[10] P.P. Vaidyanathan, Multirate Systems and Filter Banks. Prentice Hall, 1993.

[11] Y. C. Eldar, “Uncertainty Relations for Shift-Invariant Analog Signals,” IEEE

Transactions on Information Theory, vol. 55, pp. 5742–5757, Dec. 2009.

[12] Y. Eldar and T. Michaeli, “Beyond Bandlimited Sampling,” IEEE Signal Pro-

cessing Magazine, vol. 26, pp. 48–68, May 2009.

[13] M. Mishali and Y. C. Eldar, “From Theory to Practice: sub-Nyquist Sampling

of Sparse Wideband Analog Signals,” IEEE Journal of Selected Topics in Signal

Processing, vol. 4, pp. 375–391, April 2010.

[14] ——, “Blind Multiband Signal Reconstruction: Compressed Sensing for Analog

Signals,” IEEE Transactions on Signal Processing, vol. 57, pp. 993–1009, March

2009.

[15] E. Candes, J. Romberg, and T. Tao, “Robust Uncertainty Principles: Exact

Signal Reconstruction from Highly Incomplete Frequency Information,” IEEE

Transactions on Information Theory, vol. 52, pp. 489 – 509, Feb. 2006.

[16] D. Donoho, “Compressed Sensing,” IEEE Transactions on Information Theory,

vol. 52, pp. 1289 –1306, April 2006.

[17] J. A. Tropp, J. N. Laska, M. F. Duarte, J. K. Romberg, and R. G. Baraniuk, “Be-

yond Nyquist: Efficient Sampling of Sparse Bandlimited Signals,” IEEE Trans-

actions on Information Theory, vol. 56, pp. 520–544, Jan. 2010.

[18] M. Unser, “Sampling-50 years After Shannon,” Proceedings of the IEEE, vol. 88,

pp. 569 –587, April 2000.

[19] M. Unser and A. Aldroubi, “A General Sampling Theory for Nonideal Acquisition

Devices,” IEEE Transactions on Signal Processing, vol. 42, pp. 2915 –2925, Nov.

1994.

BIBLIOGRAPHY 97

[20] Y. M. Lu and M. N. Do, “A Theory for Sampling Signals From a Union of

Subspaces,” IEEE Transactions on Signal Processing, vol. 56, pp. 2334–2345,

June 2008.

[21] T. Blumensath and M. E. Davies, “Sampling Theorems for Signals From the

Union of Finite-Dimensional Linear Subspaces,” IEEE Transactions on Infor-

mation Theory, vol. 55, pp. 1872–1882, April 2009.

[22] G. G. Walter, “A Sampling Theorem for Wavelet Subspaces,” IEEE Transactions

on Information Theory, vol. 38, pp. 881–884, March 1992.

[23] S. Haddad, S. Bagga, and W. Serdijn, “Log-domain Wavelet Bases,” IEEE Trans-

actions on Circuits and Systems I, vol. 52, pp. 2023 – 2032, Oct. 2005.

[24] Sandro A. P. Haddad, Wouter A. Serdijn, Ultra Low-Power Biomedical Signal

Processing: An Analog Wavelet Filter Approach for Pacemakers. Springer, 2009.

[25] J. E. Marsden and M. J. Hoffman, Elementary Classical Analysis, Second Edi-

tion. W. H. Freeman and Company, 1993.

[26] O. Christensen, Frames and Bases, An Introductory Course. Birkhauser, 2008.

[27] J. Higgins, Sampling Theory in Fourier and Signal Analysis Foundations. Ox-

ford University Press, 1996.

[28] S. Mallat, A Wavelet Tour of Signal Processing, Second Edition. Academic

Press, 1999.

[29] Alan V. Oppenheim, Ronald W. Schafer and John R. Buck, Discrete Time Signal

Processing Second Edition. Prentice Hall, 1999.

[30] C. Eoff, “The Discrete Nature of the Paley-Wiener Spaces,” Proceedings of The

American Mathematical Society, vol. 123, no. 2, pp. 505–512, 1995.

[31] [Online]. Available: http://www.mathworks.com/help/toolbox/signal/ref/hann.

html

98 BIBLIOGRAPHY

[32] W. Chen, S. Itoh, and J. Shiki, “On Sampling in Shift Invariant Spaces,” IEEE

Transactions on Information Theory, vol. 48, pp. 2802–2810, Oct. 2002.

[33] R. M. Gray, “Toeplitz and Circulant Matrices: A review,” Foundations and

Trends in Communications and Information Theory, vol. 2, pp. 155–239, 2006.

[34] C. Lelandais-Perrault, T. Petrescu, D. Poulton, P. Duhamel, and J. Oksman,

“Wideband, Bandpass, and Versatile Hybrid Filter Bank A/D Conversion for

Software ,” IEEE Transactions on Circuits and Systems I, vol. 56, pp. 1772–

1782, Aug. 2009.

[35] A. Papoulis, “Generalized Sampling Expansion,” IEEE Transactions on Circuits

and Systems, vol. 24, pp. 652 – 654, Nov. 1977.

[36] J. Brown, J., “Multi-Channel Sampling of Low-pass Signals,” IEEE Transactions

on Circuits and Systems, vol. 28, pp. 101 – 106, Feb. 1981.

[37] S. Velazquez, T. Nguyen, and S. Broadstone, “Design of Hybrid Filter Banks for

Analog/Digital Conversion,” IEEE Transactions on Signal Processing, vol. 46,

pp. 956 –967, April 1998.

[38] Z. Song, C. Lelandais-Perrault, and P. Benabes, “Synthesis of Complex Subband

Hybrid Filter Banks A/D Converters Using Adaptive Filters,” in 16th IEEE

International Conference on Electronics, Circuits, and Systems, 2009. ICECS

2009, Dec. 2009, pp. 399 –402.

[39] D. Albuquerque, S. Soldado, J. Vieira, and N. Carvalho, “Cochlear Radio,” in

2010 European Wireless Technology Conference (EuWIT), Sep. 2010, pp. 209

–212.

[40] [Online]. Available: http://www.cirquedusoleil.com/en/shows/ka/default.aspx

[41] B. Murmann, “EE315A VLSI Signal Conditioning Circuits Lecture Notes,” Stan-

ford University, Spring 2011.

BIBLIOGRAPHY 99

[42] R. Schaumann, H. Xiao, and V. Valkenburg, Design of Analog Filters, Second

Edition. Oxford University Press, 2009.

[43] [Online]. Available: http://www.analog.com/library/analogDialogue/archives/

43-09/EDCh%208%20filter.pdf

[44] S. Darlington, “A History of Network Synthesis and Filter Theory for Circuits

Composed of Resistors, Inductors, and Capacitors,” IEEE Transactions onCir-

cuits and Systems, vol. CAS-11, pp. 1–13, Jan. 1984.

[45] A. M. Cohen, Numerical Methods for Laplace Transform Inversion. Springer,

2007.

[46] [Online]. Available: http://www.mathworks.com/help/toolbox/signal/ref/

invfreqs.html

[47] [Online]. Available: http://en.wikipedia.org/wiki/Gauss%E2%80%93Newton

algorithm

100 BIBLIOGRAPHY

Fernando Gomez-Pancorbo

I certify that I have read this dissertation and that, in my opinion, it

is fully adequate in scope and quality as a dissertation for the degree

of Doctor of Philosophy.

(Brad Osgood) Principal Adviser




(John Pauly)




(Julius Smith)

Approved for the University Committee on Graduate Studies

discrete sampling in continuous time, a dissertationch771fw4323/... · 2013. 6. 17. · john pauly,...

Documents