1 lecture 2 sampling of systems graham c. goodwin by centre for complex dynamic systems and control...

Lecture 2

Sampling of Systems

Graham C. Goodwinby

Centre for Complex Dynamic Systems and ControlUniversity of Newcastle, Australia

Presented at the “Zaborszky Distinguished Lecture Series”December 3rd, 4th and 5th, 2007

Overview

• High performance signal processing and control depends, inter-alia, on the availability of accurate models to represent systems

• Underlying physical system typically continuous• Modern data recording equipment inevitably uses

some form of sampling and quantization• This raises the question of the relationship

between the sampled data and the underlying continuous time system

• We will study this question for linear and nonlinear systems

Outline

1. The Elements of Sampling2. Sampled Data Models for Linear Deterministic Systems (A first look)3. Shift Operator4. Delta Operator5. Some advantages of delta models6. Mid course correction; beyond Euler integration7. More accurate sampled data models for deterministic linear systems8. Sampled data models for stochastic linear systems9. CAR estimation revisited10. Sampled data models for deterministic nonlinear systems11. Sampled data models for stochastic nonlinear systems12. Conclusions

Question

How do sampled signals interact with an analogue physical system?

Two Issues:

i. D/A conversion at input side (via some form of hold)

ii. A/D conversion at output side (including anti-aliasing filtering)

The Elements

PhysicalSystem

Hold PresampleFilter

Outline

1. The Elements of Sampling2. Sampled Data Models for Linear Deterministic Systems (A first look)3. Shift Operator4. Delta Operator5. Some advantages of delta models6. Mid course correction ; beyond Euler integration7. More accurate sampled data models for deterministic linear systems8. Sampled data models for stochastic linear systems9. CAR estimation revisited10. Sampled data models for deterministic nonlinear systems11. Sampled data models for stochastic nonlinear systems12. Conclusions

Deterministic Linear Systems

Once the hold and presample filter have been specified it is easy to obtain exact sampled data models for linear case.

ZOH input

• Continuous-time description:

• Discrete-time model:

[ )( )

u t ukt k k

Î D D +D

( ) ( ) ( )

( ) ( )

dx t Ax t Bu t

dty t Cx t

1x A x B uk q k q k

y C xk q k

AB e Bdq

ì Dï =ïïïï Dïï =íïïïï =ïïïî

Outline

Typically, we write the above discrete model in terms of the shift operator as

where ‘q’ is the forward shift operator

k q k q k

qx A x B u

1k kqx x +=

However, a difficulty with shift operator models is that, with fast sampling, we almost have

Thus, if we consider a model such as

Then, we might anticipate that a 1; b 0 as 0.More generally, if we consider an nth order AR description:

Then the “a coefficients” tend to the Binomial Coefficients.

1k kx x+ =

1k k kx ax bu+ = +

1 1 0k n n k n k ky a y a y bu+ - + -= + + =K

Question

What happens with coefficient quantization (i.e. finite word length representations)?

Consider the following 2 models

Note that the coefficients differ by 1%.

Hence if the coefficients were to be quantized (say to 6 bits) then the models would be identical!

Question: Does this really matter?Maybe the systems are very similar.

( ) [ ]

1.9 0.9025 0

1.9 0.8925 0b)

q q y k

Surprising Fact

Coefficients differ by 1% yet (a) is stable (b) is unstable.

( ) [ ]

1.9 0.9025 0

1.9 0.8925 0b)

q q y k

• How can we better represent the system?• Idea: Instead of modelling the absolute

displacement, i.e.,

• How about we model the difference

• This is the core idea in delta domain models.

( )1k ky f y+ =

( )1 'k k ky y f y+ - =

Outline

Delta Operator

Core attributes• This centers the calculation by subtracting what is

already known• The same idea is used in many areas:

– Sigma Delta Modulator– Predictive Coding– Optimal Noise Shaping Quantization etc

(See later lectures in this series)

( ) ( )( )

x t x tx td

é ù+D -ê úê úDë û

Coefficient Quantization Revisted• Recall shift operator example:

• Coefficients differ by 1% yet (a) is stable (b) is unstable.• Equivalent delta operator models (assuming = 0.1)

• Coefficients differ by 400%. Stability obvious by analogy with continuous time.

( ) [ ]

(a) 1.9 0.9025 0

(b) 1.9 0.8925 0

q q y k

( ) [ ]

(c) 0.25 0

(d) 0.75 0

Some History

• 17th Century Numerical Analysis.

• Harriot (teacher of Sir Walter Raleigh) developed accurate interpolation formulae based on finite differences.

• Newton and Stirling in 18th Century developed formal calculus of differences.

• Lagrange and Laplace studied relationships between linear difference equations in shift and delta form.

• Cauchy (1827) developed operational calculus for finite differences.

• Finite difference central to numerical analysis and signal processing.

Many Modern Contributors

• Tschauner (1963)• Halkin (1966) – The Discrete Maximum Principle• Jury (1971) – Unified Stability Theory• Agarwal and Burrus (1975) – Signal Processing• Kitamon (1983) – Unified Control Theory• Orlandi and Martinelli (1984) – Signal Processing• Goodall and Brown (1985) – Microprocessor Implementation• Middleton and Goodwin (1986) Filtering and Control• Goodwin, Gevers, Mayne, Middleton (1988) – Unified Theory

of Stochastic Adaptive Control• Weller, Feuer, Goodwin, Poor (1993) – Unified Lattice

Filtering

State Space EquivalenceThe discrete state space model can be expressedin Delta form as

x t x t u t

y t x t u t

where , andq q C C C D D D

12! 3!

Recall shift model

x t e x t e I u tA AA B

Thus, we recover the continuous model as ( 0).

Also, discrete results expressed in form continuous as ( 0).

1e I x t e I u tx t x tA AA B

Outline

Some Advantages of Models

i. Unification

ii. Numerical Properties

iii. Insight

(i) Unification

• Many (arguably all) results in the literature exhibit a close connection between discrete and continuous:

– Checking discrete stability – Jury type testing in delta domain converges to Routh-Hurwitz as 0. (Jury, Premaratne, …)

– Discrete-time Riccati equations for H2/H optimization converge to continuous-time Riccati equations as 0. (Middleton, Salgado, …)

– Constrained linear-quadratic optimal control. (Feuer, Yuz)

Example 1: Discrete Delta TransformWe define the Discrete Delta Transform pair as:

The Discrete Delta Transform can be related to Z-transform

by noting that

where . Conversely

D y k Y y k

D Y y k Y dj

1( )q z

( ) zqY z Y

( )qY z Z k

Discrete Delta Transform

Example 2: Stability

Z domain

domain

x Ax Bu x x

1k k kx Ax Bu 1k kqx x

1 ' 'k kk k

x xA x B u

Example 3

Continuous Riccati Discrete Riccati

APA CPC

A CPC1

P Ω PA AP H Γ H

H PC S Γ

(ii) Numerical Properties

Output

Comparison of step response of continuous-time systems to the step responses of discrete-time approximation systems using shift and operator implementations of limited word length.

It’s even worse than this suggests!

Actually 3 more significant digits were used in the shift model than in the model!

Comparison of errors for dynamic Riccati equation using delta ( ) and shift (q) operators.

Solving Discrete Riccati Equation

(iii) Insights – Generally all results expressed in terms of delta operator converge to the

continuous time result as 0. Example: State Space Models expressed using -operator

rather than q-operator• Rewriting in terms of -operator:

Then, as 0

i.e., the underlying CT system representation is recovered.

x A x B u

y Cxd ddì = +ïïíï =ïî

where1

B e Bd

ìï -ï =ïï Dïíïï =ïï Dïî ò

dA A B B

t d dd® ® ®

Outline

• The ideas presented so far have emphasized the close connections between discrete and continuous cases.

• Indeed, simply replacing (continuous) by (discrete) yields a discrete time model of accuracy . [This corresponds exactly to the use of Euler Integration].

• Is this always adequate?

D( )O D

A Motivational Problem• Continuous-time Auto-Regressive (CAR) identification:

where : continuous-time “white noise”• It is tempting to think that, for 0, we could think of

the sampled-data model as being simply

• With : discrete-time “white noise”• Then we could use ordinary least squares to estimate

d y d ya a y v

- -+ + + = &K

n nk n k k ky a y a y wd d -

-+ + + =K

[ ]1 2 0, , ,n na a aq - -= K

• However, this does not happen. Consider:

• “Euler” integration gives:

• However, L.S. estimation using the above approximate sampled-data model gives

d y dya a y v

dt dt+ + = &

21 0ˆ ˆk k k ky a y a y wd d+ + =

• What has gone wrong?It turns our that we need a more accurate representation than Euler which we recall had error O().

• The source of the difficulty is not the use of the delta operator, per se, but a problem due to relative degree as we will see below.

• We will next find alternative sampled data models that are essentially as easy to obtain as these from Euler integration but which have error O(r) where r is the relative degree!

Outline

To motivate the subsequent analysis, consider simple case of double integrator

• Continuous state space model is

é ù é ùé ù éùê ú ê úê ú êú= +ê ú ê úê ú êúë û ëûë û ë û

Say that u is generated by a ZOH and we sample output at interval .

• Associated discrete time model

21 12 2

(exact)

x x AI A A Bu

ì üé ù é ùì ü ï ïDï ïï ï ï ïê ú ê ú= + D + D + D +í ý í ýê ú ê úï ï ï ïï ïî þë û ë û ï ïî þé ùDé ùé ùD ê úê úê ú= +ê úê úê ú ê úë ûë û Dë û

é ùê ú=ê úë û

Hence exact sampled model has transfer function:

Note that the discrete time model has additional zero dynamics (which are sometimes called “sampling zeros”).

q q q qT z C ZI A B

-é ù= -ë ûD

or in form:

surprise?

More Generally

where Pr denotes the “sampling zero dynamics”.

( )1 rr r

Sampling Zeros with Zero Order Hold on Input

Relative Degree

• In general, the sampling zero dynamics are given by the Euler-Fröbenius polynomials (Åström, Hagander and Sternby, 1984)

• In -operator form (Yuz and Goodwin, 2005):

( )( )r

( ) ( )

1det1 !

é ùD Dê úê úê úê úDê ú-D = ê ú-ê úê úê úê ú-ë û

M O O M

This will reappear later

Generalization to arbitrary transfer functions

At high frequencies this model behaves like

Hence expect

Indeed, it can be shown that this approximate model has local truncation error O(r).

(Compare with O() for Euler integration.)

( )( )( )r

F sG s

m mc n r

b s bG s

( )( )

Relative degree is important!

degree relative degree

degree

mr n m

( )( ) ( )

( )rPF

Summary

• More accurate models are obtained by going beyond Euler integration.

• These more accurate models have an intuitive representation in delta form.

• We thus advocate– the use of higher order integration which, inter

alia, leads to sampling zero dynamics, and– the use of the delta operator.

Outline

Similar Ideas apply to Stochastic Linear Systems

• Stochastic system or noise models:

where v(t) is a Wiener process:

• Exact discrete-time model:

( ){ }2 2vE dv t dts=

k q k k

x A x V

+ìï = +ïíï =ïî

( ) ( ) ( )( ) ( ) ( )( ) ( )

dx t Ax t Bdv ty t H v t

y t Cx tr

ì = +ïï= íï =ïî&

{ } [ ]

Discrete white noise: he e

Tk q K

TA T Aq v

E V V k

e BB e dh h

ìï =ïïïïïïí =W -ïïïïï W =ïïî ò

There exists a parallel development for stochastic sampling zero dynamics. Here the anti-aliasing filter plays a dual role to the ZOH for the input signal.

The dual of a ZOH is an integrate and sample filter.

( ) ( )1 t

ty t y dt t

D ò( )

( )( ) ( )( ) ( )

*continu ous

B s B ss

A s A s

( )( ) ( ) ( )

( ) ( )

*2 discrete y

d dF Þ( )

( ) ( )( ) ( )

*discrete y

d dF Þ

Outline

CAR Identification Revisited

• Recall

but LS gave

• Solutions:1. Use asymptotic sampling zeros

d y dya a y v

dt dt+ = &

1ˆ ˆ 1

3 3k k k ky a y a y wd d d d

æ ö÷ç+ + = + D ÷ç ÷çè ø-

21 0ˆ ˆ? k k k ky a y a y wd d dÞ + + =

Spectral factor of ( )2P dD

Implications:

• Should use Filtered Least Squares

• Then and

( )( )

1J y a y a y

é ùê ú= + +ê úê úDë û

1 1a a® 0 0a a®

Outline

• Continuous-time nonlinear systems are described by differential equations.

• In general, exact sampled-data models are impossible to obtain for nonlinear case.

• However, approximate discrete-time descriptions can be obtain by numerical integration (analogously to linear case).

• We consider a class of deterministic nonlinear systems

• We assume that the system has nonlinear relative degree r i.e.,

( ) ( )( ) ( )( ) ( )

( ) ( )( )x t f x t g x t u t

y t h x t

( ) ( )

0 0, ,for ani

L L h x k r

L L h x-

• We express the system in normal form:

( ) ( )( )

z b a u

[ ]1 2

ìï =ïïíï é ù=ï ë ûïî

Kwhere y = h(x) = z1 and

Core Insight

• The output y = z1 has a chain of r integrators (as in the linear case). Hence one might wonder if one carries out the numerical integration in an appropriate fashion, then we may find additional discrete zero dynamics as in the linear case.

• Actually, this turns out to be true. However, we need to change the order of the Taylor series expansion depending upon the “depth” from y.

• Thus we perform Taylor series expansion for each state as follows:

( ) ( ) ( ) ( ) ( ) ( )

( ) ( ) ( )( )

( ) ( )

( ) ( ) ( ) ( )

( ) ( ) ( )

1 1 2 3

, ,2 !

, ,1 !

r r t k

z k z k z k z k b a ur

z k z k z k b a ur

z k z k b a u

k k qx

h h Vh

D D é ùD +D = D +D D + D + + +ë û

D é ùD +D = D +D D + + +ë û-

é ùD +D = D +D +ë ûé ùD +D = D +D ë û

• Converting to -form we obtain the following approximate DT model:

( ) ( )

( ) ( )( )

0 12 1 ! !

0 0 12 ! 1 !

0 0 12

0 0 0 1

S S S S S S

dV V V h V h

dh V h

é ù é ùD D Dê ú ê úê ú ê ú-ê ú ê úê ú ê úD Dê ú ê úê ú ê ú- -ê ú ê úê ú ê úê ú ê úê ú ê ú= + +ê ú ê úê ú ê úê ú ê úDê ú ê úê ú ê úê ú ê úê ú ê úê ú ê úê ú ê úê ú ê úë û ë û

M O O M M

Accuracy

• The local truncation error in the output is of order , i.e.:1r+D

( ) ( ) 1 for some c onstant S ry k y k CC +D - D < ×D

Zero Dynamics

• The DT nonlinear zero dynamics are given by:

– where

– A linear subsystem of dimension r – 1:

( )2:0, ,S S Srq zdh h= %

2: 2 , ,TS S S

r rz z zé ù=ê úë û% % %K

2: 22 2:S S

r rz Q zd =% %

• Moreover, the eigenvalues of the linear subsystem are given by

This is exactly as in the linear case!

That is we have the same (asymptotic) sampling zero dynamicsas for a linear system of the same relative degree r

( ) ( )1 22 1

!det r rr

rI Q pg g- -- = D

( ) ( )

1det1 !

é ùD Dê úê úê úê úDê ú-D = ê ú-ê úê úê úê ú-ë û

M O O M

Outline

• We consider a special class of stochastic (nonlinear) systems:

where dvt are increments of a Wiener process• The diffusion team integral cannot be

interpreted in the usual Riemann-Stieljes sense.

( ) ( ) ( )

( ) ( )00 0

, , ; ,

t t t t t t

dx a t x dt b t x dv y g t x

x x a x d b x dvt t tt t t

= + +ò ò

• To perform Taylor’s series-like expansions we need to consider the Ito rule of stochastic calculus: Given

and yt = g(t, xt), the usual chain rule has to be modified:

• We also apply the Ito rule to the process:

• Doing this, we obtain Ito-Taylor series expansion:

where depends on double stochastic integrals2sR

( ) ( ), ,t t t tdx a t x dt b t x dv= +

g g gdy dt dx dx

¶ ¶ ¶= + +

¶ ¶ ¶

( ) ( )00 0

, ,t t

tx x a x d b x dvt t tt t t= + +ò ò

( ) ( )0 0 0 20 0

tx x a x d b x dv Rtt= + + +ò ò

Different types of integration lead to discrete models of different accuracy; Specifically we will use the following:

• Strong convergence of order > 0, if there exists a constant C > 0 and a sampling period 0 > 0 such that:

• The simplest integration is the Euler-Maruyama approximation:

where and:

• This approximation contains time and Wiener integrals of multiplicity 1 only. Thus, it can be interpreted as an order 0.5 strong Ito-Taylor approximation.

( ){ } 0;kE x k x C gDD - £ D "D <D

( )kx x k= D

( ) ( )1k k k k kx x a x b x v+ = + D + D

( ){ }2

E vv dv

ì D =ïïïD = íï D =Dïïîò

• We consider a class of nonlinear stochastic systems expressed in normal form as:

( ) ( )

dy dx x dt

dx x dt

dx a X dt b X dv-

A class of nonlinear stochastic systems having relative degree n

• Expanding using Taylor series of different orders, we obtain the following approximate discrete time model:

( )( ) ( )

1 !1 !

k k k k k

X X a X b X V

é ùDé ù ê úDê úD ê úê ú- ê úê ú ê úê ú ê ú= + +ê ú ê úDê ú ê úDê ú ê úê ú ê úê úë û Dê úë û

M %O O M

Theorem:

The output converges strongly to the true output with order r/2.

( ) ( )1y k x kD = D

Some Observations

• This discrete time model has a vector noise output of dimension r.

• In the linear case, we used spectral factorization (superposition) to reduce this to a scalar input noise source.

• In the nonlinear case, we cannot use superposition.• Nonetheless, it is possible to combine the

stochastic inputs by mimicking the linear case.• Then, we find that the resultant model has

stochastic sampling zero dynamics which parallel those for the linear case.

Outline

Conclusions

• We have studied sampled data models for continuous time systems.• We have argued that coefficient quantization can be an issue with

sampled data models expressed in terms of shift operator.• Usually preferable to use Delta operator.• Delta operator provides unification with continuous time results,

improved numerical properties and insight.• Sampled data models generally contain extra sampling zero dynamics.• Sampling zero dynamics can be easily (asymptotically) characterized

in linear case.• Use of asymptotic zero dynamics leads to a model of accuracy O(r).• Ideas can be extended to classes of nonlinear deterministic and

stochastic systems.

References• Interaction between Sampled Signal and Analogue Systems

– R.H. Middleton and G.C. Goodwin, Digital Control and Estimation, Prentice Hall, Englewood Cliffs, 1990.

• Sampled Data Models for Linear Deterministic Systems– A. Feuer and G.C. Goodwin, Sampling in Digital Signal Processing and control,

Birkhäuser, Boston, 1996.• Coefficient Quantization Issues

– R.H. Middleton and G.C. Goodwin, ‘Improved finite word length characteristics in digital control using delta operators’, IEEE Transactions on Automatic Control, Vol.31, No.11, pp.1015-1021,1986.

• Delta Operator– R.M. Goodall and Donoghue, ‘Very high sample rate digital filters using the

delta operator’, Proceedings of IEE, G, Vol.140, No.3, pp.199, 1993.– G.C. Goodwin, R.H. Middleton and H.V. Poor, ‘High-speed digital signal

processing and control’, Proceedings of IEEE, Vol.80, No.2, pp.240-259, 1992.– M.J. Newman And D.G. Holmes, ‘Delta operator digital filter for high

performance inverter applications’, IEEE Transactions on Power Electronics, Vol.18,m No.1, Part2, 2003.

– K. Premaratne and E.I. Jury, ‘Tabular method for determining root distribution of delta operator formulated real polynomials’, IEEE Transactions on Automatic Control, Vol.39, No.2., pp.352-355, 1994.

References• Delta Operator Cont.

– K. Premaratne and E.I. Jury, ‘Application of polynomial array methods to discrete time system stability’, Proceedings of IEE, D, Vol.140, No.3, pp.198-204, 1993.

– K. Premaratne, S. Touset and E.I. Jury, ‘Root distribution of delta operator formulated polynomials’, Proceedings of IEE, D, Vol.147, No.1, pp.1-12, 2000.

– M.E. Salgado, R.H. Middleton and G.C. Goodwin, ‘Connections between continuous and discrete Riccati equation with applications to Kalman filtering’, Proceedings of IEE, D, Vol.135, pp.28-34, 1988.

• Exploiting Connections between Continuous and Discrete Time Models– H.H. Fan, ‘Efficient zero location tests for delta operator based polynomials’, IEEE

Transactions on automatic Control, Vol.42, No.5, pp.722-727, 1997.• Re-evaluation

– E.K. Larsson and T. Söderström, ‘Continuous-time AR parameter estimation using properties of sampled systems’, Proceedings of IFC World congress, Barcelona, 2002.

• Sampling Zeros for Linear Systems– K.J. Åström, P. Hagander and J. Sternby, ‘Zeros of sampled systems’, Automatica,

Vol.29, No.1, pp.31-38, 1984.

References

• Asymptotic Sampling Zeros– B. Wahlberg, ‘Limit results for sampled systems’, International Journal of Control,

Vol.48, No.3, pp.1267-1283, 1988.– S. Weller, W. Moran, B. Ninness and A. Pollington, ‘Sampling zeros and the Euler-

Fröbenius polynomials’, IEEE Transactions on Automatic Control, Vol.46, No.2, pp.340-343, 2001.

• Robustness Issues in System Identification arising from use of Sampled Data Models

– G.C. Goodwin, J.I. Yuz and H. Garnier, ‘Robustness issues in continuous-time system identification from sampled data’, in Proceedings of 16 th IFAC World Congress, Prague, Czech Republic, 2005.

– T. Söderström, H. Fan, B. Carlsson and S. Bigi, ‘Least squares parameter estimation of continuous-time ARX models from discrete-time data’, IEEE Transactions on Automatic Control, Vol.42, No.5, pp.959-973, 1997.

References

• Sampled Data Models for Nonlinear Systems– J.I. Yuz and G.C. Goodwin, ‘On sampled data models for nonlinear systems’, IEEE

Transactions on Automatic Control, Vol.50, No.10, pp.1477-1489, 2005.– J.I. Yuz and G.C. Goodwin, ‘Sampled data models for nonlnear stochastic systems’,

IFAC Symposium on system Identification, Newcastle, Australia, 2006.– S. Monaco and D. Normand-Cyrot, ‘Zero dynamics of aampled nonlinear systems’,

Systems and control Letters, Vol.11, pp.229-234, 1988.– S. Monaco and C. Normand-Cyrot, ‘A unified representation for nonlinear discrete-

time and sampled dynamics’, Journal of Mathematics Systems, Estimation and Control, Vol.7, No.4, pp.477-503, 1997.

– N. Kazantzis and C. Krvaris, ‘System-theoretic properties of sampled-data representations of nonlinear systems obtained via taylor-lie series’, International Journal of Control, Vol.67, No.9, pp.997-1020.1997.

– G.C. Goodwin, J.I. Yuz and M.E. Salgado, ‘Insights into the zero dynamics of sampled-data models for linear and nonlinear stochastic systems’, European Control Conference, Greece, 2-5 July, 2007.

Lecture 2

Sampling of Systems

Graham C. GoodwinUniversity of Newcastle

Australia

Centre for Complex Dynamic Systems and ControlUniversity of Newcastle, Australia

Presented at the “Zaborsky Distinguished Lecture Series”December 3rd, 4th and 5th, 2007

1 lecture 2 sampling of systems graham c. goodwin by centre for complex dynamic systems and control...

Documents

microsoft word 2010 - goodwin college home - goodwin college

microsoft excel 2010 tutorial - goodwin college home -...

goodwin 1980

pat e. goodwin pat goodwin associates...

newcastle systems reseller presentation

2069 uon cdsc inside 20.3 - university of newcastle...rick...

newcastle 2030 newcastle 2030

1 lecture 1 sampling of signals by graham c. goodwin...

information strategy of newcastle university paul hopkins...

presentation goodwin

harold goodwin

goodwin procter philip h. newman goodwin procter...goodwin...

goodwin, graebe, salgado ©, prentice hall 2000 chapter 12...

andrew goodwin

suzanne goodwin

database systems marcus kaiser school of computing science...

1.0summary - newcastle university, newcastle upon … ·...

goodwin - action_and_embodiment.pdf

marcus goodwin

goodwin growthcycle