1 lecture 2 sampling of systems graham c. goodwin by centre for complex dynamic systems and control...

75
1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the “Zaborszky Distinguished Lecture Series” December 3 rd , 4 th and 5 th , 2007

Upload: robert-barber

Post on 23-Dec-2015

218 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

1

Lecture 2

Sampling of Systems

Graham C. Goodwinby

Centre for Complex Dynamic Systems and ControlUniversity of Newcastle, Australia

Presented at the “Zaborszky Distinguished Lecture Series”December 3rd, 4th and 5th, 2007

Page 2: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

2

Overview

• High performance signal processing and control depends, inter-alia, on the availability of accurate models to represent systems

• Underlying physical system typically continuous• Modern data recording equipment inevitably uses

some form of sampling and quantization• This raises the question of the relationship

between the sampled data and the underlying continuous time system

• We will study this question for linear and nonlinear systems

Page 3: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

3

Outline

1. The Elements of Sampling2. Sampled Data Models for Linear Deterministic Systems (A first look)3. Shift Operator4. Delta Operator5. Some advantages of delta models6. Mid course correction; beyond Euler integration7. More accurate sampled data models for deterministic linear systems8. Sampled data models for stochastic linear systems9. CAR estimation revisited10. Sampled data models for deterministic nonlinear systems11. Sampled data models for stochastic nonlinear systems12. Conclusions

Page 4: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

4

Question

How do sampled signals interact with an analogue physical system?

Two Issues:

i. D/A conversion at input side (via some form of hold)

ii. A/D conversion at output side (including anti-aliasing filtering)

Page 5: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

5

The Elements

PhysicalSystem

Hold PresampleFilter

Page 6: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

6

Outline

1. The Elements of Sampling2. Sampled Data Models for Linear Deterministic Systems (A first look)3. Shift Operator4. Delta Operator5. Some advantages of delta models6. Mid course correction ; beyond Euler integration7. More accurate sampled data models for deterministic linear systems8. Sampled data models for stochastic linear systems9. CAR estimation revisited10. Sampled data models for deterministic nonlinear systems11. Sampled data models for stochastic nonlinear systems12. Conclusions

Page 7: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

7

Deterministic Linear Systems

Once the hold and presample filter have been specified it is easy to obtain exact sampled data models for linear case.

ZOH input

• Continuous-time description:

• Discrete-time model:

[ )( )

,

u t ukt k k

=

Î D D +D

( ) ( ) ( )

( ) ( )

dx t Ax t Bu t

dty t Cx t

= +

=

1x A x B uk q k q k

y C xk q k

= ++

=

0

AA eq

AB e Bdq

C Cq

h h

ì Dï =ïïïï Dïï =íïïïï =ïïïî

ò

Page 8: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

8

Outline

1. The Elements of Sampling2. Sampled Data Models for Linear Deterministic Systems (A first look)3. Shift Operator4. Delta Operator5. Some advantages of delta models6. Mid course correction ; beyond Euler integration7. More accurate sampled data models for deterministic linear systems8. Sampled data models for stochastic linear systems9. CAR estimation revisited10. Sampled data models for deterministic nonlinear systems11. Sampled data models for stochastic nonlinear systems12. Conclusions

Page 9: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

9

Typically, we write the above discrete model in terms of the shift operator as

where ‘q’ is the forward shift operator

k q k q k

k q k

qx A x B u

y C x

= +

=

1k kqx x +=

Page 10: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

10

However, a difficulty with shift operator models is that, with fast sampling, we almost have

Thus, if we consider a model such as

Then, we might anticipate that a 1; b 0 as 0.More generally, if we consider an nth order AR description:

Then the “a coefficients” tend to the Binomial Coefficients.

1k kx x+ =

1k k kx ax bu+ = +

1 1 0k n n k n k ky a y a y bu+ - + -= + + =K

Page 11: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

11

Question

What happens with coefficient quantization (i.e. finite word length representations)?

Consider the following 2 models

Note that the coefficients differ by 1%.

Hence if the coefficients were to be quantized (say to 6 bits) then the models would be identical!

Question: Does this really matter?Maybe the systems are very similar.

( ) [ ]

( ) [ ]

2

2

(a)

(

1.9 0.9025 0

1.9 0.8925 0b)

q q y k

q q y k

- + =

- + =

Page 12: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

12

Surprising Fact

Coefficients differ by 1% yet (a) is stable (b) is unstable.

( ) [ ]

( ) [ ]

2

2

(a)

(

1.9 0.9025 0

1.9 0.8925 0b)

q q y k

q q y k

- + =

- + =

Page 13: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

13

• How can we better represent the system?• Idea: Instead of modelling the absolute

displacement, i.e.,

• How about we model the difference

• This is the core idea in delta domain models.

( )1k ky f y+ =

( )1 'k k ky y f y+ - =

Page 14: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

14

Outline

1. The Elements of Sampling2. Sampled Data Models for Linear Deterministic Systems (A first look)3. Shift Operator4. Delta Operator5. Some advantages of delta models6. Mid course correction ; beyond Euler integration7. More accurate sampled data models for deterministic linear systems8. Sampled data models for stochastic linear systems9. CAR estimation revisited10. Sampled data models for deterministic nonlinear systems11. Sampled data models for stochastic nonlinear systems12. Conclusions

Page 15: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

15

Delta Operator

Core attributes• This centers the calculation by subtracting what is

already known• The same idea is used in many areas:

– Sigma Delta Modulator– Predictive Coding– Optimal Noise Shaping Quantization etc

(See later lectures in this series)

( ) ( )( )

x t x tx td

é ù+D -ê úê úDë û

@

Page 16: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

16

Coefficient Quantization Revisted• Recall shift operator example:

• Coefficients differ by 1% yet (a) is stable (b) is unstable.• Equivalent delta operator models (assuming = 0.1)

• Coefficients differ by 400%. Stability obvious by analogy with continuous time.

( ) [ ]

( ) [ ]

2

2

(a) 1.9 0.9025 0

(b) 1.9 0.8925 0

q q y k

q q y k

- + =

- + =

( ) [ ]

( ) [ ]

2

2

(c) 0.25 0

(d) 0.75 0

y k

y k

d d

d d

+ + =

+ - =

Page 17: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

17

Some History

• 17th Century Numerical Analysis.

• Harriot (teacher of Sir Walter Raleigh) developed accurate interpolation formulae based on finite differences.

• Newton and Stirling in 18th Century developed formal calculus of differences.

• Lagrange and Laplace studied relationships between linear difference equations in shift and delta form.

• Cauchy (1827) developed operational calculus for finite differences.

• Finite difference central to numerical analysis and signal processing.

Page 18: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

18

Many Modern Contributors

• Tschauner (1963)• Halkin (1966) – The Discrete Maximum Principle• Jury (1971) – Unified Stability Theory• Agarwal and Burrus (1975) – Signal Processing• Kitamon (1983) – Unified Control Theory• Orlandi and Martinelli (1984) – Signal Processing• Goodall and Brown (1985) – Microprocessor Implementation• Middleton and Goodwin (1986) Filtering and Control• Goodwin, Gevers, Mayne, Middleton (1988) – Unified Theory

of Stochastic Adaptive Control• Weller, Feuer, Goodwin, Poor (1993) – Unified Lattice

Filtering

Page 19: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

19

State Space EquivalenceThe discrete state space model can be expressedin Delta form as

k k k

k k k

x t x t u t

y t x t u t

A B

C D

where , andq q C C C D D D

2 2

0

12! 3!

e I

e d I

A

A

A

B ΩB

A AΩ

Page 20: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

20

Recall shift model

1

0

( )

I

x t e x t e I u tA AA B

Thus, we recover the continuous model as ( 0).

Also, discrete results expressed in form continuous as ( 0).

1e I x t e I u tx t x tA AA B

A

B

Page 21: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

21

Outline

1. The Elements of Sampling2. Sampled Data Models for Linear Deterministic Systems (A first look)3. Shift Operator4. Delta Operator5. Some advantages of delta models6. Mid course correction ; beyond Euler integration7. More accurate sampled data models for deterministic linear systems8. Sampled data models for stochastic linear systems9. CAR estimation revisited10. Sampled data models for deterministic nonlinear systems11. Sampled data models for stochastic nonlinear systems12. Conclusions

Page 22: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

22

Some Advantages of Models

i. Unification

ii. Numerical Properties

iii. Insight

Page 23: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

23

(i) Unification

• Many (arguably all) results in the literature exhibit a close connection between discrete and continuous:

– Checking discrete stability – Jury type testing in delta domain converges to Routh-Hurwitz as 0. (Jury, Premaratne, …)

– Discrete-time Riccati equations for H2/H optimization converge to continuous-time Riccati equations as 0. (Middleton, Salgado, …)

– Constrained linear-quadratic optimal control. (Feuer, Yuz)

Page 24: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

24

Example 1: Discrete Delta TransformWe define the Discrete Delta Transform pair as:

The Discrete Delta Transform can be related to Z-transform

by noting that

where . Conversely

0

11

1

11

2

k

k

k

D y k Y y k

D Y y k Y dj

1( )q z

Y Y z

11

( ) zqY z Y

( )qY z Z k

Page 25: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

25

Discrete Delta Transform

Page 26: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

26

Example 2: Stability

Z domain

1

domain

x Ax Bu x x

1k k kx Ax Bu 1k kqx x

1 ' 'k kk k

x xA x B u

1k kk

x xx

1

Page 27: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

27

Example 3

Continuous Riccati Discrete Riccati

T T

TI

APA CPC

A CPC1

T T

T

P Ω PA AP H Γ H

H PC S Γ

Page 28: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

28

(ii) Numerical Properties

Output

Time

Comparison of step response of continuous-time systems to the step responses of discrete-time approximation systems using shift and operator implementations of limited word length.

Page 29: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

29

It’s even worse than this suggests!

Actually 3 more significant digits were used in the shift model than in the model!

Page 30: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

30

Comparison of errors for dynamic Riccati equation using delta ( ) and shift (q) operators.

Solving Discrete Riccati Equation

Page 31: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

31

(iii) Insights – Generally all results expressed in terms of delta operator converge to the

continuous time result as 0. Example: State Space Models expressed using -operator

rather than q-operator• Rewriting in terms of -operator:

Then, as 0

i.e., the underlying CT system representation is recovered.

1qd

-=

D

k k k

k k

x A x B u

y Cxd ddì = +ïïíï =ïî

0

where1

An

A

e IA

B e Bd

d

hd h

D

D

ìï -ï =ïï Dïíïï =ïï Dïî ò

dA A B B

t d dd® ® ®

Page 32: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

32

Outline

1. The Elements of Sampling2. Sampled Data Models for Linear Deterministic Systems (A first look)3. Shift Operator4. Delta Operator5. Some advantages of delta models6. Mid course correction ; beyond Euler integration7. More accurate sampled data models for deterministic linear systems8. Sampled data models for stochastic linear systems9. CAR estimation revisited10. Sampled data models for deterministic nonlinear systems11. Sampled data models for stochastic nonlinear systems12. Conclusions

Page 33: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

33

• The ideas presented so far have emphasized the close connections between discrete and continuous cases.

• Indeed, simply replacing (continuous) by (discrete) yields a discrete time model of accuracy . [This corresponds exactly to the use of Euler Integration].

• Is this always adequate?

d

dt1qd

-=

D( )O D

Page 34: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

34

A Motivational Problem• Continuous-time Auto-Regressive (CAR) identification:

where : continuous-time “white noise”• It is tempting to think that, for 0, we could think of

the sampled-data model as being simply

• With : discrete-time “white noise”• Then we could use ordinary least squares to estimate

v&

kw

1

1 01

n n

nn n

d y d ya a y v

dt dt

-

- -+ + + = &K

11 0

n nk n k k ky a y a y wd d -

-+ + + =K

[ ]1 2 0, , ,n na a aq - -= K

Page 35: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

35

• However, this does not happen. Consider:

• “Euler” integration gives:

• However, L.S. estimation using the above approximate sampled-data model gives

0

1 1

3a a

®

2

1 02

d y dya a y v

dt dt+ + = &

21 0ˆ ˆk k k ky a y a y wd d+ + =

Page 36: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

36

• What has gone wrong?It turns our that we need a more accurate representation than Euler which we recall had error O().

• The source of the difficulty is not the use of the delta operator, per se, but a problem due to relative degree as we will see below.

• We will next find alternative sampled data models that are essentially as easy to obtain as these from Euler integration but which have error O(r) where r is the relative degree!

Page 37: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

37

Outline

1. The Elements of Sampling2. Sampled Data Models for Linear Deterministic Systems (A first look)3. Shift Operator4. Delta Operator5. Some advantages of delta models6. Mid course correction ; beyond Euler integration7. More accurate sampled data models for deterministic linear systems8. Sampled data models for stochastic linear systems9. CAR estimation revisited10. Sampled data models for deterministic nonlinear systems11. Sampled data models for stochastic nonlinear systems12. Conclusions

Page 38: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

38

To motivate the subsequent analysis, consider simple case of double integrator

• Continuous state space model is

2

2

d yu

dt=

1 1

2 2

0 1 0

0 0 1

x xdu

x xdt

é ù é ùé ù éùê ú ê úê ú êú= +ê ú ê úê ú êúë û ëûë û ë û

Page 39: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

39

Say that u is generated by a ZOH and we sample output at interval .

• Associated discrete time model

[ ]

21 12 2

2 21

2

1

2

1

2

(exact)

1

2 2

12

0 1

1 0

k k

k

k

k

x x AI A A Bu

x x

xu

x

xy

x

+

ì üé ù é ùì ü ï ïDï ïï ï ï ïê ú ê ú= + D + D + D +í ý í ýê ú ê úï ï ï ïï ïî þë û ë û ï ïî þé ùDé ùé ùD ê úê úê ú= +ê úê úê ú ê úë ûë û Dë û

é ùê ú=ê úë û

K K

Page 40: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

40

Hence exact sampled model has transfer function:

Note that the discrete time model has additional zero dynamics (which are sometimes called “sampling zeros”).

( )

[ ]

( )

1

2

2

12

1

q q q qT z C ZI A B

z

z

-é ù= -ë ûD

+=

-

( ) 2

12Gd

g

gg

D+

=

or in form:

surprise?

Page 41: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

41

More Generally

where Pr denotes the “sampling zero dynamics”.

( )1 rr r

P

s

d

d

Page 42: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

42

Sampling Zeros with Zero Order Hold on Input

Relative Degree

2

3

rP

11

2

211

6

Page 43: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

43

• In general, the sampling zero dynamics are given by the Euler-Fröbenius polynomials (Åström, Hagander and Sternby, 1984)

• In -operator form (Yuz and Goodwin, 2005):

where

( )( )r

r

PGd

gg

g

D=

( ) ( )

1

2

12! !

1det1 !

0 1

r

r

r

r

pr

gg

g

-

-

é ùD Dê úê úê úê úDê ú-D = ê ú-ê úê úê úê ú-ë û

L

L

M O O M

L

This will reappear later

Page 44: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

44

Generalization to arbitrary transfer functions

At high frequencies this model behaves like

Hence expect

Indeed, it can be shown that this approximate model has local truncation error O(r).

(Compare with O() for Euler integration.)

( )( )( )r

F sG s

E s=

( )m

m mc n r

b s bG s

s s=;

( )( )

( )F

GEd

gg

g;

Relative degree is important!

degree relative degree

degree

mr n m

n= -

( )( ) ( )

( )rPF

GEd

gg

g

gD;

Page 45: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

45

Summary

• More accurate models are obtained by going beyond Euler integration.

• These more accurate models have an intuitive representation in delta form.

• We thus advocate– the use of higher order integration which, inter

alia, leads to sampling zero dynamics, and– the use of the delta operator.

Page 46: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

46

Outline

1. The Elements of Sampling2. Sampled Data Models for Linear Deterministic Systems (A first look)3. Shift Operator4. Delta Operator5. Some advantages of delta models6. Mid course correction ; beyond Euler integration7. More accurate sampled data models for deterministic linear systems8. Sampled data models for stochastic linear systems9. CAR estimation revisited10. Sampled data models for deterministic nonlinear systems11. Sampled data models for stochastic nonlinear systems12. Conclusions

Page 47: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

47

Similar Ideas apply to Stochastic Linear Systems

• Stochastic system or noise models:

where v(t) is a Wiener process:

• Exact discrete-time model:

( ){ }2 2vE dv t dts=

1

k q k k

k q k

x A x V

y C x

+ìï = +ïíï =ïî

%

( ) ( ) ( )( ) ( ) ( )( ) ( )

dx t Ax t Bdv ty t H v t

y t Cx tr

ì = +ïï= íï =ïî&

{ } [ ]

2

0

Discrete white noise: he e

w r

Aq

k

Tk q K

TA T Aq v

A e

V

E V V k

e BB e dh h

d

s h

D

D

ìï =ïïïïïïí =W -ïïïïï W =ïïî ò

l

%

%% l

Page 48: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

48

There exists a parallel development for stochastic sampling zero dynamics. Here the anti-aliasing filter plays a dual role to the ZOH for the input signal.

The dual of a ZOH is an integrate and sample filter.

( ) ( )1 t

ty t y dt t

- D=

D ò( )

( )( ) ( )( ) ( )

*

*continu ous

k

y

y y k

B s B ss

A s A s

= D

F =

( )( ) ( ) ( )

( ) ( )

*

*2 discrete y

rB B

A A

Pdg

gd

d dF Þ( )

( ) ( )( ) ( )

*

*discrete y

B B

A A

d dg

d dF Þ

Page 49: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

49

Outline

1. The Elements of Sampling2. Sampled Data Models for Linear Deterministic Systems (A first look)3. Shift Operator4. Delta Operator5. Some advantages of delta models6. Mid course correction ; beyond Euler integration7. More accurate sampled data models for deterministic linear systems8. Sampled data models for stochastic linear systems9. CAR estimation revisited10. Sampled data models for deterministic nonlinear systems11. Sampled data models for stochastic nonlinear systems12. Conclusions

Page 50: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

50

CAR Identification Revisited

• Recall

but LS gave

• Solutions:1. Use asymptotic sampling zeros

2

1 02

d y dya a y v

dt dt+ = &

0

1 1

3a a

®

21 0

1ˆ ˆ 1

3 3k k k ky a y a y wd d d d

æ ö÷ç+ + = + D ÷ç ÷çè ø-

21 0ˆ ˆ? k k k ky a y a y wd d dÞ + + =

Spectral factor of ( )2P dD

Page 51: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

51

Implications:

• Should use Filtered Least Squares

• Then and

( )( )

2

21 01

22

1J y a y a y

Pd d

d

é ùê ú= + +ê úê úDë û

å

1 1a a® 0 0a a®

Page 52: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

52

Outline

1. The Elements of Sampling2. Sampled Data Models for Linear Deterministic Systems (A first look)3. Shift Operator4. Delta Operator5. Some advantages of delta models6. Mid course correction ; beyond Euler integration7. More accurate sampled data models for deterministic linear systems8. Sampled data models for stochastic linear systems9. CAR estimation revisited10. Sampled data models for deterministic nonlinear systems11. Sampled data models for stochastic nonlinear systems12. Conclusions

Page 53: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

53

• Continuous-time nonlinear systems are described by differential equations.

• In general, exact sampled-data models are impossible to obtain for nonlinear case.

• However, approximate discrete-time descriptions can be obtain by numerical integration (analogously to linear case).

Page 54: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

54

• We consider a class of deterministic nonlinear systems

• We assume that the system has nonlinear relative degree r i.e.,

( ) ( )( ) ( )( ) ( )

( ) ( )( )x t f x t g x t u t

y t h x t

= +

=

&

( ) ( )

( )10

0 0, ,for ani

(ii)

2, d

0

kg f

rg f

L L h x k r

L L h x-

= = -

¹

K

Page 55: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

55

• We express the system in normal form:

( ) ( )( )

1 2

1

, ,

,

r r

r

z z

z z

z b a u

q

Vh Vh

h Vh

-

=

=

= +

=

&

M&

&

&

[ ]1 2

1 2

, , ,

, , ,

T

r

T

r r n

z z z

z z z

V

h + +

ìï =ïïíï é ù=ï ë ûïî

& K

Kwhere y = h(x) = z1 and

Page 56: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

56

Core Insight

• The output y = z1 has a chain of r integrators (as in the linear case). Hence one might wonder if one carries out the numerical integration in an appropriate fashion, then we may find additional discrete zero dynamics as in the linear case.

• Actually, this turns out to be true. However, we need to change the order of the Taylor series expansion depending upon the “depth” from y.

Page 57: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

57

• Thus we perform Taylor series expansion for each state as follows:

( ) ( ) ( ) ( ) ( ) ( )

( ) ( ) ( )( )

( ) ( )

( ) ( ) ( ) ( )

( ) ( ) ( )

2

1 1 2 3

1

2 2 3

, ,2 !

, ,1 !

, ,

,

r

t k

r

t k

r r t k

t k

z k z k z k z k b a ur

z k z k z k b a ur

z k z k b a u

k k qx

Vh Vh

Vh Vh

Vh Vh

h h Vh

= D

-

= D

= D

= D

D D é ùD +D = D +D D + D + + +ë û

D é ùD +D = D +D D + + +ë û-

é ùD +D = D +D +ë ûé ùD +D = D +D ë û

K

K

M

Page 58: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

58

• Converting to -form we obtain the following approximate DT model:

( )

( ) ( )

( ) ( )( )

( )

2 1

3 2

0 12 1 ! !

0 0 12 ! 1 !

, ,

0 0 12

0 0 0 1

,

r r

r r

S S S S S S

S S S

r r

r r

b a u

q

dV V V h V h

dh V h

- -

- -

é ù é ùD D Dê ú ê úê ú ê ú-ê ú ê úê ú ê úD Dê ú ê úê ú ê ú- -ê ú ê úê ú ê úê ú ê úê ú ê ú= + +ê ú ê úê ú ê úê ú ê úDê ú ê úê ú ê úê ú ê úê ú ê úê ú ê úê ú ê úê ú ê úë û ë û

=

L

L

M O O M M

L

L

Page 59: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

59

Accuracy

• The local truncation error in the output is of order , i.e.:1r+D

( ) ( ) 1 for some c onstant S ry k y k CC +D - D < ×D

Page 60: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

60

Zero Dynamics

• The DT nonlinear zero dynamics are given by:

– where

– A linear subsystem of dimension r – 1:

( )2:0, ,S S Srq zdh h= %

2: 2 , ,TS S S

r rz z zé ù=ê úë û% % %K

2: 22 2:S S

r rz Q zd =% %

Page 61: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

61

• Moreover, the eigenvalues of the linear subsystem are given by

This is exactly as in the linear case!

That is we have the same (asymptotic) sampling zero dynamicsas for a linear system of the same relative degree r

( ) ( )1 22 1

!det r rr

rI Q pg g- -- = D

D

( ) ( )

1

2

12! !

1det1 !

0 1

r

r

r

r

pr

gg

g

-

-

é ùD Dê úê úê úê úDê ú-D = ê ú-ê úê úê úê ú-ë û

L

L

M O O M

L

Page 62: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

62

Outline

1. The Elements of Sampling2. Sampled Data Models for Linear Deterministic Systems (A first look)3. Shift Operator4. Delta Operator5. Some advantages of delta models6. Mid course correction ; beyond Euler integration7. More accurate sampled data models for deterministic linear systems8. Sampled data models for stochastic linear systems9. CAR estimation revisited10. Sampled data models for deterministic nonlinear systems11. Sampled data models for stochastic nonlinear systems12. Conclusions

Page 63: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

63

• We consider a special class of stochastic (nonlinear) systems:

where dvt are increments of a Wiener process• The diffusion team integral cannot be

interpreted in the usual Riemann-Stieljes sense.

( ) ( ) ( )

( ) ( )00 0

, , ; ,

, ,

t t t t t t

t t

t

dx a t x dt b t x dv y g t x

x x a x d b x dvt t tt t t

= + =

= + +ò ò

Page 64: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

64

• To perform Taylor’s series-like expansions we need to consider the Ito rule of stochastic calculus: Given

and yt = g(t, xt), the usual chain rule has to be modified:

• We also apply the Ito rule to the process:

• Doing this, we obtain Ito-Taylor series expansion:

where depends on double stochastic integrals2sR

( ) ( ), ,t t t tdx a t x dt b t x dv= +

( )2

2

2

1

2

g g gdy dt dx dx

t x x

¶ ¶ ¶= + +

¶ ¶ ¶

( ) ( )00 0

, ,t t

tx x a x d b x dvt t tt t t= + +ò ò

( ) ( )0 0 0 20 0

t ts

tx x a x d b x dv Rtt= + + +ò ò

Page 65: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

65

Different types of integration lead to discrete models of different accuracy; Specifically we will use the following:

• Strong convergence of order > 0, if there exists a constant C > 0 and a sampling period 0 > 0 such that:

• The simplest integration is the Euler-Maruyama approximation:

where and:

• This approximation contains time and Wiener integrals of multiplicity 1 only. Thus, it can be interpreted as an order 0.5 strong Ito-Taylor approximation.

( ){ } 0;kE x k x C gDD - £ D "D <D

( )kx x k= D

( ) ( )1k k k k kx x a x b x v+ = + D + D

{ }

( ){ }2

0kk

kk

k

E vv dv

E vt

D+D

D

ì D =ïïïD = íï D =Dïïîò

Page 66: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

66

• We consider a class of nonlinear stochastic systems expressed in normal form as:

( ) ( )

1 2

1r r

r t

dy dx x dt

dx x dt

dx a X dt b X dv-

= =

=

= +

M

A class of nonlinear stochastic systems having relative degree n

Page 67: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

67

• Expanding using Taylor series of different orders, we obtain the following approximate discrete time model:

( )( ) ( )

1

1 2

1 !1 !

0

12

0 0 1

rr

k k k k k

rr

X X a X b X V

-

+

é ùDé ù ê úDê úD ê úê ú- ê úê ú ê úê ú ê ú= + +ê ú ê úDê ú ê úDê ú ê úê ú ê úê úë û Dê úë û

L

M %O O M

M O

L

Theorem:

The output converges strongly to the true output with order r/2.

( ) ( )1y k x kD = D

Page 68: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

68

Some Observations

• This discrete time model has a vector noise output of dimension r.

• In the linear case, we used spectral factorization (superposition) to reduce this to a scalar input noise source.

• In the nonlinear case, we cannot use superposition.• Nonetheless, it is possible to combine the

stochastic inputs by mimicking the linear case.• Then, we find that the resultant model has

stochastic sampling zero dynamics which parallel those for the linear case.

Page 69: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

69

Outline

1. The Elements of Sampling2. Sampled Data Models for Linear Deterministic Systems (A first look)3. Shift Operator4. Delta Operator5. Some advantages of delta models6. Mid course correction ; beyond Euler integration7. More accurate sampled data models for deterministic linear systems8. Sampled data models for stochastic linear systems9. CAR estimation revisited10. Sampled data models for deterministic nonlinear systems11. Sampled data models for stochastic nonlinear systems12. Conclusions

Page 70: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

70

Conclusions

• We have studied sampled data models for continuous time systems.• We have argued that coefficient quantization can be an issue with

sampled data models expressed in terms of shift operator.• Usually preferable to use Delta operator.• Delta operator provides unification with continuous time results,

improved numerical properties and insight.• Sampled data models generally contain extra sampling zero dynamics.• Sampling zero dynamics can be easily (asymptotically) characterized

in linear case.• Use of asymptotic zero dynamics leads to a model of accuracy O(r).• Ideas can be extended to classes of nonlinear deterministic and

stochastic systems.

Page 71: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

71

References• Interaction between Sampled Signal and Analogue Systems

– R.H. Middleton and G.C. Goodwin, Digital Control and Estimation, Prentice Hall, Englewood Cliffs, 1990.

• Sampled Data Models for Linear Deterministic Systems– A. Feuer and G.C. Goodwin, Sampling in Digital Signal Processing and control,

Birkhäuser, Boston, 1996.• Coefficient Quantization Issues

– R.H. Middleton and G.C. Goodwin, ‘Improved finite word length characteristics in digital control using delta operators’, IEEE Transactions on Automatic Control, Vol.31, No.11, pp.1015-1021,1986.

• Delta Operator– R.M. Goodall and Donoghue, ‘Very high sample rate digital filters using the

delta operator’, Proceedings of IEE, G, Vol.140, No.3, pp.199, 1993.– G.C. Goodwin, R.H. Middleton and H.V. Poor, ‘High-speed digital signal

processing and control’, Proceedings of IEEE, Vol.80, No.2, pp.240-259, 1992.– M.J. Newman And D.G. Holmes, ‘Delta operator digital filter for high

performance inverter applications’, IEEE Transactions on Power Electronics, Vol.18,m No.1, Part2, 2003.

– K. Premaratne and E.I. Jury, ‘Tabular method for determining root distribution of delta operator formulated real polynomials’, IEEE Transactions on Automatic Control, Vol.39, No.2., pp.352-355, 1994.

Page 72: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

72

References• Delta Operator Cont.

– K. Premaratne and E.I. Jury, ‘Application of polynomial array methods to discrete time system stability’, Proceedings of IEE, D, Vol.140, No.3, pp.198-204, 1993.

– K. Premaratne, S. Touset and E.I. Jury, ‘Root distribution of delta operator formulated polynomials’, Proceedings of IEE, D, Vol.147, No.1, pp.1-12, 2000.

– M.E. Salgado, R.H. Middleton and G.C. Goodwin, ‘Connections between continuous and discrete Riccati equation with applications to Kalman filtering’, Proceedings of IEE, D, Vol.135, pp.28-34, 1988.

• Exploiting Connections between Continuous and Discrete Time Models– H.H. Fan, ‘Efficient zero location tests for delta operator based polynomials’, IEEE

Transactions on automatic Control, Vol.42, No.5, pp.722-727, 1997.• Re-evaluation

– E.K. Larsson and T. Söderström, ‘Continuous-time AR parameter estimation using properties of sampled systems’, Proceedings of IFC World congress, Barcelona, 2002.

• Sampling Zeros for Linear Systems– K.J. Åström, P. Hagander and J. Sternby, ‘Zeros of sampled systems’, Automatica,

Vol.29, No.1, pp.31-38, 1984.

Page 73: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

73

References

• Asymptotic Sampling Zeros– B. Wahlberg, ‘Limit results for sampled systems’, International Journal of Control,

Vol.48, No.3, pp.1267-1283, 1988.– S. Weller, W. Moran, B. Ninness and A. Pollington, ‘Sampling zeros and the Euler-

Fröbenius polynomials’, IEEE Transactions on Automatic Control, Vol.46, No.2, pp.340-343, 2001.

• Robustness Issues in System Identification arising from use of Sampled Data Models

– G.C. Goodwin, J.I. Yuz and H. Garnier, ‘Robustness issues in continuous-time system identification from sampled data’, in Proceedings of 16 th IFAC World Congress, Prague, Czech Republic, 2005.

– T. Söderström, H. Fan, B. Carlsson and S. Bigi, ‘Least squares parameter estimation of continuous-time ARX models from discrete-time data’, IEEE Transactions on Automatic Control, Vol.42, No.5, pp.959-973, 1997.

Page 74: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

74

References

• Sampled Data Models for Nonlinear Systems– J.I. Yuz and G.C. Goodwin, ‘On sampled data models for nonlinear systems’, IEEE

Transactions on Automatic Control, Vol.50, No.10, pp.1477-1489, 2005.– J.I. Yuz and G.C. Goodwin, ‘Sampled data models for nonlnear stochastic systems’,

IFAC Symposium on system Identification, Newcastle, Australia, 2006.– S. Monaco and D. Normand-Cyrot, ‘Zero dynamics of aampled nonlinear systems’,

Systems and control Letters, Vol.11, pp.229-234, 1988.– S. Monaco and C. Normand-Cyrot, ‘A unified representation for nonlinear discrete-

time and sampled dynamics’, Journal of Mathematics Systems, Estimation and Control, Vol.7, No.4, pp.477-503, 1997.

– N. Kazantzis and C. Krvaris, ‘System-theoretic properties of sampled-data representations of nonlinear systems obtained via taylor-lie series’, International Journal of Control, Vol.67, No.9, pp.997-1020.1997.

– G.C. Goodwin, J.I. Yuz and M.E. Salgado, ‘Insights into the zero dynamics of sampled-data models for linear and nonlinear stochastic systems’, European Control Conference, Greece, 2-5 July, 2007.

Page 75: 1 Lecture 2 Sampling of Systems Graham C. Goodwin by Centre for Complex Dynamic Systems and Control University of Newcastle, Australia Presented at the

75

Lecture 2

Sampling of Systems

Graham C. GoodwinUniversity of Newcastle

Australia

by

Centre for Complex Dynamic Systems and ControlUniversity of Newcastle, Australia

Presented at the “Zaborsky Distinguished Lecture Series”December 3rd, 4th and 5th, 2007