afosr workshop, dayton, oh, august 4-6, 1999 1 m. g. safonov robust control, feedback and learning...

AFOSR Workshop, Dayton, OH, August 4-6, 1999 1 M. G. Safonov

Robust Control, Feedback and

LearningMichael G. Safonov

University of Southern California


The Problem• Control theory relies too much on models:

– Step 1: System ID to obtain model– Step 2: Design controller for model

• System ID relies too much on assumptions:– Assume noise probabilities & model structure– Optimize model fit to data & assumptions

• Models & assumptions can be wrong• Need theory that responds appropriately

when observed data falsifies prior belief


The Approach

• Let the data speak...

• Unfalsify (validate) models and controllers against hard criteria:– Choose criteria expressible directly in terms of

observed data (sensor outputs, actuator inputs)– Avoid criteria that that rely on “noise model”

and other prior beliefs


UnfalsificationAlgorithm

Brugarolas & Safonov, CCA/CACSD ‘99

UNFALSIFIED CONTROL:

The ability of each candidate controller to meet the performance goal is treated as a hypothesis to be tested directly against evolving real-time measurement data. The controller need not be in the loop to test the hypothesis.

UnfalsifiedHypotheses

D-H Test

FalsificationTest

pass

Goal

Data

Hypotheses

pass

Tests: D-H Test: Tests data-

hypothesis consistency(Def . 3).

Falsification Test(Thm. 1).

fail

fail

Choosebest

opt

FalsifiedHypotheses


Plato introspective learning

Data approximates Unobserved TRUTH

Reality is an ideal, observable only through noisy sensors.

‘Probabilistic Estimation’

Two views on how we learn: Introspection vs. Observation

vs.

Galileo open-eyed learning

MODELS approximate Observed Data

Reality is what we observe.

‘Curve-Fitting’


• Traditional control theory (‘Platonic’):– contains many assumptions about the plant.– some assumptions are unrealistic.

• Unfalsified control theory (‘Galilean’):– eliminates hypotheses that are not consistent

with evolving experiment data.


Impact on SYSID

• Statisticians treat prior probabilities as part of model to be validated/unfalsified – validation via one- ‘confidence intervals’– ‘Platonic’ probability becomes ‘Galilean’

• Ljung (e.g., CDC ’97) proposed ‘confidence interval’ reinterpretation of SYSID– model validation/unfalsification


Two Views of SYSID: Curve-Fitting vs. Estimation

CURVE FIT: (Galilean)Given data (yi,ui), i=1,2,...

iii bauy

21||||min

(y1,u1)

(y2,u2)

(y3,u3)

(y4,u4)

(y5,u5)

(y6,u6)

(y7,u7)

height 2 {

vbauy

BAYESIAN ESTIMATE (Platonic): Given data (yi,ui), i=1,2,...

‘noise’ v=N(0, )

), ba(y |x,max probsubject to prior beliefs


CURVE FIT (Galilean):The Galilean may reject model if 2/3 of future data fails to fall in his prior 2 confidence bound

If the fit is bad, do you reject the data or the model?

(y1,u1)

(y2,u2)

(y3,u3)

(y4,u4)

(y5,u5)

(y6,u6)

(y7,u7)

height 2 {

BAYESIAN ESTIMATE (Platonic): The Platonist may reject data if 2/3 of future data fails to fall in his prior 2 confidence bound


candidate controllers

FALSIFIED

COMPUTER SIEVE

LEARNING FEEDBACK LOOPS

K

UnfalsifiedControllers

K

evolving I/O datagiven

M. G. Safonov. In Control Using Logic-Based Switching, Spring-Verlag, 1996.

How about Validation-Based Direct Controller ID?


Canonical Representation

Signals:

• manifest

control signal

measurement signal

• latent

reference signal

performance signal

controller signals

controller parameters

EXTENDED PLANT

PLANTACTUATOR SENSOR

dA dP n

UnfalsifiedController

u y

yK uK

ControlArchitecture

r

p

LearningProcessor

ControllerK

KK yu

p

r

y

u

,

Brugarolas & Safonov, CCA/CACSD ‘99


Formulation in Truncated Space

• Observations operator maps input-output signals to measurement signals.

• Truncated Space results from applying the observations operator to a signal space.

Basic sets:goalshypothesishypothesesdata

ZPzPJzPZ goals 1)(0

ZPzPKzPZ hypothesis 0)()(

ZPhypotheses 2: Z

ZPyuzPzPZ tsmeasuremenmanifestdata ),(

P

ZPZ


Formulation in Truncated Space, cont.

Problem 1 (Truncated Space Unfalsified Control Problem): Given a performance specification (goal set), a set of candidate controllers (hypotheses), and experimental data then determine the subset of candidate controllers (hypothesis) which is not falsified.

goalhypothesisdata ZZZ )(


Main Results

• Definition (Data-hypothesis consistency): Given a truncated space unfalsified control problem, we say that a hypothesis is consistent with the data if

).(hypothesisZdataZ ZPZP

manifestmanifest

• Theorem 1 (Truncated space unfalsified control): Given truncated space unfalsified control problem, then a candidate control law (hypothesis) consistent with the experimental data is unfalsified by data if and only if

.)( goalhypothesisdata ZZZ


—————

Jun & Safonov, CCA/CACSD ‘99

Example: Adaptive PID

dttff

ruwyrw2

0

2

222

2

2

1

)( where

*)(*

Performance Spec:


Adaptation Algorithm• Procedure (at each time t = kt) :

1. Measure u(kt) and y(kt).

2. For each • calculate and

• calculate

• if then delete the controller index element i from I; else continue.

3. If the set I is empty, terminate; else set the current to

and increment time.

))1((~ tkxi )(~ tkri

),(~

tkiJ ),(

~tkiJ

} ),,(~

{min arg )(ˆ ,)(ˆ I itkiJki

kiK

Ii


Controller Parameter Adaptation

• The i-th candidate PID controller Ki is unfalsified if and only if

where],,0[ ,0),(~ ttiJ

0 ))(),(),(~(),(~

0

t

ispecdxxuxyxrtiJ T

i for the signal reference fictitious :)(~dataplant past measured :)(),(

Ktr

tytu

i


Simulation



Simulation


control input u(t)

proportional gain kP (t)

integral gain kI (t)

derivative gain kD (t),

plant output y(t)

Evolution of unfalsified set Time responses

Tsao & Safonov, IEEE Trans, AC-42, 1997.


Example: Missile Autopilot• Learns control gains

• Adapts quickly to compensate for damage & failures

• Superior performance

Brugarolas, Fromion & Safonov, ACC ’98

Specified target response bound

Actual response

Commanded response

Brugarolas, Fromion & Safonov, ACC ’98

Unfalsified adaptive missile autopilot:• discovers stabilizing control gains as it flies, nearly instantaneously• maintains precise sure-footed control


Robot Example:

Tsao & Safonov, CCA/CACSC’99Tsao & Safonov, CCA/CACSD ’99


Other People’s Successes with Unfalsified Control

• Emmanuel Collins et al. (Weigh Belt Feeder adaptive PID tuning, CDC99)

• Kosut (Semiconductor Mfg. Process run-to-run tuning, CDC98)

• Woodley, How & Kosut (ECP Torsional disk control, adaptive tuning, ACC99

• … maybe others soon?


• Unfalsified Control Learning/Adaptation• Adaptation = Direct Controller ID• Unfalsified control focuses on the knowable

– ‘Galilean’ emphasis on observation – precise information in each datum– real-time learning (the essence of feedback)

• Practical, reliable adaptive feedback control• Greater tolerance of evolving uncertainties, failures &

battle damage

goalhypothesisdata ZZZ )(

Conclusions


Acknowledgment

Bob Kosut’s mid-1980’s work on time-domain model validation and identification for control played a key role in laying the foundations of this work, as did later contributions of Jim Krause, Pramod Khargonekar, Carl Nett, Kamashwar Poolla, Roy Smith and many others who have advanced the use of validation methods in control-oriented identification. Tom Mitchell’s early 1980’s “candidate elimination algorithm’’ for machine learning is closely related to the unfalsified control methods presented here. And of course, none of this would have been possible without the superb graduate education that I received at MIT so many years ago under the guidance of first Jan Willems and later Michael Athans.


References[1]M. G. SafonovandT. C. Tsao. Theunfalsi¯edcontrol concept and learning.

IEEE Trans. Autom. Control, 42(6):843{847, J une1997.

[2]M. G. Safonov. Focusing on the knowable: Controller invalidation andlearning. In A. S. Morse, editor, Control Using Logic-Based Switching,pages224{233. Springer-Verlag, Berlin, 1996.

[3]M. J un and M. G. Safonov. Automatic PID tuning: An application ofunfalsi¯ed control. In Proc. IEEE CCA/ CACSD, KohalaCoast{Island ofHawaii, HI, August 22-27, 1999(to appear).

[4]P. B. Brugarolas and M. G. Safonov. A canonical representation for unfal-si¯ed control in truncated spaces. In Proc. IEEE CCA/ CACSD, KohalaCoast{Island of Hawaii, HI, August 22-27, 1999(toappear).

[5]T.-C. Tsaoand M. G. Safonov. Unfalsi¯ed direct adpativecontrol of a two-link robot arm. In Proc. IEEE CCA/ CACSD, Kohala Coast{Island ofHawaii, HI, August 22-27, 1999(to appear).

Also, see webpage http://routh.usc.edu

afosr workshop, dayton, oh, august 4-6, 1999 1 m. g. safonov robust control, feedback and learning...

Documents