How do I know the answer if I’m not sure of the

question?

Putting robustness into estimation

K. E. Schubert11/7/00

Familiar Picture?

Basic Problem

Picture of something that has been blurred

If I know how it was blurred then I should be able to clean it up

If system is invertible then I can get the original

x b

A

xb

A†

Familiar Picture

Encountering Resistance

Consider a simpler problem.– Unknown resistor.– Take current and voltage measurements.– Plot them out.– Want to fit a line to the points.

No measurement is perfect.– No exact fit to all the points.– Want “best” fit.

Measured Values

0 2 4 6 8 10 12 14 16 180

2

4

6

8

10

12

voltage [V]

Unknown Resistor

Gauss’ Stellar Problem

Orbit of Ceres. Errors were in people’s measurements Consider distance from the measurements

to the equation to fit minimize the square of this distance

– min ||Ax-b||2

x=(ATA)

-1A

Tb=A

†b

Understanding Solution

In our problem A, b are vectors Finding nearest scaled A to b Projection

b

A

Ax

Ax-b

Resistor Solved

Want to find slope, 1/R i=(1/R)v Ax=b A vector of voltages b vector of currents x is slope 1/R=v

†i

Best line

0 2 4 6 8 10 12 14 16 180

2

4

6

8

10

12

voltage [V]

Unknown Resistor

Reasonable Question

What if I considered v=iR? Errors assumed in v now! R=i

†v

How do the measured resistances compare?

Comparison of Methods

0 2 4 6 8 10 12 14 16 180

2

4

6

8

10

12

voltage [V]

Unknown Resistor

Errors in Both

A has errors (actual is A+dA) Want to minimize distance

– min ||(A+dA)x-b||2

Need to know something about dA Worst dA in bounded region Best dA in bounded region The dA that makes Ax=b consistent

Worst in a Bounded Region

Keep worst case ok, rest will be fine

||dA||< (bounded region) Projection to farthest A+dA

b

A

(A+dA)x

(A+dA)x-bdA

Best in a Bounded Region

Pick best dA but limit options

||dA||< (bounded region) Projection to nearest A+dA

b

A(A+dA)x

(A+dA)x-bdA

Consistent Equation (TLS)

Called Total Least Squares Projection nearest to A and b in new space No bound on dA, as big as need!

b

A(A+dA)x

General Regression Problems

All of the techniques mentioned so far fall into the general category of regression (including least squares)

Find a solution for most by taking the gradient and setting it equal to zero

x=(ATA+I)

-1A

Tb

Equation for , which is solved by finding the roots of the equation (Newton’s or bisection)

Resistor by TLS

0 2 4 6 8 10 12 14 16 180

2

4

6

8

10

12

voltage [V]

Unknown Resistor

Simple Picture

Consider a city skyline.– Only consider outline of buildings.– Height is a function of horizontal distance.

Nice one dimensional picture.

Hazy Day

Smog and haze blur the image.– Rounds the corners off.– Want to get the corners back.

Actual Measured

0 10 20 30 40 50 60 70 80 90 1000

0.5

1

1.5

2

2.5

3

3.5

Least Squares Fails!

Blurring works like a Gaussian distribution Don’t know the exact blur

Measured Least Squares

0 10 20 30 40 50 60 70 80 90 100-2000

-1500

-1000

-500

0

500

1000

1500

2000

Least Squares Solution

TLS Too Optimistic!

TLS assumes things are consistent Allows dA to be large

MeasuredTLS

0 10 20 30 40 50 60 70 80 90 100-60

-40

-20

0

20

40

60

More Robust Solutions

Picking a solution with some restrictions yields good results.

Actual MeasuredMinMin MinBE

0 10 20 30 40 50 60 70 80 90 100-0.5

0

0.5

1

1.5

2

2.5

3

3.5

Sample

Sig

na

l

Conclusions

Least Squares has nice properties and generally works well.

Problems can arise in simple problems.– Fundamental errors

Must account for errors in basic system. Robust ~ works well for all nearby systems

– Can’t do as well or as bad (compromise)