Least Squares Approximation

run

So far we have been tryingto fitdata Cai exactly withfunctions

e.g polynomials splines

But often data is noisy andwhile it may originate from a

simple function e.g axt bdue to noise terror no e.gstraight line may hit the data

I intertfaynomialHi yeaL 1 shy

true function

Can we do better by tryinge.g to find the best

straight line fit

Manypossible notions of betterExamples want to pick ao a to minimize

Cao a meat Yi Capea I

1 E m

minimax quoiETCao a f I y i Cape a I

I

absolute deviation

Cao a If i Ca Ki t aDJleast squares

Each has pros cons

computational complexityrobustness to outliers etc

see book

We will focus on Least squares

Warm up e Linear least squares

regressionTo minimise

TE Cao a 2 Yi a Ki tao

we solve

IIa o IIa o

goZao.hnfsi fapciaoDZof qi Cyi Caxi iaoD

o A Yi Carita dCD

O 2h Yi Caixixao foci

InormalequationTsm m

Zegers 2 unknowns

Go save toget

iEf

ai

we have all the sci's gisso we can find ao a

Polynomial least squaresmmSame idea if Dn Lx

is a polynomial of degreesn B Cx oaixand we have m datapoints oci y i I m

with n C m I we can

minimizeE Csi PnGciD

EE 2EPnGciyitERcxiDin

daotapc azx.pt an KY

Taking derivativesZEJag

O s j n

gives us htt normal equationsmeto solve for ao an

htt unknowns

Normal Equations have a

unique solution when theXi are distinct

Can use the same idea tofit different functions

e.g g beak or g boca

Procedure e i write Eca b

2 ZEA ob

o

3 Solve to find a b