MA/CS 375 Fall 2002 1

MA/CS 375

Fall 2002

Lecture 22

MA/CS 375 Fall 2002 2

Interlude on Norms

• We all know that the “size” of 2 is the same as the size of -2, namely:

• ||2|| = sqrt(2*2) = 2• ||-2|| = sqrt((-2)*(-2)) = 2

• The ||..|| notation is known as the norm function.

MA/CS 375 Fall 2002 3

Norm of A Vector

• We can generalize the scalar norm to a vector norm, however now there are more choices for a vector :

1 2 31

2 2 2 21 2 32

1 2 3

.....

....

max , , ,.... ,

N

N

N

x x x x x

x x x x x

x x x x x

Nx

MA/CS 375 Fall 2002 4

Norm of A Vector

1 2 31

2 2 2 21 2 32

1 2 3

.....

....

max , , ,.... ,

N

N

N

x x x x x

x x x x x

x x x x x

Matlab command:

1) norm(x,1)

2) norm(x,2)

3) norm(x,inf)

MA/CS 375 Fall 2002 5

Norm of A Matrix• We can generalize the vector norm to a

matrix norm, however now there are more choices for a matrix :

1 11

2 21

11

2

1 1

max

max

max

i M

ijj N

i

x

j N

iji M

j

j Mi N

ijFi j

x

A A

A A

A A

A A

M NA

• 1-norm maximum absolute column sum

• infinity-norm, maximum absolute row sum

MA/CS 375 Fall 2002 6

Norm of A Matrix

1 11

2 21

11

2

1 1

max

max

max

i M

ijj N

i

x

j N

iji M

j

j Mi N

ijFi j

x

A A

A A

A A

A A

Matlab command:

1) norm(A,1)

2) norm(A,2)

3) norm(A,inf)

4) norm(A,’fro’)

MA/CS 375 Fall 2002 7

Matrix Norm in Action

• Theorem 5: (page 185 van Loan)– If is the stored version of

then where and

ˆ M NA ˆ A A E

M NA M NE

1 1epsE A

i.e. an error of order ||A||1eps occurs when a real matrix is stored in floating point.

MA/CS 375 Fall 2002 8

Condition Number

• Recall that when we asked Matlab to invert a matrix which was almost singular: we got this warning

MA/CS 375 Fall 2002 9

Definition of Condition Number

• rcond = 1/(condition number)

• We are going to use the 1-norm definition of the condition number given as:

11 1 1

A A A

MA/CS 375 Fall 2002 10

What’s With The Condition Number ?

• Theorem 6: (page 235 van Loan)– If is non-singular and:

In addition if then the stored linear system: is nonsingular and also:

N NA

1 11

1 1

ˆ ˆ1

x x xeps

x x

A

, where , Nx b x b A 1 1eps A

ˆ ( )fl x fl bA

MA/CS 375 Fall 2002 11

In Plain English• The theorem supposes that we can solve the Ax=b

problem without making any mistakes except for the stored approximation of A and b then:– If the condition number is small enough, then the

difference between the exact answer and the calculated answer will be bounded above by: const*condition#*eps

– Matlab value of eps is approximately 1e-16– So if A has a condition number of 1 then we can

expect to solve Ax=b to 16 decimal places at best– If A has a condition number of 1000 then in the worst

case we could make an error of 1e-13– If A has a condition number of 10^16 then we could

make O(1) errors

MA/CS 375 Fall 2002 12

cond in Matlab

• cond(A,1) will return the 1-norm condition number

• cond(A,2) will return the 2-norm condition number

MA/CS 375 Fall 2002 13

Team Exercise

• Build

• For delta=1,0.1,0.01,…,1e-16 compute:• cond(A,1)

• Plot a loglog graph of the•x=delta, y=condition number

• Figure out what is going on

1

11

A

MA/CS 375 Fall 2002 14

Team Exercise (theory)

12

1 11

1 11 1

A A

1

1max abs column sum 1

1

1max 1 , 1

1 for small

A1

21

3 2

3

11

max abs column sum 11

1 11 1max ,

1 for small

A

MA/CS 375 Fall 2002 15

Team Exercise (theory)

1

1max abs column sum 1

1

1max 1 , 1

1 for small

A1

21

3 2

3

11

max abs column sum 11

1 11 1max ,

1 for small

A

11 41 1

1

A A A

MA/CS 375 Fall 2002 16

Team Exercise (theory)

11 41 1

1

A A A

Pretty big huh Did your results concur?

MA/CS 375 Fall 2002 17

Team Exercise (theory)

11 41 1

1

A A A

Pretty big huh

Remark: remember how when we set delta = 2^(-11) and we could actually get the exact inverse. Well in this case the condition number is a little pessimistic as a guide !.

MA/CS 375 Fall 2002 18

Enough Theory !

MA/CS 375 Fall 2002 19

Interpolation

• Question: – someone gives you a function

evaluated at 10 points, how does the the function behave between those 10 points?

• Answer:– guesses

MA/CS 375 Fall 2002 20

Interpolation

• Question: – someone gives you a function

evaluated at 10 points, how does the the function behave between those 10 points?

• Answer:– there is no unique answer,

but we can make a good guess

MA/CS 375 Fall 2002 21

Polynomial Interpolation

• What’s the highest order unique polynomial that you can fit through a function evaluated at 1 point?

• That would be a constant function with the same value as the given function value.

MA/CS 375 Fall 2002 22

Polynomial Interpolation

• What’s the highest order unique polynomial that you can fit through a function evaluated at 2 points?

• That would be a linear function that passes through the two values:

MA/CS 375 Fall 2002 23

LinearInterpolation

MA/CS 375 Fall 2002 24

LinearInterpolation

exp(x)

Linear fit

Samples

MA/CS 375 Fall 2002 25

General Monomial Interpolation

• Given N function values at N distinct points then there is one unique polynomial which passes through these N points and has order (N-1)

• Guess what – we can figure out the coefficients of this polynomial by solving a system of N equations

MA/CS 375 Fall 2002 26

2nd Order Polynomial Fit

21 2 3

21 1 2 1 3 1

22 1 2 2 3 2

23 1 2 3 3 3

f

f

f

f

x a a x a x

x a a x a x

x a a x a x

x a a x a x

• we are going to use a 2nd order approximation

• let’s make sure that the approximation agrees with the sample at:

• x1,x2 and x3

MA/CS 375 Fall 2002 27

2nd Order Polynomial Fit

21 1 2 1 3 1

22 1 2 2 3 2

23 1 2 3 3 3

f

f

f

x a a x a x

x a a x a x

x a a x a x

We know:

• x1,x2 and x3

• f(x1),f(x2) and f(x3)

We do not know:

• a1,a2 and a3

MA/CS 375 Fall 2002 28

2nd Order Polynomial Fit

0 1 21 1 1 1 10 1 22 2 2 2 20 1 23 3 3 3 3

f

f

f

x x x a x

x x x a x

x x x a x

We know:

• x1,x2 and x3

• f(x1),f(x2) and f(x3)

We do not know:

• a1,a2 and a3re-written as system

MA/CS 375 Fall 2002 29

Finally A Reason To Solve A System

0 1 21 1 10 1 22 2 20 1 23 3 3

1

2

3

f

f

f

x x x

x x x

x x x

x

x

x

V

f

So we can build thefollowing then: a = V\f

MA/CS 375 Fall 2002 30

Expansion Coefficients

• Once we have the coefficients a1,a2,a3 then we are able to evaluate the quadratic interpolating polynomial anywhere.

21 2 3f x a a x a x

MA/CS 375 Fall 2002 31

Cla

ss E

xerc

ise

• Part 1: – Build a function called vandermonde.m which accepts

a vector of x values and a polynomial order P– In the function find N=length of x– Function returns a matrix V which is Nx(P+1) and

whose entries are: V(n,m) = (xn)(m-1)

• Part 2:– Translate this pseudo-code to Matlab a script:

– for N=1:5:20• build x = set of N points in [-1,1]• build f = exp(x)• build xfine = set of 10N points in [-1,1]• build Vorig = vandermonde(N-1,x)• build Vfine = vandermonde(N-1,xfine)• build Finterp = Vfine*(Vorig\f);• plot x,f and xfine,Finterp on the same graph

– end

MA/CS 375 Fall 2002 32

Next Lecture

• If you have a digital camera bring it in

• If not, bring in some jpeg, gif or tif pictures (make sure they are appropriate)

• We will use them to do some multi-dimensional interpolation

• Estimates for accuracy of polynomial interpolation