lecture16_inner1

Some Applications

Inner Product, Length, Orthogonality

Orthogonalization

Least Squares Problems

Inner Product Spaces

Orthogonality and Least Squares

Least-Sqaures Problem

Some Applications

b Ax b Ax

Least-Sqaures Lines (Linear Regression)

Some Applications

Least-Sqaures polynomials (Polynomial Regression)

Some Applications

Fourier Series

Any wave can be expressed as a series of sines and cosines

Some Applications

x

f x , x f x 2 f x2

French Scientist Jean-Baptiste Joseph Fourier

1768 - 1830

Fourier Series

Any wave can be expressed as a series of sines and cosines

Some Applications

0n n

n 1

af x a cos nx b sin nx

2

x

f x , x f x 2 f x2

Fourier Series

Any wave can be expressed as a series of sines and cosines

Some Applications

n 1

n 1

1f x sin nx

n

x

f x , x f x 2 f x2

sin x

Fourier Series

Any wave can be expressed as a series of sines and cosines

Some Applications

n 1

n 1

1f x sin nx

n

x

f x , x f x 2 f x2

12

sin x sin 2x

Fourier Series

Any wave can be expressed as a series of sines and cosines

Some Applications

n 1

n 1

1f x sin nx

n

x

f x , x f x 2 f x2

1 12 3

sin x sin 2x sin 3x

Fourier Series

Any wave can be expressed as a series of sines and cosines

Some Applications

x

f x , x f x 2 f x2

n 1

n 1

1f x sin nx

n

http://upload.wikimedia.org/wikipedia/commons/e/e8/Periodic_identity_function.gif

Some Applications of Fourier Series

1. Approximation Theory

2. Data compression

Some Applications

-0.60

-0.40

-0.20

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Am

pli

tud

e

Frequency Index

Frequncy Spectrum

n 1

n 1

1f x sin nx

n

Some Applications of Fourier Series

3. Signal Processing and Filter Design

Some Applications

Stereo Equalizer

Some Applications of Fourier Series

4. Analysis of Electric Circuits

Some Applications

Some Applications of Fourier Series

5. Solution of PDE

Vibrating String

Some Applications

2 22

2 2

y yc

x t

Some Applications of Fourier Series

5. Solution of PDE

Vibrating String

Some Applications

2 22

2 2

y yc

x t

https://www.youtube.com/watch?v=9L9AOPxhZwY

Inner Product in Rn

For U, V Rn,

The inner (dot) product is defined as:

1 1

2 2

n n

u v

u vU ,V

u v

Inner Product, Length, Orthogonality

T1 1 2 2 n nU V U V u v u v u v

Ex. Find U·V, U·W,

1 2 2

2 3 3U ,V ,W

1 1 1

3 1 1

Inner Product, Length, Orthogonality

U V 1 2 2 3 1 1 3 1 2

U W 1 2 2 3 1 1 3 1 0

Theorem

For U, V, W Rn and c R,

1. V·U =

2. (U+V)·W =

3. (cU)·V=

4. U·U

Inner Product, Length, Orthogonality

U V

U W V W

U (cV) c(U V)

0 ,U U 0 if and only if U 0

Length (norm) of a Vector For U Rn ,

The length of U is defined as: 22 2 2 T

1 2 nU U U u u u and U U U U U

1

2

n

u

uU

u

Inner Product, Length, Orthogonality

Unit Vector A unit vector is a vector whose length is 1

The unit vector in direction of V,

Ex. Find the unit vector in direction of V,

VU

V

1

2V

4

2

Inner Product, Length, Orthogonality

Note that

For a scalar c,

15

25

45

25

1

2V 1U

4V 5

2

161 4 425 25 25 25U 1

Inner Product, Length, Orthogonality

V 1 4 16 4 5

cV c V

1

2U

4

2

Distance Between Two Vectors

The distance between the U, V Rn, is the length of the

vector U-V, that is

dist U,V U V

Inner Product, Length, Orthogonality

2 2 2

1 1 2 2 n nu v u v u v

Orthogonal Vectors U and V are orthogonal, if and only if

Note that, in R2 and R3,

dist U,V dist U, V

Inner Product, Length, Orthogonality

2 2

dist U,V dist U, V

U V U V U V U V

U V 0

U Vcos

U V

90 if U V 0

Theorem For U and V Rn, the following statements are equivalent,

• U and V are orthogonal

• U·V = 0

• dist(U,V) = dist(U,-V)

• ǁU+Vǁ2 = ǁUǁ2 + ǁVǁ2

Inner Product, Length, Orthogonality

(Pythagorean Theorem)

Ex. Are U and V orthogonal?

U and V are orthogonal

Inner Product, Length, Orthogonality

U V 4 5 3 6 2 1 0

4 5

U 3 ,V 6

2 1

Orthogonal Complement • If a vector z is orthogonal to every vector in a subspace

W of Rn, then z is said to be orthogonal to W

• The set of all vectors z that are orthogonal to W is called the orthogonal complement of W and is denoted by W

Ex. For a plane W passing through the

origin in R3, and the line L passing

through the origin and orthogonal to W

W = L

L = W

Inner Product, Length, Orthogonality

Theorem

For a matrix A,

• (Row A)=

• (Col A)=

Inner Product, Length, Orthogonality

Nul A

Nul AT

Orthogonal Set A set of nonzero vectors S={V1,V2,…Vp} in Rn is an orthogonal set if Vi·Vj =0, ij

Ex. IS S={V1,V2,V3} an orthogonal set?

V1·V2 = V1·V3 = V2·V3 = 0

S is an orthogonal set

Inner Product, Length, Orthogonality

1 2 3

3 1 3

2 3 8V ,V ,V

1 3 7

3 4 0

Orthonormal Set A set of nonzero vectors S={U1,U2,…Up} in Rn is an orthonormal set if Ui·Uj =0, ij and ǁUiǁ=1 for all i

Ex. IS S={U1,U2,U3} an orthonormal set?

U1·U2 = U1·U3 = U2·U3 = 0, ǁU1ǁ=ǁU2ǁ=ǁU3ǁ=1

S is an orthonormal set

Inner Product, Length, Orthogonality

1 2 23 3 3

2 2 11 2 33 3 3

2 1 23 3 3

U ,U ,U

Orthogonal Basis If S={V1,V2,…Vp} is an orthogonal set of nonzero vectors in Rn, then S is linearly independent (orthogonal basis for Span{S})

Orthonormal Basis If S={U1,U2,…Up} is an orthonormal set of nonzero vectors in Rn, then S is linearly independent (orthonormal basis for Span{S})

Inner Product, Length, Orthogonality

Ex. Given the following vectors,

a. Are V1,V2,V3 linearly independent?

b. Do V1,V2,V3 form a basis for R3?

1 2 3

2 0 0

V 0 ,V 3 ,V 0

0 0 4

Yes

Yes (orthogonal basis)

Inner Product, Length, Orthogonality