Chapter 6 Eigenvalues

Outlines

System of linear ODE (Omit) Diagonalization Hermitian matrices Ouadratic form Positive definite matrices

Motivations To simplify a linear dynamics such

that it is as simple as possible. To realize a linear system characteristics

e.g.,

the behavior of system dynamics.

x Ax

zecharacteripofroot

bapeqch

byyay

)(

0)(..

0

2

§6-1 Eignvalues & Eignvectors

Example: In a town, each year 30% married women get divorced 20% single women get married In the 1st year, 8000 married women 2000

single women. Total population remains constant

, where

represent the numbers of married & single

women after i years, respectively.

,

,

Let ,m ii

s i

ww

w

, ,&m i s iw w

0 1

8000 0.7 0.2 &

2000 0.3 0.8i iw w w

If

Question: Why does converge?

Why does it converge to the same limit vector even when the initial condition is different?

1 0

1 12 12

0.7 0.2 8000,

0.3 0.8 2000

6000 4000, , & , 12

4000 6000

i i i

n

w Aw w w

w W w w n

0 14

14

10000 4000 (steady-state vector)

0 6000

, 14i

w w

w w i

iw

Ans: Choose a basis

Given an initial for some

for example,

Question: How does one know choosing such a basis?

1 2 1 1 2 2

2 1 1, &

3 1 2x x Ax x Ax x

1

1)4000(

3

22000

2000

8000

0 0 1 1 2 2w w c x c x 1 2&c c

0 1 1 2 2 1 1 n n nA w c A x c A x c x as n

Def: Let . A scalar is said to be an

eigenvalue or characteristic value of A if

such that

The vector is said to be an eigenvector

or characteristic vector belonging to .

is called an eigen pair of A.

Question: Given A, How to compute eigenvalues

& eigenvectors?

Ax x

nnFA

0x

x

( , )x

is an eigen pair of

Note that, is a polynomial, called

characteristic polynomial of A, of degree n in

Thus, by FTA, A has exactly n eigenvalues including

multiplicities. is a eigenvector associated

with eigenvalue while is eigenspace of A.

, 0

( ) 0, 0

( ) is singular

det( ) 0

Ax x x

A I x x

A I

A I

nnFA

.( ) det( )p A I

( ) \ 0x N A I

)( IAN

( , )x

Example:

To find the eigenspace of 2:( i.e., )

2

4 2Let

1 1

( ) det( )

4 2 det

1 1

5 6 ( 2)( 3)

2 & 3 are eignvalues of

A

p A I

A

1

2

2 2 0( 2 ) 0

1 1 0

1( 2 )

1

xA I x

x

N A I span

)2( IAN

To find the eigenspace of 3(i.e., )

Let

1

2

1 2 0( 3 ) 0

1 2 0

2( 3 )

1

xA I x

x

N A I span

)3( IAN

1

1 2

1 1

2 0

0 3

p

p Ap

Let , then

( ) is an eignvalue of ;

( ) ( ) 0 has a nontrivil solution;

( ) ( ) 0

( ) is singular;

( ) det( ) 0

( ) ( ) < ;

( ) ( ) 1

i A

ii A I x

iii N A I

iv A I

v A I

vi rank A I n

vii Nullity A I

nnFA

Let .

If is an eigenvalue of A with eigenvector

Then

This means that is also an eigen-pair of

A.

Ax x

Ax Ax Ax x x

nnA

.x

C

( , )x

Let .

where are eigenvalues of A.

(i) Let

(ii) Compare with the coefficient of

, we have

nnA

1( ) det( ) ( )

n

ii

p A I

n 1

n

iiA

1

det0

1 1( 1)n n )(

11

Atracean

iii

n

ii

Theorem 6.1.1: Let A & B be n×n matrices, if B is similar to A, then and consequently A & B have the same eigenvalues.

Pf: Let for some nonsingular matrix S.

1

1

1

( ) det( )

det( )

det ( )

det( )det( )det( )

det( ) ( )

B

A

P B I

S AS I

S A I S

S A I S

A I P

)()( BA PP

ASSB 1

§6-3 Diagonalization

Diagonalization

Goal: Given find a nonsingular matrix

S, such that is a diagonal

matrix.

Question1: Are all matrices diagonalizable?

Question2: What kinds of A are diagonalizable?

Question3: How to find S if A is diagonalizable?

nnFA

DASS 1

NOT all matrices are diagonalizable

e.g., Let

If A is diagonalizable,

nonsingular matrix S,

00

10A

2

11

0

0

d

dASS

1 2

1 2

1 2

1 1

2

( ) 0

det( ) 0

0

0

0

trace A d d

A d d

d d

dA S S O

d

To answer Q2,

Suppose that A is diagonalizable.

nonsingular matrix S,

Let

This gives a condition for diagonalizability and a way to

find S.

1 nS S S

AS SD

)( 11

ndddiagDASS

1 nAS AS

1 1 n nd S d S

, , 1, , , are eigenpairs of A. i id S i n

Theorem6.3.1: If are distinct eigenvalues of an

matrix A with corresponding eigenvectors ,

then are linearly independent.

Pf: Suppose that are linearly dependent

not all zero,

Suppose that

.... golw

2 1

1 12

( ) 0

( ) 0

kk

i i ii i

k

ii

A I c x

c x

1, , k nn

1

0k

i ii

c x

1, , kx x

1, , kx x

1, , kx x

1 kc c

1 0c

1

1

1

contradiction!

0 & are distinct.

0 ( )

are linear independent.

!i

k

x

c

x x

Theorem 6.3.2: Let is diagonalizable

A has n linearly independent eigenvectors.

Note : Similarity transformation

Change of coordinate

diagonalization

( ) ( )

V V

E E

L

A

B

1A SBS

( ) ( )

V V

F F

LSI

nnFA

y Px

1I S

Remarks: Let , and

(i) is an eigenpair of A for

(ii) The diagonalizing matrix S is not unique because

Its columns can be reordered or multiplied by

an nonzero scalar

(iii) If A has n distinct eigenvalues , A is diagonalizable.

If the eigenvalues are not distinct , then may or may

not diagonalizable depending on whether A has n

linearly independent eigenvectors or not.

(iv)

1SDSA )( 1 ndiagD 1 nS S S

i iS

ni ,,1

1 11 , .k k k k

nA SD S S diag S k

Example: Let

For

For

Let

112

202

213

A

det 0 0, 1, 1A I

1

1

1

)0(,0 SpanIAN

1

2

0

,

0

2

1

)(,1 SpanIAN

1

1 1 0 0 0 0

1 2 2 0 1 0

1 0 1 0 0 1

S D S AS

Def: If an matrix A has fewer than n linearly independent eigenvectors,we say that A is defective.

e.g.

(i) is defective

(ii) is defective

1 1 0

0 1 1

0 0 1

A

n n

00

10A

Example 4: Let A & B both have the same eigenvalues Nullity (A-2I)=1 The eigenspace associated with has only one dimension. A is NOT diagonalizable However, Nullity (B-2I)=2 B is diagonalizable

263

041

002

&

201

040

002

BA

4,2,2

2

001

020

000

)2(

rankIArank

1

063

021

000

)2(

rankIBrank

2

Question:

Are the following matrices diagonalizable ?

00

00

01

)(

00

10

01

)( BiiAi

The Exponential of a Matrix

Motivation:The general solution of

is

The unique solution of

is

Question: What is and how to compute ?

.

x Ax

( ) Atx t e c

0 0( )

x Ax

x t x

0( )0( ) A t tx t e x

Ate Ate

Note that

Define

0 !i

ia

i

ae

......!2!

2

0

AAI

i

Ae

i

iA

Suppose that A is diagonalizable with

1SDSA

1 ,k kA SD S k

1

1

0 0

1

1

( )! !

0

0

n

i i i iAt

i i

t

t

Dt

A t D te S S

i i

e

S S

e

Se S

Example 6: Compute

Sol: The eigenvalues of A are

with eigenvectors

0,1

1 2 3 1 0 ;

1 1 0 0A XDX X D

1 2

2 3

1 1x and x

2 6, for

1 3Ate A

1 10

0 1

3 2 6 6

1 3 2

A D ee Xe X X X

e e

e e

§6-4 Hermitian Matrices

Hermitian matrices

Let , then A can be written as

, where

e.g. , 2 3

4 7 2

2 3 1 1

4 7 1 2

i iA

i i

i

m nA C iCBA , m nB C R

Let , then

e.g. ,

ii

ii

ii

iiH

273

42

274

32

m nA C TH AA

HHH

HHH

HH

ACAC

BABA

AA

Def:

(a) A is said to be Hermitian if

(b) A is said to be skew-Hermitian if

(c) A is said to be unitary if

( i.e. its column vectors form an orthonormal set in )

HAA

AAH

H HA A I AA nC

Let n nA F

Theorem6.4.1: Let , then

(i)

(ii) eigenvectors belonging to distinct eigenvalues

are orthogonal.

Pf ： (i) Let be an eigenpair of A,

(ii) Let and be two eigenpairs of A,

HAA RA

, x

2 2 2

( )

H H

H H H H H

Ax x x Ax x x

x Ax x A x x Ax

x x x

1 1, x 2 2, x

Theorem6.4.1 Pf ：

(ii) Let and be two eigenpairs of A,1 1, x 2 2, x

1 1 2 1 1 2 1 2 1 2

1 2 2 1 2

1 2

1 2 1 2 1 2

=( ) ( )

, 0

H HH H H

H H

H

x x x x Ax x x A x

x Ax x x

x x x x x x

Theorem: Let and

then

Pf ： (i) Let be an eigenpair of A,

HA A be an eignvalue of ,A is purely imaginary.

( , )x 2

2 2 2

2

( ) =

is pure-imaginary.

H H

H H H H H

HH

H

x Ax x x x

x Ax x A x x A x

x x x

x Ax

x

Theorem6.4.3 (Schur’s Theorem): Let , then unitary matrix , is upper triangular.

Pf ： The proof is by mathematical induction on n. (i) The result is obvious if n=1;

(ii) Assume the hypothesis holds for k×k matrices;

(iii) let A be a (k+1)×(k+1) matrix.

nnCA AUU HU

Proof of Schur’s Theorem

Let be an eigenpair of A with

Using the Gram-Schmidt process, construct an orthonormal

basis of

Let

1

0

0

H

k k

W AWM

1 1,w 1 1.w

1 2 1{ , , , }kw w w 1.k

1 2 1 ,kW w w w

1 1 1 1 1 is unitary and H HW W Aw W w e

Proof of Schur’s Theorem By the induction hypothesis (ii)

1 1 1 1 1 unitary , , where is upper triangular.k k HV V MV T T

1

1 0 0

0Define ,

0

VV

1

1 1

1

0

is unitary and

0

H H H

T

V V W AWV TV MV

Let is unitary and .HU WV U U AU T

Theorem6.4.4: (Spectral Theorem)

If , then unitary matrix U that diagonalizes A .

Pf ： By Theorem 6.4.3 ,

unitary matrix ,

where T is upper triangular .

T is a diagonal matrix.

AAH

TAUUU H

TAUU

UAUAUUTH

HHHHH

AAH

Cor.6.4.5: Let A be real symmetric matrix . Then (i) (ii) an orthogonal matrix U, is a diagonal matrix.proof ：

RA

TU AU

is real symmetric

is Hermitian, and its eignvalues must be real

and eignvectors may be chosen to be real.

By Th 6.4.4 ,

the diagonalizing matrix U must be orthogonal.

A

A

Example 4:

Find an orthogonal matrix U that diagonalizes A.

Sol : (i)

(ii)

021

232

120

ALet

1 2 1( 5 ) , ,

6 6 6

( ) 1 , 0 , 1 , 2 , 1 , 0

T

T T

N A I Span

N A I Span

2( ) det( ) ( 1) (5 )

1, 1, 5

p A I

Example 4:

Sol :

(iii) By Gram-Schmidt process,

and 5, 1, 1 .TU AU diag

1 1 1 1 1( ) , 0, , , ,

2 2 3 3 3

TT

N A I span

3

1 1 1

6 2 32 1

The columns of 0 form an orthonormal eigenbasis of 6 3

1 1 1

6 2 3

U

Question:In addition to Hermitian matrices ,

is there any other matrices possessing

orthonormal eigenbasis?

Note ： If A has orthonormal eigenbasis

where U is unitary &diagonal.HA UDU

H H H H

H H H H

H H H H

A A UD U UDU

UD DU UDD U

UDU UD U AA

Def: A is said to be normal if

Remark ： Hermitian, Skew- Hermitian and Unitary

matrices are all normal.

HH AAAA

Theorem6.4.6: A is normal A possesses an orthonormal eigenbasisPf ：

If A has an orthonormal eigenbasis, then ,where U is unitary & diagonal.

""

HA UDU

H H H H

H H H H

H H H H

A A UD U UDU

UD DU UDD U

UDU UD U AA

proof of Theorem6.4.6: By Th.6.4.3, unitary U,

Compare the diagonal elements of

0,

is diagonal.

ijT i j

T

"" is upper triangular.

T is also normal.

H

H H H H

T U AU

A A AA T T TT

22

11 11

2 22

2 21 2

2 2

1

n

jj

n

i ji j

n

in nni

t t

t t

t t

has an orthonormal eigenbasis (WHY?)A

&H HT T TT