eigenvalue eigenvector

1

EIGENVALUES AND EIGENVECTORS

Consider the following set of coupled, linear, first order, ordinary

differential equations-initial value problems (ODE-IVPs), with constant

coefficients:

112121111 . . . bxaxaxa nndt

dx ++++= , x1(0) = x10

222221212 . . . bxaxaxa nndt

dx ++++= , x2(0) = x20

•

•

nnnnnndtndx bxaxaxa ++++= . . .2211 , xn(0) = xn0

This can be written in compact notation, as

)(tdtd bAxx += x(0) = x0

• A is an n × n matrix of constants.

• The use of vectors and matrices not only saves a great deal of space and

facilitates calculations, but also emphasizes the similarity between systems of

equations and a single equation.

• Above equation is encountered quite commonly in engineering practice.

o Systems described by lumped-parameter models in chemical engineering,

e.g., continuous-flow stirred tank reactors (CSTRs), distillation, extraction

and absorption columns, etc., are quite well described by such equations

under dynamic (unsteady) conditions.

o The behavior of batch (and plug flow) reactors under steady conditions.

2

o These equations are useful in other fields, too, as for example, for

describing the sustained harmonic oscillations of strings, membranes and

structural systems (bridges, etc.), in the presence of external forces.

o At a more abstract, mathematical level, the study of the stability of

numerical solutions of ordinary and partial differential equations (PDEs)

also requires the solution of (homogeneous) linear ODEs.


In solving above, it is necessary to solve, first, the following set of

homogeneous linear ODEs:

Axx =dtd

In order to solve above, we assume the solution to be of the following form:

x = v eλt

This is similar to the procedure used to solve single, linear, homogeneous ODEs

with constant coefficients. The constant, λ, and the constant vector, v, are to be

determined.

On substituting we obtain

v λ eλt = A v eλt

Since eλt ≠ 0, this leads to A v = λ v

or,

(A - λ I) v = 0

3

Thus, in order to solve the system of ODE-IVPs, we must solve the above

system of linear algebraic equations.

• The above system of linear algebraic equations is encountered quite often

in engineering and science, and has been studied extensively.

• This equation is satisfied by several values of λi, which are referred to as the

eigenvalues of the matrix, A.

• For each λi, there is a corresponding vector, vi. The latter is referred to as

the eigenvector of the coefficient matrix, A, associated with λi.

• The above system represents a set of linear, homogeneous algebraic

equations (with v = 0 if the rank of [A - λI] is n).

• The necessary condition for this equation to have non-trivial solutions that

are physically meaningful, is

det (A – λi I) = 0, i = 1, 2, . . . , n.

This ensures that the rank of (A - λiI) is less than n.

• Corresponding to each eigenvalue, λi, there is an eigenvector, vi.

• Thus, there are, in general, n solutions, and the general solution of the

homogeneous is a linear combination of these

x = tiei

n

i

λv∑

=1 ci

• ci are arbitrary constants that are obtained using the initial conditions.

n-λi’s exist for which det (A - λi I) V = 0

4

If λi = ai + i bi, I = 1, 2, 3, . . . , n.

Then

[ ][ ]∑ +==

n

iii

tiaii tbtbeC

1cossinvx

The significance of the eigenvalues, λi, can easily be inferred from the

above solution.

• If we consider xi to represent the deviations of the state variables from

their steady state values, this equation suggests that these deviations will

die out asymptotically with time, t, if all the eigenvalues are real and

negative.

• Similarly, if even one of the λis is positive and real, the corresponding

deviation will increase with time, and the system will be unstable.

• If any λi is complex, the corresponding deviation will be oscillatory in

nature (due to the imaginary part), and its real part will decide whether it

will die out or blow up in time.

• Thus, λi are related to the (asymptotic) stability of a system under (small)

perturbations.

5

A particular solution of


can readily be obtained if b is a constant vector, i.e., it is not a function of t.

In this case the particular solution can be obtained by substituting x = k (where k

is a constant).

0 = A k + b

or

k = - A-1 b

Therefore, the complete solution is given by

x = tiei

n

i icλ

v∑=1

- A-1 b

We now apply the initial conditions, x(t = 0) = x0. We obtain

x0 = ∑=

n

i iic1

v - A-1 b

This comprises a set of n equations in the n unknowns, ci.

It can be shown that the ci can be uniquely determined (since vi are eigenvectors,

they form a linearly independent set, and so the rank of the set of equations for ci

is n.

6

The Characteristic Polynomial

Consider the n × n matrix, A. The eigenvalues of this matrix may be obtained by

solving:

det (A – λi I) = 0, i = 1, 2, . . . , n

while the eigenvector, vi, associated with λi obtained using (for v ≠ 0)

Avi = λi vi

The eigenvalue, λi, may be real or complex.

In expanded form we have

det (A-λI) = 0

21

2222111211

)( =

−••••••••••••

••−••−

≡

λ

λλ

λ

nnanana

naaanaaa

f

We can expand the determinant to give

00 1...2 21 1)( =+++−

−+−−+= αλαλαλαλλ n

nn

nnf

Clearly, above has n roots (i.e., n-eigenvalues).

Example 1: Obtain the eigenvalues of the following matrix

111320

746

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−

−=A

det (A - λI) = det λ

λλ

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−−

−−

111320

746

⇒ f(λ) = - λ3 + 9 λ2 - 30 λ + 32 = 0

⇒ λ = 2, [7 ± i 15]/2 where i = √- 1.

7

The nth order polynomial, f(λ), can also be written in terms of the principal

minors, Sj(A), as

f(λ) = Pn(λ) = det (A - λI) = jnλ)( )(

n

0j jS −−∑=

A

In this equation, So = 1, and Sj(A) is the sum of all possible principal minors of

A, of size j × j obtained by striking out an equal number, n – j, of rows as well as

columns from A, and taking the remaining j × j elements.

While doing this, it must be remembered that identical-numbered rows and

columns need to be struck off from A in order to obtain the several minors in Sj.

The procedure is best illustrated through an example.

a a a a aa a a a a

a a a a a

a a a a a

a a a a a

k i nk i n

k k kk ki kn

i i ik ii in

n n nk ni nn

11 12 1 1 121 22 2 2 2

1 2

1 2

1 2

• • •• • •

• • • • • • • •• • •

• • • • • • • •• • •

• • • • • • • •• • •

⎡

⎣

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥

Example 2: Find the characteristic polynomial for the 3 × 3 matrix, A, in

Example 1:

f(λ) = det (A - λI) = So (-λ)3 + S1 (-λ)2 + S2 (-λ) + S3

We have

S3 = det A = 32

Principal minors

Intersection of striking out of rows and columns should be on diagonal in forming the principal

minors.

8

This is obtained by deleting 0 rows and 0 columns in A. Only one principal

minor results.

S2 can be written as

S2 = a aa a

a aa a

a aa a

11 1221 22

11 1331 33

22 2332 33

12 13 5 30+ + = + + =

S2 is the sum of three principal minors, obtained by deleting the third, second

and first rows as well as columns, respectively, from A.

S1 can be written as

S1= a11 + a22 + a33 = 6 + 2 + 1 = 9

This is the sum of three principal minors again, obtained by deleting two rows

and two (identically numbered) columns from A.

Also, So = 1

Therefore,

Pn(λ) = - λ3 + 9 λ2 - 30 λ + 32 = 0

The same result as obtained in Example 1. -----------

It can easily be deduced that

S1 = aiii

n

=∑

1= sum of diagonal terms ≡ trace of A = tr A

Also,

Sn = det A

Therefore, for a 2 × 2 matrix

P2(λ) = λ2 – (tr A) λ + (det A) = 0

9

We can also write Pn(λ) = 0 in terms of a multiple product involving the n roots,

λi, as: )( )()(0)()( 2

1

1

11

−

> =

−

==

−⎥⎦

⎤⎢⎣

⎡+−⎥

⎦

⎤⎢⎣

⎡+−==−= ∑ ∑∑∏ n

n

ij

n

iji

nn

jj

nn

jjnP λλλλλλλλλ

+ . . . . . +=∏ λ jj

n

1

A comparison of the above leads to

S1 = tr A = λ ii

n

=∑

1= sum of all the eigenvalues

Sn = det A = λ ii

n

=∏

1= product of all the eigenvalues

By a similar comparison, we can obtain

S2 = λ λii

n

j i

nj

=>∑∑

1

For n = 3, thus, we can write

S2 = λ1λ2 + λ1λ3 + λ2λ3

10

Properties of Eigenvalues and Eigenvectors

• An upper bound of the eigenvalues is given by:

Max |λi| ≤ ||A||

where the norm of matrices is discussed in Section 5.6.1.

Proof:

We can get a bound on ⏐λi⏐as follows:

Avi = λivi (vi ≠ 0) Then,

⏐λi⏐ ⏐⏐vi⏐⏐ = ⏐⏐λivi⏐⏐ = ⏐⏐Avi⏐⏐ ≤ ⏐⏐ A⏐⏐ ⏐⏐vi⏐⏐ Therefore,

⏐λi⏐ ≤ ⏐⏐ A ⏐⏐ for all i for any norm.

• When the matrix, A, has distinct eigenvalues, λj, (i.e., no repeated

eigenvalues), then the set of eigenvectors, vj, forms a linearly

independent set.

Proof:

Let vi and vj be two eigenvectors of A that correspond to eigenvalues, λi and λj,

respectively. Then vi and vj are not constant multiples of each other, because if

they were, then

vi = k vj (a)

Here, k is a non-zero constant. If we pre-multiply Eq. (a) by the matrix, A, then

Avi = k Avj (b)

This is equivalent to

λi vi = k λj vj (c)

If we now multiply both sides of Eq. (a) by λi, we have

λi vi = k λi vj (d)

11

Subtracting Eq. (c) from Eq. (d), we have

k (λi - λj) vj = 0

Since k ≠ 0, and, λi ≠ λj, this means that vj = 0, which contrary to the assumption

that vj is an eigenvector.

Therefore, the eigenvectors corresponding to different (distinct)

eigenvalues of A are linearly independent.

• If the n × n matrix, A, is symmetric, then all n of its eigenvalues, λi (i = 1,

2, . . ., n), are real, i.e., the roots of the characteristic polynomial, f(λ) = 0, have

n real roots.

However, these roots need not necessarily be distinct.

If the matrix is not symmetric, then some of the eigenvalues may be

complex.

• For a symmetric matrix, A, there are n distinct eigenvectors, vi, one

corresponding to each eigenvalue, λi. This is so even if some of the

eigenvalues are not distinct.

However, if A is not symmetric, then it may not always be possible to find

n linearly independent eigenvectors (if some of the eigenvalues are repetitive).

• The n distinct eigenvectors of a symmetric matrix, A, are orthogonal to

each other and can easily be made orthonormal. Thus,

<vi, vj > = vi† vj = δij

These eigenvectors form a complete basis set.

12

Hence, any n-dimensional vector, x, can be represented in terms of the vi as

x = ∑=

n

1i iic v

where

ci = vi† x = <x, vi >

• Let U = [v1 v2 v3 . . . vn] be the n × n matrix whose columns contain the n

orthonormal eigenvectors of the symmetric matrix, A. It is easy to show

that

U†U = I

This means that

U† ≡ (U*)T = U-1

Such matrices are referred to as unitary matrices.

If U is real, then UTU = I, and then UT = U-1. Such matrices are referred to as

orthogonal matrices, rather than unitary matrices. The rows (or columns) of a

unitary matrix form an orthogonal basis.

Example: Consider the following two U matrices:

2

12

221

2 ; 1 ⎥⎥⎦

⎤

⎢⎢⎣

⎡⎥⎦⎤

⎢⎣⎡

=−

= i

i

xcosxsinxsinxcos

UU

U1 is orthogonal since it is real, and

T

xxxx

xx 12211

1 Ucossinsincos

sincosU =

−

+=−

⎥⎦⎤

⎢⎣⎡

13

Similarly, it may be demonstrated that U2 is unitary.

=⎥⎦⎤

⎢⎣⎡=

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

−

−=−

T

i

iUU *

22

12

221

12 U2

†

• Principal Axes Theorem

Let A be a self-adjoint (or Hermitian) matrix of order n, [defined by: A† ≡

(A*)T = A]. Then, A has

o n real eigenvalues, λ1, λ2, . . ., λn, not necessarily distinct

o n corresponding distinct eigenvectors, v1, v2 , . . ., vn, that form a

complete orthonormal basis set. The components of the eigenvectors

can be complex.

o If the elements of A are real, then the components of the eigenvectors will

also be real.

o Finally, there is an unitary (or orthogonal) matrix U, for which

U†AU = D ≡ diagonal (λi)

where

000020001

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

••••••

=

nλ

λλ

D

By pre-multiplying U†AU (= D) by U, and post-multiplying it by U† (and using

the fact that U† = U-1), we can show that

A = UDU†

We can only diagonalize a matrix, A, if the eigenvectors form a basis set,

i.e., we have n linearly independent eigenvectors.

This is always true for symmetric matrices.

14

However, if A is not symmetric, then it may not always be possible to find n

linearly independent eigenvectors (if some of the eigenvalues are repetitive), and

in that case we will not be able to diagonalize the matrix.

Example: Consider the following symmetric matrix

210131012

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=A

We can find the eigenvalues of A as follows:

f(λ) = -λ3 +7λ2 - 14λ + 8 = 0

⇒ [λ1, λ2, λ3] = [1, 2, 4]; all real, as expected.

The corresponding eigenvectors can be obtained by solving for non-zero vi using

Avi = λivi or (A - λiI)vi = 0.

As the (A - λiI) matrix is singular, we have to assign one or more components of

vi arbitrarily, and then find the remaining components.

When λ = 1, we have for (A-λI) v1 = 0 as

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−=⇒

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

11

1

000

110121011

1

13

12

11

vvv

αv

Similarly, we obtain

121

γ3 ; 1

01

β2 ; 11

1 α1

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡

⎥⎥

⎦

⎤

⎢⎢

⎣

⎡=

−=−= vvv

As expected, these eigenvectors are orthogonal to each other, and <v1, v2> = <v1, v3> = <v2, v3> = 0

These vectors are also linearly independent.

15

We can, of course, make these vectors orthonormal by normalizing them to obtain

121

6

13 ;

101

2

12 ;

11

1

31

1 ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−= vvv

If we now form the U matrix as follows (each column of U contains one of the

orthonormal eigenvectors):

U

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−

−=

61

21

31

62

31

61

21

31

0

then

UTAU = D = 1 0 00 2 00 0 4

⎡

⎣

⎢⎢⎢

⎤

⎦

⎥⎥⎥

Also not that U-1 = UT.

• If all the λis are distinct (even for non-symmetric A), then A has n linearly

independent eigenvectors. This can be easily demonstrated and is shown below:

We can write, in general

01

21 =−=−−−=−= ∏=

)())....()((I)(A)( λλλλλλλλλλn

jjnn detP

When the ith eigenvalue, λi, is simple (i.e., λi has a multiplicity of 1), we can

rewrite Pn(λ) as

01

=−−= ∏

≠=

)()()( λλλλλn

jij

jin P

Differentiating it with respect to λ, and then substituting λ = λi, we get

Eigenvalues

Rank of (A-λI) < n

16

01

≠−−= ∏

≠==

)()( λλλλλ

λ n

ijj

jddP

i

n

Therefore, the rank of (A - λiI) is n - 1 (when λi is a distinct eigenvalue). Then,

Dimension of null space = n – [rank of (A - λiI)] = n – (n-1) = 1

Therefore, there is one linearly independent eigenvector for each λi. as

long as it is a distinct eigenvalue. Note that the symmetry of A was not invoked.

Hence, if all eigenvalues are distinct, then A has n linearly independent

eigenvectors, which form a complete basis set. Therefore, any n-

dimensional arbitrary vector can be represented as linear combinations of vi’s.

Example: Obtain the eigenvalues and the corresponding eigenvectors of the

following symmetric matrix

⎥⎦

⎤⎢⎣

⎡=

1111

A

We have

det (A - λI) = λ2 – (tr A) λ + det(A) = λ2 – 2λ = 0

⇒ λ = 0, 2 (i.e., the symmetric matrix, A, has two distinct eigenvalues).

To obtain the eigenvectors corresponding to λ = 0, we solve for non-trivial v

using (A - λI) v = 0:

⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡=−

00

1111

2

11

vv

v)IA( λ

or

v1 + v2 = 0

v1 + v2 = 0

Rank of (A-λiI) = n -1

17

These two equations are linearly dependent. We have to fix one of the

components of v arbitrarily to find the other one. If we fix v2 = α, then

⎥⎦

⎤⎢⎣

⎡−=

11

αv

When λ = 2:

vv

vv

⎥⎦

⎤⎢⎣

⎡==⎥

⎦

⎤⎢⎣

⎡⇒⎥

⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡−

−11

00

1111

2

1

2

1 βv

We obtained two linearly independent eigenvectors for the two distinct

eigenvalues.

Example: Now consider the following unsymmetric matrix: ⎥⎦

⎤⎢⎣

⎡−−

=11

43A

We have

det (A - λI) = λ2 – (tr A) λ + det(A) = λ2 – 2λ + 1 = 0 ⇒ λ = 1, 1

We now have two eigenvalues that are identical.

We solve for non-trivial v using (A - λI) v = 0 for λ = 1:

⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡−−

=−00

2142

2

1 vv

v)IA( λ

or

2 v1 + 4 v2 = 0

- v1 – 2 v2 = 0

The two equations are linearly dependent (rank = 1, dimension of null space =

1). We have to fix one component of v arbitrarily and evaluate the other one. If

we fix v2 = α, we obtain

⎥⎦

⎤⎢⎣

⎡−=

12

αv

18

Note that we can only find only one linearly independent eigenvector for the

eigenvalue λ = 1 with a multiplicity of two since dimension of null space is 1

(and therefore, only one independent vector exists).

We do not have two eigenvectors although we have two (identical) eigenvalues

for the non-symmetric A.

Example: Evaluate the eigenvalues and the eigenvectors of the symmetrical

identity matrix

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

100010001

A

We have

f(λ) = (1 - λ)3 = 0 ⇒ λ = 1 with a multiplicity, p of 3

Since the rank and dimension of null space for the (A-λI) matrix are 0 and 3

respectively, we can find three linearly independent eigenvectors for the above

symmetric matrix:

v1 = [1 0 0]T, v2 = [0 1 0]T, v3 = [0 0 1]T

From the principa1axes theorem, we know that for a symmetric matrix,

• All the eigenvalues are real, but all of them are not necessarily

distinct.

• However, it is always possible to find distinct eigenvectors regardless

of whether eigenvalues are distinct or repetitive.

• The corresponding eigenvectors also form a complete orthonormal

basis.

Rank of (A-λI) = 0 ⇒ Dimension of null

space = 3

19

• This means that we will always be able to find n linearly independent

eigenvectors even if all the n eigenvalues are not distinct when matrix is

symmetric.

Consider now the following non-symmetric matrix

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

100110011

A

f(λ) = (1 - λ)3 = 0 ⇒ λ = 1; p (multiplicity) = 3

Since the rank and dimension of null space for the (A-λI) matrix are 2 and 1

respectively, the matrix has only one linearly independent eigenvector for the

eigenvalue, λ = 1:

v = [1 0 0]T

• If a matrix, A, is singular, then one or more (depending on the rank of the

matrix) of the eigenvalues is zero.

We can also find the eigenvectors of A from the non-trivial columns of

adj(A - λI).

Note that when a matrix is singular, A-1 does not exist but adjugate of A exist.

A-1 = [ ]AadjA det1

Rank of (A-λI) = 2 ⇒ Dimension of

null space = 1

For non-trivial x from (A-λI) x = 0

can be obtained from non-trivial columns of adj(A-λI).

20

Example: Consider the unsymmetrical matrix

⎥⎦

⎤⎢⎣

⎡−−

=11

43A

We found λ = 1 (multiplicity = 2) for this matrix. We have, then,

2142

2142

⎥⎦

⎤⎢⎣

⎡ −−=⎥

⎦

⎤⎢⎣

⎡−−

=λ− adjadj )IA(

Example: Consider the symmetrical matrix:

⎥⎦

⎤⎢⎣

⎡=

1111

A

We had obtained λ = 0 and 2.

For λ = 0: adjadj ⎥⎦

⎤⎢⎣

⎡−

−=⎥

⎦

⎤⎢⎣

⎡=−

1111

1111

I)(A λ

For λ = 2: adjadj ⎥⎦

⎤⎢⎣

⎡−−−−

=⎥⎦

⎤⎢⎣

⎡−

−=−

1111

1111

)IA( λ

However, if repetitive eigenvalues are present and dimension of (A - λI) matrix

is greater than 1, then sometimes it may not be possible to obtain eigenvectors

from adj(A - λI) as all components of adj(A - λI) matrix may be zero.

Eigenvectors

Eigenvectors

Eigenvectors

21

Similar Matrices

Two n × n matrices, A and B, are said to be similar if there exists a non-singular

matrix, P, such that B = P-1AP, or, equivalently, A = PBP-1.

Matrices A and B are similar in the sense that

• their characteristic polynomials are identical (and so are their eigenvalues),

• their eigenvectors are related through the matrix, P.

• They are said to be represented by the same linear transformation but with

respect to different bases.

We first demonstrate that the two characteristic polynomials, fA(λ) and fB(λ), are

the same.

fB(λ) = det (B - λI) = det [P-1 A P - λP-1 I P] = det [P-1 (A - λ I) P]

= det(P-1) det (A - λI) det(P) = det (A - λI) = fA(λ)

since det(P-1) × det(P) = 1

Therefore, A and B have identical eigenvalues.

Since fA(λ) = fB(λ), the coefficients of the characteristic polynomials are the

same. This means that

trace of A = trace of B and det A = det B

When matrices A and B are similar, even though their eigenvalues are identical,

their eigenvectors are not. However, there is a one to one correspondence, as

shown below. We have

Ax = λ x

or, P-1Ax = P-1λ x

or, P-1A (P P-1) x = λ P-1 x

22

or, B (P-1x) = λ (P-1 x)

or, B z = λ z

Therefore, the eigenvectors of B are given by z = P-1x, where x is the

eigenvector of A.

In summary

• For a (n × n) matrix A, there exist n eigenvalues.

• The eigenvalues can be obtained from the characteristic polynomial

f(λ) = det (A - λI) = 0.

• The eigenvectors can be obtained from either

• The non-trivial solution v from (A - λI) v = 0, or

• Non-trivial columns of adj (A - λI) = 0.

• The eigenvalues and eigenvectors can also be obtained numerically, for

example by Inverse Power method.

• The n eigenvalues may be distinct or repetitive (multiple), and real or

complex.

• If all the eigenvalues are distinct (real or complex), then there exist n

eigenvectors, which are linearly independent to each other.

• The n eigenvectors, vi, (i = 1, 2, …n) are obtained from

either (A - λiI) vi = 0 or from adj (A - λiI) = 0.

• If we form a (n × n) U matrix such that each column of U matrix contains the

n eigenvectors, i.e., U = [v1 v2 v3 . . . vn], then U-1 A U = D, where

D is a diagonal matrix whose diagonal elements are the n eigenvalues.

23

• If all the eigenvalues are not distinct (real or complex), then we may or

may not be able to find n linearly independent eigenvectors. The

number of linearly independent eigenvectors that can be found for the

eigenvalue λi with multiplicity m will depend on the dimension of null

space for the (A - λiI) matrix.

• If the dimension of null space for the (A - λiI) matrix is 1, for the eigenvalue

λi with multiplicity m, then only one linearly independent eigenvector can be

found for corresponding to the eigenvalue λi with multiplicity m.

• If, however, the matrix A is self-adjoint (or symmetric), then

• All the eigenvalues of matrix A are always real.

• All the eigenvalues may not be distinct, but we will always be able

to find n linearly independent eigenvectors irrespective of whether

the eigenvalues are distinct or repetitive.

• The eigenvectors will not only be linearly independent, but they will

also be orthogonal.

• If we normalize the eigenvectors, and then form the U matrix, then we

will find U-1 = UT. That is the U matrix will be a unitary (or

orthogonal) matrix.

• Therefore, U-1 A U = D will be same as UT A U = D.

24

However, if the matrix is not symmetric, then it may not always be

possible to find n- linearly independent eigenvectors (particularly, if

some of the eigenvalues are repetitive), and in that case we will not be able to

diagonalize the matrix.

• Algebraic multiplicity: Multiplicity in eigenvalues as obtained from the

root of Pn(λ) or f(λ) = 0. Example: In the previous two matrices, algebraic multiplicity was 3 (for λ =

1, p was equal to 3 for both the matrices).

• Geometric multiplicity: Maximum number of linearly independent

eigenvectors possible associated with all the eigenvalues. Example: Although, algebraic multiplicity was 3 for both cases, we had 3

linearly independent eigenvectors for the first case (symmetric identity

matrix) whereas, we had only one linearly independent eigenvectors for the

second case. Therefore, geometric multiplicity is 3 for the first case (identity matrix),

whereas geometric multiplicity is only 1 for the second case. • When a matrix A (n x n) has a eigenvalue λ of algebraic multiplicity p but

geometric multiplicity less than p, then, we can form a Jordon-Canonical

Block, J, consisting of all the eigenvalues λ of A, and can find a non-singular

matrix P such that P-1AP = J.

25

• The above is similar to U-1AU = D except that • Matrix J is not a diagonal matrix but has a different form. • Each columns of U matrix consist of n- linearly independent eigenvectors,

whereas columns of P matrix consist of as many as linearly independent

eigenvectors possible to be obtained from A, plus some generalized

eigenvectors.

Note that when matrix A is symmetric, only then U-1 = UT (of course, only

if the eigenvectors were normalized), otherwise U-1 ≠ UT.

Therefore, we must use the relation U-1AU = D when matrix A is not

symmetric but still we can find n-linearly independent eigenvectors. • How to form the Jordon-Canonical Block?

When we have n-eigenvalues all equal, say λ, (i.e., algebraic multiplicity

equal to n), but it is possible to obtain only one linearly independent

eigenvector (i.e., geometric multiplicity equal to 1), then

i.e., λ at the diagonal position and 1 above λ. The rest of the elements are

zero.

00001000

00100001

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

•••••=

λλ

λλ

nJ

26

• If A is a (n x n) matrix, then, there always exist a non-singular matrix P for

which

⇒ The eigenvalues λ1, λ2 , …, λr need not be distinct.

⇒ The number of distinct blocks Jnr(λr) must be equal to the number of

independent eigenvectors.

⇒ Same eigenvalues may occur in different blocks.

Example: Let A be a (6 x 6) matrix having the following eigenvalues and the

corresponding number of linearly independent eigenvectors: λ1 = 3, p = 2 with only 1 linearly independent eigenvector.

λ1 = 4, p = 3 with only 1 linearly independent eigenvector.

λ1 = -2, p = 1 with 1 linearly independent eigenvector.

400000140000014000000300000130000002

⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢

⎣

⎡−

=J

Number of distinct blocks must be

equal to the number of

independent eigenvectors

)(000

00)(0000)(

22

11

1

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

••••=−

rnr

n

n

J

JJ

APP

λ

λλ

We can put λ1, λ2, etc. at any location on J matrix. But, when we form the P matrix, the columns of P matrix must correspond to the respective location of λ in J matrix.

27

Example: Let A be a (5 x 5) matrix having the following eigenvalues and the

corresponding number of linearly independent eigenvectors:

λ1 = 2, p = 4 with only 2 linearly independent eigenvectors.

λ1 = 3, p = 1 with 1 linearly independent eigenvector.

3000002000012000002000012

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

=J

Alternatively,

3000002000012000012000002

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

=J

• How do we form the non-singular matrix P?

Example: Consider the following matrix

P2(λ) = λ2 – (tr A) λ + det(A) = λ2 – 4λ + 4 = 0 ⇒ λ = 2, 2

Or,

A P P J

Numbers of distinct blocks are equal to the number of

independent eigenvectors. Note that

Same eigenvalue is present in two different blocks.

3111

⎥⎦

⎤⎢⎣

⎡−

=A

2012

011

⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡==−

λλ

JAPP

2012

3111

2221

1211

2221

1211⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡− pp

pppppp

28

⇒ p11 + p21 = 2 p11 ⇒ p11 = p21

⇒ - p11 + 3 p21 = 2 p21 ⇒ p11 = p21

⇒ p12 + p22 = p11 +2 p12 ⇒ p22 = p11 + p12

⇒ - p12 +3 p22 = p21 +2 p22 ⇒ p12 = p22 - p21

We have only 2 linearly independent equations for the 4 unknowns.

Let p11 = 1, and p12 = 0

Then, p21 = 1 and p22 = 1.

Therefore,

and

• Example: Consider the following matrix

P3(λ) = - λ3 + 6 λ2 + 0 λ - 32 = 0 ⇒ λ = -2, 4, 4

• For λ = -2:

181801818018180

321321

347

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡ −−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−=λ− adjIAadj )(

1101

1101 1

⎥⎦

⎤⎢⎣

⎡−

=⎥⎦

⎤⎢⎣

⎡= −PandP

2012

1101

3111

11011

⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡−⎥

⎦

⎤⎢⎣

⎡−

=− APP

121301

345

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−=A

We can take any arbitrary

multiple α

Dimension of null space = 2

29

Therefore, eigenvector, v1,

11

1 2 1

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−α=−=λ )( forv

• For λ = 4:

Therefore, eigenvector, v2,

We have only 2 linearly independent eigenvectors. Therefore, we need to

form Jordon-Canonical block and we need to find one generalized

eigenvector, p.

A [ v1 v2 p] = [v1 v2 p] J3(λ)

⇒ A v1 = -2 v1 or, (A + 2 I) v1 = 0

⇒ A v2 = 4 v2 or, (A - 4 I) v2 = 0

066066066

321341

341 )(

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−−−=− adjIAadj λ

11

1 )4 (2

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−== αλforv

400140002

1111

11

1111

11

121301

345 ,

3

2

1

3

2

1

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−

ppp

ppp

or

[ ] [ ] 400140002

, 2121⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−= pvvpvvAor

λ = - 2

λ = 4

30

⇒ A p = v2 + 4 p

Or, (A – 4I) p = v2

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−−−

11

1

321341

341

3

2

1

ppp

⇒ p1 = 1, p2 = 0, p3 = 0

Therefore, the matrix P is

and, P-1 A P = J3(λ)

• Whenever we have repeated eigenvalues with geometric multiplicity less than

algebraic multiplicity, then column vectors of the P matrix contains

generalized eigenvectors in addition to the normal eigenvectors.

Generalized eigenvectors can be obtained from

(A – λjI) pi = pi -1 ; for i = 2, 3, … with p1 = Vj

where, Vj is the linearly independent eigenvector obtained from λj.

Generalized eigenvector

(A - 4I) v2

022110110

011011111

211

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−

−−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−= −PandP

31

• Example:

λ1 = -2, m = 1 with 1 linearly independent eigenvector.

λ2 = 1, m = 1 with 1 linearly independent eigenvector.

λ3 = 3, m = 3 with only 1 linearly independent eigenvector.

where, m denotes algebraic multiplicity.

Then, we can form matrix P as follow

P = [v1 v2 v3 p1 p2]

λ1 λ2 λ3

p1 and p2 can be obtained as follows:

(A – λ3I) p1 = v3

and (A – λ3I) p2 = p1

and the Jordon-Canonical block is given by

and we have P-1 A P = J5(λ).

Example:

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−=

3111

A

f (λ) = λ2 - 4 λ + 4 = 0 ⇒ λ = 2, 2

[A - λ I] = [A - 2 I] = ⎥⎥⎦

⎤

⎢⎢⎣

⎡

−−

1111

Number of distinct blocks must be

equal to the number of independent eigenvectors.

3000013000013000001000002

)( 5

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡−

=λJ

32

Adj [A - 2 I] = ⎥⎥⎦

⎤

⎢⎢⎣

⎡

−−

1111

Therefore, v1 = α ⎥⎥⎦

⎤

⎢⎢⎣

⎡

11

Therefore, we have just 1 linearly independent eigenvector. The generalized

eigenvector p can be obtained from

(A - λ I) p = v1

or, ⎥⎥⎦

⎤

⎢⎢⎣

⎡=

⎥⎥⎦

⎤

⎢⎢⎣

⎡

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−−

11

1111

2

1pp

⇒ - p1 + p2 = 1

⇒ - p1 + p2 = 1

If we let p1 = 0, then p2 = 1.

or, ⎥⎥⎦

⎤

⎢⎢⎣

⎡=

10

αp

Therefore, matrix P is given by

P = ⎥⎥⎦

⎤

⎢⎢⎣

⎡

1101

And, P-1 A P = =⎥⎥⎦

⎤

⎢⎢⎣

⎡=

⎥⎥⎦

⎤

⎢⎢⎣

⎡

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−⎥⎥⎦

⎤

⎢⎢⎣

⎡

− 2012

1101

3111

1101

J

v1 was obtained from (A - λI) v1 = 0

v1 obtained from (A - λI) v1 = 0

p obtained from (A - λI) p = v1

33

Example: Let A be a (5 x 5) matrix having the following eigenvalues and

the corresponding number of linearly independent eigenvectors:

λ1 = -3, p = 4 with only 2 linearly independent eigenvectors.

λ1 = 2, p = 1 with 1 linearly independent eigenvector.

There many ways we can from the Jordon-canonical block, J, for this

example. However, we must form the corresponding P matrix as per we

write our J matrix

Arrangement #1: If we form the J matrix as follows

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

−−

−−

=

3000013000013000003000002

J

Form P matrix as follows

P = [v1 v2 v3 p1 p2]

Where,

v1 is obtained from (A - 2 I) v1 = 0.

v2 is obtained from (A + 3 I) v2 = 0.


p1 is obtained from (A + 3 I) p1 = v3.

p2 is obtained from (A + 3 I) p2 = p1.

3 distinct blocks for 3 linearly independent

eigenvectors

From problem statement 2 linearly independent

vectors can be obtained from

(A+ 3 I) v2 = 0 and (A+ 3 I) v3 = 0

Null space dimension must be equal to 2.

34


⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

−−

−−

=

2000003000013000003000013

J


P = [v1 p1 v2 p2 v3]

Where,







⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

−−

−−

=

3000013000002000003000013

J


eigenvectors



(A+ 3 I) v1 = 0 and (A+ 3 I) v2 = 0



eigenvectors

35


P = [v1 p1 v2 v3 p2]

Where,






In each case if we form the J and P matrix as above we would get

P-1 A P = J.

Summary

• One can find generalized eigenvector, p, which, of course, is linearly

independent to all other eigenvectors vi, from

(A - λiI) p = vi.

• One can then form a (n × n) P matrix such that each column of P matrix

contains the linearly independent eigenvectors that are possible to be

obtained plus the generalized eigenvectors.

Then we can have

P-1 A P = J

where J is called Jordon-Canonical block which is a nearly diagonal

matrix whose diagonal elements are the n eigenvalues.



(A+ 3 I) v1 = 0 and (A+ 3 I) v3 = 0


36

♦ Linear Independence of eigenvectors A set of vectors {xj} is said to be linearly independent if the equation

0.....2211 =+++ nn xcxcxc

where {cj} are constants, is satisfied only when c1 = c2 = . . . . = cn = 0.

When A has distinct eigenvalues, λj, (i.e., no repeated eigenvalues), then the

set of eigenvectors {vj} forms a linearly independent set.

Let vi and vj be eigenvectors that belongs to λ1 and λj, respectively. Then vi and

vj are not constant multiples of each other, because if they were, then

vi = k vj (a)

where k is a nonzero constant. If we pre-multiply (a) by A, then

Avi = k Avj (b)

which is equivalent to

λi vi = k λj vj (c)

If we now multiply both sides of (a) by λi, then we have

λi vi = k λi vj (d)

Subtracting (c) from (d), we have

k (λi - λj) vj = 0

Since k ≠ 0, and, λi ≠ λj, this means that vj = 0, which contrary to the assumption

that vj is an eigenvector.

Therefore, for different eigenvalues, the corresponding eigenvectors are

linearly independent.

37

♦ Bi-orthogonality property of eigenvectors

Consider two eigenvalue problems

A v = λ v (1)

and AT w = η w (2)

Since, the condition that equation (2) has nontrivial solution is

det (AT - ηI) = 0

or,

0

21

22212

12111

=

−••••••••••••

••−••−

η

ηη

nnnn

n

n

aaa

aaaaaa

Since, the value of determinant remains unchanged when its rows and columns

are interchanged, we have, λ = η, and (2) becomes

AT w = λ w (2a)

Therefore, (1) and (2a) have the same eigenvalues, λi = 1, 2, ...., n, but have

different set of eigenvectors. vj of (1) is one of the non-zero columns of

adj [A - λj I], while wj of (2a) is one of the non-zero columns of adj [AT - λj I]. It

will be same only when A is symmetric, i.e., A = AT.

However, columns of adj [AT - λj I] are rows of adj [A - λjI]. Therefore, wj is

simply non-zero rows of adj [A - λjI]. Hence, both eigenvectors, vj and wj,

can be obtained from the same calculation, adj [A - λj I].

As shown earlier, wj, can also be shown to be linearly independent like vj.

38

However, apart from being linearly independent, the vectors, vj and wj, satisfy

the bi-orthogonality property. This means that each member of one set is

orthogonal to each member of the other set, except for the one with which it has

a common eigenvalue.

By definition, A vj = λj vj (3)

and wiT A = λi wi

T, i ≠ j (4)

Pre-multiplying (3) by wiT, and post-multiplying (4) by vj, we have

wiT A vj = λj wi

T vj

wiT A vj = λi wi

T vj

Subtracting the above two equations, we have

(λi – λj) wiT vj = 0

Since, we assumed that λi ≠ λj, we have

wiT vj = <wi, vj> = 0 (5)

That is, wi and vj and are orthogonal to each other for i ≠ j.

When matrix A is symmetric, A = AT, and therefore,

adj [A – λiI] = adj [AT – λiI]

and vi = wi

Hence, equation (4) becomes

viT vj = <vi, vj> = 0 for i ≠ j.

Thus, we have an orthogonal set of vectors {vi}, that is, each member of the set

is orthogonal to every other member of the set.

39

Example:

Let A = ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−

−

320321001

f (λ) = - λ3 - 6 λ2 - 5 λ = 0 ⇒ λ = 0, -1, -5

For λ = 0:

adj [A - λI] = ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−

−

222333000

320321001

adj

For λ = -1:

adj [A - λI] = ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−

002002004

220311000

adj

For λ = -5:

adj [A - λI] = ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

12821282000

220331004

adj

The eigenvectors of A is given by the non-zero columns of adj [A - λiI] and

are denoted by vi, whereas the eigenvectors of AT is given by the non-zero

rows of adj [A - λiI] and are denoted by wi.

v1

w1

v2

w3

40

Therefore,

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=230

1v ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=111

1w

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−=

112

2v ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

001

2w

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−=11

0

3v ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−=64

1

3w

It can be easily observed that

<v1, w2> = <v1, w3> = <v2, w1> = <v2, w3> = <v3, w1> = <v3, w2> = 0

i.e., <vi, wj> = 0 for i ≠ j

or the vectors vi and wj are bi-orthogonal.

41

♦ Expansion of an arbitrary vector

We have seen that each square matrix A generates two sets of linearly

independent vectors via eigenvectors associated with the eigenvalue problems

A v = λ v and AT w = λ w

The two sets of eigenvectors, {vi} and {wi}, corresponding to eigenvalues, {λi}, i

= 1, 2, . . . . . . . . , n, are nonzero columns and rows, respectively, of adj (A – λi

I). In addition, the eigenvectors, {vi} and {wi}, satisfy the bi-orthogonality

criteria

<wj, vi > = 0, i ≠ j

<wj, vi > ≠ 0, i = j

We can now write any arbitrary vector z as

z = c1 v1 + c2 v2 + c3 v3 + . . . . . + cn vn = in

iivc∑

=1

where ci's are constant to be determined. If we take an inner product with wj, we

have

< wj, z> = < wj, in

iivc∑

=1> = cj <wj, vj>

or, ><

><=

jvjwzjw

jc ,,

and the arbitrary vector can be written as

z = ∑=∑= ><

><

=

n

jjjvjw

zjwj

n

jj vvc

1 ,,

1

If A is symmetric, then wj = vj, and

z = ∑=

><

><n

jjjvjv

zjvv

1,,

<wj, vi> = 0 except when i = j

42

♦ Application eigenvalues-eigenvectors in solving a system of

linear algebraic equations

Consider the following set of n-linear algebraic equations

A x = b (1)

Where A is a non-singular matrix (det A ≠ 0) with simple eigenvalues, b is

known vector of constants, and x is a vector of n-unknowns.

We can express any arbitrary vector x and b in terms of eigenvectors of A as

x = in

iiv∑

=1α (2)

and b = in

iiv∑

=1β (3)

where the eigenvalues {λi} and eigenvectors {vi} are obtained from

A v = λ v (4)

βi's can be determined from

><><=

iviwbiw

i ,,β , i = 1, 2, . . . . , n (5)

where {vi} and {wi} are eigenvectors, nonzero columns and rows respectively of

adj (A – λi I), corresponding to eigenvalues, λi.

Substituting (2) and (3) in (1), and using (4) we have

A in

iiii

n

iii

n

ii vvv ∑∑∑

=====

111βλαα

Rearranging, we have

[ ] 0 11

==− ∑∑==

n

iiii

n

iiii vcvβλα

43

However, the set of eigenvectors {vi} is linearly independent, therefore, the set

of constants {ci}, are equal to zero for all i. That is,

αi λi = βi, i = 1, 2, . . . . , n.

or, the unknown constants, αi, is given by

ii

i λβα = , i = 1, 2, . . . . , n. (6)

and the solution of A x = b is given by

x = iiin

iii vv

n

1i1∑ ⎥⎦

⎤⎢⎣⎡

∑ === λ

βα (7)

where βi's are obtained from (5), and λi and vi are eigenvalues and

corresponding eigenvectors of A.

Note that no solution exist for x from (7) when any λi = 0. This is expected

because if any of the eigenvalues of A is zero, then det A must be equal to zero,

i.e., A is singular, and we know that no solutions exist.

Moreover, if λ = 0 is an eigenvalue, then we have

A v = λ v = 0 [since λ = 0] (8)

Where v is the eigenvector corresponding to the zero eigenvalue. Since, non-

trivial solution of (8) can occur only if det A = 0, we must have v = 0, which

contrary to the definition of eigenvector. Therefore, if det A ≠ 0, λ = 0 can not

be an eigenvalue of A.

Example: Solve for x for the following Ax = b problem

Where, A = ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−301030103

and b = ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−22

0

The eigenvalues are 2, -3, 4 and the eigenvectors are

44

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−=

101

010

,1

01

321 vandvv

Note that the matrix is symmetric, therefore,

• vi = wi

• < vi, wj> = 0 for i ≠ j.

Now, b = ∑=

n

iii v

1β

βi can be calculated either from

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

+⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

+⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

− 22

0

101

010

1

01

321 βββ

or from

βi = ><><

ivivbiv

,,

Therefore,

β1 = 122

1,1,1 −== −

><><

vvbv

β2 = 212

2,2,2 −== −

><><

vvbv

β3 = 122

3,3,3 ==

><><

vvbv

The solution x is given by

333

222

111 vvvx ⎥⎦

⎤⎢⎣⎡+⎥⎦

⎤⎢⎣⎡+⎥⎦

⎤⎢⎣⎡= λ

βλβ

λβ

or, ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

+⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

+⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−

43

32

41

41

32

21

3

2

1

101

010

101

xxx

λ1 = 2 λ2 = -3 λ3 = 4

3 Equations 3 Unknowns

Equation (5)

Equation (7)

45

Example: Solve for x for the following Ax = b problem

Where, A = ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−−−

222214241

and b = ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

633

The eigenvalues are -3, -3, 6 and the eigenvectors are

and ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡ +−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−=

βα

βα 50

122

321

.vv,v

By arbitrarily selecting α = 1, β = 0, one can obtain ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−=

011

2v .

By arbitrarily assigning any other values for α and β, one can obtain another

eigenvector. However, we want the eigenvectors to be orthogonal. Therefore,

select α and β such that

[ ] βαβα

βα2500

5001132 .

.v,v =⇒=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡ +−•−=><

Therefore, if we arbitrarily select α = 1, we obtain ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

411

3v .

Now, b = ∑=

n

iii

1vβ

βi can be calculated from

βi = ><

><

iii

v,vb,v

Therefore,

λ1 = 6 λ2, λ3 = -3, -3

46

β1 = 211

1 =><

><v,vb,v

β2 = 022

2 =><

><v,vb,v

β3 = 133

3 −=><

><v,vb,v

The solution x is given by

333

222

111 vvv ⎥⎦

⎤⎢⎣⎡+⎥⎦

⎤⎢⎣⎡+⎥⎦

⎤⎢⎣⎡= λ

βλβ

λβx

or, ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡+

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−+

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−

−111

411

011

122

31

30

62

3

2

1

xxx

------------

Application of Eigenvalues and Eigenvectors in Solving Linear Ordinary

Differential Equations of Initial Value Problems (ODE-IVPs)

We now return to complete the solution of the first-order, linear, ODE-IVPs

with constant coefficients:

bAxx +=dtd ; x(0) = x0

The complete solution was obtained as

x = tiei

n

i icλ

v∑=1

- A-1 b

with the ci obtained from

x0 = ∑=

n

i iic1

v - A-1 b

We take the inner product with wj, to get

< wj, ∑=

n

i iic1

v > = < wj, x0> + < wj, A-1 b>

The only nonzero term in the LHS of this equation is when i = j, and, therefore,

><>−<+><

=j

jojjc vw

bAwxw,j

1, ,, j = 1, 2, . . . , n

47

The final, complete solution is then given by

bAvvwbAwxw

x 1 n

1j ,j

1, , −−∑

= ⎥⎥

⎦

⎤

⎢⎢

⎣

⎡><

>−<+><=

tjejjjoj λ

If A is symmetric, then wj = vj, and the above equation reduces to

bAvvvbAvxv

x 1 n

1j ,j

1, , −−∑

= ⎥⎥

⎦

⎤

⎢⎢

⎣

⎡><

>−<+><=

tjejjjoj λ

Example:

10121 == )(y ydt

dy

10510011000 2212 −=+−−= )(y yydt

dy

or,

⎥⎦

⎤⎢⎣

⎡−

=⎥⎦

⎤⎢⎣

⎡+⎥

⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡−−

=⎥⎦

⎤⎢⎣

⎡1

10

50

1001100010

2

1

2

1 )(y yy

yy

dtd

or,

00 y)y(bAyy =+= dtd

where

⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡−−

=50

1001100010

bA and

f(λ) = λ2 - (tr A) λ + det A = λ2 + 1001λ + 1000 = 0 ⇒ λ = -1, - 1000

For λ = -1, adj [A + I] = adj ⎥⎦

⎤⎢⎣

⎡ −−=⎥

⎦

⎤⎢⎣

⎡−− 11000

110001000100011

⇒ v1 = ⎥⎦

⎤⎢⎣

⎡−11

α , w1 = ⎥⎦

⎤⎢⎣

⎡1

1000 α

48

For λ = - 1000, adj [A + 1000I] = adj ⎥⎦

⎤⎢⎣

⎡ −−=⎥

⎦

⎤⎢⎣

⎡−− 10001000

1111000

11000

⇒ v2 = ⎥⎦

⎤⎢⎣

⎡ −1000

1 β , w2 = ⎥

⎦

⎤⎢⎣

⎡11

α

The general solution of

00 y)y(bAyy =+= dtd

is given by

bAvvbAvy 1222

111

1

1

−−

=−+=−= ∑ ttn

i

tiii ececec λλλ

or, ⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡ −−−⎥

⎦

⎤⎢⎣

⎡ −+⎥

⎦

⎤⎢⎣

⎡−= −−

50

0100011001

10001

11

100011000

21 e ce cy tt

or, ⎥⎦

⎤⎢⎣

⎡+

⎥⎥⎦

⎤

⎢⎢⎣

⎡ −+⎥⎥⎦

⎤

⎢⎢⎣

⎡−= −−

−

00050

1000 1000

1000

21.

t

t

t

t

e e c

ee cy

Apply initial conditions, at t = 0, y(0) = y0 = ⎥⎦

⎤⎢⎣

⎡−11

i.e., ⎥⎦

⎤⎢⎣

⎡+⎥

⎦

⎤⎢⎣

⎡ −+⎥

⎦

⎤⎢⎣

⎡−=⎥

⎦

⎤⎢⎣

⎡− 0

00501000

111

11

21.

c c

⇒ 1 = - c1 - c2 + 0.005

⇒ -1 = c1 + 1000 c2

or, ⎥⎦

⎤⎢⎣

⎡−

=⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡ −−1

995010001

11

2

1 .cc

or, ⎥⎥⎦

⎤

⎢⎢⎣

⎡

−

−=⎥

⎦

⎤⎢⎣

⎡−⎥

⎦

⎤⎢⎣

⎡−−

=⎥⎦

⎤⎢⎣

⎡−

9990050

999994

9991

2

1

19950

1111000

..

cc

49

The constants, ci, can also be determined from

[ ] [ ]

[ ] 999994

11

11000

00050

110001

111000

1

1111 −=

−

−+

−=><

>−<+><=

⎥⎦⎤

⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

1 oc

.

v,wbA,wy,w

[ ] [ ]

[ ] 9990050

10001

11

00050

111

111

2

1222

.

.

v,wbA,wy,w −=

−

−+

−=><

>−<+><=

⎥⎦⎤

⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

⎥⎦⎤

⎢⎣⎡

1 oc

Therefore, ⎥⎦

⎤⎢⎣

⎡+

⎥⎥⎦

⎤

⎢⎢⎣

⎡ −−⎥⎥⎦

⎤

⎢⎢⎣

⎡−−=⎥⎦

⎤⎢⎣

⎡−

−

−

−

00050

1000 1000

1000

9990050

999994

2

1 ..t

t

t

t

e e

ee

yy

------------------

We now discuss the solution of the first-order, linear, ODE-IVPs with constant

coefficients using similarity transformation

ODE-IVPs of the form dy/dt = Ay

Consider a single linear differential equation of the type

yy ay o == )(; 0dtdy

where a is a constant. The solution of the above equation can be easily obtained:

dt a ty

oy ydy

∫∫ =0

or, oyatey =

Consider, now, a system of linear ODE-IVPs with constant coefficients:

oyy nyna yayadtdy

10112121111 =+++= )(;....

oyy nyna yayadtdy

20222221212 =+++= )(;....

50

•

•

noyny nynna ynaynadtndy =+++= )(;.... 02211

The above set of equations can be written in matrix form as

00 y)y(;Ayy == dtd

where

21

)0( ;

21

2222111211

; 21

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

•=

⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

•••••

••

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

•=

noy

oyoy

nnanana

naaanaaa

ny

yy

yAy

Using analogy, we can write the solution of this equation as

ote yAy =

We now show that above is, indeed, a solution. We have

. . . .22t2

tte +++= AAIA

(this is similar to e atat a t= + + +12 22 . . . . ). We have

[ ] o. . 22t2tdt

dteodtd

dtd yAAIAyy

⎥⎦⎤

⎢⎣⎡ +++==

. . 222= . . . 2

232 , ottottdtdor yAAIAyAAAy

⎥⎦⎤

⎢⎣⎡ +++⎥⎦

⎤⎢⎣⎡ +++=

[ ] o te AyyAA ==

Alternatively, we can simplify the equation, Ayy =dtd , by using matrices, U or

P. We have seen earlier that if we have n linearly independent eigenvectors for a

matrix, A, then

U-1 A U = D

51

where D is a diagonal matrix containing the eigenvalues of A. On the other

hand, if we do not have n linearly independent eigenvectors for A, we can use

P-1 A P = J

The columns of U contain the linearly independent eigenvectors of A while the

columns of P contain all the linearly independent eigenvectors possible to be

obtained from A, plus some generalized eigenvectors.

Therefore,

..AAIA . .2ttte22

+++=

. . . 2t t 21211 +++ −−−= UD UU D UU I U

122 . . . 2t t −

⎥⎦⎤

⎢⎣⎡ +++= U DDIU

1te −= U DU

where,

tnλe00

0t2λe000t1λe

te

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

•••••

••

=D

Therefore, the solution

o(0) ; dtd yyAyy ==

can be written in terms of the eigenvalues and the corresponding eigenvectors of

A as

o te yAy =

[ ] o te 1 yUDU −=

The above can also be derived as follows:

52

We define y ≡ Uz. This satisfies dtd

dtd zUy = , since the elements of U are

constants. Hence, we have

o(0) ; dtd yyAyy ==

or,

o=(0) ; dtd yUzUz AzU =

We pre-multiply this equation by U-1 to get

o=(0) ; dtd 1-1-1-1 yUUzUUz AUzUU =-

or, o=(0) ; dtd zzDzz =

where, zo ≡ U-1 yo.

These equations can be represented by

z(0)z ; zλdtdz 1o111

1 ==

z(0)z ; zλdtdz

2o2222 ==

•

•

noz(0)nz ; nznλdtndz

==

Since, dtidz

depends only on zi, this set of equations is uncoupled and each of

these can be solved individually (instead of simultaneously).The solution of this

system is

t1λe 10z(t)1z =

t2λe 20z(t)2z =

•

53

•

tnλe n0z(t)nz =

or,

noz

2oz1oz

tnλe00

0t2λe0

00t1λe

(t)nz

(t)2z(t)1z

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

•⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

•••••

•

•

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

•

or,

z(t) = exp(Dt) zo

But,

y(t) = U z(t)

So

[ ]

(t)nz

(t)2z(t)1z

n21

(t)ny

(t)2y(t)1y

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

••=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

•uuu

or,

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

•(t)ny

(t)2y(t)1y

[ ] [ ]

noy

2oy1oy

1-

tnλe00

0t2λe0

00t1λe

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

•⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢

⎣

⎡

•••••

•

•

= UU

Therefore, solution of coupled linear ODE-IVPs can be obtained by using

similarity transformation (U matrix) to decouple the ODE-IVPs as given above.

54

Still further, note that since

U-1 A U = D

We have,

A = U D U-1

Or, A2 = (U D U-1) (U D U-1) = U D2 U-1

And therefore, An = U Dn U-1

Example: Consider the following ODE-IVPs

0)0(1 ; 3131 =+= yyydtdy

2)0(2 ; 232 −=−= yydtdy

2)0(3 ; 3313 =+= yyydt

dy

or, (0) ; dtd oyyAyy ==

where

22

0

o ;

3y2y1y

; 301030103

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−= y yA

The eigenvalues and eigenvectors of A can easily be obtained as:

λi = 4, -3, 2

The U matrix containing the corresponding eigenvectors is given by

101

010101

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−=U

λ2 = -3

λ1 = 4 λ3 = 2

55

Since A is symmetric, its eigenvectors are orthogonal to each other. If we

normalize the eigenvectors, then UT = U-1. Normalization of the eigenvectors

gives

2102

1010

2102

1

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−=U

Then,

U-1 = UT = 0

0100

21

21

21

21

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−

⇒ U-1 yo =

2102

1010

2102

1

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−

222

22

0

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−

Therefore, y(t) is given by

y(t) = U [eDt] U-1 yo

or

2

22

00

0000

0

0100

)()()(

2

3

4

21

21

21

21

3

2

1

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−

t

t

t

ee

e

tytyty

or,

ee

eee

tytyty

tt

t

tt

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

+−

−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡−

24

3

24

3

2

12

)()()(

• Note that the above differential equation has eigenvalues of 4, -3, and 2.

• These appear in the exponential terms in the final solution and control the

steady state behavior.

56

• For large value of t (i.e., near steady state – most of the time we are

interested in such steady state solutions only), y(t) approaches e4tu1, where u1

is the eigenvector corresponding to the dominant eigenvalue, λl, and blows

up (tends to infinity).

• This is why it is often important to know only the dominant eigenvalue of A

and its associated eigenvector.

• The steady state solution is stable if all the eigenvalues have negative real

parts. Otherwise, the steady state solution is unstable in the sense that it

approaches infinity as time, t, approaches infinity.

• Also, it is to be noted here that the most rapidly decreasing solution

corresponds to the eigenvalue, -3, while the most rapidly increasing solution

corresponds to the eigenvalue, 4.

• The problem (ODE-IVP) is said to be stiff if it has eigenvalues such that one

has a very large magnitude, while another has a very small magnitude, i.e.,

⏐λl ⏐>> ⏐λn⏐. This is represented mathematically in terms of a stiffness

ratio, (⏐λmax ⏐/ ⏐λmin⏐). Large values of this ratio lead to computational

problems, as discussed later when numerical solution of ODE-IVPs are

discussed.

- - - - - -

Example: Consider the following reaction network in an isothermal batch

reactor

A B C

D

with CA(0) = CA0, CB(0) = CB0, CC(0) = CC0, CD(0) = CD0. The corresponding

model equations (ODE-IVPs) are:

k1 k3

k4k2

57

dCAdt Ak k C= − +( )1 2

dCBdt A B Ck C k C k C= − +1 3 4

dCCdt B Ck C k C= 3 4 -

dCDdt Ak C= 2

or,

ddt

ABCD

ABCD

CCCC

k kk k k

k kk

CCCC

⎡

⎣

⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥

=

− +−

−

⎡

⎣

⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥

⎡

⎣

⎢⎢⎢⎢

⎤

⎦

⎥⎥⎥⎥

( )1 21 3 4

3 42

0 0 00

0 00 0 0

or,

o(0) ; dtd yyKyy ==

The rank of matrix, K, is always less than n (n = 4, in this example) because of

the law of conservation of mass (mass cannot be created or destroyed) must be

satisfied, i.e.,

dCA

dtdCB

dtdCC

dtdCD

dt+ + + = 0

One of the above equations, thus, must be a linear combination of the others.

Also, since, the determinant of K is zero, one of its eigenvalues must be zero.

We leave the further solution of this problem as an exercise.

- - - - - - - -

58


⎥⎦

⎤⎢⎣

⎡=

2112

A

(i) Determine U and D matrix such that U-1 A U = D.

(ii) Using above information, solve the equation

00 x)x(x;Ax ==−= t dtd

i.e., ⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡==

⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡−=⎥

⎦

⎤⎢⎣

⎡01

00

2112

21

21

21

)()(

;txtx

xx

xx

dtd

The eigenvalues of A is given by f(λ) = λ2 - 4λ + 3 = 0 ⇒ λ = 1, 3

The eigenvector for λ = 1:

⎥⎦

⎤⎢⎣

⎡−

−=⎥

⎦

⎤⎢⎣

⎡=−=

1111

1111

adjadj I][Av λ ⇒ ⎥⎦

⎤⎢⎣

⎡−

=1

1αv


⎥⎦

⎤⎢⎣

⎡−−−−

=⎥⎦

⎤⎢⎣

⎡−

−=−=

1111

1111

adjadj I][Av λ ⇒ ⎥⎦

⎤⎢⎣

⎡=

11

αv

Therefore, ⎥⎦

⎤⎢⎣

⎡−

=⎥⎦

⎤⎢⎣

⎡=

1111

3001

UD and

Since, A is symmetric, normalize U matrix: ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−=

21

21

21

21

U

Then, U-1 = UT = ⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡ −

21

21

21

21

59

Now, let x = Uy, then

yAx yx D dtd

dtd −=⇒−= , y(0) = U-1 x(0)

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=⎥

⎦

⎤⎢⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡ −=⎥

⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡−=⎥

⎦

⎤⎢⎣

⎡

212

1

21

21

21

21

21

21

21

01

00

3001

yy

yy

yy

dtd

)()(

;

Solve dy1/dt = -y1, y1(0) =

21

⇒ y1 = C exp[-t]

Substituting initial conditions, we have

21 = C

Therefore, y1 = 2

1 exp[-t]

Solve dy2/dt = -3 y2 , y2(0) = 2

1

⇒ y2 = C exp[-3t]

Substituting initial conditions, we have

21 = C

Therefore, y2 = 2

1 exp[-3t]

Substituting back, we have

[ ]

[ ]⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

+−

+=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−==⎥

⎦

⎤⎢⎣

⎡−−

−−

−

−

tt

tt

t

t

ee

ee

e

e

xx

321

321

32

12

1

21

21

21

21

21 Uy

-------

The above example is when we have distinct eigenvalues and n-linearly

independent eigenvectors such that we can form U matrix.

60

However, when eigenvalues are repetitive and we need to use generalized

eigenvector(s) to form P matrix, coupled ODE-IVPs can not be completely de-

coupled but it is possible to nearly de-couple as shown in the next example.


⎥⎦

⎤⎢⎣

⎡ −=

3111

A

(i) Determine P and J matrix such that P-1 A P = J.

(ii) Using above information, solve the equation

00 x)x(Ax;x ==−= t dtd

i.e., ⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡==

⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡ −−=⎥

⎦

⎤⎢⎣

⎡10

00

3111

2

1

2

1

2

1)()(

;txtx

xx

xx

dtd

The eigenvalues are given by f(λ) = λ2 - 4λ + 4 = 0 ⇒ λ = 2, 2


⎥⎦

⎤⎢⎣

⎡−−

=⎥⎦

⎤⎢⎣

⎡ −−=−=

1111

1111

adjIAadj ][v λ ⇒ ⎥⎦

⎤⎢⎣

⎡−

=1

1αv

Determine generalized eigenvector from [A - λI] p = v

⇒ ⎥⎦

⎤⎢⎣

⎡−

α=⇒⎥⎦

⎤⎢⎣

⎡−

=⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡ −−1

01

1

1111

2

1 ppp

Therefore, ⎥⎦

⎤⎢⎣

⎡−−

=⎥⎦

⎤⎢⎣

⎡−−

=⎥⎦

⎤⎢⎣

⎡= −

1101

1101

2012 1P;P;J

Now, let x = Py, then

JyAx yx −=⇒−=dtd

dtd , y(0) = P-1 x(0)

61

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−=

⎥⎥⎦

⎤

⎢⎢⎣

⎡

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−−=

⎥⎥⎦

⎤

⎢⎢⎣

⎡

⎥⎥⎦

⎤

⎢⎢⎣

⎡

⎥⎥⎦

⎤

⎢⎢⎣

⎡−=

⎥⎥⎦

⎤

⎢⎢⎣

⎡

10

10

11

01)0()0(

; 2012

2

1

2

1

2

1yy

yy

yy

dtd

Solve first dy2/dt = -2 y2 , y2(0) = -1

⇒ y2 = C exp[-2t]

Note that the second equation is function of y2 only.

Substituting initial conditions, we have -1 = C

Therefore, y2 = - exp[-2t]

Solve now dy1/dt = -2 y1 - y2, y1(0) = 0

= -2 y1 + exp[-2t]

⇒ y1 = C exp[-2t] + t exp[-2t]

Note that first equation is now function of y1 only as solution of y2 can be

substituted.

Substituting initial conditions, we have 0 = C

Therefore, y1 = t exp[-2t]

⎥⎥⎦

⎤

⎢⎢⎣

⎡

+−=

⎥⎥⎦

⎤

⎢⎢⎣

⎡

−⎥⎦

⎤⎢⎣

⎡−−

==⎥⎦

⎤⎢⎣

⎡−−

−

−

−

ttt

tt

etete

ete

xx

222

22

21

1101

Py

62

ODE-IVPs of the form dy/dt = Ay, with all eigenvalues of A being equal

We again consider the ODE-IVP:

o(0) ; dtd yyAyy ==

but, with the eigenvalues of A being all equal, say λ. We need to use the P

matrix now. We substitute y = Pz in the above equation to get

Pz AzP =dtd

or,

o(0) ;dtd 11 yPz zJ z APPz −− ===

or,

nz1nz

2z1z

λ0001000

01λ0001λ

nz1nz

2z1z

dtd

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

−

•

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

••

•••••••

=

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

−

•

This can be expanded to

2z1z λdt1dz

+=

3z2z λdt2dz

+=

•

•

nz1nz λdt1ndz

+−=−

nz λdtndz

=

The solution of the above set of ODE-IVPs is given by

zn = zn(0) eλt

zn-1 = zn-1(0) eλt + zn(0) t eλt

63

zn-2 = zn-2(0) eλt + zn-1(0) t eλt + zn(0) (t2/2) eλt

•

•

z1 = z1(0) eλt + z2(0) t eλt +. . . + zn(0) [tn-1/(n-1)!] eλt

or, in matrix form, by

(0)nz(0)1nz

(0)2z(0)1z

1000t000

2)!(n2ntt10

1)!(n1nt

22tt1

tλe

nz1nz

2z1z

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

−

•

⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

••

•••••−−

•

−−

•

=

⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢

⎣

⎡

−

•

or,

z = K zo

This leads to

y = P z = P K zo = P K P-1 yo

64

Higher-order IVPs

Let us now consider the following higher-order IVP with constant coefficients:

oy dcybadt

oyddt

odydtdy

dt

yd

dt

yd γβα ====+++ 2

2

2

2

3

3 )()( ,,)(;

We can convert this equation into a set of first order ODE-IVPs using the

procedure described below. We define

y = y1

then

dydt

dydt y= =1

2

d y

dt

dydt y

2

22

3= =

d y

dt

dydt

3

33=

Substituting the above equations in the original equation, we have

1233 dcybyaydt

dy +−−−=

or,

dy

yy

abcy

yy

dtd

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡+

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−−=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡00

100010

3

2

1

3

2

1

This may be written in matrix form as

γ β α

(0) ; dtd

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=+= yeAyy

65

Note that the vector, e, is a constant (and not a function of time). We can

transform this equation into a homogeneous one by using the concept of the

steady state (SS; represented by subscript, S) solution. We have at steady state:

=dtd Sy 0 = A yS + e

or

yS = - A-1e = [d/c 0 0]T

We now subtract the SS equation from the original one to obtain

)s( dt)sd(

yyAyy

−=−

If we define x ≡ y - yS, we have

o(0) ; dtd xxAxx ==

and the solution is

o te (t) 1 xUDUx −=

or

( ) ( )oso te oso te s(t) 111 yyUDUeAyyUDUyy −+−=−+= −−−

66

ODE-IVPs of the form dy/dt = Ay + B(t)

We now consider the more general equation

dy/dt ≡ y' = Ay + b(t)

where b is a function of time.

Using y = Uz, we obtain

U z' = y' = Ay + b(t) = A U z + b(t)

On pre-multiplying both sides of this equation with U-1, we obtain

U-1 U z' = U-1 A U z + U-1 b(t)

or,

z' = D z + U-1 b(t) ≡ D z + g(t)

Here, g(t) ≡ U-1b(t), and D is a diagonal matrix whose entries are the

eigenvalues, λ1, λ2, . . . , λn, of A, arranged in the same order as the

eigenvectors, u1, u2, . . . , un, (appearing as columns in U).

The above is a system of n-uncoupled ODE-IVPs in z1(t), z2(t), . . . , zn(t), and

may be solved individually. The ith equation may be written as

z'i (t) = dzi/dt = λi zi (t) + gi (t); i = 1, 2, . . . , n

Above has the following solution

ds (s)igt

ot

siλe tiλetiλeic(t)iz ∫−

+= ⎥⎦⎤

⎢⎣⎡

; i = 1, 2, . . . , n

This may be used with y = Uz to give the solution, y(t).

67

Example: Solve y1'' +102 y1' + 200 y1 = t, y1(0) = 1, y1'(0) = - 2

Define y1' = y2 y1(0) = 1

then y1'' = y2' = - 102 y2 - 200 y1 + t y2(0) = - 2

or, ⎥⎦

⎤⎢⎣

⎡−

=⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡+⎥

⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡−−

=⎥⎦

⎤⎢⎣

⎡2

1000

10220010

2

1

2

1

2

1)()(

yy

ty

y

yy

dtd

or, 00 y)y()g(yAy =+= tdt

d

where

⎥⎦

⎤⎢⎣

⎡−

=⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡−−

=⎥⎦

⎤⎢⎣

⎡=

210

10220010

02

1 ygAy ; t

; ; yy

Eigenvalues of A:

f(λ) = λ2 - (tr A) λ + det A = λ2 +102 λ + 200 = 0 ⇒ λ = -2, -100

Eigenvectors of A for λ = -2:

adj [A + 2I] = adj ⎥⎦

⎤⎢⎣

⎡−=⇒⎥

⎦

⎤⎢⎣

⎡ −−=⎥

⎦

⎤⎢⎣

⎡−− 2

122001100

10020012

1 αv

Eigenvectors of A for λ = -100:

adj [A + 100I] = adj ⎥⎦

⎤⎢⎣

⎡ −=⇒⎥

⎦

⎤⎢⎣

⎡ −−=⎥

⎦

⎤⎢⎣

⎡−− 100

1100200

122200

11002 βv

The matrix U, U-1 and D are given by

⎥⎦

⎤⎢⎣

⎡−

−=⎥

⎦

⎤⎢⎣

⎡−−

=⎥⎦

⎤⎢⎣

⎡ −−= −

100002

121100

100211

9811 DUU

so that U-1 A U = D.

68

Substituting y = U z in y' = Ay + g(t), we have y' = U z' = A U z + g(t)

Or, z' = U-1 A U z + U-1 g(t) = D z + U-1 g(t)

Or, ⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡−−

+⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡−

−=⎥

⎦

⎤⎢⎣

⎡− t

zz

zz

dtd 0

121100

100002

981

2

1

2

1

Or, ⎥⎦

⎤⎢⎣

⎡−

−⎥⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡−

−=

tt

zz

981

2

1

100002

Since, y = U z, we have y0 = U z0

or, z0 = U-1 y0

or, z0 = ⎥⎦

⎤⎢⎣

⎡−=⎥

⎦

⎤⎢⎣

⎡−⎥

⎦

⎤⎢⎣

⎡−−

−01

21

121100

981

Therefore, we have

102 19811 −=−−= )(z z t

dtdz

and 00100 29822 =+−= )(z z t

dtdz

or, ⎥⎦

⎤⎢⎣

⎡−= ∫

−− dsseeeczt stt

0

229812

11

or, ⎥⎦⎤

⎢⎣⎡−=

−+−−4

2129812

1tettec

Applying initial conditions: at t = 0, z1 = -1, we have

-1 = c1 - (1/98) (0) ⇒ c1 = -1

Therefore,

[ ] etez tt 2392

121 12 −− +−−−=

Note that

[ ]a

x

aaxeay xdyye 1

0−=∫

69

Similarly, z2 can be obtained as

[ ]t

xe tz 100

510891

2 1100 −+−=.

Substituting z in y = U z, we can get desired solution y.

eigenvalue eigenvector

Documents