eigenvalue eigenvector
Post on 21-Jan-2016
108 Views
Preview:
DESCRIPTION
TRANSCRIPT
1
EIGENVALUES AND EIGENVECTORS
Consider the following set of coupled, linear, first order, ordinary
differential equations-initial value problems (ODE-IVPs), with constant
coefficients:
112121111 . . . bxaxaxa nndt
dx ++++= , x1(0) = x10
222221212 . . . bxaxaxa nndt
dx ++++= , x2(0) = x20
•
•
nnnnnndtndx bxaxaxa ++++= . . .2211 , xn(0) = xn0
This can be written in compact notation, as
)(tdtd bAxx += x(0) = x0
• A is an n × n matrix of constants.
• The use of vectors and matrices not only saves a great deal of space and
facilitates calculations, but also emphasizes the similarity between systems of
equations and a single equation.
• Above equation is encountered quite commonly in engineering practice.
o Systems described by lumped-parameter models in chemical engineering,
e.g., continuous-flow stirred tank reactors (CSTRs), distillation, extraction
and absorption columns, etc., are quite well described by such equations
under dynamic (unsteady) conditions.
o The behavior of batch (and plug flow) reactors under steady conditions.
2
o These equations are useful in other fields, too, as for example, for
describing the sustained harmonic oscillations of strings, membranes and
structural systems (bridges, etc.), in the presence of external forces.
o At a more abstract, mathematical level, the study of the stability of
numerical solutions of ordinary and partial differential equations (PDEs)
also requires the solution of (homogeneous) linear ODEs.
)(tdtd bAxx += x(0) = x0
In solving above, it is necessary to solve, first, the following set of
homogeneous linear ODEs:
Axx =dtd
In order to solve above, we assume the solution to be of the following form:
x = v eλt
This is similar to the procedure used to solve single, linear, homogeneous ODEs
with constant coefficients. The constant, λ, and the constant vector, v, are to be
determined.
On substituting we obtain
v λ eλt = A v eλt
Since eλt ≠ 0, this leads to A v = λ v
or,
(A - λ I) v = 0
3
Thus, in order to solve the system of ODE-IVPs, we must solve the above
system of linear algebraic equations.
• The above system of linear algebraic equations is encountered quite often
in engineering and science, and has been studied extensively.
• This equation is satisfied by several values of λi, which are referred to as the
eigenvalues of the matrix, A.
• For each λi, there is a corresponding vector, vi. The latter is referred to as
the eigenvector of the coefficient matrix, A, associated with λi.
• The above system represents a set of linear, homogeneous algebraic
equations (with v = 0 if the rank of [A - λI] is n).
• The necessary condition for this equation to have non-trivial solutions that
are physically meaningful, is
det (A – λi I) = 0, i = 1, 2, . . . , n.
This ensures that the rank of (A - λiI) is less than n.
• Corresponding to each eigenvalue, λi, there is an eigenvector, vi.
• Thus, there are, in general, n solutions, and the general solution of the
homogeneous is a linear combination of these
x = tiei
n
i
λv∑
=1 ci
• ci are arbitrary constants that are obtained using the initial conditions.
n-λi’s exist for which det (A - λi I) V = 0
4
If λi = ai + i bi, I = 1, 2, 3, . . . , n.
Then
[ ][ ]∑ +==
n
iii
tiaii tbtbeC
1cossinvx
The significance of the eigenvalues, λi, can easily be inferred from the
above solution.
• If we consider xi to represent the deviations of the state variables from
their steady state values, this equation suggests that these deviations will
die out asymptotically with time, t, if all the eigenvalues are real and
negative.
• Similarly, if even one of the λis is positive and real, the corresponding
deviation will increase with time, and the system will be unstable.
• If any λi is complex, the corresponding deviation will be oscillatory in
nature (due to the imaginary part), and its real part will decide whether it
will die out or blow up in time.
• Thus, λi are related to the (asymptotic) stability of a system under (small)
perturbations.
5
A particular solution of
)(tdtd bAxx += x(0) = x0
can readily be obtained if b is a constant vector, i.e., it is not a function of t.
In this case the particular solution can be obtained by substituting x = k (where k
is a constant).
0 = A k + b
or
k = - A-1 b
Therefore, the complete solution is given by
x = tiei
n
i icλ
v∑=1
- A-1 b
We now apply the initial conditions, x(t = 0) = x0. We obtain
x0 = ∑=
n
i iic1
v - A-1 b
This comprises a set of n equations in the n unknowns, ci.
It can be shown that the ci can be uniquely determined (since vi are eigenvectors,
they form a linearly independent set, and so the rank of the set of equations for ci
is n.
6
The Characteristic Polynomial
Consider the n × n matrix, A. The eigenvalues of this matrix may be obtained by
solving:
det (A – λi I) = 0, i = 1, 2, . . . , n
while the eigenvector, vi, associated with λi obtained using (for v ≠ 0)
Avi = λi vi
The eigenvalue, λi, may be real or complex.
In expanded form we have
det (A-λI) = 0
21
2222111211
)( =
−••••••••••••
••−••−
≡
λ
λλ
λ
nnanana
naaanaaa
f
We can expand the determinant to give
00 1...2 21 1)( =+++−
−+−−+= αλαλαλαλλ n
nn
nnf
Clearly, above has n roots (i.e., n-eigenvalues).
Example 1: Obtain the eigenvalues of the following matrix
111320
746
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−
−=A
det (A - λI) = det λ
λλ
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−−
−−
111320
746
⇒ f(λ) = - λ3 + 9 λ2 - 30 λ + 32 = 0
⇒ λ = 2, [7 ± i 15]/2 where i = √- 1.
7
The nth order polynomial, f(λ), can also be written in terms of the principal
minors, Sj(A), as
f(λ) = Pn(λ) = det (A - λI) = jnλ)( )(
n
0j jS −−∑=
A
In this equation, So = 1, and Sj(A) is the sum of all possible principal minors of
A, of size j × j obtained by striking out an equal number, n – j, of rows as well as
columns from A, and taking the remaining j × j elements.
While doing this, it must be remembered that identical-numbered rows and
columns need to be struck off from A in order to obtain the several minors in Sj.
The procedure is best illustrated through an example.
a a a a aa a a a a
a a a a a
a a a a a
a a a a a
k i nk i n
k k kk ki kn
i i ik ii in
n n nk ni nn
11 12 1 1 121 22 2 2 2
1 2
1 2
1 2
• • •• • •
• • • • • • • •• • •
• • • • • • • •• • •
• • • • • • • •• • •
⎡
⎣
⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥
Example 2: Find the characteristic polynomial for the 3 × 3 matrix, A, in
Example 1:
f(λ) = det (A - λI) = So (-λ)3 + S1 (-λ)2 + S2 (-λ) + S3
We have
S3 = det A = 32
Principal minors
Intersection of striking out of rows and columns should be on diagonal in forming the principal
minors.
8
This is obtained by deleting 0 rows and 0 columns in A. Only one principal
minor results.
S2 can be written as
S2 = a aa a
a aa a
a aa a
11 1221 22
11 1331 33
22 2332 33
12 13 5 30+ + = + + =
S2 is the sum of three principal minors, obtained by deleting the third, second
and first rows as well as columns, respectively, from A.
S1 can be written as
S1= a11 + a22 + a33 = 6 + 2 + 1 = 9
This is the sum of three principal minors again, obtained by deleting two rows
and two (identically numbered) columns from A.
Also, So = 1
Therefore,
Pn(λ) = - λ3 + 9 λ2 - 30 λ + 32 = 0
The same result as obtained in Example 1. -----------
It can easily be deduced that
S1 = aiii
n
=∑
1= sum of diagonal terms ≡ trace of A = tr A
Also,
Sn = det A
Therefore, for a 2 × 2 matrix
P2(λ) = λ2 – (tr A) λ + (det A) = 0
9
We can also write Pn(λ) = 0 in terms of a multiple product involving the n roots,
λi, as: )( )()(0)()( 2
1
1
11
−
> =
−
==
−⎥⎦
⎤⎢⎣
⎡+−⎥
⎦
⎤⎢⎣
⎡+−==−= ∑ ∑∑∏ n
n
ij
n
iji
nn
jj
nn
jjnP λλλλλλλλλ
+ . . . . . +=∏ λ jj
n
1
A comparison of the above leads to
S1 = tr A = λ ii
n
=∑
1= sum of all the eigenvalues
Sn = det A = λ ii
n
=∏
1= product of all the eigenvalues
By a similar comparison, we can obtain
S2 = λ λii
n
j i
nj
=>∑∑
1
For n = 3, thus, we can write
S2 = λ1λ2 + λ1λ3 + λ2λ3
10
Properties of Eigenvalues and Eigenvectors
• An upper bound of the eigenvalues is given by:
Max |λi| ≤ ||A||
where the norm of matrices is discussed in Section 5.6.1.
Proof:
We can get a bound on ⏐λi⏐as follows:
Avi = λivi (vi ≠ 0) Then,
⏐λi⏐ ⏐⏐vi⏐⏐ = ⏐⏐λivi⏐⏐ = ⏐⏐Avi⏐⏐ ≤ ⏐⏐ A⏐⏐ ⏐⏐vi⏐⏐ Therefore,
⏐λi⏐ ≤ ⏐⏐ A ⏐⏐ for all i for any norm.
• When the matrix, A, has distinct eigenvalues, λj, (i.e., no repeated
eigenvalues), then the set of eigenvectors, vj, forms a linearly
independent set.
Proof:
Let vi and vj be two eigenvectors of A that correspond to eigenvalues, λi and λj,
respectively. Then vi and vj are not constant multiples of each other, because if
they were, then
vi = k vj (a)
Here, k is a non-zero constant. If we pre-multiply Eq. (a) by the matrix, A, then
Avi = k Avj (b)
This is equivalent to
λi vi = k λj vj (c)
If we now multiply both sides of Eq. (a) by λi, we have
λi vi = k λi vj (d)
11
Subtracting Eq. (c) from Eq. (d), we have
k (λi - λj) vj = 0
Since k ≠ 0, and, λi ≠ λj, this means that vj = 0, which contrary to the assumption
that vj is an eigenvector.
Therefore, the eigenvectors corresponding to different (distinct)
eigenvalues of A are linearly independent.
• If the n × n matrix, A, is symmetric, then all n of its eigenvalues, λi (i = 1,
2, . . ., n), are real, i.e., the roots of the characteristic polynomial, f(λ) = 0, have
n real roots.
However, these roots need not necessarily be distinct.
If the matrix is not symmetric, then some of the eigenvalues may be
complex.
• For a symmetric matrix, A, there are n distinct eigenvectors, vi, one
corresponding to each eigenvalue, λi. This is so even if some of the
eigenvalues are not distinct.
However, if A is not symmetric, then it may not always be possible to find
n linearly independent eigenvectors (if some of the eigenvalues are repetitive).
• The n distinct eigenvectors of a symmetric matrix, A, are orthogonal to
each other and can easily be made orthonormal. Thus,
<vi, vj > = vi† vj = δij
These eigenvectors form a complete basis set.
12
Hence, any n-dimensional vector, x, can be represented in terms of the vi as
x = ∑=
n
1i iic v
where
ci = vi† x = <x, vi >
• Let U = [v1 v2 v3 . . . vn] be the n × n matrix whose columns contain the n
orthonormal eigenvectors of the symmetric matrix, A. It is easy to show
that
U†U = I
This means that
U† ≡ (U*)T = U-1
Such matrices are referred to as unitary matrices.
If U is real, then UTU = I, and then UT = U-1. Such matrices are referred to as
orthogonal matrices, rather than unitary matrices. The rows (or columns) of a
unitary matrix form an orthogonal basis.
Example: Consider the following two U matrices:
2
12
221
2 ; 1 ⎥⎥⎦
⎤
⎢⎢⎣
⎡⎥⎦⎤
⎢⎣⎡
=−
= i
i
xcosxsinxsinxcos
UU
U1 is orthogonal since it is real, and
T
xxxx
xx 12211
1 Ucossinsincos
sincosU =
−
+=−
⎥⎦⎤
⎢⎣⎡
13
Similarly, it may be demonstrated that U2 is unitary.
=⎥⎦⎤
⎢⎣⎡=
⎥⎥
⎦
⎤
⎢⎢
⎣
⎡
−
−=−
T
i
iUU *
22
12
221
12 U2
†
• Principal Axes Theorem
Let A be a self-adjoint (or Hermitian) matrix of order n, [defined by: A† ≡
(A*)T = A]. Then, A has
o n real eigenvalues, λ1, λ2, . . ., λn, not necessarily distinct
o n corresponding distinct eigenvectors, v1, v2 , . . ., vn, that form a
complete orthonormal basis set. The components of the eigenvectors
can be complex.
o If the elements of A are real, then the components of the eigenvectors will
also be real.
o Finally, there is an unitary (or orthogonal) matrix U, for which
U†AU = D ≡ diagonal (λi)
where
000020001
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
••••••
=
nλ
λλ
D
By pre-multiplying U†AU (= D) by U, and post-multiplying it by U† (and using
the fact that U† = U-1), we can show that
A = UDU†
We can only diagonalize a matrix, A, if the eigenvectors form a basis set,
i.e., we have n linearly independent eigenvectors.
This is always true for symmetric matrices.
14
However, if A is not symmetric, then it may not always be possible to find n
linearly independent eigenvectors (if some of the eigenvalues are repetitive), and
in that case we will not be able to diagonalize the matrix.
Example: Consider the following symmetric matrix
210131012
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=A
We can find the eigenvalues of A as follows:
f(λ) = -λ3 +7λ2 - 14λ + 8 = 0
⇒ [λ1, λ2, λ3] = [1, 2, 4]; all real, as expected.
The corresponding eigenvectors can be obtained by solving for non-zero vi using
Avi = λivi or (A - λiI)vi = 0.
As the (A - λiI) matrix is singular, we have to assign one or more components of
vi arbitrarily, and then find the remaining components.
When λ = 1, we have for (A-λI) v1 = 0 as
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−=⇒
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
11
1
000
110121011
1
13
12
11
vvv
αv
Similarly, we obtain
121
γ3 ; 1
01
β2 ; 11
1 α1
⎥⎥
⎦
⎤
⎢⎢
⎣
⎡
⎥⎥
⎦
⎤
⎢⎢
⎣
⎡
⎥⎥
⎦
⎤
⎢⎢
⎣
⎡=
−=−= vvv
As expected, these eigenvectors are orthogonal to each other, and <v1, v2> = <v1, v3> = <v2, v3> = 0
These vectors are also linearly independent.
15
We can, of course, make these vectors orthonormal by normalizing them to obtain
121
6
13 ;
101
2
12 ;
11
1
31
1 ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−= vvv
If we now form the U matrix as follows (each column of U contains one of the
orthonormal eigenvectors):
U
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
−
−=
61
21
31
62
31
61
21
31
0
then
UTAU = D = 1 0 00 2 00 0 4
⎡
⎣
⎢⎢⎢
⎤
⎦
⎥⎥⎥
Also not that U-1 = UT.
• If all the λis are distinct (even for non-symmetric A), then A has n linearly
independent eigenvectors. This can be easily demonstrated and is shown below:
We can write, in general
01
21 =−=−−−=−= ∏=
)())....()((I)(A)( λλλλλλλλλλn
jjnn detP
When the ith eigenvalue, λi, is simple (i.e., λi has a multiplicity of 1), we can
rewrite Pn(λ) as
01
=−−= ∏
≠=
)()()( λλλλλn
jij
jin P
Differentiating it with respect to λ, and then substituting λ = λi, we get
Eigenvalues
Rank of (A-λI) < n
16
01
≠−−= ∏
≠==
)()( λλλλλ
λ n
ijj
jddP
i
n
Therefore, the rank of (A - λiI) is n - 1 (when λi is a distinct eigenvalue). Then,
Dimension of null space = n – [rank of (A - λiI)] = n – (n-1) = 1
Therefore, there is one linearly independent eigenvector for each λi. as
long as it is a distinct eigenvalue. Note that the symmetry of A was not invoked.
Hence, if all eigenvalues are distinct, then A has n linearly independent
eigenvectors, which form a complete basis set. Therefore, any n-
dimensional arbitrary vector can be represented as linear combinations of vi’s.
Example: Obtain the eigenvalues and the corresponding eigenvectors of the
following symmetric matrix
⎥⎦
⎤⎢⎣
⎡=
1111
A
We have
det (A - λI) = λ2 – (tr A) λ + det(A) = λ2 – 2λ = 0
⇒ λ = 0, 2 (i.e., the symmetric matrix, A, has two distinct eigenvalues).
To obtain the eigenvectors corresponding to λ = 0, we solve for non-trivial v
using (A - λI) v = 0:
⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡=−
00
1111
2
11
vv
v)IA( λ
or
v1 + v2 = 0
v1 + v2 = 0
Rank of (A-λiI) = n -1
17
These two equations are linearly dependent. We have to fix one of the
components of v arbitrarily to find the other one. If we fix v2 = α, then
⎥⎦
⎤⎢⎣
⎡−=
11
αv
When λ = 2:
vv
vv
⎥⎦
⎤⎢⎣
⎡==⎥
⎦
⎤⎢⎣
⎡⇒⎥
⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡−
−11
00
1111
2
1
2
1 βv
We obtained two linearly independent eigenvectors for the two distinct
eigenvalues.
Example: Now consider the following unsymmetric matrix: ⎥⎦
⎤⎢⎣
⎡−−
=11
43A
We have
det (A - λI) = λ2 – (tr A) λ + det(A) = λ2 – 2λ + 1 = 0 ⇒ λ = 1, 1
We now have two eigenvalues that are identical.
We solve for non-trivial v using (A - λI) v = 0 for λ = 1:
⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡−−
=−00
2142
2
1 vv
v)IA( λ
or
2 v1 + 4 v2 = 0
- v1 – 2 v2 = 0
The two equations are linearly dependent (rank = 1, dimension of null space =
1). We have to fix one component of v arbitrarily and evaluate the other one. If
we fix v2 = α, we obtain
⎥⎦
⎤⎢⎣
⎡−=
12
αv
18
Note that we can only find only one linearly independent eigenvector for the
eigenvalue λ = 1 with a multiplicity of two since dimension of null space is 1
(and therefore, only one independent vector exists).
We do not have two eigenvectors although we have two (identical) eigenvalues
for the non-symmetric A.
Example: Evaluate the eigenvalues and the eigenvectors of the symmetrical
identity matrix
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
100010001
A
We have
f(λ) = (1 - λ)3 = 0 ⇒ λ = 1 with a multiplicity, p of 3
Since the rank and dimension of null space for the (A-λI) matrix are 0 and 3
respectively, we can find three linearly independent eigenvectors for the above
symmetric matrix:
v1 = [1 0 0]T, v2 = [0 1 0]T, v3 = [0 0 1]T
From the principa1axes theorem, we know that for a symmetric matrix,
• All the eigenvalues are real, but all of them are not necessarily
distinct.
• However, it is always possible to find distinct eigenvectors regardless
of whether eigenvalues are distinct or repetitive.
• The corresponding eigenvectors also form a complete orthonormal
basis.
Rank of (A-λI) = 0 ⇒ Dimension of null
space = 3
19
• This means that we will always be able to find n linearly independent
eigenvectors even if all the n eigenvalues are not distinct when matrix is
symmetric.
Consider now the following non-symmetric matrix
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
100110011
A
f(λ) = (1 - λ)3 = 0 ⇒ λ = 1; p (multiplicity) = 3
Since the rank and dimension of null space for the (A-λI) matrix are 2 and 1
respectively, the matrix has only one linearly independent eigenvector for the
eigenvalue, λ = 1:
v = [1 0 0]T
• If a matrix, A, is singular, then one or more (depending on the rank of the
matrix) of the eigenvalues is zero.
We can also find the eigenvectors of A from the non-trivial columns of
adj(A - λI).
Note that when a matrix is singular, A-1 does not exist but adjugate of A exist.
A-1 = [ ]AadjA det1
Rank of (A-λI) = 2 ⇒ Dimension of
null space = 1
For non-trivial x from (A-λI) x = 0
can be obtained from non-trivial columns of adj(A-λI).
20
Example: Consider the unsymmetrical matrix
⎥⎦
⎤⎢⎣
⎡−−
=11
43A
We found λ = 1 (multiplicity = 2) for this matrix. We have, then,
2142
2142
⎥⎦
⎤⎢⎣
⎡ −−=⎥
⎦
⎤⎢⎣
⎡−−
=λ− adjadj )IA(
Example: Consider the symmetrical matrix:
⎥⎦
⎤⎢⎣
⎡=
1111
A
We had obtained λ = 0 and 2.
For λ = 0: adjadj ⎥⎦
⎤⎢⎣
⎡−
−=⎥
⎦
⎤⎢⎣
⎡=−
1111
1111
I)(A λ
For λ = 2: adjadj ⎥⎦
⎤⎢⎣
⎡−−−−
=⎥⎦
⎤⎢⎣
⎡−
−=−
1111
1111
)IA( λ
However, if repetitive eigenvalues are present and dimension of (A - λI) matrix
is greater than 1, then sometimes it may not be possible to obtain eigenvectors
from adj(A - λI) as all components of adj(A - λI) matrix may be zero.
Eigenvectors
Eigenvectors
Eigenvectors
21
Similar Matrices
Two n × n matrices, A and B, are said to be similar if there exists a non-singular
matrix, P, such that B = P-1AP, or, equivalently, A = PBP-1.
Matrices A and B are similar in the sense that
• their characteristic polynomials are identical (and so are their eigenvalues),
• their eigenvectors are related through the matrix, P.
• They are said to be represented by the same linear transformation but with
respect to different bases.
We first demonstrate that the two characteristic polynomials, fA(λ) and fB(λ), are
the same.
fB(λ) = det (B - λI) = det [P-1 A P - λP-1 I P] = det [P-1 (A - λ I) P]
= det(P-1) det (A - λI) det(P) = det (A - λI) = fA(λ)
since det(P-1) × det(P) = 1
Therefore, A and B have identical eigenvalues.
Since fA(λ) = fB(λ), the coefficients of the characteristic polynomials are the
same. This means that
trace of A = trace of B and det A = det B
When matrices A and B are similar, even though their eigenvalues are identical,
their eigenvectors are not. However, there is a one to one correspondence, as
shown below. We have
Ax = λ x
or, P-1Ax = P-1λ x
or, P-1A (P P-1) x = λ P-1 x
22
or, B (P-1x) = λ (P-1 x)
or, B z = λ z
Therefore, the eigenvectors of B are given by z = P-1x, where x is the
eigenvector of A.
In summary
• For a (n × n) matrix A, there exist n eigenvalues.
• The eigenvalues can be obtained from the characteristic polynomial
f(λ) = det (A - λI) = 0.
• The eigenvectors can be obtained from either
• The non-trivial solution v from (A - λI) v = 0, or
• Non-trivial columns of adj (A - λI) = 0.
• The eigenvalues and eigenvectors can also be obtained numerically, for
example by Inverse Power method.
• The n eigenvalues may be distinct or repetitive (multiple), and real or
complex.
• If all the eigenvalues are distinct (real or complex), then there exist n
eigenvectors, which are linearly independent to each other.
• The n eigenvectors, vi, (i = 1, 2, …n) are obtained from
either (A - λiI) vi = 0 or from adj (A - λiI) = 0.
• If we form a (n × n) U matrix such that each column of U matrix contains the
n eigenvectors, i.e., U = [v1 v2 v3 . . . vn], then U-1 A U = D, where
D is a diagonal matrix whose diagonal elements are the n eigenvalues.
23
• If all the eigenvalues are not distinct (real or complex), then we may or
may not be able to find n linearly independent eigenvectors. The
number of linearly independent eigenvectors that can be found for the
eigenvalue λi with multiplicity m will depend on the dimension of null
space for the (A - λiI) matrix.
• If the dimension of null space for the (A - λiI) matrix is 1, for the eigenvalue
λi with multiplicity m, then only one linearly independent eigenvector can be
found for corresponding to the eigenvalue λi with multiplicity m.
• If, however, the matrix A is self-adjoint (or symmetric), then
• All the eigenvalues of matrix A are always real.
• All the eigenvalues may not be distinct, but we will always be able
to find n linearly independent eigenvectors irrespective of whether
the eigenvalues are distinct or repetitive.
• The eigenvectors will not only be linearly independent, but they will
also be orthogonal.
• If we normalize the eigenvectors, and then form the U matrix, then we
will find U-1 = UT. That is the U matrix will be a unitary (or
orthogonal) matrix.
• Therefore, U-1 A U = D will be same as UT A U = D.
24
However, if the matrix is not symmetric, then it may not always be
possible to find n- linearly independent eigenvectors (particularly, if
some of the eigenvalues are repetitive), and in that case we will not be able to
diagonalize the matrix.
• Algebraic multiplicity: Multiplicity in eigenvalues as obtained from the
root of Pn(λ) or f(λ) = 0. Example: In the previous two matrices, algebraic multiplicity was 3 (for λ =
1, p was equal to 3 for both the matrices).
• Geometric multiplicity: Maximum number of linearly independent
eigenvectors possible associated with all the eigenvalues. Example: Although, algebraic multiplicity was 3 for both cases, we had 3
linearly independent eigenvectors for the first case (symmetric identity
matrix) whereas, we had only one linearly independent eigenvectors for the
second case. Therefore, geometric multiplicity is 3 for the first case (identity matrix),
whereas geometric multiplicity is only 1 for the second case. • When a matrix A (n x n) has a eigenvalue λ of algebraic multiplicity p but
geometric multiplicity less than p, then, we can form a Jordon-Canonical
Block, J, consisting of all the eigenvalues λ of A, and can find a non-singular
matrix P such that P-1AP = J.
25
• The above is similar to U-1AU = D except that • Matrix J is not a diagonal matrix but has a different form. • Each columns of U matrix consist of n- linearly independent eigenvectors,
whereas columns of P matrix consist of as many as linearly independent
eigenvectors possible to be obtained from A, plus some generalized
eigenvectors.
Note that when matrix A is symmetric, only then U-1 = UT (of course, only
if the eigenvectors were normalized), otherwise U-1 ≠ UT.
Therefore, we must use the relation U-1AU = D when matrix A is not
symmetric but still we can find n-linearly independent eigenvectors. • How to form the Jordon-Canonical Block?
When we have n-eigenvalues all equal, say λ, (i.e., algebraic multiplicity
equal to n), but it is possible to obtain only one linearly independent
eigenvector (i.e., geometric multiplicity equal to 1), then
i.e., λ at the diagonal position and 1 above λ. The rest of the elements are
zero.
00001000
00100001
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
•••••=
λλ
λλ
nJ
26
• If A is a (n x n) matrix, then, there always exist a non-singular matrix P for
which
⇒ The eigenvalues λ1, λ2 , …, λr need not be distinct.
⇒ The number of distinct blocks Jnr(λr) must be equal to the number of
independent eigenvectors.
⇒ Same eigenvalues may occur in different blocks.
Example: Let A be a (6 x 6) matrix having the following eigenvalues and the
corresponding number of linearly independent eigenvectors: λ1 = 3, p = 2 with only 1 linearly independent eigenvector.
λ1 = 4, p = 3 with only 1 linearly independent eigenvector.
λ1 = -2, p = 1 with 1 linearly independent eigenvector.
400000140000014000000300000130000002
⎥⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢⎢
⎣
⎡−
=J
Number of distinct blocks must be
equal to the number of
independent eigenvectors
)(000
00)(0000)(
22
11
1
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
••••=−
rnr
n
n
J
JJ
APP
λ
λλ
We can put λ1, λ2, etc. at any location on J matrix. But, when we form the P matrix, the columns of P matrix must correspond to the respective location of λ in J matrix.
27
Example: Let A be a (5 x 5) matrix having the following eigenvalues and the
corresponding number of linearly independent eigenvectors:
λ1 = 2, p = 4 with only 2 linearly independent eigenvectors.
λ1 = 3, p = 1 with 1 linearly independent eigenvector.
3000002000012000002000012
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
=J
Alternatively,
3000002000012000012000002
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
=J
• How do we form the non-singular matrix P?
Example: Consider the following matrix
P2(λ) = λ2 – (tr A) λ + det(A) = λ2 – 4λ + 4 = 0 ⇒ λ = 2, 2
Or,
A P P J
Numbers of distinct blocks are equal to the number of
independent eigenvectors. Note that
Same eigenvalue is present in two different blocks.
3111
⎥⎦
⎤⎢⎣
⎡−
=A
2012
011
⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡==−
λλ
JAPP
2012
3111
2221
1211
2221
1211⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡− pp
pppppp
28
⇒ p11 + p21 = 2 p11 ⇒ p11 = p21
⇒ - p11 + 3 p21 = 2 p21 ⇒ p11 = p21
⇒ p12 + p22 = p11 +2 p12 ⇒ p22 = p11 + p12
⇒ - p12 +3 p22 = p21 +2 p22 ⇒ p12 = p22 - p21
We have only 2 linearly independent equations for the 4 unknowns.
Let p11 = 1, and p12 = 0
Then, p21 = 1 and p22 = 1.
Therefore,
and
• Example: Consider the following matrix
P3(λ) = - λ3 + 6 λ2 + 0 λ - 32 = 0 ⇒ λ = -2, 4, 4
• For λ = -2:
181801818018180
321321
347
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡ −−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−=λ− adjIAadj )(
1101
1101 1
⎥⎦
⎤⎢⎣
⎡−
=⎥⎦
⎤⎢⎣
⎡= −PandP
2012
1101
3111
11011
⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡−⎥
⎦
⎤⎢⎣
⎡−
=− APP
121301
345
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−=A
We can take any arbitrary
multiple α
Dimension of null space = 2
29
Therefore, eigenvector, v1,
11
1 2 1
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−α=−=λ )( forv
• For λ = 4:
Therefore, eigenvector, v2,
We have only 2 linearly independent eigenvectors. Therefore, we need to
form Jordon-Canonical block and we need to find one generalized
eigenvector, p.
A [ v1 v2 p] = [v1 v2 p] J3(λ)
⇒ A v1 = -2 v1 or, (A + 2 I) v1 = 0
⇒ A v2 = 4 v2 or, (A - 4 I) v2 = 0
066066066
321341
341 )(
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−−−=− adjIAadj λ
11
1 )4 (2
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−== αλforv
400140002
1111
11
1111
11
121301
345 ,
3
2
1
3
2
1
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−
ppp
ppp
or
[ ] [ ] 400140002
, 2121⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−= pvvpvvAor
λ = - 2
λ = 4
30
⇒ A p = v2 + 4 p
Or, (A – 4I) p = v2
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−=⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−−−
11
1
321341
341
3
2
1
ppp
⇒ p1 = 1, p2 = 0, p3 = 0
Therefore, the matrix P is
and, P-1 A P = J3(λ)
• Whenever we have repeated eigenvalues with geometric multiplicity less than
algebraic multiplicity, then column vectors of the P matrix contains
generalized eigenvectors in addition to the normal eigenvectors.
Generalized eigenvectors can be obtained from
(A – λjI) pi = pi -1 ; for i = 2, 3, … with p1 = Vj
where, Vj is the linearly independent eigenvector obtained from λj.
Generalized eigenvector
(A - 4I) v2
022110110
011011111
211
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−
−−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−= −PandP
31
• Example:
λ1 = -2, m = 1 with 1 linearly independent eigenvector.
λ2 = 1, m = 1 with 1 linearly independent eigenvector.
λ3 = 3, m = 3 with only 1 linearly independent eigenvector.
where, m denotes algebraic multiplicity.
Then, we can form matrix P as follow
P = [v1 v2 v3 p1 p2]
λ1 λ2 λ3
p1 and p2 can be obtained as follows:
(A – λ3I) p1 = v3
and (A – λ3I) p2 = p1
and the Jordon-Canonical block is given by
and we have P-1 A P = J5(λ).
Example:
⎥⎥⎦
⎤
⎢⎢⎣
⎡
−=
3111
A
f (λ) = λ2 - 4 λ + 4 = 0 ⇒ λ = 2, 2
[A - λ I] = [A - 2 I] = ⎥⎥⎦
⎤
⎢⎢⎣
⎡
−−
1111
Number of distinct blocks must be
equal to the number of independent eigenvectors.
3000013000013000001000002
)( 5
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡−
=λJ
32
Adj [A - 2 I] = ⎥⎥⎦
⎤
⎢⎢⎣
⎡
−−
1111
Therefore, v1 = α ⎥⎥⎦
⎤
⎢⎢⎣
⎡
11
Therefore, we have just 1 linearly independent eigenvector. The generalized
eigenvector p can be obtained from
(A - λ I) p = v1
or, ⎥⎥⎦
⎤
⎢⎢⎣
⎡=
⎥⎥⎦
⎤
⎢⎢⎣
⎡
⎥⎥⎦
⎤
⎢⎢⎣
⎡
−−
11
1111
2
1pp
⇒ - p1 + p2 = 1
⇒ - p1 + p2 = 1
If we let p1 = 0, then p2 = 1.
or, ⎥⎥⎦
⎤
⎢⎢⎣
⎡=
10
αp
Therefore, matrix P is given by
P = ⎥⎥⎦
⎤
⎢⎢⎣
⎡
1101
And, P-1 A P = =⎥⎥⎦
⎤
⎢⎢⎣
⎡=
⎥⎥⎦
⎤
⎢⎢⎣
⎡
⎥⎥⎦
⎤
⎢⎢⎣
⎡
−⎥⎥⎦
⎤
⎢⎢⎣
⎡
− 2012
1101
3111
1101
J
v1 was obtained from (A - λI) v1 = 0
v1 obtained from (A - λI) v1 = 0
p obtained from (A - λI) p = v1
33
Example: Let A be a (5 x 5) matrix having the following eigenvalues and
the corresponding number of linearly independent eigenvectors:
λ1 = -3, p = 4 with only 2 linearly independent eigenvectors.
λ1 = 2, p = 1 with 1 linearly independent eigenvector.
There many ways we can from the Jordon-canonical block, J, for this
example. However, we must form the corresponding P matrix as per we
write our J matrix
Arrangement #1: If we form the J matrix as follows
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
−−
−−
=
3000013000013000003000002
J
Form P matrix as follows
P = [v1 v2 v3 p1 p2]
Where,
v1 is obtained from (A - 2 I) v1 = 0.
v2 is obtained from (A + 3 I) v2 = 0.
v3 is obtained from (A + 3 I) v3 = 0.
p1 is obtained from (A + 3 I) p1 = v3.
p2 is obtained from (A + 3 I) p2 = p1.
3 distinct blocks for 3 linearly independent
eigenvectors
From problem statement 2 linearly independent
vectors can be obtained from
(A+ 3 I) v2 = 0 and (A+ 3 I) v3 = 0
Null space dimension must be equal to 2.
34
Arrangement #2: If we form the J matrix as follows
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
−−
−−
=
2000003000013000003000013
J
Form P matrix as follows
P = [v1 p1 v2 p2 v3]
Where,
v1 is obtained from (A + 3 I) v1 = 0.
p1 is obtained from (A + 3 I) p1 = v1.
v2 is obtained from (A + 3 I) v2 = 0.
p1 is obtained from (A + 3 I) p2 = v2.
v3 is obtained from (A - 2 I) v3 = 0.
Arrangement #3: If we form the J matrix as follows
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
−−
−−
=
3000013000002000003000013
J
3 distinct blocks for 3 linearly independent
eigenvectors
From problem statement 2 linearly independent
vectors can be obtained from
(A+ 3 I) v1 = 0 and (A+ 3 I) v2 = 0
Null space dimension must be equal to 2.
3 distinct blocks for 3 linearly independent
eigenvectors
35
Form P matrix as follows
P = [v1 p1 v2 v3 p2]
Where,
v1 is obtained from (A + 3 I) v1 = 0.
p1 is obtained from (A + 3 I) p1 = v1.
v2 is obtained from (A - 2 I) v2 = 0.
v3 is obtained from (A + 3 I) v3 = 0.
p2 is obtained from (A + 3 I) p2 = v3.
In each case if we form the J and P matrix as above we would get
P-1 A P = J.
Summary
• One can find generalized eigenvector, p, which, of course, is linearly
independent to all other eigenvectors vi, from
(A - λiI) p = vi.
• One can then form a (n × n) P matrix such that each column of P matrix
contains the linearly independent eigenvectors that are possible to be
obtained plus the generalized eigenvectors.
Then we can have
P-1 A P = J
where J is called Jordon-Canonical block which is a nearly diagonal
matrix whose diagonal elements are the n eigenvalues.
From problem statement 2 linearly independent
vectors can be obtained from
(A+ 3 I) v1 = 0 and (A+ 3 I) v3 = 0
Null space dimension must be equal to 2.
36
♦ Linear Independence of eigenvectors A set of vectors {xj} is said to be linearly independent if the equation
0.....2211 =+++ nn xcxcxc
where {cj} are constants, is satisfied only when c1 = c2 = . . . . = cn = 0.
When A has distinct eigenvalues, λj, (i.e., no repeated eigenvalues), then the
set of eigenvectors {vj} forms a linearly independent set.
Let vi and vj be eigenvectors that belongs to λ1 and λj, respectively. Then vi and
vj are not constant multiples of each other, because if they were, then
vi = k vj (a)
where k is a nonzero constant. If we pre-multiply (a) by A, then
Avi = k Avj (b)
which is equivalent to
λi vi = k λj vj (c)
If we now multiply both sides of (a) by λi, then we have
λi vi = k λi vj (d)
Subtracting (c) from (d), we have
k (λi - λj) vj = 0
Since k ≠ 0, and, λi ≠ λj, this means that vj = 0, which contrary to the assumption
that vj is an eigenvector.
Therefore, for different eigenvalues, the corresponding eigenvectors are
linearly independent.
37
♦ Bi-orthogonality property of eigenvectors
Consider two eigenvalue problems
A v = λ v (1)
and AT w = η w (2)
Since, the condition that equation (2) has nontrivial solution is
det (AT - ηI) = 0
or,
0
21
22212
12111
=
−••••••••••••
••−••−
η
ηη
nnnn
n
n
aaa
aaaaaa
Since, the value of determinant remains unchanged when its rows and columns
are interchanged, we have, λ = η, and (2) becomes
AT w = λ w (2a)
Therefore, (1) and (2a) have the same eigenvalues, λi = 1, 2, ...., n, but have
different set of eigenvectors. vj of (1) is one of the non-zero columns of
adj [A - λj I], while wj of (2a) is one of the non-zero columns of adj [AT - λj I]. It
will be same only when A is symmetric, i.e., A = AT.
However, columns of adj [AT - λj I] are rows of adj [A - λjI]. Therefore, wj is
simply non-zero rows of adj [A - λjI]. Hence, both eigenvectors, vj and wj,
can be obtained from the same calculation, adj [A - λj I].
As shown earlier, wj, can also be shown to be linearly independent like vj.
38
However, apart from being linearly independent, the vectors, vj and wj, satisfy
the bi-orthogonality property. This means that each member of one set is
orthogonal to each member of the other set, except for the one with which it has
a common eigenvalue.
By definition, A vj = λj vj (3)
and wiT A = λi wi
T, i ≠ j (4)
Pre-multiplying (3) by wiT, and post-multiplying (4) by vj, we have
wiT A vj = λj wi
T vj
wiT A vj = λi wi
T vj
Subtracting the above two equations, we have
(λi – λj) wiT vj = 0
Since, we assumed that λi ≠ λj, we have
wiT vj = <wi, vj> = 0 (5)
That is, wi and vj and are orthogonal to each other for i ≠ j.
When matrix A is symmetric, A = AT, and therefore,
adj [A – λiI] = adj [AT – λiI]
and vi = wi
Hence, equation (4) becomes
viT vj = <vi, vj> = 0 for i ≠ j.
Thus, we have an orthogonal set of vectors {vi}, that is, each member of the set
is orthogonal to every other member of the set.
39
Example:
Let A = ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−
−
320321001
f (λ) = - λ3 - 6 λ2 - 5 λ = 0 ⇒ λ = 0, -1, -5
For λ = 0:
adj [A - λI] = ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−
−
222333000
320321001
adj
For λ = -1:
adj [A - λI] = ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−
002002004
220311000
adj
For λ = -5:
adj [A - λI] = ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
12821282000
220331004
adj
The eigenvectors of A is given by the non-zero columns of adj [A - λiI] and
are denoted by vi, whereas the eigenvectors of AT is given by the non-zero
rows of adj [A - λiI] and are denoted by wi.
v1
w1
v2
w3
40
Therefore,
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=230
1v ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=111
1w
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−=
112
2v ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
001
2w
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−=11
0
3v ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−=64
1
3w
It can be easily observed that
<v1, w2> = <v1, w3> = <v2, w1> = <v2, w3> = <v3, w1> = <v3, w2> = 0
i.e., <vi, wj> = 0 for i ≠ j
or the vectors vi and wj are bi-orthogonal.
41
♦ Expansion of an arbitrary vector
We have seen that each square matrix A generates two sets of linearly
independent vectors via eigenvectors associated with the eigenvalue problems
A v = λ v and AT w = λ w
The two sets of eigenvectors, {vi} and {wi}, corresponding to eigenvalues, {λi}, i
= 1, 2, . . . . . . . . , n, are nonzero columns and rows, respectively, of adj (A – λi
I). In addition, the eigenvectors, {vi} and {wi}, satisfy the bi-orthogonality
criteria
<wj, vi > = 0, i ≠ j
<wj, vi > ≠ 0, i = j
We can now write any arbitrary vector z as
z = c1 v1 + c2 v2 + c3 v3 + . . . . . + cn vn = in
iivc∑
=1
where ci's are constant to be determined. If we take an inner product with wj, we
have
< wj, z> = < wj, in
iivc∑
=1> = cj <wj, vj>
or, ><
><=
jvjwzjw
jc ,,
and the arbitrary vector can be written as
z = ∑=∑= ><
><
=
n
jjjvjw
zjwj
n
jj vvc
1 ,,
1
If A is symmetric, then wj = vj, and
z = ∑=
><
><n
jjjvjv
zjvv
1,,
<wj, vi> = 0 except when i = j
42
♦ Application eigenvalues-eigenvectors in solving a system of
linear algebraic equations
Consider the following set of n-linear algebraic equations
A x = b (1)
Where A is a non-singular matrix (det A ≠ 0) with simple eigenvalues, b is
known vector of constants, and x is a vector of n-unknowns.
We can express any arbitrary vector x and b in terms of eigenvectors of A as
x = in
iiv∑
=1α (2)
and b = in
iiv∑
=1β (3)
where the eigenvalues {λi} and eigenvectors {vi} are obtained from
A v = λ v (4)
βi's can be determined from
><><=
iviwbiw
i ,,β , i = 1, 2, . . . . , n (5)
where {vi} and {wi} are eigenvectors, nonzero columns and rows respectively of
adj (A – λi I), corresponding to eigenvalues, λi.
Substituting (2) and (3) in (1), and using (4) we have
A in
iiii
n
iii
n
ii vvv ∑∑∑
=====
111βλαα
Rearranging, we have
[ ] 0 11
==− ∑∑==
n
iiii
n
iiii vcvβλα
43
However, the set of eigenvectors {vi} is linearly independent, therefore, the set
of constants {ci}, are equal to zero for all i. That is,
αi λi = βi, i = 1, 2, . . . . , n.
or, the unknown constants, αi, is given by
ii
i λβα = , i = 1, 2, . . . . , n. (6)
and the solution of A x = b is given by
x = iiin
iii vv
n
1i1∑ ⎥⎦
⎤⎢⎣⎡
∑ === λ
βα (7)
where βi's are obtained from (5), and λi and vi are eigenvalues and
corresponding eigenvectors of A.
Note that no solution exist for x from (7) when any λi = 0. This is expected
because if any of the eigenvalues of A is zero, then det A must be equal to zero,
i.e., A is singular, and we know that no solutions exist.
Moreover, if λ = 0 is an eigenvalue, then we have
A v = λ v = 0 [since λ = 0] (8)
Where v is the eigenvector corresponding to the zero eigenvalue. Since, non-
trivial solution of (8) can occur only if det A = 0, we must have v = 0, which
contrary to the definition of eigenvector. Therefore, if det A ≠ 0, λ = 0 can not
be an eigenvalue of A.
Example: Solve for x for the following Ax = b problem
Where, A = ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−301030103
and b = ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−22
0
The eigenvalues are 2, -3, 4 and the eigenvectors are
44
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−=
101
010
,1
01
321 vandvv
Note that the matrix is symmetric, therefore,
• vi = wi
• < vi, wj> = 0 for i ≠ j.
Now, b = ∑=
n
iii v
1β
βi can be calculated either from
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−=⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
+⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
+⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
− 22
0
101
010
1
01
321 βββ
or from
βi = ><><
ivivbiv
,,
Therefore,
β1 = 122
1,1,1 −== −
><><
vvbv
β2 = 212
2,2,2 −== −
><><
vvbv
β3 = 122
3,3,3 ==
><><
vvbv
The solution x is given by
333
222
111 vvvx ⎥⎦
⎤⎢⎣⎡+⎥⎦
⎤⎢⎣⎡+⎥⎦
⎤⎢⎣⎡= λ
βλβ
λβ
or, ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
+⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
+⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−
43
32
41
41
32
21
3
2
1
101
010
101
xxx
λ1 = 2 λ2 = -3 λ3 = 4
3 Equations 3 Unknowns
Equation (5)
Equation (7)
45
Example: Solve for x for the following Ax = b problem
Where, A = ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−−−
222214241
and b = ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
633
The eigenvalues are -3, -3, 6 and the eigenvectors are
and ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡ +−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−=
βα
βα 50
122
321
.vv,v
By arbitrarily selecting α = 1, β = 0, one can obtain ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−=
011
2v .
By arbitrarily assigning any other values for α and β, one can obtain another
eigenvector. However, we want the eigenvectors to be orthogonal. Therefore,
select α and β such that
[ ] βαβα
βα2500
5001132 .
.v,v =⇒=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡ +−•−=><
Therefore, if we arbitrarily select α = 1, we obtain ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
411
3v .
Now, b = ∑=
n
iii
1vβ
βi can be calculated from
βi = ><
><
iii
v,vb,v
Therefore,
λ1 = 6 λ2, λ3 = -3, -3
46
β1 = 211
1 =><
><v,vb,v
β2 = 022
2 =><
><v,vb,v
β3 = 133
3 −=><
><v,vb,v
The solution x is given by
333
222
111 vvv ⎥⎦
⎤⎢⎣⎡+⎥⎦
⎤⎢⎣⎡+⎥⎦
⎤⎢⎣⎡= λ
βλβ
λβx
or, ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡+
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−+
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−
−111
411
011
122
31
30
62
3
2
1
xxx
------------
Application of Eigenvalues and Eigenvectors in Solving Linear Ordinary
Differential Equations of Initial Value Problems (ODE-IVPs)
We now return to complete the solution of the first-order, linear, ODE-IVPs
with constant coefficients:
bAxx +=dtd ; x(0) = x0
The complete solution was obtained as
x = tiei
n
i icλ
v∑=1
- A-1 b
with the ci obtained from
x0 = ∑=
n
i iic1
v - A-1 b
We take the inner product with wj, to get
< wj, ∑=
n
i iic1
v > = < wj, x0> + < wj, A-1 b>
The only nonzero term in the LHS of this equation is when i = j, and, therefore,
><>−<+><
=j
jojjc vw
bAwxw,j
1, ,, j = 1, 2, . . . , n
47
The final, complete solution is then given by
bAvvwbAwxw
x 1 n
1j ,j
1, , −−∑
= ⎥⎥
⎦
⎤
⎢⎢
⎣
⎡><
>−<+><=
tjejjjoj λ
If A is symmetric, then wj = vj, and the above equation reduces to
bAvvvbAvxv
x 1 n
1j ,j
1, , −−∑
= ⎥⎥
⎦
⎤
⎢⎢
⎣
⎡><
>−<+><=
tjejjjoj λ
Example:
10121 == )(y ydt
dy
10510011000 2212 −=+−−= )(y yydt
dy
or,
⎥⎦
⎤⎢⎣
⎡−
=⎥⎦
⎤⎢⎣
⎡+⎥
⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡−−
=⎥⎦
⎤⎢⎣
⎡1
10
50
1001100010
2
1
2
1 )(y yy
yy
dtd
or,
00 y)y(bAyy =+= dtd
where
⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡−−
=50
1001100010
bA and
f(λ) = λ2 - (tr A) λ + det A = λ2 + 1001λ + 1000 = 0 ⇒ λ = -1, - 1000
For λ = -1, adj [A + I] = adj ⎥⎦
⎤⎢⎣
⎡ −−=⎥
⎦
⎤⎢⎣
⎡−− 11000
110001000100011
⇒ v1 = ⎥⎦
⎤⎢⎣
⎡−11
α , w1 = ⎥⎦
⎤⎢⎣
⎡1
1000 α
48
For λ = - 1000, adj [A + 1000I] = adj ⎥⎦
⎤⎢⎣
⎡ −−=⎥
⎦
⎤⎢⎣
⎡−− 10001000
1111000
11000
⇒ v2 = ⎥⎦
⎤⎢⎣
⎡ −1000
1 β , w2 = ⎥
⎦
⎤⎢⎣
⎡11
α
The general solution of
00 y)y(bAyy =+= dtd
is given by
bAvvbAvy 1222
111
1
1
−−
=−+=−= ∑ ttn
i
tiii ececec λλλ
or, ⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡ −−−⎥
⎦
⎤⎢⎣
⎡ −+⎥
⎦
⎤⎢⎣
⎡−= −−
50
0100011001
10001
11
100011000
21 e ce cy tt
or, ⎥⎦
⎤⎢⎣
⎡+
⎥⎥⎦
⎤
⎢⎢⎣
⎡ −+⎥⎥⎦
⎤
⎢⎢⎣
⎡−= −−
−
00050
1000 1000
1000
21.
t
t
t
t
e e c
ee cy
Apply initial conditions, at t = 0, y(0) = y0 = ⎥⎦
⎤⎢⎣
⎡−11
i.e., ⎥⎦
⎤⎢⎣
⎡+⎥
⎦
⎤⎢⎣
⎡ −+⎥
⎦
⎤⎢⎣
⎡−=⎥
⎦
⎤⎢⎣
⎡− 0
00501000
111
11
21.
c c
⇒ 1 = - c1 - c2 + 0.005
⇒ -1 = c1 + 1000 c2
or, ⎥⎦
⎤⎢⎣
⎡−
=⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡ −−1
995010001
11
2
1 .cc
or, ⎥⎥⎦
⎤
⎢⎢⎣
⎡
−
−=⎥
⎦
⎤⎢⎣
⎡−⎥
⎦
⎤⎢⎣
⎡−−
=⎥⎦
⎤⎢⎣
⎡−
9990050
999994
9991
2
1
19950
1111000
..
cc
49
The constants, ci, can also be determined from
[ ] [ ]
[ ] 999994
11
11000
00050
110001
111000
1
1111 −=
−
−+
−=><
>−<+><=
⎥⎦⎤
⎢⎣⎡
⎥⎦⎤
⎢⎣⎡
⎥⎦⎤
⎢⎣⎡
1 oc
.
v,wbA,wy,w
[ ] [ ]
[ ] 9990050
10001
11
00050
111
111
2
1222
.
.
v,wbA,wy,w −=
−
−+
−=><
>−<+><=
⎥⎦⎤
⎢⎣⎡
⎥⎦⎤
⎢⎣⎡
⎥⎦⎤
⎢⎣⎡
1 oc
Therefore, ⎥⎦
⎤⎢⎣
⎡+
⎥⎥⎦
⎤
⎢⎢⎣
⎡ −−⎥⎥⎦
⎤
⎢⎢⎣
⎡−−=⎥⎦
⎤⎢⎣
⎡−
−
−
−
00050
1000 1000
1000
9990050
999994
2
1 ..t
t
t
t
e e
ee
yy
------------------
We now discuss the solution of the first-order, linear, ODE-IVPs with constant
coefficients using similarity transformation
ODE-IVPs of the form dy/dt = Ay
Consider a single linear differential equation of the type
yy ay o == )(; 0dtdy
where a is a constant. The solution of the above equation can be easily obtained:
dt a ty
oy ydy
∫∫ =0
or, oyatey =
Consider, now, a system of linear ODE-IVPs with constant coefficients:
oyy nyna yayadtdy
10112121111 =+++= )(;....
oyy nyna yayadtdy
20222221212 =+++= )(;....
50
•
•
noyny nynna ynaynadtndy =+++= )(;.... 02211
The above set of equations can be written in matrix form as
00 y)y(;Ayy == dtd
where
21
)0( ;
21
2222111211
; 21
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
•=
⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢
⎣
⎡
•••••
••
=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
•=
noy
oyoy
nnanana
naaanaaa
ny
yy
yAy
Using analogy, we can write the solution of this equation as
ote yAy =
We now show that above is, indeed, a solution. We have
. . . .22t2
tte +++= AAIA
(this is similar to e atat a t= + + +12 22 . . . . ). We have
[ ] o. . 22t2tdt
dteodtd
dtd yAAIAyy
⎥⎦⎤
⎢⎣⎡ +++==
. . 222= . . . 2
232 , ottottdtdor yAAIAyAAAy
⎥⎦⎤
⎢⎣⎡ +++⎥⎦
⎤⎢⎣⎡ +++=
[ ] o te AyyAA ==
Alternatively, we can simplify the equation, Ayy =dtd , by using matrices, U or
P. We have seen earlier that if we have n linearly independent eigenvectors for a
matrix, A, then
U-1 A U = D
51
where D is a diagonal matrix containing the eigenvalues of A. On the other
hand, if we do not have n linearly independent eigenvectors for A, we can use
P-1 A P = J
The columns of U contain the linearly independent eigenvectors of A while the
columns of P contain all the linearly independent eigenvectors possible to be
obtained from A, plus some generalized eigenvectors.
Therefore,
..AAIA . .2ttte22
+++=
. . . 2t t 21211 +++ −−−= UD UU D UU I U
122 . . . 2t t −
⎥⎦⎤
⎢⎣⎡ +++= U DDIU
1te −= U DU
where,
tnλe00
0t2λe000t1λe
te
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
•••••
••
=D
Therefore, the solution
o(0) ; dtd yyAyy ==
can be written in terms of the eigenvalues and the corresponding eigenvectors of
A as
o te yAy =
[ ] o te 1 yUDU −=
The above can also be derived as follows:
52
We define y ≡ Uz. This satisfies dtd
dtd zUy = , since the elements of U are
constants. Hence, we have
o(0) ; dtd yyAyy ==
or,
o=(0) ; dtd yUzUz AzU =
We pre-multiply this equation by U-1 to get
o=(0) ; dtd 1-1-1-1 yUUzUUz AUzUU =-
or, o=(0) ; dtd zzDzz =
where, zo ≡ U-1 yo.
These equations can be represented by
z(0)z ; zλdtdz 1o111
1 ==
z(0)z ; zλdtdz
2o2222 ==
•
•
noz(0)nz ; nznλdtndz
==
Since, dtidz
depends only on zi, this set of equations is uncoupled and each of
these can be solved individually (instead of simultaneously).The solution of this
system is
t1λe 10z(t)1z =
t2λe 20z(t)2z =
•
53
•
tnλe n0z(t)nz =
or,
noz
2oz1oz
tnλe00
0t2λe0
00t1λe
(t)nz
(t)2z(t)1z
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
•⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢
⎣
⎡
•••••
•
•
=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
•
or,
z(t) = exp(Dt) zo
But,
y(t) = U z(t)
So
[ ]
(t)nz
(t)2z(t)1z
n21
(t)ny
(t)2y(t)1y
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
••=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
•uuu
or,
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
•(t)ny
(t)2y(t)1y
[ ] [ ]
noy
2oy1oy
1-
tnλe00
0t2λe0
00t1λe
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
•⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢
⎣
⎡
•••••
•
•
= UU
Therefore, solution of coupled linear ODE-IVPs can be obtained by using
similarity transformation (U matrix) to decouple the ODE-IVPs as given above.
54
Still further, note that since
U-1 A U = D
We have,
A = U D U-1
Or, A2 = (U D U-1) (U D U-1) = U D2 U-1
And therefore, An = U Dn U-1
Example: Consider the following ODE-IVPs
0)0(1 ; 3131 =+= yyydtdy
2)0(2 ; 232 −=−= yydtdy
2)0(3 ; 3313 =+= yyydt
dy
or, (0) ; dtd oyyAyy ==
where
22
0
o ;
3y2y1y
; 301030103
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−= y yA
The eigenvalues and eigenvectors of A can easily be obtained as:
λi = 4, -3, 2
The U matrix containing the corresponding eigenvectors is given by
101
010101
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−=U
λ2 = -3
λ1 = 4 λ3 = 2
55
Since A is symmetric, its eigenvectors are orthogonal to each other. If we
normalize the eigenvectors, then UT = U-1. Normalization of the eigenvectors
gives
2102
1010
2102
1
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
−=U
Then,
U-1 = UT = 0
0100
21
21
21
21
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−
⇒ U-1 yo =
2102
1010
2102
1
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
−
222
22
0
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−
Therefore, y(t) is given by
y(t) = U [eDt] U-1 yo
or
2
22
00
0000
0
0100
)()()(
2
3
4
21
21
21
21
3
2
1
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−
t
t
t
ee
e
tytyty
or,
ee
eee
tytyty
tt
t
tt
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
+−
−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡−
24
3
24
3
2
12
)()()(
• Note that the above differential equation has eigenvalues of 4, -3, and 2.
• These appear in the exponential terms in the final solution and control the
steady state behavior.
56
• For large value of t (i.e., near steady state – most of the time we are
interested in such steady state solutions only), y(t) approaches e4tu1, where u1
is the eigenvector corresponding to the dominant eigenvalue, λl, and blows
up (tends to infinity).
• This is why it is often important to know only the dominant eigenvalue of A
and its associated eigenvector.
• The steady state solution is stable if all the eigenvalues have negative real
parts. Otherwise, the steady state solution is unstable in the sense that it
approaches infinity as time, t, approaches infinity.
• Also, it is to be noted here that the most rapidly decreasing solution
corresponds to the eigenvalue, -3, while the most rapidly increasing solution
corresponds to the eigenvalue, 4.
• The problem (ODE-IVP) is said to be stiff if it has eigenvalues such that one
has a very large magnitude, while another has a very small magnitude, i.e.,
⏐λl ⏐>> ⏐λn⏐. This is represented mathematically in terms of a stiffness
ratio, (⏐λmax ⏐/ ⏐λmin⏐). Large values of this ratio lead to computational
problems, as discussed later when numerical solution of ODE-IVPs are
discussed.
- - - - - -
Example: Consider the following reaction network in an isothermal batch
reactor
A B C
D
with CA(0) = CA0, CB(0) = CB0, CC(0) = CC0, CD(0) = CD0. The corresponding
model equations (ODE-IVPs) are:
k1 k3
k4k2
57
dCAdt Ak k C= − +( )1 2
dCBdt A B Ck C k C k C= − +1 3 4
dCCdt B Ck C k C= 3 4 -
dCDdt Ak C= 2
or,
ddt
ABCD
ABCD
CCCC
k kk k k
k kk
CCCC
⎡
⎣
⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥
=
− +−
−
⎡
⎣
⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥
⎡
⎣
⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥
( )1 21 3 4
3 42
0 0 00
0 00 0 0
or,
o(0) ; dtd yyKyy ==
The rank of matrix, K, is always less than n (n = 4, in this example) because of
the law of conservation of mass (mass cannot be created or destroyed) must be
satisfied, i.e.,
dCA
dtdCB
dtdCC
dtdCD
dt+ + + = 0
One of the above equations, thus, must be a linear combination of the others.
Also, since, the determinant of K is zero, one of its eigenvalues must be zero.
We leave the further solution of this problem as an exercise.
- - - - - - - -
58
Example: Consider the following matrix
⎥⎦
⎤⎢⎣
⎡=
2112
A
(i) Determine U and D matrix such that U-1 A U = D.
(ii) Using above information, solve the equation
00 x)x(x;Ax ==−= t dtd
i.e., ⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡==
⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡−=⎥
⎦
⎤⎢⎣
⎡01
00
2112
21
21
21
)()(
;txtx
xx
xx
dtd
The eigenvalues of A is given by f(λ) = λ2 - 4λ + 3 = 0 ⇒ λ = 1, 3
The eigenvector for λ = 1:
⎥⎦
⎤⎢⎣
⎡−
−=⎥
⎦
⎤⎢⎣
⎡=−=
1111
1111
adjadj I][Av λ ⇒ ⎥⎦
⎤⎢⎣
⎡−
=1
1αv
The eigenvector for λ = 3:
⎥⎦
⎤⎢⎣
⎡−−−−
=⎥⎦
⎤⎢⎣
⎡−
−=−=
1111
1111
adjadj I][Av λ ⇒ ⎥⎦
⎤⎢⎣
⎡=
11
αv
Therefore, ⎥⎦
⎤⎢⎣
⎡−
=⎥⎦
⎤⎢⎣
⎡=
1111
3001
UD and
Since, A is symmetric, normalize U matrix: ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−=
21
21
21
21
U
Then, U-1 = UT = ⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡ −
21
21
21
21
59
Now, let x = Uy, then
yAx yx D dtd
dtd −=⇒−= , y(0) = U-1 x(0)
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=⎥
⎦
⎤⎢⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡ −=⎥
⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡−=⎥
⎦
⎤⎢⎣
⎡
212
1
21
21
21
21
21
21
21
01
00
3001
yy
yy
yy
dtd
)()(
;
Solve dy1/dt = -y1, y1(0) =
21
⇒ y1 = C exp[-t]
Substituting initial conditions, we have
21 = C
Therefore, y1 = 2
1 exp[-t]
Solve dy2/dt = -3 y2 , y2(0) = 2
1
⇒ y2 = C exp[-3t]
Substituting initial conditions, we have
21 = C
Therefore, y2 = 2
1 exp[-3t]
Substituting back, we have
[ ]
[ ]⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
+−
+=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−==⎥
⎦
⎤⎢⎣
⎡−−
−−
−
−
tt
tt
t
t
ee
ee
e
e
xx
321
321
32
12
1
21
21
21
21
21 Uy
-------
The above example is when we have distinct eigenvalues and n-linearly
independent eigenvectors such that we can form U matrix.
60
However, when eigenvalues are repetitive and we need to use generalized
eigenvector(s) to form P matrix, coupled ODE-IVPs can not be completely de-
coupled but it is possible to nearly de-couple as shown in the next example.
Example: Consider the following matrix
⎥⎦
⎤⎢⎣
⎡ −=
3111
A
(i) Determine P and J matrix such that P-1 A P = J.
(ii) Using above information, solve the equation
00 x)x(Ax;x ==−= t dtd
i.e., ⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡==
⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡ −−=⎥
⎦
⎤⎢⎣
⎡10
00
3111
2
1
2
1
2
1)()(
;txtx
xx
xx
dtd
The eigenvalues are given by f(λ) = λ2 - 4λ + 4 = 0 ⇒ λ = 2, 2
The eigenvector for λ = 2:
⎥⎦
⎤⎢⎣
⎡−−
=⎥⎦
⎤⎢⎣
⎡ −−=−=
1111
1111
adjIAadj ][v λ ⇒ ⎥⎦
⎤⎢⎣
⎡−
=1
1αv
Determine generalized eigenvector from [A - λI] p = v
⇒ ⎥⎦
⎤⎢⎣
⎡−
α=⇒⎥⎦
⎤⎢⎣
⎡−
=⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡ −−1
01
1
1111
2
1 ppp
Therefore, ⎥⎦
⎤⎢⎣
⎡−−
=⎥⎦
⎤⎢⎣
⎡−−
=⎥⎦
⎤⎢⎣
⎡= −
1101
1101
2012 1P;P;J
Now, let x = Py, then
JyAx yx −=⇒−=dtd
dtd , y(0) = P-1 x(0)
61
⎥⎥⎦
⎤
⎢⎢⎣
⎡
−=
⎥⎥⎦
⎤
⎢⎢⎣
⎡
⎥⎥⎦
⎤
⎢⎢⎣
⎡
−−=
⎥⎥⎦
⎤
⎢⎢⎣
⎡
⎥⎥⎦
⎤
⎢⎢⎣
⎡
⎥⎥⎦
⎤
⎢⎢⎣
⎡−=
⎥⎥⎦
⎤
⎢⎢⎣
⎡
10
10
11
01)0()0(
; 2012
2
1
2
1
2
1yy
yy
yy
dtd
Solve first dy2/dt = -2 y2 , y2(0) = -1
⇒ y2 = C exp[-2t]
Note that the second equation is function of y2 only.
Substituting initial conditions, we have -1 = C
Therefore, y2 = - exp[-2t]
Solve now dy1/dt = -2 y1 - y2, y1(0) = 0
= -2 y1 + exp[-2t]
⇒ y1 = C exp[-2t] + t exp[-2t]
Note that first equation is now function of y1 only as solution of y2 can be
substituted.
Substituting initial conditions, we have 0 = C
Therefore, y1 = t exp[-2t]
⎥⎥⎦
⎤
⎢⎢⎣
⎡
+−=
⎥⎥⎦
⎤
⎢⎢⎣
⎡
−⎥⎦
⎤⎢⎣
⎡−−
==⎥⎦
⎤⎢⎣
⎡−−
−
−
−
ttt
tt
etete
ete
xx
222
22
21
1101
Py
62
ODE-IVPs of the form dy/dt = Ay, with all eigenvalues of A being equal
We again consider the ODE-IVP:
o(0) ; dtd yyAyy ==
but, with the eigenvalues of A being all equal, say λ. We need to use the P
matrix now. We substitute y = Pz in the above equation to get
Pz AzP =dtd
or,
o(0) ;dtd 11 yPz zJ z APPz −− ===
or,
nz1nz
2z1z
λ0001000
01λ0001λ
nz1nz
2z1z
dtd
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
−
•
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
••
•••••••
=
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
−
•
This can be expanded to
2z1z λdt1dz
+=
3z2z λdt2dz
+=
•
•
nz1nz λdt1ndz
+−=−
nz λdtndz
=
The solution of the above set of ODE-IVPs is given by
zn = zn(0) eλt
zn-1 = zn-1(0) eλt + zn(0) t eλt
63
zn-2 = zn-2(0) eλt + zn-1(0) t eλt + zn(0) (t2/2) eλt
•
•
z1 = z1(0) eλt + z2(0) t eλt +. . . + zn(0) [tn-1/(n-1)!] eλt
or, in matrix form, by
(0)nz(0)1nz
(0)2z(0)1z
1000t000
2)!(n2ntt10
1)!(n1nt
22tt1
tλe
nz1nz
2z1z
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
−
•
⎥⎥⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢⎢⎢
⎣
⎡
••
•••••−−
•
−−
•
=
⎥⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢⎢
⎣
⎡
−
•
or,
z = K zo
This leads to
y = P z = P K zo = P K P-1 yo
64
Higher-order IVPs
Let us now consider the following higher-order IVP with constant coefficients:
oy dcybadt
oyddt
odydtdy
dt
yd
dt
yd γβα ====+++ 2
2
2
2
3
3 )()( ,,)(;
We can convert this equation into a set of first order ODE-IVPs using the
procedure described below. We define
y = y1
then
dydt
dydt y= =1
2
d y
dt
dydt y
2
22
3= =
d y
dt
dydt
3
33=
Substituting the above equations in the original equation, we have
1233 dcybyaydt
dy +−−−=
or,
dy
yy
abcy
yy
dtd
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡+
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−−−=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡00
100010
3
2
1
3
2
1
This may be written in matrix form as
γ β α
(0) ; dtd
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=+= yeAyy
65
Note that the vector, e, is a constant (and not a function of time). We can
transform this equation into a homogeneous one by using the concept of the
steady state (SS; represented by subscript, S) solution. We have at steady state:
=dtd Sy 0 = A yS + e
or
yS = - A-1e = [d/c 0 0]T
We now subtract the SS equation from the original one to obtain
)s( dt)sd(
yyAyy
−=−
If we define x ≡ y - yS, we have
o(0) ; dtd xxAxx ==
and the solution is
o te (t) 1 xUDUx −=
or
( ) ( )oso te oso te s(t) 111 yyUDUeAyyUDUyy −+−=−+= −−−
66
ODE-IVPs of the form dy/dt = Ay + B(t)
We now consider the more general equation
dy/dt ≡ y' = Ay + b(t)
where b is a function of time.
Using y = Uz, we obtain
U z' = y' = Ay + b(t) = A U z + b(t)
On pre-multiplying both sides of this equation with U-1, we obtain
U-1 U z' = U-1 A U z + U-1 b(t)
or,
z' = D z + U-1 b(t) ≡ D z + g(t)
Here, g(t) ≡ U-1b(t), and D is a diagonal matrix whose entries are the
eigenvalues, λ1, λ2, . . . , λn, of A, arranged in the same order as the
eigenvectors, u1, u2, . . . , un, (appearing as columns in U).
The above is a system of n-uncoupled ODE-IVPs in z1(t), z2(t), . . . , zn(t), and
may be solved individually. The ith equation may be written as
z'i (t) = dzi/dt = λi zi (t) + gi (t); i = 1, 2, . . . , n
Above has the following solution
ds (s)igt
ot
siλe tiλetiλeic(t)iz ∫−
+= ⎥⎦⎤
⎢⎣⎡
; i = 1, 2, . . . , n
This may be used with y = Uz to give the solution, y(t).
67
Example: Solve y1'' +102 y1' + 200 y1 = t, y1(0) = 1, y1'(0) = - 2
Define y1' = y2 y1(0) = 1
then y1'' = y2' = - 102 y2 - 200 y1 + t y2(0) = - 2
or, ⎥⎦
⎤⎢⎣
⎡−
=⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡+⎥
⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡−−
=⎥⎦
⎤⎢⎣
⎡2
1000
10220010
2
1
2
1
2
1)()(
yy
ty
y
yy
dtd
or, 00 y)y()g(yAy =+= tdt
d
where
⎥⎦
⎤⎢⎣
⎡−
=⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡−−
=⎥⎦
⎤⎢⎣
⎡=
210
10220010
02
1 ygAy ; t
; ; yy
Eigenvalues of A:
f(λ) = λ2 - (tr A) λ + det A = λ2 +102 λ + 200 = 0 ⇒ λ = -2, -100
Eigenvectors of A for λ = -2:
adj [A + 2I] = adj ⎥⎦
⎤⎢⎣
⎡−=⇒⎥
⎦
⎤⎢⎣
⎡ −−=⎥
⎦
⎤⎢⎣
⎡−− 2
122001100
10020012
1 αv
Eigenvectors of A for λ = -100:
adj [A + 100I] = adj ⎥⎦
⎤⎢⎣
⎡ −=⇒⎥
⎦
⎤⎢⎣
⎡ −−=⎥
⎦
⎤⎢⎣
⎡−− 100
1100200
122200
11002 βv
The matrix U, U-1 and D are given by
⎥⎦
⎤⎢⎣
⎡−
−=⎥
⎦
⎤⎢⎣
⎡−−
=⎥⎦
⎤⎢⎣
⎡ −−= −
100002
121100
100211
9811 DUU
so that U-1 A U = D.
68
Substituting y = U z in y' = Ay + g(t), we have y' = U z' = A U z + g(t)
Or, z' = U-1 A U z + U-1 g(t) = D z + U-1 g(t)
Or, ⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡−−
+⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡−
−=⎥
⎦
⎤⎢⎣
⎡− t
zz
zz
dtd 0
121100
100002
981
2
1
2
1
Or, ⎥⎦
⎤⎢⎣
⎡−
−⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡−
−=
tt
zz
981
2
1
100002
Since, y = U z, we have y0 = U z0
or, z0 = U-1 y0
or, z0 = ⎥⎦
⎤⎢⎣
⎡−=⎥
⎦
⎤⎢⎣
⎡−⎥
⎦
⎤⎢⎣
⎡−−
−01
21
121100
981
Therefore, we have
102 19811 −=−−= )(z z t
dtdz
and 00100 29822 =+−= )(z z t
dtdz
or, ⎥⎦
⎤⎢⎣
⎡−= ∫
−− dsseeeczt stt
0
229812
11
or, ⎥⎦⎤
⎢⎣⎡−=
−+−−4
2129812
1tettec
Applying initial conditions: at t = 0, z1 = -1, we have
-1 = c1 - (1/98) (0) ⇒ c1 = -1
Therefore,
[ ] etez tt 2392
121 12 −− +−−−=
Note that
[ ]a
x
aaxeay xdyye 1
0−=∫
69
Similarly, z2 can be obtained as
[ ]t
xe tz 100
510891
2 1100 −+−=.
Substituting z in y = U z, we can get desired solution y.
top related