lecture 17 introduction to eigenvalue problems
DESCRIPTION
Lecture 17 Introduction to Eigenvalue Problems. Shang-Hua Teng. Eigenvalue Problems. Eigenvalue problems occur in many areas of science and engineering E.g., Structure analysis It is important for analyzing numerical and linear algebra algorithms - PowerPoint PPT PresentationTRANSCRIPT
Lecture 17Introduction to Eigenvalue Problems
Shang-Hua Teng
Eigenvalue Problems• Eigenvalue problems occur in many areas of
science and engineering– E.g., Structure analysis
• It is important for analyzing numerical and linear algebra algorithms– Impact of roundoff errors and precision requirement
• It is widely used in information management and web-search
• It is the key ingredient for the analysis of Markov process, sampling algorithms, and various approximation algorithms in computer science
Eigenvalues and Eigenvectors
• Standard Eigenvalue Problem: Given an n by n matrix A, find a scalar and nonzero vector x such that
A x = x is eigenvalue, and x is corresponding
eigenvector
Spectrum of Matrices
• Spectrum(A) = (A) = set of all eigenvalues of A
• Spectral radius (A) = (A) = max {||: in (A)}
• Spectral analysis
• Spectral methods
Geometric Interpretation
• Matrix expands or shrinks any vector lying in direction of eigenvector by scalar factor
• Expansion of contraction factor given by corresponding eigenvalue
• Eigenvalues and eigenvectors decompose complicated behavior of general linear transformation into simpler actions
Examples: Eigenvalues and Eigenvectors
10
,2 ,01
,1
2001
2211 xx
A
Note: x1 and x2 are perpendicular to each other
Examples: Eigenvalues and Eigenvectors
11
,2 ,01
,1
2011
2211 xx
A
Note: x1 and x2 are not perpendicular to each other
Examples: Eigenvalues and Eigenvectors
11
,4 ,11
,2
3113
2211 xx
A
Note: x1 and x2 are perpendicular to each other
Examples: Eigenvalues and Eigenvectors
11
,1 ,11
,2
5.15.05.05.1
2211 xx
A
Note: x1 and x2 are perpendicular to each other
Examples: Eigenvalues and Eigenvectors
1 :where
1, ,
1,
0110
2211
- i
ixi
ixi
A
Note: x1 and x2 are not perpendicular to each other : eigenvalues or eigenvectors may not be real!!!
Simple Facts of Eigenvalue Problem
• If (,x) is a eigenvalue-eigenvector pair of A, then for any k, (k, x) is a eigenvalue-eigenvector pair of Ak.
• If (,x) is a eigenvalue-eigenvector pair of A, then for any c, (c, x) is a eigenvalue-eigenvector pair of cA.
Algebraic Interpretation:Equation for the Eigenvalues
A x = x(A - I ) x = 0
• The eigenvectors make up the nullspace of (A – I ) if we know .
Eigenvalue First
• If (A - I ) x = 0 has a nonzero solution, then– A - I is not invertible– The determinant of A - I must be zero.
Characteristic Equation for Eigenvalues
• The number l is an eigenvalue of A if and only if (A - I ) is singular:
det( A - I ) = 0
Characteristic Polynomial for Eigenvalues
• det (A - I ) = 0 is a polynomial in of degree at most n.
• The spectrum of A is the set of roots of this characteristic polynomial:
• Fundamental Theorem of Algebra implies that n by n matrix A always has n eigenvalues, but they need be neither distinct nor real
Examples: Characteristic Polynomial
So Spectrum(A) = {1,2}
2120
01det
1001
2001
det)det(
2001
IA
A
Examples: Characteristic Polynomial
So Spectrum(A) = {2, 4}
4286
13331
13det
1001
3113
det)det(
3113
2
IA
A
Examples: Characteristic Polynomial
So Spectrum(A) = {i, -i}
ii
IA
A
11
1det
1001
0110
det)det(
0110
2
A Possible Methods for Solving the Eigenvalue Problems
• Compute the characteristic polynomial of A in by expanding det(A – I) = 0
• Find the roots of the characteristic polynomial
• For each eigenvalue , solve (A – I) x=0 to find an eigenvector x.
Practical Difficulties
• Computing eigenvalues using characteristic polynimial is not recommended or used because– Roots of polynomial of degree > 4 cannot
always be computed in finite number of steps– A lot of work is needed in computing
coefficients of the characteristic polynomial– Computer has round-off errors
Examples: Characteristic Polynomial
So Spectrum(A) = {1+, 1-}
11
111
1det )det(
an smaller thslighly number a is where 1
1
2
machine
IA
A
But in machine, 2 < machine is equal to 0So, the algorithm will returnSpectrum(A) = {1,1}
Theory and Practice
• Characteristic polynomial is a powerful theoretical tool but usually is not useful computationally.
Special Matrix
• What is Spectrum( I )?– Multiplicity is the number of times root appears
when polynomial written as product of linear factors
– det(I – I ) = (1-)n
• What is Spectrum( upper or lower triangular matrix )?
Bad News
• Elimination does not preserve the ’s.
2 and 0 has 1111
1 and 0 has 0011
21
21
Diagonalizing A Matrix• Suppose the n by n matrix A has n linearly independent
eigenvectors x1, x2,…, xn.
• Eigenvector matrix S: x1, x2,…, xn are columns of S.• Then
n
ASS
11
is the eigenvalue matrix
Matrix Power Ak
• S-1AS = implies A = S S-1
• implies A2 = S S-1 S S-1 = S S-1
• implies Ak = S kS-1
Random walks
How long does it take to get completely lost?
000001
Random walks Transition Matrix1
2
345
6
000001
021
4100
21
310
41000
31
210
21
310
00410
310
0041
210
21
31000
310
100
P
Matrix Powers
• If A is diagonalizable as A = S S-1 then for any vector u, we can compute Aku efficiently
– Solve S c = u– Aku = S kS-1 S c = S k c
• As if A is a diagonal matrix!!!!
Independent Eigenvectors from Different Eigenvalues
• Eigenvectors x1, x2,…, xk that correspond to distinct (all different) eigenvalues are linear independent.
• An n by n matrix that has n different eigenvalues (no repeated ’s) must be diagonalizable
Proof: Show that
implies all ci = 0
011 kk xcxc