lesson 16 the spectral theorem and applications
TRANSCRIPT
Lesson 16 (S&H, Section 14.6)The Spectral Theorem and Applications
Math 20
October 26, 2007
Announcements
I Welcome parents!
I Problem Set 6 on the website. Due October 31.
I OH: Mondays 1–2, Tuesdays 3–4, Wednesdays 1–3 (SC 323)
I Prob. Sess.: Sundays 6–7 (SC B-10), Tuesdays 1–2 (SC 116)
Outline
Hatsumon
Concept ReviewEigenbusinessDiagonalization
The Spectral TheoremThe split caseThe symmetric caseIterations
ApplicationsBack to FibonacciMarkov chains
A famous math problem
“A certain man had one pairof rabbits together in acertain enclosed place, andone wishes to know howmany are created from thepair in one year when it is thenature of them in a singlemonth to bear another pair,and in the second monththose born to bear also.Because the abovewritten pairin the first month bore, youwill double it; there will betwo pairs in one month.”
Leonardo of Pisa(1170s or 1180s–1250)
a/k/a Fibonacci
Diagram of rabbits
f (0) = 1
f (1) = 1
f (2) = 2
f (3) = 3
f (4) = 5
f (5) = 8
Diagram of rabbits
f (0) = 1
f (1) = 1
f (2) = 2
f (3) = 3
f (4) = 5
f (5) = 8
Diagram of rabbits
f (0) = 1
f (1) = 1
f (2) = 2
f (3) = 3
f (4) = 5
f (5) = 8
Diagram of rabbits
f (0) = 1
f (1) = 1
f (2) = 2
f (3) = 3
f (4) = 5
f (5) = 8
Diagram of rabbits
f (0) = 1
f (1) = 1
f (2) = 2
f (3) = 3
f (4) = 5
f (5) = 8
Diagram of rabbits
f (0) = 1
f (1) = 1
f (2) = 2
f (3) = 3
f (4) = 5
f (5) = 8
An equation for the rabbits
Let f (k) be the number of pairs of rabbits in month k . Each newmonth we have
I The same rabbits as last month
I Every pair of rabbits at least one month old producing a newpair of rabbits
Sof (k) = f (k − 1) + f (k − 2)
An equation for the rabbits
Let f (k) be the number of pairs of rabbits in month k . Each newmonth we have
I The same rabbits as last month
I Every pair of rabbits at least one month old producing a newpair of rabbits
Sof (k) = f (k − 1) + f (k − 2)
Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007
Page1of19
Some fibonacci numbers
k f (k)
0 11 12 23 34 55 86 137 218 349 55
10 8911 14412 233
QuestionCan we find an explicit formula for f (k)?
Outline
Hatsumon
Concept ReviewEigenbusinessDiagonalization
The Spectral TheoremThe split caseThe symmetric caseIterations
ApplicationsBack to FibonacciMarkov chains
Concept Review
DefinitionLet A be an n × n matrix. The number λ is called an eigenvalueof A if there exists a nonzero vector x ∈ Rn such that
Ax = λx. (1)
Every nonzero vector satisfying (??) is called an eigenvector of Aassociated with the eigenvalue λ.
Diagonalization Procedure
I Find the eigenvalues and eigenvectors.
I Arrange the eigenvectors in a matrix P and the correspondingeigenvalues in a diagonal matrix D.
I If you have “enough” eigenvectors (that is, one for eachcolumn of A), the original matrix is diagonalizable and equalto PDP−1.
I Pitfalls:I Repeated eigenvaluesI Nonreal eigenvalues
Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007
Page2of19
Outline
Hatsumon
Concept ReviewEigenbusinessDiagonalization
The Spectral TheoremThe split caseThe symmetric caseIterations
ApplicationsBack to FibonacciMarkov chains
QuestionUnder what conditions on A would you be able to guarantee thatA is diagonalizable?
Theorem (Baby Spectral Theorem)
Suppose An×n has n distinct real eigenvalues. Then A isdiagonalizable.
Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007
Page3of19
Theorem (Spectral Theorem for Symmetric Matrices)
Suppose An×n is symmetric, that is, A′ = A. Then A isdiagonalizable. In fact, the eigenvectors can be chosen to bepairwise orthogonal with length one, which means that P−1 = P′.Thus a symmetric matrix can be diagonalized as
A = PDP′,
Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007
Page4of19
Powers of diagonalizable matrices
Remember if A is diagonalizable then
Ak = (PDP−1)k = (PDP−1)(PDP−1) · · · (PDP−1)︸ ︷︷ ︸k
= PD(P−1P)D(P−1P) · · ·D(P−1P)DP−1 = PDkP−1
Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007
Page5of19
Another way to look at it
I If v is an eigenvector corresponding to eigenvalue λ, then
Akv =
λkv
I If v1, . . . vn are eigenvectors with eigenvalues λ1, . . . , λn, then
Ak(c1v1 + · · ·+ cnvn) = c1λk1v1 + · · ·+ cnλ
knvn
I If A is diagonalizable, there are n linearly independenteigenvectors, so any v can be written as a linear combinationof them.
Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007
Page6of19
Another way to look at it
I If v is an eigenvector corresponding to eigenvalue λ, then
Akv = λkv
I If v1, . . . vn are eigenvectors with eigenvalues λ1, . . . , λn, then
Ak(c1v1 + · · ·+ cnvn) = c1λk1v1 + · · ·+ cnλ
knvn
I If A is diagonalizable, there are n linearly independenteigenvectors, so any v can be written as a linear combinationof them.
Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007
Page7of19
Another way to look at it
I If v is an eigenvector corresponding to eigenvalue λ, then
Akv = λkv
I If v1, . . . vn are eigenvectors with eigenvalues λ1, . . . , λn, then
Ak(c1v1 + · · ·+ cnvn)
= c1λk1v1 + · · ·+ cnλ
knvn
I If A is diagonalizable, there are n linearly independenteigenvectors, so any v can be written as a linear combinationof them.
Another way to look at it
I If v is an eigenvector corresponding to eigenvalue λ, then
Akv = λkv
I If v1, . . . vn are eigenvectors with eigenvalues λ1, . . . , λn, then
Ak(c1v1 + · · ·+ cnvn) = c1λk1v1 + · · ·+ cnλ
knvn
I If A is diagonalizable, there are n linearly independenteigenvectors, so any v can be written as a linear combinationof them.
Another way to look at it
I If v is an eigenvector corresponding to eigenvalue λ, then
Akv = λkv
I If v1, . . . vn are eigenvectors with eigenvalues λ1, . . . , λn, then
Ak(c1v1 + · · ·+ cnvn) = c1λk1v1 + · · ·+ cnλ
knvn
I If A is diagonalizable, there are n linearly independenteigenvectors, so any v can be written as a linear combinationof them.
Outline
Hatsumon
Concept ReviewEigenbusinessDiagonalization
The Spectral TheoremThe split caseThe symmetric caseIterations
ApplicationsBack to FibonacciMarkov chains
Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007
Page9of19
Setting up the Fibonacci sequence
Recall the Fibonacci sequence defined by
f (k + 2) = f (k) + f (k + 1), f (0) = 1, f (1) = 1
Let’s let g(k) = f (k + 1). Then
g(k + 1) = f (k + 2) = f (k) + f (k + 1) = f (k) + g(k).
So if y(k) =
(f (k)g(k)
), we have
y(k + 1) =
(f (k + 1)g(k + 1)
)=
(g(k)
f (k) + g(k)
)=
(0 11 1
)y(k)
So if A is this matrix, then
y(k) = Aky(0).
Setting up the Fibonacci sequence
Recall the Fibonacci sequence defined by
f (k + 2) = f (k) + f (k + 1), f (0) = 1, f (1) = 1
Let’s let g(k) = f (k + 1). Then
g(k + 1) = f (k + 2) = f (k) + f (k + 1) = f (k) + g(k).
So if y(k) =
(f (k)g(k)
), we have
y(k + 1) =
(f (k + 1)g(k + 1)
)=
(g(k)
f (k) + g(k)
)=
(0 11 1
)y(k)
So if A is this matrix, then
y(k) = Aky(0).
Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007
Page11of19
Setting up the Fibonacci sequence
Recall the Fibonacci sequence defined by
f (k + 2) = f (k) + f (k + 1), f (0) = 1, f (1) = 1
Let’s let g(k) = f (k + 1). Then
g(k + 1) = f (k + 2) = f (k) + f (k + 1) = f (k) + g(k).
So if y(k) =
(f (k)g(k)
), we have
y(k + 1) =
(f (k + 1)g(k + 1)
)=
(g(k)
f (k) + g(k)
)=
(0 11 1
)y(k)
So if A is this matrix, then
y(k) = Aky(0).
Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007
Page12of19
Setting up the Fibonacci sequence
Recall the Fibonacci sequence defined by
f (k + 2) = f (k) + f (k + 1), f (0) = 1, f (1) = 1
Let’s let g(k) = f (k + 1). Then
g(k + 1) = f (k + 2) = f (k) + f (k + 1) = f (k) + g(k).
So if y(k) =
(f (k)g(k)
), we have
y(k + 1) =
(f (k + 1)g(k + 1)
)=
(g(k)
f (k) + g(k)
)=
(0 11 1
)y(k)
So if A is this matrix, then
y(k) =
Aky(0).
Setting up the Fibonacci sequence
Recall the Fibonacci sequence defined by
f (k + 2) = f (k) + f (k + 1), f (0) = 1, f (1) = 1
Let’s let g(k) = f (k + 1). Then
g(k + 1) = f (k + 2) = f (k) + f (k + 1) = f (k) + g(k).
So if y(k) =
(f (k)g(k)
), we have
y(k + 1) =
(f (k + 1)g(k + 1)
)=
(g(k)
f (k) + g(k)
)=
(0 11 1
)y(k)
So if A is this matrix, then
y(k) = Aky(0).
Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007
Page13of19
Diagonalize
The eigenvalues of A =
(0 11 1
)are found by solving
0 =
∣∣∣∣−λ 11 1− λ
∣∣∣∣ = (−λ)(1− λ)− 1
= λ2 − λ− 1
The roots are
ϕ =1 +√
5
2ϕ̄ =
1−√
5
2
Notice thatϕ+ ϕ̄ = 1, ϕϕ̄ = −1
(These facts make later calculations simpler.)
Diagonalize
The eigenvalues of A =
(0 11 1
)are found by solving
0 =
∣∣∣∣−λ 11 1− λ
∣∣∣∣ = (−λ)(1− λ)− 1
= λ2 − λ− 1
The roots are
ϕ =1 +√
5
2ϕ̄ =
1−√
5
2
Notice thatϕ+ ϕ̄ = 1, ϕϕ̄ = −1
(These facts make later calculations simpler.)
Diagonalize
The eigenvalues of A =
(0 11 1
)are found by solving
0 =
∣∣∣∣−λ 11 1− λ
∣∣∣∣ = (−λ)(1− λ)− 1
= λ2 − λ− 1
The roots are
ϕ =1 +√
5
2ϕ̄ =
1−√
5
2
Notice thatϕ+ ϕ̄ = 1, ϕϕ̄ = −1
(These facts make later calculations simpler.)
Eigenvectors
We row reduce to find the eigenvectors:
A− ϕI =
(−ϕ 11 1− ϕ
)=
(− ϕ 11 ϕ̄
)←−−ϕ̄
+
(−ϕ 10 0
)
So
(1ϕ
)is an eigenvector for A corresponding to the eigenvalue ϕ.
Similarly,
(1ϕ̄
)is an eigenvector for A corresponding to the
eigenvalue ϕ̄. So now we know that
y(k) = c1ϕk
(1ϕ
)+ c2ϕ̄
k
(1ϕ̄
)
Eigenvectors
We row reduce to find the eigenvectors:
A− ϕI =
(−ϕ 11 1− ϕ
)=
(− ϕ 11 ϕ̄
)←−−ϕ̄
+
(−ϕ 10 0
)
So
(1ϕ
)is an eigenvector for A corresponding to the eigenvalue ϕ.
Similarly,
(1ϕ̄
)is an eigenvector for A corresponding to the
eigenvalue ϕ̄.
So now we know that
y(k) = c1ϕk
(1ϕ
)+ c2ϕ̄
k
(1ϕ̄
)
Eigenvectors
We row reduce to find the eigenvectors:
A− ϕI =
(−ϕ 11 1− ϕ
)=
(− ϕ 11 ϕ̄
)←−−ϕ̄
+
(−ϕ 10 0
)
So
(1ϕ
)is an eigenvector for A corresponding to the eigenvalue ϕ.
Similarly,
(1ϕ̄
)is an eigenvector for A corresponding to the
eigenvalue ϕ̄. So now we know that
y(k) = c1ϕk
(1ϕ
)+ c2ϕ̄
k
(1ϕ̄
)
What are the constants?
To find c1 and c2, we solve(11
)= c1
(1ϕ
)+ c2
(1ϕ̄
)=
(1 1ϕ ϕ̄
)(c1
c2
)=⇒
(c1
c2
)=
(1 1ϕ ϕ̄
)−1(11
)=
1
ϕ̄− ϕ
(ϕ̄ −1−ϕ 1
)=
1
ϕ̄− ϕ
(ϕ̄− 1ϕ+ 1
)(11
)=
1√5
(ϕ−ϕ̄
)
Math 20 - October 26, 2007.GWBFriday, Oct 26, 2007
Page18of19
Finally
Putting this all together we have
y(k) =ϕ√
5ϕk
(1ϕ
)− ϕ̄√
5ϕ̄k
(1ϕ̄
)(
f (k)g(k)
)=
1√5
(ϕk+1 − ϕ̄k+1
ϕk+2 − ϕ̄k+2
)
So
f (k) =1√5
(1 +√
5
2
)k+1
−
(1−√
5
2
)k+1
Finally
Putting this all together we have
y(k) =ϕ√
5ϕk
(1ϕ
)− ϕ̄√
5ϕ̄k
(1ϕ̄
)(
f (k)g(k)
)=
1√5
(ϕk+1 − ϕ̄k+1
ϕk+2 − ϕ̄k+2
)So
f (k) =1√5
(1 +√
5
2
)k+1
−
(1−√
5
2
)k+1
Markov Chains
I Recall the setup: T is a transition matrix giving theprobabilities of switching from any state to any of the otherstates.
I We seek a steady-state vector, i.e., a probability vector usuch that Tu = u.
I This is nothing more than an eigenvector of eigenvalue 1!
Markov Chains
I Recall the setup: T is a transition matrix giving theprobabilities of switching from any state to any of the otherstates.
I We seek a steady-state vector, i.e., a probability vector usuch that Tu = u.
I This is nothing more than an eigenvector of eigenvalue 1!
Markov Chains
I Recall the setup: T is a transition matrix giving theprobabilities of switching from any state to any of the otherstates.
I We seek a steady-state vector, i.e., a probability vector usuch that Tu = u.
I This is nothing more than an eigenvector of eigenvalue 1!
TheoremIf T is a regular doubly-stochastic matrix, then
I 1 is an eigenvalue for T
I all other eigenvalues of T have absolute value less than 1.
Let u be an eigenvector of eigenvalue 1, scaled so it’s a probabilityvector. Let v2, . . . , vn be eigenvectors corresponding to the othereigenvalues λ2, . . . , λn. Then for any initial state x(0), we have
x(k) = Akx(0) = Ak (c1u + c2λ2v2 + · · ·+ cnλnvn)
= c1u + c2λk2v2 + · · ·+ cnλ
knvn
Sox(k)→ c1u
Since each x(k) is a probability vector, c1 = 1. Hence
x(k)→ c1u