scientific computing qr factorization part 1 – orthogonal matrices, reflections and rotations
TRANSCRIPT
Orthogonal Matrices
Matrix Factoring: So far, we have seen that we can factor a non-singular matrix A as A = LU, where L is lower triangular and U is upper triangular.
QR Factoring: Similar idea to factor A = QR, where R is upper triangular and Q is orthogonal.
Definition: A square matrix Q is orthogonal ifQt Q = I
Orthogonal Matrix Properties
• Multiplication of a vector by an orthogonal matrix preserves Euclidean norm
• In general, multiplication by an orthogonal matrix Q preserves inner products, angles, and distances.
Qv 2
2 (Qv)tQv v tQtQv v tv v 2
2
Rotations
• Rotation matrices are one example of orthogonal matrices. In R2 the matrix
applied to a vector x rotates x counter-clockwise about the origin.
Qcos( ) sin( )sin( ) cos( )
θ
v
Qv
Rotations
• In general, the matrix
acts as a rotation in the (xi,xj) plane
Q
1
O
1
cos( ) sin( )1
O
1
sin( ) cos( )1
O
1
i
j
Reflections
• Let w be a unit vector (column vector). The matrix acts as a reflection across the line (plane) perpendicular to w.
QI 2ww t
w
v
Qv
O
(I 2ww t )w w 2ww tw w 2w w
(I 2ww t )2 I 4ww t 4ww tww t
I 4ww t 4ww t I
Householder Transformations
• A Householder transformation (also known as Householder reflection or elementary reflector) is a linear transformation that describes a reflection about a plane (or hyperplane ) containing the origin. The Householder transformation was introduced in 1958 by Alston Scott Householder.
(Wikipedia)
Effect of Householder Transformations
• Householder transformations can be used to zero out entries of a matrix below the diagonal.
• Let A = [A1 A2 … An] be a square matrix, with columns Ai.
• Let Let v1 = A1 – α1 e1 where e1 =(1,0,….,0)
Then,
2
2121211,11 )sgn( AThenAa
(**)22
2)()(
1,1121
211,11
2
211111112
21
a
aAeAeAv
Effect of Householder Transformations
• Now, let Then, w1 is a unit vector.
Thus, H1 = I – 2 w1 w1t is a reflection (Householder transf)
The first column of H1 A will be H1 A1 .
21
11 v
vw
1111111111
2
211,1121
1,112
21111111
1121
11111111
)(,
)(**)(22
22)(22,
22
eeAAvAAHThus
slidepreviousfrombyva
aAAeAAvNow
Avv
vAAwwAAH
t
tt
QR Factorization
Theorem: If A is any square matrix of size n, then, there is an orthogonal matrix Q, and an upper triangular matrix R, such that A = QR.
Proof: The construction above shows that there is a Householder matrix H1 such that the first column of
H1 A has zeros below the first entry. Proceed by induction. Suppose, after step k-1 we have found a series of matrices H1, H2, …, Hk-1 such that
Hk-1 …H2 H1 A has all zeroes below the diagonal to position (k-1,k-1).
QR Factorization
1,
1,
1,1
1,11
1,1
11,1
12,11
121
000
00
0
knn
kkn
knk
kkkk
kn
kk
k
k
aa
aa
aaa
AHHH
QR FactorizationProof:(continued) . We will zero out the column below the (k,k)
entry as follows: Let
Then, Let
Note:
Thus, where Ik is a kxk identity matrix.
),,,,0,,0()sgn(
),,,,0,,0(
,,1,,
,,1,
kkn
kkk
kkk
kkkk
kkn
kkkk
kkkk
aaaa
aaav
kkkkkk av ,
22
222 t
kkkk
kk wwIHand
v
vw 2
2
**00
**00
0000
**0
*
*
0
tkkww
B
IwwIH tkkkk 0
0)2(
QR FactorizationProof:(continued) We see that Hk will only affect the lower (n-k)x(n-k) portion of the
current matrix. Thus, as before, Hk( Hk-1 …H2 H1 A ) will zero out the column below the (k,k) entry.
Thus, we have by induction, that we can find a sequence fo Householder transformations such that ( Hn-1 …H2 H1 A) = R,
where R is upper triangular. Then, A = H1-1
H2-1
… Hn-1-1 R. Since
Hi-1 =Hi , we have A = H1 H2 … Hn-1 R. Let Q = H1 H2 … Hn-1 . Since Q is
a product of orthogonal matrics, it is also orthogonal. QED.
B
IwwIH tkkkk 0
0)2(
QR ExampleTo derive H1 for the given matrix A :
We have :
Thus,
So,
074
212
034
4
2
4
1A
4
2
10
0
0
1
6
4
2
4
6
4
2
4
1 11 v
ttt
w
30
2,
30
1,
30
5
120
4,
120
2,
120
10
4210
4,2,10
2221
15
11
15
2
3
215
2
15
14
3
13
2
3
1
3
2
15
2
15
1
3
115
1
30
1
6
13
1
6
1
6
5
230
2,
30
1,
30
5
30
230
130
5
221 IIwwIH T
Example con’tThen,
k=2:
15
430
15
2810
3
276
074
212
034
15
11
15
2
3
215
2
15
14
3
13
2
3
1
3
2
1AH
3
1
7)2(
2A
3
101
0
0
1
0
10
3
1
0
10
3
1
0
1 22 v
1623.4101
1307.5
3,
1307.5
1623.4,
1307.5
0
9
3,,0
22
b
b
bw
tt
3162.09487.00
9487.03162.00
001
94868.120
4868.123246.170
000
3246.26
23,,0
3
0
9
22
2222 Ibbb
IwwIH t
Example con’tThen,
So, A = H1 H2 R = Q R where
RAH
6865.100
8433.01623.303
276
15
430
15
2810
3
276
3162.09487.00
9487.03162.00
001
2
1054.07379.06667.0
8433.04216.03333.0
5270.05271.06667.0
3162.09487.00
9487.03162.00
001
15
11
15
2
3
215
2
15
14
3
13
2
3
1
3
2
21HHQ
Example con’tFrom Matlab:>> A=[4 3 0; 2 1 2; 4 7 0];>> [q r] = qr(A)q = 0.6667 -0.5270 -0.5270 0.3333 -0.4216 0.8433 0.6667 0.7379 0.1054r = 6.0000 7.0000 0.6667 0 3.1623 -0.8433 0 0 1.6865 Note the slight difference between matlab’s result and our algorithm result. Matlab always uses αk values so that the diagonal of R has non-negative values.