The Islamic University of GazaFaculty of Engineering

Civil Engineering Department

Numerical Analysis ECIV 3306

Chapter 10

LU Decomposition and Matrix Inversion

Introduction• Gauss elimination solves [A] {x} ={B}

• It becomes insufficient when solving these equations for different values of {B}

• LU decomposition works on the matrix [A] and the vector {B} separately.

• LU decomposition is very useful when the vector of variables {x} is estimated for different parameter vectors {B} since the forward elimination process is not performed on {B}.

LU Decomposition

If:L: lower triangular matrixU: upper triangular matrixThen,[A]{X}={B} can be decomposed into

twomatrices [L] and [U] such that:1. [L][U] = [A] ([L][U]){X} = {B}

LU Decomposition

Consider:[U]{X} = {D}So, [L]{D} = {B}2. [L]{D} = {B} is used to generate an

intermediate vector {D} by forward substitution.

3. Then, [U]{X}={D} is used to get {X} by back substitution.

Summary of LU Decomposition

LU Decomposition

As in Gauss elimination, LU decomposition must employ pivoting to avoid division by zero and to minimize round off errors. The pivoting is done immediately after computing each column.

LU Decomposition

3

2

1

3

2

1

333231

232221

131211

bbb

xxx

aaaaaaaaa

11 12 13/ /22 23

//33

[ ] 00 0

a a aU a a

a

21

31 32

1 0 0[ ] 1 0

1L l

l l

Step 1: Decomposition

System of linear equations [A]{x}={B}

][]][[ AUL

LU DecompositionStep 2: Generate an intermediate vector {D} by

forwardsubstitution

Step 3: Get {X} by back substitution.

3

2

1

3

2

1

3231

21

101001

bbb

ddd

lll

3

2

1

3

2

1

33

2322

131211

''00''0

ddd

xxx

aaaaaa

LU Decomposition-Example

1020303071020103

A..

..

.. 3 0.1 0.20 7.003 0.2930 0.19 10.02

21 31

\32

32 \22

0.1 0.30.03333; 0.10003 3

0.19 0.027137.003

l l

ala

012100029300037020103

U..... 1 0 0

[ ] 0.03333 1 00.1000 .02713 1

L

LU Decomposition-Example (cont’d)

4.713.19

85.71

102.03.03.071.02.01.03

3

2

1

xxx

0843.705617.1985.7

4.713.19

85.7

102713.01000.0010333.0001

3

2

1

3

2

1

ddd

ddd

Step 2: Find the intermediate vector {D} by forward substitution

Use previous L and D matrices to solve the system:

LU Decomposition-Example (cont’d)

Step 3: Get {X} by back substitution.

00003.75.2

3

0843.705617.1985.7

012.10002933.00033.70

2.01.03

3

2

1

3

2

1

xxx

xxx

Decomposition Step• % Decomposition Stepfor k=1:n-1 [a,o]= pivot(a,o,k,n); for i = k+1:n a(i,k) = a(i,k)/a(k,k); a(i,k+1:n)= a(i,k+1:n)-a(i,k)*a(k,k+1:n); endend

Partial Pivoting• %Partial Pivotingfunction [a,o] = pivot(a,o,k,n)[big piv]=max(abs(a(k:n,k)));piv=piv+(k-1);if piv ~= k temp = a(piv,:); a(piv,:)= a(k,:); a(k,:)=temp; temp = o(piv); o(piv)=o(k); o(k)=temp;end

Substitution Steps

%Forward Substitution d(1)=bn(1);for i=2:n d(i)=bn(i)-a(i,1:i-1)*(d(1:i-1))'; end

• % Back Substitutionx(n)=d(n)/a(n,n);for i=n-1:-1:1 x(i)=(d(i)-a(i,i+1:n)*x(i+1:n)')/a(i,i);end

Matrix Inverse Using the LU Decomposition

• LU decomposition can be used to obtain the inverse of the original coefficient matrix [A].

• Each column j of the inverse is determined by using a unit vector (with 1 in the jth raw ).

Matrix Inverse: LU Decomposition

001

}{

1

11

xA

bxA

010

}{

2

22

xA

bxA

100

}{

3

33

xA

bxA

1st column of [A]-1

2nd column of [A]-1

3rd column of [A]-1

3211 xxxA }{}{}{

[A] [A]-1 = [A]-1[A] = I

Matrix inverse using LU decomposition Example

1020303071020103

A..

..

..

012100029300037020103

U.....1 0 0

[ ] 0.03333 1 00.1000 .02713 1

L

1009.003333.01

001

102713.01000.0010333.0001

3

2

1

3

2

1

ddd

ddd

1A. [L]{d}1 = {b}1

1B. Then, [U]{X}1={d}1

01008.000518.0

33249.0

1009.003333.01

012.10002933.00033.70

2.01.03

3

2

1

3

2

1

xxx

xxx

1st column of [A]-1

2A. [L]{d}2 = {b}2

Matrix inverse using LU decomposition Example (cont’d)

02713.010

010

102713.01000.0010333.0001

3

2

1

3

2

1

ddd

ddd

00271.0142903.0004944.0

02713.010

012.10002933.00033.70

2.01.03

3

2

1

3

2

1

xxx

xxx

2B. Then, [U]{X}2={d}22nd column of [A]-1

3A. [L]{d}3 = {b}3

Matrix inverse using LU decomposition Example (cont’d)

100

100

102713.01000.0010333.0001

3

2

1

3

2

1

ddd

ddd

09988.0004183.0006798.0

100

012.10002933.00033.70

2.01.03

3

2

1

3

2

1

xxx

xxx

3B. Then, [U]{X}3={d}33rd column of [A]-1

Matrix inverse using LU decomposition Example (cont’d)

09988.000271.001008.0004183.0142903.000518.0006798.0004944.033249.0

][ 1A

Vector and Matrix Norms

Norm is a real-valued function that provides a measure of size or “length” of vectors and matrices.

Norms are useful in studying the error behavior of algorithms.

y.repectivel axes, z and y, x,along distances theare c and b, a, where

as drepresente becan that spaceEuclidean ldimensiona-in three vector a is example simpleA

cbaF

Vector and Matrix Norms (cont’d)

Vector and Matrix Norms (cont’d)

• The length of this vector can be simply computed as222 cbaF

e

n

iji

n

je

n

iie

n

a

x

xxxX

1

2,

1

1

2

21

A

[A]matrix aFor

X

as computed is normEuclidean a

Length or Euclidean norm of [F]

• For an n dimensional vector

Frobenius norm

Vector and Matrix Norms (cont’d)

1/

1

11

,11 1

1-NormFor Vector

CFor olumn Sum Norm

X

X

( ) Matrix

A max

P-Normppn

ipi

n

ii

n

i jj n i

x

x

a

Vector and Matrix Norms (cont’d)

• Uniform vector norm

• Uniform matrix norm (row sum Norm)

inixmax

1X

n

jji

nia

1,

1maxA

Vector and Matrix Norms (cont’d)

• Matrix Condition umber Defined as:

• For a matrix [A], this number will be greater than or equal to 1.

• If the coefficients of [A] are known to t-digit precision (rounding errors~10-t) and Cond [A]=10c, the solution [X] may be valid to only t-c digits (rounding errors~10c-t).

1 AAACond

Iterative Refinement• Round-off errors can be reduced by the following procedure:

• Suppose an approximate solution vectors given by

• Substitute the result in the original system

....(Eq.1)

]~~~[~321 xxxX T

3333232131

2323222121

1313212111

~~~~

~~~~

~~~~

bxaxaxa

bxaxaxa

bxaxaxa

3333232131

2323222121

1313212111

bxaxaxabxaxaxabxaxaxa

Iterative Refinement (cont’d)

• Now, assume the exact solution is:

• Then:

… …….(Eq.2)

333

222

111

~~~

xxxxxxxxx

3333322321131

2332322221121

1331322121111

)~()~()~()~()~()~()~()~()~(

bxxaxxaxxabxxaxxaxxabxxaxxaxxa

Iterative Refinement (cont’d)

Subtract Eq.2 from Eq.1, the result is a system of linear equations

that can be solved to obtain the correction factors

The factors then can be applied to improve the solution as

specified by the equation:

333333232131

222323222121

111313212111

~

~

~

Ebbxaxaxa

Ebbxaxaxa

Ebbxaxaxa

x

333

222

111

~~~

xxxxxxxxx

Iterative Refinement- Example (cont’d)

Solve:

The exact solution is ………

1- Solve the equations using [A]-1, such as {x}=[A]-1{c}

]00.100.100.1[TX

]00.1997.0991.0[~ TX

58.232.211.285.122.598.077.653.2

28.511.206.123.4

321

321

321

xxxxxx

xxx

Iterative Refinement - Example (cont’d)

2- Substitute the result in the original system [A]{x}={c}

}~{}~]{[ CXA

59032.222246.524511.5

}~{C

0103.0

00246.00348.0

59032.258.222246.522.524511.528.5

}~{}{ CCE

Iterative Refinement - Example (cont’d)

3- Solve the system of linear equations

using [A]-1 to find the correction factors to yield

}{}~]{[ EXA

x

]00000757.000300.000822.0[ TX

00.1~00.1~999.0~

333

222

111

xxxxxxxxx