9 gauss elimination - bml.pusan.ac.krbml.pusan.ac.kr/resources/09_cna_students.pdf · hk kim...

HK Kim Slightly modified 3/1/09, 2/28/06

Firstly written at March 2005

DM21869/Computational Numerical Analysis/09_cna.doc Available at http://bml.pusan.ac.krBML/SME/PNU

1

9 Gauss Elimination

Solving Small Numbers (n ≤ 3) of Equations Naive Gauss Elimination Pivoting Tridiagonal Systems Case Study: Model of a Heated Rod

What happens behind the following commands in MATLAB?

>> x = A\b

>> x = inv(A)*b




2

Solving Small Numbers (n ≤ 3) of Equations graphical method, Cramer's rule, and elimination of unknowns

Graphical method

⇔ 221823

21

21

=+−=+

xxxx

121

923

12

12

+=

+−=

xx

xx

• singular parallel lines → no solution

coincidental lines → infinite solutions

• ill-conditioned systems closed to being singular

extremely sensitive to round-off error

Graphical depiction of singular and ill-conditioned systems: (a) no solution, (b) infinite solutions, and (c) ill-conditioned system where the slopes are so close that the point of intersection is difficult to detect




3

Determinants and Cramer's rule

Let's consider a set of three equations:

where }{}]{[ bxA =⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

333231

232221

131211

][aaaaaaaaa

A

Then, we have the determinant of this system:

Example --------------------------------------------------------------------------------------------------------------------

ompute values for the determinants of the following systems.

C

⎩⎨⎧

=+−=+

221823

21

21

xxxx

⎪⎪⎩

⎪⎪⎨

⎧

=+−

=+−

21

21

121

21

21

xx

xx

⎪⎩

⎪⎨⎧

=+−

=+−

22

121

21

21

xx

xx

⎪⎪⎩

⎪⎪⎨

⎧

=+−

=+−

121

1.153.2

21

21

xx

xx

Sol.)

---- ------------------------------------------------------------------------------------

• ill-conditioning systems → determinants close to zero

-- --------------------------------------------------

• singular systems → zero determinants




4

• Cramer's rule

Example --------------------------------------------------------------------------------------------------------------------

se Cramer's rule to solve

ol.)

------------------------------------------------------------------------------------

Cramer's rule becomes impractical for n > 3.

U

44.05.03.0 1.067.0 9.1 5.001.0 52.03.0

321

321

321

−=++=++

−=++

xxxxxxxxx

S

--------------------------------------------------------




5

Elimination unknowns

Let's consider a set of two equations:

2222121

1212111

bxaxabxaxa

=+=+

Multiplying by constants:

2112221112111

1212122111121

baxaaxaabaxaaxaa

=+=+

Subracting and eliminating x1:

1212112122122211 babaxaaxaa −=−

Then we can solve 12212211

1212112 aaaa

babax−−

= and substituting:

12212211

2121221 aaaa

babax−−

=

This elimination of unknowns is the same as Cramer's rule:

12212211

212122

2221

1211

222

121

1 aaaababa

aaaaabab

x−−

== 12212211

121211

2221

1211

221

111

2 aaaababa

aaaababa

x−−

==

The technique of the elimination of unknowns can be formalized and readily programmed for the computer.




6

Naive Gauss Elimination

• two phases of Gauss elimination: forward elimination and back substitution

manipulating the equations in order that we have one equation with one unknown

solving the one unknown and substituting in backward for the remaining unknowns

• extended to large sets of equations if we develop a systematic scheme or algorithm

• Gauss elimination is the most basic of these schemes

• "naive" Gauss elimination when the algorithm does not avoid division by zero

Let's consider a general set of n equations:

nnnnnnn

n

n

bxaxaxaxa

bxaxaxaxabxaxaxaxa

=++++

=++++=++++

L

MM

L

L

332211

222223222121

111113112111

Phase : forward elimination of unknowns (or reducing the set of equations to an upper triangular system)

Eliminate the first unknown x1 from the second through the nth equations;

multiplying: 111

211

11

21313

11

21212

11

21121a b

aaxa

aaxa

aaxa

aax nn =++++ L

pivot equation for the first unknown x1

pivot element for the first unknown x1




7

subtracting: 111

2121

11

212212

11

2122 b

aa

bxaaa

axaaa

a nnn −=⎟⎟⎠

⎞⎜⎜⎝

⎛−++⎟⎟

⎠

⎞⎜⎜⎝

⎛− L or '

2'22

'22 bxaxa nn =++L

Repeat for the remaining equations:

''3

'32

'2

'3

'33

'332

'32

'2

'23

'232

'22

11313212111

nnnnnn

nn

nn

nn

bxaxaxa

bxaxaxabxaxaxabxaxaxaxa

=+++

=+++=+++=++++

L

MM

L

L

L

Eliminate the second unknown x1 using the second pivot equation;

""3

"3

"3

"33

"33

'2

'23

'232

'22

11313212111

nnnnn

nn

nn

nn

bxaxa

bxaxabxaxaxabxaxaxaxa

=++

=++=+++=++++

L

MM

L

L

L

Continue using the remaining pivot equations and construct an upper triangular system;

Phase : back substitution

)1(

)1(

−

−

= nnn

nn

n abx

for i = n-1, n-2, …,1




8

Example --------------------------------------------------------------------------------------------------------------------

Use Gauss elimination to solve

Note that the exact solutions are x1 = 3, x2 = -2.5, and x3 = 7.

ol.)

------------------------------------------------------------------------------------------------------------------------------------------

4.71 10 .203.03.193.07 1.085.7 0.21.03

321

321

321

=+−−=−+

=−−

xxxxxxxxx

S

--




9

<An M-file to implement naive Gauss elimination>

Operation counting

• execution time ~ the amount of floating-point operations (or flops)

• some related definitions;

∑∑==

=m

i

m

iifcicf

11)()( { } ∑∑∑

===

+=+m

i

m

i

m

iigifigif

111)()()()(

mm

i=++++=∑

=

111111

L 11 +−=∑=

kmm

ki

)(22

)1(3212

1mOmmmmi

m

i+=

+=++++=∑

=

L

)(36

)12)(1(321 23

2222

1

2 mOmmmmmim

i+=

++=++++=∑

=

L

where O(mn) = terms of order mn and lower

moving below the pivot row to each of the subsequent rows

moving down the matrix from one pivot row to the next




10

For the M-file about the naive Gauss elimination:

The limits on the inner loop are from i = 2 to n because the first pass through the outer loop, k = 1.

Therefore, the number of iterations of the inner loop is that 11212

−=+−=∑=

nnn

i

• for n – 1 iterations in the inner loop;

- a division

- a multiplication for each column element from k = 1 to nb (= from 2 to n + 1 = n + 1 multiplications)

- similarly, n + 1 subtractions

- in summary, n + 2 multiplication/divisions and n + 1 subtractions

- for the first pass through the outer loop,

total (n – 1)(n + 2) multiplications/divisions and (n – 1)(n + 1) subtractions

Summarizing, Outer loop

k Inner loop

i Addition/subtraction

flops Multiplication/division

flops 1 2, n (n – 1)(n + 1) (n – 1)(n + 2) 2 3, n (n – 2)(n) (n – 2)(n + 1) M M M M k k + 1, n (n – k)(n + 2 – k) (n – k)(n + 3 – k) M M M M

n – 1 n, n (1)(3) (1)(4)

Therefore, the total addition/subtraction flops for elimination is

∑∑−

=

−

=

++−+=−+−1

1

21

1])22()1([)2)((

n

k

n

kknknnknkn

or ∑∑∑−

=

−

=

−

=

++−+1

1

21

1

1

1)22(1)1(

n

k

n

k

n

kkknnn

And finally yielding

)(3

)(31)]([)]([ 2

33323 nOnnOnnOnnOn +=⎥⎦

⎤⎢⎣⎡ +++−+




11

A similar analysis for the multiplication/division flops yields

Summing these results gives

Note that O(n2) and lower terms become negligible as n gets large.

For the back substitution;

number of addition/subtraction flops = n(n – 1)/2

multiplication/division flops = n(n + 1)/2

Therefore, the sum is

The total effort in naive Gauss elimination is

computation time increases greatly as the system gets larger

most of the effort is incurred in the elimination step

n Elimination Back substitution Total flops 2n3/3 % due to

elimination 10

100 1000

705 671550

6.67 × 108

10010000

1 × 106

805681550

6.68 × 108

667 666667

6.67 × 108

87.58%98.53%99.85%




12

Pivoting

5 63237648 32

321

321

32

=+−−=++

=+

xxxxxxxx

→ A division (or normalization) by zero occurs with naive Gauss elimination.

What happens when the pivot element is close to zero?

Round-off errors can be introduced if the magnitude of the pivot element is small compared to the other element!

Remedy

determine the coefficient with largest absolute value in the column below the pivot elements

before each row is normalized

switch the rows so that the largest element is the pivot element ⇒ partial pivoting

searching both columns and rows for the largest element and switching ⇒ complete pivoting

→ rarely used method




13

Example --------------------------------------------------------------------------------------------------------------------

Gauss elimination to solve

he computation, but

artial pivot by reversing the order of the equations. The exact solution is x1 = 1/3 and x2 = 2/3.

ol.)

------------------------------------------------------------------------------------------------------------------------------------------

Use

0000.10000.10000.1 21 =+ xx

Note that in this form the first pivot element, a11 = 0.0003, is very close to zero. Then repeat t

0001.20000.30003.0 21 =+ xx

p

S

--




14

<An M-file to implement the Gauss elimination with partial pivoting>




15

Tridiagonal Systems to avoid strong large numbers of useless zeros

Let's consider a tridiagonal system having a bandwidth of 3.

⎪⎪⎪⎪⎪

⎭

⎪⎪⎪⎪⎪

⎬

⎫

⎪⎪⎪⎪⎪

⎩

⎪⎪⎪⎪⎪

⎨

⎧

⋅⋅⋅

=

⎪⎪⎪⎪⎪

⎭

⎪⎪⎪⎪⎪

⎬

⎫

⎪⎪⎪⎪⎪

⎩

⎪⎪⎪⎪⎪

⎨

⎧

⋅⋅⋅

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎣

⎡

⋅⋅⋅⋅⋅⋅

⋅⋅⋅

−−−−−

n

n

n

n

nn

nnn

rr

rrr

xx

xxx

fegfe

gfegfe

gf

1

3

2

1

1

3

2

1

111

333

222

11

Example --------------------------------------------------------------------------------------------------------------------

Solve the following tridiagonal system:

ol.)

------------------------------------------------------------------------------------------------------------------------------------------

⎪⎪⎭

⎪⎪⎬

⎫

⎪⎪⎩

⎪⎪⎨

⎧

=

⎪⎪⎭

⎪⎪⎬

⎫

⎪⎪⎩

⎪⎪⎨

⎧

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

−−−

−−−

8.408.08.08.40

04.21104.21

104.21104.2

4

3

2

1

xxxx

S

--



DM21869/Computational Numerical Analysis/09_cna.doc Available at http://bml.pusan.ac.kr BML/SME/PNU

16

• computational effort for the tridiagonal systems ~ n (note that Gauss elimination ~ n3)

<An M-file to solve a tridiagonal system>

• most tridiagonal systems do not require pivoting, although sometimes required

9 gauss elimination - bml.pusan.ac.krbml.pusan.ac.kr/resources/09_cna_students.pdf · hk kim...

Documents