Download - Gaussian LU Eliminations Seidel
-
7/28/2019 Gaussian LU Eliminations Seidel
1/100
ES 84 Numerical Methods
Stephen H. Haim
Computer Engineering Dept./EECE
Gaussian Elimination,
LU Decomposition &
Gauss-Seidel
-
7/28/2019 Gaussian LU Eliminations Seidel
2/100
Gaussian Elimination
2
-
7/28/2019 Gaussian LU Eliminations Seidel
3/100
Nave Gaussian Elimination
One of the most popular techniques for
solving simultaneous linear equations of the
form
Consists of 2 steps
1. Forward Elimination of Unknowns.
2. Back Substitution
CXA
-
7/28/2019 Gaussian LU Eliminations Seidel
4/100
Forward Elimination
The goal of Forward Elimination is to transform
the coefficient matrix into an Upper Triangular
Matrix
7.000
56.18.40
1525
112144
1864
1525
-
7/28/2019 Gaussian LU Eliminations Seidel
5/100
Forward Elimination
Linear Equations
A set ofn equations and n unknowns
11313212111 ... bxaxaxaxa nn 22323222121 ... bxaxaxaxa nn
nnnnnnn bxaxaxaxa ...332211
. .
. .
. .
-
7/28/2019 Gaussian LU Eliminations Seidel
6/100
Forward Elimination
Transform to an Upper Triangular Matrix
Step 1: Eliminate x1 in 2nd equation using equation 1 as
the pivot equation
)(1
21
11
aa
Eqn
Which will yield
1
11
211
11
21212
11
21121 ... b
a
axa
a
axa
a
axa nn
-
7/28/2019 Gaussian LU Eliminations Seidel
7/100
Forward Elimination
Zeroing out the coefficient of x1 in the 2nd equation.
Subtract this equation from 2nd equation
1
11
2121
11
212212
11
2122 ... ba
abxaa
aaxaa
aa nnn
'
2
'
22
'
22 ... bxaxa nn
nnna
a
aaa
aa
a
aa
1
11
21
2
'
2
12
11
2122'22
Or Where
-
7/28/2019 Gaussian LU Eliminations Seidel
8/100
Forward Elimination
Repeat this procedure for the remaining
equations to reduce the set of equations as
11313212111 ... bxaxaxaxa nn '
2
'
23
'
232
'
22 ... bxaxaxa nn '
3
'
33
'
332
'
32 ... bxaxaxa nn
''
3
'
32
'
2 ... nnnnnn bxaxaxa
. . .
. . .
. . .
-
7/28/2019 Gaussian LU Eliminations Seidel
9/100
Forward Elimination
Step 2: Eliminate x2 in the 3rd equation.
Equivalent to eliminating x1 in the 2nd equation
using equation 2 as the pivot equation.
)(2
332
22
aa
EqnEqn
-
7/28/2019 Gaussian LU Eliminations Seidel
10/100
Forward Elimination
This procedure is repeated for the remaining
equations to reduce the set of equations as
11313212111 ... bxaxaxaxa nn '
2
'
23
'
232
'
22 ... bxaxaxa nn "
3
"
33
"
33 ... bxaxa nn
""
3
"
3 ... nnnnn bxaxa
. .
. .
. .
-
7/28/2019 Gaussian LU Eliminations Seidel
11/100
Forward Elimination
Continue this procedure by using the third equation as the pivot
equation and so on.
At the end of (n-1) Forward Elimination steps, the system of
equations will look like:
'
2
'
23
'
232
'
22 ... bxaxaxa nn
"3
"3
"33 ... bxaxa nn
11 nnnn
nn bxa
. .
. .
. .
11313212111 ... bxaxaxaxa nn
-
7/28/2019 Gaussian LU Eliminations Seidel
12/100
Forward Elimination
At the end of the Forward Elimination steps
)-(nnn
3
2
1
n
nn
n
n
n
b
b
b
b
x
x
x
x
a
aa
aaa
aaaa
1
"
3
'2
1
)1(
"
3
"
33
'2
'23
'22
1131211
-
7/28/2019 Gaussian LU Eliminations Seidel
13/100
Back Substitution
The goal of Back Substitution is to solve each of
the equations using the upper triangular matrix.
3
2
1
3
2
1
33
2322
131211
x
xx
00
0
b
b
b
a
aa
aaa
Example of a system of 3 equations
-
7/28/2019 Gaussian LU Eliminations Seidel
14/100
Back Substitution
Start with the last equation because it has only
one unknown
)1(
)1(
n
nn
n
n
nabx
Solve the second from last equation (n-1)th
using xn solved for previously.
This solves for xn-1.
-
7/28/2019 Gaussian LU Eliminations Seidel
15/100
Back Substitution
Representing Back Substitution for all equations
by formula
1
1
11
i
ii
n
ijjiijii
ia
xabx Fori=n-1, n-2,.,1
and
)1(
)1(
n
nn
nn
na
bx
-
7/28/2019 Gaussian LU Eliminations Seidel
16/100
Example: Rocket Velocity
The upward velocity of a rocket
is given at three different times
Time, t Velocity, v
s m/s
5 106.8
8 177.2
12 279.2
The velocity data is approximated by a polynomial as:
12.t5,322
1 atatatv
Find: The Velocity at t=6,7.5,9, and 11 seconds.
-
7/28/2019 Gaussian LU Eliminations Seidel
17/100
Example: Rocket Velocity
Assume
12.t5,atatatv 32
2
1
3
2
1
3
2
3
2
2
2
1
2
1
1
1
1
v
v
v
a
a
a
tt
tt
tt
3
2
1
Results in a matrix template of the form:
Using date from the time / velocity table, the matrix becomes:
2.279
2.177
8.106
112144
1864
1525
3
2
1
a
a
a
-
7/28/2019 Gaussian LU Eliminations Seidel
18/100
Example: Rocket Velocity
Forward Elimination: Step 1
)64(
25
12
RowRow
Yields
2.27921.96
81.106
aa
a
11214456.18.40
1525
3
2
1
-
7/28/2019 Gaussian LU Eliminations Seidel
19/100
Example: Rocket Velocity
)144(
25
13
RowRow
0.336
21.96
8.106
a
a
a
76.48.160
56.18.40
1525
3
2
1
Yields
Forward Elimination: Step 1
-
7/28/2019 Gaussian LU Eliminations Seidel
20/100
Example: Rocket Velocity
Yields
)8.16(8.4
23
RowRow
735.0
21.96
8.106
a
a
a
7.000
56.18.40
1525
3
2
1
This is now ready for Back Substitution
Forward Elimination: Step 2
-
7/28/2019 Gaussian LU Eliminations Seidel
21/100
Example: Rocket Velocity
Back Substitution: Solve fora3 using the third equation
735.07.0 3 a
70
7350
.
.a 3
0501.a 3
-
7/28/2019 Gaussian LU Eliminations Seidel
22/100
Example: Rocket Velocity
Back Substitution: Solve fora2 using the second equation
21.9656.18.4 32 aa
8.4
56.121.96 32
aa
84
05015612196
.-
...-
a 2
7019.a 2
-
7/28/2019 Gaussian LU Eliminations Seidel
23/100
Example: Rocket Velocity
Back Substitution:Solve fora1 using the first equation
8.106525 321 aaa
25
58.106 321
aaa
25
050.170.1958.106
1
a
2900.01 a
-
7/28/2019 Gaussian LU Eliminations Seidel
24/100
Example: Rocket Velocity
Solution:The solution vector is
050.1
70.19
2900.0
3
2
1
a
a
a
The polynomial that passes through the three data points is
then:
3221 atatatv
125,050.170.192900.0 2 ttt
-
7/28/2019 Gaussian LU Eliminations Seidel
25/100
Example: Rocket Velocity
Solution:
Substitute each value of t to find the corresponding velocity
.s/m1.165
050.15.770.195.72900.05.7v2
.s/m8.201
050.1970.1992900.09v2
.s/m8.252
050.11170.19112900.011v2
./69.129
050.1670.1962900.062
sm
v
-
7/28/2019 Gaussian LU Eliminations Seidel
26/100
Pitfalls
Two Potential Pitfalls-Division by zero: May occur in the forward elimination
steps. Consider the set of equations:
655
901.33099.26
7710
321
321
32
xxx
xxx
xx
- Round-off error: Prone to round-off errors.
-
7/28/2019 Gaussian LU Eliminations Seidel
27/100
Pitfalls: Example
Consider the system of equations:
Use five significant figures with chopping
515
6099.23
0710
3
2
1
x
x
x
6
901.3
7
=
At the end of Forward Elimination
1500500
6001.00
0710
3
2
1
x
x
x
15004
001.6
7
=
-
7/28/2019 Gaussian LU Eliminations Seidel
28/100
Pitfalls: Example
Back Substitution
99993.015005
150043 x
5.1001.0
6001.6 32
xx
350.010
077 321
xx
x
15004
001.67
1500500
6001.000710
3
2
1
x
xx
-
7/28/2019 Gaussian LU Eliminations Seidel
29/100
Pitfalls: Example
Compare the calculated values with the exact solution
99993.0
5.1
35.0
3
2
1
x
x
x
X calculated
1
1
0
3
2
1
x
x
x
X exact
-
7/28/2019 Gaussian LU Eliminations Seidel
30/100
Improvements
Increase the number of significant digits
Decreases round off error
Does not avoid division by zero
Gaussian Elimination with Partial Pivoting
Avoids division by zero
Reduces round off error
-
7/28/2019 Gaussian LU Eliminations Seidel
31/100
Partial Pivoting
pka
Gaussian Elimination with partial pivoting applies row switching to
normal Gaussian Elimination.
How?
At the beginning of the kth step of forward elimination, find the maximum of
nkkkkk aaa .......,,........., ,1
If the maximum of the values is In the pth row, ,npk
then switch rows p and k.
-
7/28/2019 Gaussian LU Eliminations Seidel
32/100
Partial Pivoting
What does it Mean?
Gaussian Elimination with Partial Pivoting ensures that
each step of Forward Elimination is performed with the
pivoting element |akk| having the largest absolute value.
-
7/28/2019 Gaussian LU Eliminations Seidel
33/100
Partial Pivoting: Example
Consider the system of equations
6x5xx5
901.3x3x099.2x3
7x7x10
321
321
21
In matrix form
5156099.23
0710
3
2
1
x
x
x
6901.3
7
=
Solve using Gaussian Elimination with Partial Pivoting using five
significant digits with chopping
-
7/28/2019 Gaussian LU Eliminations Seidel
34/100
Partial Pivoting: Example
Forward Elimination: Step 1
Examining the values of the first column
|10|, |-3|, and |5| or 10, 3, and 5
The largest absolute value is 10, which means, to follow therules of Partial Pivoting, we switch row1 with row1.
6
901.37
515
6099.230710
3
2
1
x
xx
5.2
001.67
55.20
6001.000710
3
2
1
x
xx
Performing Forward Elimination
-
7/28/2019 Gaussian LU Eliminations Seidel
35/100
Partial Pivoting: Example
Forward Elimination: Step 2
Examining the values of the first column
|-0.001| and |2.5| or 0.0001 and 2.5
The largest absolute value is 2.5, so row 2 is switched withrow 3
5.2
001.67
55.20
6001.000710
3
2
1
x
xx
001.6
5.27
6001.00
55.200710
3
2
1
x
xx
Performing the row swap
-
7/28/2019 Gaussian LU Eliminations Seidel
36/100
Partial Pivoting: Example
Forward Elimination: Step 2
Performing the Forward Elimination results in:
002.6
5.2
7
002.600
55.20
0710
3
2
1
x
x
x
-
7/28/2019 Gaussian LU Eliminations Seidel
37/100
Partial Pivoting: Example
Back Substitution
Solving the equations through back substitution
1002.6
002.6
3x
15.2
55.2 22
xx
010
077 321
xxx
002.6
5.2
7
002.600
55.20
0710
3
2
1
x
x
x
-
7/28/2019 Gaussian LU Eliminations Seidel
38/100
Partial Pivoting: Example
1
1
0
3
2
1
x
x
x
X exact
1
1
0
3
2
1
x
x
x
Xcalculated
Compare the calculated and exact solution
The fact that they are equal is coincidence, but it does
illustrate the advantage of Partial Pivoting
-
7/28/2019 Gaussian LU Eliminations Seidel
39/100
Summary
-Forward Elimination
-Back Substitution-Pitfalls
-Improvements
-Partial Pivoting
-
7/28/2019 Gaussian LU Eliminations Seidel
40/100
LU Decomposition
40
-
7/28/2019 Gaussian LU Eliminations Seidel
41/100
LU Decomposition
LU Decomposition is another method to solve a set of
simultaneous linear equations
Which is better, Gauss Elimination or LU Decomposition?
To answer this, a closer look at LU decomposition is
needed.
-
7/28/2019 Gaussian LU Eliminations Seidel
42/100
LU Decomposition
L
Method
A
ULA
U
For most non-singular matrix that one could conduct Nave Gauss
Elimination forward elimination steps, one can always write it as
Where
= lower triangular martix
= upper triangular martix
-
7/28/2019 Gaussian LU Eliminations Seidel
43/100
LU Decomposition
CXUL
CXA
Proof
If solving a set of linear equations
If Then
Multiply by
Which gives
Remember which leads to
Now, if then
Now, let
Which ends with
and
1L
CLXULL 11
ILL 1 CLXUI 1
UUI CLXU1
ULA
ZCL 1
CL Z
ZU X
(1)
(2)
-
7/28/2019 Gaussian LU Eliminations Seidel
44/100
LU Decomposition
CZL Z
ZU X
How can this be used?
Given
Decompose into and U L
Then solve for
And then solve for X
CXA
A
-
7/28/2019 Gaussian LU Eliminations Seidel
45/100
LU Decomposition
CZL
How is this better or faster than Gauss
Elimination?
Lets look at computational time.
n = number of equations
To decompose [A], time is proportional to
To solve and
time proportional to
3
3n
CXU
2
2n
-
7/28/2019 Gaussian LU Eliminations Seidel
46/100
LU Decomposition
Therefore, total computational time for LU Decomposition is
proportional to
23
3n
n)
2(2
3
23nn
or
Gauss Elimination computation time is proportional to
23
23nn
How is this better?
-
7/28/2019 Gaussian LU Eliminations Seidel
47/100
LU Decomposition
)2
n
3
n(m
23
)n(m3
n 23
51033.8
What about a situation where the [C] vector changes?
In LU Decomposition, LU decomposition of [A] is independent
of the [C] vector, therefore it only needs to be done once.
Let m = the number of times the [C] vector changes
The computational times are proportional to
LU decomposition = Gauss Elimination=
Consider a 100 equation set with 50 right hand side vectors
LU Decomposition = Gauss Elimination =71069.1
-
7/28/2019 Gaussian LU Eliminations Seidel
48/100
LU Decomposition
Another Advantage
Finding the Inverse of a Matrix
LU Decomposition Gauss Elimination
3
4)(
3
32
3 nnn
n
2323
3423 nnnnn
For large values of n
3
4
23
334 nnn
-
7/28/2019 Gaussian LU Eliminations Seidel
49/100
LU Decomposition
Method: [A] Decompose to [L] and [U]
33
2322
131211
3231
21
00
0
1
01
001
u
uu
uuu
ULA
[U] is the same as the coefficient matrix at the end of the forward
elimination step.
[L] is obtained using the multipliers that were used in the forwardelimination process
-
7/28/2019 Gaussian LU Eliminations Seidel
50/100
LU Decomposition
Finding the [U] matrix
Using the Forward Elimination Procedure of Gauss Elimination
112144
1864
1525
112144
56.18.40
1525
)64(25
1Row2Row
76.48.160
56.18.40
1525
)144(25
1Row3Row
-
7/28/2019 Gaussian LU Eliminations Seidel
51/100
LU Decomposition
Finding the [U] matrix
Using the Forward Elimination Procedure of Gauss Elimination
76.48.160
56.18.40
1525
7.000
56.18.40
1525
)8.16(8.4
2Row3Row
7.000
56.18.40
1525
U
-
7/28/2019 Gaussian LU Eliminations Seidel
52/100
LU Decomposition
Finding the [L] matrix
Using the multipliers used during the Forward Elimination Procedure
101
001
3231
21
56.2
25
64
11
21
21 a
a
76.525
144
11
31
31 a
a
From the first step
of forwardelimination
From the secondstep of forward
elimination
76.48.160
56.18.401525
5.38.4
8.16
22
32
32
a
a
-
7/28/2019 Gaussian LU Eliminations Seidel
53/100
LU Decomposition
15.376.5
0156.2
001
L
Does ? AUL
7.00056.18.40
1525
15.376.50156.2
001
UL
-
7/28/2019 Gaussian LU Eliminations Seidel
54/100
LU Decomposition
Example: Solving simultaneous linear equations using LU Decomposition
Solve the following set of
linear equations using LU
Decomposition
2.279
2.177
8.106
a
a
a
112144
1864
1525
3
2
1
Using the procedure for finding the [L] and [U] matrices
7.000
56.18.40
1525
15.376.5
0156.2
001
ULA
-
7/28/2019 Gaussian LU Eliminations Seidel
55/100
LU Decomposition
Example: Solving simultaneous linear equations using LU Decomposition
Set
Solve for
CZL
Z
2.279
2.177
8.106
15.376.5
0156.2
001
3
2
1
z
z
z
2.2795.376.5
2.17756.2
10
321
21
1
zzz
zz
z
-
7/28/2019 Gaussian LU Eliminations Seidel
56/100
LU Decomposition
Example: Solving simultaneous linear equations using LU Decomposition
Complete the forward substitution to solve for Z
735.0
)21.96(5.3)8.106(76.52.279
5.376.52.279
2.96
)8.106(56.22.177
56.22.177
8.106
213
12
1
zzz
zz
z
735.0
21.96
8.106
3
2
1
z
z
z
Z
-
7/28/2019 Gaussian LU Eliminations Seidel
57/100
LU Decomposition
ZXU
X
Example: Solving simultaneous linear equations using LU Decomposition
Set
Solve for
0.735
96.21-
106.8
7.000
56.18.40
1525
3
2
1
a
a
a
The 3 equations become
735.07.0
21.9656.18.4
8.106525
3
32
321
a
aa
aaa
-
7/28/2019 Gaussian LU Eliminations Seidel
58/100
LU Decomposition
Example: Solving simultaneous linear equations using LU Decomposition
From the 3rd equation
1.050
0.7
0.735a
a
3
735.07.0 3
Substituting in a3 and using the
second equation
7019.
4.8-1.0501.5696.21-
8.4
56.121.96
21.9656.18.4
3
2
32
aa
aa
-
7/28/2019 Gaussian LU Eliminations Seidel
59/100
LU Decomposition
Example: Solving simultaneous linear equations using LU Decomposition
Substituting in a3 and a2
using the first equation
2900.0
25
050.170.1958.106
25
aa58.106
8.106525
321
321
a
aaa
Hence the Solution Vector is:
050.1
70.19
2900.0
3
2
1
a
a
a
-
7/28/2019 Gaussian LU Eliminations Seidel
60/100
LU Decomposition
3
4
23
334 nnn
Finding the inverse of a square matrix
Remember, the relative computational time comparison
of LU decomposition and Gauss elimination is:
Review: The inverse [B] of a square matrix [A] is defined as
ABIBA
-
7/28/2019 Gaussian LU Eliminations Seidel
61/100
LU Decomposition
Finding the inverse of a square matrix
How can LU Decomposition be used to find the inverse?
Assume the first column of [B] to be
Using this and the definition of matrix multiplication
First column of [B] Second column of [B]
Tn112 bbb 11
0
0
1
b
b
b
A
1n
21
11
0
1
0
b
b
b
A
2n
22
12
The remaining columns in [B] can be found in the same manner
-
7/28/2019 Gaussian LU Eliminations Seidel
62/100
LU Decomposition
Example: Finding the inverse of a square matrix
112144
1864
1525
A
0.700
1.56-4.8-0
1525
15.376.5
0156.2
001
ULA
Find the inverse of [A]
Using the Decomposition procedure, the [L] and [U] matrices are found to be
-
7/28/2019 Gaussian LU Eliminations Seidel
63/100
LU Decomposition
Example: Finding the inverse of a square matrix
Solving for the each column of [B] requires to steps
1) Solve [L] [Z] = [C] for [Z] and 2) Solve [U] [X] = [Z] for [X]
Step 1: CZL
0
0
1
15.376.5
0156.2
001
3
2
1
z
z
z
This generates the equations: 11
z056.2 21 zz
05.376.5 321 zzz
-
7/28/2019 Gaussian LU Eliminations Seidel
64/100
LU Decomposition
Example: Finding the inverse of a square matrix
Solving for [Z]
2.3
56.25.3176.50
z5.3z76.50z
56.2
156.20
213
12
1
2.56z-0z
1z
2.3
56.2
1
z
z
z
Z
3
2
1
-
7/28/2019 Gaussian LU Eliminations Seidel
65/100
LU Decomposition
Example: Finding the inverse of a square matrix
Solving for [U] [X] = [Z] for [X]
3.2
2.56-
1
7.000
56.18.40
1525
31
21
11
b
b
b
1525 312111 bbb
56.256.18.4 3121 bb
2.37.0 31 b
-
7/28/2019 Gaussian LU Eliminations Seidel
66/100
LU Decomposition
Example: Finding the inverse of a square matrix
Using Backward Substitution
04762.0=25
571.4)9524.0(51=
2551=
9524.0=8.4
)571.4(560.1+56.2=
8.4
560.1+56.2=
571.4=
7.0
2.3=
3 121
1 1
3 1
2 1
3 1
bbb
bb
bSo the first column of
the inverse of [A] is:
571.4
9524.0
04762.0
31
21
11
b
b
b
-
7/28/2019 Gaussian LU Eliminations Seidel
67/100
LU Decomposition
Example: Finding the inverse of a square matrix
Repeating for the second and third columns of the inverse
Second Column Third Column
0
1
0
112144
1864
1525
32
22
12
b
b
b
000.5
417.1
08333.0
32
22
12
b
b
b
1
0
0
b
b
b
112144
1864
1525
33
23
13
429.1
4643.0
03571.0
33
23
13
b
b
b
-
7/28/2019 Gaussian LU Eliminations Seidel
68/100
LU Decomposition
Example: Finding the inverse of a square matrix
The inverse of [A] is
429.1050.5571.4
4643.0417.19524.00357.008333.04762.0
1A
To check your work do the following operation
AAIAA 11
-
7/28/2019 Gaussian LU Eliminations Seidel
69/100
Gauss-Seidel Method
69
-
7/28/2019 Gaussian LU Eliminations Seidel
70/100
Gauss-Seidel Method
An iterative method.
Basic Procedure:
-Algebraically solve each linear equation for xi
-Assume an initial guess solution array
-Solve for each xi and repeat
-Use absolute relative approximate error after each iteration
to check if error is within a pre-specified tolerance.
-
7/28/2019 Gaussian LU Eliminations Seidel
71/100
Gauss-Seidel Method
Why?
The Gauss-Seidel Method allows the user to control round-off error.
Elimination methods such as Gaussian Elimination and LU
Decomposition are prone to prone to round-off error.
Also: If the physics of the problem are understood, a close initialguess can be made, decreasing the number of iterations needed.
-
7/28/2019 Gaussian LU Eliminations Seidel
72/100
Gauss-Seidel Method
AlgorithmA set ofn equations and n unknowns:
11313212111 ... bxaxaxaxa nn
2323222121 ... bxaxaxaxa n2n
nnnnnnn bxaxaxaxa ...332211
. .
. .
. .
If: the diagonal elements arenon-zero
Rewrite each equation solvingfor the corresponding unknown
ex:
First equation, solve for x1
Second equation, solve for x2
-
7/28/2019 Gaussian LU Eliminations Seidel
73/100
Gauss-Seidel Method
AlgorithmRewriting each equation
11
13132121
1a
xaxaxacx nn
nn
nnnnnn
n
nn
nnnnnnnnn
n
nn
a
xaxaxacx
a
xaxaxaxac
x
a
xaxaxacx
11,2211
1,1
,122,122,111,11
1
22
232312122
From Equation 1
From equation 2
From equation n-1
From equation n
-
7/28/2019 Gaussian LU Eliminations Seidel
74/100
Gauss-Seidel Method
AlgorithmGeneral Form of each equation
11
11
11
1a
xac
x
n
jj
jj
22
21
22
2a
xac
x
j
n
jj
j
1,1
11
,11
1
nn
n
njj
jjnn
na
xac
x
nn
n
njj
jnjn
na
xac
x
1
-
7/28/2019 Gaussian LU Eliminations Seidel
75/100
Gauss-Seidel Method
AlgorithmGeneral Form for any row i
.,,2,1,1
nia
xac
xii
n
ijj jiji
i
How or where can this equation be used?
-
7/28/2019 Gaussian LU Eliminations Seidel
76/100
Gauss-Seidel Method
Solve for the unknowns
Assume an initial guess for [X]
n
-n
2
x
x
x
x
1
1
Use rewritten equations to solve for
each value of xi.
Important: Remember to use themost recent value of xi. Which
means to apply values calculated to
the calculations remaining in the
current iteration.
-
7/28/2019 Gaussian LU Eliminations Seidel
77/100
Gauss-Seidel Method
Calculate the Absolute Relative Approximate Error
100x
xxnew
i
old
i
new
i
ia
So when has the answer been found?
The iterations are stopped when the absolute relativeapproximate error is less than a prespecified tolerance for all
unknowns.
-
7/28/2019 Gaussian LU Eliminations Seidel
78/100
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
79/100
Gauss Seidel Method: Example
1
3
2
1
3
2
3
2
2
2
1
2
1
1
1
1
v
v
v
a
a
a
tt
tt
tt
3
2
1
Using a Matrix template of the form
The system of equations becomes
2.279
2.177
8.106
112144
1864
1525
3
2
1
a
a
a
Initial Guess: Assume an initial guess of
5
2
1
3
2
1
a
a
a
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
80/100
Gauss Seidel Method: Example
1
Rewriting each equation
2.279
2.177
8.106
112144
1864
1525
3
2
1
a
a
a
25
58.106 321
aaa
8
642.177 312
aaa
1
121442.279 213
aaa
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
81/100
Gauss Seidel Method: Example
1
Applying the initial guess and solving for ai
5
2
1
3
2
1
a
a
a 6720.325
)5()2(58.106a1
8510.7
8
56720.3642.177a2
36.155
1
8510.7126720.31442.279a3
Initial Guess
When solving for a2, how many of the initial guess values were used?
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
82/100
Gauss Seidel Method: Example
1
%76.72100x6720.3
0000.16720.31
a
%47.125100x8510.7
0000.28510.7
2
a
%22.103100x36.155
0000.536.1553
a
Finding the absolute relative approximate error
100x
xxnew
i
old
i
new
i
ia
At the end of the first iteration
The maximum absolute
relative approximate error is
125.47%
36.155
8510.7
6720.3
3
2
1
a
a
a
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
83/100
Gauss Seidel Method: Example
1
Iteration #2Using
36.155
8510.7
6720.3
3
2
1
a
a
a
056.12
25
36.1558510.758.1061
a
882.54
8
36.155056.12642.1772
a
34.798
1
882.5412056.121442.2793
a
from iteration #1
the values of ai are found:
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
84/100
Gauss Seidel Method: Example
1
Finding the absolute relative approximate error
%542.69100x056.12
6720.3056.121
a
%695.85100x
882.54
8510.7882.542
a
%54.80100x34.798
36.15534.7983
a
At the end of the second iteration
34.798882.54
056.12
3
2
1
a
a
a
The maximum absolute
relative approximate error is
85.695%
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
85/100
Gauss Seidel Method: Example
1
0858.1
690.19
29048.0
3
2
1
a
a
a
Repeating more iterations, the following values are obtained
Iteration a1 a2 a3
1
2
3
4
5
6
3.672
12.056
47.182
193.33
800.53
3322.6
72.767
67.542
74.448
75.595
75.850
75.907
-7.8510
-54.882
-255.51
-1093.4
-4577.2
-19049
125.47
85.695
78.521
76.632
76.112
75.971
-155.36
-798.34
-3448.9
-14440
-60072
-249580
103.22
80.540
76.852
76.116
75.962
75.931
%1a
%2a
%3a
! Notice The relative errors are not decreasing at any significant rate
Also, the solution is not converging to the true solution of
G S id l M th d Pitf ll
-
7/28/2019 Gaussian LU Eliminations Seidel
86/100
Gauss-Seidel Method: Pitfall
What went wrong?
Even though done correctly, the answer is not converging to the
correct answer
This example illustrates a pitfall of the Gauss-Siedel method: not all
systems of equations will converge.
Is there a fix?
One class of system of equations always converges: One with a diagonally
dominantcoefficient matrix.
Diagonally dominant: [A] in [A] [X] = [C] is diagonally dominant if:
n
jj
ijaa
i1
ii
n
ijj
ijii aa1
for all i and for at least one i
G S id l M th d Pitf ll
-
7/28/2019 Gaussian LU Eliminations Seidel
87/100
Gauss-Seidel Method: Pitfall
116123
14345
3481.52
A
Diagonally dominant: The coefficient on the diagonal must be at leastequal to the sum of the other coefficients in that row and at least one row
with a diagonal coefficient greater than the sum of the other coefficients
in that row.
1293496
55323
5634124
]B[
Which coefficient matrix is diagonally dominant?
Most physical systems do result in simultaneous linear equations that
have diagonally dominant coefficient matrices.
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
88/100
p
2
Given the system of equations
15x-3x12x 321
283x5xx 321
7613x7x3x 321
1
0
1
3
2
1
x
x
x
With an initial guess of
The coefficient matrix is:
1373
351
5312
A
Will the solution converge using the
Gauss-Siedel method?
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
89/100
p
2
1373
351
5312
A
Checking if the coefficient matrix is diagonally dominant
43155 232122 aaa
10731313 323133 aaa
8531212 131211 aaa
The inequalities are all true and at least one row is strictlygreater than:
Therefore: The solution should converge using the Gauss-Siedel Method
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
90/100
p
2
76
28
1
1373
351
5312
3
2
1
a
a
a
Rewriting each equation
12
531 321
xxx
5
328 312
xxx
13
7376 213
xxx
With an initial guess of
1
0
1
3
2
1
x
x
x
50000.0
12
150311
x
9000.4
5
135.0282
x
0923.3
13
9000.4750000.03763
x
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
91/100
p
2
The absolute relative approximate error
%662.6710050000.0
0000.150000.01a
%00.1001009000.4
09000.42a
%662.671000923.3
0000.10923.33a
The maximum absolute relative error after the first iteration is 100%
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
92/100
p
2
8118.3
7153.3
14679.0
3
2
1
x
x
x
After Iteration #1
14679.0
12
0923.359000.4311
x
7153.35
0923.3314679.0282
x
8118.3
13
900.4714679.03763
x
Substituting the x values into the equations After Iteration #2
0923.3
9000.4
5000.0
3
2
1
x
x
x
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
93/100
p
2
Iteration #2 absolute relative approximate error
%62.24010014679.0
50000.014679.01
a
%887.311007153.3
9000.47153.32
a
%876.181008118.3
0923.38118.33
a
The maximum absolute relative error after the first iteration is 240.62%
This is much larger than the maximum absolute relative error obtained in
iteration #1. Is this a problem?
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
94/100
p
2
Repeating more iterations, the following values are obtained
1a
2a
3a
Iteration a1 a
2 a
3
1
2
3
4
5
6
0.50000
0.14679
0.74275
0.94675
0.99177
0.99919
67.662
240.62
80.23
21.547
4.5394
0.74260
4.900
3.7153
3.1644
3.0281
3.0034
3.0001
100.00
31.887
17.409
4.5012
0.82240
0.11000
3.0923
3.8118
3.9708
3.9971
4.0001
4.0001
67.662
18.876
4.0042
0.65798
0.07499
0.00000
4
3
1
3
2
1
x
x
x
0001.40001.3
99919.0
3
2
1
x
x
xThe solution obtained
is close to the exact solution of
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
95/100
p
3
Given the system of equations
7613x7x3x 321
283x5xx 321
15x-3x12x 321
With an initial guess of
1
0
1
3
2
1
x
x
x
Rewriting the equations
3
13776 321
xxx
5
328 312
xxx
53121 21
3
xxx
Gauss-Seidel Method: Example
-
7/28/2019 Gaussian LU Eliminations Seidel
96/100
p
3
Conducting six iterations
1a
2a
3aIteration a1 a2 a3
1
2
34
5
6
21.000
-196.15
-1995.0-20149
2.0364x105
-2.0579x105
110.71
109.83
109.90109.89
109.90
1.0990
0.80000
14.421
-116.021204.6
-12140
1.2272x10
5
100.00
94.453
112.43109.63
109.92
109.89
5.0680
-462.30
4718.1-47636
4.8144x105
-4.8653x106
98.027
110.96
109.80109.90
109.89
109.89
The values are not converging.Does this mean that the Gauss-Seidel method cannot be used?
Gauss Seidel Method
-
7/28/2019 Gaussian LU Eliminations Seidel
97/100
Gauss-Seidel Method
The Gauss-Seidel Method can still be used
The coefficient matrix is not
diagonally dominant
5312
351
1373
A
But this is the same set of
equations used in example #2,
which did converge.
1373
351
5312
A
If a system of linear equations is not diagonally dominant, check to see if
rearranging the equations can form a diagonally dominant matrix.
Gauss Seidel Method
-
7/28/2019 Gaussian LU Eliminations Seidel
98/100
Gauss-Seidel Method
Not every system of equations can be rearranged to have adiagonally dominant coefficient matrix.
Observe the set of equations
3321 xxx9432 321 xxx
97 321 xxx
Which equation(s) prevents this set of equation from having a
diagonally dominant coefficient matrix?
Gauss Seidel Method
-
7/28/2019 Gaussian LU Eliminations Seidel
99/100
Gauss-Seidel Method
Summary
-Advantages of the Gauss-Seidel Method
-Algorithm for the Gauss-Seidel Method
-Pitfalls of the Gauss-Seidel Method
-
7/28/2019 Gaussian LU Eliminations Seidel
100/100