ee3561_unit 3(c)al-dhaifallah14351 ee 3561 : - computational methods unit 3: solution of systems of...
TRANSCRIPT
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 1
EE 3561 : - Computational MethodsUnit 3:
Solution of Systems of Linear Equations
Mujahed AlDhaifallah (Term 342)
Read Chapter 9 of the textbook
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 2
Systems of linear equations
form Matrix form Standard
7
5
3
601
315.2
342
76
535.2
3342
formsdifferent in
presented be can equationslinear of system A
3
2
1
31
321
321
x
x
x
xx
xxx
xxx
Coefficient Matrix
Unknown Vector
RHS Vector
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 3
Solutions of linear equations
52
3
equations following the tosolutiona is 2
1
21
21
2
1
xx
xx
x
x
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 4
Solutions of linear equations A set of equations is inconsistent if there exist
no solution to the system of equations
ntinconsiste are equations These
542
32
21
21
xx
xx
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 5
Solutions of linear equations Some systems of equations may have infinite
number of solutions
allfor solutionais)3(5.0
solutions ofnumber infinite have
642
32
2
1
21
21
aa
a
x
x
xx
xx
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 6
Graphical Solution of Systems ofLinear Equations
52
3
21
21
xx
xx
solution
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 7
Cramer’s Rule is not practical
way efficient in computed are tsdeterminan theif used becan It
system. 30by 30 a solve toyears10 needscomputer super A
. systems largefor practicalnot is Rule sCramer'
2
21
11
51
31
,1
21
11
25
13
system thesolve toused becan Rule sCramer'
17
21 xx
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 8
Naive Gaussian Elimination Examples
Lecture 6 Naive Gaussian
Elimination
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 9
Naive Gaussian Elimination The method consists of two steps
Forward Elimination: the system is reduced to upper triangular form. A sequence of elementary operations is used.
Backward substitution: Solve the system starting from the last variable.
'
'
'00
''0
3
2
1
3
2
1
33
2322
131211
3
2
1
3
2
1
333231
232221
131211
b
b
b
x
x
x
a
aa
aaa
b
b
b
x
x
x
aaa
aaa
aaa
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 10
Elementary Row operations
Adding a multiple of one row to another Multiply any row by a non-zero constant
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 11
ExampleForward Elimination
18
27
6
16
14320
18120
2240
4226
4,3,2equationsfrom Eliminate:Step1
nEliminatio Forward:1Part
34
19
26
16
18146
39133
106812
4226
4
3
2
1
1
4
3
2
1
x
x
x
x
x
x
x
x
x
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 12
Example Forward Elimination
3
9
6
16
3000
5200
2240
4226
4equation from Eliminate:Step3
21
9
6
16
13400
5200
2240
4226
4,3equationsfrom Eliminate:Step2
4
3
2
1
3
4
3
2
1
2
x
x
x
x
x
x
x
x
x
x
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 13
Example Forward Elimination
3
9
6
16
3000
5200
2240
4226
34
19
26
16
18146
39133
106812
4226
nEliminatio Forward theofSummary
4
3
2
1
4
3
2
1
x
x
x
x
x
x
x
x
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 14
Example Backward substitution
36
)1(4)2(2)1(216,1
4
)1(2)2(26
22
59,1
3
3
for solve,...for solvethen,for solve
3
9
6
16
3000
5200
2240
4226
12
34
134
4
3
2
1
xx
xx
xxx
x
x
x
x
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 15
Forward Elimination
ni
ba
abb
njaa
aaa
x
ni
ba
abb
njaa
aaa
x
ijj
ji
ijij
ijj
ji
ijij
3
)2(
eliminate To
2
)1(
eliminate To
222
2
222
2
2
111
1
111
1
1
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 16
Forward Elimination
.eliminated is until continue
1
)(
eliminate To
1
n
mmm
imjj
mjmm
imijij
m
x
nim
ba
abb
njmaa
aaa
x
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 17
Backward substitution
mm
n
mjjjmm
m
nn
nnnnnnnn
nn
nnnnn
nn
nn
a
xab
x
a
xaxabx
a
xabx
a
bx
,
1,
2,2
11,2,222
1,1
,111
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 18
How many solutions does a system of equations AX=B have?
0 elements0 elements
Bingcorrespond Bingcorrespond
rows zerorows zero
moreor one has moreor one hasrows zero no has
matrix reducedmatrix reducedmatrix reduced
0det(A)0det(A)0det(A)
infintesolution NoUnique
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 19
How do we know if a solution is good or not Given AX=B
X is a solution if AX-B=0 Due to computation error AX-B may not be zero Compute the residuals R=|AX-B|
One possible test is ?????
iirmax if acceptable issolution The
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 20
Determinant
13detdet
1300
410
321
A'
213
232
321
A
:Example
tdeterminan affect thenot does operations elementary The
operations Elementary
(A')(A)
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 21
Linear Systems Types
5.1!10
5.0
0
#:
0
2
00
21
1
2
00
21
1
1
20
21
4
2
42
21
3
2
42
21
2
1
43
21
solutions of # infintesolution NoUnique
XimpossibleX
solutionsInfinitesolutionNosolution
XXX
XXX
Singular Systems
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 22
Detecting Singularity After completing the elimination step, the
determinant can be evaluated as the product of the diagonal elements.
One can detect singularity by the fact that the determinant of a singular system is zero.
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 23
Problems with Naive Gaussian Elimination
o The Naive Gaussian Elimination may fail for very simple cases. (The pivoting element is zero).
o Very small pivoting element may result in , serious computation errors
2
1
11
10
2
1
x
x
2
1
11
110
2
110
x
x
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 24
Possible solution Equations Permutation: Naive Gaussian
elimination method works well in the above two examples if the equations are first permuted, i.e., arranged as
The procedure that will do so is called “Gaussian elimination with scaled partial pivoting”.
1
2
10
11
2
1
x
x
1
2
110
11
2
1
10 x
x
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 25
Gaussian Elimination with Scaled Partial Pivoting In Gaussian elimination method, the order
in which the equations are used as pivoting equations is the natural order {1, 2, 3, · · · , n}.
To overcome the problems that Naive Guassian elimination procedure face, we choose an order which is different than the natural used in forward elimination method. This is called partial pivoting.
The most important step in this method is to determine the pivot equation
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 26
Gaussian Elimination with Scaled Partial
Pivoting
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 27
Scaled Partial Pivoting Procedure1. Let l0 = {1, 2, 3, · · · , n}. This vector is called the
“index vector”.2. Define . This vector is
called the “Scale Vector” and is fixed for all operations. This means that si =absolute value of the maximum element in row i.
3. For iteration 1, define ratio#1 as That is divide the absolute value of the elements
of the first column by the corresponding elements in the “Scaled Vector”.
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 28
Scaled Partial Pivoting Procedure4. Choose the equation which give the greatest
ratio as the first pivoting equation. Assume the greatest ratio is
5. Set the new index vector to be
i.e., interchange the place of 1 and i in l0 to get l1.
6. Then do the elimination as in the Naive Gaussian elimination method by taking raw i as the pivot raw and aii as the pivot element.
7. Repeat steps 3 to 6 for n − 1 iterations.
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 29
Example 2
1
1
1
1
3524
3685
4123
1211
Pivoting Partial Scaledwith
nEliminatioGaussian using sytstem following theSolve
4
3
2
1
x
x
x
x
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 30
Example 2Initialization step
4321LVectorIndex
5842S vectorScale
1
1
1
1
3524
3685
4123
1211
4
3
2
1
x
x
x
xScale vector:
disregard sign
find largest in magnitude in each row
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 31
Why index vector? Index vectors are used because it is much
easier to exchange a single index element compared to exchanging the values of a complete row.
In practical problems with very large N, exchanging the contents of rows may not be practical since they could be stored at different locations.
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 32
Example 2Forward Elimination-- Step 1: eliminate x1
]1324[
Exchangeequation pivot first theis 4equation
toscorrespondmax 5
4,
8
5,
4
3,
2
14,3,2,1
]4321[
]5842[
1
1
1
1
3524
3685
4123
1211
equationpivot theofSelection
14
4
1,
4
3
2
1
L
landl
liS
aRatios
L
S
x
x
x
x
i
i
l
l
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 33
Example 2Forward Elimination-- Step 1: eliminate x1
1
25.2
75.1
25.1
3524
75.025.05.50
75.175.25.00
25.075.05.10
1
1
1
1
3524
3685
4123
1211
B andA Update
4
3
2
1
4
3
2
1
x
x
x
x
x
x
x
x
First pivot equation
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 34
Example 2Forward Elimination-- Step 2: eliminate x2
]2314[
2
5.1
8
5.5
4
5.0:Ratios
1
25.2
75.1
25.1
3524
75.025.05.50
75.175.25.00
25.075.05.10
equationpivot second theofSelection
4
3
2
1
L
x
x
x
x
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 35
Example 2Forward Elimination-- Step 2: eliminate x2
]2314[2
5.1
8
5.5
4
5.04,3,2:Ratios
]1324[]5842[
1
25.2
75.1
25.1
3524
75.025.05.50
75.175.25.00
25.075.05.10
equationpivot second theofSelection
2,
4
3
2
1
LiS
a
LS
x
x
x
x
i
i
l
l
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 36
Example 2Forward Elimination-- Step 3: eliminate x3
1
9
1667.2
25.1
3524
2000
8333.15.200
25.075.05.10
]3214[
1
8333.6
1667.2
25.1
3524
1667.05.200
8333.15.200
25.075.05.10
4
3
2
1
4
3
2
1
x
x
x
x
L
x
x
x
xThird pivot equation
Determinant =
(-1)3*(-1.5)(-2.5)(2)(4)
= 30-
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 37
Example 2Backward substitution
7.23334
2531
1.13335.1
75.025.025.1
2.43275.2
8333.11667.2,5.4
2
9
]3214[
1
9
1667.2
25.1
3524
2000
8333.15.200
25.075.05.10
234
1,
22,33,44,1
34
2,
33,44,2
4
3,
44,3
4,4
4
3
2
1
1
1111
2
222
3
33
4
4
xxx
a
xaxaxabx
xx
a
xaxabx
x
a
xabx
a
bx
L
x
x
x
x
l
llll
l
lll
l
ll
l
l
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 38
Example 3
1
1
1
1
3524
3685
4123
1211
Pivoting Partial Scaledwith
nEliminatioGaussian using sytstem following theSolve
4
3
2
1
x
x
x
x
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 39
Example 3Initialization step
4321LVectorIndex
5842S vectorScale
1
1
1
1
3524
3685
4123
1211
4
3
2
1
x
x
x
x
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 40
Example 3Forward Elimination-- Step 1: eliminate x1
]1324[
Exchangeequation pivot first theis 4equation
toscorrespondmax 5
4,
8
5,
4
3,
2
14,3,2,1
]4321[
]5842[
1
1
1
1
3524
3685
4123
1211
equationpivot theofSelection
14
41,
4
3
2
1
L
landl
liS
aRatios
L
S
x
x
x
x
i
i
l
l
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 41
Example 3Forward Elimination-- Step 1: eliminate x1
1
25.2
75.1
25.1
3524
75.025.05.100
75.175.25.00
25.075.05.10
1
1
1
1
3524
3685
4133
1211
B andA Update
4
3
2
1
4
3
2
1
x
x
x
x
x
x
x
x
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 42
Example 3Forward Elimination-- Step 2: eliminate x2
]1234[2
5.1
8
5.10
4
5.04,3,2:Ratios
]1324[]5842[
1
25.2
75.1
25.1
3524
75.025.05.100
75.175.25.00
25.075.05.10
equationpivot second theofSelection
2,
4
3
2
1
LiS
a
LS
x
x
x
x
i
i
l
l
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 43
Example 3Forward Elimination-- Step 2: eliminate x2
1
2.25
1.8571
0.9286
3524
75.025.05.100
1.71432.7619-00
0.35710.785700
]2314[
1
25.2
75.1
25.1
3524
75.025.05.100
75.175.25.00
25.075.05.10
B andA Updating
4
3
2
1
4
3
2
1
x
x
x
x
L
x
x
x
x
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 44
Example 3Forward Elimination-- Step 3: eliminate x3
]1234[2
0.7857
4
2.76194,3:Ratios
]1234[]5842[
1
2.25
1.8571
0.9286
3524
75.025.05.100
1.71432.761900
0.35710.785700
equationpivot third theofSelection
3,
4
3
2
1
LiS
a
LS
x
x
x
x
i
i
l
l
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 45
Example 3Forward Elimination-- Step 3: eliminate x3
1
2.25
1.8571
1.4569
3524
75.025.05.100
1.71432.761900
0.8448000
]1234[
1
2.25
1.8571
0.9286
3524
75.025.05.100
1.71432.761900
0.35710.785700
4
3
2
1
4
3
2
1
x
x
x
x
L
x
x
x
x
Determinant =
(-1)2*(0.8448)* (-2.7619)*(-10.5)*(4)
=97.9966
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 46
Example 3Backward substitution
1.86734
2531
0.3469
0.39802.7619
1.71431.8571,1.7245
0.8448
1.4569
]1234[
1
2.25
1.8571
1.4569
3524
75.025.05.100
1.71432.761900
0.8448000
234
1,
22,33,44,1
2,
33,44,2
4
3,
44,3
4,4
4
3
2
1
1
1111
2
222
3
33
4
4
xxx
a
xaxaxabx
a
xaxabx
x
a
xabx
a
bx
L
x
x
x
x
l
llll
l
lll
l
ll
l
l
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 47
How good is the solution?
0005.0
0002.0
0003.0
0001.0
:Residues
7245.1
3980.0
3469.0
8673.1
solution
1
1
1
1
3524
3685
4123
1211
4
3
2
1
4
3
2
1
R
x
x
x
x
x
x
x
x
EE3561_Unit 3 (c)AL-DHAIFALLAH1435 48
Remarks: We use index vector to avoid the need to move
the rows which may not be practical for large problems.
If you order equation as in the last value of the index vector, you have triangular form.
Scale vector is formed by taking maximum in magnitude in each row.
Scale vector does not change. The original matrices A and B are used in
Checking the residuals.