![Page 1: Introduction to Numerical Analysis for Engineers · PDF file13.002 Numerical Methods for Engineers Lecture 3 Systems of Linear Equations Gaussian Elimination Reduction Step 1 i j](https://reader030.vdocuments.site/reader030/viewer/2022021423/5ab8ef507f8b9aa6018d3c2c/html5/thumbnails/1.jpg)
Numerical Methods for Engineers13.002 Lecture 3
Introduction to Numerical Analysis for Engineers
• Systems of Linear EquationsMathews
– Cramer’s Rule– Gaussian Elimination 3.3-3.5
• Numerical implementation3.3-3.4
• Numerical stability– Partial Pivoting– Equilibration– Full Pivoting
• Multiple right hand sides• Computation count• LU factorization 3.5• Error Analysis for Linear Systems 3.4
– Condition Number• Special Matrices
– Iterative Methods 3.6• Jacobi’s method• Gauss-Seidel iteration• Convergence
![Page 2: Introduction to Numerical Analysis for Engineers · PDF file13.002 Numerical Methods for Engineers Lecture 3 Systems of Linear Equations Gaussian Elimination Reduction Step 1 i j](https://reader030.vdocuments.site/reader030/viewer/2022021423/5ab8ef507f8b9aa6018d3c2c/html5/thumbnails/2.jpg)
Numerical Methods for Engineers13.002 Lecture 3
Systems of Linear EquationsCramer’s Rule
Cramer’s Rule, n=2
Linear System of Equations
Example, n=2
Cramer’s rule inconvenient for n>3
![Page 3: Introduction to Numerical Analysis for Engineers · PDF file13.002 Numerical Methods for Engineers Lecture 3 Systems of Linear Equations Gaussian Elimination Reduction Step 1 i j](https://reader030.vdocuments.site/reader030/viewer/2022021423/5ab8ef507f8b9aa6018d3c2c/html5/thumbnails/3.jpg)
Numerical Methods for Engineers13.002 Lecture 3
Systems of Linear EquationsGaussian Elimination
ReductionStep 0Linear System of Equations
![Page 4: Introduction to Numerical Analysis for Engineers · PDF file13.002 Numerical Methods for Engineers Lecture 3 Systems of Linear Equations Gaussian Elimination Reduction Step 1 i j](https://reader030.vdocuments.site/reader030/viewer/2022021423/5ab8ef507f8b9aa6018d3c2c/html5/thumbnails/4.jpg)
Numerical Methods for Engineers13.002 Lecture 3
Systems of Linear EquationsGaussian Elimination
ReductionStep 1
i
j
![Page 5: Introduction to Numerical Analysis for Engineers · PDF file13.002 Numerical Methods for Engineers Lecture 3 Systems of Linear Equations Gaussian Elimination Reduction Step 1 i j](https://reader030.vdocuments.site/reader030/viewer/2022021423/5ab8ef507f8b9aa6018d3c2c/html5/thumbnails/5.jpg)
Numerical Methods for Engineers13.002 Lecture 3
Systems of Linear EquationsGaussian Elimination
ReductionStep n-1
ReductionStep k
Back-Substitution
![Page 6: Introduction to Numerical Analysis for Engineers · PDF file13.002 Numerical Methods for Engineers Lecture 3 Systems of Linear Equations Gaussian Elimination Reduction Step 1 i j](https://reader030.vdocuments.site/reader030/viewer/2022021423/5ab8ef507f8b9aa6018d3c2c/html5/thumbnails/6.jpg)
Numerical Methods for Engineers13.002 Lecture 3
Systems of Linear EquationsGaussian Elimination
Pivotal Elements
Step k Partial Pivoting by Columns
Row k
Row i
Required at each step!
![Page 7: Introduction to Numerical Analysis for Engineers · PDF file13.002 Numerical Methods for Engineers Lecture 3 Systems of Linear Equations Gaussian Elimination Reduction Step 1 i j](https://reader030.vdocuments.site/reader030/viewer/2022021423/5ab8ef507f8b9aa6018d3c2c/html5/thumbnails/7.jpg)
Numerical Methods for Engineers13.002 Lecture 3
Systems of Linear EquationsGaussian Elimination
Pivotal Elements
ReductionStep k
Partial Pivoting by Columns
New Row k
New Row i
Required at each step!
![Page 8: Introduction to Numerical Analysis for Engineers · PDF file13.002 Numerical Methods for Engineers Lecture 3 Systems of Linear Equations Gaussian Elimination Reduction Step 1 i j](https://reader030.vdocuments.site/reader030/viewer/2022021423/5ab8ef507f8b9aa6018d3c2c/html5/thumbnails/8.jpg)
Numerical Methods for Engineers13.002 Lecture 3
Systems of Linear EquationsGaussian Elimination
Example, n=2
Cramer’s Rule - Exact
Gaussian Elimination
2-digit Arithmetic
1% error
100% error
n=3a = [ [0.01 1.0]' [-1.0 0.01]']b= [1 1]'r=a^(-1) * bx=[0 0];m21=a(2,1)/a(1,1);a(2,1)=0;a(2,2) = radd(a(2,2),-m21*a(1,2),n);b(2) = radd(b(2),-m21*b(1),n);x(2) = b(2)/a(2,2);x(1) = (radd(b(1), -a(1,2)*x(2),n))/a(1,1);x'
tbt.m
tbt.m
![Page 9: Introduction to Numerical Analysis for Engineers · PDF file13.002 Numerical Methods for Engineers Lecture 3 Systems of Linear Equations Gaussian Elimination Reduction Step 1 i j](https://reader030.vdocuments.site/reader030/viewer/2022021423/5ab8ef507f8b9aa6018d3c2c/html5/thumbnails/9.jpg)
Numerical Methods for Engineers13.002 Lecture 3
Systems of Linear EquationsGaussian Elimination
Example, n=2
Cramer’s Rule - Exact
Partial Pivoting by ColumnsInterchange Rows
2-digit Arithmetic
1% error
1% error
![Page 10: Introduction to Numerical Analysis for Engineers · PDF file13.002 Numerical Methods for Engineers Lecture 3 Systems of Linear Equations Gaussian Elimination Reduction Step 1 i j](https://reader030.vdocuments.site/reader030/viewer/2022021423/5ab8ef507f8b9aa6018d3c2c/html5/thumbnails/10.jpg)
Numerical Methods for Engineers13.002 Lecture 3
Systems of Linear EquationsGaussian Elimination
Example, n=2
Cramer’s Rule - Exact
2-digit Arithmetic
Multiply Equation 1 by 200
1% error
100% error
Infinity-Norm NormalizationEquations must be normalized for partial pivoting to ensure stability
This Equilibration is made by normalizing the matrix to unit norm
Two-Norm Normalization
![Page 11: Introduction to Numerical Analysis for Engineers · PDF file13.002 Numerical Methods for Engineers Lecture 3 Systems of Linear Equations Gaussian Elimination Reduction Step 1 i j](https://reader030.vdocuments.site/reader030/viewer/2022021423/5ab8ef507f8b9aa6018d3c2c/html5/thumbnails/11.jpg)
Numerical Methods for Engineers13.002 Lecture 3
Systems of Linear EquationsGaussian Elimination
Example, n=2
Cramer’s Rule - Exact
Interchange Unknowns
2-digit Arithmetic
1% error
Pivoting by Rows
Full PivotingFind largest numerical value in same row and column and interchange
Affects ordering of unknowns
![Page 12: Introduction to Numerical Analysis for Engineers · PDF file13.002 Numerical Methods for Engineers Lecture 3 Systems of Linear Equations Gaussian Elimination Reduction Step 1 i j](https://reader030.vdocuments.site/reader030/viewer/2022021423/5ab8ef507f8b9aa6018d3c2c/html5/thumbnails/12.jpg)
Numerical Methods for Engineers13.002 Lecture 3
• Partial Pivoting– Equilibrate system of equations– Pivoting by Columns– Simple book-keeping
• Solution vector in original order
• Full Pivoting– Does not require equilibration– Pivoting by both row and columns– More complex book-keeping
• Solution vector re-ordered
Systems of Linear EquationsGaussian Elimination
Numerical Stability
Partial Pivoting is simplest and most commonNeither method guarantees stability
![Page 13: Introduction to Numerical Analysis for Engineers · PDF file13.002 Numerical Methods for Engineers Lecture 3 Systems of Linear Equations Gaussian Elimination Reduction Step 1 i j](https://reader030.vdocuments.site/reader030/viewer/2022021423/5ab8ef507f8b9aa6018d3c2c/html5/thumbnails/13.jpg)
Numerical Methods for Engineers13.002 Lecture 3
Systems of Linear EquationsGaussian Elimination
Example, n=2
Cramer’s Rule - Exact
Variable Transformation
2-digit Arithmetic1% error
100% error
![Page 14: Introduction to Numerical Analysis for Engineers · PDF file13.002 Numerical Methods for Engineers Lecture 3 Systems of Linear Equations Gaussian Elimination Reduction Step 1 i j](https://reader030.vdocuments.site/reader030/viewer/2022021423/5ab8ef507f8b9aa6018d3c2c/html5/thumbnails/14.jpg)
Numerical Methods for Engineers13.002 Lecture 3
• System of equations must be well conditioned– Investigate condition number
• Tricky, because it requires matrix inversion (next class)– Consistent with physics
• E.g. don’t couple domains that are physically uncoupled– Consistent units
• E.g. don’t mix meter and m in unknowns– Dimensionless unknowns
• Normalize all unknowns consistently
• Equilibration and Partial Pivoting, or Full Pivoting
Systems of Linear EquationsGaussian Elimination
How to Ensure Numerical Stability