solving a system of linear equations · solving a system of linear equations ... linear equations...

15
SOLVING A SYSTEM OF LINEAR EQUATIONS 1 Introduction In the previous chapter, we determined the value x that satisfied a single equation, f (x) = 0. Now, we deal with the case of determining the values x 1 ,x 2 , ..., x n that simultaneously satisfy a set of linear equations of the form a 11 x 1 + a 12 x 2 + .....,a 1n x n = b 1 a 21 x 1 + a 22 x 2 + .....,a 2n x n = b 2 a n1 x 1 + a n2 x 2 + .....,a nn x n = b n For small numbers of equations (n 3), linear equations can be solved readily by simple techniques. However, for four or more equations, solutions become tedious and computers must be utilized. The system of equations introduced above can be represented in a compact form [A]X = B where [A] is an n × n matrix of coefficients a 11 a 12 .... a 1n a 21 a 22 ... a 2n a n1 a n2 ... a nn B is the n × 1 column vector of constants, and X is the n × 1 column vector of unknowns: B = b 1 b 2 b n X = x 1 x 2 x n 1.1 Examples Examples of engineering problems that require a solution of a system of linear equations are: 1. Referring to fig.1 and using Kirchhoffs law, the currents i 1 , i 2 ,i 3 , and i 4 can be determined by solving the following system of four equations: 9i 1 - 4i 2 - 2i 3 = 24 -4i 1 + 17i 2 - 6i 3 - 3i 4 = -16 -2i 1 - 6i 2 + 14i 3 - 6i 4 =0 -3i 2 - 6i 3 + 11i 4 = 18 1

Upload: ledang

Post on 26-Jul-2018

242 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: SOLVING A SYSTEM OF LINEAR EQUATIONS · SOLVING A SYSTEM OF LINEAR EQUATIONS ... linear equations can be solved readily by simple ... Examples of engineering problems that require

SOLVING A SYSTEM OF LINEAR EQUATIONS

1 Introduction

In the previous chapter, we determined the value x that satisfied a single equation, f(x) = 0. Now, we dealwith the case of determining the values x1, x2, ..., xn that simultaneously satisfy a set of linear equationsof the form

a11x1 + a12x2 + ....., a1nxn = b1

a21x1 + a22x2 + ....., a2nxn = b2

an1x1 + an2x2 + ....., annxn = bn

For small numbers of equations (n ≤ 3), linear equations can be solved readily by simple techniques.However, for four or more equations, solutions become tedious and computers must be utilized.

The system of equations introduced above can be represented in a compact form

[A]X = B

where [A] is an n× n matrix of coefficientsa11 a12 .... a1na21 a22 ... a2n

an1 an2 ... ann

B is the n× 1 column vector of constants, and X is the n× 1 column vector of unknowns:

B =

b1b2

bn

X =

x1x2

xn

1.1 Examples

Examples of engineering problems that require a solution of a system of linear equations are:

1. Referring to fig.1 and using Kirchhoffs law, the currents i1, i2,i3, and i4 can be determined by solvingthe following system of four equations:

9i1 − 4i2 − 2i3 = 24

−4i1 + 17i2 − 6i3 − 3i4 = −16

−2i1 − 6i2 + 14i3 − 6i4 = 0

−3i2 − 6i3 + 11i4 = 18

1

Page 2: SOLVING A SYSTEM OF LINEAR EQUATIONS · SOLVING A SYSTEM OF LINEAR EQUATIONS ... linear equations can be solved readily by simple ... Examples of engineering problems that require

Fig.1

1.2 Overview of numerical methods for solving a system of linear algebraicequations

Two types of numerical methods, direct and iterative, are used for solving systems of linear algebraicequations. In direct methods, the solution is calculated by performing arithmetic operations with theequations. In iterative methods, an initial approximate solution is assumed and then used in an iterativeprocess for obtaining successively more accurate solutions.

1.2.1 Direct methods

In direct methods, the system of equations that is initially given in the general form, is manipulated to anequivalent system of equations that can be easily solved. Three systems of equations that can be easilysolved are the upper triangular, lower triangular, and diagonal forms.

• The upper triangular form is shown in fig.2. The system in this form has all zero coefficients belowthe diagonal and is solved by a procedure called back substitution. It starts with the last equation,which is solved for xn . The value of xn is then substituted in the next-to-the-last equation, which issolved for xn−1. The process continues in the same manner all the way up to the first equation.

Fig.2

xn =bnann

xi =

bi −n∑

j=i+1

aijxj

aiii = n− 1, n− 2, ...., 1

• The lower triangular form is shown in fig.3. The system in this form has zero coefficients above thediagonal. A system in lower triangular form is solved in the same way as the upper triangular formbut in an opposite order. The procedure is called forward substitution. It starts with the firstequation, which is solved for x1. The value of x1 is then substituted in the second equation, which issolved for x2. The process continues in the same manner all the way down to the last equation.

2

Page 3: SOLVING A SYSTEM OF LINEAR EQUATIONS · SOLVING A SYSTEM OF LINEAR EQUATIONS ... linear equations can be solved readily by simple ... Examples of engineering problems that require

Fig.3

x1 =b1a11

xi =

bi −i−1∑j=1

aijxj

aiii = 2, 3, ...., n

• The diagonal form of a system of linear equations is shown in fig.4. A system in diagonal form hasnonzero coefficients along the diagonal and zeros everywhere else. Obviously, a system in this formcan be easily solved.

Fig.4

xi =biaii

• Three direct methods for solving systems of equations: Gauss elimination, GaussJordan, andLU decomposition are described in this chapter.

1.2.2 Indirect Methods

Two indirect (iterative) methods, Jacobi and GaussSeidel are described in this chapter.

2 Direct Methods

2.1 Naive Gauss elimination method

This section includes the systematic techniques for forward elimination and back substitution that compriseGauss elimination. Although these techniques are ideally suited for implementation on computers, somemodifications will be required to obtain a reliable algorithm. In particular, the computer program mustavoid division by zero. The following method is called naive Gauss elimination because it does not avoidthis problem. Subsequent sections will deal with the additional features required for an effective computerprogram.

The approach is designed to solve a general set of n equations: The technique consists of two phases:elimination of unknowns and solution through back substitution.

3

Page 4: SOLVING A SYSTEM OF LINEAR EQUATIONS · SOLVING A SYSTEM OF LINEAR EQUATIONS ... linear equations can be solved readily by simple ... Examples of engineering problems that require

a11x1 + a12x2 + .....+ a1nxn = b1

a21x1 + a22x2 + .....+ a2nxn = b2

an1x1 + an2x2 + .....+ annxn = bn

2.1.1 Forward elimination of unknowns

The first phase is designed to reduce the set of equations to an upper triangular system. The initial stepwill be to eliminate the first unknown, x1, from the second through the nth equations. To do this, multiplythe first rwo in the previous equation by a21/a11 to give

a21x1 +a21a11

a12x2 + +a21a11

a1nxn =a21a11

b1

Now, this equation can be subtracted from the second row to give(a22 −

a21a11

a12

)x2 + ........+

(a2n −

a21a11

a1n

)xn = b2 −

a21a11

b1

or

a′22x2 + ......+ a′2nxn = b′2

where the prime indicates that the elements have been changed from their original values. The procedureis then repeated for the remaining equations. For instance, teh first row can be multiplied by a31/a11 andthe result subtracted from the third equation. Repeating the procedure for the remaining equations resultsin the following modified system:

a11x1 + a12x2 + ....., a1nxn = b1

a′22x2 + .....+ a′2nxn = b′2

a′n2x2 + .....+ a′nnxn = b′n

For the foregoing steps, row one is called the pivot equation and a11 is called the pivot coefficient orelement. Note that the process of multiplying the first row by a21/a11 is equivalent to dividing it by a11and multiplying it by a21. Sometimes the division operation is referred to as normalization. We make thisdistinction because a zero pivot element can interfere with normalization by causing a division by zero.We will return to this important issue after we complete our description of naive Gauss elimination.

Now repeat the above to eliminate the second unknown from row 3 through n in the last set of equations.To do this multiply the second row by a′32/a

′22 and subtract the result from the third equation. Perform a

similar elimination for the remaining equations to yield

a11x1 + a12x2 + a13x3 + ....., a1nxn = b1

a′22x2 + a′23x3 + ....+ a′2nxn = b′2

a′′33x3 + ...+ a′′2nxn = b′′”3

a′′n2x3 + ..+ a′′nnxn = b′′n

where the double prime indicates that the elements have been modified twice.

4

Page 5: SOLVING A SYSTEM OF LINEAR EQUATIONS · SOLVING A SYSTEM OF LINEAR EQUATIONS ... linear equations can be solved readily by simple ... Examples of engineering problems that require

The procedure can be continued using the remaining pivot equations. The final manipulation in thesequence is to use the (n− 1)th equation to eliminate the xn−1 term from the nth equation. At this point,the system will have been transformed to an upper triangular system:

a11x1 + a12x2 + a13x3 + ....., a1nxn = b1

a′22x2 + a′23x3 + ....+ a′2nxn = b′2

a′′33x3 + ...+ a′′2nxn = b′′”3

a(n−1)nn xn = b

(n−1)n

2.1.2 back substitution

xn =b(n−1)n

a(n−1)nn

xi =

b(i−1)i −

n∑j=i+1

a(i−1)ij xj

a(i−1)ii

i = n− 1, n− 2, ...., 1

Example:

Consider solving the following system using the naive Gauss elimination3x1 − 2x2 + 5x3 = 14

x1 − x2 = −1

2x1 + 4x3 = 14

2.1.3 Potential difficulties when applying the Gauss elimination method

Whereas there are many systems of equations that can be solved with naive Gauss elimination, there aresome pitfalls that must be explored before writing a general computer program to implement the method.

• Division by zero

The primary reason that the foregoing technique is called naive is that during both the eliminationand the back-substitution phases, it is possible that a division by zero can occur. For example, if weuse naive Gauss elimination to solve

2x2 + 3x3 = 8

4x1 + 6x2 + 7x3 = 3

2x1 + x2 + 6x3 = 5

the normalization of the first row would involve division by a11 = 0. Problems also can arise when acoefficient is very close to zero due to round off errors. The technique of pivoting has been developedto partially avoid these problems. It will be described in the next section. For example, considerusing Gauss elimination to solve

5

Page 6: SOLVING A SYSTEM OF LINEAR EQUATIONS · SOLVING A SYSTEM OF LINEAR EQUATIONS ... linear equations can be solved readily by simple ... Examples of engineering problems that require

0.0003x1 + 3.0000x2 = 2.0001

1.0000x1 + 1.0000x2 = 1.0000

Note that in this form the first pivot element, a11 = 0.0003, is very close to zero. The exact solutionis x1 = 1/3 and x2 = 2/3.

Multiplying the first equation by 1 /(0.0003) yields

x1 + 10, 000x2 = 6667

which can be used to eliminate x1 from the second equation:

9999x2 = 6666

which can be solved for x2 = 2/3. This result can be substituted back into the first equation toevaluate x1:

x1 =2.0001− 3(2/3)

0.0003

However, due to subtractive cancellation, the result is very sensitive to the number of significantfigures carried in the computation:

Significant x2 x1figures

3 0.667 -3.334 0.6667 0.00005 0.66667 0.300006 0.666667 0.3300007 0.6666667 0.3330000

Note how the solution for x1 is highly dependent on the number of significant figures. On the otherhand, if the equations are solved in reverse order, the row with the larger pivot element is normalized.The equations are 1.0000x1 + 1.0000x2 = 1.0000

0.0003x1 + 3.0000x2 = 2.0001

Elimination and substitution yield x2 = 2/3. For different numbers of significant figures, x1 can becomputed from the first equation, as in

x1 =1− (2/3)

1

This case is much less sensitive to the number of significant figures in the computation:

Significant x2 x1figures

3 0.667 0.3334 0.6667 0.33335 0.66667 0.333336 0.666667 0.3333337 0.6666667 0.3333333

6

Page 7: SOLVING A SYSTEM OF LINEAR EQUATIONS · SOLVING A SYSTEM OF LINEAR EQUATIONS ... linear equations can be solved readily by simple ... Examples of engineering problems that require

• Gauss elimination with pivoting

As mentioned earlier, obvious problems occur when a pivot element is zero because the normalizationstep leads to division by zero. Problems may also arise when the pivot element is close to, ratherthan exactly equal to, zero because if the magnitude of the pivot element is small compared to theother elements, then round-off errors can be introduced. Therefore, before each row is normalized, itis advantageous to determine the largest available coefficient (in absolute value) in the column belowthe pivot element. The rows can then be switched so that the largest element is the pivot element.

2.2 Gauss-Jordan elimination method

The Gauss-Jordan method is a variation of Gauss elimination. The major difference is that when anunknown is eliminated in the Gauss-Jordan method, it is eliminated from all other equations rather thanjust the subsequent ones. In addition, all rows are normalized by dividing them by their pivot elements.Thus, the elimination step results in an identity matrix rather than a triangular matrix as shown in fig.5.Consequently, it is not necessary to employ back substitution to obtain the solution.

Fig.5

• Gauss-Jordan elimination with pivoting

It is possible that the equations are written in such an order that during the elimination procedurea pivot equation has a pivot element that is equal to zero. Obviously, in this case it is impossible tonormalize the pivot row (divide by the pivot element). As with the Gauss elimination method, theproblem can be corrected by using pivoting.

• Although the Gauss-Jordan technique and Gauss elimination might appear almost identical, theformer requires approximately 50 percent more operations than Gauss elimination. Therefore, Gausselimination is the simple elimination method of preference for obtaining solutions of linear algebraicequations. One of the primary reasons that we have introduced the Gauss-Jordan, however, is thatit is still used in engineering as well as in some numerical algorithms.

Example:

Consider solving the following system using the naive Gauss elimination3x1 − 2x2 + 5x3 = 14

x1 − x2 = −1

2x1 + 4x3 = 14

2.3 LU decomposition method

As described in the previous sections, Gauss elimination is designed to solve systems of linear algebraicequations,

[A]X = B

Although it certainly represents a sound way to solve such systems, it becomes inefficient when solvingequations with the same coefficients [A], but with different right-hand-side constants (the b s). Recall that

7

Page 8: SOLVING A SYSTEM OF LINEAR EQUATIONS · SOLVING A SYSTEM OF LINEAR EQUATIONS ... linear equations can be solved readily by simple ... Examples of engineering problems that require

Gauss elimination involves two steps: forward elimination and back- substitution. Of these, the forward-elimination step comprises the bulk of the computational effort. This is particularly true for large systemsof equations.

LU decomposition methods separate the time-consuming elimination of the matrix [A] from the manip-ulations of the right-hand side B . Thus, once [A] has been decomposed, multiple right-hand-side vectorscan be evaluated in an efficient manner. Before showing how this can be done, let us first provide amathematical overview of the decomposition strategy.

2.3.1 Overview of the LU decomposition

A two-step strategy (see Fig.6) for obtaining solutions can be based explained as follow

• LU decomposition step. [A] is factored or decomposed into lower [L] and upper [U] triangularmatrices.

• Substitution step. [L] and [U ] are used to determine a solution X for a right-hand side B. Thisstep itself consists of two steps. First, an intermediate vector D is generated by forward substitution.Then, the result is substituted back to solve for X by back substitution.

[A] = [L][U ]

D = [U ]X

[L][U ]X = [L]D = B

– The forward-substitution step can be represented concisely asd1 =

b1a11

di =bi −

∑i−1j=1 aijdj

aiii = 2, 3, ...., n

– the back-substitution step can be represented concisely as

xn =dnann

xi =

di −n∑

j=i+1

aij

aiixj i = n− 1, n− 2, ...., 1

8

Page 9: SOLVING A SYSTEM OF LINEAR EQUATIONS · SOLVING A SYSTEM OF LINEAR EQUATIONS ... linear equations can be solved readily by simple ... Examples of engineering problems that require

Fig.6

For a given matrix several methods can be used to determine the corresponding [L] and [U ]. Two of themethods, one related to the Gauss elimination method and another called Crout’s method, are describednext.

2.3.2 LU decomposition using Gauss elimination procedure

When the Gauss elimination procedure is applied to a matrix , the elements of the matrices [L] and [U ]are actually calculated. The upper triangular matrix [U ] is the matrix of coefficients [A] that is obtainedat the end of the Gauss elimination procedure. For the lower triangular matrix [L], the elements on thediagonal are all 1, and the elements below the diagonal are the multipliers mij that multiply the pivotequation when it is used to eliminate the elements below the pivot coefficient. For the case of a system ofthree equations, the decomposition has the form: a11 a12 a13

a21 a22 a23a31 a32 a33

=

1 0 0m21 1 0m31 m32 1

a11 a12 a130 a′22 a′230 0 a′33

where m21 = a21/a11, m31 = a31/a11, and m32 = a′32/a

′22

2.3.3 LU decomposition using Crout’s method

In this method the matrix is decomposed into the product [L][U ], where the diagonal elements of thematrix [U ] are all 1s. It turns out that in this case, the elements of both matrices can be determined usingformulas that can be easily programmed. For example, in the case of a system of four equations

a11 a12 a13 a14a21 a22 a23 a24a31 a32 a33 a34a41 a42 a43 a44

=

L11 0 0 0L21 L22 0 0L31 L32 L33 0L41 L42 L43 L44

1 U12 U13 U14

0 1 U23 U24

0 0 1 U34

0 0 0 1

Executing the matrix multiplication on the right-hand side of the equation gives:

a11 a12 a13 a14a21 a22 a23 a24a31 a32 a33 a34a41 a42 a43 a44

=

L11 (L11U12) (L11U13) (L11U14)L21 (L21U12 + L22) (L21U13 + L22U23) (L21U14 + L22U24)L31 (L31U12 + L32) (L31U13 + L32U23 + L33) (L31U14 + L32U24 + L33U34)L41 (L41U12 + L42) (L41U13 + L42U23 + L43) (L41U14 + L42U24 + L43U34 + L44)

The elements of the matrices [L] and [U ] can be determined by solving the previous equation. The

solution is obtained by equating the corresponding elements of the matrices on both sides of the equation.

9

Page 10: SOLVING A SYSTEM OF LINEAR EQUATIONS · SOLVING A SYSTEM OF LINEAR EQUATIONS ... linear equations can be solved readily by simple ... Examples of engineering problems that require

One can observe that the elements of the matrices [L] and [U ] can be easily determined row after row fromthe known elements of [A] and the elements of [L] and [U ] that are already calculated. Starting with thefirst row, the value of L11 is calculated from L11 = a11. Once L11 is known, the values of U12, U13, andU14 are calculated by:

U12 = a12/L11 U13 = a13/L11 U14 = a14/L11

Moving on to the next row the next elements can be calculated in a similar manner.A procedure for determining the elements of the matrices [L] and [U ] can be written as follow. If [A]

is an (n× n) matrix, the elements of [L] and [U ] are given by:

• Step 1: Calculating the first column of [L]:

for i=1, 2, ...., nLi1 = ai1

• Step 2: Substituting 1s in the diagonal of [U ] :

for i=1, 2, ...., nUii = 1

• Step 3: Calculating the elements in the first row of [U ] (except U11 which was already calculated):

for j=2, 3, ...., n

U1j =a1jL11

• Step 4: Calculating the rest of the elements row after row (i is the row number and j is the columnnumber). The elements of [L] are calculated first because they are used for calculating the elementsof [U ] :

for i=2, 3, ...., n

for j=2, 3, ...., i

Lij = aij −j−1∑k=1

LikUkj

for j=(i+1), (i+2), ...., n

Uij =

aij −i−1∑k=1

LikUkj

Lii

2.3.4 LU decomposition with pivoting

Decomposition of a matrix into the matrices [L] and [U ] means that [A] = [L][U ] . In the presentation ofGauss and Crouts decomposition methods in the previous two subsections, it is assumed that it is possibleto carry out all the calculations without pivoting. In reality, as was discussed before, pivoting may berequired for a successful execution of the Gauss elimination procedure. Pivoting might also be neededwith Crouts method. If pivoting is used, then the matrices [L] and [U ] that are obtained are not thedecomposition of the original matrix [A]. The product [L][U ] gives a matrix with rows that have the sameelements as [A], but due to the pivoting, the rows are in a different order. When pivoting is used in thedecomposition procedure, the changes that are made have to be recorded and stored. This is done bycreating a matrix [P ], called a permutation matrix, such that:

[P ][A] = [L][U ]

The order of the rows of B have to be changed such that it is consistent with the pivoting. This is doneby multiplying B by the permutation matrix,[P ].

10

Page 11: SOLVING A SYSTEM OF LINEAR EQUATIONS · SOLVING A SYSTEM OF LINEAR EQUATIONS ... linear equations can be solved readily by simple ... Examples of engineering problems that require

Example:

Consider solving the following system using the LU decompositionx1 − x2 = −1

3x1 − 2x2 + 5x3 = 14

2x1 + 4x3 = 14

3 Iterative methods

Iterative or approximate methods provide an alternative to the elimination methods described previously.Such approaches are similar to the techniques we developed to obtain the roots of a single equation (fixedpoint iteration). Those approaches consisted of guessing a value and then using a systematic method toobtain a refined estimate of the root. Because the present part of the book deals with a similar problem-obtaining the values that simultaneously satisfy a set of equations-we might suspect that such approximatemethods could be useful in this context.

For a system with n equations, the explicit equations for the [xi] unknowns can be written as:

xi =1

aii

bi − n∑

j=1j 6=i

aijxj

For a system of (n = 4) equations the previous equation reduces to

Fig.7

For a system of n equations [a]X = b, a sufficient condition for convergence is that in each row of thematrix the absolute value of the diagonal element is greater than the sum of the absolute values of theoff-diagonal elements.

| aii |>j=n∑

j=1,j 6=i

| aij |

This condition is sufficient but not necessary for convergence when the iteration method is used. Whenthis condition is satisfied, the matrix is classified as diagonally dominant, and the iteration process convergestoward the solution. The solution, however, might converge even when the condition is not satisfied. Twoiterative methods are presented next

3.1 Jacobi iterative method

In the Jacobi method, an initial (first) value is assumed for each of the unknowns, x(1)1 , x

(1)2 , ..., x

(1)n . If

no information is available regarding the approximate values of the unknown, the initial value of all theunknowns can be assumed to be zero. The second estimate of the solution is calculated by substitutingthe first estimate in the right-hand side of the following equation

xi =1

aii

bi − n∑

j=1j 6=i

aijxj

11

Page 12: SOLVING A SYSTEM OF LINEAR EQUATIONS · SOLVING A SYSTEM OF LINEAR EQUATIONS ... linear equations can be solved readily by simple ... Examples of engineering problems that require

In general, the (k+1)th estimate of the solution is calculated from the (k)th estimate by:

x(k+1)i =

1

aii

bi − n∑

j=1j 6=i

aijx(k)j

The iterations continue until the differences between the values that are obtained in successive iterations

are small. The iterations can be stopped when the absolute value of the estimated relative error of all theunknowns is smaller than some predetermined value:∣∣∣∣∣x(k+1)

i − x(k)i

x(k+1)i

∣∣∣∣∣ < ε i = 1, 2, ...., n

Example:

Consider solving the following system using Jacobi iterative method (three iterations).3x1 − 2x2 + 5x3 = 14

x1 − x2 = −1

2x1 + 4x3 = 14

3.2 Gauss-Seidel iterative method

In the GaussSeidel method, initial (first) values are assumed for the unknowns x2, x3, ....., xn (all of theunknowns except x1). If no information is available regarding the approximate value of the unknowns, theinitial value of all the unknowns can be assumed to be zero. The first assumed values of the unknowns aresubstituted in the following equation with i = 1 to calculate the value of x1.

xi =1

a11

bi − n∑

j=1j 6=i

aijxj

Next, the same equation with i = 2 is used for calculating a new value for x2 . This is followed by

i = 3 for calculating a new value for x3. The process continues until i = n, which is the end of the firstiteration. Then, the second iteration starts with i = 1 where a new value for x1 is calculated, and so on. Inthe GaussSeidel method, the current values of the unknowns are used for calculating the new value of thenext unknown. In other words, as a new value of an unknown is calculated, it is immediately used for thenext application. Applying the previous equations to the GaussSeidel method gives the iteration formula:

x(k+1)1 =

1

a11

b1 − n∑j=2

a1jx(k)j

x(k+1)i =

1

aii

bi −i−1∑

j=1

aijx(k+1)j +

n∑j=i+1

aijx(k)j

i = 2, 3, ..., n− 1

x(k+1)n =

1

ann

bn − n−1∑j=1

anjx(k+1)j

• Convergence can be checked using the same criterion as in the Jacobi method

εa,i =

∣∣∣∣∣x(ik)− x(k−1)i

x(ik)

∣∣∣∣∣ < εs

for all i, where k and k − 1 are the present and previous iterations.

12

Page 13: SOLVING A SYSTEM OF LINEAR EQUATIONS · SOLVING A SYSTEM OF LINEAR EQUATIONS ... linear equations can be solved readily by simple ... Examples of engineering problems that require

Example:

Consider solving the following system using Gauss-Seidel iterative method (three iterations).3x1 − 2x2 + 5x3 = 14

x1 − x2 = −1

2x1 + 4x3 = 14

4 Use of MATLAB built in functions for solving systems of linearequations

4.1 Left division

Given a system of linear equations in the form [A]X = b, one can use the left division in MATLAB so solvefor X. The syntax is

X = [A]\b

4.2 Inverse operation

Given a system of linear equations in the form [A]X = b, one can use the inverse of the matrix to X. Thesyntax is

X = [A]−1 ∗ b or X = inv([A]) ∗ b

4.3 LU decomposition

MATLAB has a built in function for decomposition to solve for X. MATLAB uses partial pivoting. the lufunction gives [L], [U], and the permutation matrix such as [L] ∗ [U ] = [P ] ∗ [A].

[L,U, P ] = lu([A])

5 Application: Inverse of a matrix

As an application of the proceeding methods we proceed to finding the inverse of a matrix. The procedurewill be demonstrated for a 3× 3 matrix but can easily extended to a matrix of dimension n× n.

consider matrices [A] and [B] such as [A] ∗ [B] = I, where [I] is the identity matrix.

[A] =

a11 a12 a13a21 a22 a23a31 a32 a33

[B] =

b11 b12 b13b21 b22 b23b31 b32 b33

[A] ∗ [B] =

a11 a12 a13a21 a22 a23a31 a32 a33

∗ b11 b12 b13b21 b22 b23b31 b32 b33

=

1 0 00 1 00 0 1

or

13

Page 14: SOLVING A SYSTEM OF LINEAR EQUATIONS · SOLVING A SYSTEM OF LINEAR EQUATIONS ... linear equations can be solved readily by simple ... Examples of engineering problems that require

a11 a12 a13a21 a22 a23a31 a32 a33

∗ b11b21b31

=

100

a11 a12 a13a21 a22 a23a31 a32 a33

∗ b12b22b32

=

010

a11 a12 a13a21 a22 a23a31 a32 a33

∗ b13b23b33

=

001

these systems can be solved by any of the methods discussed in this chapter

6 Ill conditioned systems

A numerical solution of a system of equations is seldom an exact solution. Even though direct methods(Gauss, GaussJordan, LU decomposition) can be exact, they are still susceptible to round-off errors whenimplemented on a computer. This is especially true with large systems and with ill-conditioned systems.

An ill-conditioned system of equations is one in which small variations in the coefficients of the matrix(A) cause large changes in the solution. When an ill-conditioned system of equations is being solvednumerically, there is a high probability that the solution obtained will have a large error or that a solutionwill not be obtained at all.

To illustrate this and also be able to identify whether a system of linear equations is ill conditioned wefirst introduce the concept of norm.

By definition a norm is a real valued function that provides a measure of the size or ”length” of multiplecomponent mathematical entities such as vectors and matrices. Example of norms

• Euclidean norm: For an n-dimensional vector ~X = [x1, x2, ......, xn]

|| ~X ||e=

√√√√ n∑i=1

x2i

• Uniform vector norm:

|| ~X ||∞= max1≤i≤n

| xi |

• Frobenius norm:, for a n× n matrix A of components aij

|| A ||e=

√√√√ n∑i=1

n∑j=1

a2ij

• Uniform matrix norm:

|| A ||∞= max1≤i≤n

n∑j=1

| aij |

14

Page 15: SOLVING A SYSTEM OF LINEAR EQUATIONS · SOLVING A SYSTEM OF LINEAR EQUATIONS ... linear equations can be solved readily by simple ... Examples of engineering problems that require

Although there are theoretical benefits for using certain of the norms, the choice is sometimes influencedby practical considerations. For example, the uniform row norm is widely used because of the ease withwhich it can be calculated and the fact that it usually provides an adequate measure of matrix size.

Now that we have introduced the concept of norm, we can use it to define another quantity calledmatrix condition number

Cond[A] =|| A || · || A−1 ||

This number is always larger or equal to 1.

Cond[A] =|| A || · || A−1 ||≥ 1

It can be shown that the true relative error of the solution of ([A]X = b) or || ∆X || / || X || is less orequal to the true relative error of the residual || ∆AX || / || b || or

|| ∆ ~X |||| ~X ||

≤ Cond[A]|| ∆AX |||| b ||

where

∆AX = AXt −AXNS

and

DeltaX = Xt −XNS

If the condition number of teh matrix [A] is large there is a large probability that the true relative errorof the solution be large as well.

ExampleConsider solving the following system of linear equations 6x1 − 2x2 = 10

11.5x1 − 3.85x2 = 17

15