matrices and system of equation

8/2/2019 Matrices and System of Equation

1/81

3 Matrices and systems of equations

3.1 Systems of equations

Many engineering problems result in a system of equations for un-known quantities x1, x2, x3, x4, say.

Variables xj (j = 1, 2, 3, 4) represent, for example, production totals,

electrical currents, stresses, etc.

Often these equations will be linear, i.e., no variable is either raised toa power (other than one!) or multiplied by one of the other variables.

We will be interested in techniques which can be efficiently extendedto problems with many unknowns, as engineering applications often

demand (although in lectures, problems etc, we will rarely go beyond

three variables).

39


2/81

3.1.1 One equation for one unknown

Simplest case is of only one equation and one unknown. An importantstep in solving systems of equations will be solving a single equation.

For example, suppose we are asked to solve:

2x = 6.

Clearly the solution is x = 3.

In general if a and b are two given numbers, we might be asked to find

the values of x for which

ax = b (3.1)

is satisfied. There are three cases.

40


3/81

The three possible outcomes when solving ax = b (equation (3.1)) are:

1. If a = 0 then we can divide (3.1) by a to give x = b/a. This is theonly value of x that satisfies (3.1) and there is a unique solution.

2. If a = 0 and b = 0, (3.1) is not satisfied for any value of x no

numbers x satisfy 0 x = b (= 0). Hence there is no solution to(3.1).

3. If a = 0 and b = 0, then any value of x satisfies (3.1) because

0

x = 0 for any number x. Hence there are infinitely many solutions

to (3.1).

We will return to this situation time and time again.

41


4/81

3.1.2 A 2 2 system (2 equations in 2 unknowns) Simple example: system of two equations

3x1 + 4x2 = 2 (3.2)

x1 + 2x2 = 0 (3.3)

for the unknowns x1 and x2.

Many ways to solve this simple system of equations we describeone that is easily generalised to much larger systems of linear equations.

Step 1

Eliminate x1 from (3.3) by replacing

(3.3) 3(3.3)(3.2)

to give

3x1 + 4x2 = 2 (3.4)

2x2 = 2. (3.5)

42


5/81

Step 2

Solve (3.5) (this is easy to do divide both sides by 2). This gives

x2 = 1. (3.6)Note: (3.5) is a 1 1 system, as in (3.1).

Step 3

Substitute the result from Step 2 back into (3.4). This gives

3x1 4 = 2 3x1 = 6 x1 = 2. (3.7)The solution to equations (3.2) and (3.3) is therefore

x1 = 2 x2 = 1.

Check

3 2 + 4 (1) = 22 + 2 (1) = 0.

43


6/81

3.1.3 A 3

3 system

Similar procedure can be used to solve a system of three linear equa-tions for three unknowns x1, x2, x3.

For example, suppose we wish to solve

2x1 + 3x2 x3 = 5 (3.8)4x1 + 4x2 3x3 = 3 (3.9)2x1 3x2 + x3 = 1 (3.10)

to obtain x1, x2, x3. There are more steps involved because there aremore equations and more unknowns.

44


7/81

Step 1

Eliminate x1 from equations (3.9) and (3.10) by subtracting multiples

of (3.8).

Replace

(3.9)

(3.9)

2

(3.8)

(3.10) (3.10)(3.8)

to leave the system

2x1 + 3x2 x3 = 5 (3.11)2x2 x3 = 7 (3.12)6x2 + 2x3 = 6 (3.13).

45


8/81

Step 2

Eliminate x2 from (3.13). To do this, we subtract an appropriate mul-

tiple of (3.12) from (3.13).

(If we subtract a multiple of (3.11) from (3.13) instead, then in the

process of eliminating x2 from (3.13) we reintroduce x1!)

Replace

(3.13) (3.13) 3 (3.12)to leave

2x1 + 3x2 x3 = 5 (3.14)2x2 x3 = 7 (3.15)

5x3 = 15 (3.16).

Step 3Solve the 11 system (3.16) to find x3. In this case there is a uniquesolution to (3.16):

x3 =15

5= 3.

46


9/81

Step 4

Substitute this result back into (3.15) to give x2 from

2x2 3 = 7 2x2 = 4 x2 = 2.

Step 5

Substitute the values we have found for x2 and x3 back into (3.14) to

give

2x1 + 6 3 = 5 2x1 = 2 x1 = 1.Finally the solution to the system (3.8), (3.9), (3.10), is

x1 = 1 x2 = 2 x3 = 3.

This solution is unique.

This method is Gaussian Elimination, and can be applied to any

system where the number of equations equals the number of unknowns.

47


10/81

3.1.4 Another 3 3 systemWe apply Gaussian elimination to the equations

x1 + x2 + 2x3 = 1 R1

2x1 x2 x3 = 1 R2x1 + 4x2 + 7x3 = 2 R3

using somewhat more compact notation.

Step 1

Eliminate x1

from the last two equations

R2R22R1R3R3R1

x1 + x2 + 2x3 = 1 R1

3x2 5x3 = 1 R23x2 + 5x3 = 1 R

3

Step 2Eliminate x2 from the last equation (use R2 so x1 doesnt come back!)

R3R3+R2x1 + x2 + 2x3 = 1 R

1

3x2 5x3 = 1 R20 = 0 R3

48


11/81

Step 3

The last equation is a 1 1 system which is easily solved. In thiscase the equation

0 = 0 R3is satisfied for any value of x3 and there are infinitely many solutions.

Let

x3 = t

where t is an arbitrary number.

Step 4

Substitute x3 back in the second equation R2 to get

3x2 5x3 = 3x2 5t = 1 x2 =1

3 5t

3.

Step 5

Substitute x2 and x3 back in the first equation R1 to get

x1 + x2 + 2x3 = x1 +1

3 5t

3+ 2t = 1 x1 =

2

3 t

3.

49


12/81

Conclusion For any given value of t,

x1 =2

3 t

3

x2 =1

3 5t

3

x3 = t

is a solution of the original set of equations and there are an infinite

number of such solutions.

Exercise Check this by direct substitution in the original equations.

50


13/81

3.2 Matrices

Many engineering problems involve a large number of unknowns anda large number of equations to be solved simultaneously. These can

number in the hundreds or even the thousands in practice. Matrices.are a convenient way of representing and manipulating such problems.

For example, the equations

3x1 + 4x2 = 2 (3.17)

x1 + 2x2 = 0 (3.18)solved previously may be written more compactly by introducing

A =

3 41 2

matrix of coefficients

x = x1

x2

vector of unknowns

b =

20

vector of right-hand sides.

51


14/81

x and b are examples of a column vector, i.e., a matrix with a single

column.

Then we write the system of equations as

Ax = b (3.19)

or 3 41 2

x1x2

=

20

. (3.20)

In books, vectors are written in bold type

In handwriting, a or a.

52


15/81

For example, the 3

3 system

2x1 + 3x2 x3 = 5 (3.8)4x1 + 4x2 3x3 = 3 (3.9)2x1 3x2 + x3 = 1 (3.10)

we solved previously may be written as

Ax = b, (3.21)

where

A = 2 3 14 4

3

2 3 1 , x = x1x2x3

and b = 53

1 .

53


16/81

More generally, if we have a system of m equations in n unknowns

a11x1 + a12x2 + . . . + a1nxn = b1a21x1 + a22x2 + . . . + a2nxn = b2

... ...

am1x1 + am2x2 + . . . + amnxn = bm

(3.22)

these may be written in matrix form as

Ax = b, (3.23)

where

A =

a11 a12 . . . a1na21 a22 . . . a2n

...am1 am2 . . . amn

, x =

x1x2

...xn

and b =

b1b2...

bm

. (3.24)

A is an mn matrix (m = number of rows, n = number of columns)

x is an n 1 matrix (column vector)

b is an m

1 matrix (column vector)

54


17/81

Note that aij = element of A at the intersection of the i-th row and

the j-th column. The first subscript refers to the row, the second to

the column.

3.2.1 Order of a matrix

The order of a matrix is its size, e.g., the order of A in (3.24) is mn.

55


18/81

3.3 Matrix algebra

3.3.1 Equality

Two matrices are equal if they have the same order and if their corre-

sponding elements are equal.

3.3.2 Addition and subtraction

If A and B are of the same order then we may add corresponding

elements to obtain

A + B =

a11 + b11 a12 + b12 . . . a1n + b1n

...

am1+

bm1 am2+

bm2 . . . amn+

bmn

or subtract B from A to obtain

A B = a11 b11 a12 b12 . . . a1n b1n...

am1 bm1 am2 bm2 . . . amn bmn

.

56


19/81

3.3.3 ExampleIf

A =

1 20 4

and B =

0 31 4

then

A + B =

1 + 0 2 + 30 + 1 4 + 4

=

1 51 8

and

A B = 1 0 2 30 1 4 4

=

1 11 0 .

3.3.4 Example

If

A =

1 20 4

and B =

0 3 21 4 1

then A+B and AB do not make sense because A and B have differentorders.

57


20/81

3.3.5 Multiplication of matrices

Multiplication of one matrix by another is more involved than you mightexpect.

Example 1 12 1

0 13 2

=

3 13 4

0 13 2 1 12 1 = 2 17 1 Note: For matrices A and B, AB is not in general equal to BA.

58


21/81

Example

A =

4 1 63 2 1

is a 2 3 matrix and B =

1 11 2

1 0

is a 3 2 matrix.

A + B is not defined, nor is A B (because A and B have differentorders).

AB = 4 1 6

3 2 1

1 11 21 0

= 11 66 7

BA =

1 11 2

1 0

4 1 6

3 2 1 =

7 3 710 5 8

4 1 6

Note: AB = BA.

Note: AB is a 2 2 matrix, and BA is a 3 3 matrix.59


22/81

Let

A be an m n matrixB be an p q matrix

Then AB exists if n = p.

The result AB is an m q matrix. Thus

A B = Cm n p equal

q m q

If n = p, AB does not exist i.e. number of columns of A must equal

number of rows of B.

For example, if A is a 3 2 matrix and B is a 2 2 matrix then AB isa 3 2 matrix but BA does not make sense.

60


23/81

3.3.6 Rule for matrix multiplication

When C = AB exists, the element cij in the ith row and jth column of

C is obtained by taking the product of the ith row of A with the jthcolumn of B.

cij =

rairbrj

61


24/81

3.3.7 Zero matrix, 0

This is a matrix whose elements are all zero. For any matrix A,

A + 0 = A.

We can only add matrices of the same order, therefore 0 must be of

the same order as A.

3.3.8 Multiplication of a matrix by a scalar

If is a scalar (i.e., a number) we define

A =

a11 a12 . . . a1n...

am1 am2 . . . amn

,

i.e., we multiply every element of A by to obtain A.

For example,

3

1 20 1

=

3 60 3

.

62


25/81

3.3.9 Further properties

If is a scalar and A, B and C are matrices then, provided all the

products exist:

(A)B = (AB) = A(B);

A(BC) = (AB)Cso we may write each of these products unambiguously as ABC;

(A + B)C = AC + BC;

C(A + B) = CA + CB;

In general AB = BA, even if both AB and BA exist;

AB = 0 does not necessarily imply that A = 0 or B = 0;

A0 = 0.

63


26/81

Example

AB = 0 1

0 0

3 00 0

=

0 00 0

= 0

but neither A nor B is the zero matrix.

It follows that AB = AC does not necessarily imply that B = C be-cause

AB = AC A(B C) = 0

and as A and (B C) are not necessarily 0, B is not necessarily equalto C.

64


27/81


28/81

3.4 Special matrices

3.4.1 Square matrices

A square matrix is one where

no. of rows = no. of columns.

Example

1 2 33 1 2

2 3 1

and

1 10 4

are square matrices while1 2 32 3 1

and

1 11 0

0 4

are not.

66


29/81

3.4.2 The identity matrix

The identity matrix is a square matrix whose elements are all zero,

except those on the leading diagonal, which are unity (= 1). (The

leading diagonal is the one from the top left to the bottom right.)

The identity matrix is usually denoted by I (or sometimes by In if there

is a need to stress that it has the order n n). For example

I3 =

1 0 00 1 00 0 1

.The identity matrix has the properties that

AI = A and IA = A

for any matrix A, and

Ix = x

for any vector x.

67

3 4 3 Th t f t i


30/81

3.4.3 The transpose of a matrix

The transpose the m n matrix A is an n m matrix denoted by ATand obtained by interchanging the rows and columns of A.

ExamplesIf

A = 3 2 14 5 6 , B =

14 , C =

1 2 30 5 1

2 4 7

then their transposes are

AT =

3 42 51 6

, BT =

1 4

, CT =

1 0 22 5 43 1 7

.

Note: If D = EF then

DT = (EF)T = FTET (3.25)

68


31/81

3.4.4 The inverse of a matrix

If A is a square matrix, then its inverse matrix is denoted by A

1 and

is defined by the property that

A1A = AA1 = I.

Not every square matrix has an inverse.

Although an inverse matrix needs to satisfy both A1A = I andAA1 = I, if we can show one of these equations, then then the othermust follow, although we will not show that in this module. In other

words, it is enough to show that A

1A = I or AA

1 = I to know that

A1 is the inverse of A.

We will show how to calculate inverse matrices later in the module.

69


32/81

The inverse, if it exists, is very useful. For example, if we can findA1 then we can solve the system Ax = b because

Ax = b A1Ax = A1b

Ix = A1

b

x = A1b.Thus there is a unique solution to

Ax = b,

given by x = A1b.

If D = EF then

D1 = (EF)1 = F1E1 (3.26)

provided the inverses exist. This is easy to prove

DD1 = EF F1E1 = EIE1= EE1= I.

70


33/81

3.4.5 Orthogonal matrices

A matrix A which is such that

A1

= AT

is said to be an orthogonal matrix. Another way of saying this is that

AAT = ATA = I.

Example

A =

12 12 1

21

2

, AT =

12 12

12

12

,

and

AAT = 12 12 1

21

2

12 121

21

2

= 1 00 1

.

So AAT = I. That is, AT is the inverse of A.

71


34/81

3.4.6 Symmetric and anti-symmetric matrices

A square matrix A is said to be symmetric if

A = AT.

Example

1 0

2 3

0 3 4 72 4 1 6

3 7 6 2

is a symmetric matrix.

Note: the element aij = aji. That is, the element in the ith row and jthcolumn is the same as the element in the jth row and the ith column.

The matrix is symmetric about the leading diagonal.

72


35/81

A square matrix A is anti-symmetric if

A = AT.

Note: the element aij = aji. In particular a11 = a11, a22 = a22etc. Hence a11 = 0, a22 = 0 etc. That is, all elements on the leading

diagonal are zero.

Example

0 1 51 0 1

5 1 0

.

is an antisymmetric matrix

73


36/81

3.5 Determinants of square matrices

3.5.1 A general 2 2 system

Consider a general 2 2 system:

ax1 + bx2 = b1 (3.27)

cx1 + dx2 = b2. (3.28)

To solve this system, proceed as before.

Step 1 Eliminate x1 from (3.28) by replacing (3.28) with

a(3.28)c(3.27):

ax1 + bx2 = b1 (3.29)

(ad bc)x2 = ab2 cb1 (3.30)

Step 2 Solve (3.30) for x2. This is a 1 1 system remember thethree possibilities:

74


37/81

(a) If ad bc = 0, we may solve (3.30) to give

x2 =ab2 cb1

ad bc .

We may then carry out. . .

Step 3 Substitute the result from Step 2 back into (3.29). This

gives

ax1 + bab2 cb1

ad bc = b1and so

ax1 =adb1 abb2

ad bcand hence

x1 = db1 bb2ad bc .

(Note we have divided through by a. If a = 0 we may not do this,

but it can be shown that the result still holds in this case.)

75


38/81

(b) If ad bc = 0 and the right-hand side of (3.30) is non-zero, i.e.,

ab2 cb1 = 0,then there is no solution to (3.30), and hence no solution to the

2 2 system (3.27), (3.28).

(c) If

ad bc = 0and

ab2 cb1 = 0then any value of x2 will satisfy (3.30). We may then carry out. . .

Step 3Substitute each value of x2 back into (3.29) to give acorresponding value of x1.

There are thus infinitely many solutions to the 22 system (3.27),(3.28) in this case.

76


39/81

3.5.3 The determinant of a 2 2 system

The quantity D = ad bc is clearly important, and is called the deter-minant of the system (3.27), (3.28). It is denoted by

a bc d

or det

a bc d

.

The solution of the system (3.27), (3.28) is unique provided D = 0.

If D = 0 then there are either

- no solutions (i.e., the equations are inconsistent), or

- infinitely many solutions (i.e., the equations are not independent)

77


40/81

3.5.4 The inverse of a 2 2 matrix

If A is the 2 2 matrix a bc d

its determinant is D = ab

cd.

If D = 0, A has an inverse A1 given by

A1 = 1D

d b

c a

.

If D = 0, the matrix A has no inverse.

78

Example


41/81

Example

Find the inverse of the matrix A =

1 5

2 3

.

Step 1 Calculate the determinant

det A = 1 3 5 2 = 13.Since det A = 0 the inverse exists.

Step 2 Compute the inverse A1 = 1

13 3 52 1 .

Check:

AA1 = 113

1 5

2 3

3 5

2

1

= 113

3 10 5 56 6 10 3

=

1 00 1

= I

79

3 5 6 The determinant of a 3 3 matrix


42/81

3.5.6 The determinant of a 3 3 matrix These ideas can be extended from 2 2 matrices to 3 3, 4 4matrices and beyond.

Again the determinant dictates whether the matrix is singular (hasno inverse) or nonsingular (has an inverse).

Given a 3 3 matrix

A = a11 a12 a13

a21 a22 a23a31 a32 a33

,we define its determinant to be

det A = a11 det

a22 a23a32 a33

a12 det a21 a23a31 a33

+ a13 det

a21 a22a31 a32

.

We already know how to calculate each of the 2 2 determinants.80

3 5 7 Example


43/81

3.5.7 Example

1 2 31 3 51 5 12

= 1

3 55 12

2

1 51 12

+ 3 1 31 5

= (36 25) 2(12 5) + 3(5 3)= 11 14 + 6 = 3

Note that the definition above may be thought of as

det A = a11 det a11 a12 a13a21 a22 a23

a31 a32 a33

a

12 det

a11 a12 a13a

21a

22a

23a31 a32 a33

+ a13 det a11 a12 a13a21 a22 a23

a31 a32 a33

81


44/81

3.5.8 Chessboard determinant

det A may be obtained by expanding about any row or column, using

the chessboard pattern of signs and the cover up method.

+ + +

+ +

82

3 5 9 Example


45/81

3.5.9 Example

Let A =

1 3 02 6 41 0 2

. Here are three ways to calculate det A.

(a) Expand about the first row to give

det A = +1

6 40 2 3

2 41 2 + 0

2 61 0

= 12 24 + 0 = 12.

(b) Expand about the middle row to give

det A = 2 3 00 2

+ 6 1 01 2

4 1 31 0

= 1 2 + 1 2 12 = 12.

(c) Expand about the last column to give

det A = 0

2 61 0 4

1 31 0 + 2

1 32 6

= 0 12 + 0 = 12.

83

3 5 10 Properties of determinants


46/81

3.5.10 Properties of determinants

The properties described below follow from the definition of the deter-

minant (although proving them can be tricky for high-order matrices).

They apply to deteminants of all orders, but are illustrated by 2

2

determinants.

(i) The determinant of any matrix is equal to the determinant of its

transpose, i.e. det(A) = det(AT). Therefore any results about de-

terminants that deal with rows are also true for columns.

(ii) If all elements of one row of a matrix are multiplied by a constant

, the determinant of the new matrix is times the determinant of

the original matrix. For example

a bc d = ad bc= (ad bc)=

a bc d

.

84


47/81

The same holds if one column of a matrix is multiplied by a

constant .

Application If a factor appears in each element of a row or

column of a determinant it can be taken out as a factor: e.g. 2 121 3 = 2

1 61 3 = 2 3

1 21 1

= 2 3 (1 1 2 1) = 6.

Application If all elements of a row or column of a determinant

are zero, the value of the determinant is zero. For example

0 0c d

= 0 d 0 c = 0.

Application If A is an n n matrix,

det(A) = n det A.

This follows on applying rule (ii) n times, once for each row.

(iii) Interchanging any two rows or any two columns of a determinant


48/81

(iii) Interchanging any two rows or any two columns of a determinant

changes the sign of the determinant. For example, interchanging

rows gives

c da b = cb ad= (ad bc)=

a bc d .

Application If any two rows are identical, the determinant is zero.Similarly if any two columns are identical then the determinant is

zero.

Application (of (ii) and (iii) together) If any row is a multiple of

any other row, the determinant is zero. (Similarly if any column

is a multiple of any other column, the determinant is zero.) For

example

c dc d

=

c dc d

= 0.

(iv) The value of a determinant is unchanged by adding to any row


49/81

(iv) The value of a determinant is unchanged by adding to any row

(or column) a constant multiple of any other row (or column). For

example if Row 1 (Row 1 + Row 2),

a + c b + d

c d = (a + c)d (b + d)c= ad bc=

a bc d .

(v) If A and B are square matrices of the same order then

det(AB) = det A det B.

Application Let A = I and let B be any matrix with nonzerodeterminant. Then, because AB = IB = B, we have

det(AB) = det B

= det A det B = det Idet B

and therefore det I = 1.

Example


50/81

Example

Evaluate

1 1 20 2 43 0 6

.

1 1 20 2 43 0 6

= 2

1 1 20 1 23 0 6

(since Row 2 has factor of 2)

= 2 3 1 1 20 1 21 0 2

(since Row 3 has factor of 3)

= 2 3

0 1 00 1 2

1 0 2

(subtract Row 3 from Row 1)

= 2 3

0 21 2

= 2 3 2 = 12 85

3.6 Row reduction methods


51/81

Technique introduced in 3.1 for solving systems of equations (Gaus-

sian elimination) can be applied using matrices as follows.

Important to note that the full equations need not be written out at

each stage, only the numerical coefficients are required.

These coefficients can be written in matrix form. Solve the following

system of equations for x1, x2 and x3:

3x1 x2 + 2x3 = 32x1 + x2 + x3 = 2

x1 3x2 = 5.

In matrix form this is Ax = b, where

A =

3 1 22 1 1

1 3 0

, x =

x1x2

x3

, b =

32

5

.

86

Step 1 Write down the augmented matrix of coefficients and right-


52/81

p g ghand sides

3 1 2 : 32 1 1 : 21

3 0 : 5

.

Then, as in Gaussian elimination, we make terms below the leadingdiagonal equal to zero. Note that we need not write down equationsfor x1, x2, x3 at each stage, we can simply deal with the numericalcoefficients in the augmented matrix.

Carrying out the elimination process gives . . .

Step 2 Remove terms below leading diagonal in first column.

R1 R1R2 3R2 2R1R3

3R3

R1

3 1 2 : 30 5 1 : 120

8

2 : 12

Step 3 Remove terms below leading diagonal in second column.

R1 R1R2 R2R3 5R3 + 8R2

3 1 2 : 30 5 1 : 120 0 18 : 36

87

Step 4 Re write each row as an equation for and The last


53/81

Step 4 Re-write each row as an equation for x1, x2 and x3. The last

row gives an equation for x3:

18x3 = 36.

Solving this equation gives x3 = 2.

Step 5 Substitute x3 into the equation from row 2:

5x2 x3 = 12 5x2 = 12 + 2 = 10

x2 = 2.

Step 6 Substitute x2 and x3 into the equation from row 1:

3x1

x2 + 2x3 = 3

3x1 =

1

2

x1 = 1.

The solution is therefore x1 = 1, x2 = 2, x3 = 2.

El t ti


54/81

Elementary row operations

The operations in Steps 2 and 3 are examples of elementary row

operations (EROs).

There are three types of elementary row operations:

1. interchange any two rows;

This is equivalent to swapping the order of any two equations.

2. multiply any row by any nonzero constant;

This is equivalent to multiplying both sides of a given equation by

the constant.

3. add a multiple of one row to another.

This is equivalent to adding a multiple of one equation to another.

88


55/81

It is important to distinguish EROs from the broader range of

operations that may be applied to determinants. For example,

operations to columns are not allowed when solving a system of

equations by EROs.

3.8 Inversion by GaussJordan method


56/81

Row reduction methods can be used to find the inverse of a matrix.

We demonstrate by example.

Again only EROs may be applied: column operations are not allowed.

Example

Calculate the inverse of A = 1 1 3

2 1 11 3 5.

Step 1 Consider the augmented matrix

(A : I) =

1 1 3 ... 1 0 02 1 1 ... 0 1 0

1 3 5 ... 0 0 1

.

The method is to perform elementary row operations on this augmen-

ted matrix to reduce it to the form (I : B). The 3 3 matrix B isthen A1, the inverse of A. (Proof not required.)

89


57/81

Step 2 Clear the terms below leading diagonal of left-hand matrix. 1 1 3

... 1 0 02 1 1 ... 0 1 01 3 5 ... 0 0 1

R2R22R1R3R3R1

1 1 3... 1 0 0

0 1 5 ... 2 1 00 2 2 ... 1 0 1

R3R3+2R2

1 1 3 ... 1 0 00 1 5 ... 2 1 00 0

8 ...

5 2 1

90

Step 3 Clear the terms above leading diagonal of left-hand matrix


58/81

1 1 3

... 1 0 00 1 5 ... 2 1 00 0 8 ... 5 2 1

R28R25R3R18R1+3R3

8 8 0... 7 6 3

0 8 0 ... 9 2 50 0 8 ... 5 2 1

R1R1+R2

8 0 0 ... 2 4 2

0 8 0... 9 2 50 0 8 ... 5 2 1

R1R1/8R2,318R2,3

1 0 0 ... 1412 14

0 1 0 ... 98 14 580 0 1 ... 5

8 1

4 1

8

A1 =

14

12 14

98 14 5858 14 18

Exercise: Check that AA1 = I.

91


59/81

3.9 Iterative schemes

Gaussian elimination is often the method of choice for solving a linearsystem of n equations in n unknowns, but:

it requires a large number of algebraic manipulations, and

it is unforgiving of any errors made in the elimination procedure.

Iterative methods are an alternative. They generate successive ap-

proximations to the solution following a first guess.

Successive applications of the method give more and more accurateapproximations to the solution, until desired level of accuracy is rea-

ched.

92

3 9 1 Jacobi Iteration


60/81

3.9.1 Jacobi Iteration

Also known as method of simultaneous displacements.

Consider a system of n equations:

a11x1 + a12x2 + + a1nxn = b1a21x1 + a22x2 + + a2nxn = b2

...an1x1 + an2x2 +

+ annxn = bn.

(3.33)

Suppose:

(i) the solution to this system is unique (i.e., there is only one choice

of x1, . . . , xn that solves (3.33));

(ii) the diagonal coefficients a11, a22, . . . , ann are non-zero.

93

We can then rewrite each equation of the system (3.33) as follows:


61/81

(i) rearrange the first equation to express x1 in terms of the other xjs;

(ii) rearrange the second equation to express x2 in terms of the other

xjs;

and so on.

Hence, this procedure gives

x1 =1

a11(b1 a12x2 a13x3 a1nxn)

x2 =1

a22(b2

a21x1

a23x3

a2nxn)

...

xn =1

ann(bn an1x1 an2x2 an(n1)xn1)

(3.34)

94


62/81

If an initial approximation to the solution to (3.33) is made (e.g., by

guesswork) and then substituted into the right-hand side of (3.34), the

new values for x1, . . . , xn are an improved approximation to the solution.

If no initial guess is obvious we may take

x1 = 0, x2 = 0, . . . , xn = 0

as the initial guess.

This procedure can be repeated indefinitely, so that the refined appro-

ximation is itself substituted back into (3.34) to get a better approxi-

mation yet.

95

3.9.2 Example


63/81

Solve the system of equations

20x1 + x2 x3 = 17x1 10x2 + x3 = 13

x1+

x2+ 10

x3= 18

(3.35)

by Jacobi iteration.

Solution: First rewrite the equations as

x1 =1

20(17 x2 + x3)

x2 = 1

10(13 x1 x3)x3 =

1

10(18 + x1 x2)

i.e.,

x1 = 0.85

0.05x2 + 0.05x3

x2 = 1.3 + 0.1x1 + 0.1x3x3 = 1.8 + 0.1x1 0.1x2.

(3.36)

Since there is no obvious first guess, try

x1 = 0, x2 = 0, x3 = 0.

96


64/81

Then the refined guess is

x1 = 0.85, x2 = 1.3, x3 = 1.8.

To improve on this, substitute back into (3.36) to give

x1 = 0.85 + 0.05 1.3 + 0.05 1.8= 1.005

x2 = 1.3 + 0.1 0.85 + 0.1 1.8= 1.035

x3 = 1.8 + 0.1 0.85 0.1 (1.3)= 2.015

97


65/81


66/81

3.9.3 Convergence of iterative schemes

Sometimes, Jacobi iteration fails to give an answer, even after many

iterations. In this case, the scheme is said to diverge.

On the other hand, the scheme converges if the solution can be de-

termined to any desired level of accuracy provided enough iterations

are performed.

99

3.10 Eigenvalues and eigenvectors


67/81

Eigenvalue problems arise in many situations, for example,

calculating the natural frequences of oscillation of a vibrating system;

finding principal axes of stress and strain;

calculating the oscillations of an electrical circuit.

An eigenvalue problem takes the form

Find all the values of for which the equation

Ax = x, (3.37)

has a nonzero solutionx

, where A is an n n matrix andx

isan n 1 matrix (column vector).

Such values of are called eigenvalues of the matrix A and the cor-

responding vectors x are the eigenvectors.

100

Equation (3.37) may be written as


68/81

Ax = Ix

Ax Ix = 0

(A

I)x = 0. (3.38)

If det(A I) = 0, the matrix A I may be inverted, and the uniquesolution to (3.38) is

x = (A I)10 = 0.However, nonzero solutions x will exist if

det(A I) = 0. (3.39)

There are infinitely many eigenvectors corresponding to any giveneigenvalue. This is because: ifx is an eigenvector of A corresponding tothe eigenvalue and c is a (nonzero) scalar thencx is also an eigenvector

of A.

To see this, note that A(cx) = cAx = cx = (cx).

101

If A is the 2 2 matrix


69/81

A =

a bc d

then (3.39) is a quadratic equation for .

We can see this by calculating

|A I| =

a bc d

1 00 1

= a bc d

= 2 (a + d) + (ad bc).

So for to be an eigenvalue of A,

2 (a + d) + (ad bc) = 0.

Use standard formula for solving a quadratic to obtain eigenvalues.

102


70/81

In general, if A is an nn matrix then det(AI) is a polynomial of de-gree n in , called the characteristic polynomial. The characteristic

equation is

characteristic polynomial = 0,

i.e.,

n + cn

1n1 + cn

2

n2 +

+ c1 + c0 = 0. (3.40)

Coefficients c0, c1, . . . , cn can be found once the elements of A are

known.

The characteristic polynomial has n roots 1, 2, . . . , n. Some ofthese roots may be equal, in which case they are said to be repeated

roots.

103

3.10.1 Example


71/81

Find the eigenvalues and the corresponding eigenvectors of the matrix

A = 3 24 3 .Step 1 Find the characteristic polynomial.

det(A

I) = det 3 24 3

= (3 )(3 ) + 8= 2 9 + 8 = 2 1

Step 2 Solve the characteristic equation.

The roots of 2 1 = 0 are = 1 and = 1.

Step 3 For each eigenvalue find the corresponding eigenvector.

104

= 1


72/81

Write out

Ax = x

in full to give 3 24 3

xy

= 1

xy

=

xy

,

where x = xy . Hence3x 2y = x x = y4x 3y = y x = y.

Any vector of the form

1

1

is an eigenvector (where = 0 is an arbitrary constant).

105

= 1


73/81

Write out

Ax = x

in full to give 3 24 3

xy

= 1

xy

=

xy

Hence

3x 2y = x 2x = y4x 3y = y 2x = y.

Any vector of the form

1

2

is an eigenvector (where = 0 is an arbitrary constant).

106

3.10.2 Example


74/81

Find the eigenvalues and eigenvectors of

A = 1 2 10 1 21 0 3 .

Step 1 Find the characteristic polynomial. The determinant det(AI)is, expanding about the first column,

1 2 10 1 21 0 3

= (1 )

1 20 3

+

2 11 2

= (1 )[(3 )(1 ) 2 0]+ [2 2 1 (1 )]

= (1 )(2 + 2 3 ) + ( + 3)= (1 )( + 3)( 1 ) + ( + 3)= ( + 3)(2 + 2) = ( 2)( + 3).

107


75/81

Step 2 Solve the characteristic equation

( 2)( + 3) = 0.

The solutions are 0, 2 and 3.

Step 3 For each eigenvalue, find the corresponding eigenvector.

108

= 0


76/81

Write out

Ax = x

in full to give 1 2 10 1 2

1 0 3

xy

z

= 0

xy

z

Equivalently,

x + 2y + z = 0 (1)

y + 2z = 0 (2)

x 3z = 0 (3)

(3) x = 3z(2) y = 2z

109


77/81

Substitute into (1):

x + 2y + z = 3z 4z + z = 0.

So (1) is redundant (consistent)

Use these to write

x = 3

y = 2,where z = .

Then eigenvector corresponding to = 0 is

32

= 321

for any = 0.

110

= 2


78/81

Ax = x

1 2 10 1 2

1 0 3

xy

z

= 2

xy

z

Equivalently,

x + 2y + z = 2x

y + 2z = 2y

x 3z = 2zor

x + 2y + z = 0 (1)y + 2z = 0 (2)

x 5z = 0 (3)

(3) x = 5z(2) y = 2z

111


79/81

In (1): x + 2y + z = 5z + 4z + z = 0. So (1) is redundant

Therefore, if z = ,

52

1

is an eigenvector corresponding to = 2, for any = 0.

112

= 3


80/81

3

Ax = x 1 2 10 1 21 0 3

xyz

= 3 xyz

Equivalently,

x + 2y + z = 3xy + 2z = 3y

x 3z = 3zor

4x + 2y + z = 0 (1)

4y + 2z = 0 (2)

x = 0 (3)

113


81/81

(3)

x = 0

(2) y = z/2

and (1) is redundant

Therefore

0

121

is an eigenvector corresponding to = 3, for any = 0.

matrices and system of equation

Documents