ie301 03 review linear algebra

7/30/2019 IE301 03 Review Linear Algebra

1/68

IIKIE

Basic Linear Algebrabased on

Operations Research: Applications & Algorithms

4th edition, by Wayne L. Winston

Chapter 2

IE301 Operations Research I Spring 2013 Linear Algebra Review 1


2/68

IIKIE

Describe matrices and vectors with basic matrix operations.Describe matrices and their application to modeling systems

of linear equations.

Explain the application of the Gauss-Jordan method of

solving systems of linear equations.Explain the concepts of linearly independent set of vectors,

linearly dependent set of vectors, and rank of a matrix.

Describe a method of computing the inverse of a matrix.

Describe a method of computing the determinant of a matrix.

Chapter 2 - Learning Objectives



3/68

IIKIE

Matrix an rectangular array of numbers

1

3

2

4

1

4

2

5

3

6

1

2

2 1( )

Typical m x n matrix havingm rows and

n columns. We refer to m x

n as the order of the matrix.

The number in the ith row

and jth column of A is called

the ijth element of A and iswritten aij.

A

a 11

a 21

....

a m1

a 12

a 22

....

a m2

....

....

....

....

a 1n

a 2n

....

a mn

2.1 - Matrices and Vectors



4/68

IIKIE

Two matrices A = [aij] and B = [bij] are equal if and

only if A and B are the same order and for all i and j,

aij = bij.

If A1

3

2

4

and B

x

w

y

z

A = B if and only if x = 1, y = 2, w = 3, and z = 4




5/68

IIKIE

Vectors any matrix with only one column is acolumn vector. The number of rows in a

column vector is the dimension of the column

vector. An example of a 2 X 1 matrix or a two-

dimensional column vector is shown to the

right.

1

2

Rm will denote the set all m-dimensional column vectors

Any matrix with only one row (a 1 X n matrix) is

a row vector. The dimension of a row vector isthe number of columns.

1 2 3( )




6/68

IIKIE

Vectors appear in boldface type: for instance vectorv.

0 0 0( )

0

0

Any m-dimensional

vector (either row or

column) in which all the

elements equal zero is

called a zero vector

(written 0).

Examples are shown to

the right.




7/68

IIKIE

X1

X2

(1, 2)

(1, -3)

(-1, -2)

Any m-dimensional vector

corresponds to a directed linesegment in the m-dimensional

plane. For example, the two-

dimensional vector u corresponds

to the line segment joining thepoint (0,0) to the point (1,2)

The directed line segments

(vectors u, v, w) are shown on the

figure to the right.

u

1

2

v

1

3

w

1

2




8/68

IIKIE

The scalar product is the result of multiplying two vectors

where one vector is a column vector and the other is a rowvector. For the scalar product to be defined, the dimensions

of both vectors must be the same.

The scalar product of u and v is written:

u v u 1 v 1 u 2 v 2 .... u n v n

u 1 2 3( ) v

2

1

2

u v 1 2( ) 2 1( ) 3 2( ) 10

Example:




9/68

IIKIE2.1 - Matrices and Vectors

Then u.v is not defined

Also

u.v is not defined because the vectors are of different

dimensions.

32vand2

1u

4

3and321 vu



10/68

IIKIE

Two vectors are perpendicular if and only if their scalar

product equals 0.

[1 -1] [1 1] are perpendicular.

u.v = ||u|| ||v|| cos where ||u|| is the length of the vector

u and is the angle between the vectors u and v.

0)1.1(1.11

111




11/68


12/68

IIKIE

Addition of vectors (of

same degree). Vectors

may be added using the

parallelogram law or by

using matrix addition.

X1

X2

u

v

u+v

1 2 3

1

2

3

(1,2)

(2,1)

(3,3)

v 2 1( )

u 1 2( )

u v 2 1 1 2( )

u v 3 3( )




13/68

IIKIE

Line Segments can be defined

using scalar multiplication andthe addition of matrices.

If u = (1,2) and v = (2,1), theline segment joining u and v

(called uv) is the set of allpoints corresponding to thevectors

cu +(1-c)vwhere 0 c 1

See the example to the right.

X1

X2

u

v

1 2

1

2

c=1

c=1/2

c=0

End points




14/68

IIKIE

Transpose of a matrix

Given any m x n matrix

A

a11

a21

a31

a12

a22

a32

a13

a23

a33

AT

a11

a12

a13

a21

a22

a23

a31

a32

a33

A1

4

2

5

3

6

AT

1

2

3

4

5

6

Example

AT T AObserve:



d


15/68

IIKIE

Matrix multiplication

A1

2

1

1

2

3

B

1

2

1

1

3

2

Example

C5

7

8

11

C11

1 1 2( )

1

2

1

5 C12

1 1 2( )

1

3

2

8

C21

2 1 3( )

1

2

1

7 C22

2 1 3( )

1

3

2

11

Note: Matrix C will have the same number of rows as A and the same number of

columns as B.

Given to matrices A and B, the

matrix product of A and B (written

AB) is defined if and only if the

number of columns in A = the

number of rows in B.

The matrix product C = AB of Aand B is the m x n matrix C whose

ijth element is determined as

follows:

Cij = scalar product of

row i of A x column j of B




16/68

M i d V


17/68

IIKIE

Matrix Multiplication with Excel

A

1

2

1

1

2

3

B

1

2

1

1

3

2

A B C D

1 Matrix A 1 -1 2

2 2 1 3

3

4 Matrix B 1 1

5 2 3

6 1 2

7

8 A B = 1 2

9 7 11

Step 1 Enter matrix A intocells B1:D2 and matrix B into

cells B4:C6.

Step 2 Select (press F2) theoutput range (B8:C9) into

which the product will be

computed.

Step 3 In the upper left-handcorner (B8) of this selected

output range type the formula:

= MMULT(B1:D2,B4:C6).

Step 4 - Press CONTROL

SHIFT ENTER (not just enter)

Use the EXCEL MMULTfunction to multiply the matrices:



M t i d S t f Li E ti


18/68

IIKIE

Consider a system of linear equationsshown to the right. The variables(unknowns) are referred to as x1, x2, ,xn while the aijs and bjs are constants.

A set of such equations is called a linearsystem of m equations in n variables.

a1 1 x1 a1 2 x2 .... a1 n xn b1

a2 1 x1 a2 2 x2 .... a2 n xn b1

.... .... .... ....

am1 x1 am2 x2 .... amn xn bm

A solution to a linear set of m equations in n unknowns is a set of values for theunknowns that satisfies each of the systems m equations.

Example x1

2

x1 2x2 52x1 x2

0

x 1 1 x2 2

1 2 2( ) 5 2 1( ) 2 0

is a solution to the system

where

since (using substitution)

Given

2.2 Matrices and Systems of Linear Equations




19/68

IIKIE

Show if

is a solution to the system of linear equations ??

1

3x

t is not a solution



IE



20/68

IIKIE

Matrices can simplify and compactly represent a system of linear

equations.

A

a 11

a 21

....

a m1

a12

a 22

....

a m2

....

....

....

....

a 1n

a 2n

....

a mn

x

x 1

x 2

. . . .

x n

b

b 1

b 2

....

b m

A system of linear equations may be written Ax = b and is called its matrixrepresentation. The matrix multiplication (using only row 1 of the A matrix for

example) confirms this representation thus:

a 11 x1 a 12 x2 .... a 1n xn b 1



IE

2 2 M t i d S t f Li E ti


21/68

IIKIE

Ax = b can sometimes be abbreviated A|b.

For example, given:

A

1

2

2

1

x

x1

x2

b

5

0

A|b is written:

1

2

2

1

5

0



IIKIE

2 2 M t i e d S te of Li e E tio


22/68

IIKIE2.2 Matrices and Systems of Linear Equations

Show the system of equations the above matrix

represents?

1111

3210

2321


IIKIE

2 3 The Gauss Jordan Method


23/68

IIKIE

Using the Gauss-Jordan method, we can show that any system of

linear equations must satisfy one of the following three cases:

Case 1 The system has no solution.

Case 2 The system has a unique solution.

Case 3 The system has an infinite number of solutions.

The Gauss-Jordan method is important because many of themanipulations used in this method are used when solving

linear programming problems by the simplex algorithm (see

Chapter 4).

2.3 The Gauss-Jordan Method


IIKIE



24/68

IIKIE

Elementary row operations (ero).

An ero transforms a given matrix A into a newmatrix A via one of the following operations:

Type 1 eroMatrix A is obtained by multiplying anyrow of A by a nonzero scalar.

Type 2 ero Multiply any row of A (say, row i) by anonzero scalar c. For some j i, let:

row j of A = c*(row i of A) + row j of A.

Type 3 ero Interchange any two rows of A.



IIKIE



25/68

IIKIE

ex.

3210

6531

4321

3210

181593

4321

2*3 row

2722134

6531

4321

4321

6531

3210

31 rowrow

324 rowrow



IIKIE



26/68

IIKIE

ex.

742

2

21

21

xx

xx

2

3

2

1

2

1

x

x

742

211

2

310

2101

Multiply first equation by

-2 and add to the second

and write to the second

32

2

2

21

x

xx

2

3

2

2

21

x

xx Multiply second equation

by -1 add to the first write

to the first

divide second equation by

2



IIKIE



27/68

IIKIE

The augmented form of the matrix

742

211

2

310

2

101



IIKIE



28/68

IIKIE

Finding a solution using the Gauss-Jordan method.

The Gauss-Jordan method solves a linear equation system byutilizing eros in a systematic fashion. The steps to use theGauss-Jordan method (with an accompanying example) areshown below:

Illustration of Gauss-Jordan method is done by finding thesolution to the following linear system.

52

622

922

321

321

321

xxx

xxx

xxx



IIKIE

2 3 The Gauss-Jordan Method


29/68

IIKIE

Solve Ax = b where: A

2

2

1

2

1

1

1

2

2

b

9

6

5

Step 1 Write theaugmented matrix

representation:

2

2

1

2

1

1

1

2

2

9

6

5

A|b =

IE301 Operations Research I Spring 2013


Linear Algebra Review 29

IIKIE



30/68

IIKIE

The method uses eros to

transform the left side of A|b intoan identity matrix. The solutionwill be shown on the right side.See x to the right.

1

0

0

0

1

0

0

0

1

1

2

3

x

1

2

3

Step 2 At any stage, define a current row,current column, and current entry (the entry in

the current row and column). Begin with row

1 as the current row and column 1 as the

current column, and a11 (a11 = 2) as the

current entry.

2

2

1

2

1

1

1

2

2

9

6

5

A|b =

If the current entry (a11) is nonzero, use eros to transformthe current column (column 1) entries to:

1

0

0



IIKIE



32/68

IIKIE

Then obtain the new current row, column, and entry by

moving down one row and one column to the right and go

to Step 3.

If a11 (current entry) would have equaled zero, then do a

type 3 ero with the current row and any row that contains a

nonzero entry in the current column. Use eros totransform column 1 as shown to the right.

If there are no nonzero numbers in the current column,

obtain a new current column and entry by moving one

column to the right.

1

0

0

Step 3 - If the new current entry is non zero, use eros to transform it to 1 andthe rest of the column entries to 0. Repeat this step until finished.

2.3 The Gauss Jordan Method


IIKIE



34/68

IIKIE

Repeating Step 3 again for the another

new entry (a33) and performing anadditional three eros yields the finalaugmented array:

1

0

0

0

1

0

0

0

1

1

2

3

A9|b9 =

Special Cases: After application of the Gauss-Jordan method, linear

systems having no solution or infinite number of solutions can be

recognized.

No solution example: Infinite solutions example:

1

0

2

0

3

2

1

0

0

0

1

0

10

0

23

0



IIKIE



35/68

IIKIE

Show that

442

32

21

21

xx

xx Leads to no solution

42

3

1

321

32

21

xxx

xx

xx

Leads to infinite number of solutions




IIKIE



36/68

IIKIE

Summary of the Gauss-Jordan Method

Step 1

To solve

Write down the augmented matrix

Step 2

Use EROs to transform ex. The first column as

Step 3 write down the system of equations

bx A

b|A

0

.

0

1

0

.

1

0

1

.

0

0




IIKIE 2.3 The Gauss-Jordan Method


37/68

Let BV be the set of basic variables for Ax=b and NBV be theset of nonbasic variables for Ax=b. The character of the

solutions to Ax=b (and Ax=b) depends upon which offollowing cases occur.

For any linear system, a variable that appears with a coefficient of 1

in a single equation and a coefficient of 0 in all other equations is

called a basic variable (BV).

Any variable that is not a basic variable is called a nonbasic variable

(NBV).

Basic variables and solutions to linear equation systems





38/68

1

0

0

0

0

0

1

0

0

0

0

0

1

0

0

1

2

3

0

0

1

3

10

2

Case 1: Ax=b has atleast one row of the

form [0, 0 , , 0 | c] (c 0). Then Ax=b(and Ax = b) has no

solution. In the matrixto the right, row 5

meets this Case 1

criteria. Variables x1,

x2, and x3 are basicwhile x4 is a nonbasic

variable.

Has no solution.

A|b =

'' bx A

x1 x2 x3 x4

BV NBV





39/68

1

0

0

0

1

0

0

0

1

1

2

3

Case 2: Suppose

Case 1 does not applyand NBV, the set of

nonbasic variables is

empty. Then Ax=b

will have a uniquesolution. The matrix to

the right has a unique

solution. The set of

basic variables is x1,

x2, and x3 while the

NBV set is empty.

A|b =

This is a solution to the equations given on next slide.

x1 x2 x3

BV


3 e Gauss Jo da et od




40/68

52

6222

922

321

321

321

xxx

xxx

xxx


3




41/68

Case 3: Suppose

Case 1 does not

apply and NBV is

not empty. Then

Ax=b (and Ax = b)will have an infinite

number of solutions.

BV = {x1, x2, and x3}

while NBV = {x4 andx5}.

1

0

0

0

0

1

0

0

0

0

1

0

1

2

0

0

1

0

1

0

3

2

1

0

A|b =

x1 x2 x3 x4 x5

BV NBV


3




42/68

Does A' |b '

have a row [ 0, 0 , ..., 0 | c ] (c < > 0 ) ?

Ax = b has no

solution.Find BV and NBV.

Is NBV empty?

Ax = b has a unique

solution.

Ax = b has an infinite

number of solutions.

Yes No

Yes

No

A summary of

Gauss-Jordan

method is

shown to the

right. The end

result of theGauss-Jordan

method will be

one either Case

1, Case 2, orCase 3.


3


IIKIE 2.4 Linear Independence and Linear Dependence


43/68

Let V = {v1, v2, , vk} be a

set of row vectors allhaving the same

dimension.

A linear combination of

the vectors in V is anyvector of the form c1v1 +

c2v2+ + ckvk where c1,c2, , ck are arbitraryscalars.

Example:

If V = { [1 , 2] , [2 , 1] }

2v1 v2 = 2([1 2]) [2 1] = [0 3]

and

v1 + 3v2 = [1 2] + 3([2 1]) = [7 5]

0v1 + 3v2 = [0 0] + 3[2 1] = [6 3]


4 p p




44/68

A set of vectors is called linearly independent if the only linear

combination of vectors in V that equals 0 is the trivial linearcombination (c1 = c2= = ck = 0).

A set of vectors is called linearly dependent if there is a nontrivial

linear combination of vectors in V that adds up to 0.

Example

Show that V = {[ 1 , 2 ] , [ 2 , 4 ]} is a linearly dependent set of

vectors.

Since 2([ 1 , 2 ]) 1([ 2 , 4 }) = (0 0), there is a nontrivial linearcombination with c1 =2 and c2 = -1 that yields 0. Thus V is a

linear dependent set of vectors.


4 p p




45/68

Example 8 (0 vector makes Set LD)

Show that any set vectors containing the 0 vector is a linearly dependent set.

The V is linearly dependent because

Say then

There is a nontrivial linear combination of vectors in V that adds up to 0.

]}10[],01[],00{[ V

01c

]00[]10[0]01[0]00[1 c


p p




46/68

Example 9 Linearly independent (LI) set of vectors

Show that the set of vectors

are linearly independent set of vectors

Try to find a nontrivial linear combination of the vectors in V that yield 0.

Thus c1 and c2 must satisfy

This implies that which is the trivial linear combination of

vectors

Therefore V is linearly independent set of vectors.

]}10[],01{[ V

]00[]10[]01[21

cc

]00[][ 21cc

021cc


p p




47/68

Example 10 Linearly Dependent (LD) Set of Vectors

Show that

is a linearly dependent set of vectors.

Because

There is a nontrival linear combination with c1=2 and c2=-1 that yield 0.

Thus V is a linearly dependent vectors.

]}42[],21{[ V

]00[]42[1]21[2




48/68

X1

X2

X2

X1

1 2

1

2

2

1

1 2

v1 = (1 1)

v2 = (2 2)

v1 = (1 2)

v2 = (2 1)

Linearly Dependent

Linearly Independent

What does it mean for a set of

vectors to be linearly dependent?A set of vectors is linearly dependent

only if some vector in V can be

written as a nontrivial linear

combination of other vectors in V. If a

set of vectors in V are linearlydependent, the vectors in V are, in

some way, NOT all different vectors.By different we mean that thedirection specified by any vector in V

cannot be expressed by adding

together multiples of other vectors in

V. For example, in two dimensions,

two linearly dependent vectors lie on

the same line.




49/68

The Rank of a Matrix: Let A be any m x n matrix, and denote the rows

of A by r1, r2, , rm. Define R = {r1, r2, , rm}.The rank of A is the number of vectors in the largest linearlyindependent subset of R.

A

1

0

1

0

1

1

0

0

0

To find the rank of matrix A, apply

the Gauss-Jordan method to

matrix A. Let A be the finalresult. It can be shown that the

rank of A = rank of A. The rankof A = the number ofnonzero

rows in A. Therefore, the rank A= rank A = number of nonzerorows in A.

Given V = {[1 0 0], [0 1 0], [1 1 0]}

After theGauss-Jordan

method:

Form

matrix A

A'

1

0

0

0

1

0

0

0

0


IIKIE

2.4 Linear Independence and Linear Dependence


50/68

Example 11 Matrix with zero Rank

Show that rank A=0

For the set of vectors

t is imposible to choose a subset of R that is linearly independent

A

00

00

}00,00{R


IIKIE



51/68

Example 12 Matrix with Rank of 1

Show that rank A=1 for the following matrix

Here

The set is a linearly independent subset of R, so rank Amust be at least 1.

independent set of vectors of R will fail because

(dependent) this means rank cannot be 2.

22

11A

}22,11{R

}11{

0022112


IIKIE



52/68

Example 13 Matrix with Rank of 2

Show that rank A=2 for the following matrix

Here

From ex. 9 we know that R is linearly independent. Thus the rank

A=2.

10

01A

}10,01{R


IIKIE



53/68

Using Gauss-Jordan Method to Find Rank of Matrix

Performing a sequence of eros on a matrix does not change therank of the matrix

Thus,

320

120

001

rankA

100

010

001

100

2

110

001

200

2

110

001

320

2

110

001

320

120

001

3rankrank AA


IIKIE



54/68

A method of determining

whether a set of vectorsV = {v1, v2, , vm} islinearly dependent is to

from a matrix A whose ith

row is vi. If the rank of A

= m, then V is a linearly

independent set of

vectors. If the rank A

ie301 03 review linear algebra

Documents