math 314, lecture 14 row reductiontpantev/314/notes/math314lec14.pdf · b p km is a given vector of...

Math 314, lecture 14

Row reduction

Instructor: Tony Pantev

University of Pennsylvania

March 23, 2020

Instructor: Tony Pantev University of Pennsylvania


Outline

Simplifying linear systems

Row reduction and echelon forms

Solving systems with row reduction

Corollaries



Solving a linear system (i)

Recall: Using row and column operations we can convertevery linear system into a very simple system in which allvariables separate.

Explanation: Consider a linear system

Ax “ b

where

A P MatmˆnpKq, is a given coefficient matrix;

b P Km is a given vector of right hand sides;

x P Kn is an unknown vector.



Solving a linear system (ii)

We proved that we can find invertible matrices

R P MatmˆmpKq, C P MatnˆnpKq

such that after performing on A the row operationcorresponding to R and the column operation correspondingto C we get the simplest possible matrix

rA “ RAC “

ˆIr 00 0

˙

Where r is the rank of the matrix A.



Solving a linear system (iii)Using R and C we can simplify the system Ax “ b.Multiplying Ax “ b on the left by R gives an equivalent system

RAx “ Rb ô RACC´1x “ Rb.

Using the shortcut notation

rA :“ RAC P MatmˆnpKq,

rb :“ Rb P Km

rx :“ C´1x P Kn

we can rewrite this equivalent system as

rArx “ rb.



Solving a linear system (iv)

Therefore in terms of the new variables rx “

¨˚̋

rx1...

rxn

˛‹‚“ C´1x the

system becomes rArx “ rb or explicitlyˇ̌ˇ̌ˇ̌ˇ̌ˇ̌ˇ̌ˇ̌ˇ̌ˇ̌

rx1 “ rb1rx2 “ rb2

¨ ¨ ¨

rxr “ rbr0 “ rbr`1

¨ ¨ ¨

0 “ rbmInstructor: Tony Pantev University of Pennsylvania


Solving a linear system (v)

Conclusion: Get a complete description of the solutions ofrArx “ rb and hence of Ax “ b.

Indeed, the equations imply:

(1) The system rArx “ rb (and hence the system Ax “ b) is

consistent if and only if rbr`1 “ ¨ ¨ ¨ “ rbm “ 0.



Solving a linear system (vi)

(2) If rbr`1 “ ¨ ¨ ¨ “ rbm “ 0 then the solutions rArx “ rb are allvectors of the form ¨

˚̊˚̊˚̊˚̊˝

rb1...

rbrrxr`1

...rxn

˛‹‹‹‹‹‹‹‹‚

,

where rxr`1, . . . , rxn P K are arbitrary numbers.



Solving a linear system (vii)

(3) Similarly if rbr`1 “ ¨ ¨ ¨ “ rbm “ 0, then the solutions ofAx “ b are all vectors of the form

x “ C ¨

¨˚̊˚̊˚̊˚̊˝

rb1...

rbrrxr`1

...rxn

˛‹‹‹‹‹‹‹‹‚

,

where rxr`1, . . . , rxn P K are arbitrary numbers.



Solving a linear system (viii)

Drawback: This complete solution of the linear systemAx “ b is not constructive.





to produce R and C we had tochoose special bases in K

n andK

m that were adapted to A;

there are many such choices;





The Row reduction algorithm solves this problem.

Special features of the algorithm:

The algorithm simplifies the system systematically.

Uses only row operations (no column operations).

Done in steps with each step being a very simple rowoperation.

Does not separate the variables completely but separatesenough of them to allow us to solve the system.



Elementary row operations (i)

The row reduction algorithm is based on simple row operationsthat come in three flavors:

Definition: The elementary row operations on a linear sys-tem F pvq “ b on a vector space V are

(Type 1) exchange two equations in the system,

(Type 2) scale an equation by a non-zero scalar,

(Type 3) add to equation #k a scalar multiple of equation#j for some j ‰ k.



Elementary row operations (ii)

If we choose a basis in V and rewrite F pvq “ b as a matrixequation Ax “ b, then the elementary row operations becomespecial row operations on the augmented matrix

pA|bq P Matmˆn`1pKq.

(Type 1) exchange two rows of pA|bq,

(Type 2) scale a row of pA|bq by a non-zero scalar,

(Type 3) replace row #k of pA|bq withprow#kq ` cprow#jq for some j ‰ k and somec P K.



Elementary row operations (iii)Note:

Elementary row operations are row operations - theyreplace rows with linear combinations of rows in aninvertible way.

Inverses of elementary row operations are elementary rowoperations of the same type.

operation inverse operation

rowk Ø rowj rowk Ø rowj

rowk Ñ c ¨ rowk rowk Ñ 1c

¨ rowk

rowk Ñ rowk ` c ¨ rowj rowk Ñ rowk ´ c ¨ rowj



Elementary row operations (iv)

m ˆ m matrices which via left multiplication reproduce theelementary row operations:

Type 1

operation matrix

rowk Ø rowj

¨˚̊˚̊˚̊˚̊˝

j k

1. . .

j 0 ¨ ¨ ¨ 1.... . .

...k 1 ¨ ¨ ¨ 0

. . .1

˛‹‹‹‹‹‹‹‹‚



Elementary row operations (v)


Type 2

operation matrix

rowk Ñ c ¨rowk

¨˚̊˚̋

k

1. . .

k c

. . .1

˛‹‹‹‚



Elementary row operations (vi)


Type 3

operation matrix

rowk Ñ rowk `c ¨rowj

¨˚̊˚̊˚̊˚̊˝

j k

1. . .

j 1.... . .

k c ¨ ¨ ¨ 1. . .

1

˛‹‹‹‹‹‹‹‹‚



Compositions of row operations (i)The total result of a sequence of row operations is again a rowoperation.

Indeed if ρ1, . . . , ρs : Km Ñ K

m are invertible linear maps,then applying these as a sequence of row operations on alinear system F pvq “ b results in the equivalent system

ρspρs´1p¨ ¨ ¨ ρ1pF pvqq ¨ ¨ ¨ qq “ ρsp¨ ¨ ¨ ρ1pbq ¨ ¨ ¨ q.

But in terms of composition of maps this is the same as

pρs ˝ ¨ ¨ ¨ ˝ ρ1qpF pvqq “ pρs ˝ ¨ ¨ ¨ ˝ ρ1qpbq,

i.e. is the linear system obtained from F pvq “ b by applyingthe single row operation

ρ “ ρs ˝ ¨ ¨ ¨ ˝ ρ1.



Compositions of row operations (ii)

Let Ri P MatmˆmpKq denote the matrix of ρi in the standardbasis in K

m, i.e. ρi is given by left multiplication by Ri .Then the composite row operation ρ : Km Ñ K

m is given byleft multiplication by the matrix

R “ RsRs´1 ¨ ¨ ¨R1.

In particular we have the following

Conclusion: Performing a sequence of elementary rowoperations is the same thing as left multiplication by theproduct of the matrices of the elementary row operations.



Row reduction (i)Goal: Use elementary row operations to reduce every matrixto a simple form.

Definition: A matrix A P MatmˆnpKq is an echelon matrix or amatrix in an echelon form if it satisfies the following properties(a) If rowi “ 0, then rowj “ 0 for all j ě i .

(b) If rowi`1 ‰ 0, then its first non-zero entry (called a pivot)is strictly to the right of the first non-zero entry of rowi .

A matrix A is a reduced echelon matrix if in addition it satisfies(c) Each pivot is equal to 1.

(d) All entries of A above a pivot are equal to 0.

Note: In an echelon matrix all entries below a pivot areautomatically zero.



Row reduction (ii)

Claim: Every matrix A P MatmˆnpKq can be modified intoa reduced echelon matrix Are P MatmˆnpKq by a sequence ofelementary row operations.

Proof: The modification is achieved by the row reductionalgorithm which proceeds in steps as follows

Step 1: Find the first column of A that contains a non-zeroentry. If there is no such column then A is the zero matrixwhich is a reduced echelon matrix.

Step 2: Find the first non-zero entry, say a in the firstnon-zero column and use elementary row operations of Type 1to move a to the first row.



Row reduction (iii)Step 3: Rescale the first row by 1

a, that is - use an operation

of Type 2 to normalize the first non-zero entry (pivot) of thefirst row to be equal to 1.

Step 4: Use the pivot in row1 and operations of Type 3 toclear all entries below the pivot.







i.e. make the entriesequal to 0







Concretely: at this stage the matrix is

¨˚̊˚̊˚̋

0 ¨ ¨ ¨ 0 1 ˚ ¨ ¨ ¨ ˚0 ¨ ¨ ¨ 0 a2ℓ ˚ ¨ ¨ ¨ ˚0 ¨ ¨ ¨ 0 a3ℓ ˚ ¨ ¨ ¨ ˚... ¨ ¨ ¨

......

... ¨ ¨ ¨...

0 ¨ ¨ ¨ 0 amℓ ˚ ¨ ¨ ¨ ˚

˛‹‹‹‹‹‚



Row reduction (iv)We clear the entries below the pivot by performing the Type 3operations

row2 Ñ row2 ´ a2ℓ ¨ row1,

row3 Ñ row3 ´ a3ℓ ¨ row1,

¨ ¨ ¨

rowm Ñ rowm ´ amℓ ¨ row1.

The resulting matrix will be equal to¨˚̊˚̊˚̋

0 ¨ ¨ ¨ 0 1 ˚ ¨ ¨ ¨ ˚0 ¨ ¨ ¨ 0 0 ˚ ¨ ¨ ¨ ˚0 ¨ ¨ ¨ 0 0 ˚ ¨ ¨ ¨ ˚... ¨ ¨ ¨

......

... ¨ ¨ ¨...

0 ¨ ¨ ¨ 0 0 ˚ ¨ ¨ ¨ ˚

˛‹‹‹‹‹‚

or in block form

ˆ0 1 B1

0 0 C1

˙



Row reduction (v)

The matrix`0 0 C1

˘has m ´ 1 ă m rows so by induction we

can use Steps 1-4 to modify this matrix into a matrix

`0 0 C2

˘

where C2 is an echelon matrix in which every non-zero rowstarts with a pivot. Finally we use the pivots in the matrix

ˆ0 1 B1

0 0 C2

˙

to clear all entries above the pivots in C2. The result is thereduced echelon form of A. l.



Row reduction (vi)

Example: Find the reduced echelon form and all solutions ofthe linear system

ˇ̌ˇ̌ˇ̌

x1 ` 2x2 ´ x3 ` 3x4 “ 22x1 ` 4x2 ´ x3 ` 6x4 “ 5

x2 ` 2x4 “ 3

The augmented matrix of the system is

¨˝1 2 ´1 3 22 4 ´1 6 50 1 0 2 3

˛‚

Following the row reduction algorithm we get



Row reduction (vii)¨˝1 2 ´1 3 22 4 ´1 6 50 1 0 2 3

˛‚ row2 Ñ row2 ´ 2row1

//

¨˝1 2 ´1 3 20 0 1 0 10 1 0 2 3

˛‚ row2 Ø row3

//

¨˝1 2 ´1 3 20 1 0 2 30 0 1 0 1

˛‚ row1 Ñ row1 ´ 2row2

//

¨˝1 0 ´1 ´1 ´40 1 0 2 30 0 1 0 1

˛‚ row1 Ñ row1 ` row3

//



Row reduction (vii)¨˝1 0 ´1 ´1 ´40 1 0 2 30 0 1 0 1

˛‚ row1 Ñ row1 ` row3

//

¨˝1 0 0 ´1 ´30 1 0 2 30 0 1 0 1

˛‚

Hence the linear system is equivalent to

ˇ̌ˇ̌ˇ̌

x1 ´ x4 “ ´3x2 ` 2x4 “ 3

x3 “ 1

and x4 is free to take any value c P K.Instructor: Tony Pantev University of Pennsylvania


Row reduction (viii)

Solving the equations in terms of x4 we get that the solutionsof the system are given by

¨˚̊˝

x1x2x3x4

˛‹‹‚“

¨˚̊˝

´3 ` c

3 ´ 2c1c

˛‹‹‚, for all c P K.

Note: The method employed in the example works for solvingany linear system by using row reduction.



Solving by row reduction (i)

Let A P MatmˆnpKq, b P Kn be given coefficient matrix and

right hand side and consider the linear system

Ax “ b.

LetpAre| breq

be a reduced echelon form of the augmented matrix pA|bq.

Terminology: A variable xi is called a pivot variable for A ifthe i -th column of Are contains a pivot. A variable xi is calleda free variable for A if the i -th column of Are does notcontain a pivot.



Solving by row reduction (ii)

The systemArex “ bre

is equivalent to the system Ax “ b and has a solution if andonly if its augmented matrix pAre| breq does not contain anyrows of the form

p0, 0, . . . , 0|βq with β ‰ 0.

In other words Arex “ bre and Ax “ b are consistent if andonly if pAre|breq does not contain any non-zero rows whoseportion in Are is zero, i.e. if and only if all pivots in pAre|breqbelong to Are.



Solving by row reduction (iii)

Furthermore if the system is consistent, then moving all freevariables to the right hand side of Arex “ bre gives expressionsfor all pivot variables in terms of the free variables.Therefore all solutions of Arex “ bre and hence of Ax “ b areobtained by giving arbitrary values to the free variables andsubstituting them in the expressions for the pivot variables.

This explains how to use row reduction to check consistencyand find all solutions of a linear system.

Note: It is not hard to follow the row reduction algorithm andshow that the reduced echelon form of a matrix is unique. Wewill not be needing this in any essential way so this fact is leftas an exercise.



Corollaries of row reduction (i)

Claim: Let A P MatmˆnpKq and let Are be the reduced echelonform of A. Then

(a) If Ax “ b has a solution, then this solution is unique ifand only if there are no free variables for A, i.e. if andonly if Are has a pivot in every column.

(b) Ax “ b is consistent for every choice of b P Km if and

only if Are has a pivot in every row.

(c) Ax “ b has a unique solution for all b P Km if and only if

Are has a pivot in every column and in every row. Inparticular n “ m.



Corollaries of row reduction (ii)Proof: Part (b) is the consistency criterion we just explainedand part (a) follows from the description of all solutions interms of pAre|breq. Part (c) is the combination of (a) and (b).

l

In a similar manner the reduced echelon form of a matrix canbe used to detect the dependence/generating properties of acollection of vectors.Given a collection of vectors

v1, . . . , vn P Km

denote the m ˆ n matrix with columns v1, . . . , vn by

A “ pv1, ¨ ¨ ¨ , vnq.



Corollaries of row reduction (iii)

Proposition: Let v1, . . . , vn P Km and let Are be the reduced

echelon form of A “ pv1, ¨ ¨ ¨ , vnq. Then

(i) v1, . . . , vn are linearly independent if and only if Are hasa pivot in every column.

(ii) v1, . . . , vn span Km if and only if Are has a pivot in every

row.

(iii) v1, . . . , vn is a basis of Km if and only if Are has a pivotin every row and every column, i.e. Are “ In.



Corollaries of row reduction (iv)

Proof: Part (iii) is the combination of parts (i) and (ii). Part(i) follows from part (a) of the Claim by noticing that thelinear independence of the vj is the statement that Ax “ 0 hasonly the zero solution. Similarly part (ii) follows from part (b)of the Claim by observing that the vj ’s generate K

m if andonly if Ax “ b has a solution for any choice of b P K

m.l



Corollaries of row reduction (v)Note: The row reduction algorithm and the Claim andProposition give alternative proofs of several facts we havealready proven, e.g.

The fundamental lemma of linear algebra: more than n

linear combinations of n vectors are always linearlydependent.

If dimV “ m and v1, . . . , vn P V are linearly independent,then n ď m.

If dimV “ m and v1, . . . , vn P V span V , then n ě m.

Any two bases of a finite dimensional space have thesame cardinality.

An m ˆ n matrix A is invertible if and only if its reducedechelon form is equal to In. In particular m “ n.



Rank of a matrix

Rank of a matrix (i)

Recall: Left multiplication by matrix A P MatmˆnpKq definesa linear map

F : Kn

// Km,

v ✤

// Av.

We defined the rank of A to be the rank of the map F , i.e.the dimension of the subspace impF q Ă K

m:

rankpAq “ rankpF q “ dim impF q.

Note: It is easy to describe the subspace impF q Ă Km and

the rankpAq in terms of the matrix A directly.



Rank of a matrix

Rank of a matrix (ii)Indeed by definition

impF q “!y P K

mˇ̌ˇ y “ F pvq “ Avfor some v P K

n

)

Now note that:

Kn is spanned by the vectors of the standard basis

e1, . . . , en and therefore impF q is spanned by the vectors

Ae1, . . . ,Aen P Km;

by definition of matrix multiplication

Aei “ i -th column of A.



Rank of a matrix

Rank of a matrix (iii)

ThusimpF q “ span pcolumns of Aq .

Back



Rank of a matrix



the column space of A

Back



Rank of a matrix



the column space of AAnd so we have

rankpAq “ dimpcolumn space of Aq.

Back



math 314, lecture 14 row reductiontpantev/314/notes/math314lec14.pdf · b p km is a given vector of...

Documents