essence of linear algebra - some cool intuitions · essence of linear algebra shreedhar kodate the...

Essence of LinearAlgebra

Shreedhar Kodate

The Geometry ofLinear Equations

Vectors and Basisvectors

Linear combinationsand Span

The box game:Matrices

Elimination andMultiplication, A=LU

Transforming yourLIFE Leenearly

Cool Video, TheDeterminant

Space Tour

Column space, Nullspace, Inverses

Celebrity: TheRank

Solution concept

Some things ofEigen

Eigen values, Eigenvectors, Change ofBasis

Summary

Essence of Linear AlgebraSome cool intuitions

Shreedhar Kodate1

1Department of Computer Science and AutomationIndian Institute of Science, Bengaluru

CSA Summer School, 2017


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Disclaimer

All the credits for the content goes to the respective authorslisted in the Appendix: Further Learning.


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

The Storyline

The Geometry of Linear EquationsVectors and Basis vectorsLinear combinations and Span

The box game: MatricesElimination and Multiplication, A=LU

Transforming your LIFE LeenearlyCool Video, The Determinant

Space TourColumn space, Null space, Inverses

Celebrity: The RankSolution concept

Some things of EigenEigen values, Eigen vectors, Change of Basis


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Vectors

What even are they?

I A line with a arrowhead?OR

I A set of numbers arranged vertically?1

2

3

OR

I An abstract #»v


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Vectors

Abstract view1

#»u + ( #»v + #»w ) = ( #»u + #»v ) + #»w

#»v + #»w = #»w + #»v

There is a zero vector#»0 such that

#»0 + #»v = #»v for all #»v . For

every vector #»v there is a vector − #»v so that #»v + (− #»v ) =#»0 .

a(# »

bv) = (ab) #»v 1 #»v = #»v

a( #»v + #»w ) = a #»v + a #»w

(a + b) #»v = a #»v + b #»v

1Abstract vector spaces — Essence of Linear Algebra, Chapter 11


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Basis Vectors

These are the vectors which can define the entirecoordinate space.

I Do you recognize this special vector?#»i#»j#»

k

I Also, have you seen this?1 1

0 1

This is a special matrix called Shear matrix.


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Basis vectors

Example

The image2 below shows the basis vectors of a Shear matrix.

2Essence of Linear Algebra, Chapter 3


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

The Storyline








Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Linear combinations

Additivity 1

2

+

6

3

=

7

5

#»u + #»v = #»w

Scaling

2

1

2

=

2

4

2 #»v =# »

(2v)

Hybrid

2

1

2

+ 3

6

3

=

20

13

a #»u + b #»v = #»w


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Span

Example

The image3 below shows how two vectors can span the 2Dspace.



Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

The Storyline








Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Elimination

Gauss-Jordan EliminationA method of solving a linear system of equations. This isdone by transforming the system’s augmented matrix intoreduced row-echelon form (rref) by means of row operations.Types of row Operations:Type 1: Swap the positions of two rows.Type 2: Multiply a row by a nonzero scalar.Type 3: Add to one row a scalar multiple of another.

Example

RREF =

1 0 0 0

0 1 0 0

0 0 1 2

0 0 0 0


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Elimination using Multiplication

Example

x + 2y + z = 2

3x + 8y + z = 12

4y + z = 21 0 0

0 1 0

0 −2 1

1 0 0

−3 1 0

0 0 1

1 2 1

3 8 1

0 4 1

=

1 2 1

0 2 −2

0 0 5

E32E31E21A = U


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

A = LU

Example1 2 1

3 8 1

0 4 1

=

1 0 0

3 1 0

0 0 1

1 0 0

0 1 0

0 2 1

1 2 1

0 2 −2

0 0 5

EA = U

A = (E32E31E21)−1U

A = E−121 E−1

31 E−132 U

A = LU

So,A = E−1U.Therefore, L = E−1


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

The Storyline








Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Cool Video

Definition CoolThe phrase ”cool” is very relaxed, never goes out of style,and people will never laugh at you for using it.

Definition VideoMake a video recording of (something broadcast ontelevision). YouTube nowadays.

TheoremCool + Video = JUST START PLAYING THE VIDEO!

Example

Here You Go: HUGO


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Cool Video0000 0018 6674 7970 6d70 3432 0000 0000 6973 6f6d 6d703432 0003 34bf 6d6f 6f76 0000 006c 6d76 6864 0000 0000d42c 59a6 d42c 59a6 0000 0258 0006 06bf 0001 0000 01000000 0000 0000 0000 0000 0001 0000 0000 0000 0000 00000000 0000 0001 0000 0000 0000 0000 0000 0000 0000 40000000 0000 0000 0000 0000 0000 0000 0000 0000 0000 00000000 0000 0000 0003 0000 0015 696f 6473 0000 0000 1007004f ffff 2915 ff00 0151 fd74 7261 6b00 0000 5c74 6b686400 0000 0f00 0000 00d4 2c59 ae00 0000 0100 0000 00000606 6600 0000 0000 0000 0000 0000 0000 0000 0000 01000000 0000 0000 0000 0000 0000 0000 0100 0000 0000 00000000 0000 0000 0040 0000 0002 8000 0001 6800 0000 00002465 6474 7300 0000 1c65 6c73 7400 0000 0000 0000 01000606 6600 0000 0000 0100 0000 0151 756d 6469 6100 0000206d 6468 6400 0000 0000 0000 00d4 2c59 aa00 0075 30012d40 0355 c400 0000 0000 2d68 646c 7200 0000 0000 00000076 6964 6500 0000 0000 0000 0000 0000 0056 6964 656f4861 6e64 6c65 7200 0001 5120 6d69 6e66 0000 0014 .........


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

The Determinant

Block TitleThe determinant of a matrix A is denoted det(A) or det A.It can be viewed as the scaling factor of the transformationdescribed by the matrix.4

The formula:

det(A) =

a b

c d

= ad - bc

I The determinant can tell us whether or not a giventransformation associated with that matrix squisheseverything into a smaller dimension.

I Also, if the value of determinant is negative then thetransformation is equivalent to inverting the orientationof space.

4Source: Wikipedia


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

The Determinant

Example

The image5 below shows the significance of calculating theDeterminant of a matrix.

5The Determinant — Essence of Linear Algebra, Chapter 5


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

The Determinant

Example

The image6 below shows how to calculate the Determinantof a 2D matrix.

6The Determinant — Essence of Linear Algebra, Chapter 5


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

The Storyline








Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Column space and Null space

Column space: C(A)

The column space of a matrix A is the vector spacegenerated by all the linear combinations of the columnvectors.

Null space: N(A)

The set of all vectors #»v such that

A #»v =#»0

Example

Ax =

1 1 2

2 1 3

3 1 4

4 1 5

x

y

z

=

0

0

0

0

⇒ N(A) = c

1

1

−1


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Inverse of a matrix A

Inverse of A = A−1

In terms of transformation, the matrix that undoes all thetransformations made by matrix A is called as inverse ofmatrix A (A−1).

A−1A = AA−1 = I

Ax = b ⇒ A−1Ax = A−1b ⇒ x = A−1b

Example

A =

7 2 1

0 3 1

−3 4 2

b =

21

5

−1

x =

x

y

z

x = A−1b ⇒

x

y

z

=

−2 8 −5

3 −11 7

9 −34 21

21

5

−1

=

3

1

−2


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Inverse of A

Example

The image7 below shows how two vectors are linked to eachother via Inverse transformation.



Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

The Storyline








Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Find Mr. Rank

What is the rank of the following matrix?

Be Quick!8

12 15 14 19 13 21 05 07 41 51

22 26 26 32 27 36 21 24 59 70

10 11 12 13 14 15 16 17 18 19

24 30 28 38 26 42 10 14 82 102

30 33 36 39 42 45 48 51 54 57

15 16.5 18 19.5 21 22.5 24 25.5 27 28.5

8You’ll be given choc!


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

The Rank

Solution concept provided by The Rank

The Rank tells you everything aboutthe number of solutions to a given system of linear equations.9

Matrix A with dimensions m x n, rank r and rref(A) = R

r = m = n r = n < m r = m < n r < m, r < n

R = I R =

I0

R =[I F

]R =

I F

0 0

1 0 or 1 1 or ∞ 0 or ∞

9Gilbert Strang, Linear Algebra lecture 8


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

The Storyline








Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Some things of Eigen

Eigen values and vectors

An Eigen vector of a linear transformation is a non-zerovector whose direction does not change when that lineartransformation is applied to it. More formally, if T is a lineartransformation from a vector space V and #»v is a vector in Vthat is not the zero vector, then #»v is an eigenvector of T ifT ( #»v ) is a scalar multiple of #»v . This condition can bewritten as the equation

T ( #»v ) = λ #»v

where λ is a scalar known as the eigenvalue, characteristicvalue or root associated with the eigenvector #»v .


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary


Example

The image10 below shows how the eigenvector does notchange it’s direction after applying a linear transformation.

10Eigenvectors and eigenvalues, Essence of linear algebra, chapter 10


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary


Example

The image11 below shows how the eigenvector gets scaledafter applying a linear transformation.

11Eigenvectors and eigenvalues, Essence of linear algebra, chapter 10


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Change of Basis

New coordinates to Old coordinates2 −1

1 1

−1

2

= (−1)

2

1

+ (2)

−1

1

=

−4

1

Old coordinates to New coordinates

2 −1

1 1

−1

=

1/3 1/3

−1/3 2/3

⇒ 1/3 1/3

−1/3 2/3

3

2

=

5/3

1/3


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Change of Basis

TransformationLet #»v be a vector in the New coordinates i.e. change ofBasis vectors, A be the matrix representing thetransformation: Change of Basis, and M be thetransformation in Old coordinate system and T be the finaltransformation in the New coordinate system.

A−1MA #»v = T #»v

Example

2 −1

1 1

−1 0 −1

1 0

2 −1

1 1

#»v =

1/3 −2/3

5/3 −1/3

#»v


Shreedhar Kodate








Space Tour


Celebrity: TheRank

Solution concept

Some things ofEigen


Summary

Summary

How do you possibly hope to summarize the whole talk?

Like the OLD MAN said: TOGETHER!


Shreedhar Kodate

Appendix

For Further Learning

For Further Learning I

Gilbert Stranghttps://tinyurl.com/gt7dy36MIT OCW

Grant Sandersonhttps://goo.gl/R1kBdbEssence of linear algebra, YouTube channel3Blue1Brown

Think DifferentYou can always find more to learn... If you want to ;-)

essence of linear algebra - some cool intuitions · essence of linear algebra shreedhar kodate the...

Documents