Cs221 linear algebra

Download Cs221 linear algebra

Post on 06-May-2015

1.040 views

Category:

Technology

1 download

Embed Size (px)

TRANSCRIPT

<ul><li>1.Linear Algebra ReviewCS221: Introduction to Articial IntelligenceCarlos Fernandez-Granda10/11/2011CS221Linear Algebra Review10/11/2011 1 / 37</li></ul><p>2. Index1 Vectors2 Matrices3 Matrix Decompositions4 Application : Image Compression CS221Linear Algebra Review 10/11/2011 2 / 37 3. Vectors1 Vectors2 Matrices3 Matrix Decompositions4 Application : Image Compression CS221Linear Algebra Review 10/11/2011 3 / 37 4. VectorsWhat is a vector ? CS221 Linear Algebra Review 10/11/2011 4 / 37 5. VectorsWhat is a vector ?An ordered tuple of numbers : u 1 u2 u= . . . un CS221Linear Algebra Review10/11/2011 4 / 37 6. VectorsWhat is a vector ?An ordered tuple of numbers :u 1u2 u=. . . unA quantity that has a magnitude and a direction.CS221Linear Algebra Review 10/11/2011 4 / 37 7. VectorsVector spacesMore formally a vector is an element of a vector space. CS221 Linear Algebra Review10/11/2011 5 / 37 8. VectorsVector spacesMore formally a vector is an element of a vector space.A vector space V is a set that contains all linear combinations of itselements :CS221Linear Algebra Review10/11/2011 5 / 37 9. VectorsVector spacesMore formally a vector is an element of a vector space.A vector space V is a set that contains all linear combinations of itselements :If vectors u and v V , then u + v V .CS221Linear Algebra Review10/11/2011 5 / 37 10. VectorsVector spacesMore formally a vector is an element of a vector space.A vector space V is a set that contains all linear combinations of itselements :If vectors u and v V , then u + v V .If vector u V , then u V for any scalar .CS221Linear Algebra Review10/11/2011 5 / 37 11. VectorsVector spacesMore formally a vector is an element of a vector space.A vector space V is a set that contains all linear combinations of itselements :If vectors u and v V , then u + v V .If vector u V , then u V for any scalar .There exists 0 V such that u + 0 = u for any u V .CS221 Linear Algebra Review10/11/2011 5 / 37 12. VectorsVector spacesMore formally a vector is an element of a vector space.A vector space V is a set that contains all linear combinations of itselements :If vectors u and v V , then u + v V .If vector u V , then u V for any scalar .There exists 0 V such that u + 0 = u for any u V .A subspace is a subset of a vector space that is also a vector space.CS221 Linear Algebra Review10/11/2011 5 / 37 13. VectorsExamples Euclidean space Rn . CS221Linear Algebra Review 10/11/2011 6 / 37 14. VectorsExamples Euclidean space Rn . The span of any set of vectors {u1 , u2 , . . . , un }, dened as : span (u1 , u2 , . . . , un ) = {1 u1 + 2 u2 + + n un | i R} CS221 Linear Algebra Review10/11/2011 6 / 37 15. VectorsSubspaces A line through the origin in Rn (span of a vector). CS221 Linear Algebra Review 10/11/2011 7 / 37 16. VectorsSubspaces A line through the origin in Rn (span of a vector). A plane in Rn (span of two vectors). CS221 Linear Algebra Review 10/11/2011 7 / 37 17. VectorsLinear Independence and Basis of Vector SpacesA vector u is linearly independent of a set of vectors {u1 , u2 , . . . , un }if it does not lie in their span.CS221Linear Algebra Review 10/11/2011 8 / 37 18. VectorsLinear Independence and Basis of Vector SpacesA vector u is linearly independent of a set of vectors {u1 , u2 , . . . , un }if it does not lie in their span.A set of vectors is linearly independent if every vector is linearlyindependent of the rest.CS221Linear Algebra Review 10/11/2011 8 / 37 19. VectorsLinear Independence and Basis of Vector SpacesA vector u is linearly independent of a set of vectors {u1 , u2 , . . . , un }if it does not lie in their span.A set of vectors is linearly independent if every vector is linearlyindependent of the rest.A basis of a vector space V is a linearly independent set of vectorswhose span is equal to V .CS221Linear Algebra Review 10/11/2011 8 / 37 20. VectorsLinear Independence and Basis of Vector SpacesA vector u is linearly independent of a set of vectors {u1 , u2 , . . . , un }if it does not lie in their span.A set of vectors is linearly independent if every vector is linearlyindependent of the rest.A basis of a vector space V is a linearly independent set of vectorswhose span is equal to V .If a vector space has a basis with d vectors, its dimension is d.CS221Linear Algebra Review 10/11/2011 8 / 37 21. VectorsLinear Independence and Basis of Vector SpacesA vector u is linearly independent of a set of vectors {u1 , u2 , . . . , un }if it does not lie in their span.A set of vectors is linearly independent if every vector is linearlyindependent of the rest.A basis of a vector space V is a linearly independent set of vectorswhose span is equal to V .If a vector space has a basis with d vectors, its dimension is d.Any other basis of the same space will consist of exactly d vectors.CS221Linear Algebra Review 10/11/2011 8 / 37 22. VectorsLinear Independence and Basis of Vector SpacesA vector u is linearly independent of a set of vectors {u1 , u2 , . . . , un }if it does not lie in their span.A set of vectors is linearly independent if every vector is linearlyindependent of the rest.A basis of a vector space V is a linearly independent set of vectorswhose span is equal to V .If a vector space has a basis with d vectors, its dimension is d.Any other basis of the same space will consist of exactly d vectors.The dimension of a vector space can be innite (function spaces).CS221Linear Algebra Review 10/11/2011 8 / 37 23. VectorsNorm of a Vector A norm ||u|| measures the magnitude of a vector.CS221Linear Algebra Review10/11/2011 9 / 37 24. VectorsNorm of a Vector A norm ||u|| measures the magnitude of a vector. Mathematically, any function from the vector space to R that satises :CS221 Linear Algebra Review10/11/2011 9 / 37 25. VectorsNorm of a Vector A norm ||u|| measures the magnitude of a vector. Mathematically, any function from the vector space to R that satises : Homogeneity || u|| = || ||u||CS221 Linear Algebra Review10/11/2011 9 / 37 26. VectorsNorm of a Vector A norm ||u|| measures the magnitude of a vector. Mathematically, any function from the vector space to R that satises : Homogeneity || u|| = || ||u|| Triangle inequality ||u + v|| ||u|| + ||v||CS221Linear Algebra Review 10/11/2011 9 / 37 27. VectorsNorm of a Vector A norm ||u|| measures the magnitude of a vector. Mathematically, any function from the vector space to R that satises : Homogeneity || u|| = || ||u|| Triangle inequality ||u + v|| ||u|| + ||v|| Point separation ||u|| = 0 if and only if u = 0.CS221 Linear Algebra Review 10/11/2011 9 / 37 28. VectorsNorm of a Vector A norm ||u|| measures the magnitude of a vector. Mathematically, any function from the vector space to R that satises : Homogeneity || u|| = || ||u|| Triangle inequality ||u + v|| ||u|| + ||v|| Point separation ||u|| = 0 if and only if u = 0. Examples : Manhattan or 1 norm : ||u||1 =i |ui |. Euclidean or 2 norm : ||u||2 =i ui .2CS221 Linear Algebra Review 10/11/2011 9 / 37 29. VectorsInner Product Inner product between u and v : u, v = n uv.i =1 i i CS221Linear Algebra Review10/11/2011 10 / 37 30. VectorsInner Product Inner product between u and v : u, v = n=1 ui vi .i It is the projection of one vector onto the other. CS221 Linear Algebra Review10/11/2011 10 / 37 31. VectorsInner Product Inner product between u and v : u, v = n=1 ui vi .i It is the projection of one vector onto the other. Related to the Euclidean norm : u, u = ||u||2 . 2 CS221Linear Algebra Review 10/11/2011 10 / 37 32. VectorsInner ProductThe inner product is a measure of correlation between two vectors, scaledby the norms of the vectors :CS221Linear Algebra Review 10/11/2011 11 / 37 33. VectorsInner ProductThe inner product is a measure of correlation between two vectors, scaledby the norms of the vectors :If u, v0, u and v are aligned.CS221Linear Algebra Review 10/11/2011 11 / 37 34. VectorsInner ProductThe inner product is a measure of correlation between two vectors, scaledby the norms of the vectors :If u, v0, u and v are aligned.If u, v0, u and v are opposed.CS221Linear Algebra Review 10/11/2011 11 / 37 35. VectorsInner ProductThe inner product is a measure of correlation between two vectors, scaledby the norms of the vectors :If u, v0, u and v are aligned.If u, v0, u and v are opposed.If u, v = 0, u and v are orthogonal.CS221Linear Algebra Review 10/11/2011 11 / 37 36. VectorsOrthonormal Basis The vectors in an orthonormal basis have unit Euclidean norm and are orthogonal to each other.CS221Linear Algebra Review 10/11/2011 12 / 37 37. VectorsOrthonormal Basis The vectors in an orthonormal basis have unit Euclidean norm and are orthogonal to each other. To express a vector x in an orthonormal basis, we can just take inner products. CS221Linear Algebra Review 10/11/2011 12 / 37 38. VectorsOrthonormal Basis The vectors in an orthonormal basis have unit Euclidean norm and are orthogonal to each other. To express a vector x in an orthonormal basis, we can just take inner products. Example : x = 1 b1 + 2 b2 . CS221Linear Algebra Review 10/11/2011 12 / 37 39. VectorsOrthonormal Basis The vectors in an orthonormal basis have unit Euclidean norm and are orthogonal to each other. To express a vector x in an orthonormal basis, we can just take inner products. Example : x = 1 b1 + 2 b2 . We compute x, b1 : x, b1 = 1 b1 + 2 b2 , b1 = 1 b1 , b1 + 2 b2 , b1By linearity. = 1 + 0 By orthonormality. CS221Linear Algebra Review 10/11/2011 12 / 37 40. VectorsOrthonormal Basis The vectors in an orthonormal basis have unit Euclidean norm and are orthogonal to each other. To express a vector x in an orthonormal basis, we can just take inner products. Example : x = 1 b1 + 2 b2 . We compute x, b1 : x, b1 = 1 b1 + 2 b2 , b1 = 1 b1 , b1 + 2 b2 , b1By linearity. = 1 + 0 By orthonormality. Likewise, 2 = x, b2 . CS221Linear Algebra Review 10/11/2011 12 / 37 41. Matrices1 Vectors2 Matrices3 Matrix Decompositions4 Application : Image Compression CS221Linear Algebra Review 10/11/2011 13 / 37 42. MatricesLinear OperatorA linear operator L : U V is a map from a vector space U toanother vector space V that satises : CS221Linear Algebra Review10/11/2011 14 / 37 43. MatricesLinear OperatorA linear operator L : U V is a map from a vector space U toanother vector space V that satises :L (u1 + u2 ) = L (u1 ) + L (u2 ) . CS221Linear Algebra Review10/11/2011 14 / 37 44. MatricesLinear OperatorA linear operator L : U V is a map from a vector space U toanother vector space V that satises : .L (u1 + u2 ) = L (u1 ) + L (u2 )L ( u) = L (u)for any scalar . CS221Linear Algebra Review10/11/2011 14 / 37 45. MatricesLinear OperatorA linear operator L : U V is a map from a vector space U toanother vector space V that satises : .L (u1 + u2 ) = L (u1 ) + L (u2 )L ( u) = L (u)for any scalar .If the dimension n of U andm of V are nite, L can be represented byan m n matrix A.AAAn 1112 ...1A 21A 22 ... A n 2A= ... Am 1 Am 2 ... AmnCS221 Linear Algebra Review 10/11/2011 14 / 37 46. MatricesMatrix Vector Multiplication AAAnuv 11 12...111 A21 A 22...A nu 22 v2 Au = = ... . . . . . . Am1 Am 2... Amnun vm CS221 Linear Algebra Review 10/11/2011 15 / 37 47. MatricesMatrix Vector Multiplication AAAnuv 11 12...111 A21 A 22...A nu 22 v2 Au = = ... . . . . . . Am1 Am 2... Amnun vm CS221 Linear Algebra Review 10/11/2011 15 / 37 48. MatricesMatrix Vector Multiplication AAAnuv 11 12...111 A21 A 22...A nu 22 v2 Au = = ... . . . . . . Am1 Am 2... Amnun vm CS221 Linear Algebra Review 10/11/2011 15 / 37 49. MatricesMatrix Vector Multiplication AAAnuv 11 12...111 A21 A 22...A nu 22 v2 Au = = ... . . . . . . Am1 Am 2... Amnun vm CS221 Linear Algebra Review 10/11/2011 15 / 37 50. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A AAnBBBp 11 12...111 12... 1 A 21 A 22... BA n221B22... B p2AB = ...... Am Am12...Amn Bn 1 Bn 2 ... BnpAB AB . . . AB p 11121 ABAB2122. . . AB p 2 = ... ABm 1 ABm 2... ABmpNote that if A ism n and B is n p, then AB is m p. CS221 Linear Algebra Review10/11/2011 16 / 37 51. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A AAnBBBp 11 12...111 12... 1 A 21 A 22... BA n221B22... B p2AB = ...... Am Am12...Amn Bn 1 Bn 2 ... BnpAB AB . . . AB p 11121 ABAB2122. . . AB p 2 = ... ABm 1 ABm 2... ABmpNote that if A ism n and B is n p, then AB is m p. CS221 Linear Algebra Review10/11/2011 16 / 37 52. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A AAnBBBp 11 12...111 12... 1 A 21 A 22... BA n221B22... B p2AB = ...... Am Am12...Amn Bn 1 Bn 2 ... BnpAB AB . . . AB p 11121 ABAB2122. . . AB p 2 = ... ABm 1 ABm 2... ABmpNote that if A ism n and B is n p, then AB is m p. CS221 Linear Algebra Review10/11/2011 16 / 37 53. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A AAnBBBp 11 12...111 12... 1 A 21 A 22... BA n221B22... B p2AB = ...... Am Am12...Amn Bn 1 Bn 2 ... BnpAB AB . . . AB p 11121 ABAB2122. . . AB p 2 = ... ABm 1 ABm 2... ABmpNote that if A ism n and B is n p, then AB is m p. CS221 Linear Algebra Review10/11/2011 16 / 37 54. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A AAnBBBp 11 12...111 12... 1 A 21 A 22... BA n221B22... B p2AB = ...... Am Am12...Amn Bn 1 Bn 2 ... BnpAB AB . . . AB p 11121 ABAB2122. . . AB p 2 = ... ABm 1 ABm 2... ABmpNote that if A ism n and B is n p, then AB is m p. CS221 Linear Algebra Review10/11/2011 16 / 37 55. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A AAnBBBp 11 12...111 12... 1 A 21 A 22... BA n221B22... B p2AB = ...... Am Am12...Amn Bn 1 Bn 2 ... BnpAB AB . . . AB p 11121 ABAB2122. . . AB p 2 = ... ABm 1 ABm 2... ABmpNote that if A ism n and B is n p, then AB is m p. CS221 Linear Algebra Review10/11/2011 16 / 37 56. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A AAnBBBp 11 12...111 12... 1 A 21 A 22... BA n221B22... B p2AB = ...... Am Am12...Amn Bn 1 Bn 2 ... BnpAB AB . . . AB p 11121 ABAB2122. . . AB p 2 = ... ABm 1 ABm 2... ABmpNote that if A ism n and B is n p, then AB is m p. CS221 Linear Algebra Review10/11/2011 16 / 37 57. MatricesMatrix ProductApplying two linear operators A and B denes another linear operator,which can be represented by another matrix AB. A AAnBBBp 11 12...111 12... 1 A 21 A 22... BA n221B22... B p2AB = ...... Am Am12...Amn Bn 1 Bn 2 ... BnpAB AB . . . AB p 11121 ABAB2122. . . AB p 2 = ... ABm 1 ABm 2... ABmpNote that if A ism n and B is n p, then AB is...</p>