stk 500 pengantar teori statistika vectorpengantar teori statistika vector x 2 2 2 x 1 p2 22 1 2...
TRANSCRIPT
STK 500
Pengantar Teori Statistika
Vector
x 2 2 2x 1 p2
2 21 2
length of L x x x
x x
x'x
Vectors have geometric properties of length and direction – for a vector
we have
x 1
2
xx
2
1 x1
x2
x 1 2x x
Why?
Geometry of Vectors
2 2ah b
2 21 2
length of hypotenuse L L L
x x
x'x
Geometry of Vectors
Recall the Pythagorean Theorem: in any right triangle, the lengths of the hypotenuse and the other two sides are related by the simple formula.
2
1
side a
side b
Geometry of Vectors
Vector addition – for the vectors
we have
,x y1 1
2 2
x yx y
2
1
x 1 2x x
x y 1 1
2 2
x y+ x y
y 1 2y y
q
1 1
2 2
x yx y
xx 2 2 2 2c 1 p2
2 2 21 2
length of c L c x x x
c x x
c x'x
Geometry of Vectors
Scalar multiplication changes only the vector length – for the vector
we have
x 1
2
xx
2
1
x 1 2c c x x
Geometry of Vectors
Vector multiplication have angles between them – for the vectors
we have
,x y1 1
2 2
x yx y
2
1
x 1 2x x
q q x y x y
xy xy xyarccos cos
L L L L x'x y'y y 1 2y y
q
A Little Trigonometry Review
The Unit Circle 2
1 q Cos q = 1
0 Cos q 1
Cos q = 0
-1 Cos q 0
0 Cos q 1 -1 Cos q 0
Cos q = 0
Cos q = -1
Suppose we rotate x any y so x lies on axis 1:
2
1
x 1 2x x
y 1 2y y
qxy
A Little Trigonometry Review
What does this imply about rxy? 2
1
qxy Cos q = 1
0 Cos q 1
Cos q = 0
-1 Cos q 0
0 Cos q 1 -1 Cos q 0
Cos q = 0
Cos q = -1
A Little Trigonometry Review
What is the correlation between the vectors x and y?
2
1
,1.0 -0.50.6 -0.3
x y
y
x
Plotting in the column space gives us
Geometry of Vectors
Rotating so x lies on axis 1 makes it easier to see:
2
1
Qxy=1800
Geometry of Vectors
What is the correlation between the vectors x and y?
,1.0 -0.50.6 -0.3
x y
q
cosL L
1.0 -0.5 + 0.6 -0.3
1.0 1.0 0.6 0.6 -0.5 -0.5 -0.3 -0.3
-0.68= -1.00
1.36 0.34
x y
xy xy
x'x y'y
Geometry of Vectors
Of course, we can see this by plotting the these values in the x,y (row) space:
,1.0 -0.50.6 -0.3
x y
Y
X
1.0, -0.5
0.6, -0.3
Geometry of Vectors
What is the correlation between the vectors x and y?
,x y1.0 -0.500.4 1.25
q
x y
xy xy
x'x y'ycos
L L
1.0 -0.50 + 0.4 1.25
1.0 1.0 0.4 0.4 -0.50 -0.50 1.25 1.25
0.00= 0.00
1.16 1.8125
Geometry of Vectors
Plotting in the column space gives us 2
1
x1.00.4
y-0.50 1.25
Qxy=900
Geometry of Vectors
Rotating so x lies on axis 1 makes it easier to see:
2
1
Qxy=900
Geometry of Vectors
The space of all real m-tuples, with scalar multiplication and vector addition as we have defined, is called vector space.
The vector
is a linear combination of the vectors x1, x2,…, xk.
The set of all linear combinations of the vectors x1, x2,…, xk is called their linear span.
1 1 2 2 k ka a ay x x x
Geometry of Vectors
Here is the column space plot for some vectors x1 and x2:
1
2
3
1.000.400.20
1x
-0.50 0.50 1.50
2x
Geometry of Vectors
Here is the linear span for some vectors x1 and x2:
1
2
3
Geometry of Vectors
A set of vectors x1, x2,…, xk is said to be linearly dependent if there exist k numbers a1, a2,…, ak, at least one of which is nonzero, such that
Otherwise the set of vectors x1, x2,…, xk is said to be linearly independent
1 1 2 2 k ka a a 0x x x
Geometry of Vectors
Are the vectors
linearly independent?
Take a1 = 0.5 and a2 = 1.0. Then we have
The vectors x and y are dependent.
,1.0 -0.50.6 -0.3
x y
1 2a a 0.5 1.0x y
1.0 -0.5 0.00.6 -0.3 0.0
Geometry of Vectors
Geometrically x and y look like this: 2
1
1.00.6
x
-0.5-0.3
y
Geometry of Vectors
Rotating so x lies on axis 1 makes it easier to see: 2
1
Qxy=1800
Geometry of Vectors
Are the vectors
linearly independent?
There are no real values a1, a2 such that
so the vectors x and y are independent.
,x y1.0 -0.500.4 1.25
1 2 1 2a a a ax y
1.0 -0.50 0.00.4 1.25 0.0
Geometry of Vectors
Geometrically x and y look like this: 2
1
x1.00.4
y-0.50 1.25
Qxy=900
Geometry of Vectors
Rotating so x lies on axis 1 makes it easier to see:
2
1
Qxy=900
Geometry of Vectors
Here x and y are called perpendicular (or orthogonal) – this is written x y.
Some properties of orthogonal vectors:
x’y = 0 x y
z is perpendicular to every vector iff z = 0
If z is perpendicular to each vector x1, x2, …, xk, then z is perpendicular their linear span.
Geometry of Vectors
Here vectors x1 and x2 (plotted in the column space) are orthogonal
1
2
3
1.000.400.20
1x
-0.50 0.50 1.50
2x
Geometry of Vectors
Recall that the linear span for vectors x1 and x2 is:
1
2
3
Geometry of Vectors
Vector z looks like this:
1
2
3
0.10-0.32 0.14
z
0.10-0.32 0.14
z
Geometry of Vectors
The vector z is perpendicular to the linear span for vectors x1 and x2:
1
2
3
Check each of the dot products!
0.10-0.32 0.14
z
Geometry of Vectors
Here vectors x1, x2, and z from our previous problem are orthogonal
1
2
3
1.000.400.20
1x
-0.50 0.50 1.50
2x
0.10-0.32 0.14
z
x1 and z are perpendicular
Geometry of Vectors
Here vectors x1, x2, and z from our previous problem are orthogonal
1
2
3
1.000.400.20
1x
-0.50 0.50 1.50
2x
0.10-0.32 0.14
z
x2 and z are perpendicular
Geometry of Vectors
Note we could rotate x1, x2, and z until they lied on our three axes!
Here vectors x1, x2, and z from our previous problem are orthogonal
1
2
3
1.000.400.20
1x
-0.50 0.50 1.50
2x
0.10-0.32 0.14
z
x1 and x2 are perpendicular
Geometry of Vectors
The projection (or shadow) of a vector x on a vector y is given by:
y
x'yy
2L
If y has unit length (i.e., Ly = 1), the projection (or shadow) of a vector x on a vector y simplifies to (x’y)y
Geometry of Vectors
For the vectors
the projection (or shadow) of x on y is:
,x y0.6 0.81.0 0.3
y
x'yy
2
0.80.6 1.0
0.3 0.780.8 0.8= =
0.3 0.30.8L 0.730.8 0.30.3
0.8 0.8548 = 1.0685 =
0.3 0.3205
Geometry of Vectors
Geometrically the projection of x on y looks like this:
2
1
x0.61.0
y0.80.3
projection of x on y
Perpendicular wrt y
Geometry of Vectors
Rotating so y lies on axis 1 makes it easier to see:
2
1
Geometry of Vectors
Note that we write the length of the projection of x on y like this:
x x xy
y x y
x'y x'y= L L cos
L L Lq
For our previous example, the length of the length of the projection of x on y is:
y
x'y0.8
0.6 1.00.3
= 0.912921L 0.8
0.8 0.30.3
Geometry of Vectors
The Gram-Schmidt (Orthogonalization) Process
For linearly independent vectors x1, x2,…, xk, there exist mutually perpendicular vectors u1, u2,…, uk with the same linear span. These may be constructed by setting:
1 1
'12
12 2 '1 1
''12 k k -1
1 1k k k -1' '1 1 k -1 k -1
u x
x uu x u
u u
x ux uu x u x u
u u u u
We can normalize (convert to vectors z of unit length) the vectors u by setting
j
j '
j j
uz
u u
Finally, note that we can project a vector xk onto the linear span of vectors x1, x2,…, xk-1:
k -1
'
k jj=1
x z
The Gram-Schmidt (Orthogonalization) Process
Here are vectors x1, x2, and z from our previous problem:
1
2
3
1.000.400.20
1x
-0.50 0.50 1.50
2x
0.10-0.32 0.14
z
The Gram-Schmidt (Orthogonalization) Process
Let’s construct mutually perpendicular vectors u1, u2, u3 with the same linear span – we’ll arbitrarily select the first axis as u1:
00.000.000.1
1u
The Gram-Schmidt (Orthogonalization) Process
Now we construct a vector u2 perpendicular with vector u1 (and in the linear span of x1, x2, z):
50.150.000.0
00.000.000.1
5.050.150.050.0
00.000.000.1
00.000.000.1
00.000.000.1
00.000.000.1
50.150.050.0
50.150.050.0
2u
The Gram-Schmidt (Orthogonalization) Process
Finally, we construct a vector u3 perpendicular with vectors u1 and u2 (and in the linear span of x1, x2, z):
The Gram-Schmidt (Orthogonalization) Process
1025.03325.0000.0
00.150.000.0
25.000.000.000.1
1.014.032.010.0
00.150.000.0
00.150.000.0
00.150.000.0
00.150.000.0
14.032.010.0
00.000.000.1
00.000.000.1
00.000.000.1
00.000.000.1
14.032.010.0
14.032.010.0
'
'
'
'2
22
21
11
13 u
uu
uzu
uu
uzzu
The Gram-Schmidt (Orthogonalization) Process
Here are our orthogonal vectors u1, u2, and u3:
1
2
3
1.000.000.00
1u
0.000.501.50
2u
0.0000-0.3325 0.1025
3u
The Gram-Schmidt (Orthogonalization) Process
If we normalized our vectors u1, u2, and u3, we get:
1.00000.00000.0000
1.00001.0000 0.0000 0.0000 0.0000
0.0000
1.0000 1.00001.00.0000 0.0000
1.0 0.0000 0.0000
11 '
1 1
uz
u u
The Gram-Schmidt (Orthogonalization) Process
and:
0.00000.50001.5000
0.00000.0000 0.5000 1.5000 0.5000
1.5000
0.0000 0.00001.00.5000 0.3162
2.5 1.5000 0.9487
22 '
2 2
uz
u u
The Gram-Schmidt (Orthogonalization) Process
and:
0.0000-0.3325 0.1025
0.00000.0000 -0.3325 0.1025 -0.3325
0.1025
0.0000 0.00001.0-0.3325 -0.9556
0.1211 0.1025 0.2946
33 '
3 3
uz
u u
The Gram-Schmidt (Orthogonalization) Process
The normalized vectors z1, z2, and z3 look like this:
1
2
3
1.000.000.00
1z
0.00000.31620.9487
2z
0.0000-0.9556 0.2946
3z
The Gram-Schmidt (Orthogonalization) Process