lecture slides linalg
TRANSCRIPT
-
7/29/2019 Lecture Slides Linalg
1/23
Financial Engineering and Risk ManagementReview of vectors
Martin Haugh Garud Iyengar
Columbia UniversityIndustrial Engineering and Operations Research
-
7/29/2019 Lecture Slides Linalg
2/23
Reals numbers and vectors
We will denote the set of real numbers by R
Vectors are finite collections of real numbers
Vectors come in two varieties
Row vectors: v =v1 v2 . . . vn
Column vectors w =
w1w2
.
..wn
By default, vectors are column vectors
The set of all vectors with n components is denoted by Rn
2
-
7/29/2019 Lecture Slides Linalg
3/23
Linear independence
A vector w is linearly dependent on v1, v2 if
w =1
v1
+2
v2
for some1, 2
R
Example:
2
6
4
= 2
1
1
0
+ 4
0
1
1
Other names: linear combination, linear spanA set V= {v1, . . . , vm} are linearly independent if no vi is linearlydependent on the others, {vj : j= i}
3
-
7/29/2019 Lecture Slides Linalg
4/23
Basis
Every w Rn is a linear combination of the linearly independent set
B =
1
0
...0
,
0
1
...0
, . . .
0
0
...1
w = w1
1
0
...0
e1+w2
0
1
...0
e2+ . . . + wn
0
0
...1
enBasis any linearly independent set that spans the entire spaceAny basis for Rn has exactly n elements
4
-
7/29/2019 Lecture Slides Linalg
5/23
Norms
A function (v) of a vector v is called a norm if
(v) 0 and (v) = 0 implies v = 0
(v) = || (v) for all R
(v1 + v2) (v1) + (v2) (triangle inequality)
generalizes the notion of length
Examples:2 norm: x2 =
ni=1
|xi|2 ... usual length
1 norm: x1 =n
i=1|xi|
norm: x = max1in |x|i
p norm, 1 p < : xp =n
i=1|x|pi
1p
5
-
7/29/2019 Lecture Slides Linalg
6/23
Inner product
The inner-product or dot-product of two vector v, w Rn is defined as
v w =n
i=1
viwi
The 2 norm v2 =
v vThe angle between two vectors v and w is given by
cos() =v w
v2w
2
Will show later: v w = vw = product of v transpose and w
6
-
7/29/2019 Lecture Slides Linalg
7/23
Financial Engineering and Risk ManagementReview of matrices
Martin Haugh Garud Iyengar
Columbia UniversityIndustrial Engineering and Operations Research
-
7/29/2019 Lecture Slides Linalg
8/23
-
7/29/2019 Lecture Slides Linalg
9/23
Matrix Operations: Transpose
Transpose: A Rmd
A =
a11 a12 . . . a1da21 a22 . . . a2d
......
. . ....
am1 am2 . . . amd
=
a11 a21 . . . am1a12 a22 . . . ad2
......
. . ....
a1d a2d . . . amd
Rdm
Transpose of a row vector is a column vector
Example:
A =
2 3 7
1 6 5
: 2 3 matrix ... A =
2 1
3 6
7 5
: 3 2 matrix
v =
26
4
: column vector ... v =
2 6 4
: row vector
3
-
7/29/2019 Lecture Slides Linalg
10/23
Matrix Operations: Multiplication
Multiplication: A Rmd, B Rdp then C = AB Rmp
cij =ai1 ai2 . . . aid
b1j
b2j...bdj
row vector v R1d times column vector w Rd1 is a scalar.
Identity times any matrix AIn = ImA = A
Examples:
2 3 7
1 6 52
6
4 = 2(2) + 3(6) + 7(4)1(2) + 6(6) + 5(4) =
50
582 norm:
1
2
2
=
12 + (2)2 =
1 2
12
=
1
2
1
2
inner product: v w = vw
4
-
7/29/2019 Lecture Slides Linalg
11/23
Linear functions
A function f : Rd Rm is linear if
f(x + y) = f(x) + f(y), , R, x, y
Rd
A function f is linear if and only if f(x) = Ax for matrix A Rmd
Examples
f(x) : R3 R: f(x) = 2 3 4x1x2x3
= 2x1 + 3x2 + 4x3f(x) : R3 R2: f(x) =
2 3 4
1 0 2
x1x2x3
=
2x1 + 3x2 + 4x3
x1 + 2x3
Linear constraints define sets of vectors that satisfy linear relationships
Linear equality: {x : Ax = b} ... line, plane, etc.
Linear inequality: {x : Ax b} ... half-space
5
-
7/29/2019 Lecture Slides Linalg
12/23
Rank of a matrix
column rank of A Rmd = number of linearly independent columnsrange(A) = {y : y = Ax for some x}
column rank of A = size of basis for range(A)
column rank of A = m range(A) = Rm
row rank of A = number of linearly independent rows
Fact: row rank = column rank min{m, d}Example:
A =
1 2 3
2 4 6
, rank = 1, range(A) =
1
2
: R
A Rnn and rank(A) = n A invertible, i.e. A1 Rnn
A1A = AA1 = I
6
-
7/29/2019 Lecture Slides Linalg
13/23
Financial Engineering and Risk ManagementReview of linear optimization
Martin Haugh Garud Iyengar
Columbia UniversityIndustrial Engineering and Operations Research
-
7/29/2019 Lecture Slides Linalg
14/23
Hedging problem
d assets
Prices at time t= 0: p Rd
Market in m possible states at time t= 1
Price of asset j in state i = Sij
Sj =S1j
S2j...Smj
S = S1 S2 . . . Sd = S11 S12 . . . S1d
S21 S22 . . . S2d......
. . ....
Sm1 Sm2 . . . Smd
Rmd
Hedge an obligation X RmHave to pay Xi if state i occurs
Buy/short sell = (1, . . . , d) shares to cover obligation
2
-
7/29/2019 Lecture Slides Linalg
15/23
Hedging problem (contd)
Position Rd purchased at time t= 0j = number of shares of asset j purchased, j = 1, . . . , d
Cost of the position = dj=1
pjj = p
Payoff from liquidating position at time t= 1
payoffyi in state i: yi = dj=1 SijjStacking payoffs for all states: y = SViewing the payoff vector y: y range(S)
y = S1 S2 . . . Sd12...d
=
d
j=1
jSj
Payoff y hedges X if y X.
3
-
7/29/2019 Lecture Slides Linalg
16/23
Hedging problem (contd)
Optimization problem:
min dj=1 pjj ( p)subject to dj=1 Sijj Xi, i= 1, . . . , m ( S X)Features of this optimization problem
Linear objective function: p
Linear inequality constraints: S X
Example of a linear program
Linear objective function: either a min/max
Linear inequality and equality constraintsmax/minx c
xsubject to Aeqx = beq
Ainx bin
4
-
7/29/2019 Lecture Slides Linalg
17/23
Linear programming duality
Linear programP = minx c
x
subject to Ax bDual linear program
D = maxu bu
subject to Au = cu
0
Theorem.
Weak Duality: P DBound: x feasible for P, u feasible for D, cx
P
D
bu
Strong Duality: Suppose P or D finite. Then P= D.
Dual of the dual is the primal (original) problem
5
M d l l
-
7/29/2019 Lecture Slides Linalg
18/23
More duality results
Here is another primal-dual pair
minx c
xsubject to Ax = b = maxu b
usubject to Au = c
General idea for constructing duals
P = min{cx : Ax b} min{cx u(Ax b) : Ax b} for all u 0 bu + min{(c Au)x : x Rn}=
bu Au = c
otherwise max{bu : Au = c}
Lagrangian relaxation: dualize constraints and relax them!
6
-
7/29/2019 Lecture Slides Linalg
19/23
Financial Engineering and Risk ManagementReview of nonlinear optimization
Martin Haugh Garud Iyengar
Columbia UniversityIndustrial Engineering and Operations Research
U t i d li ti i ti
-
7/29/2019 Lecture Slides Linalg
20/23
Unconstrained nonlinear optimization
Optimization problemminxRn f(x)
Categorization of minimum pointsx global minimum if f(y) f(x) for all y
xloc local minimum if f(y) f(xloc) for all y such that y x
loc r
Sufficient condition for local min
gradient f(x) =
fx1...f
xn
= 0: local stationarity
Hessian 2f(x) = 2f
x21
2f
x1x2. . .
2f
x1xn
... ... . . . ...2f
xnx1
2f
xnx2. . .
2f
x2n
positive semidefiniteGradient condition is sufficient if the function f(x) is convex.
2
U t i d li ti i ti
-
7/29/2019 Lecture Slides Linalg
21/23
Unconstrained nonlinear optimization
Optimization problem
minxR2 x2
1+ 3x1x2 + x
3
2
Gradient
f(x) =
2x1 + 3x23x1 + 3x
22
= 0 x = 0,
9
4
3
2
Hessian at x: H =
2 3
3 6x2
x = 0: H =
2 3
3 0
. Not positive definite. Not local minimum.
x = 9
43
2
: H =
2 3
3 9
. Positive semidefinite. Local minimum
3
L i th d
-
7/29/2019 Lecture Slides Linalg
22/23
Lagrangian method
Constrained optimization problemmaxxR2 2 ln(1 + x1) + 4 ln(1 + x2),
s.t. x1 + x2 =12
Convex problem. But constraints make the problem hard to solve.
Form a Lagrangian function
L(x, v
) =2 ln
(1
+x1
) +4 ln
(1
+x2
) v
(x1
+x2
12
)
Compute the stationary points of the Lagrangian as a function of v
L(x, v) = 2
1+x1 v
4
1+x2
v = 0 x1 =2
v 1, x2 = 4
v 1
Substituting in the constraint x1 + x2 = 12, we get
6
v= 14 v= 3
7 x = 1
3
11
25
4
Portfolio Selection
-
7/29/2019 Lecture Slides Linalg
23/23
Portfolio Selection
Optimization problem
maxx x
xVx
s.t. 1x = 1
Constraints make the problem hard!
Lagrangian function
L(x, v) = x
xVx
v(1x
1)
Solve for the maximum value with no constraints
xL(x, v) = 2Vx v1 = 0 x =1
2 V1( v1)
Solve for v from the constraint
1x = 1 1V1( v1) = 2 v= 1
V1
21V11
Substitute back in the expression for x
5