Mathe IIILecture 8Mathe IIILecture 8

2

Constrained Maximization

Lagrange Multipliers

max f(x, y) s.t. g(x, y) = c

(x, y, ) = f(x, y) - g(x, y) - c L

At a maximum point of the original problem

the derivatives of the Lagrangian vanish (w.r.t. all variables).

3

Constrained Maximization Lagrange Multipliers

Intuition max f(x, y) s.t. g(x, y) = c

x

y

iso- f curves

f(x,y) = K

56

205

20

assume +

4

Constrained Maximization Lagrange Multipliers

Intuition max f(x, y) s.t. g(x, y) = c

x

y f x, y = K

x yf x, y + f x, y y = 0

y = y(x)

x

y

f x, yy = -

f x, y

5

Constrained Maximization Lagrange Multipliers

Intuition max f(x, y) s.t. g(x, y) = c

x

y f x, y = K

x yf x, y + f x, y y = 0

y = y(x)

x

y

f x, yy = -

f x, y

6

Constrained Maximization Lagrange Multipliers

Intuition max f(x, y) s.t. g(x, y) = c

x

y

max f(x, y) s.t. g(x, y)= c

x yg x, y + g x, y y = 0

x x

y y

g x, y f x, y- -

g x, y f x, y

x

y

g x, yy = -

g x, y

7

Constrained Maximization Lagrange Multipliers

Intuition max f(x, y) s.t. g(x, y) = c

x

y

x x

y y

g x, y f x, y- -

g x, y f x, y

y x

y x

f x, y f x, y

g x, y g x, yλ =

8

Constrained Maximization Lagrange Multipliers

Intuition max f(x, y) s.t. g(x, y) = c

y x

y x

f x, y f x, yλ =

g x, y g x, y

x xf x, y - λg x, y = 0

y yf x, y - λg x, y = 0

(x, y, ) = f(x, y) - g(x, y) - c L

g(x, y) = c

A stationary point of the Lagrangian

9

Constrained Maximization Lagrange Multipliers

Intuition max f(x, y) s.t. g(x, y) = c

y x

y x

f x, y f x, yλ =

g x, y g x, y

x xf x, y - λg x, y = 0

y yf x, y - λg x, y = 0

(x, y, ) = f(x, y) - g(x, y) - c L

g(x, y) = c

A stationary point of the Lagrangian

10

Constrained Maximization

The general case

max ........

1 1

1 2 n

m m

g (x) = c

f(x , x , ..x ) s.t.

g (x) = c

j 1 m m+1 n jg x , ..x , x , ..x = c

can be explicitly expressed

as functions of 1 m

m+1 n

x , ...x

x , ...x

1 m+1 n m 1 mx x , ..x , ....., x x , ...x

11

Constrained Maximization The general case j 1 m m+1 n jg x , ..x , x , ..x = c

m

j jh

h=1 h s s

g gx= 0

x x x

differentiating w.r.t. xs , s = m+1,…,n

j = 1, ....,m

s = m + 1, ....,n

12

Constrained Maximization The general case

j = 1, ....,m

mj jh

h=1 h s s

g gx= 0

x x x

.....

.....

1

m m m

m1

1 s m s s

m1

1 s m s s

1 1g xx g g

= 0x x x x x

.......

g g gxx= 0

x x x x x

13

Constrained Maximization The general case

s = m + 1, ....,n

1 1

s s

m m

s s

x g

x x

G ... ... 0

x g

x x

,

jm m j,h

h

gG G

xx.....

.....

1

m m m

m1

1 s m s s

m1

1 s m s s

1 1g xx g g

= 0x x x x x

.......

g g gxx= 0

x x x x x

14

Constrained Maximization The general case

s = m + 1, ....,n

1 1

s s

-1

m m

s s

x g

x x

... G ...

x g

x x

15

Constrained Maximization The general case

1 m+1 n m m+1 n m+1 nf x x , ..x , ....x x , ..x , x , ..x

maxm+1 n

m+1 nx ,..x

1 mf x , . x ,..x , ..x.

The derivatives w.r.t. xm+1,…..xn are zero at a max (min) point.

s = m + 1, ....,n

mh

h=1 h s s

xf f= 0

x x x

16

Constrained Maximization The general case

,...,

1

s

1 m sm

s

x

xf f f

= 0x x x

x

xs = m + 1, ....,n

m

h

h=1 h s s

xf f= 0

x x x

17

Constrained Maximization The general case

,...,

1

s

1 m sm

s

x

xf f f

= 0x x x

x

xs = m + 1, ....,n

But:

1 1

s s

-1

m m

s s

x g

x x

... G ...

x g

x x

,...,

1

s

-1

1 m sm

s

g

xf f f

G ... = 0x x x

g

x

18

Constrained Maximization The general case

,...,

-1

1

s

m

s

1 m s

g

xf

..f f

Gx x

. = 0x

g

x 1 mλ , ....λ s = m + 1, ....,n

19

Constrained Maximization The general case

,...,

-1

1

s

m

s

1 m s

g

xf

..f f

Gx x

. = 0x

g

x 1 mλ , ....λ

1

s

sm

s

1 mλ , ...

g

xf

..

x

.λ . = 0x

gs = m + 1, ....,n

We need to show this for s = 1,….m

,...,

-1

1 m1 m

f fλ , ...λ G

x x

20

Constrained Maximization The general case

,...,

-1

1 m1 m

f fλ , ...λ G

x x

,...,

-1

1 m1 m

f fλ , ...λ G G

x xG ,...,

1 m

f f=

x x

,...,

1 m

1 m

f fλ , ...λ G

x x

21

Constrained Maximization The general case

,...,

1 m

1 m

f fλ , ...λ G

x x

1

s

1 ms

m

s

g

xf

λ , ....λ ... = 0x

g

x s = 1, ..,ms = ,m + 1,1, ..,m ...,n

22

Constrained Maximization The general case

1

s

1 ms

m

s

g

xf

λ , ....λ ... = 0x

g

xs = 1, .....,n

We have shown that a solution of the original problem

max ........

1 1

1 2 n

m m

g (x) = c

f(x , x , ..x ) s.t.

g (x) = csatisfies

23

Constrained Maximization The general case

1

s

1 ms

m

s

g

xf

λ , ....λ ... = 0x

g

xs = 1, .....,n

i.e. a solution of the original problemis a stationary point of the Lagrangian :

m

1 n 1 m j j jj=1

(x , ..., x , λ , .., λ ) = f(x) - g (x) - c L

24

Constrained Maximization

Interpretation of the multipliers

max ........

1 1

1 2 n

m m

g (x) = c

f(x , x , ..x ) s.t.

g (x) = c

Let , be the solution* * * *1 n 1 mx , ..., x λ , ..., λ

,* *i 1 m j 1 mx c , ...,c λ c , ...,c

:define * *1 nf c = f x c , ..., x c

1 mc = c , ...,c

25

Constrained Maximization

Interpretation of the multipliers

* *nj

j=1i j i

f x xf c=

c x c

* *1 nf c = f x c , ..., x c

But:

* *m

h*h

h=1j j

f x g xλ

x x

* *n mh j*

hj=1 h=1i j i

g x xf c= λ

c x c

26

Constrained Maximization

Interpretation of the multipliers

* *n m

h j*h

j=1 h=1i j i

g x xf c= λ

c x c

* *m nh j*

hh=1 j=1i j i

g x xf c= λ

c x c

* *nh j

j=1 j

m*h

h 1 i=i

g x x

x

f c=

c cλ

when or

* **n

h hj

j=1 j i i

g x g xx=

x c c

= 0 h i 1 h = i

??

27

Constrained Maximization

Interpretation of the multipliers *

h hg x c = c

* *n

h j h

j=1 j i i

g x x c=

x c c

differentiate w.r.t. ci

when or = 0 h i 1 h = i

28

Constrained Maximization

Interpretation of the multipliers

* *nh j

j=1 j

m*h

h 1 i=i

g x x

x

f c=

c cλ

*i

i

f c= λ

c