nonlinear programming solution a

8/2/2019 Nonlinear Programming Solution A

1/65

Dr. A.K. Bardhan Facult of Mana ement Studies Universit of Delhi 1

Nonlinear Programming

Solutions


2/65


Difficulties of NLP Models

Nonlinear Programs:

LinearProgram:


3/65


Graphical Analysis of Non-linear programs

in two dimensions: An example

2 214 15( ) ( )x y + Minimize

subject to (x - 8)

2

+ (y - 9)

2

49x 2x 13x + y 24


4/65


Where is the optimal solution?

0 2 4 6 8 10 12 14 16 18

0

2

4

6

8

10

12

14

16

18y

x

Note: the optimalsolution is not at acorner point.It is where theisocontour first hits

the feasible region.


5/65


6/65


Local vs. Global Optima

There may be several locally optimal solutions.

x

z

10

z = f(x) max f(x)s.t. 0 x 1A

B

C

Defn: Let x be a feasible solution, then

x is a global maxif f(x) f(y) for every feasible y. x is a local maxif f(x) f(y) for every feasible y sufficiently close to x

(i.e., xj-yjxj+ for allj and some small ).


7/65


When is a locally optimal solution also globallyoptimal?

For minimization problems The objective function is convex.

The feasible region is convex.


8/65


W

P

2 4 6 8 10 12 14

2

4

6

8

10

12

14

Convexity and Extreme Points

We say that a set S is convex, if for every

two points x and y in S, and for every realnumber l in [0,1], lx + (1-l)y S.

The feasible region of a

linear program is convex.

x

y

We say that an element w S is anextreme point(vertex,corner point), if wis not the midpoint of any line segment

contained in S.

8


9/65Dr. A.K. Bardhan Facult of Mana ement Studies Universit of Delhi 9

On convex feasible regions

If all constraints are linear, then the feasible region isconvex



On Convex Feasible Regions

The intersection of convex regions is convex



S

Recognizing convex sets

Rule of thumb: suppose for all x, y S the midpoint of xand y is in S. Then S is convex.

xy

It is convex if

the entire line

segment is

always in S.

(x+y)/2



Which are convex?

CB

B C B CB C

DA

12



Joining two points on a curve

The line segment joining two points on a curve. Let f( ) be a

function, and let g(ly+(1-l)z)) = l f(y) + (1-l)f(z) for 0l.

f(y)

f(z)

(y+z)/2

f(y)/2 +f(z)/2

g(x)

g(y) = f(y)

g(z) = f(z)

g(y/2 + z/2) = f(y)/2 + f(z)/2

y z

f(x)

x



Convex Functions

Convex Functions: f(l y + (1-l)z) l f(y) + (1-l)f(z)for every y and z and for 0l.e.g., l = 1/2 f(y/2 + z/2) f(y)/2 + f(z)/2

Line joining any pointsis above the curvef(x)

xy z

f(y)

f(z)

(y+z)/2

f(y)/2 +f(z)/2

We say strict

convexityif signis



Concave Functions

Concave Functions: f(l y + (1-l)z) l f(y) + (1-l)f(z)for every y and z and for 0 l.e.g., l = 1/2 f(y/2 + z/2) f(y)/2 + f(z)/2

Line joining any pointsis below the curve

xy z

f(y)

f(z)

(y+z)/2

f(y)/2 +f(z)/2

We say strict

concavityif signis



Classify as convex or concave or both or neither.



More on convex functions

x

f(x)

-x

f(-x)

If f(x) is convex,then f(-x) isconvex.




x

y

If f(x) is convex,then K - f(x) isconcave.

x

y




If f(x) is a twice differentiable function of one

variable, and if f(x) > 0 for all x, then f(x) isconvex.

f(x) = x2.

f(x) = 2x, f(x) = 2

0

1

2

3

4

5

6

7

8

9

10

-4 -3 -2 -1 0 1 2 3 4

f(x) = - ln(x) for x > 0

f(x) = -1/x, f(x) = 1/x2

-2

-1.5

-1

-0.5

0

0.5

1

1.52

2.5

0 0.5 1 1.5 2 2.5 3 3.5 4



0

1

2

3

4

5

6

7

8

-3 -2 -1 0 1 2 3

f(x) g(x)

Even more on convex functions

If f(x) is convex and g(x) is convex,

then so is f(x) + g(x)

f(x)+g(x)



0

1

2

3

4

5

6

7

8

-3 -2 -1 0 1 2 3

f(x) g(x)

Even more on convex functions

If f(x) is convex and g(x) is convex,

then so is max [f(x), g(x)]

max[f(x), g(x)]


22/65Dr. A.K. Bardhan Facult of Mana ement Studies Universit of Delhi22

What functions are convex?

f(x) = 4x + 7 all linear functions

f(x) = 4x2 13 some quadratic functions f(x) = ex

f(x) = 1/x for x > 0

f(x) = |x|

f(x) = - ln(x) for x > 0



Convex functions vs. convex sets

If y = f(x) is convex, then{(x,y) : f(x) y} is a convex set

y

x

f(x)



Local Minimum Property

A local min of a convex function on a convex feasible region is also aglobal min.

Strict convexity implies that the global minimum is unique.

The following NLPs can be solved

Minimization Problems with a convex objective function and linearconstraints



Local minimum property

There is a unique local minimum for the function below.

y

x

f(x)

The local

minimum is

a global

minimum



Local Maximum Property

A local max of a concave function on a convex feasible region is also aglobal max.

Strict concavity implies that the global optimum is unique.

Given this, the following NLPs can be solved

Maximization Problems with a concave objective function and linearconstraints



Local maximum property

y

x

There is a unique local maximum for the function below.

The local

maximum isa global

minimum



Finding a local optimal for a single variable NLP

Solving NLP's with One Variable:max f(q)

s.t. a q b

Optimal solution is either

a boundary point or

satisfies f

(q*

) = 0 and f

(q*

) < 0.

q

f(q)

a

q *

b

q

f(q)

a q * b

q

f(q)

a q * b


29/65


Unimodal Functions

A single variable function f is unimodalif there is at most

one local maximum (or at most one local minimum) .

H fi d if f i f l i bl


30/65


How to find if a function of several variablesis Convex or Concave?

Use of Hessian Matrix


31/65


The Hessian MatrixH(X) is the nn matrix whose ith row are

j

f

x

2 2 2

2

1 1 2 1

2 2 2

2

1 2 2 2

2 2 2

2

1 2

. .

. .

..

.

. . .

n

n

n n n

f f f x x x x x

f f f

x x x x x

f f f

x x x x x

i.e. H(X) =

with respect toxi.

the partial derivates of (j =1,2,..n)

(i =1,2,..n)


32/65


++

22

26),(Then

2)(If

1

21

2221

31

xxxH

xxxxxf


33/65


Principal Minor

The ith principal minor of an nx nmatrix is the determinant of the ix imatrix

obtained by deleting (ni) rows and corresponding (ni) columns of thematrix.

Example

4122.22.6

isminorprincipalleadingsecondThe2.and6areminorsprincipalfirstThe

22

26),(Then

2)(If

11

1

1

21

2

221

3

1

++

xx

x

xxxH

xxxxxf


34/65


Leading Principal Minor

The kth leading principal minor of an nx nmatrix is the determinant of the k

x kmatrix obtained by deleting the last (nk) rows and columns of thematrix.

Example

4122.22.6

isminorprincipalleadingsecondThe6isminorprincipalleadingfirstThe

22

26),(Then

2)(If

11

1

1

21

2

221

3

1

++

xx

x

xxxH

xxxxxf


35/65


e.nonnegativareofminorsprincipalall,

eachforifonlyandifSonfunctionconvexais),...,,(

),...,,(ThenS.),...,,(pointeachforderivative

partialordersecondcontinuoushas),...,,(Suppose

21

2121

21

HSx

xxxf

xxxfxxxx

xxxf

n

nn

n

convex.is),(theorem,abovethefromTherefore

0.02(2)-2(2)minorprincipalsecondThe0).2equal(both

entriesdiagonaltheareHessiantheofminorsprincipalfirstThye

22

22

2),(

:Example

21

2

121

2

121

xxf

H

xxxxxxf

++

Convex Function

C F ti


36/65


.)1(assignsame

thehaveofminorsprincipalzero-nonall;,...,2,1and

eachforifonlyandifSonfunctionconcaveais

),...,,(ThenS.),...,,(pointeachforderivative

partialordersecondcontinuoushas),...,,(Suppose

2121

21

k

nn

n

HnkSx

xxxfxxxx

xxxf

concave.is),(theorem,abovethefromTherefore

0.7(-1)(-1)-2(-4)-

minorprincipalsecondThee.nonpositivboth0)(-2,-4

entriesdiagonaltheareHessiantheofminorsprincipalfirstThye

41

12

22),(

:Example

21

2

121

2

121

xxf

H

xxxxxxf

Concave Function


37/65


Example

A monopolist producing a single product has two types of

customers. If q1 units are produced for customer-1, thencustomer-1 is willing to pay 70 4q1 rupees. If q2 units areproduced for customer-2, then customer-2 is willing to pay 150 15q2 rupees. For q > 0, the cost of manufacturing q units is

100 - 15q rupees. To maximize profit, how much should themonopolist sell to each customer?


38/65


Solution

f(q1, q2)= q1 (70 4q1) + q2 (100 - 15q2) 100 15q1 - 15q2

-8 0

H(q1, q2 )=

0 -30

Objective function is concave and hence (55/8, 9/2) maximize

it.


39/65


Exercise-2 A DVD costs Rs. 55 to produce. We are considering

charging a price of between Rs. 110 and Rs. 150 for thisDVD. For a price of Rs. 110, Rs. 130 and Rs. 150, themarketing department estimates the demand for the DVDin the three regions where the DVD would be sold.

What price would maximize profit?

Price in Rs. Demand (in Thousands)Region 1 Region 2 Region 3

110 35 32 24

130 32 27 17

150 22 16 9


40/65


Solution

The demands curves for Region 1, Region-2, Region-3

can be estimated using nonlinear regression

The curves for Regions 2 and 3 look very much similar

Region-1

0

5

10

15

20

2530

35

40

0 50 100 150 200

Price

Demand

Demand


41/65


Region-1: Demand1= -0.87 (price)2+1.95*(price)-73.625



Total demand (TD) = Demand1+Demand2+Demand3

Profit = (price-55)*TD

Profit is a Non-linear function of one variable. The functionis concave and the optimal price is Rs. 129


42/65


Lagrange multipliers

Lagrange multipliers can be used to solve NLPs


43/65


Lagrange multipliers-Examples

A company is planning to spend $10,000 on advertising. It

costs $ 3,000 per minute to advertise on television and $ 1,000per minute to advertise on radio. If the firm buys x minutes oftelevision advertising and y minutes of radio advertising, thenits revenue in thousands of dollars is given by

f(x, y) = -2x2 y2 + xy + 8x + 3y


44/65


Kuhn-Tucker Conditions

KT Conditions, sometimes known as Karush-Kuhn-Tucker

(KKT) conditions Gives necessary and sufficient conditions for

to be an optimal solution for NLP

),...,,( 21 nxxxx

mi

bxxxg

xxxf

ini

n

,...,2,1

),...,,(S.t.

),...,,(min)(ormax

21

21


45/65


Kuhn-Tucker Conditions [contd.]

For maximization problems. If

is an optimal solution to the problem, then

must satisfy the mconstraints of the NLP, and there must existmultipliers satisfying

)

)

)m21i0

m21i0bxg

n21j0x

xg

x

xf

iii

mi

1i j

ii

j

,...,,;

,...,,;)(

,...,,;)()(

l

l

l

),...,,( 21 nxxxx

),...,,( 21 nxxxx

mlll ,...,, 21


46/65


Kuhn-Tucker Conditions [contd.]

For minimization problems. If

is an optimal solution to the problem, then

must satisfy the mconstraints of the NLP, and there must existmultipliers satisfying

)

)

)m21i0

m21i0bxg

n21j0x

xg

x

xf

iii

mi

1i j

ii

j

,...,,;

,...,,;)(

,...,,;)()(

l

l

l+

),...,,( 21 nxxxx

),...,,( 21 nxxxx

mlll ,...,, 21


47/65


Sufficiency of the KKT conditions:

Sense of

optimization

maximization

minimization

Required conditions

Objective

function

Solution

space

concave

convex

Convex set

Convex set


48/65


It is simpler to verify whether a function is

concave or convex than to prove that thesolution space is a convex set.

We thus give a set of sufficient conditions

that are easier to check that the solution

space is a convex set in terms of the

convexity or concavity of the constraint

functions.


49/65


Consider the general non-linear problem:

Maximize or minimize z =f(X)

Subject to gi(X) 0 i = 1,2,..,p

gi(X) 0 i =p+1,p+2,.., q

gi(X) = 0 i = q+1, q+2,.., r


50/65


Sufficiency of the KKT conditions:

Sense ofoptimization

maximization

minimization

Required conditions

0

0

URS

convex

concave

linear

convex

1 ip

p+1 i q

q+1 i r

f(X) gi(X) i

0

0

URS

convex

concave

linear

1 ip

p+1 i q

q+1 i r

concave


51/65


The conditions in the above table

represent only a subsetof the conditionsgiven in the earlier table.

The reason is that a solution space may be

convex without satisfying the conditions

of the constraint functions given in the

second table.


52/65


Problem Use the KKT conditions to derive

an optimal solution for the following

problem:

maximize f(x1, x2) = x1+ 2x2x23

subject to x1 + x21

x1 0

x20


53/65


Solution: Here there are three constraints

namely,

g1(x1,x2) = x1+x2- 10

g2(x1,x2) = - x1 0

g3(x1,x2) = - x2 0

Hence the KKT conditions become

01

3

3

1

2

2

1

1

1

1

x

g

x

g

x

g

x

flll

10,20, 30


54/65


1g1(x1,x2) = 0

2g2

(x1

,x2

) = 0

3g3(x1,x2) = 0

g1(x

1,x

2)0

g2(x1,x2)0

g3(x

1,x

2)

0

Note:fis concave

gi are convex,

maximizationproblem

these KKT

conditions are

sufficient at the

optimum point

02

3

3

2

2

2

2

1

1

2

x

g

x

g

x

g

x

flll

i 1 + 0 (1)


55/65


i.e. 11 +2 = 0 (1)

23x221 +3 = 0 (2)

1(x1 + x21) = 0 (3)

2x1 = 0 (4)

3x2 = 0 (5)

x1 + x21 0 (6)

x1 0 (7)

x2 0 (8) 1 0 (9)

2 0 (10) and 3 0 (11)

(1) gives 1 + 1 >0 (using 10)


56/65


(1) gives1 = 1 +2 1 >0 (using 10)

Hence (3)givesx1 + x2 = 1 (12)

Thus both x1, x2cannot be zero.

So letx1>0 (4)gives2 = 0.therefore1 = 1

if nowx2 = 0, then (2) gives 201 +3= 0or3 < 0

not possible

Therefore x2 > 0

hence (5) gives3 = 0 and then (2) givesx22 = 1/3so x2 =1/3

And sox1 = 1- 1/3

Max f = 1 - 1/3 + 2/3 1/33 =1 + (2 / 33 )


57/65


Maximizef(x) = 20x1 + 10 x2

Subject tox12 + x2

21

x1 + 2x22

x10, x20

KKT conditions become

20 - 21x12+3= 0

10 - 21x222 +4= 0

1 (x12+ x2

21) =0

1 (x1+ 2x22) =0

3x1 =0

(0,1)

(4/5,3/5)

x 0


58/65


4x2 =0

x12 + x2

2 1

x1 + 2x2 2

x1 0

x2 0

1 0

2 0

3 0

4 0

F h fi i i l h f ( ) h


59/65


From the figure it is clear that max f occurs at (x1, x2)where

x1, x2>0.

3 = 0,4 = 0supposex1 + 2x22 0

2 = 0 ,therefore we get 20 - 21x1=0

10 - 21x2=0

1x1=10 and1x2=5, squaring and adding we get

12

= 125 1 = 55therefore x1 = 2/5, x2 = 1/5, f= 50/5 >22

0 x + x 2 = 0


60/65


2 0 x1 + x22 = 0

Thereforex1=0, x2=1, f =10

Or x1= 4/5, x2= 3/5, f=22

Therefore max f occurs at x1 = 2/5, x2 = 1/5

P bl U h KKT di i d i


61/65


Problem Use the KKT conditions to derive

an optimal solution for the following

problem:

minimizef(x1, x2) = x12+ x2

subject to x12

+ x22

9x1+x2 1

Solution:Here there are two constraints,

namely, g1(x1,x2) = x12+x2

2- 9 0

g2(x1,x2) = x1 +x2 -1 0

Th h KKT di i


62/65


0,0:1 21 ll

as it is a minimization problem

0)1(

0)9(:3

212

2

2

2

11

+

+

xx

xx

l

l

1

9:4

21

2

2

2

1

+

+

xx

xx

1 1 1 2

1 2 2

2: 2 2 0

1 -2 0

x x

x

l l

l l

Thus the KKT conditions are:

0l 1l


63/65


Now (from 2) gives01 l 12 l Not possible.

Hence 01 l and so 92

2

2

1+ xx

Assume . So (1st equation of ) (2) gives02 l

0)1(2 11 lx Since we getx1= 001 l

(5)

From (5), we get 32 x

2nd equation of (2) says (with )x2 = -3,01


64/65


Problem-1

A monopolist can purchase up to 17.25 oz of a chemical for

$10/oz. At a cost of $3/oz, the chemical can be processedinto an ounce of product-1; or, at a cost of $5/oz, thechemical can be processed into an ounce of product-2. If x1oz of product-1 are produced, it sells for a price of $(30 x1)

per ounce. If x2 oz of product-2 are produced, it sells for aprice of $(50 2x2) per ounce. Determine how themonopolist can maximize profits.

P bl 2


65/65

Problem-2

A power company faces demands during both peak and off-

peak times. If a price of p1 dollars per KWH is chargedduring the peak time customers will demand (60 0.5p1)KWH of power. If a price of p2 dollars per KWH ischarged during the off-peak time customers will demand (40

p2)KWH of power. The power company must havesufficient capacity to meet demand during both the peak andoff-peak time.

It costs $10 per day to maintain each KWH of capacity.Determine how the power company can maximize dailyrevenues less operating costs.

nonlinear programming solution a

Documents