Direct optimization methods for solving a

complex state–constrained optimal control

problem in microeconomics

Helmut Maurer a Hans Josef Pesch b

aUniversitat Munster, Institut fur Numerische und Angewandte MathematikEinsteinstr. 62, 48149 Munster, Germany, Email: maurer@math.uni-muenster.de

bUniversitat Bayreuth, Fakultat fur Mathematik und Physik, Universitatstr. 30,95440 Bayreuth, Germany, Email: hans-josef.pesch@uni-bayreuth.de

Abstract

We analyze and solve a complex optimal control problem in microeconomics whichhas been investigated earlier in the literature. The complexity of the control prob-lem originates from four control variables appearing linearly in the dynamics andseveral state inequality constraints. Thus the control problem offers a considerablechallenge to the numerical analyst. We implement a hybrid optimization approachwhich combines two direct optimization methods. The first step consists in solvingthe discretized control problem by nonlinear programming methods. The secondstep is a refinement step where, in addition to the discretized control and statevariables, the junction times between bang–bang, singular and boundary subarcsare optimized. The computed solutions are shown to satisfy precisely the necessaryoptimality conditions of the Maximum Principle where the state constraints aredirectly adjoined to the Hamiltonian. Despite the complexity of the control struc-ture, we are able to verify sufficient optimality conditions which are based on theconcavity of the maximized Hamiltonian.

Key words: microeconomic control model, control of stock, labor and capital,state inequality constraints, direct optimization methods, bang–bang and singularcontrol, verification of necessary and sufficient conditionsPACS: 49J15, 49K15, 58E17, 65K05

1 Introduction

The well–known microeconomic concern model of Lesourne, Leban [10] in-volves only capital flows as control and state variables. Koslik, Breitner [8]and Winderl, Naumer [16] have developed an extended concern model that

Preprint submitted to Elsevier 13 September 2007

includes the production and employment sector. Besides its economic interest,the optimal control problem constitutes a considerable numerical challenge,since it comprises four control variables appearing linearly in the dynamicsand several pure state inequality constraints.

In [8,16], a hybrid numerical approach has been developed to determine thecomplicated control switching structure. First, a discretized version of thecontrol problem is solved by nonlinear programming methods. This methodyields reliable estimates for the control and state variables on a fixed grid.The second step is a refinement step, where the control and state estimatesare used in the so–called indirect method which requires the solution of aboundary value problem (BVP) for the state and adjoint variables. For theconcern model, it is extremely difficult to set up the BVP due to the presenceof pure state constraints. For this reason, authors [8,16] have substituted theactive state constraints by suitable mixed control–state constraints that arebetter tractable in the BVP formulation.

The purpose of the paper is twofold. First, we discuss direct optimizationmethods that provide solutions which satisfy precisely the Maximum Prin-ciple for state–constrained optimal control problems. The second goal is toshow that the computed solution satisfies a suitable type of sufficient opti-mality conditions. The organisation of the paper is as follows. The concernmodel is presented in Section 2. In Section 3, necessary optimality conditionsare discussed which are based on a Maximum Principle, where the state con-straints are directly adjoined to the Hamiltonian. In Section 4, we present ahybrid optimization approach to solve the state–constrained control problem.The first step is similar to that in [8,16] and differs only in that we apply thelarge–scale optimization methods developed by Buskens [2,3] and Wachter[15]. The second step is different from the one in [8,16]. Instead of trying tosolve the BVP of the Maximum Principle, we optimize simultaneously theswitching and junction times between bang–bang, singular and boundary arcsand the discretized control variables; cf. [5,13]. The computed control andstate variables satisfy the Maximum Principle with high accuracy. Finally,in Section 5 we show that the computed solutions satisfy a suitable type ofsufficient conditions.

2 Optimal control model for a concern with four control variablesappearing linearly and state constraints

The microeconomic control model discussed in Koslik, Breitner [8] and Winderl,Naumer [16] has six state variables and four control variables

x = (S, L, Y,X,Xm, Xr) ∈ IR6, u = (Sc, Lc, Yc, I) ∈ IR4,

2

which have the following meaning. The stock S(t) is controlled by Sc(t); thenumber L(t) of employees is controlled by the employment rate Lc(t); thecapital consists of loan capital Y (t) and equity capital X(t); the control Yc(t)describes the borrowing of loan capital while the owners of the equity cap-ital choose by means of the investment control I(t) between an investmentwithin the concern and an alternative investment Xm(t); the risk premiumXr(t) serves as a reserve fund for the safety of the capital owners; the riskpremium is denoted by ρr(t). All parameters and functions appearing in thefollowing quantities and differential equations are summarized in Table 1. Theproduction function (output) is assumed to be of Cobb–Douglas type:

F (x) = F (L, Y,X) = α(X + Y )αKLαL . (1)

Then the profit (gain) of the concern is given by

G(x, u, t) =1

d(t)[ p(t)(F (x)− Sc)− σS − ω(t)L ]− ρK(t)Y − δ(X + Y ). (2)

The discount rate d(t) is defined as the solution of the differential equation

d(t) = −d(t)ln(1 + i(t)), (3)

where i(t) is the periodic inflation rate specified in Table 1. Note that incontrast to the presentation in [8,16], we do not treat d(t) as a state variable. Insection 5, this viewpoint will allow us to apply sufficient optimality conditions.The dynamics is governed by differential equations with fixed initial values,

x(t) = f(x(t), u(t), t), x(0) = x0, (4)

which are given explicitly by

S(t) = Sc(t) , S(0) = 100,

L(t) = Lc(t) , L(0) = 30,

Y (t) = Yc(t) , Y (0) = 50,

X(t) = I(t) + (1− τ) [G(x(t), u(t), t)− ρr(t)X(t) ] , X(0) = 100,

Xm(t) = −I(t) + (1− τ)ρm(t)Xm(t) , Xm(0) = 450,

Xr(t) = (1− τ)ρr(t)X(t) , Xr(0) = 0.

(5)

3

Notation Formula / Value Meaning

tf 10 time horizon in years

F (x) α(X + Y )αKLαL production function (output)

α 100 parameter in production function

αK 0.35 elasticity of total capital K = X + Y

αL 0.5 elasticity of labor

kl 8 duration of economic cycle

kp(t)π2 + 2π

kl· t position in economic cycle

ρK(t) 0.110 + 0.030 sin kp(t) loan interest rate

ρm(t) 0.074 + 0.018 sin kp(t) current yield

ρr(t) ρK(t)− 0.05 risk premium rate

ρlowr (t) 23 (ρK(t)− 0.08) + 0.02 risk premium rate for daring investors

i(t) 0.019 + 0.029 sin kp(t) inflation rate

p 0.05 constant selling price

p(t) 0.05 + 0.01 sin( 2πkl· t) variable selling price

κ 0.8 rate of maximal borrowing

ω 2.0 constant labor cost

ω(t) 2.0 exp(0.02 · t) increasing labor cost

δ 0.440 depreciation rate

τ 0.5 tax rate

σ 0.01 storage charges

Table 1Parameter and function values for the microeconomic control model.

The economic process is considered on a time interval t ∈ [0, tf ] with fixedtime horizon tf > 0. The control constraints are given as box constraints,

Sc,min ≤ Sc(t) ≤ Sc,max , Lc,min ≤ Lc(t) ≤ Lc,max ,

Yc,min ≤ Yc(t) ≤ Yc,max , Imin ≤ I(t) ≤ Imax ,(6)

for all t ∈ [0, tf ], where we choose the following data:

Sc,min = −100, Sc,max = 100, Lc,min = −10, Lc,max = 10,

Yc,min = −100, Yc,max = 100, Imin = −100, Imax = 100.

4

The control constraints are written as u(t) ∈ U , where the cube U ⊂ IR4 isdefined in an obvious way. The state inequality constraints are

Smin = 50 ≤ S(t), Y (t) ≤ κX(t), for all t ∈ [0, tf ]. (7)

The second state constraint imposes a maximal borrowing of loan capital. Thefurther obvious state constraints 0 ≤ xi(t) (i = 2, ..., 6) do not become activeand will therefore be omitted in the analysis of necessary conditions. Thenthe optimal control problem consists in determining a piecewise continuous(measurable) control u : [0, tf ] → IR4 and an absolutely continuous statetrajectory x : [0, tf ] → IR6 that maximize the cost functional in Mayer formrepresenting the joint capital of capital owners

Φ(x(tf ), tf) := X(tf) +Xm(tf) + (1− τ)p(tf )

d(tf )S(tf) (8)

subject to the constraints (5)–(7).

3 Necessary optimality conditions

A survey on necessary and sufficient conditions for state constrained optimalcontrol problems may be found in Hartl, Sethi, Vickson [7]. The optimal con-trol (5)–(8) has the form of the control problem in Section 2 of [7], where themixed control-state constraint is given by the simple box constraint (6). Thestate constraints (7) are written as

0 ≤ h1(x) := S − Smin, 0 ≤ h2(x) := κX − Y . (9)

We choose the direct adjoining approach described in Section 4 of [7], in whichthe state constraints (9) are directly adjoined to the Hamiltonian. Necessaryconditions require regularity conditions for the state constraints which areassociated with the order of a state constraint; cf. Section 2 in [7]. Both stateconstraints in (9) have order one, since

h11(x, u, t) = h1 = S = Sc,

h12(x, u, t) = h2 = κX − Y = κ [ (I + (1− τ)(G(x, u, t)− ρr(t)X) ]− Yc .

(10)

In view of the gain function (2) we obtain

∂h11

∂u(x, u, t) = (1, 0, 0, 0) 6= 0,

∂h12

∂u(x, u, t) = (κ(1− τ)

p(t)

d(t), 0,−1, κ) 6= 0, (11)

5

which implies that the regularity condition (2.11) in [7] holds,

rank

∂h1

1/∂u

∂h12/∂u

= 2 ∀ (x, u, t).

In particular, this holds along any boundary arc with h1(x(t)) = 0 or h2(x(t)) =0 for t ∈ [ten, tex], where ten, resp., tex denotes the entry-time, resp., the exit-time of the boundary arc. On a boundary arc with h1(x(t)) = 0, the boundarycontrol is given by

h11(x(t), u(t), t) = Sc(t) ≡ 0 for t ∈ [ten, tex]. (12)

However, the boundary control on a boundary arc with h2(x(t)) = 0 is notuniquely defined by the relation

h12(x, u, t) = h1

2(x, I, Yc, t) = 0.

On such a boundary arc, further relations defining the controls Yc and I can beobtained from the following necessary conditions of the Maximum Principlewhich is stated as Informal Theorem 4.1 in [7]. Since all control variablesappear linearly in the control systems, the Hamiltonian can be written as

H(x, u, , λ, t) =λf(x, u, t) = σSc · Sc + σLc · Lc + σYc · Yc + σI · I (13)

+R(x, λ, t),

R(x, λ, t) =λX · (1− τ) ( p(t)α(X + Y )αKLαL − σS − ω(t)L )/d(t) (14)

−λX · (1− τ)(ρK(t)Y + δ(X + Y ) + ρr(t)X )

+λXm(1− τ)ρm(t)Xm + λXr(1− τ)ρr(t)X .

The adjoint variable λ = (λS, λL, λY , λX , λXm, λXr) ∈ IR6 is a row-vector.The factors to the control components in the Hamiltonian are the switchingfunctions

σSc = λS − λX(1− τ)p(t)d(t)

, σLc = λL , σYc = λY , σI = λX − λXm . (15)

Note that the switching vector σ = (σSc, σLc, σYc, σI) does not depend onthe state variable x but only on the adjoint variable λ. This property willbe important for the verification of sufficient conditions in Section 5. TheLagrangian is defined by adjoining the state constraints (9) directly to theHamiltonian by a multiplier ν = (ν1, ν2) ∈ IR2,

L(x, u, λ, ν, t) = H(x, u, λ, t) + ν1(S − 50) + ν2(κY −X). (16)

6

For the economic control problem under investigation, the Maximum Principlecan be rigorously justified using the techniques in Maurer [12]. Then the In-formal Theorem 4.1 in Hartl et al [7] gives the following necessary conditions.Let (x(t), u(t)) be an optimal pair for the control problem (5)–(8). For conve-nience, we drop asterisks or hats to denote an optimal solution. The notation[t] will be used to abbreviate arguments (x(t), u(t), λ(t), t). Then there exist aconstant multiplier λ0 ≥ 0, a piecewise absolutely continuous adjoint functionλ : [0, tf ] → IR6, a piecewise continuous multiplier function ν : [0, tf ] → IR2,a vector η(τ) = (η1(τ), η2(τ)) ∈ IR2 for each junction time τ with a boundaryarc, and a multiplier γ = (γ1, γ2) ∈ IR2 with (λ0, λ(t), ν(t), η(τ), γ) 6= 0 suchthat the following conditions hold for a.e. t ∈ [0, tf ]: the minimum condition

u(t) = arg max u∈U H(x(t), u, λ(t), t), (17)

the adjoint equation

λ(t) = −Lx[t], (18)

the jump condition at a junction time

λ(τ ∗) = λ(τ−) + η(τ)hx[τ ], (19)

the transversality condition at the terminal time

λ(tf ) = λ0Φx[tf ] + γ1(h1)x[tf ] + γ2(h2)x[tf ], (20)

and the complementarity conditions

ν(t) ≥ 0, ν(t)h[t] = 0, γi ≥ 0, γihi[tf ] = 0 (i = 1, 2). (21)

The evaluation of the adjoint equations on the basis of equations (13), (14)and (16) is left to the reader. The transversality condition (20) yields in viewof the cost functional (8) and the state constraint (9):

λS(tf ) = λ0(1− τ)p(tf )/d(tf) + γ1 , λL(tf ) = 0 , λY (tf ) = −γ2 ,

λX(tf ) = λ0 + γ2κ , λXm(tf ) = λ0 , λXr(tf ) = 0 .(22)

The numerical results in the next section show that the normality conditionλ0 = 1 holds and the multipliers γ1, γ2 are zero, though both state constraintsare active at the terminal time. The maximum condition (17) for the control

7

yields the following switching law for the control vector u = (u1, u2, u3, u4),where the switching functions σi(t), i = 1, 2, 3, 4, are defined in (15):

ui(t) =

ui,max , if σi(t) > 0

ui,min , if σi(t) < 0

singular , if σi(t) = 0 on [ten, tex] ⊂ [0, tf ]

(23)

Here, ten, resp. tex means the entry-time, resp., exit-time of a singular arc. Oninterior arcs with h(x(t)) < 0, one obtains further information on a singularcontrol ui by differentiating the switching relation σi(t) = 0, t ∈ [ten, tex].We refrain from discussing this procedure in detail. On a boundary arc, thefollowing property is noteworthy. If the component ui(t) of a boundary controllies in the interior of its control region, i.e., satisfies

ui,min < ui(t) < ui,max for ten < t < tex,

then the maximum condition (17) implies

σi(t) = 0 for ten < t < tex . (24)

Hence, the boundary control ui(t) formally behaves as a singular control. Thisproperty has been exploited in Maurer [11] to derive junction conditions forjunctions between interior arcs and boundary arcs. Though the proof tech-niques in [11] have been developed only in the case of a scalar control, aninspection of the economic problem reveals that some junction results in [11]can be extended to the vector–valued control case considered here. In partic-ular, it follows that the adjoint variables are continuous on [0, tf ], since thestate constraints are of order one and the relevant control components arediscontinuous at the entry–times of the boundary arcs; cf. Corollary 5.2 (ii)and Theorem 5.4 in [11]. Thus the multipliers η(τ) in the jump condition (19)vanish.

4 Numerical solution and verification of necessary conditions

The optimal control and state trajectory will be determined in two steps by ahybrid numerical approach where two direct optimization methods are com-bined. In the first step, the optimal control problem is discretized on a fixedgrid. This leads to a large-scale opimization problem that may be solved byvarious nonlinear programming techniques. We have used two nonlinear pro-gramming implementations. Method-1: the control package NUDOCCCS byBuskens [3,4] with up to N = 1000 grid points and a 4th order Runge–Kutta

8

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

0 2 4 6 8 10-0.01

0

0.01

0.02

0.03

0.04

0.05

0 2 4 6 8 10

Fig. 1. Left: interest rates ρK(t) (above) and ρm(t)+ρr(t); Right: inflation rate i(t).

integration scheme; Method-2: the programming language AMPL [6] and theInterior–Point Method code IPOPT developed by Wachter, Biegler [15] us-ing up to N = 50.000 grid points with an EULER or HEUN integrationscheme. Both methods are capable of detecting the correct control switchingstructure. In addition, Method (2) provides rather accurate estimates for theswitching and junction times between bang–bang and singular or boundaryarcs. Moreover, Lagrange multipliers of the nonlinear programming problemscan be identified with the values at grid points of the adjoint variables andmultipliers for the state constraints.

The second step is a refinement step, where the switching and junction timesare determined with higher accuracy. Rather then optimizing the junctiontimes directly, the arclengths of bang–bang or singular arcs are treated asadditional optimization variables. The implementation relies on a time-scalingand multiprocess control technique described in Buskens, Pesch, Winderl [5].A simplified approach avoiding the multiprocess formulation may be found inMaurer, Buskens, Kim, Kaya [13]. Both methods and the refinement step havebeen tested in the diploma theses of Balzer [1] and Lang [9].

Now we present optimal control solutions for two data sets. The interest ratesρK(t), ρm(t) + ρr(t) and the inflation rate i(t) given in Table 1 are depicted inFigure 1.

4.1 Solution for constant price p = 0.05, δ = 0.44 and constant wage ω = 2

Let us denote this data set by Data-1. The computed control u = (Sc, Lc, Yc, I)has a rather complicated switching structure with 8 bang–bang and singularsubarcs; cf. Table 2 with obvious notations. We obtain the following switchingand junction times:

t1 = 0.34711 t2 = 0.50000, t3 = 0.56660 t4 = 0.58777

t5 = 0.61542 t6 = 0.67125 t7 = 8.9394

9

t Sc Lc Yc I 50 ≤ S 0 ≤ 0.8X − Y

[0., t1] min max min min non-active non-active

[t1, t2] min max max min non-active non-active

[t2, t3] 0 max max min active nonactive

[t3, t4] 0 min max min active non-active

[t4, t5] 0 min singular min active non-active

[t5, t6] 0 min singular singular active active

[t6, t7] 0 singular singular singular active active

[t7, tf ] 0 max singular singular active active

Table 2Data-1: structure of optimal control with bang-bang, singular and boundary arcs

40

50

60

70

80

90

100

0 2 4 6 8 10 30

35

40

45

50

55

60

65

0 2 4 6 8 10 10

20

30

40

50

60

70

80

90

0 2 4 6 8 10

Fig. 2. Stock S(t), employment L(t) and loan capital Y (t) .

The optimal functional value is Φ(x(tf )) = 989.059 . The optimal state tra-jectories x(t) = (S(t), L(t), Y (t), X(t), Xm(t), Xr(t)) are shown in Figs. 2 and3. The control u(t) and the switching functions σ(t) are displayed in Figs.4–7. The adjoint variable λL and λY are shown in Figures 5, 6, while the ad-joints λS, λX , λXm and the multiplier ν2 for the state constraint 0 ≤ κY −Xare displayed in Figs. 8, 9. The adjoint variable λXr vanishes identically. Thecomputed initial values of the adjoints are

λS(0) = 0.03293358, λL(0) = 0.09139124, λY (0) = −0.03061563,

λX(0) = 1.415673, λXm(0) = 1.464419, λXr(0) = 0.0 .

The adjoint variables are continuous in [0, tf ]. Hence, the multiplier η(τ) inthe jump condition (19) vanishes. The terminal value is

λ(tf) = (0.031227953, 0.0, 0.0, 1.0, 1.0, 0.0) ∈ IR6,

which shows that the transversality condition (20) is satisfied with the multi-plier λ0 = 1 and multipliers γ1 = γ2 = 0.

The behavior of the switching functions (15) is in perfect agreement with the

10

40

50

60

70

80

90

100

110

0 2 4 6 8 10 450

500

550

600

650

700

750

800

850

900

0 2 4 6 8 10 0

5

10

15

20

25

0 2 4 6 8 10

Fig. 3. Equity capital X(t), alternative assets Xm(t) and risk premium Xr(t) .

-100

-80

-60

-40

-20

0

0 1 2 3 4 5 6 7 8 9 10-0.0025

-0.002

-0.0015

-0.001

-0.0005

0

0.0005

0 1 2 3 4 5 6 7 8 9 10

Fig. 4. Stock control Sc(t) and switching function σSc(t) .

-10

-5

0

5

10

0 1 2 3 4 5 6 7 8 9 10-0.01

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0 1 2 3 4 5 6 7 8 9 10

Fig. 5. Employment control Lc(t) and switching function σLc(t) = λL(t) .

-100

-50

0

50

100

0 1 2 3 4 5 6 7 8 9 10-0.035

-0.03

-0.025

-0.02

-0.015

-0.01

-0.005

0

0.005

0 1 2 3 4 5 6 7 8 9 10

Fig. 6. Loan capital control Yc(t) and switching function σYc(t) = λY (t) .

control law (23) and the control structure in Table 2, since we have

σSc(t)

< 0 , 0 ≤ t < t2

= 0 , t2 ≤ t ≤ 10

, σI(t)

< 0 , 0 ≤ t < t5

= 0 , t5 ≤ t ≤ tf

,

σLc(t)

> 0 , 0 ≤ t < t3

< 0 , t3 < t < t6

= 0 , t6 ≤ t ≤ t7

> 0 , t7 ≤ t ≤ t7

, σYc(t)

< 0 , 0 ≤ t < t1

> 0 , t1 < t < t4

= 0 , t4 ≤ t ≤ tf

.

11

-100

-80

-60

-40

-20

0

0 1 2 3 4 5 6 7 8 9 10-0.05

-0.04

-0.03

-0.02

-0.01

0

0 1 2 3 4 5 6 7 8 9 10

Fig. 7. Investment control I(t) and switching function σI(t) .

0.031

0.032

0.033

0.034

0.035

0.036

0.037

0 2 4 6 8 10 0.95

1

1.05

1.1

1.15

1.2

1.25

1.3

1.35

1.4

1.45

0 2 4 6 8 10

Fig. 8. Adjoint variables λS(t), λX(t) .

0.95

1

1.05

1.1

1.15

1.2

1.25

1.3

1.35

1.4

1.45

1.5

0 2 4 6 8 10 0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0 1 2 3 4 5 6 7 8 9 10

Fig. 9. Adjoint λXm(t) and multiplier ν2(t) for constraint 0 ≤ 0.8Y (t)−X(t) .

-0.0001

0

0.0001

0.0002

0.0003

0.0004

0.0005

8.6 8.8 9 9.2 9.4 9.6 9.8 10-0.001

-0.0008

-0.0006

-0.0004

-0.0002

0

0.0002

0.0004

0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65

Fig. 10. Zoom into the switching functions σLc(t) and σYc(t) .

In addition, the switching functions satisfy the so-called strict bang-bang prop-erty. Namely, σk(tj) 6= 0 holds at any switching time tj between bang-bangarcs of the control component uk. Figure 10 zooms into the switching functionσLc(t) and σYc(t) to demonstrate in greater detail that (a) the control Lc(t)has a junction of a singular arc with a bang-bang arc at t7 = 8.9394, and (b)the control Yc(t) switches between bang-bang arcs at t1 = 0.34711 and has ajunction with a singular arc at t4 = 0.58777.

12

t Sc Lc Yc I 50 ≤ S 0 ≤ 0.8X − Y

[0., t1] min max min max non-active non-active

[t1, t2] min max max min non-active non-active

[t2, t3] min max singular singular non-active active

[t3, t4] 0 max singular singular active active

[t4, t5] 0 min singular singular active active

[t5, tf ] 0 max singular singular active active

Table 3Data-2 : structure of optimal control with bang-bang, singular and boundary arcs.

50

55

60

65

70

75

80

85

90

95

100

0 2 4 6 8 10 20

25

30

35

40

45

50

55

60

65

0 2 4 6 8 10 20

40

60

80

100

120

140

160

180

0 2 4 6 8 10

Fig. 11. Data-2 : Stock S(t), employment L(t) and loan capital Y (t) .

4.2 Solution for variable price p(t) = 0.05 + 0.01 sin(2πt/8), depreciationrate δ = 0.322 and increasing wage ω(t) = 2 exp(0.02 · t)

We choose a data set denoted by Data-2 which is significantly different fromthat in Section 4.1 by considering the variable price p(t) = 0.05+0.01 sin(2πt/8),depreciation rate δ = 0.322 and increasing wage ω(t) = 2 exp(0.02 · t); cf. [1].Then the computed control u = (Sc, Lc, Yc, I) is a combination of 6 bang–bangand singular subarcs that are described in Table 3. We obtain the switchingand junction times

t1 = 0.16601 , t2 = 0.38420 , t3 = 0.50000 , t4 = 2.8496, t5 = 6.1978 .

and the optimal functional value Φ(x(tf )) = 1103.783 . The optimal statetrajectories x(t) = (S(t), L(t), Y (t), X(t), Xm(t), Xr(t)) are shown in Figs. 11and 12, while Figs 13–16 depict the optimal control components jointly withthe associated switching functions.

The adjoint variables λ ∈ IR6 and the multiplier ν2 for the state constraint0 ≤ κY −X are displayed in Figs. 14, 15, 17, 18. Again, we have λXr(t) ≡ 0

13

40

60

80

100

120

140

160

180

200

220

0 2 4 6 8 10 450

500

550

600

650

700

750

800

850

900

0 2 4 6 8 10 0

5

10

15

20

25

30

35

0 2 4 6 8 10

Fig. 12. Equity capital X(t), alternative assets Xm(t) and risk premium Xr(t) .

-100

-80

-60

-40

-20

0

0 1 2 3 4 5 6 7 8 9 10-0.00045

-0.0004

-0.00035

-0.0003

-0.00025

-0.0002

-0.00015

-1e-04

-5e-05

0

5e-05

0 1 2 3 4 5 6 7 8 9 10

Fig. 13. Stock control Sc(t) and switching function σSc(t) .

-10

-5

0

5

10

0 1 2 3 4 5 6 7 8 9 10-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

0 1 2 3 4 5 6 7 8 9 10

Fig. 14. Employment control Lc(t) and switching function σLc(t) = λL(t) .

-100

-50

0

50

100

0 1 2 3 4 5 6 7 8 9 10-0.007

-0.006

-0.005

-0.004

-0.003

-0.002

-0.001

0

0.001

0 1 2 3 4 5 6 7 8 9 10

Fig. 15. loan capital control Yc(t) and switching function σYc(t) = λY (t) .

in [0, tf ]. The computed initial values of the adjoints are

λS(0) = 0.03577027, λL(0) = 1.420754, λY (0) = −0.006379557,

λX(0) = 1.446176, λXm(0) = 1.464490, λXr(0) = 0.0,

while the terminal value is λ(tf) = (0.03747354, 0.0, 0.0, 1.0, 1.0, 0.0) ∈ IR6.

The switching functions (15) obey the control laws (23), resp., the control

14

-100

-80

-60

-40

-20

0

20

40

0 1 2 3 4 5 6 7 8 9 10-0.02

-0.018

-0.016

-0.014

-0.012

-0.01

-0.008

-0.006

-0.004

-0.002

0

0.002

0 1 2 3 4 5 6 7 8 9 10

Fig. 16. Investment control I(t) and switching function σI(t) .

0.024

0.026

0.028

0.03

0.032

0.034

0.036

0.038

0.04

0.042

0.044

0 2 4 6 8 10 0.95

1

1.05

1.1

1.15

1.2

1.25

1.3

1.35

1.4

1.45

0 2 4 6 8 10

Fig. 17. adjoint variables λS(t), λX(t) .

0.95

1

1.05

1.1

1.15

1.2

1.25

1.3

1.35

1.4

1.45

1.5

0 2 4 6 8 10 0

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

0.018

0 1 2 3 4 5 6 7 8 9 10

Fig. 18. Adjoint variable λXm(t) and multiplier ν2(t) for constraint0 ≤ 0.8Y (t)−X(t) .

structure in Table 3 and also satisfy the strict bang-bang property at switchingtimes between bang-bang arcs.

5 Verification of sufficient optimality conditions

We are going to show that the sufficient optimality conditions of Arrow–typein Hartl, Sethi, Vickson [7], Theorem 8.2, hold for both controls presented insection 4.1 and 4.2. The first assumption in Theorem 8.2 of [7] requires that thenecessary conditions be satisfied with a multiplier λ0 = 1 in the transversalitycondition (22). This property holds as stated earlier in Section 4; cf. Figs. 5,6, 8, 9 and Figs. 14, 15, 17, 18. The function

Φ(x, tf ) = X +Xm + S(1− τ)p(tf)/d(tf)

15

defining the cost functional (8) is linear in x and, hence, concave in x. Note thatwe did not treat the discount rate d(t) defined by equation (3) as an auxiliarystate variable x7 as it was done in [8,16]. This additional state variable woulddestroy the concavity of the function Φ(x, tf ). The crucial condition then isthe property that the maximized Hamiltonian

H0(x, λ, t) = max u∈U H(x, u, λ, t)

is concave for all (λ(t), t), t ∈ [0, tf ]. It can readily be seen from (13) and (14)that the maximized Hamiltonian is given by

H0(x, λ(t), t) = λX(t)(1− τ) p(t)F (x) +R0(t), (25)

where F (x) = F (L, Y,X) = α(X +Y )αKLαL is the Cobb–Douglas productionfunction (1) and R0(t) does not depend on the state variable x. The productionfunction F (x) = F (L, Y,X) is concave for L > 0, Y > 0, X > 0 in view ofthe assumption 0 < α, αK, αL and αK + αL < 1. This follows from the factthat the Hessian D2

xxF (x) is negative semi–definite since four eigenvalues arezero and two eigenvalues are negative. Moreover, Figs. 8 and 17 show thatλX(t) > 0 holds for t ∈ [0, tf ]. Since (1− τ)p(t) > 0, we finally conclude thatthe maxized Hamiltonian H0(x, λ(t), t) in (25) is concave in x, which confirmsthat the computed controls are optimal.

6 Comparison and Conclusion

The complicated control structure for Data-1 shows that many switchings be-tween bang-bang arcs occur in the very beginning of the planning period, moreprecisely for 0 ≤ t ≤ t6 = 0.671. The reason for such multiple switchings maybe that the initial values of loan and equity capital are not chosen properly. Forthe largest part of the planning period, namely for t6 = 0.671 ≤ t ≤ t7 = 8.94,all controls are singular and take values in the interior of the control region.Towards the end of the planning period, the employment rate Lc(t) is increas-ing and takes its maximum value on the final arc [t7, tf ].

The optimal solution for Data-2 is significantly different from the Data-1 so-lution as can be clearly seen in the behavior of the employment control Lc(t).This is due to the variable price function p(t) which is increasing in the periods[0, 2] and [6, 10], but decreasing in the period [2, 6]. The employment controlLc(t) is bang-bang and takes its minimum negative value Lc,min = −10, i.e.adopts a maximal dismissal rate, in the period [t4, t5] = [2.85, 6.20] before itswitches to maximal hiring in the remaining planning period. When the pricep(t) is decresaing, the loan capital control Yc(t) and the investment controlI(t) have significantly smaller values than those for Data-1.

16

Due to the complexity of the control model, the complicated control structurecan not be determined from a detailed discussion of the Maximum Princi-ple alone. We have presented a hybrid numerical approach consisting of twoconsecutive direct optimization methods which yield control and state vari-ables as well as junction times between bang-bang and singular arcs. Moreover,adjoint variables and multipliers associated with state constraints can be iden-tified with Lagrange multipliers of the optimization problems. This allows usto verify necessary optimality conditions a posteriori. Using this approach,optimal solutions can be computed for various other data scenarios in the mi-croeconomic control problem, e.g., for the risk premium rate ρlowr (t) for daringinvestors, etc. In all cases, the computed solution satisfies sufficient optimalityconditions, since the maximized Hamiltonian turns out to be a concave func-tion of the state variable.

Acknowledgement: We are grateful to Nadja Balzer [1] and Matthias Lang[9] for numerical assistance with solving the control problems in Section 4..

References

[1] Balzer, N., Optimale Steuerung okonmischer Prozesse am Beispiel eineskomplexen Unternehmensmodells mit bang–bang, singularen Steueerungenund Zustandsbeschrankungen, Diploma Thesis, Institut fur Numerische undAngewandte Mathematik, Universitat Munster, 2006.

[2] Buskens, C., Direkte Optimierungsmethoden und Sensitivitatsanalyse furoptimale Steuerprozesse mit Steuer– und Zustands–Beschrankungen.Dissertation, Institut fur Numerische und Angewandte Mathematik, UniversitatMunster, 1998.

[3] Buskens, C., and Maurer, H., SQP–methods for solving optimal controlproblems with control and state constraints: adjoint variables, sensitivityanalysis and real–time control, J. of Computational and Applied Mathematics120, 85–108 (2000).

[4] Buskens, C., and Maurer, H., Sensitivity analysis and real–time control ofparametric optimal control problems using nonlinear programming methods,in: Online Optimization of Large Scale Systems (Grotschel, M., Krumke, S.O.,and Rambau, J., eds.), pp. 57–68, Springer–Verlag, Berlin, 2001.

[5] Buskens, C., Pesch, H.J., and Winderl, S., Real–time solutions of bang–bangand singular optimal control problems, in: Online Optimization of Large ScaleSystems (Grotschel, M., Krumke, S.O., and Rambau, J., eds.), pp. 129–142,Springer–Verlag, Berlin, 2001.

[6] Fourer, R., Gay, D.M., and Kernighan, B.W., AMPL: A Modeling Language forMathematical Programming, Duxbury Press, Brooks/Cole Publishing Company,2002.

17

[7] Hartl, R.F., Sethi, S.P., and Vickson, R.G., A survey of the maximum principlesfor optimal control problems with state constraints, SIAM Review 37, 181–218(1995).

[8] Koslik, B., and Breitner, M.H., An optimal control problem in economics withfour linear controls, J. Optimization Theory and Applications 94, 619–634(1997).

[9] Lang, M., Schaltstrukturerkennung und Schaltpunkt–Optimierung beiOptimalsteuerungsproblemen mit linear eingehenden Steuerungen am Beispieleines Steuerungsproblems aus der Mikrookonomie, Diploma Thesis, Fakultatfur Mathematik und Physik, Universitat Bayreuth, 2006.

[10] Lesourne, J., and Leban, R., La substitution capital–travail au cours de lacroissance de l’entreprise, Revue d’Economie Politique 4, 540–564 (1978).

[11] Maurer, H., On optimal control problems with bounded state variables andcontrol appearing linearly, SIAM J. Control and Optimization 15, 345–362(1977).

[12] Maurer, H., On the minimum principle for optimal control problems with stateconstraints, Schriftenreihe des Rechenzentrums, Nr. 41, Universitat Munster,1979.

[13] Maurer, H., Buskens, C., Kim, J.-H.R., Kaya, Y.C., Optimization methods forthe verification of second-order sufficient conditions for bang-bang controls,Optimal Control Methods and Applications 26, 129–156 (2005).

[14] Maurer, H., Kim, J.-H.R., and Vossen, G., On a state–constrained controlproblem in optimal production and maintenance, in: Optimal Controland Dynamic Games, Applications in Finance, Management Science andEconomics, Deissenberg, C. and Hartl, R.F., eds., pp. 289–308, Springer Verlag,2005.

[15] Wachter, A., and Biegler, L.T., On the implementation of an interior–pointfilterline–search algorithm for large scale nonlinear programming, MathematicalProgramming 106, 25–57 (2006).

[16] Winderl, S, and Naumer, B., On a state constrained control problem ineconomics with four linear controls, Schwerpunktprogramm der DeutschenForschungsgemeinschaft “Echtzeitoptimierung Großer Systeme”, Report 00–9,2000.

18