an approximation algorithm for optimal consumption/investment problems

An ApproximationAlgorithm for OptimalConsumption/InvestmentProblemsSanjiv Ranjan Das1* and Rangarajan K. Sundaram2

1 Santa Clara University, USA2 New York University, USA

ABSTRACT This article develops a simple approach to solving continuous-time portfolio choiceproblems. Portfolio problems for which no closed-form solutions are available maybe handled by this technique, which substitutes the numerical solution of partialdifferential equations with a non-linear numerical algorithm approximating thesolution. This paper complements the wide literature in economics on the solutionof dynamic problems in discrete time using projection methods. Our approachextends the approximation function to power forms, which are shown to fit financetype problems well. The algorithm is parsimonious, and is first illustrated bysolving two basic examples, first, the standard Merton problem, and second, a jump-diffusion problem. Then, we demonstrate that the model is easy to implement on alarger scale, by optimizing a portfolio of six stock indexes, and stochastic volatilitydriven by two correlated state variables. Copyright 2002 John Wiley & Sons, Ltd.

INTRODUCTION

The problem of optimal consumption and port-folio choice is one with a long history. Originallyformulated in continuous time by Merton (1969,1971) the problem has been extended substan-tially and several solution approaches have beendeveloped. Barring the simplest problems, ana-lytical solutions are difficult to come by. Thispaper provides a simple numerical approach tosolving the optimal control problem using valuefunction approximation.

For simpler problems, as in the originalMerton formulation, closed-form solutions areachieved. The papers by Lehoczky et al. (1983),Karatzas et al. (1986), Jacka (1984) and Oconeand Karatzas (1991) deal with explicit solutions,

* Correspondence to: Sanjiv Ranjan Das, Leavey Schoolof Business, Santa Clara University, 500 El CaminoReal, Santa Clara, CA 95053, USA.E-mail: [email protected]

using the Bellman equation approach. The mar-tingale approach of Cox and Huang (1989) isalso a well-established one now, and has beenextended to incomplete markets by He and Pear-son (1991), Karatzas et al. (1991) and Cvitanic andKaratzas (1991). These problems are often furthercomplicated by the choice of non-additive utilityfunctions (see Duffie and Epstein, 1992 and, morerecently Dumas et al., 1997). Other complicationsarise when transactions costs are included in theanalysis, as in Constantinides (1986), Davis andNorman (1990) and Dumas and Luciano (1989).

In all these settings, simple versions admiteither closed-form solutions or problems that aresolved by applying simple numerical procedures.For example, in the case of the Bellman approach,if the number of state variables is low, thepartial differential equation of optimality maybe solved using finite-differencing methods. Ifthe objective function is simple, then easilyapplied recursive methods may be used, asin Bertsimas et al. (1997), where replication in

Copyright 2002 John Wiley & Sons, Ltd.

International Journal of Intelligent Systems in Accounting, Finance & Management

Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/isaf.213

Int. J. Intell. Sys. Acc. Fin. Mgmt. 11, 55–69 (2002)

incomplete markets is undertaken for a quadraticloss function, and the resulting system is quitetractable in low dimension. Bossaerts (1989)examines a similar problem in an Americanoption setting.

There are many approximation methods forthe solution of these problems (see the reviewarticle by Taylor and Uhlig, 1990). A wide vari-ety of numerical approaches is applied suchas iterating on the value function (Christiano,1990), quadrature methods (Tauchen, 1987, 1990),linear-quadratic approximations for the controls(Kydland and Prescott, 1982), and parameter-izations of the value function (Marcet, 1988;DeHaan and Marcet, 1990). In this paper, wedevelop an analog of the parameterizationapproach in continuous-time, and demonstrateits application to a fairly general problem incontinuous-time, that of a system driven by amixed jump-diffusion stochastic process. Ourmodel is a variant of known projection meth-ods in the literature (see Judd, 1998). Morerecent examples of approximation methods in theFinance literature include Campbell and Viceira(2001) Viceira, (2001), Brandt et al. (2000), andLongstaff (2000).

In continuous-time, solving the Bellman equa-tion is more art than science. The usual approachinvolves making a clever guess as to the formof the value function, obtaining the optimalcontrols, and then verifying the solution aftersolving the Bellman PDE subject to the guess.Solutions have been obtained for some well-known and familiar utility functions, but when-ever the number of state variables grows, orthe stochastic processes chosen are not of thecommon geometric Brownian motion form, weare usually reliant on numerical schemes. Thispaper develops value function approximation asa method of extending the Bellman approach in atractable way.

The basic idea is as follows. The optimalconsumption–investment problem is set up asa Bellman control problem in the usual way.The first-order conditions provide the functionalequations for the optimal controls, subject tosolving for the value function. Rather thanattempt to solve for the value function in closedform, we posit a very general polynomial formfor the value function, extending the linear

series forms that are prevalent in most modelsin computational economics. Thus the valuefunction is described as a general function of afinite parameter set, denoted θ . Substituting thisfunctional guess into the first-order conditionsgives us the optimal controls as a function ofθ . These optimal controls are then plugged backinto the Bellman equation which should holdfor all possible outcomes of the state variables.This will only be true when the guess for thevalue function coincides with the true valuefunction, and for complex problems, this isunlikely. However, if the approximation to thetrue value function is a good one, then thedistance between the approximate value functionand the exact one should be small over allpoints in the state space. Thus, our solutioncomprises of minimizing a ‘distance function’between exact and approximate value functions,by means of finding the best-fit parameterset θ , subject to exactly satisfying the first-order conditions.

The algorithm works well, and we provideexamples of its implementation in the paper. Thisapproach has several benefits. First, it may beused to handle higher-dimensional problems, aswell as problems with complex utility functionsand stochastic processes. As an example, we solvea jump-diffusion model in the paper. We furtherextend the problem to one of higher-dimension,with solutions over a system of six assets andstochastic volatility driven by two state variables.Second, any standard minimizer routine may beapplied for computational purposes making theproblem computationally inexpensive. In fact,the illustrations in the paper use nothing moresophisticated than the optimizer in the Excelspreadsheet. All our implementations ran inunder a minute, and were robust to differentstarting parameters. Third, general polynomialfunctions may be used to guess the valuefunction. In other methods, such as weightedresiduals methods, the approximation function isnot nonlinear in the parameters, whereas ourapproach freely permits this. Fourth, as longas analytical derivatives of the approximatedvalue function are obtainable, we are able toattain a rapid implementation. In other methods,such as finite-differencing to solve Bellman PDEs,numerical derivatives are taken, which impacts

Copyright 2002 John Wiley & Sons, Ltd. Int. J. Intell. Sys. Acc. Fin. Mgmt. 11, 55–69 (2002)

56 S. R. DAS AND R. K. SUNDARAM

the speed and convergence of the algorithmsused. Finally, examination of the numericalsolution provides hints as to the form of thetrue value function, which may lead to an explicitanalytical solution.

In the following sections, we provide theproblem set up, and the formal presentation of thesolution method. We present our approach withinthe didactic framework of projection methods.Numerical examples are also provided withappropriate discussion.

STOCHASTIC PROCESSES

Investors face a state space that is characterizedby an infinite trading interval T = [0, ∞). Theuncertainty in the portfolio choice set emanatesfrom a set of diffusion processes and Poissonjump processes, with probability spaces (�Z,FZ, QZ) and (�N, FN, QN) respectively. Zt ∈ Rn

represents a vector of Wiener processes definedon (�Z, FZ, QZ) and Nt ∈ Rm represents a vectorof orthogonal jump processes defined on (�N, FN,QN), where t ∈ T and m, n > 0. Each jump processis described by a sequence of random timesT(i) ∈ T such that Nt,i = ∑

1{t≥T(i)}. The Poissonjump arrival intensities are denoted λi, i = 1 . . . m.We allow for the jump intensities to varystochastically on T as well as functions of{Zt, Nt}.

We also allow for K state variables xt ∈ RK

which also evolve on the same jump-diffusionprobability spaces defined above. These processesare defined as follows:

dx(t) = αx(x, t)dt + σx(x, t)dZx(t)

+ Jx(x, t)dN(λ, t) (1)

where αx(x, t) ∈ RK, σx(x, t) ∈ RK×n, Jx(x, t) ∈ RK×m.The economy offers a set of L + 1 traded

assets, whose values evolve stochastically on T.These assets comprise L risky assets (S) indexedl = 1 . . . L, and a riskless asset (B) which earns aconstant return r. Hence the stochastic process forthis asset is

dB(t) = rB(t)dt (2)

The remaining assets earn a random rate of returnand obey the following SDE:

dS(t)S(t)

= α(S, x, t)dt + σ(S, x, t)dZ

+ J(S, x, t)dN(λ, t) (3)

where dS(t)/S(t), α(S, t) ∈ RL, σ (S, t) ∈ RL×n andJ(S, t) ∈ RL×m.

OPTIMAL PORTFOLIO CHOICE

The investor seeks to implement a consumption(ct) and portfolio plan (wt ∈ RL) so as to maximizehis lifetime utility. The utility of consumption isgiven by the usual class of Von-Neumann andMorgenstern utility functions, which we denoteU(ct), satisfying the usual requirements of con-cavity, and other technical regularity conditions.

The portfolio plan of the investor is a choice ofasset weights wl, l = 1 . . . L such that the amountinvested in the riskless asset is w0 = 1 − w′1. Atany time t, the investor chooses how much of hiscurrent wealth Wt to consume, and invests thebalance in the riskless and risky assets. Thus, thestochastic process for wealth taking into accountinvestment and consumption is as follows (seeMerton, 1992 for details):

dW = {W [w′(α − r1) + r] − c}dt + Ww′σdZ

+ Ww′J dN (4)

where 1 is a unit vector. At the initial time t = 0,we are interested in solving for the investor’soptimal consumption and investment program,so as to undertake the following maximization:

max{c,w}

E0

{∫ ∞

0

e−ρsU(cs)ds}

(5)

where ρ is the investor’s time preference param-eter, a scalar constant. The optimized functionat any time t is denoted as the value function,defined recursively as a function of state variables{W, x}

V(W, x; t) = maxct,wt

Et{U(ct) + V(W, x; t + dt)} (6)


APPROXIMATION ALGORITHM FOR CONSUMPTION/INVESTMENT PROBLEMS 57

Using the method of stochastic dynamic pro-gramming (see Merton, 1992), we arrive at theBellman equation of optimality:

0 = max{c,w}

H(W, x) (7)

which in full detailed form is:

0 = max{c,w}

{U(c) − ρV + VWW [w′(α − r1) + r]

− cVW + 12

VWWW2w′σσ ′w

+m∑

j=1

E(λj[V(W + w′JjW, x) − V(W, x)])

+ V ′xαx + 1

2E[(σxdZx)

′Vxx(σxdZx)]

+ WV ′WxE[(σxdZx)(σdZ)′w]

+m∑

j=1

E(λj[V(W, x + Jx) − V(W, x)])

}(8)

where Jj ∈ RL is the jth column ofmatrix J ∈ RL×m. Subscripts denote partialderivatives, i.e. Vw = ∂V/∂W, VWW = ∂2V/∂W2.Likewise, Vx = ∂V/∂x ∈ RK, Vxx = ∂2V/∂x∂x′ ∈RK×K and VWx = ∂V/∂W∂x′ ∈ RK. The solutionmethod entails taking the first-order derivativesfrom the equation of optimality to arrive at theoptimal controls {c, w}, as functions of V(W,x). Then, the substitution of these values intothe Bellman equation provides a second-order(K + 1)-dimensional partial differential equationin (W, x) which must be solved subject to suitablyimposed boundary conditions.

The first-order condition for consumption,

U′(c) = VW (9)

implies the optimal consumption rule: c∗ =I(∂V/∂W), I = [U′]−1, U′ = ∂U/∂c. Taking thefirst-order condition for optimal portfolio weightsw∗, we get the (L × 1) equation system

0 = VW(α − r1)W + VWWσσ ′wW2

+m∑

j=1

E(

λj

[∂

∂wV(W

+ w′JjW) − V(W)1

])

+ W{V ′WxE[(σxdZx)(σdZ)′]}′ (10)

where (∂/∂w)V(W + w′JjW) ∈ RL and 1 is aunit vector.

THE APPROXIMATION ALGORITHM

Exact solution of the problem in the previoussection is usually hard to achieve, except in thesimplest of cases. The problem lies in the fact thatthe optimal controls are complicated functions ofthe state variables, and value function, whichitself is the solution to a high-dimensionaldifferential equation. The usual approach is toguess a functional form for the value functionand then verify whether it satisfies the optimalityconditions of the problem. Apart from a fewwell-known cases, this approach has provenrather fruitless. Alternatively, one could attemptto solve the differential equation using numericalmethods such as finite-differencing, but achievinga stable numerical scheme with many statevariables has proven to be a daunting task.

Here, we suggest an alternative approachwhich bypasses these problems. The idea isto posit a polynomial function of the statevariables as an approximation to the valuefunction. We choose a θ -parameterized functionV(θ) ≡ V(W, x; θ), where θ ∈ RP+1 is a set ofparameters {v0, v1 . . . vP} which define the valuefunction. If we are able to find the ‘best’ possiblevalue function V∗(θ), then we have automaticallyobtained the solution to the problem, since thecontrols {c, w} derive immediately. The exactsolution will satisfy

0 = H(W, x; V∗) (11)

subject to satisfying the constraints from the first-order conditions, in equations (9) and (10). If weare not able to find the optimal value function,we can find the best V in a set of value functions{V(θ)} which may be chosen arbitrarily. To do thiswe compute the following optimization program:

min{V(θ)}

{min

θ

∑u∈U

f(H[W(u), x(u); V(W, x; θ)])

}(12)



subject to

w = w∗[V(W(u), x(u); θ)], ∀u

c = c∗[V(W(u), x(u); θ)], ∀u

Here, U represents a discrete set of choices ofstate variables, i.e. a chosen state-space for theproblem. These values u = {W(u), x(u)} may bechosen to reflect the decision-makers’ envisagedoutcomes of the state variables. The functionf (.) is the fitting function and may be chosenfrom a range of popular options. For example,a least-squares approach would set f (H) = H2.Alternatively, a probability weighted functionsuch as f (H) = H2 × prob(u) may be used. Asimple absolute valued function f (H) = |H| isalso possible. Optimization is undertaken bychoosing a specific V(θ) and then optimizing.Searching over the set {V(θ)} will produce thebest value function.

This approach has certain advantages. First,it does not require the solution of a high-dimensional differential equation. Second, thecomplexity of form of the value functional V(θ)

does not impact substantially the computationalrequirements of the algorithm. The number ofpoints in the state space U does, however, increasethe number of constraints to be satisfied in alinear way. But this was not found to be numer-ically difficult, and in fact, implementation witha spreadsheet optimizer works very well. In boththe simpler examples in the following section,and the more complex one in the ensuing sectionthereafter, we were able to solve the models ona simple Excel spreadsheet. The simple modelsbarely took a few seconds to solve. The morecomplex model, involving search over 10 param-eters in the value function, took about a minuteand roughly 50 iterations of the Newton method.Since the Excel optimizer is likely to be inferiorto those found in formal optimization software,alternative implementations using superior soft-ware will speed up the time to generate the resultseven more.1

1 We fully expected to require a formal optimizationpackage for the more complex problem we attempted.However, we decided to attempt it first in Excel, andas it turns out, this resulted in surprisingly effectiveimplementation. This suggests that the structure of the

Context and Didactic Representation of theAlgorithm

The class of methods in which our algorithmfalls is denoted as ‘projection methods’ (seeFletcher, 1984; Gaspar and Judd, 1997; Judd, 1992,1998). These methods offer a viable alternative tofinite-differencing techniques. Projection meth-ods are closely related to regression techniques,and our model, using a least-squares criterionis intuitively resident in this set of algorithms.While finite-differencing models seek to numer-ically determine the exact function by buildingup from the numerical derivatives, projectionmethods posit a parameterized approximationto the true function, substitute this into theproblem, and then search over the parameterspace for the best fit solution function. Finite-differencing solutions promise more accuracy,since they do not approximate the value function,but pay a price in convergence problems andcomputational complexity. In contrast, the costof functional approximation buys the projectionapproach much more flexibility as well as bettercomputational complexity.

We provide a brief, yet general and simpleexposition of a projection method. We seek tosolve for a function (denoted f (x)), embeddedin an equation g[x, f (x), θ ] = 0, where x proxiesfor a set of state variables, and θ is a set ofparameters in the function g[.]. The function g[.]may arise from an optimization problem. Forexample, in the framework of this paper, g[.]is the Bellman equation. In another setting itmay be a differential equation on the functionf (x) emanating from a physical or economicproblem. In this sense, power series solutions todifferential equations may also be interpreted asa special form of projection methods. Intuitively,notice that

g[x, f(x), θ ] = 0, ∀x (13)

may be interpreted as a vector of moment con-ditions. The exact function f (x) may be replacedby an approximating function f ′(x; ξ), where ξ is

problem and our implementation set up does enhancethe ease of implementing the algorithm. Hence, we didnot even require the use of more sophisticated softwarefor the solution.



a set of parameters for the approximating func-tion. Then the approach is to solve the followingproblem akin to the method of moments:

E(g[x, f ′(x; ξ), θ ].h(x)) = 0, ∀x (14)

where h(x) is a set of weighting functions. Hence,projection methods may be classified by (a) thechoice of polynomial for the approximation func-tion f ′(x; ξ), and (b) the choice of weights used.

The usual form of the polynomial is

f ′(x; ξ) = 1 +n∑

i=1

ξixi (15)

This is the ‘linear’ approximation form wherethe parameters apply linearly to the powers ofthe state variables. In this paper, we extend thespecification to a more general form where theparameters enter as power coefficients of the statevariables as well. Hence, for example, we specify:

f ′(x; ξ, δ) = δ0 +n∑

i=1

δixξi (16)

The reason for this extension is simple. Wenoticed that the solutions of the Bellman equationin the few cases in which closed-form solutionswere attained (for example, Merton, 1968, 1971)do possess such a function form for the valuefunction. This form derives from the originalpower form of the direct utility function. Hence,we decided to adopt this as a modification to theclassic polynomials used in projection methods.As will be shown, the approximation is well-founded. Other approximations may also be used,for example, the Galerkin method, which useshigher powers of x.

The other choice that is made in projectionmethods is the moment weighting function asapplied to the ‘residuals’. The classic choiceis to set h(.) = g(.), i.e. use a least-squaresobjective function. We adopt this criterion in ourimplementation of the algorithm. Many otherweighting methods may be chosen as well. Theobjective function, therefore, is usually a sumor integral of the residuals from the momentfunctions. Sometimes, parameters may be chosento force the residuals to zero at a finite number of

points on the state space grid, and this approachis called the collocation method.

Besides providing a new functional form forthe value function, other points of difference areworth mentioning. In our setting, we are able tosolve for the value function on a spreadsheet,even in complex settings as is shown in thefinal example in the paper. The problems arevery nonlinear, yet yield a solution in a fewseconds of computing time. And, just as in thefinite-differencing model, we are able to providethe values of the controls, as well as the valuefunction and all its derivatives at each point onthe state-space grid. This allows generation ofthe plots for the solution in simple and easyway as will be seen in the figures providedfor the last example in the paper. Even with avalue function with as many as 10 parameters,the spreadsheet optimizer is able to convergeeasily in under a minute using the Newton’smethod with forward derivatives. Therefore, wedid not face any severe root-finding problemsin the implementation of the models. As will beseen from the plots, the value function is verysmooth, and hence the properties of the problemitself would work in favor of this method. Thefollowing section provides illustrations of themodel implementation.

ILLUSTRATIVE EXAMPLES

A Simple Implementation Example

Consider the following simple problem. We beginwith a single asset setting, where the asset followsa geometric Brownian motion:

dS = αS dt + σS dZ (17)

The notation follows from the previous section.Assume a power utility function over consump-tion where U(c) = 1

η cη. Analogous to equation (8)the Bellman equation of optimality will be:

0 = maxc,w

{U(c) − ρV + VWW [wR + r]

−VWc + 12

VWWw2W2σ 2

}(18)



where R = α − r defines the equity pre-mium/excess return. The first-order condition forconsumption gives U′(c) = VW which implies thatthe optimal consumption is c∗ = (VW)1/(η−1). Like-wise, the first-order condition for the portfolioweights gives the optimal investment in the riskyasset via the equation VWR + VWWwWσ 2 = 0,implying that

w∗ = − Rσ 2

VW

WVWW

As is well known from Merton, 1992, the solutionto this problem provides a value function ofthe form

V(W) = AWη

η

which implies that the optimal weights are

w∗ = Rσ 2

1(1 − η)

Now suppose we did not know the valuefunction form in advance, and made a guessas to its form by choosing a somewhat moregeneral function. Let V(W) = V0 + V1WV2 where(V0,V1,V2) are unknown scalar constants.2 Ourapproach then entails substituting this positedvalue function into the Bellman equation andsolving for the best fit values of (V0,V1,V2) overthe state space, which comprises a range of valuesof W.

Let the vector of N values of W be indexedby i, such that we have Wi, i = 1 . . . N. This thendefines a vector of values of the Bellman equation.Denote the optimized Bellman equation vector asMi, i = 1 . . . N. Thus, we have

Mi = U(c∗i ) − ρV + VWWi[w∗

i R + r] − VWc∗i

+ 12

VWWw∗2i W2

i σ2, ∀Wi (19)

Note here that the equation contains the opti-mized values (c∗

i , w∗i ) from the first-order condi-

tions. Also, the derivatives VW and VWW are func-tions of Wi but the notation in the form VW(Wi)

has been suppressed for expositional reasons.

2 This approximation varies from other approachesusing simple polynomial forms, such as in the Galerkinmethod (see Judd, 1998, p. 373).

Assume any unconditional distribution forWi. Denote the probability of Wi as f (Wi), and∑N

i=1 f (Wi) = 1. For the optimal value function,it must be that Mi = 0, ∀i. However, since weare guessing the value function, and may notdetect its exact form, the best that is possible isto choose (V0,V1,V2) so as to make the valuesof |Mi|, ∀i as small as possible. This suggests arange of objective functions of which an exampleis provided below:

min{V0,V1,V2}

N∑i=1

{M2i f(Wi)} (20)

Under the assumption of equally weightedoccurrences of Wi, this is simply the leastsquares method.

We now explore the solution in more detail.First, we can compute the analytical derivativesof the value function. We get

VW = V1V2WV2−1i

VWW = V1V2(V2 − 1)WV2−2i

c∗i = [V1V2W

V2−1i ]

1η−1

w∗i = R

σ 2

1(1 − V2)

The set of Bellman equations becomes (for everyi)

Mi = 1η

[V1V2WV2−1i ]

η

η−1 − ρ[V0 + V1WV2 ]

+ V1V2WV2−1i

[Rσ 2

1(1 − V2)

+ r]

− V1V2WV2−1i [V1V2W

V2−1i ]

1η−1

+ 12

V1V2(V2 − 1)WV2−2i

[Rσ 2

1(1 − V2)

]2

× W2i σ

2, ∀i

The minimization problem may be simply statedas min{V0,V1,V2}

∑Ni=1 M2

i . Since we impose V(W =0) = 0, we get that V0 = 0. Further, it may easily bechecked that the numerical minimization in factleads to the solution V1 > 0, V2 = η. This matchesexactly the solutions in Merton (1971). Hence,the technique provides the known solution. We



now proceed to a numerical illustration of a morecomplex model.

In this method, we need to specify the state-space grid over which the value function iscomputed. The first choice to be made is the rangeof the state variable. This is easy to determinein the case of consumption/investment prob-lems, as we start our investor off with initialwealth W0 = 1. Given this, the terminal wealthmay be chosen to reside over a range from0 to 2, where the upper support refers to areturn of 100% per period, which is substan-tial. Once the overall support was chosen, wedid not find the model solution to be too sen-sitive to the number of grid points used. Thereason for this is that unlike other approaches tosolve the PDEs emanating from Bellman equa-tions, we do not need to compute derivativesnumerically in our model, as we can computethem analytically from the approximated valuefunction. For instance, in the finite-differencemethod to compute a grid over the value func-tion, derivatives are taken numerically and henceare very sensitive to mesh size on the grid. Inour case, we do not encounter this problem.Hence, mesh size is not an issue. This is an oftenoverlooked benefit of the approach we develophere.

An Example with Jump Processes

We extend the process driving the risky asset toincluding jumps with stochastic intensity. Thus,

dS = αS dt + σS dZ + JS dN(λ)

dλ = k(θ − λ) dt + δ√

λ dY

Thus, the jump intensity λ follows a mean-reverting square-root diffusion, and we assumethat dZ, dY are orthogonal diffusions. The valuefunction is now extended to cover the new statevariable λ in addition to wealth, so that wewrite it as V(W, λ). From (8) we get the Bellmanequation:

0 = maxc,w

{U(c) − ρV(W, λ) + VWW [wR + r]

−VWc + 12

VWWw2W2σ 2

+Vλk(θ − λ) + 12

Vλλδ2λ

+λE[V(W + wWJ, λ) − V(W, λ)]}

We guess the following functional form for thevalue function:

V(W, λ) = V0 + V1WV2 + V3λV4 + V5Wλ (21)

which yields the following terms:

VW = V1V2WV2−1 + V5λ

VWW = V1V2(V2 − 1)WV2−2

Vλ = V3V4λV4−1 + V5W

Vλλ = V3V4(V4 − 1)λV4−2

From the first-order condition for consumption,we obtain the optimal value of c:

c∗ = [V1V2WV2−1 + V5λ]1

η−1 (22)

The first-order condition for the risky assetweights is

VWWR + VWWwσ 2W2

+ λE[

∂

∂wV[W + wWJ]

]= 0 (23)

Thus, w∗ is implicitly defined as the solution tothe above equation. The last term requires

V[W + wWJ] = V0 + V1WV2(1 + wJ)V2

+ V3λV4 + V5W(1 + wJ)λ (24)

which provides

∂

∂wV[W + wWJ] = V1V2WV2(1 + wJ)V2−1J

+ V5WλJ (25)

For the purposes of this example we assume abinary form for the jump, i.e.

J ={+j, w/prob 1

2−j, w/prob 12

(26)



which leads to

E[V(W, λ)]

= V0 + V1WV2 + V3λV4 + V5Wλ

E{V[W + wWJ, λ]}

= V0 + V1WV2 × 12

[(1 + wj)V2 + (1 − wj)V2 ]

+ V3λV4 + V5Wλ

E{

∂

∂wV[W + wWJ, λ]

}

= 12

[V1V2WV2(1 + wj)V2−1j

− V1V2WV2(1 − wj)V2−1j]

The first-order condition for portfolio weightsmay now be written as

0 = [V1V2WV2−1 + V5λ]WR

+ [V1V2(V2 − 1)WV2−2i ]wσ 2W2

+ λ12

[V1V2WV2(1 + wj)V2−1j

− V1V2WV2(1 − wj)V2−1j] (27)

The vector of Bellman equations is now written as(i now indexes the joint space over state variablesW, λ):

Mi = U(c∗i ) − ρV(Wi, λi)

+ (V1V2WV2−1i + V5λ)Wi[w∗

i R + r]

− (V1V2WV2−1i + V5λ)c∗

i

+ 12

(V1V2(V2 − 1)WV2−2i )w∗2

i W2i σ

2

+ (V3V4λV4−1i + V5Wi)k(θ − λi)

+ 12

(V3V4(V4 − 1)λV4−2i )δ2λi

+ λi

{V0 + V1W

V2i × 1

2[(1 + w∗

i j)V2

+(1 − w∗i j)

V2 ] + V3λV4i + V5Wiλi

}− λi{V0 + V1W

V2i + V3λ

V4i + V5Wiλi}

Since the first-order condition for portfolioweights w is an implicit equation, we solve the fol-lowing optimization problem to obtain the values

(V0,V1,V2,V3,V4,V5):

min{V0,V1,V2,V3,V4,V5}

N∑i=1

M2i

subject to equations (22) and (27).This problem is solved numerically, and the

results are provided below. Additionally, sincethe jump-diffusion based problem nests the pure-diffusion model, we can examine the features ofthe first solution given above as well.

Numerical Results

The parameters in Table 1 were chosen as a basecase for the jump-diffusion model.

In order to solve the problem we need to choosea grid of points (Wi, λi), ∀i. We used a range ofvalues of W ∈ [0, 10] and for jump intensity weassumed a two-state model where λ ∈ {5, 10},i.e. low and high jump states. The algorithmwas implemented on an Excel spreadsheet, andconverges in a few seconds. The optimal values(V0,V1,V2,V3,V4,V5) are shown in Table 2.

The signs of the parameters are exactly asexpected. Note that V1, V2 are greater than zero,since indirect utility is increasing in the level ofwealth. Likewise V3, V4, V5 are less than zero,

Table 1

Parameter description Notation Value

Relative risk aversion η 0.5Mean return on risky asset α 0.07Riskless rate r 0.03Subjective discount rate ρ 0.03Volatility coefficient for risky asset σ 0.3Mean reversion for jump intensity (λ) k 0.5Mean level of λ θ 7.5Volatility coefficient of λ δ 5Jump amplitude j 0.1

Table 2. Optimal value function parameters (jump-diffusion model)

V0 V1 V2 V3 V4 V5

0 14.1290 0.4995 −1.0231 −0.0198 −0.0021



Table 3. Optimal value function parameters (pure-diffusion model)

V0 V1 V2 V3 V4 V5

0 18.0889 0.5000 0 0 0

since utility declines when jump risk increases.For the purposes of comparison, we switchedoff the jump process to reduce the problem tothe pure-diffusion model. To do so, we set theparameters as follows: (k, θ , δ, j, λ) to zero. In thissetting, the value function is as shown in Table 3.

This solution corresponds exactly to that of theknown solution in Merton (1992). Notice that thevalue of V2 = η as required in theory. In Table 4we present some of the qualitative results fromthe two models, and undertake a comparisonof outcomes.

The table presents results from the pure-diffusion model, and the jump-diffusion model.For varying levels of the state variables W andλ, we examine three values of interest: optimalconsumption, investment in the risky asset, andthe value function.

First, we note that as the level of wealthincreases, so does the value function. Second,as jump intensity increases, investor utilitydecreases since additional risk is borne. The onlyexception occurs when wealth is at a very lowlevel and the jump intensity increases from 5 to 10.This may be on account of the fact that at low lev-els of wealth, additional jumps cannot harm theinvestor given a floor level of zero on wealth. Plus,at high levels of jump intensity, mean reversionwill lower jump risk. Third, as wealth increases,the investor consumes more. Fourth, as jump risk

increases the investor also consumes more, sinceinvesting becomes less attractive, and consump-tion from the future is shifted to the present.Fifth, as jump risk increases, the investor corre-spondingly invests less in the risky asset. Sixth,when there is no jump risk, the amount investedin the risky asset is independent of wealth, as isknown from the Merton model. However, whenjump risk exists, the hedging term in equation (27)comes into play, and the choice of risky assets isno longer independent of wealth level. Finally,jump risk has a greater effect on the investor deci-sion when jump risk is small and increasing thanwhen it is large and increasing.

Computationally, the following results apply.First, the model converges to the same solution formany different starting values that we tried. Wedid not specially select any initial values. Second,the time taken for this optimization was alwaysless than 10 seconds. Hence, the spreadsheet opti-mizer does not face any root-finding problems.Third, we examined the residuals from the Bell-man equation. We did this both, in-sample andout-of-sample with reference to the state-spacegrid. While we ran wealth from 0–10 on the in-sample grid, we then tried it out-of-sample on the10–12 region. The residual statistics are shown inTable 5.

As can be seen, the residuals are extremelysmall, and attest to an accurate solution. As is to

Table 5. Residual statistics

Sample Mean Std. deviation

In-sample −1.58 × 10−7 1.7 × 10−3

Out-of-sample 2.4 × 10−3 1.9 × 10−3

Table 4. Optimal consumption and investment values

W Pure-diffusion model Jump-diffusion model

λ = 0 λ = 5 λ = 10

c∗ w∗ V(W, λ) c∗ w∗ V(W, λ) c∗ w∗ V(W, λ)

0.1 0.0012 0.8888 5.72 0.0020 0.5702 3.24 0.0020 0.4200 3.250.9 0.0110 0.8888 17.16 0.0181 0.5697 12.16 0.0182 0.4192 12.172.5 0.0306 0.8888 28.60 0.0505 0.5691 21.07 0.0507 0.4184 21.064.5 0.0550 0.8888 38.37 0.0910 0.5687 28.67 0.0916 0.4177 28.64

10.1 0.1235 0.8888 57.48 0.2052 0.5678 43.52 0.2071 0.4164 43.42



be expected the in-sample residuals are smallerthan those out-of-sample, but in all cases theresiduals are not large. We also re-optimized themodel with the additional out-of-sample pointsin the state space to check if the parameters of thevalue function changes drastically and found thechanges to be small. The statistics for the residualsafter this optimization are still very small in meanand standard deviation.

HIGHER-DIMENSIONAL PROBLEMS

In the previous sections, we considered a problemwhere there was only one risky asset and a singlestate variable driven by a jump process. Theapproach we presented in the previous sectionis easily extendable to more securities and morestate variables.

In order to demonstrate this, we extendedthe choice set of risky assets to six stocknational indexes. The countries we consideredare: United States, United Kingdom, Japan, Ger-many, Switzerland and France. We extractedindex date for the period January 1982 to Febru-ary 1997. From this data we computed the meanvector of returns (α), and the covariance matrixof returns (�). We denoted this as the ‘base’covariance matrix. Uncertainty over the indexesis driven by a vector of diffusions (dzs).

To make the problem complex as well as real-istic, we allowed the covariance matrix to varyfrom period to period, driven by two positivestate variables (x = {x1, x2}), which multiplica-tively impact the covariance matrix in a linearway. Thus the actual covariance matrix wasset to be equal to 6(x) = x1x26. The two statevariables are assumed to obey the followingstochastic processes:

dx = αxdt + σxdzx (28)

where αx ∈ R2 and dzx ∈ R2. Also note thatdzxdz′

x = Zdt, and dzs, dzx are assumed to beorthogonal. The two factors driving volatility arean open choice for the purposes of calibration.Any two state variables could be utilized.

Let the value function be an extension of theprior simpler form, i.e. V(W, x1, x2). If we denotedy = σxdzx ∈ R2, then we may write the Bellman

equation over six assets and two stochasticvolatility state variables as follows (same asbefore with additional terms):

0 = maxw,c

: U(c) − ρV

+ VWW [w′(α − r1) + r] − cVW

+ 12

VWWW2w′6(x)w

+ Vx′αx + 1

2E[dy′Vxxdy] (29)

where Vx is the gradient matrix of the valuefunction with respect to the state variables, andVxx is the Hessian matrix.

This is a much more complex problem thanthose presented previously, since the implemen-tation requires manipulation of fairly sizeablematrices. However, it is still tractable on a spread-sheet. And, of course, the value function approx-imation is also extended to a 10-parameter formas follows:

V(W, x1, x2) = V0 + V1WV2 + V3xV41 + V5x

V62

+ V7Wx1 + V8Wx2 + V9x1x2 (30)

The approach taken is the same as before. We takethe necessary derivatives of the value function,and substitute them into the Bellman function,and its first-order conditions, computed over agrid containing all three state variables. Unlike inthe past, when computation on the spreadsheetonly required a few seconds, the size of thisproblem required about 50 iterations in Excel,and took up to a minute in computationaltime. However, this is still a computationallyeconomical procedure.3

3 We used the optimizer in Microsoft excel over a rangeof values of W from 0 to 2.0, values of x1, x2 were taken tobe 0.5, 1.0 and 1.5. The settings of the optimizer were asfollows: convergence criterion was 0.0001, the precisionsetting was at 0.000001, the maximum number ofiterations set to 500 (never reached), tolerance at 5%.The estimates were based on the tangent method, withforward derivatives. Search was implemented usingNewton’s method. Finally, the results and time did notvary much at all despite a wide range of choices ofstarting parameters, even naive choices such as settingthe initial values to all be 0.1 or 0.5, for example.



Numerical Results for Higher-dimensionalProblem

In this section we present the results for theextended problem outlined above. The meansfor the six country stock index (monthly) returnswere as follows:

USA UK JP GE SW FR0.0102 0.0084 0.0080 0.0120 0.0102 0.0113

The ‘base’ covariance matrix of monthly returnsis shown in Table 6.

The state variables for stochastic volatility areassumed to evolve as follows:

(dx1

dx2

)=

(α1

α2

)+

(σ11 σ12

σ21 σ22

) (dz1

dz2

)(31)

Table 6

USA UK JP GE SW FR

0.0018 0.0014 0.0010 0.0010 0.0012 0.00130.0014 0.0032 0.0018 0.0016 0.0016 0.00190.0010 0.0018 0.0048 0.0015 0.0016 0.00200.0010 0.0016 0.0015 0.0033 0.0020 0.00230.0012 0.0016 0.0016 0.0020 0.0026 0.00190.0013 0.0019 0.0020 0.0023 0.0019 0.0037

and dz1dz2 = Corr(dz1, dz2)dt. The remaining inputparameters were as follows:

Parameters η ρ r α1 α2

0.5 0.005 0.005 0.2 0.2

Parameters σ11 σ12 σ21 σ22 Corr (dz1, dz2)

0.1 0.1 0.1 0.1 0.5

0 0.5 1 1.5 2 2.515

20

25

30

35

40

45

50

55

60

x1, x2 = 0.25

x1, x2 = 0.50

x1, x2 = 0.75

x1, x2 = 1.00

x1, x2 = 1.50

x1, x2 = 2.25

Terminal Wealth Level

Indi

rect

Util

ity

Value Function : V(W, × 1, × 2)

Figure 1 The value function. The value function V (W, x1, x2) is presented for values of W, and for values of theproduct (x1 × x2), which is the term that enters the stochastic volatility process. Our graph is two-dimensional, withterminal wealth W on the x-axis, and the value function on the y-axis



We ran the algorithm on this problem andobtained the ten parameters of the value function.These are as follows:

V0 V1 V2 V3 V4

10.1137 3.4771 0.4281 23.1806 0.0538

V5 V6 V7 V8 V9

23.1817 0.0538 0.6441 0.6441 −17.1148

The results of this model are presented usingplots, as this will make for clearer exposition.To start with, we examine the resulting valuefunction from the model. The value functionV(W,x1,x2) is hard to plot in four dimensions.Hence, we decided to present it for varyingvalues of W, and for values of the product(x1 × x2), which is the term that enters the

stochastic volatility process. Our graph is two-dimensional, with terminal wealth W on thex-axis, and the value function on the y-axis.Figure 1 presents the plots of the results. We plotdifferent lines for varying values of the volatilityfactors. First, notice that the value function(i.e. indirect utility) is increasing in wealth asexpected. Second, the numerically computedindirect utility function derives its shape from theoriginal direct one, and is therefore concave andincreasing monotonically. Third, as the volatilityof the assets increases, indirect utility drops asone would expect. However, in the presence ofrisk-aversion, the growing volatility factor resultsin greater percentage reductions in utility.

We also present (in Figure 2) the plots depictingoptimal consumption as terminal wealth changesand for different levels of the volatility state

0 0.5 1 1.5 2 2.50

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

x1 = 0.5, x2 = 0.5

x1 = 0.5, x2 = 1.0

x1 = 0.5, x2 = 1.5

x1 = 1.0, x2 = 1.5

x1 = 1.5, x2 = 1.5

Terminal wealth

Amount of consumption (c)

Figure 2 The consumption function. The consumption function c(W,x1,x2) is presented for values of W, and forvalues of the state variables, (x1,x2), which are the terms that enter the stochastic volatility process. Our graph istwo-dimensional, with terminal wealth W on the x-axis, and the value function on the y-axis



variables. The relationship is as expected. Aswealth increases, consumption also increasesbut at a decreasing rate. As volatility increases,amount of consumption declines as the amountinvested shifts to provide a greater investmentbuffer in ensuing risky periods.

CONCLUDING COMMENTS

This paper develops a simple numerical approachto solving optimal consumption and investmentproblems when analytic solutions are not achiev-able. The Bellman problem is translated into aneconometric one, where we minimize a parame-terized objective function of the Bellman equationover the state space, based on a polynomial guessfor the value function. The method is tractable,and converges quickly. The approach offers ameans to numerically guessing the form of thevalue function which, if fortuitous, may lead toan analytic solution.

The algorithm may be extended to solvingproblems in other domains in finance, such asasset-liability management, optimal replication ofderivative securities, equilibrium problems withincomplete markets and market microstructuregames.

ACKNOWLEDGMENTS

Special thanks to Luis Viceira and Raman Uppalfor several suggestions. We are very grateful tothe referees on the paper, as well as the editorChristian Haefke for several useful commentswhich guided the paper into its improved form.

References

Bertsimas D, Kogan L, Lo A. 1997. Pricing and hedgingderivative securities in incomplete markets: An ∈-arbitrage approach. Working paper No. 6250, NBER.

Bossaerts P. 1989. Simulation estimators of optimalearly exercise. Working paper, Carnegie-MellonUniversity.

Brandt M, Goyal A, Santa-Clara P. 2000. Dynamicportfolio choice with simulation methods. Workingpaper, UCLA.

Campbell J, Viceira L. 1999. Consumption andportfolio decisions when expected returns are time

varying; The Quarterly Journal of Economics 114:433–495.

Campbell J, Viceira L. 2001. Who should buy long termbonds? The American Economic Review 91: 99–127.

Christiano LJ. 1990. Solving the stochastic growthmodel by linear-quadratic approximation and byvalue-function iteration. Journal of Business andEconomic Statistics 8(1): 23–26.

Constantinides G. 1986. Capital market equilibriumwith transactions costs. Journal of Political Economy94: 842–862.

Cox D, Huang C-F. 1989. Optimal consumption andportfolio policies when asset prices follow a diffusionprocess. Journal of Economic Theory 49: 33–83.

Cvitanic J, Karatzas I. 1991. Convex duality inconstrained portfolio optimization. Working paper,Columbia University.

Davis M, Norman A. 1990. Portfolio selection withtransaction costs. Mathematics of Operations Research15: 676–713.

Den Haan WJ, Marcet A. 1990. Solving the stochasticgrowth model by parameterizing expectations.Journal of Business and Economic Statistics 8(1): 31–34.

Duffie D, Epstein L. 1992. Stochastic differential utility.Econometrica 60: 353–394.

Dumas B, Luciano E. 1989. An exact solution to adynamic portfolio choice problem with transactionscosts. Working paper, Wharton School, Universityof Pennsylvania.

Dumas B, Uppal R, Wang T. 1997. Efficientintertemporal allocations with recursive utility.Working paper, HEC School of Management, andUniversity of British Columbia.

Fleming W, Rishel R. 1975. Deterministic and StochasticOptimal Control. Springer-Verlag: Berlin.

Fletcher CAJ. 1984. Computational Galerkin Techniques.Springer-Verlag: New York.

Gaspar J, Judd K. 1997. Solving large-scale rationalexpectations models. Macroeconomic Dynamics 1:45–75.

He H, Pearson N. 1991. Consumption and portfoliopolicies with incomplete markets: the infinite-dimensional case. Journal of Economic Theory 54:259–305.

He H, Pearson N. 1991. Consumption and portfoliopolicies with incomplete markets: the finite-dimensional case. Mathematical Finance 1: 1–10.

Jacka S. 1984. Optimal consumption of an investment.Stochastics 13: 45–60.

Judd K. 1992. Projection methods for solving aggregategrowth models. Journal of Economic Theory 58:410–452.

Judd K. 1998. Numerical Methods in Economics. MITPress: Cambridge, MA.

Karatzas I, Lehoczky J, Sethi S, Shreve S. 1986. Explicitsolution of a general consumption/investmentproblem. Mathematics of Operations Research 11:261–294.

Karatzas I, Lehoczky J, Shreve S, Xu G-L. 1991.Martingale and duality methods for utility



maximization in incomplete markets. SIAM Journalof Control and Optimization 29: 702–730.

Kydland F, Prescott E. 1982. Time to build andaggregate fluctuations. Econometrica 50: 1345–1370.

Lehoczky J, Sethi S, Shreve S. 1983. Optimal consump-tion and investment policies allowing consumptionconstraints and bankruptcy. Mathematics of Opera-tions Research 8: 613–636.

Longstaff F. 2000. Optimal portfolio choice and thevaluation of illiquid securities. Review of FinancialStudies (forthcoming).

Marcet A. 1988. Solving non-linear models byparameterizing expectations. Working paper,Graduate School of Industrial Administration,Carnegie-Mellon University.

Merton RC. 1969. Lifetime portfolio selection underuncertainty: the continuous-time case. Review ofEconomics and Statistics 51: 247–257.

Merton RC. 1971. Optimum consumption and portfoliorules in a continuous time model. Journal of EconomicTheory 3: 373–413.

Merton RC. 1992. Continuous-time Finance. BasilBlackwell: New York.

Ocone D, Karatzas I. 1991. A generalized clarkrepresentation formula, with application to optimalportfolios. Stochastics and Stochastics Reports 34:187–220.

Tauchen G. 1987. Quadrature-based methods forobtaining approximate solutions to non-linear assetpricing models. Working paper, Duke University,Dept of Economics.

Tauchen G. 1990. Solving the stochastic growth modelby using quadrature methods and value-functioniterations. Journal of Business and Economic Statistics8(1): 49–51.

Taylor JB, Uhlig H. 1990. Solving non-linear stochasticgrowth models: a comparison of alternative solutionmethods. Journal of Business and Economic Statistics8(1): 1–17.

Viceira L. 2001. Optimal Portfolio Choice for LongHorizon Investors with non tradable Labor income,The Journal of Finance 56: 433–470.



an approximation algorithm for optimal consumption/investment problems

Documents