Mathematical Social Sciences 71 (2014) 142–150

Contents lists available at ScienceDirect

Mathematical Social Sciences

journal homepage: www.elsevier.com/locate/econbase

Investment–consumption with regime-switching discount rates

Traian A. Pirvu a,∗, Huayue Zhang b

a Department of Mathematics & Statistics, McMaster University, 1280 Main Street West, Hamilton, ON, L8S 4K1, Canadab Department of Finance, Nankai University, 94 Weijin Road, Tianjin, 300071, China

h i g h l i g h t s

• We further clarify the pre-commitment strategies.• We further clarify the naive strategy.• We further clarify the numerical results.

a r t i c l e i n f o

Article history:Received 17 June 2013Received in revised form30 May 2014Accepted 1 July 2014Available online 14 July 2014

a b s t r a c t

This paper considers the problem of consumption and investment in a financial market within a continu-ous time stochastic economy. The investor exhibits a change in the discount rate. The investment oppor-tunities are a stock and a riskless account. Themarket coefficients and discount factor switch according toa finite state Markov chain. The change in the discount rate leads to time inconsistencies of the investor’sdecisions. The randomness in our model is driven by a Brownian motion and a Markov chain. Follow-ing Ekeland and Pirvu (2008) we introduce and characterize the subgame perfect strategies. Numericalexperiments show the effect of time preference on subgame perfect strategies and the pre-commitmentstrategies.

© 2014 Elsevier B.V. All rights reserved.

1. Introduction

Dynamic asset allocation in a stochastic paradigm received a lotof scrutiny lately. The pioneerworks areMerton (1969) andMerton(1971). Many works then followed, most of them assuming anexponential discount function. Ekeland and Pirvu (2008) has givenan overview of the literature in the context of Merton portfoliomanagement problem with exponential discounting.

In this paper, we consider a regime switching model for thefinancialmarket. Thismodelling is consistentwith some cyclicalityobserved in financial markets. Many papers consider these typesof markets for pricing derivative securities. Here we recall onlytwo such works, Guo (2001); Elliott et al. (2005). When it comesto optimal investment in regime switching markets we point toBauerle and Rieder (2004), Sotomayor and Cadenillas (2009)and

This work is supported by NSERC grants 371653-09, 88051 and MITACS grants5-26761, 30354 and the Natural Science Foundation of China (10901086). Theauthors would like to thank anonymous referees for numerous helpful commentsand a thorough reading of the manuscript.∗ Corresponding author.

E-mail addresses: tpirvu@math.mcmaster.ca (T.A. Pirvu),hyzhang69@nankai.edu.cn (H. Zhang).

http://dx.doi.org/10.1016/j.mathsocsci.2014.07.0010165-4896/© 2014 Elsevier B.V. All rights reserved.

Zariphopoulou (1992). All these papers consider a constant rate oftime preference.

In our paper the discount rate is stochastic, exogenous and de-pends on the regime. By the best of our knowledge it is the firstwork to consider stochastic rates of time preference within theMerton problem framework. In a discrete time model (Toshihiko,2009) considers a cyclical discount factor. Next, we motivate thismodelling approach of discount rates. The issue of time discount-ing is the subject of many studies in financial economics. Severalpapers stepped away from the exponential discounting modelling,and based on empirical and experimental evidence proposed dif-ferent discount models. In fact models with time varying discountrates have a long history. More recent works like (Becker and Mul-ligan, 1997; Laibson, 1997) show that economic and social factorsimpact discount rates. Parkin (1988) examines how the businesscycles in the US affect the rate of time preference. Therefore, it ap-pears natural to assume that discount rates vary with the state ofeconomy (so are regime dependent).

Economists studied the relationship between consumption anddiscount rates by considering the following two observed phe-nomena: ‘‘decreasing marginal impatience’’ DMI and ‘‘increasingmarginal impatience’’ IMI. DMI means that the lower the level of

T.A. Pirvu, H. Zhang / Mathematical Social Sciences 71 (2014) 142–150 143

consumption, the more heavily an agent discount the future con-sumption. IMI is just the opposite: the higher the level of consump-tion, the more heavily an agent discount the future consumption.Some papers support DMI, e.g. Das (2003), others advocate for IMI,Takashi (2000).

Non constant discount rates lead to time inconsistency of thedecision maker. The resolution is to consider subgame perfectstrategies. These are strategies which are optimal to implementnowgiven that theywill be implemented in the future. Ekeland andLazrak (2010) considers a deterministic problem with continuoustime, namely the Ramsey problem of economic growth with non-exponential discounting. They define subgame perfect strategiesand characterize themby a generalizedHJB (the nonlocal extensionof HJB). Ekeland and Pirvu (2008) looks at theMerton problemwithspecial types of non-constant deterministic discount rates, andintroduced/characterize subgame perfect strategies. Ekeland et al.(2010) extends Ekeland and Pirvu (2008) by allowing general non-constant deterministic discount rates and life insurance acquisitionin their model. Related works are done in Marín-Solano andNavas (2010), Marín-Solano and Patxot (2012) and Marín-Solanoand Shevkoplyas (2011). Björk and Murgoci (2010) develops ageneral theory for stochastic control problems which are timeinconsistent and introduced the subgame perfect strategies in afairly general stochastic framework. Björk et al. (2014) looks at themean variance problem with time changing risk aversion which isalso time inconsistent.

In this paper, we consider a stochastic discount rates frame-work. Stochastic discount rates are considered in González-Hernández et al. (2007) where a dynamic programming algorithmis established in a discrete time model. Our approach is differentbecause it relies on the idea of Pontryagin’s maximum principle.

The goal of our work is to characterize the subgame perfectstrategies. The methodology developed in Ekeland et al. (2010)is employed to achieve this; it mixes the idea of value function(from the dynamic programming principle) with the idea that inthe future ‘‘optimal trading strategies’’ are to be implemented(from the maximum principle of Pontryagin). The new twist inour paper is the Markov chain, and the mathematical ingredientused is Itô’s formula for the Markov-modulated diffusions. Thevalue function is characterized by a system of four equations: firstequation says that the value function is equal to the continuationutility of subgame perfect strategies; second equation is thewealth equation generated by subgame perfect strategies; the lasttwo equations relate the value function to the subgame perfectstrategies. The end result is a complicated system of PDEs, SDE anda nonlinear equation with a nonlocal term which can be solvedfor CRRA type utility. In this case we find an ansatz for the valuefunction (by disentangling the time, the space and the Markovchain state component). This leads to subgame perfect strategieswhich are time/state dependent and linear in wealth. The infinitehorizon case is considered as well.

Subgame perfect strategies coincide with the optimal ones ifconstant discount rates. We introduce and characterize the pre-commitment strategies; these are strategies that a decision makerwould implement if there exists a commitment mechanism. Nu-merical findings show the dependence on the model’s parametersof the subgame perfect strategies and pre-commitment strategies.In one example we notice that a higher discount rate leads to ahigher subgame perfect consumption rate (relationship which re-semble IMI). This is consistent withMerton (1971) where the opti-mal consumption increases with the discount rate; consequently,the pre-commitment strategies are expected to be increasing func-tions of the discount rate. In another example we show that ahigher discount rate combined with a lower interest rate leads toa lower subgame perfect consumption rate (relationship which re-semble DMI). Thus, the relationship between time discount rates

and consumption may be affected by change in the interest rate.In our example a higher subgame perfect consumption rate occursin the state where the interest rate exceeds the discount rate. Onecandrawaparallel between this andRamsey equationwhich statesthat the consumption growth is proportional to the difference be-tween interest rate and discount rate.

Let us summarize our contributions: in the context of stochasticrates of time preference we introduce and characterize subgameperfect and pre-commitment strategies; for CRRA preferenceswe compute them in semi closed forms; by running numericalexperiments we compare subgame perfect and pre-commitmentstrategies and explore the complex dependence on the model’sparameters.

The reminder of this paper is organized as follows. In Section 2we describe the model and formulate the objective. Section 3contains the main result. Section 4 considers the special case ofCRRA preferences. Section 5 presents a numerical experiment.Section 6 addresses the infinite horizon case. The paper ends withan Appendix.

2. The model

2.1. The financial market

Consider a probability space (Ω, Ft0≤t≤T , F , P), whichaccommodates a standard Brownian motion W = W (t), t ≥ 0and a homogeneous finite state continuous time Markov Chain(MC) J = J(t), t ≥ 0. For simplicity assume that MC takes valuesin S = 0, 1. Our results hold true in the more general situationof S having finitely many states. The filtration Ft0≤t≤∞ is thecompleted filtration generated by W (t)t∈[0,∞) and J(t)t∈[0,∞),

that is Ft = FJt

F W

t . We assume that the stochastic processesW and J are independent. The MC J has a generator Λ = [λij]S×S

with λij ≥ 0 for i = j, and

j∈S λij = 0 for every i ∈ S.In our setup the financial market consists of a bank account

B and a risky asset S, that are traded continuously over a finitetime horizon [0, T ] (here T ∈ (0, ∞) is an exogenously givendeterministic time). The price process of the bank account andrisky asset are governed by the following Markov-modulated SDE:dB(t) = r(t, J(t))B(t)dt,dS(t) = S(t) [α(t, J(t)) dt + σ(t, J(t)) dW (t)] , 0 ≤ t ≤ ∞,

where B(0) = 1 and S(0) = s > 0 are the initial prices.The functions r(t, i), α(t, i), σ (t, i) : i ∈ S, are assumed to bedeterministic, positive and continuous in t . They represent theriskless rate, the stock return and the stock volatility (given thestate i of the MC at t). Moreoverµ(t, i) , α(t, i) − r(t, i)stands for the stock excess return.

2.2. Investment–consumption strategies and wealth processes

In our model, an investor continuously invests in the stock,bond and consumes. Let π(t) be the dollar value invested in stockat time t and c(t) ≥ 0be the consumption.Xu(t) , X(t) representsthe wealth of the investor at time t associated with the tradingstrategy u = (π, c); it satisfies the following stochastic differentialequation (SDE)dXu(t) =

r(t, J(t))Xu(t) + µ(t, J(t))π(t) − c(t)

dt

+ σ(t, J(t))π(t) dW (t), (2.1)where X(0) = x > 0 is the initial wealth and J0 = i ∈ S isthe initial state. This SDE is called the self-financing condition. Anacceptable investment–consumption strategy is defined below:

Definition 2.1. An R2-valued stochastic process u(t) := (π(t),c(t))t∈[0,∞) is called an admissible strategy process and write u ∈

A if it is Ft-progressively measurable and it satisfies the following

144 T.A. Pirvu, H. Zhang / Mathematical Social Sciences 71 (2014) 142–150

integrability condition

E t

0|π(s)µ(s, J(s)) − c(s)| ds

+

t

0|π(s)σ (s, J(s))|2 ds

< ∞,

a.s., for all , t ∈ [0, ∞) (2.2)

and

Xu(t) > 0, c(t) > 0, t ∈ [0, T ],

E supt≤s≤T

|U(c(s))| < ∞, E|U(Xu(T ))| < ∞. (2.3)

Here U is a utility function and is defined in the next subsection.Under the regularity condition (2.2) imposed on π(t),

c(t)t∈[0,∞) above, the SDE (2.1) admits a unique strong solution.

2.3. The risk preferences

A utility function U is a strictly increasing, strictly concave dif-ferentiable real-valued function defined on [0, ∞) which satisfiesthe Inada conditionsU ′(0) , lim

x↓0U ′(x) = ∞, U ′(∞) , lim

x→∞U ′(x) = 0. (2.4)

The strictly decreasing function U ′ maps (0, ∞) onto (0, ∞)and hence has a strictly decreasing inverse I : (0, ∞) → (0, ∞).

2.4. The discount rate

As we mentioned in the introduction, this paper considersstochastic discount rates. An easy way to achieve this is to let thediscount rate depend on the state of the MC. Thus, at some inter-mediate time t ∈ [0, ∞) the discount rate is ρJ(t), for some pos-itive constants ρ0 and ρ1. The intuition of this way of modellingdiscount rates stems from the connection between market statesand discount rates (this can be explained by some models withendogenous discount rates which are influenced by economic fac-tors).

2.5. The risk criterion

In our model, the investor decides what investment/consumption strategy to choose according to the expected utilityrisk criterion. Thus, the investor’s goal is to maximize utility of in-tertemporal consumption and finalwealth. The novelty here is thatwe allow the investor to update the risk criterion and to reconsiderthe optimal strategies she/he computed in the past. This will leadto a time inconsistent behaviour as we show below. Let the agentstart with a given positive wealth x, and assume that the currentmarket state is 0. The time 0 optimal trading strategy is chosen tomaximize the time 0 optimization criterion

E T

0e−ρ0sU(c(s)) ds + e−ρ0TU(X(T ))|X(0) = x

, (2.5)

for some finite T > 0 exogenous horizon. The corresponding valuefunction V (s, x, i, ρ0) solves the Hamilton–Jacobi–Bellman (HJB)equation∂V∂s

(s, x, i, ρ0) + supπ,c

(rx + µπ − c)

∂V∂x

(s, x, i, ρ0)

+12σ 2π2 ∂2V

∂x2(s, x, i, ρ0) + U(c)

− ρ0V (s, x, i, ρ0) +

j∈S

λijV (s, x, j, ρ0) = 0,

i ∈ S, s ∈ [0, T ]. (2.6)

If the time 0 market state is instead 1, then the value functionV (s, x, i, ρ1) solves the HJB

∂V∂s

(s, x, i, ρ1) + supπ,c

(rx + µπ − c)

∂V∂x

(s, x, i, ρ1)

+12σ 2π2 ∂2V

∂x2(s, x, i, ρ1) + U(c)

− ρ0V (s, x, i, ρ1) +

j∈S

λijV (s, x, j, ρ1) = 0,

i ∈ S, s ∈ [0, T ]. (2.7)

At a subsequent time t > 0, the t optimal trading strategy ischosen to maximize the updated time t optimization criterion

E

T

te−ρi(s−t)U(c(s)) ds

+ e−ρi(T−t)U(X(T ))|X(t) = x, J(t) = i

. (2.8)

The corresponding value function V (s, x, i, ρJ(t)) solves the HJB

∂V∂s

(s, x, i, ρJ(t)) + supπ,c

(rx + µπ − c)

∂V∂x

(s, x, i, ρJ(t))

+12σ 2π2 ∂2V

∂x2(s, x, i, ρJ(t)) + U(c)

− ρJ(t)V (s, x, i, ρJ(t)) +

j∈S

λijV (s, x, j, ρJ(t)) = 0,

i ∈ S, s ∈ [t, T ]. (2.9)

If at time t, J(t) = 1, then the Eqs. (2.6) and (2.9) differ. This leadsto different time 0 and time t optimal strategies whence the timeinconsistency (unless the investor is forced to commit to time 0optimal strategies).

There are two pre-commitment strategies corresponding toV (s, x, i, ρ0) of (2.6) and V (s, x, i, ρ1) of (2.7) which are optimalat time 0, but may fail to remain optimal afterwards if timediscount rate changes. They are computed bymaximizing the time0 optimization criterion

E T

0e−ρisU(c(s)) ds + e−ρiTU(X(T ))|X(0) = x, J(0) = i

.

Notice that the pre-commitment strategy corresponding toV (s, x, i, ρ0), s ∈ [0, T ], is alsomaximizing the time t optimizationcriterion, t ∈ [0, T ], with frozen time discount rate ρ0

E

T

te−ρ0(s−t)U(c(s)) ds

+ e−ρ0(T−t)U(X(T ))|X(t) = x, J(t) = i

.

Similarly, the pre-commitment strategy corresponding toV (s, x, i, ρ1), s ∈ [0, T ], is also maximizing the time t optimiza-tion criterion, t ∈ [0, T ], with frozen time discount rate ρ1

E

T

te−ρ1(s−t)U(c(s))ds + e−ρ1(T−t)U(X(T ))|X(t) = x, J(t) = i

.

Let us remark that in computing the pre-commitment strategiesthe time discount rate is not updated over time, i.e., it is held con-stant taking one of the two possible values ρ0 or ρ1. On the otherhand the market coefficients are updated over time. There is also

T.A. Pirvu, H. Zhang / Mathematical Social Sciences 71 (2014) 142–150 145

a naive strategy that switches between pre-commitment strate-gies; if at time t, J(t) = i then the pre-commitment strategy cor-responding to V (t, x, i, ρi) is implemented; if at a subsequent times, J(s) = j = i then the pre-commitment strategy correspondingto V (s, x, j, ρj) is implemented.

In the absence of a commitment mechanism one resolution oftime inconsistency is to introduce subgameperfect strategies. Theyare optimal now given that theywill be implemented in the future.

3. The main result

3.1. The subgame perfect trading strategies

Throughout the paper we denote Ex,it [·] , E[·|X(t) = x,

J(t) = i]. For an admissible strategy u(t) , π(t), c(t)t∈[0,T ] andits corresponding wealth process Xu(t)t∈[0,T ] given by (2.1), wedenote the expected utility functional by

Θ(t, x, i, π, c) , Ex,it

T

te−ρi(s−t)U(c(s)) ds

+ e−ρi(T−t)U(Xu(T ))

. (3.10)

Following (Ekeland and Pirvu, 2008) we shall give a rigorousmathematical formulation of the subgame perfect strategies in theformal definition below.

Definition 3.1. Let F = (F1, F2) : [0, T ] × R+× S → R × R+ be a

map such that for any t, x > 0 and i ∈ S

lim infϵ↓0

Θ(t, x, i, F1, F2) − Θ(t, x, i, πϵ, cϵ)ϵ

≥ 0, (3.11)

where

Θ(t, x, i, F1, F2) , Θ(t, x, i, π , c),

π(s) , F1(s, X(s), J(s)), c(s) , F2(s, X(s), J(s)), (3.12)

and π(s), c(s)s∈[t,T ] is admissible. Here, the process X(s)s∈[t,T ]

is the wealth corresponding to π(s), c(s)s∈[t,T ]. The processπϵ(s), cϵ(s)s∈[t,T ] is another admissible investment–consumptionstrategy defined by

πϵ(s) =

π(s), s ∈ [t, T ] \ Eϵ,tπ(s), s ∈ Eϵ,t ,

(3.13)

cϵ(s) =

c(s), s ∈ [t, T ] \ Eϵ,tc(s), s ∈ Eϵ,t ,

(3.14)

with Eϵ,t = [t, t + ϵ]; π(s), c(s)s∈Eϵ,t is any trading strategy forwhich πϵ(s), cϵ(s)s∈[t,T ] is an admissible strategy. If (3.11) holdstrue, then π(s), c(s)s∈[t,T ] is a subgame perfect strategy.

3.2. The value function

Our goal is in a first step to characterize the subgame perfectstrategies and then to find them in special cases. Inspired byEkeland and Pirvu (2008), the value function v : [0, T ] × R+

×

S × ρ0, ρ1 → R × R+ is a C1,2 function, concave in the secondvariable defined by

v(t, x, i, ρ) , Ex,it

T

te−ρ(s−t)U(F2(s, X(s), J(s))) ds

+ e−ρ(T−t)U(X(T ))

, i ∈ S, ρ ∈ ρ0, ρ1. (3.15)

Here X(s)s∈[0,T ] is the wealth process corresponding to π(s),c(s)s∈[t,T ], so it solves the SDE

dX(s) = [r(s, J(s))X(s) + µ(s, J(s))F1(s, X(s), J(s))

− F2(s, X(s), J(s))]ds

+ σ(s, J(s))F1(s, X(s))dW (s). (3.16)

Moreover, F = (F1, F2) is defined by

F1(t, x, i) , −µ(t, i) ∂v

∂x (t, x, i, ρi)

σ 2(t, i) ∂2v∂x2

(t, x, i, ρi),

F2(t, x, i) , I

∂v

∂x(t, x, i, ρi)

, t ∈ [0, T ], i ∈ S.

(3.17)

Recall that I is the inversemarginal utility. Thus, the value functionis characterized by a system of four equations: one integralequation with nonlocal term (3.15), one SDE (3.16) and two PDEs(3.17). Of course the existence of such a function v satisfying theequations above is not a trivial issue. At this point we introducethe following standing assumption.

Assumption 3.1. Assume that there exists a value function andthat the PDE systems

∂ f∂t

(t, s, x, i) + (r(t, i)x + µ(t, i)F1(t, x, i)

− F2(t, x, i))∂ f∂x

(t, s, x, i)

+σ 2(t, i)F 2

1 (t, x, i)2

∂2f∂x2

(t, s, x, i)

+

j∈S

λijf (t, s, x, j) = 0,

f (s, s, x, i) = U(F2(t, x, i)), i ∈ S,

(3.18)

∂h∂t

(t, x, i) + (r(t, i)x + µ(t, i)F1(t, x, i) − F2(t, x, i))∂h∂x

(t, x, i)

+σ 2(t, i)F 2

1 (t, x, i)2

∂2h∂x2

(t, x, i) +

j∈S

λijh(t, x, j) = 0,

h(T , x, i) = U(x), i ∈ S (3.19)

have a C1,2 solution on [0, s]×R+×S → R and [0, T ]×R+

×S →

R with exponential growth. Here t ≤ s ≤ T , and i ∈ S. Moreoverassume that the SDE (3.16) has a solution and that π(s), c(s)s∈[0,T ]

given by (3.12) with (F1, F2) of (3.17) is admissible.

Lemma 3.1. The value function can be expressed as follows:

v(t, x, i, ρ) =

T

te−ρ(s−t)f (t, s, x, i) ds

+ e−ρ(T−t)h(t, x, i), i ∈ S, ρ ∈ ρ0, ρ1. (3.20)

Proof. In the light of (3.18) and (3.19) the processes f (t, s, X(t),J(t))t∈[0,s] and h(t, X(t), J(t))t∈[0,T ] are martingales. There-fore,

f (t, s, x, i) = Ex,it [U(F2(s, X(s), J(s)))],

h(t, x, i) = Ex,it [U(X(T ))].

By (3.15) the claim follows.

The next lemma expresses the integral equation (3.15) as a systemof PDEs.

146 T.A. Pirvu, H. Zhang / Mathematical Social Sciences 71 (2014) 142–150

Lemma 3.2. The value function solves the following system of PDEs(four equations)

∂v

∂t(t, x, i, ρ) + (r(t, i)x + µ(t, i)F1(t, x, i)

− F2(t, x, i))∂v

∂x(t, x, i, ρ)

+σ 2(t, i)F 2

1 (t, x, i)2

∂2v

∂x2(t, x, i, ρ)

+U(F2(t, x, i)) + λiiv(t, x, i, ρ) − ρv(t, x, i, ρ)

= −λijv(t, x, j, ρ), i, j ∈ S, ρ ∈ ρ0, ρ1. (3.21)

Conversely, if v solves (3.21) then v solves (3.15) as well.

Proof. By differentiating (3.20) with respect to t we get

∂v

∂t(t, x, i, ρ) =

T

te−ρ(s−t) ∂ f

∂t(t, s, x, i) ds

+ e−ρ(T−t) ∂h∂t

(t, x, i)

+ ρv(t, x, i, ρ) − f (t, t, x, i). (3.22)

Moreover

∂v

∂x(t, x, i, ρ) =

T

te−ρ(s−t) ∂ f

∂x(t, s, x, i) ds

+ e−ρ(T−t) ∂h∂x

(t, x, i). (3.23)

∂2v

∂x2(t, x, i, ρ) =

T

te−ρ(s−t) ∂

2f∂x2

(t, s, x, i) ds

+ e−ρ(T−t) ∂2h

∂x2(t, x, i). (3.24)

In light of (3.18), (3.19), (3.22), (3.23) and (3.24) it follows that

∂v

∂t(t, x, i, ρ) + (r(t, i)x + µ(t, i)F1(t, x, i)

− F2(t, x, i))∂v

∂x(t, x, i, ρ) +

σ 2(t, i)F 21 (t, x, i)2

∂2v

∂x2(t, x, i, ρ)

+U(F2(t, x, i)) + λiiv(t, x, i, ρ)

− ρv(t, x, i, ρ) = −λijv(t, x, j, ρ),

which is (3.21). For the converse notice that the processM(t)t∈[0,T ],

M(t) , e−ρtv(t, X(t), J(t), ρ) +

t

0e−ρsU(F2(s, X(s), J(s))) ds

is amartingale andM(T ) = e−ρTU(X(T )). ThusM(t) = Ex,it [M(T )]

so the claim yields.

The following theorem states the central result of our paper.

Theorem 3.1. Given a value function v let us define (F1, F2) by (3.17).The trading strategy π(s), c(s)s∈[0,T ] given by (3.12) is a subgameperfect strategy.

Proof. Let us define for i ∈ S, ρ ∈ ρ0, ρ1

Γ v(t, x, i, ρ) ,∂v

∂t(t, x, i, ρ) + (r(t, i)x + µ(t, i)F1(t, x, i)

− F2(t, x, i))∂v

∂x(t, x, i, ρ)

+σ 2(t, i)F 2

1 (t, x, i)2

∂2v

∂x2(t, x, i, ρ)

+

j∈S

λijv(t, x, i, ρ) + U(F2(t, x, i)). (3.25)

Γ π,cv(t, x, i, ρ) ,∂v

∂t(t, x, i, ρ) + (r(t, i)x + µ(t, i)π − c)

×∂v

∂x(t, x, i, ρ) +

12σ 2(t, i)π2 ∂2v

∂x2(t, x, i, ρ)

+

j∈S

λijv(t, x, i, ρ) + U(c). (3.26)

By (3.21)

Γ v(t, x, i, ρ) = ρv(t, x, i, ρ), i ∈ S, ρ ∈ ρ0, ρ1.

The concavity of v and the first order conditions lead to

Γ v(t, x, i, ρ) = maxπ,c

Γ π,cv(t, x, i, ρ),

(F1(t, x, i), F2(t, x, i)) = argmaxπ,c

Γ π,cv(t, x, i, ρi).(3.27)

Let us recall that

Θ(t, x, i, F1, F2) = v(t, x, i, ρi), i ∈ S. (3.28)

Thus

Θ(t, x, i, F1, F2) − Θ(t, x, i, πϵ, cϵ)

= Ex,it

t+ϵ

te−ρi(s−t)

[U(F2(s, X(s), J(s))) − U(c(s))] ds

+ Ex,it

T

t+ϵ

e−ρi(s−t)[U(F2(s, X(s), J(s))) − U(c(s))] ds

+ Ex,i

te−ρi(T−t)(U(X(T )) − U(X(T )))

. (3.29)

In the light of inequalities (2.3) and Dominated Convergence The-orem

limϵ↓0

Ex,it

t+ϵ

t e−ρi(s−t)[U(F2(s, X(s), J(s))) − U(c(s))] ds

ϵ

= U(F2(t, x, i)) − U(c(t)).

By (3.28) it follows that

Ex,it

T

t+ϵ

e−ρi(s−t)[U(F2(s, X(s), J(s))) − U(c(s))] ds

+ Ex,i

te−ρi(T−t)(U(X(T )) − U(X(T )))

(3.30)

= Ex,it

v(t + ϵ, X(t + ϵ), J(t + ϵ), ρi)

−v(t + ϵ, X(t + ϵ), J(t + ϵ), ρi)

. (3.31)

Itô’s formula yields

Ex,it

v(t + ϵ, X(t + ϵ), J(t + ϵ), ρi)

− v(t + ϵ, X(t + ϵ), J(t + ϵ), ρi)

= Ex,i

t

t+ϵ

t[Γ v(s, X(s), J(s), ρi) − U(F2(s, X(s), J(s)))]ds

− Ex,it

t+ϵ

t[Γ π,cv(s, X(s), J(s), ρi) − U(c(s))]ds.

Therefore

limϵ↓0

Θ(t, x, i, F1, F2) − Θ(t, x, i, πϵ, cϵ)ϵ

= [Γ v(t, x, i, ρi) − Γ π,cv(t, x, i, ρi)] ≥ 0,

where the inequality follows from (3.27).

T.A. Pirvu, H. Zhang / Mathematical Social Sciences 71 (2014) 142–150 147

4. CRRA preferences

In this section we assume that the utility is of power type, i.e.,U(x) = Uγ (x) =

xγγ. Let us take advantage of this special utility

and propose an ansatz for finding v. When γ = 0 we search for vof the form:

v(t, x, i, ρi) = g(t, i)xγ

γ,

v(t, x, i, ρj) = g(t, i)xγ

γ, i, j ∈ S, x ≥ 0.

(4.32)

When γ = 0 (logarithmic utility) we look for v of the form

v(t, x, i, ρi) = g(t, i) log x + l(t, i),

v(t, x, i, ρj) = g(t, i) log x + l(t, i), i, j ∈ S, x ≥ 0.(4.33)

The functions g(t, i), g(t, i) and l(t, i), l(t, i) are to be found. Inlight of Eqs. (3.17) one gets

F1(t, x, i) =µ(t, i)x

σ 2(t, i)(1 − γ ), F2(t, x, i) = g

1γ−1 (t, i)x. (4.34)

Note that, the functions l(t, i), l(t, i) do not enter the above equa-tion so their expressions are not important. By (3.16), the associ-ated wealth process satisfies the following SDE:

dX(s) =

r(s, J(s)) +

µ2(s, J(s))σ 2(s, J(s))(1 − γ )

− g1

γ−1 (s, J(s))

X(s)ds

+µ(s, J(s))

σ (s, J(s))(1 − γ )X(s)dW (s). (4.35)

This is a linear SDE which can be easily solved. By plugging v of(4.32) into (3.15) (with F1, F2 of (4.34) and X of (4.35)), we obtainthe following system of four equations for g(t, i), g(t, i), i, j ∈ S:∂g∂t

(t, i) +

γ r(t, i) +

µ2(t, i)γ2σ 2(t, i)(1 − γ )

− ρi

g(t, i)

+ λiig(t, i) + (1 − γ )gγ

γ−1 (t, i) = −λijg(t, j), (4.36)

∂ g∂t

(t, i) +

γ r(t, i) +

µ2(t, i)γ2σ 2(t, i)(1 − γ )

− ρj

g(t, i)

+ λiig(t, i) + (1 − γ )g1

γ−1 (t, i)g(t, i) = −λijg(t, j), (4.37)with the boundary condition g(T , i) = g(T , i) = 1, i ∈ S. Nextwe show that there exists a unique solution for this ODE system,and that v of (4.32), (4.33) is a value function.We consider the caseγ = 0 (the case γ = 0 is similar).

Lemma 4.1. There exists a unique continuously differentiable uni-formly bounded solution g(t, i), g(t, i), i ∈ S for the system (4.36)and (4.37). Furthermore,

v(t, x, i, ρi) = g(t, i)xγ

γ,

v(t, x, i, ρj) = g(t, i)xγ

γ, i, j ∈ S, x ≥ 0,

is a value function. This means: v is continuously differentiable,concave in x and satisfies (3.15)with F1, F2 of (4.34) and X of (4.35).

Appendix A. proves this lemma. This leads to the central result of this section.

Theorem 4.1. The trading strategy π(s), c(s)s∈[0,T ] given by

π(s) =µ(s, J(s))X(s)

σ 2(s, J(s))(1 − γ ), c(s) = g

1γ−1 (s, J(s))X(s), (4.38)

is a subgame perfect strategy.

Proof. Let us notice that the boundedness of g , and the fact thatX of (4.35) is positive with finite moments of any order implythe acceptability of π(s), c(s)s∈[0,T ]. Next, the conclusion followsfrom Theorem 3.1.

Remark 4.1. In the case of constant discount rate, i.e., ρ0 = ρ1,the subgame perfect strategies coincide with the optimal ones.This can be seen by looking at the Eq. (3.21) which is exactly theHJB equation. The optimal investment and consumption problemis solved in closed formby Sotomayor and Cadenillas (2009)withinthis framework.

Remark 4.2. The subgame perfect investment strategy does notchange relative to the Merton model with constant discountrate; more precisely the proportion of wealth invested in thestocks is the same as in the case of Merton model with constantdiscount rate. This can be explained by the randomness of marketcoefficients being driven by the Markov chain only. We conjecturethat in amodel withmean revertingmarket price of risk this resultdoes not hold.

Remark 4.3. In the context of CRRA utilities we get semi closedform solutions for both subgame perfect strategy and the two pre-commitment strategies. Let us summarize:• the subgame perfect strategy π(s), c(s)s∈[0,T ] (with associated

wealth process X(s)s∈[0,T ]) is given by

π(s) =µ(s, J(s))X(s)

σ 2(s, J(s))(1 − γ ), c(s) = g

1γ−1 (s, J(s))X(s);

• the two pre-commitment strategies are given by

πk(s) =µ(s, J(s))Xk(s)

σ 2(s, J(s))(1 − γ ),

ck(s) = g1

γ−1k (s, J(s))Xk(s), k = 1, 2,

(with associated wealth process Xk(s)s∈[0,T ], k = 1, 2); heregk(t, i), i, j ∈ S, k = 1, 2 are given by two systems with twoequations:

∂ g1∂t

(t, i) +

γ r(t, i) +

µ2(t, i)γ2σ 2(t, i)(1 − γ )

− ρ0

g1(t, i)

+ λiig1(t, i) + (1 − γ )gγ

γ−11 (t, i) = −λijg1(t, j), (4.39)

∂ g2∂t

(t, i) +

γ r(t, i) +

µ2(t, i)γ2σ 2(t, i)(1 − γ )

− ρ1

g2(t, i)

+ λiig2(t, i) + (1 − γ )gγ

γ−12 (t, i) = −λijg2(t, j), (4.40)

with boundary condition g1(T , i) = g2(T , i), i ∈ S.

5. Numerical analysis

In this section, we use Matlab’s powerful ODE solvers (espe-cially the functions ode23 and ode45) to perform numerical ex-periments. We numerically solve ODE systems (4.36), (4.39) and(4.40) to get the subgame perfect and pre-commitment strategies.We take the Markov Chain generator to be

−2 21.5 −1.5

.

We plot the subgame perfect and pre-commitment consumptionrates denoted by

C(t, J(t)) ,F2(t, X(t), J(t))

X(t)= g

1γ−1 (t, J(t)),

Ck(t, J(t)) ,ck(t)

Xk(t)= g

1γ−1k (t, J(t)), k = 1, 2.

148 T.A. Pirvu, H. Zhang / Mathematical Social Sciences 71 (2014) 142–150

Remark 5.1. Numerical findings reveal the complex dependenceof the subgame perfect and pre-commitment consumption rateson the model parameters. In Fig. 1 we have the same marketcoefficients on the two states and a higher discount rate on the firststate. This leads to higher subgame perfect consumption rate in thefirst state. In Fig. 1 notice that C(t, 1) = C2(t, 0) = C2(t, 1); thismay seem surprising but it can be perhaps explained by ρ1 = 0.06being close to r0 = r1 = 0.05. In Fig. 2 we also let the interest ratedepend on the regime. Here although

ρ0 = 0.07 > ρ1 = 0.06,

C(t, 1) > C(t, 0). We see from the plots that higher discount rateslead to higher consumption rates and higher interest rates lead tohigher consumption rates as well. In order to gain intuition for thelast result let us notice that not only r1 = 0.09 > r0 = 0.01 butalso r1 = 0.09 > ρ1 = 0.06, r0 = 0.01 < ρ0 = 0.07. Thissuggests that consumption rates are higher in the states where thedifference between the interest rate and discount rate is higher.One can see from the plots the monotone relationship betweenpre-commitment consumption rates and discount rates. In fact inthis case one can mathematically prove that ∂ C

∂ρ> 0. Therefore,

it is not surprising that this property is inherited by the subgameperfect consumption rates. Notice from Figs. 1 and 2 that, the gapbetween the two subgame perfect consumption rates increases asthe gap between discount rates increases.

6. Infinite horizon case

In this case, we need to make the following time homogeneityassumption on the market coefficientsdB(t) = r(J(t))B(t)dt,dS(t) = S(t) [α(J(t)) dt + σ(J(t)) dW (t)] , 0 ≤ t ≤ ∞,

and restrict to time homogeneous investment–consumptionstrategy processes π(t), c(t)t∈[0,∞). Due to time homogeneitythe expected utility functional Θ(x, i, π, c) can be defined as

Θ(x, i, π, c) , Ex,i

∞

0e−ρisU(c(s)) ds

. (6.41)

Admissible strategies are required to satisfy Θ(x, i, π, c) < ∞.The definition of subgame perfect strategies in this context issimilar to finite horizon case.

Definition 6.1. Let F = (F1, F2) : R+× S → R × R+ be a map

such that for any x > 0 and i ∈ S

lim infϵ↓0

Θ(x, i, F1, F2) − Θ(x, i, πϵ, cϵ)ϵ

≥ 0, (6.42)

where

Θ(x, i, F1, F2) , Θ(x, i, π , c),

π(s) , F1(X(s), J(s)), c(s) , F2(X(s), J(s)), (6.43)

and π(s), c(s)s∈[0,∞) is admissible. Here, the process X(s)s∈[0,∞)

is the wealth corresponding to π(s), c(s)s∈[0,∞). Theprocess πϵ(s), cϵ(s)s∈[0,∞) is another admissible investment–consumption strategy defined by

πϵ(s) =

π(s), s ∈ (ϵ, ∞]

π(s), s ∈ [0, ϵ], (6.44)

cϵ(s) =

c(s), s ∈ (ϵ, ∞)c(s), s ∈ [0, ϵ], (6.45)

and π(s), c(s)s∈[0,ϵ] is any trading strategy for which πϵ(s),cϵ(s)s∈[0,∞) is an admissible strategy. If (6.42) holds true, thenπ(s), c(s)s∈[0,∞) is a subgame perfect strategy.

Inspired by the finite horizon case we define the value function

v(x, i, ρ) , Ex,i

∞

0e−ρsU(F2(s, X(s), J(s))) ds

,

i ∈ S, ρ ∈ ρ0, ρ1. (6.46)Furthermore F = (F1, F2) is defined by

F1(x, i) , −µ(i) ∂v

∂x (x, i, ρi)

σ 2(i) ∂2v∂x2

(x, i, ρi),

F2(x, i) , I

∂v

∂x(x, i, ρi)

, i ∈ S.

(6.47)

Under some technical assumptions on trading strategies thecounterpart of Theorem3.1 can be established.Weomit the details.In the special case of CRRA preferences

F1(x, i) =µ(i)x

σ 2(i)(1 − γ ), F2(x, i) = g

1γ−1 (i)x. (6.48)

The constants g(i), g(i), i, j ∈ S solve the following algebraicsystem of four nonlinear equationsγ r(i) +

µ2(i)γ2σ 2(i)(1 − γ )

− ρi

g(i) + λiig(i)

+ (1 − γ )gγ

γ−1 (i) = −λijg(j), (6.49)γ r(i) +

µ2(i)γ2σ 2(i)(1 − γ )

− ρj

g(i) + λiig(i)

+ (1 − γ )g1

γ−1 (i)g(i) = −λijg(j). (6.50)Existence of a solution for the above system can be establishedas in Lemma 4.1 of Sotomayor and Cadenillas (2009). We skipthe details. In order for the subgame perfect strategies to satisfyΘ(x, i, π , c) < ∞, one needs to impose conditions on ρ0, ρ1(sometimes referred to as transversality conditions).

Appendix

A.1. Proof of Lemma 4.1

Existence and uniqueness for the ODE system is granted locallyin time. For global existence and uniqueness we establish globalestimates. Let α(t, i) = −

γ r(t, i) +

µ2(t,i)γ2σ 2(t,i)(1−γ )

− ρi + λii

, i ∈

S, and α(t, i) = −

γ r(t, i) +

µ2(t,i)γ2σ 2(t,i)(1−γ )

− ρj + λii

, i, j ∈ S.

Then from the ODE system we get∂g∂t

(t, i) − α(t, i)g(t, i) ≤ 0,

∂ g∂t

(t, i) − α(t, i)g(t, i) ≤ 0, i ∈ S.

Integrating this from t to T and using g(T , i) = g(T , i) = 1, i ∈ Swe get a global lower bound M > 0 for g, g . Next, since γ < 1,from the ODE system∂g∂t

(t, i) ≥

α(t, i) − (1 − γ )M

γγ−1

g(t, i) − λijg(t, j), i, j ∈ S,

∂ g∂t

(t, j) ≥

α(t, j) − (1 − γ )M

γγ−1

g(t, j) − λjig(t, i), i, j ∈ S.

Let A(t, i, j) , min(α(t, i)− (1−γ )Mγ

γ−1 , α(t, j)− (1−γ )Mγ

γ−1 ),λ(i, j) , max(λij, λji). Let h(t) , g(t, i) + g(t, j). Then by addingthe above inequalities we get∂h∂t

(t) ≥ (A(t, i, j) − λ(i, j))h(t).

Integrating this from t to T yields the desired upper bound.

T.A. Pirvu, H. Zhang / Mathematical Social Sciences 71 (2014) 142–150 149

Fig. 1. Subgameperfect andpre-commitment consumption rates forµ0 = 0.1, µ1 = 0.1, σ0 = 0.2, σ1 = 0.2, r0 = 0.05, r1 = 0.05; the discount ratesρ0 = 0.3, ρ1 = 0.06.

Fig. 2. Subgame perfect and pre-commitment consumption rates for µ0 = 0.1, µ1 = 0.1, σ0 = 0.2, σ1 = 0.2, r0 = 0.01, r1 = 0.09; the discount rates ρ0 = 0.07,ρ1 = 0.06.

150 T.A. Pirvu, H. Zhang / Mathematical Social Sciences 71 (2014) 142–150

Recall that

v(t, x, i, ρi) = g(t, i)xγ

γ,

v(t, x, i, ρj) = g(t, i)xγ

γ, i, j ∈ S, x ≥ 0.

Then it can be easily shown that v solves (3.21). By Lemma 3.2, vsatisfies (3.15).Moreover v is a C1,2 function, concave in the secondvariable so it is a value function. The functions

f (t, s, x, i) , Ex,it [U(F2(s, X(s), J(s)))],

h(t, x, i) , Ex,it [U(X(T ))],

solve (3.18) and (3.19) so the Assumption 3.1 is met.

References

Bauerle, N., Rieder, U., 2004. Portfolio optimization with Markov-modulated stockprices and interest rates. IEEE Trans. Automat. Control 49, 442–447.

Becker, G.S., Mulligan, C.B., 1997. The endogenous determination of timepreference. Quart. J. Econ. 112, 729–758.

Björk, T., Murgoci, A., 2010. A general theory of Markovian time inconsistentstochastic control problems. Preprint.

Björk, T., Murgoci, A., Zhou, X.Y., 2014. Mean–variance portfolio optimization withstate-dependent risk aversion. Math. Finance 24 (1), 1–24.

Das, M., 2003. Optimal growth with decreasing marginal impatience. J. Econom.Dynam. Control 27, 1881–1898.

Ekeland, I., Lazrak, A., 2010. The golden rulewhenpreferences are time inconsistent.Math. Financ. Econ. 4, 29–55.

Ekeland, I., Mbodji, O., Pirvu, T.A., 2010. Time consistent portfolio management.SIAM J. Financ. Math. 3, 1–32.

Ekeland, I., Pirvu, T.A., 2008. Investment and consumption without commitment.Math. Financ. Econ. 2, 57–86.

Elliott, R.J., Chan, L., Siu, T.K., 2005. Option pricing and Esscher transform underregime switching. Ann. Finance 1, 423–432.

González-Hernández, J., López-Martínez, R.R., Pérez-Hernández, J.R., 2007. Markovcontrol processeswith randomized discount cost. Math.Methods Oper. Res. 65,27–44.

Guo, X., 2001. Information and option pricing. Quant. Finance 1, 38–44.Laibson, D., 1997. Golden eggs and hyperbolic discounting. Quart. J. Econ. 112,

443–478.Marín-Solano, J., Navas, J., 2010. Consumption and portfolio rules for time-

inconsistent investors. European J. Oper. Res. 201, 860–872.Marín-Solano, J., Patxot, C., 2012. Heterogeneous discounting in economic

problems. Optimal Control Appl. Methods 33, 32–50.Marín-Solano, J., Shevkoplyas, E.V., 2011. Non-constant discounting and differential

games with random horizon. Automatica 47, 2626–2638.Merton, R.C., 1969. Lifetime portfolio selection under uncertainty: the continuous-

time case. Rev. Econ. Stat. 51, 247–257.Merton, R.C., 1971. Optimum consumption and portfolio rules in a continuous-time

model. J. Econom. Theory 3, 373–413.Parkin, M., 1988. Amethod for determining whether the parameters in aggregative

models are structural. Carnegie–Rochester Conf. Ser. Public Policy 29, 215–252.Sotomayor, L.R., Cadenillas, A., 2009. Explicit solutions of consumption–investment

problems in financial markets with regime switching. Math. Finance 9 (2),251–279.

Takashi, K., 2000. Increasing marginal impatience and intertemporal substitution.J. Econ. 72, 67–79.

Toshihiko, M., 2009. A note on cyclical discount factors and labor market volatility.Preprint.

Zariphopoulou, T., 1992. Investment-consumption models with transaction feesand Markov-chain parameters. SIAM J. Control Optim. 30, 613–636.