williams (2006) (on dynamic principal agenta problems in continuous time)

8/13/2019 Williams (2006) (on Dynamic Principal Agenta Problems in COntinuous Time)

1/35

On Dynamic Principal-Agent

Problems in Continuous TimeNoah Williams *

Department of Economics, Princeton University E-mail: [email protected]

Revised December 8, 2006

I study the dynamic provision of incentives in environments with hidden ac-tions and possibly hidden states. I characterize implementable contracts by es-tablishing the applicability of the rst-order approach to incentive problems. Im-plementable contracts are history dependent, but can be written recursively with

a small number of state variables. When the agents actions are hidden, but allstates are observed, implementable contracts must take account of the agents util-ity process. When the agent has access to states which the principal cannot ob-serve, implementable contracts must also take account of the shadow value (inmarginal utility terms) of the hidden states. I then explicitly solve a simple exam-ple with linear production and exponential utility, showing how allocations aredistorted for incentive reasons and how access to hidden savings tempers altersallocations.

1. INTRODUCTION

In many economic environments, there arise natural questions as to how to provideincentives in a dynamic setting with hidden information. Some important examples in-clude the design of employment and insurance contracts and the choice of economicpolicy, specically unemployment insurance and scal policy. 1 However the analysisof dynamic hidden information models rapidly becomes complex, even more so whensome of the relevant state variables are not publicly observed or cannot be monitored. Inthis paper I develop a tractable method to analyze a class of models with hidden actionsand hidden states.

I study the design of optimal contracts in a dynamic principal-agent setting. I con-sider situations in which the principal cannot observe the agents actions, and the agentmay have access to a private unobservable state variable as well. One of the main dif-

culties in analyzing such models is in incorporating the agents incentive compatibilityconstraints. Related issues arise in static principal-agent problems, where the main sim-plifying method is known as the rst-order approach. This method replaces the incentivecompatibility constraints by the rst order necessary conditions from the agents deci-

* I thank Dilip Abreu, Jak sa Cvitani c, Narayana Kocherlakota, Nicola Pavoni, Chris Phelan, Yuliy Sannikov, IvanWerning, and Xun Yu Zhou for helpful comments.

1 The list of references here is long and expanding. A short list of papers related to my approach or setting include:Holmstrom and Milgrom (1987), Schattler and Sung (1993), Sung (1997), Fudenberg, Holmstrom, and Milgrom (1990),Rogerson (1985a), Hopenhayn and Nicolini (1997), and Golosov, Kocherlakota, and Tsyvinski (2003).

1


2/35

2 NOAH WILLIAMS

sion problem. Hence the problem becomes solvable with standard Lagrangian methods.However it has been known at least since Mirrlees (1999) (originally 1975), that this ap-proach is not generally valid. Different conditions insuring the validity of the rst-orderapproach in a static setting have been given by Mirrlees (1999), Rogerson (1985b), and Jewitt (1988).

But there have previously been no general justications of the rst-order approachin a dynamic setting with imperfect information, which has hindered the analysis of such models. Working in a continuous time framework, I establish the validity of therst-order approach in my dynamic model. These results are perhaps the rst and mostgeneral such justication in a dynamic setting. 2

In the case of only hidden actions, under relatively mild conditions the rst-orderapproach gives a complete characterization of the class of implementable contracts, thosein which the agent carries out the principals target action. Moreover these conditionsare easily stated in terms of the primitives of the model. Even in the static setting, theconditions insuring the validity of the rst order approach can be difcult to satisfy, andthey become even more complex in more than one dimension. By contrast my conditionsare quite natural, and apply to multidimensional settings. When the agents utility isseparable in consumption and leisure, I simply require that effort increase output butdecrease utility, and that output and utility each be concave in effort. Thus my resultshere are quite general.

When the agent has access to hidden states as well, I show that the rst-order ap-proach is valid under a stronger concavity assumption. As I discuss below, my assump-tion is too strong for some typical formulations with hidden savings. 3 But my results may

apply in many other potential hidden state models, such as models like Fernandes andPhelan (2000) where the current state depends on past effort choices. In Williams (2006),I show that simple extensions of the methods developed here can be used to characterizemodels with persistent private information.

After characterizing the class of implementable contracts, I briey discuss the prin-cipals choice of an optimal contract. Then I turn to a simple example illustrating myresults. In particular, I focus on a more fully dynamic extension of Holmstrom and Mil-grom (1987) in which the principal and agent have exponential preferences, and produceand save via linear technologies. In this simple setting I explicitly solve for the optimalcontracts under full information, when the agents effort action is hidden, and when the

agent has access to hidden savings as well.4

In addition to illustrating how to employ my2 Independently in concurrent research, Sannikov (2006) has given a similar proof in a simpler model with only

hidden actions. More recently (i.e. after earlier drafts of this paper), Westereld (2006) analyzes a related model usingdifferent techniques, and Cvitani c, Wan, and Zhang (2006) analyze hiddenaction modelswith only terminal payments.But there are no comparable results of my generality, and none of these papers touch on the hidden action case. Thereare some related results on the rst-order approach in other dynamic settings. For example, Phelan and Stacchetti(2001) use a rst-order approach to characterize optimal Ramsey taxation.

3 See Allen (1985), Cole and Kocherlakota (2001), Werning (2001a)-(2001b), Abraham and Pavoni (2003), Kocher-lakota (2004a), and Doepke and Townsend (2006) for some alternative formulations with hidden savings.

4 As I discuss below, the exponential preferences greatly simplify the hidden saving case as in Fudenberg, Holm-strom, and Milgrom (1990). However my results do differ from theirs, as the contract is history dependent.


3/35

ON DYNAMIC PRINCIPAL-AGENT PROBLEMS IN CONTINUOUS TIME 3

results in practice, this application allows us characterize how the informational frictionsinduce distortions along the labor/leisure margin (a labor wedge) and the consump-tion/savings margin (an intertemporal wedge). 5 A direct effect of moral hazard inproduction is that there is a wedge between the agents marginal rate of substitutionand the marginal product of labor under an optimal contract. Moreover, the dynamicnature of the contracting problem leads to an wedge between the agents intertemporalmarginal rate of substitution and the marginal product of capital. If the principal cancontrol the agents consumption (or tax his savings), the contract will thus have an in-tertemporal wedge. However when the with hidden savings, the agents consumptioncannot be directly controlled by the principal. Thus there is no intertemporal wedge, butinstead a larger labor wedge. Finally, I show how the contracts can be implemented by ahistory-dependent, performance-related payment and a tax on savings.

My analysis builds on and extends Holmstrom and Milgrom (1987) and Schattler andSung (1993), who studied incentive provision in a continuous time model. In those pa-pers, the principal and the agent contract over a period of time, with the contract speci-fying a one-time salary payment at the end of the contract period. In contrast to these in-tertemporal models, which consider dynamic processes but only a single transfer from theprincipal to the agent, in my fully dynamic model transfers occur continuously through-out the contract period. 6 My results in the hidden action case extend Schattler and Sung(1993). but in the dynamic setting, the agents optimal utility process becomes an addi-tional state variable which the principal must respect. This form of history dependenceis analogous to many related results in the literature, starting with Abreu, Pearce, andStacchetti (1986)-(1990) and Spear and Srivastrava (1987).

As noted above, the closest paper to mine is Sannikov (2006), who studies a simplemodel with hidden actions and no state variables other than the utility process. Whilethere are similarities in some of our results, there is a difference in focus. Sannikovs sim-pler setup allows for more complete characterization of the optimal contract. In contrastmy results cover much more general models with natural dynamics and with hiddenstates, but the generality implies that typically I can only provide partial characteriza-tions. However, I do show how to use thje methods in practice to solve for optimalcontracts. Apart from special cases, such as the one analyzed here, this will typicallyrequire numerical methods.

In the case of hidden states, the principal must also keep track of an additional state

variable which summarizes the shadow value of the state in terms of its contributionsto marginal utility. My results, which draw on some results in control theory due to Zhou(1996), are similar to the approach of Werning (2001a) and Abraham and Pavoni (2003)in discrete time dynamic moral hazard problems. 7 However they do not formally prove

5 See Chari, Kehoe, and McGrattan (2004) on the role of the labor and intertemporal (or investment) wedges inuctuations. Kocherlakota (2004b) discusses how intertemporal wedges result form dynamic moral hazard problems.

6 The recent paper by Cadenillas, Cvitani c, and Zapatero (2005) studies a general intertemporal model with fullobservation, and some of their results cover dynamic transfers.

7 Detemple, Govindaraj, and Loewenstein (2005) extend the Holmstrom and Milgrom (1987) model to hidden states, but retain the assumption of one-time salary payments. Their approach differs substantially from ours.


4/35

4 NOAH WILLIAMS

the validity of their approach (at least ex-ante). 8 Alternatively, Doepke and Townsend(2006) consider a recursive approach to problems with hidden states by enlarging thestate space. In their setup the state vector becomes the agents promised utility con-ditional on each value of the unobserved state. This requires discretization of the statespace, and also leads to a large number of incentive constraints which need to be checked.While they provide some ways to deal with these problems, their results remain far lessefcient than mine, albeit requiring some less stringent assumptions.

One of the main difculties in the analysis is to allow for general history dependencein contracts. I do not limit contracts to be of a particular form a priori, but instead deriveendogenously the structure of implementable contracts. I deal with this history depen-dence by following Bismut (1978) and changing state variables. Rather than directlyanalyzing how the agents choices affect the output process, I focus on how they alter itsassociated density process, which becomes the key state variable. This is analogous tothe shift in the moral hazard literature from focusing on the state space to instead con-sidering how the agents choices affect the probability of outcomes (see Stole (2001) fordiscussion). In my setting, the general history dependence in contracts then enters asrandomness in the coefcients of the density, which is signicantly easier to manage.

To characterize the agents optimality conditions, I then apply a stochastic maximumprinciple due to Bismut (1973). 9 Similar to the deterministic Pontryagin maximum prin-ciple, the stochastic maximum principle denes a Hamiltonian and expresses optimalityconditions as differentials of it. For the principals choice of a contract, either anothermaximum principle, such as Peng (1993), or dynamic programming can be used. Inthis paper, I apply dynamic programming as it lends itself more naturally to solution

methods. 10 In the end, the optimal contract is history dependent, but it can be writtenrecursively via the introduction of a small number of state variables. My results thusmake tractable the design of optimal contracts.

The rest of the paper is organized as follows. In the next section, I lay out the basicmodel. In Section 3, I formulate the agents problem, and I derive the agents optimalityconditions facing a given contract. In Section 4, I characterize the class of implementablecontracts, and in so doing establish the validity of my rst-order approach. In Section 5,I turn to the principals choice of an optimal contract, deriving necessary conditions foroptimality and discussing how to solve the principals problem explicitly. In Section 6 Isolve in detail a special case of the model, nding the optimal contracts under full infor-

mation, hidden actions, and hidden savings. Finally, Section 7 provides some concluding

8 Werning (2001b) describes how to verify ex-post whether a solution using the rst-order approach is incentivecompatible, perhaps numerically. Abraham and Pavoni (2003) provide an example of this approach. Kocherlakota(2004a) explicitly solves a special case using different means and provides a discussion. Similar state variables have been used in different contexts since Kydland and Prescott (1980).

9 The basic maximum principle is further exposited in Bismut (1978). More recent contributions are detailed inYong and Zhou (1999). Bismut (1975) and Brock and Magill (1979) give early important applications of the stochasticmaximum principle in economics.

10 An earlier version of the paper used a maximum principle to give some necessary conditions for an optimalcontract and to provide some partial characterizations.


5/35


remarks. An appendix provides proofs and technical conditions for all of the results inthe text.

2. THE MODEL2.1. The Basic Model

In this section I provide a brief description of the basic model I study. I work in acontinuous time stochastic setting, with an underlying probability space (, F , P ), onwhich is dened an (ny + nz )-dimensional standard Brownian. I partition the Brownianmotion into two independent Brownian motions W which is ny dimensional and Z whichis nz dimensional. Information is represented by a ltration {F t }, which is generated bythe Brownian motion (W t , Z t ) (suitably augmented). In this paper I consider a nitehorizon [0, T ] which may be arbitrarily long. In the application below, I let T .

The actions of the principal and the agent both affect the evolution of the state, whichis modeled as a diffusion process. For concreteness, I will refer to the state as output,denoted yt R n y , the agents action as effort denoted et A R n e , and the principalsaction as the payment denoted s t S R n s . The evolution of output is then given by:

dyt = f (t, y t , et , s t )dt + (t, y t , s t )dW t , (1)

with y0 given. Thus I have dened the drift as f : [0, T ] R n y A S R n y and thediffusion as : [0, T ] R n y S R n y n y .

Note that I do not allow the agent to directly affect the diffusion coefcient . But due

to the Brownian information structure, if the agent were to control the diffusion then hisaction would effectively be observed. For simplicity I also suppose that for given valuesof et and st the state evolution is Markov. More complex history dependence in the statecomes only through possible history dependence in the choices of the principal and theagent. Later I provide further specializations of this evolution, and introduce technicalrestrictions on the functions f and and the spaces A and S .

I consider two different cases of state observation. In the rst case, which I call hid-den actions the principal observes output y but not the agents effort e. In the secondcase, which I call hidden states (in addition to hidden actions), the agent also has accessto an additional state which the principal cannot monitor. The access to this additional

state gives the agent more exibility and may limit what incentives the principal is ableto provide through variations in the payment. I denote the hidden state m and for con-creteness I refer to it as the agents wealth (even though its evolution may be nonlinear),which is affected by a separate control c which I refer to as consumption. For notationalsimplicity only, I suppose that m is a scalar. I suppose that the principal can observe therelevant source of randomness in this state, which is the Brownian motion Z above. Theevolution of the hidden state is given by:

dm t = b(t, m t , ct , s t )dt + (t, m t )dZ t , (2)


6/35

6 NOAH WILLIAMS

with the initial level of wealth m0 given and observable to the principal. For simplicity, Iassume that the evolution of the hidden state is unaffected by output directly. Thus eventhough the principal does not observe the agents wealth, if he were to know the agentsconsumption decision he could deduce what it would be. 11

2.2. Hidden SavingsWhile I focus on the general formulation (2), a particular example motivates the ter-

minology I use. Here I suppose that the agent can borrow and lend and his asset positionis not observed by the principal, and thus is able to at least partially self-insure. In par-ticular, suppose that the agent has access to an asset with price P t which satises:

dP t = P t [P (t)dt + P (t)dZ t ] .

I assume that P and P are only functions of time, and that they satisfy regularity condi-tions which insure that this equation has a unique strong solution. The principal observesP t , which under these conditions is equivalent to observing Z t .12

Each instant the agent earns the return on his assets, gets a payment s t from the prin-cipal, and consumes ct B R + out of his assets. Thus the agents wealth mt follows:

dm t = ( m t P (t) + s t ct ) dt + m t P (t)dZ t , (3)

which is a special case of (2). If P = 0 then agent can only save (or borrow) at a knownrisk-free rate. My model thus generalizes the previous treatments of hidden savings inthe literature. However, as I will see below, my results may be more applicable whenthe drift b and/or the diffusion from (2) are strictly concave in m, as they are not here.Absent this, I require that the agents wealth directly enter the utility function. 13

2.3. More Detail and A Change of VariablesI now provide a bit more technical detail on the basic model. I let C n y and C n y + n z

be the spaces of continuous functions mapping [0, T ] into R n y and R n y + n z , respectively. Iadopt the convention of letting a bar over a variable indicate an entire time path on [0, T ].Note then that the time path of output y = {yt : t [0, T ]} is a (random) element of C n y ,and the principals observation path in the hidden state case (y, Z ) = {(yt , Z t ) : t [0, T ]}is an element of C n y + n z . I dene the ltration {Y t } to be the completion of the -algebragenerated by yt at each date, and similarly dene {Z t } as the ltration generated by

(yt , Z t ). In the hidden action case then, I dene the set of admissible contracts S a for11 Therefore I distinguish between the case of hidden states, as consdiered here, and that of partial information, in

which some random elements are not observed. Detemple, Govindaraj, and Loewenstein (2005) consider an intertem-poral partial information model.

12 See Liptser and Shiryaev (2000) for a statement of the necessary conditions for the existence of a strong solutionand a discussion of the equivalence of observing the state and the Brownian motion.

13 As noted above, the hidden state case can also capture the dependence of output on past actions as in Fernandesand Phelan (2000), say by allowing the drift f in (1) to depend on the hidden state m (which now captures a pasteffort stock) and to allow the drift b to depend on e. Such generalizations are also used in Williams (2006) for hiddeninformation problems. My more specialized formulation here allows for a simple comparison between hidden actionsand hidden states.


7/35


the principal to be those Y t -predictable functions sa : [0, T ] C n y S .14 This meansthat the contract may specify an action by the principal at date t that depends on thewhole past history of the observations of the state up to that date (but not on the future).Similarly, in the hidden state case, admissible contracts S s are those Z t -predictable func-tions s s : [0, T ] C n y + n z S . Note that in either case I allow almost arbitrary historydependence in the contract, where the relevant history is summarized by the informa-tion in the appropriate ltration. Since the agent optimizes facing the given contract,dene the set of admissible controls A for the agent as those F t -predictable functionse : [0, T ] C n y + n z A and c : [0, T ] C n y + n z B . I assume that the sets A and B can bewritten as the countable union of compact sets.

I now focus on the hidden state case, with hidden actions case as a special case. For agiven contract ss (t, y, Z ) the evolution of output (1) and wealth (2) can be written:

dyt = f (t, y, Z, e t )dt + (t, y, Z )dW t , (4)dm t = b(t, y, Z, m t , ct )dt + (t, m t )dZ t ,

where I have dened f (t, y, Z, e t ) = f (t, y t , et , s s (t, y, Z )) , (t, y, Z ) = (t, y t , s s (t, y, Z )) ,and b(t, y, Z, m t , ct ) = b(t, m t , ct , s s (t, y, Z )) . The history dependence in the contract in-duces history dependence in the state evolution. Thus (4) is a stochastic differentialequation of the functional type, with coefcients depending on elements of C n y + n z . Thisdependence would complicate a direct approach to the problem.

As in Bismut (1978), I can make the problem tractable by taking the key state variableto be the density of the output process instead of the output process directly. Different

effort choices by the agent change the distribution of output. Thus I can view the agentseffort choice as a choice of a probability measure over output. Similarly, in the hiddenstate case it is convenient to look at the evolution of the agents wealth under this newmeasure as well. As I show below, this reformulation of the problem leads to a simplecharacterization of the agents optimality conditions, and more importantly to a deriva-tion of the class of implementable contracts. Details of the change of measure are givenin Appendix A.1. 15

I now change my focus from the output process yt to the distribution over outcomes inC n y . The key state variable is the relative density t for the change of measure associatedwith different effort policies. As I show in Appendix A.1, it evolves as:

dt = t 1(t, y, Z ) f (t, y, Z, e t )dW 0t , (5)

with 0 = 1. Here W 0t is a Weiner process on C n y , and is interpretable as the distributionof output resulting from an effort policy which makes output a martingale. The covari-ation between output and the agents wealth is a key factor in the model. Thus is it also

14 See Elliott (1982) for a denition of predictability. Note that any left-continuous, adapted process is predictable.15 Similar ideas are employed by Elliott (1982) and Schattler and Sung (1993), who use a similar change of measure

in their martingale methods. The martingale approach does not apply in the hidden state case however, as the contractis conditioned directly on Z t and thus I cannot change the measure on that portion of the state.


8/35

8 NOAH WILLIAMS

useful to take xt = t m t as the relevant wealth state variable. Simple calculations showthat the evolution of this scaled wealth is given by:

dx t = t b(t, y, Z, x t / t , ct )dt + x t 1(t, y, Z ) f (t, y, Z, e t )dW 0t + t (t, x t / t )dZ t , (6)

with x0 = m0. Note that by changing variables from (y, m ) in (4) to (, x) in (5)-(6) Ihave changed the state evolution from a functional SDE to one with random coefcients.Instead the key states directly depending on their entire past history, the coefcients of the transformed state evolution depend on (y, Z ), which are random elements of theprobability space. This leads to substantial simplications, as I show below.

3. THE AGENTS PROBLEM

In this section I derive optimality conditions for the agent facing a given contract. Isuppose that the agent has a reservation utility V 0, and so I focus on contracts that satisfythis participation constraint. As in the previous section, I focus on the more generalhidden state case, with the hidden action case being a simple specialization.

I rst pose the agents decision problem. The agent has standard expected utilitypreferences dened over the states, his controls, and the principals actions (includingthe agents payment). Preferences take a standard time additive form, with ow utility uand a terminal utility v dened over the states. In particular, for an arbitrary admissiblecontrol policy (e, c) Aand a given contract ss (t, y, Z ), the agents preferences are:

V (e, c) = E e T

0 u(t, y t , m t , ct , et , s t )dt + v(yT , m T )

= E e T

0u(t, y, Z, m t , ct , et )dt + v(yT , m T )

= E T

0t u(t, y, Z, m t , ct , et )dt + T v(yT , mT ) .

Here the rst line uses the expectation with respect to the measure P e over output in-duced by the effort policy e, as discussed in Appendix A.1. The second line uses thenotation u(t, y, Z, m t , ct , et ) = u(t, y t , m t , ct , et , s s (t, y, Z )) , and the third uses the densityprocess dened above. The agents problem is to solve:

sup( e, c)A

V (e, c)

subject to (5)-(6). Any admissible control policy (e, c) that achieves the maximum iscalled an optimal control, and it implies an associated optimal state evolution (, x).

Under my change of variables, the agents problem is simply a control problem withrandom coefcients. I apply a stochastic maximum principle from Bismut (1973)-(1978)to derive the agents necessary optimality conditions. Analogous to the deterministic


9/35


Pontryagin maximum principle, I dene a Hamiltonian function H as follows:

H (t,y,Z,m,e,c,,p,Q,R ) = ( + Qm ) f (t ,y,Z,e ) + pb(t ,y,Z,m,c ) (7)

+ R (t, m ) + u(t ,y,Z,m,c,e ).

Here and Q are ny dimensional vectors, p is a scalar, and R is a nz dimensional vector.As in the deterministic theory, optimal controls maximize the Hamiltonian.

A key role in my analysis is played by the adjoint (or co-state) variables, whose evolu-tion is governed by differentials of the Hamiltonian. Thus I introduce the adjoint equa-tions corresponding to the states (t , x t ), which satisfy the following:

dq t = t f (t) + ( b(t) bm (t)m t ) pt + ( (t) m (t)m t ) R t + ( u(t) um (t)m t ) dt+ t (t)dW 0t + t dZ t (8)

q T = v(yT , mT ) vm (yT , mT )mT .dpt = bm (t) pt + Q t f (t) + Rt m (t) + um (t) dt + Qt (t)dW 0t + R t dZ t (9) pT = vm (yT , m T ).

For a given (e, c), I use the shorthand notation b(t) = b(t, y, Z, m t , ct ) and so on. Theseadjoint variables follow backward stochastic differential equations (BSDEs), as they havespecied terminal conditions but unknown initial values. 16 Below I show that the adjointprocess q t associated with the density process t corresponds to the agents optimal util-ity process, while the adjoint process pt associated with the scaled wealth xt characterizesthe agents shadow value in terms of marginal utility of the hidden state. Moreover,

these adjoint variables play a key role in characterizing the implementability of contracts below.In the following, I say that a process X t L2 if E

T 0 X

2t dt < . My rst result

gives the necessary conditions for optimality. As with all later propositions, requiredassumptions and proofs are given in Appendices A.2 and A.3, respectively.

Proposition 3.1. Suppose that Assumptions A.1, A.2, and A.3 hold, and that u also satises Assumption A.2. Let (e, c, , x) an optimal control-state pair. Then there existF t -adapted process (q t , t , t ) and ( pt , Q t , R t ), all in L2 (with t (t) and Qt (t) in L2), that satisfy (8) and(9). Moreover the optimal control (e, c) satises for almost every t [0, T ] almost surely:

H (t, y, Z, mt, e

t, c

t, t , pt , Q t , R t ) = max

(e,c )A BH (t, y, Z, m

t,e ,c , t , pt , Q t , R t ). (10)

Suppose in addition thatA and B are convex andf and u are continuously differentiable in (e, c).Then an optimal control (e, c) must satisfy for all (e, c) A B , almost surely:

H e(t, y, Z, mt , et , ct , t , pt , Q t , R t ) (e et ) 0 (11)H c(t, y, Z, mt , et , ct , t , pt , Q t , R t ) (c ct ) 0

16 See El Karoui, Peng, and Quenez (1997) for an overview of BSDEs in nance. In particular, these BSDEs dependon forward SDEs, as described in Ma and Yong (1999).


10/35

10 NOAH WILLIAMS

I stress that these are only necessary conditions for problem. As Mirrlees (1999) andRogerson (1985b) have shown, rst order conditions such as (11) may not be sufcient tocharacterize an agents incentive constraints, and so the set of implementable contractsmay be smaller than that characterized by the rst order conditions alone. However, Iestablish the validity of my rst-order approach in the next section.

Now I consider the simplications which result in the hidden action case. Here t isthe sole state variable, and thus I suppose that the agents preferences u and v dont de-pend on mt . I also suppose that the agent just consumes his payment from the principal,so I eliminate ct as well. These assumptions are simply made to make the notation moreconcise. In this case, the Hamiltonian (7) simplies to:

H (t,y,e, ) = f (t ,y,e ) + u(t ,y,e ), (12)

and the rst order condition (11) to:

f e(t ,y,e,s ) = ue(t ,y,e,s ). (13)

Thus there is a single adjoint process q t which simplies from (8) to:

dq t = t f (t) + u(t) dt + t (t)dW 0t (14)= u(t)dt + t (t)dW e

t

q T = v(yT ),

where the second line uses the change of measure for the optimal policy. This is a partic-ularly simple BSDE whose solution is easily seen to be the following:

q t = E e T

tu(t)dt + v(yT ) F t ,

so that q 0 = V (e), the agents optimal utility.Thus q t is the agents optimal utility process, the remaining expected utility at date

t when following an optimal control e. In the next section, I show that the adjointprocesses encode the necessary history dependence for implementable contracts. In thehidden action case, this simply means that the agents optimal utility process becomes a

state variable. As noted in the introduction, this is similar to the ideas of Abreu, Pearce,and Stacchetti (1986) and Spear and Srivastrava (1987), which have been widely appliedin the subsequent literature. In the hidden state case, the shadow value of the hiddenstate also becomes a state variable.

4. IMPLEMENTABILITY OF CONTRACTS

A key question in designing contracts is whether the principal can induce the agentto take desired actions by providing appropriate incentives. Thus far I have considered


11/35


the agents problem, taking as given a specied contract sa or ss . Now I characterizethe class of controls that can be implemented by the principal by appropriately tailoringthe contract. In particular, I suppose that associated with any contract is a target controlprocess (e, c), and I use the notation sa [e] and ss [e, c] to denote this association. 17 Thecontract thus gives a target level of expected utility V (e, c) to the agent, which must begreater than the reservation level V 0 for the contract to be acceptable(satisfy a participationconstraint). The issue is whether the agent actually chooses the effort and consumptionpolicies (e, c) when assigned the contract. I call a contract sa (resp. ss ) implementable if (e, c) is an optimal control when the agent faces the contract sa [e] (resp. s s [e, c]).

In this section I provide some of the main results of the paper, characterizing the classof implementable contracts in the hidden action and hidden state cases. Because theideas involved are cumulative, and each case is of interest in its own right, I characterizeimplementability separately in the two cases. In the hidden action case, the principal cancondition the contract on all relevant states for the agent, and I simply check whetherthe control is optimal given the state and the contract. To insure implementability, theprincipal must take account of the evolution of the optimal utility process. In the hiddenstate case, the agents controls also depend on the unobservable state, so I must verifythat the agent has no incentive to deviate from the principals target for it. In this case, theprincipal must also take into account both of the adjoint processes dened in (8)-(9), andthus must effectively consider both the agents optimal utility and the optimal marginalutility of wealth.

4.1. Hidden Actions

A contact sa

is contingent on the observations of output y. Thus the target level of out-put necessarily agrees with the actual level if the target effort policy e is implementable.I focus on interior target effort policies, and build in the incentive constraints via the rstorder condition (11) which thus reduces to an equality at e. Note that (11) and (12) leadto a representation for the target volatility process t :

t = ue(t, y, et ) f 1e (t, y, et ) (15)= ue(t, y t , et , s a (t, y))f 1e (t, y t , et , s

a (t, y)) (t, y, et , s at ).

Here the rst line follows from (11) assuming that f e is invertible, the second uses thedenitions of f and u, and the third introduces notation for the target volatility function.

Recalling the notion of an acceptable contract, I say that an acceptable contract sa

respects the adjoint equation if sa and its target control e imply a solution q of the BSDE (14)with volatility process t given by (15). This property is an analogue of standard promise-keeping constraints, as it insures that the policy is consistent with the utility evolution.Thus acceptable contracts which respect the adjoint equation lead to a representation of

17 Recall that in the hidden action case, I assume that the agent simply consumes his payment from the principal.


12/35

12 NOAH WILLIAMS

the terminal utility:

v(mT ) = q T = q 0

T

0u(t, y t , et , s at )dt +

T

0 t (t, y t , s at )dW

et , (16)

for some q 0 V 0.Similarly, with the specied volatility t the Hamiltonian from (12) can be represented

explicitly in terms of the target effort e:

H (t, e ) = (t, y t , et , s at )f (t, y t , e , sat ) + u(t, y t , e , s

at ). (17)

I say a contract sa has the maximization property if for almost every (t, y t ) [0, T ] R n y thefollowing holds almost surely:

H (t, et ) = maxeA

H (t, e ). (18)

This is a direct analogue of the maximum condition (10) with the given volatility process . It states that, with the specied adjoint process, the target control satises the agentsoptimality conditions, and so is an instantaneous incentive constraint. Later I providesimple conditions on the primitives of the model which insure that the maximizationproperty holds. With these denitions, I now have my main result in this case.

Proposition 4.1. Suppose that in addition the assumptions of Proposition 3.1, thatf 1e (t ,y,e,s )is invertible for every (t ,y,e,s ) [0, T ] R n A S , so that t in (15) is well-dened. Then acontract sa [e] S a is implementable in the hidden action case if and only if it: (i) is acceptable,

(ii) respects the adjoint equation, and (iii) has the maximization property.

Thus I have a characterized the validity of the rst-order approach in this dynamicsetting. I now give some sufcient conditions which simplify the matter even further by guaranteeing that the maximization condition holds. Natural assumptions are that(u,f ,b ) are all concave in (e, c) and that ue and f e have opposite signs, as increased effortlowers utility but increases output. Similarly, uc and bc have typically opposite signsas consumption increases utility but lowers wealth. These assumptions, which I stateexplicitly as Assumption A.4 in Appendix A.2, imply that the target adjoint process from (15) is positive. This allows us to provide the following sufcient conditions.

Corollary 4.1. In addition to the conditions of Proposition 4.1, suppose that Assumptions A.4 hold and the function H from (17) has a stationary point in A for almost every (t, y t ). Thena contract sa [e] S a is implementable in the hidden action case if and only if it: (i) is acceptable,(ii) respects the adjoint equation.

Under these conditions, I can reduce the implementability conditions to acceptabilityand compatibility with the adjoint equations. The key conditions are the natural curva-ture restrictions in Assumptions A.4, and thus my results here are quite general.


13/35


4.2. Hidden StatesWhen the principal cannot condition the contract on all of the states relevant for

the agent, some further conditions are needed to characterize implementable contracts.Proposition 3.1 above implies that the agents optimality conditions in this case includetwo adjoint processes, and so a contract ss must respect both of these. As above, thisinsures that the contract guarantees the agents optimal utility process. But I now alsomust guarantee the optimal shadow value of the hidden states, or else the agent mayhave an incentive to deviate from the target control policy, which I now denote (e, c). Inaddition, the principal now forms a target m for the agents wealth. Recall that the initialm0 is observed and is thus equal to m0, but from the initial period onward the principalconstructs the target as in (2):

dm t = b(t, m t , ct , s st )dt + (t, m t )dZ t .

Now the principal cannot directly distinguish whether m deviates from m .Similar to above, I say that an acceptable contract ss respects the adjoint equations if ss ,

its target control (e, c) and target hidden state m imply solutions (q, , ) and ( p, Q, R)of the adjoint equations (8)-(9). Thus, as in (15) above, each of the adjoint processes now becomes a function of (t, y, Z, m, e, c) under the contract. Thus I extend my representationof the Hamiltonian (7) with these particular adjoint processes as in (17):

H (t,m,e,c ) = ( t + Q t m) f (t, y t , e , s st ) + pt b(t,m,c,s st ) (19)+ R t (t, m ) + u(t, y t ,m,c,e,s st ).

Then parallel to (16) above, if an acceptable contract respects the adjoint equations (8)-(9)the terminal utility can be represented:

v(yT , mT ) = q T + pT mT = q 0 + p0m0 + T

0dq t +

T

0d( pm)t (20)

= q 0 + p0m0 T

0[( t + Qt m t ) f (t, y t , et , s st ) + u(t, y t , et , ct , s

st )]dt

+ T

0[ pt (t, m t ) + m t R t + t ]dZ t +

T

0[ t + Q t m t ](t, y t , s st )dW

0t .

So now V (e, c) = q 0 + p0m0.As above, I say a contract ss has the maximization property if for almost every (t, y t )

[0, T ] R ny the following holds almost surely:

H (t, m t , et , ct ) = max(e,c )A B

H (t, m t , e ,c). (21)

Now the maximization property effectively states that given the specied adjoint processes,if the agent has the target wealth level then the target control satises his optimality


14/35


15/35


Proposition 4.2. Under the assumptions of Proposition 4.1, an implementable contractss [e, c] S s in the hidden state case, with an interior target control (e, c) and target wealthm is (i) acceptable, (ii) respects the adjoint equations, and (iii) has the maximization property.If in addition, the Hamiltonian function H from (19) is concave in (m,e,c ) and the terminalutility function v is concave in m, then any admissible contract satisfying the conditions (i)-(iii)is implementable.

As above, simple sufcient conditions imply that the maximization property holds.

Corollary 4.2. In addition to the conditions of Proposition 4.2 (including the concavityrestrictions), suppose that Assumptions A.4 hold and the function H from (19) has a stationary point in A B for almost every (t, y t ). Then a contract ss [e, c] S s which (i) is acceptable and(ii) respects the adjoint equations is implementable.

Note that these sufcient conditions do not insure the additional concavity conditionsdiscussed above. Thus while my results in the hidden action case are quite general, myresults here cannot be stated directly in terms of the primitives of the model.

5. THE PRINCIPALS PROBLEM

I now turn briey to the problem of the principal who seeks to design an optimalcontract. Since there is little I can establish at this level of generality, I simply setup theprincipals problem here. The principal has his own objective function and takes theagents optimality conditions as constraints. My results above imply that I can summa-rize these constraints via a couple of additional state variables: the agents utility processin the hidden action case, and the utility and marginal adjoint processes in the hiddenstate case. With these additional states, the principals problem is essentially a Markovcontrol problem, with the complication that the new states are backward rather than for-ward stochastic differential equations. In general, one can apply a maximum principledue to Peng (1993), which handles such cases, to derive optimality conditions. In theapplication below, I consider an innite horizon version of the model where a dynamicprogramming approach is easier to apply.

Analogous to the agent, I suppose that for a given contract s in S a or S s and target

control (e, c) Athe principal has expected utility preferences given by:

J (s, e, c) = E T

0U (t, y t , m t , ct , et , s t )dt + L(yT , mT )

subject to the evolution of the states (1) and (2). In many formulations, the principalspreferences depend only output and the payment to the agent, but I also allow for thepossibility that the principal may care directly about the agents unobservable actionsand states. Also note that I allow the principal to either be risk-neutral or risk averse.


16/35

16 NOAH WILLIAMS

Since the principal must insure the agents participation constraint holds, I write theagents preferences as V (e, c, s) making explicit the dependence on the contract. For ac-ceptability, I must have V (e, c, s) V 0. I focus on implementable contracts, so I makeno distinction between targets and actual variables. I also consider a strong formula-tion of the problem, in which I take the probability space as given with xed Brownianmotions (W, Z ) (see Yong and Zhou (1999)).

In designing an optimal contract, the principal chooses a payment and target con-trols for the agent to maximize his own expected utility, subject to the constraint that thecontract be implementable. From Proposition 4.2 I can pose the principals problem asmaximizing J (s, e, c) subject to the state evolution and the adjoint equations.

In my analysis above, it was convenient to think of the principal choosing a contract ss

and target controls (e, c) which together imply a volatility process t . I now choose an al-ternative representation. Moreover, since I have assumed that is invertible I can equiv-alently reparameterize the problem in terms of t = t (t) and Qt = Qt (t) (throughoutthis section I suppress arguments of functions). Thus I solve for the agents actions viathe rst order conditions (11), which I now write as:

( + Qm )f e(t,y, e, s ) = (t,y,s )ue(t,y,m, c, e, s), (22) pbc(t,m, c, s) = uc(t,y,m, c, e, s ).

I assume that I can solve these equations for a pair of functions e = e(t,y,m,s, , Q, p)and c = c(t,y,m,s, , Q, p). In this case, I can write the state and adjoint equations as:

dyt = f (t)dt + (t)dW t (23)dm t = b(t)dt + (t)dZ tdq t = q(t) + t dW t + t dZ tdpt = p(t)dt + Q t dW t + R t dZ t ,

where I use shorthand notation for the coefcients once I substitute in the functions for eand c and evaluate at t. For example:

f (t) = f (t,y,m,s, , Q, p) = f (t,y, e(t,y,m,s, , Q, p), s).

In (23) y0 and m0 are given, and q 0 and p0 are to be determined subject to the constraints:

q 0 + p0m0 = V (e, c, s) V 0, q T = v(yT , m T ) vm (yT , m T )mT , pT = vm (yT , mT ). (24)

Thus the principals problem is to solve:

sup{s sS s }

J (s, e, c)

subject to (24), with the initial conditions for (y0, m 0), and the evolution (23). In general,this is a difcult optimization problem, which typically must be solved numerically. Inthe next section I study an example which can be solved explicitly.


17/35


6. A FULLY SOLVED EXAMPLE

While my methods can in principle lead to a full characterization of the optimal con-tracts in relatively general settings, they typically require numerical solutions in practice.In this section I show that the full information, hidden action, and hidden savings ver-sions of a model with exponential preferences and linear evolution is explicitly solvable.This allows us to fully describe the optimal contract and its implementation, as well asto directly characterize the distortions which are caused by the informational frictions.

6.1. An ExampleI now consider a simple model in which the principal hires the agent to manage a

risky project, with the agents effort choice affecting the expected return on the project.My s is a dynamic version of the model of Holmstrom and Milgrom (1987). In theirenvironment consumption and payments occur only at the end of the period and outputis i.i.d., while I include intermediate consumption (by both the principal and agent) andallow for persistence in underlying processes. In addition, mine is a closed system, whereeffort adds to the stock of assets but consumption is drawn from it. 19 I also consider anextension with hidden savings, in which the agent can save in a risk-free asset with aconstant rate of return. In this case, my results are related to the discrete time model of Fudenberg, Holmstrom, and Milgrom (1990), who show that the hidden savings problemis greatly simplied with exponential preferences. Much of the complications of hiddensavings comes through the interaction of wealth effects and incentive constraints, which Iabstract from here. However my results are not quite as simple as theirs, as the principalin my model is risk averse and the production technology is persistent. 20 I show that

hidden savings affects the contract by changing the effective rate of return that the agentcan access.

As in Holmstrom and Milgrom (1987), I assume that the principal and agent haveidentical exponential utility preferences only over consumption, while the agent hasquadratic nancial costs of effort:

u(c, e) = exp c e2

2, U (d) = exp( d).

For simplicity, I consider an innite horizon version of the model, thus letting T

in my results above. The evolution of the asset stock or cumulative output is linear withadditive noise:

dyt = ( Ay t + Be t ct dt ) dt + dW t (25)19 My preferences are also differ from Holmstrom and Milgrom (1987), as they consider time multiplicatively sepa-

rable preferences while I use time additively separable ones.20 In their model Fudenberg, Holmstrom, and Milgrom (1990) show that there are no gains to long term contracting,

and that an optimal contract is completely independent of history. The rst result relies on the risk neutrality of theprincipal, while the second relies on technology being history independent as well. Neither condition holds in mymodel, and I nd that the optimal contract is history dependent and that hidden savings alter the contract.


18/35


19/35


The optimal policies are thus:

ef i

= B, f i

(q ) = Aq

2 ,

cf i (q ) = B2

2

logA

log( q )

,

df i (y, q ) = log(J 0A)

+

log( q )

+ Ay.

The agents optimal effort is constant and his consumption does not depend directlyon output, but instead is linear in the log of the utility process. The principals consump-tion is linear in the log of the agents utility process and also linear in current output.These policies imply that the state variables evolve as follows:

dyt =2(A )

A +

2A4

dt + dW t

dq t = ( A)q t dt A

2 q t dW t .

Thus output follows an arithmetic Brownian motion with constant drift, while the utilityprocess follows a geometric Brownian motion. The expected growth rate of utility isconstant and equal to the difference between the subjective discount rate and the rateof return parameter A.

6.3. The Hidden Action CaseI now turn to the case where the principal cannot observe the agents effort et . Note

that the agents preferences are not additively separable in consumption and effort, andthus the sufcient conditions from Corollary 4.1 above are not satised. However it isstraightforward to verify that the agents Hamiltonian is concave in (c, e) so that Propo-sition 4.1 holds. Since the technology is linear, this is simply a consequence of the con-cavity of preferences. Therefore the agents effort level is determined by his rst ordercondition, which from (13) specialized to this model is:

B = e exp( (c e2

/ 2)). (27)

The principal must now choose contracts which are consistent with this effort choice bythe agent. The principals HJB equation now becomes:

J (y, q ) = maxc,d,e

exp( d) + J y(y, q ) [Ay + Be c d] + J q(y, q )[q + exp( (c e2/ 2))]

+12

J yy (y, q )2 + J yq (y, q ) (c, e)2 + 12

J qq(y, q ) (c, e)22 (28)


20/35

20 NOAH WILLIAMS

where I substitute = (c, e) using (27). The rst order conditions for (d,c,e) are then:

exp( d) = J y ,

J y J q exp( (c e2/ 2)) J yq 2 (c, e) J qq2 (c, e)2 = 0 ,

J y B + J qe exp( (c 1/ 2e2)) + J yq 21 + e2

e (c, e) + J qq2

1 + e2

e (c, e)2 = 0 .

A special feature of this example is that the value function and the optimal policiestake the same form as the full information case, albeit with different key constants. Inparticular, the value function is of the same form as above,

J (y, q ) = J 1

q exp( Ay)

for some constant J 1. The optimal policies are thus:

eha = e, ha (q ) = ekq

B ,

cha (q ) = (e)2

2

log k

log( q )

,

dha (y, q ) = log(J 1A)

+

log( q )

+ Ay,

where (e, k , J 1) are constants.Thus effort is again constant, consumption is again linear in the log of the utility

process, and the principals consumption is also linear in output. Note also that u(c(q ), e) =kq . Using this along with the form of the value function in the rst order conditions for(e, c) determines the equations that eand k must satisfy:

A k + 2A2ek/B 222(e)2k2/B 2 = 0 , (29) BA + ek 2A(1 + (e)2)k/B 22e(1 + (e)2)k2/B 2 = 0 .

These can be solved to get e(k):

e= B3A + 2BAk

B 2A + 2 2k 2 , (30)

then substituting this back into the rst equation in (29) gives an expression for k. Noticethat for = 0 these collapse to the full information solution e = B and k = A, whichof course they should since with no noise the agents action is no longer hidden. Theconstant J 1 is determined from the HJB equation after substituting in the optimal policies,which implies:

J 1 = 1kA

expA + k 2

A e B

e

2+ 22

A2

ek

B +

(e)2k2

AB 2.


21/35


Under the optimal contract, the evolution of the states is also similar to the full informa-tion case, now being:

dyt = (A + k 2)A + 2 A2 e

kB + (e

)2

k2

AB 2 dt + dW t

dq t = ( k)q t dt ek

B q t dW t .

So again output follows a Brownian motion with drift, while the utility process followsa geometric Brownian motion.

While the form of the policy functions is the same as in the full information case, theconstants dening them differ. Solving for the values of the constants is a simple nu-merical task, but explicit analytic expressions are not available. To gain some additionalinsight into the optimal contract, I expand eand k in 2 around zero. From (29) and (30)I have the following approximations:

e = B 2AB

+ o(4)

k = A 2A22 + o(4).

In turn, substituting these approximations into cha (q ) gives:

cha (q ) = cf i (q ) + o(4).

Thus the rst order effects (in the shock variance) of the information frictions are a re-

duction in effort put forth, but no change in consumption. I also show below that k

isthe agents effective rate of return, and thus this return decreases with more volatility.Moreover, the effect on effort depends on the parameters in a simple way: the higherthe rate of return parameter A or risk aversion parameter the larger is the reduction,while larger productivity values B lead to smaller reductions in effort. Below I plot theexact solutions for a parameterized version of the model and show that the results are inaccord with these rst order asymptotics. The information friction leads to a reductionin effort, but little effect on consumption.

6.4. The Hidden Saving CaseWhen the agent has access to hidden savings, some of the analysis is altered. In par-

ticular, I must now distinguish between the principals payment s t and the agents con-sumption ct . Thus the evolution of the asset stock in the project yt and the agents wealthm t are:

dyt = ( Ay t + Be t s t dt ) dt + dW tdm t = ( Am t + s t ct )dt.

Thus I assume that the agent can borrow and lend at a constant risk-free rate A A. Imostly focus on the case when A = A, in which case the agents saving is redundant, as


22/35


23/35


Based on my results in the previous section and the proportionality of utility andmarginal utility with exponential preferences, I now guess that the value function can bewritten in the form:

J (y,q,p) = F ( p/q )

q exp( Ay)

for some univariate function F . To simplify notation, I dene z = p/q as the argumentof this function, noting that z ( , 0). Thus the optimality condition for p0 translatesto F (z 0 ) = 0 , or p0 = ( F ) 1(0)q 0. Thus the initial marginal utility is proportional to theinitial promised utility.

Then I use the guess in the HJB equation (31), take the rst order condition for thedividend, and obtain a policy function similar to the hidden action case:

dhs (y,q,z ) = log(F (z )A)

+ log( q )

+ Ay.

For the volatility of marginal utility, I nd it convenient to normalize Qt = Qt pt . Thenthe optimal choices of e and Q are determined by the rst order conditions:

AF (z )(B e) + F (z )z 2 + 4 F (z )z + 2 F (z ) 2z 2e

B 2

+ A(F (z )z + F (z ))2z B

(2F (z ) + F (z )z ) Q2z 2

B = 0, (32)

F (z )z Q AF (z ) (2F (z ) + F (z )z )ez

B = 0. (33)

Thus given F , these equations determine the optimal policies as a function of z alone:ehs (z ) and Q(z ). Substituting these into (31), I nd that F is the solution of the secondorder ODE:

F (z ) = AF (z ) (F (z )z + F (z ))( + z/ ) AF (z ) Be hs (z ) ehs (z )2

2 +

log( AzF (z )/ )

+( A)zF (z ) + ( F (z )z 2 + 4 F (z )z + 2 F (z ))2ehs (z )2z 2

2B 2 + ( F (z )z + F (z ))

2Ae hs (z )z B

+ 22A2

2 F (z ) + F (z )z 2Q(z )2

2 AF (z )z Q(z ) (2F (z ) + F (z )z 2)ehs (z ) Q(z )z 2

B ,

with the boundary conditions limz0 F (z ) = and limz F (z ) = . Note that theseconditions imply that the principals utility falls without bound as the agents currentconsumption or future promised utility increase without bound.

Thus I have reduced the solution of the three dimensional PDE for the value functionto an ODE. The ODE is sufciently complex that an explicit solution for arbitrary z doesnot seem feasible. However, I can obtain more explicit results for the value and the


24/35

24 NOAH WILLIAMS

policies at the optimal starting value z 0 . In particular, note that at z 0 (33) implies

Q(z 0) =

ehs (z 0 )z 0B

and thus (32) leads to:

e0 = ehs (z 0) =

B3A 2BAz 0B 2A + 2 2(z 0 )2/

.

Then using these in the ODE we get:

F (z 0 ) = Az 0

expA z 0 2

A e0 B

e02

+ 22A2

+ e0z 0B

+ (e0z 0 )2

AB 2.

Note that if z

0 = k then these results agree with the hidden action case, as ehs

( k ) =e from (30) and F ( k ) = J 1. But this is indeed the optimal choice of z 0 when A = k.This can be veried analytically by implicitly differentiating the ODE for F and using theexpressions for ehs (z 0 ) and Q(z 0 ) to solve for z 0 . However for A = k I cannot analyticallysolve for the optimal starting value, and so I numerically solve the ODE for F (z ), thenchoose z 0 as the minimizing value. 23 For the parametrization below, I set A = A and ndthat z 0 = A to sufcient numerical precision. I have also veried the same result inalternative parameterizations with A k. In all of these cases the agent has a higherreturn on savings than his effective return from the project. Thus it appears that at leastfor these cases, the optimal choice is z 0 = A.

These results are also signicant, since they imply that z t is in fact constant over time,so that utility and marginal utility remain proportional forever under the contract. Thiscan be seen by using Itos lemma to get the evolution of z t :

dz t = (z t / + A)z t + 2ehs (z t )2z 2t

B Q(z t )

ehs (z t )z tB

dt+ z t Q(z t ) ehs (z t )z t

BdW t .

Using the expression for Q(z 0 ) above, note that dz t = 0 if z 0 = A. Thus in this case itturns out that the contract does not need to condition on the additional marginal utilitystate. These results are clearly a consequence of the absence of wealth effects, and thus

special to exponential utility.6.5. Comparing the Different Cases

Table 1 summarizes the optimal contracts under full information, hidden actions, andhidden savings (assuming z 0 = A). As weve seen, the policy functions bear a strongresemblance to each other. In each case, effort is constant, consumption is linear in the log

23 These calculations are straightforward but lengthy and thus are not included here. Details can be furnished uponrequest. For A = k when differentiating the ODE for F there is a term in F (z 0 ) which cancels in the expression,allowing for explicit solution for z 0 . But for arbitrary A the term does not cancel.


25/35


TABLE 1.

Comparison of the policies for the optimal contracts under full information, hidden actions, and hidden savings.

Full Information Hidden Action Hidden Saving (z

0 = A

)Effort, e B e (k) = B

3 A + 2 BAkB 2 A +2 2 k 2 e

( A) = B3 A + 2 BA A

B 2 A +2 2 A 2

Consumption, c B2

2 log A

log( q )

e ( k ) 2

2 log k

log( q )

e ( A ) 2

2 log A

log( q )

Dividend, d log( J 0 A ) + log( q )

+ Ay log( J 1 A )

+ log( q )

+ Ay log( F ( A ) A )

+ log( q )

+ Ay

of the promised utility q , and the principals dividend is linear log( q ) and the currentassets y. The consumption policies also depend negatively on the agents effective rateof return on assets, which is A with full information, k with hidden actions, and A withhidden savings. As weve already seen, the main effect of the hidden action case relative

to full information is the reduction of effort. Weve also seen that to rst order there is noeffect on consumption. This can be seen in the consumption policies, as effort falls but sodoes the effective return as k < A, leading to at least partially offsetting effects.

The main difference between the hidden saving and the hidden action cases is theeffective rate of return on the agents savings. When A = k then the optimal policies inthe hidden action and hidden savings cases coincide. With A > k , determining exactlywhether the effort increases or decreases under hidden savings relative to the hiddenaction case is complicated by the fact that I do not have an explicit expression for k.However if A < 4k then effort is decreasing in k, and hence consumption is as well. 24Since for small shocks weve seen that k is smaller than A by a term proportional to

the shock variance, this will certainly hold for small shocks. Thus for small enough ,with hidden savings the agent puts forth less effort and consumes less than with hiddenactions alone. This is clear since cha (q ; k) is decreasing in k if e(k) is, as effort falls butthe effective return rises. The relationships between the policies and rates of return onsaving will be also be useful in the next section, where I discuss implementation of theallocations.

In Figure 1 I plot the consumption functions for effort and consumption in the fullinformation, hidden action, and hidden savings cases for a particular parameterizationof the model. I set A = 0.15, B = 0.5, = 0.1, = 2 , A = A, and show the results forvarying . In particular, the left panel plots ef i = B, eha = e(k), and ehs (z 0 ) = e(A)

versus , while the right panel plots cf i

, cha

, and chs

(all evaluated at q = 1) versus .Clearly as 0 the cases all agree, as there is no information friction. Compared to fullinformation, both effort and consumption fall under hidden actions. Moreover, for small the approximation results in the hidden action case above are accurate, as effort falls but consumption is relatively unaffected. When the agent has access to hidden savings,

24 Simple calculations givede (k )

dk =

2 B 3 A (A 4k) 2 4 BA 2 k(B 2 A + 2 2 k 2 )2

,

which is clearly negative if A < 4k .


26/35

26 NOAH WILLIAMS

0 1 2 3

0.3

0.35

0.4

0.45

0.5

e f i

, e h

a , e

h s

Effort

FIHA

HS

0 1 2 30.98

1

1.02

1.04

1.06

1.08Agent Consumption ( q =1)

c f i

, c h

a , c

h s

FIGURE 1. The agents effort and consumption policies for different noise levels .

consumption and effort fall further. In addition, these effects are all monotone in thelevel of noise .

I can also use the policy functions to provide explicit expressions for the inefciencywedges, discussed by Kocherlakota (2004b) and others. These measure how the infor-mation frictions lead the consumption and labor allocations to be distorted for incentivereasons. In particular, suppose that the agent can borrow and lend at the same risk freerate A as the principal. When the agents saving is observable, the principal is able totax it and drive its return down to A = k. However if the principal cannot observe the

agents savings, then of course he cannot tax it and so A = A. Thus in the hidden action(but observable saving) case, the contract introduces an intertemporal wedge K , a gap

between the intertemporal marginal rate of substitution and the marginal return on as-sets. This is simply given by the tax rate which drives the after-tax rate of return downto k:

K (k) = 1 kA

.

Of course, under hidden savings K (A) = 0 .By varying the payment to the agent for incentive reasons, the optimal contract also

induces a wedge between the agents marginal productivity of labor and his marginal

rate of substitution between consumption and effort, a labor wedge L . In my example,since the marginal product of effort is simply B and the marginal rate of substitution ise( A), the labor wedge is simply:

L ( A) = 1 e( A)

B .

Thus both the intertemporal and labor wedges are constant and (negatively) propor-tional to the key terms discussed above. In Figure 2 I plot the intertemporal and labor


27/35


0 1 2 30

0.1

0.2

0.3

0.4

0.5

L

Labor Wedge

0 1 2 30

0.05

0.1

0.15Intertemporal Wedge

K

HA

HS

FIGURE 2. The labor and intertemporal wedges for different noise levels .

0 1 2 30

0.005

0.01

0.015

0.02

0.025

0.03Reduction in Principal Dividend

HAHS

FIGURE 3. Reduction in the principals dividend relative to the full information case for different noise levels .

wedges in the hidden action and hidden savings cases. The labor wedge is especially sig-nicant for this parameterization, as for > 1.5 it is comparable to labor income tax ratesof more than 30% under hidden actions and roughly 40% under hidden savings. The in-

tertemporal wedge is smaller, of course being identically zero under hidden savings andattening out near an effective tax rate of 13% under hidden actions.Finally, I provide a measure of the cost of the information frictions for the principal.

As Table 1 shows, the different informational assumptions affect the principals dividendonly through the additive constants. Thus for each level of promised utility q and assets y,the principals consumption differs by a constant amount depending on the informationstructure. In Figure 3 I plot the reduction relative to the full information case in thelevel of the principals dividend under hidden actions and hidden savings. The costsare relatively low, attening out near 0.025 units of consumption under hidden actions


28/35

28 NOAH WILLIAMS

and 0.03 units under hidden savings. These numbers are somewhat difcult to interpret,as the levels are somewhat arbitrary since output is growing over time. As an example,with y = 0 and q = 1 the dividend is exactly equal to the constant term, and these levelreductions imply a 1.5-2% fall in the principals dividend. Of course with greater outputand lower levels of promised utility, the proportional decline is much lower.

6.6. Implementing the Optimal AllocationsThus far I have focused on the direct implementation of contracts, with the princi-

pal directly assigning consumption to the agent. However in this section I show howthe same outcomes from the optimal contracts can be achieved by giving the agent aperformance-related payment but then allowing him to invest in a risk-free asset, andthus to choose his own consumption.

As in the hidden state formulation above, I now distinguish between consumption ct

and the principals payment to the agent st . The payment is set at the agents optimalconsumption under the contract, so st = s(q t ; A) = cha (q t ; A), where q t is the agentsutility process under the contract and A = k in the hidden action case and A = A inthe hidden savings case. I now directly verify that the optimal contract, expressed as apayment s(q t ; A) which in turn embodies an associated target effort e( A) and evolutionfor q t and a tax rate K ( A) = 1 A/A on saving, is implementable. Of course wevealready shown that contract is directly implementable in the hidden action case. Butrecall that in the hidden saving case my sufcient conditions guaranteeing the validityof the rst order approach failed, and thus it is important to know that the contractI deduced is indeed implementable. Moreover, this is a natural form of the contract

specication.For the agent, the state q t is simply part of the specication of the payment under thecontract. From his vantage point, it evolves as:

dq t = ( A)q t dt e( A) A

B q t dW e

t

= ( A)q t dt e( A) A

B q t

dyt (Ay t + Be( A) s(q t ; A) dt )dt

= A e( A) A(et e( A)) q t dt e( A) A

B q t dW t .

Here I use the fact that dW e

t is the driving Brownian motion under the optimal contractfor the principals information set. Moreover the agent begins with zero wealth m0 = 0 ,which then evolves as:

dm t = [1 K ( A)]Am t + s(q t ; A) ct dt

= Am t + e( A)2

2

log( A)

log( q t )

ct dt


29/35


Thus the agents value function V (q, m) solves the HJB equation:

V (q, m) = maxc,e

exp( (c e2/ 2)) + V m (q, m) Am + s(q ; A) c

+ V q(q, m)q [ A e( A) A(et e( A))] + 12

V qq(q, m)q 222e( A)2 A2

B 2 (34)

It is easy to verify that the agents value function is given by V (q, m) = q exp( Am).Substituting this into the HJB equation, and taking rst order conditions for (c, e) gives:

exp( (c e2/ 2)) = V m = Aq exp( Am),e exp( (c e2/ 2)) = e( A) AqV q = e( A) Aq exp( Am).

Taking ratios of the two equations gives e = e

, and thus the target effort level is imple-mentable. Using this, I can solve the rst equation to obtain:

c = e( A)2

2

log A

log( q )

+ Am.

Substituting this into the wealth equation gives dm t = 0. Thus, if the agent begins withm t = 0 , then he will remain at zero wealth, will consume the optimal amount underthe contract c = cha (q ; A), and will attain the value V (q, 0) = q . Therefore the policiesassociated with the optimal contracts can be implemented with this payment and savingstax scheme.

7. CONCLUSION

In this paper I have established several key results for dynamic principal-agent prob-lems. By working in a continuous time setting, I were able to take advantage of powerfulresults in stochastic control, which led to some sharp conclusions. I characterized theclass of implementable contracts in dynamic settings with hidden actions and hiddenstates via a rst-order approach. I also provided the rst general proof of the validity of this rst-order approach in a dynamic environment. I showed that implementable con-tracts must respect some additional state variables: the agents utility process and the

agents shadow value (in marginal utility terms) of the hidden states. I then developeda constructive method for solving for an optimal contract. The optimal contract is in gen-eral history dependent, but can be written recursively in terms of the state and the dualadjoint variables associated with the agents utility and shadow value processes.

In our application, we show that as in Holmstrom and Milgrom (1987) the optimalcontract is linear in our setting. However now the payment is linear in an endogenousobject, the logarithm of the agents promised utility under the contract. Moreover, weshow that the main effect of hidden actions is in the distortion of effort, along with asmaller effect on the agents implicit rate of return under the contract. Introducing hid-


30/35

30 NOAH WILLIAMS

den savings as well eliminates this second distortion, and thus increases the effort dis-tortion.

Overall, the methods developed in the paper are tractable and reasonably general,making a class of models amenable to analysis. As mentioned in the introduction, therehas been a renewed interest in dynamic models with hidden information in the past fewyears. This provides much potential scope for applications of my results.

APPENDIX

A.1. DETAILS OF THE CHANGE OF MEASURE

I now introduce more technical detail associated with the change of measure used in Section 2.3. I start by workingwith the induced distributions on the space of continuous functions, which I take to be the underlying probability

space. Thus I let the sample space be the space C n y + n z

, and let (W 0t , Z t ) = (t ) be the family of coordinatefunctions, and F 0t = {(W 0t , Z t ) s t} the ltration generated by (W 0t , Z t ). I let P be the Wiener measure on (, F 0T ),

and let F t be the completion of F 0t with the null sets of F 0T . This denes the basic (canonical) ltered probability space,on which I have the Brownian motions (W 0t , Z t ). Later I will construct another probability measure in which W 0t isreplaced with a different Brownian motion, but as contracts are conditioned on Z t I keep it xed. I now introducesome regularity conditions on the diffusion coefcient in (1) following Elliott (1982).

Assumptions A.1. Denote by ij (t ,y, s ) the (i, j ) element of the matrix (t ,y, s ) . Then I that for all i, j , for some xed K independent of t and i, j :

1. ij is continuous,

2. | ij (t ,y, s ) ij (t, y , s ) | K (|y y | + |s s |),

3. (t ,y, s ) is non-singular for each (t ,y, s ) and |( 1 (t ,y, s )) ij | K .

Of these, the most restrictive is perhaps the nonsingularity condition, which requires that noise enter every partof the state y. Under these conditions, there exists a unique strong solution to the stochastic differential equation:

dy t = (t, y, Z )dW 0t , (A.1)

with y0 given. I can interpret this as a benchmark evolutionof output under an effort policy e0 whichmakes f (t, y, Z, e 0t ) =0 at each date. Different effort choices alter the evolution of output by changing the distribution over outcomes in C n y .

Now I state some required regularity conditions on the drift function f .

Assumptions A.2. I assume that for some xed K independent of t :1. f is continuous,

2. |f ( t ,y,e,s ) | K (1 + |y | + |s |).

Note that these imply the predictability, continuity, and linear growth conditions on the concentrated function f

which are assumed by Elliott (1982). Then for e A I dene the family of F t -predictable processes:

t ( e) = exp t

0

1 (v, y, Z ) f (v, y, Z, e v )dW 0v 12

t

0| 1 (v, y, Z ) f (v, y, Z, e v ) |2 dv .

Under the conditions above, t is an F t -martingale (as the assumptions insure that Novikovs condition is satised)with E [T ( e)] = 1 for all e A. Thus by the Girsanov theorem, I can dene a new measure P e via:

dP edP

= T ( e),

and the process W et dened by:

W et = W 0t

t

0

1 (v, y, Z ) f (v, y, Z, e v )dv


31/35


is a Brownian motion under P e . Thus from (A.1), its clear that the state follows the SDE:

dy t = f (t, y, Z, e t )dt + (t, y, Z )dW et . (A.2)

Hence each effort choice e results in a different Brownian motion. Note that t dened above (suppressing e) satises t = E [T |F t ], and thus is the relative density process for the change of measure.

A.2. ASSUMPTIONS FOR RESULTS

Here I list some additional regularity conditions for my results. The rst is a basic differentiability assumptionused for the agents problem.

Assumption A.3. The functions (u,v,b , ) are continuously differentiable in m .

The next set of assumptions guarantees that the maximization property is satised.

Assumptions A.4. Suppose that u is separable in e and c and concave in (e, c ) , f is concave in e and b is concavein c, and that u e f e 0 and u c bc 0.

The following set of assumptions provide some regularity conditions which are needed for the principals problemin the hidden action case. I impose the conditions on the composite functions f and u (which clearly imply conditionson f and e as well as on e), as well as the principals preference functions U and L . Recall that in this case U isindependent of c and m .

Assumptions A.5. For some xed K independent of t I assume:

1. f , , u,U, and L are continuous, and they are continuously differentiable with respect to all variables except possibly t .

2. The derivatives of f , , and u are bounded.

3. The derivatives of U are bounded by K (1 + |y | + |s |) , and the derivative of L is bounded by K (1 + |y |) .

Further extensions of Assumptions A.5 are necessary for the principals problem in the hidden state case.

Assumptions A.6.

1. The functions p and q are continuous, and are continuously differentiable with respect to all variables except possibly t .The derivatives of p and q are bounded.

2. The derivatives of U are bounded by K (1 + |y | + |m | + |s |), and the derivative of L is bounded by K (1 + |y | + |m |).

A.3. PROOFS OF RESULTS

Proof (Proposition 3.1). This follows by applying the results in Bismut (1973)-(1978) to my problem. Under theassumptions, the maximum principle applies to the system with ( , x ) as the state variables. I now illustrate thecalculations leading to the Hamiltonian (7) and the adjoint equations (8)-(9). I rst dene the stacked system:

X t = t

x t, t =

q t pt

, t = t tQ t R t

.

Then note from (5)-(6) that X t satises (suppressing arguments):

dX t = t0b

dt + t 1 f 0m t

1 f dW 0tdZ t

= M (t )dt + ( t )[dW 0t , dZ t ]

Thus, as in Bismut (1978), the Hamiltonian for the problem is:

H = M + tr( ) + u = H,


32/35


33/35


where the term in Q t comes from the change in measure from the target control policy e to an arbitrary admissiblepolicy e. Hence by the concavity of v I have:

E e [v(T ) v(T )] E e

T

0 [ pt t + R t t [u m (

t ) +

Q t (f (t ) f (

t ))] t ]dt (A.5)

Then I proceed as in Proposition 4.1 and note that for any (e, c) A I have:

V ( e, c) V ( e, c) = E e T

0[u ( t ) u ( t )]dt +

T

0[ pt ( t ) + m t R t + t ]dZ t

+ E e T

0[ t + Q t m t ] (t )dW et + v(T ) v(T )

= E e T

0[u ( t ) u ( t ) + ( t + Q t m t )( f (t ) f ( t ))]dt

+ E e T

0[ t + Q t m t ] (t )dW et + v(T ) v(T )

E e T

0[u ( t ) u ( t ) + [ t + Q t m t ]f ( t )]dt

+ E e T

0[ pt t + R t t [u m ( t ) + Q t (f (t ) f ( t ))] t ]dt

= E e T

0[u ( t ) u ( t ) u m ( t ) t + ( t + Q t m t )( f (t ) f ( t ))]dt

+ E e T

0 pt [b( t ) b( t ) bm ( t ) t ] + R t [ (t ) ( t ) m ( t ) t ] dt= E e

T

0[H

(t ) H ( t ) H m ( t ) t ]dt 0.

Here the rst equality uses (20), the second equality uses the denitions of the change of measure between P e and P e

along with the martingale property of the stochastic integral with respect to Z

(which coincides under the two mea-sures). The rst inequality uses (A.5), while the next equality uses the denitions of t and t . The following equalityuses the denition of H function in (19), and the nal result follows from its concavity as in (A.3). Thus since (e, c)was arbitrary, we get that (e, c) is an optimal control, and the contract is thus implementable.

Proof (Corollary 4.1 and 4.2). In either case, the stated assumptions insure that H is concave in e and H

is concave in (e, c ). This is clear from (15) in the hidden action case. The parallel rst order conditions give theresult in the hidden state case, where the separability insures that I can simply look at the separate rst order con-ditions. But by assumption, each function has a stationary point on the feasible set, which is then a maximum.

References

Abraham, A. and N. Pavoni (2003). Efcient Allocations with Moral Hazard and Hid-den Borrowing and Lending. Working paper, University College London.

Abreu, D., D. Pearce, and E. Stacchetti (1986). Optimal Cartel Equilibria with ImperfectMonitoring. Journal of Economic Theory 39, 251269.

Abreu, D., D. Pearce, and E. Stacchetti (1990). Toward a Theory of Discounted RepeatedGames with Imperfect Monitoring. Econometrica 58, 10411063.

Allen, F. (1985). Repeated Principal-Agent Relationships with Borrowing and Lending.Economics Letters 17, 2731.


34/35

34 NOAH WILLIAMS

Bismut, J. M. (1973). Conjugate Convex Functions in Optimal Stochastic Control. Jour-nal of Mathematical Analysis and Applications 44, 384404.

Bismut, J. M. (1975). Growth and the Optimal Intertemporal Allocation of Risks. Journalof Economic Theory 10, 239257.

Bismut, J. M. (1978). Duality Methods in the Control of Densities. SIAM Journal onControl and Optimization 16, 771777.

Brock, W. A. and M. J. Magill (1979). Dynamics Under Uncertainty. Econometrica 47,843868.

Cadenillas, A., J. Cvitani c, and F. Zapatero (2005). Dynamic Principal-Agent Problemswith Perfect Information. Working paper, USC.

Chari, V. V., P. J. Kehoe, and E. McGrattan (2004). Business Cycle Accounting. NBERWorking Paper No. 10351.

Cole, H. L. and N. Kocherlakota (2001). Efcient Allocations with Hidden Income andHidden Storage. Review of Economic Studies 68, 523542.

Cvitani c, J., X. Wan, and J. Zhang (2006). Continuous-Time Principal-Agent Problemswith Hidden Action and Lump-Sum Payment. Working paper, USC.

Detemple, J., S. Govindaraj, and M. Loewenstein (2005). Wealth-Robust IntertemporalIncentive Contracts. Working paper, Boston University.

Doepke, M. and R. M. Townsend (2006). Dynamic Mechanism Design with HiddenIncome and Hidden Storage. Journal of Economic Theory 126, 235285.

El Karoui, N., S. Peng, and M. C. Quenez (1997). Backward Stochastic DifferentialEquations in Finance. Mathematical Finance 7, 171.

Elliott, R. J. (1982). Stochastic Calculus and Applications. New York: Springer-Verlag.Fernandes, A. and C. Phelan (2000). A Recursive Formulation for Repeated Agencywith History Dependence. Journal of Economic Theory 91, 223247.

Fudenberg, D., B. Holmstrom, and P. Milgrom (1990). Short-Term Contracts and Long-Term Agency Relationships. Journal of Economic Theory 51, 131.

Golosov, M., N. Kocherlakota, and A. Tsyvinski (2003). Optimal Indirect and CapitalTaxation. Review of Economic Studies 70, 569587.

Holmstrom, B. and P. Milgrom (1987). Aggregation and Linearity in the Provision of Intertemporal Incentives. Econometrica 55, 303328.

Hopenhayn, H. A. and J. P. Nicolini (1997). Optimal Unemployment Insurance. Journal

of Political Economy 105, 41238. Jewitt, I. (1988). Justifying the First-Order Approach to Principal-Agent Problems.

Econometrica 56, 11771190.Kocherlakota, N. R. (2004a). Figuring Out The Impact of Hidden Savings on Optimal

Unemployment Insurance. Review of Economic Dynamics 7, 541554.Kocherlakota, N. R. (2004b). Wedges and taxes. American Economic Review (Papers and

Proceedings) 94, 109113.


35/35

williams (2006) (on dynamic principal agenta problems in continuous time)

Documents