Dynamic optima with

forward-looking constraints

SGZ MACRO WK3, DAY 3

Lecture 1 part B

1

forward-looking constraints

Core reading:

Marcet and Marimon,

“Dynamic contracts”

Consider dynamic

optimization problem

• Features: constraint on controls (c,i) treated

as inequality

• State evolution equation: treated as equality

SGZ MACRO WK3, DAY 3

Lecture 1 part B

2

0 1 0{ } ,{ }0

1

max [ ( )]

. . [ ( ) ] 0

[(1 ) ] 0

t j j t j j

j

t jc kj

t j t j t j

t t t

u c

s t f k c i

k i k

β

δ

∞ ∞+ = + + =

∞

+=

+ + +

+

− − ≥

− + − =

∑

Two approaches to optimization• Lagrangian approach

0 0

( ) [ ( ) ]j j

t t j t j t j t j t j

j j

L u c p f k c iβ β∞ ∞

+ + + + +

= =

∞

= + − −∑ ∑

∑

SGZ MACRO WK3, DAY 3

Lecture 1 part B

3

• Note value, which we don’t usually bother to include but can (e.g. indirect utility)

0 1 0 0 1 0

1

0

{ } ,{ } { } ,{ }

[(1 ) ]

min maxt j j t j j t j j t j j

j

t j t j t j t j

j

t tp c k

k i k

v Lλ

β λ δ

∞ ∞ ∞ ∞+ = + + = + = + + =

∞

+ + + + +

=

+ − + −

=

∑

• Dynamic programming

1, , 1( ) max { ( ) ( )}

. ( ) 0

t t tt c i k t t

t t t

v k u c v k

s t f k c i

β

δ

+ += +

− − ≥

− + − =

SGZ MACRO WK3, DAY 3

Lecture 1 part B

4

1

1

1 1

, ,

(1 ) 0

{ ( ) [ ( ) ]

[(1 ) ] ( )}

( ) min maxt t t t

t t t

t t t t t

t t t t t

t p c k

k i k

u c p f k c i

k i k v k

v k λ

δ

λ δ β

+

+

+ +

− + − =

= + − −

+ − + − +

=

t

t

L

L

Observation

• Lagrangians play a role in two ways.

• First, in basic specification of the optimization problem in the sequential setting, with kt being a parameter of that problem so the indirect utility function – the Lagrangian maximized with respect to actions/states and minimized with respect to shadow prices is v(k )

SGZ MACRO WK3, DAY 3

Lecture 1 part B

5

with respect to shadow prices is v(kt)

• Second, as part of the “routine” of maximizing the rhs of the Bellman equation, give the accumulation constraint.

• These are different Lagrangians: one is defined over four infinite sequences and one is defined over four functions.

• Never-the-less, it is intuitive that there should be a recursive relationship of the form on the next page

Recursive Problem• What’s the outcome?

• Why? Because at the “saddle point” the

constraint binds or the multiplier is zero so

1, ,

1 1

( ) min max {[ ( ) ( ( ) )

((1 ) ) ( )}

t t t tt p c k t t t t t

t t t t t

v k u c p f k c i

k i k v k

λ

λ δ β

+

+ +

= + − −

+ − + − +

SGZ MACRO WK3, DAY 3

Lecture 1 part B

6

constraint binds or the multiplier is zero so

that the additional term not present in

traditional Bellman equations is zero, so

that this is a natural generalization of the

standard Bellman equation.• Note: “as if” standard “momentary objective” is augmented with

additional variables and is a “momentary Lagrangian”.

MM paper

• Describes a powerful recursive

methodology for problems with forward-

looking constraints

• Provides a practical recipe for macro

SGZ MACRO WK3, DAY 3

Lecture 1 part B

7

• Provides a practical recipe for macro

policy design (eg inflation tax) and micro

policy design (next lecture): don’t have to

figure things out on a case-by-case basis,

but just cast into MM form.

MM structure: objective

• Depends on endogenous state (x),

exogenous state (s) and action/control (a)

• Takes discounted present-value form

SGZ MACRO WK3, DAY 3

Lecture 1 part B

8

0

0

(1) [ ( , , )]j

t t t

t

E r x a sβ∞

=

∑

MM structure: constraint set

• Form

( , )t t ta A x s∈

SGZ MACRO WK3, DAY 3

Lecture 1 part B

9

• Motivation: general enough to have

actions (controls) that are discrete (e.g.

light switch)

MM structure: state equation

• Takes the form

1 1( , , )t t t tx l x a s+ +=

SGZ MACRO WK3, DAY 3

Lecture 1 part B

10

• Note difference from our standard setup:

future exogenous state (s) can affect

future endogenous state (x).

Standard DP in MM

• Bellman equation

1 1( , ) max { ( , , ) ( , ) | }tt t a t t t t t tv x s r x a s Ev x s sβ + += +

SGZ MACRO WK3, DAY 3

Lecture 1 part B

11

• Policy function: a(x,s)

1 1

. . ( , )

( , , )

t t t

t t t t

s t a A x s

x x x a s+ +

∈

=

MM structure:

forward-looking constraints

• Use flexible specification

1 2

1

( , , ) ( , , )jN

j n j

t t t t t n t n t n

n

g x a s E g x a sβ + + +

=

+ ∑

SGZ MACRO WK3, DAY 3

Lecture 1 part B

12

1,2.... 0for j k and t= ≥

Problem 1:

• Maximize objective (1) with respect to

sequence of contingent actions

• Subject to:

– State equation

SGZ MACRO WK3, DAY 3

Lecture 1 part B

13

– State equation

– Action/control constraint set

– Forward-looking constraints

A Saddlepoint Problem (2)

{ } { }

0

1

min max ( , , , , )

. . ( , , )

( , )

t t

t

a t t t t t

t

t t t t

h x a s

s t x l x a s

a A x s

γ β µ γ∞

=

+ =

∈

∑

SGZ MACRO WK3, DAY 3

Lecture 1 part B

14

1 1

0

0 0

( , )

( , , )

0

0

,

t t t

t t t t

t

a A x s

s

x s given

µ ϕ µ γ

γ

µ

+ +

∈

=

≥

=

A Saddlepoint Functional Equation

• Generalization of Bellman’s equation

0 ( , )

1

( , , ) min max { ( , , , , ) ( ', ', ') | }

( , , )

a A x s

t t t t

W x s h x a s EW x s s

x l x a s

γµ µ γ β µ≥ ∈

+

= +

=

SGZ MACRO WK3, DAY 3

Lecture 1 part B

15

• Optimal actions depend on new “dynamic” multipliers, a(x,µ,s). So do “point in time” multipliers, γ(x,µ,s).

• So dynamic multipliers µ evolve through time, yielding changes in actions

1

1 1

( , , )

( , , )

t t t t

t t t t

x l x a s

sµ ϕ µ γ

+

+ +

=

=

What’s the modeling gain?

• Problems with forward-looking constraints can be readily expressed in a related recursive form by two devices:

– Introduction of lagged multipliers, which can be zero set to zero

– The Saddle Point functional equation (recursive solution) methodology

SGZ MACRO WK3, DAY 3

Lecture 1 part B

16

methodology

• Concretely, if we are looking to generate a set of one-step-ahead-constraints of the form EtF(yt+1,yt, xt+1,xt)=0 then this is a very desirable methodology. We don’t need to take an entire sequence of derivatives, but just a vector of derivatives. We should have a related “envelope theorem” that would allow us to proceed just as we did earlier in week 1

Key steps in using MM

• The general transition from (1) to (2) that

they make depending on the type of

constraints (horizon N of constraints)

SGZ MACRO WK3, DAY 3

Lecture 1 part B

17

• The specific process of casting a model in

the form of (1).

One-step ahead case (section 2.2)

• User friendly result

1 2

( , , , , )

( , , ) ( , , ) ( , , )

h x a s

r x a s g x a s g x a s

µ γ

γ µ= + +

SGZ MACRO WK3, DAY 3

Lecture 1 part B

18

• Notation: row vector of multipliers, column

vector of constraint components

• Note: could dispense with γ multipliers entirely

and just work with µ and µ’

( , , )sϕ µ γ γ=

Why this form?

To see, form Lagrangian and manipulate

0 1 2 1 1 1

0 0

0 1 2 1 1 1

0 0

0 1 1 2

0

[ ( , , )] [ [ ( , , ) ( , , )]

{[ ( , , )] [ [ ( , , ) ( , , )]}

{[ ( , , )] [ ( , , )] [ [ ( ,

j j

t t t t t t t t t t t

t t

t t

t t t t t t t t t t

t t

t t t

t t t t t t t t t

t

L E r x a s g x a s E g x a s

E r x a s g x a s g x a s

E r x a s g x a s g x

β β γ β

β β γ β

β β γ β γ β

∞ ∞

+ + += =

∞ ∞

+ + += =

∞

−=

= + −

= + −

= + +

∑ ∑

∑ ∑

∑0 0

, )]}t t

t t

a s∞ ∞

= =

∑ ∑

SGZ MACRO WK3, DAY 3

Lecture 1 part B

19

• First derivation is “lagged multiplier” as in KP

• Second derivation is h function as in MM

0t= 0 0

1

0 1 2

0

1 0

0

{ [ ( , , ) ( , , ) ( , , )]}

0

t t

t

t t t t t t t t t t t

t

t t

with

E r x a s g x a s g x a s

with and

γ

β γ µ

µ γ µ

= =

−

∞

=

−

=

= + +

= =

∑

Infinite horizon forward-looking

constraints (section 2.1)

1 2

1

( , , ) ( , , )

1, 2.... 0

n

t t t t t n t n t n

n

g x a s E g x a s

for j k and t

β∞

+ + +=

+

= ≥

∑

SGZ MACRO WK3, DAY 3

Lecture 1 part B

20

• User friendly result

1 2

( , , , , )

( , , ) ( , , ) ( , , )

( , , )

h x a s

r x a s g x a s g x a s

s

µ γ

γ µ

ϕ µ γ γ µ

= + +

= +

Lagrangian

0

0

0 1 2

[ ( , , )]

[ ( ( , , ) ( , , ))]

t

t t t

t

t n

t t t t t t n t n t n

L E r x a s

E g x a s E g x a s

β

β γ β

∞

=

∞ ∞

+ + +

= =

=

+ +

∑

∑ ∑

SGZ Macro Week 3, Day 3,

Lecture 2

21

0 1t n= =

∑ ∑

Altering the last part a la Marcet-Marimon

0 2

0 1

0 2,

0 1

2 3

0 0 2, 1 2, 2 2, 3

2

0 1 2, 2 2, 3

[ ( , , )]

{ [ ]}

{ [ ....]}

{ [ ....]}

Note shorthand

t n

t t t n t n t n

t n

t n

t t n

t n

t t t

t t

E E g x a s

E g

E g g g

E g g

β γ β

β γ β

γ β β β

βγ β β

β γ β β

∞ ∞

+ + += =

∞ ∞

+= =

+ + +

+ +

=

= + +

+ + +

+ + + +

∑ ∑

∑ ∑

SGZ Macro Week 3, Day 3,

Lecture 2

22

2 2

0 2 2, 3 2, 4

0 2, 1

{ [ ....]} ....t t

t

E g g

g

β γ β β

βγ

+ +

+

+ + + +

= 2 3

0 1 2, 2 0 1 2 2, 3

2 2

0 2 1 2, 1 2 2, 2 3 2, 3

1 0

[( ) ] [( ) ] ...

...

with and 0

t t

t t t t

t t t

g g

g g g g

β γ γ β γ γ γ

µ βµ β µ β µ

µ µ γ µ

+ +

+ + +

+

+ + + + + +

= + + + +

= + =

So the MM form is

1 2

( , , , , ) ( , , )

( ( , , ) ( ( , , )

t t t t t t t t t

t t t t t t t t

h x a s r x a s

g x a s g x a s

µ γ

γ µ

=

+ +

SGZ Macro Week 3, Day 3,

Lecture 2

23

0

( , , )

0

t t t t tsϕ µ γ µ γ

µ

= +

=

Summary

• MM provide a powerful method for thinking

about economies with forward-looking

constraints

SGZ MACRO WK3, DAY 3

Lecture 1 part B

24

• Can be applied to compute commitment

optima in a wide range of optimal policy

problems