supplemental materials to dynamic random …supplemental materials to dynamic random utility by mira...
TRANSCRIPT
Supplemental Materials to DYNAMIC RANDOM UTILITY
By
Mira Frick, Ryota Iijima, and Tomasz Strzalecki
June 2017
COWLES FOUNDATION DISCUSSION PAPER NO. 2092S
COWLES FOUNDATION FOR RESEARCH IN ECONOMICS YALE UNIVERSITY
Box 208281 New Haven, Connecticut 06520-8281
http://cowles.yale.edu/
Supplementary Appendix to Dynamic Random Utility
Mira Frick, Ryota Iijima, and Tomasz Strzalecki
F Proof of Theorem 0
F.1 Preliminaries
In this section we prove Theorem 0 which extends the characterizations of REU representationsin Gul and Pesendorfer (2006) and Ahn and Sarver (2013) to allow for an arbitrary separablemetric space X of outcomes. Refer to section 2.1 of the main text for all relevant notation andterminology. Throughout, we fix some y∗ ∈ X and let RX = 0 × RXry∗ denote the set ofutility functions u in RX that are normalized by u(y∗) = 0.
We first define the static analog of S-based representations introduced in Appendix A:
Definition 11. An S-based REU representation of ρ is a tuple (S, µ, Us, τss∈S) such that
(i). S is a finite state space and µ is a probability measure on S such that supp(µ) = S
(ii). for each s ∈ S, the utility Us ∈ RX is nonconstant and Us 6≈ Us′ for s 6= s′
(iii). for each s ∈ S, the tie-breaking rule τs is a proper finitely-additive probability measureon RX endowed with the Borel σ-algebra
(iv). for all p ∈ ∆(X) and A ∈ A,
ρ(p;A) =∑s∈S
µ(s)τs(p,A),
where τs(p,A) := τs(u ∈ RX : p ∈M(M(A,Us), u)).
Analogous arguments as for the DREU part of Proposition 5 yield the equivalence of S-basedREU representations and static REU representations.
Proposition 6. Let ρ be a stochastic choice rule on A. Then ρ admits an REU representationif and only if it admits an S-based REU representation.
Proof. Analogous to Proposition 5 (i).
Thus, Theorem 0 is equivalent to the following result, which we prove throughout the restof this section.
Theorem 4. The stochastic choice rule ρ on A admits an S-based REU representation(S, µ, Us, τss∈S) if and only if ρ satisfies Axiom 0.
1
Remark 1. Note that because X may be infinite, continuity of each Us in the representationis not directly implied by linearity. However, in constructing an evolving utility representationin Section C, we can make use of Axiom 6 (iii) (Continuity) to show that each Ust is continuous(see Lemma 6). In the static setting, an alternative approach is to impose the following axiom.
As in Section 3.3, let A∗ denote the collection of menus without ties, i.e., the set of allA ∈ A such that for any p ∈ A and any sequences pn →m p and Bn →m A r p, we havelimn→∞ ρ(pn;Bn ∪ pn) = ρ(p;A).
Axiom 11 (Continuity). ρ : A∗ → ∆(∆(X)) is continuous.
Here A is endowed with the Hausdorff topology induced by the Prokhorov metric π on∆(X), and A∗ with the relative topology. We have the following proposition. Since we do notmake use of this result anywhere in the paper, the proof is omitted but available on request.
Proposition 7. Suppose ρ admits an S-based REU representation (S, µ, Us, τss∈S). Theneach utility Us is continuous if and only if ρ additionally satisfies Axiom 11. N
Additional notation: For any Y ⊆ X, let A(Y ) := A ∈ A : ∀p ∈ A, supp(p) ⊆ Y ⊆ Adenote the space of all menus consisting only of lotteries with support in Y . Note that for eachA ∈ A, there is a finite Y such that A ∈ A(Y ). We denote by ρY the restriction of ρ to A(Y ),which can be seen as a map from A(Y ) to ∆(∆(Y )). If y∗ ∈ Y , we write RY := 0×RY ry∗.
For any A ∈ A(Y ) and p ∈ ∆(X), let NY (A, p) := u ∈ RY : p ∈ M(A, u) and letN+Y (A, p) := u ∈ RY : p = M(A, u). Note that NY (p, p) = N+
Y (p, p) = RY and thatNY (A, p) = N+
Y (A, p) = ∅ if p /∈ A. Let N (Y ) := NY (A, p) : A ∈ A(Y ) and p ∈ ∆(X),N+(Y ) := N+
Y (A, p) : A ∈ A(Y ) and p ∈ ∆(X).We will consider both the Borel σ-algebra on RY and its subalgebra F(Y ) that is generated
by N (Y ) ∪ N+(Y ). A finitely-additive probability measure νY on either of these algebras iscalled proper if νY (NY (A, p)) = νY (N+
Y (A, p)) for any A ∈ A(Y ) and p ∈ ∆(X). WheneverY = X, we omit Y from the description of NY (A, p), N+
Y (A, p), N (Y ), N+(Y ), and F(Y ).
F.2 Proof of Theorem 4: Sufficiency
F.2.1 Outline
The proof proceeds as follows:
(i). In section F.2.2, we use conditions (i)–(iv) of Axiom 0 and Theorem 2 in Gul and Pe-sendorfer (2006) to construct, for each finite Y ⊆ X, a proper finitely-additive probabilitymeasure νY on F(Y ) representing ρY , in the sense that ρY (p;A) = νY (NY (A, p)) for allA, p. Given the fact that each ρY is derived from the same ρ, it is easy to check that thefamily F(Y ), νY is Kolmogorov consistent. We can then find a proper finitely-additiveprobability measure ν on F extending all the νY (and hence representing ρ).
(ii). The support of ν is defined by
supp(ν) :=(⋃V ∈ F : V is open and ν(V ) = 0
)c.
2
In section F.2.3, we use part (v) of Axiom 0 to show that supp ν is finite (up to positiveaffine transformation of utilities) and contains at least one non-constant utility function.While Axiom 0 (v) is similar to the finiteness axiom in Ahn and Sarver (2013), this steprequires more work in our setting. A key technical challenge is that unlike in Ahn andSarver, it is not clear in our infinite outcome space setting how to normalize utilities toensure that N(A, p)-sets are compact. Compact sets C have the useful property (usedrepeatedly by Ahn and Sarver) that if C∩supp ν = ∅, then ν(C) = 0. Lemma 22 exploitsthe geometry of N(A, p)-sets to show that this property continues to hold for N(A, p)-setsin our setting, even though they are not compact.
(iii). In section F.2.4, we proceed in a similar way to the proof of Theorem S3 in Ahn andSarver (2013)(again using Lemma 22 to circumvent technical difficulties). Letting S :=s1, . . . , sL denote the equivalence classes of nonconstant utilities in supp ν, we findseparating neighborhoods Bs ∈ F of each s such that ν(Bs) > 0. We then define µ(s) =
ν(Bs) and τs(V ) = ν(V ∩Bs)ν(Bs)
and show that this yields an S-based REU representation ofρ.
F.2.2 Construction of ν
In this section, we construct a proper finitely-additive probability measure ν on F that repre-sents ρ, i.e., such that for all A ∈ A and p ∈ A, we have
ρ(p;A) = ν(N(A, p)) = ν(N+(A, p))).
First consider any finite Y ⊆ X with y∗ ∈ Y . By Axiom 0 (i)–(iv) (Regularity, Linearity,Extremeness, and Mixture Continuity), Theorem 2 in Gul and Pesendorfer (2006) ensures thatthere is a proper finitely-additive probability measure νY on FY such that
ρY (p;A) = νY (NY (A, p)) = νY (N+Y (A, p))
for all A ∈ A(Y ) and p ∈ A.
Claim 4. For any finite Y ′ ⊇ Y 3 y∗, (νY′,F(Y ′)) and (νY ,F(Y )) are Kolmogorov consistent,
i.e., for any E ∈ F(Y ), we have
νY′(E × RY ′rY ) = νY (E). (29)
Proof. To see this, note first that the LHS of (29) is well-defined, since E × RY ′rY ∈ FY ′ byLemma 21 (iv). Note next that by Lemma 21 (iii), E is of the form
⋃ni=1NY (Ai, pi)∩N+
Y (Bi, qi)for some finite n and Ai, Bi ∈ A(Y ). Let E ′ be obtained from E by replacing each NY (Ai, pi)with N+
Y (Ai, pi). By Lemma 21 (ii), E ′ =⋃ni=1 N
+Y (Ci, ri) for some family Ci ⊆ A(Y ).
Moreover, since both νY and νY′
are proper, we have that νY (E) = νY (E ′) and νY′(E ×
RY ′rY ) = νY′(E ′ × RY ′rY ). Hence, it suffices to prove that νY
′(E ′ × RY ′rY ) = νY (E ′). For
this, it is enough to show that for any collection of sets N+1 , . . . , N
+n ∈ N+(Y ) := N+(A, p) :
A ∈ A(Y ), we have νY (⋃ni=1N
+i ) = νY
′(⋃ni=1 N
+i × RY ′rY ). We prove this by induction. For
3
the base case, note that for any N+(A, p) ∈ N+(Y ), we have
νY′(N+(A, p))× RY ′rY ) = ρY
′(p,A) := ρ(p;A) =: ρY (p;A) = νY (N+(A, p)).
Suppose next that the claim is true whenever m < n. Then
νY (m+1⋃i=1
N+i ) = νY (
m⋃i=1
N+i ) + νY (N+
m+1)− νY (m⋃i=1
(N+i ∩N+
m+1)) =
νY′(m⋃i=1
N+i × RY ′rY ) + νY
′(N+
m+1 × RY ′rY )− νY ′(m⋃i=1
(N+i ∩N+
m+1)× RY ′rY ) =
νY′(m+1⋃i=1
N+i × RY ′rY ),
where the second equality follows from the inductive hypothesis and the fact that N+i ∩N+
m+1 ∈N+(Y ) by Lemma 21 (ii).
Now define ν on F by setting ν(E) := νY (projRYE) for any finite Y 3 y∗ such thatE = projRYE×RXrY and projRYE ∈ FY . By Lemma 21 (iv) such a Y exists. Moreover, givenKolmogorov consistency of the family νY Y⊆X , this is well-defined. Finally, it is immediatethat ν is a proper finitely-additive probability measure and that ν represents ρ.
F.2.3 Finiteness of supp ν
The support of a finitely-additive probability measure ν is defined by
supp(ν) :=(⋃V ∈ F : V is open and ν(V ) = 0
)c.
The next lemma invokes Axiom 0 (v) (Finiteness) to show that the support of ν constructed inthe previous section contains finitely many equivalence classes of utility functions and containsat least one nonconstant function. We use 0 to denote the unique constant utility function inRX .
Lemma 18. Let K be as in the statement of the Finiteness Axiom and let Pref(∆(X)) denotethe set of all preferences over ∆(X). Then
#%∈ Pref(∆(X)) :% is represented by some u ∈ supp(ν) r 0 = L,
where 1 ≤ L ≤ K.
Proof. We first show that L ≤ K. If not, then we can find utilities u1, ..., uK+1 ⊆ supp(ν)such that each ui is non-constant over X and ui 6≈ uj for all i 6= j. By Lemma 13, we can finda menu A = pi : i = 1, . . . , K + 1 ∈ A such that ui ∈ N+(A, pi) for each i. Take any B ⊆ Awith |B| ≤ K. Then pi 6∈ B for some i.
Fix any sequences pin →m pi and Bn →m B. By definition, this means that there existsr ∈ ∆(X) and αn → 0 such that pin = αnr + (1− α)pi for all n, and that for each q ∈ B there
4
exists Bq ∈ A and βn(q) → 0 such that Bn =⋃q∈B(βn(q)Bq + (1− βn(q))q) for all n. Now,
B and each Bq are finite, and ui is linear with ui · pi > ui · q for all q ∈ B. Hence, there isN such that for all n ≥ N , ui · pin > u · qn for all qn ∈ Bn. Thus, ui ∈ N+(pin ∪ Bn, p
in)
for all n ≥ N . But since ui ∈ supp(ν) and N+(pin ∪ Bn, pin) is an open set in F , the
definition of supp(ν) then implies that ν(N+(pin ∪ Bn, pin)) > 0 for all n ≥ N . But then
ρ(pin; pin ∪Bn) = ν(N+(pin ∪Bn, pin)) > 0 for all n ≥ N , contradicting Finiteness.
Next we show that L ≥ 1. Indeed, if L = 0, then for any A ∈ A with |A| ≥ 2 and forany p ∈ A, we have (N(p,A) r 0) ∩ supp ν = ∅. By Lemma 22 below, this implies thatν(N+(p,A)) = 0 for any p ∈ A. But since ν represents ρ, ρ(p;A) = ν(N+(p,A)) for any p ∈ A,so we have
∑p∈A ρ(p;A) = 0, which is a contradiction.
F.2.4 Constructing the REU representation
Let %1, . . . ,%L denote all the preferences represented by some non-constant utility in supp(ν),where by Lemma 18 we know that L is finite and L ≥ 1. For each i = 1, . . . , L, pick someui ∈ supp ν representing %i. For any u ∈ RX , let [u] := u′ ∈ RX : u′ ≈ u. By Lemma13, we can find A := p1, . . . , pL ∈ A such that ui ∈ N+(A, pi) for all i = 1, . . . , L. LetBui := N+(A, pi) for all i. By construction, [ui] ⊆ Bui and Bui ∩Buj = ∅ for j 6= i. Moreover,by the definition of supp(ν), we have ν(Bui) > 0 for each i, since Bui ∈ F is open andui ∈ Bui ∩ supp(ν) 6= ∅.
Let S := u1, . . . , uL and define the function µ : S → [0, 1] by
µ(s) = ν(Bs) for each s ∈ S.
We claim that µ defines a full-support probability measure on S. For this it remains to showthat
∑s µ(s) = 1. Since
∑s µ(s) =
∑s ν(Bs) = ν(
⋃s∈S Bs), it suffices to prove the following
claim:
Lemma 19. ν(⋃s∈S Bs) = 1.
Proof. It suffices to prove that ν(RX r⋃s∈S Bs) = 0. Note that RX =
⋃Li=1 N(A, pi), since
A = pi, . . . , pL. Thus,
RX r⋃s∈S
Bs ⊆L⋃i=1
(N(A, pi) rN+(A, pi)).
By finite additivity of ν, this implies that
ν(RX r⋃s∈S
Bs) ≤L∑i=1
ν(N(A, pi) rN+(A, pi)) = 0,
where the last inequality follows from properness of ν.
Next, we define a set function τs : F → R+ for each s ∈ S by setting
τs(V ) :=ν(V ∩Bs)
ν(Bs)
5
for each V ∈ F . Since ν(Bs) > 0 for all s ∈ S, this is well-defined. Moreover, since ν is aproper finitely-additive probability measure on F , so is τs.
Note that for all A ∈ A and p ∈ ∆(X), u ∈ RX : p ∈ M(M(A, s), u) = N(M(A, s), p) ∈F , so τs(u ∈ RX : p ∈ M(M(A, s), u)) is well-defined. The next lemma will allow us tocomplete the representation:
Lemma 20. For each s ∈ S, A ∈ A, and p ∈ A,
ν(N(A, p)) =∑s∈S
µ(s)τs(u ∈ RX : p ∈M(M(A, s), u)).
Proof. We first show that for each s ∈ S, supp τs r 0 = [s]. To see that [s] ⊆ supp τs r 0,consider any u ∈ [s] and any open V ∈ F such that u ∈ V . By Lemma 21 (iii), V is a finiteunion of finite intersections of sets in N ∪N+. Hence, since each element of N ∪N+ is closedunder positive affine transformations so is V . Thus, u ∈ V implies s ∈ V . But then V ∩Bs ∈ Fis open and contains s, and hence ν(V ∩Bs) > 0 since s ∈ supp ν. This proves u ∈ supp τsr0.
To see that supp τsr0 ⊆ [s], consider any u 6= 0 such that u /∈ [s]. It suffices to show thatthere exists an open V ∈ F such that u ∈ V and τs(V ) = 0. If u ≈ s′ for some s′ ∈ S r s,then V = Bs′ is as required since Bs′∩Bs = ∅ and u ∈ Bs′ . If there is no s′ ∈ Srs such thatu ≈ s′, then u /∈ supp ν. But then there exists an open V ∈ F such that u ∈ V and ν(V ) = 0,so also τs(V ) = 0.
By Lemma 23 below, this implies that τs(N(A, p)) = τs(N(M(A, s), p)) for any A ∈ A andp ∈ A. This implies that for any A ∈ A and p ∈ A∑
s∈S
µ(s)τs(u ∈ RX : p ∈M(M(A, s), u)) =∑s∈S
µ(s)τs(N(M(A, s), p))
=∑s∈S
µ(s)τs(N(A, p))
=∑s∈S
ν(N(A, p) ∩Bs)
= ν(N(A, p) ∩⋃s∈S
Bs)
= ν(N(A, p),
where the last equality follows from Lemma 19.
For any s ∈ S = u1, . . . , uL, we write Us := s. We claim that (S, µ, Us, τss∈S) is anS-based REU representation of ρ. Indeed, by construction, Us is non-constant for all s, Us 6≈ Us′for any distinct s, s′ ∈ S, and µ is a full-support probability measure on S. Moreover, each τs isa proper finitely-additive probability measure on RX endowed with the algebra F . By standardarguments (cf. Rao and Rao (2012)), we can extend τs to a proper finitely-additive probabilitymeasure on the Borel σ-algebra on RX . Finally, Lemma 20 and the fact that ν represents ρimplies that for all A ∈ A and p ∈ A, we have ρ(p;A) =
∑s∈S µ(s)τs(p,A), as required.
6
F.3 Proof of Theorem 4: Necessity
Suppose that ρ admits an S-based REU representation (S, µ, Us, τss∈S). We show that ρsatisfies Axiom 0. Observe first that for any finite Y ⊆ X with y∗ ∈ Y , (S, µ, Us Y , τs Y s∈S)constitutes an S-based REU representation of ρY , where Us Y denotes the restriction of Usto Y and τs Y is given by τs Y (B) = τs(B × RXrY ) for any Borel set B on RY . Thus, byTheorem S3 in Ahn and Sarver (2013), ρY satisfies Regularity, Linearity, Extremeness, andMixture Continuity.
To show that ρ satisfies Regularity, consider any p ∈ A ⊆ A′. Pick a finite Y ⊆ X withy∗ ∈ Y such that A,A′ ∈ A(Y ). By definition, ρ(p;A) = ρY (p;A) and ρ(p;A′) = ρY (p;A′).Hence, by Regularity for ρY , we have ρ(p;A) ≥ ρ(p;A′), as required. Similarly, we can showthat ρ satisfies Linearity, Extremeness, and Mixture Continuity by using the fact that for eachfinite Y , each ρY satisfies these axioms.
Finally, to show that ρ satisfies Finiteness, let K := |S| and consider any A ∈ A. For eachs ∈ S, pick any qs ∈ M(A,Us), and define B := qs : s ∈ S. Note that |B| ≤ K. If B = A,then Finiteness is trivially satisfied. If B ( A, then pick any p ∈ A r B. We can pick a largeenough finite Y ⊆ X such that each Us is non-constant on Y and Us Y 6≈ Us′ Y for any distincts, s′ ∈ S. Let r ∈ ∆(Y ) be given by r(y) := 1
|Y | for each y ∈ Y . For each s ∈ Y , pick any
ys ∈ argmaxy∈Y Us(y). Note that Us(ys) > Us(r). Define Bn := n−1nB + 1
nys : s ∈ S and
pn := n−1np + 1
nr. Then Bn →m B and pn →m p. Moreover, for all large enough n, we have
Us(n−1nqs + 1
nys) > Us(p
n) for each s ∈ S. Thus, ρ(pn; pn ∪Bn) = 0, proving Finiteness.
F.4 Additional Lemmas for Section F
F.4.1 Properties of N(A, p) sets
Lemma 21. Fix any X ′ ⊆ X with y∗ ∈ X. For any collection S, we let U(S) denote the setof all finite unions of elements of S.
(i). If E ∈ N (X ′) (resp. E ∈ N+(X ′)), then Ec ∈ U(N+(X ′)) (resp. Ec ∈ U(N (X ′)).
(ii). If E1, E2 ∈ N (X ′) (resp. E1, E2 ∈ N+(X ′)), then E1 ∩ E2 ∈ N (X ′) (resp. E1 ∩ E2 ∈N+(X ′)).
(iii). F(X ′) is the set of all E such that E =⋃`∈LM` ∩ N` for some finite index set L and
M` ∈ N (X ′), N` ∈ N+(X ′) for each ` ∈ L.
(iv). F(X ′) is the set of all E for which there exists a finite Y ⊆ X ′ with y∗ ∈ Y and EY ∈ F(Y )such that E = EY × RXrY .
Proof.
(i): If E = N(A, p) ∈ N (X ′), then Ec =⋃q∈ArpN
+(p, q, q) ∈ U(N+(X ′)) if p ∈ A
and Ec = RX′ ∈ U(N+(X ′)) if p /∈ A. Similarly, if E = N+(A, p) ∈ N+(X ′), then Ec =⋃q∈ArpN(p, q, q) ∈ U(N (X ′)) if p ∈ A and Ec = RX′ ∈ U(N (X ′)) if p /∈ A.
(ii): If N(A1, p1), N(A2, p2) ∈ N (X ′), then N(A1, p1) ∩ N(A2, p2) = N(12A1 + 1
2A2,
12p1 +
12p2) ∈ N (X ′). The same argument goes through replacing all instances of N with N+.
7
(iii): By standard results, F(X ′) can be described as follows: Let F0(X ′) denote the setof all elements of N (X ′) ∪ N+(X ′) and their complements. Let F1(X ′) denote the set ofall finite intersections of elements of F0(X ′). Then F(X ′) is the set of all finite unions ofelements of F1(X ′). By part (i), F0(X) = U(N(X)) ∪ U(N(X ′)) is the collection of all finiteunions of elements of N (X ′) and of all finite unions of elements of N+(X ′). By part (ii),F1(X ′) = F0(X) ∪ I(X ′), where I(X ′) consists of all finite unions of the form
⋃`∈LM` ∩ N`,
where M` ∈ N (X ′) and N` ∈ N+(X ′) for each ` ∈ L. Note that RX′ ∈ N (X ′)∩N+(X ′), sinceRX′ = NX′(p, p) = N+
X′(p, p) for any p ∈ ∆(X ′). Thus, F0(X) = U(N(X)) ∪ U(N(X ′)) ⊆I(X). Hence, F1(X) = I(X) = F(X).
(iv): Note first that for any NX′(A, p) ∈ N (X ′) (resp. N+X′(A, p) ∈ N+(X ′)) and any
finite Y ⊆ X ′ with y∗ ∈ Y and A ∈ A(Y ), we have NX′(A, p) = NY (A, p) × RX′rY (resp.N+X′(A, p) = N+
Y (A, p) × RX′rY ). Now fix any E ∈ F(X ′). By part (iv), we have a finiteindex set L and M` ∈ N (X ′), N` ∈ N+(X ′) for each ` ∈ L such that E =
⋃`∈LM` ∩ N`.
By the first sentence, we can then pick a finite Y ⊆ X ′ with y∗ ∈ Y such that for each `,we have M` = MY
` × RX′rY and N` = NY` × RX′rY , where MY
` ∈ N (Y ) and NY` ∈ N+(Y ).
Then E = EY × RXrY , where EY :=⋃`∈LM
Y` ∩ NY
` ∈ F(Y ). Conversely, if EY ∈ F(Y ),then by part (iv), EY is of the form
⋃`∈LM
Y` ∩ NY
` ∈ F(Y ) for some finite collection of
MY` ∈ N (Y ) and NY
` ∈ N+(Y ). Then by the first sentence, M` := MY` ×RX′rY ∈ N (X ′) and
N` = NY` × RX′rY ∈ N+(X ′), so EY × RX′rY =
⋃L`=1 M` ∩N`) ∈ F(X ′) by part (iv).
F.4.2 Properties of proper finitely-additive probability measures on F
Lemma 22. Let ν be a proper finitely-additive probability measure on F and suppose that(N(p,A)r 0)∩ supp ν = ∅ for some A ∈ A and p ∈ A, where 0 denotes the unique constantutility in RX . Then ν(N+(A, p)) = ν(N(A, p)) = 0.
Proof. Since (N(A, p) r 0) ∩ supp ν = ∅, we have
N(A, p) r 0 ⊆ (supp ν)c :=⋃V ∈ F : V open and ν(V ) = 0.
Thus, for some possibly infinite index set I, there exists a family Vii∈I , with Vi ∈ F openand ν(Vi) = 0 for each i such that
N(A, p) r 0 ⊆⋃i∈I
Vi.
We now show that there is a finite subset i1, . . . , in ⊆ I such that
N(A, p) r 0 ⊆n⋃j=1
Vij .
To see this, define L(A, p) := (N(A, p)∩ [−1, 1]X)r0. Note that since [−1, 1]X is compact inRX (by Tychonoff’s theorem) and N(A, p) is closed in RX , C(A, p) is compact in the relativetopology on RX r 0. Hence, since L(A, p) ⊆ N(A, p)r 0 is covered by
⋃i∈I Vi and each Vi
is open, it has a finite subcover⋃nj=1 Vij .
8
We claim that N(A, p) r 0 is also covered by⋃nj=1 Vij . To see this, consider any u∗ ∈
N(A, p) r 0. We can find a finite Y ⊆ X such that y∗ ∈ Y , u∗ Y is not constant, N(A, p) =NY (A, p)×RXrY , and for each j = 1, . . . , n, Vij = V Y
ij×RXrY for some V Y
ij∈ FY (see Lemma 21
(iv)).Since Y is finite, there exists α > 0 small enough such that αu∗(y) ∈ [−1, 1] for all y ∈ Y .
Define u ∈ RX by u Y = αu∗ Y and u(x) = 0 for all x ∈ XrY . Note that u ∈ N(A, p): Indeed,u∗ ∈ N(A, p) = NY (A, p)×RXrY , u Y = αu∗ Y , and NY (A, p) is closed under positive scaling.Moreover, u is not constant, since u∗ Y is not constant. Finally, u ∈ [−1, 1]X . This showsu ∈ L(A, p). Since L(A, p) is covered by
⋃nj=1 Vij , there exists j such that u ∈ Vij = V Y
ij×RXrY .
But note that V Yij
is closed under positive scaling, since by Lemma 21 (iii) it is a finite unionof sets which are closed under positive scaling. Since u Y = αu∗ Y , this implies u∗ ∈ Vij .
The above shows that N(A, p) r 0 is covered by⋃nj=1 Vij , and hence so is N+(A, p). But
since ν(Vij) = 0 for all j = 1, . . . , n and ν is finitely additive, it follows that ν(N+(A, p)) = 0.Moreover, by properness of ν, this implies ν(N(A, p)) = 0.
Lemma 23. Suppose ν is a proper finitely-additive probability measure on F and supp ν r0 = [u] for some u ∈ RX . Then for any A ∈ A and p ∈ A, we have ν(N(A, p)) =ν(N(M(A, u), p)).
Proof. Fix any A ∈ A and p ∈ A. Note first that for any q ∈ A,
q /∈M(A, u)⇒ ν(N(A, q)) = 0. (30)
Indeed, if q /∈M(A, u), then ∅ = [u]∩N(A, q) = (N(A, q)r0)∩ supp ν. But then Lemma 22implies that ν(N(A, q)) = 0, as claimed.
Suppose now that p /∈ M(A, u). Then (30) implies that ν(N(A, p)) = 0. Moreover,N(B, p) := ∅ if p /∈ B, so also ν(N(M(A, u), p)) = 0, as required.
Suppose next that p ∈M(A, u). Then
N(A, p) ⊆ N(M(A, u), p) ⊆ N(A, p) ∪⋃
q∈ArM(A,u)
N(A, q),
so that
ν(N(A, p)) ≤ ν(N(M(A, u), p)) ≤ ν(N(A, p)) +∑
q∈ArM(A,u)
ν(N(A, q)) = ν(N(A, p)),
where the last equality follows from (30). This again shows that ν(N(A, p)) =ν(N(M(A, u), p)), as required.
G Proof of Proposition 5
The following three subsections prove Proposition 5, that is, the equivalence between DREU,evolving utility, gradual learning and their respective S-based analogs.
9
G.1 DREU
“If” direction: Suppose ρ admits an S-based DREU representation(St, µst−1
t st−1∈St−1 , Ust , τstst∈St)t=0,...,T . We will construct a DREU representation
(Ω, F∗, µ, (Ft, Ut, Wt)).Consider the space G :=
∏Tt=0
(St × RXt
)of all sequences of states and tie-breaking util-
ities. Let Ω := (s0,W0, . . . , sT ,WT ) ∈ G :∏t
k=0 µsk−1
k (sk) > 0. Let F∗ be the restric-
tion to Ω of the product sigma-algebra of the discrete sigma-algebra on∏T
t=0 St and the
product Borel sigma-algebra on∏T
t=0 RXt . For each K = (s0, K0, ..., sT, KT ) ∈ F∗, let
µ(K) =∏T
t=0 µst−1
t (st)τst(Kt); by finiteness of∏T
t=0 St, µ extends to a finitely-additive proba-
bility measure on Ω in the natural way.Let Πt be the finite partition of Ω whose cells are all the cylinders C(s0, . . . , st) := ω ∈
Ω : projS0×...×St(ω) = (s0, . . . , st). Let Ft be the sigma-algebra generated by Πt; by definition
of Ω, µ(Ft(ω)) > 0 for all ω ∈ Ω. Also, Ft(ω) =⋃ω′∈Ft(ω) Ft+1(ω′), so (Ft)0≤t≤T ⊆ F∗ is
a filtration. Define Ut : Ω → RXt by Ut(ω) = Ust where projSt(ω) = st. Note that (Ut)
is adapted to (Ft) and that Ut(ω) is nonconstant for each ω since each Ust is nonconstant.Finally, if Ft−1(ω) = Ft−1(ω′) and Ft(ω) 6= Ft(ω′), then projSt−1
(ω) = projSt−1(ω) = st−1 and
projSt(ω) = st 6= s′t = projSt(ω′) for some st−1 ∈ St−1 and st, s
′t ∈ suppµ
st−1
t . By DREU1 (a),
this implies Ut(ω) := Ust 6≈ Us′t =: Ut(ω′). Thus, (Ft, Ut) are simple.
Define Wt : Ω→ RXt by Wt(ω) = Wt where projRXt (ω) = Wt. Note that for all At, µ(ω ∈Ω : |M(At, Wt)| = 1) =
∑(s0,...,sT )
(∏Tk=0 µ
sk−1
k (sk))τst(Wt ∈ RXt : |M(At,Wt)| = 1) = 1,
since each τst is proper. Thus, (Wt) satisfies part (i) of the properness requirement for DREU.Moreover, for any FT (ω) = C(s0, . . . , sT ) and any sequence (Bt) of Borel sets Bt ⊆ RXt , thedefinition of µ implies
µ
(T⋂t=0
Wt ∈ Bt|C(s0, . . . , sT )
)=
T∏t=0
τst(Bt) =T∏t=0
µ(Wt ∈ Bt | C(s0, . . . , st)
). (31)
Since FT (ω) = C(s0, . . . , sT ) implies Ft(ω) = C(s0, . . . , st) for all t ≤ T , this shows that (Wt)also satisfies parts (ii) and (iii) of the properness requirement.
Finally, to see that (Ω, F∗, µ, (Ft, Ut, Wt)) represents ρ, fix any ht = (A0, p0, ..., At, pt) ∈ Ht.Then
µ(C(ht)) = µ(⋂t
k=0ω ∈ Ω : pk ∈M(M(Ak, Uk(ω)), Wk(ω)))
=∑C(s0,...,st)∈Πt
µ(C(s0, ..., st))µ(⋂t
k=0ω ∈ Ω : pk ∈M(M(Ak, Uk), Wk)|C(s0, ..., st))
=∑(s0,...,st)∈S0×...×St
∏tk=0 µ
sk−1
k (sk)µ(⋂t
k=0ω ∈ Ω : pk ∈M(M(Ak, Usk), Wk)|C(s0, ..., st))
=∑(s0,...,st)∈S0×...×St
∏tk=0 µ
sk−1
k (sk)τsk(pk, Ak)
where the third equality follows from the definition of µ and U , and the final equality follows
10
from (31). Thus, as required, we have
µ(C(pt, At)|C(ht−1)) =µ(C(ht))
µ(C(ht−1)=
∑(s0,...,st)
∏tk=0 µ
sk−1
k (sk)τsk(pk, Ak)∑(s0,...,st−1)
∏t−1k=0 µ
sk−1
k (sk)τsk(pk, Ak)= ρt(pt;At|ht−1),
where the final equality holds by DREU2.“Only if” direction: Take any DREU representation (Ω,F∗, µ, (Ft, Ut,Wt)) of ρ. We will
construct an S-based DREU representation (St, µst−1
t st−1∈St−1 , Ust , τstst∈St)t=0,...,T .For each t, let St := Ft(ω) : ω ∈ Ω denote the partition generating Ft, which is finite
since (Ft) is simple. Each µstt+1 is defined to be the one-step-ahead conditional of µ, i.e.,µ0(s0) := µ(s0) for all s0 ∈ S0 and µstt+1(st+1) := µ(st+1|st) for all st ∈ St, st+1 ∈ St+1. This is
well-defined since µ(Ft(ω)) > 0 for all ω. For each st ∈ St, define Ust := Ut(ω) if ω ∈ st; this iswell-defined as (Ut) is Ft-adapted and each Ust is nonconstant since each Ut(ω) is nonconstant.Finally, for any Borel set Bt ⊆ RXt , define τst(Bt) := µ(Wt ∈ Bt|st). This is well-definedsince Wt is F∗-measurable. Moreover, because µ(ω ∈ Ω : |M(At,Wt(ω)| = 1 = 1 for all Atand |St| is finite, it follows that τst(N(At, pt)) = τst(N
+(At, pt)) for all pt, i.e., τst is proper.Thus, each (St, µ
st−1
t , Ust , τstst∈St) is an REU form on Xt.Moreover, (a) for any distinct st, s
′t ∈ supp(µ
st−1
t ), we have ω, ω′ such that Ft−1(ω) = st−1 =Ft−1(ω′) and Ft(ω) = st 6= Ft(ω′) = s′t. Thus, Ust = Ut(ω) 6≈ Ut(ω
′) = Us′t , since (Ut,Ft) issimple. Also, since (Ft) is adapted, the partition St refines the partition St−1, so that (b) for
any distinct st−1, s′t−1, we have supp(µ
st−1
t ) ∩ supp(µs′t−1
t ) = ∅. Since additionally µ(st) > 0 forall st ∈ St, we have (c)
⋃st−1∈St−1
supp µst−1
t = St. Thus, DREU1 is satisfied.
To see that DREU2 holds, observe that for each ht = (A0, p0, ..., At, pt) ∈ Ht, we have
µ(C(ht) =∑
sT∈ST µ(sT )µ (C(ht)|sT )
=∑
sT∈ST µ(sT )µ(⋂t
k=0ω ∈ Ω : pk ∈M(M(Ak, Uk),Wk)|sT)
=∑
(s0,...,sT )∃ω∈Ω∀t: st=Ft(ω)
µ(sT )µ(⋂t
k=0pk ∈M(M(Ak, Usk),Wk)|st)
=∑
(s0,...,sT )∃ω∈Ω∀t: st=Ft(ω)
µ(sT )∏t
k=0 µ (pk ∈M(M(Ak, Usk),Wk)|sk)
=∑
(s0,...,st)∃ω∈Ω∀k≤t: sk=Fk(ω)
∏tk=0 µ
sk−1
k (sk)∏t
k=0 τsk(pk, Ak)
=∑
(s0,...,st)∈S0×...×St∏t
k=0 µsk−1
k (sk)∏t
k=0 τsk(pk, Ak),
where the third equality follows from the fact that (Ut) is Ft-adapted, the fourth equality followsfrom parts (ii) and (iii) of the properness assumption on (Wt), the final equality follows fromthe fact that
∏tk=0 µ
sk−1
k (sk) = 0 whenever (s0, . . . , st) 6= (F0(ω), . . . ,Ft(ω)) for all ω, and the
remaining equalities hold by definition. Since ρt(pt;At|ht−1) = µ(C(ht))µ(C(ht−1)
by (3), this shows thatDREU2 holds.
G.2 Evolving Utility
“If” direction: Suppose ρ admits an S-based evolving utility representation(St, µst−1
t st−1∈St−1 , Ust , ust , τstst∈St)t=0,...,T . Let (Ω, F∗, µ, (Ft, Ut, Wt)) denote the corre-
11
sponding DREU representation of ρ obtained in the “if” direction for DREU. In addition, defineut : Ω → RZ for each t by ut(ω) := ust whenever projSt(ω) = st. Note that the process (ut) is
Ft-adapted. Moreover, for each ω = (s0,W0, ..., sT ,WT ), we have UT (ω) = UsT = usT = uT (ω)and for each t ≤ T − 1 and (zt, At+1)
Ut(ω)(zt, At+1) = Ust(zt, At+1)
= ust(zt) +∑
st+1∈St+1
µstt+1(st+1) maxpt+1∈At+1
Ust+1(pt+1)
= ut(ω)(zt) +∑
st+1∈St+1
µ(st+1|st) maxpt+1∈At+1
Ust+1(pt+1)
= ut(ω)(zt) + E[ maxpt+1∈At+1
Ut+1(pt+1)|Ft(ω)],
where we let µ(st+1|st) := µ (C(s0, . . . , st+1) | C(s0, . . . , st)). Thus we constructed an evolvingutility representation with δ = 1.
“Only if” direction: Suppose ρ admits an evolving utility representation(Ω,F∗, µ, (Ft, Ut, ut,Wt), δ). We obtain another evolving utility representation(Ω,F∗, µ, (Ft, U ′t , u′t,Wt), 1) with discount factor 1 by setting U ′t := δtUt and u′t := δtutfor each t. Clearly this represents the same ρ. Based on this second representation, let(St, µst−1
t st−1∈St−1 , Ust , τstst∈St)t=0,...,T denote the corresponding S-based DREU repre-sentation obtained in the “only if” direction for DREU. In addition, for each st, defineust ∈ RZ by ust = u′t(ω) for any ω ∈ st; this is well-defined as (u′t) is Ft-adapted. Re-versing the argument in the previous part, we can verify that usT = UsT for each sT and
Ust(zt, At+1) = ust(zt) +∑
st+1µstt+1(st+1) maxpt+1∈At+1 Ust+1(pt+1) for each st with t ≤ T − 1.
G.3 Gradual learning
“If” direction: Suppose ρ admits an S-based gradual learning representation(St, µst−1
t st−1∈St−1 , Ust , ust , τstst∈St , δ)t=0,...,T . Let (Ω, F∗, µ, (Ft, Ut, ut, Wt), 1) denote the cor-responding evolving utility representation obtained in the “if” direction for evolving utility. Inaddition, define δ := δ. Note that for each ω = (s0,W0, .., sT ,WT ) and t ≤ T − 1, we haveut(ω) = ust = 1
δ
∑st+1
µstt+1(st+1)ust+1 = 1
δE[ut+1|Ft(ω)]. Iterating expectations, this yields
ut(ω) = δt−TE[uT |Ft(ω)] = δt−TE[UT |Ft(ω)]. Replace Ut with U ′t := δT−tUt for each t. ByProposition 1, (Ω, F∗, µ, (Ft, U ′t , Wt)) is still a DREU representation of ρ. Moreover, for eacht ≤ T − 1, we have
U ′t(ω)(zt, At+1) = δT−tut(ω)(zt) + δT−tE[ maxpt+1∈At+1
Ut+1(pt+1)|Ft(ω)]
= E[U ′T (zt) | Ft(ω)] + δE[ maxpt+1∈At+1
U ′t+1(pt+1)|Ft(ω)].
Thus, (Ω, F∗, µ, (Ft, U ′t , Wt), δ) is a gradual learning representation of ρ.“Only if” direction: Suppose that ρ admits a gradual learning representation
(Ω, µ, (Ft, Ut,Wt), δ). Let U ′t := δt−TUt for all t. By Proposition 1, (Ω, µ, (Ft, U ′t ,Wt)) is still aDREU representation of ρ. Moreover, let u′t := δt−Tut, where ut(ω) := E[UT |Ft(ω)]. By (2),
12
for each ω, we have U ′T (ω) = UT (ω) = uT (ω) = u′t(ω) and for all t ≤ T − 1
U ′t(ω)(zt, At+1) = u′t(ω)(zt) + δt−T δE[ maxpt+1∈At+1
Ut+1(pt+1) | Ft(ω)]
= u′t(ω)(zt) + E[ maxpt+1∈At+1
U ′t+1(pt+1) | Ft(ω)].
Thus, (Ω, µ, (Ft, U ′t , u′t,Wt), 1) is an evolving utility representation of ρ. Let(St, µst−1
t st−1∈St−1 , U ′st , u′st , τstst∈St)t=0,...,T denote the corresponding S-based evolving util-
ity representation of ρ obtained in the “only if” direction for evolving utility. In addition,define δ := δ. Then for each t ≤ T − 1 and ω with Ft(ω) = st, we have
u′st = u′t(ω) = δt−TE[UT |Ft(ω)] =1
δE[u′t+1|Ft(ω)] =
1
δ
∑st+1
µstt+1(st+1)u′st .
Thus (St, µst−1
t st−1∈St−1 , U ′st , u′st , τstst∈St , δ)t=0,...,T is an S-based gradual learning represen-
tation of ρ.
H Proof of Proposition 1
H.1 “If” directions:
DREU: Consider any ht = (p0, A0, ..., pt, At) ∈ Ht. Then
µ(C(ht)) =∑
FT (ω)∈ΠT
µ(FT (ω))µ
(t⋂
k=0
pk ∈M(M(Ak, Uk),Wk)|FT (ω)
)
=∑
Ft(ω)∈Πt
t∏k=0
µ(Fk(ω) | Fk−1(ω))µ (Wk ∈ N(M(Ak, Uk(ω)), pk)|Fk(ω))
=∑
Ft(ω)∈Πt
t∏k=0
µ(φk(Fk(ω)) | φk−1(Fk−1(ω)))µ(Wk ∈ N(M(Ak, Uk(ω)), pk)|φk(Fk(ω))
)
=∑
Ft(ω)∈Πt
t∏k=0
µ(Fk(ω) | Fk−1(ω))µ(Wk ∈ N(M(Ak, Uk(ω)), pk)|Fk(ω)
)
=∑
FT (ω)∈ΠT
µ(FT (ω))
(t⋂
k=0
pk ∈M(M(Ak, Uk), Wk) | FT (ω)
)= µ(C(ht)),
where the second equality follows from properness of (Wt) and Ft-adaptedness of (Ut), thethird equality follows from assumptions (i) and (iii), the fourth equality from the fact thatφt is a bijection and assumption (ii), the fifth equality from the properness of (Wt) and Ft-adaptedness of (Ut), and the first and last equalities hold by definiton. Since D represents ρ
and D represents ρ, this implies ρt(pt, At|ht−1) = µ(C(ht))µ(C(ht−1))
= µ(C(ht))
µ(C(ht−1))= ρt(pt, At|ht−1). Thus,
ρ = ρ, as required.
13
Evolving utility: By the “if” direction for DREU, D is a DREU representation of ρ.It remains to show that (D, (ut), δ) satisfies (1). From assumptions (ii), (iv), and (v) it isimmediate that UT = uT . Moreover, for all t ≤ T − 1, and ω ∈ Ω, ω ∈ φt(Ft(ω)), we have
αt(ω)Ut(ω)(z, At+1) = Ut(ω)(z, At+1)− βt(ω)
= ut(ω)(z)− βt(ω) + δEµ[ maxpt+1∈At+1
Ut+1(pt+1) | Ft(ω)]
= αt(ω)ut(ω)(z)− δEµ[βt+1|Ft(ω)] + δEµ[αt+1 maxpt+1∈At+1
Ut+1(pt+1) | Ft(ω)] + δEµ[βt+1|Ft(ω)]
= αt(ω)
(ut(ω)(z) + δEµ[ max
pt+1∈At+1
Ut+1(pt+1) | Ft(ω)]
)where the first equality follows from (ii), the second from (1) for (D, (ut), δ), the third from (i),(ii), and (v) (and the fact φt is a bijection), and the fourth by (iv). Thus, (D, (ut), δ) satisfies(1).
Gradual learning: By the “if” direction for evolving utility, (D, (ut), δ) is an evolvingutility representation of ρ. Since δ = δ by (vi), it remains to show that (D, (ut), δ) satisfies (2).For all t ≤ T − 1 and ω ∈ Ω, ω ∈ φt(Ft(ω)), we have
α0(ω)ut(ω) + γt(ω) = ut(ω)
= Eµ[UT |Ft(ω)]
= α0(ω)Eµ[UT |Ft(ω)] + Eµ[βT |Ft(ω)],
where the first equality follows from (iv), (v), and (vi), the second from (2) for (D, (ut), δ),and the third from (i), (ii), (iv), (vi) (and the fact that φt is a bijection). But since γt(ω) =
βt(ω)−δEµ[βt+1|Ft(ω)] by (v) and βk(ω) = 1−δT−k+1
1−δ Eµ[βT | Fk(ω)] for all k by (vi), we have that
γt(ω) = Eµ[βT |Ft(ω)]. Hence, the above implies that ut(ω) = Eµ[UT |Ft(ω)], whence (D, (ut), δ)satisfies (2) with ˆu := UT .
H.2 “Only if” directions:
DREU: Throughout the proof, for any t and Et = Ft(ω) ∈ Πt, we let Ut(Et) denote Ut(ω)and likewise for U ; this is well-defined by adaptedness. We construct the sequence (φt, αt, βt)inductively, dealing with the base case t = 0 and the inductive step simultaneously.
Suppose t ≥ 0 and that we have constructed (φt′ , αt′ , βt′) satisfying (i)–(iii) for all t′ < t(disregard the latter assumption if t = 0). If t > 0, fix any Et−1 = Ft−1(ω∗) ∈ Πt−1, letEt−1 := φt−1(Et−1), and let Πt(Et−1) := Et = Ft(ω) ∈ Πt : Ft−1(ω) = Et−1 and Πt(Et−1) :=Et = Ft(ω) ∈ Πt : Ft−1(ω) = Et−1. As in the proof of Lemma 2, we can repeatedlyapply Lemma 13 to find a separating history for Et−1 = Ft−1(ω∗), i.e., a history ht−1 =(B0, q0, . . . , Bt−1, qt−1) ∈ H∗t−1 such that ω ∈ Ω : qk ∈ M(Bk, Uk(ω)) = Fk(ω∗) for all
k = 0, . . . , t− 1. By inductive hypothesis ht−1 is then also a separating history for Et−1. Thus,by Lemma 14 (and the translation to S-based DREU in Proposition 5), C(ht−1) = Et−1 andC(ht−1) = Et−1. If t = 0, then in the following we let Et−1 := Ω, Et−1 := Ω, Πt(Et−1) := Π0,Πt(Et−1) := Π0, and we disregard all references to the separating history.
Enumerate Πt(Et−1) = Eit : i = 1, . . . ,m with corresponding utilities U i
t := Ut(Eit) and
14
Πt(Et−1) = Ejt : j = 1, . . . , m with corresponding utilities U j
0 := Ut(Ejt ). Since (Ft, Ut) and
(Ft, Ut) are both simple, we have µ(Eit) > 0 for all i and U i
t 6≈ U i′t for i 6= i′, and likewise
µ(Ejt ) > 0 for all j and U j
t 6≈ U j′
t for j 6= j′. Note that for every j there exists a unique i(j)
such Ui(j)t ≈ U j
t . Indeed, if such an i(j) exists it is unique because all the U it represent different
preferences. And the desired i(j) exists, since otherwise by Lemma 13, we can find a menuBt = qit : i = 1, . . . ,m ∪ qjt such that M(Bt, U
it ) = qit for each i and M(Bt, U
jt ) = qjt.
We can additionally assume (by replacing ht−1 with an appropriate mixture if need be) thatht−1 ∈ H∗t−1(Bt). Since D and D both represent ρ, we obtain
0 = µ[C(qjt , Bt)|Et−1] = ρt(qjt ;Bt|ht−1) = µ[C(qjt , Bt)|Et−1) ≥ µ(Ej
t |Et−1) > 0,
a contradiction. Similarly, for every i, there exists a unique j(i) such that Uj(i)t ≈ U i
t . Thus,
defining φt : Πt(Et−1) → Πt(Et−1) by φt(Eit) = E
j(i)t yields a bijection. By construction,
Ut(Eit) ≈ Ut(φt(E
it)) for all i, so we can find αt(E
it) ∈ R++ and β(Ei
t) ∈ R such that Ut(Eit) =
αt(Eit)Ut(φt(E
it)) + β(Ei
t). Defining α(ω) = α(Ft(ω)) and β(ω) = β(Ft(ω)) this yields Ft-measurable maps αt, βt : Et−1 → R such that (ii) holds for all ω ∈ Et−1. Moreover, applyingLemma 13 again, we can find a menu Dt = rit : i = 1, . . . , n such that M(Dt, U
it ) = rit
for each i. Again, slightly perturbing the separating history ht−1 for Et−1 if need be, we canassume that ht−1 ∈ H∗t−1(Dt). Then by the representation, µ(Ei
t |Et−1) = ρt(rit;D
it|ht−1) =
µ(φt(Eit)|Et−1) for all i, yielding (i).
To show (iii), consider any pt ∈ At, where we can again assume ht−1 ∈ H∗t−1(12At + 1
2Dt).
Let Bit := w ∈ RXt : pt ∈ M(M(At, Ut(E
it)), w). Note that by (ii), Bi
t = w ∈ RXt :pt ∈ M(M(At, Ut(φt(E
it))), w). Thus, µ(Wt ∈ Bt|Ei
t) = µ(C(pt, At)|Eit) and µ(Wt ∈
Bt|φt(Eit)) = µ(C(pt, At)|φt(Ei
t)). But since D and D both represent ρ and by choice of Dt,
µ(Eit |Et−1)µ[C(pt, At)|Ei
t ] = µ[C(1
2pt +
1
2rit,
1
2At +
1
2Dt)|Et−1] =
ρt(1
2pt +
1
2rit;
1
2At +
1
2Dt|ht−1) =
µ[C(1
2pt +
1
2rit,
1
2At +
1
2Dt)|Et−1] = µ(φt(E
it)|Et−1)µ[C(pt, At)|φt(Ei
t)],
which implies µ[C(pt, At)|Eit ] = µ[C(pt, At)|φt(Ei
t)], since by (i) we have µ(Eit |Et−1) =
µ(φt(Eit)|Et−1). Thus, µ(Wt ∈ Bt|Ei
t) = µ(Wt ∈ Bt|φt(Eit)), as required.
Finally, note that the collection Πt(Et−1) : Et−1 ∈ Πt−1 partitions Πt, and similarlyΠt(Et−1) : Et−1 ∈ Πt−1 partitions Πt. Thus, applying the above construction for everyEt−1 ∈ Πt−1 yields a bijection φt : Πt → Πt and Ft-measurable maps αt : Ω → R++ andβt : Ω→ R such that (i)–(iii) are satisfied.
Evolving utility: The “only if” part for DREU yields sequences (φt, αt, βt) such that(i)–(iii) are satisfied. It remains to show that (iv) and (v) hold. Throughout the proof, forany Et = Ft(ω) ∈ Πt, we sometimes use Ut(Et), αt(Et), βt(Et) to denote Ut(ω), αt(ω), βt(ω);this is well-defined since Ut, αt, βt are Ft-measurable. We also let Ft−1(Et) := Ft−1(ω); this iswell-defined since Ft(ω) = Ft(ω′) implies Ft−1(ω) = Ft−1(ω′), as (Ft) is a filtration.
15
For (iv), fix any ω and t ≤ T − 1. Let Et := Ft(ω) and pick any At+1, Bt+1 and zt. Then
Ut(Et)(zt, At+1)− Ut(Et)(zt, Bt+1) = αt(Et)(Ut(φt(Et))(zt, At+1)− Ut(φt(Et))(zt, Bt+1))
= αt(Et)δ∑
Et+1∈Πt+1
µ(Et+1|φt(Et))[maxAt+1
Ut+1(Et+1)−maxBt+1
Ut+1(Et+1)]
= αt(Et)δ∑
Et+1∈Πt+1
µ(φt+1(Et+1)|φt(Et))[maxAt+1
Ut+1(φt+1(Et+1))−maxBt+1
Ut+1(φt+1(Et+1))]
= αt(Et)δ∑
Et+1∈Πt+1
µ(Et+1|Et)[maxAt+1
Ut+1(φt+1(Et+1))−maxBt+1
Ut+1(φt+1(Et+1))]
= αt(Et)δ∑
Et+1 s.t. Ft(Et+1)=Et
µ(Et+1|Et)[maxAt+1
Ut+1(φt+1(Et+1))−maxBt+1
Ut+1(φt+1(Et+1))],
(32)
where the first equality holds by (ii), the second equality follows from (D, δ) being an evolvingutility representation, the third equality from the fact that φt is a bijection, the fourth equalityfrom (i), and the fifth equality from the fact that µ(Ft+1(ω′)|Et) > 0 iff Ft(ω′) = Et.
At the same time, we have
Ut(Et)(zt, At+1)− Ut(Et)(zt, Bt+1)
= δ∑
Et+1∈Πt+1
µ(Et+1|Et)[maxAt+1
Ut+1(Et+1)−maxBt+1
Ut+1(Et+1)
= δ∑
Et+1∈Πt+1
µ(Et+1|Et)αt+1(Et+1)[maxAt+1
Ut+1(φt+1(Et+1))−maxBt+1
Ut+1(φt+1(Et+1))]
= δ∑
Et+1 s.t. Ft(Et+1)=Et
µ(Et+1|Et)αt+1(Et+1)[maxAt+1
Ut+1(φt+1(Et+1))−maxBt+1
Ut+1(φt+1(Et+1))],
(33)
where the first equality follows from (D, δ) being an evolving utility representation, the secondequality from (ii), and the third equality from the fact that µ(Ft+1(ω′)|Et) > 0 iff Ft(ω′) = Et.
Combining (32) and (33), we have that for all At+1 and Bt+1,
δ∑
Et+1 s.t. Ft(Et+1)=Et
µ(Et+1|Et)αt(Et)[maxAt+1
Ut+1(φt+1(Et+1))−maxBt+1
Ut+1(φt+1(Et+1))]
= δ∑
Et+1 s.t. Ft(Et+1)=Et
µ(Et+1|Et)αt+1(Et+1)[maxAt+1
Ut+1(φt+1(Et+1))−maxBt+1
Ut+1(φt+1(Et+1))].
(34)
Since (Ft, Ut) is simple and φt is a bijection, Ut+1(φt+1(Et+1)) 6≈ Ut+1(φt+1(E ′t+1)) for all dis-tinct Et+1, E
′t+1 with Ft(Et+1) = Et = Ft(E ′t+1). So by Lemma 13, we can find a menu
At+1 := qEt+1
t+1 : Ft(Et+1) = Et such that for all Et+1 with Ft(Et+1) = Et we have
M(At+1, Ut+1(φt+1(Et+1)) = qEt+1
t+1 . Let E∗t+1 := Ft+1(ω) and let Bt+1 = At+1 rqE∗t+1
t+1 . Then
in (34), [maxAt+1 Ut+1(φt+1(Et+1))−maxBt+1 Ut+1(φt+1(Et+1))] 6= 0 iff Et+1 = E∗t+1. Hence, (34)
16
implies δδαt(ω) = δ
δαt(Et) = αt+1(E∗t+1) = αt+1(ω). Since this is true for all t ≤ T − 1, (iv)
follows.For (v), note that the claim for T is immediate from (ii) and the fact that UT = uT , UT = uT .
Next, fix any ω ∈ Ω, ω ∈ φt(Ft(ω)), t ≤ T − 1, and (z, pt+1). Then
Ut(ω)(z, pt+1) = ut(ω)(z) + δEµ[Ut+1(pt+1)|Ft(ω)]
= ut(ω)(z) + αt(ω)δEµ[Ut+1(pt+1)|Ft(ω)] + δEµ[βt+1|Ft(ω)],(35)
where the first equality follows from (D, (ut), δ) being an evolving utility representation andthe second equality from (i), (ii), (iv) (and the fact that φt is a bijection). At the same time,we have
Ut(ω)(z, pt+1) = αt(ω)Ut(ω)(z, pt+1) + βt(ω)
= αt(ω)ut(ω)(z) + αt(ω)δEµ[Ut+1(pt+1)|Ft(ω)] + βt(ω),(36)
where the first equality follows from (ii) and (iv) and the second equality from (D, (ut), δ) beingan evolving utility representation. Combining (35) and (36) yields the desired claim.
Gradual learning: The “only if” part for evolving utility yields sequences (φt, αt, βt) suchthat (i)–(v) are satisfied. It remains to show that (vi) and (vii) hold.
For (vi), note that since (D, (ut), δ) is a gradual learning representation of ρ, combining theproofs of Proposition 5 (equivalence of gradual learning and S-based gradual learning repre-sentations) and of the necessity direction of Theorem 3 yields that for all t ≤ T − 1, ht, andqt, rt ∈ ∆(Xt), we have qt %ht rt if and only if Ut(ω)(qt) ≥ Ut(ω)(rt) for all ω ∈ C(ht). Fixany ω ∈ Ω and h0 = (A0, p0) ∈ H∗0 such that ω′ : p0 ∈ M(A0, U0(ω′)) = F0(ω) (which existsby Lemma 13). Then by Lemma 14, C(h0) = F0(ω). So by Condition 1 applied to h0, we canfind `,m, n ∈ ∆Z such that U0(ω)(`, n, . . . , n) 6= U0(ω)(m,n, . . . , n). Note that by (2) and iter-ated expectations, we have U0(`0, . . . , `T ) =
∑Tk=0 δ
ku0(ω)(`k) for any stream of consumptionlotteries, (`0, . . . , `T ), where u0(ω) := E[UT |F0(ω)]. Hence, we have u0(ω)(`) 6= u0(ω)(m), andU0(ω)(`,m, n, . . . , n)−U0(ω)(η`+ (1− η)m, η`+ (1− η)m,n, . . . , n) = 0 if and only if η = 1
1+δ.
Now pick any ω ∈ φ0(F0(ω)). Then since (D, (ut), δ) is also a gradual learning representationof ρ, by the same reasoning as above we have that U0(ω)(`,m, n, . . . , n)−U0(ω)(η`+(1−η)m, η`+(1− η)m,n, . . . , n) = 0 if and only if η = 1
1+δ. By (ii), this implies that δ = δ, proving (vi).
Given (iv) and (vi) and the fact that ut(ω) = E[ut+1|Ft(ω)] and ut(ω) = E[ut+1|Ft(ω)] by
(2), applying (v) inductively then yields that βt(ω) = 1−δT−t+1
1−δ E[βT |Ft(ω)], proving (vii).
I Proofs for Section 5.2
I.1 Proof of Proposition 2
(ii)=⇒(i): Consider any (ht−1, At, pt), (ht−1, At, qt) ∈ Ht and pt+1 ∈ At+1 ∈ A∗t+1(ht−1, At, pt)∩
A∗t+1(ht−1, At, qt) such that At and At+1 are atemporal, AZt+1 ⊆ AZt and pZt+1 = pZt =: p.Let Ut(p) := ut(ω) : ω ∈ C(ht−1, At, pt) and Ut(q) := ut(ω) : ω ∈ C(ht−1, At, qt).
Note that since At+1 features no ties, Lemma 14 implies C(pt+1, At+1) = ω : pt+1 ∈M(At+1, Ut+1(ω)), which since At+1 is atemporal is in turn equal to ω : p ∈
17
M(AZt+1, ut+1(ω)). Hence, by the representation,
ρt+1(pt+1;At+1|ht−1, At, pt) = µ(p ∈M(AZt+1, ut+1)|C(ht−1, At, pt))
≥ minu∈Ut(p)
µ(p ∈M(AZt+1, ut+1)|C(ht−1) ∩ ut ≈ u). (37)
Likewise,
ρt+1(pt+1;At+1|ht−1, At, qt) = µ(p ∈M(AZt+1, ut+1)|C(ht−1, At, qt))
≤ maxu′∈Ut(q)
µ(p ∈M(AZt+1, ut+1)|C(ht−1) ∩ ut ≈ u′). (38)
Pick u ∈ Ut(p) (respectively, u′ ∈ Ut(q)) which achieve the min (respectively max) in (37) (re-spectively, in (38)). Let u1
t+1, ..., umt+1 := ut(ω) : ω ∈ C(ht−1, At, pt)∪C(ht−1, At, qt) and p ∈
M(AZt+1, ut+1(ω)) and let D := cou, u1t+1, ..., u
mt+1. Note that since AZt ⊇ AZt+1 and since pZt =
p, we have p ∈M(AZt+1, u). Hence, C(ht−1)∩ut ≈ u∩p ∈M(AZt+1, ut+1) = C(ht−1)∩ut ≈u ∩ [D], and likewise C(ht−1) ∩ ut ≈ u′ ∩ p ∈M(AZt+1, ut+1) = C(ht−1) ∩ ut ≈ u′ ∩ [D].Thus,
µ(p ∈M(AZt+1, ut+1)|C(ht−1) ∩ ut ≈ u) = µ([D]|C(ht−1) ∩ ut ≈ u) ≥µ([D]|C(ht−1) ∩ ut ≈ u′) = µ(p ∈M(AZt+1, ut+1)|C(ht−1) ∩ ut ≈ u′),
(39)
where the inequality holds by (ii). Combining (37), (38), and (39) yieldsρt+1(pt+1;At+1|ht−1, At, pt) ≥ ρt+1(pt+1;At+1|ht−1, At, qt), as required.
(i)=⇒(ii): We prove the contrapositive. Suppose that for some u, u′ ∈ RZ and ht−1 withC(ht−1) ∩ ut ≈ u 6= ∅ 6= C(ht−1) ∩ ut ≈ u′, and convex D ⊆ RZ with u ∈ D, we have
µ(ut+1 ∈ [D]|C(ht−1) ∩ ut ≈ u) < µ(ut+1 ∈ [D]|C(ht−1) ∩ ut ≈ u′). (40)
Let Ut+1 be the set of possible realizations of ut+1 conditional on the event C(ht−1) ∩ ω :ut ≈ u or ut ≈ u′. Let Ut be the set of possible realizations of ut conditional on eventC(ht−1). Utility functions in these sets are all non-constant by Condition 2 (Uniformly RankedPair). Enumerate u1
t+1, ..., umt+1 := Ut+1 ∩ [D] and u1
t+1, ..., unt+1 := Ut+1 r [D]. Fix any
pZ ∈ int∆(Z). Note that for any j = 1, ..., n, ujt+1 does not belong to [cou, u1t+1, ..., u
mt+1].
Thus, by Lemma 24, for each j = 1, ..., n, we can find a vector wj ∈ RZ with∑
z wj(z) = 0 such
that ujt+1 ·wj > 0 ≥ uit+1 ·wj, u ·wj for any i = 1, ...,m. For each j, we construct qZ(j) ∈ ∆(Z)such that the vector qZ(j)− pZ (in RZ) is proportional to wj.58 Thus for each j = 1, ..., n andi = 1, ..,m, we have ujt+1 · (qZ(j)− pZ) > 0 ≥ uit+1 · (qZ(j)− pZ), u · (qZ(j)− pZ).
Pick a uniformly ranked pair of consumption lotteries `, ` ∈ ∆Z from Condition 2. Sinceu · (` − `) > 0 and uit+1 · (` − `) > 0 for each i = 1, ..., n, setting pZ := (1 − ε)pZ + ε(p − p)for some ε > 0, we have 0 > uit+1 · (qZ(j) − pZ), u · (qZ(j) − pZ) for any i = 1, ...,m andj = 1, ..., n. We can pick ε sufficiently small so that pZ is indeed well-defined (i.e., belongsto ∆Z) and that ujt+1 · (qZ(j) − pZ) > 0 holds for each j = 1, ..., n. Finally, subject to
58Note that such a construction is possible because pZ is in interior of ∆(Z).
18
perturbations of the lotteries pZ ∪ qZ(j) : j = 1, ..., n, we can assume without loss59 thatut(q
Z(j)) 6= ut(qZ(j′)) for each ut ∈ Ut and distinct j, j′ = 1, .., n while preserving the fact that
ujt+1 · (qZ(j)− pZ) > 0 > uit+1 · (qZ(j)− pZ), u · (qZ(j)− pZ) for any i = 1, ...,m and j = 1, ..., n.Construct an atemporal menu At+1 such that AZt+1 = pZ ∪ qZ(j) : j = 1, ..., n and such
that for each p ∈ AZt+1 there is a unique pt+1 ∈ At+1 such that pZt+1 = p. Note that since
ujt+1 · (qZ(j)− pZ) > 0 > uit+1 · (qZ(j)− pZ) for any j = 1, ..., n and i = 1, ...,m, we have
µ(pZ ∈M(ut+1, AZt+1)|C(ht−1), ut ≈ u) = µ(ut+1 ∈ [D]|C(ht−1), ut ≈ u)
Let [u1t ], ..., [u
kt ] denote the collection of equivalence classes of utilities in Ut, and assume
without loss that u ∈ [u1t ]. By Lemma 13, we construct a collection of consumption lotteries
rZ(l) : l = 1, ..., k such that ult(rZ(l)) > ult(r
Z(l′)) for any distinct l, l′ = 1, ..., k.Pick ε′ > 0 sufficiently small such that pZ + ε′(rZ(l) − rZ(1)) ∈ ∆(Z) for all l = 2, . . . , k;
such an ε′ exists as pZ is in the interior of ∆(Z). Thus, we can construct an atemporal menuAt such that
AZt := pZ ∪ qZ(j) : j = 1, ..., n ∪ pZ + ε′(rZ(l)− rZ(1)) : l = 2, ..., k
and such that for each p ∈ AZt there is a unique pt ∈ At with pZt = p.Recall that by construction ut(q
Z(j)) 6= ut(qZ(j′)) for each ut ∈ Ut and distinct j, j′ = 1, .., n.
Moreover, for any ut ∈ Ut, ut(pZ + ε′(rZ(l)− rZ(1))) is non-constant in ε′. Therefore, for smallenough ε′ > 0, we can guarantee that |M(ut, A
Zt )| = 1 for all ut ∈ Ut. Since At is atemporal and
each lottery in At has a unique corresponding projection in AZt , this ensures that At ∈ A∗(ht−1)(by Lemma 14). Since ult · (rZ(l) − rZ(1)) > 0 > u · (qZ(j) − pZ), u · (rZ(l) − rZ(1)) for eachj = 1, .., n and l = 2, ..., k, we have
µ(pZ ∈M(ut, AZt )|C(ht−1)) = µ(ut ≈ u|C(ht−1)).
Let pt denote the unique lottery in At such that pZt = pZ and let pt+1 denote the uniquelottery in At+1 such that pZt+1 = pZ . Since u 6≈ u′, which implies u′(pZ + ε′(rZ(l) − rZ(1)) >u′(pZ) for the index l such that u′ ∈ [ult], there is a lottery qt ∈ At different from pt suchthat M(AZt , u
′) = qZt . Also, |M(ut+1, AZt+1)| = 1 for all ut+1 ∈ Ut+1, and thus At+1 ∈
A∗t+1(ht−1, At, pt) ∩ A∗t+1(ht−1, At, qt).
In case At 6∈ A(ht−1), we can construct another history ht−1 = λht−1 + (1 − λ)dt−1 bymixing ht−1 with an appropriate degenerate history dt−1 (as in the construction of the extendedρ, Definition 3) such that C(ht−1) = C(ht−1) and At ∈ A(ht−1). By the previous paragraphs,we have AZt ⊇ AZt+1 and ρt+1(pt+1;At+1|ht−1, At, pt) = µ(ut+1 ∈ [D]|C(ht−1), ut ≈ u) and
ρt+1(pt+1;At+1|ht−1, At, qt) = µ(ut+1 ∈ [D]|C(ht−1), ut ≈ u′). But then (40) implies thatρt+1(pt+1;At+1|ht−1, At, pt) < ρt+1(pt+1;At+1|ht−1, At, qt), which is a violation of consumptionpersistence.
Lemma 24. Take any finite Y and a finite set of non-constant utilities u1, .., um ⊆ RY . Thenfor any non-constant u ∈ RY , the following are equivalent:
59Recall that each ut ∈ Ut is non-constant.
19
(i). for any w ∈ RY such that∑
y∈Y w(y) = 0,
[∀i = 1, ...,m, ui · w ≤ 0]⇒ u · w ≤ 0
(ii). u ∈ [cou1, ..., um].
Proof. The result follows from the utilitarian aggregation theorem, e.g., Theorem 2 in Fishburn(1984).
I.2 Proof of Proposition 3
(ii)=⇒(i): Consider any (ht−1, At, pt) ∈ Ht and pt+1 ∈ At+1 ∈ A∗t+1(ht−1, At, pt) such thatAt and At+1 are atemporal with AZt+1 ⊆ AZt and pZt = pZt+1. Consider any felicity u ∈ RZ
such that µ(ut ≈ u | C(ht−1, At, pt)) > 0. Then µ(ut ≈ u | C(ht−1)) > 0, so by (ii)µ(ut+1 ≈ u | C(ht−1) ∩ ut ≈ u) > 0. Moreover, u(pZt ) ≥ u(qZt ) for all qZt ∈ AZt , sosince AZt+1 ⊆ AZt is atemporal, pZt+1 = pZt , and At+1 ∈ A∗t+1(ht−1, At, pt), Lemma 14 impliesu(pZt+1) > u(qZt+1) for all qt+1 ∈ At+1 r pt+1. Thus,
ρt+1(pt+1;At+1|ht−1, At, pt) ≥ µ(ut+1 ≈ u|C(ht−1) ∩ ut ≈ u) > 0.
(i)=⇒(ii): We prove the contrapositive. Suppose that there is some history ht−1 and felicityu ∈ RZ such that µ(ut ≈ u|C(ht−1)) > 0 but µ(ut+1 ≈ u|C(ht−1) ∩ ut ≈ u) = 0. Let([u1], ..., [um]) denote the equivalence classes of felicities that can realize in periods t or t + 1,and suppose without loss that u ∈ [u1]. Note that they are all non-constant by Condition 1(Consumption Nondegeneracy). By applying Lemma 13 to consumption lotteries, we constructa collection of consumption lotteries pZ(i) : i = 1, ...,m ⊆ ∆(Z) such that ui(pZ(i)) >ui(pZ(j)) for any distinct pair i, j = 1, ...,m.
Construct atemporal menus At and At+1 such that AZt = AZt+1 = pZ(i) : i = 1, ...,mand At+1 ∈ supp qAt for all qt ∈ At. Let pt and pt+1 denote the unique lotteries in At andAt+1 such that pZt = pZt+1 = pZ(1). By construction At ∈ A∗t and At+1 ∈ A∗t+1 (Lemma14). Moreover, conditional on C(ht−1), ut(ω) ∈ M(At, pt) if and only if ut(ω) ≈ u, so thatC(ht−1, At, pt) = C(ht−1) ∩ ut ≈ u since there is no tie at At. In case At 6∈ A(ht−1), bymixing ht−1 with an appropriate degenerate history, we choose another history ht−1 that hasthe property that C(ht−1) = C(ht−1) and At ∈ At(ht−1). But then
ρt+1(pt+1;At+1|C(ht−1, At, pt)) = µ(ut+1 ≈ u|C(ht−1) ∩ ut ≈ u) = 0,
so that ρ violates consumption inertia.
I.3 Proof of Corollary 1
first part, “only if”:We consider the case m ≥ 2 as otherwise the desired statement trivially holds with any α.
Observe first that for any distinct indices i, j ∈ 1, . . . ,m, consumption persistence and its
20
characterization (Proposition 2) implies
Mii = µ(u1 ∈ [ui]|u0 = ui) ≥ µ(u1 ∈ [ui]|u0 = uj) = Mji. (41)
(Note that by assumption both ui and uj arise with positive probability in period 0). Moreover,if D = coui, uj, then by assumption there is no k /∈ i, j such that uk ∈ [D]. Thus, byconsumption persistence and its characterization (Proposition 2),
Mii +Mij = µ(u1 ∈ [D]|u0 = ui) = µ(u1 ∈ [D]|u0 = uj) = Mjj +Mji. (42)
Suppose first that m = 2. Since 1 = M11 +M12 = M22 +M21, we have M11 −M21 = M22 −M12 := α, which is nonnegative by (41). Since the Markov chain is irreducible, M21,M12 > 0,which also ensures α < 1. Setting ν(u1) := M21
1−α and ν(u2) := M12
1−α , we obtain the desired form.
Suppose next that m ≥ 3. Take any distinct i, j, k ∈ 1, ...,m and let D′ = coui, uj, uk.By assumption there is no l /∈ i, j, k such that ul ∈ [D′]. Thus, by consumption persistenceand its characterization (Proposition 2),
Mii +Mij +Mik = µ(u1 ∈ [D′]|u0 = ui) = µ(u1 ∈ [D′]|u0 = uj) = Mjj +Mji +Mjk.
Combined with (42), this implies that Mik = Mjk for any distinct i, j, k. Thus, for any k, we candefine βk := Mik for some arbitrary i 6= k. Here βk > 0, because otherwise
∑i s.t. i 6=kMik = 0,
contradicting irreducibility of the Markov chain. By (42), Mii −Mji = Mjj −Mij for any i, j,and thus Mii − βi = Mjj − βj =: α for any i, j. By (41) α ≥ 0, and α < 1 as βk > 0 for all k.
Thus, setting ν(uj) :=βj
1−α for each j leads to the desired form.first part, “if”:
Take any t and [u], [u′] ⊆ RZ that correspond to possible realizations of period t felicities.Then for any ht−1 with µ(ut ≈ u|C(ht−1)), µ(ut ≈ u′|C(ht−1)) > 0 and any convex D withu ∈ D, we have
µ(ut+1 ∈ [D]|C(ht−1), ut ≈ u) = α + (1− α)∑uj∈[D]
ν(uj)
≥ αµ(ut ∈ [D]|C(ht−1), ut ≈ u′) + (1− α)∑uj∈[D]
ν(uj)
= µ(ut+1 ∈ [D]|C(ht−1), ut ≈ u′).
Thus, ρ features choice persistence by Proposition 2.Second part:
The proof follows immediately by applying Proposition 3 and thus is omitted.
21
J Proof of Proposition 4
Let E denote the support of the joint distribution of (ε(x,y)1 , ε
(x,z)1 ), which is also the support
of the distribution of (εy2, εz2). Fix a function σ : E → [0, 1] such that
σ(εy, εz) ∈ argmaxα∈[0,1] α(δ2v2(y) + δ2εy) + (1− α)(δ2v2(z) + δ2εz)
for each (εy, εz) ∈ E . Then
E[maxδ2v2(y) + δ2εy2, δ2v2(z) + δ2εz2]
= E[σ(εy2, εz2)(δ2v2(y) + δ2εy2) + (1− σ(εy2, ε
z2))(δ2v2(z) + δ2εz2)]
= δ2(α∗v2(y) + (1− α∗)v2(z)) + δ2E[σ(εy2, εz2)εy2 + (1− σ(εy2, ε
z2))εz2]
where α∗ := E[σ(εy2, εz2)]. Note that since ε2 has mean zero, δ2(α∗v2(y) + (1 − α∗)v2(z)) is the
expected value the agent would obtain from Alate1 if in period 2 she chooses y with probability α∗
regardless of the realization of ε2. This implies that the term δ2E[σ(εy2, εz2)εy2 + (1−σ(εy2, ε
z2))εz2]
is nonnegative. At the same time,
E[maxδ2v2(y) + δε(x,y)1 , δ2v2(z) + δε
(x,z)1 ]
≥ E[σ(ε(x,y)1 , ε
(x,z)1 )(δ2v2(y) + δε
(x,y)1 ) + (1− σ(ε
(x,y)1 , ε
(x,z)1 ))(δ2v2(z) + δε
(x,z)1 )]
= δ2(α∗v2(y) + (1− α∗)v2(z)) + δE[σ(εy2, εz2)εy2 + (1− σ(εy2, ε
z2))εz2]
where the equality used the i.i.d. assumption on ε1 and ε2. Therefore E[maxδ2v2(y) +
δε(x,y)1 , δ2v2(z) + δε
(x,z)1 ] ≥ E[maxδ2v2(y) + δ2εy2, δ
2v2(z) + δ2εz2]. Thus, the desired claimfollows from the i.i.d. assumption on ε0.
K Consumption Dependence
K.1 Enriched Primitive
To accommodate consumption dependence, we enrich the stochastic choice data studied inprevious sections: We not only keep track of past choices of lotteries from menus, but also ofthe corresponding realized consumptions.
The dynamic stochastic choice rule ρ is again defined recursively. The observed choice distri-bution at period 0 is summarized by a map ρ0 : A0 → ∆(∆(X0)) such that
∑p0∈A0
ρ0(p0;A0) =1 for all A0. The set of enriched period 0 histories H0 := (A0, p0, z0) : ρ0(p0, A0) > 0 and z0 ∈supp pZ0 summarizes all choices p0 from A0 and realized consumptions z0 that jointly occurwith positive probability. For any history h0 = (A0, p0, z0) ∈ H0, let A1(h0) := A1 ∈ A1 :(z0, A1) ∈ supp p0 denote the set of period 1 menus that follow h0 with positive probability.For each t = 1, . . . , T and history ht−1 ∈ Ht−1, observed period t choices following ht−1 aresummarized by a map ρt(·|ht−1) : At(ht−1)→ ∆(∆(Xt)) such that
∑pt∈At ρt(pt;At | h
t−1) = 1
for all At ∈ At(ht−1). The set of enriched period-t histories is denoted by
Ht := (ht−1, At, pt, zt) : ht−1 ∈ Ht−1; At ∈ At(ht−1); ρt(pt;At|ht−1) > 0; zt ∈ supp pZt .
22
For each t ≤ T − 1, the set of period t+ 1 menus that follow history ht = (ht−1, At, pt, zt) withpositive probability is At+1(ht) := At+1 ∈ At+1 : (zt, At+1) ∈ supp pt and the set of period-thistories that lead to At+1 with positive probability is Ht(At+1) := ht ∈ Ht : At+1 ∈ At+1(ht).
Finally, for each t = 1, . . . , T and consumption stream zt−1 = (z0, ..., zt−1) ∈ Zt, let Hzt−1
t−1 :=ht−1 ∈ Ht−1 : ht−1 = (A0, p0, z0, ..., At−1, pt−1, zt−1) for some A0, p0, . . . , At−1, pt−1 denote theset of histories that give rise to consumption stream zt−1; and let Hzt−1
t−1 (At) := Hzt−1
t−1 ∩Ht−1(At).
K.2 Representations
We define the consumption dependent versions of our representations by extending the S-basedrepresentations introduced in Appendix A. Equivalent analogs of the Ω-based representationsfrom the main text can be defined, but are omitted to save space.
Definition 12. A consumption-dependent DREU (CDREU) representation of ρ consists oftuples (S0, µ0, Us0 , τs0s0∈S0), (St, µst−1,zt−1
t st−1∈St−1,zt−1∈Z , Ust , τstst∈St)1≤t≤T such that forall t = 0, . . . , T , we have:
CDREU1: For all st−1 ∈ St−1 and zt−1 ∈ Z, (St, µst−1,zt−1
t , Ust , τstst∈St) is an REU formon Xt such that60
(a) Ust 6≈ Us′t for any distinct st, s′t ∈ supp(µ
st−1,zt−1
t );
(b) supp(µst−1,zt−1
t ) ∩ supp(µs′t−1,z
′t−1
t ) = ∅ for any distinct pairs (st−1, zt−1), (s′t−1, z′t−1);
(c)⋃st−1∈St−1,zt−1∈Z suppµ
st−1,zt−1
t = St.
CDREU2: For all pt, At, and ht−1 = (Ak, pk, zk)t−1k=0 ∈ Ht−1(At),
61
ρt(pt, At|ht−1) =
∑(s0,...,st)∈S0×...×St
∏tk=0 µ
sk−1,zk−1
k (sk)τsk(pk, Ak)∑(s0,...,st−1)∈S0×...×St−1
∏t−1k=0 µ
sk−1,zk−1
k (sk)τsk(pk, Ak).
A consumption-dependent evolving utility representation of ρ is a CDREU representationsuch that for all t = 0, . . . , T , we additionally have:
CEVU: For all st ∈ St, there exists ust ∈ RZ such that for all zt ∈ Z,At+1 ∈ At+1, wehave62
Ust(zt, At+1) = ust(zt) + Vst,zt(At+1),
where Vst,zt(At+1) :=∑
st+1µst,ztt+1 (st+1) maxpt+1∈At+1 Ust+1(pt+1) for t ≤ T − 1 and VsT ,zT ≡ 0.
An active learning representation is a consumption-dependent evolving-utility representationsuch that additionally:
60For t = 0, we abuse notation by letting µst−1,zt−1
t denote µ0 for all st−1, zt−1.61For t = 0, we again abuse notation by letting ρt(·|ht−1) denote ρ0(·) for all ht−1.62Note that subject to multiplying Ust and ust by δT−t for each t and st, this yields the representation in
equation (9) in the main text.
23
CGL: There exists δ > 0 such that for all t = 0, . . . , T − 1, zt ∈ Z, and st ∈ St, we have63
ust =1
δ
∑st+1
µst,ztt+1 (st+1)ust+1 .
K.3 Characterization
K.3.1 Consumption-Dependent DREU
Consumption-dependent DREU is equivalent to natural analogs of Axioms 1–4 used to charac-terize DREU. The only difference is that in defining contraction equivalence, linear equivalence,the extended version of ρ, and convergence of histories, we need to restrict attention to historieswhose sequences of realized consumptions coincide.
Given ht−1 = (A0, p0, z0, ..., At−1, pt−1, zt−1) ∈ Ht−1, let (ht−1−k , (A
′k, p′k, z′k)) denote the se-
quence of the form (A0, p0, z0, ..., A′k, p′k, z′k, ..., At−1, pt−1, zt−1).64 We say that gt−1 ∈ Ht−1 is
contraction equivalent to ht−1 if for some k, we have gt−1 = (ht−1−k , (Bk, pk, zk)), where Ak ⊆ Bk
and ρk(pk, Ak|hk−1) = ρk(pk, Bk|hk−1). We say that a finite set of histories Gt−1 ⊆ Ht−1 islinearly equivalent to ht−1 if
Gt−1 = (ht−1−k , (λAk + (1− λ)Bk, λpk + (1− λ)qk, zk)) : qk ∈ Bk
for some k, Bk, and λ ∈ (0, 1].
Axiom 12 (Contraction History Independence*). If gt−1 ∈ Ht−1(At) is contraction equivalentto ht−1 ∈ Ht−1(At), then ρt(·, At|ht−1) = ρt(·, At|gt−1).
Axiom 13 (Linear History Independence∗). If Gt−1 ⊆ Ht−1(At) is linearly equivalent to ht−1 ∈Ht−1(At), then ρt(·;At|ht−1) = ρt(·;At|Gt−1).
As before, ρt(·;At|Gt−1) is shorthand for
ρt(·;At|Gt−1) :=∑
gt−1∈Gt−1
ρt(·, At|gt−1)ρ(gt−1)∑
ft−1∈Gt−1 ρ(ft−1),
where for any gt−1 = (A0, p0, z0, . . . , At−1, pt−1, zt−1), ρ(gt−1) :=∏t−1
k=0 ρk(pk, Ak|gk−1).65
Define the set of degenerate period-t histories by Dt−1 := dt−1 ∈ Ht−1 : dt−1 =(qk, qk, zk)t−1
k=0 where qk ∈ ∆(Xk)∀k ≤ t− 1.
Definition 13. For any t ≥ 1, At ∈ At, zt−1, and ht−1 ∈ Hzt−1
t−1 , define
ρht−1
t (·;At) := ρt(·;At|λht−1 + (1− λ)dt−1) (43)
63As in footnote 62, subject to multiplying Ust and ust by δT−t for each t and st, this yields the representationin equation (10) in the main text.
64In general this is not a history in Ht−1, but it is if (zk−1, A′k) ∈ supp pk−1 and (zk, Ak+1) ∈ supp p′k and
ρk(p′k;A′k|hk−1) > 0.65As for the weights ρ(gt−1) in Section 3.1, here we again do not keep track of the probabilities pk(zk, Ak+1),
as these do not reveal any private information to the analyst.
24
for some λ ∈ (0, 1] and dt−1 ∈ Dzt−1
t−1 such that λht−1 + (1− λ)dt−1 ∈ Hzt−1
t−1 (At).66
As before, it follows from Axiom 13 that the RHS of (43) does not depend on thespecific choice of λ and dt−1. Moreover, ρh
t−1
t (·;At) coincides with ρt(·;At|ht−1) wheneverht−1 ∈ Ht−1(At). Thus, in the following we again do not distinguish between the extended andnonextended version of ρt and use ρt(·;At|ht−1) to denote both.
Axiom 14 (History-dependent REU*). For all ht−1 ∈ Ht−1, conditions (i)–(v) of Axiom 3 holdafter replacing all instances of ht−1 with ht−1.
For any 0 ≤ t ≤ T , we define the set A∗t (ht−1) of period-t menus without ties condi-tional on history ht−1 ∈ Ht−1 and the set H∗t of period-t histories without ties as in themain text, replacing each instance of h and H in Definition 4 with h and H. We extendconvergence in mixture to histories in Ht by requiring additionally that each history in thesequence gives rise to the same sequence of consumptions: We write hnt →m ht for anyht = (A0, p0, z0, ..., At, pt, zt),h
nt = (An0 , p
n0 , z0, ..., A
nt , p
nt , zt) ∈ Ht such that Ank →m Ak and
pnk →m pk for each k.
Axiom 15 (History Continuity∗). For all 0 ≤ t ≤ T − 1, At+1, pt+1, and ht ∈ Ht,
ρt+1(pt+1;At+1|ht) ∈ colimnρt+1(pt+1;At+1|ht,n) : ht,n →m ht,ht,n ∈ H∗t.
Theorem 5. The following are equivalent:
(i). ρ satisfies Axioms 12–15.
(ii). ρ admits a CDREU representation.
K.3.2 Consumption-Dependent Evolving Utility
Consumption-dependent evolving utility does not in general satisfy Separability, as currentconsumption affects the expected utility over continuation menus through its effect on thetransition distributions µst,ztt+1 . Instead, it is fully characterized by analogs of Axioms 6 (DLRMenu Preference) and 7 (Sophisticaton). To state these, we make use of the same historydependent dominance relation as in Definition 5, except that histories now live in the enrichedspace Ht:
Definition 14. For each t ≤ T − 1 and ht = (ht−1, At, pt, zt) ∈ Ht, define relation %ht , ∼ht ,and ht on ∆(Xt) as follows. For any qt, rt ∈ ∆(Xt), qt %ht rt if there exist qnt →m qt andrnt →m rt such that
ρt(1
2pt +
1
2rnt ;
1
2At +
1
2qnt , rnt |ht−1) = 0
for all n. Let ∼ht and ht respectively denote the symmetric and asymmetric component of%ht .
66As before, we define λht−1 + (1 − λ)dt−1 := (λAk + (1− λ)qk, λpk + (1− λ)qk, zk)t−1k=0, where ht−1 =
(Ak, pk, zk)t−1k=0 and dt−1 = (qk, qk, zk)t−1k=0.
25
As before, qt %ht rt reveals that at all states st consistent with ht the agent prefers qt tort. Note that since the final realized consumption zt does not reveal any additional informationabout st, %ht depends only on (ht−1, At, pt) and not on zt.
We have the following analogs of Axioms 6 and 7:
Axiom 16 (DLR Menu Preference*). For all t ≤ T − 1 and ht, conditions (i)–(iv) of Axiom 6hold after replacing all instances of ht with ht.
Axiom 17 (Sophistication*). For all t ≤ T −1, ht = (ht−1, At, pt, zt) ∈ Ht, and At+1 ⊆ Bt+1 ∈A∗t+1(ht), the following are equivalent:
(i). ρt+1(pt+1;Bt+1|ht) > 0 for some pt+1 ∈ Bt+1 r At+1
(ii). (zt, Bt+1) ht (zt, At+1).
A slight departure from Axiom 7 is that because both the agent’s preference over periodt+ 1 menus and her period t+ 1 choices may be influenced by her period t consumption, point(ii) of Axiom 17 only imposes a preference for Bt+1 over At+1 when the corresponding period tconsumption is the consumption zt that arises under ht.
Theorem 6. Suppose that ρ admits a CDREU representation. The following are equivalent:
(i). ρ satisfies Axioms 16–17.
(ii). ρ admits a Consumption-Dependent Evolving Utility representation.
K.3.3 Active Learning
As with gradual learning, active learning is characterized by additional restrictions on theagent’s preference over streams of consumption lotteries. In contrast with consumption-dependent evolving utility, active learning restores a weak form of Separability, requiring thatthe agent’s payoff to today’s consumption be independent of tomorrow’s consumption lotterystream. This captures the idea that since consumption lottery streams only entail degeneratefuture choices, today’s consumption does not have any informational value in this case andhence yields the same payoff regardless of the future stream:
Axiom 18 (Stream Separability). For all ht, zt, xt ∈ Z, (`k)Tk=t+1, (mk)
Tk=t+1 ∈ ∆(Z),
1
2(zt, `t+1, . . . , `T ) +
1
2(xt,mt+1, . . . ,mT ) ∼ht
1
2(xt, `t+1, . . . , `T ) +
1
2(zt,mt+1, . . . ,mT ).
Additionally, active learning entails analogs of Axioms 8–9 that we used to characterizegradual learning:
Axiom 19 (Stationary Consumption Preference*). For all t ≤ T − 1, `,m, n ∈ ∆(Z), and ht,(`, n, . . . , n) ht (m,n, . . . , n) if and only if (n, `, n, . . . , n) ht (n,m, n, . . . , n).
We say that `,m ∈ ∆(Z) are ht-nonindifferent if (`, n, . . . , n) 6∼ht (m,n, . . . , n) for somen ∈ ∆(Z).
26
Axiom 20 (Constant Intertemporal Tradeoff*). If `,m are ht-nonindifferent and ˆ, m are gτ -nonindifferent, then for all α ∈ [0, 1] and n ∈ ∆Z:
(`,m, n, . . . , n) ∼ht (α`+ (1− α)m,α`+ (1− α)m,n, . . . , n)
⇐⇒(ˆ, m, n, . . . , n) ∼gτ (α ˆ+ (1− α)m, α ˆ+ (1− α)m, n, . . . , n).
As before, the axiom is vacuous unless we impose Consumption Nondegeneracy:
Condition 3 (Consumption Nondegeneracy*). For all t ≤ T − 1 and ht, there exist ht-nonindifferent `,m ∈ ∆(Z).
When Condition 3 is satisfied, Axioms 18–20 fully characterize the additional behavioralcontent of active learning relative to consumption-dependent evolving utility:
Theorem 7. Suppose ρ admits a consumption-dependent evolving utility representation andsatisfies Condition 3. The following are equivalent:
(i). ρ satisfies Axioms 18–20.
(ii). ρ admits an active learning representation.
K.4 Proof Sketches for Theorems 5–7
K.4.1 Theorem 5
We first extend the terminology from Section A.3 to the enriched setting. Suppose that(St′ , µ
st′−1,zt′−1
t′ (st′−1,zt′−1)∈St′−1×Z , Ust′ , τst′st′∈St′ ) satisfy CDREU1 and CDREU2 from Defi-nition 12 for each t′ ≤ t.
Fix any state s∗t ∈ St and consumption z∗t ∈ Z. We let pred(s∗t ) denote the uniquepredecessor sequence (s∗0, z
∗0 , . . . , s
∗t−1, z
∗t−1) ∈ S0 × Z × . . . × St−1 × Z, given by assump-
tions CDREU1 (b) and (c), such that s∗k+1 ∈ supp(µs∗k,z
∗k
k+1 ) for each k = 0, ..., t − 1. Givenany history ht = (A0, p0, z0, . . . , At, pt, zt), we say that (s∗t , z
∗t ) is consistent with ht if∏t
k=0 τs∗k(pk, Ak) > 0 and zk = z∗k for all k = 0, . . . , t. A separating history for (s∗t , z∗t ) is a
history ht = (B0, q0, z0, . . . , Bt, qt, zt) ∈ H∗t such that Us∗k−1,z∗k−1
(Bk, qk) = Us∗k and zk = z∗k for
all k = 0, . . . , t, where Us∗k−1,z∗k−1
(Bk, qk) := Usk : sk ∈ suppµs∗k−1,z
∗k−1
k and qk ∈M(Bk, Usk).67
Analogs of Lemmas 1 and 2 are readily established; in particular, any (s∗t , z∗t ) ∈ St × Z
admits a separating history.
Sufficiency:The sufficiency direction of the proof proceeds as in Section B.1. Specifically, assuming that forsome t ≤ T − 1 we have constructed (St′ , µ
st′−1,zt′−1
t′ (st′−1,zt′−1)∈St′−1×Z , Ust′ , τst′st′∈St′ )satisfying CDREU1 and CDREU2 for each t′ ≤ t, we construct(St+1, µst,ztt+1 (st,zt)∈St×Z , Ust+1 , τst+1st+1∈St+1) satisfying CDREU1 and CDREU2 as fol-lows:
67As usual, Us∗−1,z∗−1
(B0, q0) denotes U0(B0, q0) := Us0 : s0 ∈ S0 and q0 ∈M(B0, Us0).
27
For any (st, zt) ∈ St × Z, we define ρst,ztt+1 (·, At+1) := ρt+1(·, At+1|ht(st, zt)), where ht(st, zt)is an arbitrary separating history for (st, zt). Using Axiom 14 and Theorem 4, we then find(St+1, µst,ztt+1 (st,zt)∈St×Z , Ust+1 , τst+1st+1∈St+1) such that CDREU1 holds and
ρst,ztt+1 (pt+1, At+1) =∑
st+1∈St+1
µst,ztt+1 (st+1)τst+1(pt+1, At+1). (44)
Analogous arguments as in Section B.1 yield analogs of Lemma 3, showing that for any historyht that can only arise at (st, zt), we have ρst,ztt+1 = ρt+1(·|ht); and of Lemma 4, showing that forany ht = (A0, p0, z0, . . . , At, pt, zt) ∈ Ht(At+1), we have
ρt+1(pt+1, At+1|ht) =
∑(s0,...,st)∈S0×···×St
∏tk=0 µ
sk−1,zk−1
k (sk)τsk(Ak, pk)ρst,ztt+1 (pt+1, At+1)∑
(s0,...,st)∈S0×···×St∏t
k=0 µsk−1,zk−1
k (sk)τsk(Ak, pk). (45)
Combining (45) and (44) shows that (St+1, µst,ztt+1 (st,zt)∈St×Z , Ust+1 , τst+1st+1∈St+1) also satisfiesCDREU2.
Necessity:Necessity of Axioms 12–15 is established using analogous arguments to Section B.2.
K.4.2 Theorem 6
We first record the following analog of Lemma 5, which admits an analogous proof. Givenht = (ht−1, At, pt, zt), we say that st is consistent with ht if (st, zt) is consistent with ht, andwe call ht a separating history for st if it is a separating history for (st, zt).
Lemma 25. Suppose that ρ admits CDREU representation. Consider any t ≤ T − 1, ht =(A0, p0, z0, . . . , At, pt, zt) ∈ Ht, and qt, rt ∈ ∆(Xt).
(i). If qt %ht rt, then Ust(qt) ≥ Ust(rt) for all st consistent with ht.
(ii). Suppose there exist g, b ∈ ∆(Xt) such that Ust(g) > Ust(b) for all st consistent with ht.If Ust(qt) ≥ Ust(rt) for all st consistent with ht, then qt %ht rt.
(iii). If ht is a separating history for st, then qt %ht rt if and only if Ust(qt) ≥ Ust(rt).
Sufficiency:We proceed by an analogous inductive argument as in Section C.2. Assume that for some t ≤T−1, we have obtained (St′ , µ
st′−1,zt′−1
t′ (st′−1,zt′−1)∈St′−1×Z , Ust′ , τst′st′∈St′ ) such that CDREU1and CDREU2 hold for each t′ ≤ t and CEVU holds for each t′ ≤ t− 1.
For any (st, zt) ∈ St×Z, define ust ∈ RZ and Vst,zt : At+1 → R by ust ≡ 0 and Vst,zt(At+1) :=Ust(zt, At+1). Applying Axiom 16 yields the following analog of Lemma 6, with an analogousproof. Note that because we do not impose Separability, Vst,zt depends on zt.
Lemma 26. For all (st, zt), Vst,zt is continuous, monotone, and linear. Moreover, there existC ′t+1, Ct+1 ∈ At+1 such that Vst,zt(Ct+1) > Vst,zt(C
′t+1) for all (st, zt).
28
Applying the “moreover” part of Lemma 26 also yields the obvious analog of Corollary C.1.Since ρ admits a CDREU representation, we can obtain
(St+1, µst,ztt+1 (st,zt)∈St×Z , Ust+1 , τst+1st+1∈St+1) satisfying CDREU1 and CDREU2 at t + 1. Forall (st, zt) ∈ St × Z, define ρst,ztt+1 by ρst,ztt+1 (pt+1, At+1) :=
∑st+1
µst,ztt+1 τst+1(pt+1, At+1).Let %st,zt denote the preference over At+1 induced by Vst,zt . Using Axiom 17, an analogous
argument as for Lemma 7 shows that for all st, zt, the pair (%st,zt , ρst,ztt+1 ) satisfies AS’s Axioms 1
and 2, and an analogous argument as for Lemma 8 establishes that %st,zt satisfies AS’s AxiomDLR 6. Given this, we proceed as in Section C.2.5 (replacing each instance of st with aninstance of (st, zt)) to obtain αst+1 > 0 and βst+1 ∈ R such that Ust+1 := αst+1Ust+1 + βst+1
satisfiesVst,zt(At+1) =
∑st+1
µst,ztt+1 (st+1) maxpt+1∈At+1
Ust+1(pt+1).
Thus, replacing Ust+1 with Ust+1 , we have that (St+1, µst,ztt+1 (st,zt)∈St×Z , Ust+1 , τst+1st+1∈St+1)satisfies not only CDREU1 and CDREU2, but also CEVU, as required.
Necessity:As in Section C.3, we first show that for all t ≤ T − 1, there exist gt, bt ∈ ∆(Xt) suchthat Ust(gt) > Ust(bt) for all st ∈ St, for which it is sufficient to find C ′t+1, Ct+1 ∈ At+1
such that Vst,zt(C′t+1) > Vst,zt(Ct+1) for all (st, zt) ∈ St × Z. As before, we let C ′t+1 :=
gt+1(st+1), bt+1(st+1) : st+1 ∈ St+1 where gt+1(st+1), bt+1(st+1) ∈ ∆(Xt+1) are such thatUst+1(gt+1(st+1)) > Ust+1(bt+1(st+1)). For each st, zt, we set At+1(st, zt) := bt+1(st+1) forsome st+1 ∈ suppµst,ztt+1 . Then Vst,zt(C
′t+1) ≥ Vst,zt(At+1(s′t, z
′t)) for all st, zt, s
′t, z′t, with strict
inequality for (st, zt) = (s′t, z′t). Hence, letting Ct+1 :=
∑(st,zt)∈St×Z
1|St×Z|At+1(st, zt), we have
Vst,zt(C′t+1) > Vst,zt(Ct+1) for all st, zt.
Given this, we can proceed as in Section C.3, using part (ii) of Lemma 25 to derive Axiom 16(i), (ii), (iv) and Axiom 17. Part (iii) of Axiom 16 is also established in an analogous mannerto Section C.3.
K.4.3 Theorem 7
We first show that when ρ admits a consumption dependent evolving utility (CEVU) represen-tation, Axiom 18 (Stream Separability) is equivalent to the following form of separability overconsumption lottery streams:
Lemma 27. Suppose ρ admits a CEVU representation with utilities Ust . Then ρ satisfiesAxiom 18 if and only if for all t ≤ T − 1, st, z, z
′ and (`k)Tk=t+1, (mk)
Tk=t+1 ∈ ∆(Z), we have
Ust(z, `t+1, . . . , `T )− Ust(z′, `t+1, . . . , `T ) = Ust(z,mt+1, . . . ,mT )− Ust(z′,mt+1, . . . ,mT ). (46)
Proof. Fix any t ≤ T − 1. By the necessity direction of the proof of Theorem 6, there existgt, bt ∈ ∆(Xt) such that Ust(gt) > Ust(bt) for all st ∈ St. Then Lemma 25 (i)–(ii) implies thatfor all ht and qt, rt, we have qt ∼ht rt if and only if Ust(qt) = Ust(rt) for all st consistent withht.
Then Axiom 18 holds if and only if for all t ≤ T − 1, st, z, z′ and (`k)
Tk=t+1, (mk)
Tk=t+1 ∈
∆(Z), we have 12Ust(z, `t+1, . . . , `T ) + 1
2Ust(z,mt+1, . . . ,mT ) = 1
2Ust(z
′, `t+1, . . . , `T ) +12Ust(z
′,mt+1, . . . ,mT ), which in turn is equivalent to (46).
29
Sufficiency:Suppose that ρ admits a CEVU representation (St, µst−1,zt−1
t st−1∈St−1,zt−1∈Z , Ust , ust , τstst∈St)0≤t≤Tand satisfies Condition 3 and Axioms 18–20. Steps 1 and 2 below first perform two nor-malizations. Step 3 then proceeds in an analogous manner to the sufficiency direction ofTheorem 3.
Step 1: We first show that replacing each Ust and ust with a suitable Ust and ust continuesto yield a CEVU representation of ρ which additionally satisfies
Ust(`t, . . . , `T ) = ust(`t) + Wst(`t+1, . . . , `T ) (47)
for all st and (`k)Tk=t ∈ ∆(Z), where Wst(`t+1, . . . , `T ) :=
∑st+1
µst,ztt+1 (st+1)Ust+1(`t+1, . . . , `T )does not depend on zt.
To see this, fix any z∗0 , . . . , z∗T ∈ Z. For all s0, define Us0 := Us0 and us0(z0) :=
Us0(z0, z∗1 , . . . , z
∗T )−Us0(z∗0 , z∗1 , . . . , z∗T ) for all z0. For all s1, define Us1 := Us1 +(us0(z0)−us0(z0))
and us1 := us1 + (us0(z0)− us0(z0)), where (s0, z0) are the unique state-outcome pair such thats1 ∈ suppµs0,z01 . Note that replacing each Us0 with Us0 , us0 with us0 , Us1 with Us1 , and us1with us1 , and keeping all utilities in periods t ≥ 2 the same continues to yield a CEVU rep-resentation of ρ: Indeed, Usi and Usi represent the same preference over ∆(Xi) so CDREU1and CDREU2 remain satisfied. Moreover, CEVU holds at s1 because Us1 and us1 are obtainedfrom Us1 and us1 by adding the same constant, and CEVU holds at s0 because Us0 = Us0 andUs1 = Us1 + (us0(z0)− us0(z0)) for all s1 ∈ suppµs0,z01 implies
Us0(z0, A1) = us0(z0) +∑s1
µs0,z01 (s1) maxp1∈A1
Us1(p1) = us0(z0) +∑s1
µs0,z01 (s1) maxp1∈A1
Us1(p1).
Then, for all z0 and (`k)Tk=1 ∈ ∆(Z), we have
∑s1µs0,z01 Us1(`1, . . . , `T ) = Us0(z0, `1, . . . , `T ) −
us0(z0) = Us0(z∗0 , `1, . . . , `T ), where the last equality follows from the fact that us0(z0) :=
Us0(z0, z∗1 , . . . , z
∗T ) − Us0(z∗0 , z∗1 , . . . , z∗T ) = Us0(z0, `1, . . . , `T ) − Us0(z∗0 , `1, . . . , `T ) by Lemma 27.
Thus, for all (`k)Tk=0 ∈ ∆(Z), we have Us0(`0, `1, . . . , `T ) = us0(`0) + Ws0(`t+1, . . . , `T ), where
Ws0(`1, . . . , `T ) :=∑
s1µs0,z01 (s1)Us1(`1, . . . , `T ) does not depend on z0, whence (47) holds at s0.
Next, suppose that for some t ≥ 1, we have obtained a CEVU representation of ρ by replacingeach Us0 , us0 , . . . , Ust , ust with Us0 , us0 , . . . , Ust , ust and keeping utilities in periods t + 1, . . . , Tthe same, and suppose that (47) holds for all t′ < t. For all st and zt, define ust(zt) :=Ust(zt, z
∗t+1, . . . , z
∗T )− Ust(z∗t , z∗t+1, . . . , z
∗T ). For all st+1, define Ust+1 := Ust+1 + (ust(zt)− ust(zt))
and ust+1 := ust+1 + (ust(zt) − ust(zt)), where (st, zt) are the unique state-outcome pair suchthat st+1 ∈ suppµst,ztt+1 . Then the same argument as in the previous paragraph shows that after
replacing ust with ust and Ust+1 and ust+1 with Ust+1 and ust+1 , we continue to have a CEVUrepresentation of ρ which now additionally satisfies (47) in period t. Proceeding inductively,we obtain the desired representation.
Step 2: Next, we show that replacing each Ust and ust with a suitable Ust and ust continuesto yield a CEVU representation of ρ which again satisfies (47) and such that additionally∑
z∈Z
ust(z) = 0 (48)
30
holds for all st. Indeed, let ust(zt) := ust(zt)−γst for all st and zt, where γst := 1|Z|∑
z∈Z ust(z).
Then (48) is immediate. Inductively define Ust by UsT := usT and Ust(zt, At+1) := ust(zt) +∑st+1
µst,ztt+1 maxpt+1∈At+1 Ust+1(pt+1) for all st. To show that Ust and ust are as required, it sufficesto prove that for all t ≤ T − 1 and st,
Ust = Ust − (γst + βst), (49)
where βst := |Z|t−T∑
(zt+1,...,zT )∈ZT−t Wst(zt+1, . . . , zT ). Indeed, (49) implies that Ust and
Ust represent the same preference over ∆(Xt), so that replacing each Ust and ust withUst and ust continues to yield a CEVU representation of ρ. Moreover, (49) implies that∑
st+1µst,ztt+1 (st+1)Ust+1(`t+1, . . . , `T ) = Wst(`t+1, . . . , `T ) − βst is independent of zt for all st,
so that (47) continues to hold as well.To show (49), note first that for each sT−1 we have
UsT−1(zT−1, AT ) = usT−1
(zT−1)− γsT−1+∑sT
µsT−1,zT−1
T (sT )( maxpT∈AT
usT (pT )− γsT ) =
UsT−1(zT−1, AT )− γsT−1
− |Z|−1∑z∈Z
∑sT
µsT−1,zT−1
T (sT )usT (z) = UsT−1(zT−1, AT )−
(γsT−1
+ βsT−1
),
where the final equality holds because µsT−1,zT−1
T (sT )usT (z) = WsT−1(z) by (47). Next, assume
that (49) holds for all t′ > t. By (47), we have for all st+1 that∑(zt+1,...,zT )∈ZT−t
Ust+1(zt+1, . . . , zT )
= |Z|T−(t+1)∑zt+1
ust+1(zt+1) + |Z|∑
(zt+2,...,zT )
Wst+1(zt+2, . . . , zT ) = |Z|T−t(γst+1 + βst+1
).
(50)
Thus,
Ust(zt, At+1) = Ust(zt, At+1)− γst −∑st+1
µst,ztt+1 (st+1)(γst+1 + βst+1
)= Ust(zt, At+1)− γst − |Z|t−T
∑st+1
µst,ztt+1 (st+1)∑
(zt+1,...,zT )∈ZT−tUst+1(zt+1, . . . , zT ) =
Ust(zt, At+1)− (γst + βst),
where the first equality holds by (49) applied to Ust+1 , the second equality follows from (50),
and the final equality holds because∑
st+1µst,ztt+1 (st+1)Ust+1(zt+1, . . . , zT ) = Wst(zt+1, . . . , zT ) by
(47). Thus, (49) holds at t as well.Step 3: Finally, we argue as in Section D.1 that the CEVU representation with utilities
Ust and ust is an active learning representation, i.e., that there exists δ > 0 such that for all stand zt we have
ust =1
δ
∑st+1
µst,ztt+1 (st+1)ust+1 . (51)
31
To see this, note first that by (47) and Condition 3, ust is nonconstant for each st. Moreover,for any sT−1 ∈ ST−1 and `T−1, `T ∈ ∆(Z), we have
UsT−1(`T−1, `T ) = usT−1
(`T−1) + E[uT (`T )|sT−1],
where E[uT (`T )|sT−1] :=∑
sTµsT−1(sT ),zT−1
T usT (`T ) does not depend on zT−1 by (47). Arguingexactly as in Lemma 10, we can invoke Axiom 19 to show that E[uT |sT−1] and usT−1
representthe same preference over ∆(Z). By (48) and because usT−1
is nonconstant, this implies thatthere exists δsT−1
> 0 for each sT−1 such that E[uT |sT−1] = δsT−1usT−1
. Invoking Axiom 20 andarguing as in Lemma 11, we can show that δsT−1
= δs′T−1=: δ for all sT−1, s
′T−1.
Next, assuming that we have established (51) for all t′ ≥ t, (47) implies that
Ust−1(`t−1, . . . , `T ) = ust−1(`t−1) +T−t∑k=0
δkE[ut(`t+k)|st−1],
where E[ut|st−1] :=∑
stµst−1(st),zt−1
t ust does not depend on zt−1 by (47). Again invoking (48),Axioms 19–20, and similar arguments as in Lemmas 10–11, we can then show that (51) alsoholds at t− 1.
Necessity:Suppose ρ admits an active learning representation. Then for each t ≤ T − 1, st, and (`k)
Tk=t ∈
∆(Z), we have Ust(`t, . . . , `T ) =∑T−t
k=0 δkust(`t+k). Then (46) holds, which by Lemma 27 implies
Axiom 18. Moreover, Axioms 19–20 are verified in an analogous manner to the necessitydirection of Theorem 3.
32