decision making under uncertainty when preference information is incomplete

MANAGEMENT SCIENCEVol. 61, No. 1, January 2015, pp. 111128ISSN 0025-1909 (print) ISSN 1526-5501 (online) http://dx.doi.org/10.1287/mnsc.2014.2059

2015 INFORMS

Decision Making Under Uncertainty WhenPreference Information Is Incomplete

Benjamin ArmbrusterDepartment of Industrial Engineering and Management Sciences, Northwestern University, Evanston, Illinois 60208,

[email protected]

Erick DelageDepartment of Decision Sciences, HEC Montral, Montral, Qubec H3T 2A7, Canada, [email protected]

We consider the problem of optimal decision making under uncertainty but assume that the decisionmakers utility function is not completely known. Instead, we consider all the utilities that meet somecriteria, such as preferring certain lotteries over other lotteries and being risk averse, S-shaped, or prudent.These criteria extend the ones used in the first- and second-order stochastic dominance framework. We then givetractable formulations for such decision-making problems. We formulate them as robust utility maximizationproblems, as optimization problems with stochastic dominance constraints, and as robust certainty equivalentmaximization problems. We use a portfolio allocation problem to illustrate our results.

Keywords : expected utility; robust optimization; stochastic dominance; certainty equivalentHistory : Received June 29, 2012; accepted August 5, 2014, by Dimitris Bertsimas, optimization.

1. IntroductionThis paper questions a key and rarely challengedassumption of decision making under uncertainty:that decision makers can always, after a tolerableamount of introspective questioning, clearly identifythe utility function that characterizes their attitudetoward risk. The use of expected utility to character-ize attitudes toward risk is pervasive. In large part,this is due to Von Neumann and Morgenstern (1944),who prove that any set of preferences that a decisionmaker may have among risky outcomes can be char-acterized by an expected utility measure if the pref-erences respect certain reasonable axioms (i.e., com-pleteness, transitivity, continuity, and independence).Specifically, there exists a utility function u2 ! so that among two random variables (or lotteries), Wand Y , the decision maker prefers W to Y if and onlyif 6u4W 57 6u4Y 57.There has been much effort on determining how to

choose a utility function for a decision maker, and thiswork plays an integral part in the design of surveysfor assessing tolerance to financial risk (Grable andLytton 1999). The method for choosing a utility func-tion proposed in most textbooks on decision analysis(see, for instance, Clemen and Reilly 2001) is to makea set of pairwise comparisons between lotteries (oftenusing the Becker-DeGroot-Marschak reference lotter-ies; Becker et al. 1964) in order to identify the valueof the utility function at a discrete set of points. Theutility function is then completed by nave interpo-lation. A more sophisticated approach assumes that

the utility function has a parametric structure suchas constant absolute or constant relative risk aver-sion. For example, if a decision maker can confirmthat he is risk averse and that his preference betweenany two lotteries is invariant to the addition of anyconstant amount to all outcomes, then that decisionmaker has constant absolute risk aversion; thus, hisutility is of the form u4y5 = 1 ey . Parameters arethen resolved using a small number of pairwise com-parisons between lotteries.These approaches have important shortcomings. If

they do not assume a parametric form, then thelarge or even continuous space of outcomes mayrequire a lot of interpolation or asking the decisionmaker many questions. Even interpolation may notbe easy, because if the questions to the decision makerare binary choices between two lotteries, then theanswers will not provide the value of the utility func-tion at any point; instead, each answer will providemerely a single linear constraint on the values of theutility function on the support of these two lotter-ies. To justify a parametric form for the utility func-tion, a decision maker must be able to confidentlyaddress a question about an infinite number of lot-tery comparisons (such as that described above forutilities with constant absolute risk aversion). A morefundamental limitation is that all these proceduresconclude by selecting a single most likely utilityfunction given the evidence. In other words, theseprocedures entirely disregard other plausible choicesand the inherent ambiguity of those choices. In this

111

Down

loade

d from

infor

ms.or

g by [

132.2

11.18

9.140

] on 2

6 May

2015

, at 0

7:24 .

For p

erson

al us

e only

, all r

ights

reserv

ed.

Armbruster and Delage: Decision Making Under Uncertainty with Incomplete Preference Information112 Management Science 61(1), pp. 111128, 2015 INFORMS

paper, we will focus on these instances where knowl-edge can only be gathered using a small number ofsimple questions, and meaningful decisions must bemade even though no single utility can be unambigu-ously identified.Our approach follows in spirit the line of work in

the artificial intelligence literature on utility elicitationusing optimization for problems that only involve afinite, although possibly large, set of outcomes. Thisline of work emphasizes that utility elicitation anddecision analysis should be combined into a singleprocess in order to use all the information collectedabout the true utility function when making a deci-sion. In this context, Chajewska et al. (2000) representsthe knowledge of the decision makers preferencesusing a probability distribution over utility functionsand then judges a decision by its expected utility aver-aged over the distribution of utilities. To increase theirknowledge of the utility function, they use a valueof information criterion to select the next question tothe decision maker. In contrast to this probabilisticapproach, in Boutilier et al. (2006), the authors con-struct the set U of all utility functions that do notcontradict the available information. They then iden-tify the decision that achieves minimum worst-caseregret (i.e., regret experienced a posteriori once thetrue utility function is revealed) using a mixed-integerprogramming approach and exploiting the assumedgeneralized additive structure of the true utility.In comparison, our paper considers uncertain real-valued outcomes and proposes formulations that aremore natural for decision making and can be reducedto convex optimization problems.We motivate our discussion with the following

stochastic program:

maxx2X 6u4h4x1 557 1

where x is a vector of decision variables, X is a setof implementable decisions, and h4x1 5 is a functionmapping the decision x to a random return indexedby the scenario ; the expectation is over the randomscenarios . We assume that we have not gatheredenough information to uniquely specify u4 5. Thuswe build on the theory developed by Aumann (1962)of expected utility without the completeness axiom.This theory suggests that our incomplete preferencescan be characterized by a set of utility functions U(Dubra et al. 2004). This set describes our incompleteinformation about u4 5 and is known to contain thetrue utility function. Another situation where prefer-ences are incomplete is when groups make decisionsby consensus: here, U contains the utility functions ofthe group members, and two lotteries are incompara-ble if the group members do not agree on which ispreferred.

The set U suggests that we face a robust optimiza-tion problem. Our approach will differ, however, fromthe typical robust optimization framework, which isrobust to the possible realizations or distributionsof (see, for example, Ben-Tal and Nemirovski 1998,Delage and Ye 2010, and references therein). Instead,we are robust to the possible utilities in U and choosethe worst-case utility function.When the range of h4x1 5 is not restricted to a dis-

crete set, the only existing way of dealing with ambi-guity in the utility function is a stochastic programwith a stochastic dominance constraint (Dentchevaand Ruszczynski 2003):

maxx2X 6f 4x1 57

s.t. h4x1 5Z1with some objective function f . In these problemsthe stochastic dominance constraint, h4x1 5 Z, isdefined as 6u4h4x1 557 6u4Z57 for all utility func-tions u 2 U. This constraint ensures that the ran-dom consequences of the chosen action, h4x1 5, arepreferred to those of a baseline random variable Zfor all utility functions in U. For first-order domi-nance, U is the set of all increasing functions, and forsecond-order dominance, it is the set of all increas-ing concave functions. The limitations of stochasticdominance constraints are threefold: first, stochasticdominance does not provide guidance with respect tochoosing an objective function f ; second, the choiceof a baseline Z is not a trivial one to make; and third,the set U is very large in the case of first- and second-order dominance, and thus the stochastic dominanceconstraint may be very restrictive.We briefly describe the four main contributions of

this paper.1. In a context where preferences information is

incomplete, to the best of our knowledge, we pro-vide for the first time tractable solution methods thatcan account for information that takes the shape ofcomparisons between specific lotteries. In particular,we will show how the worst-case difference betweenexpected utilities,

infu2U

6u4h4x1 557 6u4Z5771

or eveninfu2U

6u4h4x1 5571

can be expressed as the maximum of a linear pro-gramming problem of reasonable size. This is doneby exploiting the fact that these comparisons canbe represented as linear constraints in the space ofutility functions. The importance of this contribu-tion comes from the realization that lottery com-parisons are fundamental building blocks for repre-senting ones preferences regarding risk. It can, for

Down

loade

d from

infor

ms.or

g by [

132.2

11.18

9.140

] on 2

6 May

2015

, at 0

7:24 .

For p

erson

al us

e only

, all r

ights

reserv

ed.

Armbruster and Delage: Decision Making Under Uncertainty with Incomplete Preference InformationManagement Science 61(1), pp. 111128, 2015 INFORMS 113

instance, be observed in many risk tolerance assess-ment surveys used by financial advisers, as these typ-ically involve questions such as the following, fromGrable and Lytton (1999, p. 170):

You are on a TV game show and can choose one of thefollowing. Which would you take?

(a) $1,000 in cash(b) A 50% chance at winning $5,000(c) A 25% chance at winning $10,000(d) A 5% chance at winning $100,000

That we can handle such comparisons in a tractableway opens the door to a wide range of possibilities,one of them being the allowance of more flexibility indescribing the set of utilities U involved in a stochas-tic dominance constraint.12. We present for the first time the robust (i.e.,

worst-case) certainty equivalent formulation,

maxx2X infu2Uu

146u4h4x1 55751

(see 2 for a precise definition of certainty equiva-lent) and show how under mild conditions it can bereduced to solving a small number of linear programsof reasonable size. In fact, this performance measureis a natural one to employ when there is ambigu-ity about the decision makers risk preferences, asit provides solutions that we know are preferred tothe highest amount of guaranteed return. In partic-ular, this measure has a meaningful set of units (thesame ones as h4x1 5), unlike utility measures that canbe scaled arbitrarily. The set of utilities required fortractability is the same as in first contribution and arediscussed in 3. Note that the concept of optimizedcertainty equivalent defined in Ben-Tal and Teboulle(2007), which falls in the class of convex risk mea-sures, is a completely different concept; intuitively,it can be seen as a best-case instead of a worst-caseapproach, and it does not involve ambiguity aboutthe utility function.3. Given a number of lottery comparisons, we pro-

vide a natural way of detecting when a decisionmaker is inconsistent in his stated preferences (i.e.,makes a set of comparisons that together violate theaxioms of the expected utility framework) by verify-ing whether or not a certain linear program is fea-sible. In case of inconsistency, we are able to iden-tify the closest set of feasible preferences (or closestfeasible utility function) and quantify the degree ofinfeasibility.

1 Note that in this paper, we focus on the definition of stochasticdominance that involves the comparison of expected utility undera set of utility functions. We leave the question open as to howthe conclusions that we will draw might be interpreted in termsof comparing the results of different integration operations on thecumulative density functions.

4. We measure for the first time the potential valuethat is added to the decision as more knowledge ofthe decision makers preferences is gathered, start-ing from simple knowledge of risk aversion to exactknowledge of the utility function that characterizeshis preferences. We do this by evaluating the dif-ference in the optimal worst-case certainty equiva-lent with and without the additional information. Webelieve similar insights should be obtained in situa-tions where one is worried about worst-case expectedutility. This idea could potentially be used to helpchoose among a set of questions/comparisons orwhen deciding whether the necessary effort requiredto ask these questions is worth the gain.In the next section we describe three formulations

(including the stochastic dominance formulation) thatcan be used instead of maximizing expected utilitywhen the decision makers utility function is onlyknown to lie inside a set U. In 3 we describe the setsof utilities U and how to optimize each formulationwith these sets. We then present numerical examplesinvolving a portfolio allocation problem in 4. Sec-tion 5 describes extensions of the framework to allowthe detection and correction of inconsistent behaviorand to account for characteristics of the utility func-tion that are associated with almost stochastic dom-inance. We conclude in 6.

2. FormulationsOur work examines three formulations for decisionmaking when one knows the utility function is insome set U. These formulations involve (1) optimiz-ing with a stochastic dominance constraint,

maxx2X f 4x5

s.t. 6u4h4x1 557 6u4Z57 8u 2U1 (1)

where Z is some reference random variable; (2) max-imizing the worst-case utility,

maxx2X infu2U6u4h4x1 5573 (2)

and (3) maximizing the worst-case (or robust) cer-tainty equivalent,

maxx2X infu2Uu6h4x1 571 (3)

where the certainty equivalent of a lottery (i.e., ran-dom variable) X given a utility function u is typicallydefined as the amount for sure such that one wouldbe indifferent between it and the lottery; that is,u146u4X575. To ensure uniqueness, we slightly mod-ify this definition to u6X7 2= sup8s2 u4s5 6u4X579.The robust certainty equivalent formulation (3) maxi-mizes infu2Uu6h4x1 57, the largest amount of moneywe know for sure we would be willing to exchange

Down

loade

d from

infor

ms.or

g by [

132.2

11.18

9.140

] on 2

6 May

2015

, at 0

7:24 .

For p

erson

al us

e only

, all r

ights

reserv

ed.


for the lottery h4x1 5. In the context of group decisionmaking, using the worst-case utility function meansaccommodating the groups least favored member.Because we do not know the true utility function

in these formulations, any choice from U is as justifi-able as any other. We use the worst-case utility func-tion for convenience: that choice turns out to makethese formulations very tractable. In addition to con-venience, we can also motivate the choice of utilityfrom U with an analogy to Rawls (1971) A Theoryof Justice. Rawls proposes that one imagines decidingthe structure of society behind a veil of ignorance,i.e., without knowing ones place in society. Althoughour decision makers choices are less weighty, hisignorance of their true utility function is somewhatanalogous. Rawls then argues that this leads one tofocus on the least advantaged in society and suggestsa max-min principle for allocating goods. Similarly,we focus on the least favorable utility function usingmax-min formulations.Because we seek convex formulations, we will

assume that the feasible set X is convex, the objectivefunction f in (1) is concave, the function h4x1 5 relat-ing the action to a random outcome is concave in x,and the utilities in U are risk averse to ensure thatthe objective in (2) is concave in x (the only exceptionis when we discuss S-shaped utilities). For computa-tional tractability we also assume that all the randomvariables have finite support. We assume that thereare M scenarios for , 2= 811 0 0 0 1 M 9 with associ-ated probabilities pi 2= 6 = i7.The key to our success is determining tractable rep-

resentations of

4x3U1Z5 2= infu2U

6u4h4x1 557 6u4Z571 (4)

where we sometimes drop the dependence on U andZ from our notation. Using 4x3U1Z5, we can writethe stochastic dominance formulation (1) as

maxx2X f 4x5

s.t. 4x3U1Z5 0and the worst-case utility formulation (2) as

maxx2X 4x3U1051

where we chose Z 2= 0 a.s. Unlike the other formu-lations, the robust certainty equivalent formulationis not concave but quasiconcave (see the proof inAppendix A). Thus we can solve it using a bisectionalgorithm.

Remark 1. Although it might be tempting tostraightforwardly adopt the worst-case expected util-ity formulation (2) when considering ambiguity about

the choice of utility function, one must consider withcare that when maximizing worst-case expected util-ity, one implicitly compares random variables usinga hidden (and potentially meaningless) set of orderedlotteries, which tends in particular to favor a risk-neutral attitude. We refer the reader to Appendix Cfor a thorough discussion.

Remark 2. We do not study the worst-case regretformulation

minx2X supu2U

maxx02X

E6u4h4x01 557E6u4h4x1 557

proposed in Boutilier et al. (2006), for two reasons.First, from a decision-theoretic point of view, minimaxregret as a choice function violates the independenceto irrelevant alternatives condition, which is essentialfor rationalizing preferences (see Arrow 1959). Thatcondition states that our preference between decisionx1 and x2 should not be influenced by the set of alter-natives X . Second, it is likely to be an intractable prob-lem when U is a general convex set. Intuitively, thereason is that evaluating the worst-case regret associ-ated with a fixed x reduces to solving

supu2U1x02X

ZE6u4y5y4h4x

01 55u4y5y4h4x1 557dy1

where y4 5 is the Dirac measure. Unfortunately, thecross term u4y5y4h4x01 55 prevents this from being aconvex optimization problem.

3. Worst-Case UtilitiesThe following are three common hypotheses about adecision makers utility function.1. Risk aversion: A decision maker is risk averse if

for any lottery X, he prefers 6X7 for sure over thelottery X itself. This is characterized by the concavityof the utility function.2. S-shape: Prospect theory was proposed by Kah-

neman and Tversky (1979) to bridge the gap betweennormative theories of rational behavior and behaviorobserved by experimentalists. This theory conjecturesthat preferences are affected by four factors. First, out-comes are evaluated with respect to a reference point.Second, decision makers are more affected by lossesthan by winnings. Third, the perception of winningsor losses is diminished as they get larger. Finally, theperception of probabilities is biased (i.e., overweight-ing smaller probabilities and underweighting largerones). These observations suggest that the decisionmaker is risk averse with respect to gains and riskseeking with respect to losses. Specifically, it suggestsan S-shaped utility function that is concave for gainsand convex for losses. As is typically done in the con-text of a normative study, in what follows, we will

Down

loade

d from

infor

ms.or

g by [

132.2

11.18

9.140

] on 2

6 May

2015

, at 0

7:24 .

For p

erson

al us

e only

, all r

ights

reserv

ed.


disregard the possibility of any probability assess-ment bias and focus on how to account for informa-tion that indicates that the utility function has thisparticular shape.3 Prudence: In Eeckhoudt and Schlesinger (2006),

prudence captures the fact that a decision maker ismore risk tolerant in situations where he can achievehigher returns. In particular, given any lottery involv-ing two outcomes with equal probability of occurring,a prudent decision maker will prefer adding a zero-mean risk Z to the outcome with the largest value.2Prudence is a stronger condition than risk aversionand, as shown in Appendix E, is characterized by theexistence and convexity of the derivative of the utilityfunction. It is also commonly referred to as decreasingabsolute risk aversion.In what follows, we present tractable reformula-

tion for evaluating 43U5 for three different types ofsets U that are formed from intersections of the fol-lowing sets of utility functions:

U2 2= 8u2 u is nondecreasing and concave9 1Us 2= 8u2 u is nondecreasing, convex on 41071

and concave on 60159 1U3 2= 8u2 u0 exists and is convex9 1Un 2= 8u2 6u4W057 6u4Y057= 19 1Ua 2= 8u2 6u4Wk57 6u4Yk57 8k= 11 0 0 0 1K9 0

Here, W01 0 0 0 1WK1Y01 0 0 0 1YK are given random vari-ables representing lotteries. The set of risk-averse util-ities is denoted by U2; the set of S-shaped convexconcave utilities (and the only exception to theassumption throughout the paper that utilities areconcave) is denoted by Us ; the set of prudent utilities,those with convex u0, is denoted by U3; and the setof utilities that prefer lottery Wk to lottery Yk for all kis denoted by Ua. Since adding a constant to a util-ity or multiplying it by a positive constant results inan equivalent utility, it is often necessary to normal-ize utilities. There are multiple ways of normalizingutilities. Here, we use Un to specify the scaling, spec-ifying that the utility difference between W0 and Y0is 1. For example, assuming that W0 2= 1 and Y0 2= 0a.s. enforces that u415u405= 1.As the choices of U, we focus on U2 2=Ua \Un \U2,

Us 2=Ua \Un \Us , and U3 2=Ua \Un \U2 \U3. Thesechoices all incorporate Ua, allowing one to tailor the

2 In the economics literature (see, for instance, Leland 1968), a pru-dent attitude is said to be defined by the need for larger precaution-ary savings when facing a riskier situation. Here, we adopt a defi-nition that does not rely on comparing amounts of money receivednow versus later and is therefore closer in spirit to the definition ofrisk aversion. Both of these definitions translate as imposing thatu04y5 exists and is convex.

problem to the specific preferences of a particular deci-sion maker, whether he be entirely risk averse, riskseeking over losses, or prudent. For example, U2 withno specific preferences, i.e., K = 0, reduces to the setdefining second-order dominance. We now presentfinite dimensional linear programming reformulationsof 4x3U1Z5 for these choices of U. Although thereformulations will be exact for U2 and Us , the refor-mulations will lead to a conservative approximationof high precision for U3. In the cases of U2 and U3, thereformulations can easily be reintegrated in the opti-mization model for x and give rise to a convex opti-mization problem of reasonable size.The notation used in the following results will refer

to S as the joint support of all static random vari-ables, S 2= supp4Z5[SKk=04supp4Yk5[ supp4Wk55, andwe will use yj to denote the jth smallest entry of S .For clarity of exposure, scenarios in will always beindexed by i, outcomes in S by j , and queries by k.Thus the size of our optimization problems is speci-fied by the number of queries K, the number of sce-narios M , and the size of the support N 2= S .3.1. Incorporating Lottery ComparisonsWe first address how to account for the results ofK lottery comparisons for a decision maker knownto be risk averse. Specifically, in this case evaluating4x3U1Z5 requires characterizing the optimal valueof the infinite dimensional problem

infu2U2

6u4h4x1 557 6u4Z570

Our main result states that this value can be com-puted by solving a finite dimensional linear programof reasonable size as it involves 24N +M5 variablesand MN +K +M + 2N 1 constraints (not countingthe nonnegativity constraints).

Theorem 1. The optimal value of the linear program

min11v1w

Xi

pi4vih4x1 i5+wi5Xj

6Z= yj 7j (5a)

s.t. yjvi +wi j 8 i 2 811 0 0 0 1M91j 2 811 0 0 0 1N 91 (5b)X

j

6W0 = yj 7j 6Y0 = yj 7j

= 11 (5c)

Xj

6Wk = yj 7j Xj

6Yk = yj 7j

8k= 11 0 0 0 1K1 (5d)4j+1j5 j+14yj+1 yj5

8 j 2 811 0 0 0 1N 191 (5e)4j+1j5 j4yj+1 yj5

8 j 2 811 0 0 0 1N 191 (5f)v 01 01 (5g)

Down

loade

d from

infor

ms.or

g by [

132.2

11.18

9.140

] on 2

6 May

2015

, at 0

7:24 .

For p

erson

al us

e only

, all r

ights

reserv

ed.


equals 4x3U25. Furthermore, a worst-case utility function(i.e., one achieving the infimum in (4)) is

u4y5=

8>>>>>>>:N y yN 1j+1jyj+1 yj y+

yj+1j yjj+1yj+1 yj yjy


Theorem 2. The optimal value of the linear program

min11v1w1s

Xi

pi18h4x1i5>>>>>>>>>>>>>>>>>>>>>>:

N y yN 1j+1jyj+1 yj y+

yj+1j yjj+1yj+1 yj yj y < yj+18 j 2 J+1

maxj 028j1j+19

4j 04y yj 05+j 05 yj y < yj+18 j 2 J1

14y y15+1 y < y10(10)

This is a piecewise linear function connecting the pointsu4yj5= j .Unfortunately, the general problems (1), (2), and (3)

are probably hard to solve under Us because evenmaximizing expected utility with an S-shaped util-ity function may lead to multiple local maxima. Nev-ertheless, Theorem 2 allows us to evaluate 4x3Us5(and, potentially, its derivatives using linear program-ming sensitivity analysis), despite its infinite dimen-sional nature. This suggests that nonlinear optimiza-tion methods that accept black-box representationsof the objective function should be applicable. Suchmethods rely on an oracle that can evaluate efficientlythe objective function g4x5 for a fixed x. In particular,considering the robust certainty equivalent formula-tion presented in problem (3), one could easily con-sider applying derivative-free optimization methods(see Conn et al. 2009 for a complete survey) to theproblem maxx2X g4x5, where g4x5 2= infu2Uu6h4x1 57.

Here, g4x5 can be evaluated by applying a bisectionalgorithm to find the largest value z such that theoptimal value of the linear program (9) with Z 2= zalmost surely is greater than or equal to 0.

3.3. Incorporating Prudence InformationOur results are weaker for U3. We will assume that6a1 b7 contains the support of all the random variablesinvolved in this problem. We then discretize this inter-val, adding values to S to minimize the largest gapyj+1 yj .Theorem 3. The optimal value of the linear program

4x5 2=min

111v1w

Xi

pi4vih4x1i5+wi5Xj

6Z= yj 7j

(11a)

s.t. yjvi+wij8i28110001M91 j 28110001N 91 (11b)X

j

6W0= yj 7jXj

6Y0= yj 7j=11 (11c)Xj

6Wk= yj 7jXj

6Yk= yj 7j

8k=110001K1 (11d)j+1jj+14yj+1 yj5

8j 28110001N191 (11e)j+1jj4yj+1 yj5

8j 2811210001N191 (11f)j+1jj+14yj+1 yj5

8j 28110001N191 (11g)j+1jj4yj+1 yj5

8j 28110001N191 (11h)0101v01 (11i)

is a lower bound for 4x3U35. Furthermore, an approx-imate worst-case utility function is the piecewise linearfunction

u4y5=

8>>>>>>>>>>>>>:

N y yN 1j+1jyj+1 yj y+

yj+1j yjj+1yj+1 yj yjy


4x3U35 and its worst-case utility function. Note thatreplacing 4x5 with its lower bound 4x5 in any of thethree formulations (1), (2), or (3) will always returna solution that is conservative in the sense that it isensured to be feasible and to achieve at least the levelof performance dictated by the approximate optimalvalue.

Remark 4. This theorem can be used to solveproblems with third-order stochastic dominance con-straints because third-order dominance of h4x1 5over Z is equivalent to 4x3U35 0 with K = 0.The existing approach for such problems uses thefact that 4x3U35 0 is equivalent to 6max401y h4x1 5527 6max401y Z527 for all y 2 (Ogryczakand Ruszczynski 2001). Verifying this inequality ata discrete set of points is a tractable approximation.However, Theorem 3 leads to an approximation thathas certain advantages: (1) it only imposes linear con-straints instead of quadratic ones, (2) it provides aconservative (i.e., inner instead of an outer) approx-imation for the set of feasible x ensuring that domi-nance holds for all feasible points in the approxima-tion, and (3) it allows us to account for additionalinformation about the utility function.

4. Numerical StudyIn this section, we use a portfolio optimization prob-lem to illustrate the gains that can be achieved byadopting formulations that account for the preferenceinformation that is available. In this portfolio opti-mization problem, we assume that there are n assets,and we let xi be the proportion of the total budgetallocated to asset i. Since we do not consider shortpositions, the feasible set for the vector of alloca-tions x is the convex set X 2= 8x 2n2 x 01 x 1= 19.Let i be the random weekly return of asset i. Then welet the random outcome h4x1 5 2= x be the returnof the portfolio.We consider two formulations. First, we consider a

formulation that attempts to maximize the certaintyequivalent of the constructed portfolio:

maxx2X u4x 51 (13)

where u is the utility function that would captureexactly the complete preference of our decision maker,the investor. When preference information is incom-plete, i.e., only K pairwise comparisons have beenmade by the decision maker, the utility function isonly known to lie in a set of type U2. Hence, one caneither use this information to estimate the true utilityfunction by some function u and solve problem (13)with u instead of u or solve the robust certainty equiv-alent formulation (3) with h4x1 5 = x . The latterwill effectively return a portfolio that is preferred tothe bank account with the largest fixed interest rate.

Alternatively, our second formulation attempts tomaximize expected return of the portfolio underthe constraint that this portfolio is preferred by theinvestor to the return of a given benchmark portfo-lio Z. Specifically, we are interested in solving

maxx2X 6x 7 (14a)s.t. 6u4x 57 6u4Z570 (14b)

This time, in the case of incomplete preference infor-mation, although one could replace u by some esti-mated u, we will follow the spirit of stochastic dom-inance, as presented in Dentcheva and Ruszczynski(2006), which suggests replacing constraint (14b) with

6u4x 57 6u4Z57 8u 2U2 0

Note that this approach disregards all the preferenceinformation except for the fact that the investor is riskaverse. By allowing one to replace U2 by U2 in prob-lem (14), our approach corrects for this weakness.After presenting the data used to parameterize

these problems, in what follows we present empiricalresults that demonstrate how, in a context with incom-plete preference information, decisions can improve(1) by using a worst-case analysis that accountsappropriately for this information instead of simplyusing an estimate u or being overly conservativethrough replacing U2 with U2, and (2) by gatheringpreference information that is pertinent with respectto the nature of the decision that needs to be made.Indeed, as we ask more questions, and K increases,we expect the set of potential utilities U2 to shrinkas our knowledge becomes better, and our portfolioperformance should improve.

4.1. DataWe gathered the weekly returns of the companies inthe S&P 500 index from March 30, 1993 to July 6,2011. We focused on the 351 companies that werecontinuously part of the index during this period.Although not including companies that were removedfrom the index creates some survivorship bias, ourresults should remainmeaningful because the absolutereturns are not our focus. For each run, we randomlychose 10 companies from the pool of 351 to be ourn= 10 assets. We consideredM = 50 equally likely sce-narios for the weekly asset returns, which we chooseby randomly selecting a contiguous 50-week periodof historical returns for the selected companies fromthe data. For the stochastic dominance formulation, thedistribution of the benchmark return Z is given by theweekly return of the S&P 500 index during the sameperiod.

Down

loade

d from

infor

ms.or

g by [

132.2

11.18

9.140

] on 2

6 May

2015

, at 0

7:24 .

For p

erson

al us

e only

, all r

ights

reserv

ed.


4.2. Effectiveness of Robust ApproachOur first numerical study attempts to determinewhether there is something to gain by accountingexplicitly for available preference information in ourportfolio optimization model instead of assumingmore navely that the utility function takes on one ofthe popular shapes. In our simulation, the decisionmaker is risk averse and agrees with the axioms ofexpected utility, yet he is unaware which utility func-tion captures his risk attitude. Information about thisattitude can be obtained through comparison of ran-domly generated pairs of lotteries (using the randomutility split method described in 4.3.1) and thuscan be represented by U2.3 Although he is unawareof this, the simulated decision maker, when mak-ing a comparison, acts according to the utility func-tion u4y5=20Ei420/y5+y exp420/y5, where Ei standsfor the exponential integral Ei4y5 2=

R x exp4t5/t dt.

Our experiments consist of comparing four utilityfunction selection strategies with respect to their aver-age performance at maximizing the portfolios cer-tainty equivalent over a random sets of 10 compa-nies and 50 scenarios, which are drawn as describedin 4.1.

Remark 5. The function u4y5 = 20Ei420/y5 + y exp420/y5 was chosen because it has the property thatu004y5y2/u04y5 = 20. Hence, if the decision maker isonly asked to compare lotteries that involve weeklyreturns close to 0%, then one might conclude thatthe absolute risk aversion of this decision maker isconstant (i.e., his utility function takes the exponen-tial form) when in fact his absolute risk aversion isdecreasing and scales proportionally to 1/x2.

4.2.1. Utility Function Selection Strategies. Weconsider four different approaches to dealing withincomplete preference information that takes the formof a set of pairwise comparisons under the risk aver-sion hypothesis, i.e., U2.1. Exponential fit: This approach simply suggests

approximating u4 5 with u4 5 obtained by fitting anexponential utility function of the form uc4y5 = 41exp4cy55 to the available information. For imple-mentation details, we refer the reader to Appendix 6.It is interesting to note that, when a decision makerhas constant absolute risk aversion, it is sufficient toidentify the certainty equivalent of a single lottery tolearn exactly the values that c should take. Unfortu-nately, here, the decision maker has decreasing riskaversion; hence, as U2 ! 8u4 59, the best-fitted func-tion will become unable to fit u4 5 exactly.

3 To implement, in each simulation, we used as reference lotteriesfor W0 and Y0 the minimum and maximum return that could beachieved according to the 50 selected scenarios.

2. Piecewise linear fit: This approach simply suggestsapproximating u4 5 with u4 5 obtained by fitting apiecewise linear concave utility function of the formu14y5 =mini4iy + i5 to the available informationabout the true utility function. Our implementationfollows similar lines as used for the exponential util-ity function with the single exception that we enforcethat u1 be in U2. The best-fitted piecewise linearutility function does have a more complex represen-tation: for instance, in our implementation, the num-ber of linear pieces was comparable to the size S.For implementation details, we refer the reader toAppendix G.3. Worst-case utility function: This approach sug-

gests decisions that achieve the best worst-case per-formance over the set of potential risk-averse utilityfunctions. See 3.1 for implementation details.4. Worst-case prudent utility function: This approach

suggests decisions that achieve the best worst-caseperformance over the set of potential prudent utilityfunctions. We used a discretization of 250 points toapproximate the true problem as described in 3.3.In addition, the true utility function approach plays

the role of a reference for the best performance thatcan be achieved in each decision context. This is doneby assuming that the decision maker actually knowsthat his preference can be represented by the formu4y5=20Ei420/y5+ y exp420/y5. Although we arguethat this situation is unlikely to occur in practice, wehope to verify that the approaches based on a piece-wise linear fit or the worst-case utility functions areconsistent in the sense that the decisions they sug-gest will actually converge, as more information isobtained about the decision makers preferences, tothe decisions that should be taken if the true utilityfunction was known.

Remark 6. We performed a short experiment toverify that the approximation method based on dis-cretization was accurate enough when 250 points areused. To do so, we fixed the number of lottery com-parison to 40 and evaluated the effect of using amore refined discretization grid on the value of theapproximated optimal worst-case certainty equiva-lent on 6,000 random problem instances. We observedin these experiments that when it was possible toimprove the performance by more than 0.2 percentagepoints through a 1,000-point discretization this wasnearly always already achieved using a discretizationof 250 points. Figure 1 presents statistics of this con-vergence to the value achieved with a discretizationof 1,000 points.

4.2.2. Results. Table 1 presents a comparisonof the first percentiles and averages of certaintyequivalents achieved in 10,000 experiments when

Down

loade

d from

infor

ms.or

g by [

132.2

11.18

9.140

] on 2

6 May

2015

, at 0

7:24 .

For p

erson

al us

e only

, all r

ights

reserv

ed.


Figure 1 Statistics of Convergence of the Approximate OptimalWorst-Case Certainty Equivalent When Prudence IsAccounted for Using a Discretization Grid of Growing Size

101 102 103

1.5

1.0

0.5

0

Size of discretization (in points)

Rel

ativ

e pe

rform

ance

(in p

.p.)

Notes. The average, 10th, and 90th centiles of the performance relative tothe performance achieved with a grid of 1,000 points are presented for a setof 6,000 experiments. p.p., percentage points.

maximizing the certainty equivalent under incom-plete preference information using the four utilityfunction selection strategies that described above.Note that the certainty equivalents, whose statisticsare reported in this table, were evaluated using thetrue utility function. Because our simulations did notinclude a risk-free option, optimal portfolios had neg-ative certainty equivalents on occasion in contextswhere the 50 scenarios were taken from a periodwith a declining economy. An approximate methodmight also suggest a portfolio with negative certaintyequivalent if the utility function that is used to mea-sure performance actually overestimates its certaintyequivalent.First, we can confirm that the piecewise linear and

worst-case utility function approaches suggest deci-sions whose respective performance converges, interms of first percentile and average value, to the per-formance achieved knowing the true utility function;this is because they always employ utility functions

Table 1 Comparison of the 99% Confidence Intervals of the First Percentile and Average of Certainty Equivalents Achieved in 10,000 Experiments byMaximizing the Certainty Equivalent Under Incomplete Preference Information Using Four Utility Function Selection Strategies

Certainty equivalent (in %)

5 queries 20 queries 80 queries

Approach 1st %ile Average 1st %ile Average 1st %ile Average

Exponential fit 308 103 0005 0003 600 106 0012 0005 601 206 0013 0006Piecewise linear fit 800 100 0060 0004 308 004 0011 0002 300 003 0005 0002Worst-case utility function 206 002 0014 0001 206 002 0008 0001 203 002 0006 0001Worst-case prudent utility function 206 002 0013 0001 206 002 0005 0001 202 002 0008 0001True utility function 200 002 0012 0001 200 002 0012 0001 200 002 0012 0001Notes. An experiment consists of randomly sampling a set of 10 companies as candidates for investment; a set of 50 return scenarios; and a set of 5, 20, or80 answered queries. Also, %ile stands for percentile.

that are members of U2 and because U2 ! 8u9. It isalso as expected that making the false assumption thatabsolute risk aversion is constant, i.e., using an expo-nential utility function, can potentially lead to a sig-nificant loss in performance, especially when a largequantity of information about the decision makersrisk attitude has been gathered. Indeed, the resultsindicate that in these experiments, after 80 querieswere performed, the method that used the best-fittedexponential utility function proposed portfolios thaton average were equivalent to a negative guaranteedreturn, whereas other methods were able to suggestportfolios that on average were equivalent to a 0.06%guaranteed weekly return on investment (i.e., 3.1%annually) in terms of the decision makers prefer-ences. Finally, we can confirm that choosing a port-folio based on the worst-case utility function is sta-tistically more robust, in terms of average and firstpercentile of the performance, when little preferenceinformation is available. It is also clear from Table 1that accounting for information about prudence canincrease the worst-case certainty equivalent by anaverage of 0.02 percentage points.We wish to provide slightly more intuition about

how the uncertainty about the utility function isreduced as more questions are answered and how therespective approaches succeed at fitting the unknownutility function. For this purpose, Figure 2 presentsa set of illustrations that describe the shape of theuncertainty region together with the fitted functionsas more information was obtained in one of theabove experiments. In the five-questions scenario, itis clear that there is too little information to make agood choice of utility functionhence the need for amethod that accounts for this ambiguity. In the 20-questions scenario, all three methods seem to pro-vide a good estimate of the utility function. Notethat although in this scenario the exponential func-tion seems to fit the function best, we notice at afiner resolution (in the plot for 80 questions) that itwill never exactly replicate the true attitude towardrisk. It is harder to distinguish in these illustrations

Down

loade

d from

infor

ms.or

g by [

132.2

11.18

9.140

] on 2

6 May

2015

, at 0

7:24 .

For p

erson

al us

e only

, all r

ights

reserv

ed.


Figure 2 Evolution of the Bounding Envelope of Utility Functions in U2 and of the Utility Functions Used by the Different Approaches as Observed inOne Experiment for a Growing Number of Answered Questions

0 100

0.2

0.4

0.6

0.8

5 questions

Util

ity v

alue

0 100

0.2

0.4

0.6

0.8

1.01.0

1.0

1.0 1.01.00

1.0

1.00

1.00

20 questions

0 50.85

0.90

0.95

80 questions

10 0 10 10

10 10

0 100

0.2

0.4

0.6

0.8

Util

ity v

alue

0

0.2

0.4

0.6

0.8

0 50.85

0.90

0.95

10 0 100

0.2

0.4

0.6

0.8

Returns (in p.p.)

Util

ity v

alue

10 0 100

0.2

0.4

0.6

0.8

Returns (in p.p.)5

5

5

0 50.85

0.90

0.95

Returns (in p.p.)

Worst-caseGround truthEnvelope

Piecewise linearGround truthEnvelope

ExponentialGround truthEnvelope

Notes. The ground truth function refers to u4y 5. The worst-case utility function is obtained by solving the robust certainty equivalent optimization problem.Note also that for the 80-questions scenario, magnified versions of the curves are provided to highlight the irreducible fitting error of the exponential utility.Finally, p.p. stands for percentage points.

between the quality of the fitted piecewise linear util-ity function and the worst-case utility. Note, however,that although neither of them will ever be an exact fit(given that we can see parts of the dotted line), withthe worst-case analysis approach, we can be reassuredby the fact that the misadjusted utility function thatis used is guaranteed to provide a conservative esti-mate of the certainty equivalent.

4.3. Effectiveness of Elicitation StrategiesThe following results shed some light on how deci-sions might be improved by gaining more informationabout the preferences of the decision maker. In partic-ular, we compare how performance is improved as weincrease the number of questions the decision maker isasked using the four different elicitation strategies pre-sented in 4.3.1. For simplicity, in our simulation, thedecision makers true utility function over the weeklyreturn now has a constant absolute risk aversion level

of 10: u4y5 2= 1 e10y . Note that although the deci-sion maker is unaware that his preferences can be rep-resented by this function, we assume that he nevercontradicts the conclusions suggested by such a util-ity function when comparing lotteries. Our experi-ments consist of evaluating, as the number of queriesis increased, the average performance achieved bythe robust approach over random sets of 10 compa-nies and 50 scenarios, which are drawn as describedin 4.1.

4.3.1. Elicitation Strategies. We elicit informationabout the investors preferences by asking them tochoose between the preferred two random outcomes.For simplicity, we only consider questions that com-pare a certain outcome to a risky gamble with twooutcomes (a.k.a. the Becker-DeGroot-Marschak refer-ence lottery; Becker et al. 1964). In other words, eachquery can be described by four values r1 r2 r3 and

Down

loade

d from

infor

ms.or

g by [

132.2

11.18

9.140

] on 2

6 May

2015

, at 0

7:24 .

For p

erson

al us

e only

, all r

ights

reserv

ed.


a probability p. These four values specify the question,Do you prefer a certain return of r2 or a lottery wherethe return will be r3 with probability p and r1 withprobability 1 p? If we normalize the utilities suchthat u4r15= 0 and u4r35= 1, then this query will iden-tify whether u4r25> p or not. We now describe threedifferent schemes for sequentially choosing questionsto ask the investor.1. Random utility split: This scheme lets r1 and r3 be

the worst and best possible returns, respectively, andchooses r2 uniformly from 6r11 r37. The scheme thenseeks to reduce by half the interval I 2= 8u4r252 u 2U29of potential utility values at r2. Thus we choose p sothat pu4r35+ 41 p5u4r15 is the midpoint of I .2. Random relative utility split: This scheme differs

from the previous by choosing r1 and r3 uniformly atrandom from the range of potential returns and thensetting r2 2= 4r1 + r35/2. Like the previous scheme, weseek to reduce by half the interval I 2= 8u4r252 u 2U29,and thus, we choose p so that pu4r35+ 41 p5u4r15 isthe midpoint of I .3. Objective-driven relative utility split: Unlike the

previous schemes, this scheme takes the optimizationobjective into account and seeks to improve the opti-mal objective value as much as possible regardless ofthe answer (i.e., positive or negative) to the query. Todo so, it generates 10 queries using the random rel-ative utility split scheme and for each calculates thesmaller of the optimal objective value that would bereached either with a positive answer or a negativeanswer. It then selects among the 10 queries the querythat will give the greatest improvement in the opti-mal objective value in the most pessimistic scenariowith respect to whether the answer will be positiveor negative. Mathematically speaking, in the case ofthe robust certainty equivalent model, this elicitationscheme will suggest the ith query in the list accord-ing to

i= argmaxi2811210001109

min281119

maxx2X minu28u2U46u4Wi576u4Yi57509

u4x 51

where 2 81119 captures the fact that the answer wemight get from the investor might be that 6u4Wi576u4Yi57 or that 6u4Wi57 6u4Yi57.4.3.2. Results. Whereas Figure 3 relates to the

stochastic dominance formulation, Figure 4 relates tothe robust certainty equivalent formulation. Panel (a)in Figures 3 and 4 shows how our objective valueimproves as we gain more knowledge about theinvestors preferences. Panel (b) in Figures 3 and 4focuses on the convergence of the optimal allocation.For both formulations, we observe that the total

gain between no knowledge of preferences except riskaversion and full knowledge is worth, on average,0.4 percentage points of weekly return. We can also

Figure 3 Effect of Increasing Numbers of Questions in a StochasticDomination Formulation

0 20 40 60 800.7

0.8

0.9

1.0

1.1

1.2

Number of queries

Expe

cted

retu

rn (p

.p.)

RandomRandom relativeObjective-driven relativeFull knowledge

0 20 40 60 800

0.2

0.4

0.6

0.8

1.0

Number of queries

L 1

dista

nce

(a)

(b)

Notes. Panel (a) presents the expected return (in percentage points (p.p.)),and panel (b) presents the L1 distance between optimal allocation with Kqueries and the optimal allocation with full knowledge. Shown are averagesand standard errors from 1,500 simulations.

see that the improvement in performance is quick forthe initial 1020 queries. In fact, for the robust cer-tainty equivalent formulation, four questions chosenwith the objective-driven questioning scheme increasethe average certainty equivalent of the weekly returnby 0.2 percentage points. After these first queries,the gains from additional information decrease. Thisseems to indicate that there is considerable valuein using all the preference information that is avail-able, even if minimal, thus encouraging the use ofour stochastic dominance formulation instead of theone presented in Dentcheva and Ruszczynski (2006),which here would achieve the performance associatedto zero queries.Finally, for both formulations, it is quite notice-

able that the choice of questions to ask the decisionmaker also has an important impact on performance:the improvement is faster for the more sophisticatedobjective-driven elicitation scheme than for the sim-pler schemes. We believe this should justify furtherresearch on what constitutes an optimal learningstrategy in this context.

Down

loade

d from

infor

ms.or

g by [

132.2

11.18

9.140

] on 2

6 May

2015

, at 0

7:24 .

For p

erson

al us

e only

, all r

ights

reserv

ed.


Figure 4 Effect of Increasing Numbers of Questions in a RobustCertainty Equivalent Formulation

0 10 20 30 400

0.1

0.2

0.3

0.4

0.5

Number of queries

Certa

inty

equ

ival

ent (

p.p.)

RandomRandom relativeObjective-driven relativeFull knowledge

0 10 20 30 400

0.2

0.4

0.6

0.8

1.0

Number of queries

L 1 di

stanc

e

(a)

(b)

Notes. Panel (a) presents the certainty equivalent (in percentage points(p.p.)) of the optimal portfolio as measured with respect to the true utilityfunction, and panel (b) presents the L1 distance between optimal allocationwith K queries and the optimal allocation with full knowledge. Shown areaverages and standard errors from 500 simulations.

5. ExtensionsIn this section we discuss two extensions of the frame-work. In the first subsection we consider the casewhere the decision makers preferences among thesurveyed lotteries (i.e., that he prefers Wk to Yk for allk= 11 0 0 0 1K) is inconsistent with respect to the axiomsof the expected utility framework. Our proposed solu-tion will either correct for the inconsistency by find-ing a consistent utility function that is closest to beingable to justify the stated preferences or correct for theinconsistencies by permitting a bounded perturbationof the comparisons. The second subsection extends theframework to account for the notion of almost stochas-tic dominance. We have identified three different fla-vors of this concept and propose methods of integrat-ing each of them. In both subsections, we argue thatmany of these extensions lead to only minor modifica-tions of our framework with little loss in tractability.5.1. Accounting for Elicitation ErrorsThere are many reasons why comparisons that aremade by a decision maker might be inconsistent with

the theory of expected utility theory. This could bebecause the decision makers actual preferences donot satisfy the axioms of expected utility (such asin Allais or Ellsberg paradoxes). Alternatively, manyrecent studies have identified cognitive biases that canlead a decision maker to misperceive either the sizeof a probability or the gravity of an outcome Tver-sky and Kahneman (1974). In particular, the workof Kahneman and Tversky has led to an entirely newfield studying behavioral decision making (see Kah-neman and Tversky 1979). In view of these importantissues concerning the hypotheses made by expectedutility theory and of the possibility of inaccurate com-parisons, our proposed approach is prescriptive innature. Specifically, our main objective is to help deci-sion makers that believe in the axioms of expectedutility theory to identify which decision most truth-fully reflects their attitude toward risk. Similar towhat is done in Bertsimas and OHair (2013), whenthere is a set of preferences that does not satisfy oneof the axioms, we believe the framework should iden-tify and work with utility functions that are closest tobeing able to explain the incoherent preferences. Prac-tically speaking, this means that inconsistencies canbe treated as small measurement errors that needto be corrected for to identify how the decision makertruly wishes to act although he might be unable toexpress it. Note that one might want to report to thedecision maker the amount of correction that needsto be applied to have coherent preferences in order togive a signal regarding whether the expected utilityframework is well suited to describe his preferences.Technically speaking, in this framework a set of

comparisons (i.e., that Wk is preferred to Yk for all k=11 0 0 0 1K) can be identified as inconsistent when lin-ear programs (5), (9), or (11), depending on assump-tions made about the prudent or S-shaped attitude,are diagnosed as infeasible. When inconsistencies aredetected or assumed (indeed, to err is human), wesuggest accounting for error margins in the for-mulations. Below we describe three different typesof errors that can easily be accounted for. Note thatalthough we focus on the formulation presented in3.1, similar conclusions can be drawn for the formu-lations of 3.2 and 3.3.1. If we wish to consider that noise is corrupting

the expected utility evaluation at the moment whena comparison is made, then we can easily replace thecondition 6u4Wk57 6u4Yk57 with

6u4Yk57 6u4Wk57 k1 (15)where k 0 is some positive error term (or margin)for the kth comparison. This would lead to a minorchange in constraint (5d). The smallest total

Pk k

needed for the feasibility of problem (5) to hold can

Down

loade

d from

infor

ms.or

g by [

132.2

11.18

9.140

] on 2

6 May

2015

, at 0

7:24 .

For p

erson

al us

e only

, all r

ights

reserv

ed.


then be considered a measure of the size of incon-sistency. This quantity, together with the descriptionof the closest consistent comparisons, can easily beobtained by solving linear program (5) after replacingconstraint (5d) with (15) and replacing the objective of(5) with

Pk k. Alternatively, one could assume a bud-

get for the total amount of inconsistency,P

k k ,and propose a solution that maximizes worst-case cer-tainty equivalent in this context. This can easily bedone by replacing constraint (5d) in problem (15) andadding k 018k andX

k

k

before applying duality when formulating the equiv-alent augmented linear constraints for 4x3U1Z5 0.2. A second option assumes that the error is in

the perception of the outcome values: that randomvariable W is perceived as W + . If min maxa.s., then we could replace the condition 6u4Wk57 6u4Yk57 with 6u4Wk + max57 6u4Yk + min57. Inour formulations we would then need to replace theparameters Wk with Wk + max and Yk by Yk + min,which retains the linear structure of the problem.3. In the spirit of Bertsimas and OHair (2013), we

could require that 1 of the K lottery comparisonshold: that the decision maker is mistaken about atmost K of his lottery comparisons. In that case wewould introduce binary variables i into (5), whichwould be 1 if the decision maker is mistaken aboutlottery i, and add the constraint

PKi=1 i K. We

would then replace the condition 6u4Wk57 6u4Yk57with the two constraints iM+6u4Wk57 6u4Yk57 and41 i5M + 6u4Yk57 6u4Wk57, where M is a largeconstant (big M). Since this turns the calculation of4x5 (5) into a mixed-integer linear program, solvingthe master problem becomes harder but potentiallysolvable using cutting-plane methods.

5.2. Almost Stochastic DominanceThe idea of reducing the severity of stochastic dom-inance constraints by assuming additional structureis not a recent one. Since the introduction of thenotion of stochastic dominance, there have been a fewattempts at reducing the severity of the constraint.Of course, the earliest appearance would be the ideathat a higher-order stochastic dominance constraint isless restrictive. This translates as imposing the con-cavity/convexity of a higher-order derivative of theutility function. Three other instances are presentedbelow. The first two are close in spirit to our frame-work, as they make assumptions about the utilityfunctionthat is, properties that the first and sec-ond derivative must satisfy. Unlike our framework,however, it is unclear how one might validate withthe decision maker such hypotheses about deriva-tives and how one might perform optimization in theresulting space. The third instance is more similar in

flavor to the methods that are proposed in 5.1, as itsuggests inflating the set of feasible random variablesby adding random variable that are close enoughto a nondominated one.4 Although this approachappears more tractable, nothing is known as to whattype of preference axioms would suggest using thisapproach. For all three instances, we propose waysof extending our results to implement the proposedrelaxation.5.2.1. Meyers Relaxation. Meyer (1977) appears

to be the first mention of the idea of relaxing thestochastic dominance constraint by imposing struc-tural properties on the utility functions in U that gobeyond the sign of derivatives. Specifically, Meyersuggests imposing bounds on the ArrowPratt mea-sure of absolute risk aversion:

UM4r11 r25 2= 8u2 r14x5u004x5/u04x5 r24x590He explains how to identify for a specific pair 4X1Z5the worst-case utility function using dynamic pro-gramming. It is unclear, however, how one would goabout optimizing when the stochastic dominance con-straint involves this utility set. The following corollarysheds some light on the question.

Corollary 1. The optimal value of the linear pro-gram (11) with the additional constraints

r14yj5j j 8 j 2 811 0 0 0 1N 91r24yj5j j 8 j 2 811 0 0 0 1N 91

is a lower bound for 4x3U3 \UM5.Indeed, when we account for prudence, our approx-imate linear program optimizes j and j variablesthat play the respective role of first and second deriva-tives of the utility function; thus it is possible to fur-ther impose that the ArrowPratt measure fall in theappropriate range at the yj locations. This gives rise,through duality theory, to conservative approxima-tions for the robust certainty equivalent problem andthe stochastic dominance problem that account forinformation about absolute risk aversion. Once again,as the interval of realizations is further discretized, itis expected that the approximation will converge tothe true optimal value.5.2.2. Leshno and Levys Relaxation. To reduce

the severity of stochastic dominance constraints,Leshno and Levy (2002) suggest intersecting the util-ity sets associated with first- or second-order stochas-tic dominance constraint either with

ULL1 45=8u 2 inf

yu4y5 u4y5 inf

yu4y541/ 158y 2


or with

ULL2 45=8u 2 inf

yu4y5u4y5 inf

yu4y541/158y2


errors appropriately), (2) how to choose the questionsthat will most efficiently guide the optimization to agood decision, and (3) what other characteristics ofreasonable preference systems can be accounted for ina tractable way using this framework.

AcknowledgmentsThe authors are thankful to the referees for their con-structive criticism. E. Delange gratefully acknowledges thesupport of the Canadian Natural Sciences and EngineeringResearch Council [Grant 386416-2010].

Appendix A. Quasiconcavity of the Objectivein Problem (3)

For any fixed u, the certainty equivalent u6h4x1 57 isquasiconcave in x because u6h4x1 57 t is equivalentto 6u4h4x1 557 u4t5 and the left-hand side is concavein x. Thus, the pointwise infimum infu2U u6h4x1 57 is qua-siconcave in x with infu2U u6h4x1 57 t equivalent to4x3U1 t5 0. Hence the optimum of (3) is greater than t ifand only if the constraint 43U1 t5 0 has a feasible solu-tion x 2X . Thus if we can bound the optimum to an interval6t01 t17, then we can use the bisection algorithm on t to solvethe problem to precision by solving O4log41/55 feasibilityproblems of the type 43U1 t5 0. In fact, any bound onh4x1 5 gives a bound on the optimum: if t0 h4x1 5 t1 forall and all x 2X , then the optimum is in 6t01 t17.

Appendix B. Proof of Theorem 2The proof is similar to that of Theorem 1, and we start bydefining

U 415 2= 8u2 u4yj5= j1j 2 u04yj58 j90

Then by a similar argument,

4x3Us5=min

4x3U 415\Us5U 415\Us 6=1 U 415Ua1U 415Un0

The constraint U 415 \Us 6= is represented by (9f)(9i)and 0, U 415Un by (9d), and U 415Un by (9e).

Again, since 6u4Z57 is a constant for u 2 U 415,evaluating 4x3U 415 \ Us5 is equivalent to minimizing6u4h4x1 557 over u 2 U 415 \ Us . Among the S-shapedfunctions in U 415, this is minimized by the piecewiselinear function u in (10). The reason u is minimal is forthe concave portion is the same as in the proof of Theo-rem 1. For the convex portion, u is minimal because thelines 84y1 s52 s = j4y yj5 + j9 with j 2 J are support-ing hyperplanes for the convex portion of any function inU 415. Thus, when U 415\Us 6=,

4x3U 415\Us5 = 6u4h4x1 5518h4x1 5< 097+ 6u4h4x1 5518h4x1 5 097X

j

6Z= yj 7j 0

Since u is concave for y 0, the formulation (7) can beused for that part. When y < 0, u is convex, and thus,

u4y5=mins

s

s.t. s j4h4x1 i5 yj5+j 8 j 2 J0Putting these pieces together gives us the objective (9a) andthe constraints (9b), v 0, and (9c). Appendix C. Worst-Case Expected UtilityFormulation Biased Toward Risk NeutralityWhen using the worst-case expected utility approach, if thereis a large enough amount of ambiguity in the risk attitude toimplement, a risk-neutral attitude is used to select the deci-sion. Take, for instance, the case of U2, where no informa-tion is known about the risk attitude (i.e., K = 0 so that nolotteries have been compared). If the normalization schemeis such that u4a5 = 0 and u4b5 = 1, then one can show thatthe linear utility u4y5 = 4y a5/4b a5 is a worst-case util-ity for evaluating any lottery Z supported on the interval6a1 b7. When comparing the application of the robust cer-tainty equivalent formulation (3) to this context, one caneasily demonstrate that the infimum is achieved as goesto 0 in uZ4y5 = min413 41 54y a5/4inf Z45 a55 suchthat lim&0uZ 4Z5 = inf Z45 under this alternative formu-lation, thus implementing extreme risk aversion. Based onthis example, one might argue that the use of worst-caseexpected utility might be misleading because it actuallydoes not return cautious decisions in situations where itcould.

In fact, when maximizing the worst-case expected utility,one should be aware of the following connection with therobust certainty equivalent formulation. It is actually thecase that both methods are special cases of an approach thatseeks the optimal solution to

maximizex2X sup8v 2V 2 6u4Zv57 6u4h4x1 557 8u 2U9 1where 8Zv9v2V , for some V , is an indexed set of lot-teries parameterized by v such that v0 > v implies thatZv0 Zv. In other words, this more general formulation sug-gests selecting the decision for which the random variableh4x1 5 is known to be preferred to the most attractive lot-tery in 8Zv9v2V . When Zv is simply the certain amount v,this reduces to maximizing the robust certainty equivalentformulation. Perhaps less trivial is the fact that when Uis normalized so that all utility functions return u4a5 =0 and u4b5 = 1, then choosing 8Zv9v2V to be the lotteriesthat return b with probability v and return a with prob-ability 1 v makes this general approach reduce to theworst-case expected utility formulation. More formally, wehave that

sup8v2 601172 6u4Zv576u4h4x15578u2U9=sup8v2 601172 vu4b5+41v5u4a56u4h4x15578u2U9=sup8v2 601172 v6u4h4x15578u2U9= inf

u2U6u4h4x15570

It is because of this somewhat arbitrary choice of the set8Zv9v2V , which is implicitly used by the worst-case expected

Down

loade

d from

infor

ms.or

g by [

132.2

11.18

9.140

] on 2

6 May

2015

, at 0

7:24 .

For p

erson

al us

e only

, all r

ights

reserv

ed.


utility formulation to map and compare h4x1 5, that oneends up obtaining decisions that are biased in a potentiallyunappealing way.

Appendix D. Proof of Theorem 3As in the proof of Theorem 1, we define U 45 2= 8u2 u4yj5=j 8 j9. Then by an argument similar to one used in theproof of Theorem 1,

4x3U35=min

4x3U 45\U2 \U35U 45\U2 \U3 6=1U 45Ua1U 45Un0

The constraint U 45Un is represented by (11c) and U 45Un by (11d). Since we only seek a lower bound, we canrepresent U 45\U2 \U3 6= by the constraints (11e)(11h), 0, and 0.

Again, since 6u4Z57 is a constant for u 2 U 45, eval-uating 4x3U 45 \ U2 \ U35 is equivalent to minimizing6u4h4x1 557 over u 2U 45\U2 \U3. Among the utilities inU 45\U2 (thus giving a lower bound), this is minimized bythe function u in (12), as in the proof of Theorem 1. Thus,we seek to minimize 6u4h4x1 557Pj 6Z= yj 7j . Since uis concave, we use the formulation (7), which then gives usthe objective (11a) and the constraints (11b) and v 0. Appendix E. Prudence Implies the Existence andConvexity of u04 5Based on the definition of prudence as put forth by Eeck-houdt and Schlesinger (2006), we can conclude that for aprudent decision maker,

w v ) E6u4w+Z57u4w5 E6u4v+Z57u4v51for any pair 4v1w5 22 and for any random variable Z withzero mean. Here, we will first demonstrate that if a decisionmaker is prudent, then the derivative of u4 5 must exist onits domain. We follow with a proof that u04 5 is convex.

Proposition 1. If a decision maker is prudent and riskaverse, then the utility function that captures his attitude withrespect to risk must be differentiable everywhere in the interior ofits domain.

It is a well-known fact that risk aversion implies that theutility function is monotonic and concave. It must thereforebe differentiable almost everywhere and semidifferentiableeverywhere. Let us assume that at w0 in the interior of thedomain, the utility function is not differentiable. Since it issemidifferentiable at w0, we must have that

lim&0

u4w0+ 5u4w05

= u0+4w05exists and is strictly smaller than

lim&0

u4w05u4w0 5

= u04w05by concavity. Furthermore, since the utility function is dif-ferentiable almost everywhere, there must also exist a valuev0 < w0 where the utility function is differentiable. Hencewe have that if Z is a random variable that puts half of theweight on and half on , then

lim&0

41/54E6u4v0+Z57u4v055= lim

&041/54005u4v0 5+ 005u4v0+ 5u4v055= 00

Yet, since v0 w0, by the prudence hypothesis we must alsohave that for any > 0,

E6u4v0+Z57u4v05 E6u4w0+Z57u4w050By dividing both sides of the inequality by and taking thelimit as goes to zero, we get

lim&0

41/54E6u4v0+Z57u4v055 lim

&041/54E6u4w0+Z57u4w055

= lim&0

41/54005u4w0+ 5+ 005u4w0 5u4w055= 005u0+4w05 005u04w05< 0 0

This establishes that the expression lim&041/54E6u4v0+Z57u4v055 is both strictly smaller than zero and equal to zero,which is a contradiction.

Note that this proof excludes the boundaries of thedomain. We are left with proving that the first derivative ofthe utility function is convex.

Let w be any value in the interior of the domain of u4 5,let > 0, and let Z be any zero-mean random variable sup-ported on two points in the domain of u4 5. We have fromEeckhoudt and Schlesinger (2006) their definition of pru-dence that

E6u4w+ +Z57u4w+ 5 E6u4w+Z57u4w5and therefore that

41/54E6u4w++Z57u4w+55 41/54E6u4w+Z57u4w550Taking the limit of the difference between the left and rightsides, we get

lim&0

41/54E6u4w++Z57E6u4w+Z57u4w+5+u4w55 0

so thatE6u04w+Z57u04w5 00

The last inequality can be shown to be equivalent to thedefinition of convexity.

Appendix F. Fitting an Exponential UtilityFunction to U2To fit a utility function, common practice typically suggestsfixing the utility value at two reference points u4y05= 0 andu4w05= 1 and using queries to locate the relative utility val-ues achieved at a set of returns uj u4yj5 8 j = 1121 0 0 0 1 J .The best-fitted function is then the one that maximizesthe following mean square error problem:

mina1b1c

JXj=1

4a41 exp4cyj55+ buj52

s.t. a41 exp4cy055= 0 and a41 exp4cw055= 11a 01 c 00

Although nonconvex, this problem is typically consideredcomputationally feasible since it reduces to a search overthe single parameter c. We adapt this procedure to the

Down

loade

d from

infor

ms.or

g by [

132.2

11.18

9.140

] on 2

6 May

2015

, at 0

7:24 .

For p

erson

al us

e only

, all r

ights

reserv

ed.


context where the preference information takes the shapeof U2. Specifically, without loss of generality, we first letY0 and W0 be certain lotteries and fixed 6u4Y057 = 0 and6u4W057= 1. Next, for a set of 8yj9Jj=1, we can use the infor-mation in U2 to evaluate a range of possible utility valuesat each yj . We let uj take on the midvalue of this interval,uj 2= 4minu2U2 u4yj5+maxu2U2 u4yj55/2, hence capturing thefact that we wish the exponential utility function pass asclose as possible to the center of the intervals in which weknow the function should pass. We solve the same meansquare error problem to select our best-fitted exponentialutility function u4y5. Note that this approach reduces tothe method described above when U2 = 8u4 5u4yj5 = uj9.For computational reasons, our implementation used theset 8yj9

Jj=0 2=

SKk=04supp4Yk5 [ supp4Wk55, which uniformly

spanned the range of possible returns.

Appendix G. Fitting a Piecewise LinearUtility Function to U2To fit a piecewise linear utility function, we follow a simi-lar procedure as for fitting an exponential function. Namely,for a set of 8yj9

Jj=1 that includes y0 and w0, after consid-

ering that u4y05 = 0 and u4w05 = 1, we can use the infor-mation in U2 to evaluate a range of possible utility valuesat each yj . We let uj take on the midvalue of this interval,uj 2= 4minu2U2 u4yj5+maxu2U2 u4yj55/2, hence capturing thefact that we wish the utility function pass as close as pos-sible to the center of the intervals in which we know thefunction should pass. Based on the discretization 8yj9

Jj=1, we

parameterize the piecewise linear function using the valueand supergradient at each point in the set. We are left withsolving the following optimization problem:

min1

JXj=1

4j uj52

s.t. j+1j j+14yj+1 yj5 8 j1j+1j j4yj+1 yj5 8 j1j j1 j4yj yj15 8 j1j j1 j14yj yj15 8 j1j4y05 = 01 j4w05 = 11 01

where j4y05 and j4w05 are the respective indexes of the y0 andw0 terms in the set 8yj9

Jj=0. Again, for computational reasons,

our implementation used the set 8yj9Jj=0 2=

SKk=04supp4Yk5 [

supp4Wk55, which uniformly spanned the range of possiblereturns.

ReferencesArrow KJ (1959) Rational choice functions and orderings. Economica

26(102):121127.

Aumann RJ (1962) Utility theory without the completeness axiom.Econometrica 30(3):445462.

Becker GM, DeGroot MH, Marschak J (1964) Measuring utility by asingle-response sequential method. Behavioral Sci. 9(3):226232.

Ben-Tal A, Nemirovski A (1998) Robust convex optimization. Math.Oper. Res. 23(4):769805.

Ben-Tal A, Teboulle M (2007) An old-new concept of convex riskmeasures: The optimized certainty equivalent. Math. Finance17(3):449476.

Bertsimas D, OHair A (2013) Learning preferences under noiseand loss aversion: An optimization approach. Oper. Res. 61(5):11901199.

Boutilier C, Patrascu R, Poupart P, Schuurmans D (2006)Constraint-based optimization and utility elicitation usingthe minimax decision criterion. Artificial Intelligence 170(89):686713.

Chajewska U, Koller D, Parr R (2000) Making rational decisionsusing adaptive utility elicitation. Proc. 17th Natl. Conf. Artifi-cial Intelligence (American Association for Artificial Intelligence,Palo Alto, CA), 363369.

Clemen RT, Reilly T (2001) Making Hard Decisions with DecisionToolsSuite, 2nd ed. (Duxbury, Pacific Grove, CA).

Conn AR, Scheinberg K, Vicente LN (2009) Introduction toDerivative-Free Optimization (Society for Industrial and AppliedMathematics, Philadelphia).

Delage E, Ye Y (2010) Distributionally robust optimization undermoment uncertainty with application to data-driven problems.Oper. Res. 58(3):595612.

Dentcheva D, Ruszczynski A (2003) Optimization with stochasticdominance constraints. SIAM J. Optim. 14(2):548566.

Dentcheva D, Ruszczynski A (2006) Portfolio optimization withstochastic dominance constraints. J. Banking Finance 30(2):433451.

Dubra J, Maccheroni F, Ok E (2004) Expected utility theory withoutthe completeness axiom. J. Econom. Theory 115(1):118133.

Eeckhoudt L, Schlesinger H (2006) Putting risk in its proper place.Amer. Econom. Rev. 96(1):280289.

Grable J, Lytton RH (1999) Financial risk tolerance revisited: Thedevelopment of a risk assessment instrument. Financial ServicesRev. 8(3):163181.

Kahneman D, Tversky A (1979) Prospect theory: An analysis ofdecision under risk. Econometrica 47(2):263291.

Leland HE (1968) Saving and uncertainty: The precautionarydemand for saving. Quart. J. Econom. 82(3):465473.

Leshno M, Levy H (2002) Preferred by all and preferred bymost decision makers: Almost stochastic dominance. Man-agement Sci. 48(8):10741085.

Lizyayev A, Ruszczynski A (2012) Tractable almost stochastic dom-inance. Eur. J. Oper. Res. 218(2):448455.

Meyer J (1977) Choice among distributions. J. Econom. Theory 14(2):326336.

Ogryczak W, Ruszczynski A (2001) On stochastic dominanceand mean-semideviation models. Math. Programming 89(2):217232.

Rawls J (1971) A Theory of Justice (Harvard University Press, Cam-bridge, MA).

Tversky A, Kahneman D (1974) Judgment under uncertainty:Heuristics and biases. Science 185(4157):11241131.

Von Neumann J, Morgenstern O (1944) Theory of Games and Eco-nomic Behavior (Princeton University Press, Princeton, NJ).Do

wnloa

ded f

rom in

forms

.org b

y [13

2.211

.189.1

40] o

n 26 M

ay 20

15, a

t 07:2

4 . Fo

r pers

onal

use o

nly, a

ll righ

ts res

erved

.

decision making under uncertainty when preference information is incomplete

Documents