making sense of uncertainty: advantages and disadvantages ... ?· advantages and disadvantages of...

Download Making sense of uncertainty: Advantages and disadvantages ... ?· Advantages and disadvantages of providing…

Post on 25-Aug-2018

217 views

Category:

Documents

1 download

Embed Size (px)

TRANSCRIPT

  • In Press, Journal of Risk Research

    Making sense of uncertainty:

    Advantages and disadvantages of providing an evaluative structure

    Nathan F. Dieckmann

    Decision Research, Eugene, Oregon

    Ellen Peters

    Department of Psychology, Ohio State University, Columbus, Ohio

    Robin Gregory

    Decision Research, Eugene, Oregon

    Martin Tusler

    Department of Psychology, Ohio State University, Columbus, Ohio

    Correspondence concerning this article should be addressed to Nathan F. Dieckmann, Decision

    Research, 1201 Oak Street, Suite 200, Eugene, OR 97401. Telephone (541) 485-2400, fax (541)

    485-2403. E-mail: ndieckmann@decisionresearch.org

    Acknowledgements

    We would like to acknowledge the generous support of the National Science Foundation that

    made this work possible: NSF Award #0725025 to Decision Research (Robin Gregory, PI) and

    NSF Award #0925008 to Decision Research (Nathan Dieckmann, PI). All views expressed in

    this paper are those of the authors alone.

  • 2

    Abstract

    In many decision contexts, there is uncertainty in the assessed probabilities and expected

    consequences of different actions. The fundamental goal for information providers is to present

    uncertainty in a way that is not overly complicated, yet sufficiently detailed to prompt decision

    makers to think about the implications of this uncertainty for the decision at hand. In two

    experiments, we assess the pros and cons of providing an evaluative structure to facilitate the

    comprehension and use of uncertainty information and explore whether people who vary in

    numeracy perceive and use uncertainty in different ways. Participants were presented with

    scenarios and summary tables describing the anticipated consequences of different

    environmental-management actions. Our results suggest that different uncertainty formats may

    lead people to think in particular ways. Lay people had an easier time understanding the general

    concept of uncertainty when an evaluative label was presented (e.g., uncertainty is High or Low).

    However, when asked about a specific possible outcome for an attribute, participants performed

    better when presented with numerical ranges. Our results also suggest that there appear to be

    advantages to using evaluative labels in that they can highlight aspects of uncertainty

    information that may otherwise be overlooked in more complex numerical displays. However,

    the salience of evaluative labels appeared to cause some participants to put undue weight on this

    information, which resulted in value-inconsistent choices. The simplicity and power of providing

    an evaluative structure is a double-edged sword.

    Keywords: uncertainty; decision analysis; risk communication; ambiguity; evaluability.

    Word count: 6,992

  • 3

    Making sense of uncertainty:

    Advantages and disadvantages of providing an evaluative structure

    Introduction

    In many decision contexts, there is uncertainty in the assessed probabilities and expected

    consequences of different events. It is important that these uncertainties are communicated to

    decision makers in a manner that is easily evaluated and integrated into the decision-making

    process. Good communication should facilitate a thorough deliberation about the risks and

    benefits of different options so that decision makers can make informed, value-consistent

    choices. The fundamental issue for information providers is deciding how to present uncertainty

    in a way that is not overly complicated (which can cause some users to misunderstand or ignore

    uncertainty), yet sufficiently detailed to prompt decision makers to think about the implications

    of uncertainty for the decision at hand.

    There are several alternatives for communicating uncertainty, which include numbers (e.g.,

    probabilities, ranges, or distributions), verbal probability statements (e.g., highly unlikely), and

    evaluative labels (which might characterize uncertainty as High or Low; Peters et al. 2009).

    We focus on two common presentation formats: numerical uncertainty ranges and evaluative

    labels. In two experiments, we assess the pros and cons of providing an evaluative structure to

    facilitate the comprehension and use of uncertainty information and explore whether people who

    vary in numeracy perceive and use uncertainty in different ways. Participants were presented

    with scenarios and summary tables describing the anticipated consequences of different

    environmental-management actions. These decision problems were designed to mimic typical

    deliberative decision-making contexts.

  • 4

    1.1 Evaluative Labels

    In general, decision makers will tend to use information to the extent that they can map it to

    some evaluative scale (e.g., good/bad; Hsee et al. 1999). Some pieces of information will be

    easier to evaluate because salient comparisons can be made from experience (e.g., costs) or with

    other choice options in the decision context (e.g., alternative 1 costs twice as much as alternative

    2). However, unfamiliar numeric information can often be difficult to evaluate, and may,

    consequently, be misinterpreted or underweighted in the decision-making process. Overly

    complex representations of uncertainty often fall into this category of unfamiliar, not easily

    evaluable numeric information. Thus, providing an evaluative context could help decision

    makers make better use of this unfamiliar information.

    There are many real-world examples of simplifying evaluative structures being used to

    increase the understanding and use of uncertainty information. In the weather forecasting

    domain, color codes are often used to distinguish different levels of uncertainty in weather

    forecasts (e.g., Joslyn et al. 2005). Presumably, color coding simplifies the comparison of

    different forecasts and improves comprehension. In the intelligence domain, evaluative verbal

    labels have been used to communicate analytic uncertainty. In a recent National Intelligence

    Estimate (NIE) on Irans nuclear intentions and capabilities, analysts used (albeit inconsistently)

    verbal evaluative labels to express analytic confidence in their estimates and assessments (i.e.,

    High confidence, Medium confidence, or Low confidence). In the environmental risk

    management domain, legal mandates for increased public participation have amplified the role of

    deliberative processes in risk management decisions. Risk managers and decision makers need to

    communicate effectively with a variety of stakeholders including the lay public. Consequently,

  • 5

    verbal evaluative labels are often used to simplify communication about the uncertainty

    associated with proposed management actions.

    However, to our knowledge there has been relatively little experimental work focusing on

    how these evaluative structures affect the perception and use of uncertainty information by end-

    users. In the health communication domain, evaluative labels have been shown to improve the

    use of unfamiliar numeric information (Peters et al. 2009). In one study, participants were

    presented with a hospital judgment task in which different hospitals (i.e., the alternatives) were

    described by several different hospital quality indices (i.e., the attributes). When these quality

    indicators were presented on numeric scales only (e.g., Ease of getting a referral on 0100 point

    scale), decision makers did not weigh the information as much compared to when the attributes

    were also marked with evaluative labels that classified the score as Poor, Fair, Good, or

    Excellent. These results were found to be most consistent with an affective explanation.

    Specifically, the affect derived from the manipulation appeared to either spotlight particular

    information for use in choice, or to directly motivate choice. In another study, interpretive labels

    for test results (the test came back positive or abnormal) induced larger changes to risk

    perceptions and behavioral intentions than did numeric results alone (Zikmund-Fisher et al.

    2007).

    1.2 The Use of Numerical Ranges and Evaluative Labels

    Based on Slovics (1972) concreteness principle, decision makers will tend to use

    information in the form in which it is provided. This principle likely also applies to uncertainty

    information. Numerical uncertainty ranges and evaluative labels may highlight different aspects

    of uncertainty. As an example, consider Figure 1, which displays a range of different

    management alternatives along with the key evaluation criteria. In this case, the focus is on the

  • 6

    effects of different environmental-management strategies on costs and the local bird population.

    This type of presentation is commonly referred to as a consequence matrix (Keeney 1982) or

    facts box, and is extensively used as a decision aid to facilitate individual or group decision

    making in a variety of domains (e.g., Clemen 2004; Gregory et al. forthcoming). Its purpose is to

    clarify different alternatives and illustrate how the options differ on important attributes.

    Clarifying the primary tradeoffs is thought to allow decision makers to integrate their personal

    values and make an informed decision.

    [Figure 1 near here]

    In Figure 1, the confidence (or uncertainty) in the estimated bird population is des

Recommended

View more >