managing uncertainties by ipcc: battle of...
TRANSCRIPT
Managing uncertainties by IPCC: battle of disciplines
by Rob Swart
Which changes took place since the 1990 1st assessment and the 2007 AR4?
What are similarities and differences between Working Groups?
Uncertainty developments in IPCC
• FAR (1990): careful formulation (WG1)
•AR4 (2007): all WGs use (improved) Guidance, but select different options, and uncertainty focus mainly in SPM
• TAR (2001): Guidance for authors, adapted by WG1, applied by WG2 and rejected by WG3
• SAR (1995): WG1 includes special section, WG2 introduced “confidence levels”; WG3: silence
Working Group I (climate scientists)
SAR: special section about uncertainties, “project”, “the balance of evidence suggests”
• AR1: “certain of”, “confident about”, ”predict”
TAR: guidance note; not accepted but adapted by WG1: likelihood scale, “new and stronger evidence” (repeated in AR4)
subjective (bayesian – “degree of belief”) perspective, but hidden in definition of
‘likelihoods’ (“judgmental”)
Working Group II (impact researchers)
AR1: little attention, cautious but inconsistent formulation of findings
SAR: introduction confidence levels (H, M. L)TAR: guidance note, selectively
applied but generally as intended: confidence levels and 2D qualitative scale (also in AR4)
Explicit bayesian perspective
FAR: hardly any attention
Working Group III (economists, engineers)
SAR: description of definitions, assumptions, methodsTAR: guidance hardly used, but •stresses importance choices made
•elaborates background methodologies (top-down versus bottom-up)
•sometimes uses undefined/subjective terminology
•probabilities for reduction potential options
AR4: 2D qualitative approach
IPCC Guidelines TAR (2001): stepwise approach
1. Identification of key factors2. Document ranges and distributions3. Determine precision4. Characterize value distribution: use confidence
levels (1D) or evaluate the amount of evidence athe level of agreement (2D)
5. Value and describe the underlying information6. Describe the origin (“traceable account”)7. (Use a probabilistic framework if applicable)
This is were the emphasis has been
Issues such as “framing”, unquantifiable uncertainties, selection of indicators
missing!
Types of uncertainty (e.g., Dessai) Epistemic uncertainty: incomplete knowledge (c.f. “myopia”) e.g., climate sensitivity, terrestrial carbon uptake, structural
uncertainty in models Stochastic uncertainty: system variability (c.f. “chance”)
e.g., non-linear behaviour of climate system, randomness, initial conditions
Human reflexive uncertainty: volition (c.f. “intentionality”) e.g., climate policies, behavioural responses to scientific
knowledge IPCC acknowledges different sources of uncertainty, but
does not translate this into diversity of uncertainty management and communication guidance
Method 1
qualitative description of level of evidence and level of agreement
Established but incomplete
Speculative
Well-established
Competing explanations
Choice WG3 in AR4
TARAR4
Method 2quantitatively calibrated levels of
agreement of scientists
Choice WG2 in AR4
Method 3
probabilities that the statements are true (“judgmental”)
Choice WG1 in AR4
But remember: words have different meanings for different people
Probability that subjects associated with the qualitative description
0.00.20.40.60.81.0
Almost certain
Probable
Likely
Good chance
Possible
Tossup
Unlikely
Improbable
Doubtful
Almost impossible
range of individual upper bound estimates
range of individual lower bound estimates
range from upper to lower median estimate
Qua
litat
ive
desc
riptio
n of
unc
erta
inty
use
dIPCC “likely”
IPCC “unlikely”
Profiles according to Weiss 2003:
1. Environmental absolutist
2. Cautious environmentalist
3. Environmental centrist
4. Technological optimist
5. Scientific absolutist
Even if there is agreement on the “level of evidence”, there will be disagreement about how to respond
Level of
Evidence
Text example from the Intergovernmental Panel on Climate Change WG I (2001)
“In the light of new evidence and taking into account the remaining uncertainties, most of the observed warming over the last 50 years is likely7
to have been due to the increase in greenhouse gas concentrations.”(SPM)
Example IPCC WG I (continued)7 In this Summary for Policymakers and in the Technical Summary, the following words have been used where appropriate to indicate judgmental estimates of confidence: virtually certain (greater than 99% chance that a result is true); very likely (90–99% chance); likely (66–90% chance); medium likelihood (33–66% chance); unlikely (10–33% chance); very unlikely (1–10% chance); exceptionally unlikely (less than 1% chance). The reader is referred to individual chapters for more details.
IPCC CO2 emission scenarios in 1990
Uncertainties in the level of
policy
Not only the text, also the graphics became more complex (better?)’…
BaU
Accelerated Policies
IPCC CO2 emission scenarios in 1992
Uncertainties in the driving forces
IPCC CO2 emissions scenarios in 2001
Figure 2-12: Global CO2 emissions from energy and industry, historical development from 1900 to 1990 and in 40 SRES scenarios from 1990 to 2100, shown as an index (1990 = 1). The range is large in the base year 1990, as indicated by an “error” bar, but is excluded from the indexed future emissions paths. The dashed time-paths depict individual SRES scenarios and the blue shaded area the range of scenarios from the literature (as documented in the SRES database). The median (50th), 5th, and 95th percentiles of the frequency distribution are shown. The statistics associated with the distribution of scenarios do not imply probability of occurrence (e.g., the frequency distribution of the scenarios in the literature may be influenced by the use of IS92a as a reference for many subsequent studies). The 40 SRES scenarios are classified into six groups. Jointly the scenarios span most of the range of the scenarios in the literature. The emissions profiles are dynamic, ranging from continuous increases to those that curve through a maximum and then decline. The coloured vertical bars indicate the range of the four SRES scenario families in 2100. Also shown as vertical bars on the right are the ranges of emissions in 2100 of IS92 scenarios, and of scenarios from the literature that apparently include additional climate initiatives (designated as “intervention” scenarios emissions range), those that do not (“non-intervention”), and those that cannot be assigned to either of these two categories (“non-classified”).
Uncertainties driving forces, different models
Detailed explanation
intervention/
no intervention
statistical information
Uncertainty range
Level of knowledge
Uncertainties: targeted by sceptics
Limitation to one source (where there were several available) made this graph
particularly vulnerable
Always check the axes!
Apparent correlations do not automatically imply causal links
Improvement in AR4: hierarchy of approaches
LEVEL OF INFORMATION/AGREEMENT
LEVEL OF INFORMATION/AGREEMENT
Ambiguous or unpredictable Describe governing factors
Trend/direction can be described
Explain basis/extent to which opposite changes would not be expected; use the language options
Ranges/orders of magnitude can be given
Use (also) confidence scale
Probabilities can be given Describe assumptions, sgtructural undertainties, use likelihood scale
Probability distribution functions can be provided
Provide PDF, methods used, and structural uncertainties
NB: simplified from original source!!
Three important uncertainty dimensions
frequentist: “truth” based on observations, repeatedexperiments
bayesian: level of “belief”
precise: quantified risk, large datasets unprecise: few and/or inconsistent data
observations-theories/models human choice/intentionality/volition
-> “agree to disagree”
The three dimensions visualized
findngs about:• observed GHG concentrations• observed radiative forcing• past GHG emissions• observed temperature
mostly WG1?
findings about:findings about:-- GHG emissions reduction potential GHG emissions reduction potential -- Costs of technologiesCosts of technologies-- Future climatic changesFuture climatic changes-- Future extreme eventsFuture extreme events-- Projected impacts Projected impacts -- Emission scenariosEmission scenarios•• Costs and benefits of stabilizing Costs and benefits of stabilizing
Mostly WG3?Mostly WG3?
findings about:• observed climate impacts• attribution climate change
Mostly WG2?
attribution(models/theories)
frequentist/objective bayesian/subjective
Likelihood
Confidence scale
Level of evidence and agreement
Explanatory factors
FysicalObservations/measurements
precise
unprecise
scenarios, human choices
Human systems
Natural
systems
Personal conclusions
+ Increasing appreciation by an increasing number of authors from AR1 to AR4
+ IPCC terminology adopted by other assessments (Millennium Ecosystem Assessment, Arctic Assessment), with mixed results
+ Major differences between physicists/climate scientists (WG1), biologists/ecologists (WG2) and economists/social scientists (WG3) not consistent with one-size-fits-all approach
+ Diificult balance between scientifically comprehensive and balanced account of uncertainties and effectiveness of communation with readers/policymakers.
Ideas about managing and communicating uncertainties evolve rather fundamentally over time as we learn
Differences between Working Groups (disciplines) not only acceptable, but useful to enhance understanding about the nature of uncertainties
Distinguish between findings based on observations, on models and theories, or on scenarios including human choices
Give much and early attention to choice of indicators, graphical representation and explanation of outer ends of ranges
Provide an account of the background and history of key findings (pedigree/traceable account)
Some lessons learned
Thanks
3 paradigms risks and uncertainties
uncertainty as failure Uncertainty is temporary Reduce uncertainty, develop ever complex models Tools: quantify, Monte Carlo, Bayesian belief networks
uncertainty as lack of agreement Independent comparative evaluation of research results (evidence evaluation) Tools: scientific consensus; multi disciplinary expert panels Focus on “robust “findings
uncertainty as unavoidable and useful information Uncertainty as intrinsic in complex systems Uncertainty as a result of knowledge production Acceptance that not all uncertainty is quantifiable Deal with deeper dimensions of uncertainty openly (problem frames, indeterminacy, ignorance,
assumptions, value loadings) Tools: Knowledge Quality Assessment “Working deliberatively within imperfections”
Adapted from Jeroen van der
Sluijs
How to act upon such uncertainty? Bayesian approach: 5 priors. Average and update likelihood of
each grid-cell being red with data (but oooops, there is no data & we need decisions NOW)
IPCC approach: Lock the 5 consultants up in a room and don’t release them before they have consensus
Nihilist approach: Dump the science and decide on an other basis Precautionary robustness approach: protect all grid-cells Academic bureaucrat approach: Weigh by citation index (or H-
index) of consultant. Select the consultant that you trust most Real life approach: Select the consultant that best fits your policy
agenda Post normal: explore the relevance of our ignorance: working
deliberatively within imperfections
Adapted from Jeroen van der
Sluijs
De zekerheidstrog(McKenzie, 1990)
Some issues to take into account
More research often leads to more uncertainties: Unforseen complexities Complex systems exhibit irreducible uncertainties (intrinsic or practical)
Scientific consensus model may lead to ignoring of weak early warning signals
Neglect of uncertainty management can lead to scandals and loss of trust in science and institutions
Non-quantifiable uncertainties dominate in many complex risks High quality risk assessment low uncertainty Uncertainty information is valuable input into policy debate Shift remains required from attention to uncertainty rewduction to making
uncertainties explicit and systematically deal with it
Adapted from Jeroen van der
Sluijs
Dimensions of uncertainty
Technical: (im)precision Methodological: (un)reliability Epistemological: ignorance Societal: societal (un)robustness
Example RIVM/MNP typology of uncertaintiesDimensions of uncertainty ->
Location of uncertainty
Level of uncertainty(from determinism, through probability and possibility, to
ignorance)
Nature of uncertainty
Qualification of knowledge
base
Value-ladenness of choices
Statistical uncertainty
Scenariouncertainty
Recognisedignorance
Epis-temic
Varia-bility
– 0 + – 0 +
ContextExpert judgmentMODEL
StructureImplemen-
tationParameters
Inputs
Data Outputs
Different issues can have different dominant locations and dimensions
Elements of Tool Catalogue
Sensitivity analysis (screening, local, global) Error propagation equations (“Tier 1”) Monte Carlo Analysis (“Tier 2”) Expert elicitation NUSAP (Numeral Unit Spread Assessment
Pedrigree) Scenario analysis PRIMA (Pluralistic fRamework of Integrated
uncertainty Management and risk Analysis) Checklist for model quality assistance Critical review of assumptions
Experience with RIVM/MNP uncertainty guidance, personal observations
Enhanced awareness of importance and hence Very broad support for development and application
from both management and researchers Successful application in special dedicated projects Limited but important applications in MNP’s top
products (annual National Environmental Balance, periodic National Environmental Outlooks)
Very limited application in other projects, mainly because of time and resource constraints