PICES/NPRB PICES/NPRB Indicators Indicators WorkshopWorkshop

Points from Background Points from Background Papers and June 1 Papers and June 1

DeliberationsDeliberations

SOURCES OF SOURCES OF COMMENTSCOMMENTS

SAFE ANNEX AND PICES REPORTSAFE ANNEX AND PICES REPORT ALL THE BACKGROUND PAPERS**ALL THE BACKGROUND PAPERS** PRESENTATIONSPRESENTATIONS DISCUSSION BETWEEN PAPERS**DISCUSSION BETWEEN PAPERS**

CONTEXT OF EXPERIENCE CONTEXT OF EXPERIENCE SEVERAL PLACES (ICES, EU, SEVERAL PLACES (ICES, EU, CANADA, FAO etc)CANADA, FAO etc)

STRUCTURE OF STRUCTURE OF COMMENTSCOMMENTS

OVERALL MESSAGESOVERALL MESSAGES COMMENTS ON MAJOR REPORTSCOMMENTS ON MAJOR REPORTS FROM PAPERS AND TALKSFROM PAPERS AND TALKS

What is missingWhat is missing What is present but vagueWhat is present but vague Present but needs further discussionPresent but needs further discussion

(Gradient across topics)(Gradient across topics)

MY OWN IDEAS OF USEFUL WAYS MY OWN IDEAS OF USEFUL WAYS AHEADAHEAD

MESSAGE 1 MESSAGE 1

YOU ARE IN PRETTY GOOD SHAPE!!!YOU ARE IN PRETTY GOOD SHAPE!!! A LOT OF WORK ON OBJECTIVES A LOT OF WORK ON OBJECTIVES

APPRECIATION OF NEED FOR SPECIFICITYAPPRECIATION OF NEED FOR SPECIFICITY OBJECTIVES FROM MANY SOURCES ARE OBJECTIVES FROM MANY SOURCES ARE

CONVERGINGCONVERGING SOME SOCIAL-ECONOMIC OBJECTIVESSOME SOCIAL-ECONOMIC OBJECTIVES

INDICATORS BEING MATCHED TO INDICATORS BEING MATCHED TO OBJECTIVESOBJECTIVES

Acknowledge two modes of useAcknowledge two modes of use NOT IN DESPERATION MODENOT IN DESPERATION MODE

The Major ReportsThe Major Reports

Both very goodBoth very good Different audiences match different contentDifferent audiences match different content Both have enough detail to allow user Both have enough detail to allow user

preconceptions to guide selection of contentpreconceptions to guide selection of content Do not try to put all the detail that any Do not try to put all the detail that any

user will want. Aim for a Guidebook not user will want. Aim for a Guidebook not an Encyclopaedia. (motivate and guide)an Encyclopaedia. (motivate and guide)

Make YOUR big messages clearer.Make YOUR big messages clearer.

Papers & Talks – Papers & Talks – “Missing”“Missing”

DPSIR structure – proving useful in DPSIR structure – proving useful in Organizing dialogue and reduction in Organizing dialogue and reduction in

numbernumber Matching Indicators to use in the overall Matching Indicators to use in the overall

processesprocesses Risk management frameworkRisk management framework

Overall absence of risk language in papersOverall absence of risk language in papers Beth’s talk has one good way forwardBeth’s talk has one good way forward Need more focus on displaying uncertaintyNeed more focus on displaying uncertainty

““Missing” or Under-Missing” or Under-representedrepresented

Of the suites of indicators:Of the suites of indicators: Spatial content (missing everywhere)Spatial content (missing everywhere) Size-based – under-represented relative Size-based – under-represented relative

to their performance elsewhere (esp. to their performance elsewhere (esp. ICES)ICES)

FORMAL SELECTION PROCESSESFORMAL SELECTION PROCESSES Needs to be more that a beauty contest Needs to be more that a beauty contest Jason and Beth had some impt. pointsJason and Beth had some impt. points

PRESENT BUT VAGUEPRESENT BUT VAGUE

How to test performance of indicators How to test performance of indicators during selection processduring selection process NOT the same for indictors used in NOT the same for indictors used in

AUDIT function and indicators used in AUDIT function and indicators used in CONTROL functionCONTROL function

AUDIT – Targets primary, limits AUDIT – Targets primary, limits secondarysecondary

CONTROL – Limits primary, Targets CONTROL – Limits primary, Targets 2ndary2ndary

METHODS EXIST FOR TESTNG BOTHMETHODS EXIST FOR TESTNG BOTH

Present but VaguePresent but Vague

Where do we get the reference points?Where do we get the reference points? Differentiate Indicator [say, SSB] from Differentiate Indicator [say, SSB] from

Reference Points [Bmsy, BReference Points [Bmsy, B35%35% etc] etc] Reversibility of impact? Responsiveness to Reversibility of impact? Responsiveness to

management at all?management at all?

The “classic” three stage model should have The “classic” three stage model should have ONE biological (or socio-economic) fixed ONE biological (or socio-economic) fixed point and the rest is making uncertainty point and the rest is making uncertainty explicit – NOT two biological fixed points.explicit – NOT two biological fixed points.

More Critical Thinking, More Critical Thinking, Perhaps?Perhaps?

When and how to use absolute vs When and how to use absolute vs relative indicatorsrelative indicators Experience with IUCN decline criterion Experience with IUCN decline criterion

and marine speciesand marine species Reference Points for different Reference Points for different

regimesregimes Pop Size NO; uses of populations YESPop Size NO; uses of populations YES Especially if likelihood of prompt Especially if likelihood of prompt

detection of regime change is lowdetection of regime change is low

More Critical Thinking, More Critical Thinking, Perhaps?Perhaps?

If you like “traffic light-y” presentation, If you like “traffic light-y” presentation, then biological calibration of cut-points then biological calibration of cut-points is a crucial research topicis a crucial research topic

What to do with tough decisions and What to do with tough decisions and multiple indicatorsmultiple indicators EU experience with just B and FEU experience with just B and F US legislation on over-fishing and over-US legislation on over-fishing and over-

fished won’t transfer readily to ecosystem fished won’t transfer readily to ecosystem metricsmetrics

What other field works with What other field works with Indicators in similar Indicators in similar

contexts?contexts? PSYCHOMETRICSPSYCHOMETRICS SIMILARITIESSIMILARITIES

Fundamental underlying processes matter Fundamental underlying processes matter but are NOT accessible to direct but are NOT accessible to direct measurementmeasurement

INDIRECT indices proliferate quickly and INDIRECT indices proliferate quickly and (if flexible) easily(if flexible) easily

““Normal” is not a fixed point on ANY scaleNormal” is not a fixed point on ANY scale A LOT hinges on decisionsA LOT hinges on decisions Abuse and/or mis-interpretation is easyAbuse and/or mis-interpretation is easy

Selection – Control Selection – Control function function

SIGNAL DETECTION THEORYSIGNAL DETECTION THEORY From Human Factors ResearchFrom Human Factors Research 70 year history (WWII was first flowering)70 year history (WWII was first flowering) REQUIRES REQUIRES

Reconstruction of history of values of Reconstruction of history of values of indicator(s)indicator(s)

RECONSTRUCTION OF HISTORY OF WHAT RECONSTRUCTION OF HISTORY OF WHAT GOOD DECISIONS WOULD HAVE BEEN!!!GOOD DECISIONS WOULD HAVE BEEN!!!

If we can’t do that retrospectively how can we If we can’t do that retrospectively how can we support decision-making into the futuresupport decision-making into the future

SIGNAL DETECTION SIGNAL DETECTION THEORYTHEORY

History of indicator gives direct record History of indicator gives direct record of what the decision would have been, of what the decision would have been, had that indicator been usedhad that indicator been used

History of what good decisions would History of what good decisions would have been gives standard for whether have been gives standard for whether the decision IN HINDSIGHT would the decision IN HINDSIGHT would have been right or wrong have been right or wrong (Piet and Rice – Lower or not lower quota)(Piet and Rice – Lower or not lower quota)

SIGNAL DETECTION SIGNAL DETECTION THEORYTHEORY

Four possible Outcomes:Four possible Outcomes: HIT (something should have been done and HIT (something should have been done and

Indicator said DO IT)Indicator said DO IT) TRUE NEGATIVE (no mgmt response TRUE NEGATIVE (no mgmt response

needed and indicator said status quo OK)needed and indicator said status quo OK) MISS (something should have been done MISS (something should have been done

but indicator did not say action was needed)but indicator did not say action was needed) FALSE ALARM (nothing needed to be done FALSE ALARM (nothing needed to be done

but indicator called for management but indicator called for management intervention)intervention)

SIGNAL DETECTION SIGNAL DETECTION THEORYTHEORY

Results presented as 2 X 2 tablesResults presented as 2 X 2 tables Perfect indicator has no Misses and False Perfect indicator has no Misses and False

AlarmsAlarms COSTS of Misses and False Alarms not the COSTS of Misses and False Alarms not the

same!!!! (nor equal to different perspectives)same!!!! (nor equal to different perspectives) Simple method to choose decision point on Simple method to choose decision point on

indicator (“reference point”) to indicator (“reference point”) to Minimize overall error rateMinimize overall error rate Control ratio of Misses and False Alarms (medical)Control ratio of Misses and False Alarms (medical)

Easy to compare performance of IndicatorsEasy to compare performance of Indicators

AUDIT FUNCTION- AUDIT FUNCTION- PSYCHOMETRIC PSYCHOMETRIC DIAGNOSTICSDIAGNOSTICS

History of over a centuryHistory of over a century Many mistakes (and advocacy Many mistakes (and advocacy

abuses), many lessons already learnedabuses), many lessons already learned Uses are numerousUses are numerous

Career aptitude testingCareer aptitude testing Legal competency for actionsLegal competency for actions Personality disorders and counsellingPersonality disorders and counselling

Extensive validation testing and Extensive validation testing and codification of professional standardscodification of professional standards

PSYCHOMETRIC PSYCHOMETRIC DIAGNOSTICSDIAGNOSTICS

Reminder : Dealing with all the same Reminder : Dealing with all the same problems:problems: Fundamental underlying processes matter Fundamental underlying processes matter

but are NOT accessible to direct but are NOT accessible to direct measurementmeasurement

INDIRECT indices proliferate quickly and (if INDIRECT indices proliferate quickly and (if flexible) easilyflexible) easily

““Normal” is not a fixed point on ANY scaleNormal” is not a fixed point on ANY scale A LOT hinges on decisionsA LOT hinges on decisions Abuse and/or mis-interpretation is easyAbuse and/or mis-interpretation is easy

PSYCHOMETRIC PSYCHOMETRIC DIAGNOSTICSDIAGNOSTICS

General ApproachGeneral Approach Have large battery of “questions” (= “suites of Have large battery of “questions” (= “suites of

indicators”) – Binet, MMPI, Rorschach etcindicators”) – Binet, MMPI, Rorschach etc Test a large populace on the batteryTest a large populace on the battery Test sets of individuals KNOWN with Test sets of individuals KNOWN with

confidence to have specific disorders.confidence to have specific disorders. WHAT COMBINATION OF QUESTIONS IN WHAT COMBINATION OF QUESTIONS IN

WHAT WEIGHTING MAXIMALLY GROUPS AS WHAT WEIGHTING MAXIMALLY GROUPS AS DISTINCT THE KNOWN PATHOLOGIES DISTINCT THE KNOWN PATHOLOGIES WHILE LEAVING WHILE LEAVING MOST MOST OF POPULACE IN A OF POPULACE IN A CENTRAL CLOUDCENTRAL CLOUD

NOT TRIED IN ECOLOGY (that I know of)NOT TRIED IN ECOLOGY (that I know of)

The “Classic” three stage model

HEALTHYCAUTIOUS

CRITICAL

HA

RV

ES

T R

AT

E

BIOMASS

LRPEN TH ?

WHERE IS SC?

CONVENTIONAL FISHERIES MANAGEMENT

REBUILDING

EN

DA

NG

ER

ED

S

PE

CIE

S T

OO

LS

ONE Fixed Point in 3-stage ONE Fixed Point in 3-stage modelmodel

Some govt responsibility – law or policySome govt responsibility – law or policy ““Serious or Irreversible harm” (CBD & PA)Serious or Irreversible harm” (CBD & PA)

1. Best biological estimate of that point 1. Best biological estimate of that point (ICES is B(ICES is Blimlim – damaged productivity) – damaged productivity)

2. Estimating current status relative to that 2. Estimating current status relative to that point has uncertainty, so buffer is needed- point has uncertainty, so buffer is needed- BBpapa Point where PROBABILITY that true stock may be Point where PROBABILITY that true stock may be

at the limit is > 0.05 at the limit is > 0.05 Risk averse management relative to makes whole Risk averse management relative to makes whole

system precautionarysystem precautionary Gets F pairs from F which implies Gets F pairs from F which implies

equilibrium Bequilibrium Blimlim

The Issue of The Issue of PREDICTABILITYPREDICTABILITY

““Scenario Exploration” [use your favourite term] Scenario Exploration” [use your favourite term] in Climate Change and Marine Ecosystem in Climate Change and Marine Ecosystem Dynamics have an important difference:Dynamics have an important difference: CC - NO EXPECTATION OF ACCURACY ON CC - NO EXPECTATION OF ACCURACY ON

TIME SCALES >30 DAYS OR < 30 YEARSTIME SCALES >30 DAYS OR < 30 YEARS ECOSYSTEM – 3-7(10?) YEAR PATTERN IS ECOSYSTEM – 3-7(10?) YEAR PATTERN IS

CORE OF DECISION SUPPORTCORE OF DECISION SUPPORT CC – key decisions are long-term strategicCC – key decisions are long-term strategic Ecosystem “ “ medium-term tacticalEcosystem “ “ medium-term tactical

What to Predict?What to Predict?

Don’t try to capture the inter-annual flutterDon’t try to capture the inter-annual flutter How does probability of a an extreme event How does probability of a an extreme event

(good or bad) vary with natural or (good or bad) vary with natural or anthropogenic forcers? anthropogenic forcers?

Multi-factor non-parametric probability Multi-factor non-parametric probability density estimation methods DO show density estimation methods DO show inflections in plot of P(extreme event) as f inflections in plot of P(extreme event) as f (specified forcers)(specified forcers)

Easy to use and interpretEasy to use and interpret DO require decisions about what is “extreme”DO require decisions about what is “extreme”