1

Modeling Learning By Doing in Natural Resource Management

Half-time seminar by: Emilie LindkvistSupervisors: Jon Norberg (SRC), Maja Schlüter(SRC), Örjan Ekeberg (KTH)

A centre with:

Outline

• Short introduction

• The method – computerized learning agent

• Experiment setup, results, and conclusions of paper I and II individally

• Key findings

• Future research

Introduction

• The uncertainties humans face when dealing with natural resources are increasing as a consequence of global environmental change

?

?

IPCC (2007), World Bank(2012), Smith et al.(2011), New et al.(2011), Levin (2003)

The Solution

?

• To deal with uncertainty and surprise use Adaptive (co-)management which emphasize Learning by Doing (LBD)

• Account for resource users knowledge

Allen et al 2011, Armitage et al 2011, Folke (2004), Fazey et al.(2007), Walter&Holling (1990)

GAP

• Increased understanding of

– Individual learning process lacking (focus on social learning)

– trade-offs in the individual learning process in phase if change

• Exploration vs. exploitation

• Value future vs. present

• Stick to the past or adapt to present

• Learn from the past or trust present

Aim

• Understand

– the core of an LBD process

– impact of different learning parameters

• Study how the LBD process responds to different structures of & changes in resource dynamics

The Method & basic model setup

The Social-Ecological system

Agent = 1 Fisher (computerized learning agent)

Resource = 1 Fish stock

1 Update Rate of Mental Model2 Discount Factor3 Level of hindsight4 Exploration Level

GOAL Performance =

net income $

LEARNING PARAMETERS

LEARNING PROBLEM

?

?

The fisher repeatedly needs fish, goes fishing, harvest, learns, and updates his mental model

(Fisher)

(Fish stock)

(Effort)

$

Sutton & Barto (1998), Poggio & Girosi (1989)

Papers

I. Lindkvist, E. and Norberg, J. Modeling Experiential Knowledge: Limitations in Learning Non-Linear Dynamics for Sustainable Renewable Resource Management. Submitted to Ecological Economics

II. Lindkvist, E. and Norberg, J. Theoretical Aspects on Learning By Doing: Adapting to Effects of Environmental Change. Manuscript

Research Questions Paper I

• Are there limitations of a LBD approach for “sustainably exploiting” a renewable resource?

• What are the effects of update rate of mental model, discount factor, hindsight, exploration level, on management outcome?

1. L

ogi

stic

pro

ble

m2

. Th

resh

old

pro

ble

m(a

dd

incr

ease

d m

ort

alit

y ra

te)

Paper I

(Fisher)

(Effort)

The problems the fisher should learn to manage sustainably

Logistic Problem

• density dependent

• slower regrowth at low and high stocks

Threshold Problem

• density dependent

• slower regrowth at low and high stocks

• Threshold: hysteresis effect – if a threshold is trespassed it takes about 20 times longer to regrow IF fishing pressure is kept low

Paper I

C.W. Clark (1976, 2010 )

Increased Mortality Rate

Every fishing event with a 5% chance 5-95% of the fish stock is removed

(out of the fishers control)

Paper I

Update Rate : How fast should the fisher adapt his mental model?

Low High

Inertia Adapt instantaneously

Sticking to past perception of the resource dynamics

Over-learning (oscillating effect)

Paper I

(Update rate: parameter in mental model)

THRESHOLD

LOGISTIC

Paper I

$

A modest update rate of ones mental model is beneficial for sustainable fisheries

Discount Factor: How much should the fisher values future outcomes?

Low High

Greedy behavior Takes future fish stocks into account when learning

Optimize now Optimize over time – infinite time horizon

(Discount factor: parameter in learning method)

THRESHOLD

NO THRESHOLD

Paper I

High when logistic problem. Lower when a threshold is enforced out of the fishers control

$ LOGISTIC

Hindsight: How should the fisher evaluate previous fishing efforts, and how led up to the current state?

Low High

Adapt continuously to new experiences

Take all past experiences into account when learning from current fishing event

Don’t re-evaluate past experiences Re-evaluate all past experiences

(Hindsight : parameter in learning method)

THRESHOLD

NO THRESHOLD

Paper IHindsight should not be too high for either problem(but increased performance for threshold up

to 0.7)

$ LOGISTIC

Exploration level: How much should the fisher try other fishing efforts than what he perceives as

optimal?

Low High

Always does what is best according to his mental model

Never does what he thinks is best according to his mental model

Never deviates from his current view of the system

Always deviates from his current view of the system

(Exploration level: parameter in decision-making model)

THRESHOLD

Paper I

$LOGISTIC

Low exploration optimal for threshold problem, but higher optimal for logistic problem

Conclusions

• LDB works well but certain implications• Logistic problem

– To best manage the logistic problem a high exploration, high valuation of future outcomes, little hindsight, update mental model at 10 to 80% is optimal.

• Threshold problem – LDB ill-advised (R. Biggs et al. 2009)– However, ok if in a planned setting others can learn (C.

Walters 2007)– The fisher develops a very precise mental model of how

the fish stock behaves dependent on fish stock biomass, and develops a more careful behavior

Paper I

Research Questions Paper II

1. How does a LBD approach respond to changes in the growth rate of a renewable resource?

2. Do key learning parameters differ depending on the type change? (e.g. linear vs. abrupt)

1 Update Rate of Mental Model2 Discount Factor3 Level of hindsight4 Exploration Level

GOAL Performance =

net income $

8 s

cen

ario

s o

f ch

ange

s in

gro

wth

rat

e

Paper II

(Fisher)

(Effort)

Paper II

(growth rate change)

Paper II

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Perf

orm

ance

Con

stan

t

Line

ar In

c.

Abr

upt I

nc.

Slow C

yclic

Fast C

yclic

Abr

upt R

and

Abr

upt D

ec.

Line

ar D

ec.

Learning Agent

Static Agent

$

Paper II

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Perf

orm

ance

Con

stan

t

Line

ar In

c.

Abr

upt I

nc.

Slow C

yclic

Fast C

yclic

Abr

upt R

and

Abr

upt D

ec.

Line

ar D

ec.

Learning Agent

Static Agent

$

Paper II

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Perf

orm

ance

Con

stan

t

Line

ar In

c.

Abr

upt I

nc.

Slow C

yclic

Fast C

yclic

Abr

upt R

and

Abr

upt D

ec.

Line

ar D

ec.

Learning Agent

Static Agent

$

Conclusions

• LBD generally good• Effects of environmental change

– Tricky if abrupt decreasing (like threshold)– Always better than a non-adaptive

• Brown et al 2012. How long can fisheries management delay action in response to ecosystem and climate change?

• Niiranen et al 2012 implications for (not)modeling uncertainties in growth rates in baltic sea models

Future

Use the agent in basic networks (Motifs, BodinTengö 2012) to study impact of agent interactions (information sharing, trust, knowledge)

Future

Add governance layer, to study impact of different forms of leadership (Gutiérrez et al 2011)

Governing Agent

Supervisors, friends, family, listeners!

Subscribe to our newsletter

www.stockholmresilience.org/subscribe

35

Thank you,

A centre with:

Extra Slides

1. Make decision (on effort)

Learning by Doing (LBD)

2. Perform action (fishing effort)

3. Harvest

4. Learn (update mentalmodel of system)

Update stock

(Fisher) (Fish stock)

Mental ModelPaper I

Mental ModelPaper I

Results paper I & II

• Significant

• High for decreasing growth

• Low for

• Significant if thresholds

• Continuous adaption

• Significant

• Value future outcomes at 95%

• But less if regime shifts

• Significant

• Change mental model 20% if regime shift

• 20-80% if change

Mental Model update

rate

Discount Factor

Exploration level

Hindsight

Population Growth Function

Action PerformancePaper II

Action sensitivity dep. On Growth RatePaper I & II

Paper I & II