heterogeneous agent models lecture 4 heuristic switching...

Heterogeneous Agent Models Lecture 4 Heuristic Switching Model

Heterogeneous Agent ModelsLecture 4

Heuristic Switching Model

Mikhail Anufriev

EDG, Faculty of Business, University of Technology Sydney (UTS)

July, 2013


Outline

1 Evolutionary Model of Individual Learning

2 Positive vs. Negative Feedback


Expectations in Economic Theory

economy is an expectation feedback system

expectations affect our decisions and realizationsexpectations are affected by past experience

expectations play the key role in most economic models

30s-60s naive and adaptive expectations

70s-90s rational expectations

90s- models of learning and bounded rationalityadaptive learning (OLS-learning)belief-based learningreinforcement learning


This lecture:

Heuristic Switching Model with heterogeneous expectations

model is inspired by and tested on the experimental data

Key features:

in forecasting agents use simple rules of thumb, heuristics(Tversky and Kahneman, 1974)

in learning agents switch between different forecasting rules on thebasis of their performances

(Brock and Hommes, 1997)


Learning in a Complex Environment

Mailath (JEL, 1998) Do people play Nash equilibrium? Lessons fromevolutionary game theory, p. 1349-1350:

The typical agent is not like Gary Kasparov, the world champion chessplayer who knows the rules of chess, but also knows that he doesn’t knowthe winning strategy.In most situations, people do not know they are playing a game. Rather,people have some (perhaps imprecise) notion of the environment they are in,their possible opponents, the actions they and their opponents have available,and the possible payoff implications of different actions.

These people use heuristics and rules of thumb (generated from experience)to guide behavior; sometimes these heuristics work well and sometimes theydon’t. These heuristics can generate behavior that is inconsistent withstraightforward maximization.


Learning-to-forecast experiments: Summary 1

“Stylized facts” for two-periods ahead experiments

large bubbles in the absence of fundamentalists

qualitatively different patterns in the same environment(almost) monotonic convergence

constant oscillations

damping oscillations

coordination of individual predictions

forecasting rules with behavioral interpretation are used


40

45

50

55

60

65

70

0 10 20 30 40 50

Pric

e

Group 2

fundamental price experimental price

40

45

50

55

60

65

70

0 10 20 30 40 50

Pric

e

Group 5


40

45

50

55

60

65

70

0 10 20 30 40 50

Pric

e

Group 1


40

45

50

55

60

65

70

0 10 20 30 40 50

Pric

e

Group 6


10 20 30 40 50 60 70 80 90

0 10 20 30 40 50

Pric

e

Group 4


40

45

50

55

60

65

70

0 10 20 30 40 50

Pric

eGroup 7



Model

Model: four forecasting heuristicsadaptive rule

ADA pe1,t+1 = 0.65 pt−1 + 0.35 pe

1,t

weak trend-following rule

WTR pe2,t+1 = pt−1 + 0.4 (pt−1 − pt−2)

strong trend-following rule

STR pe3,t+1 = pt−1 + 1.3 (pt−1 − pt−2)

anchoring and adjustment heuristics with learnable anchor

LAA pe4,t+1 = 1

2

(pav

t−1 + pt−1)

+ (pt−1 − pt−2)

price dynamics

pt = 11+r

((n1,tpe

1,t+1 + n2,tpe2,t+1 + n3,tpe

3,t+1 + n4,tpe4,t+1

)+ y + εt

)


Model

Adaptive Expectations: pet+1 = w pt−1 + (1− w) pe

tDynamics globally converge to fundamental price.

45

50

55

60

65

70

0 10 20 30 40 50

Price under adaptive heuristic

w = 0.25 w = 0.65


Model

Weak-Trend Extrapolation: pet+1 = pt−1 + γ (pt−1 − pt−2)

Dynamics converge to fundamental price.

45

50

55

60

65

70

0 10 20 30 40 50

Price under weak trend following heuristic

γ = 0.4 γ = 0.99


Model

Strong-Trend Extrapolation: pet+1 = pt−1 + γ (pt−1 − pt−2)

Dynamics diverge from fundamental price...

0

20

40

60

80

100

0 10 20 30 40 50

Price under strong trend following heuristic

γ = 1.1 γ = 1.3


Model

Strong-Trend Extrapolation: pet+1 = pt−1 + γ (pt−1 − pt−2)

...and settles on the quasi-periodic attractor.

0

20

40

60

80

100

0 20 40 60 80 100

Attractor under strong trend following heuristic

γ = 1.1 γ = 1.3

200 points after 500 transitory steps


Model

Anchoring and Adjustment: pet+1 =

pf +pt−12 + (pt−1 − pt−2)

45

50

55

60

65

70

0 10 20 30 40 50

Price under anchoring and adjustment heuristic

fixed anchor learned anchor

with learning of anchor: pet+1 =

pavt−1+pt−1

2 + (pt−1 − pt−2)


Model

Model with Homogeneous Expectations

pattern of monotonic convergence can be easily reproducedadaptive rule, weak trend extrapolation

pattern of constant oscillations can be reproducedanchoring and adjustment rule without learning

pattern of damping oscillations is reproduced (very imperfectly)strong-trend extrapolations


Model

Stability conditions

-1.5

-1

-0.5

0

0.5

1

1.5

-2 -1 0 1 2

β 2

β1

Neimark-Sacker

pitch-fork

perio

d-do

ublin

g

γ = 0.4

γ = 1.3

AA

pet+1 = α+ β1pt−1 + β2pt−2


Model

Evidence for switching I

52

54

56

58

60

62

64

66

0 10 20 30 40 50

Group 6, participant 1

prediction price


Model

Evidence for switching II

48

50

52

54

56

58

60

62

64

66

0 10 20 30 40 50

Time

Group 1, participant 3

prediction price


Model

Modelling switching behavior

impacts of heuristics ni,t are evolving

performance measure of heuristic i is

Ui,t−1 = −(pt−1 − pe

i,t−1

)2+ ηUi,t−2

parameter η ∈ [0, 1] – the strength of the agents’ memory

discrete choice model with asynchronous updating

ni,t = δ ni,t−1 + (1− δ) exp(β Ui,t−1)∑4i=1 exp(β Ui,t−1)

parameter δ ∈ [0, 1] – the inertia of the tradersparameter β ≥ 0 – the intensity of choice


Model

Stability and Instability of the model

0 0.2

0.4 0.6

0.8 1 0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1 n3, fraction of STR

Stability region for model with fixed fractions

A

n1, fraction of ADA n2, fraction of WTR

n3, fraction of STR


Model

Two types of simulations

Learning parameters are fixed: β = 0.4, η = 0.7, δ = 0.9.

deterministic pathinitialized with

initial prices, p0, p1 as in the experimental groupinitial impacts of heuristics depend on the group

simulated for 47 periods with the same noise process as in theexperiment (or without noise)

one-period ahead predictionat any moment the experimental data so far are used to produce

predictions of heuristicsimpacts of heuristics


Model

Deterministic path: Monotonic convergence

Parameters: β = 0.4, η = 0.7, δ = 0.9

35

45

55

65

0 10 20 30 40 50

Pred

ictio

ns

45

55

65

Pric

e

Group 5

-2 0 2

35

45

55

65

0 10 20 30 40 50

Pred

ictio

ns

ADA WTR STR LAA 45

55

65

Pric

e

Group 5

simulation experiment

-2

0

2


Model

Deterministic path: Constant oscillations

Parameters: β = 0.4, η = 0.7, δ = 0.9

35

45

55

65

0 10 20 30 40 50

Pred

ictio

ns

45

55

65

Pric

e

Group 1

-5 0 5

35

45

55

65

0 10 20 30 40 50

Pred

ictio

ns

ADA WTR STR LAA 45

55

65

Pric

e

Group 1


-5

0

5


Model

Deterministic path: Damping oscillations

Parameters: β = 0.4, η = 0.7, δ = 0.9

45 55 65 75

0 10 20 30 40 50

Pred

ictio

ns

45

55

65

75

Pric

e

Group 7

-10 0

10 45

55

65

75

0 10 20 30 40 50

Pred

ictio

ns

ADA WTR STR LAA

45

55

65

75

Pric

e

Group 7


-10

0

10


Model

Evolution of instability

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

0 10 20 30 40 50

Group 2 Group 1 Group 7

Largest modulus of eigenvalue of HSM


Model

One-period ahead prediction: gr. 5 (convergence)

Parameters: β = 0.4, η = 0.7, δ = 0.9

35

45

55

65

0 10 20 30 40 50

Pred

ictio

ns

ADA WTR STR LAA 45

55

65

Pric

e

Group 5


-2

0

2

0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50

Fractions of 4 rules in the simulation for Group 5

ADA WTR STR LAA


Model

One-period ahead prediction: gr. 6 (constant oscillations)

Parameters: β = 0.4, η = 0.7, δ = 0.9

35

45

55

65

0 10 20 30 40 50

Pred

ictio

ns

ADA WTR STR LAA 45

55

65

Pric

e

Group 6


-5

0

5

0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50


ADA WTR STR LAA


Model

One-period ahead prediction: gr. 4 (damping oscillations)

Parameters: β = 0.4, η = 0.7, δ = 0.9

10 30 50 70 90

0 10 20 30 40 50

Pred

ictio

ns

ADA WTR STR LAA

10

30

50

70

90

Pric

e

Group 4


-50-25

0 25 50

0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50


ADA WTR STR LAA


Model

In-sample performanceMSE over 47 periods for 8 different models

Specification Group 2 Group 5 Group 1 Group 6 Group 4 Group 7

Fundamental 16.6231 10.8238 15.7581 9.3245 300.9936 21.9123ADA 0.0712 0.0378 5.6734 4.6095 210.3313 19.5158WTR 0.0862 0.1419 2.0905 1.1339 92.2163 9.2932STR 0.5001 0.6605 2.9071 0.8131 124.3494 14.7224LAA 0.4588 0.4756 0.456 0.6591 66.2637 5.8635

constant weights 0.0814 0.1698 1.2417 0.6618 70.8516 7.0956

HSM 0.0646 0.1108 0.4672 0.2917 47.2492 4.3154HSM (fitted) 0.0493 0.0353 0.4423 0.1655 34.4932 2.9358β ∈ [0, 10] 10 10 0.1 10 3 0.2η ∈ [0, 1] 0.4 0.9 1 0.1 0.8 0.5δ ∈ [0, 1] 0.9 0.6 0.5 0.7 0.6 0.4


Model

Out-of-sample performance

The Heuristic Switching Model vs. AR(2) model

Group 2 Group 5 Group 1 Group 6 Group 4 Group 7

HSM1 p ahead 0.0122 0.0321 0.479 0.1921 15.0395 0.78572 p ahead 0.0122 0.0901 1.8599 1.0792 57.5144 1.5543

AR(2) Model1 p ahead 0.3732 0.4431 0.9981 0.5568 13.4616 0.66822 p ahead 0.5052 0.4045 3.5823 1.4944 44.6453 2.0098


Model

Comparison with Homogeneous Expectations: MSE“Direct fit” with parameters β = 0.4, η = 0.7, δ = 0.9

Specification Group 2 Group 5 Group 1 Group 6 Group 4 Group 7Fundamental Prediction 18.037 11.797 15.226 8.959 291.376 22.047

ADA – exp prices 0.841 0.200 7.676 8.401 330.101 51.526WTR – exp prices 4.419 1.983 8.868 6.252 308.549 30.298STR – exp prices 585.789 478.525 638.344 509.266 1231.064 698.361AA – exp prices 39.308 21.760 17.933 17.345 289.134 87.878

LAA – exp prices 5.475 3.534 5.405 14.404 307.605 69.749ADA – fitted prices 0.514 0.199 6.832 7.431 312.564 36.436WTR – fitted prices 4.222 1.844 8.670 6.228 292.150 19.764STR – fitted prices 413.435 42.488 182.284 29.200 580.543 579.141AA– fitted prices 26.507 13.228 11.117 13.981 258.010 63.777

LAA – fitted prices 2.055 1.859 4.236 13.433 284.880 45.153

4 heuristics (plots) 0.449 0.302 8.627 14.755 526.417 29.5204 heuristics (fitted) 0.313 0.245 7.227 7.679 235.900 18.662


Model

Comparison with Homogeneous Expectations: AR2“Indirect fit” with parameters β = 0.4, η = 0.7, δ = 0.9

Specification Group 2 Group 5 Group 1 Group 6 Group 4 Group 7

Fundamental Prediction 0.946 0.671 2.673 3.610 2.311 2.002ADA – exp prices 0.239 0.006 2.182 2.898 1.691 1.494WTR – exp prices 0.066 0.529 0.383 0.627 0.203 0.165STR – exp prices 1.494 2.583 0.112 0.020 0.240 0.342AA – exp prices 1.095 1.848 0.010 0.038 0.045 0.094

LAA – exp prices 0.747 1.544 0.003 0.050 0.003 0.013ADA – fitted prices 0.100 0.000 1.584 2.159 1.385 1.157WTR – fitted prices 0.068 0.343 0.262 0.435 0.174 0.139STR – fitted prices 1.358 2.192 0.078 0.001 0.147 0.242AA– fitted prices 1.036 1.755 0.005 0.029 0.038 0.083

LAA – fitted prices 0.640 1.277 0.000 0.033 0.000 0.004

4 heuristics (plots) 0.383 0.744 0.011 0.008 0.157 0.2394 heuristics (fitted) 0.144 0.499 0.009 0.003 0.121 0.048


Model

The same experiment: group 3

Parameters: β = 0.4, η = 0.7, δ = 0.9

40

45

50

55

60

65

70

0 10 20 30 40 50

Pric

e

Experiment and simulation price for Group 3

simulation experiment 0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50


ADA WTR STR LAA


Model

Conclusion

the model with evolutionary switching between simple heuristics

dynamics of the model is path-depended (different patterns of theexperiments have been reproduced)

good in-sample and out-of-sample performance

Model applications

macroeconomics

financial bubbles and crashes

agent-based modelling

Other experiments?


Model

Bubbles in Experiment I

pt = 11+r

(pt+1 + y

)

0

100

200

300

0 10 20 30 40 50

Pric

e

HSTV08 experiment and simulation price for Group 1


0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50


ADA WTR STR LAA


Model

Bubbles in Experiment II

pt = 11+r

(pt+1 + y

)

0

200

400

600

800

1000

0 10 20 30 40 50

Pric

e

HSTV08 experiment and simulation price for Group 2

simulationexperiment

0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50


ADA WTR STR LAA


Model

Other Experiments: Smaller Fundamental Price I

y = 3 → y = 2

0

10

20

30

40

50

60

70

0 10 20 30 40 50

Pric

e



0.2

0.4

0.6

0.8

1

0 10 20 30 40 50


ADA WTR STR LAA


Model

Other Experiments: Smaller Fundamental Price II

y = 3 → y = 2

0

20

40

60

80

100

0 10 20 30 40 50

Pric

e



0.2

0.4

0.6

0.8

1

0 10 20 30 40 50


ADA WTR STR LAA


Model

Other Experiments: No Robots

pt = 11+r

((1− nt)pAE

t+1 + nt pf + y + εt

)→ pt = 1

1+r

(pAE

t+1 + y)

0

20

40

60

80

100

0 10 20 30 40 50

Pric

e



0.2

0.4

0.6

0.8

1

0 10 20 30 40 50


ADA WTR STR LAA


Model

Other Experiments: One-period ahead forecast

pt = 11+r

(pAE

t+1 + y)→ pt = 1

1+r

(pAE

t + y + εt

)

0

20

40

60

80

100

0 10 20 30 40 50

Model in Positive Feedback Environment


0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50

Frac

tions

Impacts of Heuristics in Positive Feedback Environment

ADA WTR STR LAA


Model

Other Experiments: Negative Feedback

pt = a + bpAEt+1 + εt → pt = a′ − bpAE

t+1 + εt

0

20

40

60

80

100

0 10 20 30 40 50

Model in Negative Feedback Environment


0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50

Frac

tions

Impacts of Heuristics in Negative Feedback Environment

ADA WTR STR LAA


Positive vs. Negative Feedback

Learning-to-forecast experiments: Summary 2

“Stylized facts” for one-period ahead experiment

qualitatively different aggregate patterns in differentenvironments

negative feedback: heavy fluctuation (during 5 periods) and thenfast convergence

positive feedback: no convergence, in some groups slowoscillations

coordination of individual predictions

How do people behave (form expectations and learn) in theexpectations feedback system?



Heuristic Switching ModelTwo heuristics:

adaptive rule

ADA pe1,t = 0.75 pt−1 + 0.25 pe

1,t−1

trend-following rule

TRF pe2,t = pt−1 + (pt−1 − pt−2)

price dynamicsnegative feedback market:

pt = 60− 2021(n1,tpe

1,t + n2,tpe2,t + 60

)+ εt

positive feedback market:

pt = 60 +2021(n1,tpe

1,t + n2,tpe2,t − 60

)+ εt ,



Model: time-varying impacts of heuristics

impacts of heuristics ni,t are evolving

performance measure of heuristic i is

Ui,t−1 = −(pt−1 − pe

i,t−1)2

+ ηUi,t−2

parameter η ∈ [0, 1] – the strength of the agents’ memory

discrete choice model with asynchronous updating

ni,t = δ ni,t−1 + (1− δ)exp(β Ui,t−1)∑2i=1 exp(β Ui,t−1)

parameter δ ∈ [0, 1] – the inertia of the tradersparameter β ≥ 0 – the intensity of choice



Discrete Choice Model

ni,t = δ ni,t−1 + (1− δ)exp(β Ui,t−1)∑2i=1 exp(β Ui,t−1)

-6 -4 -2 2 4 6U1 - U2

0.2

0.4

0.6

0.8

1.0Impact1

Β=10

Β=1

Β=0.3



Two types of simulations

Learning parameters are fixed: β = 1.5, η = 0.1, δ = 0.1.

deterministic pathinitialized with

initial price, p0 = 50initial impacts of heuristics, n1,0 = n2,0 = 1

2

simulated for 49 periods with the same noise process as in theexperiment (or without noise)

one-period ahead predictionat any moment the experimental data so far are used to produce

predictions of heuristicsimpacts of heuristics



Deterministic path: negative feedback

Parameters: β = 1.5, η = 0.1, δ = 0.1.

40

50

60

70

80

0 10 20 30 40 50

Pri

ce

Time

0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50Im

pac

ts

Time

adaptivetrend



Deterministic path: positive feedback

Parameters: β = 1.5, η = 0.1, δ = 0.1.

40

50

60

70

80

0 10 20 30 40 50

Pri

ce

Time

0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50Im

pac

ts

Time

adaptivetrend



One-period ahead simulation: negative feedback

Parameters: β = 1.5, η = 0.1, δ = 0.1. Group 1.

40

50

60

70

80

0 10 20 30 40 50

Pri

ce

Time

simulationexperiment 0

0.2

0.4

0.6

0.8

1

0 10 20 30 40 50Im

pac

ts

Time

adaptivetrend



One-period ahead simulation: positive feedback


20

30

40

50

60

70

80

0 10 20 30 40 50

Pri

ce

Time


0.2

0.4

0.6

0.8

1

0 10 20 30 40 50Im

pac

ts

Time

adaptivetrend



One-period ahead simulation: positive feedback


40

50

60

70

80

0 10 20 30 40 50

Pri

ce

Time


0.2

0.4

0.6

0.8

1

0 10 20 30 40 50Im

pac

ts

Time

adaptivetrend



One period ahead predictive performance I

Mean Squared Error

MSE =147

49∑t=3

(pGr X

t − pModt)2,

averaged over a number of groups

Model Negative Feedback Positive Feedback 12 groupsFundamental 2.5712 46.8344 24.7028Adaptive 2.3001 2.9992 2.6497Trend 21.1112 0.9260 11.0186Mixed 7.9798 1.0518 4.5158HSM 3.2967 0.9065 2.1016



MSE, Maximum Likelihood, AIC and BICCriterion to compare different models:

MSE of the one-period ahead forecasts (from t = 5)

MSEGr X =1

T − 4

T∑t=5

(pMod

t (θ)− pGr Xt)2, MSEGr X, Y, . . .

MSE is minimized for parameters θ which satisfy to the f.o.c. ofmaximization of the likelihood function

L(θ, σ2; exp. data) =∏

Groups

∏Periods

1σ√

2πexp

(−(pMod

t (θ)− pGr Xt )2

2σ2

)To penalize for extra parameters we use

AIC = 2n− 2 ln L(θ, σ2; exp. data) ,

BIC = n ln(kG)− 2 ln L(θ, σ2; exp. data) , using σ2 = MSE(θ)



One period ahead predictive performance II

Model Negative Feedback Positive Feedback 12 groups

benchmark

MSE 3.2967 0.9065 2.1016(β, η, δ) (1.50,0.10,0.10) (1.50,0.10,0.10) (1.50,0.10,0.10)(w, γ) (0.75,1.00) (0.75,1.00) (0.75,1.00)AIC 2273.3741 1545.1924 2019.4464BIC 2273.3741 1545.1924 2019.4464

optimized

MSE 1.9225 0.7088 1.5051(β, η, δ) (1.46,1.00,0.15) (0.48,0.85,0.00) (0.37,0.87,0.00)(w, γ) (0.98,−2.00) (0.83,0.75) (0.75,0.75)AIC 1979.2104 1416.4752 1841.1650BIC 2000.8857 1438.1504 1862.8403



Further Work

theoretical relation of an outcome with Restricted PerceptionEquilibrium

further analysis (other experiments, other methods)

comparison with other learning methods (e.g., with GA)

direct experiment on switching to estimate switching parameters

classification of behavioral types on the basis of individualpredictions



Learning-to-Forecast Experiments:Hommes, C.H., Sonnemans, J., Tuinstra, J. and Velden, H. vande, (2005), Coordination of expectations in asset pricingexperiments, Review of Financial Studies 18, 955-980.Hommes, C.H., Sonnemans, J., Tuinstra, J. and Velden, H. vande, (2008), Expectations and bubbles in asset pricingexperiments, Journal of Economic Behavior and Organization67, 116-133.Heemeijer, P., Hommes, C.H., Sonnemans, J., and Tuinstra, J.,(2009), Price stability and volatility in markets with positive andnegative expectations feedback: an experimental investigation,Journal of Economic Dynamics and Control 33, 1052-1072.Bao, T., Hommes, C., Sonnemans, J., and Tuinstra, J., (2012),Individual expectations, limited rationality and aggregateoutcomes, Journal of Economic Dynamics and Control 36,1101-1120.



Heuristic Switching Model:

Anufriev, M., and Hommes, C.H., (2012), Evolutionary selectionof individual expectations and aggregate outcomes, AmericanEconomic Journal: Microeconomics 4 (4), 35-64.

Anufriev, M., and Hommes, C.H., (2012), Evolution of marketheuristics, Knowledge Engineering Review 27, 255-271.

Anufriev, M., Hommes, C.H., and Philipse, R.H.S., (2013),Evolutionary selection of expectations in positive and negativefeedback markets, Journal of Evolutionary Economics, 23 (3),663-688.

heterogeneous agent models lecture 4 heuristic switching...

Documents