technical challenges of real-world agent-based modelling
DESCRIPTION
A presentation given to Data Science London in May 2013TRANSCRIPT
Thomas French
29th May 2013
www.sandtable.com
Technical Challenges of Real-World Agent-Based Modelling
Thursday, 30 May 13
Outline
• What is ABM?• Why use ABM?
• Classic ABM example• Real World ABM
• Three Key Technical Challenges
Thursday, 30 May 13
“Essentially, all models are wrong, but some are useful”
G.E. Box (1987)
Thursday, 30 May 13
ABM in a nutshell
AGENT
ENVIRONMENT
SENSORSMESSAGES
ACTIONS
PERCEPTS
OBJECT
ACTUATORS
Based&on&Bordini&et&al&&(2007)
Thursday, 30 May 13
Why are we talking about ABM?
• It shows promise for understanding complex systems:– heterogeneous and
adaptive actors– complex interactions:
interdependencies; feedback loops
– dynamic environment• It provides an accessible
metaphor for modelling– modelling individuals
• More and more data is available for our models- Finer levels of
granularity• Computing power is
available on-demand- Costs continue to
reduce
Thursday, 30 May 13
Classic ABM: Schelling Segregation Model
• Developed by Thomas Schelling in 1970s.• Study racial segregation of populations emerging
from individual discriminatory behaviours.
Thursday, 30 May 13
Source: Eric Fisher
Thursday, 30 May 13
Schelling Segregation Model
Thursday, 30 May 13
Schelling Segregation Model
Thursday, 30 May 13
Schelling Behaviour Tree
Thursday, 30 May 13
Real World ABM
Thursday, 30 May 13
QuitSIM Behaviour Tree
-
QUIT SIM 2QS Tree in Colour Censor
Thu May 30 2013
Thursday, 30 May 13
QuitSIM Behaviour Tree
Select State
Take up smoking?
Never smoker = 1
Become smoker
Age, gender
Prescription
age, gender, support, pregnant
NRT-OTC
age, gender, support, pregnant
NHS
age, gender, support, pregnant
CT
age, gender, support, pregnant
Choose route
Make quit attempt
Become quitter
Continue quit attempt?
Become ex-smoker
attempt length, age,, route, dependency
Become smoker
attempt length, route, age, dependency
Relapse Become smoker
Triggers, motivation, time since quit
Become smoker
age, gender, time since reduced
Do nothing
Set NRT flag
age, gender, support, pregnant
Cut down
attempt length, route, age, dependency
Become reducer
Use NRT for reduction?
Reducer
Reducer = 1
Consume media / ingest
experience
Consume media / ingest
experience
Consume media / ingest
experience
Consume media / ingest
experience
Ex-smoker
Ex-smoker = 1
Quitter
Quitter = 1
Smoker
Smoker = 1
Never Smoker
Never Smoker = 1E-cigarettes
age, gender, support, pregnant
Consume media / ingest
experience
Get support?
Set support flag
Plan to quit
Planned or Unplanned?
Do something about
smoking?
Prescription
age, gender, support, pregnant
NRT-OTC
age, gender, support, pregnant
NHS
age, gender, support, pregnant
CT
age, gender, support, pregnant
Choose route
Make quit attempt
Become quitter
Do nothing
E-cigarettes
age, gender, support, pregnant
Plan to quit
Planned or Unplanned?
Revert or take action?
Do something about
smoking?
motivation, events, price, GP, social, pregnant, media,
random
Get support? Set support flag
motivation, events, price, GP, social, pregnant, media,
random
QUIT%SIM%2QS#Tree#in#Colour
Tue#May#28#2013
Thursday, 30 May 13
Technical Challenges
BUILD VALIDATE EXPERIMENT
Designing*andbuilding*models
BuildingConfidence*in*Models
ConductingLarge?ScaleExperiments
HARD VERY,*VERY*HARD VERY*HARD
Thursday, 30 May 13
Building ModelsBUILD
VALIDATE
EXPERIMENT
Behavioural+Data
SurveyData
Assumptions
IntuitionAnalyse Build
Individual+Agent+Attributes
Behaviour+Tree
Environment(e.g.+Media)
Representative+Population
Data+SourcesSimulationComponents
Thursday, 30 May 13
Validation - Building Confidence
BUILD
VALIDATE
EXPERIMENT
Does the implemented model reflect the
real-world system?
Thursday, 30 May 13
Validation – Establishing Criteria
A framework for evaluating state of validity of models for on-going monitoring.
BUILD
VALIDATE
EXPERIMENT
VALIDATION
INTERNALVALIDATION
EXTERNALVALIDATION
Model&implemented&correctly
Behaviours&predicted&make&sense&/&are&logical
Model&stands&up&to&comparison&
with&external&data
Thursday, 30 May 13
Validation - Examples
Represented in a formal logic• linear-time temporal logic with extensions
Internal:(s_Att.gender = f) => (G (s_Att.gender = f) )G (!((s_Att.smoker = 1) && (s_Att.takeUp = 1)))G (!((s_Att.smoker = 1) && (s_Att.age < 11)))
External:n_MSE (s_Val1.prevalence, r_Val1.prevalence)n_MSE (s_Val2.quit_atts, r_Val2.quit_atts)
BUILD
VALIDATE
EXPERIMENT
Thursday, 30 May 13
Validation – Solving Multi-Criteria Problems
BUILD
VALIDATE
EXPERIMENT
Thursday, 30 May 13
Validation - Workflow
BUILD
VALIDATE
EXPERIMENT
Select&Model
Select&Tests
Select&Reference&Data
Configure&Test&Suite
Execute&Replications
Summarise&Individual&Tests
Summarise&Test&Suite
Thursday, 30 May 13
Experimentation - Approaches
• Empirical Calibration• Sensitivity Analysis• Scenario Exploration• Goal-Directed Search
BUILD
VALIDATE
EXPERIMENT
Thursday, 30 May 13
Experimentation – Exploring Parameter Spaces
BUILD
VALIDATE
EXPERIMENT
Small Large
Explore Exhaustive+Search
Simple+Random+Sampling,+Latin+Hypercube+Samplinge.g.+7+vars,+10/100+values+=+1+Trillion+parameter+sets
Seek Exhaustive+SearchNoisy,+MultiEObjective+Evolutionary+Algorithms
Parameter+Space
Search
+Typ
e
Thursday, 30 May 13
Experimentation -Handling Noise
BUILD
VALIDATE
EXPERIMENT
Thursday, 30 May 13
Experimentation – Handling Output Data
BUILD
VALIDATE
EXPERIMENT
Thursday, 30 May 13
Experimentation – Platform Architecture
BUILD
VALIDATE
EXPERIMENT
CATALOG
REST API
WORKFLOW
SCENARIOS
ANALYSIS
VALIDATION
OPTIMISATION
SERVICESmongoDB
MANAGERWORKER 1
PLATFORM
RabbitMQ
MESSAGING
http://sandtable.com
Sandtable Simulation Platform
CLIENT
simulation
analysis
validation
12
k
2
3
N
S3
Sandtable)Simulation)Platform
Thursday, 30 May 13
Experimentation - Managing Workflow
BUILD
VALIDATE
EXPERIMENT
Thursday, 30 May 13
Thursday, 30 May 13
“Nothing is built on stone; all is built on sand. But we must
build as if sand were stone.”J.L. Borges
Thursday, 30 May 13
Thanks for listening!
Thursday, 30 May 13
Further study
Book:• John Miller and Scott Page: 'Complex Adaptive
Systems: An Introduction to Computational Models of Social Life' (2007)
Coursera:• Scott Page: 'Model Thinking'• https://www.coursera.org/course/modelthinking
Thursday, 30 May 13