towards validating social network simulations
DESCRIPTION
A paper presented at ESSA 2013, Warsaw. Abstract: We consider the problem of finding suitable measures to validate simulated networks as outcome of (agent-based) social simulations. A number of techniques from computer science and social sciences are reviewed in this paper, which tries to compare and ‘fit’ various simulated networks to the avail-able data by using network measures. We look at several social network analy-sis measures but then turn our focus to techniques that not only consider the po-sition of the nodes but also their characteristics and their tendency to cluster with other nodes in the network – subgroup identification. We discuss how stat-ic and dynamic nature of networks may be compared. We conclude by urging a more comprehensive, transparent and rigorous approach to comparing simula-tion-generated networks against the available data.TRANSCRIPT
![Page 1: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/1.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 1
Towards Validating Social Network Simulations
SMA Abbas1, Shah Jamal Alam2 and Bruce Edmonds1
1Centre for Policy Modelling, Manchester Metropolitan University2School of Geosciences, University of Edinburgh
![Page 2: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/2.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 2
The Situation
A Simulation Social
“System”
GeneratesMeasured
Are these “essentially” the
same?
A Class of Networks Another Class of Networks?
Synthetic
Networks
TargetNetworks
![Page 3: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/3.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 3
Key questions
• What properties of the synthetic networks, one would expect to observe given how the model has been constructed
• Which of these properties are ‘significant’ in terms of the intended processes in the model
• Which class of target networks one might expect to observe if one could “re-run” reality under the same basic conditions as assumed in the model
• Do these classes match in important respects• How do we know they do given we only have
samples of synthetic and target networks
![Page 4: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/4.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 4
The Problem
![Page 5: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/5.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 5
Summary of Issue
• The space of possible networks is vast • But many networks will look similar to us, because
our brains can not deal with them but automatically simplifies them as part of perception
• We are not dealing with single networks but classes of networks…
• …though these classes are often implicit when a single network stands for that class (somehow)
• However, in principle, if synthetic and target networks do match (in some way) then this is potentially a strong validation
![Page 6: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/6.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 6
A Cautionary Tale – comparing two network modelsPapadopoulos et al. (2012) Popularity versus similarity in growing networks. Nature, 489:537-540.
![Page 7: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/7.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 7
But when compared in a different way…
![Page 8: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/8.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 8
Some Network Comparison Approaches
Different kinds of things to compare:• Network Measures• Network Distributions• Eigenvalue/Eigenvectors• Subgroup Identification• Functional Comparison• Likelihood of being described by an Exponential
Random Graph Model• Motif Prevalence
![Page 9: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/9.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 9
Examples that follow are from work of S.M.A. Abbas
(see papers at https://sites.google.com/site/maliabbas)
An Example of Validating Synthetic vs. Target Networks
![Page 10: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/10.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 10
An example of comparing measures
![Page 11: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/11.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 11
An example of comparing distributions
1 5 10 50 100 500 1000
5e
-04
5e
-03
5e
-02
5e
-01
Log-log Plot of Degree Distribution
Degree
Cu
mu
lative
Fre
qu
en
cy
ReferenceRandomFAOFPartyHybrid
![Page 12: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/12.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 12
Silo Index Comparison
Correlation0.83
-1.0 -0.9 -0.8 -0.7 -0.6 -0.5
-1.0
0-0
.85
-0.7
0
Dorm Silo Index
Reference Dorm
Hybrid D
orm Correlation
0.93
-1.00 -0.90 -0.80 -0.70
-1.0
0-0
.90
-0.8
0
Major Silo Index
Reference Major
Hybrid M
ajo
rCorrelation0.84
-1.0 -0.5 0.0 0.5
-1.0
-0.8
-0.6
Year Silo Index
Reference Year
Hybrid Y
ear
Correlation0.29
-1.0 -0.8 -0.6
-1.0
0-0
.90
High School Silo Index
Reference High School
Hybrid H
igh S
chool
Reference vs. Hybrid Mode Silo Indices
![Page 13: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/13.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 13
Assortativity Mixing
![Page 14: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/14.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 14
Problems in the Literature
Authors are often not clear about:• Precisely what the links in a synthetic network are
supposed to represent (in terms that would allow an in principle measurement of observed actors)
• Which aspects of the target network are subject to measurement error (or otherwise judged not to be significant) and which should be reproduced by a synthetic network
• Which aspects of the synthetic network are significant in terms of the generating process (and which are essentially accidental)
Readers often cannot judge the extent or meaning of the match/mismatch between synthetic and target networks
![Page 15: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/15.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 15
Conclusions
• Many social simulation models assume stereotypic networks (e.g. Watts-Strogatz)
• It is increasingly clear that the exact network structure matters (e.g. Holzhauer ESSA 2013)
• No single ‘Golden Bullet’ technique• More thought needed about what is significant about the
synthetic and target class networks• Multiple approaches needed to show that classes of
networks are similar – a few 1D measures is not enough to show this
• Validating networks could be quite a strong validation of our models…
• …but much more work is needed in this area!
![Page 16: Towards Validating Social Network Simulations](https://reader033.vdocuments.site/reader033/viewer/2022061110/544f4dd3af79596a6a8b87f8/html5/thumbnails/16.jpg)
Towards Validating Social Network Simulations, SMA Abbas, Shah Jamal Alam, and Bruce Edmonds, ESSA 2013, Warsaw. slide 16
Thanks!
SMA Abbas
https://sites.google.com/site/maliabbas
Shah Jamal Alam
https://sites.google.com/site/jamialam
Bruce Edmonds
http://bruce.edmonds.name
Slides at:
http://www.slideshare.net/BruceEdmonds