revealing the weakness of sna and possibly fixing it , using mas

Revealing the weakness of SNA and possibly fixing it, using MAS

Bruce EdmondsCentre for Policy Modelling

Manchester Metropolitan University

Modelling and Social Network Analysis

Revealing the weakness of SNA

and possibly fixing it, using MAS

Introduction

Revealing the weakness of SNA and possibly fixing it, using MAS, Bruce Edmonds, SNAMAS invited Talk, AISB Leicester, March 2010. slide-2


Modelling parts and relations

Object Systemknown unknown

Formal Modelinput

(parameters, initial conditions etc.)

output(results)

encoding(measurement)

decoding(interpretation)

All the stages are necessary for the model to be useful


Modelling ideas rather than observed systems

Object System

conceptual model

Model


Some ‘scientific’ uses of modelling • Prediction: Provide information about a

current unknown by inference from known information

• Explanation: Provide an explanation how an outcome resulted from some conditions

• Analogy: Provide a framework for (or a way of) thinking about a complex system

• But there are many other uses: illustration, personal exploration, persuasion, counter example, etc.

Social Network Analysis

• Abstracts a target system to a system of (possibly rich and dynamic) nodes and arcs

• It is necessary to decide what a node and an arc are (in terms of what nodes and arc represent in the target system)

• Key idea: the structure of the abstracted network tells us something useful about the properties of the system



Modelling parts and relations with a Social Network Model

Object Systemknown unknown

SNA Modeloutput in terms of

visualisation, measures etc.

representation in terms of arcs

and nodes

interpretation of output

It is not a model until there is an analysis of a network that can be interpreted in terms of the object system

About SNA Models

• Representing anything as a network involves many decisions as how to do this

• The resulting representation is only a model if one can infer anything from it

• Often this inference is implicit or informal• If the inference is specified it can be called a

Social Network Model (SNM)• Any model of an observed system is a

contingent (mini-)theory• that is, it could be wrong however plausible it

seems


Descriptive Network Analysis

Measures upon networks – the very idea!, Bruce Edmonds, Social Netwrok Conference, London, 14th July 2007. slide-9

A

B

A measure on the network, M(x)

Based on an already existing good

understanding of what is happening in

the target system

Choose and use measure etc. to

illustrate that understanding

Network Analysis as a Generative Model

Measures upon networks – the very idea!, Bruce Edmonds, Social Netwrok Conference, London, 14th July 2007. slide-10

A

B

E.g. using a measure on the network, M(x)

Given an observed system

model it with a SNM

Use the model to infer something about the

model that is meaningful in terms of the observed system

Validating a Social Network Model

• Since a SNA model is a (complex) contingent hypothesis about the target system

• To be trusted it needs to be independently validated (strong validation)

• This is very expensive to do with SNMs since not only does the data need to be collected and the model built but it also tested against what is measured in the target system

• So instead it is usual to validate weakly using the intuitions of the researcher who did the analysis which is clearly insufficient if we are to rely on it for any purpose (e.g. understaning)


Using ABS to Probe SNA Assumptions

• However we can explore the robustness of SNA against plausible social simulations

• An artificial test-bed for SNA

• This can indicate the conditions under which a particular SNM can be relied upon (or not) given which assumptions

• If a SNM of an ABS cannot be made to work how could we rely on this when considering real social phenomena?


Analysing a Simulation of a P2P System



Example 1


Example I: A Peer-to-Peer (P2P) File-sharing system

Collection of ‘servers’, each of which:– Is controlled by a user to some extent– ‘Knows’ a limited number of servers, with which

it can communicate (the network)– Makes some (or no) files available for

download by other servers– Search for files is by flood-fill: (i.e. send query

to n others who send it to n others…)– If query matches an available file it is sent back

to originator

• E.g. Bittorrent


A Simulation of a P2P System

• 50 servers, each can decide to share files (coop) or not (def) at any time

• Try collect ‘sets’ of related files stored (initially) randomly by sending queries

• Satisfaction is measured by success at collecting files – (small) cost of dealing with others’ queries (but decays over time)

• May look at and copy what a more satisfied server does, or may drop out and be replaced (especially if satisfaction is low)


Number of co-operators in a run of the simulation (out of 50)


• Key issue is number (and manner) of cooperation– Why does anyone cooperate?– How does network structure impact upon this?

Typical Emergent Network Structure

core partitition periphery

small isolated group


Suggests four types of node

• In-coop – those who share their files in core partition

• In-def – those who don’t share their files in core partition

• Out-coop – those who share their files but are outside the core partition

• Out-def – those who don’t share their files but are outside the core partition


Some General Statistics

Type Average utility Average number of

links

Average centrality

in-coop 0.79 3.0 0.41

out-coop 0.51 2.5 0.31

in-def 0.37 2.0 0.27

out-def 0.32 1.5 0.19


Over all the runs for all nodes and later times

Stop!! Time for a Thought ExperimentIf this were observations of a real P2P system

and not a simulation, what would you conclude from this analysis:

1.That the kind of node (using the above categorisation) was a significant factor in the utility of nodes?

2.That either a node’s number of links or centrality was a significant factor in achieving its utility?

Wouldn’t a paper that came to positive conclusions on these questions be publishable?



0.08

0.09

0.1

0.11

0.12

0.13

0.14

0.15

0.16

0.17

0.18

0 2 4 6 8 10 12

Lag (in cycles)

Co

rre

lati

on

cent R

num l R

Over all kinds of nodes and later times and runs

Regression coefficients with satisfaction levels of nodes

Type Number of links

Number of links

lagged 6 periods

Centrality Centrality lagged 6 periods

in-coop -0.058 0.13 -0.062 0.12

out-coop 0.073 0.17 0.065 0.16

in-def 0.039 0.074 0.067 0.087

out-def -0.15 -0.053 0.066 0.13


Other measures and lags had lower correlations, including those that just did these in aggregate

Size of partitions during a run

Blue – size of largest partition

Green – 2nd largest (if there is one)

Red, orange, etc. – even smaller ones


Conclusion of P2P Case-study

• The global measures were not very useful in providing understanding leverage

• It can be unsafe to assume that such measures derived from empirical studies give a helpful picture of the role of networks

• The structural analysis based on the detailed understanding of the dynamics created a more useful categorisation of node types (but this is precisely the kind of understanding difficult to obtain when the system is real rather than simulated)

• Given this understanding it might be possible to choose better measures etc.

• It is important to distinguish demonstrating an existing understanding of a network from fishing for understanding using SNA measures


A “simple” abstract system



Example 2


The Target System

• So here I will consider this system looking at the question of whether any measure can be relied upon to indicate eventual node importance.

• It is:– Relatively simple– Deterministic– About which we have almost complete

information about behaviour, links, etc. to help us chose our measure

Measures upon networks – the very idea!, Bruce Edmonds, Social Network Conference, London, 14th July 2007. slide-26

The Abstract System


E

1

1

1

2

2

2

20

9

?

Using the Experimental Method to Produce Reliable Self-Organised Systems, B. Edmonds, ESOA 2004, New York, July 2004, http://cfpm.org/~bruce slide-28

Basic System Outline

• Giving Agent System with Plans• Fixed number of agents: A1, A2, …An

• Each agent, Ax, has– a store, Sx

– a fixed number of plans: Px1, Px2, …

• Each Plan, Pxy, composed of instructions:– A fixed number of “give one to”– And one test: If Si is zero then do plan j next, otherwise

plan k next

• Each time click, all do: get 1 unit; use current plan to: [do giving (while they have);

test others; note next plan].


Thus all that happens in GASP systems is:• That agents have a fixed set of very simple

plans/programs

• Their state is the amount in their store and the index of the current plan

• All they do is give fixed amounts to other agents accorind to their current plan

• All they can perceive is whether an other agent’s store is zero or not…

• …which determines the index of the next plan in a fixed way


An illustration of a GASP system

Plan 1:G3

G2

JZ2,1,3

Plan 2:

JZ1,2,3

Plan 3:G2

G2

G2

JZ2,3,3

Agent 1

1 2 3

Agent 2

1 2 3

Agent 3

Etc.

Check if zero

4

27Store:

Thus the reformulated question is...

Given almost complete knowledge of a particular GASP system (except for the initial store of Agent-1), can you effectively find any measure, M, such that:

• If and only if M(A) ≥ M(B) then...

• Eventually S(t,A) ≥ S(t,B) [ where S(t,x) is the value of the store in agent x at time t ]

• That is given this system is there an M: M(A) ≥ M(B) ↔ T; for t>T S(t,A) ≥ S(t,B)


And the answer is... No!

• In other words, there are GASP systems, where even though we know: their complete behaviour (comparable to detailed interviews of all participants); everything possible about their social network (who they can make transfers to); and almost all of the initial conditions (except one value)...

• ...there is no measure that will tell us from the structure which nodes will be more influential than others once running.


Proof Sketch

• The class of GASP systems are Turing Complete, in other words they can compute anything a Turing Machine (TM) can (shown by a mapping into an Unlimited Register Machine a know TM equivalent).

• If there were a such a measure, then we could use it to check (without computation) that the results of two GASP systems (the end value in the store of Agent-1) were equal by joining the two systems into one; finding the measure, M and then using it to see if the two output nodes would be equal. This is a known uncomputable problem.


(the pessimistic) Moral!

• Even with very simple, deterministic systems, where we know everything about the behaviour and structure of the system, there are no measures that a priori inform us about node importance.

• This backs up simulation studies where a set of apparently sensible measures fail to do the same.

• Therefore the burden of proof is on those that claim, with a largely unknown complex system, that a measure will tell us such information!

• Effective measurement follows understanding


The Cure?



Part 4


Why SNM might be inadequate

• SNM are simply too abstract to adequately represent social complexity

• The jump from rich social phenomena to simple network model is too great

• This is usually masked by– The prima face plausibility of SNM– That SNA work is traditionally divided between:

• Theorists that study what can be inferred from SN• Social Scientists who represent using SN and then

trust that the theoretical techniques work


Staging Abstraction using ABS


The phenomena of

concern

The phenomena of

concern

SimulationSimulationSimulation

Social Network Models

Social Network Measures

Example II: A Simulation of Ga-Selala (in the Limpopo Valley of South Africa)• A Complex Evidence-led Simulation of a

particular village • Represents many aspects of life there,

including: sexual network and HIV/AIDS spread, friendship network, kinship network, employment, savings clubs, household structure, birth and death, government grants and health

• Purpose was to assess impacts of factors, in particular how fragile the social structure might be to these factors given the complex interplay of the various social structures and behaviours


Basic Methodology

• Repeated iterations of model development in response to stakeholder criticism, expert opinion, statistics, interviews etc.

• So that most aspects of the model had some (but varying) levels of justification from available evidence

• Result is a context-specific but dynamic “description” using a computer simulation

• Simulation is difficult to understand and slow to run, but open to experiment and inspection

• Changes in network structure can be studied in the simulation even though it is highly dynamic


Observations from running simulation experiments

• That (given the introduction of a new mining enterprise near the village) the social structure(s) collapsed

• To try and show this, snapshots of the social network taken and their degree distribution compared using non-parametric statistics (Kolmogorov-Sinai) to see if there is evidence of significant change


Comparing the social network over time with that at time 0


Initialised with Watts-Strogatz Small-world network

Initialised with Erdös random network

P-scores of K-S test on the degree distributions of the social networks

Comparing the social network over time with the previous time


Initialised with Watts-Strogatz Small-world network

Initialised with Erdös random network

P-scores of K-S test on the degree distributions of the social networks

Conclusion of talk

• SNM are weak in the sense that they are contingent and yet almost always without any independent validation

• Their apparent power comes from their simplicity and plausibility

• ABS can be used to test the assumptions behind SNA analyses in vitro

• ABS can be used to stage the abstraction from evidence to SNA, allowing chains of reference to be maintained and understanding gained to inform SNA


The End

Bruce Edmondshttp://bruce.edmonds.nameCentre for Policy Modelling

http://cfpm.orgManchester Metropolitan University Business School

http://www.business.mmu.ac.uk


revealing the weakness of sna and possibly fixing it , using mas

Documents