Statistical properties of network community structureJure Leskovec, CMUKevin Lang, Anirban Dasgupta and Michael MahoneyYahoo! Research

Network communities

Communities: Sets of nodes with lots

of connections inside and few to outside (the rest of the network)

Assumption: Networks are

(hierarchically) composed of communities (modules)

Communities, clusters, groups,

modules

Community score (quality) How community like is a

set of nodes? Want a measure that

corresponds to intuition Conductance

(normalized cut):Φ(S) = # edges cut / # edges inside

Small Φ(S) corresponds to more community-like sets of nodes

Community score (quality)

Score: Φ(S) = # edges cut / # edges inside

Community score (quality)

Score: Φ(S) = # edges cut / # edges inside

Bad communit

yΦ=5/7 = 0.7

Community score (quality)

Score: Φ(S) = # edges cut / # edges inside

Better communit

y

Φ=5/7 = 0.7

Bad communit

y

Φ=2/5 = 0.4

Community score (quality)

Score: Φ(S) = # edges cut / # edges inside

Better communit

y

Φ=5/7 = 0.7

Bad communit

y

Φ=2/5 = 0.4

Best communit

yΦ=2/8 = 0.25

Network Community Profile Plot We define:

Network community profile (NCP) plotPlot the score of best community of size k

Search over all subsets of size k and find best: Φ(k=5) = 0.25

NCP plot is intractable to compute Use approximation algorithm

NCP plot: Small Social Network Dolphin social network

Two communities of dolphins

NCP plotNetwork

NCP plot: Zachary’s karate club

Zachary’s university karate club social network During the study club split into 2 The split (squares vs. circles) corresponds to cut B

NCP plotNetwork

NCP plot: Network Science Collaborations between scientists in Networks

NCP plotNetwork

Geometric and Hierarchical graphs

Hierarchical network

Geometric (grid-like) network – Small social

networks– Geometric and– Hierarchical networkhave downward NCP plot

Our work: Large networks Previously researchers examined community

structure of small networks (~100 nodes) We examined more than 70 different large

social and information networks

Large real-world networks look

completely different!

Example of our findings Typical example:

General relativity collaboration network (4,158 nodes, 13,422 edges)

NCP: LiveJournal (N=5M, E=42M)

Better and better

communities

Worse and worse

communities

Best community

has 100 nodes

Com

mun

ity sc

ore

Community size

Explanation: Downward part Whiskers are responsible for

downward slope of NCP plot

Whisker is a set of nodes connected to the network by a single

edge

NCP plot

Largest whisker

Explanation: Upward part Each new edge inside the

community costs moreNCP plot

Φ=2/1 = 2Φ=8/3 = 2.6

Φ=64/11 = 5.8

Each node has twice as many

children

Suggested Network Structure

Network structure: Core-

periphery, jellyfish, octopus

Whiskers are responsible for

good communities

Denser and denser core

of the network

Core contains 60%

node and 80% edges

Caveat: Bag of whiskersWhat if we allow

cuts that give disconnected communities?

Cut all whiskers and compose

communities out of them

Caveat: Bag of whiskersCo

mm

unity

scor

e

Community size

We get better community scores when

composing disconnected sets of

whiskersConnected communitie

sBag of whiskers

Comparison to a rewired network

Rewired network: random

network with same degree distribution

What is a good model?

What is a good model that explains such network structure?

None of the existing models work

Pref. attachment Small World Geometric Pref. Attachment

FlatDown and Flat

Flat and Down

Forest Fire model works Forest Fire: connections spread like a fire

New node joins the network Selects a seed node Connects to some of its neighbors Continue recursively

As community grows it

blends into the core of

the network

Forest Fire NCP plot

rewired

network

Bag of whisk

ers

Conclusion and connections Whiskers:

Largest whisker has ~100 nodes Independent of network size Dunbar number: a person can maintain social

relationship to 150 people Bond vs. identity communites

Core: Newman et al. analyzed 400k node product network▪ Largest community has 50% nodes▪ Community was labeled “miscelaneous”

Conclusion and connections NCP plot is a way to analyze network

community structure Our results agree with previous work on small

networks But large networks are fundamentally

different Large networks have core-periphery structure

Small well isolated communities blend into the core of the networks as they grow