cmu scs finding patterns in large, real networks christos faloutsos cmu

CMU SCS

Finding patterns in large, real networks

Christos Faloutsos

LLNL 05 C. Faloutsos 2

CMU SCS

Thanks to

• Deepayan Chakrabarti (CMU/Yahoo)

• Michalis Faloutsos (UCR)

• Spiros Papadimitriou (CMU/IBM)

• George Siganos (UCR)

CMU SCS

Introduction

Internet Map [lumeta.com]

Food Web [Martinez ’91]

Protein Interactions [genomebiology.com]

Friendship Network [Moody ’01]

Graphs are everywhere!

CMU SCS

Outline

• Laws– static

– time-evolution laws

– why so many power laws?

• tools/results

• future steps

CMU SCS

Motivating questions

• Q1: What do real graphs look like?• Q2: How to generate realistic graphs?

CMU SCS

Virus propagation

• Q3: How do viruses/rumors propagate?

• Q4: Who is the best person/computer to immunize against a virus?

CMU SCS

Graph clustering & mining

• Q5: which edges/nodes are ‘abnormal’?

• Q6: split a graph in k ‘natural’ communities - but how to determine k?

CMU SCS

Outline

• Laws– static

• tools/results

• future steps

CMU SCS

Topology

Q1: How does the Internet look like? Any rules?

(Looks random – right?)

CMU SCS

Laws – degree distributions

• Q: avg degree is ~2 - what is the most probable degree?

degree

count ??

CMU SCS

Laws – degree distributions

• Q: avg degree is ~2 - what is the most probable degree?

degreedegree

count ??

CMU SCS

I.Power-law: outdegree O

The plot is linear in log-log scale [FFF’99]

freq = degree (-2.15)

O = -2.15

Exponent = slope

Outdegree

Frequency

Nov’97

CMU SCS

II.Power-law: rank R

• The plot is a line in log-log scale

Rank: nodes in decreasing degree order

April’98

log(rank)

log(degree)

att.com

ibm.com

CMU SCS

III.Power-law: eigen E

• Eigenvalues in decreasing order (first 20)• [Mihail+, 02]: R = 2 * E

E = -0.48

Exponent = slope

Eigenvalue

Rank of decreasing eigenvalue

Dec’98

CMU SCS

IV. The Node Neighborhood

• N(h) = # of pairs of nodes within h hops

CMU SCS

• Q: average degree = 3 - how many neighbors should I expect within 1,2,… h hops?

• Potential answer:1 hop -> 3 neighbors

2 hops -> 3 * 3

h hops -> 3h

CMU SCS

2 hops -> 3 * 3

h hops -> 3h

WE HAVE DUPLICATES!

CMU SCS

2 hops -> 3 * 3

h hops -> 3h

‘avg’ degree: meaningless!

CMU SCS

IV. Power-law: hopplot H

Pairs of nodes as a function of hops N(h)= hH

H = 4.86

Dec 98Hops

# of Pairs # of Pairs

Hops Router level ’95

H = 2.83

CMU SCS

Observation• Q: Intuition behind ‘hop exponent’?• A: ‘intrinsic=fractal dimensionality’ of the

network

N(h) ~ h1 N(h) ~ h2

CMU SCS

Hop plots

• More on fractal/intrinsic dimensionalities: very soon

CMU SCS

Any other ‘laws’?

CMU SCS

Any other ‘laws’?

• Small diameter– six degrees of separation / ‘Kevin Bacon’

– small worlds [Watts and Strogatz]

CMU SCS

Any other ‘laws’?• Bow-tie, for the web [Kumar+ ‘99]

• IN, SCC, OUT, ‘tendrils’

• disconnected components

CMU SCS

Any other ‘laws’?• power-laws in communities (bi-partite cores)

[Kumar+, ‘99]

2:3 core(m:n core)

Log(m)

Log(count)

n:2n:3

CMU SCS

More Power laws

• Also hold for other web graphs [Barabasi+, ‘99], [Kumar+, ‘99]

• citation graphs (see later)

• and many more

CMU SCS

Outline

• Laws– static

• tools/results

• future steps

CMU SCS

How do graphs evolve?

CMU SCS

Time Evolution: rank R

The rank exponent has not changed! [Siganos+, ‘03]

Domainlevel

#days since Nov. ‘97

CMU SCS

How do graphs evolve?

• degree-exponent seems constant - anything else?

CMU SCS

Evolution of diameter?

• Prior analysis, on power-law-like graphs, hints that

diameter ~ O(log(N)) or

diameter ~ O( log(log(N)))

• i.e.., slowly increasing with network size

• Q: What is happening, in reality?

CMU SCS

Evolution of diameter?

• Prior analysis, on power-law-like graphs, hints that

diameter ~ O(log(N)) or

diameter ~ O( log(log(N)))

• i.e.., slowly increasing with network size

• Q: What is happening, in reality?

• A: It shrinks(!!), towards a constant value

CMU SCS

Shrinking diameter

[Leskovec+05a]• Citations among physics

papers • 11yrs; @ 2003:

– 29,555 papers– 352,807 citations

• For each month M, create a graph of all citations up to month M

CMU SCS

Shrinking diameter• Authors &

publications

• 1992– 318 nodes

– 272 edges

• 2002– 60,000 nodes

• 20,000 authors

• 38,000 papers

– 133,000 edges

CMU SCS

Shrinking diameter• Patents & citations

• 1975– 334,000 nodes

– 676,000 edges

• 1999– 2.9 million nodes

– 16.5 million edges

• Each year is a datapoint

CMU SCS

Shrinking diameter• Autonomous

systems

• 1997– 3,000 nodes

– 10,000 edges

• 2000– 6,000 nodes

– 26,000 edges

• One graph per day

CMU SCS

Temporal evolution of graphs

• N(t) nodes; E(t) edges at time t

• suppose that

N(t+1) = 2 * N(t)

• Q: what is your guess for

E(t+1) =? 2 * E(t)

CMU SCS

• N(t) nodes; E(t) edges at time t

• suppose that

N(t+1) = 2 * N(t)

• Q: what is your guess for

E(t+1) =? 2 * E(t)

• A: over-doubled!

CMU SCS

• A: over-doubled - but obeying:

E(t) ~ N(t)a for all t

where 1<a<2

CMU SCS

• A: over-doubled - but obeying:

E(t) ~ N(t)a for all t

• Identically:

log(E(t)) / log(N(t)) = constant for all t

CMU SCS

Densification Power Law

ArXiv: Physics papers

and their citations

CMU SCS

and their citations

‘tree’

CMU SCS

and their citations

E(t)‘clique’

CMU SCS

U.S. Patents, citing each other

CMU SCS

Autonomous Systems

CMU SCS

ArXiv: authors & papers

CMU SCS

Summary of ‘laws’

• Power laws for degree distributions

• …………... for eigenvalues, bi-partite cores

• Small & shrinking diameter (‘6 degrees’)

• ‘Bow-tie’ for web; ‘jelly-fish’ for internet

• ``Densification Power Law’’, over time

CMU SCS

Outline

• Laws– static

– why so many power laws?• other power laws

• fractals & self-similarity

• case study: Kronecker graphs

• tools/results

• future steps

CMU SCS

Power laws

• Many more power laws, in diverse settings:

CMU SCS

A famous power law: Zipf’s law

• Bible - rank vs frequency (log-log)

log(rank)

log(freq)

“a”

“the”

CMU SCS

Power laws, cont’ed

• length of file transfers [Bestavros+]

• web hit counts [Huberman]

• magnitude of earthquakes (Guttenberg-Richter law)

• sizes of lakes/islands (Korcak’s law)

• Income distribution (Pareto’s law)

CMU SCS

The Peer-to-Peer Topology

• Frequency versus degree

• Number of adjacent peers follows a power-law

[Jovanovic+]

CMU SCS

Web Site Traffic

log(freq)

log(count)Zipf

Click-stream data

u-id’s url’s

log(freq)

log(count)

‘yahoo’

‘super-surfer’

CMU SCS

Lotka’s law

(Lotka’s law of publication count); and citation counts: (citeseer.nj.nec.com 6/2001)

log(#citations)

log(count)

J. Ullman

CMU SCS Swedish sex-web

Nodes: people (Females; Males)Links: sexual relationships

Liljeros et al. Nature 2001

4781 Swedes; 18-74; 59% response rate.

Albert Laszlo Barabasihttp://www.nd.edu/~networks/Publication%20Categories/04%20Talks/2005-norway-3hours.ppt

CMU SCS Swedish sex-web

Nodes: people (Females; Males)Links: sexual relationships

Liljeros et al. Nature 2001

4781 Swedes; 18-74; 59% response rate.

Albert Laszlo Barabasihttp://www.nd.edu/~networks/Publication%20Categories/04%20Talks/2005-norway-3hours.ppt

CMU SCS

Power laws

• Q: Why so many power laws?

• A1: self-similarity

• A2: ‘rich-get-richer’

CMU SCS

Digression: intro to fractals

• Fractals: sets of points that are self similar

CMU SCS

A famous fractal

e.g., Sierpinski triangle:

...zero area;

infinite length!

dimensionality = ??

CMU SCS

A famous fractal

e.g., Sierpinski triangle:

...zero area;

infinite length!

dimensionality = log(3)/log(2) = 1.58

CMU SCS

A famous fractal

equivalent graph:

CMU SCS

Intrinsic (‘fractal’) dimension

How to estimate it?

CMU SCS

Intrinsic (‘fractal’) dimension

• Q: fractal dimension of a line?

• A: nn ( <= r ) ~ r^1(‘power law’: y=x^a)

• Q: fd of a plane?

• A: nn ( <= r ) ~ r^2

fd== slope of (log(nn) vs log(r) )

CMU SCS

Sierpinsky triangle

log( r )

log(#pairs within <=r )

== ‘correlation integral’

= CDF of pairwise distances

CMU SCS

Sierpinsky triangle

log( r )

CMU SCS

log( r )

CMU SCS

2-d (Plane)

log( r )

CMU SCS

Recall: Hop Plot

• Internet routers: how many neighbors within h hops? (= correlation integral!)

Reachability function: number of neighbors within r hops, vs r (log-log).

Mbone routers, 1995log(hops)

log(#pairs)

CMU SCS

Fractals and power laws

They are related concepts:

• fractals <=>

• self-similarity <=>

• scale-free <=>

• power laws ( y= xa )– F = C r(-2)

CMU SCS

Fractals and power laws

• Power laws and fractals are closely related• And fractals appear in MANY cases

– coast-lines: 1.1-1.5– brain-surface: 2.6– rain-patches: 1.3– tree-bark: ~2.1– stock prices / random walks: 1.5– ... [see Mandelbrot; or Schroeder]

CMU SCS

Outline

• Laws– static

– why so many power laws?• other power laws

• fractals & self-similarity

• case study: Kronecker graph generator

• tools/results

• future steps

CMU SCS

Self-similarity at work

• Q2: How to generate realistic graphs?

• (Q2’) and how to grow it over time?

CMU SCS

Wish list for a generator:

• Power-law-tail in- and out-degrees

• Power-law-tail scree plots

• shrinking/constant diameter

• Densification Power Law

• communities-within-communities

Q: how to achieve all of them?

CMU SCS

Wish list for a generator:

• shrinking/constant diameter

• Densification Power Law

Q: how to achieve all of them?

A: Kronecker matrix product [Leskovec+05b]

CMU SCS

Kronecker product

CMU SCS

Kronecker product

CMU SCS

Kronecker product

N N*N N**4

CMU SCS

Properties of Kronecker graphs:

• constant diameter

• perfect Densification Power Law

CMU SCS

Properties of Kronecker graphs:

• constant diameter

• perfect Densification Power Law

and we can prove all of the above

(first and only generator that does that)

CMU SCS

Kronecker - ArXiv

(stochastic)Kronecker

Degree Scree Diameter D.P.L.

(det. Kronecker)

CMU SCS

Kronecker - patents

Degree Scree Diameter D.P.L.

CMU SCS

Kronecker - A.S.

CMU SCS

Outline

• Laws– static

• tools/results– virus propagation

– connection subgraphs

– cross-associations

• future steps

CMU SCS

Virus propagation

CMU SCS

The model: SIS

• ‘Flu’ like: Susceptible-Infected-Susceptible

• Virus ‘strength’ s= /

Infected

Healthy

Prob. β

CMU SCS

Epidemic threshold

of a graph: the value of , such that

if strength s = / < an epidemic can not happen

• given a graph

• compute its epidemic threshold

CMU SCS

Epidemic threshold

What should depend on?

• avg. degree? and/or highest degree?

• and/or variance of degree?

• and/or third moment of degree?

• and/or diameter?

CMU SCS

Epidemic threshold

• [Theorem] We have no epidemic, if

β/δ <τ = 1/ λ1,A

CMU SCS

Epidemic threshold

• [Theorem] We have no epidemic, if

β/δ <τ = 1/ λ1,A

largest eigenvalueof adj. matrix A

attack prob.

recovery prob.epidemic threshold

Proof: [Wang+03]

CMU SCS

Experiments (Oregon)

/ > τ (above threshold)

/ = τ (at the threshold)

/ < τ (below threshold)

CMU SCS

Our result:

• Holds for any graph

• includes older results as special cases

CMU SCS

Virus propagation

• A4: the one that causes the highest decrease in 1

CMU SCS

Outline

• Laws– static

• future steps

CMU SCS

(A-A) Kidman-Diaz

• What are the best paths between ‘Kidman’ and ‘Diaz’?

CMU SCS

(A-A) Kidman-Diaz

Strong, direct link

• What are the best paths between ‘Kidman’ and ‘Diaz’? [KDD’04, w/ McCurley+Tomkins]

CMU SCS

Outline

• Laws– static

• future steps

CMU SCS

Graph partitioning

• Documents x terms

• Customers x products

• Users x web-sites

CMU SCS

Graph partitioning

• Q: HOW MANY PIECES?

CMU SCS

Graph partitioning

• Q: HOW MANY PIECES?

• A: MDL/ compression

CMU SCS

Cross-associations

1x2 2x2

CMU SCS

Cross-associations

2x3 3x3 3x4

CMU SCS

Cross-associations

CMU SCS

Cross-associations

CMU SCS

• A6: choose the k that leads to best overall compression (= MDL = Minimum Description Language)

CMU SCS

Cross-associations

outlier edgemissing edge

CMU SCS

Outline

• Laws– static

• tools/results

• other related projects

• future steps

CMU SCS

Motivation

water distribution network

normal operation

May have hundreds of measurements, butit is unlikely they are completely unrelated!

Phase 1 Phase 2 Phase 3

: : : : : :

sensorsnear leak

sensorsawayfrom leak

CMU SCS

Phase 1 Phase 2 Phase 3

: : : : : :

Motivation

water distribution network

normal operation major leak

May have hundreds of measurements, butit is unlikely they are completely unrelated!

sensorsnear leak

sensorsawayfrom leak

CMU SCS

Motivation

actual measurements(n streams)

k hidden variable(s)

We would like to discover a few “hidden(latent) variables” that summarize the key trends

Phase 1

: : : : : :

Phase 1

CMU SCS

Motivation

Phase 1 Phase 1Phase 2 Phase 2

: : : : : :

CMU SCS

Motivation

Phase 1 Phase 1Phase 2 Phase 2Phase 3 Phase 3

: : : : : :

CMU SCS

Stream mining

• Solution: SPIRIT [VLDB’05]– incremental, on-line PCA

CMU SCS

Outline

• Laws

• tools/results

• other related projects

• future steps

CMU SCS

Future steps

• connection sub-graphs on multi-graphs [Tong]

• traffic matrix, evolving over time [Sun+]

• influence propagation on blogs [Leskovec+]

• automatic web site classification [Leskovec]

CMU SCS

Conclusions

• Power laws & self similarity– excellent tools, for real datasets,

– where Gaussian, uniform etc assumptions fail

• Additional tools– Kronecker, for graph generators

– eigenvalues for virus propagation

– MDL/compression for graph partitioning

CMU SCS

References

• [Aiello+, '00] William Aiello, Fan R. K. Chung, Linyuan Lu: A random graph model for massive graphs. STOC 2000: 171-180

• [Albert+] Reka Albert, Hawoong Jeong, and Albert-Laszlo Barabasi: Diameter of the World Wide Web, Nature 401 130-131 (1999)

• [Barabasi, '03] Albert-Laszlo Barabasi Linked: How Everything Is Connected to Everything Else and What It Means (Plume, 2003)

CMU SCS

References, cont’d

• [Barabasi+, '99] Albert-Laszlo Barabasi and Reka Albert. Emergence of scaling in random networks. Science, 286:509--512, 1999

• [Broder+, '00] Andrei Broder, Ravi Kumar, Farzin Maghoul, Prabhakar Raghavan, Sridhar Rajagopalan, Raymie Stata, Andrew Tomkins, and Janet Wiener. Graph structure in the web, WWW, 2000

CMU SCS

• [Chakrabarti+, ‘04] RMAT: A recursive graph generator, D. Chakrabarti, Y. Zhan, C. Faloutsos, SIAM-DM 2004

• [Dill+, '01] Stephen Dill, Ravi Kumar, Kevin S. McCurley, Sridhar Rajagopalan, D. Sivakumar, Andrew Tomkins: Self-similarity in the Web. VLDB 2001: 69-78

CMU SCS

• [Fabrikant+, '02] A. Fabrikant, E. Koutsoupias, and C.H. Papadimitriou. Heuristically Optimized Trade-offs: A New Paradigm for Power Laws in the Internet. ICALP, Malaga, Spain, July 2002

• [FFF, 99] M. Faloutsos, P. Faloutsos, and C. Faloutsos, "On power-law relationships of the Internet topology," in SIGCOMM, 1999.

CMU SCS

• [Leskovec+05a] Jure Leskovec, Jon Kleinberg and Christos Faloutsos Graphs over Time: Densification Laws, Shrinking Diameters and Possible Explanations KDD 2005, Chicago, IL. (Best research paper award)

• [Leskovec+05b] Jure Leskovec, Deepayan Chakrabarti, Jon Kleinberg and Christos Faloutsos, Realistic, Mathematically Tractable Graph Generation and Evolution, Using Kronecker Multiplication, ECML/PKDD 2005, Porto, Portugal.

CMU SCS

• [Jovanovic+, '01] M. Jovanovic, F.S. Annexstein, and K.A. Berman. Modeling Peer-to-Peer Network Topologies through "Small-World" Models and Power Laws. In TELFOR, Belgrade, Yugoslavia, November, 2001

• [Kumar+ '99] Ravi Kumar, Prabhakar Raghavan, Sridhar Rajagopalan, Andrew Tomkins: Extracting Large-Scale Knowledge Bases from the Web. VLDB 1999: 639-650

CMU SCS

• [Leland+, '94] W. E. Leland, M.S. Taqqu, W. Willinger, D.V. Wilson, On the Self-Similar Nature of Ethernet Traffic, IEEE Transactions on Networking, 2, 1, pp 1-15, Feb. 1994.

• [Mihail+, '02] Milena Mihail, Christos H. Papadimitriou: On the Eigenvalue Power Law. RANDOM 2002: 254-262

CMU SCS

• [Milgram '67] Stanley Milgram: The Small World Problem, Psychology Today 1(1), 60-67 (1967)

• [Montgomery+, ‘01] Alan L. Montgomery, Christos Faloutsos: Identifying Web Browsing Trends and Patterns. IEEE Computer 34(7): 94-95 (2001)

CMU SCS

• [Palmer+, ‘01] Chris Palmer, Georgos Siganos, Michalis Faloutsos, Christos Faloutsos and Phil Gibbons The connectivity and fault-tolerance of the Internet topology (NRDM 2001), Santa Barbara, CA, May 25, 2001

• [Pennock+, '02] David M. Pennock, Gary William Flake, Steve Lawrence, Eric J. Glover, C. Lee Giles: Winners don't take all: Characterizing the competition for links on the

web Proc. Natl. Acad. Sci. USA 99(8): 5207-5211 (2002)

CMU SCS

• [Schroeder, ‘91] Manfred Schroeder Fractals, Chaos, Power Laws: Minutes from an Infinite Paradise W H Freeman & Co., 1991 (excellent book on fractals)

CMU SCS

• [Siganos+, '03] G. Siganos, M. Faloutsos, P. Faloutsos, C. Faloutsos Power-Laws and the AS-level Internet Topology, Transactions on Networking, August 2003.

• [Wang+03] Yang Wang, Deepayan Chakrabarti, Chenxi Wang and Christos Faloutsos: Epidemic Spreading in Real Networks: an Eigenvalue Viewpoint, SRDS 2003, Florence, Italy.

CMU SCS

Reference

• [Watts+ Strogatz, '98] D. J. Watts and S. H. Strogatz Collective dynamics of 'small-world' networks, Nature, 393:440-442 (1998)

• [Watts, '03] Duncan J. Watts Six Degrees: The Science of a Connected Age W.W. Norton & Company; (February 2003)

CMU SCS

Thank you!

www.cs.cmu.edu/~christos

www.db.cs.cmu.edu

cmu scs finding patterns in large, real networks christos faloutsos cmu

Documents

carnegie mellon finding patterns in large, real networks...

1 graphs over time densification laws, shrinking diameters...

cmu scs large graph mining: power tools and a...

cmu scs graph mining: patterns and tools for static and...

cmu scs graph analytics wkshpc. faloutsos (cmu) 1 graph...

cmu scs thank you! c. faloutsos cmu. cmu scs large graph...

cmu scs mining graphs and tensors christos faloutsos cmu

cmu scs multimedia and graph mining christos faloutsos cmu

cmu scs mining billion-node graphs: patterns, generators and...

motivation - applications (cont’d)€¦ · cmu 1 cmu scs...

cmu scs 15-826: multimedia databases and data mining...

tensor analysis – applications and algorithmsusers.wfu.edu...

large graph mining - department of computer science and ......

cmu scs large graph mining - patterns, explanations and...

cmu scs c. faloutsos (cmu)#1 large graph algorithms christos...

cmu scs data mining on streams christos faloutsos cmu

cmu scs apweb 07(c) 2007, c. faloutsos 1 copyright notice...

anomaly detection and virus propagation in large graphs...

cmu scs indexing and mining biological images christos...

cmu scs graph mining and influence propagation christos...