pagerank centrality of dynamic graph structures

(PageRank) Centrality of dynamic graph

structures

David F. Gleich!Computer Science"Purdue University

1 David Gleich · Purdue AN14 · MS59

Models and algorithms for high performance !matrix and network computations

AN14 · MS59 David Gleich · Purdue 2

18 P. G. CONSTANTINE, D. F. GLEICH, Y. HOU, AND J. TEMPLETON

1

error

0

2

(a) Error, s = 0.39 cm

1

std

0

2

(b) Std, s = 0.39 cm

10

error

0

20

(c) Error, s = 1.95 cm

10

std

0

20

(d) Std, s = 1.95 cm

Fig. 4.5: Error in the reduce order model compared to the prediction standard de-viation for one realization of the bubble locations at the final time for two values ofthe bubble radius, s = 0.39 and s = 1.95 cm. (Colors are visible in the electronicversion.)

the varying conductivity fields took approximately twenty minutes to construct usingCubit after substantial optimizations.

Working with the simulation data involved a few pre- and post-processing steps:interpret 4TB of Exodus II files from Aria, globally transpose the data, compute theTSSVD, and compute predictions and errors. The preprocessing steps took approx-imately 8-15 hours. We collected precise timing information, but we do not reportit as these times are from a multi-tenant, unoptimized Hadoop cluster where otherjobs with sizes ranging between 100GB and 2TB of data sometimes ran concurrently.Also, during our computations, we observed failures in hard disk drives and issuescausing entire nodes to fail. Given that the cluster has 40 cores, there was at most2400 cpu-hours consumed via these calculations—compared to the 131,072 hours ittook to compute 4096 heat transfer simulations on Red Sky. Thus, evaluating theROM was about 50-times faster than computing a full simulation.

We used 20,000 reducers to convert the Exodus II simulation data. This choicedetermined how many map tasks each subsequent step utilized—around 33,000. Wealso found it advantageous to store matrices in blocks of about 16MB per record. Thereduction in the data enabled us to use a laptop to compute the coe�cients of theROM and apply to the far face for the UQ study in Section 4.4.

Here are a few pertinent challenges we encountered while performing this study.Generating 8192 meshes with di↵erent material properties and running independent

Tensor eigenvalues"and a power method

28

Tensor methods for network alignment

Network alignment is the problem of computing an approximate isomorphism between two net-works. In collaboration with Mohsen Bayati, Amin Saberi, Ying Wang, and Margot Gerritsen,the PI has developed a state of the art belief propagation method (Bayati et al., 2009).

FIGURE 6 – Previous workfrom the PI tackled net-work alignment with ma-trix methods for edgeoverlap:

i

j

j

0i

0

OverlapOverlap

A L B

This proposal is for match-ing triangles using tensormethods:

j

i

k

j

0

i

0

k

0

TriangleTriangle

A L B

If xi, xj , and xk areindicators associated withthe edges (i, i0), (j, j0), and(k, k0), then we want toinclude the product xixjxk

in the objective, yielding atensor problem.

We propose to study tensor methods to perform network alignmentwith triangle and other higher-order graph moment matching. Similarideas were proposed by Svab (2007); Chertok and Keller (2010) alsoproposed using triangles to aid in network alignment problems.In Bayati et al. (2011), we found that triangles were a key missingcomponent in a network alignment problem with a known solution.Given that preserving a triangle requires three edges between twographs, this yields a tensor problem:

maximizeX

i2L

wixi +X

i2L

X

j2L

xixjSi,j +X

i2L

X

j2L

X

k2L

xixjxkTi,j,k

| {z }triangle overlap term

subject to x is a matching.

Here, Ti,j,k = 1 when the edges corresponding to i, j, and k inL results in a triangle in the induced matching. Maximizing thisobjective is an intractable problem. We plan to investigate a heuris-tic based on a rank-1 approximation of the tensor T and usinga maximum-weight matching based rounding. Similar heuristicshave been useful in other matrix-based network alignment algo-rithms (Singh et al., 2007; Bayati et al., 2009). The work involvesenhancing the Symmetric-Shifted-Higher-Order Power Method due toKolda and Mayo (2011) to incredibly large and sparse tensors . On thisaspect, we plan to collaborate with Tamara G. Kolda. In an initialevaluation of this triangle matching on synthetic problems, using thetensor rank-1 approximation alone produced results that identifiedthe correct solution whereas all matrix approaches could not.

vision for the future

All of these projects fit into the PI’s vision for modernizing the matrix-computation paradigmto match the rapidly evolving space of network computations. This vision extends beyondthe scope of the current proposal. For example, the web is a huge network with over onetrillion unique URLs (Alpert and Hajaj, 2008), and search engines have indexed over 180billion of them (Cuil, 2009). Yet, why do we need to compute with the entire network?By way of analogy, note that we do not often solve partial di↵erential equations or modelmacro-scale physics by explicitly simulating the motion or interaction of elementary particles.We need something equivalent for the web and other large networks. Such investigations maytake many forms: network models, network geometry, or network model reduction. It is thevision of the PI that the language, algebra, and methodology of matrix computations will

11

maximize

Pijk

T

ijk

x

i

x

j

x

k

subject to kxk2

= 1

Human protein interaction networks 48,228 triangles Yeast protein interaction networks 257,978 triangles The tensor T has ~100,000,000,000 nonzeros

We work with it implicitly

where ! ensures the 2-norm

[x

(next)

]

i

= ⇢ · (

X

jk

T

ijk

x

j

x

k

+ �x

i

)

SSHOPM method due to "Kolda and Mayo

Big data methods SIMAX ‘09, SISC ‘11,MapReduce ‘11, ICASSP ’12

Network alignment ICDM ‘09, SC ‘11, TKDE ‘13

Fast & Scalable"Network centrality SC ‘05, WAW ‘07, SISC ‘10, WWW ’10, …

Data clustering WSDM ‘12, KDD ‘12, CIKM ’13 …

Ax = b

min kAx � bkAx = �x

Massive matrix "computations

on multi-threaded and distributed architectures

I hope to add power-grid networks soon!


Centrality measures “relative importance in a network” –Wikipedia “it’s a guess about what might be important” -Me They tell us something about a network considering it’s topology. They need to be deployed with extreme care!

AN14 · MS59 David Gleich · Purdue 4 From Wikipedia

Centrality measures of dynamic graphs Something about my network is changing, what should I do?

1.  Recompute at each change 2.  Batch up changes, and periodically recompute 3.  Efficiently update (i.e. recompute smartly!) 4.  Approximately update/compute 5.  Do something else.


What else to do??? “If the optimization is hard, you should be solving a different optimization problem” "–Cris Moore 1.  Des Higham et al. "

Adopt the fundamentals to discrete time 2.  Use dynamical system generalizations,

Gleich and Rossi 2012/2014; and "Des Higham et al. 2014

3.  Likely more too…


Smart centrality for the "smart grid?

You need to adapt your centrality measure for your application! (Or try to get lucky!)


Application to the power grid

Prior work •  Kim, Obah, 2007; Jin et al., 2010; Adolf et al., 2011; Halappanavar et

al., 2012

has found that graph properties have important correlations with power-grid vulnerabilities and contingency analysis


1.  Perspectives on PageRank

2.  PageRank as a dynamical system and time-dependent teleportation

3.  Predicting using PageRank

4.  Applications to the power-grid?


The random surfer model!At a node … 1.  follow edges with prob α 2.  do something else with prob (1-α)

Google’s PageRank is one possible answer PageRank by Google

1

2

3

4

5

6

The Model1. follow edges uniformly with

probability �, and2. randomly jump with probability

1� �, we’ll assume everywhere isequally likely

The places we find thesurfer most often are im-portant pages.

David F. Gleich (Sandia) PageRank intro Purdue 5 / 36

The important pages are the places we are most likely to find the random surfer

10

David Gleich · Purdue AN14 · MS59

My preferred version "of PageRank

A PageRank vector x is the solution of the linear system: (I – αP) x = (1 – α) v

where P is a column stochastic matrix, 0 ≤ α < 1, and v is a probability vector. PageRank details

1

2

3

4

5

6

!

2664

1/6 1/2 0 0 0 01/6 0 0 1/3 0 01/6 1/2 0 1/3 0 01/6 0 1/2 0 0 01/6 0 1/2 1/3 0 11/6 0 0 0 1 0

3775

| {z }P

P�j�0eTP=eT

“jump” ! v = [ 1n ... 1

n ]T ��0

eTv=1

Markov chainî�P+ (1� �)veT

óx = x

unique x ) �j � 0, eTx = 1.

Linear system (�� P)x = (1� �)vIgnored dangling nodes patched back to v

algorithms laterDavid F. Gleich (Sandia) PageRank intro Purdue 6 / 36

Just three ingredients!

vi � 0, eT v = 1

↵ usually 0.5 to 0.99

11


This definition applies to a remarkable variety of problems 1.  GeneRank 2.  ProteinRank 3.  FoodRank 4.  SportsRank 5.  HostRank 6.  TrustRank 7.  BadRank 8.  ObjectRank 9.  ItemRank 10.  ArticleRank 11.  BookRank 12.  FutureRank

13.  TimedPageRank 14.  SocialPageRank 15.  DiffusionRank 16.  ImpressionRank 17.  TweetRank 18.  TwitterRank 19.  ReversePageRank 20.  PageTrust 21.  PopRank 22.  CiteRank 23.  FactRank 24.  InvestorRank

25.  ImageRank 26.  VisualRank 27.  QueryRank 28.  BookmarkRank 29.  StoryRank 30.  PerturbationRank 31.  ChemicalRank 32.  RoadRank 33.  PaperRank 34.  Etc…

12


The teleportation distribution v models where surfers “restart” What if this changes with time?

13


Let’s look at how PageRank evolves with iterations

�x

(k ) = x

(k+1) � x

(k )

= ↵Px

(k ) + (1 � ↵)v � x

(k )

= (1 � ↵)v � (I � ↵P)x(k )

x

0(t) = (1 � ↵)v � (I � ↵P)x(t)

PageRank is the steady-state solution of the ODE

14


A dynamical system for "time-dependent teleportation

+ Easy to integrate + Easy to understand + Possible to treat analytically! – Need to “model time” (not dimensionless) – Still useful to have a data assimilation model

x

0(t) = (1 � ↵)v(t) � (I � ↵P)x(t)

15


Need a symplectic integrator (or self-correcting…) We use a standard RK integrator "(ode45 in Matlab) We used the formulation to maintain x(t) as a probability distribution

x

0(t) = (1 � ↵)v(t) � (�I � ↵P)x(t)

� = (1 � ↵)eTv(t) + ↵e

Tx(t)

16


Where is this model realistic?

On Wikipedia, we have hourly visit data that provides a coarse measure of outside interest

17


Now PageRank values are time-series, not static scores

1 MainPage 2 FrancisMag 3 Search 4 Pricewater 5 UnitedStat 6 Protectedr 7 administra 8 Wikipedia 9 Glycoprote 10 Duckworth−

11 501(c) 12 Searching 13 Contents 14 Politics 15 Non−profit 16 Science 17 History 18 Society 19 Technology 20 Geography

21 Maintopicc 22 Featuredco 23 administra 24 Contents/Q 25 Freeconten 26 Encycloped 27 AmericanId 28 UnitedKing 29 Mathematic 30 Biography

31 Arts 32 AmericanId 33 Englishlan 34 adminship 35 Fundamenta 36 England 37 Watchmen 38 featuredco 39 Watchmen(f 40 Earthquake

41 India 42 Sciencepor 43 Redirects 44 Articles 45 Wikipedia 46 protectedp 47 QuestCrew 48 Wiki 49 Associatio 50 Raceandeth

51 Greygoo 52 pageprotec 53 Rihanna 54 Listofbasi 55 Sciencepor 56 KaraDioGua 57 TheBeatles 58 Technology 59 London 60 Football(s

61 Science 62 Gackt 63 Teleprompt 64 Technology 65 Society 66 Outlineofs 67 ER(TVserie 68 Philippine 69 NewYorkCit 70 Australia

71 Madonna(en 72 Richtermag 73 Tobaccoadv 74 Geography 75 California 76 Constantin 77 RobKnox 78 LosAngeles 79 Canada 80 MurderofEv

81 Livingpeop 82 Mathematic 83 Societypor 84 functionar 85 March6 86 Day26 87 Skittles(c 88 EveCarson 89 Redirectsf 90 U2

91 Categories 92 Germany 93 MediaWiki 94 Rorschach( 95 EatBulaga! 96 PaulaAbdul 97 Daylightsa 98 NewYork 99 Characters 100 Scotland

Earthquake

Australian Earthquake

occurs!

Main page

Time Time

Impo

rtanc

e

18


Some quick theory

x(t) = exp[�(I � ↵P)t ]x(0)

+ (1 � ↵)

Z t

0

exp[�(I � ↵P)(t � ⌧ )]v(⌧ ) d⌧ .

x

0(t) = (1 � ↵)v(t) � (I � ↵P)x(t)

Z t

0

exp[�(I � ↵P)(t � ⌧ )]v(⌧ ) d⌧

= (I � ↵P)

�1v � exp[�(I � ↵P)t ](I � ↵P)

�1v

x(t) = exp[�(I � ↵P)t ](x(0) � x) + x

For general v(t)

For static v(t) = v

The original "PageRank vector

19


Thus we recover "the original PageRank vector "if interest stops changing.

20


Modeling cyclical behavior

Cyclically switch between teleportation vectors vj

v(t) =

1

k

kX

j=1

vj

⇣cos(t + (j � 1)

2⇡k ) + 1

⌘

0 20 40 60 800

0.05

0.1

0.15

0.2

time

Tim

e−de

pend

ent t

elep

orta

tion

Page 1Page 2Page 3Page 4

v1 v2 v1 v2

21


0 5 10 15 200.1

0.2

0.3

0.4

0.5

time

Dyn

amic

Pag

eRan

k


Cyclical behavior in the time-dependent PageRank scores

1

2

3

4

0 20 40 60 800

0.05

0.1

0.15

0.2

time

Tim

e−de

pend

ent t

elep

orta

tion


22


Modeling cyclical behavior

Cyclically switch between teleportation vectors vj

v(t) =

1

k

kX

j=1

vj

⇣cos(t + (j � 1)

2⇡k ) + 1

⌘

x(t) = x + Re {s exp(ıt)}Then the eventual solution is

(I � ↵P)x = (1 � ↵)1k

Ve

(I � ↵1+ı P)s

= (1 � ↵)

1

k (1+ı) V exp(ıf)PageRank vector with average teleportation

PageRank with complex teleportation

23


Summary

If you have cyclical interest on a node, we have a NEW centrality measure that provides the magnitude of the oscillation based on PageRank with complex valued “teleportation.”


Thus we can determine "the size of the oscillation "for the case of cyclical teleportation

25


Is it useful? Let’s try and predict retweets on Twitter

We crawled Twitter and gathered "a graph of who follows who and "how active each user is in a month This yields a graph and 6 vectors v!!Our goal is to predict how many tweets you’ll send next month based on the current month!

26


… and then there are details I can go into …


The results

Dataset Type ✓ Error Ratio

s (timescale)

1 2 6 1TWITTER stationary 0.01 0.635 0.929 0.913 0.996

0.50 0.636 0.735 0.854 0.939

1.00 0.522 0.562 0.710 0.963

non-stationary 0.01 0.461 0.841 1.001 0.992

0.50 0.261 0.608 0.585 0.929

1.00 0.137 0.605 0.617 0.918

Err Ratio = SMAPE of tweets + Time-dependent PR / SMAPE of tweets only If this ratio < 1, then using Time-dependent PR helps Stationary nodes are those with small maximum change in scores Non-stationary nodes are those with large maximum change in scores

28


Using Granger Causality to study link relationships on Wikipedia





















Earthquake Richter Mag.

Causes?

Of course! We build this into the model. But, the question is, which of these are preserved after incorporating the effects of page view data?

29


To the power grid …

Line failures in the grid can be anticipated via linearized DC dynamics Hines el al.?


c = diag(B (L)+ BT )

The PageRank problem & "the Laplacian

Combinatorial "Laplacian


1. (I � ↵AD

�1)x = (1 � ↵)v;

2. (I � ↵A)y = (1 � ↵)D�1/2v,

where A = D

�1/2AD

�1/2 and x = D

1/2y; and

3. [�D + L]z = �v where ↵ = 1/(1 + �) and x = Dz.

Let x(↵) solve PageRank and

let v

T

e = 0.

Then lim↵!1

x(↵) ! SL

+

v

where S is a scaling matrix.

Some potential applications

1.  PageRank can be thought of as a type of regularization; often helps improve on simple centrality baselines

2.  Limits of PageRank interpolate between centrality and spectral clustering [Mahoney, Orecchia, and Vishnoi]

3.  Time dependent teleportation models; adaptations to node dropouts possible.

4.  Use PageRank on the line graph?


Results on the power grid

… pending …


Questions, Conclusions, and References!

Questions!How to validate some of these ideas? Too simplistic? Other power-grid problems where similar ideas may be able to help? Collaborators?????

34


Dear David, Please remember to repeat the question!

Paper Gleich & Rossi, Internet Mathematics, 2014 Code https://www.cs.purdue.edu/homes/dgleich/codes/dynsyspr-im

Conclusions!Centrality is more complicated than just one method. It’s possible to tune centrality measures to different structures and this makes it a flexible setup."

pagerank centrality of dynamic graph structures

Technology

pxt pagerank

vwhere p

network alignmenticdm

network wikipediaits

smart centrality

powergrid vulnerabilities

powergrid networks

p ijk tijk