meta data analysis
TRANSCRIPT
Ranking on Large-Scale Graphs with Rich Metadata
Bin Gao, Taifeng Wang, and Tie-Yan Liu
Microsoft Research Asia
WWW 2011 Tutorial 1
Presenters
WWW 2011 Tutorial 2
Bin Gao Researcher, MSR Asia
http://research.microsoft.com/en-us/people/bingao/
Taifeng Wang Researcher, MSR Asia
http://research.microsoft.com/en-us/people/taifengw/
Tie-Yan Liu Lead Researcher, MSR Asia
http://research.microsoft.com/users/tyliu/
Graph
3 WWW 2011 Tutorial
Everything in the world is connected. There is graph where there is connection.
Large-scale Graph
Social graphs
โข Messenger, Facebook, Twitter, Entity Cube, etc.
Endorsement graphs
โข Web link graph, Paper citation graph, etc.
Location graphs
โข Map, Power grid, Telephone network, etc.
Co-occurrence graphs
โข Term-document bipartite, Click-through bipartite, etc.
WWW 2011 Tutorial 4
How Large Are These Graphs?
โข Web Link Graph โ Tens of billions of nodes indexed and Over one trillion
nodes discovered by major search engines
โข Facebook โ About 600 million nodes (14-Jan-2011)
โข China Telephone Networks โ 1.1 billion nodes (0.8 billion mobile + 0.3 billion land line)
(22-Jul-2010)
โข Click-through Bipartite โ Several billion queries and tens of billion URLs (recent
research papers)
5 WWW 2011 Tutorial
Properties of Real Large-scale graphs
โข Large-scale, of course
โข Very sparse
โข Rich information on nodes and edges
โข External knowledge on the graphs
6 WWW 2011 Tutorial
Rich Information on Node & Edge
โข Web Link graph โ Node: page length, creation time, etc. โ Edge: number of links, inter/intra-site link, etc.
โข Facebook โ Node: age, gender, interests, etc. โ Edge: creation time, communication frequency, etc.
โข China Telephone Networks โ Node: service category, customer profile, etc. โ Edge: communication frequency, bandwidth, types of calls, etc.
โข Click-through Bipartite โ Node: query frequency, language, page length, page
importance, dwell time, etc. โ Edge: click frequency, time of click, etc.
7 WWW 2011 Tutorial
External Knowledge
โข Point-wise
โ Entity A is popular.
โ Entity B is a spam.
โข Pair-wise
โ Entity A is more important than entity B.
โข List-wise
โ We have A > B > C, according to the user feedback on these entities.
* Here entity can be website, people, phone subscriber, query, etc.
8 WWW 2011 Tutorial
Ranking on Large-scale Graph
โข Problem definition
โ Given a large-scale directed graph and its rich metadata, calculate the ranking of the nodes in the graph according to their importance, popularity, or preference.
โข Application
โ Webpage ranking
โ Paper ranking
โ Entity ranking in social networks
9 WWW 2011 Tutorial
Example: Web Page Ranking
โข Factors to consider
โ The quality of the web page
โ The visit frequency by users
โ Userโs dwell time
โ The mutual endorsement between pages
โ โฆ
WWW 2011 Tutorial 10
Example: Paper Ranking
โข Factors to consider
โ Citation
โ Authors
โ Publication venue
โ Awards
โ Publication date
โ โฆ
WWW 2011 Tutorial 11
Example: Social Entity Ranking
โข Factors to consider
โ Account creation time
โ Account activity
โ Friends related information
โ Liked or followed by others
โ โฆ โฆ
WWW 2011 Tutorial 12
Key Questions to Answer
1. How to perform graph ranking based on graph structure?
2. How to leverage node and edge features for better graph ranking?
3. How to incorporate external knowledge in graph ranking?
4. How to implement large-scale graph ranking algorithms?
13 WWW 2011 Tutorial
Scope of the Tutorial
โข Node ranking on graphs
โ But not ranking of a number of graphs
โ But not retrieval and ranking problems for subgraphs
โข Mainly based on papers at WWW, SIGIR, KDD, ICML
โ Papers at other conferences and journals might not be well covered
โ Not necessarily a comprehensive review of the literature
โ Your are welcome to contribute by sharing and discussing with us and our audience
14 WWW 2011 Tutorial
Background Knowledge Required
โข Information Retrieval
โข Machine Learning
โข Linear Algebra
โข Probability Theory
โข Optimization
15
We assume that you are familiar with these fields, and we will not give comprehensive introduction to them in this tutorial.
WWW 2011 Tutorial
Notations
โข Graph ๐บ(๐, ๐ธ, ๐, ๐) ๐ = ๐ฃ๐ : node set, ๐ = ๐
๐ธ = ๐๐๐ : edge set, ๐ธ = ๐
๐ = ๐ฅ๐๐ : edge features, ๐ฅ๐๐ = ๐, ๐ฅ๐๐ = (๐ฅ๐๐1, ๐ฅ๐๐2, โฆ , ๐ฅ๐๐๐)๐
๐ = ๐ฆ๐ : node features, ๐ฆ๐ = ๐, ๐ฆ๐ = (๐ฆ๐1, ๐ฆ๐2, โฆ , ๐ฆ๐โ)๐
โข Matrices ๐: adjacency matrix or link matrix
๐: transition probability matrix
โข Rank score vectors ๐: authority score vector
๐: hub score vector
๐: general rank score vector
16 WWW 2011 Tutorial
Outline
I. Overview
II. Graph Ranking by Link Analysis
III. Graph Ranking with Node and Edge Features
IV. Graph Ranking with Supervision
V. Implementation for Graph Ranking
VI. Summary
17 WWW 2011 Tutorial
Link Analysis for Ranking
โข Only consider link structure, no metadata involved.
โข A link from page ๐ฃ๐ to page ๐ฃ๐ may indicate:
โ ๐ฃ๐ is related to ๐ฃ๐
โ ๐ฃ๐ is recommending, citing, voting for, or endorsing ๐ฃ๐
18 WWW 2011 Tutorial
๐บ(๐, ๐ธ)
Famous Link Analysis Algorithms
โข HITS [Kleinberg, 1997]
โข PageRank [Page et al, 1998]
19 WWW 2011 Tutorial
HITS โ Hypertext Induced Topic Selection
โข For each vertex ๐ฃ๐ in a subgraph of interest
โ ๐(๐ฃ๐) - the authority of ๐ฃ๐
โ ๐(๐ฃ๐) - the hub of ๐ฃ๐
โข Authority โ A site is very authoritative if it receives many citations.
Citation from important sites weights more than citations from less-important sites.
โข Hub โ Hub shows the importance of a site. A good hub is a site
that links to many authoritative sites.
20 WWW 2011 Tutorial
Kleinberg. Authoritative sources in a hyperlinked environment. IBM Research Report RJ 10076, 1997.
Authority and Hub
21
๐ฃ1
๐ฃ2
๐ฃ3
๐ฃ4
๐ฃ5
๐ฃ6
๐ฃ7
๐ฃ8
๐ ๐ฃ1 = ๐ ๐ฃ2 + ๐ ๐ฃ3 + ๐(๐ฃ4) ๐ ๐ฃ1 = ๐ ๐ฃ5 + ๐ ๐ฃ6 + ๐(๐ฃ7) +๐(๐ฃ8)
WWW 2011 Tutorial
Convergence of HITS
โข Recursive dependency
โข Iterative algorithm
โข Using linear algebra, it is easy to prove that ๐(๐ฃ๐) and ๐(๐ฃ๐) converge.
22
๐ ๐ฃ๐ = ๐ ๐ฃ๐๐ฃ๐โ๐๐๐๐๐๐,๐ฃ๐-
๐ ๐ฃ๐ = ๐ ๐ฃ๐๐ฃ๐โ๐๐ข๐ก๐๐๐๐,๐ฃ๐-
WWW 2011 Tutorial
๐(๐:1) ๐ฃ๐ = ๐(๐) ๐ฃ๐๐ฃ๐โ๐๐๐๐๐๐,๐ฃ๐-
๐(๐:1) ๐ฃ๐ = ๐(๐:1) ๐ฃ๐๐ฃ๐โ๐๐ข๐ก๐๐๐๐,๐ฃ๐-
๐(๐:1) ๐ฃ๐ โต๐(๐:1) ๐ฃ๐
๐(๐:1) ๐ฃ๐๐
๐(๐:1) ๐ฃ๐ โต๐(๐:1) ๐ฃ๐
๐(๐:1) ๐ฃ๐๐
Convergence of HITS
โข The authority and hub values calculated by HITS is the left and right singular vectors of the adjacency matrix of the base subgraph.
23
where ๐ = ๐ ๐ฃ1 , ๐ ๐ฃ2 , โฆ , ๐ ๐ฃ๐๐
, ๐ = ๐ ๐ฃ1 , ๐ ๐ฃ2 , โฆ , ๐ ๐ฃ๐๐
๐ = ๐๐๐๐ ๐ = ๐๐๐๐
๐ = ๐๐๐ ๐ = ๐๐
WWW 2011 Tutorial
An Example
โข Start with a root set ๐ = ๐ฃ1, ๐ฃ2, ๐ฃ3, ๐ฃ4 by nodes relevant to the topic.
โข Generate a new set ๐ (base subgraph) by expanding ๐ to include all the children and a fixed number of parents of nodes in ๐ .
24
๐ฃ1
๐ฃ2
๐ฃ3
๐ฃ4
๐ฃ5
๐ฃ6
๐ฃ7
๐ฃ8
๐ฃ9
๐ฃ10
๐ฃ11
๐ฃ12
๐
๐
WWW 2011 Tutorial
HITS of the Example
25 WWW 2011 Tutorial
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
1 2 3 4 5 6 7 8 9 10 11 12
Authority
Hub
PageRank
โข An interesting name! โ The rank of a page or the rank defined by Mr.
Page?
โข The page rank is proportional to its parentsโ rank, but inversely proportional to its parentsโ out-degrees.
26
๐(๐ฃ๐) = ๐(๐ฃ๐)
|๐๐ข๐ก๐๐๐๐,๐ฃ๐-|๐ฃ๐โ๐๐๐๐๐๐,๐ฃ๐-
30 80
30 60
20
20
20
20
30
50
WWW 2011 Tutorial
Page et al. The PageRank citation ranking: bringing order to the Web. Stanford Digital Library Technologies Project, 1998 .
๐ฃ1
๐ฃ2
๐ฃ3
๐ฃ4
Markov Chain Explanation
โข PageRank as a Random Surfer Model
โ Description of a random walk through the Web graph
โ Interpreted as a transition matrix with asymptotic probability that a surfer is currently browsing that page
โ Discrete-time Markov model
27 WWW 2011 Tutorial
An Example
28
๐ฃ1 ๐ฃ2
๐ฃ3
๐ฃ4
๐ฃ8
๐ฃ6 ๐ฃ7
๐ฃ5
Node Outlinks
๐ฃ1 ๐ฃ2, ๐ฃ3, ๐ฃ6
๐ฃ2 ๐ฃ4, ๐ฃ5
๐ฃ3 ๐ฃ4, ๐ฃ6
๐ฃ4 ๐ฃ6
๐ฃ5 ๐ฃ1, ๐ฃ4, ๐ฃ7, ๐ฃ8
๐ฃ6 ๐ฃ4
๐ฃ7 ๐ฃ3
๐ฃ8 ๐ฃ4, ๐ฃ7
WWW 2011 Tutorial
Matrix Representation
29
Adjacent Matrix
๐ =
0 1 1 0 0 1 0 00 0 0 1 1 0 0 00 0 0 1 0 1 0 00 0 0 0 0 1 0 01 0 0 1 0 0 1 10 0 0 1 0 0 0 00 0 1 0 0 0 0 0 0 0 0 1 0 0 1 0
๐ฃ1 ๐ฃ2
๐ฃ3
๐ฃ4
๐ฃ8
๐ฃ6 ๐ฃ7
๐ฃ5
WWW 2011 Tutorial
Matrix Representation
30
โข Transition Probability Matrix ๐ = ๐๐๐
๐๐๐ =
๐(๐, ๐)
๐(๐, ๐)๐ฃ๐โ๐๐ข๐ก๐๐๐๐,๐ฃ๐-
, ๐๐ข๐ก๐๐๐๐,๐ฃ๐- โ 0
๐(๐, ๐) = 0, ๐๐ก๐๐๐๐ค๐๐ ๐
๐ =
0 1/30 0
1/3 00 1/2
0 0 0 0
0 1/20 0
0 1/31/2 0
0 0 0 0
0 1/20 1
0 0 0 0
1/4 0 0 0
0 1/40 1
0 0 0 0
1 00 1/2
0 0 0 0
1/4 1/40 0
0 0 0 0
0 0 1/2 0
WWW 2011 Tutorial
PageRank of the Example
31
๐ฃ1 ๐ฃ2
๐ฃ3
๐ฃ4
๐ฃ8
๐ฃ6 ๐ฃ7
๐ฃ5
ID PR Inlink Outlink
1 0.0250 ๐ฃ5 ๐ฃ2, ๐ฃ3, ๐ฃ6
2 0.0259 ๐ฃ1 ๐ฃ4, ๐ฃ5
3 0.0562 ๐ฃ1, ๐ฃ7 ๐ฃ4, ๐ฃ6
4 0.4068 ๐ฃ2, ๐ฃ3, ๐ฃ5, ๐ฃ6, ๐ฃ8 ๐ฃ6
5 0.0298 ๐ฃ2 ๐ฃ1, ๐ฃ4, ๐ฃ7, ๐ฃ8
6 0.3955 ๐ฃ1, ๐ฃ3, ๐ฃ4 ๐ฃ4
7 0.0357 ๐ฃ5, ๐ฃ8 ๐ฃ3
8 0.0251 ๐ฃ5 ๐ฃ4, ๐ฃ7
WWW 2011 Tutorial
Rank Sink
โข Many Web pages have no inlinks/outlinks
โข It results in dangling edges in the graph
โ No inlink
โ No outlink
32
๐ฃ1 ๐ฃ2
๐ฃ3 ๐ฃ4
๐ฃ1 ๐ฃ2
๐ฃ3 ๐ฃ4
WWW 2011 Tutorial
Modification โ Transition Matrix
โข Surfer will restart browsing by picking a new Web page at random
๐ โ ๐ + ๐ธ
๐: stochastic matrix
33
๐ธ๐๐ = 0, ๐๐ ๐๐ข๐ก๐๐๐๐ ๐ฃ๐ > 01
๐, ๐๐ก๐๐๐๐ค๐๐ ๐
WWW 2011 Tutorial
Remark: ๐ = ๐๐๐ for simplicity
Further Modification โ Damping Factor
34
Stationary Distribution
Teleport Vector
Transition Matrix
๐ = ๐ผ๐๐๐ + 1 โ ๐ผ1
๐๐ , ๐ = 1,1, โฆ , 1 ๐
Damping Factor
WWW 2011 Tutorial
Other Link Analysis Algorithms
35
โข Following the success of PageRank and HITS, a lot of new algorithms were proposed.
โ Block-level PageRank
โ HostRank
โ โฆโฆ
WWW 2011 Tutorial
Block-Level PageRank
โข Web page can be divided into different vision-based segmentation (block)
36 WWW 2011 Tutorial
Block-level PageRank
โข Block-to-page matrix ๐ โ ๐ ๐: number of pages the block links to
โข Page-to-block matrix ๐
37
๐๐๐ = ๐๐ฃ๐(๐๐), ๐๐ โ ๐ฃ๐0, ๐๐ โ ๐ฃ๐
๐๐ฃ๐ ๐๐ = ฮฒ๐๐๐ง๐ ๐๐ ๐๐๐๐๐ ๐๐ ๐๐ ๐๐๐๐ ๐ฃ๐
๐ท๐๐ ๐ก๐๐๐๐ ๐๐๐๐ ๐ก๐๐ ๐๐๐๐ก๐๐ ๐๐ ๐๐ ๐ก๐ ๐ก๐๐ ๐๐๐๐ก๐๐ ๐๐ ๐ ๐๐๐๐๐
๐๐๐ =
1
๐ ๐, ๐๐ ๐ก๐๐๐๐ ๐๐ ๐ ๐๐๐๐ ๐๐๐๐ ๐๐๐๐๐ ๐๐ ๐ก๐ ๐๐๐๐ ๐ฃ๐
0, ๐๐ก๐๐๐๐ค๐๐ ๐
WWW 2011 Tutorial Cai et al. Block-level link analysis. SIGIR, 2004.
Block-level PageRank (cont.)
โข A weight matrix can be defined as ๐๐
โข A probability transition matrix ๐(๐) can be constructed by renormalizing each row of ๐๐ to sum to 1.
โข Block-level PageRank can be computed as
38
๐ = ๐ผ(๐(๐))๐๐ + 1 โ ๐ผ1
๐๐
WWW 2011 Tutorial
HostRank
WWW 2011 Tutorial 39
๐ฃ1
๐ฃ2
๐ฃ3
๐ฃ5
๐ฃ4
Upper-level Graph
Lower-level Graph
Aggregation
Super Node Super Edge
Xue et al. Exploiting the Hierarchical Structure for Link Analysis. SIGIR, 2004.
โข The Web graph has a hierarchical structure.
HostRank
โข Construct two-layer hierarchical graph
โ ๐ = ๐1, ๐2, โฆ , ๐๐ is a partition on the vertex set ๐of graph ๐บ(๐, ๐ธ)
โ Upper-layer graph contains ๐ vertices called supernodes, one for each element of the partition
โ Lower-layer graph organizes all the pages in one supernode by the node relationship.
40 WWW 2011 Tutorial
HostRank (cont.)
โข Calculate supernode importance ๐(๐ผ)
โข Calculate page importance ๐
41
๐(๐ผ) = ๐ผ(๐(๐ผ))๐๐(๐ผ) + 1 โ ๐ผ1
๐๐
๐๐๐๐๐ฅ(๐ฃ๐) = 1, ๐๐ ๐ฃ๐ ๐๐ ๐๐ ๐๐๐๐๐ฅ ๐๐๐๐
๐ฟ, ๐๐ก๐๐๐๐ค๐๐ ๐
๐๐๐๐(๐ฃ๐) = ๐ฝ๐๐ผ๐ฟ(๐ฃ๐)
๐๐ผ๐ฟ(๐ฃ๐)๐ฃ๐โ๐๐
+ (1 โ ๐ฝ)๐ผ๐ผ๐ฟ(๐ฃ๐)
๐ผ๐ผ๐ฟ(๐ฃ๐)๐ฃ๐โ๐๐
๐ค๐ = ๐ ๐๐๐๐๐ฅ(๐ฃ๐) + (1 โ ๐)๐๐๐๐(๐ฃ๐)
๐ค๐๐ = ๐พ๐ค๐
๐ฃ๐โ*๐๐๐๐๐ ๐๐๐๐ ๐ฃ๐ ๐ก๐ ๐๐๐๐ก+
๐๐ = ๐(๐ผ)๐๐ค๐๐
WWW 2011 Tutorial
๐ผ๐ผ๐ฟ(๐ฃ๐): Number of intra-link to ๐ฃ๐
๐๐ผ๐ฟ(๐ฃ๐): Number of inter-link to ๐ฃ๐
Summary
โข Link analysis is a key technology in Web search
โ Link analysis algorithms like PageRank have achieved great success and contribute significantly to todayโs search engines.
โข Link analysis technologies also have limitations
โ They only use the structure of the graph, while many other informative factors are ignored, such as user clicks and content information.
42 WWW 2011 Tutorial
Beyond Link Analysis
โข More metadata besides link structure
โ Information on nodes and edges
โ Supervision information for the ranking order
43 WWW 2011 Tutorial
Outline
I. Overview
II. Graph Ranking by Link Analysis
III. Graph Ranking with Node and Edge Features
IV. Graph Ranking with Supervision
V. Implementation for Graph Ranking
VI. Summary
44 WWW 2011 Tutorial
Beyond Link Graph
โข In conventional link analysis, link graph is simply represented by a binary adjacency matrix.
โข In practice, we have rich metadata associated with the nodes and edges, and thus the representation of the graph can be more complex.
45 WWW 2011 Tutorial
Examples
โข Non-uniform Teleport Vector โ Node metadata: bias to some nodes
โข Weighted Link Graph โ Edge metadata: number of links from one node to
another
โข User Browsing Graph โ Node metadata: user staying time on each node;
frequency of user visits on each node.
โ Edge metadata: number of user transition from one page to another.
46 WWW 2011 Tutorial
Personalized PageRank
47
Personalized Teleport Vector
๐ = ๐ผ๐๐๐ + 1 โ ๐ผ ๐
โข Change 1
๐๐ with ๐
โข Instead of teleporting uniformly to any page, we bias the jump on some pages over others
โ E.g., ๐๐ is 1 for your homepage and 0 otherwise.
โ E.g., ๐๐ prefers the topics you are interested in.
WWW 2011 Tutorial
Haveliwala et al. An analytical comparison of approaches to personalizing PageRank. Stanford University Technical Report, 2003
Examples
48 WWW 2011 Tutorial
ID ๐ ๐
1 0.0250 0.125
2 0.0259 0.125
3 0.0562 0.125
4 0.4068 0.125
5 0.0298 0.125
6 0.3955 0.125
7 0.0357 0.125
8 0.0251 0.125
ID ๐ ๐
1 0.1024 0.65
2 0.0365 0.05
3 0.0515 0.05
4 0.3774 0.05
5 0.0230 0.05
6 0.3792 0.05
7 0.0177 0.05
8 0.0124 0.05
ID ๐ ๐
1 0.0100 0.05
2 0.0103 0.05
3 0.0225 0.05
4 0.4384 0.05
5 0.0119 0.05
6 0.4825 0.65
7 0.0143 0.05
8 0.0100 0.05
uniform vector bias on ๐ฃ1 bias on ๐ฃ6
Personalized PageRank
49 WWW 2011 Tutorial
๐ = ๐ผ๐๐๐ + 1 โ ๐ผ ๐
๐ = 1 โ ๐ผ (๐ผ โ ๐ผ๐๐);1๐
Personalized Teleport Vector Fixed Matrix
Topic-Sensitive PageRank
โข Instead of using one single PageRank value to represent the importance of Web page, calculate a vector of PageRank values, according to 16 topics in ODP.
โข For each value in this vector, when making the PageRank metric primitive, use different transition matrix (only randomly jump to those pages of the given topic).
50 WWW 2011 Tutorial Haveliwala et al. Topic-sensitive PageRank. WWW, 2002 .
Topic-Sensitive PageRank (cont.)
โข Category biasing
โ ๐๐: set of pages in category ๐๐
โ ๐(๐): teleport vector
51
๐๐(๐) =
1
๐๐, ๐ โ ๐๐
0, ๐ โ ๐๐
๐(๐)
WWW 2011 Tutorial
Topic-Sensitive PageRank (cont.)
โข Query-time importance score
โ ๐: query or query context
52
๐๐ = ๐(๐๐|๐)๐๐(๐)
๐
๐ ๐๐ ๐ =๐(๐๐)๐(๐|๐๐)
๐(๐)
WWW 2011 Tutorial
Weighted Link Graph
53
๐ =
0 1 1 0 0 1 0 00 0 0 1 1 0 0 00 0 0 1 0 1 0 00 0 0 0 0 1 0 01 0 0 1 0 0 1 10 0 0 1 0 0 0 00 0 1 0 0 0 0 0 0 0 0 1 0 0 1 0
๐ =
0 2 1 0 0 3 0 00 0 0 1 2 0 0 00 0 0 4 0 1 0 00 0 0 0 0 3 0 02 0 0 3 0 0 5 10 0 0 1 0 0 0 00 0 4 0 0 0 0 0 0 0 0 2 0 0 1 0
Adjacent Matrix Weighted Adjacent Matrix
WWW 2011 Tutorial
Weighted PageRank
54 WWW 2011 Tutorial
ID ๐ ๐
1 0.0250 0.125
2 0.0259 0.125
3 0.0562 0.125
4 0.4068 0.125
5 0.0298 0.125
6 0.3955 0.125
7 0.0357 0.125
8 0.0251 0.125
ID ๐ ๐
1 0.0239 0.125
2 0.0255 0.125
3 0.0541 0.125
4 0.4142 0.125
5 0.0332 0.125
6 0.3902 0.125
7 0.0376 0.125
8 0.0213 0.125
Un-weighted Weighted
๐ =
0 2 1 0 0 3 0 00 0 0 1 2 0 0 00 0 0 4 0 1 0 00 0 0 0 0 3 0 02 0 0 3 0 0 5 10 0 0 1 0 0 0 00 0 4 0 0 0 0 0 0 0 0 2 0 0 1 0
User Browsing Graph
55
User Browsing Behaviors
Node feature โข User staying time on nodes
โข High quality pages attract longer reading time โข Spam and junk pages will be closed right after loaded โข Collected from user browsing behaviors
โข Non-uniform teleport vector โข Green traffic
Edge feature โข User transition along edges
WWW 2011 Tutorial
BrowseRank
โข Computed from User Browsing Graph
โข Random surfer model
โ Start from a page selected from the distribution of โgreen trafficโ
โ Stay a period of time
โ Jump to next page by weighted adjacent matrix
โข Challenge
โ Discrete-time Markov model does not work here โข Cannot model the non-unit staying time
56 WWW 2011 Tutorial
Liu et al. BrowseRank: Letting Web users vote for page importance. SIGIR, 2008.
Continuous-time Markov Model
โข Model the real user browsing behavior on the Web as a continuous-time Markov process on the user browsing graph
x1 x2 x3 xฯ โฆ
unit time unit time unit time unit time
PageRank
x1 x2 x3 xฯ โฆ โฆ
y1 y2 y3 yฯ
BrowseRank
Discrete-time Markov Process
Continuous-time Markov Process
57 WWW 2011 Tutorial
Stationary Distribution
โข Calculate the page importance as the stationary probability distribution of such stochastic process
58
PageRank
BrowseRank
๐ = ๐๐๐
๐ = ๐(๐ก)๐๐, โ๐ก > 0
Hard to compute
WWW 2011 Tutorial
Decomposition
59
๐ = ๐(๐ก)๐๐, โ๐ก > 0 ๐(๐ก)
Calculating ๐
๐๐ =๐ ๐ ๐๐
๐ ๐ ๐๐ ๐๐=1
Computing the stationary distribution ๐ = ๐ ๐ , ๐ = 1, โฆ , ๐ of a discrete-time Markov chain
(called embedded Markov chain)
Estimating staying time distribution
1 โ ๐;๐๐๐ก
WWW 2011 Tutorial
Staying Time Calculation
60
๐น๐๐(๐ก) = 1 โ ๐;๐๐๐ก
โข ๐๐: random variable of staying time on ๐ฃ๐
โข ๐น๐๐(๐ก): cumulative probability distribution of random variable ๐๐
๐ ๐ = ๐ธ ๐๐ = ๐ก๐น๐๐ ๐ก ๐๐ก
โ
0
=1
๐๐
WWW 2011 Tutorial
Discussions on Staying Time
โข Staying time might not only depend on the current page; It also depends on the source page from which users transits to the current page.
61
x1 x2 x3 xฯ โฆ โฆ
y1 y2 y3 yฯ
Mirror Semi-Markov Process
WWW 2011 Tutorial
From BrowseRank to BrowseRank Plus
62 WWW 2011 Tutorial
๐ฃ1
๐ฃ2
๐ฃ3
๐ฃ4
๐ฃ1
๐ฃ2
๐ฃ3
๐ฃ4
10 s 2 s 78 s 39 s 101 s 25 s
101 s 25 s
39 s
10 s 2 s 78 s
BrowseRank Plus
63
โข ๐๐: random variable of staying time on ๐ฃ๐
โข ๐น๐๐๐ (๐ก): cumulative probability distribution of random on ๐ฃ๐
from ๐ฃ๐
โข ๐๐๐: contribution probability of ๐ฃ๐ to ๐ฃ๐
๐น๐๐๐ (๐ก) = 1 โ ๐;๐๐๐๐ก
๐ ๐ = ๐ธ ๐๐ = ๐๐๐ ๐ก ร ๐น๐๐๐ (๐ก)๐๐ก
โ
0๐
= ๐๐๐
๐๐๐๐
๐๐๐ =๐๐๐๐ ๐
๐ ๐
WWW 2011 Tutorial
Gao et al. A general Markov framework for page importance computation. CIKM, 2009.
A Unified Model
64
x1 x2 x3 xฯ โฆ โฆ
y1 y2 y3 yฯ
x: Markov chain โ to model the jump of the random surfer y: Random variable dependent on x (can be understood as staying time for simplicity) โ to model page utility.
Markov Skeleton Process Model
WWW 2011 Tutorial
Algorithms in the Framework
65 WWW 2011 Tutorial
Algorithms in the Framework
66
Markov Skeleton Process
Mirror Semi-Markov Process Semi-Markov Process
Continuous-Time Markov Process
Discrete-Time Markov Process
PageRank Personalized
PageRank HostRank
BrowseRank BrowseRank
Plus
Block-level PageRank
Topic-Sensitive PageRank
WWW 2011 Tutorial
Weighted PageRank
Outline
I. Overview
II. Graph Ranking by Link Analysis
III. Graph Ranking with Node and Edge Features
IV. Graph Ranking with Supervision
V. Implementation for Graph Ranking
VI. Summary
67 WWW 2011 Tutorial
Beyond the Graph
โข In addition to the weights associated with nodes and edges, we sometimes also have supervision on the nodes.
โข Typical supervision
โ Binary labels โข Spam/non-spam, Junk/non-junk, etc.
โ Pairwise preference โข A is preferred to B
โ List of partial order โข A > B > C, according to user visiting frequency.
68 WWW 2011 Tutorial
Notations
โข Graph ๐บ(๐, ๐ธ, ๐, ๐) ๐ = ๐ฃ๐ : node set, ๐ = ๐
๐ธ = ๐๐๐ : edge set, ๐ธ = ๐
๐ = ๐ฅ๐๐ : edge features, ๐ฅ๐๐ = ๐, ๐ฅ๐๐ = (๐ฅ๐๐1, ๐ฅ๐๐2, โฆ , ๐ฅ๐๐๐)๐
๐ = ๐ฆ๐ : node features, ๐ฆ๐ = ๐, ๐ฆ๐ = (๐ฆ๐1, ๐ฆ๐2, โฆ , ๐ฆ๐โ)๐
โข Matrices ๐: adjacency matrix or link matrix
๐: transition probability matrix
โข Rank score vectors ๐: authority score vector
๐: hub score vector
๐: general rank score vector
69 WWW 2011 Tutorial
Notations for Supervision
โข Supervision ๐ต: ๐ -by-๐ supervision matrix, each row of ๐ต represents a pairwise
preference ๐ฃ๐ โป ๐ฃ๐ (๐ is the number of pairwise constraints)
โข Parameters
๐ = (๐1, ๐2, โฆ , ๐๐)๐: weight of edge features
๐ = (๐1, ๐2, โฆ , ๐โ)๐: weight of node features
70
๐ต =
0โฏ ๐ โฏ ๐ โฏ 0โฎ
0โฏ1โฏ โ 1 โฏ0โฎ ๐ ร๐
WWW 2011 Tutorial
๐ฃ๐ โป ๐ฃ๐ โ ๐๐ โฅ ๐๐ โ ๐ ๐๐๐ค ๐๐ ๐ต๐ โฅ 0
๐ฃ๐ โป ๐ฃ๐ โ min*1 โ (๐๐ โ ๐๐)+ โ minโ๐๐(๐ต๐ โ ๐)
Supervised Graph Ranking Algorithms
โข LiftHITS
โข Adaptive PageRank
โข NetRank I & II
โข Laplacian Rank
โข Semi-supervised PageRank
71 WWW 2011 Tutorial
โข Adjust adjacency matrix of HITS using one-step gradient ascent, to satisfy the supervision
โข Methodology
72
LiftHITS
๐ ๐ฃ๐ = ๐๐๐๐ ๐ฃ๐๐
๐ ๐ฃ๐ = ๐๐๐๐ ๐ฃ๐๐
๐: = ๐๐๐๐ ๐: ๐ฃ๐ = ๐๐๐๐๐๐
๐
๐(๐ฃ๐)
๐
๐๐: ๐ฃ๐
๐๐๐๐= ๐๐๐๐(๐ฃ๐)
WWW 2011 Tutorial
Chang et al. Learning to create customized authority lists. ICML, 2000.
LiftHITS
โข Algorithm (to lift the rank of ๐ฃ๐)
1. Apply HITS to compute authorities ๐ based on ๐
2. Compute gradient โ๐, ๐, ฮ๐๐๐ โ๐๐+ ๐ฃ๐
๐๐๐๐= ๐๐๐๐ ๐ฃ๐
3. Update ๐:๐๐ = ๐๐๐ + ๐พ
ฮ๐๐๐
ฮ๐๐๐๐
4. Normalize weights, setting all ๐๐๐ โฅ 0
5. Re-compute HITS authorities ๐ using updated ๐:
โข Discussion โ May affect the ranking of neighborhood nodes
โ ๐: will become denser than ๐
73 WWW 2011 Tutorial
Adaptive PageRank
โข Adjust teleport vector to produce a ranking result
โ To satisfy the supervision
โ To be as close to PageRank as possible
74 WWW 2011 Tutorial Tsoi et al. Adaptive ranking of Web pages. WWW, 2003.
Adaptive PageRank
โข Methodology
โ Transform PageRank equation
โ Optimize teleport vector ๐
WWW 2011 Tutorial 75
๐ = ๐ผ๐๐๐ + 1 โ ๐ผ ๐ โ ๐ = 1 โ ๐ผ (๐ผ โ ๐ผ๐๐);1๐ โ ๐๐
min๐
๐๐ โ ๐๐ 02
s.t. ๐ต๐๐ โฅ 0, ๐ โฅ 0
๐๐๐ = 1, ๐(0) =1
๐๐
๐ = 1,1, โฆ , 1 ๐
Adaptive PageRank
โข Reduce the complexity
โ Compute cluster-level adaptive PageRank
โ Organize nodes into clusters according to some criteria
โ Assign back the scores in node-level
โข Discussion
โ Some supervision will become invalid
76 WWW 2011 Tutorial
๐ฃ1 โป ๐ฃ2
๐ฃ1 ๐ฃ2
NetRank I
โข Adjust PageRank flow to produce a ranking result
โ To satisfy the supervision
โ To maximize the entropy of the PageRank flow
WWW 2011 Tutorial 77
All PageRank flows between nodes are equal to each other.
The sum of flows in one node equals its inlink number.
NetRank I
โข Notations โ ๐ฃ๐: dummy node having two-way edges with all ๐ฃ๐ โ ๐
โ ๐โฒ = ๐ โช ๐ฃ๐
โ ๐บโฒ = ๐โฒ, ๐ธโฒ
โ ๐(0): the set of nodes which have at least one outlink
โ ๐๐๐: PageRank flow from ๐ฃ๐ to ๐ฃ๐
78 WWW 2011 Tutorial
Agarwal et al. Learning to rank networked entities. KDD, 2006.
Optimization Problem in NetRank I
79
min0โค๐๐๐โค1
๐๐๐ log ๐๐๐(๐,๐)โ๐ธโฒ
s.t. ๐๐๐(๐,๐)โ๐ธโฒ โ 1 = 0
โ๐ฃ๐ โ ๐โฒ: โ ๐๐๐ + ๐๐๐(๐,๐)โ๐ธโฒ = 0(๐,๐)โ๐ธโฒ
โ๐ฃ๐ โ ๐ 0 : โ๐ผ๐๐๐ + (1 โ ๐ผ) ๐๐๐ = 0(๐,๐)โ๐ธ
โ๐ฃ๐ โบ ๐ฃ๐: ๐๐๐๐,๐ โ๐ธโฒ โ ๐๐๐(๐,๐)โ๐ธโฒ โค 0
Objective
Total
Balance
Teleport
Preference
WWW 2011 Tutorial
โข Discussion โ Converted to the duel formulation and solved by gradient
method โ Too many variables (edge number)
NetRank II
โข Adjust the parametric transition matrix to produce a ranking result
โ To satisfy the supervision
โ To be as close to PageRank as possible
WWW 2011 Tutorial 80
NetRank II
โข Step 1: Build a parametric transition matrix
โ ๐ฃ๐: dummy node having two-way edges with all ๐ฃ๐ โ ๐
โ ๐ฅ๐๐๐: edge features from ๐ฃ๐ to ๐ฃ๐
โ ๐ โ : edge feature combination function
81
๐ ๐, ๐ =
0, ๐ฃ๐ โ ๐ฃ๐ , ๐ฃ๐ โ ๐ฃ๐ , ๐ฃ๐ โ ๐๐๐๐(๐)
๐ผ๐ ๐, ๐ฅ๐๐ , ๐ฃ๐ โ ๐ฃ๐ , ๐ฃ๐ โ ๐ฃ๐ , ๐ฃ๐ โ ๐๐๐๐(๐)
1, ๐ฃ๐ โ ๐ฃ๐ , ๐ฃ๐ = ๐ฃ๐ , ๐ฃ๐ โ ๐๐๐๐(๐)
1 โ ๐ผ, ๐ฃ๐โ ๐ฃ๐ , ๐ฃ๐ = ๐ฃ๐ , ๐ฃ๐ โ ๐๐๐๐(๐)๐๐ ๐ฃ๐ = ๐ฃ๐ , ๐ฃ๐ โ ๐ฃ๐0, ๐ฃ๐ = ๐ฃ๐ , ๐ฃ๐ = ๐ฃ๐
WWW 2011 Tutorial
Chakrabarti et al. Learning parameters in entity relationship graphs from ranking preference. PKDD, 2006.
NetRank II
โข Step 2: Minimize the loss following function calculated from the parametric matrix
โ ๐ป: iteration number for PageRank calculation
โข Discussion
โ NetRank I II, reduce the number of parameters
โ Newton method for computation
โ Need to compute successive matrix multiplication.
82
min 1+ ((๐๐)๐ป๐(0))๐โ((๐๐)๐ป๐(0))๐
๐ฃ๐โบ๐ฃ๐
WWW 2011 Tutorial
Laplacian Rank
โข Adjust the ranking result directly
โ To satisfy the supervision
โ To make connected nodes have similar ranking
WWW 2011 Tutorial 83
Laplacian Rank
โข Laplacian matrix
โ ฮ : diagonal matrix with ฮ ๐๐ = ๐ 0 ๐ฃ๐
โ Optimization with Regularization
โข Discussion
โ Need to compute pseudo matrix inversion
84
๐ฟ = ๐ผ โฮ 12๐ฮ ;
12 + ฮ ;
12๐๐ฮ
12
2
WWW 2011 Tutorial Agarwal. Ranking on graph data. ICML, 2006.
min1
2๐๐๐ฟ๐ + ๐ ํ๐๐
๐ฃ๐โบ๐ฃ๐
s.t. ๐๐ โ ๐๐ โฅ 1 โ ํ๐๐ , โ๐ฃ๐ โบ ๐ฃ๐
Semi-Supervised PageRank
โข Adjust the parametric transition matrix and the parametric teleport vector to produce a ranking result
โ To satisfy the supervision
โ To be as close to PageRank as possible
WWW 2011 Tutorial 85
Semi-Supervised PageRank
โข Methodology โ Define parametric transition matrix
โข ๐๐๐ =
๐๐๐ฅ๐๐๐๐
๐๐๐ฅ๐๐๐๐๐๐๐๐ โ ๐ธ
0, ๐๐ก๐๐๐๐ค๐๐ ๐
โ Define parametric teleport vector โข ๐๐ ๐ = ๐๐๐ฆ๐
โ Minimize the sum of a propagation term and a loss term
86 WWW 2011 Tutorial
Gao et al. Semi-supervised ranking on very large graph with rich metadata. Microsoft Research Technical Report, MSR-TR-2011-36, 2011.
Semi-Supervised PageRank
87
min๐โฅ0, ๐โฅ0, ๐โฅ0
๐ฝ1 ๐ผ๐๐ ๐ ๐ + 1 โ ๐ผ ๐ ๐ โ ๐ 2 + ๐ฝ2 ๐(๐ โ ๐ต๐)
๐ = ๐ผ๐๐(๐)๐ + 1 โ ๐ผ ๐(๐)
Propagation term: based on PageRank propagation, combining edge features and node features by ๐(๐) and ๐(๐).
Loss term: compared with supervised information in
pairwise preference fashion.
WWW 2011 Tutorial
Gradient based Optimization
Denote
Derivatives
More Details
WWW 2011 Tutorial 89
yx G
X1 X2 X3 X4
1 2 3 4
Y1 Y2 Y3 Y4
1 2 3 4
=
X1Y1
X1 Y2
X1 Y3
X1 Y4
X2 Y1
X2 Y2
X2 Y3
X2 Y4
X3Y1
X3 Y2
X3 Y3
X3 Y4
X4 Y1
X4 Y2
X4 Y3
X4 Y4
1 2 3 4 2 4 6 8 3 6 9 12 4 8 12 16
0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 Graph Edges
A Unified Framework
โข Define the graph propagation term based on a Markov random walk on the web graph
โข Incorporate edge features into the transition probability of the Markov process, and incorporate node features to its teleport vector
โข Convert the constraints to loss functions using ๐ฟ2 distance between the ranking results given by the parametric model and the supervision
โข Keep the sparsity of the graph when updating the parameters of the model during the optimization process
90 WWW 2011 Tutorial
A Unified Framework
91
min๐โฅ0, ๐โฅ0, ๐โฅ0
๐ (๐; ๐ ๐, ๐ , ๐(๐, ๐))
s.t. ๐(๐; ๐ต, ๐) โฅ 0
min๐โฅ0, ๐โฅ0, ๐โฅ0
๐ฝ1๐ ๐; ๐ ๐, ๐ , ๐ ๐, ๐ โ ๐ฝ2๐(๐; ๐ต, ๐)
Propagation term: based on a certain graph ranking algorithm, combining graph structure and rich metadata.
Loss term: compared with supervision
WWW 2011 Tutorial
Algorithms in the Framework
92 WWW 2011 Tutorial
Algorithm Link Structure
Edge Feature
Node Feature
Supervision Objective Parameterized Model
Adaptive PageRank Yes No No Pairwise PageRank No
NetRank I Yes No No Pairwise Inlink number No
NetRank II Yes Yes No Pairwise PageRank Yes
Laplacian Rank Yes No No Pairwise Laplacian No
Semi-supervised PageRank Yes Yes Yes Pairwise PageRank Yes
Outline
I. Overview
II. Graph Ranking by Link Analysis
III. Graph Ranking with Node and Edge Features
IV. Graph Ranking with Supervision
V. Implementation for Graph Ranking
VI. Summary
93 WWW 2011 Tutorial
Many Algorithms
WWW 2011 Tutorial 94
HITS
Topic-sensitive PageRank
Block-level PageRank
BrowseRank
LiftHITS
Semi-supervised PageRank
Basic Operations in Algorithms
Algorithms Operation
Almost All Matrix-vector multiplication
NetRank II Matrix-matrix multiplication
Semi-supervised PageRank Graph-based Kronecker product between vectors
Adaptive PageRank, LaplacianRank Matrix (pseudo) inversion
โฆ โฆ
Graph Propagation
โข For many operations, propagation of values along edges in a graph is their basic computational unit.
96 WWW 2011 Tutorial
Example: Matrix-Vector Multiplication
97 WWW 2011 Tutorial
Px
A
B C
D
Graph:
2 1 1
2
3 1
A B C D
100 100 100 100
100
200
100
300
100 200
A B C D
300 400 200 100
Propagate elements in x in the graph defined by P Aggregate the propagated values per node.
๐ =
0 0 1 21 0 3 00 2 0 00 1 0 0
๐ฅ =[ -๐
๐๐ฅ =[ -๐
Example: Kronecker Product
WWW 2011 Tutorial 98
m
n
p
q
G =
m n p q
5 6 7 8 x =
m n p q
1 2 3 4 y =
Result = [mp,15] [mq,20] [nm,6] [np,18] [pn,14] [qn,16]
m,1
n,2
n,2
p,3
p,3 q,4
yx G
m n p q
[P,3] [q,4]
[m,1] [p,3]
[n,2] [n,2]
Propagate y along graph G
For each node, multiply x with the received y values
Large-scale Implementation of Graph Propagation
โข Distributed Computation Models
โ MapReduce Model
โ Bulk Synchronous Parallel(BSP) Model
99 WWW 2011 Tutorial
MapReduce
100 Dean et al. MapReduce: simplified data processing on large clusters. OSDI, 2004.
WWW 2011 Tutorial
โข Mapper โ Process input data into (key, value) pairs โ Output (key, value) pairs into MapReduce framework
โข MapReduce Framework โ Sort and aggregate (key, value) pairs according to
buckets
โข Reducer โ Receive (key, value) pairs with the same range of keys
from MapReduce infrastructure โ Aggregate the values for the same key โ Output the result
MapReduce
Mapper
Input Data Mapper
Mapper
MapReduce Framework
(Bucketing,
Sorting, Aggregating)
Reducer
Reducer
[key1, value1] [key2, value1] [key3, value1]
[key1, value2] [key3, value2] [key2, value2]
[key3, value3] [key2, value3] [key1, value3]
[key1, value1]
[key1, value2]
[key1, value3]
[key2, value1]
[key2, value3]
[key2, value2]
[key3, value3]
[key3, value1]
[key3, value2]
[key1, valueโ]
[key2, valueโ]
[key3, valueโ]
Graph Propagation on MapReduce
102 WWW 2011 Tutorial
โข Graph data is partitioned into sub graphs according to source node; input vector is also partitioned in the same way.
โข One mapper processes one partition of graph data and vector.
โข (Key, value) = (Dest node, Value on source node)
โข Reducer aggregates data according to destination node.
Discussions
โข Pros: system stability and maturity โ MapReduce has been widely used for web-scale data
processing, e.g., indexing, PageRank computation, etc. โ Simple interface: (Map, Reduce) โ Clear system-level logic and fault tolerance
โข Cons: system performance โ Data locality, graph data and output data are not
guaranteed to be on the same machine, which causes potential network transfer
โ Intensive access to disk, at every stage the system needs to serialize data to disk for the purpose of fault tolerance.
103 WWW 2011 Tutorial
BSP
SuperStep
SuperStep
SuperStep
104 WWW 2011 Tutorial http://en.wikipedia.org/wiki/Bulk_synchronous_parallel
Not necessarily โmapโ
Not necessarily โ(key value) pairsโ
BSP is more general, and MapReduce can be regarded as a special version of BSP
Graph Propagation on BSP
โข Graph propagation on MapReduce can be easily converted to that on BSP.
โข Since BSP is more flexible, it can potentially ensure locality during the iterative propagation, and thus improve the efficiency of the computation.
105 WWW 2011 Tutorial
Discussions
โข Pros โ Can support co-location of graph partition and
processing node, so as to avoid unnecessary data transfer
โ Node communication logic provides more flexible message passing
โข Cons โ Flexibility vs. usability
โ Need to implement fault tolerance logic
โ Not as widely used as MapReduce in industry.
106 WWW 2011 Tutorial
Graph Processing Systems
โข Pegasus
โข Hama
โข Pregel
โข Trinity
โข Graphor
107 WWW 2011 Tutorial
Pegasus
โข A large-scale graph mining system based on Hadoop
โข Computation model โ MapReduce โ Optimized matrix-vector multiplication by partitioning data
into blocks
โข Supported algorithms โ PageRank โ Random walk with restart โ Graph diameter computing โ Graph components mining
108 WWW 2011 Tutorial
Kang et al. PEAGSUS: a peta-scale graph mining system โ implementation and observations. ICDM, 2009. Kang et al. PEAGSUS: mining peta-scale graphs, knowledge and information systems. DOI: 10.1007/s10115-010-0305-0, 2010.
Hama
โข Graph processing library in Hadoop
โข Computation model
โ MapReduce to handle matrix computation
โ BSP to handle other graph processing
โข Supported algorithm
โ Large scale matrix multiplication
โ Shortest path finding in graph
โ PageRank
109 WWW 2011 Tutorial
The Apache Hama Project: http://incubator.apache.org/hama/ Seo et al. HAMA: an efficient matrix computation with the MapReduce framework. IEEE CloudCom Workshop, 2010.
Pregel
โข Googleโs large scale graph processing engine
โข Computation model โ BSP
โ Ram-based system
โข Supported algorithms โ PageRank
โ Single source shortest path
โ Graph component finding
110 WWW 2011 Tutorial Malewicz et al. Pregel: a system for large-scale graph processing. PODC 2009. Malewicz et al. Pregel: a System for large-scale graph processing. SIGMOD, 2010.
Trinity
โข A graph database and computation platform by Microsoft Research
โข Computation model
โ BSP, with asynchronous mode for message passing
โ Ram-based system
โข Supported algorithms
โ PageRank
โ Breadth first search
111 WWW 2011 Tutorial Trinity - a graph data base and computation platform. http://research.microsoft.com/en-us/projects/trinity/
โข A graph computation engine by Microsoft Research
โข Computation model
โ MapReduce
โ Additional logic to keep graph locality inspired by BSP
โข Supported algorithms
โ PageRank
โ Matrix-vector multiplication
โ Graph-based Kronecker product of vectors
WWW 2011 Tutorial 112 Graphor - a billion scale web graph processing platform http://research.microsoft.com/en-us/projects/graphor/
System Model Fault tolerance Supported algorithms
Pegasus MapReduce MapReduce PageRank, graph components finding, etc.
Hama MapReduce / BSP MapReduce PagRank, matrix vector multiplication, etc.
Pregel BSP Self designed PageRank, shortest path, graph components finding, etc.
Trinity BSP + None PagRank, breadth frist search on graph, etc.
Graphor MapReduce + BSP MapReduce Pagerank, multiplication of matrix and vector, graph based vector kronecker product, etc.
Comparison
113 WWW 2011 Tutorial
Outline
I. Overview
II. Graph Ranking by Link Analysis
III. Graph Ranking with Node and Edge Features
IV. Graph Ranking with Supervision
V. Implementation for Graph Ranking
VI. Summary
114 WWW 2011 Tutorial
Summary
โข Link analysis is a classical graph ranking method
โข Rich information in nodes and edges can help graph ranking
โข External knowledge on ranking orders can make graph ranking more consistent with human intuition
โข Many systems have been developed for large-scale graph ranking
115 WWW 2011 Tutorial
Answers to Question #1
โข How to perform graph ranking based on graph structure?
โ Link analysis
โ Hierarchical structure in graph
โ Random surfer model / Markov chain
116 WWW 2011 Tutorial
Answers to Question #2
โข How to leverage node and edge features for better graph ranking?
โ Node weight
โ Edge weight
โ Continuous-time Markov process / Mirror semi-Markov process / Markov skeleton process
117 WWW 2011 Tutorial
Answers to Question #3
โข How to incorporate external knowledge in graph ranking?
โ Different optimization objectives
โ Supervised learning framework
โ Large-scale optimization
118 WWW 2011 Tutorial
Answers to Questions #4
โข How to implement large-scale graph ranking algorithms?
โ MapReduce and BSP
โ Several systems
โ Select the proper platform for your application!
119 WWW 2011 Tutorial
Future Directions
โข Theoretical study
โ Various Markov processes for graph ranking
โ Learning theory for graph ranking (non-i.i.d.)
โข Novel algorithms
โ Ranking on a time series of graphs
โ Ranking on heterogeneous graphs
โข Implementation
โ Tradeoff of efficiency, flexibility, and reliability
โ Dealing with more complex graph operations
120 WWW 2011 Tutorial
References
Agarwal et al. Learning to rank networked entities. KDD, 2006.
Agarwal. Ranking on graph data. ICML, 2006.
Cai et al. Block-level link analysis. SIGIR, 2004.
Chakrabarti et al. Learning parameters in entity relationship graphs from ranking preference. PKDD, 2006.
Chang et al. Learning to create customized authority lists. ICML, 2000.
Gao et al. A general Markov framework for page importance computation. CIKM, 2009.
Gao et al. Semi-supervised ranking on very large graph with rich metadata. Microsoft Research Technical Report, MSR-TR-2011-36, 2011.
Haveliwala et al. An analytical comparison of approaches to personalizing PageRank. Stanford University Technical Report, 2003
Kang et al. PEAGSUS: a peta-scale graph mining system โ implementation and observations. ICDM, 2009.
WWW 2011 Tutorial 121
References
Kang et al. PEAGSUS: mining peta-scale graphs, knowledge and information systems. DOI: 10.1007/s10115-010-0305-0, 2010.
Kleinberg. Authoritative sources in a hyperlinked environment. IBM Research Report RJ 10076, 1997.
Liu et al. BrowseRank: Letting Web users vote for page importance. SIGIR, 2008.
Malewicz et al. Pregel: a system for large-scale graph processing. PODC 2009.
Malewicz et al. Pregel: a System for large-scale graph processing. SIGMOD, 2010.
Page et al. The PageRank citation ranking: bringing order to the Web. Stanford Digital Library Technologies Project, 1998 .
Seo et al. HAMA: an efficient matrix computation with the MapReduce framework. IEEE CloudCom Workshop, 2010.
Tsoi et al. Adaptive ranking of Web pages. WWW, 2003.
Xue et al. Exploiting the Hierarchical Structure for Link Analysis. SIGIR, 2004.
WWW 2011 Tutorial 122
Acknowledgement
โข Wei-Ying Ma (Microsoft Research Asia)
โข Hang Li (Microsoft Research Asia)
โข Haixun Wang (Microsoft Research Asia)
โข Tao Qin (Microsoft Research Asia)
โข Zhi-Ming Ma (Chinese Academy of Sciences)
โข Yuting Liu (Beijing Jiaotong University)
โข Ying Zhang (Nankai University)
โข Wei Wei (Huazhong University of Science and Technology)
โข Wenkui Ding (Tsinghua University)
โข Changhao Jiang (Tsinghua University)
โข Chenyan Xiong (Chinese Academy of Sciences)
โข Di He (Peking University)
123 WWW 2011 Tutorial
Thank You!
124
WWW 2011 Tutorial
This talk is Copyright 2011. Authors retain all rights, including copyrights and distribution rights. No
publication or further distribution in full or in part permitted without explicit written permission.