Temporal GraphsSanket Lokegaonkar

CS6604: Data Mining Large Networks

Fall 2017Virginia Tech

Outline• Motivation

• Categories

• Yu, Wenchao, Charu C. Aggarwal, and Wei Wang. "Temporally Factorized Network Modeling for Evolutionary Network Analysis.“

• Gu, Yupeng, Yizhou Sun, and Jianxi Gao. "The Co-Evolution Model for Social Network Evolving and Opinion Migration."

• Amiri, Sorour E., Liangzhe Chen, and B. Aditya Prakash. "SnapNETS: Automatic Segmentation of Network Sequences with Node Labels."

• Take-aways

Motivation

Graph Definitions

• Temporal Graphs: • G= { N, A(t) } where N is the set of nodes in the graph and A(t) represents a

set of adjacency matrices as a function of time

• For unweighted networks, the adjacency network is binary

• For weighted networks, A(t) represents adjacency matrices with weights changing over time.

Ubiquity of Temporal Graphs [1] [2]

IRIS Co-author NetworkTwitter Follower-FolloweeNetwork

And many more,Such as disease spread network

Applications

• Link Prediction with Temporal Context• Which people would you like to be friends with, given you have been

unfriending certain people?

• Community Evolution Dynamics• Which communities grow? Which communities shrink?

• Anomaly/Event Detection• When user’s connection network changes significantly in single timestep?

• Detecting change in virus behavior with mutation

• Immunization Policies• Create changes in immunization policy as disease mutates

Categories

• Maintenance Methods: • Correct your predictions/classifications/communities over time

• Learning Dynamics: • Learn how the network is changing over time

• Hybrid methods• Work that we discuss today

Method-based Characterization[3]

Graph-based Characterization[3]

• Slowly Evolving Networks• Time-stamp based

• Classical definition

• Explored in literature

• Streaming Networks• Edges and nodes come as a stream

• Real-time streaming algorithms are expected.

• Much more challenging

Yu, Wenchao, Charu C. Aggarwal, and Wei Wang. "Temporally Factorized Network Modeling for Evolutionary Network Analysis.“ [4]

Tags: Learning Dynamics, Slowly Evolving Networks, Anomaly Detection

All figures are credited to the respective paper

Core Idea [4]

• Simple Idea

• Decompose the series of adjacency matrices into a 1. changing component w.r.t. time and 2. constant component

Definitions [4]

• Given A(t), we can factorize as follows:• A(t) = f( U V(t)T)

• where U and V(t) are n x k dimensions. V(t) is a function with time t.

• k corresponds to latent dimension size

• f(.) is an element-wise function

• For undirected graphs , A(t) is symmetric. • A(t) = f( V(t) V(t)T)

• where V(t) = n x k dimensions

• can be represented with the above formulation as well, but authors took occam’s razor approach

Optimization Problem [4]

• Least Squares Optimization

• Find me f( U V(t)T) which matches A(t) as much as possible

• D(t) = Decay function with time t, current state of the network is more important , compared to the previous

• Quite memory-intensive for sparse networks.

• Better formulation for sparser networks, , let E(t) be the set of edges for which the weights in A(t) are non-zero at time t. Let S(t) be a sample of edges (i, j) at time-stamp t such that the value of edge is 0

Model Choices [4]

• V(t) is a polynomial function:

• With added regularization:

• How to optimize:• Gradient descent:

• Compute partial derivatives with respect to U and V(t), update and repeat

Algorithm [4]

Complexity: O(m + n)

• Last T-1 training data, T as test set

• Comparative algorithms:• Common Neighbors

• Adamic Adar

• Preferential Attachment

• Nonparametric Link Prediction

• Link Prediction via Matrix Factorization

• CP Tensor Model ( Only one which takes into account temporal context)

Application: Temporal Link Prediction [4]

Application: Temporal Link Prediction Results[4]

Application: Temporal Anomalies and Events Detection [4]

• Temporal anomalies/Events = unexpected changes in the network structure over time

• A(T+1) is observed matrix and መ𝐴(T+1) is predicted matrix

• If the change is large , then possible anomaly

• How do we localize the anomaly?

• Edges where the change is above threshold

Application: Temporal Anomalies and Events Detection Results [4]

Application: Community Analysis [4]

Summary (tl;dr) [4]

• Decompose the series of adjacency matrices into a 1. changing component w.r.t. time and 2. constant component

• Use least squares optimization for learning the two components

• Link prediction: Corresponds to predicting based on previous history

• Event detection: Corresponds to detecting large changes in expected vs observed

• Can be used for detecting expanding communities

• Complexity manageable

• Cons:• Does not consider the individual characteristics of the nodes

Gu, Yupeng, Yizhou Sun, and Jianxi Gao. "The Co-Evolution Model for Social Network Evolving and Opinion Migration." [5]

Tags: Learning Dynamics, Slowly Evolving Networks, Anomaly Detection

All figures are credited to the respective paper

Core Idea [5]

• Consider a dynamic graph, • Step 1 : Actors(nodes) change their opinions or behaviors, based on their

social influence

• Step 2: The network structure gets affected by these changes

• Go back to Step 1

• This paper proposes a co-evolution model which jointly models• Network changes

• Actors changes

• Allows to do lots of interesting stuff:• Opinion Convergence, Intra-community divergence

• Social reach of actors as opinions

Definitions & Background [5]

• Derives from “Influence” and “Homophily”

• Influence:• Individuals are affected by their friends, or the company they keep

• Homophily• Individuals interact with people that are similar

• Each individual can be represented by a feature vector

• Network is a reflection of “latent” variables

Definitions & Background [5]

• Collective motion:• Each particle moves at a constant rate v, while the direction of motion is

determined by the average direction of all others within its neighborhood of radius r, plus some random perturbation

Co-evolution Model [5]

• Project the nodes of a network to latent variable space ( Network generation)

• Opinions change depending on the interactions (Opinion Migration)

Co-evolution Model : Social Network Generation [5]

• Each link independent of other links

• Links specified by scoring function: RK x RK -> R, that assigns a score to a pair of node features indicating the likelihood of link between two nodes

• Dot Product-based Score Function:

• Cosine Similarity is the scoring function

• In other words, people with extreme stances (i.e. large norms of latent feature vector) will become the opinion leader.

• Not true in real-world case

• Gravity-based Score Function:

• Each node has a popularity hyper-parameter b.

• The scoring function is similar to gravity equation:

• Earlier approaches in opinion migration• is the average position of neighbors of xn at time t• Leads to a model where opinions do not change after the communities are

formed

• Does not capture the propagation model

• The paper chooses to regularize the direction instead of the position

• = average direction of neighbors• A scholar tends to raise interest in a research topic that is trending among her

collaborators.

Co-evolution Model : Opinion Migration [5]

Co-evolution Model: Unified Model Algorithm[5]

• Sparsity parameter d -> the sparsity of the graph (i.e. the average number of friends)

• Noise level σ -> the deviation of one’s direction from the expected value

Applying to Simulated Dataset [5]

• Simulated Dataset:• Node randomly initialized with initial position in a lattice [−L/2,L/2]× [−L/2,L/2] where L = 5

• Each node has popularity b ∼ Uniform([1,2]). b fixed throughout the process

• Average Normalized Velocity:

• Two hyper-parameters:.• Noise level σ

• Sparsity Parameter d

Applying to Simulated Dataset Results [5]

Applying to Simulated Dataset Results [5]

Applying to Co-sponsorship dataset [5]

• Co-sponsorship dataset:• .A sponsor of a bill is a legislator (usually a member from the congress) who introduces a bill or resolution for consideration. A

cosponsor is another congress member who adds his or her name as a supporter to the sponsor’s bill. Co-sponsorship contains important information about the social support network between legislators: the closer the relationship between a sponsor and a cosponsor, the more likely it is that the sponsor has directly petitioned the cosponsor for support

• Optimization objective:

• Method:• Step 1 : Initialize the latent variables by maximizing likelihood on the first observed graph and directions are initialized randomly• \\ Co-ordinate descent –like optimization• Step 2 :Update b• Step 3 : Update X, Θ• Go back to Step 2 \\ Stop when the likelihood does not change - converges

• Baselines:• CoNNdot : Probability of link = dot-product based similarity functions• Latent feature propagation model (LFP): binary latent feature propagation model• Dynamic social network with latent space models : Only latent vectors matter• Phase transition model :

• Large-scale information network embedding (LINE) :baseline for static methods

Applying to Co-sponsorship Prediction Results[5]

• Calculate the pairwise probability of a link from un to all other users, rank them and evaluate the AUC score

• Two metrics:• Normalized discounted cumulative gain

(NDCG)• AUC Score

Applying to Co-sponsorship Prediction Results[5]

• Calculate the pairwise probability of a link from un to all other users, rank them and evaluate the AUC score

• Two metrics:• Normalized discounted cumulative gain

(NDCG)• AUC Score

Summary (tl;dr)

• Models changes in the network structure and opinions

• Network Structure:• Embed to latent vector using HMM

• Opinion Migration:• Flocks of bird fly together.

• Results show interesting applications:• Detecting changes in opinion• Opinion Leaders

• Biologically-inspired

• Comparison might be nuanced:• Comparing against other state-of the-art approaches will be interesting

Amiri, Sorour E., Liangzhe Chen, and B. Aditya Prakash. "SnapNETS: Automatic Segmentation of Network Sequences with Node Labels." [6]

Tag: Network summarizations

Network Sequences [6]

Epidemiology: disease spreads over contact networks

Flu

Meme

G1 G2 G3 G4

Uninfected

Infected

G1 G2 G3 G4

Social Media: Information spreads over friendship networks

Inactive

2

Active

Making sense of network sequences [6]

Flu

when do the infection patterns change?

Star Bridge Near Clique

Reason:

• Virus mutation

• Vaccination

• …

G1 G2 G3 G4Uninfected

3

Infected

Making sense of network sequences [6]

Meme Reason:

• Event

• …

Star Clique

when do the activation patterns change?

G1 G2 G3 G4

Inactive

4

Active

Problem 1: Network sequence segmentation [6]

Given a sequence of networks with labeled nodes,

Find the best segmentation which captures:

Different distribution of node labels.

Near Clique

5

G1

Star

G2

Bridge

G3 G4

In this work:

Binary labels {0, 1}

Overview of SnapNETS [6]

Goal 1. Summarize each graph:Keep structural and label dependent properties

Goal 2. Construct Segmentation graph:Define nodes and edgesDefining edges weights

o extract the features of summarized graphs

Goal 3. Find the best segmentation:Define the best segmentation (path)Compute the best segmentation

15

Goal 1: Summarizing graph snapshots [6]

We want to preserve

Structural properties

Nodes labels

Role of Eigenvalue:

Epidemic threshold in most diffusion

models [Prakash et al. ICDM 2011]

Same Same diffusive properties

Leading eigenvalue of Adjacency matrix

Amiri, Chen, Prakash 18

Nodes: For each segment there is a node + {Source (‘s’), Target (‘t’)} Source (‘s’) = start time Target (‘t’) = end time

Edges: There is a directed edge between adjacent nodes

Goal 2: Segmentation graph [6]

Amiri, Chen, Prakash 23

Edge Weights [6]

How can we measure the distance between two segments?

w ?

Amiri, Chen, Prakash 24

Goal 3: Finding the best segmentation [6]

Observation:For each segmentation there is a path from ‘s’ to ‘t’

For each path from ‘s’ to ‘t’ there is a segmentation

Therefore,• Best segmentation problem ≡ Path optimization problem

Amiri, Chen, Prakash 28

Problem 3: Finding the best segmentation [6]

Our idea: Average longest path

Advantages: Parameter free

Naturally balances weight of the path with the number of segments.

Given a segmentation graph

Find the average longest path from ‘s’ to ‘t’

Amiri, Chen, Prakash 30

Summary (tl;dr)

• Presents a novel approach for summarizing temporal changes in graph

• It finds patterns where adjacent segments have different characteristics of nodes with the same label i.e. the ‘placement’ and ‘connection’ of active/inactive nodes are different

• Looks at global changes

• Can be extended to streaming or partially observed graphs

Conclusion

• Temporal networks come with massive applications • Link Prediction with Temporal Context

• Community Evolution Dynamics

• Anomaly/Event Detection

• Research Areas:• Link Prediction

• Learning Graph Dynamics

• Graph Summarization

• Opinion Migration

• And more.

References

• [1] : http://cmxp.bplaced.net/Commetrix/iris/2co-author_network_min3pap.png

• [2] : : http://cmxp.bplaced.net/Commetrix/iris/2co-author_network_min3pap.png

• [3] : Aggarwal, Charu, and Karthik Subbian. "Evolutionary network analysis: A survey." ACM Computing Surveys (CSUR) 47.1 (2014): 10.

• [4] Yu, Wenchao, Charu C. Aggarwal, and Wei Wang. "Temporally Factorized Network Modeling for Evolutionary Network Analysis.“

• [5] : Gu, Yupeng, Yizhou Sun, and Jianxi Gao. "The Co-Evolution Model for Social Network Evolving and Opinion Migration."

• [6] : Amiri, Sorour E., Liangzhe Chen, and B. Aditya Prakash. "SnapNETS: Automatic Segmentation of Network Sequences with Node Labels."