ieee transactions on knowledge and data engineering, …cslui/publication/tkdd-moss5.pdf ·...

MOSS-5: A Fast Method of ApproximatingCounts of 5-Node Graphlets in Large Graphs

Pinghui Wang , Junzhou Zhao, Xiangliang Zhang , Zhenguo Li, Jiefeng Cheng,John C.S. Lui, Fellow, IEEE, Don Towsley, Fellow, IEEE, Jing Tao, and Xiaohong Guan, Fellow, IEEE

Abstract—Counting 3-, 4-, and 5-node graphlets in graphs is important for graph mining applications such as discovering abnormal/evolution patterns in social and biology networks. In addition, it is recently widely used for computing similarities between graphs andgraph classification applications such as protein function prediction and malware detection. However, it is challenging to compute thesegraphlet counts for a large graph or a large set of graphs due to the combinatorial nature of the problem. Despite recent efforts incounting 3-node and 4-node graphlets, little attention has been paid to characterizing 5-node graphlets. In this paper, we develop acomputationally efficient sampling method to estimate 5-node graphlet counts. We not only provide a fast sampling method andunbiased estimators of graphlet counts, but also derive simple yet exact formulas for the variances of the estimators which are of greatvalue in practice—the variances can be used to bound the estimates’ errors and determine the smallest necessary sampling budget fora desired accuracy. We conduct experiments on a variety of real-world datasets, and the results show that our method is several ordersof magnitude faster than the state-of-the-art methods with the same accuracy.

Index Terms—Graphlet kernel, subgraph sampling, graph mining

Ç

1 INTRODUCTION

FOR complex networks such as online social networks(OSNs), computer networks, and biological networks,

designing tools for estimating the counts (or frequencies) of3-, 4-, and 5-node connected subgraph patterns (i.e., graph-lets) shown in Fig. 1 is fundamental for detecting evolutionand anomaly patterns in a large graph and computinggraph similarities for graph classification, which have beenwidely used for a variety of graph mining and learning

tasks. To explore patterns in a large graph, Milo et al. [1]defined network motifs as graphlets occurring in networksat numbers that are significantly larger than those found inrandom networks. Network motifs have been used for pat-tern recognition in gene expression profiling [2], evolutionpatterns in OSNs [3], [4], [5], [6], and Internet traffic classifi-cation and anomaly detection [7], [8]. In addition to mininga single large graph, graphlet counts also have been used toclassify a large number of graphs. The graphlet kernel [9](the dot product of two vectors of normalized graphletcounts) and RGF-distance [10] (euclidean distance betweentwo vectors of normalized graphlet counts) are widely usedfor graph similarity comparison, which is an importantproblem in application areas as disparate as bioinformatics,chemoinformatics, and software engineering. For example,1) protein function prediction: identifying whether a givenprotein is an enzyme is important for understanding itsfunction in biology. The biological network of a protein isusually represented as an undirected graph where a node inthe graph represents an atom and an edge represents theexistence of a chemical bond (i.e., a lasting attraction)between two atoms. Thus, one can infer whether a givenprotein is an enzyme or not by computing the similaritiesbetween the graph topologies of the protein and a large setof enzymes given in advance [11], [12]; 2) compound functionprediction. Similarly, chemical compounds are usually repre-sented as a graph, and computing the similarity betweenthem is important for applications such as predicting activ-ity or adverse effects of potential drugs [13], [14]; 3) node andcommunity clustering. In addition to biological and chemicalapplications, Yanardag and Vishwanathan [15] reveal thatcomputing similarities between the ego-networks of nodes(e.g., researchers in coauthor networks) in other networkssuch as coauthor networks and OSNs is useful for predicting

! P. Wang is with the MOE Key Laboratory for Intelligent Networks and Net-work Security, Xi’an Jiaotong University, Xianning West Road, Xi’an,Shaanxi 710049, China, and the Shenzhen Research Institute, Xi’an JiaotongUniversity, Shenzhen 518057, China. E-mail: [email protected].

! J. Zhao and J.C.S. Lui are with the Department of Computer Science andEngineering, The Chinese University of Hong Kong, Shatin, Hong Kong,Mainland China. E-mail: [email protected], [email protected].

! X. Zhang is with the King Abdullah University of Science and Technology,Thuwal 23955, Saudi Arabia. E-mail: [email protected].

! Z. Li is with the Huawei Noah’s Ark Lab, Shatin, Hong Kong.E-mail: [email protected].

! J. Cheng is with the Tencent Cloud Security Lab, Shenzhen 518057, China.E-mail: [email protected].

! D. Towsley is with the Department of Computer Science, University ofMassachusetts, Amherst, MA 01003. E-mail: [email protected].

! J. Tao is with the MOE Key Laboratory for Intelligent Networks and Net-work Security, Xi’an Jiaotong University, Xianning West Road, Xi’an,Shaanxi 710049, China. E-mail: [email protected].

! X. Guan is with the MOE Key Laboratory for Intelligent Networks andNetwork Security, Xi’an Jiaotong University, Xianning West Road, Xi’an,Shaanxi 710049, China, and the Center for Intelligent and Networked Sys-tems, Tsinghua National Lab for Information Science and Technology,Tsinghua University, Beijing, China. E-mail: [email protected].

Manuscript received 14 Jan. 2017; revised 13 Aug. 2017; accepted 14 Sept.2017. Date of publication 26 Sept. 2017; date of current version 5 Dec. 2017.(Corresponding author: Junzhou Zhao.)Recommended for acceptance by K. Yi.For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference the Digital Object Identifier below.Digital Object Identifier no. 10.1109/TKDE.2017.2756836

IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 30, NO. 1, JANUARY 2018 73

1041-4347! 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See ht _tp://www.ieee.org/publications_standards/publications/rights/index.html for more information.

the node classes (e.g., the field of researchers). Similarly, theyrepresent each online discussion thread on OSN Reddit1 as agraph where nodes correspond to users and there existsan edge between two nodes if at least one of them respondedto another’s comment. Yanardag and Vishwanathan [15]observe that computing the similarities between thesegraphs is effective for the task of identifying whether a givengraph belongs to a question/answer-based community or adiscussion-based community; 4) malware detection. Attackerscurrently use two effective and convenient ways to generateand distribute attack payloads: (a) reuse the existing mali-cious code to generate newmalware variants, (b) use repack-aging techniques to inject a small piece of malicious codeinto popular mobile Apps such as Angry Bird. Meanwhile,they can easily avoid traditional detectors based on pure syn-tax. Recently, [16], [17], [18] observe that themalwares gener-ated by the above two ways keep a large fraction ofrelationships between subroutines and classes in the originalcomputer programs, which can be recovered from disassem-bly of their executable binaries (software reverse engineer-ing). They define graphs (e.g., view graph in [16], componentgraph in [17], and call graph in [18]) to depict the relation-ships between subroutines and classes in softwares, there-fore comparing topology similarities between graphs isuseful for detecting the abovemalwares.

Due to the combinatorial explosion of the problem, it iscomputationally intensive to enumerate and computegraphlet counts even for a moderately sized graph. Forexample, for two medium-size networks Slashdot [19] andEpinions [20] that each only contains 105 nodes and 106

edges, more than 1010 4-node connected and induced sub-graphs (CISes), and 1013 5-node CISes. To address this prob-lem, approximate methods such as sampling could be usedin place of the brute-force enumeration approach. As shownin Fig. 2 (the graphical user interface of our system), a prac-tical sampling method should satisfy that it can stop as soonas possible when 1) it achieves the required accuracy or 2)the sampling budget runs out, and then returns 1) graphletcount estimates and 2) estimation errors.

Despite recent progress in counting triangles [21], [22], [23],[24], [25], [26] and 4-node graphlets [27], little attention hasbeen given to developing fast tools for characterizing andcounting 5-node graphlets. Recently, Pinar et al. [28] proposea fast method ESCAPE for counting 5-node undirected graph-lets by utilizing the relationships between 3-, 4-, and 5-nodegraphlets counts. However, ESCAPE is not scalable to largegraphs, which requires more than 10 hours to handle graphswith millions of nodes and edges. To address this challenge,in this paperwe propose a novel samplingmethodMOSS-5 toestimate the counts of 5-node graphlets. MOSS-5 consists oftwo sub-methods: T-5 and Path-5, which are customized tofast sample 5-node CISes in two specific graphlet groupsrespectively. Based on the samples of T-5 and Path-5, we esti-mate all 5-node graphlet counts and bound the estimates’errors. Our contributions are summarized as:

! Our method for sampling 5-node CISes and estimat-ing 5-node graphlet counts is scalable and computa-tionally efficient.

! To validate our method, we perform an in-depthanalysis to demonstrate the accuracy of our method.

Fig. 2. The graphical user interface of our system.

Fig. 1. 3-, 4-, and 5-node undirected graphlets. Combinatorial explosion: A node v with degree dv in the graph of interest is included in at least12 dvðdv # 1Þ 3-node CISes, 16 dvðdv # 1Þðdv # 2Þ 4-node CISes, and 1

24 dvðdv # 1Þðdv # 2Þðdv # 3Þ 5-node CISes.

1. http://www.reddit.com

74 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, VOL. 30, NO. 1, JANUARY 2018

We find that our method provides unbiased esti-mates of 5-node graphlet counts. We also derive sim-ple and exact formulas for the variances of theestimators, which is critical in practice such asbounding the estimates’ errors and determining aproper sampling budget to achieve a desired accu-racy. This has been lacking for previous estimators.

! We conduct experiments on a variety of publiclyavailable datasets, and experimental results demon-strate that our method significantly outperforms thestate-of-the-art methods. To guarantee the reproduc-ibility of the experimental results, we release thesource code of MOSS-5 in open source.2

The rest of this paper is organized as follows. The prob-lem formulation is presented in Section 2. Section 3 presentsour 5-node graphlet sampling method MOSS-5. The perfor-mance evaluation and testing results are presented inSection 4. Section 5 summarizes the related work. Conclud-ing remarks then follow.

2 PROBLEM FORMULATION

Let G ¼ ðV;EÞ be an undirected graph, where V and E arethe node set and edge set respectively. To define graphletcounts ofG, let us first introduce some notation. A subgraphG0 of G is a graph of which node set and edge set are bothsubsets of V and E respectively. An induced subgraph of G,G0 ¼ ðV 0; E0Þ, is a subgraph that consists of some nodes of Gand all of the edges that connect these nodes in G, i.e., V 0 & V ,E0 ¼ fðu; vÞ : u; v 2 V 0; ðu; vÞ 2 Eg. Until we explicitly say“induced” in this paper, otherwise a subgraph is not necessarilyinduced. All undirected graphs’ 5-node graphlets Gð5Þ

1 ; . . . ;Gð5Þ

21 studied in this paper are shown in Fig. 1. Denote by Cð5Þ

the set of 5-node CISes in G, and Cð5Þi the set of 5-node CISes

in G isomorphic3 to graphlet Gð5Þi . The graphlet count of Gð5Þ

i

is defined as hi ¼ jCð5Þi j, 1 ' i ' 21. As we discussed above, it

is computationally expensive to enumerate and count all 5-node CISes in large graphs. In this paper, we develop a fastsampling method to accurately estimate 5-node graphletcounts h1; . . . ; h21. For ease of reading, we list notation usedthroughout the paper in Table 1.

3 OUR METHOD OF ESTIMATING 5-NODE

UNDIRECTED GRAPHLET COUNTS

In this section, we introduce our method MOSS-5. Weobserve that 1) except CISes in Cð5Þ

1 [ Cð5Þ2 [ C

ð5Þ6 , 5-node

CISes include at least one subgraph isomorphic to graphlet

Gð5Þ3 ; 2) except CISes in Cð5Þ

2 [ Cð5Þ3 [ C

ð5Þ8 , 5-node CISes

include at least one subgraph isomorphic to graphletGð5Þ1 . Let

V1 ¼ f1; . . . ; 21g# f1; 2; 6g and V2 ¼ f1; . . . ; 21g# f2; 3; 8g.Inspired by the above two observations, as shown in Fig. 3,we develop a method MOSS-5 consisting of two sub-meth-ods: T-5 and Path-5, where T-5 is customized to fast sample

5-node CISes isomorphic to Gð5Þi , i 2 V1, and Path-5 is cus-

tomized to fast sample 5-node CISes isomorphic to Gð5Þj ,

j 2 V2. For any i 2 V1, we provide an unbiased estimate hð1Þiof hi based on sampled CISes of T-5, and derive a closed-formformula of the variance of hð1Þi . For any j 2 V2, similarly, we

provide an unbiased estimate hð2Þj of hj based on sampled

CISes of Path-5, and derive a closed-form formula of the vari-

ance of hð2Þj . Based on hð1Þi and h

ð2Þj , we propose a more accu-

rate estimator hk of hk, k 2 V1 [V2 ¼ f1; . . . ; 21g# f2g andprovide an unbiased estimator h2 of h2. To bound the error ofhk, k 2 f1; . . . ; 21g, we also derive the variance of each hk.

3.1 The T-5 Sampling MethodDenote

Gð1Þv ¼ ðdv # 1Þðdv # 2Þ

X

x2Nv

ðdx # 1Þ; v 2 V:

We assign a weight Gð1Þv to each node v 2 V . Define

Gð1Þ ¼P

v2V Gð1Þv and rð1Þv ¼ G

ð1Þv

Gð1Þ. To sample a 5-node CIS, T-5

uses six steps:

Step 1. Sample a node v from V according to the distributionrð1Þ ¼ frð1Þv : v 2 V g;

Step 2. Sample a node u from Nv according to the distribu-tion sðvÞ ¼ fsðvÞ

u : u 2 Nvg, where sðvÞu is defined as

sðvÞu ¼ du # 1P

x2Nvðdx # 1Þ ; u 2 Nv;

TABLE 1Table of Notation

G ¼ ðV;EÞ G is an undirected graphNv the set of neighbors of a node v in Gdv dv ¼ jNvj, the cardinality of setNv

Gð5Þ1 ; . . . ; Gð5Þ

21 5-node graphlets

Gð5ÞðsÞ 5-node graphlet ID of CIS sCð5Þ the set of all 5-node CISes in G

Cð5Þ1 ; . . . ; Cð5Þ

21 Cð5Þi is the set of 5-node CISes in G

isomorphic to graphlet Gð5Þi , 1 ' i ' 21

h1; . . . ; h21 hi ¼ jCð5Þi j, 1 ' i ' 21

K1,K2 sampling budgets of T-5 and Path-5K ¼ K1 þK2 sampling budgets of MOSS-5

fð1Þi ; 1 ' i ' 21 the number of subgraphs in a CIS s 2 Cð5Þ

i

that are isomorphic to Gð5Þ3


i



i


Fig. 3. Overview of MOSS-5. MOSS-5 smartly combines two T-5 andPath-5 sampling methods, which are two novel sampling methods pro-posed in this paper.

2. http://nskeylab.xjtu.edu.cn/dataset/phwang/code/mosscode.zip

3. Two graphsG1 ¼ ðV1; E1Þ andG2 ¼ ðV2; E2Þ are said to be isomor-phic if there exists at least one bijection f : V1 ! V2 such that any twonodes u and v in V1 are adjacent in G1 if and only if fðuÞ and fðvÞ areadjacent in G2.

WANG ET AL.: MOSS-5: A FAST METHOD OF APPROXIMATING COUNTS OF 5-NODE GRAPHLETS IN LARGE GRAPHS 75

Step 3. Sample a node w from Nv # fug at random;Step 4. Sample a node r from Nv # fu;wg at random;Step 5. Sample a node t fromNu # fvg at random;Step 6.Return the CIS s that includes nodes v, u,w, r, and t.

Onemaywonder why not sample v from V and u fromNv

uniformly in the first two steps? This is because it is difficultto compute and remove the sampling bias of s when sam-pling v from V and u from Nv uniformly. In contrast, sam-pling v and u according to specific distributions rð1Þ and sðvÞ

leads to the sampling bias of T-5 that can be easily derivedand removed, which will be discussed later (Theorems 1 and2). We run the above procedureK1 times to obtainK1 CISessð1Þ1 ; . . . ; sð1ÞK1

. The pseudo-code of T-5 is shown inAlgorithm1.In Algorithm 1, functionWeightRandomVertexðV; rð1ÞÞ returnsa node sampled from V according to the distributionrð1Þ ¼ frð1Þv : v 2 V g, function RandomVertexðXÞ returns anode sampled from X at random, and function CISðfv; u;w; r; tgÞ returns the CIS with the node set fv; u; w; r; tg inG.

Algorithm 1. The Pseudo-Code of T-5

input: G ¼ ðV;EÞ andK1.output: hð1Þi .for i 2 V1 dohð1Þi 0;

endfor k 2 ½1;K1* dov WeightRandomVertexðV; rð1ÞÞ;u WeightRandomVertexðNv; sðvÞÞ;w RandomVertexðNv # fugÞ;r RandomVertexðNv # fu;wgÞ;t RandomVertexðNu # fvgÞ;sð1Þk CISðfv; u; w; r; tgÞ;if t 6¼ w and t 6¼ r thenz

i Gð5Þðsð1Þk Þ;hð1Þi h

ð1Þi þ 1

K1pð1Þi

;end

end

For a CIS s isomorphic to graphlet Gð5Þi , 1 ' i ' 21,

denote fð1Þi as the number of subgraphs in s that are isomor-

phic to graphlet Gð5Þ3 . In other word, s contains f

ð1Þi sub-

graphs that are isomorphic to Gð5Þ3 . The value of fð1Þ

i is given

in Table 2. The following theorem shows the sampling biasof T-5, which is critical for estimating hi.

Theorem 1. Using the sampling procedure once (i.e., K1 ¼ 1),T-5 returns a CIS s 2 Cð5Þ

i sampled with probability

pð1Þi ¼ 2fð1Þi

Gð1Þ ; 1 ' i ' 21:

Proof. As shown in Fig. 4, we can easily find that there existtwo ways to sample a subgraph isomorphic to graphlet

Gð5Þ3 by T-5. Each happens with probability

rð1Þv + sðvÞu + 1

dv # 1+ 1

dv # 2+ 1

du # 1¼ 1

Gð1Þ :

For a 5-node CIS s 2 Cð5Þi , s has f

ð1Þi different subgraphs

isomorphic to graphlet Gð5Þ3 , 1 ' i ' 21. Therefore, the

probability of T-5 sampling s is2f

ð1Þi

Gð1Þ. tu

We let Gð5ÞðsÞ be the 5-node graphlet ID of s when s is a5-node CIS, and -1 otherwise. We define

mð1Þi ¼

XK1

k¼1

1ðGð5Þðsð1Þk Þ ¼ iÞ:

It is easy to obtain the expectation ofmð1Þi as

Eðmð1Þi Þ ¼ K1p

ð1Þi hi:

For i 2 V1, pð1Þi is larger than zero and thus we estimate hi as

hð1Þi ¼ mð1Þ

i

K1pð1Þi

:

Theorem 2. For i 2 V1, hð1Þi is an unbiased estimator of hi, i.e.,

Eðhð1Þi Þ ¼ hð1Þi , and the variance of hð1Þi is

Varðhð1Þi Þ ¼ hiK1

1

pð1Þi

# hi

!

: (1)

We estimate Varðhð1Þi Þ by replacing hi with hð1Þi in (1). We also

compute the covariance of hð1Þi and hð1Þj as follows,

Covðhð1Þi ; hð1Þj Þ ¼ #hihjK1

; i 6¼ j; i; j 2 V1;

which is used to compute the variance of the estimate of h2given in Section 3.3.

Proof. For i 2 V1 and 1 ' k ' K1, we have

P ðGð5Þðsð1Þk Þ ¼ iÞ ¼X

s2Cð5Þ

P ðsð1Þk ¼ sÞ1ðGð5Þðsð1Þk Þ ¼ iÞ

¼ pð1Þi hi:

Since sð1Þ1 ; . . . ; sð1ÞK1are sampled independently, the ran-

dom variable mð1Þi follows the binomial distribution with

parameters K1 and pð1Þi hi. Then, the expectation and vari-ance ofmð1Þ

i are

TABLE 2

Values of fð1Þi , fð2Þ

i , and fð3Þi

i 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

fð1Þi 0 0 1 1 2 0 2 2 4 4 5 4 6 10 9 12 10 20 20 36 60

fð2Þi 1 0 0 2 2 5 1 0 4 7 2 4 6 10 6 6 14 24 18 36 60

fð3Þi 0 1 0 0 0 0 0 1 0 0 1 1 0 1 1 2 0 1 2 3 5

Fig. 4. The ways of T-5 sampling a subgraph s isomorphic to graphlet

Gð5Þ3 , where v, u, w, r, and t are the variables in Algorithm 1, i.e., the

nodes sampled at the 1st, 2nd, 3rd, 4th, and 5th steps, respectively.


Eðmð1Þi Þ ¼ K1p

ð1Þi hi;Varðmð1Þ

i Þ ¼ K1pð1Þi hið1# pð1Þi hiÞ:

Therefore, the expectation and variance of hð1Þi are com-puted as

Eðhð1Þi Þ ¼ Emð1Þ

i

K1pð1Þi

!

¼ Eðmð1Þi Þ

K1pð1Þi

¼ hi;

VarðhiÞ ¼ Varmð1Þ

i

K1pð1Þi

!

¼ hiK1

1

pð1Þi

# hi

!

:

tu

For i 6¼ j and i; j 2 V1, the covariance of hð1Þi and h

ð1Þj is

Covðhð1Þi ; hð1Þj Þ

¼ Covmð1Þ

i

K1pð1Þi

;mð1Þ

j

K1pð1Þj

!

¼Covð

PK1k¼1 1ðGð5Þðsð1Þk Þ ¼ iÞ;

PK1l¼1 1ðGð5Þðsð1Þl Þ ¼ jÞÞ

K21p

ð1Þi pð1Þj

¼PK1

k¼1

PK1l¼1 Covð1ðGð5Þðsð1Þk Þ ¼ iÞ; 1ðGð5Þðsð1Þl Þ ¼ jÞÞ

K21p

ð1Þi pð1Þj

:

Each sampled 5-node CIS is obtained independently, so wehave

Covð1ðGð5Þðsð1Þk Þ ¼ iÞ; 1ðGð5Þðsð1Þl Þ ¼ jÞÞ ¼ 0; k 6¼ l:

In addition, we have P ðGð5Þðsð1Þk Þ ¼ i ^Gð5Þðsð1Þk Þ ¼ jÞ ¼ 0when i 6¼ j. Therefore, we obtain

Covð1ðGð5Þðsð1Þk Þ ¼ iÞ; 1ðGð5Þðsð1Þk Þ ¼ jÞÞ

¼ Eð1ðGð5Þðsð1Þk Þ ¼ iÞ1ðGð5Þðsð1Þk Þ ¼ jÞÞ

# Eð1ðGð5Þðsð1Þk Þ ¼ iÞÞEð1ðGð5Þðsð1Þk Þ ¼ jÞÞ

¼ 0# pð1Þi hipð1Þj hj

¼ #pð1Þi pð1Þj hihj:

Now, we easily have

Covðhð1Þi ; hð1Þj Þ

¼PK1

k¼1 Covð1ðGð5Þðsð1Þk Þ ¼ iÞ; 1ðGð5Þðsð1Þk Þ ¼ jÞÞK2

1pð1Þi pð1Þj

¼ #hihjK1

:

3.2 The Path-5 Sampling MethodThe pseudo-code of Path-5 is shown in Algorithm 2. Let

Gð2Þv ¼

X

x2Nv

ðdx # 1Þ

!2

#X

x2Nv

ðdx # 1Þ2; v 2 V:

We assign a weight Gð2Þv to each node v 2 V . Define Gð2Þ ¼

Pv2V Gð2Þ

v and rð2Þv ¼ Gð2Þv

Gð2Þ. To sample a 5-node CIS, Path-5

mainly consists of six steps:

Step 1. Sample a node v from V according to the distributionrð2Þ ¼ frð2Þv : v 2 V g;

Step 2. Sample a node u from Nv according to the distribu-tion tðvÞ ¼ ftðvÞu : u 2 Nvg, where we define

tðvÞu ¼ðdu # 1Þð

Py2Nv#fugðdy # 1ÞÞGð2Þv

; u 2 Nv;

Step 3. Sample a nodew fromNv # fug according to the distri-butionmðv;uÞ ¼ mðv;uÞ

w : w 2 Nv # fug! "

, wherewe define

mðv;uÞw ¼ dw # 1P

y2Nv#fugðdy # 1Þ; w 2 Nv # fug;

Step 4. Sample a node r from Nu # fvg at random;Step 5. Sample a node t from Nw # fvg at random;Step 6. Return the CIS s that includes nodes v, u, w, r, and t.

Algorithm 2. The Pseudo-Code of Path-5

input: G ¼ ðV;EÞ andK2.output: hð2Þi .for i 2 V2 dohð2Þi 0;

endfor k 2 ½1;K2* dov WeightRandomVertexðV; rð2ÞÞ;u WeightRandomVertexðNv; tðvÞÞ;w WeightRandomVertexðNv # fug;mðv;uÞÞ;r RandomVertexðNu # fvgÞ;t RandomVertexðNw # fvgÞ;sð2Þk CISðfv; u; w; r; tgÞ;if t 6¼ u and r 6¼ w and t 6¼ r then

i Gð5Þðsð2Þk Þ;hð2Þi h

ð2Þi þ 1

K2pð2Þi

;end

end

We run the above procedure K2 times to obtain K2 CISes

sð2Þ1 ; . . . ; sð2ÞK2. For a CIS s isomorphic to graphlet Gð5Þ

i ,

1 ' i ' 21, let fð2Þi denote the number of subgraphs in s that

are isomorphic to Gð5Þ1 . In other word, s contains f

ð2Þi sub-

graphs that are isomorphic to Gð5Þ1 . The value of fð2Þ

i is given

in Table 2. The following theorem shows the sampling bias

of Path-5.

Theorem 3. Using the sampling procedure once (i.e., K2 ¼ 1),Path-5 samples a CIS s 2 Cð5Þ

i with probability

pð2Þi ¼ 2fð2Þi

Gð2Þ ; 1 ' i ' 21:

Proof. As shown in Fig. 5, we can see that there exist twoways to sample a subgraph isomorphic to graphlet Gð5Þ

1by Path-5. Each one happens with probability

rð2Þv + tðvÞu + mðv;uÞw + 1

du # 1+ 1

dw # 1¼ 1

Gð2Þ :

For a 5-node CIS s 2 Cð5Þi , s has f

ð2Þi subgraphs isomor-

phic to graphlet Gð5Þ1 , 1 ' i ' 21. Thus, the probability of

Path-5 sampling s is2f

ð2Þi

Gð2Þ. tu


Denote

mð2Þi ¼

XK2

k¼1

1ðGð5Þðsð2Þk Þ ¼ iÞ:

Then, we haveEðmð2Þ

i Þ ¼ K2pð2Þi hi:

For i 2 V2, pð2Þi is larger than zero and we then estimate hi as

hð2Þi ¼ mð2Þ

i

K2pð2Þi

:

Theorem 4. For i 2 V2, hð2Þi is an unbiased estimator of hi and

its variance is

Varðhð2Þi Þ ¼ hiK2

1

pð2Þi

# hi

!: (2)

We estimate Varðhð2Þi Þ by replacing hi with hð2Þi in (2). The

covariance of hð2Þi and hð2Þj is

Covðhð2Þi ; hð2Þj Þ ¼ #hihjK2

; i 6¼ j; i; j 2 V2;

which is used to compute the variance of the estimate of h2given in Section 3.3.

Its proof is similar to the proof of Theorem 2.

3.3 Hybrid Estimator of 5-Node Graphlet CountsWe estimate hi as h

ð1Þi and h

ð2Þi for i 2 V1 #V2 and i 2 V2#

V1 respectively. When i 2 V1 \V2, according to [29], weestimate hi based on its two unbiased estimates hð1Þi and h

ð2Þi .

Formally, let

!ð1Þi ¼ Varðhð2Þi Þ

Varðhð1Þi Þ þVarðhð2Þi Þ;!ð2Þ

i ¼ Varðhð1Þi ÞVarðhð1Þi Þ þVarðhð2Þi Þ

:

Here Varðhð1Þi Þ and Varðhð2Þi Þ are estimated by Theorems 2and 4. For i 2 V1 [V2 ¼ f1; 3; 4; 5; . . . ; 21g, we finally esti-mate hi as

hi ¼!ð1Þi h

ð1Þi þ !ð2Þ

i hð2Þi ; i 2 V1 \V2;

hð1Þi ; i 2 V1 #V2;

hð2Þi ; i 2 V2 #V1:

8>><

>>:(3)

Note that V1 [V2 ¼ f1; 2; . . . ; 21g# f2g. Next, we dis-cuss our method for estimating h2. For a CIS s isomorphic

to graphlet Gð5Þi , 1 ' i ' 21, let fð3Þ

i denote the number of

subgraphs in s that are isomorphic to graphlet Gð5Þ2 . In other

word, s contains fð3Þi subgraphs that are isomorphic to Gð5Þ

2 .

The value of fð3Þi is given in Table 2. Let

L4 ¼X

v2V

dv4

# $:

Then, the number of all 5-node subgraphs (not necessarily

induced) in G isomorphic to graphlet Gð5Þ2 is L4. Let

V3 ¼ fj : fð3Þj > 0g and V,

3 ¼ V3 # f2g. We observe that

X

i2V3

fð3Þi hi ¼ L4:

Since fð3Þ2 ¼ 1, we estimate h2 as

h2 ¼ L4 #X

i2V,3

fð3Þi hi:

Theorem 5. hi is an unbiased estimator of hi, 1 ' i ' 21. Fori 2 V1 [V2 ¼ f1; 2; . . . ; 21g# f2g, the variance of hi is

VarðhiÞ ¼

Varðhð1Þi ÞVarðhð2Þi Þ

Varðhð1Þi ÞþVarðhð2Þi Þ; i 2 V1 \V2;

Varðhð1Þi Þ; i 2 V1 #V2;

Varðhð2Þi Þ; i 2 V2 #V1:

8>>>><

>>>>:

(4)

In the above equation, we estimate Varðhð1Þi Þ and Varðhð2Þi Þusing the methods in Theorems 2 and 4. We compute the vari-ance Varðh2Þ ¼

X

i2V,3

ðfð3Þi Þ2VarðhiÞ þ

X

i;j2V,3;i 6¼j

fð3Þi f

ð3Þj Covðhi; hjÞ;

where Covðhi; hjÞ ¼

#P

l¼1;2

!ðlÞi !

ðlÞj hihj

Kl; i; j 2 V1 \V2;

#!ð1Þj hihj

K1; i 2 V1 #V2; j 2 V1 \V2;

# !ð1Þi hihjK1

; i 2 V1 \V2; j 2 V1 #V2;

# !ð2Þi hihjK2

; i 2 V1 \V2; j 2 V2 #V1;

#!ð2Þj hihj

K2; i 2 V2 #V1; j 2 V1 \V2;

0; i 2 V1 #V2; j 2 V2 #V1;

0; i 2 V2 #V1; j 2 V1 #V2:

8>>>>>>>>>>>>>>>>>>><

>>>>>>>>>>>>>>>>>>>:

Proof. For i 2 V1 [V2, Theorems 2 and 4 tell us that hð1Þi

and hð2Þi are unbiased estimators of hð1Þi , and they are inde-

pendent. Moreover, !ð1Þi þ !ð2Þ

i ¼ 1. Therefore, we easily

find that hi is also an unbiased estimator of hð1Þi , and its

variance is (4). Next, we study the expectation and vari-

ance of h2. The expectation of h2 is computed as

Eðh2Þ ¼ L4 #X

i2V,3

fð3Þi EðhiÞ ¼ L4 #

X

i2V,3

fð3Þi hi ¼ h2:

Before we compute the covariance of hi and hj for

i; j 2 V1 [V2 and i 6¼ j, we first introduce three equa-

tions: (I) for any i; j 2 V1 [V2, we have Covðhð1Þi ; hð2Þj Þ ¼ 0

Fig. 5. The ways of Path-5 sampling a subgraph s isomorphic to graphlet

Gð5Þ1 , where v, u, w, r, and t are the variables in Algorithm 2, i.e., the

nodes sampled at the 1st, 2nd, 3rd, 4th, and 5th steps, respectively.


because hð1Þi and h

ð2Þj are independent; (II) from Theo-

rem 2, we have Covðhð1Þi ; hð1Þj Þ ¼ # hihjK1

, i 6¼ j and i; j 2 V1;

(III) from Theorem 4, we have Covðhð2Þi ; hð2Þj Þ ¼ # hihjK2

,

i 6¼ j and i; j 2 V2. Based on these three equations and

eq. (3), we compute hi and hj as follows:

! When i 2 V1 #V2 and j 2 V2 #V1, we have

Covðhi; hjÞ ¼ Covðhð1Þi ; hð2Þj Þ ¼ 0.! When i 2 V2 #V1 and j 2 V1 #V2, we have

Covðhi; hjÞ ¼ Covðhð2Þi ; hð1Þj Þ ¼ 0.! When i 2 V1 #V2 and j 2 V1 \V2, we have

Covðhi; hjÞ ¼ Covðhð1Þi ;!ð1Þj h

ð1Þj þ !ð2Þ

j hð2Þj Þ ¼ #

!ð1Þj hihjK1

:

! When i 2 V1 \V2 and j 2 V1 #V2, we have

Covðhi; hjÞ ¼ Covð!ð1Þi h

ð1Þi þ !ð2Þ

i hð2Þi ; hð1Þj Þ ¼ #

!ð1Þi hihjK1

:

! When i 2 V1 \V2 and j 2 V2 #V1, we have


ð1Þi þ !ð2Þ

i hð2Þi ; hð2Þj Þ ¼ #

!ð2Þi hihjK2

:

! When i 2 V2 #V1 and j 2 V1 \V2, we have

Covðhi; hjÞ ¼ Covðhð2Þi ;!ð1Þj h

ð1Þj þ !ð2Þ

j hð2Þj Þ ¼ #

!ð2Þj hihjK2

:

! When i; j 2 V1 \V2 and i 6¼ j, we have


ð1Þi þ !ð2Þ

i hð2Þi ;!ð1Þ

j hð1Þj þ !ð2Þ

j hð2Þj Þ

¼ #hihj!ð1Þi !ð1Þ

j

K1þ!ð2Þi !ð2Þ

j

K2

!:

Finally, we have Varðh2Þ ¼ VarðL4 #P

i2V,3fð3Þi hiÞ ¼

Pi2V,

3ðfð3Þ

i Þ2VarðhiÞ þP

i;j2V,3;i 6¼j f

ð3Þi f

ð3Þj Covðhi;

hjÞ. tu

3.4 Implementation and ComplexitiesIn this subsection, we introduce our methods of implement-ing the functions in Algorithms 1 and 2. We also analyzetheir time and space complexities.

Function Gð5ÞðsÞ. Let A be the adjacent matrix of s. Wecompute a bit string by concatenating all elements abovethe main diagonal of A, that is,

str ¼ A1;2jj . . . jjA1;5jjA2;3jj . . . jjA2;5jjA3;4jj . . . jjA3;5jjA4;5:

Then, we compute Gð5ÞðsÞ by looking up key str in hashtable HT . Therefore, the average computational complexityof function Gð5ÞðsÞ is Oð1Þ. Hash table HT is generated inadvance as: For each graphlet Gð5Þ

j , 1 ' j ' 13, we computea key str for each permutation of nodes in GðkÞ

j , and thenstore key str and its value j in hash table HT , i.e.,HT ½str* ¼ j. There exist 5! ¼ 120 different permutations for5 nodes. Therefore, it is memory and computationally effi-cient to generateHT in advance.

Functions Gð1Þv and Gð2Þ

v . For each node v, we store itsdegree dv and use a list to store its neighbors’ degrees.

Clearly, it requires OðdvÞ operations to compute Gð1Þv and

Gð2Þv . Therefore, the space and time complexities of process-

ing all nodes are both OðjEjÞ.WeightRandomVertexðV;rð1ÞÞ. We use a list V ½1; . . . ; jV j*

to store the nodes in V . We store an array ACC Gð1Þ½1;. . . ; jV j* in memory, where ACC Gð1Þ½i* is defined as

ACC Gð1Þ½i* ¼Pi

j¼1 Gð1ÞV ½j*, 1 ' i ' jV j. Note that ACC Gð1Þ

½jV j* ¼ Gð1Þ. Let ACC Gð1Þ½0* ¼ 0. Then, WeightRandom

VertexðV;rð1ÞÞ is easily achieved by the following threesteps: step 1) select a number rnd from f1; . . . ;Gð1Þg atrandom; step 2) find i such that ACC Gð1Þ½i# 1* <rnd ' ACC Gð1Þ½i*, which is solved by binary search; step3) return V ½i*. Thus, the space and time complexities ofWeightRandomVertexðV;rð1ÞÞ are OðjV jÞ and Oðlog jV jÞrespectively.

WeightRandomVertexðV; rð2ÞÞ. It is achieved similarly tothat forWeightRandomVertexðV; rð1ÞÞ.

WeightRandomVertexðNv; sðvÞÞ. We use a list Nv½1; . . . ; dv*to store the neighbors of v. We store an array ACC sðvÞ½1;. . . ; dv* in memory, where ACC sðvÞ½i* is defined as

ACC sðvÞ½i* ¼Pi

j¼1ðdNv½j* # 1Þ, 1 ' i ' dv. Let ACC sðvÞ½0* ¼0. Then, WeightRandomVertexðNv; sðvÞÞ is easily achieved

by the following three steps: step 1) select a number rnd

from f1; . . . ; ACC sðvÞ½dv*g at random; step 2) find i such that

ACC sðvÞ½i# 1* < rnd ' ACC sðvÞ½i*, which is solved by

binary search; step 3) return Nv½i*. Thus, the space and time

complexities of WeightRandomVertexðNv; sðvÞÞ are OðdvÞand Oðlog dvÞ respectively.

WeightRandomVertexðNv; tðvÞÞ. It is achieved similarly tothat forWeightRandomVertexðNv; sðvÞÞ.

WeightRandomVertexðNv # fug;mðv;uÞÞ. As mentioned,

we use a list Nv½1; . . . ; dv* to store the neighbors of v, and

store an array ACC sðvÞ½1; . . . ; dv* in memory, where

ACC sðvÞ½i* ¼Pi

j¼1ðdNv½j* # 1Þ, 1 ' i ' dv. Let POSv;u be the

index of u in Nv½1; . . . ; dv*, i.e., Nv½POSv;u* ¼ u. Then, func-

tion WeightRandomVertexðNv # fug;mðv;uÞÞ is easily achieved

by the following three steps: step 1) select a random num-

ber rnd from 1; . . . ; ACC sðvÞ½dv*! "

# ACC sðvÞ½POSv;u # 1* þ!

1; . . . ; ACC sðvÞ½POSv;u*g; step 2) find i such that ACC sðvÞ

½i# 1* < rnd ' ACC sðvÞ½i*, which is solved by binary

search; step 3) return Nv½i*. Therefore, the time complexity

of WeightRandomVertexðNv # fug;mðv;uÞÞ is Oðlog dvÞ.RandomVertexðNv # fugÞ. Function RandomVertexðNv#

fugÞ is achieved by two steps: step 1) select a number rndfrom f1; . . . ; dvg# fPOSv;ug at random; step 2) returnNv½rnd*. Thus, the computational complexity of RandomVertexðNv # fugÞ is Oð1Þ.

RandomVertexðNv # fu;wgÞ. It is achieved by two steps:step 1) select a number rnd from f1; . . . ; dvg# fPOSv;u;POSv;wg at random; step 2) return Nv½rnd*. Thus, the compu-tational complexity of RandomVertexðNv # fu;wgÞ is Oð1Þ.

In summary, the space and time complexities of T-5 sam-pling K1 CISes are OðjV jþ jEjÞ and OðjEjþK1log jV jÞrespectively, and the space and time complexities of Path-5sampling K2 CISes are OðjV jþ jEjÞ and OðjEjþK2log jV jÞrespectively. Therefore, the space and time complexities ofMOSS-5 are OðjV jþ jEjÞ and OðjEjþ ðK1 þK2Þlog jV jÞrespectively.


3.5 Parameter SettingsFrom Theorem 5, we can see that the variance of hi greatlydepends on the sampling budgetK1 for i 2 V1 #V2. In con-trast, K2 is used to guarantee the desired variance of hi,i 2 V2 #V1. Thus, K1 and K2 can be set according to theabove observations. Given a total sampling budget K (i.e.,K ¼ K1 þK2), how to set K1 and K2? Our empirical studyshows that pð1Þi and pð2Þi have similar values. Therefore, T-5and Path-5 exhibit similar estimation errors when K1 ¼ K2

and we setK1 ¼ K2 ¼ K2 in this paper for simplicity.

4 EVALUATION

4.1 DatasetsWe perform our experiments on a variety of publicly avail-able graph datasets ranged from 0.1 to 117 million edgestaken from the Stanford Network Analysis Platform(SNAP),4 which are summarized in Table 3. We use thestate-of-the-art method ESCAPE [28] to exactly compute 5-node graphlet counts h1; . . . ; h21 for all these graphs. Fig. 6shows the real values of h1; . . . ; h21.

4.2 MetricWe study the Normalized Root Mean Square Error(NRMSE) to measure the relative error of the graphlet countestimate hi with respect to its true value hi, i ¼ 1; . . . ; 21. Itis defined as

NRMSEðhiÞ ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiMSEðhiÞ

p

hi; i ¼ 1; ; 21;

whereMSEðhiÞ denotes the mean square error of hi, i.e.,

MSEðhiÞ ¼ Eððhi # hiÞ2Þ ¼ VarðhiÞ þ EðhiÞ # hið Þ2:

We can see thatMSEðhiÞ decomposes into a sum of the vari-ance and bias of the estimator hi. Both quantities are impor-tant and need to be as small as possible to achieve a goodestimation performance. When hi is an unbiased estimatorof hi, we have MSEðhiÞ ¼ VarðhiÞ and then NRMSEðhiÞ isequivalent to the normalized standard error of hi, i.e.,NRMSEðhiÞ ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiVarðhiÞ

p=hi. In addition, we define jhi#hij

hias

the relative error of hi, and also study the CCDF (comple-mentary cumulative distribution function) of jhi#hij

hi, that is,

CCDFjhi # hij

hi; x

# $¼ P

jhi # hijhi

> x

# $:

In our experiments, we calculate the NRMSE and CCDFover 1,000 runs. Our experiments are conducted on a serverwith a Quad-Core AMD Opeteron (tm) 8379 HE CPU 2.39GHz processor and 128 GB DRAMmemory.

4.3 RuntimeTable 4 shows the computational time of MOSS-5. We cansee that MOSS-5 is quite computationally efficient, whichtakes less than 200 seconds to sample 10 million 5-nodeCISes for all graphs studied in this paper. We observe thatthe sampling budget K does not offer a strictly linearincrease in running time. This is because the time cost ofcomputing Gð1Þ

v and Gð2Þv cannot be neglected for all nodes

v 2 V , which equals 68, 30, 20, 10, 9, 4, 3, 1, and 0.05 secondsfor graphs com-Orkut, Livejournal, Pokec, Filckr, Xiami,Wiki-Talk, Web-Google, YouTube, and ca-HepPh respec-tively. Moreover, we observe that sampling large graphssuch as com-Orkut is computationally expensive than sam-pling small graphs such as ca-HepPh. From the experimen-tal results in Section 4.5 (Table 5), we show that MOSS-5requires less than 2 minutes to compute 5-node graphletcounts with NRMSEs smaller than 0.1 for graphs with mil-lions of nodes and edges.

4.4 AccuracyFig. 7 shows NRMSEs of MOSS-5 with sampling budgetK ¼ 100; 000; 1; 000; 000; 10; 000; 000. For all graphs, most 5-node graphlets’ NRMSEs are smaller than 0.1 whenK ¼ 100; 000, and all 5-node graphlets’ NRMSEs are smallerthan 0.1 when K ¼ 10; 000; 000. In addition, we observe thatNRMSEs are proportional to 1ffiffiffiffi

Kp , which is consistent with

Theorem 5. For example, in Fig. 7 we see that a one orderof magnitude increase in K decreases NRMSEs by 1ffiffiffiffi

10p .

Fig. 8 shows the CCDF of relative error jhi#hijhi

, 1 ' i ' 21,

given by MOSS-5 with sampling budget K ¼ 1; 000; 000.We can see that more than 99 percent of estimates hiobtained by 1,000 independent runs have a relative errorsmaller than 3NRMSEðhiÞ. From Figs. 6, 7 and 8, weobserve that MOSS-5 exhibits smaller estimation errorsfor graphlets with large graphlet counts (i.e., frequentgraphplets) than graphlets with small graphlet counts(i.e., rare graphlets).

TABLE 3Undirected Graph Datasets Used in Our Experiments. Where“max-degree” Represents the Maximum Number of Edges

Incident to a Node in the Undirected Graph

Graph nodes edges max-degree

com-Orkut [30] 3,072,441 117,185,803 33,313LiveJournal [31] 5,189,809 48,688,097 15,017Pokec [32] 1,632,803 22,301,964 14,854Flickr [31] 1,715,255 15,555,041 27,236Xiami [33] 1,753,620 16,018,571 19,727Wiki-Talk [34] 2,394,385 4,659,565 100,029Web-Google [35] 875,713 4,322,051 6,332YouTube [31] 1,138,499 2,990,443 28,754ca-HepPh [36] 12,008 118,490 491

TABLE 4Computational Time (Seconds) of MOSS-5

Graph sampling budgetK

100,000 1,000,000 10,000,000

com-Orkut 69.1 s 80.4 s 193.8 sLiveJournal 31.5 s 40.4 s 128.7 sPokec 20.7 s 30.2 s 125.2 sFlickr 10.4 s 19.6 s 111.6 sXiami 10.4 s 18.6 s 99.3 sWiki-Talk 4.6 s 12.9 s 95.7 sWeb-Google 4.1 s 10.9 s 79.3 sYouTube 2.5 s 9.4 s 78.6 sca-HepPh 0.53 s 4.8 s 46.3 s

4. www.snap.stanford.edu


4.5 Comparison with Prior ArtMOSS versus Fast Exact Counting Method ESCAPE [28].Table 5 shows the expected smallest computational time ofMOSS-5 required to obtain all estimates h1; . . . ; h21 withNRMSE smaller than 0.1. To compute h1; . . . ; h21, the state-of-the-art exact computing method ESCAPE requires52 hours, 32 hours, 23 hours, 1 hour, 31 minutes, 10 minutes,8 minutes, 3 minutes, and 2 minutes for graphs Flickr, com-Orkut, LiveJournal, Pokec, Wiki-Talk, ca-HepPh, Xiami,YouTube, and Web-Google respectively. We can see thatthe computational time of ESCAPE does not strictly increasewith the graph size. For example, graph ca-HepPh is morethan ten times smaller than graphs YouTube and Web-Google. To compute h1; . . . ; h21, however, ESCAPE requiresmuch more time for ca-HepPh than for YouTube and Web-Google. From Table 5, we see that our method MOSS-5 is 2to 18,945 times faster than ESCAPE when providing accu-rate estimates with NRMSE smaller than 0.1. From theresults in Section 4.4 (Fig. 7), we observe that when the max-imum of NRMSEs of all graphlets’ estimates equals 0.1,NRMSEs of many graphlets’ estimates are much smaller

than 0.1. In Table 5, we show the average NRMSE121

P21i¼1 NRMSEðhiÞ when maxi¼1;...;21NRMSEðhiÞ ¼ 0:1. We

can see that the average NRMSE varies from 0.01 to 0.04 forall graphs studied in this paper.

Fig. 6. Real values of 5-node graphlet counts.

TABLE 5Computational Time (Seconds) and Accuracy

of MOSS-5 in Comparison with State-of-the-Art ExactCounting Method ESCAPE

Graph ESCAPE MOSS-5, maxi¼1;...;21NRMSEðhiÞ ¼ 0:1

(time) time 121

P21i¼1 NRMSEðhiÞ

Flickr 189,450 s 10 s 0.039

com-Orkut 116,029 s 103 s 0.015

LiveJournal 82,445 s 31 s 0.037

Pokec 3,696 s 31 s 0.024

Wiki-Talk 1,877 s 47 s 0.018

Xiami 518 s 82 s 0.013

Web-Google 112 s 25 s 0.013

YouTube 193 s 96 s 0.011

ca-HepPh 589 s 64 s 0.011


MOSS versus Sampling Methods Guise [37] and Graft [38].Most previous work focuses on estimating 5-node motifconcentrations, i.e., vi ¼ hiP21

j¼1hj, i ¼ 1; . . . ; 21. We run MOSS-

5 and the state-of-the-art sampling methods Guise [37] andGraft [38] on all above graphs and increase their samplingbudgets until the estimation errors of motif concentrationsare within 10 percent. Guise uses a Metropolis-HastingsRandom Walk (MHRW) method to uniformly sample CISesfrom all 3-, 4-, and 5-node CISes. To conduct a fair compari-son, we adapt GUISE to focus on 5-node CISes similarlyto [39]. Graft samples a fraction of edges from G at random,and then enumerates all 5-node CISes that include at leastone edge in the set of sampled edges. In practice, it is noteasy to obtain an estimation with a desired accuracy forGuise [37] and Graft [38]. In our experiments, we increasetheir sampling budgets until the relative errors of their

estimates are no more than 10 percent with respect to thereal values of all graphlet concentrations. Fig. 9 shows theruntime of Graft and Guise normalized with respect to theruntime of MOSS-5 (i.e., the runtime of MOSS-5 of unit 1).We can see that our method MOSS-5 is 2 to 3 orders of mag-nitude faster than Graft and Guise.

5 RELATED WORK

In this paper, we study the problem of estimating the countsof 3-, 4-, and 5-node graphlets for a single large graph, whichis much different from the problem of computing the num-ber of subgraph patterns appearing in a large set ofgraphs [40]. A variety of centralized and distributed algo-rithms have been developed to enumerate and count all tri-angles in large undirected graphs [41], [42], [43], [44], [45].Recently, a considerable attention has been given to

Fig. 7. NRMSEs of estimates of 5-node graphlet counts hi, 1 ' i ' 21, given by MOSS-5 with sampling budgetK ¼ 105; 106; 107.


Fig. 9. Runtime of the-state-of-art methods normalized with respect to runtimes of MOSS-5 for estimating 5-node graphlet concentrations.

Fig. 8. (Flickr) CCDFðjhi # hij=hi; xÞ, 1 ' i ' 21, given by MOSS-5 with sampling budgetK ¼ 1;000;000.


designing fast algorithms for counting higher order sub-graph patterns such as 4- and 5-node graphlets. [28], [44],[45], [46], [47] develop fast algorithms for counting 4- and 5-node undirected graphlets by utilizing the relationshipsbetween 3-, 4-, and 5-node graphlet counts. In addition, quitea few efforts have been devoted to designing samplingmeth-ods for computing a large graph’s graphlet concentrations(or, motif concentrations) [37], [38], [39], [48], [49], [50], [51],but they fail to compute graphlet counts. Alon et al. [52] pro-pose a color-coding method to reduce the computational costof counting subgraphs. Color-coding reduces the computa-tion by coloring nodes randomly and enumerating only col-orful CISes (i.e., CISes that consist of nodes with distinctcolors), but [27] reveals that the color-coding method is notscalable and is hindered by the sheer number of colorfulCISes. [21], [22], [23], [24] develop sampling methods to esti-mate the number of triangles of static and dynamic graphs.Jha et al. [27] develop sampling methods to estimate countsof 4-node graphlets. Thesemethods cannot be used to sampleand estimate 5-node grahplet counts. When the graph ofinterest is not available but given a RESampled graph thatis obtained by sampling each edge with a fixed probability,Minfer [53] aims to infer the underlying graph’s graphletconcentrations from the RESampled graph. Moreover, [39],[51], [54] assume that the graph of interest is not given inadvance, and they focus on designing crawling methods toquery as less nodes as possible to characterize graphlets. Inthis paper, we assume that the entire graph of interest isgiven in advance, and aim to design a fast samplingmethod to reduce the time of computing graphlet counts.When the entire graph is given in advance, Minfer [53]and crawling methods in [39], [51], [54] exhibit muchlarger errors than sampling methods such as our methodMOSS-5 (aim to estimate 5-node graphlet counts), 3-pathsampling [27] (aim to estimate 4-node graphlet counts),and wedge sampling [25] (aim to estimate 3-node graphletcounts) under the same computational time. In contrary,these sampling methods require the statistics of all nodesand edges such as degree, so they cannot be used to esti-mate graphlet counts when the entire graph is not given inadvance. In addition, [55], [56] accelerate the speed ofexactly counting graphlets, and [47] develops a parallelalgorithm to exactly count 3- and 4-node graphlets.

6 CONCLUSIONS AND FUTURE WORK

We develop a computationally efficient sampling methodMOSS-5 to estimate the counts of 5-node graphlets in a largegraph. We provide unbiased estimators of 5-node graphletcounts, and derive simple yet exact formulas for the varian-ces of the estimators. Meanwhile, we conduct experimentson a variety of publicly available datasets, and experimentalresults demonstrate that our method significantly outper-forms the state-of-the-art methods. In future, we plan toextend MOSS-5 on parallel and distributed computing sys-tems for greater scalability.

ACKNOWLEDGMENTS

The authors wish to thank the anonymous reviewers for theirhelpful feedback. In addition, the authors also wish to thankMr. Yiyan Qi and Miss Xiaotong Ren for discussions. Thiswork was supported in part by Army Research Office

Contract W911NF-12-1-0385, and ARL under CooperativeAgreement W911NF-09-2-0053. The views and conclusionscontained in this document are those of the authors andshould not be interpreted as representing the official policies,either expressed or implied of the ARL, or the U.S. Govern-ment. The research presented in this paper is supported inpart by the National Natural Science Foundation of China(U1301254, 61603290, 61602371), the Ministry of Education &China Mobile Research Fund (MCM20160311), the NaturalScience Foundation of Jiangsu Province (SBK2014021758),111 International Collaboration Program of China, theProspective Joint Research of Industry-Academia-ResearchJoint Innovation Funding of Jiangsu Province (BY2014074),Shenzhen Basic Research Grant (JCYJ20160229195940462),the China Postdoctoral Science Foundation (2015M582663),and the Natural Science Basic Research Plan in Shaanxi Prov-ince of China (2016JQ6034). Junzhou Zhao is the correspond-ing author.

REFERENCES

[1] R. Milo, E. Al, and C. Biology, “Network motifs: Simple buildingblocks of complex networks,” Sci., vol. 298, no. 5549, pp. 824–827,October 2002.

[2] S. S. Shen-Orr, R. Milo, S. Mangan, and U. Alon, “Network motifsin the transcriptional regulation network of escherichia coli,”Nature Genetics, vol. 31, no. 1, pp. 64–68, May 2002.

[3] H. Chun, Y.-Y. Ahn, H. Kwak, S. Moon, Y. ho Eom, and H. Jeong,“Comparison of online social relations in terms of volume versusinteraction: A case study of cyworld,” in Proc. 8th ACMSIGCOMM Conf. Internet Meas., Nov. 2008, pp. 57–59.

[4] J. Kunegis, A. Lommatzsch, and C. Bauckhage, “The slashdot zoo:Mining a social network with negative edges,” in Proc. 18th Int.Conf. World Wide Web, Apr. 2009, pp. 741–750.

[5] J. Zhao, J. C. S. Lui, D. Towsley, X. Guan, and Y. Zhou, “Empiricalanalysis of the evolution of follower network: A case study ondouban,” in Proc. NetSciCom, April 2011, pp. 941–946.

[6] J. Ugander, L. Backstrom, and J. Kleinberg, “Subgraph frequen-cies: Mapping the empirical and extremal geography of largegraph collections,” in Proc. 22nd Int. Conf. World Wide Web, 2013,pp. 1307–1318.

[7] Y. Jin, E. Sharafuddin, and Z.-L. Zhang, “Unveiling core network-wide communication patterns through application traffic activitygraph decomposition,” in Proc. 11th Int. Joint Conf. Meas. ModelingComput. Syst., 2009, pp. 49–60.

[8] M. Iliofotou, M. Faloutsos, and M. Mitzenmacher, “Exploitingdynamicity in graph-based traffic analysis: Techniques andapplications,” in Proc. 5th Int. Conf. Emerging Netw. Exp. Technol.,2009, pp. 241–252.

[9] N. Shervashidze, S. V. N. Vishwanathan, T. Petri, K. Mehlhorn, andK. M. Borgwardt, “Efficient graphlet kernels for large graphcomparison,” inProc. 12th Int. Conf. Artif. Intell. Stat., 2009, pp. 488–495.

[10] N. Przulj, D. G. Corneil, and I. Jurisica, “Modeling interactome:Scale-free or geometric?”Bioinf., vol. 20, no. 18, pp. 3508–3515, 2004.

[11] P. D. Dobson and A. J. Doig, “Distinguishing enzyme structuresfrom non-enzymes without alignments,” J. Mol. Biol., vol. 330,no. 4, pp. 771–783, 2003.

[12] S. V. N. Vishwanathan, N. N. Schraudolph, R. Kondor, andK. M. Borgwardt, “Graph kernels,” J. Mach. Learn. Res., vol. 11,pp. 1201–1242, 2010.

[13] N. Wale, I. A. Watson, and G. Karypis, “Comparison of descriptorspaces for chemical compound retrieval and classification,” inProc. Int. Conf. Data Min., 2006, pp. 678–689.

[14] P. Yanardag and S. Vishwanathan, “A submodular frameworkfor graph comparison,” in Proc. NIPS Workshop on Networks, 2015,pp. 1–7.

[15] P. Yanardag and S. Vishwanathan, “Deep graph kernels,” in Proc.21th ACM SIGKDD Int. Conf. Knowl. Discovery Data Min., 2015,pp. 1365–1374.

[16] K. Chen, et al., “Finding unknown malice in 10 seconds: Mass vet-ting for new threats at the Google-play scale,” in Proc. 24th USE-NIX Conf. Secur. Symp., 2015, pp. 659–674.


[17] T. Shen, Y. Zhongyang, Z. Xin, B. Mao, and H. Huang, “Detectandroid malware variants using component based topologygraph,” in Proc. IEEE 13th Int. Conf. Trust, Secur. Privacy Comput.Commun., 2014, pp. 406–413.

[18] H. Gascon, F. Yamaguchi, D. Arp, and K. Rieck, “Structural detec-tion of android malware using embedded call graphs,” in Proc.ACMWorkshop Artif. Intell. Secur., 2013, pp. 45–54.

[19] J. Leskovec, K. J. Lang, A. Dasgupta, and M. W. Mahoney,“Community structure in large networks: Natural cluster sizesand the absence of large well-defined clusters,” Internet Math.,vol. 6, no. 1, pp. 29–123, 2009.

[20] M. Richardson, R. Agrawal, and P. Domingos, “Trust manage-ment for the semantic web,” in Proc. 2nd Int. Semantic Web Conf.,Oct. 2003, pp. 351–368.

[21] C. E. Tsourakakis, U. Kang, G. L. Miller, and C. Faloutsos,“Doulion: Counting triangles in massive graphs with a coin,” inProc. 15th ACM SIGKDD Int. Conf. Knowl. Discovery Data Min.,2009, pp. 837–846.

[22] A. Pavany, K. Tangwongsan, S. Tirthapuraz, and K.-L. Wu,“Counting and sampling triangles from a graph stream,” Proc.VLDB Endowment, vol. 6, no. 14, pp. 1870–1881, 2013.

[23] M. Jha, C. Seshadhri, and A. Pinar, “A space efficient streamingalgorithm for triangle counting using the birthday paradox,” inProc. 19th ACM SIGKDD Int. Conf. Knowl. Discovery Data Min.,2013, pp. 589–597.

[24] N. Ahmed, N. Duffield, J. Neville, and R. Kompella, “Graph sam-ple and hold: A framework for big-graph analytics,” in Proc. 20thACM SIGKDD Int. Conf. Knowl. Discovery Data Min., 2014,pp. 1446–1455.

[25] C. Seshadhri, A. Pinar, and T. G. Kolda, “Wedge sampling forcomputing clustering coefficients and triangle counts on largegraphs,” Stat. Anal. Data Min., vol. 7, no. 4, pp. 294–307, 2014.

[26] B. Wu, K. Yi, and Z. Li, “Counting triangles in large graphs byrandom sampling,” IEEE Trans. Knowl. Data Eng., vol. 28, no. 8,pp. 2013–2026, Aug. 2016.

[27] M. Jha, C. Seshadhri, and A. Pinar, “Path sampling: A fast andprovable method for estimating 4-vertex subgraph counts,” inProc. World Wide Web, 2015, pp. 495–505.

[28] A. Pinar, C. Seshadhr, and V. Visha, “Escape: Efficiently counting all5-vertex subgraphs,” inProc.WorldWideWeb, 2017, pp. 1431–1440.

[29] F. A. Graybill and R. B. Deal, “Combining unbiased estimators,”Biometrics, vol. 15, no. 4, pp. 543–550, Dec. 1959.

[30] J. Yang and J. Leskovec, “Defining and evaluating network com-munities based on ground-truth,” in Proc. IEEE 12th Int. Conf.Data Min., Dec. 2012, pp. 745–754.

[31] A. Mislove, M. Marcon, K. P. Gummadi, P. Druschel, and B.Bhattacharjee, “Measurement and analysis of online social networks,”inProc. 7th ACMSIGCOMMConf. Int.Meas., Oct. 2007, pp. 29–42.

[32] L. Takac and M. Zabovsky, “Data analysis in public socialnetworks.” in Proc. Int. Sci. Conf. Int. Workshop Present Day TrendsInnovations, May 2012, pp. 1–6.

[33] P. Wang, J. Zhao, J. C. S. Lui, D. Towsley, and X. Guan, “Samplingcontent distributed over graphs,” CoRR, vol. abs/1311.3882,pp. 1–14, 2013.

[34] J. Leskovec, D. Huttenlocher, and J. Kleinberg, “Predicting posi-tive and negative links in online social networks,” in Proc. 19thInt. Conf. World Wide Web, Apr. 2010, pp. 641–650.

[35] “Google programming contest,” 2002. [Online]. Available: http://www.google.com/programming-contest/

[36] J. Leskovec, J. Kleinberg, and C. Faloutsos, “Graph evolution:Densification and shrinking diameters,” ACM Trans. Knowl. Dis-covery Data, vol. 1, no. 1, Mar. 2007.

[37] M. A. Bhuiyan, M. Rahman, M. Rahman, and M. A. Hasan,“Guise: Uniform sampling of graphlets for large graph analysis,”in Proc. IEEE 12th Int. Conf. Data Min., Dec. 2012, pp. 91–100.

[38] M. Rahman, M. Bhuiyan, and M. A. Hasan, “Graft: An approxi-mate graphlet counting algorithm for large graph analysis,” IEEETrans. Knowl. Data Eng., vol. 26, no. 1, pp. 2466–2478, Oct. 2014.

[39] P. Wang, J. C. Lui, J. Zhao, B. Ribeiro, D. Towsley, and X. Guan,“Efficiently estimating motif statistics of large networks,” ACMTrans. Knowl. Discovery Data, 2014, Art. no. 8.

[40] M. A. Hasan and M. J. Zaki, “Output space sampling for graphpatterns,” Proc. VLDB Endowment, vol. 2, no. 1, pp. 730–741,Aug. 2009.

[41] S. Chu and J. Cheng, “Triangle listing in massive networks and itsapplications,” in Proc. 17th ACM SIGKDD Int. Conf. Knowl. Discov-ery Data Min., Aug. 2011, pp. 672–680.

[42] S. Suri and S. Vassilvitskii, “Counting triangles and the curseof the last reducer,” in Proc. Int. Conf. World Wide Web, 2011,pp. 607–614.

[43] T. Schank, “Algorithmic aspects of triangle-based networkanalysis,” Ph.D. dissertation, der Fakultat€a f€ur Informatik,der Universit€at Fridericiana zu Karlsruhe, Karlsruhe, Germany,2007.

[44] E. R. Elenberg, K. Shanmugam, M. Borokhovich, and A. G.Dimakis, “Beyond triangles: A distributed framework for estimat-ing 3-profiles of large graphs,” in Proc. KDD, 2015, pp. 229–238.

[45] E. R. Elenberg, K. Shanmugam, M. Borokhovich, and A. G.Dimakis, “Distributed estimation of graph 4-profiles,” in Proc.25th Int. Conf. World Wide Web, 2016, pp. 483–493.

[46] D. Marcus and Y. Shavitt, “Rage–A rapid graphlet enumerator forlarge networks,” Comput. Netw., vol. 56, no. 2, pp. 810–819, 2012.

[47] N. K. Ahmed, J. Neville, R. A. Rossi, and N. G. Duffield, “Efficientcounting for large networks,” in Proc. IEEE Int. Conf.Data Min.,2015, pp. 1–10.

[48] N. Kashtan, S. Itzkovitz, R. Milo, and U. Alon, “Efficient sam-pling algorithm for estimating subgraph concentrations anddetecting network motifs,” Bioinf., vol. 20, no. 11, pp. 1746–1758, 2004.

[49] S. Wernicke, “Efficient detection of network motifs,” IEEE/ACMTrans. Comput. Biol. Bioinf., vol. 3, no. 4, pp. 347–359, Oct.-Dec. 2006.

[50] S. Omidi, F. Schreiber, and A. Masoudi-nejad, “Moda: An efficientalgorithm for network motif discovery in biological networks,”Genes Genet Syst., vol. 84, no. 5, pp. 385–395, 2009.

[51] X. Chen, Y. Li, P. Wang, and J. C. Lui, “A general framework forestimating graphlet statistics via random walk,” Proc. VLDBEndowment, vol. 10, no. 3, pp. 253–264, 2016.

[52] N. Alon, R. Yuster, and U. Zwick, “Color-coding,” J. ACM, vol. 42,no. 4, pp. 844–856, Jul. 1995. [Online]. Available: http://doi.acm.org/10.1145/210332.210337

[53] P. Wang, J. C. S. Lui, D. F. Towsley, and J. Zhao, “Minfer: Amethod of inferring motif statistics from sampled edges,” in Proc.IEEE 32nd Int. Conf. Data Eng., 2016, pp. 1050–1061.

[54] X. Chen and J. C. S. Lui, “Mining graphlet counts in online socialnetworks,” in Proc. IEEE 16th Int. Conf. Data Min., Dec. 2016,pp. 71–80.

[55] W. Lin, X. Xiao, X. Xie, and X. Li, “Network motif discovery: AGPU approach,” in Proc. IEEE 31st Int. Conf. Data Eng., 2015,pp. 831–842.

[56] R. A. Rossi and R. Zhou, “Leveraging multiple gpus and cpus forgraphlet counting in large networks,” in Proc. 25th ACM Int. Conf.Inf. Knowl. Manag., 2016, pp. 1783–1792.

Pinghui Wang received the BS degree in infor-mation engineering and the PhD degree in auto-matic control from Xi’an Jiaotong University,Xi’an, China, in 2006 and 2012, respectively. Hewas a postdoctoral researcher in the Departmentof Computer Science and Engineering, TheChinese University of Hong Kong and School ofComputer Science, McGill University, Qu"ebec,Canada, and was a researcher with HuaweiNoah’s Ark Lab, Hong Kong. He is currently anassociate professor in the MOE Key Laboratory

for Intelligent Networks and Network Security, Xi’an Jiaotong University,and is also with the Shenzhen Research Institute at Xi’an Jiaotong Uni-versity, Shenzhen, China. His research interests include Internet trafficmeasurement and modeling, traffic classification, abnormal detection,and online social network measurement.

Junzhou Zhao received the BS degree in infor-mation engineering and the PhD degree in auto-matic control from Xi’an Jiaotong University,Xi’an, China, in 2006 and 2012, respectively.He is currently a postdoc fellow in the Departmentof Computer Science and Engineering, TheChinese University of Hong Kong, Shatin, HongKong. His research interests include online socialnetwork measurement and modeling.


Xiangliang Zhang received the PhD degree incomputer science from INRIA-Universite Paris-Sud, France, in 2010. She is currently an assistantprofessor and directs the Machine Intelligenceand Knowledge Engineering (MINE) Laboratory,King Abdullah University of Science and Technol-ogy (KAUST), Saudi Arabia. Her main researchinterests and experiences are in machine learn-ing, datamining, and cloud computing.

Zhenguo Li received the BS and MS degreesfrom the Department of Mathematics at PekingUniversity, in 2002 and 2005, respectively, and thePhD degree from the Department of InformationEngineering at the Chinese University of HongKong, in 2008. He is currently the director of AI The-ory Lab and principal researcher in Huawei Noah’sArk Lab. He was an associate research scientistin the Department of Electrical Engineering atColumbia University. His research interests includemachine learning and artificial intelligence.

Jiefeng Cheng received the PhD degree fromThe Chinese University of Hong Kong, in 2007.From 2007 to 2010, he worked as a research fel-low with The Chinese University of Hong Kongand the Hong Kong University, respectively. Heis currently an expert engineer with TencentCloud Security Lab, Shenzhen, China. Prior tojoining Tencent, he was a researcher in HuaweiNoah’s Ark Lab, Hong Kong, and a faculty mem-ber with The University of Chinese Academy ofSciences, in China. His research interests includegraph mining and management.

John C.S. Lui received the PhD degree in com-puter science from UCLA. He is currently a pro-fessor in the Department of Computer Scienceand Engineering, The Chinese University ofHong Kong. His current research interestsinclude communication networks, network/sys-tem security, network economics, network scien-ces, cloud computing, large-scale distributedsystems, and performance evaluation theory. Heserves on the editorial boards of the IEEE/ACMTransactions on Networking, the IEEE Transac-

tions on Computers, the IEEE Transactions on Parallel and DistributedSystems, the Journal of Performance Evaluation, and the InternationalJournal of Network Security. He was the chairman of the CSE Depart-ment from 2005-2011. His personal interests include films and generalreading. He received various departmental teaching awards and theCUHK Vice-Chancellor’s Exemplary Teaching Award. He is also a core-cipient of the IFIP WG 7.3 Performance 2005 and IEEE/IFIP NOMS2006 Best Student Paper Awards. He is an elected member of the IFIPWG 7.3, fellow of the ACM, fellow of the IEEE, and croucher seniorresearch fellow.

Don Towsley received the BA degree in physicsand the PhD degree in computer science fromthe University of Texas, in 1971 and 1975,respectively. From 1976 to 1985, he was a mem-ber of the faculty in the Department of Electricaland Computer Engineering, University of Massa-chusetts, Amherst. He is currently a distinguishedprofessor in the Department of Computer Sci-ence, University of Massachusetts. He hasheld visiting positions at the IBM T.J. WatsonResearch Center, Yorktown Heights, New York;

Laboratoire MASI, Paris, France; INRIA, Sophia-Antipolis, France;AT&T Labs-Research, Florham Park, New Jersey; and MicrosoftResearch Lab, Cambridge, United Kingdom. His research interestsinclude network measurement and modeling, networks, and perfor-mance evaluation. He served as editor-in-chief of the IEEE/ACM Trans-actions on Networking and on the editorial boards of the Journal of theACM, the IEEE Journal on Selected Areas in Communications, andnumerous other editorial boards. He was program co-chair of the jointACM SIGMETRICS and PERFORMANCE 92 conference and the Per-formance 2002 conference. He has received the 2007 IEEE KojiKobayashi Award, the 2007 ACM SIGMETRICS Achievement Award,the 1998 IEEE Communications Society William Bennett Best PaperAward, and numerous best conference/workshop paper awards. He is afellow of the IEEE and ACM, member of the ACM and ORSA, and chairof the IFIP Working Group 7.3.

Jing Tao received the BS and MS degrees inautomatic control from Xi’an Jiaotong University,Xi’an, China, in 2001 and 2006, respectively.He is currently a teacher with Xi’an JiaotongUniversity and is on-the-job working toward thePhD degree in the Systems Engineering Instituteand SKLMS Laboratory, Xi’an Jiaotong Universityunder the supervision of Prof. Xiaohong Guan.His research interests include Internet trafficmeasurement and modeling, traffic classification,abnormal detection, and botnet.

Xiaohong Guan received the BS and MSdegrees in automatic control from Tsinghua Uni-versity, Beijing, China, in 1982 and 1985, respec-tively, and the PhD degree in electricalengineering from the University of Connecticut,Storrs, in 1993. He is currently a professor in theSystems Engineering Institute, Xi’an JiaotongUniversity, Xi’an, China. From 1993 to 1995, hewas a consulting engineer in PG&E. From 1985to 1988, he was with the Systems EngineeringInstitute, Xi’an Jiaotong University, Xi’an, China.

From January 1999 to February 2000, he was with the Division of Engi-neering and Applied Science, Harvard University, Cambridge, Massa-chusetts. Since 1995, he has been with the Systems EngineeringInstitute, Xi’an Jiaotong University, and was appointed Cheung Kongprofessor of systems engineering, in 1999, and the dean of the Schoolof Electronic and Information Engineering, in 2008. Since 2001, he hasbeen the director of the Center for Intelligent and Networked Systems,Tsinghua University, and served as head of the Department of Automa-tion, 2003-2008. He is an editor of the IEEE Transactions on Power Sys-tems and an associate editor of Automata. His research interestsinclude allocation and scheduling of complex networked resources, net-work security, and sensor networks. He is a fellow of the IEEE.

" For more information on this or any other computing topic,please visit our Digital Library at www.computer.org/publications/dlib.


ieee transactions on knowledge and data engineering, …cslui/publication/tkdd-moss5.pdf ·...

Documents