![Page 1: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/1.jpg)
1
KONECT Cloud
Large Scale Network Mining in the Cloud
Jérôme Kunegis Future SOC Lab Day, 18.04.2012
![Page 2: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/2.jpg)
Networks are Everywhere
Communication
Authorship
Friendship
c
Interaction
Trust
Co-occurrence
![Page 3: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/3.jpg)
Social Networks
friend
![Page 4: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/4.jpg)
Trust Networks
trust
![Page 5: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/5.jpg)
Friend/Enemy Network
enemy
frien
d
![Page 6: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/6.jpg)
Interaction Networklisten
![Page 7: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/7.jpg)
KONECT – Koblenz Network Collection
148 network datasets
26 are undirected 38 are directed 84 are bipartite 59 have unweighted edges 77 allow multiple edges 04 have signed edges 08 have ratings as edges 78 have edge arrival times
konect.uni-koblenz.de
![Page 8: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/8.jpg)
Largest Network
Directed “who follows who” network
0 041 652 230 users
1 468 365 182 edges
konect.uni-koblenz.de/networks/twitter
![Page 9: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/9.jpg)
148 Network Datasets
authorshipcommunicationco-occurrence
featuresfolksonomyinteraction
physicalratings
referencesemantic
socialtrust
![Page 10: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/10.jpg)
What We Computed
Connected componentsNetwork diameterClustering coefficientsDegree distributionsSpectral distributionEigenvector centralityGraph drawingTemporal AnalysisLink prediction
←at Future SOC Lab
![Page 11: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/11.jpg)
Network Diameter
6
![Page 12: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/12.jpg)
90 Percentile Effective Diameter
5
![Page 13: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/13.jpg)
90 Percentile Effective Diameter
3
![Page 14: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/14.jpg)
90 Percentile Effective Diameter
3.75
![Page 15: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/15.jpg)
Computing the Effective Diameter
for each node i { |V| count hops needed to reach 90% |E|
}
Total runtime: |E| × |V|
![Page 16: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/16.jpg)
Graph Sampling
KeepX% of edges
![Page 17: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/17.jpg)
Computation
× 1 000 vertices (sampled)× 120 840 391 edges× 20 sample sizes (5%, 10%, …, 100%)× 50 random samplings
Evaluation on single machine:
1 TiB memory 64 cores Matlab 64 bit
![Page 18: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/18.jpg)
Results
![Page 19: KONECT Cloud – Large Scale Network Mining in the Cloud](https://reader036.vdocuments.site/reader036/viewer/2022062617/54c662194a79594b538b46ae/html5/thumbnails/19.jpg)
Dr. Jérôme Kunegis
west.uni-koblenz.de
Thank You!
konect.uni-koblenz.de