2014.06.24.graph support
DESCRIPTION
overview and discussion of graphx integration into Ubix.TRANSCRIPT
![Page 1: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/1.jpg)
Graph Support by Alexander Albul
| UBIX |
UBIX.IO
![Page 2: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/2.jpg)
Architecture
UBIX.IO
+ +
![Page 3: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/3.jpg)
Idea
UBIX.IO
![Page 4: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/4.jpg)
Phase one: Denial and Isolation
UBIX.IO
![Page 5: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/5.jpg)
Possible problems
UBIX.IO
● Our engine did not support it. ● How to construct graphs? ● What to do with commands?
![Page 6: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/6.jpg)
Problem #1. Engine
UBIX.IO
![Page 7: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/7.jpg)
Problem #1. New Engine
UBIX.IO
![Page 8: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/8.jpg)
Problem #2. Graph construction
UBIX.IO
![Page 9: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/9.jpg)
Problem #2. Graph construction
UBIX.IO
![Page 10: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/10.jpg)
Problem #2. Graph construction
UBIX.IO
val context = new SparkContext("local", "graphx app") val verticesRdd = context.textFile("vertices.csv").map { case line => val lineSplit = line.split(",") lineSplit.head.toLong -> lineSplit.last } val edgesRdd = context.textFile("edges.csv").map { case line => val lineSplit = line.split(",") val fromId = lineSplit.head.toLong val toId = lineSplit.tail.head.toLong val attribute = lineSplit.last Edge(fromId, toId, attribute) } val graph = Graph(verticesRdd, edgesRdd) println(graph.numVertices)
![Page 11: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/11.jpg)
Problem #2. Graph construction
UBIX.IO
● Extract data from raw file ● Transform to GraphX format ● Build Graph
![Page 12: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/12.jpg)
Problem #2. Graph construction
UBIX.IO
![Page 13: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/13.jpg)
Problem #2. Graph construction
UBIX.IO
![Page 14: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/14.jpg)
Problem #2. Graph construction
UBIX.IO
![Page 15: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/15.jpg)
Problem #3. Graph commands
UBIX.IO
Can i use graphs now?
![Page 16: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/16.jpg)
Problem #3. Implicit type conversion
UBIX.IO
![Page 17: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/17.jpg)
Problem #3. Graph to table
UBIX.IO
…. val graph = Graph(verticesRdd, edgesRdd) val table = graph.triplets.map { case triplet => triplet.srcId :: triplet.srcAttr :: triplet.dstId :: triplet.dstAttr :: triplet.attr :: Nil } table.foreach(println)
![Page 18: 2014.06.24.graph support](https://reader033.vdocuments.site/reader033/viewer/2022060111/556263d9d8b42aab1a8b4b08/html5/thumbnails/18.jpg)
Q & A
UBIX.IO