Neo4jAnd TelCo goodness

Peter NeubauerNeo Technology

#neo4j@peterneubauerpeter@neotechnology.com

Some usecases

NOSQL data models

Bigtable clones

Key-value stores

Document databases

Graph databases

Data complexity

Dat

a si

ze

The Neo4j model: Property GraphCore abstractions:

NodesRelationships between nodesProperties on both

name = “Emil”age = 29sex = “yes”

type = KNOWStime = 4 years

type = carvendor = “SAAB”model = “95 Aero”

11 22

33

The NoSQL landscapeNeo4j introExamples for TelCos

CDRRoutingSocial graphsMaster Data ManagementSpatial and LBSNetwork topology analysisNeo4j and Android

Agenda

Most widely deployed graph db in the worldACID, persistent, embedded/serverRobust: 24/7 production since 2003Mature: lots of production deploymentsScalable: High Availability, Master failoverCommunity: ecosystem of tools, bindings, frameworksProduct: OSGi, Spatial, RDF, languages

Available under AGPLv3 and as commercial productBut the fi rst one is free! For ALL use-cases.

Linkshttp://neo4j.org http://lists.neo4j.org

Neo4j?

Neo4j – Nodes, Relationships, Properties

Nodes have different propertiesMatrix characters: People vs. Programs

Build structure as you goWho loves Neo?

Building a node spaceGraphDatabaseService graphDb = ... // Get factory

// Create Thomas 'Neo' AndersonNode mrAnderson = graphDb.createNode();mrAnderson.setProperty( "name", "Thomas Anderson" );mrAnderson.setProperty( "age", 29 );

// Create MorpheusNode morpheus = graphDb.createNode();morpheus.setProperty( "name", "Morpheus" );morpheus.setProperty( "rank", "Captain" );morpheus.setProperty( "occupation", "Total bad ass" );

// Create a relationship representing that they know each othermrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS );// ...create Trinity, Cypher, Agent Smith, Architect similarly

Building a node spaceGraphDatabaseService graphDb = ... // Get factoryTransaction tx = graphdb.beginTx();

// Create Thomas 'Neo' AndersonNode mrAnderson = graphDb.createNode();mrAnderson.setProperty( "name", "Thomas Anderson" );mrAnderson.setProperty( "age", 29 );

// Create MorpheusNode morpheus = graphDb.createNode();morpheus.setProperty( "name", "Morpheus" );morpheus.setProperty( "rank", "Captain" );morpheus.setProperty( "occupation", "Total bad ass" );

// Create a relationship representing that they know each othermrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS );// ...create Trinity, Cypher, Agent Smith, Architect similarlytx.commit();

Code (2): Traversing a node space

// Instantiate a traverser that returns Mr Anderson's friendsTraverser friendsTraverser = mrAnderson.traverse(

Traverser.Order.BREADTH_FIRST,StopEvaluator.END_OF_GRAPH,ReturnableEvaluator.ALL_BUT_START_NODE,RelTypes.KNOWS,Direction.OUTGOING );

// Traverse the node space and print out the resultSystem.out.println( "Mr Anderson's friends:" );for ( Node friend : friendsTraverser ){

System.out.printf( "At depth %d => %s%n",friendsTraverser.currentPosition().getDepth(),friend.getProperty( "name" ) );

}

Rubygem install neo4j

require ”rubygems”require 'neo4j'

class Person include Neo4j::NodeMixin property :name, :age, :occupation index :name has_n :friendsend

Neo4j::Transactoin.run do neo = Person.new :name=>'Neo', :age=>29 morpheus = Person.new :name=>'Morpheus', :occupation=>'badass' neo.friends << morpheusend

neo.friends.each {|p|...}

Spatial and social data

name = “Omni Hotel”lat = 3492848long = 283823423

11

length = 7 miles

name = ...lat, long = ...

name = “Swedland”lat = 23410349long = 2342348852

33

1313

ROAD ROOOAD

name = “The Tavern”lat = 1295238237long = 234823492

length = 3 miles

name = ...

77

22

ROAD

ROADR

OAD

name = ...

4242

ROAD

Financial data – fraud detection

name = “Mr Godfather”karma = veeeery-lowcash = more-than-you

11

amount = $1000

name = “Emil”cash = always-too-li'l

title = “ATM @ Wall St”id = 230918484233cash_left = 384204

33

1313

TRANSFER WITHDRAW

name = “The Tavern”lat = 1295238237long = 234823492

amount = $1000

name = ...

77

22

OWNS

DEPOSITTR

ANSF

ER

name = ...

4242

WITHDRAW

CDR analysis

name = “Mr Godfather”karma = veeeery-lowcash = more-than-you

11

time = 2min

name = “Emil”cash = always-too-li'l

Name= customer_support

33

1313

CALLED CALLED

name = “Bob”lat = 1295238237long = 234823492

time =13min

name = Alice

77

22

CALLED

CALLEDFA

MIL

Y

name = flat_business

4242

DATA_P

LAN

Call Data Records (CDR)Forming a graphLocation basedPossible uses:

Find clusters (better plans)Build social connectionsFind influencers

Routing

Routing with Neo4j and A*

Routing

Connecting and Splitting

Routing

Social graphs

RecommendationsLocation based servicesInfluencersShortest path

Recommendations and big graphsGlobal heuristics

Page rankLocal recommendations

Shortest pathsHammock functionsRandom walksDijkstra, A*, Shooting star etc

Grammar based Random Walks

Pic from gremlin.tinkerpop.com / Marko Rodriguez

Impact, Dependency Analysis

Master Data Management

SpatialComplex data

Multiple indexing (domain, Spatial, temporal)Location entering many domainsGIS going mainstream, topologies explodeNo good systems out thereProprietary stacks rule (ESRI, Oracle)Open Government DataShapefiles suck.

Current challenges in SpatialDomain and Spatial interconnectionsUnstructured domain dataRoutingTopology handlingNo good OSS full GIS stack

Multiple indexes - GIS

OpenStreetMap

The OpenStreetMap datasetWiki for Spatial infoFreely available dataVery unstructured, free tagging

Points, Ways, Relations, Tags, ChangesetsChanges can be pushed back

Used for other purposesGreat coverage in interesting places (towns, disasters etc)

OpenStreetMap

OpenStreetMap

Web exposure of Spatial Data

Davide SavazziGeotools & GeoServerRoutinguDig

GSoC 2010 - uDig

Neo4j dynamic layers

Layer1

Layer2

Layer3

GeometryEncoder

DynamicQuery

DynamicStyles

DynamicMeta-Inf

Connected domain data Neo4j Spatial GIS and Spatial stacks

OpenStreetMap

OSM

Index (RTree)

DynamicLayers

Dynamic Layers

Network Topology analysis

Analytics of network coverage and frequenciesCell towersDrive dataInfrastructure

AnalyticsSpatial signal strengthAntenna placement and azimuthFrequency planningNetwork differences over timeReporting and charting

Small footprintRunning almost unmodifiedInteresting scenarios

Semantic homescreenConnected devicesLocal cachesDisconnected databases

Neo4j and Android

Connected devices

Emils PC Marcus

Emil

LeighJohan

“Link all pictures that any of my friends has taken of me into my PicturesOfMe folder and update this every hour”

Interesting scenariosSemantic homescreenConnected devicesLocal cachesDisconnected databases

How ego are you? (aka other impls?)Franz’ AllegroGraph (http://agraph.franz.com)

Proprietary, Lisp, RDF-oriented but real graphdb

Sones graphDB (http://sones.com)

.NET, Deutsche Telekom VC backed

Twitter's FlockDB (http://github.com/twitter/flockdb)

Twitter's (graph) database for large and shallow graphs

Google Pregel (http://bit.ly/dP9IP)

We are oh-so-secret

Objectivity's InfniteGraph (http://infinitegraph.com)

New, closed OODB with Graph Layer on top

API ReferencesWiki, Code, API references

http://wiki.neo4j.org/content/Neo4j_Spatialhttp://github.com/neo4j/neo4j-spatialhttp://components.neo4j.org/neo4j-spatialMailing list: neo4j@lists.neo4j.orghttp://neo4j.org/community/list/

Questions?

Image credit: lost again! Sorry :(

http://neotechnology.com