graphday noble/coolio

Time Travel with Neo4jAleksa Vukotic Noble Group

Noble Group

• Commodity trading

• Supply chain management

• 70 locations

• $100 B revenue

The Vision

• Data consolidation strategy

• Polyglot persistence (time series, objects, reference data)

• Selecting reference data master store

Requirements• Diverse data/schemas

• Multi-source data flows

• Complex Hierarchy Trees

• Search

• All or nothing updates

• Versioning and time travel

Other Options

• Relational DB

• Index Store (Elastic Search/Lucene)

Problem

• Being able to travel in time

• Two use cases to manage:

1. How entities change over time

2. How relationships change over time

1. Entity Versioning

id: 1name: VATrate: 20%

TAX

id: 1name: VATrate: 15%

TAX

13/11/2014 00:00

Entity Versioningid: 1

TAX

13/11/2014 00:00

name:VATrate: 20%

versionSince:-1versionClose:

1415836800000

name:VATrate: 15%

versionSince: 1415836800000versionClose:

9223372036854775807

HAS_VERSION HAS_VERSION

TAX_STATE

TAX_STATE

2. Relationship Versioning

• Credit risk & counter-party tree

• Who owns who affects the credit risk

• Legal entities change ownership

• Need to keep history of all changes

Relationship Versioningid: 1

name: Money uLTD

id: 2name: ACME

IS_PARENT_OF

COUNTERPARTY

COUNTERPARTY

13/11/2014 00:00

id: 3name:

Pear Inc

IS_PARENT_OF

id: 2name: ACME

COUNTERPARTY

COUNTERPARTY

Relationship Versioning

id: 1name:

Money uLTD

COUNTERPARTY

id: 2name: ACME

COUNTERPARTY

13/11/2014 00:00

id: 2name:

Pear Inc

IS_PARENT_OF IS_PARENT_OFversionSince: -1

versionClose: 1415836800000

versionSince: 1415836800000versionClose: 9223372036854775807

Results

• Ref Data Platform launched recently

• 3 nodes (1 per region)

• >5 M nodes, >100 M relationships and growing

• Query times <300ms

• Data Clarity and Agility

Thank you

Paul%Gilligan%Coolio%Ltd%

@brothergilligan%%%#graphday%

•  Overview%

•  Future%plans%

•  Descrip>on%of%problem%

•  Solu>on%of%problem%using%Neo4j%

Paul%Gilligan%Coolio%Ltd%

@brothergilligan%%%#graphday%

•  Overview%

•  Future%plans%

•  Descrip>on%of%problem%

•  Solu>on%of%problem%using%Neo4j%

Overview%

@brothergilligan%%%#graphday%

Overview%

@brothergilligan%%%#graphday%

Overview%

@brothergilligan%%%#graphday%

Overview%

@brothergilligan%%%#graphday%

•  Simple%buEon%press%to%place%a%Coolio%

•  En>rely%binary,%en>rely%subjec>ve%

•  Algorithm%ascribes%a%value%to%each%Coolio%%%

•  Coolios%drive%the%Coolth%of%Places%and%People%

•  Each%Coolio%is%born,%degrades%then%dies%

•  Dispropor>onate%power%of%the%nega>ve%

•  Everyone’s%‘Cool’%is%different%

Highly%social%and%highly%

personalised,%up%to%the%minute%

ra>ngs%and%recommenda>ons,%

with%no%pollu>on%from%the%

dispropor>onate%power%of%the%

nega>ve.%

The%Problem%

@brothergilligan%%%#graphday%

•  Many%users%placing%many%Coolios%could%get%too%big%too%quickly.%

•  Degrading%Coolios%would%need%to%use%update%queries%

•  Each%new%Coolio%needs%many%Coolio%values%to%be%retrieved%from%many%users%and%many%places.%

•  Ins>nc>vely%knew%that%this%was%too%messy%and%too%cumbersome%

The%Problem%

@brothergilligan%%%#graphday%

The%Problem%

@brothergilligan%%%#graphday%

The%Problem%

@brothergilligan%%%#graphday%

The%Solu>on%

@brothergilligan%%%#graphday%

The%Solu>on%

@brothergilligan%%%#graphday%

Coolio •  Hit%index%once%and%find%user%•  Retrieve%all%User’s%Coolio’s%

values%by%going%straight%to%them%•  Don’t%have%to%search%every%

record%of%a%single%table%

CoolioCoolio

•  Immediately%simpler%and%easier%than%original%SQL%schema%

•  Update%values%every%>me%we%hit%a%Coolio%

Coolio

Coolio

Coolio

The%Solu>on%

@brothergilligan%%%#graphday%

Coolio

Coolio

User yCoolio

gcpvj6bPlace c

gcpvj6c

gcpvj6f

Place d

Place e

gcpvj4zPlace f gcpvj4

The%Solu>on%

@brothergilligan%%%#graphday%

GeoTree%

The%Friend%Zone%

The%EcoYSystem%

The%Solu>on%

@brothergilligan%%%#graphday%

The%Future%

@brothergilligan%%%#graphday%

Coolio

Mon Tue Wed Thu Fri Sat Sun

17.0016.0015.0014.0013.00

May

Jun

Jul

Spr

Sum

Aut

The%Future%

@brothergilligan%%%#graphday%

Technical) Cypher)–)quick,)easy,)powerful,)expressive)Flexibility)–)server)instance)or)embedded)in)Java)applica?on)Languages)–)Python,)NodeJS,)PHP,)Java,)Ruby))API)–)own)powerful)RESTful)API)

Support) Provide)training)and)facilitate)a)growing)developer)community)Generate)opportuni?es)for)discussion)and)access)to)their)developer)with)conferences,)meetups)and)events)like)this)

Commercial) Suppor?ve)and)understand)commercial)pressures)My)success)equals)their)success)