intro to graph world

1

Andreas Kollegger@akollegger #neo4j

Introduction toNeo4j

Thursday, January 24, 13

1


2


The Path Forward

2


The Path Forward

1.Why graphs?

2


The Path Forward

1.Why graphs?

2.What's a graph database?

2


The Path Forward

1.Why graphs?

2.What's a graph database?

3.How do people use Neo4j?

2


3


Why graphs, why now?

3

!⛵☕

$

%⚾'

()* +,-

.

✈⛽ 1

23

4☕ 5

6

7 89:

;<

=

>

?@

⚽

B

C

D

E $F%

GHI

J

K

L

M

()

NOP

,Q

-*


1.Big Data is the trend


3

! " #

$

%

&

✈

⛽

⛵

*+

,⚽

.

/ 0

1

☕

3

4

5 ⚾

7

8

9

:

;<

=> ?

@ A B

C!D

E F

G

H"

$

✈⛽,

.

0

1

I☕J

<

@KBL

MNG

OJ

#

%&

P

⛵

*

+Q

⚽

R

/

S

3

T

4U5

O

⚾7

8 V

9

W

:

X

;

=>

YZ

C

[

D\E

F]

H

AX

?!⛵☕

$

%⚾'

()* +,-

.

✈⛽ 1

23

4☕ 5

6

7 89:

;<

=

>

?@

⚽

B

C

D

E $F%

GHI

J

K

L

M

()

NOP

,Q

-*



2.NOSQL is the answer


3

!"#

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

23

4

5

☕

7

8

9

:

;

<⚾

>

?@

A

B

C

D E

FGH

IJ

KL

M

N

O !

PQ

R

ST

UV

W

X

Y

Z"

$

✈⛽ -

1

3

4

5☕ [

E

I KMP

S WX

\ [

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾ >

?@

A

B

C

]

D

FG

LN

O

Q

R TU

VY

Z

J]

H

! " #

$

%

&

✈

⛽

⛵

*+

,⚽

.

/ 0

1

☕

3

4

5 ⚾

7

8

9

:

;<

=> ?

@ A B

C!D

E F

G

H"

$

✈⛽,

.

0

1

I☕J

<

@KBL

MNG

OJ

#

%&

P

⛵

*

+Q

⚽

R

/

S

3

T

4U5

O

⚾7

8 V

9

W

:

X

;

=>

YZ

C

[

D\E

F]

H

AX

?!⛵☕

$

%⚾'

()* +,-

.

✈⛽ 1

23

4☕ 5

6

7 89:

;<

=

>

?@

⚽

B

C

D

E $F%

GHI

J

K

L

M

()

NOP

,Q

-*




3.Large in volume, and in variety

!"#

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

23

4

5

☕

7

8

9

:

;

<⚾

>

?@

A

B

C

D E

FGH

IJ

KL

M

N

O !

PQ

R

ST

UV

W

X

Y

Z"

$

✈⛽ -

1

3

4

5☕ [

E

I KMP

S WX

\ [

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾ >

?@

A

B

C

]

D

FG

LN

O

Q

R TU

VY

Z

J]

H


3

!"#

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

23

4

5

☕

7

8

9

:

;

<⚾

>

?@

A

B

C

D E

FGH

IJ

KL

M

N

O !

PQ

R

ST

UV

W

X

Y

Z"

$

✈⛽ -

1

3

4

5☕ [

E

I KMP

S WX

\ [

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾ >

?@

A

B

C

]

D

FG

LN

O

Q

R TU

VY

Z

J]

H

! " #

$

%

&

✈

⛽

⛵

*+

,⚽

.

/ 0

1

☕

3

4

5 ⚾

7

8

9

:

;<

=> ?

@ A B

C!D

E F

G

H"

$

✈⛽,

.

0

1

I☕J

<

@KBL

MNG

OJ

#

%&

P

⛵

*

+Q

⚽

R

/

S

3

T

4U5

O

⚾7

8 V

9

W

:

X

;

=>

YZ

C

[

D\E

F]

H

AX

?!⛵☕

$

%⚾'

()* +,-

.

✈⛽ 1

23

4☕ 5

6

7 89:

;<

=

>

?@

⚽

B

C

D

E $F%

GHI

J

K

L

M

()

NOP

,Q

-*




3.Large in volume, and in variety

!"#

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

23

4

5

☕

7

8

9

:

;

<⚾

>

?@

A

B

C

D E

FGH

IJ

KL

M

N

O !

PQ

R

ST

UV

W

X

Y

Z"

$

✈⛽ -

1

3

4

5☕ [

E

I KMP

S WX

\ [

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾ >

?@

A

B

C

]

D

FG

LN

O

Q

R TU

VY

Z

J]

H


3

!"#

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

23

4

5

☕

7

8

9

:

;

<⚾

>

?@

A

B

C

D E

FGH

IJ

KL

M

N

O !

PQ

R

ST

UV

W

X

Y

Z"

$

✈⛽ -

1

3

4

5☕ [

E

I KMP

S WX

\ [

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾ >

?@

A

B

C

]

D

FG

LN

O

Q

R TU

VY

Z

J]

H

! " #

$

%

&

✈

⛽

⛵

*+

,⚽

.

/ 0

1

☕

3

4

5 ⚾

7

8

9

:

;<

=> ?

@ A B

C!D

E F

G

H"

$

✈⛽,

.

0

1

I☕J

<

@KBL

MNG

OJ

#

%&

P

⛵

*

+Q

⚽

R

/

S

3

T

4U5

O

⚾7

8 V

9

W

:

X

;

=>

YZ

C

[

D\E

F]

H

AX

?!⛵☕

$

%⚾'

()* +,-

.

✈⛽ 1

23

4☕ 5

6

7 89:

;<

=

>

?@

⚽

B

C

D

E $F%

GHI

J

K

L

M

()

NOP

,Q

-*

! " #

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

2 3

4

5

☕

7

8

9

:

;

< ⚾

>

?@

A

B

C

DE

FG H

I J

KL

M

N

O!

PQ

R

S T

U V

W

X

Y

Z"

$

✈⛽-

1

3

4

5☕[

E

IKMP

SWX

\[

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾>

? @

A

B

C

]

D

FG

LN

O

Q

RTU

VY

Z

J]

H

! " #

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

2 3

4

5

☕

7

8

9

:

;

< ⚾

>

?@

A

B

C

DE

FG H

I J

KL

M

N

O!

PQ

R

S T

U V

W

X

Y

Z"

$

✈⛽-

1

3

4

5☕[

E

IKMP

SWX

\[

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾>

? @

A

B

C

]

D

FG

LN

O

Q

RTU

VY

Z

J]

H!"#

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

23

4

5

☕

7

8

9

:

;

<⚾

>

?@

A

B

C

D E

FGH

IJ

KL

M

N

O !

PQ

R

ST

UV

W

X

Y

Z"

$

✈⛽ -

1

3

4

5☕ [

E

I KMP

S WX

\ [

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾ >

?@

A

B

C

]

D

FG

LN

O

Q

R TU

VY

Z

J]

H

!"#

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

23

4

5

☕

7

8

9

:

;

<⚾

>

?@

A

B

C

D E

FGH

IJ

KL

M

N

O !

PQ

R

ST

UV

W

X

Y

Z"

$

✈⛽ -

1

3

4

5☕ [

E

I KMP

S WX

\ [

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾ >

?@

A

B

C

]

D

FG

LN

O

Q

R TU

VY

Z

J]

H

!"#

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

23

4

5

☕

7

8

9

:

;

<⚾

>

?@

A

B

C

D E

FGH

IJ

KL

M

N

O !

PQ

R

ST

UV

W

X

Y

Z"

$

✈⛽ -

1

3

4

5☕ [

E

I KMP

S WX

\ [

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾ >

?@

A

B

C

]

D

FG

LN

O

Q

R TU

VY

Z

J]

H

!"#

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

23

4

5

☕

7

8

9

:

;

<⚾

>

?@

A

B

C

D E

FGH

IJ

KL

M

N

O !

PQ

R

ST

UV

W

X

Y

Z"

$

✈⛽ -

1

3

4

5☕ [

E

I KMP

S WX

\ [

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾ >

?@

A

B

C

]

D

FG

LN

O

Q

R TU

VY

Z

J]

H

!"#

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

23

4

5

☕

7

8

9

:

;

<⚾

>

?@

A

B

C

D E

FGH

IJ

KL

M

N

O !

PQ

R

ST

UV

W

X

Y

Z"

$

✈⛽ -

1

3

4

5☕ [

E

I KMP

S WX

\ [

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾ >

?@

A

B

C

]

D

FG

LN

O

Q

R TU

VY

Z

J]

H

!"#

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

23

4

5

☕

7

8

9

:

;

<⚾

>

?@

A

B

C

D E

FGH

IJ

KL

M

N

O !

PQ

R

ST

UV

W

X

Y

Z"

$

✈⛽ -

1

3

4

5☕ [

E

I KMP

S WX

\ [

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾ >

?@

A

B

C

]

D

FG

LN

O

Q

R TU

VY

Z

J]

H

!"#

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

23

4

5

☕

7

8

9

:

;

<⚾

>

?@

A

B

C

D E

FGH

IJ

KL

M

N

O !

PQ

R

ST

UV

W

X

Y

Z"

$

✈⛽ -

1

3

4

5☕ [

E

I KMP

S WX

\ [

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾ >

?@

A

B

C

]

D

FG

LN

O

Q

R TU

VY

Z

J]

H

!"#

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

23

4

5

☕

7

8

9

:

;

<⚾

>

?@

A

B

C

D E

FGH

IJ

KL

M

N

O !

PQ

R

ST

UV

W

X

Y

Z"

$

✈⛽ -

1

3

4

5☕ [

E

I KMP

S WX

\ [

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾ >

?@

A

B

C

]

D

FG

LN

O

Q

R TU

VY

Z

J]

H

!"#

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

23

4

5

☕

7

8

9

:

;

<⚾

>

?@

A

B

C

D E

FGH

IJ

KL

M

N

O !

PQ

R

ST

UV

W

X

Y

Z"

$

✈⛽ -

1

3

4

5☕ [

E

I KMP

S WX

\ [

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾ >

?@

A

B

C

]

D

FG

LN

O

Q

R TU

VY

Z

J]

H

!"#

$

%

&

'

✈

⛽

⛵

+,

-

.

⚽

0

1

23

4

5

☕

7

8

9

:

;

<⚾

>

?@

A

B

C

D E

FGH

IJ

KL

M

N

O !

PQ

R

ST

UV

W

X

Y

Z"

$

✈⛽ -

1

3

4

5☕ [

E

I KMP

S WX

\ [

#

%&

'

⛵

+

,.

⚽

0

2

7

8

9

:;<

\

⚾ >

?@

A

B

C

]

D

FG

LN

O

Q

R TU

VY

Z

J]

H


3


4


4

Everyone is talking about graphs...


4

Everyone is talking about graphs...

Facebook Open Graph


5

Text


Graphs are everywhere! I even have one...

5

Text


A graph...

6


A graph...

6

๏you know the common data structures


A graph...

6


•linked lists, trees, object "graphs"


A graph...

6



๏a graph is the general purpose data structure


A graph...

6




•suitable for any data that is related


A graph...

6





๏well-understood patterns and algorithms


A graph...

6






•studied since Leonard Euler's 7 Bridges (1736)


A graph...

6







•Codd's Relational Model (1970)


A graph...

6







•Codd's Relational Model (1970)

•not a new idea, just an idea who's time is now


A graph database...

7


A graph database...

7

๏optimized for the connections between records


A graph database...

7


๏really, really fast at querying across records


A graph database...

7



๏a database: transactional with the usual operations


A graph database...

7



๏a database: transactional with the usual operations

๏“A relational database may tell you the average age of everyone in this webinar,

but a graph database will tell you how to drive from Baltimore to San Francisco.”


That last bullet is the key insight...

8


9


You know relational

9


You know relational

9

foo


You know relational

9

foo bar


You know relational

9

foo barfoo_bar


You know relational

9

now consider relationships...


9


10


We're talking about aProperty Graph

10



10

Nodes



10

Nodes

Relationships


Emil

Andrés

Lars

Johan

Allison

Peter

Michael

Tobias

Andreas

IanMica

Delia

knows

knows

knowsknows

knows

knows

knows

knows

knows

knowsMica

knowsknowsMica

Delia

knows


10

Nodes

Relationships

Properties (each a key+value)

+ Indexes (for easy look-ups)


10


Neo4j - the Graph Database

11


Google "neo4j"

12


Google "neo4j"๏neo4j.org

12



๏neotechnology.com

12




๏github.com/neo4j

12




๏github.com/neo4j

๏neo4j.meetup.com

12




๏github.com/neo4j

๏neo4j.meetup.com

๏graphconnect.com

12


How to get started?

13


How to get started?๏ Documentation

13



•docs.neo4j.org - tutorials+reference

13




•Neo4j in Action

13




•Neo4j in Action

•Good Relationships

13




•Neo4j in Action


๏ Get Neo4j

13




•Neo4j in Action


๏ Get Neo4j

•http://neo4j.org/download

13


http://neo4j.org/download




•Neo4j in Action


๏ Get Neo4j


•http://addons.heroku.com/neo4j/

13




http://addons.heroku.com/neo4j/




•Neo4j in Action


๏ Get Neo4j



๏ Participate

•ask questions on Stack Overflow

•http://groups.google.com/group/neo4j

•http://neo4j.meetup.com

13






http://groups.google.com/group/neo4j


http://neo4j.meetup.com




•Neo4j in Action


๏ Get Neo4j



๏ Participate

•ask questions on Stack Overflow

•http://groups.google.com/group/neo4j

•http://neo4j.meetup.com

•webinars, every month on everything from intro to production13










Neo4j is a Graph Database

14


Neo4j is a Graph Database• A Graph Database:

14



• a Property Graph with Nodes, Relationshipsand Properties on both

14




• perfect for complex, highly connected data

14





• A Graph Database:

14






• reliable with real ACID Transactions

14






• reliable with real ACID Transactions• scalable: 32 Billion Nodes, 32 Billion Relationships, 64 Billion

Properties

14







Properties

• Server with REST API, or Embeddable on the JVM

14







Properties

• Server with REST API, or Embeddable on the JVM• high-performance with High-Availability (read scaling)

14


15


And, but, so how do you query this "graph" database?

15


16


16

Cypher - a graph query language๏a pattern-matching query language

๏declarative grammar with clauses (like SQL)

๏aggregation, ordering, limits

๏create, read, update, delete


16

Cypher - a graph query language๏a pattern-matching query language

๏declarative grammar with clauses (like SQL)

๏aggregation, ordering, limits

๏create, read, update, delete// get node from an index named “foo”start foo=node:people(name=‘Andreas’) return foo

// find “bar” nodes related to Andreasstart foo=node:people(name=‘Andreas’)match (foo)-->(bar) return bar

// create a nodecreate (me {name:'Andreas'})


17


Is it production ready?

17


18


Neo4j HA - High Availability Cluster

18



18

•master-slave replication

Load Balancer



18


• read-scaling

Load Balancer



18


• read-scaling

• single datacenter, or global zones

Load Balancer



18


• read-scaling


• tolerance for high-latency

Load Balancer



18


• read-scaling



• redundancy provides improved uptime

Load Balancer



18


• read-scaling



• redundancy provides improved uptime

• automatic failoverLoad Balancer



18

Load Balancer


19

Real World Use Cases:


19

[A] Mmm Pancakes



19

[A] Mmm Pancakes[B] ACL from Hell



19

[A] Mmm Pancakes[B] ACL from Hell[C] Master of your Domain



[A] Mmm Pancakes


[A] Mozilla Pancake

This Material is subject to the terms of the Mozilla Public # License, v. 2.0. If a copy of the MPL was not distributed with this # file, You can obtain one at http://mozilla.org/MPL/2.0/


http://mozilla.org/MPL/2.0/


[A] Mozilla Pancake

• Experimental cloud-based browser

• Built to improve how users Discover, Collect, Share & Organize things on the web

• Goal: help users better access & curate information on the net, on any device





Why Neo4J?





Why Neo4J?

• The internet is a network of pages connected to each other. What better way to model that than in graphs?

• No time lost fighting with less expressive datastores

• Easy to implement experimental features





Cute meta + data





Neo4J Co-Existence

• Node uuids as refs in external ElasticSearch also in internal Lucene

• Custom search ranking for user history based on node relationship data

• MySQL for user data, Redis for metrics





Mozilla Pancake

Available on BitBucket:https://bitbucket.org/mozillapancake/pancake

Questions?Olivier Yiptong: [email protected]



https://bitbucket.org/mozillapancake/pancake

https://bitbucket.org/mozillapancake/pancake

mailto:[email protected]

mailto:[email protected]



[B] ACL from Hell

One of the top 10 telcos worldwide


Background

Telenor calculated in 2010 that it´s self-service web solution for companies, MinBedrift, would not scale with the projected customer and subscription growth beyond 2012.

Projected Growth

2010 2015

Limit


[B] Telenor Background

Background


Projected Growth

2010 2015

Limit


[B] Telenor Background

Background


Projected Growth

2010 2015

Limit

• MinBedrif, a self service web solution for companies

• 2010 - calculated that it would not scale with projected growth


Example Access Authorization

Access may be given directly or by inheritance

U

C

S

C

A A A

S S S S S

User

Customer

Account

Subscription Inherit = true U

Inherit = true

Inherit = false

C

A

S S

C

A

S S


Business Case

The business case is built on the negative consequence of NOT addressing the problem.

Loss of customers (income)

Reduced sales transactions (income)

Other

Increased manual support (expenses)


30


Current ACL Service

30


Current ACL Service

•Stored procedure in DB calculating all access

• cached results for up to 24 hours

• minutes to calculate for large customers

• extremely complex to understand (1500 lines)

• depends on temporary tables

• joins across multiple tables

30


31


ACL With Neo4j

31


ACL With Neo4j๏Faster than current solution

๏Simpler to understand the logic

๏Avoid large temporary tables

๏Tailored for service (resource authorization)

31


[C] Master of your Domain


[C] MDM within Ciscomaster data management, sales compensation management, online customer support

Architecture3-node Enterprise cluster with mirrored disaster recovery clusterDedicated hardware in own datacenterEmbedded in custom webapp

Sizing35 million nodes50 million relationships600 million properties

DescriptionReal-time conflict detection in sales compensation management. Business-critical “P1” system. Neo4j allows Cisco to model complex algorithms, which still maintaining high performance over a large dataset.

BackgroundNeo4j replaces Oracle RAC, which was not performant enough for the use case.

BenefitsPerformance : “Minutes to Milliseconds”Outperforms Oracle RAC, serving complex queries in real timeFlexibilityAllows for Cisco to model interconnected data and complex queries with easeRobustnessWith 9+ years of production experience, Neo4j brings a solid product.


34


34

Really, once you start thinking in graphs it's hard to stop

Recommendations MDM

Systems ManagementGeospatial

Social computing

Business intelligence

Biotechnology

Making Sense of all that data

your brainaccess control

linguistics

catalogs

genealogy routing

compensationmarket vectors


34

Really, once you start thinking in graphs it's hard to stop

Recommendations MDM

Systems ManagementGeospatial

Social computing

Business intelligence

Biotechnology

Making Sense of all that data

your brainaccess control

linguistics

catalogs

genealogy routing

compensationmarket vectors

What will you build?


35


35

and, Thanks :)

Neo4j


intro to graph world

Documents