A GAME OF DATA AND

GRAPHQL

Michael Hunger, Head of Developer Relations, Neo4j (@mesirii)

GraphQL Meetup Berlin, Aug 24 2017

Looking for

GoT Data

AN API OF ICE AND

FIRE

https://anapioficeandfire.com/About

https://github.com/joakimskoog/AnApiOfIceAndFire

An API ofIce and Fire

• Data sourced from Wiki of Ice And Fire (AWOIAF)

• Well documented .Net powered API

• Data on

– Books (5)

– Characters (2400)

– Houses (444)

• 100 entries per page

An API ofIce and Fire

– https://www.anapioficeandfire.com/api/characters/1303

GRAPHQL

Quick Graphql Schema

type Seat {

name: String!

houses: [House] @relation(name:"SEAT_OF")

}

type Region {

name: String!

houses: [House] @relation(name:"IN_REGION",

direction:IN)

}

type House {

id: ID!

name: String!

founded: String

titles: [String]

ancestralWeapons: [String]

coatOfArms: String

words: String

seats: [Seat] @relation(name:"SEAT_OF",

direction:IN)

region: Region @relation(name:"IN_REGION")

leader: Person @relation(name:"LED_BY")

founder: Person @relation(name:"FOUNDED_BY")

allies: [House] @relation(name:"ALLIED_WITH",

direction:IN)

follows: House @relation(name:"SWORN_TO")

followers: [House] @relation(name:"SWORN_TO",

direction:IN)

heir: [Person] @relation(name:"HEIR_TO",

direction:IN)

}

type Person {

id: ID!

name: String!

aliases: [String]

books: [Int]

tvSeries: [String]

playedBy: [String]

isFemale: Boolean

culture: String

died: String

titles: [String]

founded: [House] @relation(name:"FOUNDED_BY",

direction:IN)

leads: [House] @relation(name:"LED_BY",

direction:IN)

inherits: [House] @relation(name:"HEIR_TO")

spouse: [Person] @relation(name:"SPOUSE",

direction:BOTH)

parents: [Person] @relation(name:"PARENT_OF",

direction:IN)

children: [Person] @relation(name:"PARENT_OF")

houses: [House] @relation(name:"ALLIED_WITH")

}

SPIN UP AN ENDPOINT

$ npm install –g neo4j-graphql-cli

$ neo4j-graphql got-schema.graphql

# configure graphql-cli

$ npm install –g graphql-cli

$ graphql init

# add Auth Headers

Visualize Schema

call graphql.schema();

Visualize Schema

$ graphql voyager

LOADING THE

DATA

LOAD VIA API

• Iterate over batches of pages

• Data cleanup

• Replace URLs with IDs

• Throttle

• INSERT COMPLEX SCRIPT HERE

unwind range(1,43) as page

call apoc.util.sleep(1)

with page, 'characters' as type

call apoc.load.jsonArray('https://www.anapioficeandfire.com/api/'+type+'?pageSize=50&page='+page) yield value

with apoc.convert.toMap(value) as data

MERGE (p:Person {id:split(data.url,"/")[-1]})

SET

p += apoc.map.clean(data, ['allegiances','books','father','spouse','mother'],['',[''],[]]),

p.books = [b in data.books | split(b,'/')[-1]],

p.name = colaesce(p.name,head(p.aliases))

FOREACH (a in data.allegiances | MERGE (h:House {id:split(a,'/')[-1]}) MERGE (p)-[:SWORN_TO]->(h))

FOREACH (f in case coalesce(data.father,"") when "" then [] else [data.father] end | MERGE (o:Person {id:split(f,'/')[-1]})

MERGE (o)-[:PARENT_OF {type:'father'}]->(p))

FOREACH (f in case coalesce(data.mother,"") when "" then [] else [data.mother] end | MERGE (o:Person {id:split(f,'/')[-1]})

MERGE (o)-[:PARENT_OF {type:'mother'}]->(p))

FOREACH (f in case coalesce(data.spouse,"") when "" then [] else [data.spouse] end | MERGE (o:Person {id:split(f,'/')[-1]})

MERGE (o)-[:SPOUSE]-(p))

return p.id, p.name;

BETTER!FULL JSON DATA

• Full JSON data in Github Repository

• Load all characters and houses in one go

• No URL conversion

• But lowercasing keys

• Offline and much faster

call apoc.load.jsonArray('https://raw.githubusercontent.com/joakimskoog/AnApiOfIceAndFire/master/data/houses.json') yield value

with apoc.convert.toMap(value) as data

with apoc.map.clean(data, [],['',[''],[],null]) as data

with apoc.map.fromPairs([k in keys(data) | [toLower(substring(k,0,1))+substring(k,1,length(k)), data[k]]]) as data

MERGE (h:House {id:data.id})

SET

h += apoc.map.clean(data, ['overlord','swornMembers','currentLord','heir','founder','cadetBranches'],['',[''],[],null])

FOREACH (id in data.swornMembers | MERGE (o:Person {id:id}) MERGE (o)-[:ALLIED_WITH]->(h))

FOREACH (s in data.seats | MERGE (seat:Seat {name:s}) MERGE (seat)-[:SEAT_OF]->(h))

FOREACH (id in data.cadetBranches | MERGE (b:House {id:id}) MERGE (b)-[:BRANCH_OF]->(h))

FOREACH (id in case data.overlord when null then [] else [data.overlord] end | MERGE (o:House {id:id}) MERGE (h)-[:SWORN_TO]->(o))

FOREACH (id in case data.currentLord when null then [] else [data.currentLord] end | MERGE (o:Person {id:id}) MERGE (h)-[:LED_BY]->(o))

FOREACH (id in case data.founder when null then [] else [data.founder] end | MERGE (o:Person {id:id}) MERGE (h)-[:FOUNDED_BY]->(o))

FOREACH (id in case data.heir when null then [] else [data.heir] end | MERGE (o:Person {id:id}) MERGE (o)-[:HEIR_TO]->(h))

FOREACH (r in case data.region when null then [] else [data.region] end | MERGE (o:Region {name:r}) MERGE (h)-[:IN_REGION]->(o))

return h.id, h.name;

ITS A GRAPH

ITS A

GRAPH

QuERY Data with Cypher

MISSING DATA !?

MATCH (p:Person)

WHERE size(p.tvSeries) > 1

AND NOT exists((p)-[:PARENT_OF]-())

RETURN p LIMIT 10;

• Walder • The waif• High Septon• Margaery Tyrell• Tywin Lannister• Unella• Aemon Targaryen• Alliser Thorne• Arya Stark • Asha Greyjoy

QuERY Data

WITH

Graphql

RESOURCES

Code and instructions for

todays presentation

• GitHub

– https://github.com/neo4j-examples/game-of-thrones

• Medium Post

– medium.com/@mesirii/a-game-of-data-and-graphql-97ee2ca297ce

• Graph of Thrones 7 week contest

– https://neo4j.com/blog/graph-of-thrones/

• An API of Ice and Fire

– https://github.com/joakimskoog/AnApiOfIceAndFire

– https://anapioficeandfire.com/About

• GraphQL CLI Load

– https://npmjs.org/package/graphql-cli-load

Neo 4J GraphQL

• http://neo4j.com/developer/graphql

• https://npmjs.org/package/neo4j-graphql-cli

• http://communitygraph.org/graphql

• http://github.com/neo4j-graphql

– graphiql4all

– graphql-cli-load

– neoj4-graphql-cli

– community graph

– ...

More Fun

with Graphs and

Game of Thrones

• Character Interactions

– https://networkofthrones.wordpress.com/

– :play https://guides.neo4j.com/got

• Graph Analytics of Character Interactions

– http://www.lyonwj.com/2016/06/26/graph-of-thrones-neo4j-social-network-analysis/

• NLP on the book texts

– https://graphaware.com/neo4j/2017/07/24/reverse-engineering-book-stories-nlp.html

• The Maths of Game of Thrones

– https://anthonybonato.com/2016/04/13/the-mathematics-of-game-of-thrones/

• Kaggle GoT Data (Battles)

– https://tbgraph.wordpress.com/?s=Game+of+Thrones

QUESTIONS

@mesirii | neo4j.com/slack