nosql @ codemash 2010

bensco'ield  –  viget  labscodemash14  january  2010

NoSQLDeath  to  Relational  Databases(?)

1

Motivations

2

Performance

3

Scalability

4

Meh

5

Flexibility

6

Complexity

7

“Comics”  Is  Hard

8

9

Functionality

10

Charlie  Chaplin

Jet  Li

Marian  Collier

Hank  Mann

11

Taxonomy

12

Key-­‐Value  Stores

13

distributed  hash  tables

14

PerformanceScalabilityFlexibilityComplexityFunctionality

highhighhighnonevariable  (none)

15

DynamoGT.MPStoreRedis

16

Column-­‐Oriented  Stores

17

semi-­‐structured

18

PerformanceScalabilityFlexibilityComplexityFunctionality

highhighmoderatelowminimal

19

BigTableCassandraHBase

20

Document-­‐Oriented  Stores

21

also  semi-­‐structured

22

PerformanceScalabilityFlexibilityComplexityFunctionality

highvariable  (high)highlowvariable  (low)

23

CouchDBMongoDBRDDBRiak

24

Graph  Databases

25

graph  theory

26

PerformanceScalabilityFlexibilityComplexityFunctionality

variablevariablehighhighgraph  theory

27

ActiveRDF  AllegroGraphNeo4J

28

Relational  Databases

29

PerformanceScalabilityFlexibilityComplexityFunctionality

variablevariablelowmoderaterelational  algebra

30

Examples

31

Redis

32

Data  Typesstringslistssetssorted  sets

33

In-­‐Memoryperiodic  snapshots  /  append  only  'ilemaster-­‐slave  replicationmemory-­‐bound

34

require 'redis'

gl = Redis.new

# A stringgl['name'] = 'Kyle Rayner'gl['name']gl.delete('name')

# A listgl.push_tail 'to-dos', 'Lose Ion power'gl.push_tail 'to-dos', 'Mourn dead loved ones'gl.push_tail 'to-dos', 'Blow up zombie lanterns'

gl.list_range('to-dos', 0, -1)

35

Tokyo  Cabinet

36

Data  Typesbinary  datastrings

37

Tables!?

38

Related  Projectstyrantdystopiapromenade

39

require 'rufus/tokyo'

# Key-valuejli = Rufus::Tokyo::Cabinet.new('jl.tch')jli['members'] = [ 'Batman', 'Black Canary', 'Blue Beetle', 'Captain Marvel', 'Doctor Light', 'Doctor Fate', 'Guy Gardner', 'Martian Manhunter', 'Mister Miracle'].to_yaml

YAML.load(jli['members'])

40

require 'rufus/tokyo'

# Tablebig7 = Rufus::Tokyo::Table.new('big7.tct')

big7['s'] = {'name' => 'Superman', 'role' => 'deus ex machina'}big7['b'] = {'name' => 'Batman', 'role' => 'mastermind'}big7['gl'] = {'name' => 'Green Lantern', 'role' => 'space cop'}big7['f'] = {'name' => 'Flash', 'role' => 'speedster'}big7['mm'] = {'name' => 'Martian Manhunter', 'role' => '?'}big7['ww'] = {'name' => 'Wonder Woman', 'role' => 'hitter'}big7['a'] = {'name' => 'Aquaman', 'role' => 'fish-talking'}

big7.query {|q| q.add_condition 'role', :streq, 'fish-talking'}

41

Cassandra

42

GenealogyDynamoBigTable

43

Column-­‐Orientedcolumnssupercolumnscolumn  families

44

Distributedautomatic  replicationeventual  consistencyeasy  expansion

45

Availabilityweak  readsquorum  reads

46

require 'cassandra'

op = Cassandra.new('OnePiece')

op.insert(:People, '1', {'name' => 'Luffy'})op.insert(:People, '2', {'name' => 'Crocodile'})op.insert(:People, '3', {'name' => 'Mr. 3'})

op.insert(:Fights, '1', {'opponents' => {UUID.new => '2'}})op.insert(:Fights, '1', {'opponents' => {UUID.new => '3'}})

luffy_fights = op.get(:Fights, '1', 'opponents')luffy_fights.map {|t, opp| op.get(:People, opp, 'name')}

47

CouchDB

48

Web-­‐InspiredJSON  storageHTTP  /  RESTful  interface

49

Viewsprede'ined,  updated  incrementallyjavascript  for  map/reduce

50

Updatesfull,  including  embedded  documents

51

require 'couchrest'

konoha = CouchRest.database!('http://127.0.0.1:5984/konoha')naruto = konoha.save_doc { 'name' => 'Naruto Uzumaki', 'chakra' => 'wind'}shikamaru = konoha.save_doc { 'name' => 'Shikamaru Nara', 'chunin' => true}

konoha.save_doc { '_id' => '_design/first', :views => { :chunin => { :map => 'function(doc){if(doc.chunin){emit(null, doc);}}' } }}

puts konoha.views('first/chunin')['rows'].inspect

52

MongoDB

53

Storagebinary  JSON  documents

54

Accessnative  clients

55

Queriesdynamicindex-­‐based

56

Updatesallows  partial  updates

57

require 'mongo'

avengers = Mongo::Connection.new.db('avengers')members = avengers.collection('members')

members.insert {'name' => 'Ant-Man'}members.insert {'name' => 'Hulk'}members.insert {'name' => 'Iron Man'}members.insert {'name' => 'Thor'}members.insert {'name' => 'Wasp'}

members.create_index('name')

pym = members.find {'name' => 'Ant-Man'}pym['name'] = 'Giant-Man'pym.save

members.remove {'name' => 'Hulk'}

members.insert {'name' => 'Captain America'}

58

Riak

59

also  Web-­‐InspiredJSON  storageHTTP  /  RESTful  interfacelinks  for  relationships

60

Decentralizedno  privileged  nodes

61

Con'igurablestore  /  read  /  write

62

require 'jiak'

jc = JiakClient.new('127.0.0.1', 8098)jc.set_bucket_schema('supervillains', { 'allowed_fields' => ['name', 'alias', 'power']})

jc.store({ 'bucket' => 'supervillains', 'key' => 'Normie', 'object' => { 'name' => 'Norman Osborn', 'alias' => 'Green Goblin', 'power' => 'Supreme jerkfacedness' }, 'links' => []})

kth = jc.fetch('supervillains', 'Normie')

63

Neo4J

64

Structurenodes  and  edgeskey-­‐value  pairs

65

Querieslucenegremlin

66

require 'neo4j'

class Person include Neo4j::NodeMixin property :name, :mutant index :name, :mutant has_n :crushes has_n :hookups has_n :marriages def initialize(name, mutant = true) name = name mutant = mutant endend

67

Neo4j::Transaction.run do magneto = Person.new('Magneto') esme = Person.new('Esme') rogue = Person.new('Rogue') magda = Person.new('Magda', false) wasp = Person.new('Wasp', false) magneto.crushes << wasp magneto.hookups << rogue magneto.marriages << magda esme.crushes << magneto rogue.hookups << magneto magda.marriages << magnetoend

68

magneto = Person.find(:name => 'Magneto')

# Who likes Magneto?magneto.relationships.incoming(:crushes).nodes

# Which non-mutants has Magneto dated?magneto.hookups{ !mutant? }.to_a

69

Simulations

70

Structure

71

people{ ‘name’:‘Jimmy Olsen’ ‘title’:‘Superman’s Pal’ ‘company_id’:12441}

companies{ _id:12441 ‘name’:‘Daily Planet’}

72

Lack  of  Structure

73

mysql> SELECT * FROM people LIMIT 1 \G*************************** 1. row *************************** id: 1content: --- company: Daily Planetname: Jimmy Olsentitle: Superman’s Pal

74

But  wait!friendfeedfriendly

75

mysql> desc people;+-------+-------------+------+-----+---------+-------+| Field | Type | Null | Key | Default | Extra |+-------+-------------+------+-----+---------+-------+| id | int(11) | YES | | NULL | || name | varchar(50) | YES | | NULL | |+-------+-------------+------+-----+---------+-------+

mysql> desc attributes;+-----------+--------------+------+-----+---------+-------+| Field | Type | Null | Key | Default | Extra |+-----------+--------------+------+-----+---------+-------+| id | int(11) | YES | | NULL | || person_id | int(11) | YES | | NULL | || attribute | varchar(50) | YES | | NULL | || value | varchar(100) | YES | | NULL | |+-----------+--------------+------+-----+---------+-------+

76

Not  Only  SQL

77

Polyglot  Persistence

78

Caching

79

Already  in  Usememcached

80

Queues

81

Long-­‐running  processesresque

82

Logging

83

Rails  Log  Replacementhttp://github.com/peburrows/mongo_db_logger

84

Hybrid  Domains

85

different  domains

86

Publishinge-­‐commercedocuments

87

Datinge-­‐commercesocial  graph

88

different  scales

89

Photo  Sharinguser  accountsuploaded  photos

90

Next  Steps

91

Explore

92

Database  Listhttp://internetmindmap.com/database_software

NoSQL  Google  Grouphttp://groups.google.com/group/nosql-­‐discussion

93

Ignore  the  Database

94

Logical  Modeling  Firstbe  mindful

95

Change  the  Default

96

Application  Templatesstart  with  something  new

97

@bsco'ieldben.sco'ield@viget.comhttp://spkr8.com/bsco'ieldhttp://viget.com/extendhttp:  //bensco'ield.com

bensco'ield

98