yes,%sql!% - qcon · 2014. 9. 16. ·...
TRANSCRIPT
![Page 1: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/1.jpg)
Yes, SQL!
Uri Cohen
![Page 2: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/2.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
> SELECT * FROM qcon.speakers WHERE name=‘Uri Cohen’
+-----------------------------------------------------+ | Name | Company | Role | Twitter | +-----------------------------------------------------+ | Uri Cohen | GigaSpaces | Product Manager | @uri1803 | +-----------------------------------------------------+
2
> db.speakers.find({name:”Uri Cohen”}) { “name”:”Uri Cohen”, “company”: { name:”GigaSpaces”, products:[“XAP”, “IMDG”] domain: “In memory data grids” } “role”:”product manager”, “twitter”:”@uri1803” }
![Page 3: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/3.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Agenda
3
• SQL – What it is and isn’t good for
• NoSQL – Mo;va;on & Main Concepts of Modern Distributed Data Stores
– Common interac;on models
• Key/Value, Column, Document
• NOT consistency and distribu;on algorithms
• One Data Store, Mul;ple APIs – (Really) brief intro to GigaSpaces
– SQL challenges: Add-‐hoc querying, Rela;onships (JPA)
![Page 4: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/4.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
A FEW (MORE) WORDS ABOUT SQL
4
![Page 5: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/5.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
SQL
(Usually) Centralized à TransacAonal, consistent à Hard to Scale
5
![Page 6: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/6.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
SQL
Sta;c, normalized data schema • Don’t duplicate, use FKs
6
![Page 7: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/7.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
SQL
Add hoc query support à Model first, query later
7
![Page 8: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/8.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
SQL
Standard à Well known à Rich ecosystem
8
![Page 9: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/9.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
(BRIEF) NOSQL RECAP
9
![Page 10: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/10.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
NoSql (or a Naive AYempt to Define It)
A loosely coupled collecAon of non-‐rela;onal data stores
10
![Page 11: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/11.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
NoSql (or a Naive AYempt to Define It)
(Mostly) d i s t r i b u t e d
11
![Page 12: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/12.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
NoSql (or a Naive AYempt to Define It)
scalable (Up & Out)
12
![Page 13: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/13.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
NoSql (or a Naive AYempt to Define It)
Not (always) ACID • BASE anyone?
13
![Page 14: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/14.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Why Now?
Timing is everything… • ExponenAal Increase in data & throughput • Non or semi structured data that changes frequently
14
![Page 15: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/15.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
A Universe of Data Models
15
Key / Value Column { “name”:”uri”, “ssn”:”213445”, “hobbies”:[”…”,“…”], “…”: { “…”:”…” “…”:”…” } }
{ { ... } }
{ { ... } }
Document
![Page 16: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/16.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Key/Value
• Have the key? Get the value – That’s about it when it comes to querying
– Map/Reduce (someAmes)
– Good for • cache aside (e.g. Hibernate 2nd level cache)
• Simple, id based interacAons (e.g. user profiles)
• In most cases, values are Opaque
16
K1 V1 K2 V2 K3 V3 K4 V1
![Page 17: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/17.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Key/Value
Scaling out is rela;vely easy (just hash the keys) • Some will do that automaAcally for you • Fixed vs. consistent hashing
17
![Page 18: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/18.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Key/Value
• Implementa;ons: – Memcached, Redis, Riak
– In memory data grids (mostly Java-‐based) started this way
• GigaSpaces, Oracle Coherence, WebSphere XS, JBoss Infinispan, etc.
18
![Page 19: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/19.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Column Based
19
![Page 20: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/20.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Column Based
• Mostly derived from Google’s BigTable / Amazon Dynamo papers
• One giant table of rows and columns – Column == pair (name and a value, someAmes Amestamp)
– Each row can have a different number of columns
– Table is sparse: (#rows) × (#columns) ≥ (#values)
20
![Page 21: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/21.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Column Based
• Query on row key – Or column value (aka secondary index)
• Good for a constantly changing, (albeit flat) domain model
21
![Page 22: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/22.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Document
Think JSON (or BSON, or XML)
22
_id:1
_id:2
_id:3
{ “name”:”Lady Gaga”, “ssn”:”213445”, “hobbies”:[”Dressing up”,“Singing”], “albums”: [{“name”:”The fame” “release_year”:”2008”}, {“name”:”Born this way” “release_year”:”2011”}] }
{ { ... } }
{ { ... } }
![Page 23: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/23.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Document
• Model is not flat, data store is aware of it – Arrays, nested documents
• Beler support for ad hoc queries – MongoDB excels at this
• Very intuiAve model • Flexible schema
![Page 24: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/24.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
What if you didn’t have to choose?
24
JPA
JDBC
![Page 25: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/25.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
A Brief Intro to GigaSpaces
In Memory Data Grid • With op;onal write behind to a secondary storage
25
![Page 26: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/26.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
A Brief Intro to GigaSpaces
Tuple based • Aware of nested tuples (and soon collec;ons)
– Document like
• Rich querying and map/reduce seman;cs
26
![Page 27: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/27.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
A Brief Intro to GigaSpaces
Transparent par;;oning & HA • Fixed hashing based on a chosen property
27
JAVA Virtual MachineJAVA Virtual Machine JAVA Virtual MachineJAVA Virtual Machine
Replication
Primary 1Backup 1
Replication
Backup 2Primary 2
![Page 28: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/28.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
A Brief Intro to GigaSpaces
Transac;onal (Like, ACID) • Local (single par;;on) • Distributed (mul;ple par;;ons)
28
![Page 29: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/29.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Use the Right API for the Job
29
• Even for the same data… – POJO & JPA for Java apps with complex domain model
– Document for a more dynamic view
– Memcached for simple, language neutral data access
– JDBC for: • InteracAon with legacy apps • Flexible ad-‐hoc querying (e.g. projecAons)
![Page 30: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/30.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Memcached (the Daemon is in the Details)
Memcached Client
![Page 31: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/31.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Memcached Client
Memcached (the Daemon is in the Details)
![Page 32: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/32.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
SQL/JDBC – Query Them All
32
Query may involve Map/Reduce • Reduce phase includes merging and sorAng
![Page 33: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/33.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
SQL/JDBC – Things to Consider
• Unique and FK constraints are not prac;cally enforceable
• Sor;ng and aggrega;on may be expensive • Distributed transac;ons are evil
– Stay local…
33
![Page 34: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/34.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
JPA
It’s all about rela;onships…
34
![Page 35: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/35.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
JPA Rela;onships
To embed or not to embed, that is the ques;on….
ü Easy to par;;on and scale ü Easy to query: user.accounts[*].type = ‘checking’
× Owned rela;onships only
![Page 36: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/36.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
JPA Rela;onships
To embed or not to embed, that is the ques;on….
ü Any type of rela;onship × Par;;oning is hard × Querying involves joining
![Page 37: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/37.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Summary
• One API doesn’t fit all – Use the right API for the job
• Know the tradeoffs – Always ask what you’re giving up, not just what you’re gaining
37
![Page 38: Yes,%SQL!% - QCon · 2014. 9. 16. · ®"Copyright2010"Gigaspaces"Ltd."All"Rights"Reserved" AFEW%(MORE)%WORDS%ABOUT%SQL% 4"](https://reader035.vdocuments.site/reader035/viewer/2022071008/5fc5bf7713483d7e60457329/html5/thumbnails/38.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
THANK YOU!
@uri1803
hYp://blog.gigaspaces.com
38