couchconf israel 2013_developing with cb ii
TRANSCRIPT
1 1
Developing With Documents: Schemas and Rela8onships
Ma: Ingenthron Director, Developer Solu8ons
2
What we’ll talk about
• Working with JSON documents • Run8me-‐driven pa:erns for
• linking between documents • embedding data • fetching mul8ple documents
• Considera8ons with mul8-‐datacenter deployments
3
JSON DOCUMENTS
3
4
Couchbase Server is a Document Database
h:p://mar8nfowler.com/bliki/AggregateOrientedDatabase.html
5
Document Database
• Easy to distribute data • Makes sense to applica8on programmers
This synergy between the programming model and the distribu5on model is very valuable. It allows the database to use its knowledge of how the applica5on programmer clusters the data to help performance across the cluster.
h:p://mar8nfowler.com/bliki/AggregateOrientedDatabase.html
6
JSON Documents
• Maps more closely to external API • CRUD Opera8ons, lightweight schema
• Stored under an iden8fier key
{ ! “fields” : [“with basic types”, 3.14159, true], ! “like” : “your favorite language”!} !!!client.set(“mydocumentid”, myDocument); !mySavedDocument = client.get(“mydocumentid”); !
7
Meta + Document Body
{ "brewery": "New Belgium Brewing", "name": "1554 Enlightened Black Ale", "abv": 5.5, "descrip8on": "Born of a flood...", "category": "Belgian and French Ale", "style": "Other Belgian-‐Style Ales", "updated": "2010-‐07-‐22 20:00:20" } { "id" : "beer_Enlightened_Black_Ale”, ... {
Document user data,
can be anything
unique ID
Metadata identifier,
expiration, etc
“vintage” date format from an SQL dump >_<
8
No More ALTER table
• No More alter table • More produc8ve developers! • Emergent schema— – the next session is about views – This session is about working directly with documents from interac8ve applica8on code
• Schema driven by code
9
LIVE DATA
9
10
Real8me Interac8ve CRUD
• Requirement: high-‐performance with high-‐concurrency and dynamic scale
• Key/value API is the interac8ve low-‐latency path
11
Run8me Driven Schema
• What’s in the database looks more like your code • Thinking about throughput, latency, update and read pa:erns is
the new data modeling • Data flows get more a:en8on than data at rest
• When should I split a data-‐structure into mul8ple documents? • Generally the more useful your document is as a standalone en8ty, the be:er.
• Documents that grow without bound are bad
12
LINK BETWEEN DOCUMENTS
12
13
Let’s Add Comments and Ra8ngs to the Beer
• Challenge linking items together • Whether to grow an exis8ng item or store independent documents
• No transac8onality between documents!
I give that a 5!
good w/ burgers
tastes like college!
{ "brewery": "New Belgium Brewing", "name": "1554 Enlightened Black Ale", "abv": 5.5, "descripAon": "Born of a flood...", "category": "Belgian and French Ale", "style": "Other Belgian-‐Style Ales", "updated": "2010-‐07-‐22 20:00:20" }
14
Let’s Add Comments and Ra8ngs to the Beer
• We’ll put comments in their own document • And add the ra8ngs to the beer document itself.
{ "type": "comment", "about_id": "beer_Enlightened_Black_Ale", "user_id": 525, "text": "tastes like college!", "updated": "2010-‐07-‐22 20:00:20" }
I give that a 5!
{ "brewery": "New Belgium Brewing", "name": "1554 Enlightened Black Ale", "abv": 5.5, "descripAon": "Born of a flood...", "category": "Belgian and French Ale", "style": "Other Belgian-‐Style Ales", "updated” : "2010-‐07-‐22 20:00:20", “raAngs” : { “525” : 5, “30” : 4, “1044” : 2 }, “comments” : [ “f1e62”, “6ad8c” ] }
15
Do it: save the comment document
• Set at the id “f1e62”
client.set(“f1e62”,{ !!
!!!!!!}); !
create a new document
{ "id": "f1e62" }
"type": "comment", "about_id”: "beer_Enlightened_Black_Ale", "user_id": 525, "text": "tastes like college!", "updated": "2010-‐07-‐22 20:00:20"
16
Link between comments and beers
"name": "1554 Enlightened Black Ale", "abv": 5.5, "descripAon": "Born of a flood...", "category": "Belgian and French Ale", "style": "Other Belgian-‐Style Ales", "updated": "2010-‐07-‐22 20:00:20", “ra8ngs” : { “525” : 5, “30” : 4, “1044” : 2 }, “comment_ids” : [ “f1e62”, “6ad8c” ] }
16
{ "type": "comment", "about_id": "beer_Enlightened_Black_Ale", "user_id": 525, "text": "tastes like college!", "updated": "2010-‐07-‐22 20:00:20" }
link to comments
link to beer
{ "id": "f1e62" }
17
How to: look up comments from a beer
• SERIALIZED LOOP beer = client.get(“beer:A_cold_one”); !beer.comment_ids.each { |id| ! comments.push(client.get(id)); !} !
• ASYNC VIEW QUERY comments = client.asyncGet(“myapp”,“by_comment_on”, ! {:key => “beer:A_cold_one”}); !!
• FAST MULTI-‐KEY LOOKUP beer = client.get(“beer:A_cold_one”); !comments = client.multiGet(beer.comment_ids) !!
18
How to: add a ra8ng to a beer
• Other users are ra8ngs beers also, so we use a CAS update – we don’t want to accidentally overwrite another users ra8ng that is being saved at
the same 8me as ours
• Retry the opera8on, if appropriate – Also useful if you have internal structure that you want to maintain
cb.cas("mykey") do |doc| ! doc["ratings"][current_user.id] = my_rating! doc !end !!
Actor 1 Actor 2
Couchbase Server
CAS mismatch & retry
Success
19
Object Graph With Shared Interac8ve Updates
• Challenge: higher level data structures • Objects shared across mul8ple users • Mixed object sets (upda8ng some private and some shared objects)
Only marginally related figure courtesy of h:p://www.ibm.com/developerworks/webservices/library/ws-‐sdoarch/
20
Get With Lock (GETL)
• Owen referred to as “GETL” • Pessimis8c concurrency control • Locks have a short TTL • Locks released with CAS opera8ons • Useful when working with object graphs
S8ll only marginally related figure courtesy of h:p://www.ibm.com/developerworks/webservices/library/ws-‐sdoarch/
21
MULTI-‐DATACENTER CONSISTENCY?
21
22
You Want Datacenter Affinity
• No ACID across documents, need resilient code • Locks and counters should be per-‐datacenter • Impacts opera8ons like INCR DECR and CAS
US DATA CENTER EUROPE DATA CENTER ASIA DATA CENTER
Replica8on Replica8on
Replica8on
23
Conclusion and Next Session Summary
• JSON documents • Run8me-‐driven schema NEXT UP: Views • See inside the data • Prac8cal pa:erns
