btree nosql oak
DESCRIPTION
An introduction to CouchDB's append-only storage model.TRANSCRIPT
CouchDBApache
Monday, November 2, 2009
how couchdb treats the disksappend only btree
Replication
JSON
Map Reduce
HTTPMonday, November 2, 2009
intro:offline appssimple developmenteasy to deploy
! !
Stuart Langridge - Canonical
Monday, November 2, 2009
15 million desktops by the end of the year.solve the data island problem
Monday, November 2, 2009
raindrop - next gen messaging platformname: your own personal piece of the cloud
Monday, November 2, 2009
Monday, November 2, 2009
Couchapps -- easiest way to deploy an Ajax application that needs to store data.-- capable of scaling up with CouchDB. (etags etc)
http://www.flickr.com/photos/mcpig/872293700/
“Ground Computing”@jhuggins
Monday, November 2, 2009
app runs near the user. lower latency, lower cost.
Monday, November 2, 2009
http://www.flickr.com/photos/shane-h/280084650
Offline by Default
Monday, November 2, 2009
reliabletelephone poles / like akamai but for dynamic database backed applications. - same application on the client and server or the cloud
http://jacobian.org/writing/of-the-web/
http://apod.nasa.gov/apod/ap050930.html
“Of the Web”
Monday, November 2, 2009
Let me tell you something: Django may be built for the Web, but CouchDB is built of the Web. I've never seen software that so completely embraces the philosophies behind HTTP. ... this is what the software of the future looks like. Jacob Kaplan-Moss -- October, 2007
perfect spot - discovered not invented
When you can move data via replication, it
changes the physics of the web
so data is useful independent of location.
LocklessStorage
Monday, November 2, 2009
html5 story js ruby transactions?
-- getting CouchDB right involves going all the way to core lockless storage engine.
performance, map reduce, and deployment
B-Tree
Monday, November 2, 2009
data structuresorted by keysminimize disk seeks
Monday, November 2, 2009
logical btree
Monday, November 2, 2009
indexes - efficientkey lookupsrange scans
Append
OnlyMonday, November 2, 2009
robust / reliablenever touch bytes that are on diskcan’t be inconsistent
Monday, November 2, 2009
append only - in actionupdate J to a new value (rewrite j’s leaf node)
Monday, November 2, 2009
rewrite the path along the tree to Jup the parent path until we write the root.
Monday, November 2, 2009
all pointers point backwards into the file
Monday, November 2, 2009
update cost?
only has O(log n) cost where n is the # of keys
Monday, November 2, 2009
another update - add 5.
rewrite path to Z node to add 5
Monday, November 2, 2009
time follows the blue arrow. top half is actually an accumulation of updates like the bottom.note: batch efficiency -- save block updates
Monday, November 2, 2009
2 day test runcouchdb over long haul4 writers, 16 readers, random access, small keyspace - lots of 409s etc.
Incremental
ReduceMonday, November 2, 2009
takes advantage of the write pattern we’ve seensetup main storage doc count example
Monday, November 2, 2009
rereduce = falsecount # of keys in the block
Monday, November 2, 2009
rereduce = truesum # of docs in child nodesflexible range querying
Monday, November 2, 2009
intermediate reductions stored during a view calculationupdating J doesn’t change any counts, adding the 5 key does
Monday, November 2, 2009
we are rewriting the parent nodes anyway, so it makes sense to run the reduce calculation
Concurrent Access
Monday, November 2, 2009
readers see a consistent viewwe can support many readersreads don’t block writeswrites don’t block reads
Monday, November 2, 2009
concurrent readerslist all docs, then a write comes in, you don’t want to be surprised by an extra docreaders can be as long lived as they want without blocking writers or seeing inconsistencies
Local Web Platform
Monday, November 2, 2009
user interviews
http://www.longnow.org/projects/clock/orrery/Monday, November 2, 2009
append-onlyno fixup phasereliable and concurrentsequential write pattern (few writer seeks)hot backupscompaction