rolling with riak
DESCRIPTION
Overview of Riak's NoSQL distributed key/value data store by John Lynch from Rigel Group.TRANSCRIPT
![Page 2: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/2.jpg)
Web App Developers
ORMFocus on the AppSpeed of DevelopmentDB Agnostic(ish)
Fixed SchemaLimits design choicesMigration HellScaling at DB layer
Rails!
![Page 3: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/3.jpg)
Shiny New Toys
Decades of research and best practicesAwesome ad-hoc query capabilityZillions of vendors/tools/libraries/code
Flexibility of schema-less designAbility to scale…. Web-scale
![Page 4: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/4.jpg)
Web Scale Changing App Types
SocialGamesMarketing / Advertising
Freemium Business Models1M free => 10K paying customers
![Page 5: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/5.jpg)
NoSQL Landscape
Pure Key/Value (Redis/Tokyo Cabinet/etc)
Key/Value+ (CouchDB/MongoDB/Riak)
BigTable Type (Hbase, HyperTable)
Choose wisely! No standard API.
(Good general overview can be found here: http://cattell.net/datastores/Datastores.pdf)
![Page 6: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/6.jpg)
MongoDB
Popular with Ruby community Combines Key/Value with ability to do
Indexed Queries
![Page 7: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/7.jpg)
Scaling MongoDB
Master, Slave, Replica Set, Replica Pair, Shard Server, Connection Pool, ack!
![Page 8: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/8.jpg)
Scaling MongoDB
![Page 9: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/9.jpg)
If all you want is NoSQL…
NoSQL on MySQL
Leverages all MySQL skills, tools, techniques, stability, dependability
If you want NoSQL + Scalability…
…not so much.
![Page 10: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/10.jpg)
Riak
Developed by Basho.com Used on several large production sites Written in Erlang Distributed – Fault Tolerant Buckets – Keys – Values Values can be anything (json,binary,etc) Ruby & Rails Client (Ripple project @ Github)
![Page 11: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/11.jpg)
Riak speaks HTTP
> curl –i http://host:8098/riak/bucket1/key1
HTTP/1.1 200 OK X-Riak-Vclock: awpcFAA== Content-Type: text/plain Content-Length: 9Last-Modified: Wed, 01 Se…Etag: 45364657
I am a value
Leverage existing HTTP infrastructure, tools, etc
![Page 12: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/12.jpg)
Scaling Riak
Riak Riak Riak Riak Riak
Http Load BalancerVarnish (cache)
Standard HTTP Protocol
Rails Rails Rails Rails Rails
![Page 13: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/13.jpg)
Scaling Riak (alt)
RiakRailsNginx
RiakRailsNginx
RiakRailsNginx
RiakRailsNginx
RiakRailsNginx
Http Load BalancerVarnish (cache)
Standard HTTP Protocol
![Page 14: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/14.jpg)
Key Differentiator - Distributed Inspired by Amazon’s Dynamo Uses consistent hashing algorithim No “Master Node” No single point of failure Any node can service any request Automatically rebalances as nodes join Tunable CAP Properties
Consistency, Availability, Partition Tolerance
![Page 15: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/15.jpg)
N R W
N = # of copies of the data R = # of nodes necessary to read W = # of nodes necessary to write
Tunable by the application, on a per-bucket and per-query basis
![Page 16: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/16.jpg)
Riak cluster of 4 Physical Computers
Low Value Data (N=2 R=1 W=1)Logging
Web Content (N=4 R=1 W=4)Maximum availability and consistency
Financial Data (N=4 R=1 W=4 DW=4)DW is “Durable Write”
![Page 17: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/17.jpg)
Network Split
![Page 18: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/18.jpg)
Network Split
![Page 19: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/19.jpg)
Network Split
![Page 20: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/20.jpg)
Map/Reduce
Map steps run on each node Final reduce runs on single node
results = Riak::MapReduce.new(client). add(“albums”). map("function(v){ return [JSON.parse(v.values[0].data).title]; }", :keep => true).run
![Page 21: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/21.jpg)
Links
Riak documents can have links to other documents, each link can be “tagged”
Link data is separate from doc data Easy URL access to walk these links
GET /riak/artists/TheBeatles/albums,_,_/tracks,_,1
![Page 22: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/22.jpg)
When NOT to use Riak
Single machine Small scale or bog-standard apps Need rich ad-hoc indexed queries Need mature tools and libraries
![Page 23: Rolling With Riak](https://reader036.vdocuments.site/reader036/viewer/2022081414/54b7600e4a7959f71f8b465a/html5/thumbnails/23.jpg)
Any questions?
(First round at Rock Bottom generously sponsored by Basho.com)