cassandra froscon 10
DESCRIPTION
TRANSCRIPT
![Page 2: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/2.jpg)
Saturday, August 21, 2010
![Page 3: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/3.jpg)
Why NoSQL?
✤ Relational databases don’t scale
✤ The relational model maps poorly to some problems
✤ Relational databases are slow
Saturday, August 21, 2010
![Page 4: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/4.jpg)
Myth 1
✤ “NoSQL is for people who don’t understand {SQL, denormalization, query tuning, ...}”
✤ Similarly: “Only users of [database X] are turning to NoSQL databases, because X sucks.”
Saturday, August 21, 2010
![Page 5: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/5.jpg)
eBay: NoSQL pioneer
✤ “BASE is diametrically opposed to ACID. Where ACID is pessimistic and forces consistency at the end of every operation, BASE is optimistic and accepts that the database consistency will be in a state of flux. Although this sounds impossible to cope with, in reality it is quite manageable and leads to levels of scalability that cannot be obtained with ACID.”
✤ --BASE: An Acid Alternative, Dan Pritchett, eBay
✤ See also: The eBay Architecture, Randy Shoup and Dan Pritchett
Saturday, August 21, 2010
![Page 6: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/6.jpg)
(“The eBay Architecture,” Randy Shoup and Dan Pritchett)
Saturday, August 21, 2010
![Page 7: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/7.jpg)
Myth 2
✤ “NoSQL is nothing new because we had key/value databases like bdb years ago.”
Saturday, August 21, 2010
![Page 8: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/8.jpg)
Myth 3
✤ “Only huge web 2.0 sites like Facebook and Twitter care about scalability.”
Saturday, August 21, 2010
![Page 9: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/9.jpg)
Cassandra
✤ Relational databases don’t scale
✤ The relational model maps poorly to some problems
✤ Relational databases are slow
Saturday, August 21, 2010
![Page 10: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/10.jpg)
Saturday, August 21, 2010
![Page 11: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/11.jpg)
Saturday, August 21, 2010
![Page 12: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/12.jpg)
Saturday, August 21, 2010
![Page 13: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/13.jpg)
Saturday, August 21, 2010
![Page 14: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/14.jpg)
Saturday, August 21, 2010
![Page 15: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/15.jpg)
Saturday, August 21, 2010
![Page 16: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/16.jpg)
Use cases
✤ Digital Reasoning: NLP + entity analytics
✤ OpenX: largest publisher-side ad network in the world
✤ Cloudkick: performance data & aggregation
✤ SimpleGEO: location-as-API
✤ Ooyala: video analytics and business intelligence
✤ ngmoco: massively multiplayer game worlds
Saturday, August 21, 2010
![Page 17: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/17.jpg)
Cassandra
✤ Relational databases don’t scale
✤ The relational model maps poorly to some problems
✤ Relational databases are slow
Saturday, August 21, 2010
![Page 18: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/18.jpg)
Saturday, August 21, 2010
![Page 19: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/19.jpg)
Cassandra memory-aware index
<row data 0><row data 1>
...<row data 127>
...<row data 255>
...
<key 127><key 255>
...
Saturday, August 21, 2010
![Page 20: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/20.jpg)
Commitlog
MemtableWriterReader
The Log-Structured Merge-Tree,Bigtable: A Distributed Storage System for Structured Data
Saturday, August 21, 2010
![Page 21: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/21.jpg)
Saturday, August 21, 2010
![Page 22: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/22.jpg)
Durable
✤ Write to commitlog
✤ fsync is cheap since it’s append-only
✤ Write to memtable
✤ [amortized] flush memtable to sstable
Saturday, August 21, 2010
![Page 23: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/23.jpg)
Scaling
Saturday, August 21, 2010
![Page 24: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/24.jpg)
A
L
T
W
Saturday, August 21, 2010
![Page 25: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/25.jpg)
A
L
T
W
F
Saturday, August 21, 2010
![Page 26: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/26.jpg)
A
L
T
W
F(A-L]
Saturday, August 21, 2010
![Page 27: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/27.jpg)
A
L
T
W
F(A-F]
(F-L]
Saturday, August 21, 2010
![Page 28: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/28.jpg)
A
L
T
W
F
Key “C”
Saturday, August 21, 2010
![Page 29: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/29.jpg)
Reliability
✤ No single points of failure, clean design
✤ Multiple datacenters
✤ Monitorable
Saturday, August 21, 2010
![Page 30: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/30.jpg)
Saturday, August 21, 2010
![Page 31: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/31.jpg)
Some headlines
✤ “Resyncing Broken MySQL Replication”
✤ “How To Repair MySQL Replication”
✤ “Fixing Broken MySQL Database Replication”
✤ “Replication on Linux broken after db restore”
✤ “MySQL :: Repairing broken replication”
Saturday, August 21, 2010
![Page 32: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/32.jpg)
The opposite of heroes
✤ “If your software wakes someone up at 4 AM to fix it, you’re doing it wrong.”
Saturday, August 21, 2010
![Page 33: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/33.jpg)
Good architecture solves multiple problems at once
✤ Availability in single datacenter
✤ Availablility in multiple datacenters
✤ Built-in repair rather than an afterthought
✤ or worse, having to restart replication entirely
Saturday, August 21, 2010
![Page 34: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/34.jpg)
A
LT
W
F
P
YKey “C”
U
Saturday, August 21, 2010
![Page 35: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/35.jpg)
A
LT
W
P
Y
UF
Saturday, August 21, 2010
![Page 36: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/36.jpg)
A
LT
W
F
P
Y
U
Saturday, August 21, 2010
![Page 37: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/37.jpg)
A
LT
W
F
P
YKey “C”
U
Saturday, August 21, 2010
![Page 38: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/38.jpg)
A
LT
W
F
P
YKey “C”
U
Saturday, August 21, 2010
![Page 39: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/39.jpg)
A
LT
W
F
P
YKey “C”
U
Saturday, August 21, 2010
![Page 40: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/40.jpg)
A
LT
W
F
P
YKey “C”
U
Saturday, August 21, 2010
![Page 41: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/41.jpg)
A
LT
W
F
P
YKey “C”
U
Saturday, August 21, 2010
![Page 42: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/42.jpg)
A
LT
W
F
P
YKey “C”
U
XSaturday, August 21, 2010
![Page 43: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/43.jpg)
A
LT
W
F
P
YKey “C”
U
Xhint
Saturday, August 21, 2010
![Page 44: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/44.jpg)
A
LT
W
F
P
YKey “C”
U
Xhint
Saturday, August 21, 2010
![Page 45: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/45.jpg)
A
LT
W
F
P
Y
U
Saturday, August 21, 2010
![Page 46: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/46.jpg)
A
LT
W
F
P
Y
U
Saturday, August 21, 2010
![Page 47: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/47.jpg)
A
LT
W
F
P
Y
U
Saturday, August 21, 2010
![Page 48: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/48.jpg)
Saturday, August 21, 2010
![Page 49: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/49.jpg)
Eventual Tuneable consistency
✤ ONE, QUORUM, ALL
✤ R + W > N
✤ Choose availability vs consistency (and latency)
Saturday, August 21, 2010
![Page 50: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/50.jpg)
Monitorable
Saturday, August 21, 2010
![Page 51: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/51.jpg)
Events
Saturday, August 21, 2010
![Page 52: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/52.jpg)
0
375
750
1125
1500
Jan(0.5)
Feb(0.5.1) Mar
Apr(0.6, 0.6.1)
May(0.6.2)
Jun(0.6.3)
Jul(0.6.4)
mails sent
Saturday, August 21, 2010
![Page 53: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/53.jpg)
Data model
✤ Twitter: “Fifteen months ago, it took two weeks to perform ALTER TABLE on the statuses [tweets] table.”
Saturday, August 21, 2010
![Page 54: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/54.jpg)
Columns
ColumnFamilies
Saturday, August 21, 2010
![Page 55: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/55.jpg)
Saturday, August 21, 2010
![Page 56: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/56.jpg)
SELECT * FROM tweetsWHERE user_id IN (SELECT follower FROM followers WHERE user_id = ?)ORDER BY time_tweeted DESCLIMIT 40
followers
?
tweets
timeline
?
uuid:tweet
Cassandra “indexes” are materialized views
SORT
Saturday, August 21, 2010
![Page 57: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/57.jpg)
Analytics in Cassandra
✤ @afex: “Cassandra + Pig (Hadoop) is very exciting. A 7 line script to analyze data from my entire cluster transparently, with no ETL? Yes, please”
Saturday, August 21, 2010
![Page 58: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/58.jpg)
TaskTracker
JobTracker
Saturday, August 21, 2010
![Page 59: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/59.jpg)
Coming in 0.7 [beta1 released 13/8]
✤ Secondary indexes and expressions
✤ Large (> 2GB) rows
✤ More control over replica placement
✤ Online schema changes
✤ Flow control
✤ Dynamic routing around slow nodes
Saturday, August 21, 2010
![Page 60: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/60.jpg)
More
✤ http://wiki.apache.org/cassandra/ArticlesAndPresentations
✤ http://wiki.apache.org/cassandra/ArchitectureInternals
Saturday, August 21, 2010
![Page 61: Cassandra FrOSCon 10](https://reader033.vdocuments.site/reader033/viewer/2022051322/545826d7af79590b088b5334/html5/thumbnails/61.jpg)
Questions
Saturday, August 21, 2010