cassandra @formspring
DESCRIPTION
TRANSCRIPT
![Page 1: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/1.jpg)
Cassandra @Formspring
Tuesday, August 30, 2011
![Page 2: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/2.jpg)
Cassandra @Formspring
Yet another Knight, Dragon and Princess Story.
Tuesday, August 30, 2011
![Page 3: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/3.jpg)
Formspring helps people find out more about each other by sharing interesting & personal responses
Tuesday, August 30, 2011
![Page 4: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/4.jpg)
25M users
Tuesday, August 30, 2011
![Page 5: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/5.jpg)
3.5B responses
Tuesday, August 30, 2011
![Page 6: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/6.jpg)
Tuesday, August 30, 2011
![Page 7: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/7.jpg)
Tuesday, August 30, 2011
![Page 8: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/8.jpg)
Tuesday, August 30, 2011
![Page 9: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/9.jpg)
Tuesday, August 30, 2011
![Page 10: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/10.jpg)
Tuesday, August 30, 2011
![Page 11: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/11.jpg)
Tuesday, August 30, 2011
![Page 12: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/12.jpg)
Tuesday, August 30, 2011
![Page 13: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/13.jpg)
Tuesday, August 30, 2011
![Page 14: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/14.jpg)
Tuesday, August 30, 2011
![Page 15: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/15.jpg)
Tuesday, August 30, 2011
![Page 16: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/16.jpg)
Tuesday, August 30, 2011
![Page 17: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/17.jpg)
Tuesday, August 30, 2011
![Page 18: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/18.jpg)
Tuesday, August 30, 2011
![Page 19: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/19.jpg)
Tuesday, August 30, 2011
![Page 20: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/20.jpg)
Tuesday, August 30, 2011
![Page 21: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/21.jpg)
Tuesday, August 30, 2011
![Page 22: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/22.jpg)
Follow[ing|ers]
Tuesday, August 30, 2011
![Page 23: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/23.jpg)
Tuesday, August 30, 2011
![Page 24: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/24.jpg)
Tuesday, August 30, 2011
![Page 25: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/25.jpg)
Tuesday, August 30, 2011
![Page 26: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/26.jpg)
Tuesday, August 30, 2011
![Page 27: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/27.jpg)
Our situation before Cassandra
Tuesday, August 30, 2011
![Page 28: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/28.jpg)
MySQL + Memcache
Tuesday, August 30, 2011
![Page 29: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/29.jpg)
MySQL + Memcache
• Flat MySQL table
Tuesday, August 30, 2011
![Page 30: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/30.jpg)
MySQL + Memcache
• Flat MySQL table
• Indexes (Followers vs Following)
Tuesday, August 30, 2011
![Page 31: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/31.jpg)
MySQL + Memcache
• Flat MySQL table
• Indexes (Followers vs Following)
• Memcache
Tuesday, August 30, 2011
![Page 32: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/32.jpg)
Memcache
• Stored as JSON list
Tuesday, August 30, 2011
![Page 33: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/33.jpg)
Memcache
• Stored as JSON list
• Can get out of sync
Tuesday, August 30, 2011
![Page 34: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/34.jpg)
MySQL
• Hundreds of millions of rows
Tuesday, August 30, 2011
![Page 35: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/35.jpg)
MySQL
• Hundreds of millions of rows
• m2.4xl EC2 instance
Tuesday, August 30, 2011
![Page 36: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/36.jpg)
It all started with a Feature.
Tuesday, August 30, 2011
![Page 37: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/37.jpg)
QOTD
Tuesday, August 30, 2011
![Page 38: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/38.jpg)
QOTD
Tuesday, August 30, 2011
![Page 39: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/39.jpg)
QOTD
• Feb: 35K Followers
Tuesday, August 30, 2011
![Page 40: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/40.jpg)
QOTD
• Feb: 35K Followers
• Mid-Feb: 100K
Tuesday, August 30, 2011
![Page 41: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/41.jpg)
QOTD
• Feb: 35K Followers
• Mid-Feb: 100K
• Growing about 30K a week.
Tuesday, August 30, 2011
![Page 42: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/42.jpg)
QOTD
• Feb: 35K Followers
• Mid-Feb: 100K
• Growing about 30K a week.
• 200K: closer to Memcache 1MB limit.
Tuesday, August 30, 2011
![Page 43: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/43.jpg)
QOTD
• Feb: 35K Followers
• Mid-Feb: 100K
• Growing about 30K a week.
• 200K: closer to Memcache 1MB limit.
• (with gz compaction . . .)
Tuesday, August 30, 2011
![Page 44: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/44.jpg)
Then we added more features . . .
Tuesday, August 30, 2011
![Page 45: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/45.jpg)
Tuesday, August 30, 2011
![Page 46: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/46.jpg)
Tuesday, August 30, 2011
![Page 47: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/47.jpg)
Temporary solution
Tuesday, August 30, 2011
![Page 48: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/48.jpg)
Redis
• Key : JSON blob of IDs
Tuesday, August 30, 2011
![Page 49: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/49.jpg)
Redis
• Key : JSON blob of IDs
• ~2-3s / insert
Tuesday, August 30, 2011
![Page 50: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/50.jpg)
Frontend Ask
MySQLMemCache
Tuesday, August 30, 2011
![Page 51: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/51.jpg)
OutboxQW
Frontend Ask
MySQLMemCache
Tuesday, August 30, 2011
![Page 52: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/52.jpg)
OutboxQW
Frontend Ask
SELECT account_idFROM follow WHERE following = 1234 MySQL
MemCache
Tuesday, August 30, 2011
![Page 53: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/53.jpg)
OutboxQW
Frontend Ask
MySQLMemCache
OH SHIT
Tuesday, August 30, 2011
![Page 54: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/54.jpg)
OutboxQW
Frontend Ask
OH SHIT
MySQLMemCache
Tuesday, August 30, 2011
![Page 55: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/55.jpg)
OutboxQW
Frontend Ask
Gazillion messages
OH SHIT
MySQLMemCache
Tuesday, August 30, 2011
![Page 56: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/56.jpg)
OutboxQW
Frontend Ask
Gazillion messagesInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWs
OH SHIT
MySQLMemCache
Tuesday, August 30, 2011
![Page 57: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/57.jpg)
OutboxQW
Frontend Ask
Gazillion messagesInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQW
Inbox Object
InboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWs
OH SHIT
MySQLMemCache
Tuesday, August 30, 2011
![Page 58: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/58.jpg)
OutboxQW
Frontend Ask
Gazillion messagesInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQW
Inbox Object
InboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWs
*HANGING*OH SHIT
MySQLMemCache
Tuesday, August 30, 2011
![Page 59: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/59.jpg)
OutboxQW
Frontend Ask
Gazillion messagesInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQW
Inbox Object
InboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWs
*OOM*OH SHIT
MySQLMemCache
Tuesday, August 30, 2011
![Page 60: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/60.jpg)
OutboxQW
Frontend Ask
Gazillion messagesInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQW
Inbox Object
InboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWs
OutboxQW
OutboxQW
OutboxQW
OutboxQW
OutboxQW
OutboxQW
OH SHIT
MySQLMemCache
Tuesday, August 30, 2011
![Page 61: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/61.jpg)
OutboxQW
Frontend Ask
Gazillion messagesInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQW
Inbox Object
InboxQWInboxQWInboxQWInboxQWInboxQWInboxQWInboxQWs
OutboxQW
OutboxQW
OutboxQW
OutboxQW
OutboxQW
OH SHIT
MySQLMemCacheOOM
Tuesday, August 30, 2011
![Page 62: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/62.jpg)
We need a solution
Tuesday, August 30, 2011
![Page 63: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/63.jpg)
Fast
Tuesday, August 30, 2011
![Page 64: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/64.jpg)
Tuesday, August 30, 2011
![Page 65: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/65.jpg)
Step 1: Research
Tuesday, August 30, 2011
![Page 66: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/66.jpg)
Tuesday, August 30, 2011
![Page 67: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/67.jpg)
Tuesday, August 30, 2011
![Page 68: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/68.jpg)
Tuesday, August 30, 2011
![Page 69: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/69.jpg)
Tuesday, August 30, 2011
![Page 70: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/70.jpg)
Tuesday, August 30, 2011
![Page 71: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/71.jpg)
Our first shot with Cassandra
Tuesday, August 30, 2011
![Page 72: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/72.jpg)
All Responses
Tuesday, August 30, 2011
![Page 73: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/73.jpg)
Tuesday, August 30, 2011
![Page 74: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/74.jpg)
Explain our stack here
Tuesday, August 30, 2011
![Page 75: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/75.jpg)
Index of responses to questions on the site.
Tuesday, August 30, 2011
![Page 76: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/76.jpg)
Index of responses to questions on the site.
Tuesday, August 30, 2011
![Page 77: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/77.jpg)
Explain our stack here
Tuesday, August 30, 2011
![Page 78: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/78.jpg)
Tuesday, August 30, 2011
![Page 79: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/79.jpg)
Ask Followers
Tuesday, August 30, 2011
![Page 80: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/80.jpg)
How to scale ?
Tuesday, August 30, 2011
![Page 81: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/81.jpg)
Build a Social Graph
Tuesday, August 30, 2011
![Page 82: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/82.jpg)
Social Graph
Tuesday, August 30, 2011
![Page 83: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/83.jpg)
Build a Social Graph
• Python
Tuesday, August 30, 2011
![Page 84: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/84.jpg)
Build a Social Graph
• Python
• Cassandra (Obviously..)
Tuesday, August 30, 2011
![Page 85: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/85.jpg)
Build a Social Graph
• Python
• Cassandra (Obviously..)
• PyCassa
Tuesday, August 30, 2011
![Page 86: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/86.jpg)
Build a Social Graph
• Python
• Cassandra (Obviously..)
• PyCassa
• Thrift
Tuesday, August 30, 2011
![Page 87: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/87.jpg)
4 types of actions
• Following
Tuesday, August 30, 2011
![Page 88: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/88.jpg)
4 types of actions
• Following
• Followers
Tuesday, August 30, 2011
![Page 89: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/89.jpg)
4 types of actions
• Following
• Followers
• Blocks
Tuesday, August 30, 2011
![Page 90: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/90.jpg)
4 types of actions
• Following
• Followers
• Blocks
• Blockers
Tuesday, August 30, 2011
![Page 91: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/91.jpg)
Thrift
Tuesday, August 30, 2011
![Page 92: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/92.jpg)
Tuesday, August 30, 2011
![Page 93: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/93.jpg)
Tuesday, August 30, 2011
![Page 94: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/94.jpg)
Cassandra
Tuesday, August 30, 2011
![Page 95: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/95.jpg)
Cassandra
Tuesday, August 30, 2011
![Page 96: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/96.jpg)
Cassandra
Tuesday, August 30, 2011
![Page 97: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/97.jpg)
Cassandra
Tuesday, August 30, 2011
![Page 98: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/98.jpg)
Cassandra
Tuesday, August 30, 2011
![Page 99: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/99.jpg)
Cassandra
Tuesday, August 30, 2011
![Page 100: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/100.jpg)
Python + Pycassa
Tuesday, August 30, 2011
![Page 101: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/101.jpg)
Python + Pycassa
Column Families
Tuesday, August 30, 2011
![Page 102: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/102.jpg)
Python + Pycassa
Get list of Follow[ers/ing]
Tuesday, August 30, 2011
![Page 103: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/103.jpg)
Python + Pycassa
More Queries
Tuesday, August 30, 2011
![Page 104: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/104.jpg)
Python + Pycassa
Follow user
Tuesday, August 30, 2011
![Page 105: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/105.jpg)
Switch to Production
Tuesday, August 30, 2011
![Page 106: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/106.jpg)
Simultaneous writes to MySQL and Cassandra
Tuesday, August 30, 2011
![Page 107: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/107.jpg)
Cassandra MySQL
Read x
Write x x
Switch to Production
Tuesday, August 30, 2011
![Page 108: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/108.jpg)
Idempotence is your friend.
Tuesday, August 30, 2011
![Page 109: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/109.jpg)
Migration from MySQL to Cassandra
Tuesday, August 30, 2011
![Page 110: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/110.jpg)
Load batches of user info into Cassandra.
Tuesday, August 30, 2011
![Page 111: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/111.jpg)
It’s OK if it fails.
Tuesday, August 30, 2011
![Page 112: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/112.jpg)
Remember idempotence is your
friend.
Tuesday, August 30, 2011
![Page 113: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/113.jpg)
Adjust your capacity
Tuesday, August 30, 2011
![Page 114: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/114.jpg)
From 4 instancesto
a 12 instances
Tuesday, August 30, 2011
![Page 115: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/115.jpg)
New Ask Followers Infrastructure
Tuesday, August 30, 2011
![Page 116: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/116.jpg)
Frontend Ask
CassandraCluster
Tuesday, August 30, 2011
![Page 117: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/117.jpg)
OutboxQW
Frontend Ask
CassandraCluster
Tuesday, August 30, 2011
![Page 118: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/118.jpg)
OutboxQW
Frontend Ask
CassandraCluster
IterativeKW
Tuesday, August 30, 2011
![Page 119: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/119.jpg)
OutboxQW
Frontend Ask
CassandraCluster
IterativeKW
get_followers(user_id, 0, N)
Tuesday, August 30, 2011
![Page 120: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/120.jpg)
OutboxQW
Frontend Ask
CassandraCluster
IterativeKW
[‘1234, ‘1238’, ..., ‘1999’]
Tuesday, August 30, 2011
![Page 121: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/121.jpg)
OutboxQW
Frontend Ask
CassandraCluster
IterativeKW
[‘1234, ‘1238’, ..., ‘1999’]
N Messages
Tuesday, August 30, 2011
![Page 122: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/122.jpg)
OutboxQW
Frontend Ask
CassandraCluster
IterativeKW
[‘1234, ‘1238’, ..., ‘1999’]
InboxQW
N Messages
Tuesday, August 30, 2011
![Page 123: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/123.jpg)
OutboxQW
Frontend Ask
CassandraCluster
IterativeKW
[‘1234, ‘1238’, ..., ‘1999’]
InboxQW
N Inbox Object
Tuesday, August 30, 2011
![Page 124: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/124.jpg)
OutboxQW
Frontend Ask
CassandraCluster
IterativeKW
InboxQW
N Inbox Object
get_followers(user_id, N, N+N)
Tuesday, August 30, 2011
![Page 125: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/125.jpg)
Count Following
14K/min hits at peak time
Tuesday, August 30, 2011
![Page 126: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/126.jpg)
Count Followers
40K/min hits at peak time
Tuesday, August 30, 2011
![Page 127: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/127.jpg)
Reduce our #instances
InboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWs
InboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsOutboxQWs
12x 12x
Tuesday, August 30, 2011
![Page 128: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/128.jpg)
InboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWs
12x
InboxQWsInboxQWsInboxQWsInboxQWsOutboxQWs
4x
Reduce our #instances
Tuesday, August 30, 2011
![Page 129: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/129.jpg)
InboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWsInboxQWs
Inbox+
Iterator QW
InboxQWsInboxQWsInboxQWsInboxQWsOutboxQWs
12x 4x
Reduce our #instances
Tuesday, August 30, 2011
![Page 130: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/130.jpg)
Counts for free with 0.8.1
Tuesday, August 30, 2011
![Page 131: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/131.jpg)
Counting . . .
Tuesday, August 30, 2011
![Page 132: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/132.jpg)
Counting is hard
Let’s go Shopping.
Tuesday, August 30, 2011
![Page 133: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/133.jpg)
Counting
• Fast
Tuesday, August 30, 2011
![Page 134: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/134.jpg)
Counting
• Fast
• Not-accurate
Tuesday, August 30, 2011
![Page 135: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/135.jpg)
Counting
• Fast
• Not-accurate
• Faking “Read-repair”
Tuesday, August 30, 2011
![Page 136: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/136.jpg)
Counting
Tuesday, August 30, 2011
![Page 137: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/137.jpg)
Ghetto Count
Tuesday, August 30, 2011
![Page 138: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/138.jpg)
Cassandra allowed us:
• To create new features.
Tuesday, August 30, 2011
![Page 139: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/139.jpg)
Cassandra allowed us:
• To create new features.
• To scale.
Tuesday, August 30, 2011
![Page 140: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/140.jpg)
Cassandra allowed us:
• To create new features.
• To scale.
• To save money.
Tuesday, August 30, 2011
![Page 141: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/141.jpg)
Cassandra allowed us:
• To create new features.
• To scale.
• To save money.
• To sleep. (Almost there)
Tuesday, August 30, 2011
![Page 142: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/142.jpg)
Cassandra allowed us:
• To create new features.
• To scale.
• To save money.
• To sleep. (Almost there)
• Lots of fun :)
Tuesday, August 30, 2011
![Page 143: Cassandra @Formspring](https://reader034.vdocuments.site/reader034/viewer/2022042606/54b799834a79591d4a8b47c0/html5/thumbnails/143.jpg)
Come work with us !
Tuesday, August 30, 2011