cassandra summit keynote 2014
DESCRIPTION
Cassandra Summit SF 2014 keynote, covering Cassandra 2.1.TRANSCRIPT
©2014 DataStax Confidential. Do not distribute without consent.
CTO, DataStax
Jonathan EllisProject Chair, Apache Cassandra
State of Cassandra 2014
2.1http://cassandra.apache.org/download/
Cluster cluster = Cluster.builder() .addContactPoint("127.0.0.1") .withRetryPolicy( DefaultRetryPolicy.INSTANCE) .withLoadBalancingPolicy( new TokenAwarePolicy( new DCAwareRoundRobinPolicy())) .build();Session session = cluster.connect("demo");
cluster = Cluster( contact_points=['127.0.0.1'], load_balancing_policy=TokenAwarePolicy( DCAwareRoundRobinPolicy( local_dc='datacenter1')), default_retry_policy = RetryPolicy())session = cluster.connect('demo’)
Cluster cluster = Cluster.Builder() .AddContactPoints("127.0.0.1") .WithRetryPolicy(DowngradingConsistencyRetryPolicy.INSTANCE) .WithLoadBalancingPolicy( new TokenAwarePolicy( new RoundRobinPolicy()) .Build();ISession session = cluster.Connect("demo");
Cassandra’s peer to peer architecture, linear scalability and multi data center active-active deployment enable mission critical eBay features for hundreds of millions users every day.
Feng Qu, principal DBA at eBay Inc
Client Coordinator
40%busy
90%busy
30%busy
Client Coordinator
40%busy
90%busy
30%busy
Client Coordinator
40%busy
90%busy
30%busy
Client Coordinator
40%busy
90%busy
30%busyX
Client Coordinator
40%busy
90%busy
30%busyX
Client Coordinator
40%busy
90%busy
30%busyX
Client Coordinator
40%busy
90%busy
30%busyXsuccess
“One of the great things about NoSQL is the ability to iterate on one's data model as one's business requires it.”
“Everything in one place is convenient until it fails. If you want to scale and be highly available, use Cassandra.”
Matt AsayMongoDB
Adrian CockcroftBattery Ventures
2.1: Performance
Read performance, 2.0 vs 2.1
Reads, post-compaction (2.0)
Reads, post-compaction (2.1)
Write performance, 2.0 vs 2.1
Competitive performance
Competitive performance
https://github.com/jbellis/YCSB
2.1: Data modeling
Lightweight transactions
Lightweight transactionsINSERT INTO users (username, name, email, password, created_date)VALUES ('pmcfadin', 'Patrick McFadin', ['[email protected]'], 'ba27e03fd9...', '2011-06-20 13:50:00')IF NOT EXISTS;
Lightweight transactionsINSERT INTO users (username, name, email, password, created_date)VALUES ('pmcfadin', 'Patrick McFadin', ['[email protected]'], 'ba27e03fd9...', '2011-06-20 13:50:00')IF NOT EXISTS;
INSERT INTO users (username, name, email, password, created_date)VALUES ('pmcfadin', 'Patrick McFadin', ['[email protected]'], 'ea24e13ad9...', '2011-06-20 13:50:01')IF NOT EXISTS;
Lightweight transactionsINSERT INTO users (username, name, email, password, created_date)VALUES ('pmcfadin', 'Patrick McFadin', ['[email protected]'], 'ba27e03fd9...', '2011-06-20 13:50:00')IF NOT EXISTS;
[applied]----------- True
INSERT INTO users (username, name, email, password, created_date)VALUES ('pmcfadin', 'Patrick McFadin', ['[email protected]'], 'ea24e13ad9...', '2011-06-20 13:50:01')IF NOT EXISTS;
Lightweight transactions
[applied] | username | created_date | name -----------+----------+----------------+---------------- False | pmcfadin | 2011-06-20 ... | Patrick McFadin
INSERT INTO users (username, name, email, password, created_date)VALUES ('pmcfadin', 'Patrick McFadin', ['[email protected]'], 'ba27e03fd9...', '2011-06-20 13:50:00')IF NOT EXISTS;
[applied]----------- True
INSERT INTO users (username, name, email, password, created_date)VALUES ('pmcfadin', 'Patrick McFadin', ['[email protected]'], 'ea24e13ad9...', '2011-06-20 13:50:01')IF NOT EXISTS;
Static columns (2.0.6)CREATE TABLE bills ( user text, balance int static, expense_id int, amount int, description text, paid boolean, PRIMARY KEY (user, expense_id));
Static columns + LWTCREATE TABLE bills ( user text, balance int static, expense_id int, amount int, description text, paid boolean, PRIMARY KEY (user, expense_id));
BEGIN BATCH UPDATE bills SET balance = -116 WHERE user='user1' IF balance = 84; INSERT INTO bills (user, expense_id, amount, description, paid) VALUES ('user1', 2, 200, 'hotel room', false);APPLY BATCH;
Atomic log appends with LWTCREATE TABLE log ( log_name text, seq int static, logged_at timeuuid, entry text, primary key (log_name, logged_at));
INSERT INTO log (log_name, seq) VALUES ('foo', 0);
Atomic log appends with LWTBEGIN BATCH
UPDATE log SET seq = 1WHERE log_name = 'foo'IF seq = 0;
INSERT INTO log (log_name, logged_at, entry)VALUES ('foo', now(), 'test');
APPLY BATCH;
User defined typesCREATE TYPE address ( street text, city text, zip_code int, phones set<text>)
CREATE TABLE users ( id uuid PRIMARY KEY, name text, addresses map<text, frozen<address>>)
SELECT id, name, addresses.city, addresses.phones FROM users;
id | name | addresses.city | addresses.phones--------------------+----------------+-------------------------- 63bf691f | jbellis | Austin | {'512-4567', '512-9999'}
Collection indexingCREATE TABLE songs ( id uuid PRIMARY KEY, artist text, album text, title text, data blob, tags set<text>);
CREATE INDEX song_tags_idx ON songs(tags);
SELECT * FROM songs WHERE tags CONTAINS 'blues';
id | album | artist | tags | title----------+---------------+-------------------+-----------------------+------------------ 5027b27e | Country Blues | Lightnin' Hopkins | {'acoustic', 'blues'} | Worrying My Mind
2.1: Operations
Counters, 2.0
Counters, 2.0
MemtableMemtableMemtable
A0 1 1
CommitlogCommitlogCommitlog
A0 1 1
1. Node A, increment by 1
Value: 1
Counters, 2.0
CommitlogCommitlogCommitlog
A0 1 1
A0 1 1
MemtableMemtableMemtable
A0 2 2
2. Node A, increment by 1
Value: 2
MemtableMemtableMemtable
A0 1 1
CommitlogCommitlogCommitlog
A0 1 1
1. Node A, increment by 1
Value: 1
Counters, 2.0
CommitlogCommitlogCommitlog
A0 1 1
A0 1 1
MemtableMemtableMemtable
A0 2 2
2. Node A, increment by 1
Value: 2
MemtableMemtableMemtable
A0 1 1
CommitlogCommitlogCommitlog
A0 1 1
1. Node A, increment by 1
Value: 1
MemtableMemtableMemtable
A0 3 0
3. Node A, decrement by 2
Value: 0
Commit LogCommit LogCommit Log
A0 1 1
A0 1 1
A0 1 -2
Counters, 2.1
Counters, 2.1
MemtableMemtableMemtable
A0 1 1
CommitlogCommitlogCommitlog
A0 1 1
1. Node A, increment by 1
Value: 1
Counters, 2.1
CommitlogCommitlogCommitlog
A0 1 1
A0 2 2
MemtableMemtableMemtable
A0 2 2
2. Node A, increment by 1
Value: 2
MemtableMemtableMemtable
A0 1 1
CommitlogCommitlogCommitlog
A0 1 1
1. Node A, increment by 1
Value: 1
Counters, 2.1
CommitlogCommitlogCommitlog
A0 1 1
A0 2 2
MemtableMemtableMemtable
A0 2 2
2. Node A, increment by 1
Value: 2
MemtableMemtableMemtable
A0 1 1
CommitlogCommitlogCommitlog
A0 1 1
1. Node A, increment by 1
Value: 1
MemtableMemtableMemtable
A0 3 0
3. Node A, decrement by 2
Value: 0
Commit LogCommit LogCommit Log
A0 1 1
A0 2 2
A0 3 0
(low contention)
(high contention)
Regular repair
Regular repair
Regular repair
Incremental repair
Implications for LCS (and STCS)bin/nodetool upgradesstables
tools/bin/sstablerepairedset --is-repaired
bin/nodetool repair
bin/nodetool repair -inc
The new row cacheCREATE TABLE notifications ( target_user text, notification_id timeuuid, source_id uuid, source_type text, activity text, PRIMARY KEY (target_user, notification_id))WITH CLUSTERING ORDER BY (notification_id DESC) AND caching = 'rows_only' AND rows_per_partition_to_cache = '3';
The new row cachetarget_user notification_id source_id source_type activity
nick e1bd2bcb- d972b679- photo tom liked
nick 321998c- d972b679- photo jake commented
nick ea1c5d35- 88a049d5- user mike created account
nick 5321998c- 64613f27- photo tom commented
nick 07581439- 076eab7e- user tyler created account
mike 1c34467a- f04e309f- user tom created account
target_user notification_id source_id source_type activity
nick e1bd2bcb- d972b679- photo tom liked
nick 321998c- d972b679- photo jake commented
nick ea1c5d35- 88a049d5- user mike created account
nick 5321998c- 64613f27- photo tom commented
nick 07581439- 076eab7e- user tyler created account
mike 1c34467a- f04e309f- user tom created account
The new row cache
The new row cachetarget_user notification_id source_id source_type activity
nick e1bd2bcb- d972b679- photo tom liked
nick 321998c- d972b679- photo jake commented
nick ea1c5d35- 88a049d5- user mike created account
nick 5321998c- 64613f27- photo tom commented
nick 07581439- 076eab7e- user tyler created account
mike 1c34467a- f04e309f- user tom created account
2.1: Testing & QA
