cassandra summit 2014: lesser known features of cassandra 2.1

83
Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License #CASSANDRASUMMIT LESSER KNOWN FEATURES OF CASSANDRA 2.1 Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License Aaron Morton @aaronmorton Co-Founder & Principal Consultant

Upload: planet-cassandra

Post on 13-Dec-2014

1.954 views

Category:

Technology


4 download

DESCRIPTION

Presenter: Aaron Morton, Apache Cassandra Committer & Co-Founder of The Last Pickle Apache Cassandra 2.0 and 2.1 include a wealth of new and updated features. Some are well known, others are known to only a few. But any of them could help you reduce latency, improve throughput, or make operations easier. This talk will take a deep dive into features that improve: Compaction, Write Performance, Memory Management, CQL 3, TTL and Tombstones, & Repair. Existing and new users will benefit from this wide ranging view of the features Apache Cassandra offers.

TRANSCRIPT

Page 1: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License

#CASSANDRASUMMIT

LESSER KNOWN FEATURES OF CASSANDRA 2.1

Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License

Aaron Morton @aaronmorton

Co-Founder & Principal Consultant

Page 2: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

About The Last Pickle. !

Work with clients to deliver and improve Apache Cassandra based solutions.

Apache Cassandra Committer, DataStax MVP, Hector

Maintainer, Apache Usergrid Committer. Based in New Zealand & USA.

Page 3: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Configuration CQL3 Tools

Internals

Page 4: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Change Logging to Logback (CASSANDRA-5883)

Faster Auto Reloading Auto Archiving logback.xml

Page 5: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Change Logging to Logback (CASSANDRA-5883) <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${cassandra.logdir}/system.log</file> <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy"> <fileNamePattern>${cassandra.logdir}/system.log.%i.zip</fileNamePattern> <minIndex>1</minIndex> <maxIndex>20</maxIndex> </rollingPolicy> </appender> !

<logger name=“org.apache.cassandra.Foo” level="ERROR"/>

Page 6: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Change Logging to Logback (CASSANDRA-5883)

TIP: Change compression so zless and zgrep work.

!

${cassandra.logdir}/system.log.%i.gz

Page 7: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Change Logging to Logback (CASSANDRA-5883) $ ls -lah logs/ 963K system.log 129K system.log.1.gz 158K system.log.2.gz 158K system.log.3.gz 158K system.log.4.gz 158K system.log.5.gz 158K system.log.6.gz 158K system.log.7.gz 158K system.log.8.gz 157K system.log.9.gz

Page 8: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Command Line Parameters

Page 9: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

New Default Paths (CASSANDRA-7136) // bin/cassandra.in.sh CASSANDRA_HOME="`dirname "$0"`/.." cassandra_storagedir=“$CASSANDRA_HOME/data" !

// bin/cassandra -Dcassandra.logdir=$CASSANDRA_HOME/logs" -Dcassandra.storagedir=$cassandra_storagedir"

Page 10: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

New Default Paths (CASSANDRA-7136)

No change for package installs. !

Leave YAML settings blank for new defaults.

Page 11: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

New Default Paths (CASSANDRA-7136) $ ls -lh cassandra/v2.1/ -rw-r--r--@ CHANGES.txt -rw-r--r--@ LICENSE.txt -rw-r--r--@ NEWS.txt -rw-r--r--@ NOTICE.txt drwxr-xr-x@ bin drwxr-xr-x@ conf drwxr-xr-x data drwxr-xr-x@ lib drwxr-xr-x logs drwxr-xr-x@ pylib drwxr-xr-x@ tools

Page 12: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

New Default Paths (CASSANDRA-7136) $ ls -lah cassandra/ drwxr-xr-x@ apache-cassandra-1.0.12 drwxr-xr-x@ apache-cassandra-1.1.12 drwxr-xr-x@ apache-cassandra-1.2.18 drwxr-xr-x@ apache-cassandra-2.0.10 drwxr-xr-x@ apache-cassandra-2.1.0-rc7 lrwxr-xr-x current -> v2.0/ lrwxr-xr-x v1.0 -> apache-cassandra-1.0.12 lrwxr-xr-x v1.1 -> apache-cassandra-1.1.12 lrwxr-xr-x v1.2 -> apache-cassandra-1.2.18/ lrwxr-xr-x v2.0 -> apache-cassandra-2.0.10 lrwxr-xr-x v2.1 -> apache-cassandra-2.1.0-rc7

Page 13: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Custom Configuration Loader (CASSANDRA-5045) -Dcassandra.config.loader

!

Provide a custom class that implements

o.a.c.config.ConfigurationLoader.

Page 14: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Custom Configuration Loader (CASSANDRA-5045)

Useful when embedding or storing configuration elsewhere.

Page 15: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Add QueryHandler Interface (CASSANDRA-6659)-Dcassandra.custom_query_handler_class

!

Provide a custom class that implements

o.a.c.cql3.QueryHandler.

Page 16: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Add QueryHandler Interface (CASSANDRA-6659)

Custom implementation to handle all CQL3 queries.

!

For example logging, slow query log, stubbing Cassandra.

Page 17: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Add QueryHandler Interface (CASSANDRA-6659)public interface QueryHandler { ResultMessage process(String query, QueryState state,…); !

ResultMessage.Prepared prepare(String query, QueryState state); !

ParsedStatement.Prepared getPrepared(MD5Digest id); !

CQLStatement getPreparedForThrift(Integer id); !

ResultMessage processPrepared(CQLStatement statement, …); !

ResultMessage processBatch(BatchStatement statement, …);

Page 18: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Pluggable o.a.c.metrics Reporters (CASSANDRA-4430)!

-Dcassandra.metricsReporterConfigFile !

https://github.com/addthis/metrics-reporter-config

Page 19: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Pluggable o.a.c.metrics Reporters (CASSANDRA-4430)

Reporters push metrics to Graphite, Ganglia, Riemann

etc.

Page 20: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Pluggable o.a.c.metrics Reporters (CASSANDRA-4430)riemann: - period: 1 timeunit: 'SECONDS' hosts: - host: 'localhost' port: 5555 predicate: color: "white" useQualifiedName: true patterns: - "^org.apache.cassandra.metrics.ClientRequest.+" - "^org.apache.cassandra.metrics.DroppedMessage.+"

Page 21: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Selective Commit Log Replay (CASSANDRA-4809)

-Dcassandra.replayList !

-Dcassandra.replayList=Foo.Bar,…

Page 22: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Selective Commit Log Replay (CASSANDRA-4809)

Use for selective Point in Time recovery using Commit Log

Archiving.

Page 23: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Add Option To Disable STCS In L0 (CASSANDRA-6621)!

-Dcassandra.disable_stcs_in_l0

!

May be useful when Bootstrapping.

Page 24: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Use JMX_PORT For The RMI Port (CASSANDRA-7087) !

// conf/cassandra-env.sh -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT

Page 25: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

YAML Configuration

Page 26: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Enabled Hints Per Data Center (CASSANDRA-6157) !

hinted_handoff_enabled: DC1,DC2 !

Disable Hints for a troubled DC.

!

!

Page 27: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Add Paranoid Disk Failure Option (CASSANDRA-6646) !

disk_failure_policy: stop_paranoid !

Implement STOP behaviour for CorruptSSTableException.

Page 28: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Commit Log Failure Handling (CASSANDRA-6364) !

commit_failure_policy: stop !

Page 29: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Commit Log Failure Handling (CASSANDRA-6364) !

ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2014-… CommitLog.java:350 - Failed to persist commits to disk. Commit disk failure policy is stop; terminating thread !

java.io.IOException: Something broke. at org.apache.cassandra.db.commitlog.AbstractCommitLogService $1.run(AbstractCommitLogService.java:123) at java.lang.Thread.run(Thread.java:744)

Page 30: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Add file_cache_size_in_mb setting (CASSANDRA-5661) !

# Total memory to use for sstable-reading buffers. # Defaults to # the smaller of 1/4 of heap or 512MB # file_cache_size_in_mb: 512

Page 31: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Add file_cache_size_in_mb setting (CASSANDRA-5661)

File Readers have a 64 KB buffer.

Compressed File Readers have two 64 KB buffers.

Page 32: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Add file_cache_size_in_mb setting (CASSANDRA-5661)

Discard cached file readers 512ms after last access for a

file.

Page 33: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Add file_cache_size_in_mb setting (CASSANDRA-5661)

Maintain a maximum combined buffer size for all file

readers.

Page 34: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Improve 2.1 flush defaults (CASSANDRA-7551) !

# memtable_flush_writers defaults to the smaller of (number of # disks, number of cores), with a minimum of 2 and a maximum of 8. # # If your data directories are backed by SSD, you should increase # this to the number of cores. # memtable_flush_writers: 8

Page 35: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

New concurrent_compactors Defaults (CASSANDRA-7139)!

# concurrent_compactors defaults to the smaller of (number # of disks, number of cores), with a minimum of 2 and a # maximum of 8. # # If your data directories are backed by SSD, you should # increase this to the number of cores. # concurrent_compactors: 1

Page 36: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Throw Error When Frame Too Large (CASSANDRA-5981)!

# native_transport_max_frame_size_in_mb: 256 !

Improves Existing Behaviour, InvalidRequestException now

thrown.

Page 37: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Add broadcast_rpc_address Option (CASSANDRA-5899) !

broadcast_rpc_address: 1.2.3.4 !

Broadcast to clients via node discovery.

Page 38: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Add listen_interface and rpc_interface (CASSANDRA-7417)!

rpc_interface: eth1 !

listen_interface: eth0

Page 39: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Add listen_interface and rpc_interface (CASSANDRA-7417)

Must have a only one IP Address.

Page 40: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Drop queries reading many Tombstones (CASSANDRA-6117)

Replaces tombstone_debug_threshold

Page 41: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Drop queries reading many Tombstones (CASSANDRA-6117)!

tombstone_warn_threshold: 1000

Page 42: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Drop queries reading many Tombstones (CASSANDRA-6117)WARN [main] … !

SliceQueryFilter.java:236 - Read 2 live and 4 tombstoned cells in system.schema_columnfamilies (see tombstone_warn_threshold). 2147483631 columns was requested, slices=[-], delInfo={deletedAt=-9223372036854775808, localDeletion=2147483647}

Page 43: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Drop query that using Tombstones (CASSANDRA-6117)!

tombstone_failure_threshold: 100000

Page 44: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Drop query that using Tombstones (CASSANDRA-6117)!

ERROR [SharedPool-Worker-7] … !

SliceQueryFilter.java:212 - Scanned over 100000 tombstones in system.schema_columns; query aborted (see tombstone_fail_threshold)

Page 45: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Log a Warning for Large Batches (CASSANDRA-6487)!

batch_size_warn_threshold_in_kb: 5

Page 46: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Log a Warning for Large Batches (CASSANDRA-6487)!

WARN [Native-Transport-Requests:553]… !

BatchStatement.java (line 228) Batch of prepared statements for [Keyspace1.Standard1] is of size 6165, exceeding specified threshold of 5120 by 1045.

Page 47: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Preemptive opening compaction result (CASSANDRA-6916)

Replaces compaction_preheat_key_cache: true, preheat_kernel_page_cache: false

and populate_io_cache_on_flush

table property.

Page 48: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Preemptive opening compaction result (CASSANDRA-6916)

With sstable_preemptive_open_interval_in_mb: 50

Page 49: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Preemptive opening compaction result (CASSANDRA-6916)

Pre heats Key Cache and IO Cache, and reads from partial

SSTables.

Page 50: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Throttle Inter-DC Streaming Traffic (CASSANDRA-6596) !

inter_dc_stream_throughput_outbound_megabits_per_sec: 0

Applied after stream_throughput_outbound_megabits_per_sec: 200

Page 51: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Configuration CQL3 Tools

Internals

Page 52: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Alias Support For SELECT Statement (CASSANDRA-5075) !

cqlsh:dev> select foo as total_foo from foo; !

total_foo ----------- bar baz !

(2 rows)

Page 53: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

An Official Way To Disable Compaction (CASSANDRA-5074) !

create table no_compaction_for_you ( foo text, bar text, primary key (foo, bar) ) with compaction = { 'enabled' : false, 'class' : 'SizeTieredCompactionStrategy' };

Page 54: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Aggressive Tombstone Compactions (CASSANDRA-6563) !

CREATE TABLE foo (      foo text primary key,      bar text ) WITH compaction = { 'class' : 'SizeTieredCompactionStrategy', 'unchecked_tombstone_compaction' : 'true' };

Page 55: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Aggressive Tombstone Compactions (CASSANDRA-6563)

Disables check for other row fragments.

Page 56: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Configuration CQL3 Tools

Internals

Page 57: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

cqlsh

Page 58: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Pass cqlsh Statements via Command Line CASSANDRA-7172 !

bin/cqlsh --execute="select distinct foo from dev.foo;" !

foo ------- three one five two four !

(5 rows)

Page 59: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Include CQL in Error Messages (CASSANDRA-7111)!

cqlsh:dev> SELECT foo bar from foo; !

<ErrorMessage code=2000 [Syntax error in CQL query] message="line 1:11 no viable alternative at input 'bar'"> cqlsh:dev> !

Page 60: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

nodetool

Page 61: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Set & Get Logging Levels via nodetool (CASSANDRA-7090)!

nodetool getlogginglevels Logger Name Log Level ROOT INFO com.thinkaurelius.thrift ERROR !

nodetool setlogginglevel org.apache.cassandra.db.Row DEBUG !

nodetool getlogginglevels Logger Name Log Level ROOT INFO com.thinkaurelius.thrift ERROR org.apache.cassandra.db.Row DEBUG

Page 62: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Add listsnapshots To nodetool (CASSANDRA-5742)!

nodetool listsnapshots !

Snapshot Details: Snapshot name Keyspace name Column family True size Size on disk pre-migration Keyspace1 Standard1 508.03 MB 555.59 MB all-good Keyspace1 Standard1 549.01 MB 803.68 MB !

Total TrueDiskSpaceUsed: 1.03 GB

Page 63: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Add Snapshot Space Used To cfstats (CASSANDRA-6231) !

Table: Standard1 SSTable count: 4 Space used (live), bytes: 76875505 Space used (total), bytes: 76875505 Space used by snapshots (total), bytes: 1108391939

Page 64: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Save Compaction History (CASSANDRA-5078)!

nodetool compactionhistory !

id keyspace columnfamily compacted_at bytes_in bytes_out 31...9d Keyspace1 Standard1 1410287780455 488451903 174360500 51...9d Keyspace1 Standard1 1410287834272 527847260 256812363

Page 65: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Repairing Between Specific Replicas (CASSANDRA-6440)!

nodetool repair [(-dc <specific_dc> | --in-dc <specific_dc>)...] [(-hosts <specific_host> | --in-hosts <specific_host>)]

Page 66: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Reset SSTable LCS Compaction Level (CASSANDRA-5271)!

sstablelevelreset --really-reset Keyspace1 Standard1 !

!

Changing level from 1 to 0 on …/Keyspace1-Standard1-ka-35-Data.db !

Skipped .../Keyspace1-Standard1-ka-33-Data.db since it is already on level 0 !

Changing level from 1 to 0 on …/Keyspace1-Standard1-ka-32-Data.db !

Skipped .../Keyspace1-Standard1-ka-34-Data.db since it is already on level 0 !

Page 67: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Configuration Schema Tools

Internals

Page 68: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Promote Partition Index (CASSANDRA-2319)

Index of Cells created when partition is larger than

column_index_size_in_kb (64).

Page 69: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Promote Partition Index (CASSANDRA-2319)

Column Index promoted into the Key Cache.

Improves reading from wide Partitions.

Page 70: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Remove Keyspace.switchLock (CASSANDRA-5549)

Removed memtable_flush_queue_size

Added memtable_cleanup_threshold

Page 71: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Remove Keyspace.switchLock (CASSANDRA-5549)

When the Flush Queue was full Mutations would be

blocked. (Blocked FlushWriter tasks.)

Page 72: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Unique Column Family Directories (CASSANDRA-5202)

Prevents “re-using”data from dropped Column Families.

Page 73: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Unique Column Family Directories (CASSANDRA-5202) ls -lah data/data/Keyspace1/ !

drwxr-xr-x Counter1-89df6990386511e48bb41d4c6fbb569d drwxr-xr-x Counter3-89f20730386511e48bb41d4c6fbb569d drwxr-xr-x Standard1-89df4280386511e48bb41d4c6fbb569d

Page 74: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Specify Cells To Cache Per Row (CASSANDRA-5357)create table row_cache ( foo text, bar text, primary key (foo, bar) ) with rows_per_partition_to_cache = 10;

Page 75: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Use hibernate when join_ring is false (CASSANDRA-6961)

Use when returning a node that as been down for longer

than the Hint window.

Page 76: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Use hibernate when join_ring is false (CASSANDRA-6961)

nodetool join

Page 77: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Add Manifest Describing Snapshot Files (CASSANDRA-6326)!

{ "files": [ "Keyspace1-Standard1-ka-1-Data.db", "Keyspace1-Standard1-ka-3-Data.db", "Keyspace1-Standard1-ka-2-Data.db", "Keyspace1-Standard1-ka-5-Data.db", "Keyspace1-Standard1-ka-4-Data.db" ] }

Page 78: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Track min/max Clustered Values (CASSANDRA-5514)!

CREATE TABLE temperature ( weatherstation_id text, event_time timestamp, temperature text, PRIMARY KEY (weatherstation_id,event_time) );

Page 79: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Track min/max Clustered Values (CASSANDRA-5514)!

SELECT event_time, temperature FROM temperature WHERE weatherstation_id='1234ABCD' AND event_time >= '2013-04-03 07:01:00' AND event_time <= '2013-04-03 07:04:00';

Page 80: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Track min/max Clustered Values (CASSANDRA-5514)

Skips SSTables that will not contain relevant data.

Page 81: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

The Take Away

Use Cassandra v2.0.10 !

Get testing on v2.1.0

Page 82: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Thanks. #CASSANDRASUMMIT

!

Page 83: Cassandra Summit 2014: Lesser Known Features of Cassandra 2.1

Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License

Aaron Morton @aaronmorton

!

Co-Founder & Principal Consultant www.thelastpickle.com

!

Licensed under a Creative Commons Attribution-NonCommercial 3.0 New Zealand License