cassandra 2.1 boot camp, protocol, queries, cql
DESCRIPTION
Cassandra Summit Boot Camp, 2014 Protocol, Queries, and Cell Names Tyler Hobbs presenterTRANSCRIPT
©2014 DataStax
Tyler Hobbs
Protocol, Queries, and Cell Names
1
[email protected] @tylhobbs
Friday, September 12, 14
Native Protocol•Driver <=> Cassandra Communications•doc/native_protocol_v3.spec
Friday, September 12, 14
Native Protocol•Handshake•Set protocol version, compression, auth, etc
Friday, September 12, 14
Native Protocol•Multiplexed operations•Unique request ID per-operation per-connection•Responses can come out of order•Pushed notifications from the server
driver c*123
213
Friday, September 12, 14
Native Protocol•PREPARE•EXECUTE•QUERY•BATCH
Friday, September 12, 14
Statement Preparation•Driver sends PREPARE with query•C* replies with:• Statement ID• Query parameter metadata• Result set metadata
•Driver prepares against all nodes it will use
Friday, September 12, 14
Lexing and Parsing•ANTLR•Cql.g is the grammar•Minimal checking of statement validity•Creates “Raw” statements and terms• No type checking yet
Friday, September 12, 14
Raw -> Prepared•Term.Raw -> Term.(Non)Terminal• Terminal is fully processed, nothing to do after preparing
• String literals, int literals, etc• Pure function calls (e.g. intAsBlob(42))
• NonTerminal has bind markers or non-pure function calls• Example: now()
•Type checking performed where possible•Term.Raw.prepare()
Friday, September 12, 14
Raw -> Prepared•Statement.Raw -> ParsedStatement.Prepared• Is the query valid and supported?• Organize restrictions on partitioning, clustering and
normal columns• Check validity of restrictions, secondary indexes,
ordering, filtering, etc.• SelectStatement.Raw.prepare()
Friday, September 12, 14
Raw -> Prepared•ParsedStatement.Prepared is cached, ID returned
to driver•QueryProcessor
Friday, September 12, 14
Statement Execution•EXECUTE with ID and binary parameters
Friday, September 12, 14
Statement Execution•Check permissions• QueryProcessor.processStatement -> Statement.checkAccess()
•Check Statement Validity•Bind parameters• Term.NonTerminal -> Term.Terminal• Evaluate functions• Check parameter types
Friday, September 12, 14
Statement Execution•Build slice ends for reads• Pick start, end based on reversal• Are we getting multiple slices?• SelectStatement.makeFilter()
Friday, September 12, 14
Statement Execution•Set up paging• Use last seen primary key as starting point• Paging state held by driver
Friday, September 12, 14
Statement Execution•Pass command to StorageProxy• StorageProxy.read(), mutate(), cas()
• Dispatched to replicas• Low-level cells returned
Friday, September 12, 14
Statement Execution•Process Results• Order columns correctly• Filter empty rows• Order rows• Trim excess rows to match LIMIT• Aggregate (count(), for now)
Friday, September 12, 14
Statement Classes•SchemaAlteringStatement• CREATE/ALTER/DROP KEYSPACE, etc
•ModificationStatement• INSERT, UPDATE, DELETE
•SelectStatement
Friday, September 12, 14
SchemaAlteringStatement•Update local schema•Broadcast to all live nodes•Notify drivers with pushed notification
Friday, September 12, 14
ModificationStatement•When doing LWT/CAS• Prepare -> Read -> Propose -> Commit• Build slice ends for read, execute read• Check conditions against read row• If CAS fails, return existing row as result
Friday, September 12, 14
Cells•Subject to change for 3.0•Name/value pair
• Sorted by name
Friday, September 12, 14
CellsCREATE TABLE versioned_docs (id uuid,
version timeuuid,
fieldname text,
fieldvalue blob,
PRIMARY KEY (id, version, fieldname))
•id is the partitioning key•version and fieldname are clustering columns•Within a partition, rows are ordered by clustering
columns
Friday, September 12, 14
Sparse Format•CQL3 Format•Cell name format:
• (clustering_1, ..., clustering_n, column_name)
Friday, September 12, 14
Sparse FormatCREATE TABLE versioned_docs (id uuid,
version timeuuid,
fieldname text,
fieldvalue blob,
PRIMARY KEY (id, version, fieldname))
Friday, September 12, 14
Sparse FormatCREATE TABLE versioned_docs (id uuid,
version timeuuid,
fieldname text,
fieldvalue blob,
PRIMARY KEY (id, version, fieldname))
INSERT INTO ... VALUES( D5B...0A5, 3AF4...D93, ‘title’, ‘Monthly Report’)
(3AF4..D93, 'title', 'fieldvalue')
Friday, September 12, 14
Sparse Format•Add one extra component for collections:
• (clustering_1, ..., column_name, collection_element)
Friday, September 12, 14
Sparse FormatCREATE TABLE versioned_docs (id uuid,
version timeuuid,
fields map<text, blob>,
PRIMARY KEY (id, version))
Friday, September 12, 14
Sparse FormatCREATE TABLE versioned_docs (id uuid,
version timeuuid,
fields map<text, blob>,
PRIMARY KEY (id, version))
INSERT INTO ... VALUES ( D5B...0A5, 3AF4...D93, {‘title’: ‘Monthly Report’})
(3AF4..D93, 'fields', 'title')
Friday, September 12, 14
Sparse Format• If there are no clustering columns, the cell name
is just the column name
CREATE TABLE users ( id uuid PRIMARY KEY, name text, age int)
•Cell names:(‘name’) and (‘age’)
Friday, September 12, 14
Sparse Format• If COMPACT STORAGE is used, we drop the
composite
CREATE TABLE users ( id uuid PRIMARY KEY, name text, age int)WITH COMPACT STORAGE
•Cell names: ‘name’ and ‘age’
Friday, September 12, 14
Dense Format•Used for COMPACT STORAGE tables with
clustering columns
CREATE TABLE sensor_readings ( id uuid, time timestamp, value float, PRIMARY KEY (id, time))WITH COMPACT STORAGE
Friday, September 12, 14
Dense Format•Don’t store column name in cell name
CREATE TABLE sensor_readings ( id uuid, time timestamp, value float, PRIMARY KEY (id, time))WITH COMPACT STORAGE
INSERT INTO ... VALUES ( D5B...0A5, 1410279227535, 0.64)
1410279227535
Friday, September 12, 14
Dense Format•Multiple clustering keys result in composite cell
nameCREATE TABLE sensor_readings ( id uuid, time timestamp,
attribute text, value float, PRIMARY KEY (id, time, attribute))WITH COMPACT STORAGE
•Cell name: (1410279227535, ‘temperature’)
Friday, September 12, 14
Formats•Sparse Composite
• CQL3 tables
•Sparse Simple• “Static” compact storage, no clustering columns• e.g. “users” table
•Dense Simple• compact storage w/ one clustering column• e.g. sensor readings
•Dense Composite• compact storage w/ multiple clustering columns• e.g. sensor readings w/ multiple fields
Friday, September 12, 14
Internal Classes• CellName and CellNameType
• Sparse/Dense
• Simple/Composite
Friday, September 12, 14
Questions?
@tylhobbs
Friday, September 12, 14