cassandra drivers and libraries

@doanduyhai

Cassandra Drivers And Tools DuyHai DOAN, Technical Advocate

@doanduyhai

Agenda!

2

Drivers •  architecture, policies, Java driver API

DevCenter (live coding demo!) Cassandra Unit (+ live coding demo!) Object Mapper module (+ live coding demo!) Achilles Object Mapper (+ live coding demo!)

Cassandra Drivers Architecture!

Architecture!Policies!

Java driver API!

@doanduyhai

Drivers list!

4

•  Java •  C# •  Python •  Node.js •  Ruby (1.0.0.rc1) •  C++ (beta) •  ODBC (beta) •  Clojure (community) •  Go (community) •  PHP (to be announced)

@doanduyhai

Connection pooling!

5

n2

n3

n4

Driver

Pool1

Pool2

Pool3

Client Thread1

Client Thread2

Client Thread3

@doanduyhai

Connection pooling!

6

n2

n3

n4

Driver

Pool1

Pool2

Pool3

Client Thread1

Client Thread2

Client Thread3

1

2 3

4 5

6

@doanduyhai

Request Pipelining!

7

Client Cassandra

@doanduyhai

Request Pipelining!

8

Client Cassandra

StreamID

StreamID

@doanduyhai

Nodes Discovery!

9

n2

n3

n4

n5

n6

n7

n8

n1 Driver

Control Connection

@doanduyhai

Round Robin Load Balancing!

10

n2

n3

n4

n5

n6

n7

n8

n1 Client 1

2 3

4

@doanduyhai

DC Aware Load Balancing!

11

Client1 DC1

DC2 Client2

⤫

@doanduyhai

DC Aware Load Balancing!

12

Client1 DC1

DC2 Client2

⤫

@doanduyhai

Token Aware Load Balancing!

13

n2

n3

n4

n5

n6

n7

n8

n1 Client

1

2

3

⤫

@doanduyhai

Combining Load Balancing Policies!

14

Load Balancing Policy

Round Robin DC Aware Round Robin

Token Aware

extends

wraps

Default config

@doanduyhai

Automatic Failover!

15

n2

n3

n4

Driver

Pool1

Pool2

Pool3

Client Thread

⤫

1

2 3

4 5

6 7 8

@doanduyhai

Other policies!

16

Retry policy •  write/read timeout •  node unavailable Reconnection policy •  constant schedule •  exponential schedule

@doanduyhai

Statements!

17

Plain statement •  convenient, one-off query •  plain string ☞ parsing overhead

INSERT INTO user(login, name, age) VALUES(‘jdoe’, ‘John DOE’, 33)’;

@doanduyhai

Statements!

18

Prepared statements •  avoid parsing overhead •  query structure should be known ahead of time •  bound values •  named parameters

INSERT INTO user(login, name, age) VALUES(?, ?, ?)’;

INSERT INTO user(login, name, age) VALUES(:login, :name, :age)’;

@doanduyhai

Statements!

19

Parameterized statements •  same as plain statement •  pass bound values as bytes ☞ avoid ser/deser of values

INSERT INTO user(login, name, age) VALUES(?, ?, ?)’;

@doanduyhai

Java Driver!

20

Reference implementation Base on asynchronous Netty library Configurable policies Query tracing support Client-node compression & SSL

@doanduyhai

Maven dependency!

21

<dependency> <groupId>com.datastax.cassandra</groupId> <artifactId>cassandra-driver-core</artifactId> <version>2.1.3</version> </dependency>

depends on Netty, Guava, Metrics

Available on Maven Central

@doanduyhai

Connect and Write!

22

Cluster cluster = Cluster.builder() .addContactPoints("127.0.0.1", “another-host").build();

seed nodes (IP or DNS name)

Session session = cluster.connect("my_keyspace");

session.execute("INSERT INTO user (user_id, name, email) VALUES (12345, 'johndoe', 'john_doe@fiction.com’)");

@doanduyhai

Configuration!

23

Cluster cluster = Cluster.builder() .addContactPoints("127.0.0.1", “another-host") .withLoadBalancingPolicy(new DCAwareRoundRobinPolicy("DC1") .withRetryPolicy(DowngradingConsistencyRetryPolicy.INSTANCE) .withReconnectionPolicy(new ConstantReconnectionPolicy(1000)) .build();

@doanduyhai

Read!

24

ResultSet resultSet = session.execute("SELECT * FROM user");

List<Row> rows = resultSet.all();

for (Row row : rows) { Long userId = row.getLong("user_id"); String name = row.getString("name"); String email = row.getString("email"); }

Stateless ☞ thread-safe

@doanduyhai 25

Asynchronous Read!

ResultSetFuture future = session.executeAsync("SELECT * FROM user");

ResultSet resultSet = future.get(); //blocking call

List<Row> rows = resultSet.all();

for (Row row : rows) { Long userId = row.getLong("user_id"); String name = row.getString("name"); String email = row.getString("email"); }

@doanduyhai 26

Asynchronous Read with CallBack!

ResultSetFuture future = session.executeAsync("SELECT * FROM user");

future.addListener(new Runnable() { public void run() { // Process the results here } }, executor);

executor = Executors .newCachedThreadPool(); or

executor = Executors .sameThreadExecutor();

@doanduyhai 27

Query Builder!

Query query = QueryBuilder .select() .all() .from( "my_keyspace", "user") .where(eq("login", "jdoe"));

query.setConsistencyLevel(ConsistencyLevel.ONE);

ResultSet rs = session.execute(query);

@doanduyhai

Old manual paging!

28

Some time you need to fetch all table content Manual paging:

SELECT * FROM users WHERE token(login) >= token(<last_fetched_login>) LIMIT 100;

@doanduyhai

New automatic paging!

29

n2

n3

n4

n5

n6

n7

n8

n1 Driver

Query

Page 1 + paging state 1

@doanduyhai

New automatic paging!

30

n2

n3

n4

n5

n6

n7

n8

n1 Driver

Query

Page 2 + paging state 2

@doanduyhai

New automatic paging!

31

Paging state ≈ stateless cookie Resilient to node failure

@doanduyhai

Paging during node failure!

32

n2

n3

n4

n5

n6

n7

n8

n1 Driver

Query

Page 1 + paging state 1

@doanduyhai

Paging during node failure!

33

n2

n3

n4

n5

n6

n7

n8

n1 Driver

Query

Page 2 + paging state 2

⤫

Q & R

! " !

Dev Center Demo

@doanduyhai

Cassandra Unit!

36

•  start an embedded Cassandra server

•  useful for unit testing

•  mature project, created at August 5th 2011

•  designed around Thrift & Hector

•  propose a JUnit rule for CQL

Cassandra Unit Demo

@doanduyhai

Java Driver Object Mapper!

38

•  simple mapper

•  KISS

•  annotations à-la JPA (but no JPA dependencies)

•  templating system à-la Spring Data

@doanduyhai

Java Driver Object Mapper!

39

@Table(keyspace =  "mapper_module", name = "users") public class User {

@PartitionKey private String login;

private String name; // getters and setters omitted... }

Mapping

@doanduyhai

Java Driver Object Mapper!

40

MappingManager manager = new MappingManager(session); Mapper mapper = manager.mapper(User.class);

User user = mapper.get("jdoe@fiction.com");

mapper.saveAsync(new User("hsue@fiction.com"));

mapper.delete("jdoe@fiction.com");

Usage

@doanduyhai

Java Driver Object Mapper!

41

@Accessor interface UserAccessor { @Query("SELECT * FROM users LIMIT :max") Result<User> firstNUsers(@Param("max") int limit); }

Accessors (SpringData template-like) definition

@doanduyhai

Java Driver Object Mapper!

42

Accessors usage

UserAccessor accessor = manager.createAccessor(UserAccessor.class); List<User> users = accessor.firstNUsers(10).all(); for (User user : users) { System.out.println( profile.getAddress().getZip() ); }

Java Driver Object Mapper Demo

@doanduyhai

Achilles!

44

Why ? •  started in late 2012, when mapper module did not exists

•  more involved and more features than the mapper module

•  different annotations set (may converge)

Achilles Demo

@doanduyhai

Dirty Checking!

46

Dirty checking, why is it important ? •  1 user ≈ 8 mutable fields

•  × n denormalizations = n update combinations

•  and not even counting multiple fields updates …

@doanduyhai

Dirty Checking!

47

•  Are you going to manually generate n prepared statements for all possible updates ?

•  Or just use dynamic plain string statements and get some perf

penalty ?

@doanduyhai

Dirty Checking!

48

//No read-before-write ContactEntity proxy = manager.forUpdate(ContactEntity.class, contactId); proxy.setFirstName(…); proxy.setLastName(…); //type-safe updates proxy.setAddress(…);

manager.update(proxy);

@doanduyhai

Dirty Checking!

49

Proxy Setters interception

Empty Entity

DirtyMap

PrimaryKey

@doanduyhai

Dirty Checking!

50

•  Dynamic statement generation

UPDATE contacts SET firstname=?, lastname=?,address=? WHERE contact_id=?

prepared statements are cached, of course

@doanduyhai

Main API!

51

manager.insert(entity) manager.update(entity) manager.remove(entity) manager.find(Entity.class, primaryKey)

@doanduyhai

Advanced Features!

52

Counter Batch mode Strategies (insert, naming) Options Asynchronous

@doanduyhai

Documentation!

53

Comprehensive Github WIKI Twitter-clone demo app (demo.achilles.io) Versioned documentation (HTML & PDF) JavaDoc

@doanduyhai

RoadMap!

54

C* 2.1 user defined types (UDT) Query-templates à-la Spring Data Reactive ? (RxJava) ElasticSearch integration (@olivierbourgain)

Q & R

! " !

Thank You @doanduyhai

duy_hai.doan@datastax.com

https://academy.datastax.com/