microservices with node.js and apache cassandra

Microserviceswith Node.js and Apache Cassandra

1

Jorge Bay Gondra

@jorgebg

Software Engineer, Drivers team at DataStax

https://twitter.com/jorgebg

© 2015 DataStax, All Rights Reserved. Company Confidential

Topics

1- Microservices

2- Why Node.js and Cassandra

3- The Node.js driver

4- Putting it all together: killr-service

2

1- Microservices

3


1- Microservices

Defining microservices

4

- Service as software component

- 1 service per process

- Different data storage per service


1- Microservices

From monolithic server architecture

5

- Single process contains all the logic

- Simple to develop

- Simple to deploy


1- Microservices

From monolithic server architecture

6

- Large code bases

- Hard add new features

- Hard to train new developers

- Long term technology commitment


1- Microservices

Why Microservices

7

- Codebase <---> teams

- Independently deployable: Versioning

- Fault isolation

- Best technology for the job

- Long lived systems, short lived services


1- Microservices

But some things get harder

8

- Inter-service communication

- Deployment


1- Microservices

9

Monolithic architecture

UI

Services

Data Access

Browser Monolithic Application Relational database


1- Microservices

10

Monolithic architecture

Store UI Order UI Customer UI

Catalog Svc Order Svc Customer Svc

Product DA Order DA Customer DA

Browser Monolithic Application Relational database


1- Microservices

11

Microservices architecture

Browser Front end Services Data storage

Order

Product

Customer


1- Microservices

12

Inter-service communication

Products

Orders

Message Broker


1- Microservices

Inter-service communication

13

- Event based

- Async

- Start simple


1- Microservices

14

Microservices architecture

Browser Front end Services Data storage

Order

Product

Customer

Message broker


1- Microservices

Recap

15

- Service as component

- Deploy / Versioning

- Own your data

2- Node.js + Cassandra

16



Node.js

17

- Single paradigm: async IO / event loop

- Minimum overhead per connection

- Predictable amount of memory usage under load



Cassandra

18

- Tunable consistency

- No master / slave

- Fault tolerant

3- The Node.js Driver

for Apache Cassandra

19


3- The Node.js Driver for Cassandra

Features

20

- Automatic failover

- Node discovery

- Tunable policies

- Request pipelining

- TLS



Automatic failover

21Client 1

DC 1

DC 2

Requests



Node discovery

22

Client

DC 1

Request

Events

Response



Tunable policies

23

- Load balancing

- Retry

- Reconnection



Load balancing policy sample

24

function BlackListPolicy(blackListedHost, childPolicy) {this.blackListedHost = blackListedHost;this.childPolicy = childPolicy;

}util.inherits(BlackListPolicy, LoadBalancingPolicy);

BlackListPolicy.prototype.newQueryPlan = function (keyspace, queryOptions, callback) {this.childPolicy.newQueryPlan(keyspace, queryOptions, function (err, iterator) {callback(err, filter(iterator));

});};function *filter () {/**/}



Ecmascript 6

25

- Load balancing policies: generators

- Encoding / decoding: ES6 Map / Set



Request pipelining

26

Client

Sending first request

(1)

Cassandra Nodetim

e

Received first response

(2)



Recap

27

- Failover built-in

- Connection pooling

- Defaults

4- Putting it all together

Demo: Microservices with

Node.js and Cassandra Demo

28


4- Microservices Demo

Functionality

29

- Video Comments

- Video Ratings



Schema

30

CREATE TABLE comments_by_video (

videoid uuid,

commentid timeuuid,

userid uuid,

comment text,

PRIMARY KEY (videoid, commentid))

WITH CLUSTERING

ORDER BY (commentid DESC);

CREATE TABLE comments_by_user (

userid uuid,

commentid timeuuid,

videoid uuid,

comment text,

PRIMARY KEY (userid, commentid))

WITH CLUSTERING

ORDER BY (commentid DESC);



Schema

31

CREATE TABLE video_rating (

videoid uuid,

rating_counter counter,

rating_total counter,

PRIMARY KEY (videoid));

CREATE TABLE video_ratings_by_user (

videoid uuid,

userid uuid,

rating int,

PRIMARY KEY (videoid, userid));



Service methods

32

- GET comments by video

- POST comment

- GET rating by video

- POST rating



GET comments by video

33

var cassandra = require('cassandra-driver');var client = new cassandra.Client({ contactPoints: ['localhost']});var query = 'SELECT videoid, commentid, userid, comment FROM comments_by_video WHERE videoid = ?';app.get('/v1/comments/:videoId([a-f0-9\\-]{36})', function (req, res, next) {

client.execute(query, [req.params.videoId], { prepare: true }, function (err, result) {if (err) return next(err);res.json(result.rows);

});});



GET comments by video

34

var repository = new Repository(client);//...app.get('/v1/comment/:videoId([a-f0-9\\-]{36})', function (req, res, next) {

repository.getCommentsByVideo(req.params.videoId, function (err, comments) {if (err) return next(err);res.json(comments);

});});



POST comment

35

app.post('/v1/comment/:videoId([a-f0-9\\-]{36})', function (req, res, next) {

repository.insertComment(req.params.videoId, req.body.userId, req.body.comment, (err, id) {

if (err) return next(err);

res.send(id.toString());

});

});



Repository: Insert comment

36

Repository.prototype.insertComment = function (videoId, userId, comment, callback) {

var commentId = cassandra.types.TimeUuid.now();

var queries = [

{ query: 'INSERT INTO comments_by_video (videoid, commentid, userid, comment) VALUES (?, ?, ?, ?)',

params: [videoId, commentId, userId, comment]},

{ query: 'INSERT INTO comments_by_user (userid, commentid, videoid, comment) VALUES (?, ?, ?, ?)',

params: [userId, commentId, videoId, comment]}];

this.client.batch(queries, { prepare: true }, callback);

});



Repository: Insert comment

37

Repository.prototype.insertComment = function (videoId, userId, comment, callback) {

var commentId = cassandra.types.TimeUuid.now();

var queries = [/*...*/];

var bus = this.bus;

this.client.batch(queries, { prepare: true }, function (err) {

if (!err) {

bus.publishNewComment(videoId, commentId, userId, comment);

}

callback(err, commentId);

});

});



Message broker client

38

function Bus() {

}




39

function Bus() {

}

Bus.prototype.publishNewComment = function (videoId, /*...*/) {

//Insert magic here

};




40

function Bus() {

events.EventEmitter.call(this);

var self = this;

serverLib.on('whatever.topic.event.name', function (data) {

self.emit('whatever-client-name', data);

});

}

util.inherits(Bus, events.EventEmitter);

Bus.prototype.publishNewComment = function () {/*...*/};



Seneca sample: GET comments

41

seneca.add( {role: 'comment', cmd: 'get'}, function (args, done) {repository.getCommentsByVideo(args.id, done);

});



Recap

42

- 3 classes

- ~150 LoC

- Boundaries


Thanks

@jorgebg

github.com/jorgebay/killr-service

datastax.com/dev/blog

bit.ly/nodejs-cassandra-user

43

https://twitter.com/jorgebg

https://github.com/jorgebay/killr-service

http://www.datastax.com/dev/blog

http://bit.ly/nodejs-cassandra-user


Thanks

Further reading

- Sample project: killr-service

- Presentation: Implementing Micro-Service Architecture by Fred George

- Book: Building Microservices By Sam Newman (O'Reilly)

- Article: Microservices Architecture By James Lewis and Martin Fowler

44

https://github.com/jorgebay/killr-service

http://new.livestream.com/websummit/DeveloperStage2013/videos/33534085

http://shop.oreilly.com/product/0636920033158.do

http://martinfowler.com/articles/microservices.html

microservices with node.js and apache cassandra

Software

introduction of microservices

brief theory microservices

single process

new features hard

company confidentialits

large code bases hard

monolithic applications

single unit