full stack development with node.js and nosql (nic raboy & arun gupta)

Full Stack Development with Node.js and NoSQL

Nic Raboy (@nraboy)Arun Gupta (@arungupta)

The Role of a Full Stack Developer

Application Stacks

The Power of the Flexible JSON Schema

• Ability to store data in multiple ways

• De-normalized single document, as opposed to normalizing data across multiple table

• Dynamic Schema to add new values when needed

Where is NoSQL a Good Fit?

Low-latency CriticalHigh Throughput or Large Numbers of Users

Unknown Demand with Sudden growth

Predominantly Direct Document Access

Read / Mixed / Write-heavy Workloads

Traditional Business ApplicationsTransaction-heavy AppsLegacy HardwareFull ACID support

Web / Mobile / IoT

Legacy Business Apps

What is Couchbase?

Couchbase is a distributed operational database that enables you to develop with agility and operate at any scale.

Managed Cache Key-Value Store Document Database

Embedded Database

Sync Management

Develop with Agility

Easier, Faster Development

Flexible Data Modeling

Powerful Querying

SQL Integration & Migration

Big Data Integration Mobile / IoT

Operate at Any Scale

Elastic Scalability Consistent High Performance

Always-on Availability

Multi-Data Center Deployment

Simple, Powerful Administration

Enterprise Grade Security

Couchbase Developer

Couchbase integrates with the Big Data ecosystem

Couchbase Server – Single Node Architecture Data Service – builds and

maintains local view indexes

Indexing Engine – builds and maintains Global Secondary Indexes

Query Engine – plans, coordinates, and executes queries against either Global or Local view indexes

Cluster Manager – configuration, heartbeat, statistics, RESTful Management interface

Simplified Administration• Online upgrades and operations• Built-in enterprise class Admin Console• RESTful APIs

Accessing Data From Couchbase

Key access using Document ID

• Operations are extremely fast with consistent low latency

• Reads and writes are evenly distributed across Data Service nodes

• Data is cached in built-in Managed Caching layer and stored in persistent storage layer

Queries using N1QL

• SQL-like : SELECT * FROM WHERE, LIKE, GROUP, etc.,

• JOINs• Powerful Extensions

(nest, unnest) for JSON to support nested and hierarchical data structures.

• Multiple access paths – Views and global secondary indexes

• ODBC/JDBC drivers available

Views using static queries

• Pre-computed complex Map-Reduce queries

• Incrementally updated to power analytics, reporting and dashboards

• Strong for complex custom aggregations

The Couchbase Node.js SDK

Uses the high performance Couchbase C library Compatible with Node.js frameworks

– Express, Sails, Etc. Minimal coding

– No database maintenance via code– No parsing queries via application code

Building Applications with Node.js SDK//including the Node.js dependencyvar Couchbase = require(”couchbase");

//connecting to a Couchbase clustervar cluster = new Couchbase.Cluster("couchbase://localhost:8091");

//opening a bucket in the clustervar myBucket = cluster.openBucket("travel-sample", "");

//preparing N1qlvar myQuery = Couchbase.N1qlQuery();

//creating and saving a Documentvar document = { firstname: "Nic", lastname: "Raboy"};myBucket.insert("my-key", document, function(error, result) {});

Node.jsfunction query(sql, done) { var queryToRun = myQuery.fromString(sql); myBucket.query(queryToRun, function(error, result) { if(error) { console.log(“ERROR: “, error); done(error, null); return; } done(null, result); return; });}

Demo Time!

The CEAN Stack

Stack Design

AngularJSClient Frontend

Node.jsServer Backend

Node.js Separation of Frontend and Backend No Jade markup API driven Less client and server coupling The backend can evolve without affecting the frontend Frontend can be extended to web as well as mobile

Multiple Frontends & One Backend

The Node.js Backend

Node.js Configuration// config.json

{ "couchbase": { "server": "couchbase://localhost", "bucket": "restful-sample" }}

Node.js Configuration// Project’s app.js file

var express = require("express");var bodyParser = require("body-parser");var couchbase = require("couchbase");var path = require("path");var config = require("./config");var app = express();

Node.js Configuration// app.js continued…

app.use(bodyParser.json());app.use(bodyParser.urlencoded({ extended: true }));

module.exports.bucket = (new couchbase.Cluster(config.couchbase.server)).openBucket(config.couchbase.bucket);

app.use(express.static(path.join(__dirname, "public")));

var routes = require("./routes/routes.js")(app);

var server = app.listen(3000, function () { console.log("Listening on port %s...", server.address().port);});

Node.js Create or Update Endpoint// routes.js

app.post("/api/create", function(req, res) { if(!req.body.firstname) { return res.status(400).send({"status": "error", "message": ”First Name?"}); } // … RecordModel.create(req.body, function(error, result) { if(error) { return res.status(400).send(error); } res.send(result); });});

Node.js Get Document Endpoint// routes.js continued…

app.get("/api/get", function(req, res) { if(!req.query.document_id) { return res.status(400).send({"status": "error", "message": ”Document ID?"}); } RecordModel.getByDocumentId(req.query.document_id, function(error, result) { if(error) { return res.status(400).send(error); } res.send(result); });});

Node.js Delete Endpoint// routes.js continued…

app.post("/api/delete", function(req, res) { if(!req.body.document_id) { return res.status(400).send({"status": "error", "message": ”ID?"}); } RecordModel.delete(req.body.document_id, function(error, result) { if(error) { return res.status(400).send(error); } res.send(result); }); });

Node.js Upsert Document Function// recordmodel.jsRecordModel.save = function(data, callback) { var jsonObject = { type: “user”, firstname: data.firstname, lastname: data.lastname, email: data.email } var documentId = data.document_id ? data.document_id : uuid.v4(); db.upsert(documentId, jsonObject, function(error, result) { if(error) { return callback(error, null); } callback(null, {message: "success", data: result}); });}

Couchbase JSON Document

Node.js Get Document with N1QL Function// recordmodel.js continued…

RecordModel.getByDocumentId = function(documentId, callback) { var statement = "SELECT firstname, lastname, email " + "FROM `" + config.couchbase.bucket + "` AS users " + "WHERE META(users).id = $1"; var query = N1qlQuery.fromString(statement); db.query(query, [documentId], function(error, result) { if(error) { return callback(error, null); } callback(null, result); });};

Node.js Delete Document Function// recordmodel.js continued…

RecordModel.delete = function(documentId, callback) { db.remove(documentId, function(error, result) { if(error) { callback(error, null); return; } callback(null, {message: "success", data: result}); });};

The AngularJS Frontend

Get all documents// AngularJS app.js

$scope.fetchAll = function() { $http( { method: "GET", url: "/api/getAll" } ) .success(function(result) { for(var i = 0; i < result.length; i++) { $scope.items[result[i].id] = result[i]; } }); }

Save a document// AngularJS app.s

$scope.save = function(firstname, lastname, email) { $http( { method: "POST", url: "/api/save", data: { firstname: firstname, lastname: lastname, email: email, document_id: $stateParams.documentId } } )}

More Complex Node.js Queries

Node.js Travel SampleFlightPath.findAll = function(from, to, callback) { var statement = "SELECT faa AS fromAirport, geo " + "FROM `" + config.couchbase.bucket + "` r" + "WHERE airportname = $1 " + "UNION SELECT faa AS toAirport, geo " + "FROM `" + config.couchbase.bucket + "` r" + "WHERE airportname = $2"; var query = N1qlQuery.fromString(statement); db.query(query, [from, to], function(error, result) { if(error) { return callback(error, null); } callback(null, result); });};

Node.js Travel SampleFlightPath.findAll = function(queryFrom, queryTo, leave, callback) { var statement = "SELECT r.id, a.name, s.flight, s.utc, r.sourceairport, r.destinationairport, r.equipment " + "FROM `" + config.couchbase.bucket + "` r" + "UNNEST r.schedule s " + "JOIN `" + config.couchbase.bucket + "` a ON KEYS r.airlineid " + "WHERE r.sourceairport = $1 AND r.destinationairport = $2 AND s.day = $3 ” "ORDER BY a.name"; var query = N1qlQuery.fromString(statement); db.query(query, [queryFrom, queryTo, leave], function(error, result) { if(error) { return callback(error, null); } callback(null, result); });};

Node.js Sample Applications

https://github.com/couchbaselabs/try-cb-nodejshttps://github.com/couchbaselabs/restful-angularjs-nodejs

Couchbase N1QL Tutorial

http://query.pub.couchbase.com/tutorial/

Ottoman ODM

Ottoman Model

var RecordModel = ottoman.model("Record", { firstname: {type: "string”}, lastname: {type: "string"}, email: {type: "string"}, created_at: {type: “Date”, default: Date.now}});

Ottoman Saving

var myRecord = new RecordModel({ firstname: “Nic”, lastname: “Raboy”, email: “nic@couchbase.com”});

myRecord.save(function(error) { if(error) { // Error here } // Success here});

Ottoman Finding

RecordModel.find({}, function(error, result) { if(error) { // Error here } // Array of results here});

Couchbase Mobile

Couchbase LiteEmbedded NoSQL Database

Sync GatewaySecure Synchronization

Couchbase Server

Cloud NoSQL Database

Couchbase Mobile

Couchbase Lite

Full FeaturedLightweightNativeSecureJSON

Sync Gateway

Secure SynchronizationAuthenticationData Read AccessData Write Access

Couchbase Server

Highly scalableHigh PerformanceAlways onJSON

Couchbase Lite NoSQL Mobile database Runs in-process Small footprint

var database = new Couchbase("nraboy-database");var documentId = database.createDocument({ "firstname": "Nic", "lastname": "Raboy", "address": { "city": "San Francisco", "state": "CA", "country": "USA" } "twitter": "https://www.twitter.com/nraboy"});

Map Reduce Indexes• Building indexes in your native language• Results are persisted for fast querying• Just set breakpoints to debug

// Create an indexdatabase.createView("people", "1", function(document, emitter) { if(document.twitter) { emitter.emit(document._id, document); }});

// Now query itvar rows = database.executeQuery("people");for(var i = 0; i < rows.length; i++) { // print row[i]}

Change Notifications

Change notifications• Listen for changes• Cuts down on a ton of cruft code• Data, queries, replications – even documents

database.addDatabaseChangeListener(function(changes) { for(var i = 0; i < changes.length; i++) { if(!changes[i].isConflict()) { console.log(changes[i].getDocumentId()); } }});

Sync• Full multi-master replication• Ability to minimize battery drain• Change notifications and conflict detection

var push = database.createPushReplication("http://syncgateway");var pull = database.createPullReplication("http://syncgateway");

push.setContinuous(true);pull.setContinuous(true);

push.start();pull.start();

JCenter

Maven Central

Github

Demo Time!

What is Couchbase?

Where do you find us?• developer.couchbase.com• blog.couchbase.com• @couchbasedev or @couchbase• forums.couchbase.com• stackoverflow.com/questions/tagged/couchbase

POWER UP and PARTY DOWN with Red Hat

Mobile, Middleware and OpenShift.

Wednesday evening 9PM – 12AM

Pick up your invitation for the party that beats all parties at:

Mobile, Middleware or OpenShift demo pods in

the Red Hat Booth, Partner Pavilion

full stack development with node.js and nosql (nic raboy & arun gupta)

Software

oracle academy cloud program...y asistentes digitales, y...

node.js workshop- node.js unit testing

kullanilan node.js mongodb · mongodb ölçeklenebilir,...

full stack development with node and nosql - austin node.js...

healthcare messaging application - xb software€¦ ·...

node.js driver developer's guide...you access the data in...

full stack development with node.js and nosql

sql vs nosql: the nosql way

who am i?assets.astrails.com/.../wtf-is-mysql.pdf ·...

node.js workshop- node.js basics

raboy servicio publico en siglo xxi

node.js primer -...

developing for node.js with mysql and nosql

node.js workshop- node.js databases

node.js i getting started -...

node.js and nosql

introducing documentdb - david · pdf fileintroducing...

article by marc raboy and claudia padovani

nosql or not nosql?

web - oracle · •node.js nosql api couples high...