NoSQL: Which way to go?

DDD Melbourne 2015

Himanshu Desai, Ahmed Elharouny & Daniel Janczak

About us

• Himanshu Desai (@H_Desai)• Ahmed Elharouny(@Harouny)• Daniel Janczak (@DanielJanczak)

And we all work for Readify

Page

Why NoSQL?

Scale

Pros NoSQL technologies can offer more scalability than relational databases

ConsOften lose some benefits of relational databases e.g. database-wide transactions

Better Formats

ProsAvoiding ORMs

ConsLimited BI tooling, Data hard to share outside the application

Work in more flexible ways

Pros No fixed schemas

ConsFixed schemas help prevent errors

Page

Document DatabaseDatabase

CollectionsCollections

Database

{ “title”: “Justin …”, “genre”: “Pop”, “price”: 36}

Document 1

{ “title”: “Don’t Spe..”, “genre”: “Techno”, “price”: 42.99}

Document 2

{ “numberOfSongs”: 2, “last”: “Jan 1, 2015”}

Document 3

Request

Application

{ }

RavenDB

What is RavenDB?› Open source NoSQL database for .NET

› Developed by Hibernating Rhinos Ltd

› Available› As a standalone package› As embedded version› On Azure and Amazon

Schema-freeStore data as Plain Old C# Object (POCO)

Or directly as JSON

var author = new Author {

Name = "George Martin", Books = new[] {

new Book {Title = "A Game of Thrones"},new Book {Title = "A Clash of Kings"}

} };

session.Store(author);

{ "Name": "George Martin", "Books": [ { "Title": "A Game of Thrones", "Genre": "Fantasy"

}, { "Title": "A Clash of Kings", "Genre": "Fantasy

} ]}

Scalable› Replication

› “Keep copies of my data somewhere else”

› Sharding› “Split my huge database into smaller parts”

› Multi-tenancy› “Single instance serves multiple customers = many

databases per server”

Extensible› Use existing bundles

› Compression› Encryption› Expiration› Periodic Export› Quotas› Replication› Scripted Index› SQL Replication› Versioning

› Or write your own plugins› Triggers› Codecs› Tasks› Compilation Extensions› Analyzer Generators› Database configuration› Custom sorters

Easy to use› .NET and JAVA client

› LINQ to query data

› REST API› GET http://localhost:8080/docs/bobs_address› PUT http://localhost:8080/docs/bobs_address -d "{ FirstName:

'Bob', LastName: 'Smith', Address: '5 Elm St' }“› And so on…

Easy to manage› .Built-in management tool – no need to download anything extra

Optimized › Transactional (ACID) on document operations

› In plain English “your data is safe”

› But only BASE for queries› “Basically Available, Soft state, Eventual Consistency”› In plain English:

› you will always get the results of your queries as fast as possible

› but the results won’t always be up to date until database finishes indexing

› If you want them up to date, you have to demand it explicitly

Eventual consistency Good or bad ?Newspaper Good! Very few writes, thousands of

reads, doesn’t have to be immediateProduct signup Good! All applicant information held

togetherBanking system What if the credits and debits

sometimes don’t match?Patient drug monitoring

What if the drug doses are not updated on time?

Our test datapublic class Author {

public string Name {get; set;} public string Surname {get; set;} public Book[] Books {get; set;}

} public class Book {

public string Title {get; set;} public string Genre {get; set;} public DateTime? PublishedOn {get; set;} public decimal? Price {get; set;}

}

Our test data continues…• George Martin

• A Game of Thrones, Fantasy, 6/8/1996, $12.34• A Clash of Kings, Fantasy, 16/11/1998

• Dan Simmons• Hyperion, Science Fiction, 1/1/1989, $34.56• Terror, Thriller, 8/1/2007, $23.45

• John Tolkien• Hobbit, Fantasy, 21/9/1937, $12.34

• Stephen King• The Stand, Horror, 1/9/1978, $12.34

A quick look at code

Code samples:

https://github.com/danieljanczak/ddd-melb2015-ravendb-samples

MongoDB

A scalable, high-performance, open source, schema-free, document-oriented database.

- mongodb.org

What is MongoDB?

MongoDB Popularity

Source: db-engins.com

MongoDB FeaturesThe best features of key/values stores, document databases and relational databases in one.

• Open source

MongoDB FeaturesThe best features of key/values stores, document databases and relational databases in one.

• Open source• Cross platform

The best features of key/values stores, document databases and relational databases in one.

MongoDB Features

• Open source• Cross platform• Strong indexing

The best features of key/values stores, document databases and relational databases in one.

MongoDB Features

• Open source• Cross platform• Strong indexing• Strong query language

The best features of key/values stores, document databases and relational databases in one.

MongoDB Features

• Open source• Cross platform• Strong indexing• Strong query language• Replication

The best features of key/values stores, document databases and relational databases in one.

MongoDB Features

• Open source• Cross platform• Strong indexing• Strong query language• Replication• Sharding

The best features of key/values stores, document databases and relational databases in one.

MongoDB Features

• Open source• Cross platform• Strong indexing• Strong query language• Replication• Sharding• Native drivers

The best features of key/values stores, document databases and relational databases in one.

MongoDB Features

• Open source• Cross platform• Strong indexing• Strong query language• Replication• Sharding• Native drivers• File storage (GridFS)

The best features of key/values stores, document databases and relational databases in one.

MongoDB Features

• Open source• Cross platform• Strong indexing• Strong query language• Replication• Sharding• Native drivers• File storage (GridFS)• Aggregation framework + MapReduce

MongoDB Getting Started1

DOWNLOADMongoDB 64 Bit for Windows(Link)

2

Create Data Directory

3

Start MongoDB

Demo

Code samples:https://gist.github.com/harouny/a29e5455d9ea9d773a6ehttps://gist.github.com/harouny/a00db1065fbbcfa8d79b

DocumentDB

How would you access it?

Application

Database

Collections

StoredProcedures Triggers UDFs

Documents

Libraries: .NET, JavaScript, Node.js, Python,

…REST

Collections

Database

DocumentDB Working with data

RESTful access methods

For Create/Read/Update/Delete (CRUD) operations

DocumentDB SQL

A query language with SQL-derived syntax

Example:SELECT c.BooksFROM Authors c WHERE c.name = “George"

Executing logic in the DB ( Javascript)

Stored procedures

Triggers

User-defined functions (UDFs)- Allow extending

DocumentDB SQL

Sharding and Transaction

Collection

Collection

Collection

Database

JSON

JSON

JSON

JSON

JSON

JSON

JSON

JSON

Collection

JSON

JSON

JSON

The unit of Sharding is a collection is a

collection

Atomic transactions can span only a single collection

collection

.NET Development

• You need following from the portal after provision a database• Endpoint Url• Authorization key

• Nu-get package• Install-Package Microsoft.Azure.DocumentDB

• Use them to create DocumentClient

Demo

Page

In a nutshell

In a nutshell - ACID

• RavenDB• Fully transactional writes on document level, even between collections• Reads use eventual consistency• Support for DTS

• MongoDB• Only supports ACID transactions at the document level.• Multi-collection (table) transactions not supported.• Users can choose the level of consistency using (WriteConcern and ReadPreference)

• DocumentDB• Full ACID support on the collection level• 4 levels of consistency, Extremely rich query model.

In a nutshell - availability and tooling• RavenDB

• On-prem (Windows), Cloud (RavenHQ) on Azure and Amazon, Embedded version / unit testing• Built in web based Management Studio• .NET and JAVA drivers, REST API• Possible access through LinqPad

• MongoDB• On-prem (Windows, Linx, Mac), Cloud (i.e MongoLab)• MongoDB Cloud Manager (monitoring, automated management)• Mongo Shell• Lots of 3rd party GUIs i.e. Robomongo

• DocumentDB• Accessed from virtually “any” system. Available in “Azure” only at this point.• Azure Web Portal• DocumentDB Studio• LinqPad• SDK available for various platform and REST based access

In a nutshell – querying and indexing• RavenDB

• Only very basic queries supported without static indexes• Map-reduce• Full text search• Eventual consistency by default

• MongoDB• Strong indexing (_id indexed by default, custom indexes, compound, unique, text). Async indexing.• Query from primary by default (Strong consistency) • Query from “nearest” if needed (Eventual consistency)• When writing you choose when to consider write operation successful.

• DocumentDB• Very rich query model – Supports Linq to objects, Lambda expression and SQL style syntax. Supports

Stored procedure, triggers and UDFs• Auto index by default.• Support lazy indexing if required.• 4 levels of consistency to choose from (Strong, Session, Bounded Stalesness,Eventual consistency

In a nutshell - performance

• RavenDB• Writes directly to disk (safety over speed)• Indexing in the background• Query results are instantly available in indexes• May need to wait for specific update

• MongoDB• TCP/IP based communication.• Uses BSON (lightweight, traversable, efficient).• MMAPv1 storage engine uses OS memory mapped files.• Wiredtiger storage engine (7,000,000 op/second 28 threads).• Choose level of performance vs consistency.

• DocumentDB• 3 Levels of performance offered. • Normalised CPU, memory, Disk and network as Request Units (RU)• TCP/HTTPS based access available

In a nutshell – gotchas!

• RavenDB• Need to reset underlying Lucene indexes – they do crash• Database defaults (winning numbers: 15, 50, 16384; 128, 1024)• Static indexes• Eventual consistency

• MongoDB• 32-bit limitations• Max document size 16 MB• Modeling currency

• DocumentDB• Max response size : 1MB• Max request size including attachment: 512KB.• Consistency is set at database level not at collection level.

Thank you