radu pintilie + liviu mazilu document db

Managed NoSQL databaseDocumentDB

RADU PINTILIELIVIU MAZILU

October 25, 2014

DocumentDB

© EXPERT NETWORK

CODECAMP

Challenges in distributed applicationsSQL Azure FederationHDInsight

Previous subjects

October 25, 2014

DocumentDB

© EXPERT NETWORK

DocumentDB

The need for storageDocumentDB OverviewDevelopmentCase Scenarios

Agenda

October 25, 2014

DocumentDB

© EXPERT NETWORK

The need for storage

Why do we store data?How do we store it?What’s important?

October 25, 2014

DocumentDB

© EXPERT NETWORK

What are the options

Flat filesRelationalNon-relational

Key-valueTabularDocument

October 25, 2014

DocumentDB

© EXPERT NETWORK

What’s important

CAP theoremConsistency – each unit always has the same view of the dataAvailability – all units can always read or writePartition tolerance – system works well across physical network partitions

Plot twist : you can choose only two

October 25, 2014

DocumentDB

© EXPERT NETWORK

Consistent, Available (CA) SystemsCA Systems have trouble with partitions and typically deal with it with replication. Examples of CA systems include:

Traditional RDBMSs like Postgres, MySQL, etc (relational)Vertica (column-oriented)Aster Data (relational)Greenplum (relational)

October 25, 2014

DocumentDB

© EXPERT NETWORK

Consistent, Partition-Tolerant (CP) Systems

CP Systems have trouble with availability while keeping data consistent across partitioned nodes. Examples of CP systems include:

BigTable (column-oriented/tabular) Hypertable (column-oriented/tabular) HBase (column-oriented/tabular) MongoDB (document-oriented) Terrastore (document-oriented) Redis (key-value) Scalaris (key-value) MemcacheDB (key-value) Berkeley DB (key-value)

October 25, 2014

DocumentDB

© EXPERT NETWORK

Available, Partition-Tolerant (AP) Systems

AP Systems achieve "eventual consistency" through replication and verification. Examples of AP systems include:

Dynamo (key-value) Voldemort (key-value) Tokyo Cabinet (key-value) KAI (key-value) Cassandra (column-oriented/tabular) CouchDB (document-oriented) SimpleDB (document-oriented) Riak (document-oriented)

October 25, 2014

DocumentDB

© EXPERT NETWORK

DocumentDB

Fully managedSchema-less, NoSQL document databaseStored entities are JSON documentsTunable consistencyDesigned to scale into petabytes

Features

October 25, 2014

DocumentDB

© EXPERT NETWORK

Databases in Azure

RelationalSQL Database (PaaS)SQL Server (IaaS)

NoSQLAzure Tables – structured, non-relational

dataDocumentDB – document database

October 25, 2014

DocumentDB

© EXPERT NETWORK

Resource Model

Database AccountDatabase

CollectionDocument

AttachmentStored ProcedureTriggerUser-defined functionsUser

Permission

Media

October 25, 2014

DocumentDB

© EXPERT NETWORK

Resource Addresing

Interface is RESTful

Each resource has a unique ID

API URL : codecamp.documents.azure.com

Document path : /dbs/{database id}/colls/{collection id}/docs/{document id}

Example URL : dbs/Cv8kAA==/colls/Cv8kAMUKpAA=/docs/Cv8kAMUKpAACAAAAAAAAAA==/

October 25, 2014

DocumentDB

© EXPERT NETWORK

Operations

For each resourceCreateReplaceDeleteReadQuery

Read – GET Operation on a specified ID, returns a single resource.Query – POST Operation on a collection with a request containing DocumentDB SQL text, returning a collection

October 25, 2014

DocumentDB

© EXPERT NETWORK

DocumentDB SQL

SELECT <select-list>

FROM <from-specification>

WHERE <filter-condition>

Similar to normal SQL

Ability to reach into JSON tree to:Access values for filter conditionShape select list

User-defined functions

October 25, 2014

DocumentDB

© EXPERT NETWORK

Consistency Levels

Strong - the operation will not return until the query has been made durable

Bounded Staleness - guarantees the order of propagation of writes but with reads potentially lagging behind the writes - useful for applications dealing with time and ordered operations

Session - strong consistency scoped to a single client session. This consistency level is usually sufficient

Eventual - the weakest form of consistency where a client may get the values which are older than the ones it had seen before, over time. Lowest latency for reads and writes

October 25, 2014

DocumentDB

© EXPERT NETWORK

Indexing Policy

Specified at the collection levelAutomatic indexing

By default all properties indexed automatically. This is tunable for individual documents and paths within a document – either inclusion or exclusion of a pathIndex precision can be specified for strings and numbers

Indexing modeConsistent – By default indexes synchronously updated on insert, replace or delete Lazy – asynchronous index update (targeted at bulk ingestion)

October 25, 2014

DocumentDB

© EXPERT NETWORK

Performance

Capacity UnitSpecified amount of storage capacity and operational throughputCollection quota per capacity unitProvisioning unit for scaleout for both performance and storageConfigured at the database account levelPreview limit is 10GB, 3 collections per capacity unit

Storage is SSD backedMicrosoft has used databases with terabytes of storage (designed for petabytes)

October 25, 2014

DocumentDB

© EXPERT NETWORK

Stored Procedures,Triggers and UDFs

DocumentDB supports server-side JavaScriptStored Procedures:

Registered at collection levelOperate on any document in the collectionInvoked inside transaction Triggers:Pre- or Post: create, replace or delete operationsInvoked inside transaction

User-Defined FunctionsScalar functions invoked only inside queries

October 25, 2014

DocumentDB

© EXPERT NETWORK

RESTful API

Core interface to DocumentDBUsed by all client libraries

Standard operations against all DocumentDB resources:

CREATE, DELETE, PUT, GET, POSTReturns permanent resource URL on creation

DocumentDB request headers

October 25, 2014

DocumentDB

© EXPERT NETWORK

SCENARIOS

Good for unstructured dataDenormalized schemaNeed to scaleHybrid solutions (RDBMS + NoSQL)

USE CASE

October 25, 2014

DocumentDB

© EXPERT NETWORK

Conclusions

The need for storageDocumentDB OverviewDevelopmentCase Scenarios

radu pintilie + liviu mazilu document db

Technology