dotnettoscana: nosql revolution - ravendb

www.dotnettoscana.org

NoSQL revolutionAn introduction to the NoSQL world with real

lifeexamples using RavenDB and Redis

Luigi Berrettini

Nicola Baldi

Matteo Baglinihttp://it.linkedin.com/in/matteobaglini

http://it.linkedin.com/in/nicolabaldi

http://it.linkedin.com/in/luigiberrettini 15/12/2012

The movement

Document databases in practice

Luigi Berrettini

Nicola Baldihttp://it.linkedin.com/in/nicolabaldi

http://it.linkedin.com/in/luigiberrettini

Overview

15/12/2012 Document databases in practice 4

5

Safe by default

Unbounded result sets problem

Unbounded number of requests problem

15/12/2012 Document databases in practice - Overview

GRACEFULLY HANDLED

6

Document stores basics

15/12/2012 Document databases in practice - Overview

They favor denormalization overcomposition and joins

Relations are different than in RDBMSs

They are schema-less, but attention should be paid in designing documents

7

Domain model and implementation details

15/12/2012 Document databases in practice - Overview

« a conceptual model should be drawn with little or no regard for the software that might implement it » (Martin Fowler, UML Distilled)

A domain model should be independent from implementation details like persistence

In RavenDB this is somewhat true

8

RDBMSs vs document DBs

15/12/2012 Document databases in practice - Overview

RDBMS are schema-full• tuples = sets of key-value pairs ⇒ flat structure• more complex data structures are stored as

relations

Document databases are schema-less• object graphs stored as docs ⇒ no flat structure• each document is treated as a single entity

RavenDB suggested approach is to follow the aggregate pattern from the DDD book

9

The aggregate pattern (1)

ENTITY

15/12/2012 Document databases in practice - Overview

Some objects are not defined primarily by their attributes

They represent a thread of identity that runs through time and often across distinct representations

Mistaken identity can lead to data corruption

10

The aggregate pattern (2)

VALUE OBJECT

15/12/2012 Document databases in practice - Overview

When you care only about the attributes of an element of the model, classify it as a value object

Make it express the meaning of the attributes it conveys and give it related functionality

Treat the value object as immutable

Don't give it any identity and avoid the design complexities necessary to maintain entities

11

The aggregate pattern (3)

AGGREGATE

15/12/2012 Document databases in practice - Overview

Invariants are consistency rules that must be maintained whenever data changes

They’ll involve relationships within an aggregate(relations & foreign keys: order / orderlines)

Invariants applied within an aggregate will be enforced with the completion of each transaction

12

The aggregate pattern (4)

15/12/2012 Document databases in practice - Overview

Cluster entities and value objects into aggregates and define boundaries around each

Choose one entity to be the root of each aggregate and control all access to the objects inside the boundary through the root

Allow external objects to hold references to the root only

Transient references to internal members can be passed out for use within a single operation only

13

The aggregate pattern (5)

15/12/2012 Document databases in practice - Overview

Because the root controls access, it cannot be blindsided by changes to the internals

This arrangement makes it practical to enforce all invariants for objects in the aggregate and for the aggregate as a whole in any state change

14

Document design (1)

15/12/2012 Document databases in practice - Overview

Nested child document

Auction

15

Document design (2)

15/12/2012 Document databases in practice - Overview

Document referenced by ID

Auction AuctionBids

16

Document design (3)

Denormalized reference

15/12/2012 Document databases in practice - Overview

we clone properties that we care about when displaying or processing a containing document

avoids many cross document lookups and results in only the necessary data being transmitted over the network

it makes other scenarios more difficult: if we add frequently changing data, keeping details in synch could become very demanding on the server

use only for rarely changing data or for data that can be dereferenced by out-of-sync data

17

Document design (4)

15/12/2012 Document databases in practice - Overview

18

Document design (5)

Order contains denormalized datafrom Customerand Product

Full data aresaved elsewhere

15/12/2012 Document databases in practice - Overview

19

Document design (6)

DNT.LearnRaven.Aggregates

15/12/2012 Document databases in practice - Overview

Document databases in practice 20

Querying

15/12/2012

21

Getting started (1)

15/12/2012 Document databases in practice – Querying

DocumentStore• used to connect to a RavenDB data store• thread-safe• one instance per database per application

Session• used to perform operations on the database• not thread-safe• implements the Unit of Work pattern

in a single session, a single document (identified by its key) always resolves to the same instance

change tracking

22

Getting started (2)

15/12/2012 Document databases in practice – Querying

23

Document keygeneration options

15/12/2012 Document databases in practice – Querying

Sequential GUID key• when document key is not relevant (e.g. log entries)• entity Id = sequential GUID (sorts well for indexing)• Id property missing / not set ⇒ server generates a key

Identity key• entity Id = prefix + next available integer Id for it• Id property set to a prefix = value ending with slash• new DocumentStore ⇒ server sends a range of HiLo

keys

Assign a key yourself• for documents which already have native id (e.g.

users)

24

Simple queries

15/12/2012 Document databases in practice – Querying

25

Paging (1)

15/12/2012 Document databases in practice – Querying

soft-limit = 128no Take() replaced by Take(128)

hard-limit = 1024if x > 1024 Take(x) returns 1024 documents

26

Paging (2)

15/12/2012 Document databases in practice – Querying

RavenDB can skip over some results internally ⇒ TotalResults value invalidated

For proper paging use SkippedResults:Skip(currentPage * pageSize + SkippedResults)

Assuming a page size of 10…

27

Paging (3)

15/12/2012 Document databases in practice – Querying

28

More advanced queries

15/12/2012 Document databases in practice – Querying

29

Aggregation (1)

15/12/2012 Document databases in practice – Querying

RavenDB supports Count and Distinct

SelectMany, GroupBy and Join are not supported

The let keyword is not supported

For such operations an index is needed

30

Indexes

15/12/2012 Document databases in practice – Querying

All queries use an index to return results

Dynamic = created automatically by the server

Static = created explicitly by the user

31

Dynamic indexes

15/12/2012 Document databases in practice – Querying

no matching static index to query ⇒ RavenDB automatically creates a dynamic index on the fly (on first user query)

based on requests coming in, RavenDB can decide to promote a temporary index to a permanent one

32

Static indexes (1)

15/12/2012 Document databases in practice – Querying

permanent

expose much more functionality

low latency: on first run dynamic indexes have performance issues

map / reduce

33

Static indexes (2)

15/12/2012 Document databases in practice – Querying

34

Static indexes (3)

15/12/2012 Document databases in practice – Querying

35

Static indexes (4)

15/12/2012 Document databases in practice – Querying

Document databases in practice 36

Advanced topics

15/12/2012

37

Lucene.NET indexing (1)

15/12/2012 Document databases in practice – Advanced topics

an index is made of documents

document• atomic unit of indexing and searching• flat ⇒ recursion and joins must be denormalized• flexible schema• made of fields

38

Lucene.NET indexing (2)

15/12/2012 Document databases in practice – Advanced topics

field• a name-value pair with associated info• can be indexed if you're going to search on it

⇒ tokenization by analysis• can be stored in order to preserve original

untokenized value within document

example of physical index structure{“__document_id”: “docs/1”, “tag”: “NoSQL”}

39

Lucene.NET indexing (3)

DNT.LearnRaven.IndexAndAnalysis

15/12/2012 Document databases in practice - Overview

40

Index and polymorphism (1)

15/12/2012 Document databases in practice – Advanced topics

41

Index and polymorphism (2)

15/12/2012 Document databases in practice – Advanced topics

42

Includes (1)

15/12/2012 Document databases in practice – Advanced topics

One to one

43

Includes (2)

15/12/2012 Document databases in practice – Advanced topics

One to many ⇒ SELECT N+1

44

Includes (3)

15/12/2012 Document databases in practice – Advanced topics

Value type

45

Stale indexes (1)

15/12/2012 Document databases in practice – Advanced topics

indexing: thread executed on creation or update server responds quickly BUT you may query

stale indexes (better stale than offline)

46

Stale indexes (2)

15/12/2012 Document databases in practice – Advanced topics

47

Stale indexes (3)

documentStore.Conventions.DefaultQueryingConsistency

15/12/2012 Document databases in practice – Advanced topics

ConsistencyOptions.QueryYourWritessame behavior ofWaitForNonStaleResultsAsOfLastWrite

ConsistencyOptions.MonotonicReadyou never go back in time and read older data than what you have already seen

48

Optimistic concurrency

DNT.LearnRaven.OptimConcurrency

15/12/2012 Document databases in practice - Overview

49

Using RavenDB in an ASP.NET MVC application

DNT.LearnRaven.MVCScenario

15/12/2012 Document databases in practice - Overview

50

DNT.RavenQA (1)

Overview

Indexes

Queries

Shardingpolymorphism supportpolymorphism and document design

15/12/2012 Document databases in practice - Overview

Key-value databases in practice