techtalk #14 grokking: couchbase - nosql + memcached + real-time + offline!

ScalabilityofNoSQL.FlexibilityofJSON.PowerofSQL.

VuNguyen

WhatisCouchbase?

Couchbase Server CouchbaseMobile

• DocumentDatabase

• Key/ValueStore• DistributedCache

• EmbeddedDatabase

• Offline&Real-timesync

DatainCouchbase

• Key:“DocumentID”

• Anystringupto250bytes• Uniqueinsideabucket

• Value• Anyvalueupto20MB

• Anything:JSON,XML,serializedobject,string,etc.

• Metadata

• CAS(check-and-set)• TTL(time-to-live)

• Optionalflags

Architecture

• NodeACouchbase instance.

Architecture


• ClusterAscalable,networkedsetofnodes.

Architecture


• ClusterAscalable,networkedsetofnodes.

• BucketAlogicalcollectionofdocumentswithuniquekeys.

Bucketsareevenlydistributedacrossacluster.

[key1:value]

[key2:value]

[key3:value]

[key4:value]

[key5:value]

[key6:value]

Architecture

• NodeACouchbase instance

• ClusterAscalable,networkedsetofnodes

• BucketAlogicalcollectionofdocumentswithuniquekeys.

Bucketsareevenlydistributedacrossacluster.

• Document

Akey/valuepairwithinabucket.Keysareunique

withinabucket.

[key1:value]

[key2:value]

[key1:value]

[key2:value]

[key3:value]

[key4:value]

[key3:value]

[key4:value]

[key5:value]

[key6:value]

[key5:value]

[key6:value]

Howtoaccessdata?

1. Read/writedocumentsbytheirkeys

• Extremelyfastduetocache

2. MapReduceviews

• Distributedsecondaryindexes,builtusingmap-reduce

3. N1QL(“nickel”)query

• SQLsupersetforindexingandqueryingJSONstructure4. Fulltextsearch

• Couchbase FTS

Single-NodeArchitecture

ClusterManager

DataService

IndexService

QueryService

ManagedCache

Storage

Couchbase Servernodesareidentical.

Twocorecomponents

- ClusterManager

- NodeManagers

Threeindependentservices

- DataService

- IndexService

- QueryService

ClusterArchitecture

ClusterManager

DataService

IndexService

QueryService

ManagedCache

Storage

ClusterManager

DataService

IndexService

QueryService

ManagedCache

Storage

ClusterManager

DataService

IndexService

QueryService

ManagedCache

Storage

ClusterManager

DataService

IndexService

QueryService

ManagedCache

Storage

Cluster

Howdoesaget operate?

DataService

ManagedCache

Storage

doc2 doc3

doc1 doc2 doc3 doc4

doc2 doc4

Replication

queue

Persistence

queue

Getisextremelyfast:

- Commondocs

arecached

- BinaryTCPconnection

Twoconfigs:

- Value-onlyejection

- Fullmeta-dataejection

Howdoesaset operate?

DataService

ManagedCache

Storage

doc4

doc1 doc2 doc3 doc4

doc4

Replication

queue

Persistence

queue

Acknowledgetype:

- WheninRAM(default)

- Wheninstorage

- Whenreplicated

Replication

- Veryfast,RAMtoRAM

WhyCouchbase serverfast?

• Getandsetareservedprimarilyfromcache

• Diskwritesareappendonly

doc1 doc2 doc3




doc1 doc2 doc3 doc1




doc1 doc2 doc3 doc1 doc3



• Diskwritesareappendonly• Datafilesareperiodicallycompressed

doc2 doc1 doc3doc1 doc3



• Diskwritesareappendonly• Datafilesareperiodicallycompressed

doc2 doc1 doc3

MapReduce

Viewsaresecondaryindexes

• WritteninJavaScript

• ExecutedbyV8engine

function (doc, meta) {if (doc.age > 30) {

emit(doc.department, null)}

}

N1QLQueries

SQLforJSONdata

• SELECT, INSERT, UPDATE, DELETE

• JOIN, WHERE, GROUP BY

• CREATE INDEX, DROP INDEX

• MIN, MAX, COUNT

• UNION, INTERSECT, EXCEPT, ...

SELECT count(*), city FROM customer

WHERE customer.type = “vip”

GROUP BY customer.city ORDER BY customer.city

LIMIT 5 OFFSET 5

Bucket&vBucket

• Eachbucketisdividedinto1024vBucket.• vBuckets aredistributedevenlyacrossallnodesinthecluster.

• Asnodesjoin/leavecluster,vBuckets adjustautomatically.

Active

vBuckets

Replicated

vBuckets

Active

vBuckets

Replicated

vBuckets

Active

vBuckets

Replicated

vBuckets

CouchbaseMobile

• EmbeddeddatabaseforiOS&Android

• UnofficialJavaScriptdatabase(browser&NodeJs)

• Real-timesynchronization

• CouchDB replicationprotocol

Architecture

CouchDB protocol

GET/source/_changes?feed=normal&style=all_docs&heartbeat=10000{

"results":[{"seq":14,"id":"f957f41e","changes":[{"rev":"3-46a3"}],"deleted":true},{"seq":29,"id":"ddf339dd","changes":[{"rev":"10-304b"}],

],"last_req": 78,

}

GET/source/_changes?feed=continuous&style=all_docs&heartbeat=10000

{"seq":14,"id":"f957f41e","changes":[{"rev":"3-46a3"}],"deleted":true}{"seq":29,"id":"ddf339dd","changes":[{"rev":"10-304b"}]

SyncFunctionfunction (doc, oldDoc) {

if (doc._deleted) {requireRole("role:editor");requireUser(oldDoc.writers);return;

}

if (!doc.title || !doc.creator || !doc.channels || !doc.writers) {throw({forbidden: "Missing required properties"});

} else if (doc.writers.length == 0) {throw({forbidden: "No writers"});

}if (oldDoc == null) {

requireRole("role:editor");requireUser(doc.creator)

} else {requireUser(oldDoc.writers);if (doc.creator != oldDoc.creator) {

throw({forbidden: "Can't change creator"});}

}

channel(doc.channels);}

ScalabilityofNoSQL.FlexibilityofJSON.PowerofSQL.

VuNguyen

techtalk #14 grokking: couchbase - nosql + memcached + real-time + offline!

Technology