couchconf tokyo tour of server and demo

1

Couchbase Server

Perry KrugSr. Solutions Architect

2

Couchbase Server

=• Simple• Fast• Elastic• NoSQL Database

Formerly known as Membase Server

3

Couchbase Server Features

• Built-in clustering – All nodes equal

• Data replication with auto-failover

• Zero- downtime maintenance

• Clone to grow and scale horizontally

4

Auto-sharding: vBuckets

5

Couchbase Server Basic Operation

COUCHBASE CLIENT LIBRARY

Docs distributed evenly across servers in the cluster

Each server stores both active & replica docs Only one server active at a time

Client library provides app with simple interface to database

Cluster map provides map to which server doc is on App never needs to know

App reads, writes, updates docs

Multiple App Servers can access same document at same time

Doc 4

Doc 2

Doc 5

SERVER 1

Doc 6

Doc 4

SERVER 2

Doc 7

Doc 1

SERVER 3

Doc 3

Read/Write/Update


Read/Write/Update

Doc 9

Doc 7

Doc 8 Doc 6

Doc 3

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

Doc 9

Doc 5

DOC

DOC

DOC

Doc 1

Doc 8 Doc 2

Replica Docs Replica Docs Replica Docs

Active Docs Active Docs Active Docs

CLUSTER MAP CLUSTER MAP

APP SERVER 1 APP SERVER 2

COUCHBASE SERVER CLUSTER

6

Add Nodes

Two servers added to cluster One-click operation

Docs automatically rebalanced across cluster Even distribution of

docs Minimum doc

movement Cluster map updated

App database calls now distributed over larger # of servers

Read/Write/Update Read/Write/Update

Doc 7

Doc 9

Doc 3

Active Docs

Replica Docs

Doc 6


CLUSTER MAP

APP SERVER 1


CLUSTER MAP

APP SERVER 2

Doc 4

Doc 2

Doc 5

SERVER 1

Doc 6

Doc 4

SERVER 2

Doc 7

Doc 1

SERVER 3

Doc 3

Doc 9

Doc 7

Doc 8 Doc 6

Doc 3

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

Doc 9

Doc 5

DOC

DOC

DOC

Doc 1

Doc 8 Doc 2



SERVER 4 SERVER 5

Active Docs Active Docs

Replica Docs Replica Docs


7

Fail Over Node

Doc 7

Doc 9

Doc 3

Active Docs

Replica Docs

Doc 6


CLUSTER MAP


CLUSTER MAP

Doc 4

Doc 2

Doc 5

SERVER 1

Doc 6

Doc 4

SERVER 2

Doc 7

Doc 1

SERVER 3

Doc 3

Doc 9

Doc 7 Doc 8

Doc 6

Doc 3

DOC

DOC

DOCDOC

DOC

DOC

DOC

DOC

DOC DOC

DOC

DOC

DOC

Doc 9

Doc 5DOC

DOC

DOC

Doc 1

Doc 8

Doc 2



SERVER 4 SERVER 5

Active Docs Active Docs

Replica Docs Replica Docs


DOC

DOC

App servers happily accessing docs on Server 3

Server fails App server requests to server 3 fail Cluster detects server has failed

Promotes replicas of docs to active Updates cluster map

App server requests for docs now go to appropriate server

Typically rebalance would follow

8

Couchbase Server Features

• Memcached compatible (built-in caching)

• Monitoring and administration APIs and GUI

• Reliable storage architecture

9

Couchbase Server 1.8 Architecture

Hea

rtbe

at

Proc

ess

mon

itor

Glo

bal s

ingl

eton

sup

ervi

sor

Confi

gura

tion

man

ager

on each node

Reba

lanc

e or

ches

trat

or

Nod

e he

alth

mon

itor

one per cluster

vBuc

ket s

tate

and

repl

icati

on m

anag

er

httpRE

ST m

anag

emen

t API

/Web

UI

HTTP8091

Erlang port mapper4369

Distributed Erlang21100 - 21199

Erlang/OTP

Cluster Manager

Persistence Layer

storage interface

Memcached

Couchbase EP Engine

11210Memcapable 2.0

Moxi

11211Memcapable 1.0

Data Manager

10


Persistence Layer

storage interface

Hea

rtbe

at

Proc

ess

mon

itor

Glo

bal s

ingl

eton

sup

ervi

sor

Confi

gura

tion

man

ager

on each node

Reba

lanc

e or

ches

trat

or

Nod

e he

alth

mon

itor

one per cluster

vBuc

ket s

tate

and

repl

icati

on m

anag

er

httpRE

ST m

anag

emen

t API

/Web

UI

Memcached

Couchbase EP Engine

11210Memcapable 2.0

Moxi

11211Memcapable 1.0

HTTP8091



Erlang/OTP

11


Hea

rtbe

at

Proc

ess

mon

itor

Confi

gura

tion

Man

ager

Glo

bal s

ingl

eton

sup

ervi

sor

on each node

Reba

lanc

e or

ches

trat

or

Nod

e he

alth

mon

itor

one per cluster

vBuc

ket s

tate

and

repl

icati

on m

anag

er

httpRE

ST M

anag

emen

t /W

ebU

I

HTTP8091



Erlang/OTP

Persistence Layer

storage interface

Memcached

Couchbase EP Engine

11210Memcapable 2.0

Moxi

11211Memcapable 1.0

12

Couchbase SDKs

Java SDK

.Net SDK

PHP SDK

Ruby SDK

…and many more

Java client API

User Code

Couchbase Server

CouchbaseClient cb = new CouchbaseClient(listURIs,"aBucket", "letmein");// this is all the same as beforecb.set("hello", 0, "world");cb.get("hello");

http://www.couchbase.com/develop

Couchbase Java Library (spymemcached)

13

DEMO TIME

14

Couchbase Server 2.0

• Next major release of Couchbase Server• Currently in Developer Preview

What’s new:• Indexing and Querying• Incremental Map Reduce• Cross Data Center Replication

• Fully backwards compatible with existing Couchbase Server

15


Hea

rtbe

at

Proc

ess

mon

itor

Glo

bal s

ingl

eton

sup

ervi

sor

Confi

gura

tion

man

ager

on each node

Reba

lanc

e or

ches

trat

or

Nod

e he

alth

mon

itor

one per cluster

vBuc

ket s

tate

and

repl

icati

on m

anag

er

httpRE

ST m

anag

emen

t API

/Web

UI

HTTP8091



Erlang/OTP

storage interface

Couchbase EP Engine

11210Memcapable 2.0

Moxi

11211Memcapable 1.0

Memcached

New Persistence Layer

8092Query API

Que

ry E

ngin

e

16

Indexing and Querying

APP SERVER 1

COUCHBASE CLIENT LIBRARY Indexing work is distributed amongst nodes Large data set possible Parallelize the effort

Each node has index for data stored on it

Queries combine the results from required nodes

CLUSTER MAP

Doc 4

Doc 2

Doc 5

SERVER 1

Doc 6

Doc 4

SERVER 2

Doc 7

Doc 1

SERVER 3

Doc 3

APP SERVER 2


CLUSTER MAP

Doc 9

Doc 7

Doc 8 Doc 6

Doc 3

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

DOC

Doc 9

Doc 5

DOC

DOC

DOC

Doc 1

Doc 8 Doc 2



COUCHBASE CLIENT LIBRARY COUCHBASE CLIENT LIBRARY

CLUSTER MAP CLUSTER MAP

APP SERVER 1 APP SERVER 2

QueryResponse

17

Cross Data Center Replication

Want data close to user Want multiple locations for disaster recovery

Multi-Master: Can write to same document in all different regions & it will sync (eventually consistent, always available)

US DATA CENTER

EUROPE DATA CENTER

ASIA DATA CENTERReplication Replication

Replication

18

DEMO TIME

19

Demo: The next big social game

3 Objects (documents) within game:• Players•Monsters• Items

Gameplay:• Players fight monsters•Monsters drop items• Players own items

20

Player Document

{"_id": "Keith4540","jsonType": "player","uuid": "35767d02-a958-4b83-8179-616816692de1","name": "Keith4540","hitpoints": 75,"experience": 663,"level": 4,"loggedIn": false

}

21

Item Document

{"_id": "Katana_e5890c94-11c6-48-65746ce6c560","jsonType": "item","name": "Katana_e5890c94-11c6-65746ce6c560","uuid": "e5890c94-11c6-4856-a7a6-65746ce6c560","ownerId": "Dale9887"

}

Player “_id”

22

Monster Document

{"_id": "Bauchan9932”,"jsonType": "monster","name": "Bauchan9932","uuid": "d10dfc1b-0412-4140-b4ec-affdbf2aa5ec","hitpoints": 370,"experienceWhenKilled": 52,"itemProbability": 0.5050581341872865

}

23

GAME ON!

24

QUESTIONS?

25

THANK YOU!

Get Couchbase Server 2.0 at http://www.couchbase.com/downloads

Give us feedback at:http://www.couchbase.com/forums

couchconf tokyo tour of server and demo

Technology