orientdb distributed architecture v2.0

Distributed DBMS

Copyright (c) - Orient Technologies LTD 2

Short history !

In 2012, we had a Master/Slave replication !

While it scaled up well on reads, users complained of a single Master node bottleneck

It’s quite easy to scale up reads, the hard part is to scale up both reads and writes

Copyright (c) - Orient Technologies LTD

How Master/Slave works

3

Master Node

CC C

Slave Node

Slave Node

Writes

Master node is the bottleneck

Copyright (c) - Orient Technologies LTD 4

Master/Slave !

PROS: - Relatively easy to develop !

CONS: - The master is the bottleneck for writes - No matter how many servers you have, the

throughput is limited by the Master node

Copyright (c) - Orient Technologies LTD 5

What happened to OrientDB's M/S architecture?

This is the old MASTER/SLAVE replication

Copyright (c) - Orient Technologies LTD 6

2012: new architectural goals

Multi-Master: all the nodes must accept writes

Sharding: split data in multiple partitions

Better Fail-Over

Simplified configuration with Auto-Discovery

Copyright (c) - Orient Technologies LTD 7

Master Node

Auto-Discovery

I’m the only one!

C

Copyright (c) - Orient Technologies LTD 8

Auto-Discovery

Master Node

Master Node

Connected!

C

Copyright (c) - Orient Technologies LTD 9

Master Node

Master Node

C

updated distributed configuration is broadcasted to

all the connected clients

Clients see the distributed configuration

Copyright (c) - Orient Technologies LTD 1010

Master Node

Master Node

CC

Master Node

Auto-reconnect in case of failure

In case of failure, the clients auto-reconnect to

the available nodes

Copyright (c) - Orient Technologies LTD 1111

Master Node

Master Node

C

DB are automatically deployed

to the new joining nodes

DB DB

Auto-deploy of databases

CC

C

Copyright (c) - Orient Technologies LTD 12

Customer

customer

Classes rely on Cluster to store records

By default 1 class -> 1 cluster Class

Cluster

Copyright (c) - Orient Technologies LTD 13

Classes can be split into more clusters

13

Customer

customer_usa

customer_europe

customer_china

Class

Cluster

ClusterCluster

Define multiple clusters

and assign them to each node

Copyright (c) - Orient Technologies LTD 14

Assign 1 cluster per Node

1414

customer_usa customer_europe customer_china

Master Node

Master Node

Master Node

Customer

Copyright (c) - Orient Technologies LTD

What about sharing + replication?

!

We used a solution similar to RAID for HardDrives

15

Copyright (c) - Orient Technologies LTD 16

RAID for databases

161616

customer_usa customer_europe customer_china

Master Node

Master Node

Master Node

Customer

customer_europecustomer_usacustomer_china

Replica factor = 2

Copyright (c) - Orient Technologies LTD 17

RAID for databases

17171717

customer_usa customer_europe customer_china

Master Node

Master Node

Master Node

Replica factor = 3

customer_usa customer_europecustomer_china

customer_usacustomer_europe customer_china

Each node owns all customers

Copyright (c) - Orient Technologies LTD 18

Replication: under the hood Client sends an INSERT request

1818181818

Master Node

Master Node

Master Node

HZ Queue

HZ Queue

HZ Queue

C

INSERT

Requests

Copyright (c) - Orient Technologies LTD 19191919191919

Master Node

Master Node

Master Node

HZ Queue

HZ Queue

HZ Queue

C

HZ Queue

HZ Queue

HZ Queue

Replication: under the hood Response handling

WriteQuorum = 2

Sends OK

OK

Requests

Responses

Copyright (c) - Orient Technologies LTD 20

Replication: under the hood Fix the unaligned node

20202020202020

Master Node

Master Node

Master Node

HZ Queue

HZ Queue

HZ Queue

HZ Queue

HZ Queue

HZ Queue

Requests

Responses

Fix

Copyright (c) - Orient Technologies LTD 21

Master Node

Master Node

CC

C

CC

C Master Node

CC

C

Master Node

CC

C

Master Node

CC

C

Master Node

CC

C

Master Node

C

C

Linear and Elastic scalability on both read & writes!

Copyright (c) - Orient Technologies LTD 22

Hazelcast’s role

Auto-Discovering (Multicast/TCP-IP/Amazon)

Queues for requests and responses

Store metadata in distributed Maps

Distributed Locks

2222

Copyright (c) - Orient Technologies LTD 23

OrientDB 2.0 (Sept 2014) has even better performance: +300% improvement on all the distributed operations

Pluggable conflict resolution strategy

Auto-discovery also by Clients

OrientDB’s Future Roadmap