multi-data center mysql with continuent tungsten · 4 key reasons for multi-data center mysql •...

Peter Boros, Consultant, Percona Robert Hodges, CEO, Continuent

Percona Technical Webinars March 2, 2013

Multi-Data Center MySQL with Continuent Tungsten

www.percona.com 2 www.continuent.com

About This Presentation

•  Why is multi-data center MySQL important?

•  How does Continuent Tungsten fill the gaps?

•  Demonstration •  Practical steps to multi-data center

MySQL

Why is multi-data center MySQL important?

Robert Hodges


Key reasons for multi-data center MySQL

•  High availability – Moving data away from problems

•  Performance – Moving data closer to users

•  Legal – Keeping data within some locations but not others


HA Example – Amazon Failure

US-East-1 AZ AZ

AZ AZ

EU- West-1 AZ

AZ AZ US- West-1 AZ

AZ AZ AZ AZ AZ AZ AZ

Availability Zones Region

AZ AZ AZ AZ

EBS fails in a single availability zone



US-East-1 AZ AZ

AZ AZ

EU- West-1 AZ

AZ AZ US- West-1 AZ



AZ AZ AZ AZ

Leading to failures within the region due to coupled Amazon services



US-East-1 AZ AZ

AZ AZ

EU- West-1 AZ

AZ AZ US- West-1 AZ



AZ AZ AZ AZ

Leading to failure of tightly coupled applications in other regions


Requirements for successful multi-DC MySQL

•  Local HA within geographic regions •  Failover •  Planned maintenance and upgrade

•  Advanced multi-site replication •  Flexible topologies •  Tolerant of faults and WAN latency •  Transaction filtering

•  Minimal changes to apps / migration

How does Continuent Tungsten fill the gaps?

Robert Hodges


Tungsten picks up where MySQL leaves off

•  Handle local DBMS server failures •  Perform maintenance while apps are

running •  Load balance SQL across DBMS •  Create disaster recovery sites •  Enable multi-master replication •  Replicate transactions in real-time to

other DBMS types


Tungsten Cluster Architecture

Manager

db2

Manager

Replicator

db1

Manager

db3

Slave

Application Stack

Tungsten Connector

Application Stack

Tungsten Connector

Replicator Replicator

Slave Master

•  Off-the-shelf MySQL build

•  Async replicators

•  Peer-to-peer management

•  Network-based connectors


How connectivity works •  Transparent: Implements

MySQL wire protocol

•  Runs anywhere on network

•  Routes transactions to master by default

•  Optional read/write splitting

•  Reads load-balanced across all slaves Manager

db2

Manager

db1

Slave

Application Stack

Tungsten Connector

Master

TCP/IP


Graceful failover

Manager

db2

Manager

Replicator

db1

Manager

db3

Slave

Application Stack

Tungsten Connector

Application Stack

Tungsten Connector


Slave Master

•  Triggered by command

•  Zero-data loss

•  Transparent to applications

•  Automatic reconfiguration


Hard failover – MySQL crash

Manager

db2

Manager

db1

Manager

db3

Slave

Application Stack

Tungsten Connector

Application Stack

Tungsten Connector


Slave Master

•  Triggered by automatic rules

•  Within seconds for MySQL crash

•  Up to 1 minute after host crash

•  Admin required for recovery

Shunned


Hard failover – quorum loss

Manager

db2

Manager

db1

Manager

db3

Slave

Application Stack Application Stack

Tungsten Connector


Slave Master

Tungsten Connector •  Managers

maintain quorum

•  Connectivity stops when lost

•  Works across Amazon AZs

•  …But not regions

Shunned

Failed Failed

Shunned Shunned

X X X X


How replication works

Manager

•  Moves transactions between points

•  Can replicate between clusters

•  Can run multiple services

•  Global transaction IDs

•  Very flexible topologies

•  Transaction filtering Replicator

Replicator

Replicator

west

Replicator

Replicator

east DBMS

east

west


Cross-site multi-master with independent clusters

Replicator

Replicator

west

west Replicator

Replicator

east east

west east

Application Stack

Tungsten Connector Application Stack

Tungsten Connector

Application Stack


Tungsten Connector

Normal operation


Multi-master replicator failure

Replicator

Replicator

west

west Replicator

Replicator

east east

west east

Application Stack


Tungsten Connector

Application Stack


Tungsten Connector

Automatic rerouting after

replicator failure


Multi-master node failure

Replicator

Replicator

west

west Replicator

Replicator

east east

west east

Application Stack


Tungsten Connector

Application Stack


Tungsten Connector

Automatic failover following

node failure


Full site failure

Replicator

Replicator

west

west Replicator

Replicator

east east

west east

Application Stack


Tungsten Connector

Application Stack


Tungsten Connector

West site down for multiple days


Cross-site primary/DR with composite data service

Replicator

Replicator

west Replicator

Replicator

east

Application Stack


Tungsten Connector

Application Stack


Tungsten Connector

Normal operation

us


Composite service node failure

Replicator

Replicator

west Replicator

Replicator

east

Application Stack


Tungsten Connector

Application Stack


Tungsten Connector

Failover to slave node

us


Composite service site failure

Replicator

Replicator

west Replicator

Replicator

east

Application Stack


Tungsten Connector

Application Stack


Tungsten Connector

Manual failover to slave site

us

Demonstration

Peter Boros


Demo Architecture

Multi-Master Topology

EU-West-1

US-East-1 US-West-1

US-West-1 EU-West-1

Composite Data Service

World

Practical steps to multi-data center MySQL

Robert Hodges


Setting Up Multi-DC Clustering

•  Tungsten tpm – Sets up local clusters and replication links between them

1.  Put tungsten.ini file on host 2.  Download software (rpm or tarball) 3.  Run ‘tpm install’


Ini File for Composite Service [defaults]!home-directory=/opt/continuent!user=tungsten!replication-user=tungsten!replication-password=password!application-user=sbtest!application-password=sbtest!mysql-connectorj-path=/home/tungsten/mysql-connector-java-5.1.26-bin.jar!start-and-report=true![eu]!master=tw-eu1!slaves=tw-eu2,tw-eu3!connectors=tw-eu1,tw-eu2,tw-eu3![west]!master=tw-usw1!relay-enabled=true!relay-source=eu!slaves=tw-usw2,tw-usw3!connectors=tw-usw1,tw-usw2,tw-usw3![world]!composite-datasources=eu,west!


Composite Data Services – Pros and Cons

•  Key benefits: •  No conflicts •  No app changes •  Failover is transparent •  Works with statement replication

•  Things to consider and test: •  High latency writes to primary master •  Managing unplanned site failover •  Potential coupling between sites


Multi-Master Data Services – Pros and Cons

•  Key benefits •  Sites are completely uncoupled •  No failover! •  Minimal write latency

•  Things to consider and test carefully: •  Potential for conflicts •  App changes often required •  Row replication strongly recommended


Workloads that handle multi-master very well

•  Append-only data

•  Tables that use only auto-increment keys

•  Sharded apps where writes to each shard are in a particular region (system of record)


Where to get started

•  Cluster and replication documentation •  https://docs.continuent.com

•  Software •  https://code.google.com/p/tungsten-

replicator/ •  http://www.continuent.com/downloads/

software •  Percona and Continuent Websites


Peter Boros [email protected]

Robert Hodges

[email protected]

Thank You!

www.percona.com www.continuent.com

multi-data center mysql with continuent tungsten · 4 key reasons for multi-data center mysql •...

Documents