©Continuent 2012.
Multi-Master, Multi-Site MySQL Databases Made Easy
with Continuent TungstenRobert Hodges, CEO
Giuseppe Maxia, Director of QA
©Continuent 2012
Introducing Continuent
2
• The leading provider of clustering and replication for open source DBMS
• Our Product: Continuent Tungsten
• Clustering - Commercial-grade HA, performance scaling and data management for MySQL
• Replication - Flexible, high-performance data movement
©Continuent 2012
Quick Continuent Facts
• Largest Tungsten clustering installation processes about 500M transactions each day
• Tungsten Replicator was application of the year at the 2011 MySQL User Conference
• Continuent is well known in the MySQL community for its clustering and replication
• Over 90 customers currently deployed
3
©Continuent 2012
Select Continuent Customers
4
©Continuent 2012 5
Why Tungsten?
©Continuent 2012
MySQL is a Great OLTP Database
6
But how do I...
• Handle DBMS failures?
• Perform maintenance and upgrades without taking applications o!ine?
• Load balance SQL across replicas?
• Create a disaster recovery site?
• Set up multi-master replication?
• Integrate with another DBMS?
©Continuent 2012 7
Tungsten Master/Slave Clustering
©Continuent 2012
Introducing Tungsten Clustering
8
Tungsten clusters combine o"-the-shelf open source DBMS servers into data services with:
• 24x7 data access • Scaling of load on replicas• Simple management commands
...without app changes or data migration
AmazonUS West
apache/php
GonzoPortal.com
Connector Connector
©Continuent 2012
Doesn’t Replication Do This Already?
9
Application
Libmysqlclient.a
Application
Libmysqlclient.a
Replicator
Db2
Replicator
Db3Db1
Replicator
Slave SlaveMaster
©Continuent 2012
Replication Limitations
Replication alone cannot enable constant access to data...
• What happens if the master fails?
• What happens if a slave is lagging too far behind to be usable?
• How do I maintain the DBMS without stopping apps?
10
©Continuent 2012
From Replicating to Clustering
11
Application
Tungsten Connector
Application
Tungsten Connector
Replicator
Db2
Replicator
Db3Db1
Replicator
Slave SlaveMaster
Manager Manager Manager
Monitoring and Control
Monitoring and Control
Data Service: sjc
©Continuent 2012
Basic Cluster Operations
12
• Switch -- Promote a slave node to master
• Shun -- Kick node out of cluster
• Welcome -- Return node to cluster
• Backup -- Run a backup
• Restore -- [Re-]provision a node
• Recover -- Return a broken node to slave
• Heartbeat -- Check cluster liveness
©Continuent 2012
Automatic Failover
13
Application
Tungsten Connector
©Continuent 2012
Automatic Failover
14
Application
Tungsten Connector
1. Detect non-responsivemaster
2. Halt master connections
©Continuent 2012
Automatic Failover
15
Application
Tungsten Connector
3. Select most advanced slave and wait for pending transactions to apply
©Continuent 2012
Automatic Failover
16
Application
Tungsten Connector
4. Promote to master, reconfigure slave(s), and re-enable connections
©Continuent 2012
Automatic Failover
17
Application
Tungsten Connector
5. Recover failed master
©Continuent 2012
Rolling SQL Schema Upgrade
18
Final UpgradeSwitchSlave Upgrade Slave Upgrade
• Shun slave 1, run upgrade, and return to cluster. Discard and recover on failure
• Upgrade old master. Cluster schema is now fully upgraded
• Repeat for additional slave(s)
• Switch master to promote an upgraded slave
©Continuent 2012
Backup, Recovery, and Restore
19
• Backups
/cluster > datasource DB1 backup
• Recover
/cluster > datasource DB2 recover
• Restore
/cluster > datasource DB2 restore
• Out-of-box support for mysqldump/XtraBackup; simple integration for others
©Continuent 2012 20
Managing Data Across Multiple Sites
©Continuent 2012
Topics:
• Tungsten Replicator basics
• Disaster recovery sites
• Multi-master topology
• Fan-in
21
©Continuent 2012
Tungsten Replicator Overview
22
Master
(Transactions + Metadata)
Slave
THL
DBMSLogs
Replicator
(Transactions + Metadata)
THLReplicator
Download transactions via network
Apply using JDBC
©Continuent 2012
Tungsten Replication Service
23
Extract Filter Apply
StageExtract Filter Apply
StageExtract Filter Apply
Stage
Pipeline
MasterDBMS
TransactionHistory Log
In-MemoryQueue
SlaveDBMS
©Continuent 2012
Parallel Replication of Sharded Data
24
Extract Filter Apply
StageExtract Filter Apply
StageStage
Pipeline
MasterDBMS
TransactionHistory Log
In-MemoryQueue
SlaveDBMS
Extract Filter ApplyExtract Filter ApplyExtract Filter Apply
©Continuent 2012
Multiple Services Per Replicator
25
NYCReplicator
London
Service nyc
FrankfurtReplicator
Service fra
Replicator
Service nyc
Service fra
©Continuent 2012
Multi-Master Replication
26
• Updates on 2+ sites (active-active mode)
• Enables geographic distribution of data
• No failover necessary if network fails or site becomes unavailable
• Not all applications can handle multi-master
• Applications must avoid con#icts
• Careful testing required
• Restoration of broken systems may not be easy
©Continuent 2012
Simple Multi-Master Con$guration
27
NYC FrankfurtReplicator
fra (master)
nyc (slave)
Replicator
fra (slave)
nyc (master)
Database-to-Database
©Continuent 2012
Clustered Multi-Master Con$guration
28
NYC Frankfurt
master
slave fra
fra
nyc
nyc
master
slave
©Continuent 2012
Clustered Multi-Master Con$guration
29
NYC Frankfurt
slave
master fra
fra
nyc
nyc
master
slave
NYC Master Offline
©Continuent 2012
Clustered Multi-Master Con$guration
30
NYC Frankfurt
master
slave fra
fra
nyc
nyc
master
slave
NYC Replicator Offline
©Continuent 2012
Complex Multi-Master Topologies
31
StarReplication
AllMasters
©Continuent 2012
Disaster Recovery Sites
32
• Updates go to primary site master
• DR site receives updates in real time
• Simple command promotion
• Tungsten connectors can route to primary and DR sites
• No application changes required
• Simplest way to protect against region/site failures
©Continuent 2012
Tungsten Composite Data Services
33
App Server+ Connector
App Server+ Connector
Master Slave
San Jose New York
sj_nyc
©Continuent 2012
Switching and failover
34
• Planned switch
/sj_nyc > switch to nyc
• Failover
/cluster > failover
• Failover between sites is not automated!
©Continuent 2012
System-of-Record
35
• Application data are “homed” in di"erent locations
• Each MySQL schema has a single master
• Within a single DBMS there can be both masters and slave schemas
• Minimal application changes required
• Allows you to move chunks of data closer to users
©Continuent 2012
System of Record Con$guration
36
NYC Frankfurt
Replicator
fra (master)
nyc (slave)
Replicator
fra (slave)
nyc (master)
acme
spitze
superb
acme
spitze
superb
Acme & Spitze mastered in NYC
Superb mastered in Frankfurt
©Continuent 2012
Implementing System of Record
37
• Con$gure shard $lters on installation to set policies for unknown/bad shards
• De$ne shard homes in shard.map $le
shard_id master criticalacme nyc falsespitze nyc falsesuperb fra false
• Load shard de$nitions
trepctl shard -‐insert < shard.map
• Move shard homes by editing reloading the shard map
©Continuent 2012
Fan-In Replication
38
• Transactions feed from multiple locations into a single slave
• Aggregate data for reporting
• Consolidate data from multiple servers into one server
• Transfer shards between non-connected clusters/locations
• Con#ict avoidance is up to applications
©Continuent 2012
Fan-In Topology
39
Replicator
Replicator
nyc (master)
fra (master)
sfo (master)
nyc (slave)
fra (slave)
sfo (slave)
Replicator
New York
Frankfurt
Replicator
San Francisco
London
©Continuent 2012
How Do I Obtain Tungsten?
40
• Register at Continuent to get downloads for Oracle replication: http://www.continuent.com/downloads/software
• Documentation is available on Continuent website
• MySQL replication is available from Tungsten Replicator project on code.google.comhttp://code.google.com/p/tungsten-replicator/
©Continuent 2012
One more thing:
WE’RE HIRING!!!
41
©Continuent 2012.
Continuent Web Page:http://www.continuent.com
Tungsten Replicator 2.0:http://code.google.com/p/tungsten-replicator
Our Blogs:http://scale-out-blog.blogspot.comhttp://datacharmer.org/bloghttp://www.continuent.com/news/blogs
560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009e-mail: [email protected]