harnessing the power of master/slave clusters to operate data-driven businesses on mysql
DESCRIPTION
Users seeking high availability, disaster recovery and zero downtime maintenance operation for business-critical MySQL applications face confusing choices. Is multi-master or master/slave clustering better? What about synchronous versus asynchronous replication? Using a plain vanilla, stock MySQL or a modified version of it? Which of these choices are right for data-driven businesses that depend on fast, reliable data access? This no-BS webinar cuts through the FUD to explore the real trade-offs between the different clustering and replication methods, then show you how Continuent Tungsten asynchronous master/slave clusters support these important capabilities for business-critical applications: - High application write rates - Mixed workloads consisting of large and small transactions - Data across multiple geographically distributed locations - Failures and more importantly recovery from them - Zero downtime maintenance and software upgrades - Use of off-the-shelf MySQL/MariaDB to avoid application changes and allow clusters to improve as MySQL itself does. We illustrate key points with demonstrations and case studies from deployed systems. Join us for an educational session on the cutting edge of MySQL data management!TRANSCRIPT
©Continuent 2014
Harnessing the Power of Master/Slave Clusters for Data-
Driven Business on MySQL
Robert Hodges, CEO Giuseppe Maxia, Director of QA
©Continuent 2014 2
Introducing Continuent
©Continuent 2014
Introducing Continuent
3
• The leading provider of clustering and replication for open source DBMS
• Our Product: Continuent Tungsten
• Clustering - Commercial-grade HA, performance scaling and data management for MySQL
• Replication - Flexible, high-performance data movement
©Continuent 2014
Quick Continuent Facts
• Largest Tungsten installation by data volume processes over 800 million transactions per day on 225 terabytes of relational data
• Largest installation by transaction volume handles up to 8 billion transactions daily
• Wide variety of topologies including MySQL, Oracle, Vertica, and Hadoop in production
• Cross-site topologies widely deployed
4
©Continuent 2014©Continuent 2014
Continuent Tungsten Customers
5
1
©Continuent 2014 6
Too Good to Be True
©Continuent 2014
The Dream: multiple, active DBMS servers with identical data
7
Server A
Server B
Server C
Transparent read/write
to any server
Updates propagated immediately
High Performance
High Availability
©Continuent 2014
Multi-Master Clusters in Action: Galera
8
Group Ordering and
Delivery
Server A
Server B
Server C
foo pk=1, v=5
foo pk=6, v=25
foo pk=1, v=6
[1] pk=1, v=6
[2] pk=1, v=5
[3] pk=6, v=25
[1] pk=1, v=6
[2] pk=1, v=5
[3] pk=6, v=25
[1] pk=1, v=6
[2] pk=1, v=5
[3] pk=6, v=25
©Continuent 2014
Synchronous Optimistic Locking
9
Group Ordering and
Delivery
Server A
Server B
Server C
foo pk=1, v=5
foo pk=6, v=25
foo pk=1, v=6
[1] pk=1, v=6
[2] pk=1, v=5
[3] pk=6, v=25
[1] pk=1, v=6
[2] pk=1, v=5
[3] pk=6, v=25
[1] pk=1, v=6
[2] pk=1, v=5
[3] pk=6, v=25
COMMIT
COMMITDEADLOCK
©Continuent 2014
Trouble in Paradise
• Transaction failures due to conflicts
• Operations like SELECT FOR UPDATE not supported
• Slow writes due to synchronous messaging
• Large transactions lock system/cause failures
• Unstable cross-site replication
10
©Continuent 2014
Can Master/Slave Offer Same Benefits?
11
Server A
Server B Server C
Updates propagated
immediately?
High Performance?
High Availability?
Transparent read/write
to any server?
©Continuent 2014 12
Tungsten Master/Slave Clusters
©Continuent 2014
Tungsten Database-as-a-Service
13
Tungsten clusters combine off-the-shelf open source DBMS servers into data services with: !
• 24x7 data access • Scaling of load on replicas • Simple management commands !...without app changes or data migration
Amazon US West
apache /php
GonzoPortal.com
Connector Connector
©Continuent 2014
Cluster Architecture
14
Application
Tungsten Connector
Application
Tungsten Connector
Replicator
Db2
Replicator
Db3Db1
Replicator
Slave SlaveMaster
Manager Manager Manager
Monitoring and Control
Monitoring and Control
Data Service: sjc
©Continuent 2014
Tungsten Connector Basics
• Any MySQL client can connect
• Initiates connections using JDBC
15
App Native protocol
!COM_QUERY
COM_INIT_DBCOM_DROP_DB
Connector
©Continuent 2014
Tungsten Connector Basics
• Pass-through operation after connection
• Full transparency & low overhead for clients
16
App
Protocol packet COM_QUERY
select * from t
ConnectorProtocol packet
OKResultSet Rows:1
©Continuent 2014
SmartScale Session Load Balancing
17
App
Connect/Write
Connector
Transaction History Log
Slave replay
position
Session x Session y
Slave DBMS
Master DBMS
Slave DBMS
Initial Write
©Continuent 2014
SmartScale Session Load Balancing
18
AppRead
Connector
Transaction History Log
Slave replay
position
Session x Session y
Slave DBMS
Master DBMS
Slave DBMS
Read from master
NO read from slave!
©Continuent 2014
SmartScale Session Load Balancing
19
AppRead
Connector
Transaction History Log
Slave replay
position
Session x
Slave DBMS
Master DBMS
Slave DBMS
Read from master
Read from slave
©Continuent 2014
Connectivity Example
20
Tungsten Connector Tungsten Connector
Db2 Db3Db1
SlaveMasterSlave
(SmartScale) (Strict Consistency)
©Continuent 2014 21
Demo: Transparent Connectivity to Replicas
©Continuent 2014 22
Transaction Scaling with Master/Slave Clusters
©Continuent 2014
Understanding Replication
23
Master
(Transactions + Metadata)
Slave
THL
DBMS Logs
Replicator
(Transactions + Metadata)
THLReplicator
Download transactions via network
Apply using JDBC
©Continuent 2014
Master Replication Service
24
Extract Filter Apply
StageExtract Filter Apply
Stage
Pipeline
MySQL Master
Transaction History Log
In-Memory Queue
Slave ReplicatorsBinlog
tcp/ip
©Continuent 2014
Slave Service with Parallel Apply
25
Extract Filter Apply
StageExtract Filter Apply
StageStage
Pipeline
Transaction History Log
Parallel Queue
Slave DBMS
Extract Filter ApplyExtract Filter ApplyExtract Filter Apply
Master Replicator
tcp/
ip
©Continuent 2014
Understanding Transaction Loads
26
Large TransactionsSlow
queriesReports
Db2 Db3Db1
SlaveMasterSlave
Small Transactions
©Continuent 2014 27
Demo: Scalable Transaction Processing
©Continuent 2014 28
Master/Slave Clustering Across Regions
©Continuent 2014
Primary/Disaster Recovery Clustering
29
App Server + Connector
App Server + Connector
Master Slave
San Jose NYC
©Continuent 2014
Further Cross-Site Topologies
30
Data Warehouse
All Masters
Fan-In
©Continuent 2014 31
Failover and Maintenance
©Continuent 2014
Automatic Failover
32
Application
Tungsten Connector
©Continuent 2014
Automatic Failover
33
Application
Tungsten Connector
1. Detect non-responsive master
2. Halt master connections
©Continuent 2014
Automatic Failover
34
Application
Tungsten Connector
3. Select most advanced slave and wait for pending transactions to apply
©Continuent 2014
Automatic Failover
35
Application
Tungsten Connector
4. Promote to master, reconfigure slave(s), and re-enable connections
©Continuent 2014
Automatic Failover
36
Application
Tungsten Connector
5. Recover failed master
©Continuent 2014
Rolling MySQL maintenance
• Task: change the innodb log file size
• Problem: it requires a database server restart
• Constraint: we do not want to restart
• Solution: rolling maintenance
• shun node2 (slave) - do maintenance - put node online
• shun node3 - do maintenance - put node online
• promote node2 to master
• shun node1 - do maintenance - put node online
• promote again node1 to master
37
©Continuent 2014
Rolling MySQL Maintenance
38
Final UpgradeSwitchSlave Upgrade Slave Upgrade
• Shun slave 1, run upgrade, and return to cluster. Discard and recover on failure
• Upgrade old master. Cluster schema is now fully upgraded
• Repeat for additional slave(s)
• Switch master to promote an upgraded slave
©Continuent 2014 39
Demo: Zero-Downtime Upgrade and Failover
©Continuent 2014 40
Wrap-Up
©Continuent 2014
Master/Slave Performs Very Well!
41
Server A
Server B Server C
Updates propagated
without cost to applications
Very High Performance
Very High Availability
Transparent connectivity with full SQL
semantics
©Continuent 2014
In Conclusion: Tungsten Offers...
• Transparent access to a highly available cluster of off-the-shelf MySQL replicas
• High performance transaction processing regardless of cluster size or distance between nodes
• Zero-downtime maintenance and upgrade of DBMS and applications
42
©Continuent 2014
www.continuent.com Follow us on Twitter @continuent
!
Tungsten Replicator: http://code.google.com/p/tungsten-replicator
Our Blogs: http://scale-out-blog.blogspot.com http://datacharmer.org/blog http://www.continuent.com/news/blogs http://flyingclusters.blogspot.com/
560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009 e-mail: [email protected]