20141011 my sql clusterv01pptx

MySQL Cluster Driving high performance

Ivan Ma 11th October 2014

Open Source Tech Talk 2014.10

Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

MySQL Cluster Overview

• Auto-Sharding, Multi-Master

• ACID Compliant, OLTP + Real-Time Analytics HIGH SCALE, READS +

WRITES

• Shared nothing, no Single Point of Failure

• Self Healing + On-Line Operations 99.999% AVAILABILITY

• In-Memory Optimization + Disk-Data

• Predictable Low-Latency, Bounded Access Time REAL-TIME

• Key/Value + Complex, Relational Queries

• SQL + Memcached + JavaScript + Java + HTTP/REST & C++ SQL + NoSQL

• Open Source + Commercial Editions

• Commodity hardware + Management, Monitoring Tools LOW TCO

MySQL Cluster Architecture

MySQL Cluster Data Nodes

Clients

Application Layer

Data Layer

MySQL Cluster Scaling

Clients

Application Layer

Data Layer

MySQL Cluster HA

Clients

Application Layer

Data Layer

MySQL Cluster 7.3: 1.2 Billion UPDATEs per Minute

• NoSQL C++ API, flexaSynch benchmark

• 30 x Intel E5-2600 Intel Servers, 2 socket, 64GB

• ACID Transactions, with Synchronous Replication

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30

Millio

Scaling & Linear Scaling

• Partitioning & Sharding

• Push Down Query

• Explain Query

• Multi-threaded Data Node – ndbd vs ndbmtd

• SQL and NoSQL

Data Node 1

Data Node 2

Primary Fragment

Secondary Fragment

Data Node 3

Data Node 4 Fx

Table T1

Px Partition

4 Partitions * 2 Replicas = 8 Fragments

Automatic Data Partitioning Nodes & Node Groups

Data Node 1

Data Node 2

Primary Fragment

Secondary Fragment

Data Node 3

Data Node 4 Fx

Table T1

Px Partition

Data Node 1

Data Node 2

Primary Fragment

Secondary Fragment

Data Node 3

Data Node 4 Fx

Table T1

Px Partition

Data Node 1

Data Node 2

Primary Fragment

Secondary Fragment

Data Node 3

Data Node 4 Fx

Table T1

Px Partition

Data Node 1

Data Node 2

Primary Fragment

Secondary Fragment

Data Node 3

Data Node 4

Node Group 1

Node Group 2 Fx

Table T1

Px Partition

Scalability: Auto-Sharding

• Partitioning happens automatically & transparent to the application

• Transparency maintained during failover, upgrades and scale-out

• No need for application-layer sharding logic

• Transactional across shards (COMMIT transaction across shards/partitions)

http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster_perfomance.php

Adaptive Query Localization Scaling Distributed Joins

• Perform Complex Queries across Shards • JOINs pushed down to data nodes

• Executed in parallel

• Returns single result set to MySQL

• Opens Up New Use-Cases

• Real-time analytics

• Recommendations engines

• Fraud-detection

mysqld

Data Nodes

mysqld

Data Nodes

AQL – How to Use it

•Activated when ndb_join_pushdown is on (default)

•Run ANALYZE TABLE <tab-name> on each table once

•Use EXPLAIN to see what components are being pushed down: – Extra: Child of 'd' in pushed join@1

Data Node 1 Data Node 2

Need more throughput? Scale Out

Oops, need to increase capacity as well!

Scale Out

Simple access patterns

Compromise on consistency for performance

Ad-hoc data format

Simple operation

Complex queries with joins

ACID transactions

Well defined schemas

Rich set of tools

• No best single solution fits all • Mix and match

Still a role for SQL (RDBMS)?

Scalability

Performance

Ease of use

SQL/Joins

ACID Transactions

Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps Apps

Cluster JPA

PHP Perl Python Ruby JDBC Cluster J JS Apache Memcached

MySQL JNI Node.JS mod_ndb ndb_eng

NDB API (C++)

NoSQL Access to MySQL Cluster data

MySQL Cluster 7.4.1 DMR Available Now!

Better performance and operational simplicity

MySQL Cluster 7.4.1 DMR

• Performance gain over 7.3

– 47% (Read-Only)

– 38% (Read-Write)

• Faster maintenance operations

– Nodal & Rolling restarts

– Upgrades

• Asynchronous replication between MySQL Clusters

• Active-Active

– Update anywhere

– Conflict detection • Application notified through exception tables

• Can opt to have conflicts resolved automatically

– Auto-conflict-resolution • Conflicting transaction and dependent ones

are rolled-back

• No changes to application schema

Active-Active Geo-Replication

Restart Times

• Restarting data node with locally checkpointed data

– Major improvement

• Restarting data node which must recover data from peer

– Further speedups to come in 7.4.X (greater parallelization)

• Upgraded/rolling restarts

• Cluster shutdown and restart

– Minor improvement

What operations benefit?

Oracle MySQL HA & Scaling Solutions MySQL

Replication MySQL Fabric

Oracle VM Template

Oracle Clusterware

Solaris Cluster

Windows Cluster

DRBD MySQL Cluster

App Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔

Data Layer Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔

Platform Support All All Linux Linux Solaris Windows Linux All

Clustering Mode Master +

Slaves Master +

Slaves Active/Pas

sive Active/Passi

ve Active/P

assive Active/Pas

sive Active/P

assive Multi-Master

Failover Time N/A Secs Secs + Secs + Secs + Secs + Secs + < 1 Sec

Scale-out Reads ✔ ✖ ✖ ✖ ✖ ✖ ✔

Cross-shard operations N/A ✖ N/A N/A N/A N/A N/A ✔

Transparent routing ✖ For HA ✔ ✔ ✔ ✔ ✔ ✔

Shared Nothing ✔ ✔ ✖ ✖ ✖ ✖ ✔ ✔

Storage Engine InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ NDB

Single Vendor Support ✔ ✔ ✔ ✔ ✔ ✖ ✔ ✔

When to Consider MySQL Cluster

• Scalability demands

– Sharding for write performance?

• Latency demands

– Cost of each millisecond?

• Uptime requirements – Cost per minute of downtime?

– Failure versus maintenance?

• Application agility

– Developer languages and frameworks?

– SQL or NoSQL?

20141011 my sql clusterv01pptx

oracle andor

nosql copyright

oracle corporation

sole discretion of oracle

minute nosql c api

multimaster acid compliant

open source tech talk

intel servers

Technology

my sql lec2

php + my sql

20141011 mastering mysqlnd

my sql optimization

my sql crashcourse_intro_kdl

my sql 5.6_replwebinar_may12

my sql vivo_5.5_product_update_pt

my sql regis_handsonlab

20141011 moss9aoki

my sql susecon_crashcourse_2012

4081 my sql

20141011 echo camp 2014 "about openstreetmap"

my sql 101

php & my sql

my sql cluster_taipei_event

comparacion entre my sql y sql server

my sql proxy

my sql administration

conference unit 20141011 vf

私のsql －my sql