Download - MySQL Cluster - voxxed days belgrade 2015
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
1/32
MySQL Cluster
tips & tricks
Bogdan KecmanMySQL Principal Technical [email protected]
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
2/32
Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purnot be incorporated into any contract. It is not a commitment to deliver any material, code, or functnot be relied upon in making purchasing decisions. The development, release, and timing of any feafunctionality described for Oracles products remains at the sole discretion of Oracle.
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
3/32
Cloud
Web & Enterprise OEM & ISVs
Industry Leaders Rely on MySQL
http://images.google.com/imgres?imgurl=http://www.infinitytechnology.com/uploadedImages/Home/TechFocus_News/AlcatelLucent_Hor_2col_lrg.gif&imgrefurl=http://www.infinitytechnology.com/phonesystems.aspx&h=251&w=720&sz=11&hl=en&start=4&sig2=urYvX47BOQRqvADY_UMA0w&um=1&tbnid=pA91nzVKrUFImM:&tbnh=49&tbnw=140&ei=94SgSOaRCaqkpATZu-jgBw&prev=/images?q=alcatel+lucent&um=1&hl=en&suggon=0&rls=com.microsoft:en-us&sa=Nhttp://images.google.com/imgres?imgurl=http://www.infinitytechnology.com/uploadedImages/Home/TechFocus_News/AlcatelLucent_Hor_2col_lrg.gif&imgrefurl=http://www.infinitytechnology.com/phonesystems.aspx&h=251&w=720&sz=11&hl=en&start=4&sig2=urYvX47BOQRqvADY_UMA0w&um=1&tbnid=pA91nzVKrUFImM:&tbnh=49&tbnw=140&ei=94SgSOaRCaqkpATZu-jgBw&prev=/images?q=alcatel+lucent&um=1&hl=en&suggon=0&rls=com.microsoft:en-us&sa=N -
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
4/32
MySQL Powers The Web
Over 50 million Tweets/day. 143,200 Tweets/sec in Aug 2013
Many petabytes of data. 11.2 Million Row changes & 2.5 birows read /sec handled in MySQL
6 billion hours of video watched each month
Globally-distributed database with 100 terabytes of user-reladata based on MySQL Cluster
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
5/32
Hosting IaaS, PaaS
SaaS
The #1 Database in the Cloud
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
6/32
MySQL Cluster Data Nodes InnoDB
Architectural overview
SE s1 SE s2MyISAM
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
7/32
Whos Using MySQL Cluster?
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
8/32
MySQL Cluster Architecture
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
9/32
MySQL Cluster NODES (1/2)
Data nodes (ndbd or ndbmtd) Stores data and indexes
In memory Non-indexed data can be stored on disk
Stores schema definition Check pointed to disk Transaction coordination Online backup All connect to each other
Management nodes (ndb_mgmd) Distributing configuration Logging Monitoring Act as Arbitrator to prevent split-brain scenarios Not crucial for Cluster operation (if not working cluster still works properly,
only limit is that no new nodes can start if no mgm nodes are around)
One is enough, two are perfect, three are too much
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
10/32
MySQL Cluster NODES (2/2)
SQL nodes (mysqld) The NDBCLUSTER Storage Engine is actually API node Transparent for most applications Used to create tables Used for geo-replication Can act as arbitrator Connects to all Data Nodes
API nodes (your own executable) Application written using NDB API
C C++ Java
FAST No SQL parsing No Optimizer
Examples
Ndbcluster storage engine, ndb_restore, memcached-ndbcluster plugin, l
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
11/32
High Availability
Fragmentation Table data is split among data nodes
Synchronous replication Each fragment is stored NoOfReplicas times
Heartbeating
Automatic failover
Online backups
Online Updates
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
12/32
High Performance
Performance boost In-memory Shared IO load As many SQL/API nodes as you like (up to 200) Direct access trough NDB API
Performance killers
Network (latency creates big problem) With 1gbit nic up to 10 data nodes works without problem With modern 10gbit nic up to 40 data nodes can run ok
Joins Huge improvements in 7.4 but still joins on distributed data a
going to suffer performance, especially on slower network Blobs
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
13/32
Scaling
Up to 254 nodes
Add data nodes on-line Note that adding data nodes does not only increase storage capa
but also increase your IO capacity too.
Geographical Replication
Multi channel replication (note it is always idempotent)
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
14/32
Accessing NDBCLUSTER data
SQL (via MySQL connector/php,java,ruby,python..., odbc, MySQL C-API.
MEMCACHED (add ndbcluster driver to memcached server)
ndbAPI (C/C++ API)
ClusterJ, JPA, ClusterJPA, LDAP
ClusterJMySQL
JDBC
Apps
JPA
JNIPython Ruby
ClusterJPA
Apps Apps Apps Apps Apps
Node.js
JSON
Apps
mod-ndb
Apache
Apps
ndb-eng
Memcached
Apps Apps
NDB API (C++)
MySQL Cluster Data Nodes
Apps
PHP PERL
Apps
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
15/32
Domain Object Model Persistence API
Java API
High performance, low latency
Feature rich
JPA interface built upon this new Java l
Java Persistence API compliant
Implemented as an OpenJPA plu
Uses ClusterJ where possible, revertoperations
Higher performance than JDBC
More natural for most Java designer
Easier Cluster adoption for web app
ClusterJ/JPA
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
16/32
Memcached is a distributed memory b
key/value store with no persistence to NoSQL, simple API, popular with develo
MySQL Cluster already provides scalab
performance with NoSQL (hashed) acce
persistence
Provide the Memcached API but map to N
Writes-in-place, so no need to invalidat Simplifies architecture as caching & dat
into 1 tier
Access data from existing relational tab
Memcached
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
17/32
Traditional Memcached Architecture
Memcache
httpd memcached
memcached
memcachedmemcache key
PHP/Perl
friends:12389
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
18/32
NDB & Memcache Architecture: Memcache protocol + NDB stora
Dmemcached
Memcache
Client
Application
D
NDB Engine
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
19/32
Memcached/MySQL Cluster latency
memcachetest -
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
20/32
Cluster & Memcached Configured Schema
prefix key value
key value
Prefix Table Key-col Val-col policy
town: map.zip town code cluster
Config tables
town ... code
maidenhead ... SL6
map.zip
Application view
SQL view
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
21/32
Node.js NoSQL API
Native JavaScript access to MySQL C
End-to-End JavaScript: browser to the a
Storing and retrieving JavaScript object
MySQL ClusterEliminate SQL transformation
Implemented as a module for node.j
Integrates full Cluster API library within
Couple high performance, distribute
performance distributed database
V8 JavaScript Engine
MySQL Cluster Node.js Module
MySQL Cluster Data Nodes
Clients
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
22/32
MySQL Cluster NoSQL API for Node.js
// Constructor
function Tweet(user, message) {this.id= UUID.generate();this.timestamp = Date.now();this.user = user;this.message = message;
}
// Server Startup
var nosql = require('mysql-js');
var sessionFactory =nosql.connectSync('ndb');
nosql.mapClass(Tweet, 'tweets');
Application Code
// Create a tweet
function newTweet(err, dbSession, httpvar tweet = new TweethttpReq.user,httpReq.message);
dbSession.persist(twe
}
function onNewTweetReqerr, httpReq){
essionFactory.openSenull, newTweet, ht
}
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
23/32
Need for Speed?
C++ programming interface
Provides direct access to data nodesNo MySQL server needed (still it is recommended to MySQL to manipulate schema)
No SQL layer (no parser, no optimizer, no )
Query batchingAsync transactions
NDB Events
NDB API
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
24/32
Cant all be that good?
Portability (all requests are hardcoded into your appl
Less flexibilityschema changes need to be hardcoded into your app
to change a simple query you have to change and recompile y
No privileges (everyone have access to everything)
No security (if you can telnet to a data node port youaccess to everything)
No triggers, views, stored procedures
No auditting
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
25/32
NDB API
1. Start transaction
2. Define operations3. Execute operations
4. Commit / Abort transaction
Life of a transaction
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
26/32
NDB API
A transaction is started by getting an NdbTransaction
An Ndb object can have maximum of 1024 parallel tr
NdbTransaction * t = ndb->startTransactiif (t == NULL){
printerr(could not start transaction\return (-1);}
Starting a Transaction
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
27/32
NDB API
An NdbOperation object is created with the getNdbO
A table name or a NdbDictionary::Table* needs to be
NdbOperation * op = t->getNdbOperation(
if (op == NULL){//handle error
}
Getting an NdbOperation Object
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
28/32
NDB API
insertTuple()
readTuple()writeTuple()
updateTuple()
deleteTuple()
op->readTuple();
Defining the operation type
NDB API EXAMPLE ( k )
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
29/32
NDB API EXAMPLE (pk access)NdbTransaction * trans = ndb->startTransaction();if (trans == NULL){ printerr(could not start transaction\n); retuNdbOperation * op = trans->getNdbOperation(City);if (op == NULL) return -1;op->readTuple();
int idvalue = 3236;op->equal(ID, idvalue);
int population = 0;op->getValue(Population, (char*)&population);
char name[35];op->getValue(Name, name);
if (trans->execute(NdbTransaction::Commit, NdbOperation::AbortOnErr== -1){
printerr(transaction was not successful\n);return (-1);
}trans->close();printf(The City %s has the population of %d\n, name, population)
NDB API
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
30/32
NDB API
Joining tables with NDB API is way more complex the
The basic principle is easy nested FOR loopsThe method for retrieving the rows depends on the tinvolved and possible indexes
It is very difficult to do dynamic optimization
In principle the join method has to be decided when the program (coding time)
Joining tables
SQL NDB API d i ( d / d t l ith
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
31/32
SQL vs NDB API speed comparison (reads/second note logarithm
-
7/23/2019 MySQL Cluster - voxxed days belgrade 2015
32/32
Thank You!
Questions?
Bogdan KecmanMySQL Principal Technical [email protected]