Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Getting started with MySQL Cluster Hands-on Lab
Benedita Paúl Vasconcelos Principal Technical Support Engineer Oracle, MySQL Support October 29, 2015
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
3
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Aim of this Hands-on Lab
Brief Introduction to MySQL Cluster
Installation of MySQL Cluster
Start & Monitoring of MySQL Cluster
Connecting to MySQL Cluster
Safe Shutdown of MySQL Cluster
Overview of MySQL Cluster’s Admin Commands & Operations
MySQL Cluster CGE & Conclusions
1
2
3
4
5
6
7
8
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda with Highlight
Aim of this Hands-on Lab
Brief Introduction to MySQL Cluster
Installation of MySQL Cluster
Start & Monitoring of MySQL Cluster
Connecting to MySQL Cluster
Safe Shutdown of MySQL Cluster
Overview of MySQL Cluster’s Admin Commands & Operations
MySQL Cluster CGE & Conclusions
1
2
3
4
5
5
6
7
8
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Aim of this Hands-on Lab
• Objectives
1. Learn the basics of the MySQL Cluster Architecture
2. Learn the basics of MySQL Cluster Configuration and Administration
3. Learn how to start a new Cluster for evaluation purposes and how to connect to it
• Benefits
– Familiarize with MySQL Cluster
– Know where to go next to start (or continue) Cluster evaluation
6
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda with Highlight
Aim of this Hands-on Lab
Brief Introduction to MySQL Cluster
Installation of MySQL Cluster
Start & Monitoring of MySQL Cluster
Connecting to MySQL Cluster
Safe Shutdown of MySQL Cluster
Overview of MySQL Cluster’s Admin Commands & Operations
MySQL Cluster CGE & Conclusions
1
2
3
4
5
7
6
7
8
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Brief Introduction to MySQL Cluster
• Memory optimized tables with durability
• Predictable Low-Latency, Bounded Access Time REAL-TIME
• Auto-Sharding, Multi-Master
• ACID Compliant, OLTP + Real-Time Analytics HIGH SCALE, READS + WRITES
• Shared nothing, no Single Point of Failure
• Self Healing + On-Line Operations 99.999% AVAILABILITY
• Key/Value + Complex, Relational Queries
• SQL + Memcached + JavaScript + Java + HTTP/REST & C++ SQL + NoSQL
• Open Source + Commercial Editions
• Commodity hardware + Management, Monitoring Tools LOW TCO
MySQL Cluster Overview – WHAT
8
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Brief Introduction to MySQL Cluster MySQL Cluster GA – WHAT
9
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Brief Introduction to MySQL Cluster
Web
High volume OLTP
eCommerce
User Profile Management
Session Management & Caching
Content Management
On-Line Gaming
MySQL Cluster Deployments – WHERE
10
Telecoms
Subscriber Databases (HLR / HSS)
Service Delivery Platforms
VAS: VoIP, IPTV & VoD
Mobile Content Delivery
Mobile Payments
LTE Access
Case studies: www.mysql.com/customers/cluster/
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
• Scalability demands
– Sharding for write performance?
• Latency demands
– Cost of each millisecond?
• Uptime requirements – Cost per minute of downtime?
– Failure versus maintenance?
• Application agility
– Developer languages and frameworks?
– SQL or NoSQL?
• MySQL Cluster is designed for: – Short transactions
– Many parallel transactions
Brief Introduction to MySQL Cluster
11
MySQL Cluster Architecture – WHEN
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Brief Introduction to MySQL Cluster MySQL Cluster Architecture – HOW
12
MySQL Cluster Data Nodes
Clients
Application Layer
Data Layer Management
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Brief Introduction to MySQL Cluster MySQL Cluster Scaling – HOW
13
MySQL Cluster Data Nodes
Clients
Management
Application Layer
Data Layer
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Brief Introduction to MySQL Cluster MySQL Cluster HA – HOW
14
Clients
Management Data Layer
Application Layer
MySQL Cluster Data Nodes
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Brief Introduction to MySQL Cluster
15
MySQL Cluster Customers – WHO
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda with Highlight
Aim of this Hands-on Lab
Brief Introduction to MySQL Cluster
Installation of MySQL Cluster
Start & Monitoring of MySQL Cluster
Connecting to MySQL Cluster
Safe Shutdown of MySQL Cluster
Overview of MySQL Cluster’s Admin Commands & Operations
MySQL Cluster CGE & Conclusions
1
2
3
4
5
17
6
7
8
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Installation of MySQL Cluster
Open the HOL instructions and enter the VM:
Open the terminal:
• gedit HOL3348_Instructions.txt &
• ssh [email protected] (Password: ouser)
It’s your turn
18
Hands-on Lab
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 19
Generic Linux package
Installation of MySQL Cluster
Unpack of the package
Creation of the needed directories
Installation of SQL nodes
Creation of the configuration
files
1 2 3 4
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Installation of MySQL Cluster
Execute the commands at section #1 of the HOL instructions
It’s your turn
20
Hands-on Lab
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Section #1.1 of HOL instructions
Commands:
• cd $HOME/software/generic
• tar -xzfv mysql-cluster-gpl-7.4.7-linux-glibc2.5-i686.tar.gz
21
Unpack of the package
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Section #1.2 of HOL instructions
Commands:
• mkdir $HOME/cluster
• mkdir $HOME/cluster/7.4.7
• mkdir $HOME/cluster/7.4.7/conf $HOME/cluster/7.4.7/mysqld1_data
$HOME/cluster/7.4.7/mysqld2_data
$HOME/cluster/7.4.7/ndb_data
• ls -l $HOME/cluster/7.4.7/
22
Creation of the needed directories
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Section #1.3 of HOL instructions
Commands:
• cd $HOME/software/generic/mysql-cluster-gpl-7.4.7-linux-glibc2.5-i686
• scripts/mysql_install_db --no-defaults --datadir=$HOME/cluster/7.4.7/mysqld1_data/
• scripts/mysql_install_db --no-defaults --datadir=$HOME/cluster/7.4.7/mysqld2_data/
23
Installation of SQL nodes
Initialize MySQL Data Directory
Do not read any option files
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Section #1.4 of HOL instructions
Commands:
• cd $HOME/cluster/7.4.7/conf/
The files to be created are:
–my1.cnf
–my2.cnf
–config.ini
24
Creation of the configuration files
Specifies options for all MySQL Cluster executables
Contains the configuration of each node involved in the Cluster
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Section #1.4 of HOL instructions
[mysqld]
ndbcluster
datadir=/home/ouser/cluster/7.4.7/mysqld1_data
basedir=/home/ouser/software/generic/mysql-cluster-
gpl-7.4.7-linux-glibc2.5-i686
port=3307
log_error=/home/ouser/cluster/7.4.7/mysqld1_data/mys
qld.log
25
my1.cnf
Enables NDBCLUSTER storage engine
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Section #1.4 of HOL instructions
[ndb_mgmd]
hostname=localhost
PortNumber=1186
datadir=/home/ouser/cluster/7.4.7/ndb_data
NodeId=49
ArbitrationRank=1
[ndbd default]
NoOfReplicas=2
datadir=/home/ouser/cluster/7.4.7/ndb_data
[ndbd]
hostname=localhost
NodeId=1
[ndbd]
hostname=localhost
NodeId=2
[mysqld]
NodeId=50
[mysqld]
NodeId=51
26
config.ini
The [ndb_mgmd] sections are used to configure the behavior of each one of the Cluster’s Management server: • PortNumber: port number (default value) on which the management server listens for configuration requests and management commands • NodeID: each node in the cluster has an unique identity • ArbitrationRank: used to define which nodes can act as arbitrators (1 is the default value)
The [ndbd default] section is used to configure the behavior of all the Cluster's Data nodes: • NoOfReplicas: defines the number of replicas for each table stored in the cluster (2 is the default value)
The [ndbd] sections are used to configure the behavior of each one of the Cluster's Data nodes: • NodeID: each node in the cluster has an unique identity
The [mysqld] sections are used to define the behavior of the MySQL servers (SQL nodes): • NodeID: each node in the cluster has an unique identity
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda with Highlight
Aim of this Hands-on Lab
Brief Introduction to MySQL Cluster
Installation of MySQL Cluster
Start & Monitoring of MySQL Cluster
Connecting to MySQL Cluster
Safe Shutdown of MySQL Cluster
Overview of MySQL Cluster’s Admin Commands & Operations
MySQL Cluster CGE & Conclusions
1
2
3
4
5
27
6
7
8
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Start of MySQL Cluster
28
Ordered start of the nodes
Management node (ndb_mgmd)
Data nodes (ndbd/ndbmtd)
SQL nodes (mysqld)
1 2 3
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Start of MySQL Cluster
Execute the commands at section #2 of the HOL instructions
It’s your turn
29
Hands-on Lab
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Section #2.1 of HOL instructions
Command:
• ndb_mgmd -f $HOME/cluster/7.4.7/conf/config.ini --configdir=$HOME/cluster/7.4.7/conf/ --initial
-f : specify the cluster configuration file
--configdir : specify the cluster management server's configuration cache directory
--initial : causes the management server reload its configuration data from the configuration file, bypassing the configuration cache
Start of Management node
30
Suggestion: open now a new terminal to monitor the Cluster log: tail –f ndb_49_cluster.log
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Sections #2.2 & #2.3 of HOL instructions
Commands:
• ndbd -c localhost:1186 --initial --ndb-nodeid=1
• ndbd -c localhost:1186 --initial --ndb-nodeid=2
-c : set connection string for connecting to ndb_mgmd
--initial : perform initial start of the Data node, including cleaning the file system
--ndb-nodeid : set the respective Data node’s ID
31
Start of Data nodes (ndbd/ndbmtd)
Note: an --initial start is to be used only when starting the ndbd process under very special circumstances; this is because this option causes all files to be removed from the MySQL Cluster file system and all redo log files to be re-created.
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Sections #2.4 & #2.5 of HOL instructions
Commands:
• mysqld --defaults-file=$HOME/cluster/7.4.7/conf/my1.cnf &
• mysqld --defaults-file=$HOME/cluster/7.4.7/conf/my2.cnf &
32
Start of SQL nodes (mysqld)
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Monitoring of MySQL Cluster
•Cluster log: ndb_<mgm-nodeid>_cluster.log
•Data node’s logs: output ; error ; trace
Log files
•shell> ps aux | grep -i ndb
•shell> ps aux | grep -i mysql
Cluster processes
•SHOW
•ALL | <nodeid> STATUS
•ALL | <nodeid> REPORT MEMORY | BackupStatus | EventLog
ndb_mgm commands
•memoryusage
•nodes
•cluster_transactions
•restart_info
•disk_write_speed_aggregate
•etc
ndbinfo database
•MySQL Cluster Memory
•MySQL Cluster Log buffers and files
•MySQL Cluster cache
•MySQL Cluster node status
MySQL Enterprise Monitor (MEM)
33
Manual: ndbinfo
Manual: MEM
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Monitoring of MySQL Cluster processes
Execute the commands at section #3 of the HOL instructions
It’s your turn
34
Hands-on Lab
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Section #3 of HOL instructions
Are the Cluster processes running?
• ps aux | grep -i ndb
• ps aux | grep –i mysqld
Tail of the Management log:
• tail –f ndb_49_cluster.log
35
Monitoring of MySQL Cluster processses & logs
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda with Highlight
Aim of this Hands-on Lab
Brief Introduction to MySQL Cluster
Installation of MySQL Cluster
Start & Monitoring of MySQL Cluster
Connecting to MySQL Cluster
Safe Shutdown of MySQL Cluster
Overview of MySQL Cluster’s Admin Commands & Operations
MySQL Cluster CGE & Conclusions
1
2
3
4
5
36
6
7
8
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Connecting to MySQL Cluster
37
Administration + SQL & NoSQL access to Cluster data
Management client (administration)
MySQL client (SQL access)
Connectors & APIs (SQL or NoSQL access)
1 2 3
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Connecting to MySQL Cluster
Execute the commands at section #4 of the HOL instructions
It’s your turn
38
Hands-on Lab
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Section #4.1 of HOL instructions
Command in shell:
• ndb_mgm -c localhost:1186
Command in Management Client (ndb_mgm):
• SHOW
• EXIT
Management Client (administration)
39
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Section #4.2 of HOL instructions
Command in shell:
• mysql -uroot -P3307
Commands in MySQL Client (mysql):
• USE ndbinfo
• SHOW TABLES;
• SELECT * FROM memoryusage;
• SELECT * FROM nodes;
• exit
MySQL Client (SQL access)
40
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda with Highlight
Aim of this Hands-on Lab
Brief Introduction to MySQL Cluster
Installation of MySQL Cluster
Start & Monitoring of MySQL Cluster
Connecting to MySQL Cluster
Safe Shutdown of MySQL Cluster
Overview of MySQL Cluster’s Admin Commands & Operations
MySQL Cluster CGE & Conclusions
1
2
3
4
5
41
6
7
8
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Shutdown of MySQL Cluster
42
A two-step process
Use of the Cluster’s “shutdown” command
Shutdown (or not) all the SQL nodes
1 2
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster shutdown
• Cluster's "shutdown" command: ndb_mgm> shutdown
Node 1: Cluster shutdown initiated
Node 2: Cluster shutdown initiated
Node 2: Node shutdown completed.
Node 1: Node shutdown completed.
3 NDB Cluster node(s) have shutdown.
Disconnecting to allow management server to shutdown.
• Tail of the Management log: 2015-08-26 12:30:15 [MgmtSrvr] INFO -- Node 1: Cluster shutdown initiated
2015-08-26 12:30:15 [MgmtSrvr] INFO -- Node 2: Cluster shutdown initiated
2015-08-26 12:30:22 [MgmtSrvr] ALERT -- Node 49: Node 2 Disconnected
2015-08-26 12:30:22 [MgmtSrvr] INFO -- Node 2: Node shutdown completed.
2015-08-26 12:30:22 [MgmtSrvr] INFO -- Node 1: Node shutdown completed.
2015-08-26 12:30:22 [MgmtSrvr] ALERT -- Node 49: Node 1 Disconnected
2015-08-26 12:30:25 [MgmtSrvr] INFO -- Shutting down server...
2015-08-26 12:30:33 [MgmtSrvr] INFO -- Shutdown complete
43
Sample output
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda with Highlight
Aim of this Hands-on Lab
Brief Introduction to MySQL Cluster
Installation of MySQL Cluster
Start & Monitoring of MySQL Cluster
Connecting to MySQL Cluster
Safe Shutdown of MySQL Cluster
Overview of MySQL Cluster’s Admin Commands & Operations
MySQL Cluster CGE & Conclusions
1
2
3
4
5
44
6
7
8
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Overview of main MySQL Cluster’s Admin Commands & Operations
• Management Client's (ndb_mgm) Commands: HELP, SHOW, CREATE | DROP NODEGROUP
<nodeid> START | STOP | RESTART
START BACKUP, ENTER | EXIT SINGLE USER MODE
CLUSTER LOG ON | OFF | INFO | FILTER <severity_level>
<nodeid> CLUSTERLOG category=<threshold>
ALL | <nodeid> REPORT MEMORY | BackupStatus | EventLog
SHUTDOWN, QUIT | EXIT
• Cluster Operations:
– On-line Rolling Restart • On-line Scaling (add/remove nodes)
• On-line Reconfiguration
• On-line Upgrades
– On-line Backup & Restore
45
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Section #6 of HOL instructions
Command in shell:
• ndb_mgm -c localhost:1186
Command in Management Client (ndb_mgm):
• ALL REPORT MEMORY
• ALL REPORT BackupStatus
46
MySQL Cluster’s Admin Commands
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Section #6 of HOL instructions
Command in Management Client (ndb_mgm):
• 1 RESTART
• ALL STATUS Note: keep running ALL STATUS during the restart of node 1 so you may see the node passing through some different start phases.
47
MySQL Cluster’s Admin Commands
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Program Agenda with Highlight
Aim of this Hands-on Lab
Brief Introduction to MySQL Cluster
Installation of MySQL Cluster
Start & Monitoring of MySQL Cluster
Connecting to MySQL Cluster
Safe Shutdown of MySQL Cluster
Overview of MySQL Cluster’s Admin Commands & Operations
MySQL Cluster CGE & Conclusions
1
2
3
4
5
48
6
7
8
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster Carrier Grade Edition
49
More: www.mysql.com/buy-mysql
No Trade-Offs
On the Top
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster Manager (MCM)
50
Automated Management
• Start / Stop node or whole cluster
• On-Line Scaling
• On-Line Reconfiguration
• On-Line Upgrades
• On-Line Backup & Restore
• Import Running Cluster
Self-Healing
• Node monitoring
• Auto-recovery extended to SQL + mgmt nodes
HA Operations
• Cluster-wide configuration consistency
• Persistent configurations
• HA Agents
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Enterprise Monitor (MEM) Enterprise Monitor, Advisors & Query Analyzer
51
Your virtual MySQL DBA Assistant
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
MySQL Cluster Auto-Installer
• Fast configuration
• Auto-discovery
• Workload optimized
• Repeatable best practices
52
Specific Workload
Auto-Discover
Define Topology
Deploy
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. | 53
MySQL Cluster Auto-Installer
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Oracle Premier Support for MySQL
• Straight from the Source • Largest Team of MySQL Experts • Backed by MySQL Developers • Forward Compatible Hot Fixes • MySQL Maintenance Releases • MySQL Support in 29 Languages • 24/7/365 • Unlimited Incidents • Knowledge Base • MySQL Consultative Support
54
Rely on The Experts - Get Unique Benefits
Only From
Oracle
"The analyst quickly answered my questions, and went beyond what I expected, providing resources to get additional information and understanding.“
mysql.com/support/quotes
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Resources m
ysq
l.co
m TCO calculator
White papers
dev
.mys
ql.c
om
Community downloads
Documentation
Dev articles
Forums
PlanetMySQL
eDel
iver
y.co
m Oracle Software Delivery
Cloud
Download and evaluate all MySQL products
sup
po
rt.o
racl
e.co
m Knowledge Base
• Doc ID 1389140.1: Is MySQL Cluster the Right Choice Compared with InnoDB?
• Doc ID 1023119.1: How to Add New Nodes To an Existing MySQL Cluster Setup?
• Doc ID 1926680.1: Best Practices For MySQL Cluster
• Doc ID 2028705.1: How to Upgrade/Downgrade MySQL Cluster When Using MySQL Cluster Manager (MCM)
• Doc ID 1941472.1: How to Upgrade MySQL Cluster using RPM
Service request (SR)
Patches
Bugs/Defects
Etc
55
MySQL Support: www.mysql.com/support
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Summary
• Web-Scale Performance with Carrier-Grade Availability
• Scale-Out
• Real Time Performance
• 99.999% Uptime
• SQL & NoSQL Access Methods
56
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
58