mysql cluster nosql memcached api
DESCRIPTION
MySQL Cluster 7.2 added support for the Memcached API, enabling web services to directly query MySQL Cluster using the Memcached API, and adding a persistent, scalable, fault tolerant backend to Memcached. The slides take you through the design concepts and introduce a sample social media app built using memcached and MySQL ClusterTRANSCRIPT
MySQL Cluster: NoSQL Memcached API
• Meet New Requirements: Add persistence, sharding, fault tolerance
• Ease-of-Use: Standard Memcached libraries
• Simplification: Eliminate cache-coherency & consolidate data tiers
• Flexible Data Model: Schema / Schema-less storage
• Flexible Access Patterns: SQL and NoSQL
• Flexible Deployments: values stored in Cluster, Memcached or both
• Key / Value Performance: Native access to MySQL Cluster
Copyright © 2012 Oracle Corp.
<Insert Picture Here>
Program Agenda
• Overview, MySQL Cluster
• Memcached API
• Memcache Overview • MySQL Cluster (NDB) & Memcache setup • Sample Application
• Resources to Get Started
Copyright © 2012 Oracle Corp.
MySQL Cluster
• Auto-Sharding, Multi-Master • ACID Compliant, OLTP + Real-Time Analytics
HIGH SCALE, READS + WRITES
• Shared nothing, no Single Point of Failure • Self Healing + On-Line Operations
99.999% AVAILABILITY
• Open Source + Commercial Editions • Commodity hardware + Management, Monitoring Toos LOW TCO
• Key/Value + Complex, Relational Queries • SQL + Memcached + Java + JPA + HTTP/REST + C++ SQL + NoSQL
• High Load, Real Time Performance • Predictable Low-Latency, Bounded Access Time REAL-TIME
Copyright © 2012 Oracle Corp.
MySQL Cluster 7.2 70x Faster JOINs
Fastest Ramp…Ever
8x Higher Per Node Performance
writes
NoSQL Memcached API
Geo-Distributed Clusters
Memcached API
Copyright © 2012 Oracle Corp.
Copyright © 2012 Oracle Corp.
Memcached Overview
Memcache Client
Application memcached
memcached
memcached memcache key friends:12389!
hash key to find data
hash key to pick server
Copyright © 2012 Oracle Corp.
Memcache Protocol Elements
• Key Up to 250 characters • Expire Time A number in seconds, up to 30 days • Flags 32-bit number for application use • CAS ID 64-bit incrementing version number on value • Value Value stored for key; traditionally up to 1 MB.
Copyright © 2012 Oracle Corp.
Memcache Protocol Commands (1)
• get key Fetch value for this key • set key value Store this data • add key value Store this data (strict insert) • replace key value Store this data (strict update) • cas key value cas-id Update with version number check
Copyright © 2012 Oracle Corp.
Memcache Protocol Commands (2)
• append key value Append text to current value • prepend key value Insert text before current value • incr key amount Add to current stored integer value • decr key amount Subtract from current stored value • delete key Delete a key and value • flush_all Delete all stored values on the server • stats stat-name Fetch server statistics
Copyright © 2012 Oracle Corp.
Traditional architecture: Memcached & MySQL
MySQL Slave
more storage (on disk)!more powerful protocol (SQL)!
higher throughput !
lower latency!
mysql client
memcached Memcache
Client
Application
Copyright © 2012 Oracle Corp.
NDB & Memcache Architecture: Memcache protocol + NDB storage
MySQL Cluster
Data Node memcached
Memcache Client
Application
MySQL Cluster
Data Node
NDB Engine
Copyright © 2012 Oracle Corp.
Goals • Access stored data directly from memcache client • Memcached perspective:
• MySQL Cluster is a write-scalable, replicated data store • with reliable in-memory storage, • plus on-disk storage when data is too big for memory.
• MySQL Cluster perspective: • memcache is a high performance API • providing easy access to in-memory data, • plus an extra layer of caching when data is on disk.
Copyright © 2012 Oracle Corp.
Measured Latency memcachetest -t 2 -M 7000 -c 25000 !
Copyright © 2012 Oracle Corp.
Configuration Overview NDB Memcache
Copyright © 2012 Oracle Corp.
Anatomy of a Memcache key
user:1248
the prefix the database key
Copyright © 2012 Oracle Corp.
A key-prefix mapping
Memcache key
prefix Cluster Container Cache
Policy
Copyright © 2012 Oracle Corp.
key prefix Cluster Con-
tainer Cache Policy
Server Role
ID
A memcache server role
key prefix Cluster Con-
tainer Cache Policy
key prefix Cluster Con-
tainer Cache Policy
key prefix Cluster Con-
tainer Cache Policy
Copyright © 2012 Oracle Corp.
Getting Started MySQL Cluster and Memcached
Copyright © 2012 Oracle Corp.
“Cluster on a laptop”
• Configuration file cluster.ini • Management server ndb_mgmd • Data node ndbd • MySQL server mysqld • Memcached server memcached
(All of these are present in MySQL Cluster 7.2 distribution)
Cluster Configuration File
A single central file for the whole cluster Loaded by the management server on --initial And then stored in cache for future restarts Sections:
[MGM] [DB DEFAULT] [NDBD] [API]
Copyright © 2012 Oracle Corp.
[MGM] section: Management Server
[MGM]NodeId=1datadir=/Users/jdd/sandbox/ndbhostname=localhost
Copyright © 2012 Oracle Corp.
[DB DEFAULT] Section: data node parameters
[DB DEFAULT]NoOfReplicas=1TimeBetweenGlobalCheckpoints=12000HeartbeatIntervalDbApi=15000DataMemory=180MIndexMemory=60M
Copyright © 2012 Oracle Corp.
[NDBD] Sections: individual data node
[NDBD]NodeId=2
Copyright © 2012 Oracle Corp.
[API] Sections for MySQL & Memcached servers
[API][API][API][API][API][API][API]
Copyright © 2012 Oracle Corp.
Complete cluster.ini file [MGM]NodeId=1datadir=/Users/jdd/sandbox/ndbhostname=localhost[DB DEFAULT]NoOfReplicas=1TimeBetweenGlobalCheckpoints=12000HeartbeatIntervalDbApi=15000DataMemory=180MIndexMemory=60M[NDBD]NodeId=2[API][API] ...
Copyright © 2012 Oracle Corp.
Start the Management Server
% ndb_mgmd -f config-file-path
Copyright © 2012 Oracle Corp.
Start the Data Node
% ndbd
Copyright © 2012 Oracle Corp.
Configure & start mysqld (my.cnf file)
[mysqld]ndbcluster
Copyright © 2012 Oracle Corp.
Load NDB Memcache configuration tables
% cd /usr/local/mysql/share/memcache-api% mysql -u root < ndb_memcache_metadata.sql
Copyright © 2012 Oracle Corp.
Copyright © 2012 Oracle Corp.
Sample Application
Copyright © 2012 Oracle Corp.
"Twitter-like" Application
• Two Tables • authors • tweets
• Three Perl Functions • Create a new author record • Insert a new tweet • Increment an author's tweet count
Required Perl Modules
% cpan install Cache::Memcached% cpan install UUID::Tiny
Copyright © 2012 Oracle Corp.
Sample Application
% mysql -u rootmysql> CREATE DATABASE demo; use demo;
Copyright © 2012 Oracle Corp.
Data Definition: Authors
CREATE TABLE authors ( author varchar(15) not null, date_created timestamp, tweets bigint unsigned not null default 0, followers bigint unsigned not null default 0, following bigint unsigned not null default 0, PRIMARY KEY (author)) ENGINE=ndb;
Copyright © 2012 Oracle Corp.
Data Definition: Tweets
CREATE TABLE tweets ( id varchar(36) primary key, author varchar(15), time_stamp timestamp, tweet varchar(140), index(time_stamp), index(author))ENGINE=ndb;
Copyright © 2012 Oracle Corp.
Configuration: Use the ndbmemcache config schema
mysql> use ndbmemcache; show tables;
Copyright © 2012 Oracle Corp.
INSERT into containers SET name = 'tweets_table' , db_schema = 'demo' , db_table = 'tweets', key_columns = 'id' , value_columns = 'time_stamp,author,tweet';
INSERT into key_prefixesSET key_prefix = 'tweet:', policy = 'ndb-only', container = 'tweets_table';
Configuration: Create container & key_prefixes
Copyright © 2012 Oracle Corp.
Copyright © 2012 Oracle Corp.
Two key_prefixes for authors table
• newauthor: • ADD an author record and set date_created • tweetcount: • INCR tweet count for author
INSERT into containers SET name = 'author_new' , db_schema = 'demo' , db_table = 'authors',key_columns = 'author' ,
value_columns = 'date_created';
INSERT into key_prefixesSET key_prefix = 'newauthor:', policy = 'ndb-only', container = 'author_new';
Configuration: New Author
Copyright © 2012 Oracle Corp.
INSERT into containers SET name = 'author_count', db_schema = 'demo', db_table = 'authors', key_columns = 'author', increment_column = 'tweets';
INSERT into key_prefixesSET key_prefix = 'tweetcount:', policy = 'ndb-only', container = 'author_count';
Configuration: Tweet Count
Copyright © 2012 Oracle Corp.
% cd /usr/local/mysql% bin/memcached -E lib/ndb_engine.so
Memcached: Start the server
17-Aug-2012 13:53:32 PDT NDB Memcache 5.5.25-ndb-7.2.7 ...Contacting primary management server (localhost:1186) ... Connected to "localhost:1186" as node id 4.Retrieved 6 key prefixes for server role "default_role".Server started with 4 threads.Priming the pump ... done [0.661 sec].
Copyright © 2012 Oracle Corp.
use Cache::Memcached;use UUID::Tiny;our $mc = new Cache::Memcached( { 'servers' => [ "localhost:11211" ]});
Perl: Startup Code
Copyright © 2012 Oracle Corp.
Perl: Add a new author
## Takes one parameter: authorsub new_author { my $author = shift; my $time = time(); my $key = "newauthor:$author"; $mc->add($key, $time);}
Copyright © 2012 Oracle Corp.
## Take two parameters: author and tweetsub insert_tweet { my $author = shift; my $tweet = shift; my $id = create_UUID_as_string(UUID_V4); my $time = time(); # Insert the tweet my $key = "tweet:$id"; my $value = "$time" ."\t". $author ."\t". $tweet; $mc->add($key, $value);}
Perl: Insert a Tweet
Copyright © 2012 Oracle Corp.
## Increment the author's tweet count.## Take one parameter: authorsub increment_tweetcount { my $author = shift; my $key = "tweetcount:$author"; $mc->incr($key, 1);}
Perl: Increment Tweet Count
Copyright © 2012 Oracle Corp.
Download MySQL Cluster + Memcached API
http://www.mysql.com/downloads/cluster/#downloads
Most secure, scalable MySQL Database, Development / Management / Monitoring Tools, backed by Oracle Premier Lifetime Support
Oracle Premier Support
Oracle Product Certifications/Integrations
MySQL Enterprise Security
MySQL Enterprise Scalability
MySQL Enterprise Backup
MySQL Enterprise Monitor/Query Analyzer
MySQL Workbench
MySQL Cluster CGE
MySQL Cluster
Manager
• Straight from the Source • Largest Team of MySQL Experts • Direct Access to MySQL Support Engineers • Backed by MySQL Developers • Global 24/7/365, 29 languages • Forward Compatible Hot Fixes • MySQL Maintenance Releases • Unlimited Incidents • Knowledge Base • MySQL Consultative Support
Oracle Premier Support for MySQL Rely on The Experts - Get Unique Benefits
"The MySQL support service has been essential in helping us with troubleshooting and providing recommendations for the production cluster, Thanks." -- Carlos Morales – Playfulplay.com
Only From Oracle
Getting Started
Learn More
Get MySQL Cluster 7.2 Review Documentation
Scaling Web Databases Guide www.mysql.com/cluster/
Download Today http://www.mysql.com/downloads/cluster/
Copyright 2011 Oracle Corporation 50
https://dev.mysql.com/doc/refman/5.5/en/mysql-cluster-ndbmemcache.html
Summary
Scale Web Services with 99.999% Availability
Native Memcached NoSQL Access
Try it out Today!
Copyright 2011 Oracle Corporation 51
MySQL Cluster Architecture
Data Nodes
Node Group 1
F1
F3
F3
F1
Nod
e 1
Nod
e 2
Node Group 2
F2
F4
F4
F2
Nod
e 3
Nod
e 4
Application Nodes
Cluster Mgmt
Cluster Mgmt
REST JPA
MySQL Cluster - Extreme Resilience
Data Nodes
Node Group 1
F1
F3
F3
F1
Nod
e 1
Nod
e 2
Node Group 2
F2
F4
F4
F2
Nod
e 3
Nod
e 4
Application Nodes
Cluster Mgmt
Cluster Mgmt
REST JPA