©Continuent 2013.

How Gittigidiyor (a subsidiary of eBay) Replicates Data In Real Time

From MySQL To OracleLevent Kurt, Site Operations Manager, Gittigidiyor

Giuseppe Maxia, Director of Quality Assurance, Continuent Linas Virbalas, Senior Software Engineer, Continuent

©Continuent 2013

Introducing Continuent

���2

• The leading provider of clustering and replication for open source DBMS

• Our Product: Continuent Tungsten

• Clustering - Commercial-grade HA, performance scaling and data management for MySQL

• Replication - Flexible, high-performance data movement

©Continuent 2013

A Review of Tungsten Replicator

���3

©Continuent 2013

Tungsten Replicator Overview

���4

Master

(Transactions + Metadata)

Slave

THL

DBMS Logs

Replicator

(Transactions + Metadata)

THLReplicator

Download transactions via network

Apply using JDBC

©Continuent 2013

Master Replication Service

���5

Extract Filter Apply

StageExtract Filter Apply

Stage

Pipeline

MySQL Master

Transaction History Log

In-Memory Queue

Slave ReplicatorsBinlog

tcp/ip

©Continuent 2013

Slave Replication Service

���6

Extract Filter Apply

StageExtract Filter Apply

StageExtract Filter Apply

Stage

Pipeline

Transaction History Log

In-Memory Queue

Slave DBMS

Master Replicator

tcp/

ip

©Continuent 2013

star

master-slave Heterogeneous

fan-in slave all-masters

MySQL

Oracle

Oracle

MySQL Oracle

Oracle

MySQL MySQL

���7

©Continuent 2013

Heterogeneous Replication

©Continuent 2013

Steps to Heterogeneous Replication

1. Prepare (translate) schema for the slave DBMS

2. Set up replication

3. Provision initial data

4. Continue real-time replication

ddlscan

tpm

Parallel ApplyMySQL Blackhole

Tungsten Replicator

©Continuent 2013

Replicating from MySQL to Oracle

©Continuent 2013

Use Case: Real-Time Sales/Order Fulfillment

Real-Time Replication

Web-based Sales Purchase Ordering

3M Order Items

~1M Xacts/ Day

©Continuent 2013

1. Translating schema for the slave

©Continuent 2013

Translating Schema

���13

• Goal - set up heterogeneous replication

Replication

©Continuent 2013

Translating Schema

���14

• Beginning - how to convert tables?

empty

Tables

©Continuent 2013

Translating Schema

���15

• Beginning - how to convert tables?

empty

Tables

•Data types? •Column lengths? •Naming conventions? •Reserved words?

©Continuent 2013

Translating Schema

���16

• ddlscan automates these conversions

empty

Tables

ddlscan

©Continuent 2013

ddlscan

���17

• Part of Tungsten Replicator, GPL v2

• Translates schema with replication in mind

• Provides errors and warnings

• Can rename schema/tables/columns

©Continuent 2013

Usage (MySQL to Oracle Example)

���18

$ cd tungsten-replicator/bin !

$ ./ddlscan \ -db test \ -template ddl-mysql-oracle.vm \ -user tungsten \ -pass secret

©Continuent 2013

Translating Schema

���19

• ddlscan looks into source schema

empty

Tables

©Continuent 2013

Translating Schema

���20

• ddlscan translates and renders DDL commands

empty

Tables

oracle-ddl.sql

©Continuent 2013

Result of ddlscan

���21

CREATE TABLE 3colors /* ERROR: table starts with a number, use rename option and a filter */

( id NUMBER(10, 0), color VARCHAR2(1) /* ENUM('R','G','B') */, enabled NUMBER(1) /* BIT(1) - no constraints for 0 or 1 */, acolumnthatdoesntfittooracleat NUMBER(10, 0)

/* WARN: truncated column name exceeding 30 characters (acolumnthatdoesntfittooracleatall) */

! /* ERROR: table must have a primary key! */ ); !CREATE TABLE talks ...

©Continuent 2013

Translating Schema

���22

• You run resulting SQL file on Oracle

empty

Tables

oracle-ddl.sql

©Continuent 2013

Translating Schema

���23

• Tables are ready!

Tables Translated tables (empty)

©Continuent 2013

2. Set Up Replication

©Continuent 2013

Replicator Installation

���25

./tools/tpm configure my_ora \ --enable-heterogenous-service=true \ --install-directory=/opt/mysql_to_ora/continuent \ --members=alpha,bravo \ --master=alpha !./tools/tpm configure my_ora --hosts=alpha \ --replication-user=tungsten \ --replication-password=secret !./tools/tpm configure my_ora --hosts=bravo \ --replication-user=tungsten_frommysql \ --replication-password=secret \ --datasource-type=oracle \ --datasource-oracle-service=ORCL \ --svc-applier-filters=dropstatementdata !./tools/tpm install my_ora !./tools/tpm start my_ora

!

©Continuent 2013

Replication Under the Hood

���26

MySQL Binlog

binlog_format=row

Service frommysql

Tungsten Master Replicator

MySQLExtractor Special Filters •Transform

ENUM to string •Transform SET

to string

Service frommysql

Tungsten Slave Replicator

Special Filters •Drop DDL •Map names to upper case •Rename too long or reserved

objects •Optimize updates to remove

unchanged columns •Ignore extra tables

OracleApplierMySQLExtractor

©Continuent 2013

3. Provisioning an Oracle slave with

auditing trail

©Continuent 2013

The Requirements

• Export data from MySQL master to the Oracle slave before starting replication

• The Oracle slave should also take trace of the changes

• Convert the MySQL tables to stream of raw data in THL format

• Data is about 400 GB

���28

©Continuent 2013

The Practical Steps

1. Export data from master

2. Keep track of replication position

3. Start the replicator from the sandbox

4. Load data into slave with parallel apply

5. Switch replicator to real master

6. Dismiss the sandbox

7. Replication from MySQL to Oracle live

���29

©Continuent 2013

Provisioning (1)

���30

slave

Oracle

master

MySQL

table data

table structuretable data

table structuretable data

table structure

©Continuent 2013

table structure

table data

table structure

Provisioning (2)

���31

slave

Oracle

master

MySQL

table data

table structuretable data

table structure table structuretable structureddlscan

©Continuent 2013

table structure

Provisioning (3)

���32

slave

Oracle

master

MySQL

table structuretable structure

MySQL sandbox

CONVERT to BLACKHOLE

table data

table structuretable data

table structuretable data

table structure

©Continuent 2013

Provisioning (4)

���33

slave

Oracle

master

MySQL

MySQL sandbox

table data

table structuretable data

table structuretable data

table structure

not replicating

backup

©Continuent 2013

Provisioning (5)

���34

slave

OracleMySQL sandbox

backup

bin logs

tungsten replicator THL

tungsten replicator

(ROW)

©Continuent 2013

How Parallel Apply Works

ExtractFilter

(Shard)Apply

Extract/Filter/Apply

Stage Stage StagePipeline

Master DBMS

Slave DBMS

Transaction History Log

Parallel Queue

Extract/Filter/ApplyExtract/Filter/Apply

Extract/Filter/Apply

ProvisioningMutli-tenant source

©Continuent 2013

4. Continue Real-Time Replication

©Continuent 2013

Provisioning (6)

���37

slave

Oracle

master

MySQL

MySQL sandbox

table data

table structuretable data

table structuretable data

table structure

tungsten replicator

bin logs

THLtungsten replicator

©Continuent 2013

Gittigidiyor (subsidiary of eBay)

architecture and needs

©Continuent 2013 1

• Gittigidiyor • A subsidiary of eBay • Founded in 2001, joined the eBay family in June 2011 • Most widely-used e-shopping site in Turkey • More than 9 million registered users • Over 900 thousand daily visitors and • 5 million listings at any given time. • 700 thousand sellers all around Turkey • Selling 1 item every 3 seconds

©Continuent 2013

Gittigidiyor Datawarehouse The Problem !• 400 GB of data • 450 tables • over 150 million rows • 20GB binlog/day • MySQL inefficiency for large

data handling • Migration to Oracle • Need to keep auditing trail • Tried Golden Gate, but

• technical problems • too expensive

Replication

Master

ETL

CDC

DWH

���40

©Continuent 2013

Tungsten Solution !• DDL generation • Replication to Oracle master • Handling data correctly • CDC generation • Fail-safe

Master

ETL

Replicator

ORA Master

ORA CDC ORA DWH

Replication

Tungsten Replication

Tungsten Filters

���41

©Continuent 2013

The steps we took !1. Proof of Concept

a. replicated 5 tables b. created the DDL

Generator. 2. Set up the replication to

Oracle Master 3. Set up filters for Oracle

CDC generation. 4. Dumped all data in MySQL

Master and MySQL CDC to Oracle equivalents.

Master

ETL

Replicator

ORA Master

ORA CDC ORA DWH

Replication

Tungsten Replication

Tungsten Filters

���42

©Continuent 2013

MySQL to Oracle Demo

©Continuent 2013

Support for Tungsten Replicator

���44

• Community support is available on project mailing list

• Continuent offers implementation services and annual 24/7 support subscriptions

• For high availability or disaster recovery needs, check out Continuent Tungsten clustering

• Meanwhile, have fun and try it out!

©Continuent 2013 ���45

Continuent Web Page: http://www.continuent.com

!

Tungsten Replicator: http://tungsten-replicator.org

Our Blogs: http://flyingclusters.blogspot.com http://scale-out-blog.blogspot.com http://datacharmer.blogspot.com http://continuent-tungsten.blogspot.com

560 S. Winchester Blvd., Suite 500 San Jose, CA 95128 Tel +1 (866) 998-3642 Fax +1 (408) 668-1009 e-mail: sales@continuent.com