mysql_pres

8/7/2019 mysql_pres

1/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster

InfiniBandTransporter

Benchmarks

Summary

A native InfiniBand Transporter for

MySQL Cluster

Frank Mietke, Dirk Dunger, Torsten Mehlan,Torsten Hfler and Wolfgang Rehm

Computer Architecture GroupDepartment of Computer Science

Chemnitz University of Technology

February 8, 2007
http://find/http://goback/

8/7/2019 mysql_pres

2/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster


Benchmarks

Summary

Outline

1 Introduction

2 MySQL Cluster

3 InfiniBand Transporter

4 Benchmarks

5 Summary

8/7/2019 mysql_pres

3/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster


Benchmarks

Summary

General Motivation

Doubling of Storage Needs every 18 Months

(IDC)

(some apps grow faster)

Faster Internet Connectivity (Consumer) up to

100 Mbit/s

Ubiquitious for Business Processes

Globalization 24/7

Driven by User Need (Tax Offices)

8/7/2019 mysql_pres

4/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster

InfiniBand

Transporter

Benchmarks

Summary

Scalability Issues

Where to store?

No. of Nodes

No.

ofProcessors

ScaleOut

ScaleU

p

8/7/2019 mysql_pres

5/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster

InfiniBand

Transporter

Benchmarks

Summary

Clustered Database Systems

Examples:

Oracle RAC 10g

Shared Storage Approach (SAN, NAS or DAS)

Using Local Caching

Fast Recovery but Cache Coherence

IBM DB2 UDB ESE

Database Partitioning Feature (DPF)

Shared Nothing Approach

No Increase of Availability

MySQL Cluster

Shared Nothing/In-Memory Approach

Increase of Availability

Based on NDB Cluster

8/7/2019 mysql_pres

6/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster

InfiniBand

Transporter

Benchmarks

Summary

InfiniBand Interconnect

Most affordable 10Gbit/s Interconnect

Low Latency

RDMA Capabilities

HPC / Storage / Data Center

Wide Vendor Acceptance (OFA)

8/7/2019 mysql_pres

7/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster

InfiniBand

Transporter

Benchmarks

Summary

MySQL Cluster

Transporter for SHM, TCP/IP and SCI

Network Database (NDB)

8/7/2019 mysql_pres

8/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster

InfiniBand

Transporter

Benchmarks

Summary

Data Partitioning

8/7/2019 mysql_pres

9/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster

InfiniBand

Transporter

Benchmarks

Summary

Access the Cluster Data

Primary Key:

Using Hash Value

One Communication Step

Unique Key:Keyword UNIQUE (Hidden Table)

Two Communication Steps

Ordered Index:

Using T-Tree (Range Search)Communicate to all NDB Nodes

Complete Scan:

SQL Node asks each NDB Node (MySQL 5.x)

8/7/2019 mysql_pres

10/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster

InfiniBand

Transporter

Benchmarks

Summary

Transporter Registry

Handles different Transporter Types

(Creation and Management)

Provides Methods to NDB Process

prepareSend, performSend, pollReceive and

performReceive

8/7/2019 mysql_pres

11/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster

InfiniBand

Transporter

Benchmarks

Summary

Transporter Implementation

C++ Class

Exchange of Information through TCP Channel

Using Mellanox Verbs API (Channel/MemorySemantic)

Collecting short Messages

Handling of CQE (many Send Operations)

One CQ for all Instances of Transporter ClassImpact of Buffer Structure (Small Packets)

Usage of Inline Sends

8/7/2019 mysql_pres

12/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster

InfiniBand

Transporter

Benchmarks

Summary

Benchmark Setup

8/7/2019 mysql_pres

13/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster

InfiniBand

Transporter

Benchmarks

Summary

OSDL Database Test 2

Based on TPC-C

Simulates an Online Store

New Order Transactions per Minute

Background Transactions

Delivery

Order Status

Payment

Stock Level

Two Modes of Benchmarking

Realistic (with Pauses)

Full Load

8/7/2019 mysql_pres

14/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster

InfiniBand

Transporter

Benchmarks

Summary

Results OSDL DB Test 2

8/7/2019 mysql_pres

15/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster

InfiniBand

Transporter

Benchmarks

Summary

NDB testReadPerf

Measure the Impact of Interconnects on NDB

Cluster

No SQL node necessary

Benchmarks:serial pk

batch pk

serial uniq index

batch uniq indexindex eq-bound

index range

index ordered (batch)

interpreted scan

8/7/2019 mysql_pres

16/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster

InfiniBand

Transporter

Benchmarks

Summary

Results NDB testReadPerf

native InfiniBand

TCP (IP over InfiniBand)

SCI

TCP Ethernet (1 GBit/s)

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

interpreted_scanindex_eqboundserial_uniq_indexserial_pk

Verarbeitungsgescheindigkeit(rows/s)

native InfiniBand


SCI


0

20,000

40,000

60,000

80,000

100,000

120,000

140,000

160,000

180,000

index_orderedindex_rangebatch_uniq_indexbatch_pk

Verarbeitungsgescheindigkeit(rows/s)

native InfiniBand


SCI


0

200

400

600

800

1,000

1,200

1,400

1,600

1,800

interpreted_scanindex_eqboundserial_uniq_indexserial_pk

durchschnittlicheAntwortzeit(M

ikrosekunden)

native InfiniBand


SCI


0

5

10

15

20

25

30

35

40

45

50

index_orderedindex_rangebatch_uniq_indexbatch_pk

durchschnittlicheAntwortzeit(M

ikrosekunden)
http://goback/http://find/http://goback/http://find/http://goback/http://find/http://goback/http://find/http://find/http://goback/

8/7/2019 mysql_pres

17/17

MySQL + IB

Frank Mietke

Introduction

MySQL Cluster

InfiniBand

Transporter

Benchmarks

Summary

Summary

Relatively easy Implementation Hurdles

Good Replacement for SCI

Up to 40% faster Response time (testReadPerf)

Up to 70% more Processing Power

(testReadPerf)

Outlook

Finish RDMA Transporter

Use OFED Verbs API

mysql_pres

Documents