mongiops: your favorite datastore, only faster
DESCRIPTION
Matthew Kennedy, Fusion-io Big Data Solutions Architect, presented at MongoSV on December 4, 2012. Joining him were Dale Russell and Brian Knox of Talksum.TRANSCRIPT
MONGIOPS Your Favorite Datastore, Only Faster…
A MONGO SERVER
This is a typical server…
December 5, 2012 Fusion-io Confidential 2
Dell R720 CPU Cores: 32 w/HT Memory: 128 GB
…is your working set > 128GB?
DO
LL
AR
S
COST OF DRAM MODULES
0
200
400
600
800
1000
1200
1400
1600
4GB 8GB 16GB 32GB
Fusion-io Confidential December 5, 2012 3
$ $$ $$$
$$$$$$
A MONGO CLUSTER
With NoSQL Databases, we tend to scale out for DRAM
December 5, 2012 Fusion-io Confidential 4
Combined Resources CPU Cores: 96! Memory: 384 GB
Do we need that many cores to serve reads and writes?
RE
LA
TIV
E P
ER
FO
RM
AN
CE
(L
OG
AR
ITH
MIC
SC
AL
E)
STORAGE IS SLOW
December 5, 2012 5 Fusion-io Confidential Source: http://www.micronblogs.com/category/history-of-digital-storage/
1
100,000
100
1,000
10,000
10
1988 2008
HDD Performance
TECHNOLOGY ENABLERS
© Fusion-io
FLASH MEMORY
SOFTWARE-ENABLED REPROGRAMMABLE
CONTROLLERS
PCIe Ecosystem
SSD TREATS MEMORY LIKE DISK
© Fusion-io
MONGO ON FUSION-IO
August, 2012 Fusion-io Confidential 8
Step 1: Mount ioDrive at /var/lib/mongo Step 2:
IODRIVE2 SPECIFICATIONS
ioDrive2 Capacity 400GB 600GB 365GB 785GB 1.2TB
NAND Type SLC (Single Level Cell)
SLC (Single Level Cell)
MLC (Multi Level Cell)
MLC (Multi Level Cell)
MLC (Multi Level Cell)
Read Bandwidth (1 MB) 1.4 GB/s 1.5 GB/s 910 MB/s 1.5 GB/s 1.5 GB/s
Write Bandwidth (1 MB) 1.3 GB/s 1.3 GB/s 590 MB/s 1.1 GB/s 1.3 GB/s
Read IOPS (512 Byte) 190,000 193,000 137,000 141,000 143,000
Write IOPS (512Byte) 535,000 535,000 535,000 535,000 535,000
Read Access Latency 47 µs 47 µs 68 µs 68 µs 68 µs
Write Access Latency 15 µs 15 µs 15 µs 15 µs 15 µs
Bus Interface PCI-Express 2.0 x4
Weight Less than 11 ounces
Operating Systems 64-Bit Microsoft XP/Vista/Win7/Server 2003/Server 2008, RHEL 4/5/6, SLES 10/11, OEL v5/6,
VMware ESX 4.0/4.1/ESXi 4.1/5.0*, Solaris 10 U8/U9, OSX* 10.6.7/10.7, HP-UX* 11i *ioDrive2 and ioDrive2 Duo
August, 2012 Fusion-io Confidential 9
IODRIVE2 DUO SPECIFICATIONS
ioDrive2 Capacity 1.2TB 2.4TB
NAND Type SLC (Single Level Cell) MLC (Multi Level Cell)
Read Bandwidth (1 MB) 3.0 GB/s 3.0 GB/s
Write Bandwidth (1 MB) 2.5 GB/s 2.5 GB/s
Read IOPS (512 Byte) 385,000 285,000
Write IOPS (512Byte) 725,000 725,000
Read Access Latency 47 µs 68 µs
Write Access Latency 15 µs 15 µs
Bus Interface PCI-Express 2.0 x8
Weight Less than 16 ounces
Operating Systems 64-Bit Microsoft XP/Vista/Win7/Server 2003/Server 2008, RHEL 4/5/6, SLES 10/11, OEL v5/6,
VMware ESX 4.0/4.1/ESXi 4.1/5.0*, Solaris 10 U8/U9, OSX* 10.6.7/10.7, HP-UX* 11i *ioDrive2 and ioDrive2 Duo
August, 2012 Fusion-io Confidential 10
GO TO OUR WEBSITE
December 5, 2012 Fusion-io Confidential 11
ONE IODRIVE VS A 10-DISK RAID-0
December 5, 2012 Fusion-io Confidential 12
vs
ioDrive 2 – 1.2TB 10 x 7200 RPM SAS disks in raid-0 mdarray
YCSB DETAILS
▸ 2 x 8-core 2.9GHz Xeon/64GB/2x10GbE ▸ XFS
Test ▸ Workload A: 50/50% read/write mix ▸ Workload B: 95/5% read/write mix ▸ Workload C: 100% read ▸ Workload F: 50/50% read/read+modify+write mix
December 5, 2012 Fusion-io Confidential 13
Workload A - 50/50% read/write mix
December 5, 2012 Fusion-io Confidential 14
Workload B - 95/5% read/write mix
December 5, 2012 Fusion-io Confidential 15
Workload C - 100% read
December 5, 2012 Fusion-io Confidential 16
Workload F - 50/50% read/read+modify+write mix
December 5, 2012 Fusion-io Confidential 17
AV
G R
EA
D L
AT
EN
CY
IN
MIL
ISE
CO
ND
S
YCSB: LOWEST READ LATENCY WINS
December 5, 2012 18
0
20
40
60
80
100
120
140
160
180
W O R K L O A D A W O R K L O A D B W O R K L O A D F : U P D A T E W O R K L O A D F : R M W
Fusion-io Disk
Fusion-io Confidential
30x
Fast
er
27x
Fast
er
20x
Fast
er
40x
Fast
er
AV
G U
PD
AT
E L
AT
EN
CY
IN
MIL
ISE
CO
ND
S
YCSB: LOWEST UPDATE LATENCY WINS
December 5, 2012 19
0
50
100
150
200
250
300
350
400
450
W O R K L O A D A W O R K L O A D B W O R K L O A D F : U P D A T E W O R K L O A D F : R M W
Fusion-io Disk
Fusion-io Confidential
18x
Fast
er
4x F
aste
r
14x
Fast
er
16x
Fast
er
ARCHITECTURAL IMPACT OF FUSION-IO
▸ Avoid scaling out for DRAM • Nodes can handle higher transaction load • Terabytes of low latency persistent storage on single nodes • Potentially avoid sharding
▸ Use less DRAM per node • Lower cost servers • DRAM available for applications
December 5, 2012 20 Fusion-io Confidential
Talksum Data Stream Real Time for Big Data
Talksum Data Stream Appliance
• Real Time Data Management & Analy1cs
• Filters, Monitors, Counts, Transforms & Reduces
• OS Level Processing for High Performance
Talksum Data Stream
System Logs
App Data
Device Stats
OtherData
Accepts Data fromany source
Filter &Route
Real-TimeMonitors & Analytics
Real Time Reduced
Data
RemoteSites
Outputs to Storage,Applications or other Systems
BI Tools
Analytic Apps
StoredProcess
Third Party
API
Device
Storage
WHY DO WE USE MONGO?
▸ First Class JSON Support • Perfect match for CEE enhanced RFC3164 • “Schemaless” = great for log data with minor variations
▸ Capped Collections • Control over space utilization important on an appliance
▸ Atomic Operations • Great for working with metrics
▸ Aggregation Framework • Convenience of SQL “group by”
▸ Map Reduce • Familiar paradigm
December 5, 2012 Fusion-io Confidential 23
ANALYZING NETFLOW DATA
Analyzing a day’s worth of Netflow Data to generate sent octets metrics for each source ip, destination ip, and source->destination ip combination using atomic increment operations
{ _id : src_key }, { $inc : { sntOcts : snt_oct } } { _id : dst_key }, { $inc : { sntOcts : snt_oct } } { _id : src_dst_key }, { $inc : { sntOcts : snt_oct }}
To test IO, messages sent as fast as possible.
No Queue! Messages dropped when IO capacity overflows.
December 5, 2012 Fusion-io Confidential 24
NETFLOW MESSAGES PROCESSED A SECOND
Atomic Updates: each message is 3 Mongo operations.
December 5, 2012 25 Fusion-io Confidential
RESULTS OVER 1 HOUR (3600 SECONDS)
▸ SATA Drive • Total messages processed: 2,602,550 • Messages Per Second: 3,614 • Total Mongo Operations: 7,807,650 • Mongo Operations Per Second: 10,481
▸ Fusion-io Drive • Total messages processed: 7,811,851 • Messages Per Second: 10,849 • Total Mongo Operations: 23,435,553 • Mongo Operations Per Second: 30,549
December 5, 2012 26 Fusion-io Confidential
T H A N K Y O U