dbms workloads in online services - tpc · 2009-09-05 · dbms workloads in online services swaroop...
TRANSCRIPT
![Page 1: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/1.jpg)
DBMS workloads in online services
Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar,Eno Thereska, Kushagra Vaid, and Bruce Worthington
Microsoft Redmond and Microsoft Research Cambridge
![Page 2: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/2.jpg)
Large-scale online services
• 1000s of servers
• Millions of users
• In mega-scale data centers
– Each hosting many such services
• Server, infrastructure costs dominate
• Rightsizing is key
– pick the right #servers
2
![Page 3: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/3.jpg)
Large-scale online services
3
Structuredstorage tier
I/O
![Page 4: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/4.jpg)
Load variation over time
• User-facing services show diurnal pattern
– “Pacific Ocean trough”
• Important to understand
– Consolidate un/anti correlated workloads
– Schedule background tasks intelligently
– Power down resources at low load
• Potentially big $$$ at mega-DC scale
4
![Page 5: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/5.jpg)
Challenges
• Rightsizing
– How many servers, and what hardware?
– How much disk space v. IOPS v. CPU ...
• Consolidation
– Which workloads are un(anti)correlated
• Power-proportionality
– Maximize work done / Joule
5
![Page 6: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/6.jpg)
In this talk
• Analyze I/O traces of real workloads
– from structured storage in online services
• Characterize the workloads
– Using a variety of metrics
• Compare with standard TPC benchmarks
– How well do they match?
6
![Page 7: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/7.jpg)
Outline
• Motivation
• Online workload analysis
• Conclusion
7
![Page 8: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/8.jpg)
Workloads studied
• IM-DB– Messenger user profiles, buddy lists
• MSN-DB:– Web content for online portal
• EMAIL-DB– E-mail service metadata
• BLOB-DB– Metadata for blob store (blobs = photos, videos, ...)
8
![Page 9: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/9.jpg)
Production server tracing
• Gives a very realistic picture of workload
• Low-overhead tracing infrastructure
– Event Tracing for Windows (since Win 2000)
• Not trivial to setup (but worth it)
– Avoid operations impact
– Anonymize PII
– Build trust with stakeholders
9
![Page 10: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/10.jpg)
Production server tracing
• 4 services, 1 representative server each
• Traced every block-level read and write
– Device number, offset, size, timestamp
• Trace periods vary (25 min – 24 hrs)
• Below the buffer cache
• Also traced runs of TPC-C, TPC-E, TPC-H
10
![Page 11: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/11.jpg)
Workload traces
Workload Trace length Storage arrays Total disks
IM-DB25 min 5 x RAID-10 34
MSN-DB24 hrs 10 x RAID-10 46
EMAIL-DB2 hrs 4 x RAID-10 34
BLOB-DB24 hrs 10 x RAID-10 46
TPC-C6 min 14 x RAID-0 392
TPC-E17 min 12 x RAID-0 336
TPC-H1.5 hrs 4 x RAID-0 36
11
![Page 12: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/12.jpg)
Workload trace observations
• Data file I/Os dominate
– Log traffic is 11-12% for BLOB-DB, MSN-DB
– < 2% for others
• Traced servers provisioned differently
– 34 – 392 spindles
• Need to normalize load “per unit storage”
– We normalize by data size, e.g. IOPS/GB
12
![Page 13: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/13.jpg)
Workload metrics extracted
• Peak non-sequential request rate (IOPS)
• Peak sequential transfer rate (MB/s)
• Peak-to-mean ratios (for IOPS, MB/s)
• Data set size (GB)
– Based on highest LBN accessed in trace
• Sequential fraction of I/Os
• Read/write ratio
13
![Page 14: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/14.jpg)
Peak IOPS v data size (log-log)
14
![Page 15: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/15.jpg)
I/O rate v transfer rate (log-log)
15
![Page 16: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/16.jpg)
IOPS v data size
• Order-of-magnitude differences
– Between all workloads (online & TPC)
• But, servers provisioned differently
– TPC-C had 10x the spindles of EMAIL-DB
• We should look at load per unit storage
– IOPS/GB, not IOPS/traced server
• IOPS and MB/s highly correlated
– SQL Server uses mostly 8KB requests16
![Page 17: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/17.jpg)
IOPS/GB (peak IOPS)
17
![Page 18: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/18.jpg)
IOPS/GB (log scale)
18
![Page 19: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/19.jpg)
Peak-to-mean load ratios
19
![Page 20: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/20.jpg)
Online workloads have ...
• much lower IOPS/GB than TPC
– Even when considering peak IOPS
– Except IM-DB: roughly same as TPC-C
• higher peak/mean ratios than TPC-C,E
– Except IM-DB
– TPC-H comparable to BLOB-DB, MSN-DB
• But for different reasons (TPC-H has phases)
– EMAIL-DB has very high peak/mean ratio
20
![Page 21: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/21.jpg)
R/w ratio and sequentiality
21
![Page 22: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/22.jpg)
Time variation: BLOB-DB
22
![Page 23: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/23.jpg)
Time variation (MSN-DB)
23
![Page 24: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/24.jpg)
Summary
• R/w ratio, sequentiality vary hugely
– Some workloads close to TPC benchmarks
• But differ on other metrics (like IOPS/GB)
• Online workloads have time variation
– Periodic (diurnal, hourly)
– Noise (high-frequency variation)
– Load spikes
• TPC benchmarks do not have this notion24
![Page 25: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/25.jpg)
Outline
• Motivation
• Online workload analysis
• Conclusion
25
![Page 26: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/26.jpg)
Analysis summary
• Online workloads vary widely
– Differ from TPC benchmarks and each other
– IM-DB is the most “TPC-like”
• Sometimes like TPC-C, sometimes like TCP-E
• Still not a great match
• Low IOPS/GB ratio even at peak
• High peak-to-mean ratios
• Time variation in load26
![Page 27: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/27.jpg)
How do we measure perf?
• Current benchmarks not representative
– For these workloads
• Devise new benchmarks?
– Workloads also vary widely among each other
– Would need one benchmark per service
• Measure using I/O trace replay?
– Effective, but has its limitations
27
![Page 28: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/28.jpg)
Trace replay advantages
• Captures properties of real workload
• We used traces to drive many evaluations
– Disk spin-down depends on idle times
– Burst absorbtion depends on burstiness
– SSD v disk depends on IOPS/GB
• Benchmarks would not have worked here
28
![Page 29: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/29.jpg)
Trace replay limitations
• Trace replay captures real workload
• But has limitations vis-a-vis benchmarks
– I/O trace replay only measures disk resources
– “Open loop” problems
– Hard to scale (up or down)
– Not standardized for comparison of systems
29
![Page 30: DBMS workloads in online services - TPC · 2009-09-05 · DBMS workloads in online services Swaroop Kavalanekar, Dushyanth Narayanan, Sriram Sankar, Eno Thereska, Kushagra Vaid, and](https://reader034.vdocuments.site/reader034/viewer/2022042416/5f30daf4fd22363f65690d8e/html5/thumbnails/30.jpg)
Future directions
• End-to-end tracing
– All resources (CPU, network, user think time)
• Parameterize the benchmarks
– Set IOPS/GB, r/w ratio, ... to measured values
– Need to allow orders of magnitude variation
– Need to model/express “time variation”
• Trace repository a la IOTTA
– Maybe TPC can help set this up?30