1 wenguang wangrichard b. bunt department of computer science university of saskatchewan november...
TRANSCRIPT
![Page 1: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/1.jpg)
1
Wenguang Wang Richard B. BuntDepartment of Computer Science
University of Saskatchewan
November 14, 2000
Simulating DB2 Buffer Pool Management
![Page 2: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/2.jpg)
2
Outline
• Background
• Problem
• Methodology
• Simulation results
• Future work
![Page 3: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/3.jpg)
3
Background• What is buffer pool
Buffer Pool
Database on Disks
Writes Reads
Applications
Upper layer of DBMSDBMS
![Page 4: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/4.jpg)
4
Problem
• Buffer pool management is important to the performance of any DBMS
• The config and tuning of buffer pool is not a easy problem for the database administrator
• The buffer pool management of a DBMS is very complex
• It is hard to study and test the buffer pool management algorithm directly
![Page 5: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/5.jpg)
5
Methodology
• Trace-driven simulation provides an effective approach
• Compare to the real DBMS:– Simulator is easier to be controlled
– Simulator requires much lower computing resources (CPU, memory, disk, running time)
– New algorithm is easier to be implemented and tested in the simulator
– Changes that cannot be done or are not easy to do in the real system can be simulated in the simulator
![Page 6: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/6.jpg)
6
Methodology
• Create trace-driven simulation tools– Collect trace– Process trace– Develop simulator– Verify simulator
• Perform experiments by the simulator– Understand the effect of buffer pool parameters– Give suggestions to the tuning of buffer pool– Design and test alternate buffer pool algorithms
![Page 7: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/7.jpg)
7
System Configuration
• DBMS — IBM DB2– Relational DBMS– Distributed DBMS which supports multiple
nodes. Because buffer pools on different nodes are independent, only the single node DB2 is studied
• Workload — the TPC-C benchmark– An On-Line Transaction Processing benchmark– Many clients send simple queries simultaneously
to the DBMS on the server side– A large amount of data are updated by the queries
![Page 8: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/8.jpg)
8
System Configuration (cont.)
• DB2 version 6.1 running on Windows NT Server 4.0
• TPC-C database – Small application: 50-warehouses (5GB data)
spanning over 9 physical disks
![Page 9: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/9.jpg)
9
Trace Collection• Trace tools of DB2
• Suspend the TPC-C benchmark periodically to record big enough trace
Buffer Pool
Database on Disks
Writes Reads
Applications
Upper layer of DBMSDBMSTrace point
![Page 10: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/10.jpg)
10
Trace Volume
• 60M buffer pool requests
• 200K TPC-C transactions
• Equivalent to 30 minutes TPC-C run when no traces are recorded
![Page 11: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/11.jpg)
11
Buffer Pool Simulator
• To simulate the buffer pool management algorithm and the disk activities
• About 8000 lines C++ code
![Page 12: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/12.jpg)
12
Architecture of the Simulator
![Page 13: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/13.jpg)
13
Clock-Pointer
Page Cleaners
Cleaning pages
DB2 Buffer Pool Algorithm
Clock Algorithm
Threshold: triggers the page cleaning activity
Database on Disks
![Page 14: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/14.jpg)
14
Buffer Pool Algorithm (cont.)
Threshold
Asynchronous writesperformed by page
cleaners
Dirty pages
Clean Region
Dirty Region
Expand
BufferPool
Reads
Synchronouswrites
Database on Disks
TPC-C
![Page 15: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/15.jpg)
15
Simulator Verification
• Compare the throughput curve
• Compare the run-time statistics– Hit ratio– Dirty page percentage
• Test the effect of parameters– Dirty page threshold– Number of page cleaners
![Page 16: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/16.jpg)
16
Simulator Verification— Similar Throughput Curve
![Page 17: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/17.jpg)
17
Page Distribution of Buffer Pool
![Page 18: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/18.jpg)
18
I/O Activities of the Buffer Pool
![Page 19: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/19.jpg)
19
Simulation Results Under Default Configuration
• Page cleaners cannot clean out pages fast enough
under the default configuration (2 page cleaners)
• Too many dirty pages (87%) in the buffer pool
under the default configuration
• The existence of too many dirty pages lowers the
buffer pool hit ratio and performance
![Page 20: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/20.jpg)
20
Effect of More Page Cleaners
![Page 21: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/21.jpg)
21
IO Activities Under More Page Cleaners
![Page 22: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/22.jpg)
22
Effect of Number of Page Cleaners
![Page 23: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/23.jpg)
23
Effect of Buffer Pool Parameters
• Threshold cannot affect performance when the number of page cleaners is small
• Setting an appropriate number of page cleaners is important to performance
• Appropriate number of page cleaners are different for different workloads
![Page 24: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/24.jpg)
24
Future work• Gain more understanding of the buffer pool
algorithm from the simulator and DB2
• Extend the work to a much larger TPC-C database
• Investigate alternative algorithms of the buffer pool management algorithm which are easier to be managed and tuned
• Test the alternative algorithms first in the simulator and then in the real system
![Page 25: 1 Wenguang WangRichard B. Bunt Department of Computer Science University of Saskatchewan November 14, 2000 Simulating DB2 Buffer Pool Management](https://reader036.vdocuments.site/reader036/viewer/2022062517/56649ec75503460f94bd3c3e/html5/thumbnails/25.jpg)
25
Questions?