online feedback-based estimation of dynamic page service time
DESCRIPTION
Online Feedback-based Estimation of Dynamic Page Service Time. Ashwini Kumar Kaushik Veeraraghavan Benjamin Wester Kang Shin. Motivation. Dynamic pages Increasingly prevalent Pages difficult to cache Service time applications Differentiated QoS (prioritizing requests) Load balancing - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/1.jpg)
Online Feedback-based Estimation of Dynamic Page
Service Time
Ashwini KumarKaushik Veeraraghavan
Benjamin WesterKang Shin
![Page 2: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/2.jpg)
2
Motivation
• Dynamic pages– Increasingly prevalent– Pages difficult to cache
• Service time applications– Differentiated QoS (prioritizing requests) – Load balancing– Virtual host services– Estimates: credit card payment times
![Page 3: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/3.jpg)
3
Idea
• History– Maintain records of past service times– Predict based on existing records
• Assume a correlation between past and future– What is the invariant property?– What system state can we store to track it?
• We explore three estimators and their design
![Page 4: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/4.jpg)
4
Outline• Motivation
• Design & Implementation
• Evaluation
• Conclusion and Future Work
![Page 5: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/5.jpg)
5
History
1. Maintain per-URL fixed-size history table2. Data depends on the estimator used3. Tag incoming requests4. Update history when request processing finishes
![Page 6: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/6.jpg)
6
SEDA: Haboob
![Page 7: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/7.jpg)
7
Sirocco
![Page 8: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/8.jpg)
8
Estimators
Scheme Store in history Estimation
Average
STQ
STQT
Length Queue
Time Service
Time Service
Length Queue
Threads Time Service
)(historyavg
Length Queuehistoryavg )(
Threads
Length Queuehistoryavg )(
![Page 9: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/9.jpg)
9
Outline• Motivation
• Design & Implementation
• Evaluation
• Conclusion and Future Work
![Page 10: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/10.jpg)
10
Evaluation
• How close are our estimators?
• Load patterns– Steady load: httperf
• 50ms / request
– Changing load: SURGE
0
20
40
60
80
100
0 5 10 15 20 25
Re
que
sts
Time (seconds)
![Page 11: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/11.jpg)
11
Average Estimator (steady)
• Request takes ~35ms to complete
• Estimator should not be too responsive
![Page 12: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/12.jpg)
12
Average Estimator (changing)
• Each point averages 20 requests
• Vertical bars indicate the addition of a handling thread
![Page 13: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/13.jpg)
13
Average Estimator
• How to read the graph:• Enters system: t = 5.4s• Estimated service time: 1.6s• Real service time: 4.9s
![Page 14: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/14.jpg)
14
Average Estimator
• History becomes out-of-date
• Base estimation on the current state of the system
![Page 15: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/15.jpg)
15
STQ Estimator
• It’s better than the Average Estimator…
• Inaccurate around a thread change
![Page 16: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/16.jpg)
16
STQ Estimator
• Estimations will be inaccurate unless the estimator knows about the coming change in the number of threads
• After a change, recently-completed requests show a mixed view of system state and will pollute the history
![Page 17: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/17.jpg)
17
STQT Estimator
• Overshoots target because of a polluted history
• Fix: Integrate request times across the thread change
![Page 18: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/18.jpg)
18
Outline• Motivation
• Design & Implementation
• Evaluation
• Conclusion and Future Work
![Page 19: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/19.jpg)
19
Concluding Remarks
• Takeaways– Don’t adapt to transient delays– Use current state along with the history– Be aware of underlying changes
• Future Directions– Extrapolate requests’ service times– URL service time as a distribution
![Page 20: Online Feedback-based Estimation of Dynamic Page Service Time](https://reader035.vdocuments.site/reader035/viewer/2022062809/56815884550346895dc5e65f/html5/thumbnails/20.jpg)
Questions?