multi-class latency bounded web services vikram kanodia and edward knightly rice networks group
TRANSCRIPT
![Page 1: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/1.jpg)
Multi-Class Latency Bounded Web Services
Vikram Kanodia and Edward Knightly
Rice Networks Group
http://www.ece.rice.edu/networks
![Page 2: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/2.jpg)
Vikram Kanodia 2
Motivation
Poor end-to-end performance of web traffic. Excessive latencies due to overloaded
servers a dominant factor.
Present day web servers provide only FCFS.
Need Mechanisms to: Reduce server latency; and Control server latency.
![Page 3: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/3.jpg)
Vikram Kanodia 3
Web Hosting Example
A A A B B B B B
B .C O M
A .C O M
![Page 4: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/4.jpg)
Vikram Kanodia 4
Steps Towards Web QoS - 1
SBAC – Session Based Admission Control [CherkPhaal99]. Blocks sessions if load above a certain threshold.
Pros: Prevents server from going into overload.
Cons: Only ensures better service to all admitted
requests. Cannot ensure that requested service is met.
![Page 5: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/5.jpg)
Vikram Kanodia 5
Steps Towards Web QoS - 2
Operating system hooks: Mechanisms to support resource reservation
among different domains at OS level. Resource Containers [BangDrsch99]. Eclipse/BSD operating system [Silber99].
Prioritizing incoming requests provides class differentiation [BhattiFried99].
Distributed server architecture for better throughput [Vpai98].
![Page 6: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/6.jpg)
Vikram Kanodia 6
What is Lacking ?
No mechanism to meet a requests’ targeted delay.
No class based service model: Multiple user classes. Each class has a different response time target. All classes contending for the same resource.
No means of statistically quantifying the service received .
![Page 7: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/7.jpg)
Vikram Kanodia 7
Key Challenges
Net service rate is a complex, unknown function of CPU / disk/ cache behavior.
Very difficult to model a requests’ service demand in terms of low level system resources.
Interaction between requests belonging to different classes difficult to predict a priori.
All present day web QoS schemes coupled tightly with server architecture .
![Page 8: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/8.jpg)
Vikram Kanodia 8
System Model
L IS T E N Q U E U E A D M IS S IO NC O N TR O L
M EA S UR EM ENT
S C H E D U L IN G
B A CK E N DN O D E S
IN C O M IN GR E Q U E S T S
D R O P P E DR E Q U E S T S
B L O C K E DR E Q U E S T S
S E R V E DR E Q U E S T S
F R O N T E N D
![Page 9: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/9.jpg)
Vikram Kanodia 9
First Cut: Baseline Scheme
Latency targeted service model: Single user class with a targeted delay to be
met by some percentage of all serviced requests.
Goals: Illustrate an abstraction of the server
resources into a simple queuing model. Highlight key issues for managing multi-
class web services. Use for experimental comparisons.
![Page 10: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/10.jpg)
Vikram Kanodia 10
Baseline Scheme: Problem formulation
Assumption: Stationary and homogeneous arrivals.
Some maximum service rate which satisfies QoS requirements. All arrival greater than the maximum
service rate need to be be blocked.
How to determine the maximum service rate ?
![Page 11: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/11.jpg)
Vikram Kanodia 11
Model for Baseline Scheme
A D M I S S I O NC O N T R O L F O R
C L A S S i
i i
i i
a i , d i
j j
![Page 12: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/12.jpg)
Vikram Kanodia 12
Baseline Scheme: M/M/1 model
Approximate a class’ service by an M/M/1 queue with an unknown service rate. Abstracts the low level server resources into
a virtual server.
Unknown Service rate is given by:
d
1
![Page 13: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/13.jpg)
Vikram Kanodia 13
Baseline Scheme: Admission Control
A new request leads to an increase in load to ’. Delay violation probability under load ’:
If P( D > d*) is greater than the targeted fraction of requests meeting the delay target , block the new request.
))' (*exp(*)( ddDP
![Page 14: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/14.jpg)
Vikram Kanodia 14
Limitations of Baseline Scheme
No support for multiple service classes M/M/1 models each class as independent of
other classes. Cannot capture inter class interference.
Assumption of independent and exponentially distributed service times is faulty. Does not account for highly variable service
time. Ignores temporal correlation among different
requests for the same document.
![Page 15: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/15.jpg)
Vikram Kanodia 15
Solution
LMAC : Latency Targeted Multi-Class Admission Control
Service model: A minimum fraction of accepted requests
will be serviced within the class delay target. Mechanism to characterize and control inter-
class relationships. Decouples access control from actual server.
architecture or the operating system.
![Page 16: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/16.jpg)
Vikram Kanodia 16
Our Technique: Envelopes
Envelopes: arrival/service rates over intervals of time.
Deterministic [Cruz95] and statistical [QK99,CK00] envelopes are used to manage network QoS.
Envelopes represent net service received in the presence of other concurrent requests being processed by the server at the same time.
![Page 17: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/17.jpg)
Vikram Kanodia 17
What do Envelopes Buy Us ?
A general yet accurate way of describing a class’ service and demand.
A higher level of abstraction of low level system resources.
Capture effects of temporal correlation and high variability in requests and server latencies.
Model relationship among different user classes in a tractable manner.
![Page 18: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/18.jpg)
Vikram Kanodia 18
Measured Based Service Envelope
Envelope is service received versus interval length when backlogged.
Given the number of concurrently backlogged requests:
Compute the request latency mean and variance.
Use gaussian approximation to get the targeted percentile delay.
![Page 19: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/19.jpg)
Vikram Kanodia 19
Model for LMAC
A D M I S S I O NC O N T R O L F O R
C L A S S i
i
i i
a1 ,a2 ,.....,a n s1 ,s2 ,.....,sn
i
![Page 20: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/20.jpg)
Vikram Kanodia 20
LMAC Algorithm
Ensure that a arrival maintains the latency target of its own class Maintain a maximum horizontal distance
between the requests and service envelopes less than the targeted latency.
How to ensure that the service of other classes is not disrupted ?
![Page 21: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/21.jpg)
Vikram Kanodia 21
LMAC Algorithm (cont.)
To ensure that other classes do not suffer:
Assume that the new arrival has strict priority over all other requests.
This is a worst case assumption.
For all other classes, the request workload remains the same, but there is a reduction in service.
![Page 22: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/22.jpg)
Vikram Kanodia 22
Simulation Details
Simulations performed using a simulator which approximates the behavior of OS management for CPU, disk, caching etc.
Use a trace generated from the CS departmental server logs at Rice University.
Assume arrival rate is poisson with a given mean rate.
![Page 23: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/23.jpg)
Vikram Kanodia 23
Experiment 1
Targeted delay of 1 second for 95 percentile of all admitted requests.
Demonstrates overload protection properties similar to SBAC.
![Page 24: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/24.jpg)
Vikram Kanodia 24
Experiment 2
Single class-single node case.
Baseline scheme does meet its delay target, but is too conservative.
![Page 25: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/25.jpg)
Vikram Kanodia 25
Multi-Class Performance
In the absence of any server level support : Performance of each class bounded by the
most stringent class.
To investigate a true multi-class scenario: Devise an artificial resource allocation
policy.
![Page 26: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/26.jpg)
Vikram Kanodia 26
Experiment 3: Setup
IS O L A TIO NF R O NT
END
BAC KEND
BAC KEND
A
B
A
B
B E X P L O ITSIN TE R -C L A S SG A IN S
F R O NTEND
BAC KEND
BAC KEND
A
B
B
A + B
![Page 27: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/27.jpg)
Vikram Kanodia 27
Experiment 3 (cont.)
Class A: Arrival rate 300 reqs/sec, target delay .5 sec
Class B: Arrival rate 200 reqs/sec, target delay 1 sec
Class IsolationMulti-class with
SharingThroughp
ut(reqs/sec)
Delay(sec)
Throughput(reqs/sec)
Delay(sec)
A 147 .467 141 .501
B 92 .912 145 .935
![Page 28: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/28.jpg)
Vikram Kanodia 28
Conclusions
Scheme to ensure that a minimum fraction of all accepted requests meet latency targets.
A way to model system resources into a high level server: Makes our approach general and
independent of OS/ server architecture.
Ability to exploit additional features within the server architecture for higher utilization.
![Page 29: Multi-Class Latency Bounded Web Services Vikram Kanodia and Edward Knightly Rice Networks Group](https://reader033.vdocuments.site/reader033/viewer/2022051515/551a0de755034619378b4d56/html5/thumbnails/29.jpg)
Vikram Kanodia 29
Future Work
Address Heterogeneous Content Content with different service demands , e.g
dynamic content.
Perform experiments with additional traces.
Incorporate LMAC into a real server and test its performance.