shanjiang tang, bu-sung lee, bingsheng he, haikun liu school of computer engineering nanyang...
TRANSCRIPT
Shanjiang Tang, Bu-Sung Lee, Bingsheng He, Haikun Liu
School of Computer Engineering
Nanyang Technological University
Long-Term Resource FairnessTowards Economic Fairness on Pay-as-you-use Computing Systems
Pay-As-You-Use is Pervasive• Charge users based on the amount of resources
used over time (e.g., Hourly).• Advantages
– Elasticity– Flexibility– Cost efficiency
• Pay-as-you-use is becoming common and popular.– Supercomputing, Cloud Computing
2
Twitter’s Cluster
Twitter’s Cluster
One week data from Twitter production cluster [Delimitrou et. Al. ASPLOS’14]
Resource Utilization =
• User resource demands are heterogeneous.– Users have different demands.– A user’s demand is changing over time.Static provisioning/partitioning causes
underutilization. • Resource utilization is a critical problem in such
pay-as-you-use environments.– Providers waste resources( waste investment and lose profit).– Users waste money.
3
• Resource Sharing can improve resource utilization.– Allow underloaded users to release resources to other
users.– Allow overloaded users to temporarily use more resources (from others). Reduce the idle resources at runtime. Resolve resource contention across users.
• What about fairness?– If the fairness is not solved, resource sharing is unlikely
to achieve in pay-as-you-use environments.
To Share or Not To Share?
4
Pay-as-you-use Fairness: Resource-as-you-pay• The total resources a user gained should be
proportional to her payment.• This is a Service-Level Agreement (SLA).
60 $
40 $40 $
A:A:
B:B:
60% 40%40%
Resource ServiceResource Service
AA BB
Resource Service = Resources-per-time X service timeResource Service = Resources-per-time X service time
5
Fair Policy in Existing Systems• State-of-the-art: Max-min fairness
– Select the user with the minimum allocation/share ratio every time.
– Consider the present requirement only (memoryless).• Memoryless fairness has severe problems in pay-as-you-use
environments, violating the following properties:– Resource-as-you-pay fairness guarantee.– Non-Trivial workload incentive and sharing incentive.– Truthfulness (Users may get benefits by cheating).
8
Problems with MemoryLess Fairness• Resource-as-you-pay Fairness Problem
– E.g., A, B equally pay for total resource of 100 units.
Time New Demand
A B
t1 20 100
AA BB
Accumulate Resource
Usage:
Accumulate Resource
Usage:
20
8080
2020 8080
Unsatisfied DemandUnsatisfied Demand
AA BB00 2020
9
Current Allocation at t1:
Current Allocation at t1:
Problems with MemoryLess Fairness• Resource-as-you-pay Fairness Problem
– E.g., A, B equally pay for total resource of 100 units.
Time New Demand
A B
t1 20 100
t2 40 60
AA BB
Accumulate Resource
Usage:
Accumulate Resource
Usage:
40
6060
6060 140140
AA BB
Unsatisfied DemandUnsatisfied Demand
00 2020
10
Current Allocation at t2:
Current Allocation at t2:
Problems with MemoryLess Fairness• Resource-as-you-pay Fairness Problem
– E.g., A, B equally pay for total resource of 100 units.
Time New Demand
A B
t1 20 100
t2 40 60
t3 80 50
AA BB
Accumulated resource usage:
Accumulated resource usage:
50
5050
110110 190190
AA BB
Unsatisfied DemandUnsatisfied Demand
3030 2020
11
Current Allocation at t3:
Current Allocation at t3:
Problems with MemoryLess Fairness• Resource-as-you-pay Fairness Problem
– E.g., A, B equally pay for total resource of 100 units.
Time New Demand
A B
t1 20 100
t2 40 60
t3 80 50
t4 60 50
AA BB
Accumulated resource usage:
Accumulated resource usage:
50
5050
160160 240240
AA BB
Unsatisfied DemandUnsatisfied Demand
4040 2020
Existing Fair Policy fails to satisfy Resource-as-you-pay fairness!!!Existing Fair Policy fails to satisfy Resource-as-you-pay fairness!!!12
Current Allocation at t4:
Current Allocation at t4:
MemoryLess Fairness Violates Sharing Incentives• Non-trivial workload and sharing incentive Problem
– Yielding resources to others have no benefits. – Suppose A, B, and C equally pay for total resource of 100
units. A has 13 idle resource units. In that case, A can be selfish, either idle or running trivial workloads.
CPUCPU2020
3333
3333
A:A:
B:B:
C:C:
A’s idle resourceA’s idle resource
13
1313
Cheating User Benefits on MemoryLess Fairness • Truthfulness Problem
– Suppose A, B, C equally pay for a cluster of 100 units, with true demand to be 33, 21 and 80, respectively.
– Case 1: all are honest. – Case 2: User A cheats and claims the demand to be 40.
14
3333
2121
3333
A:A:
B:B:
C:C:
66
66
A’s cheating get benefits
A’s cheating get benefits
3333
2121
3333
A:A:
B:B:
C:C:1212
Case 1: A is honest
Case 1: A is honest
Case 2: A is cheating
Case 2: A is cheating
Our Work
• Challenges: can we find a fair sharing policy that satisfies the following properties?– Resource-as-you-pay fairness– Non-trivial workload and sharing incentives– Truthfulness
• Our Solution: Long-Term Resource Fairness– Ensure resource fairness over a period of time. – With historical information considered.
15
Long-Term Resource Fairness
• Basic Concept: Loan agreement (Lending w/o interests)– When resources are not needed, users can lend the
resources to others. – When more resources are needed, others should give back. Benefit others and user herself.
16
Long-Term Resource Fairness
• Satisfy Pay-as-you-use Fairness
Time
New Demand
A B
t1 20 100
AA BB
Accumulated resource usage:
Accumulated resource usage:
20
8080
2020 8080
Unsatisfied DemandUnsatisfied Demand
AA BB00 2020
17
Current Allocation at t1:
Current Allocation at t1:
AA
BB
Lend Resources:
Lend Resources:
3030
-30-30
Long-Term Resource Fairness
• Satisfy Pay-as-you-use Fairness
Time
New Demand
A B
t1 20 100
t2 40 60
AA BB
Accumulated resource usage:
Accumulated resource usage:
40
6060
6060 140140
AA BB
Unsatisfied DemandUnsatisfied Demand
00 2020
18
Current Allocation at t2:
Current Allocation at t2:
AA
BB
Lend Resources:
Lend Resources:
4040
-40-40
Long-Term Resource Fairness
• Satisfy Pay-as-you-use Fairness
Time
New Demand
A B
t1 20 100
t2 40 60
AA BB
Accumulated resource usage:
Accumulated resource usage:
40
6060
6060 140140
AA BB
Unsatisfied DemandUnsatisfied Demand
00 2020
19
Current Allocation at t2:
Current Allocation at t2:
AA
BB
Lend Resources:
Lend Resources:
4040
-40-40 t3 80 50
Long-Term Resource Fairness
• Satisfy Pay-as-you-use Fairness
Time
New Demand
A B
t1 20 100
t2 40 60
t3 80 50
AA BB
Accumulated resource usage:
Accumulated resource usage:
80
2020
140140 160160
AA BB
Unsatisfied DemandUnsatisfied Demand
00 5050
20
Current Allocation at t3:
Current Allocation at t3:
AA
BB
Lend Resources:
Lend Resources:
1010
-10-10
Long-Term Resource Fairness
• Satisfy Pay-as-you-use Fairness
Time
New Demand
A B
t1 20 100
t2 40 60
t3 80 50
AA BB
Accumulated resource usage:
Accumulated resource usage:
80
2020
140140 160160
AA BB
Unsatisfied DemandUnsatisfied Demand
00 5050
21
Current Allocation at t3:
Current Allocation at t3:
AA
BB
Lend Resources:
Lend Resources:
1010
-10-10
t4 60 50
Long-Term Resource Fairness
• Satisfy Pay-as-you-use Fairness
Time
New Demand
A B
t1 20 100
t2 40 60
t3 80 50
t4 60 50
AA BB
Accumulated resource usage:
Accumulated resource usage:
60
4040
200200 200200
AA BB
Unsatisfied DemandUnsatisfied Demand
00 6060
Long-Term Resource Fairness satisfy Resource-as-you-pay fairness.Long-Term Resource Fairness satisfy Resource-as-you-pay fairness.22
Current Allocation at t4:
Current Allocation at t4:
AA
BB
Lend Resources:
Lend Resources:
00
00
Other Properties of Long-Term Resource Fairness• Satisfy non-trivial workload and sharing incentives
– Running trivial workload can waste money.– Not sharing idle resource can waste money.
• Users cannot get benefits by lying (strategy proof).
23
Proof sketches are in the paper.
LTYARN
• Implement Long-Term Resource Fairness in YARN– Extend memoryless max-min fairness to long-term max-
min fairness.– Add a few components into resource manager• Support full long-term and time window-based
requirements.• Currently support a single resource type (main
memory).
24
LTYARN Design
• Quantum Updater (QU)– Estimates task execution time.– Updates the resource usage history periodically. • Resource Controller (RC)– Manages and updates resource for each queue.• Resource Allocator (RA)– Performs long-term resource allocation.– Runs when there are pending tasks and idle resources.
25
Evaluation
• A Hadoop Cluster– 10 nodes, each with two Intel X5675 CPUs (6 cores per
CPU with 3.07 GHz), 24GB DDR3 memory, 56GB hard disks.
– YARN-2.2.0, configured with 24GB memory per node.
• Macro-benchmarks– Synthetic Facebook Workload– Purdue Workload– HIVE/TPC-H– Spark
26Detailed setups are in the paper.
Metrics
• Evaluation metrics– Fairness degree for each user (>1 for sharing
benefits; <1 for sharing loss) – Resource-as-you-pay fairness– Application performance
• Benchmark scenario– The four macro benchmarks equally share the
cluster.– Each benchmark runs in a separate queue.– Window size =1 day.
27
Sharing Benefit/Loss
• LTYARN enables sharing benefits for all applications.
(b). LTYARN(b). LTYARN
29
(a). YARN(a). YARN
Resource-as-you-pay Fairness Results• LTYARN achieves resource-as-you-pay fairness.
30
Performance Results
31
• Sharing always achieves a better performance.• Long-term fairness is comparable to memory-less
fairness (max-min).
Conclusions
• Max-min resource fairness is memoryless and unsuitable for pay-as-you-use computing.
• We define long-term resource fairness that can satisfy the desirable properties.
• We develop LTYARN by integrating long-term resource fairness into YARN
– Homepage: http://sourceforge.net/projects/ltyarn/
32
We are Hosting IEEE CloudCom 2014 in Singapore
• Deadline for paper submissions: July 31, 2014• Notification of Paper acceptance: September 2,
2014• Conference: December 15-18, 2014
34
Thanks!
Question?
35