1

Incentive-Based Scheduling for Market-Like Computational GridsIncentive-Based Scheduling for

Market-Like Computational GridsLijuan Xiao, Yanmin Zhu, Member, IEEE, Lionel M. Ni, Fellow,

IEEE, and Zhiwei Xu, Senior Member, IEEELijuan Xiao, Yanmin Zhu, Member, IEEE, Lionel M. Ni, Fellow,

IEEE, and Zhiwei Xu, Senior Member, IEEE

Present by Ting-Wei, Chen

2

Index Introduction Problem Formulation The Incentive-Based Scheduling Scheme Performance Evaluation Conclusions

3

Introduction (cont.)

Market-like computational grid’s Characteristic– Allow providers and consumers to make autonomous

scheduling decisions– Have sufficient incentives to stay and play in the market

Formulate a intuition of optimizing incentives as dual-objective scheduling problem– Maximize the success rate of job execution– Minimize fairness deviation among resources

4

Introduction (cont.)

Present an incentive-based scheduling scheme– IB– Peer-to-peer decentralized scheduling framework– A set of local heuristic algorithms– Market instrument

• Job announcement

• Price

• Competition degree (CD)

5

Introduction (cont.)

Object– Make autonomous decisions – Producing a desirable emergent property in the

grid system

6

Problem Formulation (cont.)

Define a market-like computational grid Four-tuple G=(R, S, J, M)

Provider R0

Capability C0

Queue Q0

Provider Ri

Capability Ci

Queue Q i

Provider Rm-1

Capability Cm-1

Queue Qm-1

Consumer S0 Consumer Sj Consumer Sk-1

Scheduling Scheme M

Send job Resource price bidJ ob announcement

J ob announcement Resource price bid Receive job

7

Problem Formulation (cont.)

Consumers and jobs– Only consider computation-intensive jobs– Job announcement

• Job length• Job deadline

Providers and resources– Be modeled with three parameters

• Capability• Job queue• Unit price

8

Problem Formulation (cont.)

Incentives for consumers and providers– High quality of computational service at low cost– High successful-execution rate of jobs– Successful-execution rate θ

1, if

0, if

0 ,i n i wheren

i i iC DT T

i iC DT T

9

Problem Formulation (cont.)

Fairness deviation σ of the grid system

where

Maximize θ Minimize σ

0 1

0

0

_ ( ,..., )

/

/

m

j l m lj

j l m l

std dev

P P

C C

10

The Incentive-Based Scheduling Scheme (cont.)

Characterized– Consumer or provider autonomously makes

scheduling decisions– Scheduling algorithms are local to a resource

provider– Three market instruments

• Job announcement

• Price

• CD

11

The Incentive-Based Scheduling Scheme (cont.)

Peer-to-Peer Scheduling Framework– Decentralization– Scalability– Dynamics of grid environments

12

The Incentive-Based Scheduling Scheme (cont.)

– Computational grid G• Via one of which a provider can join the grid

• Get the information of designated neighbors

• Connect into the P2P network

– Consumer• Submit a job announcement to the computational grid

• Spread throughout the P2P network

• Provider receive a job announcement

13

The Incentive-Based Scheduling Scheme (cont.)

– Realize the complete competition • All providers should have an equal chance to compete

for any job

• The number of providers will not be too large

– Solve the Blind-flooding-based broadcasting’s problem

• Building overlay networks

• Efficient broadcasting mechanism

14

The Incentive-Based Scheduling Scheme (cont.)

J ob Announcement

Result

Bid

Bid

Bid

P2P Network

Step 1

Step 3Step 2

Step 2

Step 2

J obStep 4

15

The Incentive-Based Scheduling Scheme (cont.)

Incentive-Based Scheduling Algorithms

– Job competing algorithm• Provider receives a job announcement

16

The Incentive-Based Scheduling Scheme (cont.)

• Step 1

The provider estimates whether it is able to meet the job deadline

1 if TL > TA then2 can meet ←true;3 reordered ← false;4 insert place ← Pq;5 else // TL is covered by the execution of Ji in the queue6 if insert s at Pi-1, none of Ji ~Jq will miss its deadline then7 can meet ← true;8 reordered ← true;9 insert place ← Pi-1;10 else11 can meet ← false;12 endif13 endif

The Incentive-Based Scheduling Scheme (cont.)

• Step 2

The provider offers a price for the job

• Step 3

The provider sends the price as a bid and inserts the job at the place that the variable insert_place indicates at the probability of 1-CD.

17

1 price ← p*Ls;2 if reordered then3 price ← λ*price;4 endif

18

The Incentive-Based Scheduling Scheme (cont.)

– Heuristic Local Scheduling Algorithm• Punishment mechanism• How much time the completion time TC exceeds the

deadline TD

• Every time a provider is offered a job that is not kept in the job queue

The Incentive-Based Scheduling Scheme (cont.)1 insert place ← Pq;

2 penalty ← calculate the penalty of inserting the job at Pq;

3 for i ← q-1 to 0 do

4 penaltyi ← calculate the penalty of inserting the job at Pi;

5 if penaltyi < penalty then

6 penalty ← penaltyi;

7 insert_place ← Pi;

8 endif

9 endfor

10 insert the job at insert_place

19

The Incentive-Based Scheduling Scheme (cont.)

– Price-Adjusting Algorithm• Price mechanism

– Make prices different

– Differentiate the chances of providers to be chosen

– Eventually realize the fair allocation of profits

• All the providers need to know some global information

• Every time a provider is offered a job or deletes an unconfirmed job, then start the price-adjusting algorithm

• The price will fluctuate around the market price

20

The Incentive-Based Scheduling Scheme (cont.)1 r1 ← LO/LT ;

2 r2 ← C/∑0≤j<m Cj;

3 if offered a job then

4 if r1 > r2 and p <=PM then

5 p ← α*p;

6 endif

7 else // delete an unconfirmed job

8 if r1 < r2 and p >=PM then

9 p ← β*p;

10 endif

11 endif

21

The Incentive-Based Scheduling Scheme (cont.)

– Competition-Degree-Adjusting Algorithm• Keep unconfirmed jobs in their job queues

22

//Every time the penalty increases1 if Rp >=THp and CD >=ε then2 CD ← CD - ε;3 endif

//Every time a certain interval such as 1 day1 if Rp < THp and RJ >=THJ and CD <=1- ε then2 CD ← CD+ ε;3 endif

23

Performance Evaluation (cont.)

Evaluation Methodology– Discrete event-driven simulator– Mainly drive:

• The network delay of communication (Ignore)

• Job execution

– Four experiments• The impact of CD on performance

• Analyzes the incentive-based scheduling scheme by disabling the CD-adjusting algorithm

• Compare IB, FCFS, SJF, EDF, and FirstReward under synthetic workloads and real workloads

24

Performance Evaluation (cont.)

– System load• Over a period of time T is defined as the ratio of the

aggregated length of jobs submitted to the aggregated job length that the computational grid is capable to execute.

• System load varies from 0.1 to 0.7

0

0*i n i

j m j

L

T C

Performance Evaluation (cont.)

– Consumers• Job generation is modeled as a Poisson process

• The deadline is uniformly distributed as well

– Providers• Distributed on account of the observation

• Predominate in computer market

25

26

Performance Evaluation (cont.)

Four experiment– Impact of CD on performance

• Failure Rates of Jobs

27

Performance Evaluation (cont.)

• Deadline Missing Rates of Jobs

• The successful-execution rate θ can be calculated

(1 )*(1 )

28

Performance Evaluation (cont.)• Successful-execution rates of jobs

– The conservative attitude (CD=0) toward competing for jobs is nota desirable one, considering the successful-execution rate.

29

Performance Evaluation (cont.)

• The Total Penalty of Providers

Performance Evaluation (cont.)

• Total profit of providers

31

Performance Evaluation (cont.)

– Analyzes the incentive-based scheduling scheme• Analysis of IB on the successful-execution rate

32

Performance Evaluation (cont.)

– Compare our scheme with four other schemes under synthetic workloads

• FCFS

• SJF

• EDF

• FirstReward

• IB

33

Performance Evaluation (cont.)

• Comparison on the successful-execution rate

34

Performance Evaluation (cont.)

• Comparison on the fairness deviation

35

Performance Evaluation (cont.)

– Compare our scheme with four other schemes under real workload

• Chose the LPC EGEE (Laboratoire de Physique Corpusculaire Enabling Grids for E-sciencE) trace

36

Performance Evaluation (cont.)

• Fair profit allocation of IB versus unfair profit allocation of EDF

37

Performance Evaluation (cont.)

• Balanced utilization of IB versus imbalanced utilization of EDF

38

Conclusion (cont.) Incentive-based scheduling scheme IB feature

– Consumer and provider autonomously makes scheduling decisions

– All scheduling algorithms are local to a resource provider

– Three market instruments

39

Conclusion (cont.)

Advantages– Participant makes local/autonomous decision– High successful-job-execution rate– Fair allocation of profits– Balanced utilization of resource

40

Thank you for your attention