Resource allocation in a Cloud partially powered byrenewable energy sources

Supervisor:Anne-Cecile Orgerie (CNRS)Jean-Marc Menaud (EMN)

Myriads - IRISA - Rennes

Yunbo LI - March 17, 2015

1 / 30

Project EPOC

Figure : Collaboration

Project EPOC: EnergyProportional and OpportunisticComputing system.

4 Phd students

5 partners

6 teams

! It aim at optimizing energyconsumption at the level ofhardware, software, network and thetrade-o↵ between energy costs andperformance.

2 / 30

My subject

My subject: Resource allocation in a Cloud partially powered byrenewable energy sources.

! At the infrastructure level, by designing an energy-aware distributed system incharge of optimizing the energy consumed by the infrastructure running the jobs.

3 / 30

Analysis the energy consumption in datacenters

Figure : Energy consumption in datacenters

How can we reduce the energy consumption in datacenters?

! Point in common: Minimize the number of powered-on PhysicalMachines (PMs)Ref: Eugen Feller Phd thesis

4 / 30

Problem

Actual

Survey from Google: 5000 servers, average CPU utilization [10%, 50%] [1].An idle server consumes between 50% and 80% of its peak power[2].

How can we reduce the energy consumption in datacenters?! Increase the CPU/RAM utilization for each Physical Machine (PM)and switch-o↵ the unnecessary powered-on PMs.

Advantage:

More green by using the solar/wind power

Reduce CO2 emission footprint

Reduce brown energy consumption and save money

[1]Luiz Andr e Barroso and al. The case for energy-proportional computing. Computer, 40 :33–37, December 2007.[2]Stephen Dawson-Haggerty and al. Power optimization, a reality check. Technical Report UCB EECS-2009-140, EECSDepartment, University of California, Berkeley, Oct 2009.

5 / 30

System assumption

In a mono-site [50 servers]. PMs hardware can be eitherhomogeneous or heterogeneous.

Three kinds of modes to provide power (Green energy, brwon energy,hybrid)

No batteries to store electricity

6 / 30

Job scheduling

When

Time is divided into slot.Job classification:

1 web-job - Non-Interruptible2 batch-job - Interruptible within deadline

Schedule/Reschedule jobs at the beginning of each slot:

Schedule web-job first (Algo-webjob)Then schedule the batch-job (Algo-batchjob)If there is not enough green energy (Algo-consolidation if necessary)

Where (e.g. #node/#server) The chose serve should satisfy job’sdemand

Constraints of servers:1. CPU resource (e.g. number of cores)2. Memory resource

7 / 30

The system model

JobUser Job filterWeb Job

Batch Job Global Manager

Local Manager VMM

VM1 VM2 VM n...

PM 1

Local Manager VMM

VM1 VM2 VM n...

PM m

...

web job/batch job/consolidation algorithm

Figure : The system model

8 / 30

Switch ON servers

ON# ON# OFF# OFF# OFF#

0/10 0/10 0/10 0/10 0/10

OFF# OFF#

0/10 0/10

OFF# OFF# OFF#

0/10 0/10 0/10 Receive the green energy and Compute the #servers to switch on

Figure : Switch ON servers

9 / 30

Job placement algorithm

4" 4" 3" 2"4"

New Arrival

OFF" OFF" OFF"

0/10 0/10 0/10 0/10 0/10

6"

batch"web"

Reject:

Place the first web-job:

Figure : Job placement algorithm

10 / 30

Job placement algorithm

4" 4" 3" 2"4"

New Arrival

OFF" OFF" OFF"

6/10 0/10 0/10 0/10 0/10

6"

batch"web"

Reject:

Figure : Job placement algorithm

11 / 30

Job placement algorithm

4" 4" 3" 2"

New Arrival

OFF" OFF" OFF"

10/10 0/10 0/10 0/10 0/10

6"

batch"web"

Reject:

4"

Figure : Job placement algorithm

12 / 30

Job placement algorithm

4"4/10

4" 3" 2"

New Arrival

OFF" OFF" OFF"

10/10 0/10 0/10 0/10

6"

batch"web"

Reject:

4"

Figure : Job placement algorithm

13 / 30

Job placement algorithm

4"8/10

4"

3" 2"

New Arrival

OFF" OFF" OFF"

10/10 0/10 0/10 0/10

6"

batch"web"

Reject:

4"

Place the first batch-job:

Figure : Job placement algorithm

14 / 30

Job placement algorithm

3"Reject:

4"8/10

4"

2"

New Arrival

OFF" OFF" OFF"

10/10 0/10 0/10 0/10

6"

batch"web"

4"

Figure : Job placement algorithm

15 / 30

Job placement algorithm

3"Reject:

4"10/10

4"2"

New Arrival

OFF" OFF" OFF"

10/10 0/10 0/10 0/10

6"

batch"web"

4"

End

Figure : Job placement algorithm

16 / 30

Switch OFF servers

Figure : Switch OFF servers

17 / 30

Switch OFF servers

Figure : Switch OFF servers

18 / 30

Switch OFF servers

Figure : Switch OFF servers

19 / 30

Switch OFF servers

Figure : Switch OFF servers

20 / 30

Switch OFF servers

Figure : Switch OFF servers

21 / 30

Switch OFF servers

Figure : Switch OFF servers

22 / 30

Switch OFF servers

Figure : Switch OFF servers

23 / 30

Switch OFF servers

Figure : Switch OFF servers

24 / 30

Switch OFF servers

Figure : Switch OFF servers

25 / 30

Solar power (1)

Figure : 4 sites’ coordinate in France

History from the 1 jan 2005to 31 dec 2012

26 / 30

Solar power (2) - Real trace

0

100000

200000

300000

400000

500000

600000

700000

800000

900000

1e+06

04/0104:00

04/0108:00

04/0112:00

04/0116:00

04/0120:00

04/0200:00

Pow

er (W

atts

)

time

CPUMhz, GreenEnergy from 01-04-2014 to 02-04-2014

Workload energy consumptionGreen energy

Figure : 2009 solar radiation, site North near Calais

Database provided by EasyVirt.27 / 30

Validation

End of development a simple simulator to valid our algorithm.Code source : JavaAfter valid this prototype, we will experimenter it on Grid’5000.

28 / 30

Conclusion

Topic: Resource allocation in a Cloud partially powered by renewableenergy sourcesGeneral problem to solve :

Resource allocation

Maximize on using the renewable energy instead of the Brown energy .

For VM consolidation, our research is focus on the three following points.

VM placement ! The server manager rank the PM and VM bycombining di↵erent weighting factors.

Reduce the number of powered-on physical machines by increasingthe VM consolidation ratio

29 / 30

The EndMerci:)

30 / 30