greensoftware : managing datacenters powered by renewable energy
DESCRIPTION
GreenSoftware : Managing Datacenters Powered by Renewable Energy. Íñigo Goiri, William Katsak, Md E Haque , Kien Le, Ryan Beauchea, Jordi Guitart, Jordi Torres, Thu D. Nguyen, Ricardo Bianchini Department of Computer Science . Motivation. Datacenters consume large amounts of energy - PowerPoint PPT PresentationTRANSCRIPT
GreenSoftware:Managing Datacenters Powered by Renewable
Energy
Íñigo Goiri, William Katsak, Md E Haque, Kien Le,Ryan Beauchea, Jordi Guitart, Jordi Torres,
Thu D. Nguyen, Ricardo Bianchini
Department of Computer Science
2
Motivation• Datacenters consume large amounts of energy• High energy cost and carbon footprint– Brown electricity: coal and natural gas
• Connect datacenters to green sources: solar, wind
Apple DC in Maiden, NC 40MW solar farm
Green datacenter
3
Challenges and opportunities
• Scheduling workload/energy sources– Lower costs: brown energy, peak brown power, capital
• Study opportunities in green datacenters– Build hardware/software
Power
Time
Load
VariableSolar power
Workload
4
GreenSoftware
How to build software for green datacenters?1. Malleable energy demand– Idle nodes → Turn off/Sleep (S3) [COLP’01]– Reduce frequency (DVFS) → Lower quality
2. Move computation under renewables– Weather forecast → Green energy forecast– Delay computation or degrade quality– Leverage energy storage
5
Outline
• Motivation• GreenSoftware– GreenSlot– GreenHadoop– GreenSwitch– GreenCassandra– … and others
• Conclusion
6
GreenSlot [SC’11]
• Batch jobs on SLURM (& Hadoop)• Send idle nodes to S3• Predict solar availability• Delay jobs within deadlines– Known jobs characteristics (length, deadline, size…)– Heuristic
Time
Job 1 Power
Deadline
Job 2
Job 3
Job 4
7
GreenSlot [SC’11]
• Batch jobs on SLURM (& Hadoop)• Send idle nodes to S3• Predict solar availability• Delay jobs within deadlines– Known jobs characteristics (length, deadline, size…)– Heuristic
Time
Job 1 Power
Deadline
Job 2 Job 3
Job 4
8
GreenHadoop [Eurosys’12]
• Batch jobs on Hadoop• Send idle nodes to S3• Make required data available– Move data blocks
• Predict solar availability• Delay jobs within deadlines– Predict global jobs energy consumption– Heuristic
Map1Map2Map3Map4Map5
Reduce
Reduce 6
7
Shuffle
9
Covering subset
GreenHadoop: Data management• Deactivate servers to save energy
– Some data might become unavailable• Prior solution: covering subset [Leverich’09]
– Set of servers always running has ALL data
7
3
45
21 6
8
7 1
4 5
6
3
2
8 1
7 3
• Our approach• Only required data has to be available• We usually require fewer active servers
Server
Block
10
GreenHadoop: Data managementServer 1
1 72Active
Decommission
Down
Server 24
356
Server 3
46
Required fileNon-required file
Server 42
3 84
Server 5
3 67
JobA 4
JobB 5
JobC 1
6
Running queue:
11
GreenHadoop: Data management
Server 42
3 84
Server 5
3 67
Active
Decommission
Down
GreenHadoop (computation) requires only 2 servers
Server 1
1 72
Server 1
1 72
Server 24
356
Server 3
46
Required fileNon-required file JobA 4
JobB 5
JobC 1
6
Running queue:
12
GreenHadoop: Data management
Active
Decommission
Down
Move required files to Active servers
Server 1
1 72
Server 24
356
Server 3
46
1
Server 42
3 84
Server 5
3 67
ReplicateJobA 4
JobB 5
JobC 1
6
Running queue:
13
Server 1
1 72
GreenHadoop: Data management
Active
Decommission
Down
Decommissioned server can be sent to Down
Server 1
1 72
Server 24
356
Server 3
46
Required fileNon-required file
1
Server 42
3 84
Server 5
3 67
JobA 4
JobB 5
JobC 1
6
Running queue:
14
Server 1
1 72
GreenHadoop: Data management
Active
Decommission
Down
Jobs to be executed change → Required files change
Server 24
356
Server 3
46
Non-required file
1
Server 42
3 84
Server 5
3 67
JobA 4
JobB 5
JobC 1
6
JobD 8
Required file
646
4
648
Required file
Running queue:
15
Server 42
3 84
Server 1
1 72
GreenHadoop: Data management
Active
Decommission
Down
Make missing data available
Server 24
356
Server 42
3 84
Server 5
3 67
Server 3
46
1
Required file
Non-required file
JobB 5
JobC 1
JobD 8
Required fileRunning queue:
16
Server 42
3 84
Server 1
1 72
GreenHadoop: Data management
Active
Decommission
Down
Server 24
356
Server 42
3 84
Server 5
3 67
GreenHadoop (computation) requires 3 servers
Server 3
46
1
Non-required file
JobB 5
JobC 1
JobD 8
Required fileRunning queue:
17
GreenSwitch [ASPLOS’13]
• Batch jobs on Hadoop• Similar to GreenHadoop• Energy storage– Battery– Net metering
• Schedule workload and energy sources– Optimization
• Evaluation on Parasol(Presented on Monday by Thu)
18
GreenCassandra
• Distributed DB/storage on Cassandra• Add an optional ring
• Degrade quality when no green
1
4
6 2
35
DHT Ring
A
A
A
1
4
35
6 2
DoubleDHT Ring
Optional
AA
Server
Data
19
GreenSoftware summaryType Malleable energy Green adaptability
GreenSlot Batch jobs Delay jobsSleep servers Delay until green
GreenHadoop Batch jobsDelay jobsSleep serversData management
Delay until green
GreenSwitch Batch/interactive jobs Delay jobsSleep servers
Delay until greenEnergy storage
GreenCassandra Distributed storage Optional ring Degrade quality
GreenSLA VMs Migrate VMsSleep servers
Route green energyto racks
GreenPar MPI jobs Change parallelismSleep servers
Greater parallelismon green
GreenScale Non-deferrable jobs CPU and mem DVFS Faster on green
GreenNebula Geo distributed VMs Migrate VMs “Follow the renewables”
20
Conclusions
• Green datacenters– Challenges & opportunities– Hardware/software solution
• GreenSoftware– Adapt software to green datacenters– Malleable energy demand– Match computation and renewables
GreenSoftware:Managing Datacenters Powered by Renewable
Energy
Íñigo Goiri, William Katsak, Md E Haque, Kien Le,Ryan Beauchea, Jordi Guitart, Jordi Torres,
Thu D. Nguyen, Ricardo Bianchini
Department of Computer Science
22
Other GreenSoftware• GreenSLA [IGCC’13]
– Bringing green energy to users– New hardware to route green energy
• GreenPar– MPI jobs with sub linear speedup– Use “Free” green energy
• GreenNebula– VMs in multiple geo distributed datacenters– Follow the sun
• GreenScale– Change frequency (DVFS)
27
Parasol without GreenSwitch
Green use
Green available
Net metering
Brown use
IT load
28
GreenSwitch: deferrable workload
Battery discharge
Battery charge
IT load
Net meteringGreen available
Green use