1

Request Dispatching for Cheap Energy Prices in Cloud Data CentersHiroshi Yamada (TUAT)In cooperation withTakumi Sakamoto( takus@sslab.ics.keio.ac.jp ), Hikaru Horie, Kenji Kono(Keio Univ.)1Cloud Service Providers Make use of geographically-distributed data centers Offer users a large amount of resources in a pay-as-you-go manner Enable us to easily use resources as needede.g. Amazon has more than tens of DCs over the world2

TokyoLondonDallasAmsterdam

MiamiSeattleCloud platforms are very common. The current could service providers, like Amazon, Google, Microsoft, commonly make use of geographically-distributed data centers. They orchestrate these data centers to provide users a large amount of resources. Cloud users can utilize them in a pay-as-you-go manner, which means they can easily use resources as needed.

For example, Amazon has more than tens of DCs over the world. It has DCs in London , Tokyo, US Dallas, Seattle and so on.If a cloud user need more resources, he or she can use these DCs resources to provide the services.2Energy cost is EXTREMELY high Millions of dollars of electric costs per year [Qureshi+ SIGCOMM 04] Google spends $38 million on electricity Other large companies pay tens of million dollars

Significant financial overhead in the management cost Energy prices can be higher and higher Limited fossil fuel Restriction to build nuclear power plants Lead to higher charges for the use of clouds3In the management of these DCs, energy cost is a significant concern. It is extremely high recently.Some document reports that energy costs are equal to millions of dollars per year.For example, google annually spends 38 million dollars for electricity. Additionally, other large companies pay tens of million dollars for electricity.

This means that energy cost is significant financial overhead in the management cost.Unfortunately, energy prices tend to become more and more expensive due to limited fossil fuel and restriction to build nuclear power plants. This situation is more severe in Japan.

This fact can lead to higher charges for the use of clouds so we want to avoid this situation.3Existing solution for reducing energy costs Forwards client requests to DCs in cheaper energy price regions [Qureshi+ SIGCOMM 09] Average utilization in DCs is 2030% Servers can be replicated in different DCs4

Energy prices are fluctuated over time in the US.The previous work has tackled this problem. To reduce energy cost, the previous work dispatches client requests to DCs in cheaper energy price regions.The insight of this work is to pay attention to the fact that energy prices are different in different places. And, the average utilization in DCs is 20 to 30 % for successfully handling request bursts, and server instances can be easily replicated among DCs like Amazon EC2.

This graph shows energy prices change over time in the US. This shows three places energy price, California, Midwest, and New York.X-axis is time, and Y-axis is electric price.We can see that electric price change over time, and the price is different in each place.

4Challenges for dispatching client requests in clouds DCs capacities The previous work dispatches requests only to the DC in the cheapest energy price region Even if the DC is overloaded Service qualities can be degraded Lose 1 % income if the response time is 10 % increased in Amazon [Kohavi+ 07]

Various factors to be considered DC loads, response time, Cannot dispatch requests based on only energy prices

5However, the existing solution cannot be applied to clouds. There are two challenges in energy-price-driven request dispatching.

First is DCs capacities. The existing solution just consider energy price, so it dispatches requests to only the DC in the cheapest energy price region even if the DC is overloaded, which means the services running on the DC are severely degraded. And the DC cannot handle suddenly-occurred request burst.This is critical for services. For example, if response time is 10 % increased in Amazon, income is 1 % decreased.

Second is to consider various factors in dispatching requests in clouds. For example, such as DC loads, response time, and so on.Since the existing work takes into account only energy prices, these demands are not satisfied. 5Proposal Dispatching client requests to DCs in cheaper energy price regions, considering: DCs capacities Various factors such asDC loads and response time

Assumptions Know the real-time energy prices that vary hourly Use a function to automatically launch servers inside DCs like Amazon auto scaling function6

Dispatcher$1 / MWh$2 / MWh$3 / MWhRequestsTo address this issue, we presens an approach to dispatching client requests to DCs in cheaper energy price regions, considering DCs capacities and additional factors such as DC loads and response time.

Our dispatcher receives client requests, and dispatches them to DCs in cheaper energy price regions, satisfying predefined demands such as DC loads.

We have the same assumptions with Qureshis SIGCOMM paper.First we know the real-time energy prices that vary hourly.Second, we use a function to automatically launch servers inside DCs like Amazon auto-scaling function.6Contributions Propose a mechanism to dispatch requests to save energy prices, satisfying pre-defined demands

Show an example algorithm that satisfies pre-defined demands and can be integrated to our dispatcher

Implement our prototype and perform simulation evaluation using real data7Our contributions are here.

We propose a mechanism to dispatch requests to save energy prices with pre-defined demands satisfied.

We also show an simple algorithm that satisfies pre-defined demand, DC loads and response time, and can be integrated to our dispatcher.

Finally, we implement our prototype and evaluate it in simulation using realistic data.7Design of request dispatcher Requirements Scalability Has to handle a huge amount of requests sent to DCs Ability to follow frequent-redirection-changes Energy price and DC loads are changed frequently Availability Easy to fail to redirect requests if the dispatcher is fragile

Solution: Mapping Nodes Run between clients and DCs in a P2P fashion Satisfy the three requirements

8To design our dispatcher, we have to consider three requirements. First is scalability. The dispatcher has to successfully handle a huge amount of requests sent to DCs.

Second, the dispatcher needs to follow frequent-redirection-changes since energy price and DC loads are changed frequently. So, it needs to have an ability to do it.

Third is availability. If the dispatcher is fragile, client request redirection is easily failed.

To achieve the requirements, we prepare mapping nodes that run between clients and DCs in a P2P fashion. 8Mapping Node Constructs a distributed hash table (DHT) Key: a host name, Value: Mapping Entry Mapping entry contains the hosts IP and RTT (for cache) Returns IP address like DNS servers Clients use the returned address to connect9

Mapping Node in details.

Mapping nodes work together. They construct a distributed hash table (DHT) to achieve the three requirements I said early, availability, scalability and so on. They work like a DNS server for client.

Mapping nodes manage a host name as a key, and mapping entry as a value. Mapping entry is a data object that contains hosts IP and RTT for cache in a mapping node. Mapping node returns an IP address to the client when its request is received.

First, client sends a request to a known mapping node to get an IP address of the server. Mapping node searches the mapping entry of the given server name.It returns the IP address to the client after finding the mapping entry.9A Request Dispatching Algorithm Used for mapping nodes to return IP addresses Calculates dispatching request ratio in each DC A DC to forward is decided based on the ratio Cloud service providers can change weights10

Example: request dispatching based on DC load, response time, and energy pricesweightDC loads. Weight is decreased as DC loads are more severeResponse time. Weight is 0 when latency is over 70 msecEnergy prices. Weight is smoothly decreased as the price is higher In this work, our mapping node employs this request dispatching algorithm to decide IP addresses to be returned.This is very simple so there are a lot of rooms to improve and can borrow novel algorithms from various previous work. But an important point is that we can develop an algorithm that is applicable to mapping nodes.

This algorithm decides percentages of requests to send to a DC, following pre-defined threshold. It considers three factors, DC load, response time, and energy prices. Each factor can be set weight that is adjusted based on which factor is more important.

In this algorithm, the weigh changes in a situation. We set these functions to reduce percentages when response time and DC loads are over the threshold. 10Simulation Evaluation Implement a prototype on Overlay Weaver [Shodo+ 06] DC emulator: CloudSim [Buyya+ 09] Parameters Energy prices data: 1st week in the US on Aug. 2010 Workloads: E-commerce DC workload, borrowing from [Heller+ NSDI 10] Constraints (Threshold) DC loads: less than 80 % Response time: less than 70 ms11RandomDispatch requests randomlyLatency & PriceConsider response time and energy priceLatency & LoadConsider response time and DC loadsOur ProposalConsider response time, DC loads, and energy price (shown in the previous slide)Compared with the following policies

We implemented our prototype on Overlay weaver that is a DHT framework and evaluate it on a DC emulator named CloudSIm.

We also used the following parameters. We used energy price data for the first week in the US on August 2010, and E-commerce DC workload borrowed from Hellers NDSI paper.

In addition, we set thresholds for DC loads and response time to 80% and 70 msecond respectively.

For comparison, we prepared 4 request dispatching policies. Random, Latency & price, Latency & load, and our proposal.Random dispatches request randomly, Latency & price consider response time and energy price, Latency & load considers response time and DC loads,And, our proposal consider all.11ResultEnergy price and constraint violation Our proposal lowers energy prices without the constraint violation 28 % better in energy prices than Random Saves several hundred dollars if annual energy cost is several thousand dollars12Energy price nomalized by RandomConstraint ViolationLower is betterLower is betterThe results are here.

The left graph is energy price, and the right graph is constraint violation which means whether DC loads and response time are over the threshold.

We can see that our proposal successfully lowers energy prices without constraint violation. Random and Latency & Price violate the constraints.Specifically, our proposal is 28 % better in energy prices than Random, which means it saves several hundred dollars if annual energy cost is several thousand dollars.12ResultDC loads Our proposal successfully regulates DC loads DC loads in Latency & Price are fluctuated severely13

Latency & PriceLoads are sharply fluctuated.Our ProposalLoads are changed along with the workload.RandomLoads are changed along with the workload.Latency & LoadLoads are changed along with the workload.Threshold = 80%Threshold = 80%Threshold = 80%Threshold = 80%1234567Day1234567Day1234567Day1234567DayThe results of DC loads are like this.

We can see that our proposal almost successfully regulates DC loads under the threshold. In latency & price, the loads are severely fluctuated since it does not consider any DC loads situation.13Related Work Energy-price-driven request dispatcher [Qureshi+ SIGCOMM 09] Dispatches requests to DCs in a cheap energy price region Only consider energy price

Load-balancer for DCs [Wendell+ SIGCOMM 10] Dispatches requests to DCs based on their loads Does not consider energy prices

DHT-based DNS[Ramasubramanian+ SIGCOMM 04] Achieves fault-tolerant and high performance DNS using DHT Not focus on energy prices and cloud user policy14Related work.

Qureshi proposes energy-price-drive request dispatcher.However, as I said, it considers only energy prices so it cannot take into account other factors like response time and DC loads and so on.

The Load-balancer for DCs dispatches requests to DCs based on their loads.This balancer does not consider energy prices.

DHT-base DNS is proposed and it achieves fault-tolerant and high performance DNS using DHT.We can use this techniques to mapping node to enhance it.14Conclusions Proposal: Energy-price-driven request dispatcher for clouds Takes response time and DC loads into account Leverages a DHT mechanism

Our simulation result shows that our dispatcher saves energy costs without violation of pre-defined constraints

Future work Enhance policies used in mapping nodes Evaluate our prototype in the real world15Conclude this talk.

We propose an energy-price-driven request dispatcher for clouds that can take into account various factors like not only energy prices but DC loads, response time and so on.It leverages a DHT mechanism to achieve its scalability and availability.

Our simulation result shows that our dispatcher saves energy costs without violation of pre-defined constraints.

Future work includes to enhance request dispatching policies used in mapping nodes, and to evaluate our prototype in the real world.

15ResultMapping node execution time 16LatencyMeanMedian90th %Legacy DNS (original)382 ms39 ms337 msLegacy DNS (revised)246 ms30 ms163 msMapping Node102 ms52 ms204 ms[ms]