Lecture 4: Power Provisioning

Prof. Fred Chong290N Green Computing

Power Provisioning

• $10-22 per deployed IT Watt• Given 10 year depreciation cycle

– $1-2.20 per Watt per year• Assume $0.07 per kilowatt-hr and PUE 2.0

– 8766 hours in a year– (8766 / 1000) * $0.07 * 2.0 = $1.22724

• Up to 2X cost in provisioning– eg. 50% full datacenter = 2X provisioning cost

Power Distribution Revisited

Measured Load vs Power

Modeled vs. Measured PDU Power

Methodology

Workloads

• Websearch – high request throughput and large data size

• Webmail – high I/O• Mapreduce – large offline batch jobs

Websearch Results

Webmail Results

Mapreduce Results

Mixed Load

Real Datacenter

Time at Power Level

80 servers800 servers8000 servers

Oversubscription Opportunity

• 7% for racks (80)• 22% for PDUs (800)• 28% for clusters (8000)

– Could have hosted almost 40% more machines

Power Capping

Observed Power

CPU DVS

Idle Power

Energy Savings

Underdeployment

• New facilities plan for growth• Also discretization of capacity

– Eg 2.5kW circuit may have four 520W servers• 17% underutilized, but can’t have one more

Modeling Costs

TCO = datacenter depreciation + datacenter opex +server depreciation + server opex

$ per critical watt

Cost / W

Source

$12-25 Uptime Institute; the lower value is for “Tier 1” designs that are rarely used in practice [http://www.upsite.com/TUIpages/downloads/TUI808DollarsPerkW_WP.pdf]$10 Microsoft’s purchase of two 10MW datacenters in for $200M; this cost excludes the value of land and buildings http://www.savvis.net/corp/News/Press+Releases/Archive/SAVVIS+Sells+Assets+Related+to+Two+Data+Centers+for+200+Million.htm

$10-16 Dupont Fabros S-1 filing, discussing plans to build several 18MW datacentershttp://www.secinfo.com/d14D5a.u5dFg.htm (page 6). A more recent article (http://www.reuters.com/article/pressRelease/idUS12552+06-Nov-2008+PRN20081106?symbol=DFT.N), shows their facility having reached just over $10/W.

Case A

• Dell 2950 III EnergySmart– 16GB of RAM and 4 disks– 300 Watts – $6K

Assumptions

• The cost of electricity is the 2006 average US industrial rate ay 6.2 cents/kWh.

• The interest rate a business must pay on their loans is 12%.• The cost of datacenter construction is $15/W amortized

over 12 years.• Datacenter opex is $0.03/W/month.• The datacenter has a PUE of 2.0.• Server lifetime is 4 years, and server repair and

maintenance is 5% of capex per year.• The server’s average power draw is 75% of peak power.

Cost Breakdown A

Case B

• higher-powered server– 500W – $2K

• energy cost of $0.10/kWh • datacenter related costs rise to 46% of the total• energy costs to 25%• server costs falling to 31%. • hosting cost of such a server, i.e., the cost of all

infrastructure and power to house it, is more than twice the cost of purchasing and maintaining the server.

Cost Breakdown B

Utilization

• CPU Utilization of 50% => 75% Peak Power• Nameplate 500W server

– with all options (max mem, disk, PCI cards)– but more commonly 300W– Thus 60% utilized => 1.66x OPEX

• Vendor power calculator assumes 100% CPU utilization

•

Power Provisioning Problems

• Assume 30% CPU utilization and provision power accordingly– 200W instead of 300W– Variations could cause server to overhead or trip a

breaker– Adding memory or disk would require physical

decompaction of racks• Thus 20-50% slack space common

– Eg 10MW provisioned power => 4-6 MW actual power (plus PUE overhead)

Case B with 50% Occupancy

Partial Utilization Costs

• Partially utilized servers use less power – Appear to cost less in OPEX cost per server– But produce less value in terms of applications

• Need metric for application value– Eg number of transactions, number of web searches– Divide TCO by metric– Eg TCO = $1M/month, 100M transactions/month => 1

cent / transaction– Eg TCO = $1M/month, 50M transactions/month => 2

cents / transaction (2X cost)