Towards VM Consolidation Using Idle States

Rayman Preet Singh, Tim Brecht, S. Keshav University of Waterloo

@ACM VEE ‘15

1  

Traditional VM Consolidation

•  Re-package and save

•  Power(idle machine) > 50% of peak power [Gandhi ‘09]

Hardware  

Hypervisor  

VM  VM  

Hardware  

Hypervisor  

VM   Hardware  

Hypervisor  

VM  VM   VM  

2  

Consolidating Further

•  More inactive states

–  Frozen [LXC]

–  Substrate [Wang ‘11], Fast-resume [Zhang ’11]

Booted

Inactive

VM  

VM  

VM  

VM  

3  

VM  

VM  

1

Inactive 2

VM  

VM  

VM  

VM  

VM  

VM  

VM  

VM  

VM  

Example: DreamServer

•  Web-hosting •  Density improvement: over 46%

•  Miss penalty: ~1 sec

Booted

VM  Suspended

VM  

VM  

VM  

VM  

VM  

VM  

VM  

VM  

[Knauth et al. DreamServer: Truly On-Demand Cloud Services. SYSTOR ’14]  

4  

Question

Goals –  Maximize VM density

–  Minimize average miss penalties

What policy should we adopt to manage VMs across the different inactive states?

5  

Low Duty-cycle Workloads

•  High idle times

•  Relatively uncorrelated active times

•  Only a small fraction simultaneously active –  Large fraction inactive

VM 1

VM 2

25 % duty-cycle

6  

Low Duty-cycle Workloads

•  Notable examples –  Web hosting [Knauth ‘14]

–  Personal servers [Elsmore ‘12, Mortier ’10]

–  Cyber-foraging [Satyanarayanan ’09, Ha ‘13]

App  VEE  

7  

Problem Formulation

VM

VM

VM

Booted

Inactive 1

Inactive i

Inactive N

Bi VM

Ti,0

•  Find policy P – Maximize #VMs – Average miss penalty(P) < Limit

8  

Policy-based Resource Provisioning

•  Multi-level cache management

–  Eviction

–  Miss penalty = F(hit rates)

–  Exclusive caching

9  

Disk

Memory

L3

L2

L1

Booted

Frozen

VM

Suspended

VM

VM

VM Hierarchy (LXC) Memory Hierarchy

Policy-based Resource Provisioning

•  Page replacement

–  Temporal locality

–  Reactive (demand-based) vs. Proactive (prefetching)

10  

Main Memory

Swap space

Booted

Frozen

VM

Suspended

VM

VM

− VM eviction

− VM active duration

− Writeback

− Pinned page

•  Demand-driven

•  LRU, NRU, Second-chance, Clock, …

•  Optimal policy

–  Belady’s MIN optimal demand policy [Aho et al. ’71]

–  Unknown for multi-state hierarchy[Gill et al. ’08]

Reactive Policies

Main Memory

Swap space

Booted

Frozen

VM

Suspended

VM

VM

11  

Reactive Policies: Lower Bound

•  Miss penalty ≥ Ti,0

•  Total miss penalty(P, ω) ≥ Σ hi.Ti,0

•  Lower bound on Σ hi.Ti,0 –  Lower bound on miss penalty

12  

VM

Booted

Inactive i Bi

VM

Ti,0

Proactive Policies •  Prefetching

•  Further reduce #page faults –  Optimal: DPMIN [Trivedi et al. ‘76]

13  

M Time

Main Memory

Swap space

Proactive Policy: Sliding Window

14  

VM

VM

VM

Time

VM Request

B0 B1 B2

VM Idle

Tnext

B0

B1

B2

•  Online implementation –  Predict Tnext : next arrival per VM –  e.g., using ARMA

Measuring Model Parameters

•  Model input –  Transition times (Ti,j) –  State capacities (Bi)

•  Experiments –  Sensitivity analysis –  Density analysis

•  Example virtualization solution: LXC –  Open source, Mainstream Linux, CCC, Dockr –  States: booted, suspended, frozen [Menage et al. ’07]

•  Experiment setup –  Server machine: 24 cores 3.46 GHz, 128 GB RAM

15  

B F S

Measuring Model Parameters

Suspended-to-booted vs. #Booted VMs

16  

0

500

1000

1500

2000

2500

3000

t2 t1 t2 t1 t2 t1 t2 t1 t2 t1 t2 t1 t2 t1 t2 t1 t2 t1

Tim

e (m

s)

Number of Booted VMs

Transition to Booted

400350300250200150100500

Measuring Model Parameters

•  Frozen-to-booted v/s #Booted VMs –  Similar behavior with #frozen VMs

•  Identify bottlenecks to LXC density

17  

0

10

20

30

40

50

60

0 50 100 150 200 250 300 350 400

Tim

e (m

s)

Number of Booted VMs

Frozen-to-booted (t1)

Model Parameters for LXC

•  Mean-value analysis

•  Similar analysis for other virtualization solutions

•  Stochastic-value analysis

18  

How do different policies effect miss penalty? – Reactive vs. Lower bound vs. Proactive

19  

Policy Evaluation

•  Sample low duty cycle workload: personal servers –  Topic of active research [Shakimov et al. ‘11, Elsmore et al. ‘12,

Ha et al. ’13, Singh et al. ‘13, Gupta et al. ’14]

–  Request inter-arrivals and durations

•  Machine generated requests vs. User-generated

–  Periodic data uploads, VISs, cloud-offloading

20  

Inter-­‐arrival  ,me   Dura,on  

Rela>vely  fixed   Rela>vely  fixed  

Stochas-c   Rela>vely  fixed  

Stochas-c   Stochas-c  

Fixed Inter-arrivals + Fixed Duration

21  

0.1

1

10

100

1000

250 350 450 550 650 750 850 950 1050

Av

g M

iss

Pen

alty

(in

ms)

VM density

Suspended-to-booted

Frozen-to-booted

Fixed Inter-arrivals + Fixed Duration

22  

0.1

1

10

100

1000

250 350 450 550 650 750 850 950 1050

Av

g M

iss

Pen

alty

(in

ms)

VM density

Suspended-to-booted

Frozen-to-booted MinΦ

(ω) / |ω|

Fixed Inter-arrivals + Fixed Duration

23  

0.1

1

10

100

1000

250 350 450 550 650 750 850 950 1050

Av

g M

iss

Pen

alty

(in

ms)

VM density

Suspended-to-booted

Frozen-to-booted MinΦ

(ω) / |ω|LRU

Fixed Inter-arrivals + Fixed Duration

24  

0.1

1

10

100

1000

250 350 450 550 650 750 850 950 1050

Av

g M

iss

Pen

alty

(in

ms)

VM density

Suspended-to-booted

Frozen-to-booted MinΦ

(ω) / |ω|LRU

SlidingWindow+ARMASlidingWindow+Ground Truth

Stochastic Inter-arrivals + Stochastic durations

•  Datasets: Newton et al. ‘13, Arlitt et al. ’96 25  

0.1

1

10

100

1000

250 350 450 550 650 750

Av

g M

iss

Pen

alty

(in

ms)

VM density

Suspended-to-booted

Frozen-to-booted

Stochastic Inter-arrivals + Stochastic durations

•  Datasets: Newton et al. ‘13, Arlitt et al. ’96 26  

0.1

1

10

100

1000

250 350 450 550 650 750

Av

g M

iss

Pen

alty

(in

ms)

VM density

Suspended-to-booted

Frozen-to-booted MinΦ

(ω) / |ω|

Stochastic Inter-arrivals + Stochastic durations

•  Datasets: Newton et al. ‘13, Arlitt et al. ’96 27  

0.1

1

10

100

1000

250 350 450 550 650 750

Av

g M

iss

Pen

alty

(in

ms)

VM density

Suspended-to-booted

Frozen-to-booted MinΦ

(ω) / |ω|LRU

Stochastic Inter-arrivals + Stochastic durations

•  Datasets: Newton et al. ‘13, Arlitt et al. ’96 28  

0.1

1

10

100

1000

250 350 450 550 650 750

Av

g M

iss

Pen

alty

(in

ms)

VM density

Suspended-to-booted

Frozen-to-booted MinΦ

(ω) / |ω|LRU

SlidingWindow+Ground Truth

Stochastic Inter-arrivals + Stochastic durations

•  Datasets: Newton et al. ‘13, Arlitt et al. ’96 29  

0.1

1

10

100

1000

250 350 450 550 650 750

Av

g M

iss

Pen

alty

(in

ms)

VM density

Suspended-to-booted

Frozen-to-booted MinΦ

(ω) / |ω|LRU

SlidingWindow+Ground TruthSlidingWindow+ARMA

Policy Evaluation

•  Proactive vs. Reactive

–  If miss penalties are small => use proactive, else reactive

•  Proactive works well for relatively predictable workloads –  Upto 2.2× density, 1 ms miss penalty

30  

Conclusion

•  State-based VM consolidation improves VM density –  Legacy compatible, can leverage transient idleness

•  Imperative to keep miss penalty low –  Optimize policies (control plane) and mechanisms

•  Future work: lots! –  VM heterogeneity—capacity, SLAs, .. –  Workloads, bottlenecks, .. –  Native integration of inactive states

31