next stop, the cloud: understanding modern web...

Next Stop, the Cloud: Understanding Modern Web Service

Deployment in EC2 and Azure

Keqiang He, Alexis Fisher, Liang Wang, Aaron Gember, Aditya Akella, Thomas Ristenpart

University of Wisconsin-Madison

1

PrivateDatacenters

Co-locationCenters

Move to IaaS clouds

2

Incomplete view of cloud use

3

Want a global, in-depth understanding of IaaS cloud usage patterns

Fundamental questions

1. Who is using public clouds?

→Percentage of cloud-using domains, traffic profile

2. How are Web services using the cloud?

→ Impact of failures; ways to improve availability

→ Design of new systems/services

4

Datasets

• University packet capture

– Deep, but local perspective

• Alexa subdomains DNS records

– Broad, high-level perspective

5

Top 1 million Subdomains

Zone transferDNS

records

1.4TB Capture

IP RangesConn

recordsCampusTraffic

Fundamental questions

1. Who is using public clouds?

→Percentage of cloud-using domains, traffic profile

6

40,333Cloud-using domains

How many popular web services use the cloud?

7

94.2%

5.1% 0.7%Both

9. 13.29.35.36.

2 of 68

3 of 142

4 of 181

18 of 24

14 of 89

Cloud availability and performance issues may impact over 4%

of popular web services

18.3%

81.7%

Traffic Volume (Bytes)

Which (locally) popular web services use the cloud?

8

1.2.3.4.5.

68.2%

1.7%

1.1%

0.7%

0.6%

1.2.3.4.5.

atdmt.com

s-msn.com

msecnd.net

3.1%

2.4%

2.3%

1.5%

1.4%

Fundamental questions

2. How are Web services using the cloud?

→ Impact of failures; ways to improve availability

→ Design of new systems/services

9

VM ?

VM

VMLB

Proxy ?

Which front ends are used?

10

sub.abc.com A 72.44.32.1

sub.abc.com CNAME abc.elb.aws.com

abc.elb.aws.com A 72.44.42.1

sub.abc.com CNAME proxy.heroku.com

proxy.heroku.com A 72.44.62.1

?PaaSNode

DNSServer

VM

LBProxy

PaaSNodeDNS

records

72%

4%

8%

0% 50% 100%

VM front end

LB front end

PaaS front end

50%35%

15%

1 VM

2 VMs

3+ VMs

97%

3%Elastic Beanstalk

Which front ends are used?

11

There is limited uptake of value-added features

How many and which regions are used?

12

DNS records

How many and which regions are used?

13

521,681

116,366

40,548 Virginia

Ireland

California

Oregon

Singapore

Tokyo

Sao Paulo(74%)

(16%)

How many and which regions are used?

14

862

558

2,071

1,395

1,035

1,205

632502 Virginia

California

Illinois

Texas

Ireland

Netherlands

Singapore

Hong Kong

How many and which regions are used?

92%

8%

1 region

2+ regions

97%

3%

1 region

2+ regions

9. 13.29.35.36.

1

2

1

1

2

7. 18.20.31.

3

5

1

5

Single region failures can take down a large fractionof cloud-using subdomains

How many and which availability zones are used?

• Latency measurements

– Low latency => instances are in same zone

– High coverage, low accuracy (noisy)

• IP address correlation

– IPs are in the same /16 subnet => instances are in the same zone

– Higher accuracy, low coverage

16

How many and which availability zones are used?

17

33.2%

44.5%

22.3%

1 Zone

2 Zones

3+ Zones

Single zone failures can take down 16% of subdomains

and affect many others

Highlights

• The first comprehensive study of Web service deployment in public clouds

• 4% of popular web services use EC2/Azure

• VMs are the most popular EC2 front end, but value-added features are used by top domains

• Limited region and zone usage (and diversity) makes web services vulnerable to failures

18