next stop, the cloud: understanding modern web...
TRANSCRIPT
Next Stop, the Cloud: Understanding Modern Web Service
Deployment in EC2 and Azure
Keqiang He, Alexis Fisher, Liang Wang, Aaron Gember, Aditya Akella, Thomas Ristenpart
University of Wisconsin-Madison
1
PrivateDatacenters
Co-locationCenters
Move to IaaS clouds
2
Incomplete view of cloud use
3
Want a global, in-depth understanding of IaaS cloud usage patterns
Fundamental questions
1. Who is using public clouds?
→Percentage of cloud-using domains, traffic profile
2. How are Web services using the cloud?
→ Impact of failures; ways to improve availability
→ Design of new systems/services
4
Datasets
• University packet capture
– Deep, but local perspective
• Alexa subdomains DNS records
– Broad, high-level perspective
5
Top 1 million Subdomains
Zone transferDNS
records
1.4TB Capture
IP RangesConn
recordsCampusTraffic
Fundamental questions
1. Who is using public clouds?
→Percentage of cloud-using domains, traffic profile
6
40,333Cloud-using domains
How many popular web services use the cloud?
7
94.2%
5.1% 0.7%Both
9. 13.29.35.36.
2 of 68
3 of 142
4 of 181
18 of 24
14 of 89
Cloud availability and performance issues may impact over 4%
of popular web services
18.3%
81.7%
Traffic Volume (Bytes)
Which (locally) popular web services use the cloud?
8
1.2.3.4.5.
68.2%
1.7%
1.1%
0.7%
0.6%
1.2.3.4.5.
atdmt.com
s-msn.com
msecnd.net
3.1%
2.4%
2.3%
1.5%
1.4%
Fundamental questions
2. How are Web services using the cloud?
→ Impact of failures; ways to improve availability
→ Design of new systems/services
9
VM ?
VM
VMLB
Proxy ?
Which front ends are used?
10
sub.abc.com A 72.44.32.1
sub.abc.com CNAME abc.elb.aws.com
abc.elb.aws.com A 72.44.42.1
sub.abc.com CNAME proxy.heroku.com
proxy.heroku.com A 72.44.62.1
?PaaSNode
DNSServer
VM
LBProxy
PaaSNodeDNS
records
72%
4%
8%
0% 50% 100%
VM front end
LB front end
PaaS front end
50%35%
15%
1 VM
2 VMs
3+ VMs
97%
3%Elastic Beanstalk
Which front ends are used?
11
There is limited uptake of value-added features
How many and which regions are used?
12
DNS records
How many and which regions are used?
13
521,681
116,366
40,548 Virginia
Ireland
California
Oregon
Singapore
Tokyo
Sao Paulo(74%)
(16%)
How many and which regions are used?
14
862
558
2,071
1,395
1,035
1,205
632502 Virginia
California
Illinois
Texas
Ireland
Netherlands
Singapore
Hong Kong
How many and which regions are used?
92%
8%
1 region
2+ regions
97%
3%
1 region
2+ regions
9. 13.29.35.36.
1
2
1
1
2
7. 18.20.31.
3
5
1
5
Single region failures can take down a large fractionof cloud-using subdomains
How many and which availability zones are used?
• Latency measurements
– Low latency => instances are in same zone
– High coverage, low accuracy (noisy)
• IP address correlation
– IPs are in the same /16 subnet => instances are in the same zone
– Higher accuracy, low coverage
16
How many and which availability zones are used?
17
33.2%
44.5%
22.3%
1 Zone
2 Zones
3+ Zones
Single zone failures can take down 16% of subdomains
and affect many others
Highlights
• The first comprehensive study of Web service deployment in public clouds
• 4% of popular web services use EC2/Azure
• VMs are the most popular EC2 front end, but value-added features are used by top domains
• Limited region and zone usage (and diversity) makes web services vulnerable to failures
18