Taming Anycast in a Wild Internet
Stephen McQuistinUniversity of GlasgowVerizon Digital Media Services
Sree Priyanka UppuMarcel Flores Verizon Digital Media Services
2
192.0.2.0/24192.0.2.0/24
San Jose, CA Denver, CO
3
4
Performance is impacted by taking longer routes to a farther site
35ms
36ms
45ms
IP anycast
AS A AS A
5
Site 1 Site 2
IP anycast
Peer X
Transit Z
AS A AS A
Site 1 Site 2
6
IP anycast
AS BPeer X
Transit Z
AS A AS A
Site 1 Site 2
7
IP anycast
AS BPeer X
Transit Z
AS A AS A
Site 1 Site 2
8
Desirable route/site is via Z to Site 2
IP anycast
AS BPeer X
Transit Z
AS A AS A
Site 1 Site 2
9
… but AS B takes route through X to Site 1
IP anycast
AS BPeer X
Transit Z
AS A AS A
Site 1 Site 2
Opaque, upstream provider policies determine end-user performance
10
How do anycast operators influenceinbound routing?
Example
12
• Impact of adding a new peer is significant: two thirds of affected ASes see RTTs more than half
Announcement configurations
13
Site 1 Site 2 Site 3 Site 4 Site 5 …
Transit B
Transit C
Transit D
Peer E
Peer F
Peer G
…
Announcement configurations
14
Site 1 Site 2 Site 3 Site 4 Site 5 …
Transit B
Transit C
Transit D
Peer E
Peer F
Peer G
…
Announcement configurations
15
Site 1 Site 2 Site 3 Site 4 Site 5 …
Transit B
Transit C
Transit D
Peer E
Peer F
Peer G
…
Many-provider networks have a larger announcement configuration space
Many-provider networks
16
Many-provider networks
17
DNS root servers
Many-provider networks
18
A large commercial CDN
Many-provider networks
19
A large DNS provider
Many-provider networks
20
Many-provider networks
21
Few providers
Many-provider networks
22
Many providers
Many-provider networks
23
Many-provider networks
24
Many-provider networks generally have lower hegemony
Many-provider networks
25
Many-provider networks use a diversity of provider networks
What are the potential network impacts of altering announcement configurations?
Network impacts
27
AS A
192.0.2.0/24
TransitB
TransitC Transit
D
Control configuration: announcements to a limited set of
transit providers
Network impacts
28
AS A
198.51.100.0/24
TransitB
TransitC Transit
D
Experimental configuration: announcements to a limited set of transit providers and to nearly all
other providers
Peer E
Peer F
Peer G
Grouping vantage points
29
• Measurements taken from RIPE Atlas probes
• We don't want to rely on results from individual probes
• Group together, first by AS, then by geolocation
• More grouping functions evaluated in the paper
Control Experimental
Round-trip time impacts
30
�1.00 �0.75 �0.50 �0.25 0.00 0.25 0.50 0.75 1.00Relative Change
0.0
0.2
0.4
0.6
0.8
1.0
CD
F of
Gro
ups
�1.00 �0.75 �0.50 �0.25 0.00 0.25 0.50 0.75 1.00Relative Change
0.0
0.2
0.4
0.6
0.8
1.0
CD
F of
Gro
ups
Round-trip time impacts
31
Performance improves for 60% of groups
�1.00 �0.75 �0.50 �0.25 0.00 0.25 0.50 0.75 1.00Relative Change
0.0
0.2
0.4
0.6
0.8
1.0
CD
F of
Gro
ups
Round-trip time impacts
32
.. and degrades for 40% of groups
Catchment shift impacts
33
Catchment shift impacts
34
Majority of differences (and the greatest changes) come from groups that shift catchment
Catchment shift impacts
35
In the paper, we further explore performance impacts in terms of
paths taken and inbound providers
Who should we announce to for optimal performance?
DailyCatch
37
AS A
192.0.2.0/24
TransitB
TransitC Transit
D
Control configuration
AS A
198.51.100.0/24
TransitB
TransitC Transit
DPeer E
Peer F
Peer G
Experimental configuration
DailyCatch
38
AS A
192.0.2.0/24
TransitB
TransitC Transit
D
Control configuration
AS A
198.51.100.0/24
TransitB
TransitC Transit
DPeer E
Peer F
Peer G
Experimental configuration
DailyCatch
39
AS A
192.0.2.0/24
TransitB
TransitC Transit
D
Control configuration
AS A
198.51.100.0/24
TransitB
TransitC Transit
DPeer E
Peer F
Peer G
Experimental configuration
Snapshot A Snapshot B
DailyCatch
40
AS A
192.0.2.0/24
TransitB
TransitC Transit
D
Control configuration
AS A
198.51.100.0/24
TransitB
TransitC Transit
DPeer E
Peer F
Peer G
Experimental configuration
Snapshot A Snapshot B
Output: a net score and group-level scores
Scoring & Comparison
41
1 Take two snapshots
Control
Experiment
traceroute
Scoring & Comparison
42
2 For each group, assign a score, by passing the change in RTT through a logistic function
Control
Experiment
traceroute
Scoring & Comparison
43
3 Weight each group score by the volume of traffic it represents
Control
Experiment
traceroute
Scoring & Comparison
44
4 Sum for the overall score
Control
Experiment
traceroute ∑
45
Under the control configuration, probes are routed to San Jose
35ms
36ms
45ms
46
Under the experimental configuration, probes are routed to Denver
35ms → 21ms
36ms → 21ms
45ms → 20ms
Mixed impacts in North America
47
Mixed impacts in North America
48
Group size doesn’t correlate with observed RTT
Mixed impacts in North America
49
RTT changes are not uniform within a single AS
Mixed impacts in North America
50
More results and case studiesin the paper
Taming Anycast in a Wild Internet
51
• Anycast networks with many providers interact with the Internet in an observably different way than those with few providers
• This provides a configuration space of where and to who anycast announcements are given
• Announcement configurations can have significant network impacts
• DailyCatch enables a systematic, active measurement approach to configuration management