Performance Comparison: IPv4 and IPv6
Lee Howard Geoff Huston
John Brzozowski Chris Donley
February 2014
Geoff Huston, APNIC
The Environment
We use Google Ads to deliver test scripts to a very broad cross-‐secQon of Internet Users
– We use a script that requests dual-‐stack end clients to fetch unique V6 and a V4 URLs from our servers
(servers located in the US, Germany and Australia)
– We have some 300,000 -‐ 400,000 ads delivered per day – We packet dump all acQvity on the server
ConnecQon Reliability
Looking at the TCP handshake, what proporQon of IPv6 clients send us a SYN, but no following ACK?
SYN
SYN+ACK
ACK
ConnecQon Reliability
ConnecQon Reliability
IPv4 failure rate: 0.1% -‐ 0.3% IPv6 failure rate: 1.2 – 2.1% This appears to indicate that a visible proporQon of IPv6-‐capable end user devices are located behind firewall/filter setups that deny incoming IPv6 packets
IPv6 ConnecQon Failure Rate by Origin AS
AS Samples Failure Rate (%) AS Name 14210 22 95.45 EDGECAST-‐DCA -‐ EdgeCast Networks, Inc. US
132497 295 78.64 DNA-‐AS-‐AP SMARTLINK BROADBAND SERVICES PVT LTD IN
55680 53 75.47 KSI-‐UAJY-‐AS-‐ID Kantor Sistem Informasi Universitas Atma Jaya Yogyakarta ID
198864 341 75.37 QMW-‐AC-‐UK Queen Mary and Wesiield College, University of London GB
18106 45 73.33 VIEWQWEST-‐SG-‐AP Viewqwest Pte Ltd SG
25592 51 62.75 NETIS-‐AS NETIS TELECOM Inc. Yaroslavl region ISP provider Russia RU
2856 27 59.26 BT-‐UK-‐AS BTnet UK Regional network GB
17660 141 58.16 DRUKNET-‐AS DrukNet ISP BT
22773 33 57.58 ASN-‐CXA-‐ALL-‐CCI-‐22773-‐RDC -‐ Cox CommunicaQons Inc. US
17705 37 51.35 INSPIRENET-‐AS-‐AP InSPire Net Ltd NZ
55947 128 46.09 BBNL-‐IN Bangalore Broadband Network Pvt Ltd IN
4755 175 43.43 TATACOMM-‐AS TATA CommunicaQons formerly VSNL is Leading ISP IN
278 45 40.00 Universidad Nacional Autonoma de Mexico MX
53187 28 35.71 UNIVERSIDADE ESTADUAL DE CAMPINAS BR
23148 108 27.78 TERREMARK Terremark US
1930 4808 25.21 RCCN Rede Ciencia Tecnologia e Sociedade (RCTS) PT
10429 60 25.00 Telefonica Data S.A. BR
17832 73 23.29 SIXNGIX-‐AS-‐KR Korea Internet Security Agency KR
45133 39 23.08 SINGAPORE-‐POLYTECHNIC-‐AS-‐AP Singapore Polytechnic SG
21366 50 22.00 KYMP KYMP OY FI
This table looks at the origin AS’s with more than 20 sample points – these are the origin AS’s with the highest failure rates
IPv6 ConnecQon Failure Rate by Origin AS
AS Samples Failure Rate (%) AS Name 786 458,143 1.22 JANET JISC CollecQons And Janet Limited GB 15169 106,221 0.08 GOOGLE -‐ Google Inc. US 8708 75,393 5.70 RCS-‐RDS RCS & RDS SA RO 7922 57,659 1.84 COMCAST-‐7922 -‐ Comcast Cable CommunicaQons, Inc. US 7018 50,489 1.22 ATT-‐INTERNET4 -‐ AT&T Services, Inc. US 2516 50,358 0.16 KDDI KDDI CORPORATION JP 12322 38,125 5.02 PROXAD Free SAS FR 6147 32,512 1.75 Telefonica del Peru S.A.A. PE 22394 27,237 0.34 CELLCO -‐ Cellco Partnership DBA Verizon Wireless US 23910 23,372 2.98 CNGI-‐CERNET2-‐AS-‐AP China Next GeneraQon Internet CERNET2 CN 4739 19,398 0.77 INTERNODE-‐AS Internode Pty Ltd AU 3303 17,319 4.45 SWISSCOM Swisscom (Switzerland) Ltd CH 4773 13,963 0.57 MOBILEONELTD-‐AS-‐AP MobileOne Ltd. Internet Service Provider Singapore SG 19782 8,172 0.06 INDIANAGIGAPOP -‐ Indiana University US 7575 7,863 3.84 AARNET-‐AS-‐AP Australian Academic and Reasearch Network (AARNet) AU
This table looks at the origin AS’s where we have the highest number of samples
IPv6 ConnecQon Failure Rate for US + CA Nets
AS Samples Failure Rate (%) AS Name
14210 22 95.45 EDGECAST-‐DCA -‐ EdgeCast Networks, Inc. US 22773 33 57.58 ASN-‐CXA-‐ALL-‐CCI-‐22773-‐RDC -‐ Cox CommunicaQons Inc. US
23148 108 27.78 TERREMARK Terremark US 237 265 21.13 MERIT-‐AS-‐14 -‐ Merit Network Inc. US
5707 24 20.83 UTHSC-‐H -‐ The University of Texas Health Science Center at Houston US
812 1139 17.12 ROGERS-‐CABLE -‐ Rogers Cable CommunicaQons Inc. CA 11351 471 15.50 RR-‐NYSREGION-‐ASN-‐01 -‐ Time Warner Cable Internet LLC US
2055 325 14.46 LSU-‐1 -‐ Louisiana State University US
174 25 12.00 COGENT Cogent/PSI US 12271 1850 11.24 SCRR-‐12271 -‐ Time Warner Cable Internet LLC US
11426 209 9.09 SCRR-‐11426 -‐ Time Warner Cable Internet LLC US 1312 527 8.54 VA-‐TECH-‐AS -‐ Virginia Polytechnic InsQtute and State Univ. US
10796 669 7.77 SCRR-‐10796 -‐ Time Warner Cable Internet LLC US
1280 97 6.19 ISC-‐AS1280 Internet Systems ConsorQum, Inc. US 11427 2102 4.52 SCRR-‐11427 -‐ Time Warner Cable Internet LLC US
3356 316 4.43 LEVEL3 Level 3 CommunicaQons US 12222 24 4.17 AS12222 Akamai Technologies US
20001 4505 3.26 ROADRUNNER-‐WEST -‐ Time Warner Cable Internet LLC US
6939 4592 3.11 HURRICANE -‐ Hurricane Electric, Inc. US 33522 37 2.70 CPANEL-‐INC -‐ cPanel, Inc. US
RTT Measurements
Data performance is highly dependent on the RTT across the data connecQon Is IPv6 faster or slower than IPv4 in terms of an RTT comparison?
RTT EsQmate
Measuring the Qme interval for the compleQon of the TCP handshake
SYN
SYN+ACK
ACK
RTT Interval
Paired RTT DistribuQon
Take the TCP handshake and measure the elapsed Qme at the server between the iniQal SYN and the following ACK packet
– this Qme value is an indicator of the RTT
Take the measurements where we have web log evidence that the IPv4 and IPv6 addresses correspond to a single experiment, and generate a raQo of the two RTT values
Paired RTT DistribuQon
IPv6 is faster than IPv4 IPv4 is faster than IPv6
Then: July 2012
% of R
TT m
easuremen
ts
3x 3x 2x 2x 1.5x 1.5x =
RTT RaQo
Paired RTT DistribuQon
IPv6 is faster than IPv4 IPv4 is faster than IPv6
Then: July 2012
Now: December 2013
% of R
TT m
easuremen
ts
3x 3x 2x 2x 1.5x 1.5x =
RTT RaQo
RTT DistribuQon
• There is a slight change in the RTT distribuQons over the past 12 months favoring IPv6 being slightly faster than IPv4 – This could be due to different network paths between IPv4 and IPv6 – Or less deployment of port 80 trapping middleware in IPv6 as compared to IPv4
Lee Howard
Performance Dashboard
IPv4 Average 39.3ms IPv6 Average 33.4ms Difference 15%
Avg speed to site Avg speed from hub:
Green = IPv6 >30% faster Pink = IPv6 >30% slower
atl20-‐tbp1 chi10-‐tbp1 chi30-‐tbp1 clt01-‐tbp1 dca10-‐tbp1 dca20-‐tbp1 dfw10-‐tbp1
hou30-‐tbp1 lax00-‐tbp1 lax30-‐tbp1 nyc20-‐tbp1 nyc30-‐tbp1 sea20-‐tbp1 sjc10-‐tbp1 sjc30-‐tbp1 IPv4
Avg IPv6 Avg
A 14.2 12.3 17.7 8.7 17.4 7.9 7.9 7.7 1.8 3.6 2.4 4.2 9.2 0.7 14.9 4.9 1.2 7.8 5 9.2 3.1 9.2 2.6 8.5 27.1 38.1 9.2 19.6 12.1 21 9.72 10.89
B 11.9 12.6 29.7 27.1 11.1 7.8 0.8 1.1 5.9 1.7 7 11.2 20.1 14 9.4 7.3 8.6 6.8 1.1 2.5 3.9 10.95 6.22
C 14.8 15.6 179.9 181.5 176.1 183.4 16.1 16.5 9.6 10 10 10.8 47.9 35.8 48 39.8 83 30.6 82.8 32.5 15.6 15.3 14.9 15.3 98.8 53.7 82.8 22.8 83.4 23.3 64.25 45.79
D 14.9 15.4 179.7 181.8 176.2 183.6 16.2 16.6 9.9 9.9 10.2 10.6 47.9 36.2 47.9 39.3 83.1 30.6 82.7 32.5 15.8 16 15.2 15.2 98.8 53.7 82.8 22.8 83.4 23.3 64.31 45.83
E 7.8 6.4 32.6 38.2 32.5 37.8 15.8 15.4 12.2 13.7 10.1 38.8 17.1 18.1 21.1 21.5 0.6 0.8 2.9 3 10.3 6.1 5.7 6.7 19.7 25.6 11.1 11.6 12.1 12.3 14.11 17.07
F 15 15.9 41.7 35.6 35.9 31.5 8.5 9.4 1.7 3.1 2.5 4.1 47.6 30.8 47.9 34.7 66.8 9.1 66.5 16.3 15.8 16.1 15.1 16.5 93.1 61.1 77 17.4 77.4 18 40.83 21.31
G 15.1 16.3 177.1 182.4 173.8 183.9 10.7 11.8 4.3 5.6 4.9 6 43.4 24.8 43.5 30.9 66.8 8.9 66.4 16.5 11.3 11.4 9.7 11.2 90.9 65.5 74.9 15.8 75.4 17.6 57.88 40.57
H 16.3 15.1 48.3 44 48.3 40.2 10.7 9.9 7.5 2.9 4.3 22.2 83.1 63 85.7 68.2 60.8 39.5 43 38.7 10 8.5 9.5 8.9 67.8 56.7 48.9 37.8 50.5 38.9 39.65 32.97
I 15.1 16.3 28.5 27.5 25.2 25.2 10.4 10 3.7 1.5 6.6 6.1 50.2 47.8 47.9 47.3 80.7 65.5 79.9 79.8 10.9 10.8 9.7 7.8 82.1 80.7 77.1 75.8 76.6 74 40.31 38.41
J 39.3 71.4 41.5 39.2 43.1 38.6 35 61.8 27.4 58.5 27 65.9 57.7 59.8 55.6 64.4 34.6 69.5 36.3 84 16.7 52.7 17.7 52.9 11.3 61 27.9 73.9 30 63.4 33.41 61.13
K 14.9 15.1 177 182 173.9 183.5 8.4 16.5 1.7 9.9 2.5 11.1 43.5 36.9 43.5 38.9 66.8 30.7 66.4 32.7 8 15.6 7.6 15.5 90.9 53.9 74.9 22.8 75.4 23.3 57.03 45.89
Average 16.3 19.3 86.7 92.1 84.5 91.6 13.7 16.7 7.3 10.9 7.9 16.5 41.3 35.4 42.5 39.0 51.3 29.3 49.6 34.5 11.5 15.4 10.6 15.0 62.0 55.0 51.7 32.0 52.7 31.5 64.3 47.4
95th percen3le: 16.3 16.3 179.7
182.0 176.1 183.6 16.2 16.6 12.2 13.7 10.2 38.8 57.7 60.1 55.6 64.8 83.0 65.9 82.7 80.2 15.8 16.1 15.2 16.5 98.8 67.0 82.8 74.1 83.4 64.5 64.3 47.4
lax30-‐tbp1 nyc20-‐tbp1
IPv4 IPv6 IPv4 IPv6 A 5 9.2 3.1 9.2 B 14 9.4 7.3 C 82.8 32.5 15.6 15.3 D 82.7 32.5 15.8 16 E 2.9 3 10.3 6.1 F 66.5 16.3 15.8 16.1 G 66.4 16.5 11.3 11.4 H 43 38.7 10 8.5 I 79.9 79.8 10.9 10.8 J 36.3 84 16.7 52.7 K 66.4 32.7 8 15.6 Average 49.6 34.5 11.5 15.4 95th %ile 82.7 80.2 15.8 16.1
Performance Dashboard (extract)
Performance Dashboard: Hop Count
atl20-‐tbp1 chi10-‐tbp1 chi30-‐tbp1 clt01-‐tbp1 dca10-‐tbp1 dca20-‐tbp1 dfw10-‐tbp1 hou30-‐tbp1 lax00-‐tbp1 lax30-‐tbp1 nyc20-‐tbp1 nyc30-‐tbp1 sea20-‐tbp1 sjc10-‐tbp1 sjc30-‐tbp1
A 4.8 5 6.8 4.3 5 4.5 3.6 5 3.6 4 4 4 3.7 4 3.6 4 2.6 4.3 3.3 3.2 3.9 2.6 3.9 4 5.4 6.4 4.4 5.5 4.4 5.5
B 8 8 10 8 10 8.9 10 5.8 8 9.8 10 4.9 10 8 10 9.1 9.4 9.8 9 9.8 9 9 9 10.9 10 8 9 13.6 10 14.4
C 8 8 10 8 10 8.9 10 6.2 8 9.8 10 4.5 10 8 10 9.2 9.4 9.8 9 9.8 9 9 9 10.9 10 8.1 9 13.6 10 14.4
D 9.3 8.6 9 6.9 8 6.9 12.9 10.4 12.8 8.7 11 8.5 6 6 7 6.9 6 6 7.6 6.9 11.3 5.6 9.4 7.5 11 9 10 7 11 7.9
E 9 9.9 7.1 6.1 7.1 7 7.1 5.6 5 7.9 7 3.5 8.6 7.1 8.7 8 8.2 9.2 6.5 9.2 9.4 9 9.4 10.9 9.7 10.6 6.7 13.6 7.7 14.5
F 9 10 7.3 6.2 7.3 7 7.1 5.3 5 7.9 7 3.6 7 6 7 6.9 7.6 7.9 6 7.9 10 9 10 10.9 11 11 8 13.6 9 14.4
G 11 7.9 10 7.9 10 7.9 11.9 8.9 10 6.9 12 6.9 13 12 13 13 12.6 11 11 9.1 10 6 10 7.9 8 7 13.6 10 13.6 11
H 4 6 5 6.4 6 7.3 5 6 5 4 3 3.3 5 8.9 6 9.7 8.6 9.3 7 10.7 4 4.4 4 6.4 7 8.4 5 6.4 6 7.4
J 11 6.1 9.8 7.3 10.8 8.1 12 7.8 12 6.1 10 4.6 12.8 8.3 13.8 9.2 12.8 10.6 11.2 10 8 4 11 6 8.3 8 10.2 10 11.2 10.8
K 9 8 7 8 7 8.8 7.1 5.6 5 9.8 7 4.2 7 8 7 9.1 7.6 9.8 6 9.8 10 9 10 10.9 11 8 8 13.6 9 14.4
Average 8.3 7.8 8.2 6.9 8.1 7.5 8.7 6.7 7.4 7.5 8.1 4.8 8.3 7.6 8.6 8.5 8.5 8.8 7.7 8.6 8.5 6.8 8.6 8.6 9.1 8.5 8.4 10.7 9.2 11.5 95th percen3le:
11.0 9.9 10.0 8.0 10.1 8.9 12.1 9.1 12.1 9.8 11.1 7.1 12.8 9.2 13.1 10.0 12.6 10.6 11.0 10.1 10.1 9.0 10.1 10.9 11.0 10.6 10.5 13.6 11.4 14.4
IPv4 Average 8.38
IPv6 Average 7.54
Difference 10%
Dashboard History: AvgRTT over 1 week
Dashboard History: Speed Download over 1 month
Conclusions
• Of the sites we’re measuring, IPv6 is generally faster • ExcepQons:
– Sites that tunnel IPv6 over IPv4 – Hubs that are not closest exit
• No conclusions about variability—IPv4 and IPv6 performance both vary
• More research needed
John Brzozowski
IPv6 Performance
John Jason Brzozowski, Comcast February 11, 2014
Methodologies
• Background • IniQal focus to test, validate, and audit IPv6 • Goal is to ensure ideal customer experience as IPv6
deployment advances and evolves • Various approaches measure and test key elements in an
isolated and integrated manner • Variety of applicaQons, tools, and protocols used to
perform tesQng and gather • Third party and internally developed leveraging well
known protocols -‐ HTTP, ICMP, DNS • iPerf • Speed test
25
Approaches
Experience Index
Broadband Network
Core Network
Internet
ObservaQons • IPv6 performance across broadband ecosystem appears to lead that of IPv4 in some cases, specifically for broadband data
• Factors include the chipsets and implementaQons, specifically related to home networking
• Broadband access implementaQons are likely more recent and with simpler deployment models
• Core network performance is largely even across the core in typical scenarios
• Core to Internet adds variability • PotenQal opportuniQes exist to opQmize, larger MTU for IPv6
27
ObservaQons (conQnued)
• Internet ecosystem introduces relevant variables • DNS lookup Qme, connected Qme, and transfer Qme are key metrics
• CDN readiness and availability specific to IPv6 • Observed to result in inefficient and/or slow content distribuQon
• Many CDNs, however, have exactly the same performance over IPv6 and IPv4 • Some have greater performance over IPv6
• ImplementaQon quality for non-‐CDN hosted content varies greatly, which also impacts performance
28
ObservaQons (conQnued)
• And… • Average speed tests indicate that IPv6 out performs IPv4 naQonwide
• Large sample size, all speed test servers are dual stack enabled • Comcast has ~30% of customers acQvely provisioned with dual stack enabled broadband
• Even with highs and lows dropped average IPv6 speed test is faster than IPv4
29
Next steps
• ConcatenaQon of all scenarios seems yield posiQve performance differences
• Increase and improve measurements from the customer’s premises
• Core network and Internet oriented measurements
• Consider customer premises variables • Wired versus wireless • Wi-‐Fi radio (2.4Ghz versus 5Ghz) • Other network acQvity within the customer’s premises
• IdenQfy opQmizaQon opportuniQes 30
John Jason Brzozowski [email protected]
Chris Donley
© 2013 CableLabs®. All rights reserved.
IPv6-IPv4 Performance Comparison The Effect of NAT
Chris Donley
Is IPv6 faster than IPv4?
• CableLabs has been conducQng IPv6 interops since 2009 – Observed subtle but persistent v6 performance improvements
• Conducted lab tesQng to measure v4/v6 performance – NaQve IPv6 – IPv4 with one layer of NAT – IPv4 with two layers of NAT
Slide 34
Methodology • Conducted matched v4-‐v6 ping tests over the course of an IPv6 interop – First sets: 2 NATS enabled – Second sets: 1 NAT enabled
• IdenQfy effect of NAT without v4-‐v6 path differences
• Did NOT measure effects of traffic engineering or differenQated peering
Slide 35
Test Server
Switch
CMTS
CM
Low-‐end
Router
Home Router
Test PC
Configurable NAT
NAT Enabled
Results – One Layer of NAT
• Could NAT be responsible for slower v4 performance?
• IPv6 performed measurably be{er than IPv4 – Lower mean, median, standard deviaQon, and minimum
– StaQsQcally significant differences
Common Scenario Today
Slide 36
IPv4 IPv6
Mean 12.02
9.39
Median 9.42 8.75
Standard Dev.
5.76 3.51
Max 29.79
46.17
Min 7.68 7.38
Samples 122 129
Results – Two Layers of NAT
• WHAT! • One NAT showed differences, two show nearly idenQcal results – 81% chance that the means are really the same
– V4 in both cases sQll shows higher variability – V6 performed be{er half the Qme across 6 test runs
Possible CGN effect
Slide 37
IPv4 IPv6
Mean 10.00
10.06
Median 9.48 9.65
Standard Dev.
2.03 1.74
Max 17.79
15.99
Min 7.90 7.60
Samples 112 111
Conclusions (?)
• IPv4 NAT IN SOME ROUTERS does affect performance • Your mileage may vary – NAT performance seems to vary across different plaiorms
• Other network effects such as tunnels, traffic engineering, network congesQon are likely to produce larger effects
Slide 38
Conclusions