Iden%fyingandMeasuringPointsofConges%on

GeorgiosSmaragdakis

ATeamEffort

DavidClark,SteveBauer,WilliamLehr

2

KimberlyClaffy,Ma>hewLuckie,AmoghDhamdhere,BradleyHuffaker

ArthurBerger,KCNg,BalaChandrasekaran(internfromDuke)

WhyStudyConges%on?-  TheInternetisnotanymorea“nicetohave”service;networkdelaysaffectproducOvity

-  RevenuecanbesensiOvetomillisecondsdelay(seereferencesin[1])-  Amazonfoundthat100msecoflatencycost1%ofsales.-  GooglefoundthatdelaysindisplayingwebpagesleadtorevenuereducOon(upto

20%fora500msecdelay).

-  EngagementofusersisalsosensiOvetodelay[2]-  Usersstarttoabandonavideostreamingisstart-upOme>2seconds.-  Usersexperiencere-bufferingfor1%ofvideoduraOonplay5%lessvideo(and

watchfewerads).

3

[1]“PracOcalGuidetoControlledExperimentsontheWeb:ListentoYourCustomersnottotheHiPPO”KDD’07[2]“VideoStreamQualityImpactsViewerBehavior:InferringCausalityUsingQuasi-ExperimentalDesigns”IMC’12

WhyStudyConges%on?-  Unintendedconsequences,e.g.,congesOoncancauseerrorstoNTPaccuracy

-  LocaOonwherecongesOonoccursma>ers:-  CongesOoninaccesslinksaffectsusersinaregion-  CongesOonintransit/interconnecOonsaffectsthousandsofusers!

-  ShedlightontherootcausesofcongesOon(economic,technical,architectural)towardsbuildingabe>erInternet/informpolicymakers.

4

Conges%on:AnecdoteorEvidence?

5

WhyYouTubebuffers:Thesecretdealsthatmake—andbreak—onlinevideo

Nejlix’sDisputesWithVerizon,ComcastUnderInvesOgaOon

Nejlixwarisover,butmoneydisputessOllharmInternetusers.

NejlixtoPayComcastforSmootherStreaming

Europe'scompeOOonwatchdogisinvesOgaOngsomeoftheregion'sbiggesttelecomscompaniesoverwhethertheyabusedtheirmarketposiOon

BuildingaConges%onMeasurementPlaBorm

ObjecOves:-  CollectandanalyzedatatoprovideunbiasedevidenceofcongesOon.

-  Developtoolstoconstructadetailed“heatmap”ofcongesOonpercity,peeringlocaOon,theinterconnecOonbetweentwonetworks.

Focuson:-  PersistentcongesOon;cleardailypa>ernsthatspanmulOpledays

Requirements:-  Large-scalebutlightweightmeasurements

6

Iden%fyingandMeasuringPointsofConges%on

PartI:TargetedInterconnecOonsorInternalLinks

-“ChallengesinInferringInternetInterdomainCongesOon”,Luckieetal.,IMC2014-“MeasurementandAnalysisofInternetInterconnecOonandCongesOon”Clarkatal.,TPRC2014

Methodology:TimeSequenceLatencyProbes(TSLP)

8

VantagePoint

BorderRouter#A

BorderRouter#B

AccessRouter DesOnaOon:

VideoServer

Methodology:TimeSequenceLatencyProbes(TSLP)

9

VantagePoint

BorderRouter#A

BorderRouter#B

AccessRouter DesOnaOon:

VideoServer

“Near” “Far”

Methodology:TimeSequenceLatencyProbes(TSLP)

10

VantagePoint

BorderRouter#A

BorderRouter#B

AccessRouter DesOnaOon:

VideoServer

“Near” “Far”

SendTTL-limitedpacketsthatexpireinthe“Near”and“Far”router

Methodology:TimeSequenceLatencyProbes(TSLP)

11

VantagePoint

BorderRouter#A

BorderRouter#B

AccessRouter DesOnaOon:

VideoServer

“Near” “Far”

SendTTL-limitedpacketsthatexpireinthe“Near”and“Far”router

Frequentlymeasure:

RTT#A

RTT#B

TTL=2

TTL=3

AnExample(November2013)

12

AnExample(November2013)

13

ToinferdiurnalpaWern:FFTanalysisof%meserieswithfrequency1/day.

Limita%ons-  AsymmetricRouOng

-  ReverseTraceroute[1]mayunveilthereversepath(usingIPopOons)

-  Bothforwardandbackwardpathshouldbemonitored;vantagepointsareneededatbothends

-  RouterQueuingManagement

-  Measuringpackets(ICMPpackets)maybeassignedtolowpriorityqueues

-  RandomEarlyDetecOon(RED)beforequeuebecomesfull

-  RouterOwnership-  Itisnottrivialtomapaninterface/routertoanetwork;itrequiresanalysisofmassiveamountofmeasurements(aliasing)

14

[1]“ReverseTraceroute”NSDI’10

Iden%fyingandMeasuringPointsofConges%on

PartII:AtInternet-wideScale-“AServer-to-ServerViewoftheInternet”,Chandrasekaranetal.,CoNEXT2015

U%lizingaHighlyDistributedPlaBorm

16

-Large-scalemeasurementsu%lizing5,000+serverclusters(oneserverpercluster)-2,000+loca%ons:coloca%onfacili%es,IXPs,datacenters,residen%alnetworks,enterprisenetworks.-1,200+networks

U%lizingaHighlyDistributedPlaBorm

17

Measurementtools(opera%onalversions):-  ping-  traceroute(Paris)

Methodology

18

FrequentServer-to-Server

pingMeasurements

ApplyFFTtoselectcandidatepairswith

“conges%on”

Performtraceroutecampaigns

Infertheloca%onofconges%on

BootstrapPhase:Server-to-ServerPingMeasurements

19

0

10

20

30

40

50

60

70

80

00:0001/052015

00:0001/062015

00:0001/072015

00:0001/082015

00:0001/092015

00:0001/102015

00:0001/112015

00:0001/122015

1

late

ncy

[ms]

1 if

any

pack

et lo

ss

d-trip time and Packet Loss between regions 19247__23336 (SH-SJC3__TWC-DF95th minus 5th percentile = 29 ms. Power Ratio = 0.845

latencyany packet loss

Mon Tue Wed Thu Fri Sat Sun Mon

BootstrapPhase:Server-to-ServerPingMeasurements

20

-  Wecollectedandanalyzedaround2MillionOmeseriesofpings

-  Frequency:1sampleper15minutesfor1week

-  TheFFTanalysisshowedthataround6%arepotenOalcandidatepairsforcongesOon-  NoOcethatrouOngmayplayarole-  NoOcethattheincreaseofdelaymaynotbealwayssignificant

Server-to-ServerTracerouteMeasurements

-  UnfortunatelywithpingmeasurementsisnotpossibletolocatewherethecongesOonoccurs.

-  Weperformserver-to-servertraceroutemeasurementsinbothdirecOons,foraround100Kpairs

-  Measurementsspantwoweekswithfrequency1tracerouteevery30mins.

21

Loca%ngConges%onPoints

%me

hops

hop1 hop2 hop3 hop4 Lasthop

Loca%ngConges%onPoints

23

%me

hops

hop1 hop2 hop3 hop4 Lasthop

Loca%ngConges%onPoints

24

%me

hops

hop1 hop2 hop3 hop4 Lasthop

Tolocatethecongestedlink:ComputethePearsoncorrela%oncoefficientρin(-1,1).

Loca%ngConges%onPoints

25

%me

hops

hop1 hop2

Tolocatethecongestedlink:ComputethePearsoncorrela%oncoefficientρin(-1,1).

ρ=0.004 ρ=0.005 ρ=0.60 ρ=0.60

hop3 hop4 Lasthop

Loca%ngConges%onPoints

26

%me

hops

hop1 hop2

Tolocatethecongestedlink:ComputethePearsoncorrela%oncoefficientρin(-1,1).

ρ=0.004 ρ=0.005 ρ=0.60 ρ=0.60

hop3 hop4 Lasthop

Loca%ngConges%onPoints

-SymmetricRouOng:Forwardandreverseinferthesamerouter

-AsymmetricRouOng:Wecanonlyargueinternal/interconnecOonlink

27

SomeObserva%ons

-  WeinvesOgated310Klinks;weinferredaround3,000linkswithpersistentcongesOon

-  BothinternalandinterconnecOonlinkswerecongested

-  But,interconnecOonlinkswereinferredfromalargenumberoftraceroutes,insomecasesby>300probes.

-  Bothcustomer-providerandpeer-peerinterconnecOonswerecongested

-  Publicpeeringlinks(atIXPs)werelesscongestedthanprivateinterconnects.

28

WhatistheOverheadofConges%on?

29

“uniform”overheadinUS-USlinks;around25+msec

Bestprac%cesinrouterconfigura%ons?

WhatistheOverheadofConges%on?

30

less“uniform”overheadinlinksaroundtheglobe

Bestlocalprac%ces?Longer/transcon%nentaldistances?

WhatistheOverheadofConges%on?

31

No%cethatrou%ngchangesmayincreasethedelayby50+milliseconds.

Summary

32

-  AswerelyonasmoothoperaOonoftheInternet,anydisrupOonsuchascongesOon,hasanegaOveimpactonuserexperienceandproducOvity

-  WepresentedtechniquestomeasurecongesOonandlocalizeittoalinkoranetwork

-  Ourlarge-scalestudyshowsthatcongesOonisnotthenorm,butinsomepathsitcontributestotheend-to-enddelay

NextSteps

33

-  ConOnuetomeasuretheInternetandseekforpointsofcongesOon

-  Improveourtechniquesanddealwith“blackbox”behavior;wewelcomeyourhelp!

-  Scaleupouranalysis

-  MakeanInternet“heatmap”ofcongesOonpubliclyavailable

34

Thank you!

34