the internet is flat: revisited...sep 21, 2018 · a small transit provider case study 2017 –...
TRANSCRIPT
The Internet is Flat: Revisited
A Small Transit Provider Case Study
Bob Bender Steve Surdock (CTS Telecom)
Amogh Dhamdhere
(CAIDA/UCSD)
The Internet is Flat: Revisited
The Internet is Flat: Revisited A Small Transit Provider Case Study
In2011atNANOG52,asmallTier3ISPAS19653joinedNANOG.Alsoin2011,thissmallISPreadthepaper-“TheInternetisFlat:ModelingtheTransitionfromaTransitHierarchytoaPeeringMesh”Theforecastsinthispaperwereusedtoinformbusinessandnetworkplanning.ActualnetworkdatawascollectedfromAS19653from2010topresent.Thissmalltransitproviderdataisavignetteofthefactorsthat“cantransformtheInternetecosystemfromamulti-tierhierarchythatreliesmostlyontransitlinkstoadensemeshofhorizontalinterconnectionsthatreliesmostlyonpeeringlinks”
The Internet is Flat: Modeling the Transition from a Transit Hierarchy to a Peering Mesh
OriginallyPresentedatACMCoNEXT2010Nov.30–Dec3,2010--Philadelphia,PAUSA
AmoghDhamdhere(CAIDA)ConstantineDovrolis(GeorgiaTech)
The Internet is Flat: Revisited
The Internet is Flat: Revisited
The Internet in 2011: What did the future hold?
InternetEcosystemEvents
2004-GoogleIPO
2007-AppleiPhoneIntroduced
2007-Netflixbeginsstreaming
2008-HuluLaunched
2011-PandoraIPO
2012-FacebookIPO
The“TheInternetisFlat”paperofferedananalysisofwhatwesawhappeninganecdotallyasasmallISP.AsaTier3ISPitbecameclearthatamovetobecomeaTier2ISPwouldbepossibleinthenewInternetecosystem.MostimportantlythepaperdrovehomethattheimportanceofTier1TransitwasdiminishedandpeeringwithcontentintheIXPwasparamount.
The ITER Model
5The Internet is Flat: Revisited
Agent-basedcomputationalmodeltoanswer“what-if”questionsaboutInternetevolution
Inputs• Networktypesbasedonbusinessfunction• Pricing/costparameters• Interdomaintrafficmatrix• Geographicalconstraints• Peer/providerselectionmethods
Output:Equilibriuminternetworktopology,trafficflow,per-networkfitness
A.DhamdhereandC.Dovrolis.TheInternetisFlat:ModelingtheTransitionfromaTransitHierarchytoaPeeringMeshACMCoNEXT2010–Page2
ITERModelPreviousApplicationsareinDemography,Social,EconomicandEnvironmentalSciences.
The ITER Approach
Analyticallyintractable.Findequilibriumcomputationally,usingagent-basedsimulationsEquilibrium:nonetworkhastheincentivetochangeitsproviders/peers
TrafficFlowInterdomainTopology
Per-ASEconomicFitness
Cost/PriceParameters
BGP-likeRouting
InterdomainTrafficMatrix
ProviderSelection
The Internet is Flat: Revisited
ASOptimizations
PeerSelection
A.DhamdhereandC.Dovrolis.TheInternetisFlat:ModelingtheTransitionfromaTransitHierarchytoaPeeringMeshACMCoNEXT2010–Page3
The “Hierarchical” and “Flat” Internet
TheHierarchicalInternet(late90s–2007)• Topcontentprovidersgeneratedsmallfractionof
totaltraffic• Contentprovidersweretypicallyservedfromorigin• Peeringwasrestrictive
TheFlatInternet(2007onwards)• Topcontentprovidersgeneratelargefractionof
totaltraffic• Contentprovidershaveexpandedgeographically• Peeringismoreopen
“InternetInterdomainTraffic”,Labovitzetal.,Sigcomm2010The Internet is Flat: Revisited
Content Consolidation
The Internet is Flat: Revisited
Interdomain Routing and Traffic flow
• Moretrafficflowsoverpeeringlinksthantransitlinksinthe“Flat”Internet
• Trafficfollowsshorterroutingpathsduetodirectpeeringinthe“Flat”Internet
• Thiseffectisevenmorepronouncedwhenpathsareweightedbytrafficvolume:pathscarryingthemosttrafficareshorter
A.DhamdhereandC.Dovrolis.TheInternetisFlat:ModelingtheTransitionfromaTransitHierarchytoaPeeringMeshACMCoNEXT2010–Page2
Simulatedtwo“instances”oftheITERmodel.Firstwasparameterizedtoresemblethe“HierarchicalInternet”.Secondwasparameterizedtoresemblethe“FlatInternet”.Thencomparedvariouspropertiesoftheequilibriumthatwegetfromthetwoinstancesofthemodel.
Predictions of Transition Impacts
ContenttrafficbypassesTier-1providersinthe“flat”Internet:ProducesconditionsforTier1consolidationItispossibleforaTransitProviderstoenhanceprofitabilityinthe“flat”bypeeringstrategicallywithlargeContentProvidersContentproviderscalepromotespeering
The Internet is Flat: Revisited A.DhamdhereandC.Dovrolis.TheInternetisFlat:ModelingtheTransitionfromaTransitHierarchytoaPeeringMeshACMCoNEXT2010–Page12
InboththeHierarchicalandFlatInternet,thereisastrongcorrelationbetweenaTransitProvider’sfitnessandthesizeofitscustomerbase.(need“eyeballs”topeer)
IntheFlatInternet,however,strategicpeeringbecomesmoreimportantforSmallTransitProviders(STP)andLTPs;bothcanbeprofitablebypeeringselectivelywiththelargestcontentproviders.
IntheFlatInternet,itispossibleforaTransitProvidertotransitionfromunprofitabilitytoprofitabilitybypeeringstrategically,particularlywithlargeContentProviders;suchatransitionislesslikelyintheHierarchicalInternet.
The Opportunity Presented by Peering Content instead of relying on Tier 1 Transit
The Internet is Flat: Revisited A.DhamdhereandC.Dovrolis.TheInternetisFlat:ModelingtheTransitionfromaTransitHierarchytoaPeeringMeshACMCoNEXT2010–Page12
A Small Transit Provider Case Study
2017–(after18NANOGs)PacketOpticalServiceProviderTier2ISP80%Peering20%transitMorethan100Ginupstreamports
2011–JoinedNANOGTelephoneCompany(ILEC-CLEC)Tier3ISP100%transit(twoOC-12s)
A Small Transit Provider Case Study
AS19653–SmallTransitProviderinClimax,MichiganFoundedin1911asClimaxTelephonedbaCTSTelecom
IndependentILEC-CLEC-ISP.CLLI=CLMXMIXI
Network Data Source for Graphs
A Small Transit Provider Case Study
DailySolarWindsNPM95thPercentilereportscollectedsince2010
SOURCE:CTSTelecom
0Mbps1,000Mbps2,000Mbps3,000Mbps4,000Mbps5,000Mbps6,000Mbps7,000Mbps8,000Mbps9,000Mbps10,000Mbps11,000Mbps12,000Mbps13,000Mbps14,000Mbps15,000Mbps16,000Mbps17,000Mbps18,000Mbps19,000Mbps20,000Mbps21,000Mbps22,000Mbps23,000Mbps24,000Mbps
Nov-10
Nov-11
Nov-12
Nov-13
Nov-14
Nov-15
Nov-16
Nov-17
CabinetinIXPPeeringExchange
1stGigPNItoContent
FirstCDNServer
CDNServer
CDNServer
Transitiontoall10GPorts
SecondIXPCabinet
L-RootHost
J-RootHostThirdIXP
PeeringDB
RadBRoutingRegistry
3rdPeeringExchange
A Small Transit Provider Case Study
“IntheFlatInternet,however,strategicpeeringbecomesmoreimportantforSmallTransitProviders…”
A Small Transit Provider Case Study
A.DhamdhereandC.Dovrolis.TheInternetisFlat:ModelingtheTransitionfromaTransitHierarchytoaPeeringMeshACMCoNEXT2010–Page12
SOURCE:CTSTelecom
AS19653 Traffic Mirrors the US IP Traffic Curve
A Small Transit Provider Case Study
U.S.IPtrafficisprojectedtogrow2.5xinthenextfiveyears
AS19653 Evolution of Transit to Peering
A Small Transit Provider Case Study SOURCE:CTSTelecom
0Mbps
2,000Mbps
4,000Mbps
6,000Mbps
8,000Mbps
10,000Mbps
12,000Mbps
14,000Mbps
16,000Mbps
18,000Mbps
20,000Mbps
22,000Mbps
2010 2011 2012 2013 2014 2015 2016 2017 2018
Peering Transit
Peering 0% 0% 36% 66% 73% 80% 86% 88% 82%
Transit 100% 100% 64% 34% 27% 20% 14% 12% 18%
AS19653 From Transit to IXP Peering to CDN
A Small Transit Provider Case Study SOURCE:CTSTelecom
0Mbps1,000Mbps2,000Mbps3,000Mbps4,000Mbps5,000Mbps6,000Mbps7,000Mbps8,000Mbps9,000Mbps10,000Mbps11,000Mbps12,000Mbps13,000Mbps14,000Mbps15,000Mbps16,000Mbps17,000Mbps18,000Mbps19,000Mbps20,000Mbps21,000Mbps22,000Mbps23,000Mbps24,000Mbps
Nov-10
Jan-11
Mar-11
May-11
Jul-1
1
Sep-11
Nov-11
Jan-12
Mar-12
May-12
Jul-1
2
Sep-12
Nov-12
Jan-13
Mar-13
May-13
Jul-1
3
Sep-13
Nov-13
Jan-14
Mar-14
May-14
Jul-1
4
Sep-14
Nov-14
Jan-15
Mar-15
May-15
Jul-1
5
Sep-15
Nov-15
Jan-16
Mar-16
May-16
Jul-1
6
Sep-16
Nov-16
Jan-17
Mar-17
May-17
Jul-1
7
Sep-17
Nov-17
Jan-18
Mar-18
May-18
Jul-1
8
Sep-18
Transit IXPeering CDNPeering
Percentage of Total Traffic AS19653 Transit/Peering/CDN
A Small Transit Provider Case Study SOURCE:CTSTelecom
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
11/10/2010
1/10/2011
3/10/2011
5/10/2011
7/10/2011
9/10/2011
11/10/2011
1/10/2012
3/10/2012
5/10/2012
7/10/2012
9/10/2012
11/10/2012
1/10/2013
3/10/2013
5/10/2013
7/10/2013
9/10/2013
11/10/2013
1/10/2014
3/10/2014
5/10/2014
7/10/2014
9/10/2014
11/10/2014
1/10/2015
3/10/2015
5/10/2015
7/10/2015
9/10/2015
11/10/2015
1/10/2016
3/10/2016
5/10/2016
7/10/2016
9/10/2016
11/10/2016
1/10/2017
3/10/2017
5/10/2017
7/10/2017
9/10/2017
11/10/2017
1/10/2018
3/10/2018
5/10/2018
7/10/2018
9/10/2018
Transit IXPeering CDNPeering
A Small Transit Provider Case Study
Network Snapshot AS19653
CAIDAASrank:1483IPsinCustomerCone(v4):143,104InternetExchanges:3PrefixesOriginated(all):12PrefixesOriginated(v4):8PrefixesOriginated(v6):4PrefixesAnnounced(all):57PrefixesAnnounced(v4):46PrefixesAnnounced(v6):11BGPPeersObserved(all):423BGPPeersObserved(v4):417BGPPeersObserved(v6):261IPsOriginated(v4):90,624ASPathsObserved(v4):91,578ASPathsObserved(v6):19,424
SOURCE:A.DhamdhereCAIDA
PeersvisibleafterjoiningRouteViews
A Small Transit Provider Case Study
Game Changers
• JoiningNANOGCommunity• EstablishingIXPpresence• JoinedPeeringExchange• JoinedPeeringDB• Read“TheInternetisFlat”• ImplementedNetFlowanalysis• DevelopingNANOG“savoirfaire”• “Dr.Peering”Website(ThankstoBillNorton!)• SupportofContentProviders• MentoringfromtheNANOGcommunity
A Small Transit Provider Case Study
Challenges and Cautions for Small Providers
• Unlessyouhavealargeenoughnumberof“eyeballs”onyournetworkandahighenoughtrafficlevel,peeringdoesnotmakeeconomicsense
• Peeringrequiresasignificantamountoftechnicalexpertiseandcommitmentsofresources.
• ConnectivitytoInternetExchangePointsisnottrivial.IdeallyaprovidershouldbeattwoIXPsandredundantnetworkconnectionsarebest.SelectiveContentProvidersrequirepeeringatmultiplelocations.
• ThefallingpriceofTransitmakesthecaseforpeeringforasmallprovidereconomicallychallenging:sometimesbuyingTransitiseasier.
• YoumusthaveeconomicalaccesstofibertransporttoreachtheIXP.