Download - Peering & Internet eXchange Points AfNOG 2001 Meeting 12 th May, 2001. Miklin Hotel Accra – Ghana
Peering & Internet eXchange Points
AfNOG 2001 Meeting12th May, 2001.
Miklin HotelAccra – Ghana
Abstract
Internet Service Provider (ISP) Peering has emerged as one of the most important and effective ways of improving efficiency of service and operation. Peering is the interconnection mutual business arrangement between at least two Service Providers whereby each directly exchanges traffic to and from each other's clients. Peering relationships are sought primarily because peering reduces cost and reliance on purchased Internet bandwidth and/or transit. Secondly, peering lowers inter-AS traffic latency. By avoiding a transit provider hop in between ISPs traffic between peering ISPs has lower latency. So how is peering done and what are the issues and structures of peering?
In peering arrangements, especially when two ISPs of similar size want to interconnect it becomes a problem. The setting up of Internet Exchange Paints (IXP), which is the neutral interconnection point of traffic exchange between peering ISPs has been a way to solving peering problems.
In this paper, focus is made on issues regarding peering and Internet Exchanges, structures, roles and the decision making process which further breaks down into three distinct major processes as follows: Identification (Traffic Engineering Data Collection Analysis), Contact and Qualification (Initial Peering Negotiations) and Implementation Discussion (Peering Methodology). The first phase identifies 'the who‘and 'the why', while the second phase focuses on 'the how'.
Background to ISP Peering
The global telephone system and the Internet share a major common attribute of value to all users of their respective services - that of comprehensive connectivity
Preservation of such connectivity became a problem due to the growth of the Internet.
Transition and Financial consideration necessary to guarantee the long term viability of the medium were also problems
Peering is a one way to solving these problems The Interconnection points for traffic exchange between peering ISPs on the
Internet is the Internet eXchange Point Exchange Points are high-speed (not less than 100mb) switch network, with
current physical configuration of today being a mix of FDDI/ATM switches
General Peering Models
Takes place within one of three general models First is that of bilateral settlements The second model is that of the Sender Keep All (SKA) The third is that of transit fees Within the Internet to date only the second and third models can be observed
within the deployed infrastructure SKA arrangement is only stable where the parties involved perceive equal benefit
from the interconnection The third model (transit fees) is commonly seen as a component of the
relationship between a transit provider and a local access provider, where the local provider is charged for transit services within a larger service agreement.
There is also that of multilateral settlement, which has been developed at neutral exchanges.
Issues & Structures
Within the Internet the common basis of the settlement appears to be one of flat fees, without reference to exchanged data volumes
Significant issues include the status of an ISP - a peer to, or a client of another ISP Another is that of imbalance in traffic exchange There are three generic peering structure options:
direct point-to-point interconnection, multilateral exchange points and mutual bilateral exchange points
Issues & Structures
Direct point-to-point peering Each bilateral peer arrangement is implemented through a physical
interconnection Multilateral exchange points
Seen within the CIX exchange architecture, and within the European EBONE structure
Architecture is the enforcement of a single multilateral peering structure Stable when all parties are in a position of approximate equal positioning with
respect to the mutual traffic exchange Mutual bilateral exchange points
Offers the greatest degree of flexibility Have been used for some years within the United States
Decision Making Process
The decision making process involves three main phases These phases identifies ‘the why’, ‘the who’ and ‘the how’ of Peering and
Exchange Points Setup
The Three Major Decision Making Process Identification of Potential Peer: Traffic Engineering Data Collection and
Analysis Contact & Qualification: Initial Peering Negotiation Implementation Discussions: Peering Methodology
Phase One:
Identification of Potential PeerTraffic Engineering Data Collection and Analysis
An interview conducted by W.B. Norton with some ISPs highlighted a common concern: traffic destined for a competitors customer located across the street may need to traverse a couple of transit providers across great distances (with high latency) before interconnecting. The worst example was that traffic between the United Arab Emirates and Saudi Arabia must traverse an exchange point in Washington DC. But through direct interconnections (through direct circuits or regional exchange points) ISP customers realize better performance
Reduces Transit Costs. Internet transit costs dominates the decision making process Peering reduces transit costs
Lower Latency Direct interconnection with peers lowers latency in traffic between peering
ISP's High Speed, low latency access to sites for end-users
Scaling Bandwidth Continued success of the Internet depends on increasing bandwidth Since packet loss and latency slows traffic consumption, there is benefit from
a low latency, low packet loss Internet Enables POP Activity
Provides a hardened and scalable environment in which to operate POPs
Motivations: Why do you need to peer?
ISP A
ISP B
Transit ISP
Transit Cost ($$)
Transit Cost ($$)
Peering provides routes only to each others customers
Transit vs. Peering Interconnection
Peering between ISP A and ISP B
Transit from ISP A and ISP B to Upstream or Transit ISP
Variety of Criteria are used to identify Potential Peers, such may include the following:
Quantities of traffic Business Credibility Interconnection Policies
The Criteria: Whom do you peer with?
Yes
Yes
Yes
Yes
Yes
Part of broadBusiness
Relationship?
DominantTraffic Flow?
Large new Customer impact?
Traversing Expensive transit
cost?
Will PeeringHave a positive
effect on mynetwork?
Proceed to Phase 2Contact Peer
A Simplified Peering Selection Decision Tree
Migration Path from Transit to Peering
Interviews conducted with some tier 2 ISPs highlighted an emerging peering transition strategy:
Access the Internet via transit from a global provider Pursue peering arrangements on public switches at exchange points to
reduce load on transit links and improve performance Migrate high traffic peering interconnections to private interconnections (via
fiber or direct circuits) Ultimately migrate traffic away from transit purchase and negotiate (free or
for-fee) peering with former transit provider
Interview was conducted by W. B. Norton (Equinix Inc.) in the U.S.
Phase Two:
Contact & QualificationInitial Peering Negotiation
ISPs should have a person or group in the organization specifically tasked with traffic engineering issues
The first step in a peering arrangement is for one the parties to initiate the contact with the potential peer partner
Finding the right person to speak with is a difficult intensive task Second step, though optional is the negotiation and signing of mutual non-
disclosures The third step is to evaluate requirements and capability Each party then decides whether to stay or to walk away until criteria and
requirements are met
Principles for Peering Arrangements
However, some principles are needed to be considered during preparations for peering arrangements. These include:
providing interconnection agreement under non-discriminatory and
transparent terms and conditions; providing interconnection agreement at non-discriminatory cost-oriented
rates and comparable quality to that which it provides itself; negotiate interconnection in good faith; and ensure that interconnection agreements, or a reference offer, are made
public.
Interconnection is a necessary preliminary to traffic exchange
Contact & Qualification Decision Tree
Proceed to Phase 3Implementation
of Discussion
Peering @ orpersonal contact
Exchange Point Contact
list
tech-c oradmin-c in DNS/ASNRegistry
Operations Forum
Trade Shows
Sales Force
Larger business
transactions
InitialContact
Do both parties findmotivation to continue
peering discussion?
Close Discussion
Yes No
Sign NDA,See Policies
Share trafficdata, BLPA
Phase Three:
Implementation DiscussionsPeering Methodology
Since peering is of mutual benefit, both parties next explore the interconnection method(s) that most effectively exchange traffic to and from each other's customers
The primary goal is to establish mutual point(s) of interconnection, and secondarily detail optimal traffic exchange behaviours
ISPs face three options for interconnections: Direct Circuit Interconnection, Exchange-Based Interconnection or Some global combination thereof
The preferred methodology to adopt depends on the number of peers participating and interconnecting at the Exchange Point
And also on the bandwidth required for the Exchange Interconnections ISPs that expect to interconnect at high or rapidly increasing bandwidth within the
region, or expect interconnections with more than five parties in the region often prefer the exchange-based solution where we have the multilateral exchange points structure
Those that do not anticipate a large number of regional interconnects prefer direct-circuit (the direct point to point structure) and typically decide to split the costs of interconnection with the peer by region
Peering Arrangement Goals
get peering set up as soon as possible, minimize the cost of the interconnection and their transit costs maximize the benefits of a systematic approach to peering execute the regional operations plan as strategy dictates (may be architecture /
network development group goal), and fulfill obligations of larger business agreement.
Direct-Circuit Interconnection
Interconnection takes place in one of two forms: Peering and Transit In either case, it requires provision of bandwidth between parties Direct Point-to-Point interconnection model requires the lease of point-to-point
circuits between parties that scale linearly with the number of ISPs The circuits traverse miles of fiber underground, subject to outages due to
construction (and the rapid proliferation of backhoes) As the bandwidth requirements grow, ISPs need to upgrade circuits to many
different places; they are unable to take advantage of traffic aggregation over a very high bandwidth circuit
Cost of this interconnection strategy does not scale either
Direct-Circuit Interconnection
SSS
GGG
UUU
A A A
S
G
U
A
CCCC
Point-to-point circuits for ISP
interconnection in an exchange
Direct-Circuit Interconnection
Point-to-point circuits for ISP interconnection in an exchange (a simpler diagram)
ISP B
ISP A
ISP C
R2
R1
R3
Exchange-Based Interconnection
An alternative to the direct circuit interconnection model is for ISPs to purchase instead a much larger circuit into the Neutral Internet Business Exchange to take advantage of the relatively inexpensive and the rich connectivity mesh within the exchange
Efficient to scale a single high bandwidth, point-to-point circuit than many lower speed point-to-point circuits
The greater the dependence on the interconnection, the more hardened and scalable that interconnection environment should be
In a Neutral Internet Business Exchange Model, ISPs have access to an additional source of revenue: transit sales to content providers and ISPs
Exchange-Based Interconnection
ISP A R1
ISP C R3
ISP E R5 ISP FR6
ISP DR4
ISP BR2
R0ExchangePoint
Exchange PointHigh-Speed Switch
Each ISP has its own transit uplink
Peering Implementationand Operation
Exchange Environment Selection Criteria
Deployment Issues
ISP CurrentPresences
TelecomAccessIssues
CostIssues
BusinessIssues
Operations Issues
CredibilityIssues
Exchange Population
Existing Vs.Emerging
Exchange Environment Selection Criteria
Telecommunication Cost Issues
Have to do with getting telecommunications services into the exchange How fast can circuits be bought into the interconnection environment? How many carriers compete for my business for circuits back to my local Point of
Presence (POP)? For facilities-based ISPs, what is the cost of trenching into the exchange (how far
away and what obstacles present themselves)? Are there nearby fiber providers that lease strands? These answers will answer the most important question to ISPs:
How fast can my peer and I get connectivity into the exchange?
The answers to these questions strongly impact the desirability of the exchange environment
Deployment Issues
Have to do with getting equipment into the exchange How do I get my equipment into the exchange (assuming it supports collocation)?
Do I ship equipment in or do I have to bring it with me as I fly in? Will someone act as remote hands and eyes to get the equipment into the
racks or do I do the installation myself? What are the costs associated with deployment (travel, staff time, etc.)? Does the exchange have sufficient space, power?
The answers to these questions impact the deployment schedule for the ISP engineers and the costs of the interconnection method
ISP Current Presences
Based on the observations that the most inexpensive and expedient peering arrangements are the ones made between ISPs that are already located in the same exchange
The assumption here is that there is sufficient capacity to interconnect. Cross-connects or switching fabrics can easily establish peering within hours or at most days
Operations Issues
Focus should at this stage shift to operations activities allowed within the Exchange
Does the exchange allow private network interconnections? Are there requirements to connect to a central switch? How secure is the facility? Is there sufficient power, capacity at the switch, space for additional racks, real
time staff support? Is it easy to upgrade my presence over time?
Upgrading in this context means the ability to increase the speed of circuits into the exchange, the ability to purchase dark fiber, the ability to increase the number of racks and cross connects in the exchange, the ease of increasing the speed of interconnection. ISPs will prefer bandwidth-rich, ISP-friendly exchanges to those with restrictions over future operations.
Business Issues
Perhaps the most far-reaching business issues are strategic, as an ISP, ask these questions:
Do we want to support this exchange operator Do the interests of Exchange partners enhance or conflict with ours?
“Bandwidth, strategic partner alliances, and corporate ties often override the technical justification.”
Will using this exchange support a competitor (contribute to their net income, their credibility, their positioning)?
Does the exchange have requirements (require use of their carrier or ISP services) that limit the market for services within the exchange?
Cost Issues
Crosses all other criteria What is the cost of using this exchange? What are the rack fees, cross connect fees, port fees, and installation fees? What is the future operating fees going to be? What are the motivations and parameters surrounding these fees?
Exchange Population Issues
These issues go to the other (side) benefits to using this exchange, Are there other ISPs there that are peering candidates?
Are there transit sales possible at the exchange? In the context of the credibility issue discussed above, who will likely be at the
exchange in the future, and when will the cost of participation equal the value of the interconnection (also known as the Critical Mass Point)?
Existing vs. New Exchanges
There are many existing exchanges and also emerging ones Which one would benefit you as an ISP intending to go into peering with others? Preferences for an New Exchange
Chronic traffic congestion can influence the decision to plan to peer in an existing exchange or wait until a better exchange opens
Long-term benefits (scalability) may lead to preferring a next generation exchange. However, all else considered equal, ISPs generally prefer an existing exchange to an emerging one
How important are these Issues?
These issues where not listed in any particular order
An interview conducted by W.B. Norton with some ISPs in the U.S. reveals that they shared varied weightings of importance of each of these issues. To some the most important issues where business related, and others weighted more highly the flexibility of ongoing operations. Each ISP will place higher or lower importance on different issues and not surprisingly select operations environment based on their specific criteria. It was noted that they all seem to agree upon the general issues within which their criteria operate.
Summary
Looked through a rough description of the decision process ISPs follow to obtain peering relationships and establish peering
It focuses on the elements of the decision that lead to the selection of a specific exchange environment for peering
It can be concluded that Peering has some important advantages, with the issue of reduced cost and improvement in performance (lower latency) as the primary benefits
The goals of peering ISPs include: to get peering set up in the soonest possible time; to minimize the cost of the interconnection and their transit costs; to maximize the benefits of a systematic approach to peering; to execute the regional operations plan as strategy dictates (may be
architecture/network development group goal) and finally, to fulfill obligations of larger business agreement.
It should also be of critical note to potential peers that the need to make proper decision before joining an Internet eXchange is very critical
One major challenge facing Peering Coordinators is the identification of potential peers and initiating discussions