1

Internet Routing:Border Gateway Protocol (BGP)

Jennifer Rexford

Princeton University

http://www.cs.princeton.edu/~jrex/bgp-tutorial

2

Goals of This Section• Path-vector routing

– Limitations of shortest-path routing and flooding– Path-vector routing as alternative to distance-vector

• Border Gateway Protocol – BGP session running over reliable transport protocol– Announcement and withdrawal messages– Path exploration during convergence

• BGP routing policy– Import policy, decision process, and export policies

• BGP inside of an AS

• Applications of BGP– Anycast, mobility, monitoring, blackholing, …

3

Path-Vector Routing

4

Shortest-Path Routing is Restrictive• All traffic must travel on shortest paths

• All nodes need common notion of link costs

• Incompatible with commercial relationships

Regional ISP1

Regional ISP2

Regional ISP3

Cust1Cust3 Cust2

National ISP1

National ISP2

YES

NO

5

Link-State Routing is Problematic• Topology information is flooded

–High bandwidth and storage overhead–Forces nodes to divulge sensitive information

• Entire path computed locally per node–High processing overhead in a large network

• Minimizes some notion of total distance–Works only if policy is shared and uniform

• Typically used only inside an AS–E.g., OSPF and IS-IS

6

Distance Vector is on the Right Track

• Advantages–Hides details of the network topology–Nodes determine only “next hop” toward the dest

• Disadvantages–Minimizes some notion of total distance, which is

difficult in an interdomain setting–Slow convergence due to the counting-to-infinity

problem (“bad news travels slowly”)

• Idea: extend the notion of a distance vector–To make it easier to detect loops

7

Path-Vector Routing• Extension of distance-vector routing

–Avoid count-to-infinity problem–Support flexible routing policies

• Key idea: advertise the entire path–Distance vector: send distance metric per dest d–Path vector: send the entire path for each dest d

3 2 1

d

“d: path (2,1)” “d: path (1)”

data traffic data traffic

8

Faster Loop Detection• Node can easily detect a loop

–Look for its own node identifier in the path–E.g., node 1 sees itself in the path “3, 2, 1”

• Node can simply discard paths with loops–E.g., node 1 simply discards the advertisement

3 2 1“d: path (2,1)” “d: path (1)”

“d: path (3,2,1)”

9

Flexible Policies• Each node can apply local policies

–Path selection: Which path to use?–Path export: Which paths to advertise?

• Examples–Node 2 may prefer the path “2, 3, 1” over “2, 1”–Node 1 may not let node 3 hear the path “1, 2”

2 3

1

2 3

1

10

Border Gateway Protocol (BGP)

11

• Interdomain routing protocol for the Internet –Prefix-based path-vector protocol–Policy-based routing based on AS Paths–Evolved during the past 20 years

• 1989 : BGP-1 [RFC 1105], replacement for EGP• 1990 : BGP-2 [RFC 1163]• 1991 : BGP-3 [RFC 1267]• 1995 : BGP-4 [RFC 1771], support for CIDR • 2006 : BGP-4 [RFC 4271], update

Border Gateway Protocol

12

BGP Operations

Establish session on TCP port 179

Exchange all active routes

Exchange incremental updates

AS1

AS2

While connection is ALIVE exchangeroute UPDATE messages

BGP session

13

Incremental Protocol• A node learns multiple paths to destination

–Stores all of the routes in a routing table–Applies policy to select a single active route–… and may advertise the route to its neighbors

• Incremental updates–Announcement

Upon selecting a new active route, add node id to path … and (optionally) advertise to each neighbor

–Withdrawal If the active route is no longer available … send a withdrawal message to the neighbors

14

BGP Route• Destination prefix (e.g., 128.112.0.0/16)

• Route attributes, including– AS path (e.g., “7018 88”)– Next-hop IP address (e.g., 12.127.0.121)

AS 88Princeton

128.112.0.0/16AS path = 88Next Hop = 192.0.2.1

AS 7018AT&T

AS 11Yale

192.0.2.1

128.112.0.0/16AS path = 7018 88Next Hop = 12.127.0.121

12.127.0.121

15

ASPATH Attribute

AS7018128.112.0.0/16AS Path = 88

AS 1239Sprint

AS 1755Ebone

AT&T

AS 3549Global Crossing

128.112.0.0/16AS Path = 7018 88

128.112.0.0/16AS Path = 3549 7018 88

AS 88

128.112.0.0/16Princeton

Prefix Originated

AS 12654RIPE NCCRIS project

AS 1129Global Access

128.112.0.0/16AS Path = 7018 88

128.112.0.0/16AS Path = 1239 7018 88

128.112.0.0/16AS Path = 1129 1755 1239 7018 88

128.112.0.0/16AS Path = 1755 1239 7018 88

16

BGP Path Selection• Simplest case

–Shortest AS path–Arbitrary tie break

• Example–Three-hop AS path preferred

over a five-hop AS path–AS 12654 prefers path

through Global Crossing

• But, BGP is not limited to shortest-path routing–Policy-based routing AS 3549

Global Crossing

128.112.0.0/16AS Path = 3549 7018 88

AS 12654RIPE NCCRIS project

AS 1129Global Access

128.112.0.0/16AS Path = 1129 1755 1239 7018 88

17

2 AS hops, 8 router hops

AS Path Length != Router Hops• AS path may be longer than shortest AS path

• Router path may be longer than shortest path

s d

3 AS hops, 7 router hops

18

BGP Routing Policy

19

BGP Policy: Applying Policy to Routes

• Import policy–Filter unwanted routes from neighbor

E.g. prefix that your customer doesn’t own–Manipulate attributes to influence path selection

E.g., assign local preference to favored routes

• Export policy–Filter routes you don’t want to tell your neighbor

E.g., don’t tell a peer a route learned from other peer–Manipulate attributes to control what they see

E.g., make a path look artificially longer than it is

20

BGP Policy: Influencing Decisions

Best Route Selection

Apply Import Policies

Best Route Table

Apply Export Policies

Install forwardingEntries for bestRoutes.

ReceiveBGPUpdates

BestRoutes

TransmitBGP Updates

Apply Policy =filter routes & tweak attributes

Based onAttributeValues

IP Forwarding Table

Apply Policy =filter routes & tweak attributes

Open ended programming.Constrained only by vendor configuration language

21

Import Policy: Local Preference• Favor one path over another

–Override the influence of AS path length–Apply local policies to prefer a path

• Example: prefer customer over peer

AT&T Sprint

Yale

Tier-2

Tier-3

Local-pref = 100

Local-pref = 90

22

Import Policy: Filtering• Discard some route announcements

–Detect configuration mistakes and attacks

• Examples on session to a customer–Discard route if customer doesn’t own the prefix–Discard route containing other large ISPs

Patriot

Princeton

USLEC

128.112.0.0/16

23

Export Policy: Filtering• Discard some route announcements

–Limit propagation of routing information

• Examples–Don’t announce routes from one peer to another

AT&T SprintUUNET

24

Export Policy: Filtering• Discard some route announcements

–Limit propagation of routing information

• Examples–Don’t announce routes for network-management

hosts or the underlying routers themselves

USLEC

Princeton

network operator

25

Export Policy: Attribute Manipulation

• Modify attributes of the active route–To influence the way other ASes behave

• Example: AS prepending–Artificially inflate AS path length seen by others–Convince some ASes to send traffic another way

Patriot

Princeton

USLEC

128.112.0.0/16

Sprint

88 88 88

26

BGP Policy Configuration• Policy languages are vendor-specific

–Not part of the BGP protocol specification–Different languages for Cisco, Juniper, etc.

• Still, all languages have some key features–Policy as a list of clauses–Each clause matches on route attributes–… and discards or modifies the matching routes

• Configuration done by human operators–Implementing the policies of their AS–Biz relationships, traffic engineering, security, …

27

BGP Inside an AS

28

An AS is Not a Single Node• Multiple routers in an AS

–Need to distribute BGP information within the AS–Internal BGP (iBGP) sessions between routers

AS1

AS2

eBGP

iBGP

29

Internal BGP and Local Preference• Example

– Both routers prefer the path through AS 100 on the left– … even though the right router learns an external path

I-BGPAS 256

AS 300

Local Pref = 100 Local Pref = 90

AS 100

AS 200

30

Joining BGP and IGP Information• Border Gateway Protocol (BGP)

–Announces reachability to external destinations–Maps a destination prefix to an egress point

128.112.0.0/16 reached via 192.0.2.1

• Interior Gateway Protocol (IGP)–Used to compute paths within the AS–Maps an egress point to an outgoing link

192.0.2.1 reached via 10.1.1.1

192.0.2.1

10.1.1.1

31

IGP

Joining BGP with IGP Information

AS 7018 AS 88192.0.2.1

128.112.0.0/16

10.10.10.10

BGP

192.0.2.1128.112.0.0/16

destination next hop

10.10.10.10192.0.2.0/30

destination next hop

128.112.0.0/16Next Hop = 192.0.2.1

+ Forwarding Table

128.112.0.0/16

destination next hop

10.10.10.10192.0.2.0/30 10.10.10.10

32

An AS May Learn Many Routes• Multiple connections to neighboring ASes

–Multiple border routers may learn good routes–… with the same local-pref and AS path length

1

2

34

5

67

Multiple links

33

Hot-Potato (Early-Exit) Routing• Hot-potato routing

– Each router selects the closest egress point– … based on the path cost in intradomain protocol

• BGP decision process– Highest local preference– Shortest AS path– Closest egress point– Arbitrary tie break

hot potato

A B

C

DG

EF4

5

39

34

108

8

A Bdst

34

Example Applications of BGP

35

Conventional Use of BGP• An AS owns a prefixes

–Originates that prefix into BGP

• Neighboring ASes propagate the route–Add themselves to the AS path–Apply local policy for selecting and exporting

Patriot

Princeton

USLEC

128.112.0.0/16

Sprint

88 88

36

Conventions Not Always True• IP prefix corresponds to a single location?

– Multiple locations for a replicated service (e.g., DNS)

• The location stays the same over time?– Wide-area mobility (e.g., WiFi on an airplane)

• The BGP session connects immediate neighbors?– BGP monitoring (e.g., RouteViews, blackholing service)

• The originating AS owns the prefix?– Malicious or misconfigured AS (e.g., prefix hijacking)

37

IP Anycast

38

Many Services are Replicated

• Servers in many locations–Reliability: copies that fail independently–Performance: clients directed to nearby replicas

39

Anycast

• Anycast–One-to-many association of name to endpoints–Each destination represents a set of receivers–Only one receives information from the sender

• Questions–How to name the (replicated) service?

URL, host name, IP address, … –How to decide which instance receives traffic?

Network proximity, load balancing policies, …–How “sticky” should the binding be?

Each packet independent? Connection-oriented?

40

IP Anycast• Announce IP prefix in interdomain routing

– At each replica location

• Rely on global routing to direct traffic– To the “nearest” replica

63.251.179.13

63.251.179.13

63.251.179.13

41

IP Anycast: Pros and Cons

• Advantages–Completely transparent to clients and routers–Scales well for a large group of replicas–End-to-end paths automatically efficient

• Disadvantages–Pollutes the global routing system–Separate /24 for each replicated service–Does not consider server load–Different packets may reach different replicas–Slow BGP convergence after a withdrawal

42

Application-Level Anycast

• URL rewriting–Server dynamically rewrites HTML page–E.g., image at foo23.bar.com vs. foo46.bar.com

• Application-level redirection–Explicit redirection of a request to new location–E.g., HTTP 302 “Moved Temporarily”

• DNS redirection–Change mapping of domain name to address–E.g., www.cnn.com to 8.15.7.117

43

Application-Layer Anycast: Pros and Cons

• Advantages–Fine-grain control of load across group members–Can easily incorporate variety of criteria–Successive packets delivered to same replica

• Disadvantages–Need to identify location of the requesting client

Especially difficult for DNS-based redirection–Extra round-trip times for redirection–Small TTLs to prevent long DNS caching–Boot-strapping to find redirecting/lookup server

44

Open Questions• Hybrids of IP and application-level anycast

–IP anycast proxies–Route servers that “pin” the route from a sender

• New routing architectures–Designed with replicated services in mind–Support from the end-host protocols?

• Content-oriented networking–Naming and addressing content–Rather than end-host computers

45

Wide-Area Mobility

46

Maintaining Ongoing Transfers • Seamless transmission to a mobile host

A B

47

Could Let Routing Protocol Handle It• Mobile node has a single, persistent address

• Address injected into routing protocol

B12.34.45.0/24

A

12.34.45.7/32

Mobile host with IP address 12.34.45.7

48

Example: Boeing Connexion Service

• Boeing Connexion service–Mobile Internet access provider–WiFi “hot spot” at 35,000 feet moving 600 mph–Went out of business in December 2006…

• Communication technology–Plane antenna to leased satellite transponders–Ground stations serve as Internet gateways

• Using BGP for mobility–Address block (a /24) per international airplane–Ground station advertises the prefix into BGP

49

Example: Boeing Connexion Service

Internet

12.78.3.0/24

50

Boeing Approach• Advantages

–No changes to the end host–Traffic follows an efficient path to new location

• Disadvantages–Does not scale to large number of mobile hosts

Large number of routing-protocol messages Larger routing tables to store smaller address blocks

• Open questions–Is this fair to the rest of the Internet???–How should we support wide-area mobility?

51

Route Monitoring

52

Motivations for BGP Monitoring• Improve the health of the Internet

– Analyze update messages to detect attacks– Identify destinations that have become unreachable

• Characterize the Internet topology– Snapshots of the AS-level topology– Additions of new prefixes and new prefix-AS mappings– Inference of business relationships between AS

• Study routing dynamics– Transient behavior of BGP during convergence

• Input to “what if” tools– To guide possible changes to BGP routing policy

53

BGP Monitoring: A Wish List

• Ideally: knowing what the router knows–All externally-learned routes–Before policy has modified the attributes–Before a single best route is picked

• How to achieve this–Special monitoring session on routers that tells

everything they have learned–Packet monitoring on all links with BGP sessions

• If you can’t do that, you could always do…–Periodic dumps of routing tables–BGP session to learn best route from router

54

Using Routers to Monitor BGP

Talk to operational routers using SNMP ortelnet at command line

(-) BGP table dumps are expensive

(+) Table dumps show all alternate routes

(-) Update dynamics lost

(-) restricted to interfaces provided by vendors

Establish a “passive” BGPsession from a workstationrunning BGP software

(+) BGP table dumps do not burden operational routers

(-) Receives only best routes from BGP neighbor

(+) Update dynamics captured

(+) not restricted to interfaces provided by vendors

eBGP or iBGP

Atlanta

St. LouisSanFrancisco

Denver

Cambridge

Washington, D.C.

Orlando

Chicago

Seattle

Los Angeles

Detroit

Houston

New York

PhoenixSan Diego

Austin

Philadelphia

Dallas

2

Kansas City

Collect BGP Data From Many Routers

Route MonitorBGP is not a flooding protocol

56

Wide-Area BGP Monitoring• Public BGP monitors

– RouteViews, RIPE RIS, …– Both BGP table dumps and update-message logs

1

2

34

5

67

57

Example BGP Table (“show ip bgp” at RouteViews)

AS 80 is General Electric, AS 701 is UUNET, AS 7018 is AT&TAS 3786 is DACOM (Korea), AS 1221 is Telstra

Network Next Hop Metric LocPrf Weight Path* 3.0.0.0 205.215.45.50 0 4006 701 80 i* 167.142.3.6 0 5056 701 80 i* 157.22.9.7 0 715 1 701 80 i* 195.219.96.239 0 8297 6453 701 80 i* 195.211.29.254 0 5409 6667 6427 3356 701 80 i*> 12.127.0.249 0 7018 701 80 i* 213.200.87.254 929 0 3257 701 80 i* 9.184.112.0/20 205.215.45.50 0 4006 6461 3786 i* 195.66.225.254 0 5459 6461 3786 i*> 203.62.248.4 0 1221 3786 i* 167.142.3.6 0 5056 6461 6461 3786 i* 195.219.96.239 0 8297 6461 3786 i* 195.211.29.254 0 5409 6461 3786 i

58

Open Questions• BGP data widely used in research studies

– Characterize today’s Internet– Evaluate proposed protocol enhancements

• However, view of the Internet is incomplete– Feeds available only from certain vantage points

What edges in the graph do we miss?– Feeds from just 1-2 routers in participating ASes

How representative are they of other routers in the AS?– Our understanding of routing policy is incomplete

How well can we infer how the ASes are making decisions

• Still a very active area of research

59

Blackholing Service

60

Blocking Malicious Hosts• Malicious hosts send unwanted traffic

–Spam, denial-of-service, scanning, viruses, etc.–(Or, politically or socially sensitive content)

• ASes would like to block the traffic–Identify the offending IP addresses–Block communication with them

• One solution–Stop sending traffic to known bad IP addresses–Prevent bidirectional communication with them

61

BGP Blackholing Service• Identify offending IP address blocks

• Announce them (via BGP) to participants

• Participants associate prefixes with “null” routes

1

2

34

5

67

feed of bad prefixes

62

BGP Blackholing

drop packetsto 18.4.2.0/24

18.4.2.3

18.4.2.0/24is bad!

ISP

63

Conclusions• Path-vector routing

–Faster convergence than distance-vector routing–Hides information and enables flexible policies

• Border Gateway Protocol (BGP)–Prefix-based, path-vector, policy-based routing–Incremental update (announcement, withdrawal)

• Interdomain routing policy–Import policy, decision process, export policy–Policies based on biz relationships, security, …

• Other uses (abuses?) of BGP routing