1 © 2001, cisco systems, inc. all rights reserved. deploying mpls traffic engineering rodrigo...

1© 2001, Cisco Systems, Inc. All rights reserved.© 2001, Cisco Systems, Inc. All rights reserved.© 2001, Cisco Systems, Inc. All rights reserved.

Deploying MPLS Traffic Deploying MPLS Traffic Engineering Engineering

Rodrigo [email protected]

Consulting Systems EngineeringLatin America Core Technologies Group

mailto:[email protected]

2© 2001, Cisco Systems, Inc. All rights reserved.© 2001, Cisco Systems, Inc. All rights reserved.© 2001, Cisco Systems, Inc. All rights reserved.

What It Is, How It Works, and How to Use It

2

© 2001, Cisco Systems, Inc. All rights reserved. 3© 2001, Cisco Systems, Inc. All rights reserved. 3© 2001, Cisco Systems, Inc. All rights reserved. 3

Agenda

• How MPLS-TE Works

• Basic Configuration

• Knobs! Knobs! Knobs!

• Deploying and Designing


How MPLS-TE Works

• How MPLS-TE works

What good is MPLS-TE?

Information distribution

Path calculation

Path setup

Forwarding traffic down a tunnel


What Good Is MPLS-TE?

• There are three kinds of networks

1. Those that have plenty of bandwidth everywhere

2. Those with congestion in some places, but not in others

3. Those with constant congestion everywhere

• The first kind always evolves into the second kind!



• MPLS-TE introduces a 4th kind:1. Those that have plenty of bandwidth everywhere

2. Those with congestion in some places, but not in others

3. Those with constant congestion everywhere

4. Those that use all of their bandwidth to its maximum efficiency, regardless of shortest-path routing!

• MPLS-TE can help turn #2 into #4If you have #1, you probably don’t need MPLS-TE—yet

If you have #3, you’re stuck—you either need morebandwidth (or less traffic)



• Multi protocol label switching—traffic engineering

• Magic problemsolving labor substitute which is totally effortless

This Stuff Takes Work, but It’s Worth It!!!

What Is It Not?What Is MPLS-TE?


Information Distribution

• You need a link-state protocol as your IGP

IS-IS or OSPF

• Link-state requirement is only for MPLS-TE!

Not a requirement for VPNs, etc!


Need for a Link-State Protocol

• Why do I need a link-state protocol?

To make sure info gets flooded

To build a picture of the entire network


Router F

The Problem with Shortest-Path

• Changing to A->C->D->E won’t help

Router C Router D

Router G

80Mb Traffic

80Mb Traffic

35Mb Drops!

35Mb Drops!Router A

Router B

NodeNode Next-HopNext-Hop CostCostBB 1010BB

FF 3030BB

CC 1010CCDD 2020CCEE 2020BB

GG 3030BB

OC-3

OC-3

DS3

DS3

DS3OC-3

OC-3

• Some links are DS3, some are OC-3

• Router A has 40Mb of traffic for Route F, 40Mb of traffic for Router G

• Massive (44%) packet loss at Router B->Router E!

Router E


Router F

40Mb40Mb

What MPLS-TE Address

• Router A sees all links

• Router A computes paths on properties other than just shortest cost

• No link oversubscribed!

OC-3

OC-3

DS3

DS3

DS3OC-3

Router C

Router E

Router D

Router G

Router A

Router B

40Mb40Mb


F 30Tunnel 0


GG 3030Tunnel 1Tunnel 1

OC-3


How MPLS-TE Works




Path calculation

Path setup




• IS-IS

Uses Type 22 TLVs

See draft-ietf-isis-traffic

• OSPF

Uses type 10 (opaque area—local) LSAs

See draft-katz-yeung-ospf-traffic



• IS-IS and OSPF propagate the same information!

Link identification

TE metric

Bandwidth information (physical, reserveable, available)

Attribute flags



• TE flooding is local to a single {area|level}

• Inter-{area|level} TE harder, but possible (think PNNI)


How MPLS-TE Works




Path calculation

Path setup



Path Calculation

• Modified Dijkstra at tunnel head-end

• Often referred to as CSPF

Constrained SPF

• …or PCALC (path calculation)



F 30Tunnel 0


GG 3030Tunnel 1Tunnel 1

Router F

Path Calculation

• PCALC takes bandwidth, other constraints into account

• Paths calculated, resources reserved if necessary

• End result: Bandwidth used more efficiently!

40Mb40Mb

OC-3

OC-3

DS3

DS3

DS3OC-3

Router C

Router E

Router D

Router G

Router A

Router B

40Mb40Mb

OC-3


Path Calculation

• What if there’s more than one path that meets the minimum requirements (bandwidth, etc.)?

• PCALC algorithm:

Find all paths with the lowest IGP cost

Then pick the path with the highest minimum available bandwidth along the path

Then pick the path with the lowest hop count (not IGP cost, but hop count)

Then just pick one path at random


How MPLS-TE Works




Path calculation

Path setup



Path Setup

• Cisco MPLS-TE uses RSVP

• RFC2205, plus draft-ietf-mpls-rsvp-lsp-tunnel (RSVP-TE)

• Once the path is calculated, it is handed to RSVP

• RSVP uses PATH and RESV messages to request an LSP along the calculated path


Router F

Path Setup

• PATH message: “Can I have 40Mb along this path?”

• RESV message: “Yes, and here’s the label to use”

• LFIB is set up along each hop

Router B

Router C

Router E

Router D

Router G

Router A

= PATH messages

= RESV messages


How MPLS-TE Works




Path calculation

Path setup



Forwarding Traffic Down a Tunnel

• There are three ways traffic can be forwarded down a TE tunnel

Auto-route

Static routes

Policy routing

• With the first two, MPLS-TE gets you unequal cost load balancing


Auto-Route

• Auto-route = “Use the tunnel as a directly connected link for SPF purposes”

• This is not the CSPF (for path determination), but the regular IGP SPF (route determination)


Auto-Route

This Is the Physical Topology

Router FRouter H

Router B

Router C

Router E

Router D

Router G

Router A

Router I


Auto-Route

• This is Router A’s logical topology

• By default, other routers don’t see the tunnel!

Tunnel1

Router FRouter H

Router B

Router C

Router E

Router D

Router G

Router A

Router I


Auto-Route

• Router A’s routing table, built via auto-route

• Everything “behind” the tunnel is routed via the tunnel

Tunnel1

Router FRouter H

Router B

Router C

Router E

Router D

Router G

Router A

Router I


FF 3030BB


GG 3030Tunnel 1Tunnel 1HH 4040Tunnel 1Tunnel 1II 4040Tunnel 1Tunnel 1


Unequal Cost Load Balancing

• IP routing has equal-cost load balancing, but not unequal cost*

• MPLS-TE does unequal cost load balancing, using 16 hash buckets for next-hop, shared in rough proportion to configured tunnel bandwidth or load-share value

*EIGRP Has ‘Variance’, but That’s Not As Flexible


Unequal Cost: Example

Router A Router E

Router F

Router G

gsr1#show ip route 192.168.1.8Routing entry for 192.168.1.8/32 Known via "isis", distance 115, metric 83, type level-2 Redistributing via isis Last update from 192.168.1.8 on Tunnel0, 00:00:21 ago Routing Descriptor Blocks: * 192.168.1.8, from 192.168.1.8, via Tunnel0 Route metric is 83, traffic share count is 2 192.168.1.8, from 192.168.1.8, via Tunnel1 Route metric is 83, traffic share count is 1

40MB

20MB


Unequal Cost: Example

Note That the Load Distribution Is 11:5—Very Close to 2:1, but Not Quite!

gsr1#sh ip cef 192.168.1.8 internal………Load distribution: 0 1 0 1 0 1 0 1 0 1 0 0 0 0 0 0 (refcount 1) Hash OK Interface Address Packets Tags imposed

1 Y Tunnel0 point2point 0 {23} 2 Y Tunnel1 point2point 0 {34}………

Router A 40MB

20MBRouter G

Router E

Router F


Static Routing

RtrA(config)#ip route H.H.H.H 255.255.255.255 Tunnel1

Router FRouter H

Router B

Router C

Router E

Router D

Router G

Router A

Router I


Tunnel1

Static Routing

• Router H is known via the tunnel

• Router G is not routed to over the tunnel, even though it’s the tunnel tail!

Router FRouter H

Router B

Router C

Router E

Router D

Router G

Router A

Router I


FF 3030BB


GG 3030BBHH 4040Tunnel 1Tunnel 1II 4040BB


Static Routing

Static Routes Inherit Unequal Cost Load-Sharing When Recursing through a Tunnel

gsr1(config)#ip route 1.2.3.4 255.255.255.255 192.168.1.11

gsr1#sh ip cef 1.2.3.4 ………Load distribution: 0 1 0 1 0 1 0 1 0 1 0 0 0 0 0 0 (refcount 1) Hash OK Interface Address Packets Tags imposed

1 Y Tunnel0 point2point 0 {23} 2 Y Tunnel1 point2point 0 {34}………

Router A 40MB

20MBRouter G

Router E

Router F


Policy Routing

RtrA(config-if)#ip policy route-map set-tunnel

RtrA(config)#route-map set-tunnel

RtrA(config-route-map)#match ip address 101

RtrA(config-route-map)#set interface Tunnel1

Router FRouter H

Router B

Router C

Router E

Router D

Router G

Router A

Router I

Tunnel1


Policy Routing

• Routing table isn’t affected by policy routing

• Need (12.0(16)ST or 12.2T) or higher for ‘set interface tunnel’ to work

Router FRouter H

Router B

Router C

Router E

Router D

Router G

Router A

Router I


FF 3030BB


GG 3030BBHH 4040BBII 4040BB

Tunnel1Tunnel1


Forwarding Traffic down a Tunnel

• You can use any combination of auto-route, static routes, or PBR

• …But simple is better unless you have a good reason

• Recommendation: Either auto-route or statics to BGP next-hops, depending on your needs


Agenda

• Prerequisites






Basic Midpoint/Tail Configuration

(globally)

ip cef {distributed}

mpls traffic-eng tunnels

(per interface)

mpls traffic-eng tunnels



(if IGP = OSPF)

router ospf <x>

mpls traffic-eng router-id Loopback0

mpls traffic-eng area <y>



(if IGP = IS-IS)

router isis <x>

mpls traffic-eng router-id Loopback0

mpls traffic-eng level-{1,2}

metric-style wide


Basic Head-End Configuration

• Head-end needs the 4–5 ‘mid/tail’ lines

• But wait—there’s more!



• Create the tunnel interface

interface Tunnel0

ip unnumbered Loopback0

tunnel mode mpls traffic-eng

tunnel source Loopback0

tunnel destination <tunnel endpoint>

tunnel mpls traffic-eng autoroute announce

tunnel mpls traffic-eng path-option 10 dynamic



• Total configuration:

1 line globally

1 line per interface

2 lines if OSPF

3 lines if IS-IS

+ 7 lines per tunnel at head-end

Not really much to the basic configuration


Agenda

• Prerequisites






Knobs! Knobs! Knobs!

• Influencing the path selection

• Auto-bandwidth

• Fast reroute

• DiffServ-Aware Traffic Engineering




Bandwidth

Priority

Administrative weight

Attributes and affinity


Bandwidth

• Per-interface command

• X = amount of reservable BW, in K

• Default: X=75% of link bandwidth

ip rsvp bandwidth <x>


Bandwidth

• Per-tunnel command

• Tunnel default: 0 Kb

tunnel mpls traffic-eng bandwidth <Kb>


Priority

• Configured on tunnel interface

• S = setup priority (0–7)

• H = holding priority (0–7)

• Lower number is more important, or better

tunnel mpls traffic-eng <S> {H}


Administrative Weight


• X = 0–4,294,967,295

• Gives a metric that be considered for use instead of the IGP metric

• This can be used as a per-tunnel delay-sensitive metric for doing VoIP TE

mpls traffic-eng administrative-weight <X>


Delay-Sensitive Metric with Administrative Weight

• Configure admin weight = interface delay

• Configure VoIP tunnels to use TE metric to calculate the path cost (see the PCALC algorithm earlier in these slides)

tunnel mpls traffic-eng path-selection metric {te|igp}


Attributes and Affinity


mpls traffic-eng attribute-flags <0x0-0xFFFFFFFF>




• Mask is a collection of do-care bits

• ‘affinity 0x2 mask 0xA’means ‘I care about bits 1 and 3 (with the values 2 and 8); bit 1 must be set, bit 3 must be 0’

tunnel mpls traffic-eng affinity<0x0-0xFFFFFFFF> {mask <0x0-

0xFFFFFFFF>}



• Q: How should I use link attributes?

• A: To exclude some links from consideration by some tunnels

• …So give a satellite link an attribute of 0x2, and any VoIP tunnels can be configured with ‘affinity 0x0 mask 0x2’




• Auto-bandwidth

• Fast reroute



Auto-Bandwidth


• Periodically changes tunnel BW reservation based on traffic out tunnel

• Timers are tunable to make auto-bandwidth more or less sensitive

tunnel mpls traffic-eng auto-bw ? collect-bw Just collect Bandwidth info on this tunnel frequency Frequency to change tunnel BW max-bw Set the Maximum Bandwidth for auto-bw on this tunnel min-bw Set the Minimum Bandwidth for auto-bw on this tunnel <cr>




• Auto-bandwidth

• Fast reroute



Fast Reroute

• In an IP network, a link failure causes several seconds of outage

Link Failure DetectionLink Failure Detection

Information PropagationInformation Propagation

Route RecalculationRoute Recalculation

ThingThing

IGP Timers, NetworkSize, Collective

Router Load


Router Load

Media- and Platform-specific

Media- and Platform-specific ~μsecs (POS + APS)~μsecs (POS + APS)

~5–30 sec~5–30 sec

LSDB Size, CPU Load LSDB Size, CPU Load ~1–3 sec~1–3 sec

DependencyDependency TimeTime


Fast Reroute

• In an MPLS network, there’s more work to be done, so a (slightly) longer outage happens


Route RecalculationRoute Recalculation

ThingThing

~Usecs (POS + APS)~Usecs (POS + APS)

~5–30 sec~5–30 sec

LSDB Size, CPU Load LSDB Size, CPU Load ~1–3 sec~1–3 sec


New LSP SetupNew LSP SetupNetwork Size,

CPU Load Network Size,

CPU Load ~5–10 sec~5–10 sec



Router Load


Router Load




Three Kinds of Fast Reroute

• Link protection

• Node protection

• Path protection


Link Protection

• TE Tunnel A -> B -> D -> E

Router DRouter B

Router C

Router ERouter A


Link Protection

• B has a pre-provisioned backup tunnel to the other end of the protected link (Router D)

• B relies on the fact that D is using global label space

Router D

Router C

Router A Router B Router E


Link Protection

• B -> D link fails, A -> E tunnel is encapsulated in B -> D tunnel

• Backup tunnel is used until A can re-compute tunnel path as A -> B -> C -> D -> E (10–30 seconds or so)

Router C

Router DRouter A Router B Router E


Link Protection

• On tunnel head-end:

tunnel mpls traffic-eng fast-reroute

• On protected link:mpls traffic-eng backup-path <backup-tunnel>

Router DRouter B Router ERouter ERouter A


Node Protection

• Solution: protect tunnel to the hop past the protected link

Router D Router FRouter B Router ERouter A


Path Protection

• Path protection: Multiple tunnels from TE head to tail, across diverse paths

Router D Router FRouter B Router ERouter A


Path Protection

Head-end Switch-overto Protect LSP

Head-end Switch-overto Protect LSP

Network Size, CPU Load

Network Size, CPU Load

~Msec~Msec

Path vs. Local Protection

Local (Link/Node) Protection


Local Switch-over toProtect Tunnel

Local Switch-over toProtect Tunnel

ThingThing

RP-> Communication Time

RP-> Communication Time


Media- and Platform-specific ~Usecs (POS + APS)~Usecs (POS + APS)

~Few msec or less~Few msec or less



ThingThing


Media- and Platform-specific ~Usecs (POS + APS)~Usecs (POS + APS)




Router Load


Router Load~5–30+ sec~5–30+ sec




• Auto-bandwidth

• Fast reroute



DiffServ-Aware Traffic Engineering

• MPLS can advertise and reserve bandwidth on a link

• Works great, but what if you send a mix of LLQ (EF) and BE traffic down a TE tunnel?

• Need some way to differentiate and reserve LLQ (EF) bandwidth on a link



• 100MB reservable on C<->E, with a 30MB LLQ/EF (QoS Config)

• 2 tunnels across C<->E link

• 40MB each tunnel

• What happen as when both tunnels send 20MB of VoIP traffic?

Router A

Router B

Router C

Router E

Router D Router F

Router G



• Problem: Only one pool on an interface, no way to differentiate what types of traffic are carried!

• Solution: Advertise more than one pool!

30MB LLQ - 40MB EF traffic = 10MB not LLQ’d!Router A

Router B

Router C

Router E

Router D Router F

Router G



ip rsvp bandwidth <x> sub-pool <y>

• ‘this link has available bandwidth of X, Y of which is in a sub-pool’

• Not quite two pools, really—no sense in withholding bandwidth from global availability if it’s not in use

• …Which means sub-pool tunnels need to have a better priority than non-sub-pool tunnels



tunnel mpls traffic-eng bandwidth <x> sub-pool

• ‘This tunnel wants to reserve X Kbps from a sub-pool’

• Sub-pool bandwidth is looked at instead of global pool bandwidth

• If sub-pool bandwidth is not available, tunnel won’t come up


Agenda

• Prerequisites






Deploying and Designing

• Deployment methodologies

• Scalability

• Management

• Security


Deployment Methodologies

• Two ways to deploy MPLS-TE

As needed to clear up congestion

Full mesh between a set of routers

• Both methods are valid, both have their pros and cons


As Needed

• All links are OC12

• A has consistent 700MB to send to C

• ~100MB constantly dropped!

Case Study: A Large US ISP

Router A

Router B

Router D Router E

Router C


As Needed

• Solution: Multiple tunnels, unequal cost load sharing!

• Tunnels with bandwidth in 3:1 (12:4) ratio

• 25% of traffic sent the long way

• 75% sent the short way

• No out-of-order packet issues— CEF’s normal per-flow hashing is used!

Router A

Router B

Router D Router E

Router C


As Needed

• From Router A’s perspective, topology is:

Router A

Router B

Router D Router E

Router C


As Needed

• As needed—Easy, quick, but hard to track over time

• Easy to forget why a tunnel is in place

• Inter-node BW requirements may change, tunnels may be working around issues that no longer exist


Full Mesh

• Put a full mesh of TE tunnels between routers

• Initially deploy tunnels with 0 bandwidth (some folks deploy full mesh just to get router-to-router (pop-to-pop) traffic matrix)

• Watch tunnel interface statistics, see how much bandwidth you are using between router pairs

Tunnels are interfaces—use IF-MIB!

Make sure that tunnel <= network BW


Full Mesh

• Physical topology is:

Router A

Router B

Router D Router E

Router C


Full Mesh

• Logical topology is**Each link is actually 2 unidirectional tunnels

• Total of 20 tunnels in this network

Router A

Router B

Router D Router E

Router C


Full Mesh

• Things to remember with full mesh

N routers, N*(N-1) tunnels

Routing protocols not run over TE tunnels— unlike an ATM/FR full mesh!

Tunnels are unidirectional—this is a good thing

…Can have different bandwidth reservations in two different directions




• Scalability

• Management

• Security


Scalability

• Tests were done on a GSR

• RSP4, RSP8, VXR300, VXR400 will be similar

How Many Tunnels on a Router?

Number of Head-End

Tunnels

Number of Head-End

Tunnels

Number of Tail-End

Tunnels

Number of Tail-End

Tunnels

Number of Mid-Points

Number of Mid-PointsCodeCode

12.0ST12.0ST 600600 10,00010,000 5,0005,000


Scalability

http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/120newft/120limit/120st/120st14/scalable.htm

• Or just search CCO for “Scalability Enhancements for MPLS Traffic Engineering”







• Scalability

• Management

• Security


Traffic Engineering MIBs

• Interfaces MIB

• MPLS-TE-MIB

• CISCO-TE-MIB

• MPLS-DS-TE-MIB


TunnelVision

• Need a tool to help manage TE LSPs?

• TunnelVision (server and client component, will run on Solaris and Windows 2000)

• Not a network modeling tool!

Use WANDL, Orchestream, MakeSys, Opnet, and others


TunnelVision

• Cisco is also working with an external partner to add protection path calculation

• The partner has world-class algorithm development experience

• TunnelVision will feed topology to this tool, tool will calculate backup paths




• Scalability

• Management

• Security


Security

• MPLS-TE is not enabled on externally facing interfaces

• Biggest security risk is spoofed RSVP

Hacker would have to know a lot about your topography to do anything

RSVP authentication exists (rfc2747), not yet implemented, on the radar

• If you’re concerned about spoofed RSVP, then add RSVP to the ACLs you probably already use to stop spoofed BGP, OSPF, etc.

• uRPF also helps here


Conclusion

Basically, TE helps you to optimize your network resources utilization, provide a better quality of service and enhance the network and services availability.

Obrigado!Obrigado!

1 © 2001, cisco systems, inc. all rights reserved. deploying mpls traffic engineering rodrigo...

Documents