raszuk mpls te

1© 1999, Cisco Systems, Inc.

MPLS Traffic MPLS Traffic EngineeringEngineering

NANOG18NANOG18

Robert Raszuk - IOS Engineering Robert Raszuk - IOS Engineering [email protected]@cisco.com

2NANOG18 - Robert Raszuk © 2000, Cisco Systems, Inc.

Location of filesLocation of files

This presentation, handouts & demo are located at: This presentation, handouts & demo are located at: ftp://ftpeng.cisco.com/rraszuk/nanog18ftp://ftpeng.cisco.com/rraszuk/nanog18

• RR_MPLS_TE_Nanog.pdf - this presentationRR_MPLS_TE_Nanog.pdf - this presentation

• TE_Monitor.pdf - show & debug commandsTE_Monitor.pdf - show & debug commands

• TE_Config.pdf - full configuration syntaxTE_Config.pdf - full configuration syntax

• TE_SampleCfg.pdf - configuration sampleTE_SampleCfg.pdf - configuration sample

• TE_DEMO.tar - Tared TE offline demo (HTML)TE_DEMO.tar - Tared TE offline demo (HTML)

• TEisistdp_1.pdf - Demo’s Lab TopologyTEisistdp_1.pdf - Demo’s Lab Topology


Traffic Engineering: MotivationsTraffic Engineering: Motivations

• Reduce the overall cost of operations by more efficient use of bandwidth resources

– by preventing a situation where some parts of a service provider network are over-utilized (congested), while other parts under-utilized

The ultimate goal is cost saving !cost saving !


Traffic Engineering: MotivationsTraffic Engineering: Motivations

• MPLS and Traffic Eng allows for one to MPLS and Traffic Eng allows for one to spread the traffic and distribute it across spread the traffic and distribute it across the entire network infrastructure like the entire network infrastructure like magnetic fields between poles while magnetic fields between poles while also providing the redundancy required also providing the redundancy required for high availability service.for high availability service.

(Eric Dean)(Eric Dean)


Without Traffic EngineeringWithout Traffic Engineering

Cars:

SFO-LAX

LAX-SFO

SAN-SMF

SMF-SAN

No Traffic Engineering

analogy

to Human Drivers


With Traffic EngineeringWith Traffic Engineering

Cars:

SFO-LAX

LAX-SFO

SAN-SMF

SMF-SAN

Traffic Engineering

analogy

to Auto Pilot


Routing solution to Traffic Routing solution to Traffic Engineering Engineering

• Construct routes for traffic streams within a service provider in such a way, as to avoids causing some parts of the provider’s network to be over-utilized, while others parts remain under-utilized

R2

R3

R1


The “Overlay” SolutionThe “Overlay” Solution

• Routing at layer 2 (ATM or FR) is used for traffic engineering

• Analogy to direct highways between SFO-LAX & SAN-SMF. Nobody enters the highway in between.

L3L3

L3L3

L3L3

L3L3

L3L3

L3L3

L3L3

L2L2

L2L2

L2L2

L2L2

L2L2

L2L2

L3L3

L3L3

L3L3

L3L3 L3L3

Physical Logical


Traffic engineering with overlayTraffic engineering with overlay

R2

R3

R1

PVC for R2 to R3 trafficPVC for R2 to R3 traffic

PVC for R1 to R3 trafficPVC for R1 to R3 traffic


““Overlay” solution: drawbacksOverlay” solution: drawbacks

• Extra network devices (cost)

• More complex network management (cost)

– two-level network without integrated network management

– additional training, technical support, field engineering

• IGP routing scalability issue for meshes

• Additional bandwidth overhead (“cell tax”)


Traffic engineering with Layer 3Traffic engineering with Layer 3R2

R3

R1

IP routing: destination-based least-cost routingIP routing: destination-based least-cost routing

under-utilized alternate pathunder-utilized alternate path

Path for R2 to R3 trafficPath for R2 to R3 traffic



Traffic engineering with Layer 3Traffic engineering with Layer 3

R2

R3

R1

IP routing: destination-based least-cost routingIP routing: destination-based least-cost routing

under-utilized alternate pathunder-utilized alternate path




Traffic engineering with Layer 3 Traffic engineering with Layer 3 what is missing ?what is missing ?

• Path computation based just on IGP metric is not enough

• Support for “explicit” routing (aka “source routing”) is not available

• Analogy: San

Jose

San

Jose


MPLS Traffic MPLS Traffic EngineeringEngineering



TE - key mechanismsTE - key mechanisms

• “Explicit” routing (aka “source routing”)

– Constrained-based Path Selection Algorithm (Example: Choose path with no congestion, avoid highways, select scenic roads etc…)

– Extensions to OSPF/ISIS for flooding of resources / policy information (Live collection of traffic statistics - pilot tests in Europe)

– MPLS as the forwarding mechanism (Auto Pilot programmed in each car when entering city)


TE - key mechanismsTE - key mechanisms

• “Explicit” routing (aka “source routing”)

– RSVP as the mechanism for establishing Label Switched Paths (LSPs)

– use of the explicitly routed LSP’s in the forwarding table


What is a “traffic trunk” ?What is a “traffic trunk” ?

• Aggregation of (micro) flows that are:

– forwarded along a common path (within a service provider)

– often from a POP to another POP

– share a common QoS requirement (if L-LSPs are used)

• Essential for scalability

D

A B

C


TE basicsTE basics

• Traffic within a Service Provider as a collection of “POP to POP traffic trunks” with known bandwidth and policy requirements

• TE provides traffic trunk routing that meets the goal of Traffic Engineering

– via a combination of on-line and off-line procedures


Requirements:Requirements:

• Differentiating traffic trunks:

– large, ‘critical’ traffic trunks must be well routed in preference to other trunks

• Handling failures:

– automated re-routing in the presence of failures

• Pre-configured paths:

– for use in conjunction with the off-line route computation procedures

• Support of multiple Classes of Service


Requirements (cont.)Requirements (cont.)

• Constraining sub-optimality:

– should re-optimize on new/restored bandwidth• in a non-disruptive fashion - maintain the existing route until the new route is established, without any double counting

• Ability to “spread” traffic trunk across multiple Label Switched Paths (LSPs)

– could provide more efficient use of networking resources

• Ability to include / exclude certain links for certain traffic trunks


Design ConstraintsDesign Constraints

• Constrained to a single routing domain

– initially constrained to a single area

• Requires OSPF or IS-IS

• Unicast traffic

• Focus on supporting routing based on a combination of administrative + bandwidth constraints


Trunks AttributesTrunks Attributes



Trunk AttributesTrunk Attributes

• Configured at the head-end of the trunk

• Bandwidth

• Priorities

– setup priority: priority for taking a resource

– holding priority: priority for holding a resource


Trunk attributesTrunk attributes

• Ordered list of Path Options

– possible administratively specified paths (via an off-line central server) - {explicit list}

– Constrained-based Dynamically computed paths based on combo of Bw and policies

• Re-optimization

– each path option is enabled or not for re-optimization, interval given in seconds.

– Max 1 week (7*24*3600), Disable 0, Def 1h.


Trunk AttributesTrunk Attributes

• Resource class affinity (Policy)

– supports the ability to include/exclude certain links for certain traffic trunks based on a user-defined Policy

– Tunnel is characterized by a

• 32-bit resource-class affinity bit string

• 32-bit resource-class mask (0= don’t care, I care)

– Link is characterized by a 32-bit resource-class attribute string

– Default-value of tunnel/link bits is 0

– Default value of the tunnel mask = 0x0000FFFF


Example0: 4-bit string, defaultExample0: 4-bit string, default

• Trunk A to B:

– tunnel = 0000, t-mask = 0011

• ADEB and ADCEB are possible

A B

0000

0000 0000

00000000

C

D E


Example1a: 4-bit stringExample1a: 4-bit string

• Setting a link bit in the lower half drives all tunnels off the link, except those specially configured

• Trunk A to B:

– tunnel = 0000, t-mask = 0011

• Only ADCEB is possible

A B

0000

0000 0000

00100000

C

D E


Example1b: 4-bit stringExample1b: 4-bit string

• A specific tunnel can then be configured to allow such links by clearing the bit in its affinity attribute mask

• Trunk A to B:

– tunnel = 0000, t-mask = 0001

• Again, ADEB and ADCEB are possible

A B

0000

0000 0000

00100000

C

D E


Example1c: 4-bit stringExample1c: 4-bit string

• A specific tunnel can be restricted to only such links by instead turning on the bit in its affinity attribute bits

• Trunk A to B:

– tunnel = 0010, t-mask = 0011

• No path is possible

A B

0000

0000 0000

00100000

C

D E


Example2a: 4-bit stringExample2a: 4-bit string

• Setting a link bit in the upper half drives has no immediate effect

• Trunk A to B:

– tunnel = 0000, t-mask = 0011

• ADEB and ADCEB are both possible

A B

0000

0000 0000

01000000

C

D E


Example2b: 4-bit stringExample2b: 4-bit string

• A specific tunnel can be driven off the link by setting the bit in its mask

• Trunk A to B:

– tunnel = 0000, t-mask = 0111

• Only ADCEB is possible

A B

0000

0000 0000

01000000

C

D E


Example2c: 4-bit stringExample2c: 4-bit string

• A specific tunnel can be restricted to only such links

• Trunk A to B:

– tunnel = 0100, t-mask = 0111

• No path is possible

A B

0000

0000 0000

01000000

C

D E


Trunk AttributeTrunk Attribute Resource Class Affinity (Policy)Resource Class Affinity (Policy)

• The user defines the semantics:

– this bit/mask says “low-delay path excluded”

• Flexible (maybe too flexible :)

– 1c vs 2c ?… in 1c, the default tunnels will not be willing to flow via the special links


Link Attributes and Link Attributes and their floodingtheir flooding



Link Resource AttributesLink Resource Attributes

• Resource attributes are configured on every link in a network

– bandwidth

– Link Attributes

– TE-specific link metric


Link Resource AttributesLink Resource Attributes

• Resource attributes are flooded throughout the network

– bandwidth per priority (0-7)

– Link Attributes (Policy)

– TE-specific link metric

– draft-li-mpls-igp-te-00.txt


Per-Priority Available BWPer-Priority Available BW

DT=0 Link L, BW=100 D advertises: AB(0)=100=…= AB(7)=100

AB(i) = ‘Available Bandwidth at priority I”

DT=2 Link L, BW=100 D advertises: AB(0)=AB(1)=AB(2)=100

AB(3)=AB(4)=…=AB(7)=70

T=1 Setup of a tunnel over L at priority=3 for 30 units

DT=4 Link L, BW=100

D advertises: AB(0)=AB(1)=AB(2)=100 AB(3)=AB(4)=70 AB(5)=AB(6)=AB(7)=40

T=3 Setup of an additional tunnel over L at priority=5 for 30 units


Information DistributionInformation Distribution

• Re-use the flooding service from the Link-State IGP

– opaque LSA for OSPF• draft-katz-yeung-ospf-traffic-00.txt

– new wide TLV for IS-IS• draft-ietf-isis-traffic-00.txt


Information DistributionInformation Distribution

• Periodic (timer-based)

• On significant changes of available bandwidth (threshold scheme)

• On link configuration changes

• On LSP Setup failure


Periodic TimerPeriodic Timer

• Periodically, a node checks if the current TE status is the same as the one lastly broadcasted.

• If different, it floods its updated TE Links status


Significant ChangeSignificant Change

• Each time a threshold is crossed, an update is sent

• Denser population as utilization increases

• Different thresholds for UP and Down (stabler)

50%

100%

70%85%92%

Update

Update


LSP Setup FailureLSP Setup Failure

• Due to the threshold scheme, it is possible that one node thinks he can signal an LSP tunnel via node Z while in fact, Z does not have the required resources

• When Z receives the Resv message and refuses the LSP tunnel, it broadcasts an update of its status


Constrained-based Constrained-based ComputationComputation



Constrained-Based RoutingConstrained-Based Routing

• “In general, path computation for an LSP may seek to satisfy a set of requirements associated with the LSP, taking into account a set of constraints imposed by administrative policies and the prevailing state of the network -- which usually relates to topology data and resource availability. Computation of an engineered path that satisfies an arbitrary set of constraints is referred to as "constraint based routing”.

Draft-li-mpls-igp-te-00.txt


Path ComputationPath Computation

“On demand” by the trunk’s head-end:

– for a new trunk

– for an existing trunk whose (current) LSP failed

– for an existing trunk when doing re-optimization



Input:

– configured attributes of traffic trunks originated at this router

– attributes associated with resources

• available from IS-IS or OSPF

– topology state information

• available from IS-IS or OSPF



• Prune links if:

– insufficient resources (e.g., bandwidth)

– violates policy constraints

• Compute shortest distance path

– TE uses its own metric

– Tie-break: selects the path with the highest minimum bandwdith so far, then with the smallest hop-count



Output:

– explicit route - expressed as a sequence of router IP addresses

• interface addresses for numbered links

• loopback address for unnumbered links

– used as an input to the path setup component


ExampleExample

• Tunnel’s request:

– Priority 3, BW = 30 units,

– Policy string: 0000, mask: 0011

A B

0000

1000 0100

0000 0000

C

D E

10000010

G

BW(3)=60

BW(3)=50

BW(3)=80

BW(3)=20

BW(3)=50 BW(3)=70

BW(3)=80


MPLS as the forwarding MPLS as the forwarding mechanismmechanism



MPLS LabelsMPLS Labels

Two types of MPLS Labels:

Prefix Labels & Tunnel Labels

LDP RSVP

MP-BGP CR-LDP

PIM

Distributed by:


MPLS as forwarding engineMPLS as forwarding engine

• Traffic engineering requires explicit routing capability

• IP supports only the destination-based routing

– not adequate for traffic engineering

• MPLS provides simple and efficient support for explicit routing

– label swapping

– separation of routing and forwarding


LSP tunnel SetupLSP tunnel Setup



RSVP Extensions to RFC2205RSVP Extensions to RFC2205for LSP Tunnelsfor LSP Tunnels

• downstream-on-demand label distribution

• instantiation of explicit label switched paths

• allocation of network resources (e.g., bandwidth) to explicit LSPs

• rerouting of established LSP-tunnels in a smooth fashion using

• the concept of make-before-break

• tracking of the actual route traversed by an LSP-tunnel

• diagnostics on LSP-tunnels

• the concept of nodal abstraction

• preemption options that are administratively controllable

draft-ietf-mpls-rsvp-lsp-tunnel-0X.txt


RSVP Extensions: new objectsRSVP Extensions: new objects

• LABEL_REQUESTLABEL_REQUEST found in Path

• LABEL LABEL found in Resv

• EXPLICIT_ROUTE EXPLICIT_ROUTE found in Path

• RECORD_ROUTERECORD_ROUTE found in Path, Resv

• SESSION_ATTRIBUTESESSION_ATTRIBUTE found in Path 0x01 Fast Reroute Capable, 0x02 Permit Merging, 0x04 May Reoptimize => SE

• New C-Types are also assigned for the SESSIONSESSION, SENDER_TEMPLATESENDER_TEMPLATE, FILTER_SPECFILTER_SPEC, FLOWSPECFLOWSPEC objects.

• All new objects are optional with respect to RSVP (RFC2205).

• The LABEL_REQUEST and LABEL objects are mandatory with respect to MPLS LSP signalisation specification.


LSP SetupLSP Setup

• Initiated at the head-end of a trunk

• Uses RSVP (with extensions) to establish Label Switched Paths (LSPs) for traffic trunks


Path Setup - ExamplePath Setup - Example

Setup: Path (ERO = R1->R2->R6->R7->R4->R9)Setup: Path (ERO = R1->R2->R6->R7->R4->R9)

Reply: Resv communicates labels andReply: Resv communicates labels andreserves bandwidth on each linkreserves bandwidth on each link

Pop

Label 22

Label 49Label 17

R8

R2

R6

R3

R4

R7

R1R5

R9

Label 32


Path Setup - more details Path Setup - more details

R2 R3R1

Path: Common_HeaderSession(R3-lo0, 0, R1-lo0)PHOP(R1-2)Label_Request(IP)ERO (R2-1, R3-1)Session_Attribute (S(3), H(3), 0x04) Sender_Template(R1-lo0, 00) Sender_Tspec(2Mbps)Record_Route(R1-2)

2 21 1


Path Setup - more detailsPath Setup - more details

R3R1

Path State: Session(R3-lo0, 0, R1-lo0)PHOP(R1-2)Label_Request(IP)ERO (R2-1, R3-1)Session_Attribute (S(3), H(3), 0x04) Sender_Template(R1-lo0, 00) Sender_Tspec(2Mbps)Record_Route (R1-2)

2 1R2

21



R3R1

Path: Common_Header Session(R3-lo0, 0, R1-lo0)PHOP(R2-2)Label_Request(IP)ERO (R3-1)Session_Attribute (S(3), H(3), 0x04) Sender_Template(R1-lo0, 00) Sender_Tspec(2Mbps)Record_Route (R1-2, R2-2)

2 1R2

21



R3R12 1

R221

Path State: Session(R3-lo0, 0, R1-lo0)

PHOP(R2-2)Label_Request(IP)

ERO ()Session_Attribute (S(3), H(3), 0x04)

Sender_Template(R1-lo0, 00) Sender_Tspec(2Mbps)

Record_Route (R1-2, R2-2, R3-1)


Resv: Common_Header

Session(R3-lo0, 0, R1-lo0)PHOP(R3-1)

Style=SE FlowSpec(2Mbps)

Sender_Template(R1-lo0, 00) Label=POP

Record_Route(R3-1)


R3R12 1

R221



R3R12 1

R221

Resv State Session(R3-lo0, 0, R1-lo0)

PHOP(R3-1)Style=SE

FlowSpec (2Mbps) Sender_Template(R1-lo0, 00)

OutLabel=POPIntLabel=5

Record_Route(R3-1)



R3R12 1

R221

Resv:Common_Header


Style=SE FlowSpec (2Mbps)

Sender_Template(R1-lo0, 00) Label=5

Record_Route(R2-1, R3-1)



R3R12 1

R221

Resv state:Session(R3-lo0, 0, R1-lo0)PHOP(R2-1)Style=SEFlowSpec (2Mbps) Sender_Template(R1-lo0, 00)Label=5Record_Route(R1-2, R2-1, R3-1)


Trunk Admission ControlTrunk Admission Control

• Performed by routers along a Label Switched Path (LSP)

• Determines if resources are available

• May tear down (existing) LSPs with a lower priority

• Does the local accounting

• Triggers IGP information distribution when resource thresholds are crossed


Link Admission ControlLink Admission Control

• Already invoked by Path message

– if BW is available, this BW is put aside in a waiting pool (waiting for the RESV msg)

– if this process required the pre-emption of resources, LCAC notified RSVP of the pre-emption which then sent PathErr and/or ResvErr for the preempted tunnel

– if BW is not available, LCAC says “No” to RSVP and a Path error is sent. A flooding of the node’s resource info is triggered, if needed

– ”draft-ietf-mpls-rsvp-lsp-tunnel-02.txt”


Path MonitoringPath Monitoring

• Use of new Record Route Object

– keep track of the exact tunnel path

– detects loops

– copy of RRO to ERO allows for route pinning


Path Re-OptimizationPath Re-Optimization

• Looks for opportunities to re-optimize

– make before break

– no double counting of reservations

– via RSVP “shared explicit” style!


Non-disruptive rerouting - new Non-disruptive rerouting - new path setuppath setup

Current Path (ERO = R1->R2->R6->R7->R4->R9)Current Path (ERO = R1->R2->R6->R7->R4->R9)

New Path (ERO = R1->R2->R3->R4->R9) - shared with Current PathNew Path (ERO = R1->R2->R3->R4->R9) - shared with Current Path

Until R9 gets new Path Message, current Resv is refreshedUntil R9 gets new Path Message, current Resv is refreshed

Pop

22

4917

R8

R2

R6

R3

R4

R7

R1R5

R9

32


Non-disruptive rerouting - Non-disruptive rerouting - switching pathsswitching paths

PopPop

22

3849 17

R8

R2

R6

R3

R4

R7

R1R5

R9

32

Resv: allocates labels for both pathsResv: allocates labels for both pathsReserves bandwidth once per linkReserves bandwidth once per link

PathTear can then be sent to remove old path (and release PathTear can then be sent to remove old path (and release resources)resources)

8926


Reroute - More DetailsReroute - More Details

R2 R3R1

ERO (R2-1, R3-1R2-1, R3-1)Sender_Template(R1-lo0, 0000)

2

3

1

3

12

Session(R3-lo0, 0, R1-lo0)

ERO (R2-1, …, R3-3R2-1, …, R3-3)Sender_Template(R1-lo0, 0101)

00

01

0101

Resource Sharing



R2 R3R1

Path: Common_HeaderSession(R3-lo0, 0, R1-lo0)PHOP(R1-2)Label_Request(IP)ERO (R2-1, …,R3-3)Session_Attribute (S(3), H(3), 0x04) Sender_Template(R1-lo0, 01) Sender_Tspec(3Mbps)Record_Route(R1-2)

2

3

1

3

12



R2 R3R12 31 3

Path State: Session(R3-lo0, 0, R1-lo0)PHOP(R1-2)Label_Request(IP)ERO (R2-1, …,R3-3)Session_Attribute (S(3), H(3), 0x04) Sender_Template(R1-lo0, 01)Sender_Tspec(3Mbps) Record_Route (R1-2)



R2 R3R12 31 3



R2 R3R12 31 3

RSVP:Common_Header


Style=SE FlowSpec(3Mbps)

Sender_Template(R1-lo0, 01) Label=POP

Record_Route(R3-3)



R2 R3R12 31 3



R2 R3R12 31 3

RSVP:Common_Header


Style=SE FlowSpec (3Mbps)

Sender_Template(R1-lo0, 01) Label=6

Record_Route(R2-1, …, R3-3)Sender_Template(R1-lo0, 00)

Label=5Record_Route(R2-1, R3-1)



R2 R3R12 31 3

RSVP state:Session(R3-lo0, 0, R1-lo0)PHOP(R2-1)Style=SEFlowSpec Sender_Template(R1-lo0, 01)Label=6Record_Route(R2-1, …, R3-3)Sender_Template(R1-lo0, 00)Label=5Record_Route(R2-1, R3-1)


Fast RestorationFast Restoration

Handling link failures - two complementary mechanisms:

• Path protection

• Link/Node protection


Path ProtectionPath Protection




• Step1: link failure detection

– O(depends on L2/L1)

• Step2a: IGP reaction (ISIS case)

– Either via Step1 or via IGP hello expiration (30s by default for ISIS)

– 5s (default) must occur by default before the generation of a new LSP

– 5.5s (default) must occur before a change of the LSPDB and the consecutive SPF run. The next SPF run can only occur 10s after (default)

– Flooding time (LSP are paced (16ms for first LSP, 33ms between LSP’s, depend also on link speed)

– Once the RIB is updated, this change must be incorporated into CEF.

The Head-end finally computes the new topology and finds out that some established LSP’s are affected. It schedules a reoptimization for them



• Step2b: RSVP signalisation

– rsvp path states with the failed intf as oif is detected

– check if another oif available (if loose ero)

– if not, clear path state and send tear to head-end

• Step2: Either stepA or stepB alarms the head-end

• Step3: Re-optimization

– dijkstra computation: O(0.5)ms per node (rule of thumb)

– RSVP signalisation time to instal rerouted tunnel

convergence in the order of several seconds (at least).


Path ProtectionPath ProtectionSpeed it UpSpeed it Up

• Fine Tune the IGP convergence– Through adequate tuning, ISIS could be tuned to converge in 2-3s, this ensuring that the convergence time bottleneck is the signalisation time for the new tunnel.

• Several tunnels in parallel with load-babalancing– if combined with the IGP convergence, the path resilience could be brought to around 2-3s

• One end-2-end tunnel in parallel but in backup mode– feature under development (Fast Path Protection)


Fast ReRouteFast ReRoute(aka Link Protection)(aka Link Protection)

An OverviewAn Overview



ObjectiveObjective

• FRR allows for temporarily routing around a failed link or node while the head-end may reoptimize the entire LSP

– rerouting under 50ms

– scalable (must support lots of LSP’s)


Fast reroute OverviewFast reroute Overview

• Controlled by the routers at ends of a failed link

– link protection is configured on a per link basis

– Session_Attribute’s Flag 0x01 allows the use of Link Protection for the signalled LSP

• Uses nested LSPs (stack of labels)

– original LSP nested within link protection LSP


Static backup TunnelStatic backup Tunnel

Setup: Path (R2->R6->R7->R4)Setup: Path (R2->R6->R7->R4)

Labels Established on Resv messageLabels Established on Resv message

Pop

22

17

R8

R2

R6

R4

R7

R1R5

R9


R8

R2

R6

R4

R7

R1 R5

R9

Setup: Path (R1->R2->R4->R9) Setup: Path (R1->R2->R4->R9)

Labels Established on Resv messageLabels Established on Resv message

37

14

Pop

Routing prior R2-R4 link failureRouting prior R2-R4 link failure


Link Protection ActiveLink Protection Active

R8

R2

R6

R4

R7

R1 R5

R9

On failure of link from R2 -> R4, R2 simply changes outgoingLabel Stack from 14 to <17, 14>


Link Protection ActiveLink Protection Active

R8

R2

R6

R4

R7

R1 R5

R9

PushPush 37

PopPop 22

SwapSwap 37->14PushPush 17

SwapSwap 17->22

PopPop 14

Label Stack: R1 R2 R6 R7 R4 R937 17 22 14 None

14 14


Fast ReRouteFast ReRouteMore details on Link More details on Link Protection (FRR v1)Protection (FRR v1)



V1 ConstrainV1 Constrain

• We protect the facility (link), not individual LSP’s

– scalability vs granularity

• No node resilience

• Static backup tunnel

• The protected link must use the Global Label space

• A backup tunnel can backup at most one link, but n LSPs travelling via this link


TerminologyTerminology

• LSP: end-to-end tunnel onto which data normally flows (eg R1 to R9)

R8

R2

R6

R4

R7

R1 R5

R9

• BackUp tunnel: temporary route to take in the event of a failure


TerminologyTerminology

• Link Protection

– In the event of a link failure, an LSP is rerouted to the next-hop using a preconfigured backup tunnel


How to indicate a link is protected and How to indicate a link is protected and which tunnel is the backup?which tunnel is the backup?

• On R2 (For LSP’s flowing from R2 to R4):

interface pos <r2tor4>

mpls traffic-eng backup tunnel 1000 link

• LSP’s are unidirectional, so the same protection should be enable for the opposite direction if reverse LSP is conf.


How to setup the backup tunnel?How to setup the backup tunnel?

• Just as a normal tunnel whose head-end is R2 and tail-end is R4

– v1 requires a manually configured ERO

interface Tunnel1000 ip unnumbered Loopback0 tunnel destination R4 tunnel mode mpls traffic-eng tunnel mpls traffic-eng priority 7 7 tunnel mpls traffic-eng bandwidth 800 tunnel mpls traffic-eng path-option 1 explicit name backuppath1

ip explicit-path name backuppath1 enable next-address R6 next-address R7 next-address R4


Which LSP’s can be rerouted on R2 in Which LSP’s can be rerouted on R2 in the event of R2-R4 failure?the event of R2-R4 failure?

• The LSP’s flowing through R2 that

– have R2-R4 as Outgoing Interface

– have been signalled by their respective head-ends with a session attribute flag 0x01=ON (may use fast-reroute tunnels)• int tunnel 1 ## config on the head-end

tunnel mpls traffic-eng fast-reroutefast-reroute


Global Label AllocationGlobal Label Allocation

• For the blue LSP, R4 bound a global label of 14

• Any MPLS frame received by R4, with label 14, will be switched onto the link to R9 with a POP, whatever the incoming interfacewhatever the incoming interface

R8

R2

R6

R4

R7

R1 R5

R914

POP


How fast is fast?How fast is fast?

• Link Failure Notification

– Usual PoS alarm detection

– PoS driver optimisationPoS driver optimisation to interrupt RP in < 1ms< 1ms

– Expected call to net_cstate(idb, UP/DOWN) identifying the Expected call to net_cstate(idb, UP/DOWN) identifying the DOWN state of the protected int to start our protection action.DOWN state of the protected int to start our protection action.

• RP updates the master TFIB (replace a swap by a swap-push)

– < 1ms< 1ms

• Master TFIB change notified to the linecards

– < 1ms< 1ms


Path state while ReroutingPath state while Rerouting

R8

R2

R6

R4

R7

R1 R5

R9

BackUP tunnel

Path (…, PHOP=R2, …)

Path statePath state

PathError (Reservation in Place)


Path & Resv Msgs [Error & Tear]Path & Resv Msgs [Error & Tear]

R4R1 R2 R3

Path Tear

Resv Tear

Conf.Resv Tear Conf.

When no link protection:

R4 waits for refresh

Path Error

Resv in place

When link protection:


LSP reoptimizationLSP reoptimization

• Head-end notified by PathError

– special flag (reservation in place) indicates that the path states must not be destroyed. It is just a hint to the head-end that the path should be reoptimized

• Head-end notified by IGP


Why the Patherror?Why the Patherror?

• The Patherror might be faster

• In case of multi-area IGP, the IGP will not provide the information

• In case of very fast up-down-up, the LSP will be put on the backup tunnel and will stay there as the IGP will not have originated a new LSP/LSA

– a router waits a certain time before originating a new LSP/LSA after a topological change

• Reliable PathErr optimization


R8

R2

R6

R4

R7

R1 R5

R9

BackUP tunnel

Resv

Resv stateResv state

Resv state while ReroutingResv state while Rerouting

• Resv Message is unicast to the Phop (R2)

• R2’s Path State has been informed that the Resv might arrive over a different intf as the one used by the Path message

The loss of the interface does not affect the Path and Resv states for the LSP’s received on that

interface that are marked fast reroutable!


DiffServ and LSP ReoptimizationDiffServ and LSP Reoptimization

• In order to optimize the bandwdith usage, backup tunnels might be configured with 0kbps

– no ‘non-working’ bandwdith as in SDH!no ‘non-working’ bandwdith as in SDH!

• Although usually the backbone is though as being congestion-free, during rerouting some local congestion might occur

– Use diffservdiffserv to handle this short-term congestionshort-term congestion

– Use LSP reoptimizationLSP reoptimization to handle the long-term long-term congestioncongestion


Layer1/2 and Layer3Layer1/2 and Layer3

• Backup Tunnel should not use

– the protected L3 link

– the protected L1/L2 links!!!

• Use WANDL (loaded with both L3 and L1/2 topologies) to compute the best paths for backup tunnels

– Download this as static backup tunnels to the routers


Fast ReRouteFast ReRouteNode ProtectionNode Protection



R8

R2

R6

R3

R7

R1 R5

R9

R4

OverviewOverview

Backup Tunnel to the next-hop of the LSP’s next-hop


A few More detailsA few More details

• Assume

– R2 is configured with resilience for R3

– R2 receives a path message for a new LSP whose ERO is {R3, R4, …}, whose Session is (R9, 1, R1), whose sender is (R1, 1) and whose session attribute is (0x01 ON, 0x02 OFF)

• 0x01: may use local fast-reroute if available

• 0x02: merge capable



• Then

– R2 checks if it already has a tunnel to R4

– If not, R2 builds a backup tunnel to R4 (currently just like in link protection - manual explicit setup).

– R2 sends a Path onto the tunnel with Session (R9, 1, R1), Sender (R2, 1), Session Attribute (0x01 OFF, 0x02 ON) and PHOP R2



• When R4 receives this Path message,

– it matches the session with the LSP’s one

– merge (and thus stop) this path message

– sends a RESV back to R2 (unicast) and allocate the appropriate label L



• When R2 detects R3’s failure,

– For the TFIB entry for the LSP, R2 changes the existing ‘swap’ by a ‘swap to L’ and a ‘push of the backup tunnel label’

• R4’s states are refreshed by the secondary path messages (over the backup tunnels)

• ERO of the original path is adjusted at R2

• NHOP is modified in R2 (from R3 to R4)

• PHOP is modified in R4 (from R3 to R2)


A few More DetailsA few More Details

• RESV is being sent back from R4 to R2 directly

• If R3 is still active and just the R2-R3 link failed R4 needs to ignore & drop any Tear-Down msg R3 would be sending after the termination of reception of path refreshes from R2.


How to detect R3’s failure?How to detect R3’s failure?

• A node may fail while the link is still up

• A node’s linecard processes might survive, a main process failure (freeze of the RP process)


A possible solutionA possible solution

• Keepalives between LC’s

• Keepalives between a LC and its master RP

LC

RP

...

LC

RP

LC


Assigning traffic to PathsAssigning traffic to Paths (aka autoroute) (aka autoroute)



Enhancement to SPFEnhancement to SPF

• During SPF each new node found is moved from a TENTative list to PATHS list. Now the first-hop is being determined via:

A. Check if there is any TE tunnel terminating at this node from the current router and if so do the metric check

B. If there is no TE tunnel and the node is directly connected use the first-hop from adj database

C. In non of the above applies the first-hop is copied from the parent of this new node.


Enhancement to SPF - metric checkEnhancement to SPF - metric check

Tunnel metric:

A. Relative +/- X

B. Absolute Y

The default is relative metric of 0.

Example:

Metric of native IP path to the found node = 50

1. Tunnel with relative metric of -10 => 40

2. Tunnel with relative metric of +10 => 60

3. Tunnel with absolute metric of 10 => 10


Enhancement to SPF - metric checkEnhancement to SPF - metric check

• If the metric of the found TE tunnel at this node is higher then the metric for other tunnels or native IGP path this tunnel is not installed as next hop

• If the metric of the found TE tunnel is equal to other TE tunnels the tunnel is added to the existing next-hops

• If the metric of the found TE tunnel is lower then the metric of other TE tunnels or native IGP the tunnel replaces them as the only next-hop.


Other TE New FeaturesOther TE New Features



Auto-BandwidthAuto-Bandwidth

• Monitor marked tunnels’ 5-min average counters every X minutes

– default: X = 300 (seconds)

– (config)# mpls traffic-eng auto-bw timers frequency <seconds>

Global command:


Auto-BandwidthAuto-Bandwidth

• Every Y minutes, update the BW constraint of the tunnel with the maximum of:

– the largest 5-min values sampled during the last Y minutes (Def Y = 24 * 3600sec) - 24h

– a configured maximum value

– (config-if)# tunnel mpls traffic-eng auto-bw {frequency <seconds>} {max-bw <kbs>}

– if the new Bw is not available, the old one is maintained (the new BW is signalled via a 2nd tunnel to follow make before break model)

Per tunnel command:


ExampleExample


VerbatimVerbatim

• Applies to explicitly routed LSP’s

• Disable any check against TE/IGP database of the head end

• RSVP still check BW (and policy when this will be in Path) hop by hop

• Application: manual TE through multi-area IGP

CLI: tunnel mpls traffic-eng path-option verbatim


In-ProgressIn-Progress

• Allows an end-head to account for bw consumed by tunnels that it has just signalled and for whom the IGP LSA/LSP update has not reflected the available bandwdith


ExampleExample

Avail Bw: 100

In-Prog Bw: 55In-Prog Bw: 10

All tunnels require 45 units of BW

• In-progress counters reset upon new LSA/LSP reception

• In-progress counter decremented upon receipt of path-error


BenefitsBenefits

• Speed-up the installation of tunnels as it avoids spending time trying not working solutions

• Allows for better load-balancing

– igp metric then max(min(path-bw)!


Under/OverbookUnder/Overbook

• ML: Maximum link bandwidth:–This sub-TLV contains the maximum bandwidth that can be used on this link in this direction (from the system originating the LSP to its neighbors). This is useful for traffic engineering.

• MR: Maximum reservable link bandwidth:– This sub-TLV contains the maximum amount of bandwidth that can be reserved in this direction on this link. Note that for oversubscription purposes, this can be greater than the bandwidth of the link.

• UR(I): Unreserved bandwidth at Priority i:– This sub-TLV contains the amount of bandwidth reservable on this direction on this link, at a certain priority. Note that for oversubscription purposes, this can be greater than the bandwidth of the link.


Under/OverbookUnder/Overbook

• ML is set to B1 (eg 1500)

• MR is set to B2 (eg 4000)

• At t=0, for all i 0 to 7, UB(i) = M = (eg 4000)

• routerA's LCAC will not accept an LSP tunnel asking more than ML even if there is available bandwdith at the requested priority.

• However, LCAC would allow for example 5 trunks each asking 700 kbps (thus each asking less than ML) while the aggregate is smaller than MR: because { 700 < ML=1500 } and { 3500 < MR=4000 }

A B... ...Physical T1

s0

A’s config:A’s config:int s0int s0 bandwidth <B1> (eg 1500 kbps)bandwidth <B1> (eg 1500 kbps) ip rsvp bandwdith <B2> (eg 4000 kbps)ip rsvp bandwdith <B2> (eg 4000 kbps)

A’s config:A’s config:int s0int s0 bandwidth <B1> (eg 1500 kbps)bandwidth <B1> (eg 1500 kbps) ip rsvp bandwdith <B2> (eg 4000 kbps)ip rsvp bandwdith <B2> (eg 4000 kbps)


StandbyStandbyCurrent solutionCurrent solution

• Solution:

– 4 tunnels from A to B:

– Tu1’s relative metric: -3

– Tu2 and tu3’s relative metric: -2

– Tu4’s relative metric: -1

A BTu1: bw1

Tu2: bw2

Tu3: bw3

Tu4: bw4


Last hop labelLast hop label

• IETF draft-ietf-mpls-label-encaps-07.txt– A value of 0 represents the "IPv4 Explicit NULL Label”

– A value of 1 represents the "Router Alert Label”

– A value of 2 represents the "IPv6 Explicit NULL Label" – A value of 3 represents the "Implicit NULL Label”

New cli forces tailend to send implicit-null (3) instead of explicit null (0) - default.

# [no] mpls traffic-eng signalling advertise implicit-null [<acl>]

On receipt (n-1) node we must map 0, 1 or 3 to internal Implicit Null [1 only for historical reasons]


QoS and RRRQoS and RRR




• MPLS TE can operate simultaneously (and orthogonally) with MPLS Diff-Serv

• All Precedence/DSCP packets follow the same TE tunnels

•Diff-Serv provides selective discard (via WRED), and selective scheduling (via WFQ)



• Future:

– Scalable per-tunnel scheduling and policing

• Guaranteed PIPE in MPLS-VPN CoS

– per-DSCP/per-FEC traffic engineering

• diffserv backbone capacity management


DiffServ and fast-reroute/TEDiffServ and fast-reroute/TE

• In order to optimize the bandwdith usage, backup tunnels might be configured with 0kbps

– no ‘non-working’ bandwdith as in SDH!no ‘non-working’ bandwdith as in SDH!

• Although usually the backbone is though as being congestion-free, during rerouting some local congestion might occur

– Use diffservdiffserv to handle this short-term congestionshort-term congestion

– Use LSP reoptimizationLSP reoptimization to handle the long-term long-term congestioncongestion


RSVP RSVP LSP Signalling Protocol LSP Signalling Protocol for Traffic Engineeringfor Traffic Engineering



MPLS-TE Signalling ProtocolMPLS-TE Signalling Protocol

• Two proposed signaling mechanisms for MPLS traffic engineering are being considered by the IETF’s MPLS work group

– RSVP (Cisco and a number of Gigabit router startups (Avici, Argon, Ironbridge, Juniper, and Torrent))

– CR-LDP (Ericsson, Ennovate, GDC, Nortel)


Why RSVPWhy RSVP ? ?

• What is needed:

– ability to establish and maintain Label Switched Path along an explicit route

– ability to reserve resources when establishing a path

• Interdependent, not independent tasks

– benefit from consolidation

An IP signalling Protocol!


Do I need RSVP only for TE ?Do I need RSVP only for TE ?

Other uses of RSVP in today’s networks:

• Voice over IP call setup, Video (IPTV)

• Hybrid deployments (only where needed)

• QoS DiffServ Engineering (Cops)

• Qualitative Service for DiffServ with RSVP

(as opposed to Quantitative RSVP IntServ model)

NO !


RSVP is a natural choiceRSVP is a natural choice

• RFC2205: “provides a general facility for creating and maintaining distributed reservation state across a mesh of multicast and unicast delivery paths”

• TE: use as a general facility for creating and maintaining distributed forwarding & reservation state across a mesh of delivery paths



• RFC2205: “transfers and manipulates QoS control parameters as opaque data, passing them to the appropriate traffic control module for interpretation”

• TE: transfer and manipulate explicit route and label control parameters as opaque data pass explicit route parameter to the appropriate routing module, and label parameter to the MPLS module



• Leverage Standardized Protocols

– PIM for Multicast MPLS

– BGP for MPLS VPN’s

– RSVP for MPLS Traffic Engineering

– LDP (TDP) has been designed because it was easier than fixing all IGP’s (RIP, EIGRP, OSPF, ISIS)

– fast deployments and engineering consistency

• Leverage Deployed Experience

– RSVP deployed since 1996 (IOS 11.2)

– ww.isi.edu/rsvp/DOCUMENTS/ietf_rsvp_qos_survey for a list of RSVP implementations



• RSVP easily supports

– Dynamic resizing of tunnels or paths through refresh messages

–Supports strict as well as loose source routes

–No double counting of bandwidth when re-routing sub-optimal routes

• Extensible via definition of new objects


RSVP/TE and ScalabilityRSVP/TE and Scalability

• State applies to a collection of flows (i.e. a traffic trunk), rather than to a single (micro) flow

• RSVP sessions are used between routers, not hosts

• Sessions are long-lived (up to a few weeks)

• Paths are not bound by destination-based routing

• Reference: ‘Applicability Statement for Extensions to RSVP for LSP-Tunnels’ (draft-awduche-mpls-rsvp-tunnel-applicability-01.txt)

Very Different than IntServ context


• RFC2208: “the resource requirements for running RSVP on a router increases proportionally with the number of separate sessions”

• TE: that is why using traffic trunks to aggregate flows is essential

• RFC2208: “supporting numerous small reservations on a high-bandwidth link may easily overtax the routers and is inadvisable”

• TE : n/a in the context of TE - traffic trunks aggregate multiple flows

RSVP/TE and ScalabilityRSVP/TE and ScalabilityVery Different than IntServ context


TE/RSVP Scalability TE/RSVP Scalability

• With basic RSVP (RFC2205), 10000 RRR LSP tunnels flowing through a 75x0 or 12000 is not a problem

• Already Deployed on a number of Tier-1 ISP backbones

– http://www.nanog.org/mtg-9905/hanna.html

– Ship with 12.0(5)S

• Refresh Aggregation work will again enhance this scalability


ConclusionConclusion

• Using RSVP as MPLS/TE signalling protocol is the natural and consistent choice

• It is however only one part of a whole solution:

– MPLS as forwarding engine

– IGP (OSPF/ISIS) extensions

– Constrained Base Routing (RRR)

– RSVP as MPLS/TE Signalling Protocol

– Installation of Tunnels in the FIB


SummarySummary



Traffic EngTraffic Eng

• Provides traffic engineering capabilities at Layer 3

– above and beyond of what is provided with ATM

• Could be used for other applications as well

• Shipping and deployed in production

raszuk mpls te

Documents