solving the softwire mesh problem

1

Solving the Softwire Mesh Problem

Chris Metz, chmetz@cisco.com

IETF Softwire WG Interim Meeting

Hong Kong

February 2006

2

Contents

• Some Terminology

• Basic Problem to Solve

• Similarities with L3VPN

• Solution Overview– Encapsulations– BGP Protocol Elements

• Examples

• References

3

Terminology (1)

• AF(i) Transit Core – single AF IPv4 or IPv6 backbone network• AFBR – Address Family Border Routers, dual-stack (I,j)• AF(j) Access Islands – single AF(j) or dual-stack (i,j) access

networks connected to AFBR

AF = Address Family

4

Terminology (2): What it looks like with IPv4 and IPv6 Plugged In …

5

So what is the problem we need to solve?

• Support inter-AF(j) island routing and forwarding across a single AF(i) transit core.

6

Problem to Solve? IPv4 Islands across an IPv6 Core and …

7

… IPv6 Islands across an IPv4 Core

8

Some quick observations of what is needed here (1)

• Multi-AF Route Distribution– ex. so that routers in AF(j) Access Island-1(including AFBR-1)

learn about AF(j) prefixes located in other AF(j) access islands reachable thru AFBR-2, .. AFBR-N

9

Some quick observations of what is needed here (2)

• AF(i) Encapsulation of AF(j) Packets– ex. AFBR-1 encapsulates AF(j) packet in AF(i) “wrapper” so that packet can be forwarded across AF(i) core; wrapper is then removed at egress AFBR– also need to figure out how AFBRs agree on what “wrappers” to use and how to correlate this with the AFBR and AF(j) reachability …

10

So big picture at this point ..

• We have:– requirement to distribute multi-AF routes (IPv4 or

IPv6) between AF access islands connected to a single AF backbone network

– requirement to use that reachability information to forward AF packets (IPv4 or IPv6) across that backbone network from one access island network to another

– requirement to encapsulate AF island-sourced IPv4 or IPv6 packets for transport across AF backbone network

• This has similarities to the classic L3VPN problem and solution space. Let’s take a look …

11

Classic MPLS VPN (1)

• Define a new IPv4 VPN address family (VPNv4) to identify and store customer VPN IPv4 routes inside VPN routing tables (VRFs) on PE nodes

• Use MP-BGP to distribute VPNv4 routes, VPN labels, Next-Hop information, etc. between PE nodes only

MP-BGP = multi-protocol BGPVRF = VPN Routing/Forwarding Table

12

Classic MPLS VPN (2)

• Native IPv4 customer VPN packets are encapsulated in MPLS labels for transport across the MPLS backbone– IGP label(s) provide the label switched path (LSP) from PE-1 to PE-2– VPN label identifies which destination customer site to forward IPv4 packet to

13

Classic MPLS VPN (3)• Defined in RFC2547/RFC4364• Many interoperable implementations and deployments• Can even support IPv6 VPNs

– draft-ietf-l3vpn-bgp-ipv6-07.txt• Extended for Multicast VPN

– draft-ietf-l3vpn-2547bis-mcast-01.txt– only IPv4 at the moment

• Scalability– Per-PE routing table: O(# of Internet Routes + # of VPN routes for

attached customers)– per-PE peering: O(# of remote PEs + # of attached customer routers)– per-local PE-to-remote PE paths: O(# of remote PEs)

• Security– Discussed in RFC4111, “Security Framework for Provider-

Provisioned Virtual Private Networks (PPVPNs)”

14

Classic MPLS VPN (4)

• What happens if the backbone IS NOT MPLS? Can we still do MPLS VPNs?

• Yes, we can nail up inter-PE IP tunnels (e.g. GRE) and then tunnel the VPN-labeled customer packets thru or …

15

MPLS VPN over IP (1)

• Extend MP-BGP to advertise IP tunnel info along with VPNv4 prefixes, VPN labels, etc.– ex. now PE-1 learns of remote VPNv4 prefixes, the VPN labels, the next_hop and an IPv4

tunnel to use to reach that next_hop

16

MPLS VPN over IP (2)

• Native IPv4 customer VPN packets encapsulated in VPN label and IP Tunnel Header (e.g. GRE, L2TPv3, IPsec) for transport across IP backbone

• Current deployments include:– static GRE tunnels between PE nodes; BGP-advertised L2TPv3 tunnel encaps

17

Building the solution with some of these pieces …

• MP-BGP – efficient and scalable one (egress AFBR) to

many (ingress AFBR) delivery of multi-AF reachabililty and IP tunnel information

• Standard Encapsulation Techniques– IP/IP, GRE, L2TPv3, MPLS-in-IP, IPsec, etc.

• Interoperable L3VPN deployments– VPNv4 over MPLS and IP– VPNv6 over MPLS

18

One more bit of Terminology - Softwire

• Defined as a logical pt-pt (or pt-mpt) tunnel established between participating AFBR nodes

• Purpose is to transport packets of AF(j) across the AF(i) backbone

19

Solution Overview (1)Basic Idea

• Leverage and reuse existing L3VPN functions and protocols where appropriate

• Identify/develop a set of Softwire encapsulations using standard/existing encapsulations

• Extend MP-BGP to– enable egress AFBR(s) to advertise their softwire

tunnel capabilties, encapsulation parameters and preferences to participating ingress AFBR nodes … thus forming the softwire mesh

– enable egress AFBR(s) to advertise AF prefixes and associated softwire(s) to use to reach those prefixes

20

Solution Overview (2) A Picture

21

Solution Overview (3)

• AF Access Islands can be single or dual-stack• AFBR may support more than one softwire type

– ex. egress AFBR-2 may support GRE and L2TPv3 encaps and will tell other AFBRs about this along with which one AFBR-2 would prefer to be used.

• No new AF/SAF needed to define IPv4 and IPv6 addresses for transport in MP-BGP

22

Solution Overview (4)

• Establishment of inter-AFBR softwires is decoupled from the distribution of AF reachability information– advertise softwire tunnel encapsulation and preferences once

and then many AF prefixes and which softwire tunnel to use. – more efficient BGP packing and processing by eliminating

advertisement of duplicate softwire tunnel info for each prefix– enables policy control on AFBR for softwire installation and

selection

• Not mandated to store AF prefixes in VRFs on AFBR– only needed to support overlapping address requirement or if

operator prefers this configuration

23

Note on VRF and Global Tables

• AF Island prefixes VRFs– MP-BGP advertises as VPN:AF with VPN label, RT, etc.

• AF Island prefixes Global– MP-BGP advertises as AF

• In either case:– softwire tunnels setup is separate from AF island prefix distribution– AF island prefix distribution (VPN or Global) can include softwire tunnel ID

24

Softwire Encapsulation Possibilities(over IPv4 Transit)

• IP – IPv6/IPv4 – IPv6/VPN label/IPv4 -

• UDP/IP – IPv6/UDP/IPv4

• GRE– IPv6/GRE/IPv4– IPv6/VPN Label/GRE/IPv4

• IPsec– IPv6/IPsec/IPv4

• MPLS– if IPv4 transit is MPLS-

enabled then MPLS label may be pushed on top or replace outer IPv4 header

• L2TPv3– IPv6/L2TPv3/IPv4– IPv6/VPN label/L2TPv3/IPv4– IPv6/L2TPv3/IPsec/IPv4– IPv6/VPN

label/L2TPv3/IPsec/IPv4– IPv6/L2TPv3/UDP/IPv4

25

Softwire Encapsulation Possibilities(over IPv6 Transit)

• IPv6 only– IPv4/IPv6 – IPv4/VPN label/IPv6

• UDP/IP only– IPv4/UDP/IPv6

• GRE– IPv4/GRE/IPv6– IPv4/VPN Label/GRE/IPv6

• IPsec– IPv4/IPsec/IPv6

• MPLS– if IPv6 transit is MPLS-

enabled then MPLS label may be pushed on top or replace outer IPv6 header

• L2TPv3– IPv4/L2TPv3/IPv6– IPv4/VPN label/L2TPv3/IPv6– IPv4/L2TPv3/IPsec/IPv6– IPv4/VPN

label/L2TPv3/IPsec/IPv6– IPv4/L2TPv3/UDP/IPv6

26

Quick MP-BGP NoteMP_REACH_NLRI Attribute

IPv4=1, IPv6=2

Unicast=1Multicast=2....Tunnel SAFI=64MPLS VPN=128

http://www.iana.org/numbers.html

27

BGP Solution Elements

1. Distribution of Softwire Tunnel capabilities, encapsulation(s) types, parameters and preferences

2. Distribution of AF Island Prefixes

3. Distribution of Softwire Tunnel IDs

28

BGP Solution Elements (1a)

• How does egress AFBR tell (N number of) candidate ingress AFBR(s) what softwire tunnel types, parameters and preferences it can support?

• Answer: BGP Tunnel SAFI

BGP RR = BGP Route Reflector

29

BGP Solution Elements (1b)BGP Tunnel SAFI

• MP_REACH_NLRI attribute with a SAFI=64 indicates tunnel attributes are present– AFI=1 and SAFI=64 point to IPv4-specific parameters– AFI=2 and SAFI=64 point to IPv6-specific parameters

• NLRI of Tunnel SAFI contains address of tunnel end-point on AFBR– same address can be used by many different tunnels

thus conserving address space on the AFBR that terminates the tunnel

• draft-nalawade-kapoor-tunnel-safi-04.txt

30

BGP Solution Elements (1c)Tunnel Encapsulation Attribute

• Also present when Tunnel SAFI=64 are one (or more) Tunnel Encapsulation Attributes (TLVs)– egress AFBR-2 tells the peering ingress AFBR(s) (1-N) what

parameters and preferences of specific encap types it can support• Defined values so far:

– Type 1: L2TPv3 Tunnel information– Type 2: mGRE Tunnel information– Type 3: IPSec Tunnel information– Type 4: MPLS Tunnel information– Type 5: L2TPv3 in IPSEC Tunnel information– Type 6: mGRE in IPSEC Tunnel information

• Includes a preference field that indicates the egress AFBR’s preferred ordering of softwire encapsulations that the ingress AFBR should consider when selecting a softwire tunnel.

• draft-nalawade-kapoor-tunnel-safi-04.txt

31

BGP Solution Elements (1d)Tunnel SAFI + Tunn Encapsulation Attributes

10.1.2.1

10.1.2.1

• AFBR-2 is telling the other AFBRs that– it can terminate an L2TPv3/IPv4

softwire at 10.1.2.1

32

BGP Solution Elements (1e)After BGP Tunnel SAFI

• Softwire established to AFBR-2– it is possible to establish more than one softwire to an egress AFBR

33

BGP Solution Elements (2)• Used existing MP-BGP protocols to distribute native or VPN-

specific AF Island Prefixes between AFBR nodes

Prefix Type Received Into:

AF SAF Reference

Island IPv4 native

Global 1 1 RFC2858

Island IPv4 native

VRF 1 128 RFC4364

Island IPv6 native

Global 2 1 RFC2858

Island IPv6 native

VRF 2 128 draft-ietf-l3vpn-bgp-ipv6-07.txt

34

BGP Solution Elements (3a)

• Final piece is for egress AFBR to advertise specific tunnel identifier that ingress AFBR(s) should use to reach a particular destination AF island prefix– ingress AFBR uses this to determine which tunnel to forward

packets through to reach the advertised destination

• Use BGP Connector Attribute. Defined value are:– Type 1 = IPv4 address (for inter-as MDT case)– Type 2 = Tunnel ID: Tunnel End-Point Address (IPv4/6 address)– Type 3 = Tree ID: Tunnel End-Point Address (IPv4/6 address)

(for multicast case)

• draft-nalawade-l3vpn-bgp-connector-00.txt

35

BGP Solution Elements (3b)

• BGP AF island prefix advertisement includes connector attribute that informs ingress AFBRs which softwire tunnel to use

36

BGP Solution Elements

1. BGP Updates contains Tunnel SAFI and tunnel encapsulation TLV to announce softwirecapabilities, encapsulation parameters and preferences

2. BGP updates include AF Island Prefix and Connector Attribute that points to softwire to use.

37

Examples

1. Native IPv6 over IPv4 Core

2. VPNv6 over L2TPv3/IPv4 Core

3. VPNv4 over GRE IPv6 Core

38

Example 1a: IPv6 over GRE/IPv4

16410.1.2.1Type 2 (GRE)99

10.1.2.1 GRE

IPv4

GRE

IPv4

39

Example 1b: IPv6 over GRE/IPv4

3FFE:1234:1111/48IPv6

10.1.2.1

3FFE:1234:1111/48noneegress AFBRtunn ID: 10.1.2.1 (type 2)

IPv4GREnoneIPv6IPv6 IPv6

IPv6 glbl glblIPv4

40

Example 2a: VPNv6 over L2TPv3/IPv4

16410.1.2.1Type 1 (L2TPv3)99

10.1.2.1 L2TPv3

IPv4

L2TPv3

IPv4

41

Example 2b: VPNv6 over L2TPv3/IPv4

3FFE:1234:1111/48IPv6

10.1.2.1

RD:3FFE:1234:1111/4855egress AFBRtunn ID: 10.1.2.1 (type 2)

IPv4L2TPv3

55IPv6IPv6 IPv6

IPv6 VRF VRFIPv4

42

Example 3a: IPv4 over GRE/IPv6

2642002:1111::1Type 2 (GRE)99

GRE

IPv6

GRE

IPv6

2002:1111::1

43

Example 3b: IPv4 over GRE/IPv6

200.1/20IPv4

200.1/20noneegress AFBRtunn ID: 2002:1111::1(Type 2)

IPv6GRE

noneIPv4IPv4 IPv4

IPv4 GBL GBLIPv6

2002:1111::1

44

Additional Functions

• Inter-AS– advertise softwire tunnel attributes and AF

reachability (to egress AFBR) across AS boundaries then …

– advertise AF prefixes and connector attributes using MP-eBGP across AS boundaries

• Two options for Multicast: – Traditional IPv4 or IPv6 multicast tunneled across

softwire mesh– Extend mVPNv4 model to include multicast IPv6

reachability and forwarding over inclusive and selective P-multicast service interfaces (PMSI)

45

Summarizing Key Aspects of this Solution (1)

• Leverages existing and deployed L3VPN protocols (e.g. MP-BGP) and IP encapsulation techniques (e.g. GRE, L2TPv3)

• Scalability:– Per-AFBR routing table: O(# of Internet Routes + # of AF island

prefixes of attached islands)– per-AFBR peering: O(# of remote AFBRs + # of attached AF

island routers)– per-local AFBR-to-remote AFBR paths: O(# of remote AFBRs)

• Security:– RFC4111 provides framework– Control Plane: BGP/TCP MD5, BGP/TCPoIPsec– Data Plane: GRE keys, L2TPv3 cookie, IPsec

• Multicast:– traditional multicast over softwire tunnels– mVPN extensions

46

Summarizing Key Aspects of this Solution (2)

• OAM– can employ existing (e.g. Netflow, interface counters per softwire)

accounting mechanisms– feasible to run tunnel “health probes” (e.g. LSP Ping/VCCV/BFD) along

with the usual ICMP ping/trace• Multihoming

– no problem with multihoming from AF island into multiple AFBRs announcing same AF prefix

• Multi-Softwire Support– AFBR can announce different softwires (e.g. GRE and L2TPv3/IPsec), a

preference for one over the other and even can have specific prefixes use different softwires if desired

• L2VPN– pseudowires could provide the signaling and encapsulation to transport

L2-encapsulated IPv4 or IPv6 packets between AFBRs– but there is O(N2) provisioning to consider plus O(N) of L2 interfaces per

AFBR

47

In Conclusion

• BGP-based VPNs (IPv4 and IPv6) as deployed and in operation today form the foundation for a softwire mesh solution

• Modest extensions– support global and VRF tables– agree on set of softwire encaps and add to

BGP Tunnel SAFI – support BGP Connector Attribute

• Done

48

Question?

• Currently Tunnel SAFI and Connector Attribute are not Inter-domain Routing (IDR) WG documents.

Should we do the work here in Softwires or take it to IDR?

Quick Note on this: Review of Paris and Vancouver IDR meeting notes implies that IDR would review and bless the encodings once some other WG (e.g. L3VPN, now softwires) figured out what application and solution and proposes encodings– References: http://www3.ietf.org/proceedings/05nov/index.html,

http://www3.ietf.org/proceedings/05aug/index.html

49

References• RFCs:

– RFC2858 - Multiprotocol Extensions for BGP-4– RFC4364 - BGP/MPLS IP Virtual Private Networks (VPNs)– RFC4023 - Encapsulating MPLS in IP or Generic Routing Encapsulation

(GRE)

• Internet Drafts:– draft-ietf-l3vpn-gre-ip-2547-05, Use of PE-PE GRE or IP in BGP/MPLS

IP Virtual Private Networks– draft-ietf-l3vpn-bgp-ipv6-07.txt, BGP-MPLS IP VPN extension for IPv6

VPN– draft-nalawade-kapoor-tunnel-safi-04.txt, Tunnel SAFI– draft-nalawade-l3vpn-bgp-connector-00.txt, BGP Connector Attribute– draft-townsley-l2tpv3-mpls-02.txt, Encapsulation of MPLS over Layer 2

Tunneling Protocol Version 3 (expired)

50

Backup Notes follow …

51

Notes (1)

• Advantages of this solution– employs well-understood, deployed BGP protocol– more efficient BGP processing/packing as AF NLRIs DO NOT

carry softwire tunnel header information; there is a decoupling of the softwire tunnel header distribution from AF reachability distribution

– multiple softwires can be set up between ingress and egress AFBR pair and egress AFBR can express a preference for one over the other; also possible to have one set of NLRIs use one softwire and another set of NLRIs use a different softwire

– extensible to accommodate existing and future address families, softwire tunnel encapsulation attributes, preferences, etc.

– Enables “3rd party” operation where “tunnel broker” injects BGP Tunnel SAFI into system identifying softwire tunnel encaps, end-points, etc.

52

Notes (2)

• Disadvantages of this solution– might be viewed as cumbersome by some to

associate different connector attributes for each (set of) AF NLRIs

53

Notes (3)

• Why not just advertise AF NLRI with different AF next_hop?– violates BGP spec which says NLRI and

next_hop must be same address family– can’t communicate softwire tunnel encap

parameters and preferences in next_hop– major change to BGP implementations

54

Notes (4)• What about the Extended Communities approach?

– idea is to advertise AF NLRI reachability along with a new Extended Community that carries IP tunnel capabilities

– therefore each egress AFBR must advertise the same tunnel information O(# of AF updates) times. Example: if AFBR advertises 1000 BGP updates for prefixes in that AF, then same IP tunnel information is advertised 1000 times. This is 999 more times than is necessary.

– Extended community only defines a bit-mask indicating the type of tunnel supported. No means exists to define a set of tunnels, the encapsulation parameters of the tunnels in the set or, the order of preference of tunnels in the set.

– also assumes that IP tunnel end-point is the same as the BGP next_hop. True when using MPLS LSP but perhaps not true when using IP tunnels. In fact IPsec will likely use an IP address that is completely different from BGP next_hop. Therefore IPSec protection will clearly require special tunnel capability advertisements that identify the IPSec tunnel end-points which Extended Communities does not support

– References: draft-raggarwa-l3vpn-tunnel-attribute-00.txt