© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Ethernet OAM - Technical Overview and
Deployment Scenarios BRKNMS-2202
2
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Agenda
3
Protocol Overview
‒ IEEE 802.3ah (Link OAM)
‒ IEEE 802.1ag(CFM)
‒ ITU-T Y.1731
‒ MEF-16 (E-LMI)
‒ OAM Inter-Working
‒ Ethernet Data-Plane Loopbacks
CFM Design Considerations / Cisco Implementation
Fault Management Scenarios
Performance Management
Platform Support
Summary
Ethernet OAM
Protocol Overview
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Drivers for Ethernet OAM
5
OAM benchmarks
‒ Set by TDM and existing WAN technologies
Operational Efficiency
‒ Reduce OPEX, avoid truck-rolls
‒ Downtime cost
Management Complexity
‒ Large Span Networks
‒ Multiple constituent networks belong to disparate organizations/companies
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Problem Taxonomy
6
Fault Management Fault Detection
Fault Verification
Fault Isolation
Fault Recovery
Fault Notification
Configuration Management Service Provisioning
Performance Management Frame Loss Measurement
Delay Measurement
Delay Variation Measurement
Availability Measurement
Carrier
Ethernet
Services
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Ethernet OAM
7
Protocol Positioning
Access Access Core Customer
Provider Bridges
Provider Bridges
IP/MPLS
Business
Residential
Business
Residential
UNI UNI NNI NNI NNI
Backbone Bridges
Backbone Bridges
Customer
Ethernet Link OAM
Access E-LMI
MPLS OAM
MSE/BNG
Y.1731 Performance Management
Access
Connectivity Fault Management
E-LMI - User to Network Interface (UNI)
Link OAM - Any point-to-point 802.3 link
CFM / Y.1731 - End-to-End UNI to UNI
MPLS OAM - within MPLS cloud
Link OAM
IEEE 802.3-2008 (Clause 57)
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Link OAM
Provides mechanisms useful
for ―monitoring link operation‖,
such as:
‒ Link Monitoring
‒ Remote Failure Indication
‒ Remote Loopback Control
Defines an optional OAM sublayer
Intended for single point-to-point IEEE 802.3 links
Uses ―Slow Protocol‖1 frames called OAMPDUs which are
never forwarded by MAC clients
Standardized: IEEE 802.3ah, clause 57 (now in IEEE 802.3-2008)
(IEEE 802.3ah, Clause 57)
(1) No more than 10 Frames Transmitted in Any One-Second Period
Higher Layers
LLC
OAM (Optional)
MAC
Physical Layer
LAN CSMA/CD
Layers
OSI Model
Application
Presentation
Session
Transport
Network
Data Link
Physical
9
Connectivity Fault Management (CFM)
IEEE 802.1ag
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Overview
11
Family of protocols that provides capabilities to detect, verify, isolate
and report end-to-end ethernet connectivity faults
Employs regular Ethernet frames that travel in-band with the customer
traffic
‒ Devices that cannot interpret CFM Messages forward them as normal data frames
CFM frames are distinguishable by Ether-Type (0x8902) and dMAC
address (for multicast messages)
Standardized by IEEE in 2007
‒ IEEE std. 802.1ag-2007
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Overview (Cont.)
Key CFM mechanisms include:
‒ Nested Maintenance Domains (MDs) that break up the responsibilities for network
administration of a given end-to-end service
‒ Maintenance Associations (MAs) that monitor service instances under a given MD
‒ Maintenance Points (MPs) that generate and respond to CFM PDUs
‒ Protocols (Continuity Check, Loopback and Linktrace) used for Fault
Management activities
12
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Concepts
Defined by Operational/Contractual Boundaries e.g. Customer/Service Provider/Operator
MD may nest and touch, but never intersect
Up to 8 levels of ―nesting‖: MD Level (0..7) ‒ The higher the level, the broader its reach
MD Name Format: null, MAC address, DNS or string-based
Maintenance Domain (MD)
13
Operator A Operator B
Service Provider
Customer
CE CE
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Concepts
Maintenance Association (MA)
14
CE CE Operator A Operator B
Monitors connectivity of a particular service instance in a given MD e.g. 1 service traversing 4 MDs = 4 MAs
Defined by a set of Maintenance End Points (MEP) at the edge of a domain
Identified by MAID == ―Short MA‖ Name + MD Name
Short MA Name Format: Vlan-ID, VPN-ID, integer or string-based
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Concepts Maintenance Point (MP) - MEP
15
CE CE Operator A Operator B
MEP MEP
MEP MEP
MEP MEP MEP MEP
Maintenance Association End Point (MEP)
Define the boundaries of a MD
Support the detection of connectivity failures between any pair of MEPs in an MA
Associated per MA and identified by a MEPID (1-8191)
Can initiate and respond to CFM PDUs
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Concepts Maintenance Point (MP) - MIP
16
MEP MEP MIP MIP
MEP MEP MIP MIP
MEP MEP MEP MEP
MIP MIP
MIP MIP
MIP MIP
MIP MIP MIP MIP
Maintenance Domain Intermediate Point (MIP)
Support the discovery of paths among MEPs and location of faults along those paths
Can be associated per MD and VLAN / EVC (manually or automatically created)
Can add, check and respond to received CFM PDUs
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Protocols
17
There are three (3) protocols defined by CFM
Continuity Check Protocol
‒ Fault Detection
‒ Fault Notification
‒ Fault Recovery
Loopback Protocol
‒ Fault Verification
Linktrace Protocol
‒ Path Discovery and Fault Isolation
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Protocols Continuity Check Protocol
18
CE CE Operator A Operator B
MEP MEP MIP MIP
Catalogue Catalogue
Catalogue and Terminate
1 2 3
1. Continuity Check Message (CCM) X
Used for Fault Detection, Notification and Recovery
Per-Maintenance Association multicast ―heart-beat‖ messages Configurable transmission interval by MEPs (3.3ms, 10ms, 100ms, 1s, 10s, 1min, 10min)
Uni-directional (no response required)
Carries status of port on which MEP is configured
Catalogued by MIPs at the same MD-Level and terminated by remote MEPs in the same MA
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Protocols Loopback Protocol
19
CE CE Operator A Operator B
MEP MEP MIP MIP
S D
1. Loopback Message (LBM)
2. Loopback Reply (LBR) 1
2
1
2
Used for Fault Verification—Ethernet Ping
MEP can transmit a unicast LBM to a MEP or MIP in the same MA
Receiving MP responds by transforming the LBM into a unicast LBR sent back to the originating MEP
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Protocols Linktrace Protocol
20
CE CE Operator A Operator B
MEP MEP MIP MIP
S D
1 3 5
2 4
6
1. 1, 3, 5 Linktrace Message (LTM)
2. 2, 4, 6 Linktrace Reply (LTR)
X
Y
Used for Path Discovery and Fault Isolation—Ethernet Traceroute
MEP can transmit a multicast message (LTM) in order to discover the MPs and path to a MIP or MEP in the same MA
MIPs along the path relay LTM message only when target MAC is known
Each MIP along the path and the terminating MP return a unicast LTR to originating MEP
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Protocols Putting Everything Together
21
1. Run Connectivity Check to proactively detect a soft or hard failure
2. Upon a failure detection, use Loopback to verify it
3. Upon verification, run Traceroute to isolate it; multiple segment LPs can also be used to isolate the fault
4. If the isolated fault points to a virtual circuit, then the OAM tools for that technology can be used to further fault isolation—e.g., for MPLSPW (VCCV) and MPLSLSP (LSP ping and traceroute) can be used
Connectivity Fault Management (CFM) Use Cases
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Use Cases Ethernet L2VPN
Enterprise
Branch
Enterprise
Branch
EVC
EVC
NID /
Access /
Demarcation Device
NID
Unmanaged
L3 CE
IP-based probe
IP-based probe
Ethernet Service Provider
Ethernet Connectivity
IP-based performance probes between HQ and Branch sites
L3 CE
L3 CE
Shadow
Router
Optional
EVC = Ethernet Virtual Circuit
NID = Network Interface Device
= CFM Maintenance Association End Point (MEP)
23
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Use Cases L3VPN with Ethernet Access (CE-PE)
24
Shadow
Router
Optional
L3VPN
Customer
Ethernet Service Provider
EVC
Managed
L3 CE
L3 PE
IP / MPLS Core
VRF
Internet
NID
Third Party Ethernet Service Provider
Or Transport / Aggregation Department
IP Department IP Department
Managed
L3 CE
DSLAM
G.SHDSL
Aggregator
Demarc Device
EVC
IP-based probe
EVC = Ethernet Virtual Circuit
NID = Network Interface Device
= CFM Maintenance Association End Point (MEP)
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Use Cases Mobile Backhaul
25
EVC Demarc
Demarc EVC Demarc
Ethernet Service Provider
Wireline Provider / Mobile Backhaul Operator
RNC
BSC
SAEGW
eNodeB (4G)
NodeB (3G)
BTS (2G)
4G
3G
2G
Ethernet backhaul
Ethernet-based performance probes between Distribution and Cell Site routers
Cell Site
Router
Distribution
Router
Distribution
Router
eNode B = Enhanced Node B
SAEGW = System Arch Evolution GW
RNC = Radio Network Controller
MTSO = Mobile Telephone Switching Office
BTS = Base Transceiver Station
BSC = Base Station Controller
EVC = Ethernet Virtual Circuit
NID = Network Interface Device
= CFM Maintenance Association End Point (MEP)
Mobile Operator Cell Site(s) Mobile Operator MTSO
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Use Cases Broadband/Wholesale Access
26
EVC Aggregator BNG/
BRAS SP Retail ISP
VRF
Internet
Aggregator DSLAM
PONOLT
EVC
(Bitstream Wholesale)
Retail ISP
Residential
Subscribers
Internet
Wholesale SP
subscribers
Ethernet Service Provider
Broadband Department
EVC = Ethernet Virtual Circuit
NID = Network Interface Device
= CFM Maintenance Association End Point (MEP)
Third Party Ethernet Service Provider
Or Transport / Aggregation Department
Broadband Department /
Wholesale SP
OAM Functions and Mechanisms for
Ethernet-Based Networks ITU-T Y.1731
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
ITU-T Y.1731 Overview
ITU-T recommendation for user-plane OAM functionality in Ethernet networks. It defines:
Fault Management mechanisms
Performance Management mechanisms
Standardized by ITU-T SG 13 in May 2006
[ITU-T Y.1731 (2008)] - Published after IEEE 802.1ag standardization
[ITU-T Y.1731 (2008) Amend.1] - Amendment released in July 2010
[ITU-T G.8013/Y.1731 (07/2011)] Pre-published state
Frame formats (Multicast Address, Ethertype, and common OAMPDU fields) and base functionality are generally agreed upon across IEEE 802.1ag and Y.1731
Fault Management
Performance Management
Detection
Verification
Isolation
Notification
Delay
Loss
ITU-T Y.1731
28
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
ITU-T Y.1731 Terminology Comparison with IEEE 802.1ag
29
IEEE 802.1ag ITU-T Y.1731
ME Maintenance Entity ME Maintenance Entity
MA Maintenance Association
MEG ME Group
MAID MA Identifier MEGID MEG Identifier
MD Maintenance Domain --- No such construct available
MD Level
MD Level MEG Level
MEG Level
MEP MA End Point MEP MEG End Point
MIP MD Intermediate Point MIP MEG Intermediate Point
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
ITU-T Y.1731 Overview (cont.)
30
OAM Functions for Fault Management
Ethernet Continuity Check (ETH-CC)
Ethernet Loopback (ETH-LB)
Ethernet Linktrace (ETH-LT)
Ethernet Remote Defect Indication (ETH-RDI)
Ethernet Alarm Indication Signal (ETH-AIS)
Ethernet Locked Signal (ETH-LCK)
Ethernet Client Signal Fail (ETH-CSF)
In addition: ETH-TEST, ETH-APS, ETH-MCC, ETH-EXP, ETH-VSP
OAM Functions for Performance Management
Frame Loss Measurement (ETH-LM)
Synthetic Frame Loss Measurement (ETH-SLM)
Frame Delay Measurement (ETH-DM)
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
ITU-T Y.1731 vs IEEE CFM OAMPDUOpCode Assignments
OpCode
Value
OAMPDU
type
ITU-T
Y.1731
IEEE
802.1ag
1 CCM √ √
2 LBR √ √
3 LBM √ √
4 LTR √ √
5 LTM √ √
33 AIS √ --
35 LCK √ --
37 TST √ --
39 Linear APS √ --
40 Ring APS √ --
41 MCC √ --
42 LMR √ --
43 LMM √ --
OpCode
Value
OAMPDU
type
ITU-T
Y.1731
IEEE
802.1ag
45 1DM √ --
46 DMR √ --
47 DMM √ --
48 EXR √ --
49 EXM √ --
50 VSR √ --
51 VSM √ --
52 CSF √ --
54 SLR √ --
55 SLM √ --
32,34,36,38,
44,53,56-63
Reserved
ITU-T √ --
0, 6-31, 64-
255
Reserved
IEEE -- √
Common OAMPDUs between CFM
and Y.1731
31
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
ITU-T Y.1731 vs IEEE CFM
32
Type Length Value (TLV) Assignments
Type value TLV Name ITU-T
Y.1731
IEEE
802.1ag
0 End TLV √ √
1 Sender ID TLV -- √
2 Port Status TLV -- √
3 Data TLV √ √
4 Interface Status TLV -- √
5 Reply Ingress TLV √ √
6 Reply Egress TLV √ √
7 LTM Egress ID TLV √ √
8 LTR Egress ID TLV √ √
31 Org-Specific TLV -- √
32 Test TLV √ --
33-63 Reserved ITU-T √ --
9-30, 64-255 Reserved IEEE -- √
Most TLVs
defined and
used by IEEE
CFM only
Ethernet Local Management Interface (E-LMI) MEF-16
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Ethernet LMI Overview
Provides protocol and mechanisms used for:
‒ Notification of EVC addition, deletion or
status (Active, Not Active, Partially Active)
to CE
‒ Communication of UNI and EVC attributes
to CE (e.g. CE-VLAN to EVC map)
‒ CE auto-configuration
‒ Notification of Remote UNI name and status to CE
Asymmetric protocol based on
Frame Relay LMI, mainly applicable
to the UNI (UNI-C and UNI-N)
Specification completed by MEF:
http://www.metroethernetforum.
org/PDFs/Standards/MEF16.doc
CE
User Network Interface (UNI)
UNI-C UNI-N
E-LMI
Metro Ethernet Network
Cisco
Enhancement
34
Ethernet OAM Interworking
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Ethernet over SONET
Ethernet over MPLS
10G Ethernet
ATM RFC1483
Edge
Brid
ge
Brid
ge
Brid
ge
Brid
ge
Edge
Ro
ute
r
switch
switch
Ro
ute
r
Ro
ute
r
Ro
ute
r
ATM
Switch
ATM
Switch
Operator A Operator B
Network OAM Network OAM
Cu
stom
er B
ridge
Cu
stom
er B
ridge
Transport OAM
Transport OAM
Transport OAM
Transport OAM
Server Layers feed events into Client Layers
Event Translation inter-domains intra-layer
Service OAM
What Is OAM Interworking?
Strict OAM layering should be honored: messages should not cross layers
OAM Messages should not leak outside domain boundaries within a layer
Interworking is event translations & not necessarily 1:1 message mapping
Interworking may be inter-layer and intra-layer
36
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Interworking Scenarios Main Examples
CFM
Link OAM
MPLS PW OAM
E-LMI
CFM
E-LMI
37
Ethernet Data-Plane Loopbacks
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Throughput Measurements Ethernet Data-Plane Loopback
Ethernet data traffic can be looped back on a per port / per VLAN basis
Use cases:
‒ Service turn-up
‒ Post service turn-up troubleshooting
‒ Out-of-service throughput testing
Enabled via CLI configuration
Configurable SRC and DST MAC Addresses swap
Configurable direction:
‒ Facility Loopback (facing wire)
‒ Terminal Loopback (facing bridge)
Complements CFM Loopback
External central Test Head allows for flexible and sophisticated test traffic patterns UNI Customer
CPE
Central Test Head
Carrier Ethernet Network
NNI Facility Loopback
Terminal
Loopback
39
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Ethernet Loopback Comparison
Functionality IEEE 802.1ag /
Y.1731 Loopback
IEEE 802.3ah Remote
Loopback
Ethernet Data-Plane Loopback
Triggering Mechanism CLI CLI / In-Band (OAMPDUs)
signal from Master NE CLI
Loopback Type Per-Port / Per-VLAN
(according to initiator MEP configuration)
Per-Port Per-Port / Per-VLAN
Test Intrusiveness In-Service and Out-of-Service Out-of-Service Out-of-Service
Looped Frames OAM frames (LBM/LBR) Data frames Data frames
SA and DA MAC swap YES NO YES
Loopback Direction
Follows direction of responding MEP: Facility / Terminal (down / up mep
respectively)
Facility Configurable (Facility /
Terminal)
Test Head to Test Point connectivity
Single-Hop / Multi-Hop bridged networks
Single-Hop bridged network (between Master -
Slave)
Single-Hop / Multi-Hop bridged networks
40
Connectivity Fault Management (CFM) Design Considerations
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Design Considerations Maintenance Domain
CFM Design ‒ Align CFM domains with operational responsibilities / boundaries
‒Domains expanding different organizations technically possible but …
‒Consider challenges such as MEP alarm reporting with multiple NOCs involved
CFM Maintenance Domains (MD) ‒ Allocate efficiently number of MDs in the network
‒ A domain hosts multiple services / MAs
‒Typically, one (1) domain used to host all Maintenance Associations within organizational boundaries
MD Level Allocation
Source: IEEE 802.1ag-2007
42
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Design Considerations Maintenance Associations and MEPs
CFM Maintenance Association (MA) ‒ CFM MAID must be the same at end points of a monitored service, otherwise considered as an
error and alarm raised
‒ CFM MAID = Maintenance Association name + Maintenance Domain name
MAs must be terminated by MEPs ‒ Typical deployments start with MEP-only MAs (i.e. no MIPs)
‒ MEPs placed at UNI for e2e fault detection and verification
MEP ID value allocation ‒ MEPs must have a unique MEP ID (mpid) value within a given service
‒ Mpid value can be reused across MEPs on different services
‒ Consider use of same mpid for all MEPs in a given Network Element
MEP’s direction determined by monitoring objectives ‒ UP MEP
‒ Down MEP
43
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Design Considerations UP MEP
CFM PDUs generated by the MEP are sent towards the Bridge’s Relay Function and not via the wire connected to the port where the MEP is configured
CFM PDUs to be responded by the MEP are expected to arrive via the Bridge’s Relay Function
Applicable to switches
UP MEPs typically found on UNIs (e.g. Port A)
Port A
Bridge
Relay Entity
Port B
CFM PDUs
44
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Design Considerations DOWN MEP
CFM PDUs generated by the MEP are sent via the wire connected to the port where the MEP is configured
CFM PDUs to be responded by the MEP are expected to arrive via the wire connected to the port where the MEP is configured
Port MEP – special Down MEP at level zero (0) used to detect faults at the link level (untagged - rather than service)
Applicable to routers and switches
Port A
Bridge / Router
Relay Entity
Port B
CFM PDUs
45
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Monitored area
CFM Design Considerations MAs and UP/DOWN MEPs
MEPs of different direction can be configured on the same MA
Bridge
Port
Bridge 1
Relay Entity
Bridge
Port
Bridge
Port
Bridge 2
Relay Entity
Bridge
Port
Monitored area
Bridge
Port
Bridge 1
Relay Entity
Bridge
Port
Bridge
Port
Bridge 2
Relay Entity
Bridge
Port
UP MEP to UP MEP
DOWN MEP to UP MEP
46
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Design Considerations Etherchannel considerations
NE NE Demarc / NID
Transport Dark Fiber
SONET/SDH
DWDM
Switched Eth
EoMPLS
Demarc / NID
CFM MIP / Down MEP (shown)
on Port-Channel (sub-)interface
Untagged CFM Down MEPs
on bundle member interfaces
47
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Design Considerations MIPs
MIPs configured at intermediate nodes ‒ Typical deployment adds MIPs on follow-up phase (e.g. due to HW/SW considerations at
intermadiate devices)
MIP configuration ‒ Manual MIPs
‒ Auto-created MIPs (according to policies)
(Optional) Create manual MIPs at UNIs
Use Default-domain MIP auto-create policies for intermediate nodes
Use MA MIP auto-create policies for devices hosting MEPs
48
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Design Considerations Continuity Check Protocol
CCM Scalability ‒ Timer-based detection protocol
‒ Same scale implications as with any other protocols like BFD
CCM Transmission Interval ‒Intervals are discrete steps (e.g. no intermediate values between 100msec and 1 sec)
Transmission interval should set according to intended use
‒ End-to-end Service Monitoring - 1 sec most common value
‒ Resiliency – 1 sec or lower
CCM could even be turned OFF or run at slow rate (e.g. 10sec) in normal circumstances and be accelerated during troubleshooting periods / after customer complains
49
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Design Considerations CC Protocol – Interop with Y.1731 ETH-CC
Feature ITU-T Y.1731
IEEE CFM
Comments
Ethernet Continuity Check
ETH-CC CC
Proactive fault detection, fault notification
No compatibility between IEEE 802.1ag-2007 (CFM) and ITU-T Y.1731-2008 messages due to different formats for shortMA / MEG ID fields. Overall, IEEE CFM offers more flexibility/choices concerning formats for MD and ShortMA/MEG ID fields
IEEE CFM defines:
MD name format: NULL, MAC address, DNS or string-based
ShortMA name format: Vlan-ID, VPN-ID, integer or string-based
ITU-T Y.1731-2008 defines:
MD name format: NULL only
MEG-ID name format: ICC-based only
Compatibility between 802.1ag-2007 and Y.1731-2010 Amendment.
Y.1731 extends MEG-ID name formats to includes those in IEEE CFM.
ICC-based MEG ID continues to be an ITU-only format
IEEE CFM allows optional TLVs in CCM to improve fault notification for UP MEPs (i.e. Port Status and Interface Status TLVs). IEEE CFM also allows Sender ID TLVs and Organization-Specific TLVs for vendor extensions. No optional TLVs in Y.1731 ETH-CC
IEEE CFM defines multicast Destination Address for CCM, ITU-T defines both multicast and unicast DA CCM. Most fault management scenarios relay on multicast DA
50
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Design Considerations
Provisioning service verification utility executes after every CCM received
for a service (Starts checking after loss threshold * CCM interval delay)
Compares remote MEPs CCM received versus expected MEPs
(configured static list) for a given service
Support defect detections for
‒ Unexpected end-point / Missing end-point
CC Protocol - Remote MEP Learning (Static)
CE 1 CE 4
MEP 1
CE 2 CE 3
MEP 4
MEP 2 MEP 3
mpid 1 mpid 3 mpid 4
mpid 1 mpid 2 mpid 4
mpid 1 mpid 2 mpid 3
mpid 2 mpid 3 mpid 4
Crosscheck List:
51
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Design Considerations CC Protocol - Remote MEP Learning
Dynamic
Does NOT require static rmep list
Extension to standard
Simpler provisioning especially for multipoint services
ALL incoming CCMs arriving on a given MA / MEG are accepted and dynamically placed in the MEP CCDB and MIP CCDB
Static
User does configure static rmep list
Behavior described in the standard
ONLY incoming CCMs arriving on a given MA / MEG that match the static rmep list are accepted in the MEP CCDB and MIP CCDB
Incoming CCMs from MEPs not defined in the list are rejected and stored as ―Unknow MEP‖ in Error DB
‒DefErrorCCM alarm raised
Missing CCMs from MEPs defined in the list are stored as ―Remote MEP missing‖ in Error DB
‒DefRemoteCCM alarm raised
52
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Design Considerations Fault Notification and Instrumentation
Fault Notification (Alarms) accomplished by MEPs ‒ No alarms raised by MIPs
Various instrumentation alternatives
Console Syslogs
IEEE CFM MIB (802.1ap) (IEEE-defined alarm)
‒ Note that Clear-alarm notifications not included in standard
Vendor enterprise MIBs (e.g. Cisco-defined alarms)
‒ Support alarm and clear notifications
MEF FM MIB (MEF 31)
‒ Addresses IEEE gap with clear notifications
‒ Alarms for Y.1731 FM mechanisms (No MIBs defined by ITU-T for Y.1731)
53
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Design Considerations Instrumentation – IEEE-defined alarm
IEEE Alarm Name
CCM-Based Defects
DefXconCCM Reception by a MEP of a CCM with an incorrect MAID (cross connect error)
Reception by a MEP of a CCM with an MD Level lower than that of the MEP
DefErrorCCM
Reception by a MEP of a CCM from a MEPs not included in a static list
Reception by a MEP of a CCM with an incorrect MEPID (duplicate mpid error)
Reception by a MEP of its own CCM
Reception by a MEP of a CCM with an incorrect transmission interval
DefRemoteCCM Inability to receive consecutive CCMs from any one of the other MEPs in its MA
Inability to receive CCMs from any one of the MEPs configured in a static list
DefMACstatus Reception by a MEP of a CCM containing a Port Status TLV or Interface Status TLV indicating a failed port
DefRDICCM Reception by a MEP of a CCM with the Remote Defect Indicator (RDI) bit set Lowest
Highest
Ala
rm P
rio
rity
54
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
CFM Design Considerations
VLAN Acrobatics / Manipulation
‒ VLAN IDs are not carried inside CFM PDUs, thus VLAN rewrites should not
impact CFM operation
Congruency of CFM / Service frame forwarding paths
‒ Ensure that Ethernet vlan stack applied to service frames is the same applied to
CFM frames used for monitoring
55
Connectivity Fault Management (CFM) Cisco Implementation
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Cisco IOS CFM Implementation Example 1: CFM on Switchport
CE CE Operator A Operator B
UNI
Global Configuration
interface GigabitEthernet2/1 switchport trunk allowed vlan 500 switchport mode trunk ethernet cfm mip level 4 vlan 500
MD and MD Level
MA
UP MEP Manual MIP
MEP GE2/7
MIP GE2/1
NNI
interface GigabitEthernet2/7 switchport trunk allowed vlan 500 switchport mode trunk ethernet cfm mep domain Domain_L4 mpid 111 vlan 500
ethernet cfm ieee ethernet cfm global ! ethernet cfm domain Domain_L4 level 4 service customer_500_provider vlan 500 continuity-check continuity-check interval 1s ! ethernet cfm logging alarm cisco ethernet cfm logging alarm ieee
ME-3400 3750-ME
6500 MWR 2941
For your reference
57
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Cisco IOS CFM Implementation Example 2: CFM on Layer 3 interfaces
For your reference
interface GigabitEthernet0/1 ethernet cfm mep domain DOMAIN-L7 mpid 111 vlan 4000 ethernet cfm mep domain DOMAIN-L7 mpid 111 vlan 3000 ethernet cfm mep domain DOMAIN-L7 mpid 111 vlan 2000 ! interface GigabitEthernet0/1.2000 encapsulation dot1Q 2000 ip address 1.1.1.1 255.255.255.0 ! interface GigabitEthernet0/1.3000 encapsulation dot1Q 3000 ip address 2.2.2.1 255.255.255.0 ! interface GigabitEthernet0/1.4000 encapsulation dot1Q 4000 ip address 3.3.3.1 255.255.255.0
ethernet cfm ieee ethernet cfm global ! ethernet cfm domain DOMAIN-L7 level 7 service EVPL-2000 vlan 2000 direction down continuity-check service EVPL-3000 vlan 3000 direction down continuity-check service EVPL-4000 vlan 4000 direction down continuity-check
DOWN MEPs
Global Configuration Interface Configuration
ISRrouters ASR 1000
58
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Cisco IOS CFM Implementation Example 3: CFM on Service Instance with bridge-domain
For your reference
interface GigabitEthernet2/1 description UNI service instance 665 ethernetsampleEVC encapsulation dot1q100 rewrite ingress tag pop 1 symmetric bridge-domain 777 cfmmep domain Domain_L5mpid 1111 cfmmip level 6 ! interface GigabitEthernet2/2 description EVCNNI service instance 666 ethernetsampleEVC encapsulation dot1q200 rewrite ingress tag pop 1 symmetric bridge-domain 777 cfmmip level 5 ! interface GigabitEthernet2/3 description switchportNNI switchport trunk allowed vlan777 switchport mode trunk ethernetcfmmip level 5 vlan 777
ethernetcfmieee ethernetcfm global ! ethernetcfm domain Domain_L5 level 5 service E2E_MA_nameevcsampleEVCvlan 777 continuity-check continuity-check interval 1s ! ethernetevcsampleEVC ! ethernetcfm logging alarm cisco ethernetcfm logging alarm ieee
Global Configuration Interface Configuration
Example includes Service Instance (with BD) and switchports in the same broadcast domain
2/3 dot1q 777 switchport
2/2 dot1q 200
BD 777 UP MEP
7600
59
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Cisco IOS CFM Implementation Example 4: CFM on Service Instance with Xconnect
For your reference
7600
interface GigabitEthernet2/1 description PW AC service instance 665 ethernet sampleEVC encapsulation dot1q 100 rewrite ingress tag pop 1 symmetric xconnect 2.2.2.2 123 pw-class vlan-xconnect cfm mip level 4 cfm mep domain Domain_L3 mpid 1111 !
ethernet cfm ieee ethernet cfm global ! ethernet cfm domain Domain_L3 level 3 service sampleMA evc sampleEVC continuity-check continuity-check interval 1s ! ethernet evc sampleEVC ! pseudowire-class vlan-xconnect encapsulation mpls ! ethernet cfm logging alarm cisco ethernet cfm logging alarm ieee
Global Configuration Interface Configuration
2/1 dot1q 100 xconnect
PW
UP MEP
60
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Cisco IOS CFM Implementation Example 5: CFM on Layer 2 and Layer 3 Interfaces
For your reference
ASR9000
interface GigabitEthernet0/1/0/1.156 l2transport encapsulation dot1q 156 ethernet cfm mep domain Domain_L2 service sampleMA1 mep-id 15 ! interface GigabitEthernet0/1/0/2 ipv4 address 20.20.14.15 255.255.255.0 ethernet cfm mep domain Domain_L2 service sampleMA2 mep-id 15
ethernet cfm domain Domain_L2 level 2 service sampleMA1 down-meps continuity-check interval 1s log continuity-check errors log continuity-check mep changes ! service sampleMA2 down-meps continuity-check interval 1s log continuity-check errors log continuity-check mep changes ! ! !
Global Configuration Interface Configuration
0/1/0/1.156 dot1q 156 EoMPLS AC
PW
DOWN MEP 0/1/0/2 L3 interface
61
Fault Management Scenarios
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
Access Access Aggregation Aggregation CE CE
Point-to-Point Ethernet Service
63
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
Access Access Aggregation Aggregation CE CE
CFM E-LMI Link OAM
E-LMI Link OAM
OAM protocol positioning
CFM to E-LMI IW
CFM to E-LMI IW
Link OAM to CFM IW
Link OAM to CFM IW
64
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
CFM Continuity Check Messages (CCM)
UPE11#show ethernet cfm maintenance-points remote -------------------------------------------------------------------------------- MPID Domain Name MacAddress IfSt PtSt Lvl Domain ID Ingress RDI MA Name Type Id SrvcInst EVC Name Age -------------------------------------------------------------------------------- 3100 PROVIDER_DOMAIN aabb.cc00.0599 Up Up 4 PROVIDER_DOMAIN Et0/1.100 - customer_100_provider Vlan 100 N/A N/A 0s Total Remote MEPs: 1
CCM Database
t0
t1
t2
CE 11 CE 31 uPE 31 uPE 11 AGG 31 AGG 11
Proactive End-to-End Service Monitoring
65
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
UPE11#ping ethernet mpid 3100 domain PROVIDER_DOMAIN vlan 100 Type escape sequence to abort. Sending 5 Ethernet CFM loopback messages to aabb.cc00.0599, timeout is 5 seconds :!!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 4/5/12 ms
CFM Loopback Message (LBM)
CFM Loopback Reply (LBR)
uPE11# ping ethernet
End-to-end Service/Failure Verification
CE 11 CE 31 uPE 31 uPE 11 AGG 31 AGG 11
66
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
CFM Linktrace Reply (LTR) CFM Linktrace Message (LTM)
uPE11# traceroute ethernet
Service Path Discovery/Failure Isolation
CE 11 CE 31 uPE 31 uPE 11 AGG 31 AGG 11
67
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
UPE11#traceroute ethernet mpid 3100 domain PROVIDER_DOMAIN vlan 100 Type escape sequence to abort. TTL 64. Linktrace Timeout is 5 seconds Tracing the route to aabb.cc00.0599 on Domain PROVIDER_DOMAIN, Level 4, vlan 100 Traceroute sent via Ethernet0/1.100, path found via MPDB B = Intermediary Bridge ! = Target Destination * = Per hop Timeout -------------------------------------------------------------------------------- MAC Ingress Ingr Action Relay Action Hops Host Forwarded Egress Egr Action Previous Hop -------------------------------------------------------------------------------- B 1 AGG11 aabb.cc00.0399 Et0/0.100 IngOk RlyMPDB Forwarded Et0/1.100 EgrOK aabb.cc00.0299 B 2 AGG31 aabb.cc00.0499 Et0/0.100 IngOk RlyMPDB Forwarded Et0/1.100 EgrOK aabb.cc00.0399 ! 3 UPE31 aabb.cc00.0599 Et0/0.100 IngOk RlyHit:MEP Not Forwarded aabb.cc00.0499
uPE11# traceroute ethernet
Service Path Discovery/Failure Isolation
CE 11 CE 31 uPE 31 uPE 11 AGG 31 AGG 11
68
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
ELMI Status Enquiry message (Full Status report)
ELMI Status message (Full Status report)
Local UNI ID CE11_UNI CE-VLAN/EVC Map type Service_Multiplexing EVC ID EVC_P2P_100 EVC Type Point_to_Point CE-VLAN/EVC Map vlan 100 EVC Status New, Active Remote UNI count – configured 1 Remote UNI count – active 1 Remote UNI ID CE31_UNI Remote UNI status UP
Example:
CE 11 CE 31 uPE 31 uPE 11 AGG 31 AGG 11
Cisco enhancements to ELMI
CE Notification
69
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
CE Notification
CE 11 CE 31 uPE 31 uPE 11 AGG 31 AGG 11
CE11#show ethernet lmi evc detail EVC_P2P_100 EVC Id: EVC_P2P_100 interface GigabitEthernet0/0 Time since Last Full Report: 00:21:23 Ether LMI Link Status: Up UNI Status: Up UNI Id: CE11_UNI CE-VLAN/EVC Map Type: Bundling VLAN: 100 EVC Status: Active EVC Type: Point-to-Point Remote UNI Count: Configured = 1, Active = 1 UNI Id UNI Status Port ------ ---------- ---- CE31_UNI Up Remote
CE11#show ethernet lmi evc map interface gig0/0 UNI Id: CE11_UNI St Evc Id CE-VLAN --- ----------------- -------- A EVC_MP_250 250 A EVC_P2P_100 100 Key: St=Status, A=Active, P=Partially Active, I=Inactive, *=Default EVC, ?=Link Down
CE11#show ethernet lmi evc detail EVC_MP_250 EVC Id: EVC_MP_250 interface GigabitEthernet0/0 Time since Last Full Report: 00:25:54 Ether LMI Link Status: Up UNI Status: Up UNI Id: CE11_UNI CE-VLAN/EVC Map Type: Bundling VLAN: 250 EVC Status: Active EVC Type: Multipoint-to-Multipoint Remote UNI Count: Configured = 2, Active = 2 UNI Id UNI Status Port ------ ---------- ---- CE12_UNI Up Remote CE31_UNI Up Remote
70
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
CE Notification—VLAN ID Missmatch
CE 11 CE 31 uPE 31 uPE 11 AGG 31 AGG 11
CE11(config)#interface gig0/0.100 CE11(config-subif)#encapsulation dot1Q 100 CE11#show ip interface brief Interface IP-Address OK? Method Status Protocol <snip> GigabitEthernet0/0.100100.100.100.11 YES NVRAMup up
CE Configured with the correct C-VLAN (e.g. vid 100)
71
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
CE Notification—VLAN ID Missmatch
CE 11 CE 31 uPE 31 uPE 11 AGG 31 AGG 11
CE Configured with the incorrect C-VLAN (e.g. vid 1300) CE11(config)#interface gig0/0.100 CE11(config-subif)#encapsulation dot1Q 1300 Jan 26 00:15:39.546:%ETHER_LMI-6-MISMATCHED_VLAN_NOT_CONFIGURED: VLAN 100 not Configured but in VLAN mapping for UNI GigabitEthernet0/0 Jan 26 00:15:39.546:%ETHER_LMI-6-MISMATCHED_VLAN_CONFIGURED: VLAN 1300 configured but not in VLAN mapping for UNI GigabitEthernet0/0 Interface CE11#show ip interface brief Interface IP-Address OK? Method Status Protocol <snip> GigabitEthernet0/0.100100.100.100.11 YES NVRAM down down
Proactive ELMI Action at CPE
72
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
Access Access Aggregation
ELMI Status message Async EVC report
ELMI Status message Async EVC report
ELMI action: CE brings down (sub)interface
ELMI action: CE brings down (sub)interface
CFM remote MEP timeout MEP Down (timeout) alarm DefRemoteCCM IEEE alarm EVC declared Inactive CFM to ELMI Interworking
CFM remote MEP timeout MEP Down (timeout) alarm DefRemoteCCM IEEE alarm EVC declared Inactive CFM to ELMI Interworking
Failure Scenario: Network Failure
Aggregation
X CE CE
73
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
Error DB
UPE11# *Apr 8 04:33:44.911: %E_CFM-3-REMOTE_MEP_DOWN: Remote MEP mpid 3100 vlan 100 MA name customer_100_provider in domain PROVIDER_DOMAIN changed state to down with event code TimeOut. *Apr 8 04:33:44.911: %ETHER_SERVICE-6-EVC_STATUS_CHANGED: status of EVC_P2P_100 changed to InActive *Apr 8 04:33:47.587: %E_CFM-3-FAULT_ALARM: A fault has occurred in the network for the local MEP having mpid 1100 vlan 100 for service MA name customer_100_provider with the event code DefRemoteCCM. UPE11#show ethernt cfm errors -------------------------------------------------------------------------------- MPID Domain Id Mac Address Type Id Lvl MAName Reason Age -------------------------------------------------------------------------------- 3100 PROVIDER_DOMAIN aabb.cc00.0599 Vlan 100 4 customer_100_provider Lifetime Timer Expired 119s
CE 11 CE 31 uPE 31 uPE 11
X AGG 31 AGG 11
IEEE-defined alarm
Cisco-defined alarm
74
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
UPE11#ping ethernet aabb.cc00.0599 domain PROVIDER_DOMAIN vlan 100 Type escape sequence to abort. Sending 5 Ethernet CFM loopback messages to aabb.cc00.0599, timeout is 5 seconds :..... Success rate is 0 percent (0/5) UPE11#traceroute ethernet aabb.cc00.0599 domain PROVIDER_DOMAIN vlan 100 Type escape sequence to abort. TTL 64. Linktrace Timeout is 5 seconds Tracing the route to aabb.cc00.0599 on Domain PROVIDER_DOMAIN, Level 4, vlan 100 Traceroute sent via Ethernet0/1.100, path found via MPDB B = Intermediary Bridge ! = Target Destination * = Per hop Timeout -------------------------------------------------------------------------------- MAC Ingress Ingr Action Relay Action Hops Host Forwarded Egress Egr Action Previous Hop -------------------------------------------------------------------------------- B 1 AGG11 aabb.cc00.0399 Et0/0.100 IngOk RlyMPDB Forwarded Et0/1.100 EgrOK aabb.cc00.0299 * * * *
CE 11 CE 31 uPE 31 uPE 11
X AGG 31 AGG 11
75
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
CE11# *Apr 8 04:33:44.991: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet0/0.100, changed state to down CE11#show ethernet lmi evc detail EVC_P2P_100 EVC Id: EVC_P2P_100 interface Ethernet0/0 Time since Last Full Report: 00:01:13 Ether LMI Link Status: Up UNI Status: Up UNI Id: CE11_UNI CE-VLAN/EVC Map Type: Service Multiplexing with no bundling VLAN: 100 EVC Status: Inactive EVC Type: Point-to-Point Remote UNI Count: Configured = 1, Active = 0 UNI Id UNI Status Port ------ ---------- ---- CE31_UNIUnreachable Remote
CE 11 CE 31 uPE 31 uPE 11
X AGG 31 AGG 11
Network Failure: Remote UNI shows UNREACHABLE
76
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
ELMI action: CE brings down (sub)interface
UNI Link Down
ELMI Status message Async EVC report
CFM MEP Up (port state Down) alarm DefMACstatus IEEE alarm EVC declared Inactive CFM to ELMI InterWorking
EVC declared Inactive
CFM CCM Interface Status TLV “isDown”
X
Failure Scenario: UNI Link Down
Access Access Aggregation Aggregation CE CE
77
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
UPE11# *Apr 8 04:41:54.823: %E_CFM-6-REMOTE_MEP_UP: Continuity Check message is received from a remote MEP with mpid 3100 vlan 100 MA name customer_100_provider domain PROVIDER_DOMAIN interface status Down event code PortState. *Apr 8 04:41:54.823: %ETHER_SERVICE-6-EVC_STATUS_CHANGED: status of EVC_P2P_100 changed to InActive *Apr 8 04:41:57.451: %E_CFM-3-FAULT_ALARM: A fault has occurred in the network for the local MEP having mpid 1100 vlan 100 for service MA name customer_100_provider with the event code DefMACstatus. UPE11#show ethernet cfm maintenance-point remote -------------------------------------------------------------------------------- MPID Domain Name MacAddress IfSt PtSt Lvl Domain ID Ingress RDI MA Name Type Id SrvcInst EVC Name Age -------------------------------------------------------------------------------- 3100 PROVIDER_DOMAIN aabb.cc00.0599 Down Up 4 PROVIDER_DOMAIN Et0/1.100 - customer_100_provider Vlan 100 N/A N/A 0s Total Remote MEPs: 1
X CE 11 CE 31 uPE 31 uPE 11 AGG 31 AGG 11
78
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
CE11# *Apr 8 04:41:54.907: %LINEPROTO-5-UPDOWN: Line protocol on Interface Ethernet0/0.100, changed state to down CE11#show ethernet lmi evc detail EVC_P2P_100 EVC Id: EVC_P2P_100 interface Ethernet0/0 Time since Last Full Report: 00:01:07 Ether LMI Link Status: Up UNI Status: Up UNI Id: CE11_UNI CE-VLAN/EVC Map Type: Service Multiplexing with no bundling VLAN: 100 EVC Status: Inactive EVC Type: Point-to-Point Remote UNI Count: Configured = 1, Active = 0 UNI Id UNI Status Port ------ ---------- ---- CE31_UNIDown Remote
X CE 11 CE 31 uPE 31 uPE 11 AGG 31 AGG 11
UNI Failure: Remote UNI shows DOWN
79
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
CE CE PE PE EoMPLS Pseudowire
Point-to-Point Ethernet Service
80
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
CE CE PE PE EoMPLS Pseudowire
Directed LDP session E-LMI E-LMI
OAM Protocol Positioning
PW OAM To E-LMI IW
PW OAM To E-LMI IW
81
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
ELMI action: CE brings down (sub)interface
CE CE PE PE EoMPLS Pseudowire
CE UNI failure: Admin “shutdown”
ELMI Status message Async EVC report
PW declared DOWN (syslog) EVC declared Inactive (syslog) PW OAM to ELMI InterWorking
EVC declared Inactive PW declared DOWN (syslog)
Tx LDP TLV Status (PW status: AC DOWN)
X
Failure Scenario: UNI Failure
82
Ethernet Performance Management
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Key Performance Indicators (KPIs)
Frame Loss Ratio(FLR) – percentage (%) of service
frames* (or synthetic frames) not delivered / Total
number of service frames (or synthetic frames)
transmitted in T time interval
Frame Delay (FD) – round-trip/one-way delay for a
service frame
Frame Delay Variation(FDV) – Variation in frame delay
between a pair of service frame
Service Availability
* Service frames (Green) are frames that conform to
agreed upon level of bandwidth profile conformance
84
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Ethernet Performance Management ITU-T Y.1731 – Technology Overview
Single-Ended
ETH-SLM
• Unidirectional frame
loss measurements
• Applicable to P2P
and MP services
• SLM / SLRPDUs
• Based on statistical
sampling
• Exchange of synthetic
frame counters
Ethernet
Synthetic Loss
Measurement
ETH-SLM
NEW
Dual-Ended
ETH-LM
Single-Ended
ETH-LM
Ethernet Loss Measurement
ETH-LM
• Unidirectional frame
loss measurements
• Applicable to P2P
services only
• LMM / LMRPDUs
• On-demand operation
• Based on actual
Service Frame Loss
• Exchange of service
frame counters
• Unidirectional frame
loss measurements
• Applicable to P2P
services only
• CCMPDUs
• Proactive operation
• Based on actual
Service Frame Loss
• Exchange of service
frame counters
Ethernet Delay Measurement
ETH-DM
One-Way ETH-
DM
Two-Way ETH-
DM
• One-Way delay
measurements
• 1DM PDU
• Synthetic traffic with
two (2) timestamps
• Need for Time-of-Day
synchronization
• Two-Way delay
measurements
• Also One-Way delay
when synchronized
• DMM / DMRPDUs
• Synthetic traffic with
two (2) mandatory
and two (2) optional
timestamps
Scope
Specifics
85
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Standard CFM PDUs
• Two-Way delay measurements
• Standard LBM / LBRPDU
• Interoperable with non-Cisco CFM-
capable devices (NO need for Cisco
Responder)
• RTT measurement includes remote
end-point processing (no timestamps
added by Responder)
Vendor-specific CFM PDUs
• Two-Way delay measurements
• One-Way delay when synchronized
• Unidirectional frame loss
measurements (applicable to P2P and
MP services)
• Proprietary CFM PDUs
• Requires Cisco Responder
• Frame loss based on statistical
sampling
Sp
ecif
ics
Sco
pe
Ethernet Performance Management CFM-based Alternatives (non-ITU-T Y.1731)
86
Platform Support
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Ethernet OAM Cisco Platform Support
Access
Cisco 7600
ASR 9000
XR 12000
Catalyst 6500
Catalyst 4500
Catalyst 4900
Cisco ONS 15454
Cisco CGS 2520
Cisco ME3400
Cisco ME3400E
Cisco ME3600X
Cisco ME3800X
Cisco ME4924
Cisco ME6524
Catalyst 3750-ME
ASR 901
ASR 903
Cisco 880/890
Cisco 1800/2800/3800
Cisco 1900/2900/3900
Cisco 7200/7201/7301
Cisco MWR 2941
Cisco CGR 2010
Cisco ASR 1000
Aggregation
Customer
Premise
Core
CRS-1
CRS-3
88
Summary
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Cisco Carrier Ethernet OAM
Industry’s Only Solution with End-to-End Manageability and Customer Loyalty: Service Level Assurance
MPLS OAM
Ethernet LMI
Ethernet Link
OAM
Embedded Event
Manager EEM
Connectivity Fault
Management CFM
IP Service Level
Agreement
OAM Interworking
90
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Complete Your Online
Session Evaluation Give us your feedback and you
could win fabulous prizes.
Winners announced daily.
Receive 20 Passport points for each
session evaluation you complete.
Complete your session evaluation
online now (open a browser through
our wireless network to access our
portal) or visit one of the Internet
stations throughout the Convention
Center.
Don’t forget to activate your
Cisco Live Virtual account for access to
all session material, communities, and
on-demand and live activities throughout
the year. Activate your account at the
Cisco booth in the World of Solutions or visit
www.ciscolive.com.
91
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Acronyms Acronym
AIS Alarm Indication Signal
CCM Continuity Check Message
CCMDB CCM Data Base (see CCM)
CE Customer Edge
CFM Connectivity Fault Management
EFM Ethernet in the First Mile
E-LMI Ethernet LMI (see LMI)
E-OAM Ethernet OAM (see OAM)
EVC Ethernet Virtual Connection
IEEE Institute of Electrical and Electronics Engineers
ITU International Telecommunication Union
LBM Loopback Message
LBR Loopback Reply
LMI Local Management Interface
LTM Linktrace Message
LTR Linktrace Reply
MA Maintenance Association
MAID MA Identifier (see MA)
MD Maintenance Domain
Acronym
MEF Metro Ethernet Forum
MEN Metro Ethernet Network
MEP Maintenance Association End Point
MEPID MEP Identifier (see MEP)
MHF MIP Half Function (see MIP)
MIB Management Information Base
MIP Maintenance Domain Intermediate Point
MP Maintenance Point
OAM Operations, Administration and Maintenance
PDU Protocol Data Unit
PE Provide Edge
RDI Remote Defect Indicator
RFI Remote Failure Indicator
TLV Type, Length, Value
UNI User to Network Interface
UNI-C Customer side of UNI (see UNI)
UNI-N Network side of UNI (see UNI)
VID VLAN Identifier
VLAN Virtual LAN
93
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
E-OAM Documentation at Cisco.com
IOS Carrier Ethernet Configuration Guide
‒ http://www.cisco.com/en/US/docs/ios-xml/ios/cether/configuration/15-2s/ce-15-2s-book.html
IOS XE Carrier Ethernet Configuration Guide
‒ http://www.cisco.com/en/US/docs/ios-xml/ios/cether/configuration/xe-3s/ce-xe-3s-book.html
Service Diagnostics Documentation and Scripts
‒ http://www.cisco.com/go/iossd
Link OAM Configuration Guide
‒ http://www.cisco.com/en/US/docs/ios/cether/configuration/guide/ce_oam.html
Y.1731 Fault Management Configuration Guides
‒ http://www.cisco.com/en/US/docs/ios/cether/configuration/guide/ce_cfm-ieee_y1731.html
Y.1731 Performance Management Configuration Guides
‒ http://www.cisco.com/en/US/docs/ios-xml/ios/cether/configuration/xe-3s/ce-y1731-perfmon.html
Ethernet Local Management Interface (E-LMI) Configuration Guides
‒ http://www.cisco.com/en/US/docs/ios/cether/configuration/guide/ce_elmi-pe.html (PE role)
‒http://www.cisco.com/en/US/docs/ios/cether/configuration/guide/ce_elmi_ps10591_TSD_Products_Configuration_Guide_Chapter.html (CE role)
94
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
E-OAM Documentation at Cisco.com
IP SLAs for Metro Ethernet
‒ http://www.cisco.com/en/US/docs/ios/ipsla/configuration/guide/sla_metro_ethernet.html
IP SLAs LSP Health Monitor
‒ http://www.cisco.com/en/US/docs/ios/ipsla/configuration/guide/sla_lsp_mon_autodisc.html
Cisco ME3600X/ME3800X Ethernet OAM, CFM and E-LMI Configuration Guide
‒http://www.cisco.com/en/US/docs/switches/metro/me3600x_3800x/software/release/15.1_2_ey/configuration/guide/swoam.html
Cisco ME3400E CFM, Y.1731, E-LMI and Link OAM Configuration Guide
‒http://www.cisco.com/en/US/docs/switches/metro/me3400e/software/release/12.2_58_se/configuration/guide/swoam.html
Cisco ME3400 CFM, Y.1731, E-LMI and Link OAM Configuration Guides
‒http://www.cisco.com/en/US/docs/switches/metro/me3400/software/release/12.2_58_se/configuration/guide/swoam.html
Cisco Catalyst 3750-ME CFM, E-LMI and Link OAM Configuration Guides
‒http://www.cisco.com/en/US/docs/switches/metro/catalyst3750m/software/release/12.2_58_se/configuration/guide/swoam.html
95
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
E-OAM Documentation at Cisco.com
Cisco Catalyst 4500 CFM and OAM Configuration Guide
‒ http://www.cisco.com/en/US/docs/switches/lan/catalyst4500/12.2/3.1.1SG/configuration/guide/CFM.html
‒ http://www.cisco.com/en/US/docs/switches/lan/catalyst4500/12.2/15.02SG/configuration/guide/E_OAM.html
Cisco Catalyst 4500 Y.1731 AIS and RDI Configuration Guide
‒ http://www.cisco.com/en/US/docs/switches/lan/catalyst4500/12.2/3.1.1SG/configuration/guide/Y1731.html
‒ http://www.cisco.com/en/US/docs/switches/lan/catalyst4500/12.2/15.02SG/configuration/guide/Y1731.html
Cisco ONS 15454 ML-MR-10 CFM, E-LMI and Link OAM Configuration Guide
‒ http://www.cisco.com/en/US/docs/optical/15000r9_0/ethernet/454/guide/45490a_eoamonmlmr.html
EoMPLS Remote Link Failure Notification via E-LMI (aka Remote Port Shutdown)
‒ http://www.cisco.com/en/US/docs/ios/12_2sr/12_2srb/feature/guide/srbrpsdn.html
Cisco IOS Carrier Ethernet Command Reference
‒ http://www.cisco.com/en/US/docs/ios/cether/command/reference/ce_book.html
Cisco ASR 9000 Ethernet OAM Configuration Guide
‒http://www.cisco.com/en/US/docs/routers/asr9000/software/asr9k_r4.1/interfaces/configuration/guide/hc41eoam.html
Cisco ASR 9000 Ethernet OAM Command Reference
‒http://www.cisco.com/en/US/docs/routers/asr9000/software/asr9k_r4.1/interfaces/command/reference/b_interfaces_cr41asr9k_chapter_0101.html
96
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
E-OAM Documentation at Cisco.com
Cisco CRS-1 Ethernet OAM Configuration Guide
‒ http://www.cisco.com/en/US/docs/routers/crs/software/crs_r4.1/interfaces/configuration/guide/hc41eoam.html
Cisco CRS-1 Ethernet OAM Command Reference
‒http://www.cisco.com/en/US/docs/routers/crs/software/crs_r4.1/interfaces/command/reference/b_interfaces_cr41crs_chapter_0110.html
Cisco XR 12000 Ethernet OAM Configuration Guide
‒http://www.cisco.com/en/US/docs/routers/xr12000/software/xr12k_r4.1/interfaces/configuration/guide/hc41eoam.html
Cisco XR 12000 Ethernet OAM Command Reference
‒http://www.cisco.com/en/US/docs/routers/xr12000/software/xr12k_r4.1/interfaces/command/reference/b_interfaces_cr41xr12k_chapter_0101.html
Cisco ASR 1000 CFM Configuration Guide
‒ http://www.cisco.com/en/US/docs/ios-xml/ios/cether/configuration/xe-3s/ce-cfm-ieee-xe.html
‒ http://www.cisco.com/en/US/docs/ios-xml/ios/cether/configuration/xe-3s/ce-y1731-perfmon.html
Cisco ASR 901 Ethernet OAM Configuration Guide
‒ http://www.cisco.com/en/US/docs/wireless/asr_901/Configuration/Guide/oam.html
Cisco Active Network Abstraction (ANA) Guide and Reference
‒ http://www.cisco.com/en/US/products/ps6776/products_user_guide_list.html
97
CFM Appendix
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Cisco IOS CFM Implementation Example 6: CFM on VPLSL2 Virtual Forwarding Instance (VFI)
interface vlan 10 description SVI interface xconnect vfi foo-core
ethernet cfm ieee ethernet cfm global ! ethernet cfm domain Domain_L3 level 3 service sampleMA evc sampleEVCvlan 10 direction down continuity-check continuity-check interval 1s ! ethernet evc sampleEVC ! pseudowire-class HubClass encapsulation mpls ! l2 vfi foo-core manual sampleEVC vpn id 100 bridge-domain 10 vlan neighbor 11.0.0.1 pw-class HubClass neighbor 11.0.0.2 pw-class HubClass cfm mep domain Domain_L3 mpid 1100 ! ethernet cfm logging alarm cisco ethernet cfm logging alarm ieee
Global Configuration Interface Configuration
Bridge
Physical
Interface
Full Mesh
Pseudowires
VPLS
Forwarder
Emulated
LAN Interface
L2 VFI
Spoke
Pseudowires CFM Down MEP
7600
For your reference
99
CFM and Y.1731 Comparison Appendix
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
ITU-T Y.1731 vs IEEE CFM OAM Functions for Fault Management
Feature ITU-T Y.1731
IEEE CFM
Comments
Ethernet Continuity Check
ETH-CC CC
Proactive fault detection, fault notification
No compatibility between IEEE 802.1ag-2007 (CFM) and ITU-T Y.1731-2008 messages due to different formats for shortMA / MEG ID fields. Overall, IEEE CFM offers more flexibility/choices concerning formats for MD and ShortMA/MEG ID fields
IEEE CFM defines:
MD name format: NULL, MAC address, DNS or string-based
ShortMA name format: Vlan-ID, VPN-ID, integer or string-based
ITU-T Y.1731-2008 defines:
MD name format: NULL only
MEG-ID name format: ICC-based only
Compatibility between 802.1ag-2007 and Y.1731-2010 Amendment.
Y.1731 extends MEG-ID name formats to includes those in IEEE CFM.
ICC-based MEG ID continues to be an ITU-only format
IEEE CFM allows optional TLVs in CCM to improve fault notification for UP MEPs (i.e. Port Status and Interface Status TLVs). IEEE CFM also allows Sender ID TLVs and Organization-Specific TLVs for vendor extensions. No optional TLVs in Y.1731 ETH-CC
IEEE CFM defines multicast Destination Address for CCM, ITU-T defines both multicast and unicast DA CCM. Most fault management scenarios relay on multicast DA
101
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
ITU-T Y.1731 vs IEEE CFM OAM Functions for Fault Management (cont.)
Feature ITU-T Y.1731
IEEE CFM
Comments
Ethernet Remote Defect Indication
ETH-RDI RDI
Fault propagation in the backward direction (opposite direction of the fault).
IEEE RDI flag and Y.1731 ETH-RDI are in the same position in CCM header. However, formats maynot be compatible between IEEE CFM and Y.1731 (see CCM section)
Ethernet Link Trace ETH-LT LT
On-demand path discovery / fault isolation
Compatible operations between IEEE and ITU
However, Y.1731 has no concept of MIP CCM database.
Thus linktrace operation would always fail in situations
where active CAM table has aged out. IEEE CFM on the
other hand, has the option to rely on MIP CCM database
(slower aging than CAM table) when determining the
egress port for an LTM
In addition, IEEE CFM LTRPDUs contain more information
than Y.1731 LTR ones; e.g. interface names for ingress and
egress interfaces. IEEE CFM LTR can also contain a
Sender ID TLV (not defined in Y.1731) to carry hostname of
the responder
102
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
ITU-T Y.1731 vs IEEE CFM OAM Functions for Fault Management (cont.)
Feature ITU-T Y.1731
IEEE CFM
Comments
Ethernet Loopback ETH-LB LB
On-demand service / fault verification
Compatible operations between IEEE and ITU
IEEE CFM defines unicast DA LBM, ITU defines both multicast and unicast DA LBM. For compatibility, IEEE CFM allows replies to incoming multicast LBMs
IEEE CFM defines optional Data TLV (arbitrary data pattern) in LBM as well as Organization-Specific TLV for vendor extensions
ITU Y.1731 allows optional Data TLV and Test TLV (pre-defined data pattern) in LBM
In fact, Test TLV is the only TLV defined exclusively by Y.1731. A subset of IEEE CFM TLVs are used in Y.1731
Ethernet Alarm Indication Signal
ETH-AIS --
Fault propagation in the forward direction (same direction of the fault)
Also used to suppress Loss of Continuity alarms following detection of defect conditions
Ethernet Locked Signal
ETH-LCK -- Communicates administrative locking of a MEP
Allows intrusive OAM loopback tests (e.g. during turn up)
Ethernet Test Signal ETH-Test -- One-way on-demand diagnostics
103
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
ITU-T Y.1731 Overview OAM Functions for Fault Management (cont.)
Feature ITU-T Y.1731
IEEE CFM
Comments
Ethernet Automatic Protection Switching
ETH-APS -- Applications for this PDU are defined in ITU-T G.8031 and G.8032 (Ethernet Linear and Ring Protection Switching)
Ethernet Maintenance Communication Channel
ETH-MCC -- No application / use defined in Y.1731 for this PDU
Ethernet Experimental OAM
ETH-EXP -- No application / use defined in Y.1731 for this PDU
Ethernet Vendor-specific OAM
ETH-VSP --
No application / use defined in Y.1731 for this PDU
Y.1731 allows vendor extensions of new PDUs
IEEE CFM allows vendor extensions to standard PDUs using Organization-Specific TLVs
Ethernet Client Signal Fail
ETH-CSF -- Applications for this PDU are defined in ITU-T G.806
PDU added with ITU-T Y.1731 (2008) Amend.1
Manageability (SNMP MIB)
No Yes
IEEE 802.1ap includes MIB modules for CFM (IEEE8021-CFM and IEEE8021-CFM-V2MIB modules)
No MIB definitions for Y.1731 from ITU-T
104
Cisco IOS-Embedded Event Manager (EEM) and Service Diagnostics
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Embedded Event Manager What and Why?
Onboard automation
Automate troubleshooting
Automate commands
Automatic actions based on
real-time events—improved
availability
Automatic data collection
for capacity planning or
performance analysis
Components:
‒ Event Detectors
‒ EEM Server
‒ Policies (scripts)
106
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Service Diagnostics Introduction
Tcl-based embedded automation scripts using EEM
Automated troubleshooting for Ethernet services using CFM (IEEE 802.1ag)
Automated interface state actions based on CFM failure detection
Developed and supported by Cisco
Fully tested on ME-3400 and ME-3400E platforms (IOS 12.2(52)SE)
Features user-friendly menu driven installation and configuration
Scripts and documentation available at: http://www.cisco.com/go/iossd
Auto Traceroute
Automatic Diagnostics
ON-Demand Diagnostics
Shutdown Port
Unshut Port
107
© 2012 Cisco and/or its affiliates. All rights reserved. BRKNMS-2202 Cisco Public
Deploying Carrier Ethernet OAM Ethernet Layer 2 VPN Services
EoMPLS Pseudowire Access Access Aggregation Aggregation CE CE
User at NOC or onsite launches On-Demand script to assess condition of a given service
Service Diagnostics – ON-Demand
Sample Syslog Notification (SNMP and email also available)
108