dubai partner bootcamp part1 without password protection

© Cisco Systems 2007 CISCO PARTNER CONFIDENTIAL

Course Objective Focus of this course

COURSE OBJECTIVE: Introduce the student to…

: the Catalyst 6500 Hardware Platform: the Hardware Accelerated features available on the Catalyst 6500: the Catalyst 6500 Hardware Architecture: the Catalyst 6500 IOS Software and its Architecture: a feature deep dive: a discussion on 6500 roadmap

COURSE OBJECTIVE: Introduce the student to…

: the Catalyst 6500 Hardware Platform: the Hardware Accelerated features available on the Catalyst 6500: the Catalyst 6500 Hardware Architecture: the Catalyst 6500 IOS Software and its Architecture: a feature deep dive: a discussion on 6500 roadmap

The course is a “101” course and makes the presumption that the student knows very little about the Catalyst 6500.


Course Objective Focus of this course

HARDWARE6500 Chassis, Power Supplies, Fan

Tray, 6500 Architecture & Backplane, Supervisor 720 and 32, MSFC3,

PFC3x, Ethernet Linecards, Hardware Accelerated Features

SOFTWARE12.2SX IOS Train, IOS Architecture, CLI Structure, Software Modularity,

Key Software Features

WHAT WILL BE COVERED WHAT WILL NOT BE COVERED

HARDWAREOptical Switch Modules, FlexWAN,

7600 Chassis, Supervisor 1A, Supervisor 2

SOFTWARE12.2SR (Cascades) IOS Train,Catalyst OS (CatOS), MPLS

Agenda – Catalyst 6500 Bootcamp


• Chapter 1: Course Objective• Chapter 2: Introduction• Chapter 3: Chassis, Fan Tray, Power Supplies• Chapter 4: Supervisors• Chapter 5: Line Cards• Chapter 6: Switching Basics and Internals• Chapter 7: Packet Walks• Chapter 8: IPv6• Chapter 9: VRF/MPLS• Chapter 10: Control Plane Features• Chapter 11: Netflow• Chapter 12: Security• Chapter 13: QoS• Chapter 14: GRE• Chapter 15: High Availability• Chapter 16: IOS Architecture• Chapter 17: IOS Software Modularity• Chapter 18: Embedded Event Manager• Chapter 19: Other Switch Features• Chapter 20: Summary


CHAPTER 2Introduction To The Catalyst 6500


Catalyst Switching Portfoliofor Enterprises (Small, Medium, Large)

Catalyst 29xx

Catalyst 3750Catalyst 3560

Catalyst 4500Catalyst 6500

Catalyst Express 500

Catalyst 4500

Catalyst 6500

Catalyst 4948

Catalyst 6500

Small Medium-sized Large

Employee Size/Density

Feat

ures

, Sca

labi

lity,

Lon

gevi

ty

Wiring Closet

Datacenter Access

Distribution/Core

Blade Switches


The Catalyst 6500 was announced in 1999 and is targeted to have a continued lifecycle for at least another 7 years from 2005 …

Expected Catalyst 6500 lifecycle

20121999 2004

Catalyst6500

Announced

Supervisor 2SFM

Sup72067xx Linecards

IPv6

DistributedForwarding

ServicesModules

Sup720-3BXLSup720-3B

Enabling MPLSSup32

2002

Higher Density 10-GEApplication FluencyVirtualizationNetwork Admission ControlHardware-based NBARAnd more….

Catalyst 6500Long-Term Investment Platform

2007 2010

Catalyst 6500 The Leading Platform for Advanced Technologies


Wireless Module

IP Security Module

Voice & Video

Intrusion Detection & Prevention ModuleApplication

Oriented Networking

Telepresence

Application Control Module

High Density PoE

Catalyst 6500 FamilyNew “E” Series Chassis


New E Series chassis designed to support much higher loads of power across the backplane - ideally suited for large deployments of inline powered devices…

Support up to 12000W of power

Chassis is S/W transparent

Same Pricing as existing chassis

New Catalyst 6500 “E” Series Chassis BenefitsIncreases inline

device count

6504-E6503-E

6509-E

6506-E

Ready for 80G / slot


Supervisor 720Core, Distribution & Data Center

Supervisor 32Access & WAN Edge

Hardware Accelerated Services and Forwarding Across Both Supervisors:IPv4 IPv6 QoS MPLS Port ACL's NAT GRE Multicast

Data Center WAN Edge Core Distribution Access

Security• Catalyst Security Toolkit• Identity Based Networking (IBNS)• Network Admission Control (NAC)• Control Plane Policing• Svc Modules (FW, IDS, AD, VPN)• HW-based L2 MAC Learning

Multicast

• Bi-Directional PIM• IGMP Querier• RGMP, MBGP• PIM Snooping• IGMP v3 and SSM• MSDP

High Availability & Quality• Software Modularity • Soft HA• GOLD • L2 / L3 Sub-Second Switchover• Non-Stop Forwarding• Safe Harbor and FTL

Catalyst 6500 SupervisorsEnable Consistent Features Across Your Entire Network

End-to-End feature consistency & IOS Software Modularity!


System Architectures

Layer 3 – Network Switching Architecture - Flow based vs Topology based

Flow-based Switching(Competitor)

Control Plane

Data Plane

CPU

LineCard

1st Packet2nd Packet

3rd Packet

4th Packet

LineCardFlow

Cache

Network

Topology-based (Cisco Express Forwarding)

Control Plane

Data Plane

MSFC2

PFC2FIB TABLE

1st Packet

2nd Packet

3rd Packet

4th Packet

Network

1st packet of every “NEW” flow is handled by the CPU ( Slow path)In a dynamic environment, the actual performance of switch is limited by

the capacity of the Switch CPU. Important during route/network flaps or new flows being learnt


Catalyst 6500 PFC3: Securing the NetworkProtection Against DoS Attacks

Control Plane Rate Limiting– Hardware Based Control Plane

ProtectionWhat is it?Protects switch (supervisor) performance from

potentially crippling Denial of Service attacks

• Supports more than 20 Different Conditions• User configurable rate limit – all in hardware• Additional capabilities supplemented by s/w

based Control Plane Policing

Example: SQL and Slammer mitigated by this feature

MSFC

PFC

Linecard Linecard

RP

Control PlaneProtection

Hacker


Catalyst 6500 Access Layer Security Enhancements Comprehensive Edge Security

• Industry’s leading LAN Switching Security portfolio -802.1x, Catalyst Integrated Security Toolkit

• A combination of authentication, access control, and user policies to secure network connectivity and resources

• Greater flexibility and mobility for a stratified user community

• Reduced OPEX

• DHCP Snooping• Dynamic ARP Inspection• IP Source Guard• SSHv2 support • SCP (with SSHv1)• Comprehensive 802.1x enhancements

802.1X with VLANs Assignment802.1X with Port Security802.1X with VVID (IP Telephony)802.1X Guest VLANs802.1X with DHCP Snooping802.1x and QoS802.1x with Wake on LAN802.1x Accounting Enhancements802.1x - Authenticated Identity to Port

Description Mapping802.1x - One-to-Many logical VLAN name

to ID mapping802.1x – DNS Resolution for RADIUS Server802.1x and ACL / VACL propagation

Secure Mobility & Workforce Optimization Enhanced Productivity

New


Cisco Catalyst 6500 High Availability LeadershipMaximizing Uptime

Physical Redundancy• Redundant supervisors, power supplies,

switch fabrics, and clocks

Non-Stop Forwarding / Stateful Switch Over (NSF/SSO)• Traffic continues flowing after a

primary supervisor failure• Sub-second recovery in

L2 and L3 networks• No line card reset

Generic Online Diagnostics(GOLD)• Proactively detect and address

potential hardware and software faults in the switch before they adversely impact network traffic

Catalyst 6500

Cisco IOS Software Modularity• Subsystem In-Service Software Upgrades (ISSU)• Stateful Process Restarts• Fault Containment, Memory Protection

New!


CHAPTER 3Chassis, Fan Tray, Power Supply

Chassis, Fan, Power Supply The Chassis



Chassis, Fan, Power Supply Chassis Architecture - 6503/6503-E

Power Supply

Fan

Tray

Power Supply

SLOT 1

SLOT 2

SLOT 3

Clock Clock EEPROM

Dual Channels

Dual Channels

Dual Channels

Switch Fabric Shared Bus

SupervisorSlots

The three slot chassis supports a Supervisor in either Slot 1 or 2 - Power supplies are installed into the rear of the chassis - a power entry module (PEM) is used at the front of the chassis to provide an interface from the power cable to the power supplies at the back of the chassis…

NOTE: A Different PEM is used for the 1400W Power Supply to the 950W Power Supply


Chassis, Fan, Power Supply Chassis Architecture - 6504-E

Power Supply

Fan

Tray

Power Supply

SLOT 1

SLOT 2

SLOT 3

Clock Clock EEPROM

Dual Channels

Dual Channels

Dual Channels


The four slot chassis supports a Supervisor in either Slot 1 or 2 - Power supplies are installed into the rear of the chassis…

SupervisorSlots

SLOT 4 Dual Channels



Power Supply

Fan

Tray

Power Supply

SLOT 1SLOT 2SLOT 3

Clock Clock EEPROM

Dual ChannelsDual ChannelsDual Channels


SLOT 4 Dual ChannelsSLOT 5 Dual ChannelsSLOT 6 Dual Channels

SupervisorSlots

The six slot chassis supports a Supervisor in either Slot 5 or 6 - Power supplies are installed in the bottom of the chassis…



Power Supply

Fan

Tray

Power Supply

SLOT 1SLOT 2SLOT 3

Clock Clock EEPROM

Dual ChannelsDual ChannelsDual Channels




SupervisorSlots

The nine slot chassis supports a Supervisor in either Slot 5 or 6 - Power supplies are installed in the bottom of the chassis…


Chassis, Fan, Power Supply Chassis Architecture - 6509-NEB-A

Power Supply Power Supply

SLOT

1

Clock Clock EEPROM

Fan Tray Fan Tray

SLOT

2

SLOT

3

SLOT

4

SLOT

5

SLOT

6

SLOT

7

SLOT

8

SLOT

9

Dual

Chan

nels

Dual

Chan

nels

Dual

Chan

nels

Dual

Chan

nels

Dual

Chan

nels

Dual

Chan

nels

Dual

Chan

nels

Dual

Chan

nels

Dual

Chan

nels

Switc

h Fa

bric

Shar

ed B

us

The nine slot vertical chassis supports a Supervisor in either Slot 5 or 6

Power supplies are installed in the bottom of the chassis

This chassis incorporates redundant fan trays


Chassis, Fan, Power Supply Chassis Architecture - 6513

Power SupplyFan

Tray

Power Supply

SLOT 1SLOT 2SLOT 3

Clock Clock EEPROM

Single ChannelSingle ChannelSingle Channel


SLOT 4 Single ChannelSLOT 5 Single ChannelSLOT 6 Single ChannelSLOT 7 Single ChannelSLOT 8 Single ChannelSLOT 9 Dual ChannelsSLOT 10 Dual ChannelsSLOT 11 Dual ChannelsSLOT 12 Dual Channels

Dual ChannelsSLOT 13

Power SupplyFan

Tray

Clock Clock

The thirteen slot chassis supports a Supervisor in either Slot 7 or 8

Power supplies are installed in the bottom of the chassis

Slots 1-8 have a single fabric channel

Slots 9-13 have dual fabric channels


Chassis, Fan, Power Supply The Fan Tray

Fan Trays are used to draw cool air into chassis to cool components

- Typically mounted vertically down left side of chassis (6509-NEB-A is exception)- Right to Left airflow- Three Fan Tray Types

Standard Fan TrayUsed in base Catalyst 6500 Chassis

High Speed Fan (Fan2)Used in base Catalyst 6500 Chassis with Supervisor 720 and Supervisor 32

E Series FANUse in Catalyst 6500 E series Chassis

Fan Tray Types ARE NOT interchangeable outside of their supported chassis types

Chassis, Fan, Power Supply Fan2


Fan Tray 2 is a new HIGH SPEED fan for non “E” series chassis…

Fan2 is designated for use in the following chassis

- Catalyst 6503- Catalyst 6506- Catalyst 6509- Catalyst 6513

Fan designation in chassis is noted on the FAN

Mandatory for use in chassis running a Supervisor 720 or Supervisor 32

Also mandates a minimum power supply of 2500W or higher


Chassis, Fan, Power Supply E-Fan

E-Fan Tray is standard HIGH SPEED fan for “E” series chassis…

Designated for use in the following chassis- Catalyst 6503-E- Catalyst 6504-E- Catalyst 6506-E- Catalyst 6509-E

WILL NOT WORK in a NON E Series ChassisWILL NOT WORK in a NON E Series Chassis


Chassis, Fan, Power Supply Power Supplies

Power Supplies for six, nine and thirteen slot chassis are located at front bottom of chassis

Power Supplies for three and four slot chassis are located in the rear

AC Power SuppliesAC Power Supplies DC Power SuppliesDC Power Supplies950W

1000W1300W1400W2500W2700W3000W4000W6000W8700W

950W1000W1300W1400W2500W2700W3000W4000W6000W8700W

950W1300W2500W2700W4000W6000W

950W1300W2500W2700W4000W6000W

******

**

* future product ** Now EOS*

*


Chassis, Fan, Power Supply 6000W Power Supply

1 x 110V Input = No Power2 x 110V Input = 2900W1 x 220V Input = 2900W2 x 220V Input = 6000W

1 x 110V Input = No Power2 x 110V Input = 2900W1 x 220V Input = 2900W2 x 220V Input = 6000W

6000 Watt Power Supply (WS-CAC-6000W)

Currently there is only an AC version of the 6000W Power Supply

All inputs to the power supply are single phase

Each input supports a max of 17.6A

Max chassis heat dissipation ~ 24K BTU/hr

Supported in the six, nine and thirteen slot chassis

In non E series 6506 and 6509, only 4000W can be obtained from this power supply (due to backplane limitation)


Chassis, Fan, Power Supply 8700W Power Supply

1 x 110V Input = No Power2 x 110V Input = 2800W3 x 110V Input = 4200W1 x 220V Input = 2800W2 x 220V Input = 5800W3 x 220V Input = 8700W

1 x 110V Input = No Power2 x 110V Input = 2800W3 x 110V Input = 4200W1 x 220V Input = 2800W2 x 220V Input = 5800W3 x 220V Input = 8700W

8700 Watt Power Supply (WS-CAC-8700W)Expected to FCS towards the end of CQ2 2007

There will only be an AC version of the 8700W Power Supply

All inputs to the power supply are single phase

Each input supports a max of 17.6A (3 inputs)

Max chassis heat dissipation ~ 37K BTU/hr

Supported in the six, nine and thirteen slot chassis

In non E series 6506 and 6509, only 4000W can be obtained from this power supply (due to backplane limitation)

Chassis, Fan, Power Supply Understanding Power Redundancy


The 6500 can utilize two power supplies to work in either combined or redundant mode

Redundant Mode Combined Mode

In redundant mode, each power supply operates at 50% capacity and provides the same total power as a single power supply

– if one fails, the backup reverts to providing 100% power

In combined mode, each power supply operates at 83% - if one fails, then the

running supply provides 100% of its power capacity

Power Supply 1 Power Supply 2

50% 50%

Switch

Power Supply 1 Power Supply 2

83% 83%

Switch


When booting with unequal power supplies in redundant mode, the following occurs

Switch Switch

1. System log and SYSLOG messages generated2. System does not allow different sized power supplies to be booted in redundant mode3. Smaller Power Supply is shut down4. Only the larger power supply is booted – this is run at 100% capacity

PowerSupplyX Watts

Redundant Mode Redundant Mode

100%PowerSupplyY Watts

Chassis, Fan, Power Supply Booting with Unequal PSU in Redundant configuration

Chassis, Fan, Power Supply Booting with Unequal PSU in Combined configuration


When booting with unequal power supplies in combined mode, the following occurs

Switch Switch

1. System log and SYSLOG messages generated2. System allows different sized power supplies to be booted in combined mode3. Both Supplies run up at 167%

PowerSupplyX Watts

Redundant Mode Combined Mode

PowerSupplyY Watts

83% 83%


Chassis, Fan, Power Supply Power Management

All Supervisors and Linecards have power value preprogrammed in EPROM - this is used to identify how much power reserved in chassis…

Use the Power Calculator on CCO to determine the power supply and minimum power requirements - http://www.cisco.com/go/powercalculator

If insufficient power available, system powers down Power Devices, then switching modules, then services modulesPowered Devices and modules powered off from highest numbered to lowest numbered (port or slot)


Chassis, Fan, Power Supply CPC - Cisco.com Power Calculator

http://www.cisco.com/go/powercalculatorhttp://www.cisco.com/go/powercalculator

Chassis, Fan, Power Supply CPC - Cisco.com Power Calculator



CHAPTER 4Catalyst 6500 Supervisors

Agenda


• Supervisor Engines• Multi Layer Switching Card – MSFC• Policy Feature Card – PFC• Performance

Catalyst 6500 SupervisorsSupervisor 720


The Supervisor 720 is designed for deployment in the Core and Distribution Layers of the Network - it is the highest performing Supervisor option available for the Catalyst 6500 platform…


Catalyst 6500 SupervisorsSupervisor 720 - Some Facts

Supervisor 720 Quick FactsSupervisor 720 Quick FactsThree Models are available- Supervisor 720- Supervisor 720-3B- Supervisor 720-3BXL

Other Quick Facts- Integrated 720Gb Switch Fabric- Integrated Policy Feature Card 3 supporting hardware acceleration for select features- Integrated Multilayer Switch Feature Card 3 supporting two CPU’s for Layer 2 and Layer 3 functionality- Two external compact flash slots- Two Uplink Ports

- Port 1: SFP- Port 2: SFP or GE-TX *

- Console Port

Three Models are available- Supervisor 720- Supervisor 720-3B- Supervisor 720-3BXL

Other Quick Facts- Integrated 720Gb Switch Fabric- Integrated Policy Feature Card 3 supporting hardware acceleration for select features- Integrated Multilayer Switch Feature Card 3 supporting two CPU’s for Layer 2 and Layer 3 functionality- Two external compact flash slots- Two Uplink Ports

- Port 1: SFP- Port 2: SFP or GE-TX *

- Console Port* The user tells the switch which media type to use

Catalyst 6500 SupervisorsSupervisor 720 - The Options


Supervisor 720-3B - Integrated MSFC3 and PFC3B which incorporates some new hardware features like MPLS VPN, EoMPLS, ACL Counters, 4K ACL’s, TCAM utilization improvements and more…

Original Supervisor 720 announced in March 2003 -Added HW support for IPV6, GRE, NAT, PAT, Egress Policing and a host of other accelerated features…

Supervisor 720-3BXL - Same hardware features as the Supervisor 720-3B with the additional support of up to 1 Million IPV4 routes and support for up to 256K Netflow table entries


Catalyst 6500 SupervisorsSupervisor 720 - The Elements

All Supervisor 720 models incorporates three main elements - the Multilayer Switch Feature Card 3 (MSFC3), the Policy Feature Card 3 (PFC3) and the Switch Fabric - each of which is highlighted below…

Switch FabricSwitch Fabric

PFC3PFC3

MSFC3MSFC3


Catalyst 6500 Supervisors Supervisor 32

The Supervisor 32 is designed for deployment in the Access Layer - there are two Supervisor 32 options, one with 8 GE SFP ports and the other with 2 10GE ports on the front panel…

Sup32-8GESup32-8GE

Sup32-10GESup32-10GE


Catalyst 6500 Supervisors Supervisor 32 - Some facts

Supervisor 32 Quick FactsSupervisor 32 Quick Facts

Two Models are available- Supervisor 32-8GE- Supervisor 32-10GE

Other Quick Facts- NO Switch Fabric- Integrated Policy Feature Card 3 (PFC3B) supporting hardware acceleration for select features- Integrated Multilayer Switch Feature Card 2a(MSFC2a) supporting two CPU’s for Layer 2 and Layer 3 functionality- One external compact flash slot- Two 10GE ports OR Eight GE SFP port- One 10/100/1000 port- Console Port- USB Ports

Two Models are available- Supervisor 32-8GE- Supervisor 32-10GE

Other Quick Facts- NO Switch Fabric- Integrated Policy Feature Card 3 (PFC3B) supporting hardware acceleration for select features- Integrated Multilayer Switch Feature Card 2a(MSFC2a) supporting two CPU’s for Layer 2 and Layer 3 functionality- One external compact flash slot- Two 10GE ports OR Eight GE SFP port- One 10/100/1000 port- Console Port- USB Ports

Catalyst 6500 Supervisors Supervisor 32 - The Elements


Both Supervisor 32 models incorporates a Multilayer Switch Feature Card 2a (MSFC2a) and the Policy Feature Card 3 (PFC3) - each of which is highlighted below…

PFC3PFC3

MSFC2aMSFC2a


Catalyst 6500 Supervisors What Supervisors Should I Be Selling?If we take the typical Campus Design, then the following Supervisors are suggested for each layer…

CORECORE

DISTRIBUTIONDISTRIBUTION

ACCESSACCESS

Supervisor 720-3B, Supervisor 720-3BXL

Supervisor 720-3B, Supervisor 720-3BXL

Supervisor 720-3B, Supervisor 720-3BXL, Supervisor 32-8GE, Supervisor32-10GE


Catalyst 6500 Supervisors MSFC and PFC in more detail

Lets look at the MSFC and PFC in more detail - what they are and what they do…

Multilayer Switch Feature Card (MSFC)Multilayer Switch Feature Card (MSFC) Policy Feature Feature Card (PFC)Policy Feature Feature Card (PFC)

Provides Control Plane FunctionalityProvides Control Plane Functionality Provides Data Plane FunctionalityProvides Data Plane Functionality

So what is the Control Plane and the Data Plane???So what is the Control Plane and the Data Plane???


Catalyst 6500 Supervisors Control and Data Plane

There are two parts to the processing functionality of the Switch - the Control Plane and the Data Plane…

SWITCH CONTROL PLANESWITCH CONTROL PLANE

SWITCH DATA PLANESWITCH DATA PLANE

Switch Features that are processed in SOFTWAREby a CPU…

There are two control planes - the Switch Processor Control Plane (SP) which processes software based Layer 2 features - the Route Processor Control Plane (RP) which processes software based Layer 3 features

Switch Features that are processed in SOFTWAREby a CPU…

There are two control planes - the Switch Processor Control Plane (SP) which processes software based Layer 2 features - the Route Processor Control Plane (RP) which processes software based Layer 3 features

Switch Features that are processed in HARDWARE by Application Specific Integrated Circuits (ASICs)

Feature performance usually ROCKS!!!!

Switch Features that are processed in HARDWARE by Application Specific Integrated Circuits (ASICs)

Feature performance usually ROCKS!!!!


Catalyst 6500 Supervisors MSFC3 Introduction

MSFC3MSFC3


Catalyst 6500 Supervisors MSFC3 Components

Route ProcessorBootflash 64MB

Switch ProcessorBootflash 64MB

Switch Processor

Route Processor

Switch Processor DRAM512MB

Route Processor DRAM512MB


Catalyst 6500 Supervisors MSFC3 - Switch Processor

MSFC3

RouteProcessor

(RP)

RouteProcessor

(RP)

SwitchProcessor

(SP)

SwitchProcessor

(SP)

Both the RP and SP perform distinct functions during both the booting of the operating system and the ongoing operation of the switch…

- The SP owns the switch at initial boot up before handing over to the RP

- The SP runs all layer 2 operations like VTP, Spanning Tree, Chassis and Power Management, etc

- Supports other Layer 2 features like CDP, SPAN, Broadcast Suppression, Etherchannel, etc

The Switch Processor is physically located on the MSFC3

Logically considered as the Network Management Processor (NMP)

Dedicated CPU, DRAM and Flash


Catalyst 6500 Supervisors MSFC3 - Route Processor

MSFC3

Both the RP and SP perform distinct functions during both the booting of the operating system and the ongoing operation of the switch…

The Route Processor is physically located on the MSFC

Logically considered as the MSFC

Dedicated CPU, DRAM and Flash

- The RP runs the Layer 3 routing protocols like OSPF, EIGRP, BGP, etc

- Other layer 3 features like IPX and Appletalk

- Manages the user interface (CLI)- All show and configuration commands are

processed on the RP then sent to the SP for execution

RouteProcessor

(RP)

RouteProcessor

(RP)

SwitchProcessor

(SP)

SwitchProcessor

(SP)


Catalyst 6500 Supervisors MSFC3 - Bootflash

MSFC3

RPSP

RP BOOTFLASHSP BOOTFLASH

The RP and the SP both have their own set of Bootflash…SP Bootflash is used to store the boot image and is referred to as SUP-BOOTFLASH during normal operationRP Bootflash is referred to as BOOTFLASH during normal operation

6500# dir sup-bootflash:Directory of sup-bootflash:/

1 -rwx 78958736 May 4 2006 20:07:18 +00:00 s72033-advipservicesk9_wan-vz.122-18.SXF4.bin

512040960 bytes total(433082224 bytes free)


Catalyst 6500 Supervisors MSFC3 - RP and SP DRAM

MSFC3RPSP

RP DRAMSP DRAM

The RP and the SP both have their own set of DRAM…RP/SP DRAM (each is 512MB) is used to store the running configuration, the running IOS image, the routing table, etc – the amount of SP and RP DRAM available can be seen using the following commands…

6500# show version<snip>cisco Catalyst 6000 (R7000) processor with 458752K/65536K bytes of memory.

6500# remote command switch show version<snip>cisco Catalyst 6000 (R7000) processor with 458752K/65536K bytes of memory.


Catalyst 6500 SupervisorsMSFC3 New Bootflash Option

COMPACT FLASH ADAPTER CF Adapter Specifications

• Upgrade P/N: WS-CF-UPG=• Upgrade contains one CF adapter with

a single 512MB CF

• Supported on SP of Sup720(PFC3A and later)

• Requires SP ROMMON 8.4(2)(ROMMON image is field upgradeable)

• Requires >= IOS 12.2(18)SXE5

• Supervisor must be removed to install the adapter

• Allows scaling flash >=512MB

A new Compact Flash Adapter that can take the place of existing Bootflash on the Supervisor to alleviate current Bootflash size limitations…

Note: once the CF adapter is installed the SP bootflash is referred to as sup-bootdisk:


Catalyst 6500 Supervisors MSFC3 New Bootflash Option

MSFC3 SP and RP Bootflash Slots with standard Bootflash

installed…

MSFC3 SP Bootflash Slot with CF

Bootflash Adapter installed…

Catalyst 6500 Supervisors PFC3 Introduction



Catalyst 6500 Supervisors PFC3 Introduction

PFC3x Hardware Features SummaryIPV4 and IPV6 CEF SwitchingIPV6 TunnelingIPV4 NAT/PAT in HardwareGRE and IP in IP Tunnels (HW)WCCP V2 enabled

From 256K-1M IPV4 RoutesIngress/Egress PolicingUser Based Rate LimitingHardware based ClassificationBi-Directional PIM

From 128K-500K IPV6 RoutesPort Access Control ListsMulti-path URPF4K ACL Labels (3B/3BXL only)ACL Counters (3B/3BXL only)

PFC3PFC3PFC3A - Base PFC

PFC3B - Adds MPLS, 4K ACL Labels, ACL Counters, improved hash algorithm for Netflow entries

PFC3BXL - Upgrades FIB to 1M entries and Netflow table to 256K Entries


Catalyst 6500 Supervisors PFC3 Components

FIBTCAM

AdjacencyTable

QoSACL

SecurityACL

NetflowTCAM

NetflowTable

NetflowStatistics

Counters3B/3BXL

Layer 3Forwarding

Engine

Layer 2Forwarding

Engine

CAMTable

Contains IPV4, IPV6 prefixes and MPLS entries

Contains Layer 2 rewrite information

Contains QoS ACL entries (up to 32K)

Contains Security ACL entries (up to 32K)

Contains location of flow in Netflow Table

Contains key packet fields for flow

Collection of statistics for each active flow

Security ACL Counters

64K CAM containing MAC addresses

To DBUS To RBUS To EOBC


Catalyst 6500 Supervisors PFC3 Comparison

Feature PFC2 PFC3A PFC3B PFC3BXLFIB TCAM 256K

256K

128K (32K)128K (32K)SoftwareSoftware

NoNo

VRF Lite No Yes Yes Yes

NAT Software Hardware Hardware + UDP Hardware + UDP

Software

256K 256K 1MAdjacency Table

1M 1M 1M

Netflow Table 128K (64K) 128K (115K) 256K (230K)MAC Table 64K (32K) 64K (32K) 64K (32K)IPv6 128K 128K 500KBi-Dir PIM Hardware Hardware HardwareNative MPLS No Yes YesEoMPLS No Yes Yes

Tunnels Hardware Hardware + QoS Policies

Hardware + QoS Policies


Catalyst 6500 Supervisors FIB TCAM – what does it mean?

Feature PFC2 PFC3A PFC3B PFC3BXLFIB TCAM 256K

256K

128K (32K)128K (32K)SoftwareSoftware

NoNo

VRF Lite No Yes Yes Yes

NAT Software Hardware Hardware + UDP Hardware + UDP

Software

256K 256K 1MAdjacency Table

1M 1M 1M

Netflow Table 128K (64K) 128K (115K) 256K (230K)MAC Table 64K (32K) 64K (32K) 64K (32K)IPv6 128K 128K 500KBi-Dir PIM Hardware Hardware HardwareNative MPLS No Yes YesEoMPLS No Yes Yes

Tunnels Hardware Hardware + QoS Policies

Hardware + QoS Policies

PFC3A/B default FIB TCAM allocation:6500-3B#show mls cef maximum-routes FIB TCAM maximum routes :=======================Current :--------IPv4 + MPLS - 192k (default)IPv6 + IP Multicast - 32k (default)

PFC3BXL default FIB TCAM allocation:6500-3BXL#show mls cef maximum-routes FIB TCAM maximum routes :=======================Current :--------IPv4 + MPLS - 512k (default)IPv6 + IP Multicast - 256k (default)

IPv6 and IP Multicast entries take two entries:

PFC3A/B: 192K + 2* 32K = 256K

If PFC3A/B based system is acting as Internet peering device

change default TCAM allocation


Catalyst 6500 Supervisors PFC3 Comparison

Feature PFC2 PFC3A PFC3B PFC3BXLACL TCAM 32K/4K

NoNo

32K/4K51232No

NoNo

uRPF CheckYes - Singlepath Yes (Multipath) Yes (Multipath) Yes (Multipath)

32K/4K 32K/4K 32K/4KPACLs Yes Yes YesACE Counters No Yes YesQoS TCAM 32K/4K 32K/4K 32K/4KACL Labels 512 4K 4KACL LOUs 32 64 64

User-Based Policing

Yes Yes Yes

Egress Policing Yes Yes Yes

Unique MAC/Interface

Yes Yes Yes


Catalyst 6500 Supervisors Supervisor Engine 32 Architecture Paper

http://www.cisco.com/en/US/products/hw/switches/ps708/products_white_paper0900aecd803e508c.shtml

Provides an insight into the architecture of Supervisor Engine 32 on the Catalyst 6500

http://www.cisco.com/en/US/products/hw/switches/ps708/products_white_paper0900aecd803e508c.shtml

Catalyst 6500 Supervisors Distributed Forwarding Performance


Another often quoted number is the 400Mpps forwarding number – again this is a number derived from utilizing distributed forwarding cards (DFC’s) to optimize the forwarding performance of the switch…

SLOT 1SLOT 2SLOT 3

Single ChannelSingle ChannelSingle Channel

SLOT 4 Single ChannelSLOT 5 Single ChannelSLOT 6 Single ChannelSLOT 7 Single ChannelSLOT 8 Single ChannelSLOT 9 Dual ChannelsSLOT 10 Dual ChannelsSLOT 11 Dual ChannelsSLOT 12 Dual Channels

Dual ChannelsSLOT 13

Using a WS-X6724 w/DFC yields 24Mpps Using a WS-X6724 w/DFC yields 24Mpps Using a WS-X6724 w/DFC yields 24Mpps Using a WS-X6724 w/DFC yields 24Mpps Using a WS-X6724 w/DFC yields 24Mpps Using a WS-X6724 w/DFC yields 24Mpps

Using a WS-X6724 w/DFC yields 24Mpps Supervisor slot Using a WS-X6748 w/DFC yields 48Mpps Using a WS-X6748 w/DFC yields 48Mpps Using a WS-X6748 w/DFC yields 48Mpps Using a WS-X6748 w/DFC yields 48Mpps

Using a WS-X6748 w/DFC yields 48Mpps

Total: 408Mpps


Catalyst 6500 Supervisors The 720Gb Switch Fabric

- Integrated 720Gb/sec Switch Fabric- Provides backplane interconnects between linecards

- Consists of 18 Fabric Traces which are distributed across each linecard slot

- Each Fabric Trace can run at 8Gb/sec OR 20Gb/sec

- Integrated 720Gb/sec Switch Fabric- Provides backplane interconnects between linecards

- Consists of 18 Fabric Traces which are distributed across each linecard slot

- Each Fabric Trace can run at 8Gb/sec OR 20Gb/sec

Switch FabricSwitch Fabric


Catalyst 6500 Supervisors The Backplanes

Classic BUS

Two backplanes exist in the Catalyst 6500, the “Classic BUS” and the “Switch Fabric”…Classic BUS Switch FabricSwitch Fabric

Backplane TypeBackplane Type BUSBUS CrossbarCrossbar

Supported by Sup720Supported by Sup720

Supported by Sup32Supported by Sup32YesYes YesYes

YesYes NoNo

SpeedSpeed 16Gb16Gb 8Gb or 20Gb8Gb or 20Gb

Full DuplexFull Duplex YesYes YesYes

Linecard ConnectionLinecard Connection Single ConnectionSingle Connection Single or Dual ChannelSingle or Dual Channel

Backplane is Shared Medium?Backplane is Shared Medium? Yes - All modules connect to same BUS

Yes - All modules connect to same BUS

No - Each module has discrete connection(s)No - Each module has discrete connection(s)

Supports Classic LinecardSupports Classic Linecard YesYes NoNo

Supports CEF256 LinecardSupports CEF256 Linecard YesYes YesYes

Supports CEF720 LinecardSupports CEF720 Linecard NoNo YesYes

Supports Linecard with DFCSupports Linecard with DFC NoNo YesYes


Catalyst 6500 Supervisors The 32Gb Bus

Supervisor

The 32Gb Classic Bus is a legacy backplane that originated with the first release of the Catalyst 6500 back in 1999.

Supervisor

DBUSRBUSEOBC

LinecardLinecard LinecardLinecard

DBUS - Data Bus - path over which data is transferred between linecardsRBUS - Results Bus - path over which results of forwarding lookups by the Supervisor are passed back to linecardsEOBC - Ethernet Out of Band Channel - path Supervisor uses for internal communication with linecards

DBUS - Data Bus - path over which data is transferred between linecardsRBUS - Results Bus - path over which results of forwarding lookups by the Supervisor are passed back to linecardsEOBC - Ethernet Out of Band Channel - path Supervisor uses for internal communication with linecards

Catalyst 6500 Supervisors The 720Gb Switch Fabric


The Supervisor 720 supports a 720Gb Switch Fabric which offers each connected linecard a set of discrete communication paths into the switch backplane…

LinecardSlot #3

LinecardSlot #3

LinecardSlot #4

LinecardSlot #4

SupervisorSlot #5

SupervisorSlot #5

LinecardSlot #6

LinecardSlot #6

LinecardSlot #7

LinecardSlot #7

LinecardSlot #2

LinecardSlot #2

LinecardSlot #1

LinecardSlot #1

LinecardSlot #9

LinecardSlot #9

LinecardSlot #8

LinecardSlot #8

Data Flows


CHAPTER 5Catalyst 6500 Linecards

Agenda


• LAN Line Cards• 10GE Transceivers• Daughter Cards

– Centralized Forwarding Card (CFC)– Distributed Forwarding Card (DFC)– Power over Ethernet (PoE)

• WAN Carrier Cards (SIP/SPA)• Service Modules


Catalyst 6500 Linecards Introduction

There are a multitude of linecards available for the Catalyst 6500 platform - they essentially fall into one of a number of categories defined below…

ClassicClassic

CEF256CEF256

dCEF256dCEF256

CEF720CEF720

10M Ethernet100M EthernetGE Ethernet

10GE EthernetCopper (TX)

Fiber (FL)Fiber (GBIC)Fiber (SFP)

Fiber (Xenpak)Fiber (X2)

10M Ethernet100M EthernetGE Ethernet

10GE EthernetCopper (TX)

Fiber (FL)Fiber (GBIC)Fiber (SFP)

Fiber (Xenpak)Fiber (X2)

dCEF720dCEF720

Catalyst 6500 Linecards Classic Linecards


The Classic linecard connects to the 16Gb BUS - typically it has a number of Port ASICs on board that provide this connection as well as providing the front facing ports that hosts connect into…

DBUSRBUSEOBC

SupervisorSupervisor

PORTPORT PORTPORT PORTPORT PORTPORTLinecard

The BUS is often referred to as a 32Gb

BUS

It is in fact a 16Gb BUS however, as it

supports Full Duplex communication is was marketed as 32Gb (i.e.

16Gb Read + 16Gb Write)

The BUS is often referred to as a 32Gb

BUS

It is in fact a 16Gb BUS however, as it

supports Full Duplex communication is was marketed as 32Gb (i.e.

16Gb Read + 16Gb Write)

16Gb BUS

Catalyst 6500 Linecards Classic Linecards - WS-X6148X2-RJ-45


The WS-X6148X2-RJ-45 supports the following…

Classic Linecard48 Physical 10/100 RJ-45 ports96 10/100 RJ-45 ports via Splitter (Splitter ships with module)32-Gbps Shared Bus ConnectionSupports IEEE Inline Power Daughtercard

~1MB Buffering per portSupports 1P1Q0T on RXSupports 1P3Q1T on TXSupports Weighted Round Robin and Strict Priority queuing


Catalyst 6500 Linecards Classic Linecards - WS-X6196-RJ-21

The WS-X6196-RJ-21 supports the following…

32Gb Bus connectivity only96 ports 10/100 with RJ21 TelcoSupports IEEE Inline Power DaughtercardSupports up to 96 Cisco inline powered ports

Support up to 48 IEEE Class 3 devicesSupport up to 96 IEEE Class 2 devicesSupports strict priority queueReceive queue type 1p1q0tTransmit queue type 1p3q1tBuffers – 28K Rx, 1088K Tx


Catalyst 6500 Linecards Classic Linecards - WS-X6148A-GE-TX

WS-X6148A-GE-TX wiring closet linecard for 10/100/1000 deployment

- Adds support for Jumbo Frames and Q-in-Q Tunnelling- Supports 4 Transmit queues including one strict priority queue- Adds support for WRED- Supports optional IEEE POE daughter linecard- 8:1 oversubscribed

- 48 ports of 10/100/1000 RJ-45- Increased per port buffering (5.2MB per port)- Integrated TDR for cable fault detection

Catalyst 6500 Linecards Classic Linecards - WS-X6148A-RJ-45


New improved WS-X6148A-RJ-45 wiring closet linecard for 10/100 deployment

Incorporates some of the following features- 48 ports of 10/100 RJ-45- Increased per port buffering (5.2MB per port)- Integrated TDR for cable fault detection

- Supports 4 Transmit queues including one strict priority queue- Adds support for WRED- Supports optional IEEE POE daughter linecard

Catalyst 6500 Linecards Classic Linecards - WS-X6148-FE-SFP


New improved WS-X6148-FE-SFP wiring closet linecard for 100FX to the desktop

Incorporates some of the following features- 48 ports of 100FX Fiber ports- Increased per port buffering (5.2MB per port)

- Supports 4 Transmit queues including one strict priority queue- Adds support for WRED


Catalyst 6500 Linecards CEF256 Linecards - WS-X6516A-GBIC

The WS-X6516a-GBIC supports the following…

16 ports 1000MB (GE) GBICSupports range of GBIC opticsConnection to the Shared BusSingle 8Gb Connection into the Switch Fabric

Supports optional DFC1MB Buffering per portEgress Multicast Replication2 receive queues and 3 transmit queuesSupports strict priority RX and TX queue

Catalyst 6500 Linecards CEF256 Linecards - WS-X6548-GE-TX


The WS-X6548-GE-TX supports the following…

48 ports 10/100/1000 RJ45Connection to the Shared BusSingle 8Gb Connection into the Switch FabricSupport for Cisco and IEEE Inline Power

Interoperates with all SupervisorsSome caveats…8:1 oversubscriptionNo Jumbo FramesNo support for ISL VLAN TrunkingNo Q-in-Q Trunking


Catalyst 6500 Linecards dCEF256 Linecards - WS-X6816-GBIC

The WS-X6816-GBIC supports the following…

16 ports 1000Mb (GE) GBICSupports range of GBIC opticsNO connection to the Shared BusDual 8Gb Connection into the Switch Fabric

Supports integrated DFC512K Buffering per port2 receive queues and 3 transmit queuesSupport strict priority RX and TX queue


Catalyst 6500 Linecards CEF720 Linecards - WS-X6748-SFP

Up to 48Mpps when DFC3 is used1.2MB Buffering per portEgress Multicast ReplicationSupports Strict Priority queue on transmitSupports 2 receive queues per portSupports 4 transmit queues per portSupports Weighted Round Robin

The WS-X6748-SFP supports the following…

CEF720 Linecard48 ports 1000B SFPConnection to the Shared BusTwo x 20Gb Connections into the Switch FabricSupports optional DFC3

Catalyst 6500 Linecards CEF720 Linecards - WS-X6748-GE-TX


The WS-X6748-GETX supports the following…

CEF720 Linecard48 ports 10/100/1000 RJ45Connection to the Shared BusTwo x 20Gb Connections into the Switch FabricSupports optional DFC3

Up to 48Mpps when DFC3 is used1.2MB Buffering per portEgress Multicast ReplicationSupports Strict Priority queue on transmitSupports 2 receive queues per portSupports 4 transmit queues per portSupports Weighted Round Robin

Catalyst 6500 Linecards CEF720 Linecards - WS-X6704-10GE


The WS-X6704-10GE supports the following…

CEF720 Linecard4 ports 10GE (Xenpak based)Connection to the Shared BusTwo x 20Gb Connections into the Switch FabricSupports optional DFC3

Up to 48Mpps when DFC3 is used16MB Buffering per portEgress Multicast ReplicationSupports Strict Priority queue on transmitSupports 2 receive queues per portSupports 8 transmit queues per portSupports Weighted Round Robin


Catalyst 6500 Linecards CEF720 Linecards - WS-X6708-10G-3C

The WS-X6708-10G-3C/3CXL supports the following…

dCEF720 Linecard8 ports 10GE (X2 based)No Connection to the Shared BusTwo x 20Gb Connections into the Switch FabricSupports integrated DFC3C / DFC3CXL

Up to 48Mpps local forwarding256MB Buffering per portEgress Multicast ReplicationSupports Strict Priority queue on transmitSupports 2 receive queues per portSupports 8 transmit queues per portSupports Weighted Round Robin and

Shaped Round Robin


Feature WS-X6704-10GE WS-X6708-10GE-3C/XL10 GE Ports 4Maximum 10 GE Density per Chassis 32 64

Fabric Connection 2x20G 2x20G

Fast Switchover Subseconds <200msec

Xenpak (LX4, SR, LR, ER, CX4, DWDM, WANPHY)

2MB/14MBOptional

Queue Structure (RX/TX) – Cos-Q 8q8t/1p7q8t 8q4t/1p7q4tQueue Structure (RX/TX) – DSCP-Q No 8q4t/1p7q4t

DWRR with SP

YesYes

VLAN Translation Yes (128 per port) Yes (128 per port)VSL Capable No Yes

8

Optics X2 (LX4, SR, LR, ER, CX4)

Per-port Buffers (RX/TX) 128MB/128MBDFC Integrated

Scheduling DWRR with SP, SRR (TX)

Storm Control Yes802.1q Tunneling Yes

Catalyst 6500 Linecards 10GE Linecard summary


Catalyst 6500 Linecards Optics - XENPAK for 10GE

Xenpak Modular Optics

IEEE 802.3ae 10 GbE MSA

Hot Pluggable 10GbE ‘GBIC’

SC duplex fiber optic connector

Compliant for all IEEE 802.3ae and 802.3ak mediums

Xenpak Modular Optic Options include

- WS-XENPAK-ZR—80km Single Mode Fiber

- WS-XENPAK-LR—10km Single Mode Fiber

- WS-XENPAK-ER—40km Single Mode Fiber

- WS-XENPAK-SR—66m to 300m Multi-mode Fiber

- WS-XENPAK-LX4—300m Multi-mode Fiber

- WS-XENPAK-CX4—10GbE Copper Short Reach up to 15m using Infini-band cabling

Xenpak optics are used in the new 10GE modules. They allow flexibility in choice of optics per port on each 10GE module. Xenpak optic options include the following…


Catalyst 6500 Linecards Optics - DWDM and Receive Only Xenpak

DWDM Xenpak Modular Optics


Hot Pluggable 10GbE ‘GBIC’

SC duplex fiber optic connector


- Support 10GBase Ethernet- 32 different Xenpak options each supporting a non-tuneable ITU 100Ghz wavelength with the Cisco ONS DWDM channel plan

- Supports digital optical monitoring- Dual SC/PC connection

Xenpak optics also now include DWDM and Receive Only Optics

Xenpak DWDM Modular Optic Options specifications

Xenpak Receive Only Modular Optic Options specifications- WDM-XENPAK-REC- Used when creating a Unidirectional Ethernet (UDE) Link


Catalyst 6500 Linecards Optics - X2 for 10GE

X2 Modular Optics


Hot Pluggable 10GbE ‘Optic’

SC/PC duplex fiber optic connector


X2 Modular Optic Options include

- X2-10GB-ER—40km Single Mode Fiber- X2-10GB-LR—10km Single Mode Fiber - X2-10GB-SR—26m to 300m Multi-mode Fiber

- X2-10GB-LX4—300m Multi-mode Fiber- X2-10GB-CX4—10GbE Copper Short Reach up to 15m using Infiniband cabling

X2 optics are used in the new 6708 10GE module. They provide full interoperability with the equivalent Xenpak optic. X2 optics include the following…

Catalyst 6500 Linecards What LAN Linecards Should I Be Selling?


If we take the typical Campus Design, then the following linecards are suggested for each layer…

CORECORE

DISTRIBUTIONDISTRIBUTION

ACCESSACCESS

WS-X6708-10G-3C, WS-X6708-10G-3CXL, WS-X6724-SFP, WS-X6748-SFP, WS-X6748-GE-TX

WS-X6708-10G-3C, WS-X6708-10G-3CXL, WS-X6724-SFP, WS-X6748-SFP, WS-X6748-GE-TX, WS-X6516A-GBIC, WS-X6816-GBIC

WS-X6708-10G-3C, WS-X6708-10G-3CXL, WS-X6724-SFP, WS-X6748-SFP, WS-X6748-GE-TX, WS-X6516A-GBIC, WS-X6148A-GE-TX, WS-X6148-RJ-45, WS-X6148-SFP, WS-X6148X2-RJ-45, WS-X6196-RJ21, WS-X6548-GE-TXAlso the inline power versions of the above linecards are applicable for this layer…

Catalyst 6500 Linecards Mixing Different Linecards Architectures


When transferring data between linecards, the backplane will operate in one of three modes –these modes are determined by the combination of linecards installed in the chassis and which module the traffic sourced from and destined to…

Mode Description

FLOW THROUGH / BUSUsed for traffic between non fabric (classic) enabled modules and for traffic between a non fabric and a fabric enabled linecard…centralized performance to 15Mpps

COMPACT / FABRIC ONLYUsed when only ALL fabric enabled linecards used in a chassis – this mode uses a compact from of DBus header which optimizes centralized performance to 30Mpps

TRUNCATED / MIXEDUsed for traffic between fabric enabled linecards when a non fabric enabled linecard is installed in the chassis. In this mode, centralized forwarding reverts back to 15Mpps.

Cat6k#show fabric switching-mode | inc GlobalGlobal switching mode is Compact


Catalyst 6500 Linecards Centralized Forwarding Cards (CFC)

The CFC sits on all CEF720 linecards and

provides the connection to the

Classic BUS

CFC is used to communicate with the

Supervisor when centralized forwarding

is used

The Centralized Forwarding Card (CFC) provides BUS connectivity for the CEF720 linecards…

The CFC sits on all CEF720 linecards and

provides the connection to the

Classic BUS

CFC is used to communicate with the

Supervisor when centralized forwarding

is used


Catalyst 6500 Linecards Distributed Forwarding Card 3 (DFC3)

The DFC3 is an optional extra - it is used to provide local switching for the linecard to incrementally boost overall switch performance - if installed on a CEF720 linecard, it takes the place of the CFC…

The DFC3 supports forwarding rates up to 48Mpps

The DFC3 also stores a local copy of the forwarding tables, as well as

Security and QoS ACL’s that are centrally defined

The DFC3 supports forwarding rates up to 48Mpps

The DFC3 also stores a local copy of the forwarding tables, as well as

Security and QoS ACL’s that are centrally defined

Three different versions of the DFC are supported…

DFC3aDFC3B/DFC3BXLDFC3C/DFC3CXL

Three different versions of the DFC are supported…

DFC3aDFC3B/DFC3BXLDFC3C/DFC3CXL

Catalyst 6500 Linecards DFC3 Interoperability with PFC3


The use of a DFC3 requires it to operate with the equivalent PFC3 version - a mix of PFC3 and DFC3 versions will result in the system operating at the lowest common denominator…

show platform hardware pfc mode

PFC3A PFC3B PFC3BXLDFC3 Yes Operate as PFC3A Operate as PFC3A

DFC3B Operate as DFC3A Yes Operate as PFC3BDFC3BXL Operate as DFC3A Operate as DFC3B Yes

DFC3C Operate as DFC3A Operate as DFC3B Operate as PFC3B and DFC3B

DFC3CXL Operate as DFC3A Operate as DFC3B Operate as DFC3BXL

ExampleCat6k#show platform hardware pfc mode PFC operating mode : PFC3B


Catalyst 6500 Linecards Inline Power

WS-F6K-FE48-AF for WS-X6148-RJ45WS-F6K-GE48-AF for WS-X6148-GETXWS-F6K-GE48-AF also for WS-X6548-GETXWS-F6K-FE48X2-AF for WS-X6148X2-RJ45WS-F6K-48-AF (supports fixed power)

Maximum number of Inline Devices supported in AC power mode and DC power modeAC - Class 2Chassis AC - Class 3 DC - Class 2 DC - Class 3

70Catalyst 6503-E 48 70 48227Catalyst 6504-E 110 227 110480Catalyst 6506-E 284 381 173587Catalyst 6509-E 288 374 184574Catalyst 6513 281 357 176

** Numbers derived from running the CCO power calculator against the largest available single power supply for a given chassis

Class 2 – 7W (4ma-5ma, and 19V to 21V)Class 3 – 15.4W (6ma-7ma, and 19V to 21V)

Catalyst 6500 Linecards Understanding IP Phone Support


There are three main areas where the switch interacts with the Cisco IP Phone – they are in the areas of delivery of inline power ,VLAN tagging, and extended trust settings.

Segregated VLAN’s for Voice and Data traffic

Primary VLANAuxiliary (Voice) VLAN

Inline power delivery to power the Phone Re-tagging packets from downstream device to a value pre-determined by the administrator


Catalyst 6500 Linecards Inline Device Discovery

When a switchport configured for inline power comes online, the switch will send a Fast Link Pulse (FLP) to the attached device – in the Cisco IP phone, it will loop the FLP back to the switch, indicating its inline power capable - at this point the switch is ready to deliver inline power…

IP Phone Switch Port

RX

TX

FLPFLP

It’s an inline device

Catalyst 6500 Linecards Default Inline Allocation


When a switch port is enabled for inline power, upon discovery of an attached inline device, it will deliver a default amount of power to the attached device -

12345678

Switch Port

12345678

Each UTP cable contains 8 wires – On the Catalyst 6500 Inline powered capable line cards, inline power is delivered over pins 1, 2, 3 and 6

The default inline power allocation is 15 Watts – once the inline device is enabled, it will use CDP to adjust its power to the actual requirement of the inline device…

Catalyst 6500 Linecards Adjusting the Default Inline Allocation


After the phone is powered up, it will use Cisco Device Protocol (CDP) to instruct the switch to adjust the delivered power to a specific value – non IEEE and IEEE based Cisco phones will negotiate the default allocation of 15.4 Watts down to their required power value. This allows the switch to avoid wasting unnecessary power…

Here is the default allocation of 15.4W

Can you adjust the power down to 12W

Sure – power now delivered at 12W

NOTE: Future proposed Cisco IEEE phones will also use a CDP like feature (LLDP) to negotiate default IEEE Class power values down to actual phone power requirements


Catalyst 6500 Linecards New Inline Power Daughter Card

- Provides 15.4W (IEEE Class 3) for all 48 ports on the above linecards- Supported in linecards for both 6500 and 6500-E chassis- Supports bi-directional CDP support for negotiation of power- Supports power shut down if port draws more power than configured

A new Inline Powered daughter card has been introduced which allows an administrator to define the maximum amount of power that can be delivered to a port on that module…


Catalyst 6500 Linecards WAN - Enhanced FlexWAN

The WS-X6582-PA supports the following…

Legacy PA Carrier CardUp to 625Kpps without QoSUp to 2 x OC3 throughputDual 400Mhz SiByte CPU’sUp to 128MB of packet buffer per CPU

Distributed QoS featurescRTPLFI - ATM, FR, MLPPPClassificationPolicingMarkingCBWFQ/LLQ and WREDPer Class Traffic Shaping


Catalyst 6500 Linecards WAN - SIP200

The 7600-SIP-200 supports the following…

Cisco 7500 Feature ParityUp to 1.1Mpps without QoSUp to 622Mbps performanceDual Core CPU’s4 bays for SPA’sUp to 128MB of packet buffer per CPU

Distributed HQoS featurescRTPMLPPP and LFI in hardwareClassificationPolicingMarkingCBWFQ/LLQ and WREDPer Class Traffic Shaping



The 7600-SIP-200 supports the following Supervisor options…

Sup720Sup720-3BSup720-3BXLSup32-8GESup32-10GE

Supported in all 6500 chassis except 6503

Requires IOS 12.2(18)SXE (for Sup720)Requires IOS 12.2(18)SXF for Sup32



SPA Name Description Height

SPA-CH8TE1 8- port ChannelizedT1/E1 SPA Single height

SPA-4XT3/E3 4-port T3/E3 SPA Single height

SPA-4XCT3/DS0 4- port Channelized T3 SPA Single height

SPA-2XCT3/DS0 2- port Channelized T3 SPA Single height

SPA-2XT3/E3 2-port T3/E3 SPA Single height

SPA-4XOC3-POS 4- port POS OC3 SPA Single heightSPA-2XOC3-POS 2- port POS OC3 SPA Single heightSPA-4XOC3-ATM 4- port ATM OC3 SPA Double heightSPA-2XOC3-ATM 2- port ATM OC3 SPA Double height




Network Processor based carrier cardUp to 6.2Mpps @ 40 byte packetsUp to 4Gbps performance @ 64 byte packetsDual Network Processors4 bays for SPA’sReplacement for OSM

32,000 Queues3 level Hierarchical QoSDual Rate, 3 Color PolicerCBWFQ + LLQ + WREDEoMPLSFRoMPLSATM AAL5 over MPLSATM Cell Relay over MPLS



SPA Ports Interface Form FactorOC-3/STM-1 POS 2, 4 SFP Single Height

Single HeightOC-3/STM-1 ATM 2, 4 SFP Double HeightOC12/STM-4 ATM 1 SFP Double HeightOC48/STM-16 POS/RPR 2, 4 SFP Single HeightOC48/STM-16 ATM 1 SFP Double HeightT3/E3 ATM 2, 4 Copper Single HeightChannelized T1/E1 8 Copper Single HeightClear Channel T3/E3 2, 4 Copper Single HeightChannelized T3 2, 4 Copper Single HeightChannelized OC-3/STM-1 1 SFP Single HeightFE 4, 8 TX, FX Single HeightGE 1, 2 SFP Single Height

OC-12/STM-4 POS 1, 2, 8 SFP

SPA’s supported by the SIP400 include the following…




Integrated DFC38000 full duplex queuesUp to 10Gbps performance @ 64 byte packetsDual Network Processors1Gb local Memory20Gb Switch Fabric ConnectionNo connection to the Classic BUS

Sup720-3B and Sup720-3BXL only

VPLS/H-VPLSSecurity ACL’sPolicingClassificationMarkingCBWFQ/LLQHierarchical Traffic ShapingShaping/WRED


Catalyst 6500 Linecards WAN – where to use the SIPs?

SiSi

SiSi

Aggregation Site

Branch 1

Branch 2

Branch 3

Branch 4

6500

6500

ISR

3750

ME3400

ME3750

2 Mbps

50 Mbps

200 Mbps

200 Mbps

Hierarchical QoS Required:-per vlan/subinterface shaping-per shaped queue LLQ-per shaped queue CBWFQ

SIP-400 with SPA-2X1GE


Catalyst 6500 Linecards Services Modules

WLSM, WiSM

Supported by the Wireless BU (WNBU)

JUST A QUICK NOTE: Services Modules are not directly supported by the ISBU - rather they are supported by the respective BU that developed them…

WLSM, WiSM

Supported by the Wireless BU (WNBU)

FWSM, VPNSM, IDSM, VPN SPA, CSM, SSL,

ACE, Anomaly Detector, Anomaly

Guard

Supported by the Security Technology Group (STG)

FWSM, VPNSM, IDSM, VPN SPA, CSM, SSL,

ACE, Anomaly Detector, Anomaly

Guard

Supported by the Security Technology Group (STG)

NAM-1, NAM-2

Supported by the Network Management BU (NMBU)

NAM-1, NAM-2

Supported by the Network Management BU (NMBU)

AONS

Supported by the Application Orientated Network BU (AONBU)

AONS

Supported by the Application Orientated Network BU (AONBU)

MWAM, CSG, PSD

Supported by the Routing and Service Provider TG (RSPTG)

MWAM, CSG, PSD

Supported by the Routing and Service Provider TG (RSPTG)

Services ModulesServices Modules


Catalyst 6500 Linecards Services Modules - ACE

ACE

Virtual partitioning (up to 250 Contexts)Up to 16Gbps performance6.5Mpps350,000 SYSLOG’s per second4 Million concurrent connections16K Real or Virtual Servers

Multiple probes (ICMP, TCP, UDP, etc)HTTP deep packet inspectionBi-Directional NAT/PATTCP Connection State TrackingTCP Header validation and window size checkingURPF check at session establishment

Catalyst 6500 Linecards Services Modules - FWSM


Multiple Security ContextsRouted and Transparent FirewallUp to 5Gbps of PerformanceBi-Directional NATPolicy Based NAT

URL FilteringOSPF Routing Protocol SupportObject GroupingActive-Standby Inter/Intra FailoverARP Inspection

FWSM

Catalyst 6500 Linecards Services Modules - IDSM


Up to 600Mbps performance (passive)Up to 500Mbps (inline)Advanced false positive reductionSupports multiple capture techniquesSingle device management using CLI or

IPS device manager

Up to 6000 HTTP transactions per secondUp to 6000 TCP connections pe secondUp to 60,000 concurrent connectionsHot swap moduleStandard code base 100% compatible with Cisco standalone IPS devices

IDSM


Catalyst 6500 Linecards Services Modules - NAM

NAM

Application MonitoringPerformance ManagementFault IsolationTroubleshootingTrend AnalysisCapacity planning VOIP Monitoring

QoS and DSCP monitoringMIB II – RFC1213RMON (RFC2819) All groupsRMON2 (RFC2021) All groupsSMON (RFC2613)DSMON (Rmon extensions for Diffserv)ART MIBHCRMON


Catalyst 6500 Linecards Services Modules - VPN SIP + SPA

DES, 3DES and AES Encryption ServicesUp to 2.5Gbps of AES EncryptionUp to 60 Tunnels/sec setup rateUp to 8000 TunnelsUp to 10 VPN SPA's per Catalyst 6500Jumbo Frame SupportHardware Acceleration for IPSec and GREDynamic Multipoint VPN (DMVPN)VRF Aware

VPN

Catalyst 6500 Linecards Services Modules - WiSM


Incorporates some of the following features- CEF720 based module- IEEE 802.11a, 802.11b, 802.11g, 802.11d, 802.11h- Intelligent RF Control plane supporting self-configuration, self-healing and self-optimization- Intrusion Detection / Mobility Management

- HTTP, HTTPS, CLI, Telnet, SSH management interface options- Encryption options include WEP and TKIP-MIC: RC4 40, 104 and 128 bits (both static and shared keys) Secure Sockets Layer (SSL) and TLS: RC4 128-bit and RSA 1024- and 2048-bit AES: CCM, CCMP

WiSM


CHAPTER 6Catalyst 6500 Switching Basics and Internals

Agenda


• VLANs and Port Configuration• Basic Layer 2 and Layer 3 forwarding• PFC3 Layer 2 forwarding• PFC3 Layer 3 forwarding• Multicast forwarding

Catalyst 6500 Internals Introduction


What lies within the Catalyst 6500 and how does it perform its magic? This section will look at the internals of the Catalyst 6500: it will first explore the linecard port configuration and then explore how the Catalyst 6500 performs the task of forwarding packets…


Catalyst 6500 Internals Understanding VLANs

A Virtual LAN allows the grouping of different switch ports into the same broadcast domain as though they were connected via the same physical switch. A VLAN can span across non contiguous ports, across different modules and across different switch’s.

In the above diagram, there are three VLANs, Red, Green and Blue – all hosts belonging to a particular VLAN need to traverse a Layer 3 device to reach a host in another VLAN…


Catalyst 6500 Internals Broadcast Domain

A VLAN creates a broadcast domain such that any broadcasts generated by hosts within the VLAN do not (by default) cross into another VLAN boundary…

In the above example, a broadcast sent by “Red” host A will be forwarded to all other hosts in the RED VLAN, but not to hosts in the BLUE or GREEN VLAN…


Catalyst 6500 Internals VLANs and IP Subnets

While not common, it is valid for multiple subnets to exist wholly within the same VLAN but in this case each subnet needs a layer 3 device to communicate to another subnet…

It is common practice for a Virtual LAN to be associated with a single IP Subnet as follows.

VLAN A - IP Subnet A

VLAN B - IP Subnet BVLAN C - IP Subnet A & B


Catalyst 6500 Internals VLAN Number Range

When a VLAN is created, it has to be assigned a valid number within a specified range. Currently the VLAN number range is as follows…

VLAN # Range Usage VTP Support0 Reserved System Use only N/A1 Normal Cisco Default – Usable but cannot be deleted Yes

2 - 1001 Normal Can be created, used and deleted

1002 - 1005 Normal Defaults for Token Ring and FDDI – Cannot be deleted

1006 - 4094 Extended For Ethernet VLAN’s only - Can be created, used and deleted No

4095 Reserved System Use only N/A

Yes

Yes

NOTE: Configuring extended VLAN’s required additional configuration


Catalyst 6500 Internals Extended VLANs

Each VLAN consumes a MAC address (used by Spanning Tree to build a bridge ID). As the switch only has 1024 MAC addresses, using extended VLAN’s (1006 – 4024) requires users to enable the “extended system-id” feature – this enables switch to build a unique bridge ID for all potential 4094 VLAN’s…

Normal Spanning Tree Bridge ID is built as follows…

Bridge Priority MAC Address

2 bytes – 16 bits 6 bytes – 48 bits

Bridge Priority without extended system-id configured…

Bridge Priority with extended system-id configured…

Bridge Priority Extended System ID (VLAN)Bridge Priority

2 bytes – 16 bits 4 bits 12 bits


Catalyst 6500 Internals Internal VLANs

The Catalyst 6500 uses a VLAN number internally to represent a layer 3 port – that being a physical layer 3 port (like a FlexWAN or a routed Ethernet port) or a logical layer 3 port (like a sub-interface on a FlexWAN port, etc)…

STDVLAN1-1001

EXTDVLAN1006

to4094

Standard Ethernet layer 2 port can be placed in any VLAN

VLAN interface can use any VLAN number

A layer 3 Ethernet port or a FLEXWAN/OSM layer 3 port each consumes 1 extended VLAN number

A sub-interface consumes 1 extended VLAN number


Catalyst 6500 Internals Internal VLANs

Once an extended VLAN is consumed by a layer 3 port, it cannot be used for other purposes…The switch can be configured to define the allocation policy – that is should extended VLAN numbers be allocated bottom up (from 1006 up) or top down (from 4094 down)…

STDVLAN1-1001

EXTDVLAN1006

to4094

1006100710081009…..

……4091409240934094

Allocation policy of ascending indicates the VLAN’s allocated to layer 3 interfaces will be assigned from 1006 and upwards…

INTERNAL VLAN ALLOCATION POLICY

Allocation policy of descending indicates the VLAN’s allocated to layer 3 interfaces will be assigned from 4094 and downwards…

Catalyst 6500 InternalsLinecard Port Configuration


When running IOS, Ethernet ports can be configured with one of three interface types: Access, Trunk or Router. Interfaces in IOS assume a different default behavior than those same ports under CatOS in that they are shutdown on initial startup.

SupervisorSupervisor MSFCMSFC

SVI SVI

Supervisor Engine

VLANInterface

Linecard

AccessPorts

TrunkPort

ShutdownShutdown

L3 RoutedPorts


Catalyst 6500 Internals VLAN Port Types

Switch Ports defined as an access port are placed in a VLAN. They can only belong to one VLAN at a time. Special Switch Ports can be defined as a VLAN Trunk Port which I designed to carry traffic from multiple VLANs… Trunk ports tend to be defined for links to other switches or routers…

Port 2/2 – VLAN 10






Access Ports Trunk Ports


Catalyst 6500 InternalsVLAN Trunks - Tagging

A VLAN trunk will tag data with its VLAN number, so the destination switch will know which VLAN to forward to packet to – There are two technologies supported in the Catalyst 6500 to “tag” VLANs and they are ISL and 802.1Q – these are typically implemented in ASICs to maximize performance

VLAN 10

VLAN 20

VLAN 30

VLAN 10

VLAN 20

VLAN 30

Trunk Port to carry traffic from Multiple VLANs

Individual VLANs running on Access Ports


Catalyst 6500 Internals VLAN Tagging – ISL

Inter Switch Link (ISL) was the first VLAN tagging mechanism released by Cisco. It is a “two level” tagging mechanism as it pre-appends and appends tags both at the front and back of the encapsulated frame… Its supports 1024 VLAN numbers

Data DataData

ISL Header 26 Bytes Data FCS 4 Bytes

DA Type User SA LEN AAAA03 HSA VLAN BPDU INDEX RES

VLAN 10

VLAN 20

VLAN 30

VLAN 10

VLAN 20

VLAN 30


Catalyst 6500 InternalsVLAN Tagging – 802.1Q

802.1Q is an IEEE standard for VLAN Tagging - It is a “one level” tagging mechanism inserting a single tag within the Ethernet frame… Unlike ISL, it supports the full 4096 VLAN numbers…

Data DataData

DA SA ETH-TYPE TAG TYPE/LEN DATA

User Priority CFI VLAN Number

VLAN 10

VLAN 20

VLAN 30

VLAN 10

VLAN 20

VLAN 30

Catalyst 6500 Internals Packet Forwarding in the Catalyst 6500


What lies within the Catalyst 6500 and how does it perform its magic? This section will explore how the Catalyst 6500 performs the task of forwarding packets…


Catalyst 6500 Internals The Packet

Preamble

Everything a Switch does is centered on the data it forwards…

Preamble DEST AddressDEST Address SRC AddressSRC Address LEN/ETYPELEN/ETYPE DATADATA FCSFCS

Ethernet Frame

A Layer 2 Switch will use the Destination MAC Address to make forwarding decisions

IPv4 HeaderIPv4 Header

VersionVersion IHLIHL Type of ServiceType of Service LengthLengthIdentificationIdentification FlagsFlags Fragment OffsetFragment Offset

Time to Live (TTL)Time to Live (TTL) ProtocolProtocol Header ChecksumHeader ChecksumSource IPv4 AddressSource IPv4 Address

Destination IPv4 AddressDestination IPv4 Address

OptionsOptions

DataData

A Layer 3 Switch will use the Destination IP Address to make forwarding decisions

Catalyst 6500 Internals The Layer 2 Boundary


A host can talk to any host at Layer 2 within the same VLAN (or broadcast domain) without requiring a Layer 3 Switch (or router) to forward the data…

VLAN 10 VLAN 20

A

B

C

D

E

F

A <> BA <> CB <> C

D <> ED <> FE <> F

Layer 3 operationA/B/C <> DA/B/C <> EA/B/C <> F

D/E/F <> AD/E/F <> BD/E/F <> C

Layer 2 operation

Catalyst 6500 Internals Layer 2 and Layer 3 Forwarding


What is the difference between a Layer 2 and Layer 3 Forwarding operation? It’s all about the address that is used to forward the data – let’s look at a Layer 2 and Layer 3 operation…

A B

C

D

VLAN 10SUBNET X

SUBNET Y

SUBNET ZA B



We will explore a Layer 2 operation first - packet from host A to host B…

IP: 10.1.1.2/24MAC: A1VLAN 10

IP: 10.1.1.2/24MAC: A1VLAN 10

IP: 10.1.1.3/24MAC: B1VLAN 10

IP: 10.1.1.3/24MAC: B1VLAN 10

IP: 10.1.1.1/24MAC: C1VLAN 10

IP: 10.1.1.1/24MAC: C1VLAN 10

A B

11

11 A sends ARP - who is 10.1.1.3DMAC: All FF’s SMAC: A1

22

22 ARP is broadcast so switch forwards out all ports

33

33 B Replies to ARPDMAC: A1 SMAC:B1

44

5544 A sends to B

DMAC: B1 SMAC:A1DIP: 10.1.1.3 SIP: 10.1.1.2

55 Switch performs CAM lookup using DMAC and forwards packet to B1



Now lets look at a Layer 3 operation - from host A to host D - We will use the following addressing…

IP: 10.1.1.2/24MAC: A1VLAN 10

IP: 10.1.1.2/24MAC: A1VLAN 10

IP: 10.5.1.2/24MAC: F2IP: 10.5.1.2/24MAC: F2

IP: 10.1.1.1/24MAC: C1VLAN 10

IP: 10.1.1.1/24MAC: C1VLAN 10

IP: 10.2.1.1/24MAC: C2IP: 10.2.1.1/24MAC: C2

IP: 10.3.1.2/24MAC: D1IP: 10.3.1.2/24MAC: D1

IP: 10.5.1.1/24MAC: D2IP: 10.5.1.1/24MAC: D2

D

A

IP: 10.3.1.1/24MAC: C3IP: 10.3.1.1/24MAC: C3



The operation proceeds as follows…

IP: 10.1.1.2/24MAC: A1VLAN 10

IP: 10.1.1.2/24MAC: A1VLAN 10

IP: 10.5.1.2/24MAC: F2IP: 10.5.1.2/24MAC: F2

IP: 10.1.1.1/24MAC: C1VLAN 10

IP: 10.1.1.1/24MAC: C1VLAN 10

IP: 10.3.1.2/24MAC: D1IP: 10.3.1.2/24MAC: D1

IP: 10.5.1.1/24MAC: D2IP: 10.5.1.1/24MAC: D2

D

A

11

11 A sends ARP - who is 10.5.1.2DMAC: All FF’s SMAC: A1

22

22 Switch replies to ARP - saying send it to me

1133

33 A sends to SwitchDMAC:C1 SMAC:A1DIP: 10.5.1.2 SIP: 10.1.1.2

IP: 10.3.1.1/24MAC: C3IP: 10.3.1.1/24MAC: C3



The operation proceeds as follows…

IP: 10.1.1.2/24MAC: A1VLAN 10

IP: 10.1.1.2/24MAC: A1VLAN 10

IP: 10.5.1.2/24MAC: F2IP: 10.5.1.2/24MAC: F2

IP: 10.1.1.1/24MAC: C1VLAN 10

IP: 10.1.1.1/24MAC: C1VLAN 10

IP: 10.3.1.1/24MAC: C3IP: 10.3.1.1/24MAC: C3

IP: 10.3.1.2/24MAC: D1IP: 10.3.1.2/24MAC: D1

IP: 10.5.1.1/24MAC: D2IP: 10.5.1.1/24MAC: D2

D

A

44 Switch does a L3 lookup

44

55Packet forwardedDMAC:D1 SMAC:C3DIP: 10.5.1.2 SIP: 10.1.1.2

55

6677

66 Switch does a forwarding lookup

77Packet forwardedDMAC:F2 SMAC:D2DIP: 10.5.1.2 SIP: 10.1.1.2

Catalyst 6500 Internals Basics of a Layer 2 Forwarding Operation


The MAC Address Table (or CAM Table) is a piece of memory in a switch that is used to store MAC addresses and the ports from which they were learnt…

CAM Table

CAM tables range in size across the different

switch platforms

CAM table can also store VLAN within which MAC

was learnt

MACABCDEF

Port123456

A

B

C

D

E

F

12

3

45

6



A

B

C2

3

The normal CAM behavior in a switch is as follows…

D4

1

123

ABC

CAM

DST MAC = CDST MAC = C

For MAC addresses that are found in the

CAM table, the switch will forward that packet

only to that host…



A

B

C2

3

The normal CAM behavior in a switch is as follows…

D4

1

123

ABC

CAM

DST MAC = DDST MAC = D

For MAC addresses that are NOT found in

the CAM table, the switch will forward that packet to ALL hosts in

that VLAN…


Catalyst 6500 Internals Layer 2 Forwarding on the PFC3

MAC Table

16 pages4096 rows

PFC3

4K*16=64K entries

On the PFC3 is an integrated CAM Table that supports up to 64,000 MAC address entries…



1. Hash result identifies starting a row in MAC table2. Lookup key (VLAN and MAC) compared to contents of indexed line on each page in

parallel3. Destination lookup: Match returns destination interface(s), miss results in flood Source

lookup: Match updates age of matching entry, miss installs new entry in table

Destination interface(s)

DMAC lookup

UpdateEntry

SMAC lookup

6

MAC Table16 pages

4096 rows20 | 0000.cccc.cccc

10 | 0000.bbbb.bbbb

30 | 0000.dddd.dddd

10 | 0000.aaaa.aaaa

Compare

10 | 0000.aaaa.aaaaVLAN MAC Address

Lookup Key

Frame

Hash Function

MAC TableRow

HIT!

1

2

3

4

5



Cisco IOSCisco IOS show mac-address-tableshow mac-address-table

6509#show mac-address-table dynamic vlan 30Codes: * - primary entry

vlan mac address type learn qos ports------+----------------+--------+-----+---+-----------------------* 30 0003.a088.c408 dynamic Yes -- Fa3/18* 30 0012.d949.04d2 dynamic Yes -- Gi5/1* 30 0003.a08a.15f3 dynamic Yes -- Fa3/24* 30 0090.a400.1850 dynamic Yes -- Fa3/14* 30 0003.a08a.15f9 dynamic Yes -- Fa3/25<…>6509#

6509#show mac-address-table dynamic vlan 30Codes: * - primary entry

vlan mac address type learn qos ports------+----------------+--------+-----+---+-----------------------* 30 0003.a088.c408 dynamic Yes -- Fa3/18* 30 0012.d949.04d2 dynamic Yes -- Gi5/1* 30 0003.a08a.15f3 dynamic Yes -- Fa3/24* 30 0090.a400.1850 dynamic Yes -- Fa3/14* 30 0003.a08a.15f9 dynamic Yes -- Fa3/25<…>6509#

The MAC addresses that have been learned by the Switch can be viewed from the switch CLI using the following command - note that for each MAC address learned, the port from where theAddress arrived is stored along with the VLAN of which the host is a part …

NOTE: You can have duplicate MAC addresses as long as they appear in a different VLANNOTE: You can have duplicate MAC addresses as long as they appear in a different VLAN

NOTE: MAC address learning is done in HARDWARENOTE: MAC address learning is done in HARDWARE



Layer 3 Forwarding is controlled by the CEF (Cisco Express Forwarding) process - the elements include…

Routing ProtocolsOSPF, EIGRP, ISIS, BGP, etc

Routing protocols receive routing information from the network Control Plane (RP)Holds routing tables in

Routing information Base (RIB)from all running routing

protocols

CEFTakes RIB and builds a

Forwarding Information Base(FIB) containing prefixes

CEFLoads FIB into PFC3

and DFC3s

FIB (on PFC3/DFC3)FIB is used by PFC3/DFC3 hardware to

perform L3 lookups and forwarding

CEFLoads FIB into PFC3

and DFC3s

FIB (on PFC3/DFC3)FIB is used by PFC3/DFC3 hardware to

perform L3 lookups and forwarding



Control Plane (RP)Interacts with the network - where all the routing protocols run (i.e. OSPF, BGP, EIGRP, etc) – Route Processor is located on the MSFC…

Data Plane (PFC3)Holds the FIB, Adjacency, ACL’s and Netflow Statistics - performs the hardware based switching of packets

Hardware Based CEF Process

FIB lookup based on destination prefix (longest-match)

FIB “hit” returns adjacency, adjacency contains rewrite information (next-hop)

ACL, QoS, and NetFlow lookups occur in parallel and effect final result



The FIB contains the following• IPv4 entries logically arranged from most

to least specific• Overall FIB hardware shared by

–IPv4 Unicast–IPv4 Multicast–IPv6 Unicast–IPv6 Multicast–MPLS

Adjacency table:• Hardware adjacency table also shared

among protocols• Actual adjacency table entries are NOT

shared

10.1.0.0172.16.0.0

…

172.20.45.110.1.1.100

…10.1.3.010.1.2.0

…

0.0.0.0

MASK (/24)

MASK (/16)

MASK (/32)

MASK (/0)

FIB TCAM

IF, MACs, MTU

IF, MACs, MTU

IF, MACs, MTU

IF, MACs, MTU

Adjacency Table

Located on the PFC3 is the FIB and Adjacency Table…


Catalyst 6500 Internals IPv4 Layer 3 Forwarding on the PFC3

10.1.0.0172.16.0.0

…

172.20.45.110.1.1.100

…10.1.3.010.1.2.0

…

0.0.0.0

MASK (/24)

MASK (/16)

MASK (/32)

MASK (/0)

Assuming a lookup was performed for a packet with a destination of 10.1.5.2, then the following would occur…

PacketPacket11

22 Key GenKey Gen22 Key GenKey Gen

33 Lookup KeyLookup Key

HIT!

IF, MACs, MTU

IF, MACs, MTU

IF, MACs, MTU

IF, MACs, MTU

Load-SharingHash

44

55

66 66

Adjacency Table

FIB TCAM

Catalyst 6500 Internals Load Balancing


The Catalyst 6500 also supports load balancing - two forms of load balancing exist…

EtherchannelEtherchannel

Up to 8 links can be bundled together to

make them logically look like a single link

Up to 8 links can be bundled together to

make them logically look like a single link

Equal Cost Multi-pathRouting (ECMP)

Equal Cost Multi-pathRouting (ECMP)

Up to 8 links (paths) can be used to get to another

network node(e.g. from A to B)

Up to 8 links (paths) can be used to get to another

network node(e.g. from A to B)

A

B

Catalyst 6500 Internals Etherchannel - How it works?


Etherchannel uses a load balancing algorithm to determine which link in the bundle to use - the inputs to the algorithm are a combination of L2, L3 or L4 addresses…

6500(config)# port-channel load-balance ?dst-ip Dst IP Addrdst-mac Dst Mac Addrdst-port Dst TCP/UDP Port mpls Load Balancing for MPLS packets src-dst-ip Src XOR Dst IP Addrsrc-dst-mac Src XOR Dst Mac Addrsrc-dst-port Src-Dst TCP/UDP Port src-ip Src IP Addrsrc-mac Src Mac Addrsrc-port Src TCP/UDP Port

6500(config)# port-channel load-balance ?dst-ip Dst IP Addrdst-mac Dst Mac Addrdst-port Dst TCP/UDP Port mpls Load Balancing for MPLS packets src-dst-ip Src XOR Dst IP Addrsrc-dst-mac Src XOR Dst Mac Addrsrc-dst-port Src-Dst TCP/UDP Port src-ip Src IP Addrsrc-mac Src Mac Addrsrc-port Src TCP/UDP Port


Catalyst 6500 Internals Etherchannel - What Link is Used?

Load Balance Option

The output of the Algorithm is a hex string which matches a bit out of an 8 bit string - this in turn identifies the link in the bundle that will be used to transmit the data…

Load Balance Option Bit ResultBit Result

Hash Bit 7 Bit 6 Bit 5 Bit 4 Bt 3 Bit 2 Bit 1 Bit 02 Links Link 1 Link 1 Link 1 Link 1 Link 2 Link 2 Link 2 Link 23 Links Link 1 Link 1 Link 1 Link 2 Link 2 Link 2 Link 3 Link 34 Links Link 1 Link 1 Link 2 Link 2 Link 3 Link 3 Link 4 Link 45 Links Link 1 Link 1 Link 2 Link 2 Link 3 Link 3 Link 4 Link 56 Links Link 1 Link 1 Link 2 Link 2 Link 3 Link 4 Link 5 Link 67 Links Link 1 Link 1 Link 2 Link 3 Link 4 Link 5 Link 6 Link 78 Links Link 1 Link 2 Link 3 Link 4 Link 5 Link 6 Link 7 Link 8

ExampleHash result = 0x2 for 6 links

Result link is link 4








Catalyst 6500 Internals ECMP - How it works?

6500#show mls cef lookup 10.10.10.1

Codes: decap - Decapsulation, + - Push LabelIndex Prefix Adjacency1874 10.10.10.0/24 Gi3/1, 0000.0000.0013

Gi3/2, 0000.0000.0014Gi4/1, 0000.0000.0015Gi4/2, 0000.0000.0016

6500#show mls cef lookup 10.10.10.1

Codes: decap - Decapsulation, + - Push LabelIndex Prefix Adjacency1874 10.10.10.0/24 Gi3/1, 0000.0000.0013

Gi3/2, 0000.0000.0014Gi4/1, 0000.0000.0015Gi4/2, 0000.0000.0016

A BG3/1

G3/2

G4/1

G4/210.10.10.110.10.10.1

The Catalyst 6500 maintains an adjacency entry for each link that can be used to reach the destination - this can be viewed using the following command…


Catalyst 6500 Internals ECMP - The caveats

Up to 8 hardware load-sharing paths per prefix

IPv4 CEF load-balancing is per-IP flow

Per-packet load-balancing NOT supported

Load-sharing based on Source and Destination IP addresses by default

Configuration option supports inclusion of L4 ports in the hash (mls ip cef load-sharing full)

Up to 8 hardware load-sharing paths per prefix

IPv4 CEF load-balancing is per-IP flow

Per-packet load-balancing NOT supported

Load-sharing based on Source and Destination IP addresses by default

Configuration option supports inclusion of L4 ports in the hash (mls ip cef load-sharing full)


Catalyst 6500 Internals ECMP - What Link is Used?

(TCAM)

Prefix Entries Results Memory

Adjacency Table

10.7.1.0

10.3.2.0

10.1.5.0

Mask /24

10.5.1.0

11.1.1.0

10.2.6.0

Adjacency Entry #1Adj Index: 15 / Number of paths: 3

Adjacency Entry #3Adjacency Entry #4Adjacency Entry #5Adjacency Entry #6Adjacency Entry #7

Source IPDest IP

Optional L4 portsUnique ID

Load-Balancing Hash

FIB (SSRAM)Result/ Next HopResult/ Next HopAdjAdj Index Offset: 0Index Offset: 0AdjAdj IndexIndex Offset: 1Offset: 1AdjAdj IndexIndex Offset: 2 Offset: 2

New MAC and VLANNew MAC and VLANNew MAC and VLAN

Adj Idx 15: Rewrite info

New MAC and VLANNew MAC and VLANNew MAC and VLANNew MAC and VLANNew MAC and VLANNew MAC and VLAN

Adj Idx 15+2: Rewrite infoAdj Idx 15+1: Rewrite info

AdjAdj IndexIndex Offset: 1Offset: 1

Adj Idx 15+2: Rewrite info

In the FIB, a matching prefix is found - this points to an Adjacency pointer that contains the number of paths for that Adjacency Index - a hash is computed returning a result (an index offset) into the Adjacency table that points to the rewrite info and the outbound interface…


Catalyst 6500 Internals ECMP - PolarizationPolarization is an issue that occurs in a multi-hop environment where a common link is used continuously due to ECMP hashing…

“Unique ID” in Supervisor 720 prevents polarization (can be changed with ip cef load-sharing algorithm universal command)

Switch will ALWAYS chose link “L” if the inputs into the hash are the same for a packet that goes at different layers…

L R

R

R

L

L

IP: 10.5.3.2

IP: 10.22.8.17

Source IPDest IP

Optional L4 portsUnique ID

Load-Balancing Hash Result

Catalyst 6500 Internals Multicast Forwarding on the PFC3


Multicast is the act of forwarding a single packet that will reach multiple hosts - the Catalyst 6500 has a number of hardware resources that are used to facilitate Multicast forwarding in hardware…

Receivers Group B

SenderGroup A

Receivers Group A

SenderGroup B

Catalyst 6500 Internals IPv4 Multicast Forwarding


• Implements centralized and distributed IPv4 multicast hardware switching– Off-loads majority of forwarding tasks from

RP CPU• Supports PIM-SM (*,G) mroute

forwarding in hardware• Supports PIM-SM and PIM-SSM (S,G)

mroute forwarding in hardware• Supervisor 720 and Supervisor 32

support Bidir (*,G) forwarding in hardware

• Supervisor 2 and Supervisor 720 support distributed multicast packet replication


• IGMP support through Cisco IOS software– IGMP v1/v2/v3 protocol support for PIM-SM and

Bidir PIM– IGMP v3 protocol support for PIM-SSM– Option for SSM mapping to translate IGMPv2

joins to PIM-SSM joins• IGMP snooping support leveraging both hardware

and software – Snooping support for all IGMP versions– PFC performs hardware redirection of

IGMP packets to SP CPU for analysis

Catalyst 6500 Internals IGMP and Group Membership signaling

DR

Receivers



The hardware elements used to facilitate Multicast forwarding are…

FIBFIB

ADJACENCYTABLE

ADJACENCYTABLE

MULTICASTEXPANSIONTABLE (MET)

MULTICASTEXPANSIONTABLE (MET)

Contains the Multicast routes - also known as mroutes Contains the Multicast routes - also known as mroutes

Contains the rewrite information and index into MET table Contains the rewrite information and index into MET table

Contains Output Interface Lists (OIL) - list of interfaces requiring replication

Contains Output Interface Lists (OIL) - list of interfaces requiring replication

FIBFIB Contains the Multicast routes - also known as mroutes Contains the Multicast routes - also known as mroutes

RP/SP CPURP/SP CPU RP builds MROUTE Table, SP downloads into FIB RP builds MROUTE Table, SP downloads into FIB



123456

78

Adj IndexRPF VLAN

Result

Replication Engine(s)

Compare

FIB TCAMMasks Values

FFFFFFFF FFFFFFFF

10.1.1.10, 239.1.1.110.1.1.10, 239.1.1.1

10.1.1.10, 239.1.1.110.1.1.10, 239.1.1.110.1.1.10, 239.1.1.110.1.1.10, 239.1.1.110.1.1.10, 239.1.1.1

10.1.1.10, 239.1.1.1

Lookup Key

Generate Lookup

Key

S,G10.1.1.10, 239.1.1.1

Multicast Packet

MAC, MET Index

MAC, MET Index

MAC, MET Index

MAC, MET Index

Adjacency Table

1

2

3

4

S,G compares all bits in SIP

and GIP

MET

OIL #1OIL #2OIL #3OIL #4

5

10.1.1.10, 239.1.1.1

HIT!

6


100 fa4/12,fa4/19

OIF VLAN Port List

101 fa9/1

102 fa9/25,fa9/29

100 fa4/12,fa4/22,fa4/30

1019 gig1/1

4030 gig8/1

4031 gig8/3

4032 gig2/1,gig2/2,gig2/9

700 fa4/1,fa4/2,fa4/3

Index A from ADJ MET Block

Index B from ADJ

Index C from ADJ

EntryID

A

B

C

Catalyst 6500 Internals Multicast Expansion Table (MET)

ReplicationEngine MET

Catalyst 6500 Internals Multicast Replication Modes


• Replication mode refers to where in the system multicast replication occurs

• In classic system, replication always occurs centrally on the supervisor engine

• In fabric-enabled system, two possible replication modes:– Ingress replication– Egress replication


A

• Supported on Supervisor 2 (with fabric) and Supervisor 720

• Requires fabric-enabled modules• Replication load distributed—

Supervisor and switching modules perform replication

• Replication engine on ingress module performs replication for all OIFs

• Input and replicated packets get lookup on PFC or ingress DFC

• Replicated copies pass over fabric to egress modules

• Multiple MET tables, but MET on all replication engines synchronized

B

D

CSwitchFabric

Three Packets Cross Fabric

RE

RE = Replication Engine

RE

RE

RE

Catalyst 6500 Internals Multicast Ingress Replication Model


A

D

C

• Supported on Supervisor 720 with certain switching modules only (CEF720, 6516A, 6548-GETX, SIPs)

• Replication load distributed—Supervisor and switching modules perform replication

• All modules in chassis must be egress-capable • Egress mode not optimized unless DFCs present

on modules• Input packets get lookup on ingress

DFC, replicated packets get lookup on egress DFC

• For OIFs on ingress module, local engine performs the replication

• For OIFs on other modules, ingress engine replicates a single copy of packet over fabric to all egress modules

• Engine on egress module performs replication for local OIFs

• MET tables on different modules can be asymmetric

B

SwitchFabric

One Packet Crosses Fabric

RE = Replication Engine

RE

RE

RE

RE

Catalyst 6500 Internals Multicast Egress Replication Model

Catalyst 6500 Internals Reference: Additional Forwarding Entry Details


• FIB TCAM is a shared resource for IPv4/v6 unicast, IPv4/v6 multicast, and/or MPLS

• Each multicast FIB entry consumes at least two physical FIB TCAM entries

• Upper limit reserved for IP multicast is configurable in PFC3– mls cef maximum-routes ip-multicast <entries>

• Adjacency table is also a shared resource• Each multicast forwarding entry consumes at least one adjacency entry

– PFC2 supports 256K hardware adjacencies– PFC3 supports 1M hardware adjacencies

• MET contains up to 64K OIFs (central or ingress replication) or (64K * number of forwarding engines) OIFs (egress replication)

• Hardware Bidir-PIM RP-to-DF table supports up to four RPs per VRF


CHAPTER 8IPv6 on the Catalyst 6500

IPv6 Coexistence in the Enterprise


IPv6 Network

IPv6 Network

IPv6 Host

Configured/6to4 Tunnel

Configured/6to4 Tunnel

IPv6 Host

IPv4

IPv4: 192.168.99.1

IPv6: 2001:db8:1::1/64IPv6/IPv4

Dual Stack

IPv6

NAT-PT

IPv6ISATAPRouter

IPv4 ISATAP Tunneling(Intra-Site Automatic Tunnel Addressing Protocol)

IPv4 only Server

IPv4-Only Segment

IPv6 on the Catalyst 6500 IPv6 Support on the PFC3


The PFC3 supports the following IPv6 features in hardware…

IPv6 Hardware Support on the PFC3IPv6 Hardware Support on the PFC3FIB Forwarding based on v6 destination addressFIB Forwarding based on v6 destination address

Full Netflow classification and policingFull Netflow classification and policingNetflow statistics and forwardingNetflow statistics and forwarding

ACL lookups using IPv6 SRC/DST addressingACL lookups using IPv6 SRC/DST addressingQoS lookups using IPv6 SRC/DST addressingQoS lookups using IPv6 SRC/DST addressing

Link Local and Site Local AddressingLink Local and Site Local Addressing

Up to 128k IPv6 routes on the PFC3A, PFC3B and PFC3C

Up to 128k IPv6 routes on the PFC3A, PFC3B and PFC3C

Up to 512k IPv6 routes on the PFC3BXL and PFC3CXL

Up to 512k IPv6 routes on the PFC3BXL and PFC3CXL

6-to-4 and Automatic Tunneling6-to-4 and Automatic Tunneling


CHAPTER 9VRF/MPLS on the Catalyst 6500


VRF/MPLSNetwork Virtualization Solution Overview

802.1x Identity

NAC/CCA

MAC Auth Bypass

Web Based Proxy Auth

VLAN/.1Q GRE

VRF-Lite

MPLS ACL

Firewall

Content Switching (ACE)

Policy Management

Guest VLAN

Research VLAN

Admin/Faculty VLAN

Quarantine VLAN

Authentication VLAN

IT Staff VLAN

Guest VLAN

Research VLAN

Admin/Faculty VLAN

Quarantine VLAN

Authentication VLAN

IT Staff VLAN

Distribution Layer

Access Layer

Data Center

Research Virtual

Domain

GuestVirtual

Domain

Admin/Faculty Virtual

Domain

IT StaffVirtualDomain

AuthenticationVirtualDomain

QuarantineVirtualDomain

Core Layer

Services Edge

Access Control

Path IsolationSiSi SiSi

SiSi SiSi SiSi SiSi

SiSi SiSi SiSi SiSi


VRF/MPLSWhat is a VRF (Virtual Routing and Forwarding)?

• Typically all route processes and static routes are populating one routing table

• All interfaces are part of the global routing table

router eigrp 1network 10.1.1.0 0.0.0.255

!router ospf 1network 10.2.1.0 0.0.0.255 area 0

!router bgp 65000neighbor 192.168.1.1 remote-as 65000

!ip route 0.0.0.0 0.0.0.0 140.75.138.114

global routing table


VRF/MPLSWhat is a VRF (Virtual Routing and Forwarding)?

• VRFs allow dividing up your routing table into multiple virtual tables

• Routing protocol extensions allow binding a process/address family to a VRF

• Interfaces are bound to a VRF usingip vrf forwarding <vrf-name>

router eigrp 1network 10.1.1.0 0.0.0.255

!router ospf 1 vrf orangenetwork 10.2.1.0 0.0.0.255 area 0

!router bgp 65000address-family ipv4 vrf blue…

!ip route vrf green 0.0.0.0 0.0.0.0 …

global routing table


VRF/MPLSHow are VRFs used?

VRFs can be used in conjunction with VRF-lite or MPLS VPN

MPLS VPNsVRF-lite(aka Multi-VRF CE)

VLAN ID

L2 Header IP SRC PAYLOADIP DST802.1q

TAG

MACDST

MACSRC

ETHERTYPE0x8100

802.1pCoS CFI Label

(VPN ID)MACDST

MACSRC

ETHERTYPE0x8847

L2 Header

MPLSLabel IP SRC PAYLOADMPLS

Label IP DST

EXP TTLS

MPLS802.1q

• Defines from which VRF traffic was sourced / for which VRF traffic is destined• FIB table needs to have this information for each prefix


VRF/MPLSDeployment Examples of VRFs and MPLS

Guest Access Network Virtualization

(m)GRE + VRF-lite VRF-lite (end-to-end) MPLS VPN

P

PE PE

P

P P

PE PE

VRF/MPLSIntroduction to VRF (Virtual Route Forwarding)


All routes learned are, by default, placed into a consolidated routing table, which is referred to as the “Global” routing table…

Global RoutingTable

Global RoutingTable

Routes10.1.2.010.2.1.010.5.3.0

10.1.2.010.2.1.010.5.3.010.6.3.010.6.7.010.9.2.0

10.6.3.010.6.7.010.9.2.0

Routes

VRF/MPLSIntroduction to VRF (Virtual Route Forwarding)


VRF allows multiple “virtual” routing tables to exist on the 6500 at any one point in time - each VRF table contains a unique set of routes - routing protocol instances can be bound to different VRFs…

VRF #1VRF #1

Routes

Routes

OSPF #110.2.1.010.5.3.0

OSPF #210.4.6.010.5.1.010.9.2.010.7.5.0

VRF #2VRF #2

OSPF #110.2.1.010.5.3.010.6.7.010.1.8.0

OSPF #210.4.6.010.5.1.0

OSPF #110.6.7.010.1.8.0

OSPF #210.9.2.010.7.5.0


VRF/MPLSWhere is the VRF Information Stored?

Sup720

MSFC3

Switch Processor (SP)takes RIB and programs the PFC FIB with this

information - holds VRF to VPN Map Table

Switch Processor (SP)takes RIB and programs the PFC FIB with this

information - holds VRF to VPN Map Table

Route Processor(RP)

Route Processor(RP)

DRAM holds Routing Information Base (RIB)

PFC3

Tycho (Layer 3 Engine)Tycho (Layer 3 Engine)

Superman (Layer 2 Engine)Superman (Layer 2 Engine)

FIB holds IPv4 Routes as a prefix and

associates a VPN ID with each prefix Runs VRF Aware Routing Protocols


VRF/MPLSHow VRFs are stored in the FIB

1000

When VRFs are enabled, the VRF number (or VPN number) is loaded with the prefix entry in the FIB for each prefix that exists in the forwarding table

1000 XTAGXTAG

IPv4 Destination Lookup FIB Entry

VPN #VPN # FlagsFlags ReservedReserved TOSTOS IPv4 Destination ADDRIPv4 Destination ADDR

IPv4 Address in FIB is associated with a VPN IDwhich represents the VRF that the prefix is a part of…

6500-2#remote command switch show platform software vpn mapping

Type | VRF Name | Table id || HW table id | App Bitmask | App Data mask | Reference --------+-------------+----------++--------------+-------------+---------------+-----------IOS | Default | 0 || 0 | 0x00000031 | 0x00000000 | IOS | Admin | 1 || 257 | 0x00000001 | 0x00000000 | R[0]:1 IOS | Engineering | 2 || 256 | 0x00000001 | 0x00000000 | R[0]:1 IOS | Marketing | 3 || 258 | 0x00000001 | 0x00000000 | R[0]:1

6500-2#remote command switch show platform software vpn mapping

Type | VRF Name | Table id || HW table id | App Bitmask | App Data mask | Reference --------+-------------+----------++--------------+-------------+---------------+-----------IOS | Default | 0 || 0 | 0x00000031 | 0x00000000 | IOS | Admin | 1 || 257 | 0x00000001 | 0x00000000 | R[0]:1 IOS | Engineering | 2 || 256 | 0x00000001 | 0x00000000 | R[0]:1 IOS | Marketing | 3 || 258 | 0x00000001 | 0x00000000 | R[0]:1

A VRF name to VPN-ID mapping table is maintained by the control plane


VRF/MPLSVRF Forwarding Process

10.1.0.0/VPN 4172.16.0.0/VPN 67

…

172.20.45.1/VPN 610.1.1.100/VPN 22

…10.1.3.0/VPN 210.1.3.0/VPN 8

…

10.0.0.0/VPN 3

MASK (/24)

MASK (/16)

MASK (/32)

MASK (/8)

PacketPacket11

22 Key GenKey Gen22 Key GenKey Gen

33 Lookup KeyLookup Key

IF, MACs, MTU

IF, MACs, MTU

IF, MACs, MTU

IF, MACs, MTU

Load-SharingHash

44 55 66 66

When the lookup is initiated, the VPN ID is used as input to the key gen for the lookup key - as the same routes can exist in different VRFs, it important that the VPN is also considered in the forwarding operation to ensure the packet is correctly forwarded in the right VRF

HIT!

In this case we are looking for a forwarding entry for destination address 10.1.3.5 in VRF Engineering (VPN-ID 2)…

Adjacency Table

FIB TCAM


VRF/MPLSVRF Features on the 6500

VRF-aware feature IOS release

Import/export from global Whitney-1.0HSRP 12.2(17d)SXBNetflow*** 12.2(18)SXE

Unicast traffic forwarding 12.2(17d)SXBMulticast traffic forwarding* 12.2(18)SXEDirected broadcast fwd 12.2(18)SXEStatic routes 12.2(17d)SXBRIP (v1 and v2) 12.2(17d)SXBEIGRP 12.2(18)SXEOSPF** 12.2(17d)SXBeBGP 12.2(17d)SXB

• Software Modularity images support VRFs as of 12.2(18)SXF5.• MPLS support is committed for 12.2(33)SXH

* Multicast forwarding is HW based. This is applicable for VRF-lite as well as MPLS VPN (mVPN) scenarios. As of February 2007, the 3750 and 4500 do not support multicast on VRF interfaces

** For more than 28 OSPF processes the minimum release is 12.2(18)SXE*** Ingress Netflow on a CE-to-PE link

VRF-aware feature IOS releaseTelnet 12.2(17d)SXB

IPSec (VPNSM based) 12.2(18)SXD1IPSec (SSC/SPA based) 12.2(18)SXE2NAT (FWSM based) 12.2(17d)SXBStatic ARP entries 12.2(17d)SXB

IP SLA Whitney-1.0Standard/Extended ACLs 12.2(18)SXD

Ping 12.2(17d)SXBTraceroute 12.2(17d)SXBTFTP/FTP 12.2(17d)SXBSYSLOG Whitney-1.x


VRF/MPLSVRF Feature Roadmap on the 6500

• This is a list of features that are not VRF-aware yet but are being considered for future releases on the Catalyst 6500

VRF-aware featureAAACoPPDHCP RelayDynamic ARP inspection (DAI)IS-ISNTPOSPFv3PBR set VRF

RCP/SCPSNMP Access RestrictionsSSHTACACSuRPFVRF aware PBRVRRP/GLBPWCCP


VRF/MPLSIntroduction to MPLS Forwarding

LSR LSR

LSR LSR

LER LER

Basic MPLS forwarding involves two types of devices - Label Edge Router and Label Switch Router -these devices will use MPLS labels to determine next hop forwarding - LER is also able to push (add) and pop (remove) labels…

MPLS Core 172.168.1.0/24

In I/FIn I/F

11

In LabelIn Label

NANA

ADDR PrefixADDR Prefix

172.168.1172.168.1

Out I/FOut I/F

33

Out LabelOut Label

55

In I/FIn I/F

11

In LabelIn Label

55

ADDR PrefixADDR Prefix

172.168.1172.168.1

Out I/FOut I/F

22

Out LabelOut Label

7 7

LFIB Lookup Key is in-labelFIB Lookup Key is prefix

VRF/MPLSIntroduction to MPLS VPN


An MPLS VPN network can loosely be defined as one where multiple virtual networks exist in single physical network topology each with their own network topology, security and access policy…

VPN1

VPN1

VPN2

VPN2

MPLS VPN Backbone

P

P P

P

PE

CE

CE

CE

CE

PE

PE

CE - Customer EdgePE - Provider Edge = Label Edge Router (LER)P - Provider = Label Switch Router (LSR)


VRF/MPLSIntroduction to MPLS Header

MPLS LabelMPLS Label EXPEXP SS TTLTTL

An MPLS header is a 32 bit header that sits between a Layer 2 and Layer 3 header in the packet…

MPLS Header ( 32 Bits)

Layer 2 Header

MPLS Label - 20 bitsExperimental bits (equivalent to Class of Service) - 3 bitsBottom of Stack - 1 bitTime to Live - 8 bits

Layer 2 Header Label 6Label 6 Label 5Label 5 Label 4Label 4 Label 3Label 3 Label 2Label 2 Label 1Label 1 Layer 3 HeaderLayer 3 Header

MPLS Label Stack


VRF/MPLSSup720 Resources - RIB/FIB/LIB/LFIB Linkage

RoutingInformation Base

(RIB)


(RIB)

RoutingProtocolsRouting

Protocols

LabelInformation Base

(LIB)

LabelInformation Base

(LIB)

Learn Routes from

Routing peers

Control Plane

Data Plane

ForwardingInformationBase (FIB)

ForwardingInformationBase (FIB)

Label bindings learned

from LDP peers

LabelForwardingTable (LFIB)

LabelForwardingTable (LFIB)

OutgoingIP Packet

IncomingIP Packet

IncomingMPLS Packet

OutgoingMPLS Packet


(RIB)


(RIB)

RoutingProtocolsRouting

Protocols


VRF/MPLSSup720 MPLS/VRF Resources

Sup720

MSFC3

Switch Processor (SP)Takes RIB/LIB and

programs the PFC FIB with this information

Switch Processor (SP)Takes RIB/LIB and

programs the PFC FIB with this information

Route Processor(RP)

Route Processor(RP)

DRAM holds Routing

Information Base (RIB) and Label

Information Base (LIB)

PFC3



FIB holds IPv4 Entries and MPLS Entries - MPLS entries logically part of the Label Forwarding Table (LFIB)

Adjacency Table holds rewrite info (MAC) for IPv4 and label info for pushing/popping MPLS labels

VLAN RAM maps VLAN to VPN ID

VPN CAM maps MPLS Label to an index which is used as a lookup key into the FIB

MPLS VPN RAM maps VPN CAM lookup results into a VPN and TOS values used as a lookup into the FIB


VRF/MPLSFIB Entries and MPLS

1000

When VRFs are enabled, the VRF number (or VPN number) is loaded with the prefix entry in the FIB for each prefix that exists in the forwarding table

1000 XTAGXTAG

IPv4 Destination Lookup FIB Entry - used during imposition

VPN #VPN # FlagsFlags ReservedReserved TOSTOS IPv4 Destination ADDRIPv4 Destination ADDR

IPv4 Address in FIB is associated with a VPN IDwhich represents the VRF of which that prefix is a part…

10111011 XTAGXTAG

MPLS Lookup FIB Entry - used when switching labels

VPN #VPN # FlagsFlagsMPLS Label 1MPLS Label 1

LabelLabel EXPEXP SSVV

MPLS Label 0MPLS Label 0

LabelLabel EXPEXP SS

VRF/MPLSSup720 Resources - Adjacency Table Entries


Adjacency Table entries exist for MAC and MPLS packets

FlagsFlags MAC DestinationAddress

MAC DestinationAddress

MAC SourceAddress

MAC SourceAddress

Used for- Layer 2 Encapsulation Change- Destination/Source MAC

Address Rewrite

MAC Adjacency Entry

FlagsFlags MAC DestinationAddress

MAC DestinationAddress

MAC SourceAddress

MAC SourceAddress

Used for- Add Label/ Remove Label/Replace Label- Destination/Source MAC

Address Rewrite

MPLS Adjacency Entry

Label 2Label 2 Label 1Label 1 Label 0Label 0

NOTE: The PFC can push (add) 3 labels or pop (remove) up to 2 labels in a single lookup operation


VRF/MPLSWhere does the HW get involved with MPLS?

PE PE

P

P

IPIP DataData

LBL=10LBL=10 IPIP DataData

IPIP DataData

LBL=20LBL=20 IPIP DataData

1. Pushing (Imposition of) labels onto IP packets as they enter the MPLS Network

3. Popping (disposition of) labels from MPLS tagged packets as they leave the MPLS Network

2. Switching of labeled packets (and swapping labels) when they are within the MPLS Network


VRF/MPLSLabel Switching Routing

Label FIBLabel FIBIn LabelIn Label Out LabelOut Label

3030 5050

L2 HDRL2 HDR MPLS Label (30)MPLS Label (30) IP SRCIP SRC IP DSTIP DST DataData L2 HDRL2 HDR MPLS Label (50)MPLS Label (50) IP SRCIP SRC IP DSTIP DST DataData

Normal Label Switching typically involves performing a label swap -

the incoming packet label is used as a lookup into the LFIB to determine

the outgoing label to be used…

Label swapped


VRF/MPLSLabel Edge Routing - Pushing (adding) a Label

HW FIB TableHW FIB TableVPN IDVPN ID IP FIBIP FIB

00 10.1.1.010.1.1.0

L2 HDRL2 HDR IP SRCIP SRC IP DSTIP DST DataData L2 HDRL2 HDR MPLS Label (40)MPLS Label (40) IP SRCIP SRC IP DSTIP DST DataData

Normal Label Edge Routing typically involves adding (pushing) or removing (popping) a label…

The following example shows a label being added…

HW ADJ TableHW ADJ TableInterfaceInterface Out LabelOut Label

G3/1G3/1 4040

Label added (pushed)


VRF/MPLSLabel Edge Routing - Popping (removing) a Label

ACTUALLY - BEFORE WE GO INTO THAT LETS LOOK AT SOME OTHER FACTORS WHICH IMPACT HOW THE

HARDWARE WILL POP LABELS

ACTUALLY - BEFORE WE GO INTO THAT LETS LOOK AT SOME OTHER FACTORS WHICH IMPACT HOW THE

HARDWARE WILL POP LABELS


VRF/MPLSPFC3 VPN Capacity

Supported VPN Upper Limit

Supported VPN Upper Limit

Number of VPN’s with

Optimal Performance

Number of VPN’s with

Optimal Performance

Hardware Max # of VPN’s

Hardware Max # of VPN’s

LFIB Entries

(256K FIB)

LFIB Entries

(256K FIB)

LFIB Entries

(1M FIB)

LFIB Entries

(1M FIB)

10241024 511511 40954095 256K256K 1M1M

The sum of routes in all VRFs must be lower than the total FIB TCAM capacity.

Example:10 VRFs with 3,000 routes each and global table with 20,000 routes10* 3,000 + 20,000 = 50,000 FIB TCAM entries

If the VRFs are transported over an MPLS VPN network the user also needs to account for the Label Information Base (LIB) see architecture paper.


VRF/MPLSWhy only 512 VRF’s?

Sup720

MSFC3

Switch Processor (SP)Switch Processor (SP)

Route Processor(RP)

Route Processor(RP)

PFC3



FIB

Adjacency Table

VLAN RAM

VPN CAM is the VPN to VLAN map table and holds 512 entries !!!!

MPLS VPN RAM


VRF/MPLSAggregate Labels and VPN CAM

An aggregate label is assigned by an LER for directly connected or summarized routes - aggregate labels indicate the arriving packet should have the label popped and then use the underlying IP prefix to make the next forwarding decision…

Label 5

Label 8

PFC

Label 22

PFC

Layer 2 EngineLayer 2 Engine

Layer 3 EngineLayer 3 Engine

VPN CAMVPN CAMLabelLabel IndexIndex

55 3388 12122222 3131…… ……

A hit in the VPN CAM will provide a index key for the PFC to use when looking up the IP prefix in the FIB

The VPN CAM can only hold 512

entries so maximum MPLS lookup

performance is achieved when # VRFs is < 512…

NOTE: One entry is reserved for the explicit

null entry hence why performance is optimized

for 511


VRF/MPLSLabel Edge Routing - Popping an Aggregate Label

For MPLS aggregate labeled packets and (VPN CAM entries < 512) transiting an LER to be switched as a normal IP packet, the following happens …

L2 HDRL2 HDR MPLS Label 40MPLS Label 40 IP SRCIP SRC IP DSTIP DST DataData


88 552626 18184040 3636…… ……

Layer 2 Engine Layer 3 Engine

MPLS Tag removed and the underlying IP prefix passed to next stage

MPLS Tag removed and the underlying IP prefix passed to next stage

MPLSVPN RAM

MPLSVPN RAMIndexIndex VPNVPN

2121 772929 18183636 112112…… ……

FIBFIB ADJ TableADJ Table10.3.1.0/VPN8610.3.1.0/VPN86 MAC/VLANMAC/VLAN

MAC/VLANMAC/VLANMAC/VLANMAC/VLANMAC/VLANMAC/VLAN

10.3.1.0/VPN010.3.1.0/VPN010.3.1.0/VPN11210.3.1.0/VPN11210.4.1.0/VPN9110.4.1.0/VPN91

….….

L2 HDRL2 HDR IP SRCIP SRC IP DSTIP DST DataData


VRF/MPLSLabel Edge Routing - Popping a Label

For MPLS aggregate labeled packets when the VPN CAM is full (i.e. > 512 entries) transiting an LER to be switched as a normal IP packet, the following happens …

L2 HDRL2 HDR MPLS Label 729MPLS Label 729 IP SRCIP SRC IP DSTIP DST DataData


88 552626 1818

511511 612612

Layer 2 Engine Layer 3 Engine

VLAN RAMVLAN RAMVLANVLAN VPNVPN

5858 9292241241 657657816816 82282213641364 00

FIBFIB ADJ TableADJ TableDEF/VPN522DEF/VPN522 MAC/VLANMAC/VLAN

POP LabelPOP LabelMAC/VLANMAC/VLANMAC/VLANMAC/VLAN

10.5.1.0/VPN42210.5.1.0/VPN422

Use input port VLAN as inputUse input port VLAN as input


ABC/VPN822ABC/VPN82210.4.1.0/VPN2510.4.1.0/VPN2510.5.1.0/VPN010.5.1.0/VPN0

….….MISSMISS

RECIRCULATE PACKETRECIRCULATE PACKET


VRF/MPLSOther Recirculation Instances

Other examples where recirculation is required is when more than three labels are pushed or more than two labels are popped…


L2 HDRL2 HDR LBL 1LBL 1 LBL 2LBL 2 LBL 3LBL 3 LBL 4LBL 4 IP SRCIP SRC IP DSTIP DST DataData

L2 HDRL2 HDR LBL 1LBL 1 LBL 2LBL 2 LBL 3LBL 3 IP SRCIP SRC IP DSTIP DST DataData


L3Engine

L3Engine

L3Engine

L3Engine

Pushing > 3 labels

Popping > 2 labels


VRF/MPLSMPLS and Load Balancing

L2 HDR

The hardware supports Layer 3 load balancing options in the following manner for MPLS tagged packets…

L2 HDR LBL 1LBL 1 IP SRCIP SRC IP DSTIP DST DataData

L2 HDRL2 HDR LBL 1LBL 1 LBL 2LBL 2 IP SRCIP SRC IP DSTIP DST DataData

L2 HDRL2 HDR LBL 1LBL 1 LBL 2LBL 2 IP SRCIP SRC IP DSTIP DST DataDataLBL 3LBL 3

L2 HDRL2 HDR LBL 1LBL 1 LBL 2LBL 2 IP SRCIP SRC IP DSTIP DST DataDataLBL 3LBL 3 LBL 4LBL 4

L2 HDRL2 HDR LBL 1LBL 1 LBL 2LBL 2 IP SRCIP SRC IP DSTIP DST DataDataLBL 3LBL 3 LBL 4LBL 4 LBL 5LBL 5

SRC/DST IP address used for load balancing when underlying packet type is IPv4 and up to 3 MPS labels appear in the packet…

The 4th Label is used for load balancing when 4 labels exist in the packet…

Packets with 5 or more labels - the 5th Label is used for load balancing…

L2 HDRL2 HDR LBL 1LBL 1 LBL 2LBL 2 IP SRCIP SRC IP DSTIP DST DataDataLBL 3LBL 3The bottom most label is used for load balancing when 3 or less labels exist in the packet and packet is NOT IPv4…


VRF/MPLSMPLS and Load Balancing

L2 HDR

The hardware supports Etherchannel load balancing options in the following manner for MPLS tagged packets…

L2 HDR LBL 1LBL 1 IP SRCIP SRC IP DSTIP DST DataData

L2 HDRL2 HDR LBL 1LBL 1 LBL 2LBL 2 IP SRCIP SRC IP DSTIP DST DataData

L2 HDRL2 HDR LBL 1LBL 1 LBL 2LBL 2 IP SRCIP SRC IP DSTIP DST DataDataLBL 3LBL 3

L2 HDRL2 HDR LBL 1LBL 1 LBL 2LBL 2 IP SRCIP SRC IP DSTIP DST DataDataLBL 3LBL 3 LBL 4LBL 4

L2 HDRL2 HDR LBL 1LBL 1 LBL 2LBL 2 IP SRCIP SRC IP DSTIP DST DataDataLBL 3LBL 3 LBL 4LBL 4 LBL 5LBL 5

SRC/DST IP address used for load balancing when underlying packet type is IPv4 and up to 3 MPS labels appear in the packet…

The lowest two labels are used for load balancing when 4 labels exist in the packet…

Packets with 5 or more labels - the 4th and 5th Label is used for load balancing…

L2 HDRL2 HDR LBL 1LBL 1 LBL 2LBL 2 IP SRCIP SRC IP DSTIP DST DataDataLBL 3LBL 3The lowest two labels are used for load balancing when 3 or less labels exist in the packet and packet is NOT IPv4…


VRF/MPLSMPLS Features supported

IOS release Feature12.2(17d)SXB(Tetons)

12.2(18)SXD(Rockies-1)

12.2(18)SXD1(Rockies-1A)12.2(18)SXE (Rockies-2)

12.2(18)SXF(Rockies-3)

MPLSoGRE on SIP-400Virtual Private LAN Service (VPLS) on SIP-600

Future MPLS IONization, Extranet mVPN, 6VPE, MPLS HA

P & PERFC2547 VPNs, OSPF, OSPF Sham link, RIP, Statics, eBGPCarrier Support Carrier, Inter-AS VPNsAny Transport over MPLS – EoMPLS, FRoMPLS, ATMoMPLSVirtual Private LAN Service (VPLS) on OSMs6PEMIBs LSR, LDP, VPNTraffic Engineering (OSPF, ISIS with interarea support)DiffServ Aware Traffic EngineeringPE-CE EIGRP IP Event DampeningMIBs TETraffic Engineering Fast ReRouteVPN Aware IPSec with VPNSM (ASWAN 2.0)Multicast VPN (Limited Release MVPN in 12.2(17d)SXB6)PPP/HDLCoMPLSMPLS OAM (Ping, Traceroute, VCCV)Hierarchical Shaping for WAN EoMPLSOSPF more than 28 processes


CHAPTER 10Catalyst 6500 Control Plane Features

dubai partner bootcamp part1 without password protection

Documents

catalyst express

expected catalyst

e new catalyst

ios architecture chapter

security chapter

ipv6 chapter

supervisors chapter

qos chapter