Download - Best Practice Design for Campus Networks
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF
March 25 – 27th, 2014 І Orlando, FL
#AvayaATF© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya.
Best Practice Design for Campus NetworksSteve Emert, Avaya
Keith Nuehring, City of Cedar Rapids
#AvayaATF
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF#AvayaATF
Best Practice Design for Campus Networks
City of Cedar Rapids Fabric Connect Case Study
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 3
Agenda
Campus Best Practices Design with Fabric Connect Deploying Fabric Connect in the Campus Design Options with Compact Form Factor Switches Capabilities to Ease Integration with Conventional Networks Models to Extend the Fabric to the Wiring Closet Edge
City of Cedar Rapids, Iowa case studyKeith Nuehring, IT Operations Manager, City of Cedar Rapids Cedar Rapids network before SPB and Fabric Connect Considerations and the decision to move to Fabric Connect Network Redesign Goals and Objectives Planning and Staging the Network Cutover Cutover Weekend Observations and Lessons Learned
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 4
Best Practice Network Designs for the Campus
First…. The sales pitch… not that you haven’t heard it before! Use Fabric Connect! Whether a small/medium enterprise, large campus network, or a campus
network that is distributed across a city or even a larger geography Why?
It will make your life easier It will free up your time and your staff’s time to work on more strategic
projects It will allow you to support new initiatives
improving your business – improve time to service
It will help simplify your efforts to maintain PCI DSS or other security compliance requirements or regulations
It will save you time and money in operational expenditures
It will make your network run more smoothly
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 5
Deploying Fabric Connect in the Campus
What SPB Network services to use, and where … Layer 2 VSNs
Stretching VLANs across the campus Rack to Rack, Row to Row, Data Center to Data Center for VM Moves Special purpose L2 networks
Totally constrained networks if no IP interface created on BEBs Routable if IP interface configured
STP BPDUs not transported across ISID STP becomes a “local construct” only for edge protection
BEB BEBBCBBCB
VLAN 200
I-SID200
VLAN 200
I-SID200
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 6
Deploying Fabric Connect in the Campus
What SPB Network services to use, and where … IP Shortcuts (GRT Route Redistribution to ISIS)
One-hop IP routing across the fabric Eliminates “transit” IP Subnets, simplifying the routing table Enhances security – end users cannot determine “core” routers by
workstation Traceroute Typical Uses
Simplest migration from conventional IP routed network to Fabric Connect
Simply enable route redistribution to SPB/ISIS
BEB BEBBCBBCBVLAN 100
VLAN 300
VLAN 400
VLAN 200
Route redistributionacross ISIS
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 7
Deploying Fabric Connect in the Campus
What SPB Network services to use, and where … Layer 3 VSNs
Multiple isolated/segregated IP routed networks within a single fabric infrastructure More efficient than conventional VRFs with a single instance of the routing
protocol (ISIS for SPB), single LSDB with info for all VRFs Typical Uses
Fully routed private networks for security segregation/isolation PCI DSS, IP Video surveillance networks, SCADA or HVAC Mergers and acquisitions, partnerships – duplicated IP addresses are allowed
when in different VRFs
BEB BEBBCBBCB
VLAN 100
VLAN 200
VLAN 300
I-SID500
I-SID500
VRF VRF
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 8
Deploying Fabric Connect in the Campus What SPB Network services to use, and where
ERS 4800Unicast only
ERS 4800Multicast
Clients
ERS 4500or
4500/4800
ERS 5000
VSP 4450 /4850VSP 8284
L2 VSN
IP Shortcuts
NNI
NNI
VSP 7024
IP Shortcuts
L2 VSN
Q Tagged
NNI
Q Tagged
NNI
VSP 7024
IP Shortcuts
IP Shortcuts
L2 VSN
IP Shortcuts
Q Tagged
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 9
Design Options with Compact Form Factor Switches
VSP 8284
VSP 7024
VSP 4850
VSP 4450
ERS 4800
Port Capacity Fabric Services
80 10Gig, 4 40Gig BCB, L2 VSN, IP Shortcuts,L3 VSN in 2015
24 10Gig (SFP+, 10GBase-T), 8 port MDA
BCB, L2 VSN
48 UTP 10/100/1000, 2 shared SFP, 2 10Gig SFP
BCB, L2 VSN, IP Shortcuts, L3 VSN
12 UTP, 36 SFP, 2 10Gig SFP
BCB, L2 VSN, IP Shortcuts,L3 VSN
48 UTP 10/100/1000 PoE+ or non PoEStackable
L2 VSN only
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 10
Fabric Connect Deployment Best Practices
Establish a consistent, understandable and useful naming/numbering system Backbone VLAN IDs 4051 and 4052 – per informational RFC System IDs – 00xx.xxxx.xx00
Stay away from first two hex digits and last two hex digits
Building or network . Subnet or other identifier . Switch number in location Example: 0001.07a1.0100
Nick-Names (ISIS Source Address) – x.xx.xx Base it on the System ID for ease of correlation
Example: 1.a1.01 System names (CLI Prompt names)
Meaningful, avoid special characters – make it easy to type!
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 11
Fabric Connect Deployment Best Practices
Ensure STP/MSTP/RSTP is disabled on NNI ports Default behavior on most switches Faster convergence when STP does not interfere with NNI
Avoid putting CVLANs on NNI ports Except where needed during conventional network to Fabric
migration Network Edge connectivity and protection
Spanning Tree Fast Learning or MSTP Edge-Port SLPP and SLPP Guard
Can use SLPP even in some non-traditional locations VLACP
Use is optional, but can improve NNI failover times Can specify Ethertype to be unique for the specific link
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 12
Capabilities to Ease Integration with Conventional Networks
Current capabilities Split BEB for SMLT compatibility 802.1Q tagged interfaces to non-SPB switches Transparent UNI Route redistribution between ISIS/SPB and other IGPs
Upcoming capabilities Virtual IST Fabric Attach Fabric Connect over IP SFP Adapter
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 13
Route Redistribution between SPB/ISIS and Other IGPs
Avaya SPB Domain
IP RoutedDomain
Avaya SPB Domain
Layer 3 Switch
Layer 2 Switch
OSPF/RIP/BGP enabled on external facing interfaces.
Redistribute ISIS Routes into other IGPs:ip <rip|ospf|bgp> redistribute isis <create|enable|apply>
Redistribute direct/static/other IGP routes into ISIS:ip isis redistribute <direct|static> <create|enable|apply>ip isis redistribute <rip|ospf|bgp> <create|enable|apply>
Layer 2 Switch
Edge VLANs/IP Subnets are present on VLAN interfaces within the SPB BEB
switch, associated with ISIDs
Redistribute Direct to ISIS
Layer 3 Switch
RIP or OSPF used to exchange routes with external routing switch
Redistribute RIP or OSPF to ISIS
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 14
Virtual IST for SMLT (vIST) Virtual IST Concepts
Delivers SMLT/RSMLT w/ virtual IST capability. Ability to run IST over SPB fabric. Removing the
need for direct links between IST peers. Can be deployed as conventional SMLT/RSMLT solution with direct IST links.
First phase: configure ISIS and SPB & IST protocol between a cluster pair
Virtual IST Benefits Further increased resiliency & more flexible
routing for IST connectivity. (no need for direct IST trunk)
Allows mixing of IST node types (VSP8k, VSP4k, …)
Future benefits for FA and distributed LAG Platform Implementation Timelines
VSP8k Release 4.0 1H2014 VSP4k Release 4.1 2H2014
Seamless & Painless Scalability
Virtual IST
Single, Unified, Logical Core
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 15
Models to Extend the Fabric to the Wiring Closet Edge
ERS 4800 SPB L2 VSN
Fabric Attach
L2 VSN
NNI
NNI
ERS 5600
FA Client
FA Hosts
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 16
Fabric Attach (FA)
Concepts Automatic attachment of non-fabric switches (e.g.
ERS 56xx) and hosts/devices (Servers, Cameras, APs) to Fabric Connect networks.
Introduces FA Server, FA Switch & FA Devices Uses signaling protocol to signal VLAN/ISID
memberships between attached hosts and non-fabric switches/devices to FA Server switches (BEBs).
First step to Zero-Config-Edge: Establishes node connectivity for FA hosts/devices out of the box and auto attaches it to fabric
Timelines (solution more widely available in 2015) FA Server: VSP7k, ERS4k demo now, VSP8k,
VSP9k, VSP4k 2015 FA Switch: ERS4k, ERS5k demo now FA Devices: TBD
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 17
Fabric Attach Extends the Benefits of Fabric to the Wiring Closet & Network Attached Devices
Customer Value Automated identification and provisioning of
end points (e.g., wireless AP’s and cameras) Simplified network provisioning for devices
outside the Fabric Reduction in network configuration errors Simplifies adds, moves, and changes
Technology Builds on top of Fabric Connect architecture Extends Fabric benefits to non-Fabric
Connect platforms AND endpoints / users Client/Device identification, authentication
and authorization via Identity Engines
Fabric ConnectSwitch Stack of ERS 4800
Fabric AttachSwitch ERS 4800
FabricAttach
FabricAttach
Employee Zone
Contractor Zone
Management Zone
Guest Zone
Authentication &Authorization
Fabric AttachSwitch ERS 5600
Authentication &Authorization
Fabric AttachServer ERS 4800
DHCP
Fabric AttachAccess Point
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 18
Fabric Connect SPB over IP
Concepts Solution allows extension of Avaya’s Fabric Connect fabrics
over IP networks. Introduces 100Mbps/1Gbps SFP compatible adapter which
tunnels Ethernet VLANs over IP. The SFP “sleeve” can be inserted into an SFP/SFP+ NNI port
and is used in conjunction with any supported SFP (sorry, no 10Gig SFP+).
SFP adapter provides IP tunneling capabilities for SPB NNI connections by adding additional IP tunnel header to the SPB MAC-in-MAC packets.
IP connection MTU requirements: 1582 to 2000 bytes – work with the carrier to ensure compatibility!
Benefits Allows extending SPB/Fabric Connect over an IP network. IP
network can be campus backbone or MAN/WAN IP MPLS network.
Full fabric capabilities remain intact over IP (except MTU considerations).
Support for Hub and Spoke topologies (with up to 64 tunnels per adapter)
Allows extending IST over WAN solution with vIST
Timelines Proof Of Concept occurring now Production anticipated 2H2014, early 2015
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 19
Fabric Connect over IP Deployment Scenario – Hub and Spoke
Adapter establish multiple tunnels per device (POC restricted to 2) For POC hub site requires translation bridge (ERS4800) to convert
NNI interfaces into 1 uplink port
Main Site
Site 1
VSP4000
VSP4000
WAN
SPB Fabric
Site 2
Site 3
Site 4
VSP4000
VSP4000
VSP4000
4 interfaces
ERS4800
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF
March 25 – 27th, 2014 І Orlando, FL
#AvayaATF© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya.
City of Cedar Rapids, Iowa Fabric Connect/SPBm Implementation Case Study
Keith Nuehring – IT Operations Manager
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 21
Cedar Rapids Network before SPB and Fabric Connect
Many device types – ERS 8300, ERS 5530, ERS 4548, ERS 2526, SR 1004, SR 3120
With three primary locations, SMLT (two-switch cluster) didn’t really fit well ERS 5530’s required at least annual reboot to maintain stability Maintained a Spanning Tree ring to support smaller venues within the City
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 22
Considerations and the Decision to move to Fabric Connect
Should we stay with Avaya or move to another vendor? Nortel’s Chapter 11 Uncertainty over whether Avaya would continue to support and expand
the networking business acquired with Nortel Enterprise Solutions Concerns with technical support after Avaya’s NES acquisition Concerns with stability and product life of existing equipment
ERS 8300 ERS 5530
Attended ATF Orlando February 2012 Came away from first ATF with decision to implement SPB
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 23
Network Redesign Goals and Objectives
New network in conjunction with new City Services Center Upgrade from 1Gigabit to 10Gigabit City-wide network Increase network resilience – target of milliseconds, not seconds Consolidate Layer 3 routing into three main sites Reduce number of devices types to support
From … ERS 8300 and ERS 5500 both performing IP routing ERS 4000 series and ERS 2500 series edge switching
Different platforms, different capabilities, different CLIs To …
VSP 7000 SPB Fabric Switches and IP Routing ERS 4000 series and ERS 3500 series edge switching
Single CLI common across entire network
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 24
Planning and Staging the Network Cutover
Considerations in selecting the network products to use Large chassis switches not practical for City of Cedar Rapids VSP 7000 – 10 Gigabit switching, SPB Fabric, but no
simultaneous SPB Fabric and IP Routing VSP 4000 – Limited number of 10 Gigabit interfaces, semi-
external USB flash, does have IP Shortcut Routing Selected VSP 7000 both for SPB Switching Fabric and for IP
Routing external to the SPB Fabric Network design
Three-site mesh topology with all links active NetApp storage and servers sync between sites across L2 VSN
Training and Staging Used to-be live equipment for on-site hands-on training for staff Pre-built and tested entire network in City Services Center lab
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 25
New City of Cedar Rapids SPB Network Design
Involta Data CenterVSP 7000
L2 Fabric Core and in-building IDF Distribution
VSP 7000L3 Fabric Services
ERS 48001Gig Server Switching
VSP 7000L2 Fabric Core and in-
building IDF Distribution
VSP 7000L3 Fabric Services
ERS 48001Gig Server Switching
Central FireVSP 7000
L2 Fabric Core and in-building IDF Distribution
VSP 7000L3 Fabric Services
ERS 48001Gig Server Switching
Public Works
City Hall
Police Department
Water
Shown:• Main site network core switching
and routing only
Not shown:• Small, seasonal venues (parks,
rinks, etc.)• In-building wiring closets
City Services Center
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 26
Cutover Weekend
Moved pre-configured equipment from City Services Center lab to final location
Plan and document all steps, label all cables Walk through migration steps in a dry run to test procedures With single-strand BX optics, could bring up basics of new network
simultaneously with old network for easing the migration Proactively open up Avaya Support case, have support engineer
review plan and configurations before cutover Problems? … a few …
PVID set wrong on a switch Missed moving a cable, caused a routing loop VSP 7000 in Involta Data Center random rebooting
Updated boot loader to fix
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF 27
Observations and Lessons Learned
SPB resiliency works One optic failed some time after cutover and the NNI was
bouncing No visible effect to users – even while link was bouncing for 2-
1/2 hours Simply disabled link over weekend until optic could be
changed the next Monday Proved during cutover that even with a VSP 7000 failing, the
alternate switch and path would work to maintain the network VSP 7000 that was rebooting was one of two switches
connecting the NetApp storage! Simplified IP routing by consolidating into VSP 7000’s with VRRP Standardized on Direct and Static routing and VRRP Single CLI syntax on VSP 7000, ERS 4000, ERS 3500 is nice Was it worth making the change?
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF#AvayaATF
Thank You!
© 2014 Avaya Inc. Avaya – Confidential & ProprietaryDo not duplicate, publish or distribute further without the express written permission of Avaya. #AvayaATF
BE SURE TO TWEET YOUR FEEDBACK ON THIS PRESENTATION
#AvayaATF
29
BEST OF ATF SPEAKER AND TEAM AWARD
Winners will be announced at closing of event