celerra nsx architectural overview

Copyright 2006 EMC Corporation. Do not Copy - All Rights Reserved.

Celerra NSX Architectural Overview - 1

2006 EMC Corporation. All rights reserved.

Celerra NSX Architectural OverviewCelerra NSX Architectural Overview

Welcome to Celerra NSX Architectural Overview.

The AUDIO portion of this course is supplemental to the material and is not a replacement for the student notes accompanying this course. EMC recommends downloading the Student Resource Guide from the Supporting Materials tab, and reading the notes in their entirety.

These materials may not be copied without EMC's written consent.

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

EMC2, EMC, Symmetrix, Celerra, CLARiiON, CLARalert, Connectrix, Dantz, Documentum, HighRoad, Legato, Navisphere, PowerPath, ResourcePak, SnapView/IP, SRDF, TimeFinder, VisualSAN, and where information lives are registered trademarks,

and EMC Automated Networked Storage, EMC ControlCenter, EMC Developers Program, EMC OnCourse, EMC Proven, EMC Snap, Access Logix, AutoAdvice, Automated Resource Manager, AutoSwap, AVALONidm, C-Clip, Celerra Replicator, Centera, CentraStar, CLARevent, CopyCross, CopyPoint, DatabaseXtender, Direct Matrix, Direct Matrix Architecture, EDM, E-Lab, Enginuity, FarPoint, FLARE, GeoSpan, InfoMover, MirrorView, NetWin, OnAlert, OpenScale, Powerlink, PowerVolume, RepliCare, SafeLine, SAN Architect, SAN Copy, SAN Manager, SDMS, SnapSure, SnapView, StorageScope, SupportMate, SymmAPI, SymmEnabler, Symmetrix DMX, Universal Data Tone, and VisualSRM are trademarks of EMC Corporation.

All other trademarks used herein are the property of their respective owners.



2006 EMC Corporation. All rights reserved. Celerra NSX Architectural Overview - 2

Course Objectives

Upon completion of this course, you will be able to:

y Describe the architecture of the Celerra NSX y Identify the new hardware components of the NSX y Describe the differences between the NSX and the NS

Series

y Describe the new software needed for the Celerra NSX family

y Describe the NSX backend support

The objectives for this course are shown here. Please take a moment to read them.




Introduction to the NSX

Upon completion of this module, you will be able to:

y Identify the new features of the Celerra NSXy Identify the similarities between the NSX and NS seriesy Explain the differences between the NSX and NS series

The objectives for this module are shown here. Please take a moment to read them.




Celerra NSX Overview y NSX System

4 to 8 Blades Minimum 1 Standby Blade required

2 to 4 Blade Enclosures daisy-chained by redundant Ethernet networks

Always ordered in a Gateway Configuration

Redundant Management Switches in each enclosure

Dual Control Stations CLARiiON and Symmetrix back-

end arrays

y Managed Dual Redundant UPS

The new NSX system is the most highly redundant model in the Celerra family of products. Compared to the NS Series, it is always ordered in a Gateway configuration and consists of the following:y 4 to 8 X-Blade 60 blades that come in a Blade Enclosure. Each system is required to have a

minimum of one blade configured as a standby for high availabilityy 2 to 4 Blade Enclosures daisy-chained together with dual, redundant Ethernet networks used

to manage and monitor the system y 2 Management Switches in each Blade Enclosure which have Ethernet Switches and serial

connections y Two new Control Stations (for redundancy) to manage the system y Support for both CLARiiON and Symmetrix arrays for backend storage y the ability to upgrade from both the CNS and NS Celerra families

Note that any upgrade will be a box replacement and, depending on how many DMs are being replaced and what the backend storage array is, could be very difficult and time consuming.

Dual redundant UPS (Uninterruptible Power Supply) is used to keep data flowing, in case a power outage occurs.




Differences Between NSX and NS

y Blades use Small Form Pluggable (SFPs) instead of Media Interface Adapters (MIAs) for all Fibre connections

y No serial connections from the Control Station to the Blades Users can still access Blade console if needed

y Two redundant Ethernet Management Networksy No eight-port Ethernet network switch on the CS

The switch functionality has been built into the new CS used on the NSX

The NSX is a new family of Celerra. In designing this system, more redundancy was built in to avoid single points of failure. Some of the key differences between the NSX and the older NS500/700 families are as follows:y There is no longer a serial connection between the CS and each Blade y If a serial connection is needed by Customer Service, one is provided through the

Management Module y Like the earlier CNS models, the NSX has two independent Ethernet networks for

management of the system y All of the fibre connections in the NSX use SFP type connectors rather than the MIAs used

in the earlier NS family y The eight port Ethernet switch that used to reside on the CS has been replaced by redundant

Management Modules. These have built-in Ethernet switches and serial connections and are located in each Blade Enclosure.




Differences Between NSX and NS y Redundant Management Modules in each enclosurey Redundant Uninterruptible Power Supplies (UPS) and AC

Transfer Switches (ACTS)

y Several more Field Replaceable Units (FRUs) Blade enclosure Frontend personality card and Backend personality card Blade motherboard (4 GB) without personality cards System management switch module Power supplies Blower module The Control Station (CS) can be replaced The automatic transfer switches (ATSs) can be replaced The uninterruptible power supplies (UPSs) can be replaced

The new NSX has dual UPS to maintain the Control Stations and the blades, in the event of a power loss. The UPS maintains power to the backends, while the ACTS supplies power from the UPSs to the Control Stations. Because the CS has a single power plug, the ACTS are used to route power from both UPS, in case either fails. The earlier NS families only had power protection for the backend array.

The NSX family has many more Field Replaceable Units (FRUs) than earlier models. Rather than having to replace large parts of the system, smaller pieces, such as blade memory modules, can be replaced onsite at the customers location.




Similarities Between NSX and NS Families

y Same NAS code starting with v5.4 y CLARiiON and Symmetrix backend arrays can be usedy Fibre Channel zoning requirements are the samey All Celerra NAS software features are supported

Similarities between the NSX and older NS families are shown here. Starting with NAS code 5.4, the same code is used on all models of the Celerra series.

All storage arrays in the CLARiiON and Symmetrix families that were supported by the NS500/600/700 are supported for the NSX.

Fibre Channel requirements for switches and zoning are the same in all Celerras.

All software and hardware features, which were supported by earlier NS families, are supported in the NSX.




NSX Control Station


y Explain the concept of the new Control Stationy Identify the main differences and similarities to the

previous versions





NSX Control Station y Intel Pentium IV 2.8 GHz with 512 Kb cache processory 2 Gb Memory Installedy One 10/100/1000 Mb/s port Gig-E controllery Two 10/100/1000 Mb/s Ethernet LAN porty One 10/100 Mb/s Ethernet porty 80 GB internal SATA disk drivey Two integrated serial ports, one for laptop/console redirection and

one for the CallHome modem

y 3-1/2 inch floppy drivey DVD-ROM drivey Linux operating system software

With the introduction of the NSX family, a new Control Station was also introduced.

The Control Station monitors and communicates with the blades through the Ethernet link. It uses the Ethernet link to initiate failover. This communication is done through the internal private Ethernet network between the blades and the system management switches.

The Control Station can also monitor the backend storage systems through the Ethernet links for status and to extract information. It cannot control the backend storage systems.

When the NSX server contains multiple blades, different failover options can be configured. The NSX series server ships with four, six, or eight blades. A blade can have multiple standby blades configured for it.

The Control Station is connected to an external SYMMOD-US1 serial modem through the COM 1 port, and optionally to a service laptop using the COM 2 port, located in the front of the Control Station.




NSX Control Station

Celerra NSX Front view

LEDs and Switches

Power Switch

NMI Switch

Reset Switch ID Switch

Power Boot Sequence LED

Status LED

HDD Act LED

HDD Fault LED Gb # 1 and Gb # 2 LED

USB Connectors 2 and 3

ID LED

Serial Port COM2

The Control Station is a dedicated management, Intel processor-based computer that monitors and sends commands to the blades. The private network connects the two Control Stations (always shipped on NSX systems) to the blades through the system management switch modules. Like previous versions, it provides software installation and upgrade services, and high-availability features such as fault monitoring, fault recovery, fault reporting (CallHome), and remote diagnosing. Two Control Stations can be connected to a public or private network for remote administration. Each Control Station has a serial port that connects to an external modem so that the Control Station can call home to EMC or a service provider if a problem should arise.




NSX Control Station

eth3 - Public LAN Port

Celerra NSX Rear view

COM1 - To serial modem (for Call-Home)

eth0 Internal Network (To Mgmt. Switch-A in

Enclosure 0)

Video Port

Gb2 Internal Network(To Mgmt. Switch-B in

Enclosure 0)

Gb1 IPMI (To eth1 of the other

Control Station)

This slide displays the rear view of the Next Generation Control Station. Note the lack of a 25-pin quad serial port and spider cable.




NSX Control Station

y Comparison of NSX to NS Control Station No quad serial port and cables for connection to Blades No attached Ethernet switch mounted on chassis function now in

CS 2GB of main memory now standard Dual Control Stations are now standard

y Redundant connectivity for monitoring, administration Control now provided through the Management Switches

The differences between this CS and the one used in other NS500/600/700 families are as follows:y There is no longer a quad serial port on the rear of the CS with a spider cable connecting to

each of the Blades. With the development of the dual redundant Management Switches and Ethernet networks, there is no longer a need for this type of connection.y The eight port Ethernet switch has been removed from the CS tray because the switch

functionality has been built into the CS itself. y The CS now comes standard with 2GB of main memory. y Dual Control Stations come standard with each Celerra NSX. y If a t2cab is run on the Celerra NSX CS, it will identify which model it is.y The management of the NSX is done through the redundant Management Switches built into

each Blade Enclosure.




Celerra NSX Blades


y Explain the concept of Bladesy Identify the main differences and similarities to the

previous versions





NSX X-Blade 60 Specifications

y 64 bit capable, 4GB standard, 16GB memory capabley 3.4 GHz Pentium 4 processor with HT (Hyper-Threading)

Technology

y 800 MHz FSB, 4 GB memory y Two I/O Modules (FRUs)

4 FC, 2 Fibre GbE, 6 Cu GbE ports

y I/O expansion capable for future releasesy Supports up to 16TB of Fibre Channel storage

For the Celerra NSX series, Data Movers will be identified as Blades. The X-Blade 60 is the newest CPU module in the Celerra line. The X-Blade 60 specifications are shown here.




NSX Blade = CPU Blade + Fibre I/O Module + GbE I/O Module

NSX Blade

Fibre I/O Module GbE I/O Module

Label under modulesBlade Motherboard

The NSX X-Blade 60 is made up of three components: the CPU board, the Fibre I/O module, and the GbE I/O module.

This slide displays the face of both of the I/O modules.

The Backend personality card (Fibre I/O Module) connects the system to the FC switches and the SAN fabric.

The Frontend personality card (GbE I/O Module) connects the system to the customers Ethernet network.




NSX Blade - Fibre I/O Module

Backend Personality Card

Label under module

The left module, inserted into the Blade Enclosure, is the Fibre module.

It has four 2GB Fibre ports which are labeled from left to right: BE0, BE1, AUX0 and AUX1. The AUX ports are used to connect to tape devices for backup. The BE0-1 ports are used to connect the Blade to either a CLARiiON or Symmetrix array for booting and users filesystems.

NOTE: All four ports must be connected through a switch. The labels are on a Mylar card under the I/O module. This is due to the limited space on the front of the module and the cooling requirements. It can be pulled out for viewing.




NSX Blade - GbE I/O Module

Frontend Personality Card

Label under module

The right module, inserted into a Blade, is the GbE I/O module.

It has two Fibre and four copper Ethernet ports. The Fibre ports are labeled fge0 and fge1 from left to right. The copper Ethernet ports are labeled cge0-cge2 on the top row and cge3-cge5 on the bottom row. These ports are used to connect to the customers Ethernet network for sharing and exporting filesystems.

NOTE: The labels are on a Mylar card under the I/O module; This is due to the limited space on the front of the module and the cooling requirements. It can be pulled out for viewing.




NSX Blade - Connections

Fibre connections

NSX Blade

Instead of Media Interface Adapters (MIAs) for all Fibre connections, NSX X-Blade 60 uses Small Form Pluggable (SFPs). Small form-factor pluggable (SFP) modules connect optical cables to special SFP sockets on the blades (BE 0, BE 1, AUX 0, AUX 1, fge0 e fge1).




NSX Blade Enclosure = 2 Blades + 2 Mgmt. Switches + 2 Power Supplies

NSX Blade Enclosure Back View

Power Supply A

Blade BBehind Ethernet and

Fibre Boards Blade ABehind Ethernet and

Fibre Boards

Fibre I/O Modules

GbE I/O Modules

Mgmt Switch-B

Mgmt Switch-A

Power Supply B

The NSX Blade Enclosure is made up of 2 X-Blade 60 Blades, 2 Management Switches, and 2 power supplies.

The module and the power supply on the right side are labeled A, and the two on the left side are labeled B. The top blade is always the higher number server, so if this was the first enclosure, the top blade would be server_3 and the bottom blade would be server_2.




NSX Blade Enclosure Front View with Bezel Removed

NSX Blade Enclosure Front View

Power Supply BPower Supply A Blower Modules

Blower Module Fault LED

Enclosure Fault LED

Power LED

Power Supply Fault LED

The NSX Blade Enclosure is 4U (7 inches) high.

Besides the previously standard high-availability features explained (2 redundant X-blade 60s, 2 system management switch modules and 2 redundant power supplies), the enclosure is also supplied with four system cooling modules (four blowers total for N+1 operation).

The blower modules draw ambient room air through the chassiss front bezel and force it out through the back of the enclosure.

The 2 power supplies have separate 12 volt output regulators, one for each blade. Also, an adjustable 9-volt/12-volt line supplies energy to the blowers, allowing them to speed up if one of them fails.




Components Overview


y Identify hardware componentsy Understand new software components and changes





Celerra NSX Cabinet Rear Component Vy Two to four blade

enclosureEach enclosure contain two X-Blades 60s

y Two Control Stationsy Two uninterruptible

power supplies (UPS)

y Two automatic transfer switches

y Two Management Switches per Enclosure

The components are shown here.

The Celerra NSX series servers connect to either network attached storage (NAS) or to a combination of storage area network (SAN) and NAS. The NSX gateway server cabinet has four, six, or eight blades (four blade enclosures each containing two blades).

With upgrade kits, you can convert a four-blade model to a six- or eight-blade model. Or you can upgrade a six-blade model to an eight-blade model.

The model NSX system has two Control Stations. The second Control Station provides high availability. The Celerra NSX ships pre-configured and fully cabled from the factory. It is shipped installed in an EMC 40U rackmount cabinet.




Management Switch y Two Management Switches per Enclosure for

redundancy

y Each Switch consists of: 5 port Ethernet switch (Broadcom 5325M) Motorola Coldfire Micro-controller Downloadable firmware (upgraded during NAS install/upgrade) Connection to I2C buses Interconnection to peer module through enclosure backplane LED indicators for Enclosure ID USB serial ports for Blade console and debug access

y Switches provide power-control and monitoring Replaces the RS4XX bus from CNS cabinet Replaces serial lines and Peer-Power from NS

With the introduction of the NSX family comes a new Management Switch. These switches replace the 8-port switches that were mounted on the rear of the CS tray in older NS families. The features of each module are shown here.

These switches also provide peer-power control and box monitoring, which was previously provided by the RS4XX bus in the CNS cabinet and the serial cables in the older NS Celerras.




Management Switch

Port 0 (Uplink) Port 0 (Uplink)RS232 Serial Ports

A BPort 4 (from CS1 or UPS)Port 3 (Downlink)

Fault LED

Power LED

Power LED

This is a picture of the new Management Switch. The switches on the right side of the enclosure, for the primary management network, are labeled A. Those on the left side, for the secondary network, are labeled B.




NSX Internal Management Network Cabling

Secondary Network

Primary Network

This diagram shows the Ethernet connections for an NSX with 4 Enclosures and no UPS. Facing the rear of the rack, the primary Ethernet management network is on the right and the secondary is on the left.




NSX Internal Management Network Cabling (cont.)

This shows the NSX Celerra with the UPS installed. The Ethernet connection on the UPS is plugged into port 4 on the Management Switch of the second Blade Enclosure in the chain. UPS 0 is connected to the Primary network and UPS 1 is connected to the Secondary network.




NSX Power Cabling with UPS

If the UPS has been installed, the power strips are not used. The main power cords of the UPS will be plugged into separate power sources for redundancy. All power supplies on the right side will be plugged into UPS 1 and all power supplies from the left side will be cabled into UPS 0.

The Control Stations will get their power from ACTS (AC Transfer Switches) 0 and 1. Because the Control Stations only have one power input, the ACTS are fed from both of the UPSs and then supply the CSs with power. In this way, if either of the power circuits or UPSs fail, the other one will support the Control Stations. Unlike in older NS Celerras, the NSX family will continue to serve data to the customers network when power is lost.




NSX Power Cabling with UPS Schematic

This illustration displays the power distribution schematic for the NSX with the UPS, and part numbers for the UPS assemblies. Please note that this view is from the front of the system so Power Supply A is on the left and B is on the right.




Blade Enclosure N+1 Cooling - Front View

Blower Modules

Blower ModulesFault LED

The 4 blower modules located in front of each blade provides cooling for the blades and the power supplies from front to rear. N+1 Cooling means that one fan can go down and the system will continue to operate. Since each of the fan packs has 2 fans, removing a fan pan pack will cause a graceful shut down if not replaced within the 2 minute time period.

The blowers operate at a low speed during normal operation. If a blower fails, the remaining blowers speed up to maintain airflow. Latches on the blower module make it easy to replace without tools.

Each blower module connects directly to the midplane and is powered by the enclosures power supplies. Latches on the blower module make it easy to replace without tools.

Each blower module has a status LED indicator.




New NAS Software Components

y Management Switch Firmware Maintains status of all FRUs in the enclosure Peer Mgmt. Switch power control Blade Power control Enclosure ID assignment, reason codes Isolates failing FRUs

y T2net Library / Enclosure-ID utility UDP based network relay between the Mgmt Switches and Tier-2 Provides abstractions for Tier-2 to obtain enclosure information and

manage FRUs Commands to setup the enclosures during installs/upgrades and

FRU replacements

With the release of the NSX, new pieces of software had to be developed to manage the new system. The first is new firmware for the Management Switches. This firmware takes care of the items shown here.

Another new piece of software is the enclosure-ID utility and the T2net library. Information about these are displayed here.




Celerra NSX NAS Software Changes

y The EnclosureID concept is new for NAS Blue LED on the Mgmt Switch indicates enclosure ID Based on position on the Management Ethernet network daisy chain

y NAS Installation Setting up Enclosure IDs during install DHCP server always up Optional - Creation of system LUNs can be automated Optional - Automated zoning and Storage Group configuration Faster NAS installation

With the introduction of the NSX family and NAS v5.4, changes have been made in system monitoring, NAS installation, and how the system is configured. One new concept is the Enclosure ID. In older systems, the ID was based on the positioning of the serial cables at installation time and then stored in NVRam on the enclosure.

In the NSX system, a new command is run by the setup script which will probe the management Ethernet network and assign Enclosure IDs based on the order the enclosure is on the Ethernet network chain. This ID is then displayed by the LEDs on the Management Switch in that enclosure.

Changes that have occurred in the NAS installation are shown here.




Celerra NSX Backend Support


y Describe the backend storage systems support provided by Celerra NSX





Celerra NSX Backend Supporty NSX connects to Symmetrix

and/or CLARiiON

y Always connected via fabricy Mixed Symmetrix and CLARiiON

backend is supported Recommended that Control

Volumes/LUNs reside on the Symmetrix

y One NSX can connect to up to 4 storage arrays

y Up to 4 NSX can connect to one storage array

y Small Form Pluggable (SFP) instead of MIAs

The Celerra NSX connects to Symmetrix or CLARiiON arrays. A mixed CLARiiON and Symmetrix backend is also supported.

The NSX is always configured as a Fabric-connected gateway system. It is cabled to a Fibre Channel switch using fibre-optic cables and small form pluggable (SFP) optical modules, rather than the MIAs used in the earlier NS family. You can use two switches for high-availability.

One NSX can attach to up to four storage arrays. EMC best practices says that if attaching to both Symmetrix and CLARiiON, place the Celerra Control Volumes on the Symmetrix. Up to four NSX can be attached to a singe storage array. EMC recommends a CLARiiON maximum of two.




NSX-Symm Single Switch Fibre Configuration

Fibre Channel Switch

Symm DirectorsBlade 2

Blade 3

BE-1

BE-0

BA BA

BB

AA

AB

AA

BB

AB

BE-1

BE-0

The slide shows the fibre layout for a 2 Blade through a single switch configuration. Note that the two connections to the SYMM are to different Directors and are on ports on different busses for maximum load balancing.




NSX Symm Dual Switch Fibre Configuration

Switch A Symm DirectorsBlade 2

Blade 3Switch B

BA

AA

BB

AB

BA

BB

AA

AB

BE-1

BE-0

BE-1

BE-0

In a two Blade dual switch configuration, we are still using the same FA configuration, but all the Blades BE-0 ports are going to one switch and the BE-1 ports are going to the other switch.




Course Summary

Key points covered in this course:

y The new features of the Celerra NSX y The differences between the Celerra NSX and the NS

Series

y Description of the physical hardware of the Celerra NSXy Discussion about the new software needed for the

Celerra NSX family

y Description of the NSX backend support

These are the key points covered in this training. Please take a moment to review them.

This concludes the training. In order to receive credit for this course, please proceed to the Course Completion slide to take the assessment.

Celerra NSX Architectural OverviewCourse ObjectivesIntroduction to the NSXCelerra NSX Overview Differences Between NSX and NSDifferences Between NSX and NS Similarities Between NSX and NS FamiliesNSX Control StationNSX Control Station NSX Control Station NSX Control Station NSX Control Station Celerra NSX BladesNSX X-Blade 60 SpecificationsNSX BladeNSX Blade - Fibre I/O ModuleNSX Blade - GbE I/O ModuleNSX Blade - ConnectionsNSX Blade Enclosure Back ViewNSX Blade Enclosure Front ViewComponents OverviewCelerra NSX Cabinet Rear Component VManagement Switch Management SwitchNSX Internal Management Network CablingNSX Internal Management Network Cabling (cont.)NSX Power Cabling with UPSNSX Power Cabling with UPS SchematicBlade Enclosure N+1 Cooling - Front ViewNew NAS Software ComponentsCelerra NSX NAS Software ChangesCelerra NSX Backend SupportCelerra NSX Backend SupportNSX-Symm Single Switch Fibre ConfigurationNSX Symm Dual Switch Fibre ConfigurationCourse Summary

celerra nsx architectural overview

Documents