atm ip solution level fault management cvm12 nn10408 900_07.04

768
Carrier VoIP Nortel ATM/IP Solution-Level Fault Management Release: (I)CVM12 Document Revision: 07.04 www.nortel.com NN10408-900 .

Upload: rosembergp

Post on 11-Nov-2014

241 views

Category:

Documents


9 download

TRANSCRIPT

Page 1: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Carrier VoIP

Nortel ATM/IP Solution-LevelFault ManagementRelease: (I)CVM12Document Revision: 07.04

www.nortel.com

NN10408-900.

Page 2: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Carrier VoIPRelease: (I)CVM12Publication: NN10408-900Document release date: 30 April 2009

Copyright © 2008-2009 Nortel NetworksAll Rights Reserved.

Printed in Canada, the United States of America, and the United KingdomLEGAL NOTICE

While the information in this document is believed to be accurate and reliable, except as otherwise expresslyagreed to in writing NORTEL PROVIDES THIS DOCUMENT "AS IS" WITHOUT WARRANTY OR CONDITION OFANY KIND, EITHER EXPRESS OR IMPLIED. The information and/or products described in this document aresubject to change without notice.

Nortel, the Nortel logo, Business Made Simple and the Globemark are trademarks of Nortel.

All other trademarks are the property of their respective owners.

.

Page 3: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

3.

ContentsNew in this release 11Features 11Other changes 11

Introduction 13

Fault management fundamentals 15Communication Server 2000 Management Tools 16Carrier VoIP lines testing 113Fault management trunk testing 114PT-AAL1 trunk testing capabilities 114PT-IP trunk testing capabilities 118Checking for Media Gateway logs 125Ethernet Routing Switch 8600 Ethernet port configuration, and 100Base-T cable

replacement 125Add carriers to a GWC 126Geographic survivability 145Understanding Link States 166Communication Server 2000 Management Tools fault management

overview 167Perform an audit 172

Fault management tasks 175

View and search log files 177Running the CopyLogs log viewer utility 178Running the LogExtractor log viewer utility 180Running the LogBrowser log viewer utility 182

Communication Server 2000 Management Tools softwaredowngrade 185Cloning the image of one node in a cluster to the other node 186Migrating the SAM21 network elements back to the CS 2000 SAM21 Manager

on the core manager 193Launching the Communication Server 2000 SAM21 Manager Client that resides

on the Communication Server 2000 Management Tools Server 197Rolling back the Shelf Controller software 199

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 4: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

4

Performing a rollback of the CS 2000 SAM21 Manager 202Performing a full system restore on an SPFS-based server 204Restoring the Oracle/MySQL data on an SPFS-based server 209Clearing the JWS Cache on a Client Workstation 215Routing log streams from an SPFS-based server to a remote host 218

Breaker module replacement 221Removing a breaker module 222Installing a BIP alarm module 224Removing a BIP alarm module 226Replacing a BIP alarm module 228Replacing a (BIM) Breaker Interface Module 229

Maintaining Communication Server 2000 GWC alarms and logs231

Pinging the Communication Server 2000 Core Manager 233Pinging the core and billing manager 236Ensuring that the OSS Comms Svcs and OSS and Application Svcs are in

service 238Checking the status of the DDMS proxy 240Adding the Communication Server 2000 Management Tools Server as a DDMS

client of the CBM 241Adding the Communication Server 2000 Management Tools Server as a DDMS

client of the Communication Server 2000 Core Manager Core 245Restarting the Apache Web server 248Checking the occupancy of the /opt/ directory on the Communication Server 2000

Management Tools server 249Adding LEN information back into the XA-Core 251Checking for MSC Server 1000 XACoreCS 2000 or CS 2000-Compact logs 254Deleting carriers from a GWC 255Viewing carrier provisioning data for a GWC node 257Adding carriers to a GWC 261Viewing media proxy associations 284Viewing media proxies associated with a GWC node 286Taking a manual GWC software image 287Reconfiguring NTP service 290Using the Table Editor to Delete a Tuple from a Table 293Changing the APS Oracle account password 295Viewing GWC platform hardware alarms 297Viewing GWC PM logs 299Viewing GWC logs in syslog files 300

Deleting a line 305Removing a line 306

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 5: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

5

Maintaining GWC alarms and logs 313Viewing GWC service alarm history 314Viewing GWC service alarms 316Viewing and interpreting the operational status of a GWC node 318Viewing switch-wide GWC status 322Monitoring operational and activity status of GWCs 329Restarting or rebooting a GWC card 331Restarting GWC card services 333Performing a Communication Server 2000 data integrity audit 335Recovery of the GWC320 certificate expiry alarm 338Increasing the size of a logical volume 340

I/O module replacement in an ERS 8600 in a carrier CS-LAN 343Configuring the SESM Server Application 345Starting the SESM server application 347Stopping the SESM server application 349Viewing SESM configuration settings 351Stopping and starting the DDMS proxy 354Stopping the SAM21 Manager server application 356Stopping the APS server application 358Stopping the NPM server application 360Starting the SAM21 Manager server application 362Starting the APS server application 364Starting the NPM server application 367Starting and stopping CINotifier 369Changing the Oracle user password on an SPFS-based server 372Disabling Ethernet Routing Switch 8600 WAN Edge routing 374Disabling and reenabling the router ports on the Ethernet Routing Switch

8600 376Installing the remote backup server 378Viewing configuration information for remote server backups 383Replacing an I/O module in an ERS 8600 in a carrier CS-LAN 385Locking the USP-Compact 400Taking down the paths to the ERS 8600 chassis X 408Determining if STORM devices require rotating, demounting and busying 410Locking the inactive Shelf Controller (SC) blade connected to chassis X 415Unlocking the Border Control Point 7100 419Unlocking the inactive SC 422Unlocking the USP-Compact 424Returning the paths to the ERS 8600 chassis X to service 432Enabling the Ethernet links between the XA-Core and the ERS 8600 434Accessing the Network Patch Manager CLUI 436

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 6: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

6

Powering up the XA-Core 437XA-Core power up procedures 438Powering up from the power distribution center 440Powering up from the XA-Core cabinet and terminal 443Powering up from the MAP terminal 445Powering up at the IOC/PDC 449Powering up at the PM frames 452Completing the power up of the XA-Core 454

Powering up the network 459Powering up the Message Switch 461Powering up a Ethernet Routing Switch 8600 464Powering up the Enhanced Network 465Powering up the Link Peripheral Processors 467Powering up the SDM 470Powering up a Sun Netra server 471Powering up the Border Control Point Manager 472Powering up the SAMF frame 473Power-On and boot a Policy Controller unit 475Powering on and booting an SST unit 476Viewing the operational status of the SIP Gateway application 477Powering up the Call Control Frame 481Unlocking the Call Agent 483Unlocking a GWC card 484Locking a GWC card 486Manually re-provisioning GWC cards 488Powering up a USP 492Powering up a Media Gateway/Multiservice Switch 15000 494Powering up a Media Gateway/Multiservice Switch 7400 495Powering up the MDM workstation 497Powering up an SPM device 498Powering up the MCS servers 500Powering up a Border Control Point 504Powering up the IPmedia 2000 shelves 505Powering up the SAM16 shelves 506Powering up an MG 9000 device 507Viewing logs from a remote backup 512Setting date and time 513

Server shut down and backup 515Scheduling automatic backups on the remote server 516Initiating a recovery back to the cluster 519Initiating a switch over to the remote backup server 520Shutting down an SPFS-based server 522

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 7: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

7

Communication Server 2000 Management Tools faultmanagement 527Configuring SESM log reporting 529Viewing debug logs 532Viewing OMPUSH logs 533Viewing customer logs on an SPFS-based server 535Posting a line by directory number 537Posting a line by gateway 539Busying a line 542Installation busying a line 544Force releasing a line 546Returning a line to service 548Clearing one or more posted lines from the display 550Retrieving line properties 552Querying line gateways in a trouble state 554Performing trunk maintenance using the Trunk Maintenance Manager 559Posting PRI Group D-channels 573Displaying trunk CLLI codes by gateway 575Performing an ISUP Continuity Test 577Accessing the Alarm Manager 579Retrieving details about an alarm 586Filtering alarms 589Resetting the filters for alarms 592Defining alarms using the NPM CLUI 594Defining alarms using the NPM GUI 596Performing an audit 600Replacing one or more failed disk drives on an SPFS-based server 621Replacing a DVD drive on an SPFS-based server 629Replacing a failed SPFS-based server 632Replacing failed Ethernet interfaces 635Re-establishing the connection between the NPM GUI and the server 637Restoring SSH communication between the CS 2000 Management Tools server

and the core manager 639Enabling X11-forwarding 642Disabling X11-forwarding 645Displaying X11-forwarding status 648

CORBA configuration issue correction between theCommunication Server 2000 server and the MG 9000 Managerserver 651Correcting a CORBA configuration issue between the CS 2000 Management

Tools server and the MG 9000 Manager server 652Recovering from having two inactive servers in a cluster 657Performing a restore of APS audio files from DVD 660

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 8: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

8

Performing an APS-only Database restore from a CD 661Verifying the state of a cluster 663Initiating a manual failover on a Sun Netra server pair 667Querying the state of the SPFS platform 669Rebooting an SPFS-based server 671

Communication Server 2000 troubleshooting 673Troubleshooting multiple DS1 carriers in a Carrier Fail Loss (CFL) state 674Troubleshooting DS1 carriers in an intermittent Carrier Fail Loss (CFL) state 676Troubleshooting DS0s trunks that are in a Carrier Fail Loss (CFL) state 678Troubleshooting DS0s trunks that are in a peripheral manual busy (PMB)

state 680Troubleshooting DS0s trunks that are in a System Busy (SB) state 681Troubleshooting DS0s trunks that are in a lock out (LO) state 682Troubleshooting DS0s trunks that are in a remote manual busy (RMB) state 683Troubleshooting line(s) in a permanent lock out (PLO) state 684Troubleshooting lines in a manual busy (MB) state 686Troubleshooting lines in a system busy (SB) state 687

GWC troubleshooting 689Troubleshooting a GWC that continuously initializes and fails to return to service

(RTS) 690Troubleshooting a GWC that fails to boot 691Troubleshooting a GWC that has a poor call completion rate 692Troubleshooting PRI trunks that remain in a lockout state 693Troubleshooting a failed attempt to create a GWC image 694

Trunk testing 697Performing a TL 100 trunk test 698Performing a TL 101 trunk test 700Performing a TL 102 trunk test 701Performing a TL 104 trunk test 703Performing a TL 105 trunk test 705Performing a TL 108 trunk test 711Performing a ISUP continuity test (ICOT) 713Performing a CVTEST 714Performing a QRYSIG 715

Communication Server 2000 management tools troubleshooting 717Troubleshooting a failed attempt to add and provision a GWC or to associate a

gateway with a GWC 719Troubleshooting a failure to add or delete a GWC 720Troubleshooting when an OSSGate session fails to launch from a PC 721Troubleshooting a connection failure when adding a GWC node 722Troubleshooting a failure to start the SESM server application 724

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 9: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

9

Troubleshooting a provisioning error with a GWC card or node 725Troubleshooting a failure of the Communication Server 2000 Management Tools

alarm manager to properly set alarm date and time 726Troubleshooting small gateway IP addresses that are reset to 0.0.0.0 in the

Communication Server 2000 Management Tools server 728Troubleshooting a failure to RTS a DMS Maintenance Application that is

IsTb 729Troubleshooting a failure to add a fourth Media Gateway 15000 to a GWC 730Troubleshooting a failure to perform maintenance actions while using the

Communication Server 2000 Management Tools 731Troubleshooting a failure to launch Communication Server 2000 SAM21 Manager,

Communication Server 2000 Management Tools GUI, or Network PatchManager 732

Diagnosing problems with a GWC card that cannot be booted 734Troubleshooting problems arising when an SSL certificate is changed and the CM

hostname is new or changed 738Troubleshooting combined Communication Server 2000 Management Tools

failures 739Troubleshooting a failure to delete a line from a media gateway in the IAC or Intl

IAC solution 740Troubleshooting a failed attempt to input the SDNA command in OSSGate 742Troubleshooting a failure to add an endpoint to a GWC during line provisioning

of the IAC or Intl IAC solution 743Troubleshooting a failed attempt to create a GWC image from the Communication

Server 2000 Management Tools GUI 744Troubleshooting APS logon problems 745

Communication Server 2000 Management Tools geographicsurvivability reference 751Logs and operational measurements supported by CEM 752

Preventative maintenance for CVoIP components 757Preventative maintenance tasks 758Downgrading the CS 2000 Management Tools software 763

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 10: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

10

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 11: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

11.

New in this releaseThe following sections detail what is new in Nortel ATM/IP Solution-levelFault Management (NN10408-900) for release (I)CVM12:

• “Features” (page 11)

• “Other changes” (page 11)

FeaturesThere are no new feature changes in this release.

Other changesSee the following section for information about changes that are notfeature-related:

• This document has been restructured according to Nortel CustomerDocumentation Standards (NCDS).

• This document has been modified to include new naming conventionsfor Audiocodes due to re-branding.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 12: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

12 New in this release

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 13: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

13.

IntroductionThis document describes the tasks and procedures involved withmanaging faults in the Carrier Voice over IP (CVoIP) solutions. Thisdocument also provides fundamental concepts and methodologies forhandling fault management in the network.

• hardware replacement and activation

• troubleshooting

• working with logs and alarms

Navigation• “Fault management fundamentals” (page 15)

• “Fault management tasks” (page 175)

• “View and search log files” (page 177)

• “Communication Server 2000 Management Tools software downgrade” (page 185)

• “Breaker module replacement ” (page 221)

• “Maintaining Communication Server 2000 GWC alarms and logs ”(page 231)

• “Deleting a line ” (page 305)

• “Maintaining GWC alarms and logs ” (page 313)

• “I/O module replacement in an ERS 8600 in a carrier CS-LAN ” (page343)

• “Powering up the XA-Core” (page 437)

• “Powering up the network” (page 459)

• “Server shut down and backup” (page 515)

• “Communication Server 2000 Management Tools fault management ”(page 527)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 14: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

14 Introduction

• “CORBA configuration issue correction between the CommunicationServer 2000 server and the MG 9000 Manager server” (page 651)

• “Communication Server 2000 troubleshooting” (page 673)

• “GWC troubleshooting ” (page 689)

• “Trunk testing” (page 697)

• “Communication Server 2000 management tools troubleshooting”(page 717)

• “Communication Server 2000 Management Tools geographicsurvivability reference” (page 751)

• “Preventative maintenance for CVoIP components” (page 757)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 15: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

15.

Fault management fundamentalsThis section provides the overall concepts of fault management in theCarrier Voice over IP (CVoIP) solutions. This section also provides anoverview of the various components in the CVoIP solutions, and basicinformation on testing and troubleshooting them. The various faultindicators, such as errors, logs, and alarms, are also explained in thissection.

Navigation• “Communication Server 2000 Management Tools” (page 16)

• “Fault management strategy” (page 21)

• “Line and loop testing” (page 34)

• “Trunk and carrier testing” (page 34)

• “ATM loopback testing” (page 37)

• “Misconnected DS512 link diagnostics” (page 40)

• “Multiservice Switch/I.610 Loopback feature” (page 45)

• “USP message tracing tool” (page 53)

• “MAP-based MG 9000 testing” (page 54)

• “OSS interface” (page 55)

• “Tools and utilities” (page 56)

• “Integrated Element Management System” (page 81)

• “Core and Billing Manager” (page 85)

• “Media Gateway 9000 Manager” (page 86)

• “Communication Server 2000 Gateway Controller Manager” (page 88)

• “Session Server Manager” (page 92)

• “Universal Audio Server Manager” (page 94)

• “Audio Provisioning Server (APS)” (page 95)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 16: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

16 Fault management fundamentals

• “Network Patch Manager (NPM)” (page 97)

• “Universal Signalling Point Manager” (page 97)

• “Multiservice Data Manager” (page 98)

• “Device Manager for Ethernet Routing Switch 8600” (page 110)

• “Storage Management Manager (STORM Manager)” (page 112)

• “Nuera Configurator” (page 113)

• “NueraView” (page 113)

• “HP OpenView” (page 113)

• “Carrier VoIP lines testing” (page 113)

• “Overview of line testing” (page 114)

• “MG 9000 lines testing” (page 114)

• “Fault management trunk testing” (page 114)

• “PT-AAL1 trunk testing capabilities” (page 114)

• “PT-IP trunk testing capabilities” (page 118)

• “TRKQRY” (page 124)

• “Naming conventions for Media Gateway carrier endpoint names”(page 125)

• “Checking for Media Gateway logs” (page 125)

• “Ethernet Routing Switch 8600 Ethernet port configuration, and100Base-T cable replacement” (page 125)

• “Geographic survivability” (page 145)

• “Communication Server 2000 Management Tools geographicsurvivability” (page 165)

• “Understanding Link States” (page 166)

• “Communication Server 2000 Management Tools fault managementoverview” (page 167)

• “User interface” (page 168)

• “Perform an audit” (page 172)

• “Media gateway endpoint names” (page 172)

Communication Server 2000 Management ToolsThe Communication Server 2000 (Communication Server 2000)Management Tools configuration for Geographic Survivability for the SN08release consists of two Communication Server 2000 Management Tools

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 17: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Communication Server 2000 Management Tools 17

active servers at site A which are commissioned as a local HA cluster andone Communication Server 2000 Management Tools (hitherto referred toas CKMT) server at site B which acts as a standby. The standby server isbrought into service using a recovery procedure if there is a catastrophicsite outage at site A. This section contains the Communication Server2000 Management Tools manual recovery procedure for disaster recoveryby switching activity between sites in the event of a catastrophic faultcondition on the CKMT cluster.

Terminology

Table 1CKMT Geographic Survivability procedure terms

Term Definition

CKMT HA The high-availability CKMT product which consists of two Sun Netra240 servers both commissioned with the CKMT SSPFS profile, andinstalled with cluster=ON.

CKMT for GS An additional standby server located at a different physical location.This unit is prepared for implementation of the manual recoveryprocedure in the event of a catastrophic fault condition at the local,main CKMT HA site.

Cluster Two interconnected redundant units. One unit is active, and theother is inactive. The active unit has applications in-service (INSV),and ensures that the inactive unit replicated file systems areupdated in real-time. In the event of an automatic cluster failover,the currently inactive unit becomes the active unit and the formerlyactive unit becomes the inactive unit.

Sites A and B The two geographic survivability sites. Each site has duplicateconfigured hardware, with the exception of the CBM 850 and CKMTunits.

CKMT HA u0 Unit 0 of the CKMT cluster at site A. One half of the clusterconfiguration formed by units A0 and A1. This unit is the mate ofCKMT HA u1.

CKMT HA u1 Unit 1 of the CKMT cluster at site A. One half of the clusterconfiguration formed by units A0 and A1. This unit is the mate ofCKMT HA u0.

CKMT u0 The single standby CKMT server at site B. Although clustering isconfigured on this unit because it will be restored from CKMT HA u0or u1, it does not have a mate unit.

Active CKMT unit The CKMT HA unit at site A (either CKMT HA u0 or CKMT HA u1)which has the application states as in-service (INSV).

Inactive CKMT unit The CKMT HA unit at site A (either CKMT HA u0 or CKMT HA u1)which has the application states as STANDBY.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 18: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

18 Fault management fundamentals

The Communication Server 2000 (Communication Server 2000)Management Tools configuration for Geographic Survivability for the SN08release consists of two Communication Server 2000 Management Toolsactive servers at site A which are commissioned as a local HA cluster andone Communication Server 2000 Management Tools (hitherto referred toas CKMT) server at site B which acts as a standby. The standby server isbrought into service using a recovery procedure if there is a catastrophicsite outage at site A. This section contains the Communication Server2000 Management Tools manual recovery procedure for disaster recoveryby switching activity between sites in the event of a catastrophic faultcondition on the CKMT cluster.

Terminology

Table 2CKMT Geographic Survivability procedure terms

Term Definition

CKMT HA The high-availability CKMT product which consists of two Sun Netra240 servers both commissioned with the CKMT SSPFS profile, andinstalled with cluster=ON.

CKMT for GS An additional standby server located at a different physical location.This unit is prepared for implementation of the manual recoveryprocedure in the event of a catastrophic fault condition at the local,main CKMT HA site.

Cluster Two interconnected redundant units. One unit is active, and theother is inactive. The active unit has applications in-service (INSV),and ensures that the inactive unit replicated file systems areupdated in real-time. In the event of an automatic cluster failover,the currently inactive unit becomes the active unit and the formerlyactive unit becomes the inactive unit.

Sites A and B The two geographic survivability sites. Each site has duplicateconfigured hardware, with the exception of the CBM 850 and CKMTunits.

CKMT HA u0 Unit 0 of the CKMT cluster at site A. One half of the clusterconfiguration formed by units A0 and A1. This unit is the mate ofCKMT HA u1.

CKMT HA u1 Unit 1 of the CKMT cluster at site A. One half of the clusterconfiguration formed by units A0 and A1. This unit is the mate ofCKMT HA u0.

CKMT u0 The single standby CKMT server at site B. Although clustering isconfigured on this unit because it will be restored from CKMT HA u0or u1, it does not have a mate unit.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 19: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Communication Server 2000 Management Tools 19

Table 2CKMT Geographic Survivability procedure terms (cont’d.)

Term Definition

Active CKMT unit The CKMT HA unit at site A (either CKMT HA u0 or CKMT HA u1)which has the application states as in-service (INSV).

Inactive CKMT unit The CKMT HA unit at site A (either CKMT HA u0 or CKMT HA u1)which has the application states as STANDBY.

Distribution of OAMP platformThe components in an OAMP platform are distributed as follows. Thescenarios assume redundant configurations are located in two separatebuildings:

• CKMT HA - The CKMT and Integrated Element Management Systems(IEMS) high-availability (HA) server pairs are located at site A.

• CBM 850 HA - The Core and Billing Manager (CBM) high-availabilityserver pair are located at site B.

• Standby Sun servers - In the event of a catastrophic fault conditionat one site, the standby Sun servers are brought into service throughthe manual recovery procedure. The Sun servers are positioned asfollows:

— The CBM 850 standby server is located in the CentralizedOperations Administration and Maintenance (COAM) frame at siteA.

— The Communication Server 2000 Management Tools/IEMS standbyserver is located in the COAM frame at site B.

— The standby server is brought into service using a manual recoveryprocedure if there is a total loss of connectivity at the other site.

All three servers must be able to connect to the OAMP LAN. However, ifthe servers at both site locations are permitted to function on the OAMPLAN at the same time, networks problems with clustering and duplicate IPaddresses will occur.

Required backups on the CKMT serversThere are two types of required backups needed to restore the standbyserver to functionality following a catastrophic fault condition:

• a full file system backup that makes a copy of all the file systemson the server with the exception of the Oracle file system. The OSand third party software application configuration data is captured on

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 20: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

20 Fault management fundamentals

DVD-RW discs. The amount of data on the server determines thenumber of DVD-RW discs required to complete the backup.

• a data-only backup for the data stored in the Oracle database andother application-specific data.

Meet all prerequisitesAdherence to the procedure prerequisites ensures successful completionof geographical survivability manual recovery while holding minimizing theout-of-service time for the failover to approximately four hours.

If the documented procedures are not adhered to strictly, longer restorationtimes will result. Failure to perform the following tasks properly results inproblems:

• full file system backup and data backup on the CKMT cluster

• full file system restore and data restore to the standby CKMT/IEMSserver

• ensuring the CKMT cluster at site A (CKMT HA u0 and u1) and thestandby CKMT unit in site B (CKMT u0) are not on-line at the sametime

Nortel recommends that you perform a full file system backup before andafter every installation, upgrade, or patch application to the CKMT server.The application data in the Oracle database is not automatically backupup, therefore, Nortel recommends daily backups of this data for the bestrecovery outcomes.

Communication Server 2000 Management Tools geographicsurvivability

The following procedures contain information for OA&M auto backup andaccelerated restore capability:

• “Installing the remote backup server” (page 378)

• “Scheduling automatic backups on the remote server” (page 516)

• “Viewing configuration information for remote server backups” (page383)

• “Performing a rollback of the CS 2000 SAM21 Manager” (page 202)

• “Viewing logs from a remote backup” (page 512)

• “Initiating a recovery back to the cluster” (page 519)

• “Initiating a switch over to the remote backup server” (page 520)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 21: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Fault management strategy 21

Fault management strategyIn all CVoIP solutions, fault management capabilities provide three mainfunctions:

• “Fault Delivery” (page 21)

• “Testing and Diagnostics” (page 22)

• “OSS interface” (page 55)

Fault DeliveryLog report is a record of a message that the system generates when asignificant event occurs in a network element. Log reports include statusand activity reports, as well as reports on hardware or software faults, testresults, changes in state, and other events or conditions likely to affectthe performance of the switch. Either a system or a manual action cangenerate a log report.

Log adapter The SCC2/STD log adapter receives CommunicationServer 2000, MG 4000, IW SPM, DPT SPM, MDM, and MultiserviceSwitch/Media Gateway logs. The adapter is a one-way translator thattranslates events received from network managers and network elementsinto the format required by the customer log system. The interfacebetween the Log Adapter and Communication Server 2000 Core Managerapplications is the CORBA interface. When the system starts up, the LogAdapter application registers with the Communication Server 2000 CoreManager applications for events that the Log Adapter will report to the faultOSS.

The primary events are

• alarm-raised

• alarm-cleared

• service state change

When the Log Adapter receives an event, it converts the event intoan SCC2 or DMS STD format and sends the log to the Log Deliveryapplication.

For more information on the CVoIP Log Delivery application, see Core andBilling Manager 850 Fault Management (NN10351-911).

Log Delivery application The Log Delivery application provides logreports in Switching Control Center 2 (SCC2) format. Log reports in SCC2format do not specify the switch or network element name. The LogDelivery application transfers log reports in normal ASCII text across aTCP/IP connection to the fault OSS.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 22: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

22 Fault management fundamentals

The log categories are fault and status changes. The network elementsdetect, timestamp, and log the event on a log report.

For more information on the CVoIP Log Delivery application, see Core andBilling Manager 850 Fault Management (NN10351-911).

AlarmsAlarms provides notice of problems or conditions that can change theperformance or working state of a network or network element. Dailyoperation of the network requires monitoring for alarms and checking thatfunctions continue without interruption. Alarms provide notification that asystem hardware or software-related event has occurred.

The following list shows the fault delivery features available:

• fault collecting and reporting

• fault isolation and identification

• storage, delivery, and formatting (Operational Support Systems [OSS]interfaces)

Testing and DiagnosticsThe following list shows the testing and diagnostic capabilities available:

• line and loop testing

• trunk and carrier testing

• ATM loopback testing

• routine exercise testing

• Misconnected link diagnostics

• USP message tracing tool

• MG 9000 testing

DPT trunk testing with the DPT-SPM and MG 4000

ATTENTIONThis section applies only to the North American PT-AAL1.

Digital Remote Office Test Line (ROTL) is a DMS feature to remotelytest trunks via a digital, four wire E&M trunk. This CAS trunk interfacepermits a test head to be connected to the Communication Server 2000and maintenance dial plan, select an outgoing trunk to be tested. A testhead is connected to the Communication Server 2000 by a T1, on whichseveral channels are provisioned with a ROTL trunk type. Test calls canthen be outpulsed from the test head over the ROTL trunk members using

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 23: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Fault management strategy 23

a defined maintenance dial plan which instructs the Communication Server2000 software to select an outgoing trunk circuit and generate a test callto the far-end switch. A connection is then established between the testequipment and the trunk circuit being tested. Once all connections havebeen made, the test head conducts the desired trunk test.

As of Release (I)SN09, the digital ROTL functionality has been enhancedto permit test connections over DPTs hosted by an DPT-SPM or MG 4000.The customer can directly select the DPT group, node and TID to test.

The following figure illustrates the configuration of the Digital ROTLfunctionality on the DPT-SPM and MG 4000.

Figure 1Digital ROTL functionality

There are currently two solutions for trunk troubleshootingin the PT-IP solution:

• hybrid switch solution (AAL5 IP only)

• non-hybrid/Communication Server 2000-Compactsolution (AAL2 and AAL5)

Hybrid switch solution (AAL5 IP only) Developed for the AAL5 hybridswitch configuration, which contains an ENET and an Integrated ServiceModule. The ISM is specifically designed to accommodate a variety oftest hardware which can perform maintenance testing on TDM trunks.This solution takes advantage of the TDM to IP bridge capabilities of the

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 24: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

24 Fault management fundamentals

IW-SPM, to provide connectivity between the Gateway TDM trunk undertest and the test hardware located in the ISM.

Non-hybrid/Communication Server 2000-Compact solution (AAL2and AAL5) Developed for the AAL2 and AAL5 switch configurationsthat do not contain an ENET or ISM, this solution utilizes third-party testequipment, Media Server 2000 Series (MS 2000) or the Universal AudioServer (UAS) with SAGE 945 as a limited replacement for the ISM. TheMS 2000 Series and UAS provides the trunk test functionality to theGateway TDM trunk under test.

Both solutions are fully integrated with the Communication Server 2000.This means that test execution, results and logs are equivalent to theLegacy TDM environment. However, since the Hybrid solution utilizes theISM, there are more trunk testing options available to the customer thanthe initial non-hybrid/Communication Server 2000-Compact solution.

The Hybrid Switch Solution supports the following trunk test functionality.Customers can test TDM trunks via the MAPCI TTP level and ATT levelinterfaces.

Table 3TTP level commands

Command Description

CktInfo Displays the name and state of the PM and thestate of the circuit of a trunk

Ckt Connects a specified circuit to a trunk

StkSdr Allows access to the Stuck Sender feature,which identifies trunks with outpulsingproblems.

Table 4Manual level commands

Command Description

LOSS Manually measures transmission loss. (Fordetails, see TGEN, LOSS and CKTMONCommand additional information.).

TGEN Manually generates a tone of desiredfrequency and level. (For details, see TGEN,LOSS and CKTMON Command additionalinformation.).

Noise Manually measures transmission noise.

OP Permits the ability to seize and outpulse a setof desired digits over a trunk.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 25: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Fault management strategy 25

Table 4Manual level commands (cont’d.)

Command Description

TDet Manually measures the presence of anincoming tone.

HSet Connects an available headset to a trunk.

SGNL Executes a bit signaling test on a trunk. (Fordetails, see SGNL Command additionalinformation.).

TST Permits the user to run a number of supportedtests, including the following:

• ISUP continuity test (ICOT)

• T100 (originating and terminating)

• T102 (originating and terminating)

• T104 (originating and terminating)

• T105 (originating and terminating)

Table 5Monitor level commands

Command Description

Montalk (L) Listen provides the user the ability to monitora trunk circuit in both directions in a listen onlymode.

Montalk (T) Talk provides the user the ability to monitora trunk circuit in both directions whileparticipating in the call (listen and talk mode).

CktMon Displays messages on the MAP from thedigital PM of the circuit without interruptingnormal call processing. (For details, seeTGEN, LOSS and CKTMON Commandadditional information.)

CPos Executes a test to a Centralized AutomaticMessage Accounting (CAMA) or RemoteOperator Number Identification (RONI) trunk.

Table 6DATATTP level commands

Command Description

Berttime Sets the length of time a BERT will run

BTerm Sets the type of outpulse or digits for a BERT

BERT Executes a Bit Error Rate Test on a trunk

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 26: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

26 Fault management fundamentals

Table 7C7TTP level commands

Command Description

CVTest Performs a Circuit Validation Test on a trunk

TrkQry Displays the local or remote state of a trunk

QrySig Queries the signaling availability and the stateof a trunk

RouteSet Displays the route set CLLI of a CCS7 trunk

CIC Displays the circuit identification code of atrunk

ATTENTIONAll commands at the Automatic Test Trunk (ATT) level are supported.

SGNL Command additional information The SGNL command supportsthe following North American MG-based PTS trunk variants which use thestandard/TDM PTS call model for the H.248 based protocol.

Trunk type Start signalling type Pulse type Card type

OP Wink MF DSISIG

ES Wink MF DSISIG

ATC Wink MF DSISIG

IT Wink MF DSISIG

CELL Wink MF DSISIG

IBNT2 Wink MF, DP DSISIG

IBNTO Wink MF, DP DSISIG

IBNTI Wink MF, DP DSISIG

The SGNL command support on MG15000 PTS trunks has the followingqualifications:

• No international trunk variants are supported.

• Delay Dial and Immediate start signaling types are not supported.

• Not supported in the IAC solution which employs PTS trunk variantsOP and ES hosted on TGCP protocol-based trunk gateways.

• Wink indication is represented by the single flash of a ‘+’ sign under theR field to the right of the posted tunk. The ‘+’ sign is only an indicationof the reception of a Wink signal and should not be interpreted asactual length of the Wink.

• Current MG15000-based PTS trunk architecture does not supportonhook signal reporting during digit collection. If the user is executing

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 27: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Fault management strategy 27

the SGNL command whose far-end trunk is an MG15000-based PTStrunk, the following procedure applies:

— User posts supported PTS trunk circuit and executes the SGNL’onoffhk’ command.

— Far-end MG15000 PTS trunk connects to an available digit receiverand outputs a Wink signal reply.

— Upon successful reporting of Wink signal, user releases postedPTS trunk circuit trunk via the available RLS release command.

— User waits for far-end trunk digit receiver timer to expire. Uponexpiration far-end trunk will return to idle state.

The following table lists support information applicable toNorth American MG-based PTS trunk variants that usethe standard/TDM PTS call model for the H.248-basedprotocol.

Trunk type Start signalling type Pulse type Card type

OP Wink MF DSISIG

ES Wink MF DSISIG

ATC Wink MF DSISIG

IT Wink MF DSISIG

CELL Wink MF DSISIG

IBNT2 Wink, DT, IM MF, DP, DD DSISIG

IBNTO Wink, DT, IM MF, DP, DD DSISIG

IBNTI Wink, DT, IM MF, DP, DD DSISIG

TGEN, LOSS and CKTMON commands support onMG15000 PTS trunks, has the following qualifications:

• No international trunk variants are supported.

• Not supported in the IAC solution which employsPTS trunk variants OP and ES hosted on TGCPprotocol-based trunk gateways.

• The LOSS command line option E for the echo test isnot supported.

• The CKTMON command resides on the MAPCI TTPMonitor level. No physical hardware connectionsassociated with audible circuit monitoring will beattempted at MAPCI TTP Monitor level Monitorlevel commands MonPost, MonLink, MonBoth andMonTalk are not supported. If the user is executing

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 28: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

28 Fault management fundamentals

the CKTMON command on a supported MG-basedPTS trunk the following procedure applies.

— User posts a supported PTS trunk circuit in anIDLE state.

— User enters the Monitor level by executing theMONITOR command.

— User enables the CKTMON command. This willturn ‘on’ the message intercept function.

— A call is received on the posted trunk circuit.

— Incoming messages from the Gateway Controllerare displayed on the MAPCI above the postedtrunk.

— User disables the CKTMON command byexecuting the CKTMON ‘off’ command.

— User quits the Monitor level to return back to theTTP level.

The following tests are supported but are not supportedby the MAPCI:

• T108 Terminating Test Line - a non-inverting digitalloopback connection

• T101/C101 trunk call

• Digital Remote Office Test Line (ROTL) - 100/200trunk types only

• Automatic Trunk Routing (ATR) - 250 trunk types only

ATTENTIONGateway-based TDM trunks do not support ATR.

ATTENTIONAdditional equipment is required for ROTL and ATR.

Non-hybrid/Communication Server 2000-Compact switchsolution This solution supports the following trunk test functionality.Customers can test TDM trunks via the MAPCI TTP level and ATT levelinterfaces*. Trunk test functionality is provided either by the MediaServer 2000 series products, MS 2010 (AAL5) and MS 2020 (AAL2)or the Universal Audio Server with SAGE 945. The applicable Norteldocumentation reference is provided for each MAPCI level.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 29: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Fault management strategy 29

Command Description

CktInfo Displays the name and state of the PM and the state of thecircuit of a trunk

StkSdr Allows access to the Stuck Sender feature, which identifiestrunks with outpulsing problems

Command Description

OP Permits the ability to seize and outpulse a set of desired digitsover a trunk

MWTSwap Permits the ability to send and receive a 1004 Hz test tone andmeasure transmission loss on a received test tone.

ATTENTIONCommand support available only on MS 2000 Series.

Command support for ISUP and PTS ES/OP trunk types only.

SGNL Executes a bit signaling test on a trunk. (See SGNL Commandadditional information for details.)

TST Permits the user to run a number of supported tests, includingthe following: ISUP continuity test (ICOT), T100 (originatingand terminating), T102 (originating and terminating), and T105(originating and terminating).

T105 termination on the MS 2000 series supports only the LSC,L, NSC, RN, N, and NT subtests.

CktMon Displays messages on the MAP from the digital PM of thecircuit without interrupting normal call processing.

Command Description

CVTest Performs a Circuit Validation Test on a trunk

TrkQry Displays the local or remote state of a trunk

QrySig Queries the signaling availability and the state of a trunk

RouteSet Displays the route set CLLI of a CCS7 trunk

CIC Displays the circuit identification code of a trunk

• T108 Terminating Test Line - a non-inverting digital loopbackconnection

ATTENTIONAll commands at the Automatic Test Trunk (ATT) level are supported.

SGNL Command additional information The SGNL command supportsthe following North American MG based PTS trunk variants which use thestandard/TDM PTS call model for the H.248 based protocol.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 30: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

30 Fault management fundamentals

Trunk type Start signalling type Pulse type Card type

OP Wink MF DSISIG

ES Wink MF DSISIG

ATC Wink MF DSISIG

IT Wink MF DSISIG

CELL Wink MF DSISIG

IBNT2 Wink MF, DP DSISIG

IBNTO Wink MF, DP DSISIG

IBNTI Wink MF, DP DSISIG

The SGNL command support on MG15000 PTS trunks, has the followingqualifications:

• No international trunk variants are supported

• Delay Dial and Immediate start signaling types are not supported

• Not supported in the IAC solution which employs PTS trunk variantsOP and ES hosted on TGCP protocol based trunk gateways

• Wink indication is represented by the single flash of a ‘+’ sign under theR field to the right of the posted tunk. The ‘+’ sign is only an indicationof the reception of a Wink signal and should not be interpreted asactual length of the Wink.

• Current MG15000 based PTS trunk architecture does not supportonhook signal reporting during digit collection. If the user is executingthe SGNL command whose far-end trunk is an MG15000-based PTStrunk, the following procedure applies:

— User posts supported PTS trunk circuit and executes the SGNL’onoffhk’ command.

— Far-end MG15000 PTS trunk connects to an available digit receiverand outputs a Wink signal reply.

— Upon successful reporting of Wink signal, user releases postedPTS trunk circuit trunk via the available RLS release command.

— User waits for far-end trunk digit receiver timer to expire. Uponexpiration, far-end trunk returns to idle state.

Trunk testing with the UAS Trunk testing with the UAS is provided incombination with a Sage 945RTS test unit, which has T1 interfaces directlyconnected to the UAS (to the AG4000 card in an ATM configuration orCG6000 card in an IP configuration).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 31: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Fault management strategy 31

Configure the Sage test unit to support a combination of types of testsbased on the engineering of the customer’s network. Each DS0 circuit tothe Sage is dedicated to one type of trunk test. Parallel provisioning existson the UAS node to group circuits supporting the same test type into trunkgroups. Each test type has its own trunk group. The provisioning and datarepresentation for these resources are provided by separate features withthe UAS.

The Test Line trunk tests currently supported by the UAS with Sage:

• TL100, which provides far-to-near end signal loss and noisemeasurements.

ATTENTIONThe Sage 945RTS does not provide a TL100 trunk test as defined byGR-822 (Issue 1, December 1995). It provides instead a silent terminationTL100 trunk terminator.

• TL102, which provides far-to-near end signal loss measurement.

• TL105, which provides two-way signal loss and noise measurementsalong with noise and loss self check (if available).

The above tests are available for both origination and termination and arefully integrated with the XA-Core. This means that test execution, resultsand logs are equivalent to the traditional TDM environment. Available forIP (AAL5) from SN06 and for ATM (AAL2) from SN06.2.

Trunk testing using the MS 2000 Series An H.248 configured MS 2010or MS 2020 supports testing trunks on TDM trunk gateways in CVoIPnetworks deployed in North America. Trunk testing is provided directlyfrom the MS 2000 Series and requires no additional hardware unlike theUAS.

The Test Line trunk tests currently supported by the MS 2000 Seriesinclude the following tests:

• TL100, which provides far-to-near end signal loss and noisemeasurements.

• TL102, which provides far-to-near end signal loss measurement.

• TL105, which provides two-way signal loss and noise measurementsalong with noise and loss self check (if available). **T105 terminationon the MS 2000 series supports only the LSC, L, NSC, RN, N, and NTsubtests.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 32: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

32 Fault management fundamentals

• TL904 which provides two-way signal loss at 800 Hz. (International testtrunk utilized in the Israeli telephone network.)

• MWTSwap, which permits the ability to send and receive a 1004 Hztest tone and measures transmission loss on the received 1004 Hz testtone.

The above tests are available for both origination and termination and arefully integrated with the XA-Core. This means that test execution, resultsand logs are equivalent to the traditional TDM environment. Available forIP (AAL5) is the MS2010 from (I)SN06.2 and for ATM (AAL2) is the MS2020 from (I)SN07.

Routine Exercise (REX) testingREX testing for SPM nodes REX testing is used to improve faultcoverage and system reliability on SPM nodes (including MG 4000,IW-SPM, and DPT-SPM). A REX test generally consists of a combinationof out-of-service diagnostics, in-service diagnostics and activity switchingfor duplex nodes. Some examples of REX tests are:

• CM REX test

• MS REX test

• ENET REX test

• XPM REX test

REX test of CEMs includes the following steps:

• Reset on Inactive CEM

• OOS Test on Inactive CEM

• RTS on Inactive CEM

• SWACT to the Inactive CEM

The default scheduled frequency is currently once every 7 days during theREX maintenance window. This means that ideally, REX test on an SPMnode is run once every week.

For additional information on REX testing, see the following documents:

• MG 4000 Fault Management module (NN10076-911)

• IW SPM-ATM Fault Management module (NN10077-911)

• IW SPM-IP Fault Management module (NN10078-911)

• DPT SPM ATM Fault Management module (NN10080-911)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 33: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Fault management strategy 33

REX testing for the MG 9000 Routine Exercise (REX) of intelligentcard pairs performs the diagnostics of the inactive card. During REX, theinactive card of a card pair will be diagnosed. Followed by the inactivecard, REX is executed on the active card. This occurs after a SWACT isinitiated on the active card, which now becomes inactive.

Users can input the schedule for REX on a particular NE. Intelligent cardpairs considered for REX on MG9000 are SCO/SCI/SGE (DCC), ITP, andITX cards. User can see the progress of the REX on a particular NE fromthe REX GUI of Subnet View and on NE Desktop View. Success/failure ofthe operations in REX for a given NE are also logged.

The REX NE option is provided in the configuration menu of Desktopview. On clicking the REX NE menu, the REX GUI will be opened. Usercan initiate REX on NE(s) using the REX GUI. It contains options to Add,Remove, Suspend, and Resume an NE for REX. User can perform thefollowing operations from the REX GUI:

• REX on an NE can be scheduled for a particular day and time fromthe GUI or can be executed immediately. Scheduling of REX is similarto that of Audit.

• An option is provided to remove an NE for which REX is alreadyscheduled. When removing the NE, if the REX is in progress, it wouldbe suspended.

• User can suspend the already scheduled REX.

• User can resume the suspended REX.

• User can view the results/progress details of REX executed on theselected NE.

During REX, the inactive card of the intelligent card is diagnosed. Theresults of all these operations are logged and progress of REX is shownon the progress details field of the REX GUI. Progress details are alsoshown on the REX tab of NE Desktop View. Success/Failure of SWACTand diagnostics are displayed to the user as part of the progress detailsand also in the Logs.

REX testing for the GWC REX testing scheduling facility is used toperform routine data refresh operations on inactive units of the GWCnodes. The routine data refresh operation refreshes the inactive GWCunits to ensure that the inactive units are capable and ready to take activitywhen a switch of activity (SWACT) occurs. The Core informs individualGWC units when it is time to perform their REX action, which is the datarefresh operation. During the refresh operation, the system busies theinactive unit, downloads data from the Core and the SESM to the inactiveunit, and returns the inactive unit to service.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 34: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

34 Fault management fundamentals

These routine refreshes reduce instances of data mismatch that wouldprevent a successful GWC SWACTs. By default, routine data refreshoperations occur automatically as scheduled in tables REXSCHED andOFCVAR. However, you can invoke a manual refresh using commandGWCREX NOW. The REX controller coordinates interactions betweenmanual and system-initiated routine data refresh operations. The systemrejects a manual request if the automatic refresh operation is in progresson a particular node and vice versa.

For information about GWC routine data refresh operation, see NortelGateway Controller Administration and Security ( (NN10213-611)) .

Line and loop testingFor additional information on line or loop test, see the following componentdocumentation:

• Access Care product documentation

• MG 9000 Fault Management module (NN10074-911)

• DMS legacy documentation

Trunk and carrier testingPT-AAL1 trunk testing capabilities

Developed for the AAL1 hybrid switch configuration which containsan ENET and an Integrated Service Module. The ISM is specificallydesigned to accommodate a variety of test hardware which can performmaintenance testing on TDM trunks. This solution takes advantage ofthe TDM to IP bridge capabilities of the IW-SPM, to provide connectivitybetween the Gateway TDM trunk under test and the test hardware locatedin the ISM.

This solution is fully integrated with the XA-Core. This means that testexecution, results and logs are equivalent to the Legacy TDM environment.

The Hybrid Switch Solution supports the following trunk test functionality.Customers can test TDM trunks via the MAPCI TTP level and ATT levelinterfaces.

Table 8TTP level commands

Command Description

CktInfo Displays the name and state of the PM and thestate of the circuit of a trunk

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 35: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Trunk and carrier testing 35

Table 8TTP level commands (cont’d.)

Command Description

Ckt Connects a specified circuit to a trunk

StkSdr Allows access to the Stuck Sender feature,which identifies trunks with outpulsingproblems.

Table 9Manual level commands

Command Description

LOSS Manually measures transmission loss

TGEN Manually generates a tone of desiredfrequency and level

Noise Manually measures transmission noise

OP Permits the ability to seize and outpulse a setof desired digits over a trunk

TDet Manually measures the presence of anincoming tone

HSet Connects an available headset to a trunk

SGNL Executes a bit signaling test on a trunk

TST Permits the user to run a number of supportedtests, including the following:

• ISUP continuity test (ICOT)

• T100 (originating and terminating)

• T102 (originating and terminating)

• T104 (originating and terminating)

• T105 (originating and terminating)

Table 10Monitor level commands

Command Description

MonPost Provides the user the ability to monitor a trunkcircuit in posted position

MonLink Provides the user the ability to monitor a trunkcircuit in the linked position

Montalk (L) Listen provides the user the ability to monitora trunk circuit in both directions in a listen onlymode.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 36: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

36 Fault management fundamentals

Table 10Monitor level commands (cont’d.)

Command Description

Montalk (T) Talk provides the user the ability to monitora trunk circuit in both directions whileparticipating in the call (listen and talk mode).

CktMon Displays incoming messages on the MAP fromthe digital PM of the circuit without interruptingnormal call processing.

CPos Executes a test to a Centralized AutomaticMessage Accounting (CAMA) or RemoteOperator Number Identification (RONI) trunk.

Table 11C7TTP level commands

Command Description

CVTest Performs a Circuit Validation Test on a trunk

TrkQry Displays the local or remote state of a trunk

QrySig Queries the signaling availability and the stateof a trunk

RouteSet Displays the route set CLLI of a CCS7 trunk

CIC Displays the circuit identification code of atrunk

ATTENTIONAll commands at the Automatic Test Trunk (ATT) level are supported.

The following tests are supported but are not supported by the MAPCI.For more information, see DMS-100 Family Maintenance and OperationsManual (297-8991-500), or the DMS-100 Family Trunks MaintenanceGuide (297-1001-595).

• T108 Terminating Test Line - a non-inverting digital loopbackconnection

• T101/C101 trunk call

• Digital Remote Office Test Line (ROTL) - 100/200 trunk types only

ATTENTION— Additional equipment is required.

— This is available for both TDM and DPT-SPM/MG 4000 trunk types

• Automatic Trunk Routing (ATR) - 250 trunk types only

ATTENTIONAdditional equipment is required.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 37: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

ATM loopback testing 37

Testing for DMS and SPM-based equipmentFor DMS and SPM-based equipment, the customer can test equipmentand the ATM layer using the test command that is supported on theCommunication Server 2000 Core Manager CLUIs. The equipmentconsists of hardware including the CEM and RMs. All other diagnostics,including testing of trunks and carriers, are tested at the carrier and trunktest position (TTP) levels of the MAP interface.

ATTENTIONFor additional information on trunk testing, see “Fault management trunk testing”(page 114).

The customer cannot set parameters for testing. There are only two typesof service diagnostics:

• in-service (InSv) test that runs while the system is InSv

• out-of-service (OOS) test that runs while the system is OOS

The InSv test can change states, but the OOS test cannot. If the InSv testfails, noncritical test failure causes the unit to become in-service trouble(ISTb). However, the system continues to generate the noncritical faultlog. Critical test failure results in trigger recovery and causes the unit tobecome system busy (SysB). The system discontinues fault reportingduring critical test failure.

ATM loopback testingATM Loopback testing is used for on-demand connectivity monitoring andfault localization on AAL1 bearer connections, as well as AAL5 PVCs andSVCs, and pre-service connectivity verification on AAL5 PVCs. A loopbacktest or trace through I.610 or Multiservice Switch proprietary OAM cellson ATM connections can be performed on any ATM-equipped CVoIPSpectrum peripheral (MG 4000, IW -SPM (ATM), DPT-SPM, MG 9000).This functionality is available through three different MAP levels:

• CoreConn level

• PeerConn level

• BearConn level

In addition, the AtmLb command is available for use at any MAP level forloopback requests.

CoreConn levelThe CoreConn level gives access to the AAL5 PVC Messagingconnections between the core and an MG 4000. Two commands areavailable: Tst and Trace.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 38: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

38 Fault management fundamentals

The Tst command performs an I.610 loopback test on the specified coreconnection for the posted MG 4000. The user specifies the connection totest (0 or 1), and whether to perform a segment test or an end-to-end test.The test is always originated on the posted MG 4000. A segment test isterminated at the first defined segment endpoint along the connection (maybe an ATM switch or the XA-Core). An end-to-end test is terminated atthe connection endpoint (XA-Core).

The Trace command performs an I.610 or Multiservice Switch loopbacktrace on the specified core connection for the posted MG 4000. Theuser specifies the connection to test (0 or 1), and whether to performan I.610 type trace or a Multiservice Switch proprietary trace. The traceis originated on the posted MG 4000 and terminated at the far end ofthe connection (the XA-Core). The loopback trace sends a result backfrom every port between the originating and terminating nodes. EachMultiservice Switch node returns two results - one for the ’ingress’ port(where the segment enters the Multiservice Switch node) and one for the’egress’ port (where it exits the node).

PeerConn levelThe PeerConn level gives access to the AAL5 SVC messagingconnections between an MG 4000 and its MG 4000s. PeerConn onlysupports the MG 4000. As with the CoreConn level, a Tst and Tracecommand are offered.

The Tst command performs an I.610 loopback test on the specified peerconnection for the posted MG 4000. The user specifies the peer MG4000 connection to test, and whether to perform a segment test or anend-to-end test. The test is always originated on the posted MG 4000. Asegment test is terminated at the first defined segment endpoint along theconnection (may be an ATM switch or the peer MG 4000). An end-to-endtest is terminated at the peer MG 4000 (connection) endpoint.

The Trace command performs an I.610 or Multiservice Switch loopbacktrace on the specified peer connection for the posted MG 4000. The userspecifies the peer MG 4000 connection to test, and whether to performan I.610 type trace or a Multiservice Switch proprietary trace. The traceis originated on the posted MG 4000 and terminated at the far end ofthe connection (the peer MG). The loopback trace sends a result backfrom every port between the originating and terminating nodes. EachMultiservice Switch node returns two results - one for the ’ingress’ port(where the segment enters the Multiservice Switch node) and one for the’egress’ port (where it exits the node).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 39: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

ATM loopback testing 39

BearConn levelThe BearConn level gives access to the AAL1 SVC Bearer connectionson the ATM Framework of a given SPM. At this time, BearConn supportsthe MG 4000, IW-SPM, DPT-SPM, and MG 9000. The Tst and Tracecommands are available.

ATTENTIONDPT connections are not supported by the Tst and Trace commands.

The Tst command performs an I.610 loopback test on the specified bearerconnection from the posted SPM. The user can specify the connection totest by supplying either a TID or a trunk name. The user also indicateswhether to perform a segment test or an end-to-end test. The test isalways originated on the posted SPM. A segment test is terminated atthe first defined segment endpoint along the connection (may be an ATMswitch or an end node). An end-to-end test is terminated at the end nodeof the connection.

The Trace command performs an I.610 or Multiservice Switch loopbacktrace on the specified bearer connection for the posted SPM. The usercan specify the connection on which to perform a trace by supplyingeither a TID or a trunk name. The user also specifies whether to performan I.610 type trace or a Multiservice Switch proprietary trace. The traceis originated on the posted SPM and terminated at the far end of theconnection. The loopback trace sends a result back from every node andport between the originating and terminating nodes. Each MultiserviceSwitch node returns two results - one for the ’ingress’ port (where thesegment enters the Multiservice Switch node) and one for the ’egress’ port(where it exits the node).

AtmLb commandThe AtmLb command performs an ATM LoopBack request on the ATMconnection identified by the command parameters. The connection canbe identified by a TID (node number, terminal number) or TRK name(CLLI and member number). Options are also provided for Core and Peerconnections. Parameters are provided to specify what type of loopbackto perform (i.e. Test versus Trace).

The command will determine which SPM hosts the connection in question,and will send a request to that SPM to perform the loopback. The resultsare returned to the MAP and displayed.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 40: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

40 Fault management fundamentals

Additional informationFor additional information on ATM loopback testing, see the followingdocuments:

• MG 4000 Fault Management module (NN10076-911)

• IW SPM-ATM Fault Management module (NN10077-911)

• DPT SPM ATM Fault Management module (NN10080-911)

Misconnected DS512 link diagnosticsThis PT-AAL1 feature detects DS512 host links that are misconnectedbetween the ENET (enhanced network) and the CEMs (commonequipment modules) of legacy SPMs (Spectrum peripheral modules), orDPT SPMs, or IW SPMs. You must connect DS512 host links between theENET and the SPM according to the following rules:

• Links from ENET plane 0 must terminate on CEM 0 (of the SPM) andlinks from ENET plane 1 must terminate on CEM 1.

• You must ensure that the datafill in table MNLINK, that defines whichENET links connect to which CEM ports, exactly matches the physicalconnections.

The Misconnected Link Diagnostics feature only detects misconnectedDS512 links that are connected to the wrong CEM, but that are connectedto the right port (according to table MNLINK). The message switch (MS)port that is associated with the misconnected ENET link remains systembusy until the link is properly connected. The detection of misconnectedlinks is only be possible if the CEM has software on it and the CEM is notisolated.

Figure 2 "Correct method of connecting DS512 links" (page 41) showsthe correct way to connect DS512 links from the ENET to the CEM portsof an SPM.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 41: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Misconnected DS512 link diagnostics 41

Figure 2Correct method of connecting DS512 links

Figure 3 "Misconnection of DS512 links within a CEM" (page 42) shows anexample of an improper method of connecting DS512 links from the ENETto the CEMs of an SPM. In this case, link 1 and 2, from ENET plane 0, toCEM 0 are swapped. Although this is an incorrect method of connectingDS512 links, in this example, the CEM software is not swapped acrossthe CEMs.

ATTENTIONThe Misconnected Link Diagnostics feature does not detect this type of incorrectconnection.

In the example in Figure 3 "Misconnection of DS512 links within a CEM"(page 42), the misconnected links (1 and 2) between ENET plane 0 andCEM 0 remain in-service. The MS ports associated with the misconnectedlinks are out-of-service. The system does not generate logs to indicatethat there are misconnected links.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 42: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

42 Fault management fundamentals

Figure 3Misconnection of DS512 links within a CEM

Figure 4 "Misconnection of DS512 links across CEMs (software isswapped)" (page 43) shows an example of an improper method ofconnecting DS512 links from the ENET to the CEMs of an SPM. Inthis case, link 1 from ENET plane 0, and link 1 from ENET plane 1 areswapped. With this configuration, if a misconnected link is used to load theCEM, the CEM software is swapped.

ATTENTIONThe Misconnected Link Diagnostics feature does detect this type of incorrectconnection.

In the example in Figure 4 "Misconnection of DS512 links across CEMs(software is swapped)" (page 43), the misconnected link 1 from ENETplane 0, and ENET plane 1 (to the CEMs) remains in service. Links 2, 3,and 4 to CEM 0 and CEM 1 remain in-service. The MS ports associatedwith links 2, 3, and 4 are out-of-service. The MS port associated with themisconnected links (link 1) remain in-service. The system generates Link300 logs for the two link 1s that are swapped and misconnected across theCEMs. The system does not generate a log when the fault clears.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 43: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Misconnected DS512 link diagnostics 43

Figure 4Misconnection of DS512 links across CEMs (software is swapped)

Figure 5 "Complete misconnection of DS512 links" (page 44) shows anexample of an improper method of connecting DS512 links from the ENETto the CEMs of an SPM. In this case, all the links between the ENETand CEM are swapped exactly across CEM 1 and 0 of an SPM. Thisconfiguration causes the software load to be swapped between the twoCEMs.

ATTENTIONThe Misconnected Link Diagnostics feature does detect this type of incorrectconnection.

In this example in Figure 5 "Complete misconnection of DS512 links"(page 44), all four misconnected links from ENET plane 0 to CEM 1, andall four links from ENET plane 1 to CEM 0 remain in service. The MSport associated with each of the misconnected links remains in service.The system generates Link 300 logs for each link that is swapped andmisconnected across the CEMs. The system does not generate a logwhen the fault clears.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 44: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

44 Fault management fundamentals

Figure 5Complete misconnection of DS512 links

Figure 6 "Misconnection of DS512 links across CEMs (software is notswapped)" (page 45) shows an example of an improper method ofconnecting DS512 links from the ENET to the CEMs of an SPM. Inthis case, link 1 from ENET plane 0, and link 1 from ENET plane 1 areswapped. With this configuration, if a misconnected link is used to load theCEM, the CEM software is swapped.

ATTENTIONThe Misconnected Link Diagnostics feature does detect this type of incorrectconnection.

In the example in Figure 6 "Misconnection of DS512 links across CEMs(software is not swapped)" (page 45), the misconnected link 1 from ENETplane 0, and ENET plane 1 (to the CEMs) remains in service. The MS portassociated with the misconnected links (link 1) is taken out-of-service.The system generates Link 300 logs for the two link 1s that are swappedand misconnected across the CEMs. The system does not generate a logwhen the fault clears.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 45: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Multiservice Switch/I.610 Loopback feature 45

Figure 6Misconnection of DS512 links across CEMs (software is not swapped)

The Wrong Application Data Detection feature provides detectionand diagnostic support for situations where an SPM with DS512misconnections to the ENET are mistakenly loaded with the wrongapplication data. If one or more of the DS512 links between the ENETand the SPM are misconnected, then there is a possibility of the CEM(common equipment module) receiving the wrong application data during areload restart. The phrase "wrong application data" means that the CEM inslot 7, of the SPM, receives application data for CEM 1, and the CEM, inslot 8, receives application data for CEM 0. The NODE 303 log indicatesthat a CEM contains wrong application data. In addition, MAPCI responsesat the SPM common equipment level (SPMCEDIR) and the SPM upgradelevel (SPMUPGDIR) notify the user when a CEM has been loaded withthe wrong application data.

ATTENTIONA CEM that is loaded with the wrong application data cannot return to service.

Multiservice Switch/I.610 Loopback featureThe Multiservice Switch/I.610 Loopback feature allows you to isolate faultson connections across an ATM (asynchronous transfer mode) network.Table 12 "Connection paths" (page 45) lists the connection paths that youcan troubleshoot using the Multiservice Switch/I.610 Loopback feature.

Table 12Connection paths

MAP levelSource oftrace

Destinationof trace

SignalType

CircuitType

ATMconn/CoreConn MG 4000 XA-Core AAL5 PVC

ATMconn/PeerConn MG 4000 MG 4000 AAL5 SVC

ATMconn/BearConn MG 4000 MG 4000 AAL1 SVC

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 46: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

46 Fault management fundamentals

Table 12Connection paths (cont’d.)

MAP levelSource oftrace

Destinationof trace

SignalType

CircuitType

ATMconn/BearConn MG 4000 IW SPM AAL1 SVC

ATMconn/BearConn IW SPM MG 4000 AAL1 SVC

Figure 7 "MG 4000 to XA-Core connection" (page 46) shows an MG 4000to XA-Core AAL5 PVC (permanent virtual circuit) messaging connection.

Figure 7MG 4000 to XA-Core connection

Figure 8 "MG 4000-to-MG 4000 peer AAL5 messaging connection" (page47) shows a peer SVC AAL5 messaging connection between two MG4000s.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 47: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Multiservice Switch/I.610 Loopback feature 47

Figure 8MG 4000-to-MG 4000 peer AAL5 messaging connection

Figure 9 "MG 4000-to-MG 4000 AAL1 bearer connection" (page 48) showsan SVC AAL1 bearer connection between two MG 4000s.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 48: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

48 Fault management fundamentals

Figure 9MG 4000-to-MG 4000 AAL1 bearer connection

Figure 10 "MG 4000-to-IW SPM AAL1 bearer connection" (page 49) showsan SVC AAL1 bearer connection between an MG 4000 and an IW SPM.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 49: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Multiservice Switch/I.610 Loopback feature 49

Figure 10MG 4000-to-IW SPM AAL1 bearer connection

The Multiservice Switch/I.610 Loopback feature displays the results of bothproprietary and standards-based ATM loopback queries. The four types ofqueries are as follows:

• I.610 trace

• Multiservice Switch trace (provides proprietary information forMultiservice Switch 15000 switches)

• I.610 segment end point loopback test (provides information on thesegment end-node)

• I.610 end-to-end endpoint loopback test (provides information on theconnection end-node)

Figure 11 "i.610 end-to-end loopback test and, I.610 segment end pointloopback test" (page 50) shows an I.610 end-to-end loopback test, and anI.610 segment-end-point loopback test. These two types of I.610 loopbacktests, test the connectivity between two points. The I.610 end-to-end test,tests from the beginning end point to the connection end point. The I.610segment-end-point loopback test, tests from the beginning end point to the

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 50: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

50 Fault management fundamentals

segment-end-point. In Figure 11 "i.610 end-to-end loopback test and, I.610segment end point loopback test" (page 50), the end points are shown asMG 4000s and the ATM switches as Multiservice Switch 15000s.

Figure 11i.610 end-to-end loopback test and, I.610 segment end point loopback test

Figure 12 "Multiservice Switch trace" (page 51) shows an example of theMultiservice Switch trace. You would only use the Multiservice Switchtrace on Multiservice Switch 15000 ATM switches. During the trace,the system software forwards a loopback cell from one MultiserviceSwitch 15000 to the next. Each Multiservice Switch 15000 returns twoloopback responses, one for the ingress and, and another for the egressport. In other words, each ATM node along the connection path, returnsinformation.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 51: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Multiservice Switch/I.610 Loopback feature 51

Figure 12Multiservice Switch trace

Figure 13 "I.610 trace" (page 51) shows an example of the I.610 trace.The I.610 trace is similar to the Multiservice Switch trace, except that youuse the I.610 trace on ATM switches that are supplied by vendors otherthan Nortel (in other words, ATM switches that are not Multiservice Switch15000s).

Figure 13I.610 trace

Typically, to troubleshoot an ATM connection where the ATM networkconsists of Multiservice Switch 15000s, first run an I.610 end-to-end test tocheck for continuity. If that test fails, run a Multiservice Switch trace.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 52: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

52 Fault management fundamentals

The MAPCI (maintenance and administration position commandinterpreter) is the tool used to perform Multiservice Switch/I.610 Loopbacktests or traces. The Multiservice Switch/I.610 Loopback feature uses theATMConn directory, as well as three sublevel directories below ATMConn.Perform a Multiservice Switch/I.610 Loopback test or trace by running thetest command at the appropriate sublevel of the ATMConn directory.

Figure 14 "MAPCI levels used for Multiservice Switch/I.610 Loopbackfeature" (page 52) shows the MAPCI hierarchy for the ATMConn directory.

Figure 14MAPCI levels used for Multiservice Switch/I.610 Loopback feature

ATMConn is the ATM (asynchronous transfer mode) connectivity MAPCIlevel. This MAPCI directory deals with the Spectrum Peripheral Module(SPM) interfaces to the ATM network. These interfaces are also known asthe ATM framework. The commands available at ATMConn sublevels arevalid for MG 4000s, and IW SPMs. CoreConn, PeerConn, and BearConnare the three MAPCI sublevels below the ATMConn directory.

The CoreConn directory provides maintenance, testing, and performancemonitoring commands for AAL5 PVC (permanent virtual circuit)connections from an MG 4000 to the Communication Server 2000. ThePeerConn directory provides maintenance, testing, and performancemonitoring commands for the Switched Virtual Circuit (SVC) AAL5 peermessaging connections between two MG 4000s. The BearConn directoryprovides maintenance, testing, and performance monitoring commands forthe AAL1 SVC bearer connections.

For detailed procedures on how to perform a Multiservice Switch/I.610Loopback trace or test that originates from an MG 4000, see MG 4000Fault Management (NN10076-911) on this disk. To perform a MultiserviceSwitch/I.610 Loopback test or trace that originates from a IW SPM, see IWSPM-ATM Fault Management (NN10077-911).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 53: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

USP message tracing tool 53

USP message tracing toolBefore (I)SN08, message tracing on the USP had to be done byconfiguring gateway screening criteria on a per-linkset basis. Defining agateway screening rule for the purpose of debugging the network can becomplex and may result in discarding messages unnecessarily.

(I)SN08 provides a message monitoring tool to capture messages on thesignaling links. It can be used to filter messages at the M3UA, MTP, SCCPand/or ISUP levels. The tool is independent of gateway screening.

All ANSI and ITU MTP3, ANSI SCCP and ISUP messages are decoded inthe MSU-trace on the GUI presented in a tree diagram as of USP8.1.0.This feature will add the support of M3UA message decode. Messagedecoding will be only done on the message types defined in the M3UAprotocol. Any message type that is used by the other Sigtran AdaptationLayers, such as the provisioning messages to the Core, will be displayedin hexadecimal byte string.

If the tool is invoked from the Command Line Interface, no decode isprovided; the message is captured and forwarded to the interface, but thedata remains displayed in raw data format.

The tool provides two methods to define message tracing criteria.

• First, the tool will allow filtering messages by providing byte string tomatch with, a given offset to the message, and the length of the givenbyte string.

• Secondly, the tool will allow specific field/value matching as a filter.

The fields that may be used as filters are as follows:

• Originating Point Code

• Destination Point Code

• Network Indicator

• Message Priority

• MTP3 Network Management Message Types H0H1

• SCCP Message Types

• M3UA Message Types

• ISUP Message Types

• Affected Point Code in SCMG messages

• Affected Point Code in M3UA messages

• SCCP Subsystem Number

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 54: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

54 Fault management fundamentals

• SCCP Point Code

• SCCP GT Translation Type

• SCCP GT Nature of Address

• SCCP GT Addressing Information (dialed digits)

• SCMG Format Identifier

• ISUP Circuit Identification Code (new)

• ISUP IAM CdPN (dialed digits)

Only the following ISUP messages are fully decoded:

• IAM

• SAM

• ACM

• REL

• SUS

• RES

Other ISUP message types (BLO, CQM, CON, COT, etc.) may be tracedby type, but the ISUP portion of the message is not decoded beyond themessage type.

TCAP decode is not supported. ITU14 country variants for Point Codesare supported. Partial Point Codes are not supported.

MAP-based MG 9000 testingMAP-based line testing for the MG 9000 is supported on Intl UA-IPsolutions. It uses the legacy DMS MTM/ISM hardware or RMM hardwarefor remote sites and software along with the MG 9000 Line Test Systemto perform MG 9000 line testing from the Maintenance and AdministrationPosition (MAP) terminal.

MG 9000 native lines in the central office are tested by using MTM/ISMhardware. MG 9000 native lines in the remote side are tested by usingRMM hardware.

ABI lines are tested using MTM/ISM hardware in the Central office. ABIlines in the remote site are tested using RMM hardware. To test ABIlines, the Core sends the MAP-based line test request to the GatewayControl (GWC). The GWC encapsulates this test request into an H.248package and sends the gateway of the MG 9000. the MG 9000 receivesthe MAP-based line test request from H.248 package and then sends it tothe LGC. After testing, test results are sent to Core with same route.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 55: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

OSS interface 55

OSS interfaceFault aggregation can be performed by the IEMS. Among the featuresoffered are a consolidated fault feed (ntstd, scc2, snmp or custolog) for aCVoIP network to the OSS.

North-bound fault interfaces are available for Communication Server2000, SAM21, Universal Audio Server (UAS), Gateway Controller (GWC),Multiservice Switch 15000/Media Gateway 15000 and Multiservice Switch7400/Media Gateway 7400, Ethernet Routing Switch 8600, and UniversalSignaling Point (USP). The OSS interfaces include the following interfaces:

• A single SCC2/NT Standard fault interface is provided for faults comingfrom the Communication Server 2000 Core, SAM21, Call Agent,STORM, SDM platform, Media Gateway 9000 (MG 9000), APS, SPFSplatform, UAS, and GWC.

• The Simple Network Management Protocol (SNMP) interface isavailable for Ethernet Routing Switch 8600, MCS, and USP.

• A single CORBA fault interface is provided for faults generated by theUAS, Audio Provisioning Server (APS), MG 9000, and GWC.

• An API is available on Multiservice Data Manager (MDM) for faults forthe Multiservice Switch 15000/Media Gateway 15000 or MultiserviceSwitch 7400/Media Gateway 7400. The GW service fault alarms areraised by GWCs for Multiservice Switch 15000/Media Gateway 15000.

• A SYSLOG is available for GWC, SAM21, SPFS Platform, UAS, APS,and MG 9000.

ATTENTIONThe Communication Server 2000 Core Manager also supports alarmreporting. See “Alarm Reporting” (page 61)

The user can view the hardware status information through eachcomponent’s element manager graphical user interface (GUI). Thisincludes alarms related to the TDM core, UAS, APS, GWC, MultiserviceSwitch 15000/Media Gateway 15000, Multiservice Switch 7400/MediaGateway 7400, and Ethernet Routing Switch 8600.

The SNMP/Corba interfaces described above do not report faults fromline gateways. Line gateways must have a compatible third-party elementmanager. Typically, a line gateway communicates with its elementmanager using SNMP. The methods used for communication between theelement manager and the network management layer are vendor-specific;for details, refer to the vendor’s documentation. It is the gateway supplier’sresponsibility to ensure that line gateway faults can be obtained from thegateway when required.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 56: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

56 Fault management fundamentals

In (I)SN06.2, an alternative was introduced to log/fault delivery fromeach separate element management system. Syslog can be used toaggregate some of the individual fault/log streams into the SCC2/NT STDstream on the Communication Server 2000 Core Manager/SDM. The faultstream from MDM can be passed to the SDM and converted to SCC2/NTSTD. Also, the fault streams from the Gateway Controller (GWC) andUniversal Audio Server (UAS) can be passed to the SDM using syslog,and converted to SCC2/NT STD.

Tools and utilitiesIAC Tools and Utilities

Fault management for the IAC solution is performed at the elementmanagement layer (EML) using the following managers:

• Integrated Element Management System (IEMS)

• SDM/Communication Server 2000 Core Manager

• Core and Billing Manager

• Communication Server 2000 Service Application Module 21 (SAM21)Manager

• Communication Server 2000 Gateway Controller (GWC) Manager

• Session Server Manager

• Universal Audio Server (UAS) Manager

• Media Server 2000 (MS 2000) Series

• Audio Provisioning Server (APS)

• Universal Signalling Point (USP) Manager

• Multiservice Data Manager (MDM)

• Device Manager for Ethernet Routing Switch 8600

• Call Agent Manager

• STORage Management Manager (STORM Manager)

• Nuera Configurator

• NueraView

• HP Openview

Intl IAC Tools and UtilitiesFault management for the Intl IAC solution is performed at the elementmanagement layer (EML) using the following managers:

• Integrated Element Management System (IEMS)

• SDM/Communication Server 2000 Core Manager

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 57: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Tools and utilities 57

• Core and Billing Manager

• Communication Server 2000 Service Application Module 21 (SAM21)Manager

• Communication Server 2000 Gateway Controller (GWC) Manager

• Universal Audio Server (UAS) Manager

• Media Server 2000 (MS 2000) Series

• Audio Provisioning Server (APS)

• Universal Signalling Point (USP) Manager

• Multiservice Data Manager (MDM)

• Device Manager for Ethernet Routing Switch 8600

• Call Agent Manager

• STORage Management Manager (STORM Manager)

Intl IAW Tools and UtilitiesFault management for the Intl IAW solution is performed at the elementmanagement layer (EML) using the following managers:

• Integrated Element Management System (IEMS)

• SDM/Communication Server 2000 Core Manager

• Core and Billing Manager

• Communication Server 2000 Service Application Module 21 (SAM21)Manager

• Communication Server 2000 Gateway Controller (GWC) Manager

• Universal Audio Server (UAS) Manager

• Media Server 2000 (MS 2000) Series

• Audio Provisioning Server (APS)

• Universal Signalling Point (USP) Manager

• Multiservice Data Manager (MDM)

• Device Manager for Ethernet Routing Switch 8600

• Call Agent Manager

• STORage Management Manager (STORM Manager)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 58: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

58 Fault management fundamentals

PT-AAL1 Tools and UtilitiesFault management for the PT-AAL1 solution is performed at the elementmanagement layer (EML) using the following managers:

• SDM/Communication Server 2000 Core Manager

• Multiservice Data Manager (MDM)

• Device Manager for Ethernet Routing Switch 8600

PT-IP Tools and UtilitiesFault management for the PT-IP solution is performed at the elementmanagement layer (EML) using the following managers:

• Integrated Element Management System (IEMS)

• SDM/Communication Server 2000 Core Manager

• Communication Server 2000 Service Application Module 21 (SAM21)Manager

• Communication Server 2000 Gateway Controller (GWC) Manager

• Session Server Manager

• Universal Audio Server (UAS) Manager

• Media Server 2000 (MS 2000) Series

• Audio Provisioning Server (APS)

• Universal Signalling Point (USP) Manager

• Multiservice Data Manager (MDM)

• Device Manager for Ethernet Routing Switch 8600

• Call Agent Manager

• STORage Management Manager (STORM Manager)

Intl PT-IP Tools and UtilitiesFault management for the Intl PT-IP solution is performed at the elementmanagement layer (EML) using the following managers:

• SDM/Communication Server 2000 Core Manager

• Core and Billing Manager

• Communication Server 2000 Service Application Module 21 (SAM21)Manager

• Communication Server 2000 Gateway Controller (GWC) Manager

• Session Server Manager

• Universal Audio Server (UAS) Manager

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 59: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Tools and utilities 59

• Media Server 2000 (MS 2000) Series

• Audio Provisioning Server (APS)

• Universal Signalling Point (USP) Manager

• Multiservice Data Manager (MDM)

• Device Manager for Ethernet Routing Switch 8600

• Call Agent Manager

• STORage Management Manager (STORM Manager)

PT-XA-Core and PT-SN70EM Tools and UtilitiesFault management for the PT-XA-Core and PT-SN70EM solutions isperformed at the element management layer (EML) using the followingmanagers:

• SDM/Communication Server 2000 Core Manager

UA-AAL1 Tools and UtilitiesFault management for the UA-AAL1 solution is performed at the elementmanagement layer (EML) using the following managers:

• Integrated Element Management System (IEMS)

• SDM/Communication Server 2000 Core Manager

• Core and Billing Manager

• Media Gateway 9000 (MG 9000) Manager

• Communication Server 2000 Service Application Module 21 (SAM21)Manager

• Communication Server 2000 Gateway Controller (GWC) Manager

• Universal Audio Server (UAS) Manager

• Media Server 2000 (MS 2000) Series

• Universal Signalling Point (USP) Manager

• Multiservice Data Manager (MDM)

• Device Manager for Ethernet Routing Switch 8600

UA-IP Tools and UtilitiesFault management for the UA-IP solution is performed at the elementmanagement layer (EML) using the following managers:

• Integrated Element Management System (IEMS)

• SDM/Communication Server 2000 Core Manager

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 60: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

60 Fault management fundamentals

• Core and Billing Manager

• Media Gateway 9000 (MG 9000) Manager

• Communication Server 2000 Service Application Module 21 (SAM21)Manager

• Communication Server 2000 Gateway Controller (GWC) Manager

• Universal Audio Server (UAS) Manager

• Media Server 2000 (MS 2000) Series

• Audio Provisioning Server (APS)

• Universal Signalling Point (USP) Manager

• Multiservice Data Manager (MDM)

• Device Manager for Ethernet Routing Switch 8600

• Call Agent Manager

• STORage Management Manager (STORM Manager)

Intl UA-IP Tools and UtilitiesFault management for the Intl UA-IP solution is performed at the elementmanagement layer (EML) using the following managers:

• Integrated Element Management System (IEMS)

• SDM/Communication Server 2000 Core Manager

• Core and Billing Manager

• Communication Server 2000 Service Application Module 21 (SAM21)Manager

• Communication Server 2000 Gateway Controller (GWC) Manager

• Universal Audio Server (UAS) Manager

• Media Server 2000 (MS 2000) Series

• Audio Provisioning Server (APS)

• Universal Signalling Point (USP) Manager

• Multiservice Data Manager (MDM)

• Device Manager for Ethernet Routing Switch 8600

• Call Agent Manager

• STORage Management Manager (STORM Manager)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 61: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Tools and utilities 61

SDM/Communication Server 2000 Core ManagerThe SDM/Communication Server 2000 Core Manager performs thefollowing fault management functions:

• alarm surveillance

• alarm history retrieval and browsing

• test invocation (access only)

• fault correction (access only)

• fault verification (access only)

• maintenance

• alarm configuration per device

• log collection configuration

The user can view the alarms and the status of hardware componentsthrough the SDM/Communication Server 2000 Core Manager which alsosupports the following tasks:

• Alarm Reporting

— MAP Alarm banner

— MAP subsystem and status summary field

• Troubleshooting tools and utilities

Alarm ReportingIn addition to the fault management functions listed above,SDM/Communication Server 2000 Core Manager also supportsalarms reporting. The tasks that allow generation of reports aboutalarms or inhibit the generating of reports are all performed on theSDM/Communication Server 2000 Core Manager. The tasks that allowgeneration of and viewing of information about logs are all performed onthe SDM/Communication Server 2000 Core Manager. For information onhow to configure reporting of alarms and view information about logs, seethe following component documentation:

• Communication Server 2000 Fault Management module(NN10083-911)

• Communication Server 2000 Core Manager Fault Management module(NN10082-911)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 62: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

62 Fault management fundamentals

SDM/Communication Server 2000 Core Manager events are recordedinternally to the SDM/Communication Server 2000 Core Manager in aseries of log reports. SDM/Communication Server 2000 Core Manager logreports are local and do not appear in the generic DMS log utility streamexcept log reports SDM550 and SDM650.

The SDM/Communication Server 2000 Core Manager provides anetwork-level view of SDM/Communication Server 2000 Core Manager,XA-Core or SN70EM, Communication Server 2000, IW SPM, and MG4000 fault data through the MAP interface.

For Multi-Service Gateway 4000 (MG 4000) and Interworking SPM (IWSPM), the alarm reporting system integrates event detection and alarmnotification functions. An alarm becomes active when a reduced service,reliability, or test condition occurs in the network or network element.The alarm remains active until a system event or activity performed byoperating company personnel clears the alarm condition. The alarmsystem includes audible notification and visual display through warninglights and the MAP terminal.

The MAP terminal displays alarm codes in the banner and the subsystemstatus summary field (SSSF). The alarm banner displays alarm codes thatindicate the effect of the alarm event on the network or network element.The SSSF displays alarm codes that indicate equipment faults of systemstates.

MAP Alarm bannerMAP alarm banner software monitors for equipment or software eventsor faults. For events that affect performance or redundancy, the systemraises and displays an appropriate system alarm in the MAP alarm banner.The MAP alarm banner displays alarm codes that indicate the effect of thealarm event on the system. For an example of a memory limit (MemLim)alarm with a critical severity level, see Figure 15 "MAP alarm banner"(page 62).

Figure 15MAP alarm banner

For information on the alarm banner, see Table 13 "MAP alarm bannersummary field" (page 63).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 63: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Tools and utilities 63

Table 13MAP alarm banner summary field

Alarm banner Description

Communication Server2000 systemheaders

The Communication Server 2000 headers permanently display aset of titles indicating the different switch Communication Server2000 nodes. The XA-Core MAP system displays the CommunicationServer 2000 headers on all XA-Core MAP levels.

Alarm status codes The alarm status codes indicate the system alarm status. Thealarm status fields display the alarm status code under each of theCommunication Server 2000 headers. If there are multiple faultsin a single switch system, the alarm status field displays the mostimportant fault.

Alarm severity The alarm severity field displays the severity of the alarm condition.Each Communication Server 2000 subsystem reports on critical,major and minor alarm conditions. If three are multiple faults ina single switch system, the alarm severity field displays the mostimportant fault. The alarm banner displays the alarm severity underthe alarm indicator in XAC header.

For information on possible alarm severity conditions, their status anddescription, see Table 14 "Alarm severity conditions" (page 63).

Table 14Alarm severity conditions

Alarm Status Description

"." No alarm A no alarm indicates that there are no problems in the network.

"m" minor alarm A minor alarm indicates a small loss of redundancy. The next faultof the same type does not cause degradation of service.

"M" Major alarm A major alarm indicates lost redundancy. The next fault of the sametype can cause a reduction or complete loss of service. There isno backup if another fault occurs on the active system. This alarmlevel can be generated when service decreases below an operatingcompany defined threshold.

"C" Critical alarm A critical alarm indicates a reduced service condition or completeloss of service. A critical alarm indicates that the system can nolonger perform its design function. The alarm condition requiresimmediate correcting action so that the performance of the systemcan return to its design function.

MAP subsystem and status summary field The subsystem statussummary field (SSSF) displays the working status of the equipmentsubsystems. The SSSF has three rows and four columns of fields that areshown in Figure 16 "XA-Core MAP subsystem and status summary field"(page 64).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 64: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

64 Fault management fundamentals

Figure 16XA-Core MAP subsystem and status summary field

Table 15 "MAP subsystem and status summary field" (page 64) showsthe MAP subsystem and status summary field. The alarm banner showsone active alarm at a time in the alarm status code field. The alarm bannerdisplays the most important alarm.

Table 15MAP subsystem and status summary field

MAP subsystem Description

Subsystem headers The subsystem headers permanently display a set of titles thatdescribe the equipment subsystems.

Hardware alarm codes The hardware alarm code fields display hardware alarms for eachsubsystem type. If there are multiple faults in a single CP/packlettype, the hardware alarm field displays the most important.

Out-of-service (OOS) circuitpack/packlet count

The OOS CP/packlet count fields display the total number ofout-of-service CPs or packlets in each of the subsystems. The countcan increase for each alarm condition. The count does not increasefor noncritical device faults and trouble conditions.

Analyzing fault data from DMS and SPM-based equipment DMSand SPM-based equipment uses the DMS proprietary states to report thestatus of network devices. The following list identifies the DMS proprietarystates:

• Offline (Offl)

• Manual Busy (ManB)

• In Service (InSv)

• In-service Trouble (ISTb)

• C-side Busy (CBsy)

• System Busy (SysB)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 65: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Tools and utilities 65

Troubleshooting tools and utilitiesThe Communication Server 2000 Core Manager uses tools that are withinthe software to gather information for troubleshooting. These tools apply tothe different elements of CVoIP Solutions:

• Traver - used to show the path a call takes from the switch perspective

• logutil - used to view logs that have been recently generated

• Dlog - disk volume stores logutil messages

• OMSHOW - displays performance metrics collected by CommunicationServer 2000 equipment as well gateway equipment.

Recording information from the Communication Server 2000Core Manager, XA-Core, SN70EM, or SPM based logFigure 17 "Log with SCC2 header format" (page 65) shows an example ofan XAC 300 log with SCC2 header. The major alarm priority (**) is shown(see arrow 1). The log name and number (XAC300) is shown (see arrow2). The date and time (JUL 28 01:38:37) is shown (see arrow 3). Thealarm name (LowSM (Low Shared Memory)) is shown (see arrow 4).

Figure 17Log with SCC2 header format

Enhanced log information for SPM logsThe Enhanced Logs feature provides additional text for the followingclasses of log reports:

• ATM

• CARR

• CCMT

• DPTM

• IWBM

• SPM

Two enhancements are made:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 66: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

66 Fault management fundamentals

• identifying the SPM node type for ATM and CARR logs

• identifying the fabric for all SPM-related logs

The ATM and CARR logs provide information on the four SPM (Spectrumperipheral module) network elements: IW SPM, DPT SPM, MG 4000 andlegacy SPMs. The additional text, in the ATM, and CARR logs, consistsof the SPM location (SPM number) and SPM type (DMSCP, IW, SMG4,DPT). Table 16 "Mapping type value to SPM node" (page 66) maps thefour SPM network elements to the values in the SPM type field.

Table 16Mapping type value to SPM node

Value in the typefield of the ATM log Corresponding SPM network element

DMSCP Legacy SPM

SMG4 MG 4000

IW IW SPM

DPT DPT SPM

The value for Fabric comes from table MNCKTPAK and the Fabric valueswill be assigned according to the following table.

Table 17Fabric values from table MNCKTPAK

Datafilled CPKTYPE Fabric

GEM IP

ATM ATM

anything other value UNK

not datafilled UNK

ATTENTIONIf the Type in table MNODE is DMSCP, then the fabric portion of the line willnot be used.

Figure 18 "ATM300 log with SCC2 header format" (page 67) shows anexample of an SCC2 header format for an ATM300 log. The log name andnumber (ATM300) is shown (see arrow 1). The alarm name (ATM LCDAlarm) is shown (see arrow 2). The location of the network element andthe network element type (in this case a DPT SPM) is shown (see arrow3). Additionally, the fabric (in this case ATM) is shown (see arrow 4).

For the ATM and CARR logs, the location and type field (see arrow 3) onlyappears in the SN05 and following software release and only if the officeparameter SPM_ENHANCED_OUTPUT in table OFCVAR is set to ON.The fabric field (see arrow 4) only appears in SN06 and following software

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 67: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Tools and utilities 67

releases if the office parameter SPM_ENHANCED_OUTPUT is set to ON.If the office parameter SPM_ENHANCED_OUTPUT is set to OFF, thelocation and type fields do not appear on the ATM and CARR logs. NewATM and CARR logs created in SN05 or later display the location and typefields by default, regardless of the value of SPM_ENHANCED_OUTPUT.

Figure 18ATM300 log with SCC2 header format

Table 18 "Comparison of an ATM log with and without enhanced text"(page 67) shows the ATM300 log on both NT Standard and SCC2 logformats. In addition Table 18 "Comparison of an ATM log with and withoutenhanced text" (page 67) shows the ATM300 log with and without theenhanced text that gives the location and type of SPM (see bold text inrow two and four of Table 18 "Comparison of an ATM log with and withoutenhanced text" (page 67)). The enhanced text is only visible when officeparameter SPM_ENHANCED_OUTPUT is set to ON in table OFCVAR.You can turn the office parameter OFF and ON as frequently as youwish without causing negative effects. Changing this office parameterdoes not require a restart or any other switch action. The settings of theSPM_ENHANCED_OUTPUT parameter are able to survive all restarts.

Table 18Comparison of an ATM log with and without enhanced text

Log Format Four variations of the ATM300 log

NT Standard(no enhancedtext)

RPTN04BF**

ATM300 FEB16 09:22:24 3496 FLT ATMLCD Alarm RaisedLocation: SPM 0 ATM RM in Slot:10 (InAct )Description: ATM LCD alarm raised.

NT Standard(with enhancedtext)

RPTN04BF**

ATM300 FEB16 09:22:24 3496 FLT ATM LCD AlarmRaisedLocation: SPM 0 ATM RM in Slot:10 (InAct )Description: ATM LCD alarm raised.Location: SPM 0 Type: DPT Fabric: ATM

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 68: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

68 Fault management fundamentals

Table 18Comparison of an ATM log with and without enhanced text (cont’d.)

Log Format Four variations of the ATM300 log

SCC2(no enhancedtext)

00ATM

300 6217 INFO ATM LCD Alarm RaisedLocation: SPM 0 ATM RM in Slot:10 (InAct )Description: ATM LCD alarm raised.

SCC2(with enhancedtext)

00ATM

300 6217 INFO ATM LCD Alarm RaisedLocation: SPM 0 ATM RM in Slot: 10(InAct )Description: ATM LCD alarm raised.Location: SPM 0 Type: DPT Fabric: ATM

Figure 19 "CCMT501 log with SCC2 header format" (page 68) shows anexample of an SCC2 header format for a CCMT501 log. The log nameand number (CCMT501) is shown (see arrow 1). The location of thenetwork element, the network element type (in this case an MG 4000), andthe fabric (in this case ATM) is shown (see arrow 2).

For the CCMT, DPTM, IWBM, and SPM logs, the location and typefield and the fabric field (see arrow 2) only appears in the (I)SN06and following software release and only if the office parameterSPM_ENHANCED_OUTPUT in table OFCVAR is set to ON. If the officeparameter SPM_ENHANCED_OUTPUT is set to OFF, the location andtype fields do not appear on the logs. New logs created in (I)SN06 or laterdisplay the location and type fields by default, regardless of the value ofSPM_ENHANCED_OUTPUT.

Figure 19CCMT501 log with SCC2 header format

Table 18 "Comparison of an ATM log with and without enhanced text"(page 67) shows the CCMT501 log on both NT Standard and SCC2 logformats. In addition Table 18 "Comparison of an ATM log with and without

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 69: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Tools and utilities 69

enhanced text" (page 67) shows the CCMT501 log with and without theenhanced text that gives the location and type of SPM (see bold text inrow two and four of Table 18 "Comparison of an ATM log with and withoutenhanced text" (page 67)). The enhanced text is only visible when officeparameter SPM_ENHANCED_OUTPUT is set to ON in table OFCVAR.You can turn the office parameter OFF and ON as frequently as youwish without causing negative effects. Changing this office parameterdoes not require a restart or any other switch action. The settings of theSPM_ENHANCED_OUTPUT parameter are able to survive all restarts.

Table 19Comparison of a CCMT log with and without enhanced text

Log Format Two variations of the CCMT501 log

NT Standard(no enhancedtext)

RTPN05AV**

CCMT501 JUN12 02:15:09 9700 INFO Core ConnectionState ChangeCore: LINK 2 CONN 1 VPI 0 VCI 182TAG 0Node: SMG4 20 VPI 0 VCI 51Failed to enable PVC segment.PVC segment is disabled.0000 0000 0000 0000 0000 0000 0000 00000000 0000 0000 0000 0000 0000 0000 0000

NT Standard(with enhancedtext)

RTPN05AV**

CCMT501 JUN12 02:15:09 9700 INFO Core ConnectionState ChangeCore: LINK 2 CONN 1 VPI 0 VCI 182 TAG0Node: SMG4 20 VPI 0 VCI 51Failed to enable PVC segment.PVC segment is disabled.0000 0000 0000 0000 0000 0000 0000 00000000 0000 0000 0000 0000 0000 0000 0000Location: SPM 20 Type: SMG4 Fabric: ATM

SCC2(no enhancedtext)

00CCMT

501 5921 INFO Core Connection State ChangedCore: LINK 2 CONN 1 VPI 0 VCI 182 TAG 0Node: SMG4 20 VPI 0 VCI 51Failed to enable PVC segment.PVC segment is disabled.0000 0000 0000 0000 0000 0000 0000 00000000 0000 0000 0000 0000 0000 0000 0000

SCC2(with enhancedtext)

00CCMT

501 5921 INFO Core Connection State ChangedCore: LINK 2 CONN 1 VPI 0 VCI 182 TAG 0Node: SMG4 20 VPI 0 VCI 51Failed to enable PVC segment.PVC segment is disabled.0000 0000 0000 0000 0000 0000 0000 0000

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 70: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

70 Fault management fundamentals

Table 19Comparison of a CCMT log with and without enhanced text (cont’d.)

Log Format Two variations of the CCMT501 log

0000 0000 0000 0000 0000 0000 0000 0000Location: SPM 0 Type: SMG4 Fabric: ATM

Table 20 "SPM-related logs that have the optional location and type fields"(page 70) lists the SPM-related logs that display the optional SPM locationand type field when office parameter SPM_ENHANCED _OUTPUT is setto ON (in table OFCVAR).

Table 20SPM-related logs that have the optional location and type fields

Log number Description

ATM300 ATM LCD Alarm raised

ATM301 ATM LCD alarm cleared

ATM500 ATM ILMI status

ATM501 ATM Signaling status

ATM600 ATM Address registration

ATM601 ATM Address de-registration

ATM604 ATM Recovery audit

ATM605 ATM Overload event

CARR300 Carrier failure event set

CARR310 Carrier failure event cleared

CARR330 Facility protection switch

CARR331 Failed facility protection switch

CARR340 Simplex set

CARR341 Simplex clear

CARR500 Carrier state change to InSv from ManB or SysB

CARR501 Carrier state change to CBsy from SysB or ManB

CARR510 Carrier state change to ManB from InSv, SysB, orC-side CBsy

CARR511 Carrier state change to SysB from InSv or CBsy

CARR512 OC3 carrier state changes to CBsy from InSv,ManB, or SysB

CARR800 Threshold crossing alert (TCA)

CARR801 Maintenance limits reset

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 71: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Tools and utilities 71

Table 20SPM-related logs that have the optional location and type fields (cont’d.)

Log number Description

CARR810 Threshold crossing alert (TCA) set (LBC,OPT,OPR)

CARR811 Threshold crossing alert (TCA)

CCMT301 Core PVC Segment Fault

CCMT501 Core Connection State Change

CCMT502 Peer Connection State Change

CCMT601 Core PVC Segment Fault Cleared

DPTM500 INSV State change

DPTM501 SYSB State change

DPTM502 MANB State change log

DPTM503 PMB State change log

DPTM504 INB State change

DPTM 700 DPT DDM Data Mismatch

DPTM701 DPT Data Mismatch Clear

IWBM500 IW_Bridges OOS

IWBM501 IW_Bridges INSV

IWBM600 Connectivity Mismatch

IWBM601 Audit Action

IWBM900 Bridge depleted

SPM300 Device fault

SPM301 Clock Range

SPM310 Suspend Echo Canceller Report

SPM311 SW Exception Report

SPM312 TBL Trap

SPM313 MIM Fault

SPM314 TBL IMC

SPM330 SPM Datasync Report

SPM331 Failed Protection Switch

SPM332 Sync Reference Switched

SPM333 SPM REX Failed

SPM340 FAIL CM WarmSwact

SPM350 Free Resources Low

SPM500 Device State Change

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 72: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

72 Fault management fundamentals

Table 20SPM-related logs that have the optional location and type fields (cont’d.)

Log number Description

SPM502 ATM Connection State Change

SPM503 ATM Carrier State Change

SPM504 ATM Device State Change

SPM600 MS Mode Change

SPM630 Successful Sparing Event

SPM632 SPM REX start time

SPM633 SPM REX Success

SPM650 SPM Device Loader Success

SPM651 SPM Device Loader Failure

SPM660 Continuous Echo Canceller Performance

SPM661 Continuous Echo Canceller Performance Change

SPM680 MBM log message

SPM681 MBM Pool Low

SPM700 DDM Audit Fail

SPM701 DDM Audit Succeeded

SPM702 DDM Dynamic Update Failed

SPM703 DDM Audit Updated Trunk

SPM704 DDM Dynamic Update failed for trunk

SPM705 Trunk State Change

SPM706 Trunk Automatic RTS

SPM707 ISDNPARAMETER DDM Dynamic Update

SPM708 ISDNPARAMETER DDM Audit

SPM709 ISDNPROT DDM Dynamic Update

SPM710 ISDNPROT DDM Audit

For detailed information on setting office parameter SPM_ENHANCED_OUTPUT in table OFCVAR see the following:

• MG 4000 Operational Configuration (NN10098-511)

• IW SPM (ATM) Operational Configuration (NN10099-511)

• IW SPM (IP) Operational Configuration (NN10100-511)

• DPT SPM (ATM) Operational Configuration (NN10102-511)

For additional information on the Enhanced Logs feature, see thefollowing:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 73: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Tools and utilities 73

• MG 4000 Fault Management (NN10076-911)

• W SPM (ATM) Fault Management (NN10077-911)

• IW SPM (IP) Fault Management (NN10078-911)

• DPT SPM (ATM) Fault Management (NN10080-911)

Logging in to the Map and accessing the maintenance levelFor detailed information on logging in to the MAP and understandingthe general display, see the DMS-100 Family Maintenance SystemMan-Machine Interface Description, 297-1001-520 on Helmsman.

After logging in to the maintenance level of the MAP, the alarm statusindicators along the top status area will be seen (see Figure 20 "Examplescreen showing the maintenance level of the MAP" (page 73)). The alarmstatus indicators appear under the subsystem codes (for example, XAC,MS, and IOD are subsystem). The dot (*) under a subsystem indicatesthere is no alarm. When an alarm is present, the priority is shown (*C*) iscritical, M is major, and blank is minor) with a numerical value to indicatethe number of alarms of a given priority.

The log information recorded earlier will assist in finding the alarm indicatorin the status banner, under the appropriate subsystem. If there is morethan one alarm, troubleshoot the highest priority (most critical) alarm first.

Figure 20Example screen showing the maintenance level of the MAP

Navigating to the appropriate subsystem of the MAPFigure 21 "Example MAP display of the XAC subsystem" (page 75) showsan example of the MAP maintenance level for the XAC subsystem andFigure 22 "Example MAP display of the PM subsystem" (page 75) showsan example of the MAP maintenance level for the PM (peripheral module)subsystem.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 74: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

74 Fault management fundamentals

ATTENTION• Although DPT (dynamic packet trunk) alarms appear under the PM

subsystem on the alarm banner, you obtain additional informationabout DPT alarms by accessing the DPTTRM (DPT Terminal ResourceManager) level of MAP. The DPTTRM level is found under the TRKS(trunks) subsystem. To view the DPTTRM level of MAP, use the followingpath: MAPCI;MTC;TRKS;DPTTRM. For information on clearing DPTalarms appearing under the PM banner, see MG 4000 Fault Management(NN10076-911), and DPT SPM ATM Fault Management (NN10080-911).

• The DPTRKS level of the MAP is the subsystem where you can view thestatus of the CICs (circuit identification codes) that are datafilled againsteach DPT. For an example of the DPTRKS level of MAPCI, see Figure 23"DPTRKS level of MAPCI" (page 76). The DPTRKS level is found underthe TRKS (trunks) subsystem. To view the DPTRKS level of MAP, use thefollowing path: MAPCI;MTC;TRKS;DPTRKS. The DPTRKS level also allowsyou to reset single CICs or ranges or CICs, and to busy and return to serviceCICs.

• There are three ways to detect STS-1 carrier faults on the MG 4000. Thefirst way is by means of the DMS LOGUTIL system (logs 300 and 310).Secondly, STS-1 carrier faults appear as SYSB at the carrier MAP level.Lastly, use the Listlalarm command at the carrier MAP level, to display anyfaults for a posted carrier.

• You can use the QueryPM command with the Reason option (for example,QueryPM Flt Reason), for posted SPMs (Spectrum peripheral modules)to find out the reason why the RM (resource module), or CEM (commonequipment module) is in an in-service trouble or system busy state. You canuse this command on SPMs such as the IW SPM, MG 4000, DPT SPM,as well as legacy SPMs connected to the ENET. The QueryPM commandwith the Reason option is available at the PM level of the MAP once you’veposted one or more SPMs.

A list of alarms can be displayed at the PM level of the MAP by using theListAlm (list alarms) and Disp_ (display alarms) commands. Current logscan be viewed at any time on the MAP by using the LOGUTIL tool. Fordetailed information on the log system see North American Log ReportReference Manual (297-8021-814) on Helmsman.

For detailed information on telescoping between subsystems of theMAP, see DMS-100 Family Maintenance System Man-Machine InterfaceDescription (297-1001-520) on Helmsman.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 75: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Tools and utilities 75

Figure 21Example MAP display of the XAC subsystem

Figure 22 "Example MAP display of the PM subsystem" (page 75) showsan example of the MAP maintenance level for the PM (peripheral module)subsystem.

Figure 22Example MAP display of the PM subsystem

DPT and DPT trunk maintenance MAPCI level enhancementsThe DPT trunks (DPTRKS) MAPCI level includes a display of thenumber of call identifier codes (CICs) that are in the maintenance (MTC)state. There are three sub-statuses of this maintenance state: PV(protocol violation), FE_BLK (far_end blocked), and FE_UnEQ (far_endunequipped). (See Figure 23 "DPTRKS level of MAPCI" (page 76).) InFigure 23 "DPTRKS level of MAPCI" (page 76), the three sub-statuses areshown in bold type with red underline. Beside each sub-status name is thenumber of CICs currently in that maintenance state. In addition, the busy(BSY_) command (without any arguments) and return to service (RTS_)command (without any arguments) are available at the DPTRKS level.

For procedures on how to use the commands at the DPTRKS level ofMAPCI, see MG 4000 Configuration Management (NN10098-511), or DPTSPM ATM Operational Configuration (NN10102-511).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 76: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

76 Fault management fundamentals

When BICC DPT trunk groups go out of service, the local switchautomatically sends group blocking messages to the far-end switch.Reacting to the group blocking messages, the far end switch sets theassociated CICs to a state of FE BLK (far_end blocked) so that no traffic isdirected from the far end switch to the local switch. When BICC DPT trunkgroups return to service, group unblocking messages are automaticallysent from the local switch to the far-end switch. Once the group unblockingmessages are received, the far-end switch sets the associated CICs to astate of FE IDL (far_end idle) to allow them to handle traffic again.

Figure 23DPTRKS level of MAPCI

The local switch does not send out a group blocking message to a far-endswitch if the affected CICs (either the starting CICs or the entire range ofCICs) at the far-end switch are not provisioned. In other words the CICsare in a FE_UNEQ (far_end unequipped) state, the local switch does notsend out a group blocking message.

If the far-end switch receives a group blocking message and the startingCIC in the range (or the entire range of CICs) are not provisioned, or therange state of the DPT trunk is in INB, then the far-end switch sends outone UCIC (unequipped circuit identification code) message to the localswitch. Next, the far-end switch discards the group blocking message.

Fast routeset recovery after switch initializationNortel has defined a proprietary group reset message that resets all of theCICs (circuit identification codes) in one routeset. This super group resetmessage reduces the time necessary for routeset recovery. In SN05 aswitch can have up to 38 million CICs.

In order to use the super group routeset message, provision theSGRPRSC field in table ADJNODE (adjacent node).The super grouprouteset message only works in networks where the far end switch is aNortel DMS (digital multiplex system) switch running a CVoIP (I)SN05(or higher) software load. Any far-end switch that does not meet these

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 77: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Tools and utilities 77

requirements treats the group reset message as unrecognizable anddiscards the message. For information on provisioning the SGRPRSCfield in table ADJNODE, see Communication Server 2000 ConfigurationManagement (NN10201-511).

Assuming the far-end switch is a DMS running (I)SN05 or higher, the localswitch continues to send the group reset message in 60 second intervalsuntil one of the following events occurs:

• the local switch receives the group reset acknowledgement from thefar-end switch

• all of the ranges within the routeset are placed in an INB (installationbusy) state through manual intervention at the MAPCI using the busyor installation busy commands

• the super group reset option (SGRPRSC) is deleted in tableADJNODE.

During a reload or a cold restart, the DPT range state is set to the RINI(range initialization) state if the SGRPRSC field is set in table ADJNODE.After the local switch receives an acknowledgement for the super groupreset message, the local switch changes the RINI state to the recoverystate for the routeset to which the DPT trunk group belongs.

The RPV (range protocol violation state) for DPT trunks indicates that thefar-end switch has not acknowledged a message from the local switch.The RPV state is replaced by the recovery state after the local switchreceives the acknowledgement. When the RPV state is set on a range ofCICs, the call processing software can not use any CICs within that range.You can use the manual MAPCI commands "bsy inb" to change the RINIor RPV state to INB (installation busy).

Clearing flash memory before a CEM is moved orde-commissionedFlash memory in the common equipment module (CEM) of an SPM(Spectrum peripheral module) stores the IP (Internet Protocol) address,and the load of the CEM. If you move the CEM from the current SPM shelfto a different location or platform (or de-commission the CEM), you mustclear the flash memory of the CEM. Otherwise, the information in the CEMmay cause the following problems to occur:

• there are two CEMs trying to own the same IP address

• the CEM is not able to recover due to a mismatch in the IP address

• the CEM may auto-boot an incorrect software load

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 78: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

78 Fault management fundamentals

The erase flash (ERASEFL) command can be used at the MAPCI CEMlevel to erase the flash memory of CEMs. The erase flash command issupported for legacy SPMs, IW SPMs, MG 4000s, and DPT SPMs.

For detailed procedures telling you how to erase CEM flash memory, seethe following component documentation:

• IW SPM (ATM) Fault Management (NN10077-911)

• MG 4000 Fault Management (NN10076-911)

• DPT SPM Fault Management (NN10080-911)

• SPM Fault Management (NN10075-911)

Additional informationFor additional details on fault management tools and utilities, see thefollowing component documentation:

• SPM Fault Management (NN10075-911)

• MG 4000 Fault Management (NN10076-911)

• IW-SPM (ATM) Fault Management (NN10077-911)

• IW-SPM (IP) Fault Management (NN10078-911)

• SDM Fault Management (NN10081-911)

• Communication Server 2000 Core Manager Fault Management(NN10082-911)

• Communication Server 2000 Fault Management (NN10083-911)

Communication Server 2000 SAM21 Manager

ATTENTIONThe Communication Server 2000 SAM21 Manager does not apply to thePT-AAL1 solution.

The Communication Server 2000 SAM21 Manager performs the followingfault management functions:

• Manages faults in the individual shelf slots and the hardwareconfigured and installed in the application slots

• Manages Maintenance functions associated with the individual shelfslots and the hardware configured and installed in the application slots

• Uses the provisioning information to deliver a graphical representationof each SAM21 shelf within the call server

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 79: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Tools and utilities 79

• Uses the graphical representation of each SAM21 shelf to identify,acknowledge, study, resolve and clear alarms

• Generates SAM21 log reports to indicate

— raising an clearing of alarms for all cards in the shelf

— state transition for all card types in the shelf

— card insertion and removal for all card types

— ATM faults if the Shelf Controller uses the ATM interface and isdeployed in an ATM network

Log reporting for Communication Server 2000 ManagementTools applicationsAll applications and devices under the Communication Server 2000Management Tools use the SPFS Logging API to record their logs. ThisAPI doubles as the custlog feed to the OSS Fault Collector. It supportsfive levels of logs:

• critical

• major

• minor

• warning

• none

The API also provides four types of logs

• customer logs - alarms and other customer visible events

• audit logs - user actions taken

• authentication logs -security events

• debug logs - software or hardware errors

Controlling the forwarding of logs is done through use of the syslog.conffile on the SPFS machine. This file (a standard syslog file) is used bysyslog to dictate its behavior. This file is a simple text file that can beedited to define whether to record the logs on the local machine and/orwhether to send them to another machine (i.e. the OSS Fault Collector).The level of control is on a log level basis, so critical logs can be handledone way and warning logs another. By default, the only logs that areforwarded are the customer logs (alarms), although, as noted above, thisis customizable.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 80: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

80 Fault management fundamentals

ATTENTIONCurrently, the Communication Server 2000 GWC Mgr and CommunicationServer 2000 UAS Mgr send logs northbound though other 3rd party interfacesas well.

Media Server 2000 (MS 2000) Series

ATTENTIONThe MS 2000 Series does not apply to the PT-AAL1 solution.

MS 2000 Series fault reporting is based on Simple Network ManagementProtocol (SNMP). The MS 2000 Series carrier-grade alarm systemprovides a reliable alarm reporting mechanism that takes into accountelement manager outages, network outages, and an unreliable transportmechanism (SNMP over UDP). A carrier-grade alarm system ischaracterized by the following capabilities:

• a mechanism to allow an EM to determine which alarms are currentlyactive in the NE. (The NE maintains an active alarm table.)

• a mechanism to allow an EM to detect lost alarm raise and clearnotifications. [seq no in trap, current seq no mib object]

• a mechanism to allow an EM to recover lost alarm raise and clearnotifications [keep a log history]

• the ability to send a cold start trap to indicate that it is starting. Thisallows the EM to synchronize its view of the NE’s active alarms.

• clear alarms before shutting down if possible.

Both a listing of active alarms and a history of alarm are maintained on theMS 2000 Series servers using the standard NOTIFICATION-LOG- MIBand draft of the ALARM-MIB for the alarm history and active alarms.

The source for these traps is a proprietary enterprise SNMP MIB for the2000 Series, called "AcBoard". Traps are generated off the Trunk PackModules (TPMs). Either TPM for the MS 2010 can generate traps. In thecase of the MS 2020, there is only one TPM.

Customer logsThe traps mentioned above are sent from the MS 2000 series. Thesetraps are received and processed by the IEMS system in the network. TheMS 2000 server must be added to IEMS for monitoring and management.MS numbers are added to the MS 2000 SNMP traps as they areprocessed by the management server and are visible in the IEMS userinterface.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 81: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Integrated Element Management System 81

BootP operationA BootP server is configured on the SDM or CBM for the MS 2000 Series.The BootP server is needed, initially, for assigning an IP address for a MS2000 Series node when it is installed. After installation is complete, theBootP server is used as a backup configuration repository for MS 2000Series software loads (CMP file) and the basic configuration file (INI file).

A request to reload the CMP and INI files to a MS 2000 Series node fromthe BootP server is generated in response to the following events:

• the MS 2000 Series node is power-cycled

• the MS 2000 series is reset

• a watchdog process running on the MS 2000 Series node determinesthat a fatal error has occurred in the IPM-1610 board

Integrated Element Management SystemFault management for IEMS consists of working with alarms and events,as well as configuring Northbound fault feeds. IEMS supports both a JavaWeb Start and HTML client.

Working with Alarms in the Java Web Start Client and HTML ClientThe IEMS Alarm browser provides a consolidated real-time view of theevents that have occurred in a Communication Server 2000 central office.It provides a tool to view and page through the alarms from the NortelEMs, NEs, platforms, and applications in a common graphical interface.The Alarms browser provides a wide range of features to manage andview alarms in a centralized location. Alarms are generated when a faultis detected in a network device. The devices forward these alarms to theIEMS. The alarms have one of the following severities:

• Critical

• Major

• Minor

• Warning

The significance of the color (background color of rows in Alarms panel)for each severity levels is listed in the following table.

Color Severity Level

Critical

Major

Minor

Warning

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 82: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

82 Fault management fundamentals

To access the Java Web Start Client Alarms browser, select the Alarmsnode (sub-node of Fault Management) in the IEMS tree. The followingfigure illustrates the Java Web Start Client Alarms browser.

Figure 24Java Web Start Client Alarms browser

To access the HTML Client Alarms browser, select the Fault Managementtab at the top of the page, and then select Alarms on the left side of therefreshed browser window. The following figure illustrates the HTML ClientAlarms browser.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 83: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Integrated Element Management System 83

Figure 25HTML Client Alarms browser

For additional information, see IEMS Fault Management (NN10334-911).

Working with Events in the Java Web Start Client and HTML ClientThe IEMS event browser provides a consolidated historical and real-timeview of the events that have occurred in a Communication Server 2000central office. Event browser is a tool to view the events from the NortelEMSs, NEs, platforms, and applications in a common GUI. The eventbrowser serves as a key tool that enables the monitoring and debugging ofnetwork activities and issues.

To access the Java Web Start Client Events browser, select the NetworkEvents node (sub-node of Fault Management) in the IEMS tree. Thefollowing figure illustrates the Java Web Start Client Event browser.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 84: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

84 Fault management fundamentals

Figure 26Java Web Start Client Events browser

To access the HTML Client Events browser, select the Fault Managementtab at the top of the page, and then select Network Events on the left sideof the refreshed browser window. The following figure illustrates the HTMLClient Events browser.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 85: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Core and Billing Manager 85

Figure 27HTML Client Events browser

For additional information, see IEMS Fault Management (NN10334-911).

Configuring Northbound fault feedsThe IEMS standardizes the fault interfaces from the various EMSs, NEs,applications, and platforms that it manages. It normalizes the eventsreceived from these interfaces and converts them into a common format.The IEMS supports the following northbound event interfaces.

• SCC2

• SNMP

• Customerlog SYSLOG

• NTSTD

Providing a common set of northbound OSS interfaces that are based oncommon standards simplifies the effort for third party vendors to integrateand monitor the event stream from a Nortel office.

For additional information, see IEMS Fault Management (NN10334-911).

Core and Billing ManagerATTENTIONThe Core and Billing Manager (CBM) does not apply to the PT-AAL1 solution.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 86: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

86 Fault management fundamentals

The CBM provides fault management functions for alarms and logs.

CBM alarmsThe CBM provides the alarm status dynamically in the CBM maintenanceinterface (MI). The status of the CBM and its components are alsoavailable through command line query commands.

The CBM’s aggregated product state will also be reflected in the serverLEDs for critical, major and minor alarms as well as through the officealarm unit (OAU) in the breaker interface panel of the PTE2000, MIS orCMIS frame. The CBM800 link alarms are reported on the Core. TheCBM800 aggregated node state will be aggregated with the link states andalarmed on the Core.

CBM logsCBM applications and services generate customer logs. These customerlogs are written to persistent storage in a simple text file. These files arerotated daily or when disk space is limited. The CBM base will generate anaggregated system log describing the overall system state.

These CBM logs as well as the managed Core logs are provided to thecustomer in SCC2 or Nortel Standard format through the Log DeliveryService (linke here?). The Log Delivery Service provides the ability toconfigure multiple log streams (or devices) which can be files or incomingor outgoing TCP-IP connections. In addition the customer logs are storedlocal on the CBM, for a configure-able number of days, available for reviewthrough the LOGQUERY interface which operates similarly the DMS CI;LOGUTIL interface.

Media Gateway 9000 ManagerATTENTIONThe MG 9000 Manager only applies to the UA-AAL1, UA-IP, and Intl UA-IPsolutions.

The Media Gateway 9000 (MG 9000) Manager is a server/clientapplication that runs on a Sun Solaris workstation. A graphical userinterface (GUI) allows fault handling and reporting.

The MG 9000 Manager allows the following fault management functions:

• retrieve, view, acknowledge/unacknowledge, and report alarms

• retrieve and view logs

• test lines

• perform line circuit diagnostics

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 87: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Media Gateway 9000 Manager 87

AlarmsAlarm surveillance, correlation, and reporting is performed at the AlarmBrowser screen. The Alarm Browser screen is accessed from the Alarmmenu option, which appears at the top of all screens. The Alarm Browserscreen consists of multiple sections used to view and retrieve and viewframe and shelf level MG 9000 alarms. The alarms are displayed as tablelist in the Alarm Browser. To maximize the performance of the AlarmBrowser, alarms are displayed one page at a time. By selecting one ofthe alarms, the alarm related information can be seen at the alarm textpanel at the lower portion of the Alarm Browser. The alarm counts canbe viewed from each Subnet Icon, Frame, Shelf, Card, Port, and CircuitListing GUIs. The generated alarm events are reported to NML’s OSSclient through a dedicated OSS Corba Event Channel.

LogsWhen MG 9000 logs are generated, they are written to the MG 9000Manager screen and are then directed to log files. The logs are writteninto the following two files:

• /data/mg9kem/logs/mg9kem.custlog (customer log file)

• /data/mg9kem/logs/mg9kem.deslog (designer log file)

The log files are named according to the type of log. The customer logsare directed to the custlog file using Syslog.The design logs from all theprocesses are directed to the deslog file using the Unix Syslog daemon.

The custlog seen at the MG9000 Manager correlates to alarms seen at theAlarm Browser screen and the traps generated from Gateway when thestatus changes. The designer logs seen at the MG 9000 Manager recordstracing and debugging history of the manager execution. The designer canuse it to correct the bug when the MG 9000 Manager in a unstable state.In addition to the alarms being viewed at the MG 9000 Manager, alarmsare also forwarded to the Operator Services System (OSS) for networklevel alarm monitoring.

Line test and diagnosticsThe following functions are available for line test:

• MAPCI user interface

Testing and troubleshooting of lines faults is available using aMaintenance and Administration Position (MAP) user interface at thelevels below the LNS MAP level. The LNS MAP level contains the linestest position (LTP) menu commands, the automatic line testing (ALT)menu commands and the lines service trouble (LNSTRBL) commands.

• MTAP test manager

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 88: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

88 Fault management fundamentals

The metallic test access point (MTAP) line test screen provides aninterface to pass line test commands from the manager to external testequipment.

The Nortel Access Care external test system has direct access to testheads collocated with the MG 9000 for line testing and communicateswith the manager line test manager through an ethernet port. Themanager software provides the line test management applicationto convert the Access Care parameters into commands that can beforwarded and processed in the MG 9000. The GUI specifies the testheads for Access Care use when testing a specified line circuit.

• DTA test manager

DTA (Digital Test Access) is the functionality used to test privatelines circuits by setting up test access connections with external testequipment. Compatible external test systems pass TL1 (TransactionLanguage 1) requests through an ethernet port to the MG 9000Manager to set up appropriate MG 9000 connections to collocatedtestheads or remote test units.

Line circuit diagnosticsThe Line Circuit Diagnostic subsystem provides utilities that performdiagnostics on line circuits on the following cards:

• POTS multi-circuit line card (POTS-32)

• Service Adaptive Access circuits (SAA-12 card)

• ADSL DMT Combo 8x8 card (8x8 ADSL)

For more information on Media Gateway 9000 Manager fault management,see the following documents:

• MG 9000 Fault Management module (NN10074-911)

• Access Care documentation

Communication Server 2000 Gateway Controller ManagerATTENTIONThe Communication Server 2000 GWC Manager does not apply to the PT-AAL1solution.

The Communication Server 2000 Gateway Controller Manager GUI/CLUIperforms the following fault management functions:

• Service data required for call processing

• Interfaces required to collect and distribute the data

• Provisioning data rules

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 89: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Communication Server 2000 Gateway Controller Manager 89

• Manages Maintenance state

• Manages alarm collection and reporting

• fault verification

• maintenance

The user can view the alarms and the status of hardware componentsthrough the Communication Server 2000 Gateway Controller Managerwhich also supports the following activities:

• Alarm Reporting

• Alarm History retrieval and browsing

Alarm ReportingThe Communication Server 2000 Gateway Controller Manager AlarmManager is launched from the ’Fault’ pull-down menu on the Call ServerManagement Tools GUI. The Alarm Manager pops up to the foreground.Figure 28 "Alarm Manager without details" (page 89) shows the AlarmManager without details.

Figure 28Alarm Manager without details

After the browser pops up, the user has the ability to filter for severityand acknowledgement state. Alarms are displayed in the center alarmtable panel. The headers for the center alarm table panel are NetworkElement Name, Alarm Level, Category, Alarm Time, Probable Cause,Acknowledge Time, System Uptime, Component ID, Alarm Description,and Specific Problem. Category and Network Element Name can befiltered by selecting the Advanced Filters button. Below the filters are thealarm counters. From here the user can see the total number of alarmspresent in the system, broken down by severity. Also, the display containsa total count and the time in which the last alarm was received or the lasttime the GUI was refreshed.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 90: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

90 Fault management fundamentals

The retrieve button activates the search for the desired alarms as per thecustomer settings. The Alarm Manager components display an activelist of the most current alarms on the system. It is updated automaticallywhen an alarm occurs on a managed device. When an alarm is cleared, itdisappears from the display.

ATTENTIONThe CMT GUI "Alarm Manager" allows viewing of the last 1,000 logs/alarms.

The new Alarm Manager also gives the user the ability to acknowledge analarm. This acknowledgment state is applied to all active Alarm Managers,and is persisted with the alarm in the database.

At the bottom of the display is the details panel. The details of an alarmcan be displayed by ensuring the details panel is showing (select the arrowbutton to point down), and then selecting the alarm from the Alarm Table.The user can show and hide this panel by selecting the arrow buttonbeside the Details label. This panel shows all information in the alarm. Inorder to display the information for the current alarm in the details panel,the user can select the row of interest. This processes a request to thedatabase to gather this information.

Alarm Totals can be viewed at the bottom left just above the details panel.Alarm totals are for critical, major, minor, and warning alarms. Once analarm has been cleared, its corresponding field is decremented. There isalso a total alarm count which includes the time of the last update to thedisplay. Figure 29 "Alarm Manager details" (page 90) shows the Detailspanel of the Alarm Manager window.

Figure 29Alarm Manager details

Alarm History retrieval and browsingUnlike the Alarm Manager, the Alarm History is a static view of the system.The Alarm History window is launched from the ’Fault’ pull-down menuon the Call Server GUI. The Alarm History pops up to the foreground.Figure 30 "Alarm History without details" (page 91) shows the alarm historywithout details.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 91: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Communication Server 2000 Gateway Controller Manager 91

Figure 30Alarm History without details

The Alarm History gives the ability to search based on Network ElementName, Alarm Level, Category, Alarm Time, Probable Cause, AcknowledgeTime, System Uptime, Component ID, Alarm Description, and SpecificProblem.To update the list of network elements currently displayed, theuser can refresh the list by the Action pull-down menu, and RefreshNetwork Element List. All alarms are stored in the Alarm History tableand in the event history syslog. The event history syslog resides in theCommunication Server 2000 Gateway Controller Manager or the UniversalAudio Server Manager server under "ptmlog" file name. Location of thefile is /var/log.

The Alarm History gives the user the ability to view both active and inactivealarms that have existed on the system for some time. It uses the conceptof paging to avoid keeping many alarms in memory at a given time. Itdisplays the same information that the Alarm Manager does, as well as thetime in which the alarm was cleared, if it was. The Alarm History is a frameand as such is independent from the main Communication Server 2000Gateway Controller Manager or the Universal Audio Server Manager GUI.

Once the user selects Search, one page of a fixed size displays on thescreen. To advance the display, the user can select Next Page, andcan go back to the beginning by selecting First Page.The Alarm Historydisplay has a details panel that is almost identical to the Alarm Manager.In addition, it also has the Clear Time. This can be accessed by the arrowbutton, just as the details panel in the Alarm Manager.

If the number of alarms from the current search is greater than 25, theuser can select the Next Page button to see the next batch of alarms.There is a First page button which resets the search to the beginning. The

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 92: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

92 Fault management fundamentals

details panel works exactly as it does in the Alarm Manager. First the usermust ensure it is displayed using the arrow button in the lower left of thedisplay. Then select an alarm from the Alarm Table, and the details displaysimilar to how they are displayed in the Alarm Manager.

Communication Server 2000 GWC Manager logsFor information on Communication Server 2000 GWC Manager logs,see “Log reporting for Communication Server 2000 Management Toolsapplications” (page 79).

Session Server ManagerATTENTIONThe Session Server Manager only applies to the IAC, PT-IP, and Intl PT-IPsolutions.

Alarms and logs for the Session Server are viewed through the SessionServer Manager.

AlarmsThe Session Server Manager has the capability for applications or thesystem itself to generate alarms. These alarms can be viewed on theSession Server Manager’s Alarm web page. Information shown aboutparticular alarms include the type, identifier, time-stamp, host (unit fromwhich the alarm was raised), severity, and a description of the alarm.

The user may view alarms by going to the alarms web page. The alarmweb page is located in the Monitoring menu folder.

Once the craftsperson clicks on Alarms, the alarm web page is presentedin the right frame with the current alarm list. The alarm web page updatesevery 30 seconds and the user also has the option of invoking a re-queryof alarms. This is similar to the Alarm implementation for the Stormsystem.

The following figure shows a sample alarms web page for the SessionServer Manager.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 93: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Session Server Manager 93

Figure 31Session Server alarms web page

Additional information is available in Session Server Fault Management(NN10332-911).

LogsThe Session Server Manager has the capability for applications or thesystem itself to generate logs. These logs can be viewed on the SessionServer Manager’s log web page. Information shown about a particular logincludes the following data:

• Timestamp

• Session Server device name

• Process that generated the log

• Log ID and number

• Alarm severity

• Log type

• Log label

• Additional information

The user may view logs by going to the logs web page located in theMonitoring menu folder.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 94: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

94 Fault management fundamentals

Once the craftsperson click on Logs, the log web page is presented in theright frame with the current log list. Unlike the alarms page, the logs pagedoes not update automatically. A sample log web page is shown below.

Figure 32Session Server logs web page

ATTENTIONThe body of log formats shown in this document are the formats for SessionServer logs if viewed from the IEMS. Log formats may differ if logs are viewedfrom the Session Server Web Interface. The content of the logs will not differ,only the formatting.

Additional information is available in Session Server Fault Management(NN10332-911).

Universal Audio Server ManagerATTENTIONThe UAS Manager does not apply to the PT-AAL1 solution.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 95: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Audio Provisioning Server (APS) 95

The Universal Audio Server (UAS) Manager performs the following faultmanagement functions:

• allows for retrieval of logs and responses to clear system faults thatgenerated the logs

• allows for retrieval of alarms and responses to clear the faults thatgenerated the alarms

UAS fault reporting is based on Simple Network Management Protocol(SNMP). Both alarm and log retrieval, and system correction tasks areperformed either through the Universal Audio Server Manager GUI orthrough commands entered at a command line or Windows interface. TheUniversal Audio Server Manager also supports the following functions:

• Alarm Reporting

• Alarm History retrieval and browsing

Alarm ReportingThe alarm reporting for the Universal Audio Server Manager is the sameas the alarm reporting for Communication Server 2000 Gateway ControllerManager. For details, refer to the Alarm Reporting information underCommunication Server 2000 Gateway Controller Manager of this section.

Alarm History retrieval and browsingThe alarm history retrieval and browsing for the Universal Audio ServerManager is the same as the alarm history retrieval and browsing for theCommunication Server 2000 Gateway Controller Manager. For details,refer to the Alarm History retrieval and browsing information underCommunication Server 2000 Gateway Controller Manager in this section.

UAS Manager logsFor information on UAS Manager logs, see “Log reporting forCommunication Server 2000 Management Tools applications” (page 79).

Audio Provisioning Server (APS)ATTENTIONAPS does not apply to the PT-AAL1 and UA-AAL1 solutions.

APS Fault Management consists of handling logs and alarms. Both logsand alarms for the APS utilize a management station. A managementstation can be one of the following platforms:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 96: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

96 Fault management fundamentals

• a desktop PC running Windows NT 4.0 with Service Pack 4 or higherand all PTM daemons

• a Solaris workstation running the PTM software and an Oracledatabase

• a desktop PC running an MIB browser

APS logsAPS logs are collected from various sources in the APS and are optionallysent to a management station through SNMP traps. Sources of APS logsinclude the following locations:

• APS administration

• APS Audio Management

• APS File Upload

• APS Servlet Requests

• IPS software

Logs from each of the above sources can be of type information, warning,or error.

APS alarmsAlarms are sent from the various APS software components to themanagement station. The following table lists the fields that are associatedwith APS alarms.

Table 21APS alarm fields

Field Description

severity warning, minor, major, or critical

probable cause Qualifies the cause of the alarm condition. Takes theform of an integer from a range of standardized valuesfrom the NORTEL-NMI-TC-MIB. Possible valuesincludes the following causes:

• 1-adapter error

• 7-configuration or customization error

• 8-congestion

• 17-file error

• 46-software error

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 97: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Universal Signalling Point Manager 97

Table 21APS alarm fields (cont’d.)

Field Description

problem type This type categorizes the alarm. Possible valuesinclude the following types:

• communication

• environmental

• equipment

• quality of service

• processing error

specific problem A number that uniquely identifies the alarm.

component name Name of the software or hardware component forwhich this alarm is associated.

additional text Text description of the alarm.

Network Patch Manager (NPM)ATTENTIONThe NPM application does not apply to the PT-AAL1 solution.

The NPM application utilizes an SPFS-based API interface to capture logsand alarms and send them northbound to the Communication Server 2000Core Manager or any type of OSS Fault Collector.

Universal Signalling Point ManagerATTENTIONThe USP Manager does not apply to the PT-AAL1 solution.

USP Manager performs the following fault management functions:

• viewing alarms

• selecting alarms

• printing alarms

• clearing alarms

• viewing logs

• retrieving logs

• printing logs

• testing links

• maintenance

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 98: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

98 Fault management fundamentals

For additional details on the fault management aspects of the USPManager, see USP Fault Management (NN10071-911).

Multiservice Data ManagerMDM performs the following fault management functions:

• supports maintenance

• supports configuration

• supports reporting of faults for the Multiservice Switch 15000/MediaGateway 15000 or Multiservice Switch 7400/Media Gatewaycomponent faults

• supports the API to deliver log and alarm data to an OSS

• supports carrier maintenance activity for Multiservice Switch15000/Media Gateway 15000 and Multiservice Switch 7400/MediaGateway 7400

The user can view the status of hardware components through the MDMwhich also supports the following tasks:

• Alarm Surveillance

• Alarm Reporting

— Network status bar

— Network viewer

— Component information viewer

• Alarm History retrieval and browsing

• Alarm Display

• Alarm Help

Alarm SurveillanceMDM Advisor toolset provides alarm surveillance to the MultiserviceSwitch 15000/Media Gateway 15000 and Multiservice Switch 7400/MediaGateway 7400. The MDM tools are launched from a launch pad calledthe MDM toolset. This set of tools provides a common view of networksurveillance data across all supported devices. The Advisor toolsetprovides both alarm and state based surveillance. For detailed information,see the MDM User Guide (241-6001-011).

Alarm ReportingThe MDM collects Multiservice Switch 15000/Media Gateway 15000,Multiservice Switch 7400/Media Gateway 7400, and MDM alarm andlog data, through Ethernet connections to each. The MDM includesboth graphical and command-line user interfaces to perform the various

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 99: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Multiservice Data Manager 99

network management activities. Within the MDM, device states, devicesubcomponents, and links are represented by a specific color. Themapping from state to color is user configurable. Alarms are presented onthe MDM with a color to distinguish the alarm severity. The color to alarmseverity mappings are configurable.

Table 22 "Default state to color mapping" (page 99) shows default stateto color mapping.

Table 22Default state to color mapping

State Color Explanation

Unknown (UNKN) Grey The component is unreachable

Out-of-Service (OOS) Red The component is no longeroperational

In Service Trouble(ISTB)

Orange At least one important sub-componentis out-of-service

In Service Trouble Yellow Secondary sub-components areout-of-service

In Service (INSV) Green The component is operational

Table 23 "Default alarm severity to color mapping" (page 99) shows defaultalarm severity to color mapping.

Table 23Default alarm severity to color mapping

Alarm Severity State

Unknown Light blue

Warning Dark blue

Minor Yellow

Major Orange

Critical Reg

Network status barThe MDM tool provides a high-level view of all network element alarmsin the network (Figure 33 "MDM Network Status bar" (page 100)). TheNetwork Status bar gives the total and the severity of the alarms. Inaddition, the Network Status bar performs the following functions:

• Provides a high-level view of component states

• Displays only the troubled components (if requested)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 100: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

100 Fault management fundamentals

• Allows users to acknowledge the current state of the network

• Allows users to drag and drop troubled components on to theComponent Information Viewer and the Command Console GUIs

Figure 33MDM Network Status bar

Network viewerThis MDM tool presents a real-time graphical network map of allMultiservice Switch 15000/Media Gateway 15000 and Multiservice Switch7400Perhaps/Media Gateway 7400 configuration elements, includingcomponents, trunks, and links. The Network Viewer of the MDM providesthe following functionality:

• Represents different device types by the shape of the icon andrepresents the states of components by the color of the icon.

• Monitors and displays component states for first-alert surveillance.

• Provides flexibility to display an organization view of MultiserviceSwitch 15000/Media Gateway 15000 and Multiservice Switch7400/Media Gateway 7400 configuration elements by grouping theobjects in a way that reflects the required view by function.

• Displays a high-level view of each Multiservice Switch 15000/MediaGateway 15000 and Multiservice Switch 7400/Media Gateway 7400shelf for quick identification of areas that require action.

• Provides access to other Advisor tools through main menus andpop-up menus displayed after selecting a network element in the map.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 101: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Multiservice Data Manager 101

Component information viewerThe MDM tool provides diagnostic analysis of a component in relation toan event identified by the other surveillance tools such as Network Viewerand Alarm Display. The Component Information Viewer displays an activeor recent alarm list as well as component status records, and correlatesfault impacts and causes. This tool also has filtering options that allowviewing of impact on related components.

Alarm history retrieval and browsingThe MDM command-line tool allows a query of old alarms, and supportsvarious search criteria including start and end time, fault code, componentname and criticality.

Short-term Historical Alarm Reporting provides the ability to query oldalarms and supports the following various search criteria:

• start time

• end time

• fault code

• component name

• criticality

Short-term Historical Alarm Reporting is a command-line tool. Searchresults are viewed from a window or saved to a file. Figure 34 "Queryresults from Short-term Historical Alarm Reporting" (page 102) shows theHistorical Alarm reporting tool.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 102: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

102 Fault management fundamentals

Figure 34Query results from Short-term Historical Alarm Reporting

Alarm DisplayThe MDM tool, Alarm Display, provides a common format for viewing logsfor Multiservice Switch 15000/Media Gateway 15000 and MultiserviceSwitch 7400/Media Gateway 7400 (Figure 35 "MDM Alarm Display" (page103)). Alarm Display supports both Alarm Log and Active Alarm displaymodes, and provides both accept and reject filters to allow viewing of aspecific subset of alarms. This tool also provides access to diagnosticinformation for a selected alarm by using Alarm Help. In addition, AlarmDisplay has an Alarm Acknowledgment utility that allows the user toindicate that an alarm is being investigated. The indication shows the userwho is responsible for the acknowledgement (or non-acknowledgement)and the reason for the acknowledgement (or non-acknowledgement).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 103: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Multiservice Data Manager 103

Figure 35MDM Alarm Display

Alarm HelpThis MDM tool provides a view of diagnostic information for a selectedalarm. The description provides details on what the alarm means and alsothe action that is necessary to fix the problem. This tool can be started byselecting an alarm from the Alarm Display and choosing the Alarm Helpsubmenu item. Alarm Help uses the Netscape Web browser to display therequested alarm text.

Recording alarm information from a Multiservice Switch/Media GatewaySCC2 log

Figure 36 "Example of a Multiservice Switch/Media Gateway SCC2 alarmlog" (page 104) shows an example of an SCC2 log. Field 1 in the header,identifies the log name and number (P15K025). In the example, the logname P15K corresponds to a Multiservice Switch 15000/Media Gateway15000 network element.

The Multiservice Switch 15000/Media Gateway 15000 is identified in field 2as EM/P15E LP/5 Sonet/2. The NETTypeID (network type ID) is identifiedas 138 in field 3. The specificProblems (specific problems) eight-digit codeis identified in field 4 as 70115202.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 104: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

104 Fault management fundamentals

Figure 36Example of a Multiservice Switch/Media Gateway SCC2 alarm log

Determining a Multiservice Switch/Media Gateway logFigure 37 "Example SCC2 log identifying Multiservice Switch 15000/MediaGateway 15000 major alarm" (page 105) shows an example of an SCC2log. In Figure 37 "Example SCC2 log identifying Multiservice Switch15000/Media Gateway 15000 major alarm" (page 105), the log nameand number appears as P15K162 (see field 1). The log name P15Kcorresponds to a Multiservice Switch/Media Gateway network element.There are two possible Multiservice Switch/Media Gateway networkelement types shown in the following table.

Multiservice Switchlog name

Type ofMultiserviceSwitch/MediaGateway networkelement

Network elementType ID

P15K Multiservice Switch 15000 orMedia Gateway 15000

138

MDM MDM 86

Also note (in Figure 37 "Example SCC2 log identifying Multiservice Switch15000/Media Gateway 15000 major alarm" (page 105)) the value of themanagedObjectInstance field which is EM/15K COL/LOG SP/$ (see field2), the NetTypeID which is 138 (see field 3) and the specificProblems codewhich is 70030002 (shown in field 4).

If the log name, on the SCC2 log, does not match one of the twoMultiservice Switch/Media Gateway log names, then the log is not aMultiservice Switch/Media Gateway log and must be a log associated withthe Communication Server 2000 Core Manager, Communication Server2000, MG 4000, or IW SPM.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 105: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Multiservice Data Manager 105

Figure 37Example SCC2 log identifying Multiservice Switch 15000/Media Gateway15000 major alarm

Logging in to the MDM and accessing the Component InformationViewer

After logging in to the MDM, launch the Component Information Viewertool from the MDM Toolset (see Figure 38 "Example view of theComponent Information Viewer" (page 106)). Then, input the networkelement information (EM/P15E LP/5 Sonet/2) from the SCC2 log onto thefield called Related Components for (see field 1). As a result of this actionthe related components will be displayed in field 2.

Next, input EM/P15E LP/5 Sonet/2 into the Information For field (see field3 in Figure 38 "Example view of the Component Information Viewer" (page106)) to obtain a list of alarms for this component. Select the requiredalarm (choose the highest priority first) from the list (see field 4) and rightclick the mouse to launch the Alarm Help utility in the Web browser.

Alarm Help displays detailed information on specific alarms and alsoprovides suggestions for remedial action.

For detailed information on logging into the MDM and launching theComponent Information Viewer, and acknowledging, clearing anddisplaying Multiservice Switch alarms, see Multiservice Data ManagerFault Management Tools (241-6001-011).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 106: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

106 Fault management fundamentals

Figure 38Example view of the Component Information Viewer

Obtaining remedial action for a Multiservice Switch alarmThe Alarm Help utility can be used to obtain specific information forMultiservice Switch alarms (see Figure 39 "Example alarm informationfrom the Alarm Help utility" (page 107)). The Alarm Help utility usesthe eight-digit alarm code that appears on the SCC2 alarm log on thespecificProblems field.

ATTENTIONThe alarm described in the example Alarm Help screen in Figure 39 "Examplealarm information from the Alarm Help utility" (page 107) does not relate to theSCC2 alarm log shown earlier.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 107: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Multiservice Data Manager 107

Figure 39Example alarm information from the Alarm Help utility

Fault management suggestions for the Multiservice Switch 15000The Multiservice Switch 15000 switch is routinely monitored to determine ifproblems exist and if traffic is properly handled. Regularly monitoring thesystem helps identify and prevent potential problems. The MDM AdvisorPerformance Viewer tool is used to generate statistics for ATM services onthe following ATM equipment: ATM function processors, ATM interfaces,ATM connections. For more information on these Multiservice Switch15000 tests, see Multiservice Switch Fault Information, and MDM faultInformation.

Monitoring ATM function processors on the Multiservice Switch15000Begin monitoring the Multiservice Switch ATM system by examining theATM function processors. If the function processors are not operatingproperly, none of the other elements of the system can effectively processATM traffic.

Monitoring ATM interfaces on the Multiservice Switch 15000After the function processors have been examined, the ATM interfaces oneach function processor are the next system component to be examined.Each interface supports a variety of traffic types and connections, andprovides a summary of all connections on that link.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 108: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

108 Fault management fundamentals

Monitoring ATM connections on the Multiservice Switch 15000Once you have determined how the function processors and ATMinterfaces are operating, start examining the individual connections thatare supported on each ATM interface. When monitoring each connection,review how the configured routing and signaling are operating.

Hardware LED status indicators on Multiservice Switch 15000The status of hardware equipment on Multiservice Switch 15000 isindicated by a system of colored and shaped LEDs. A lit LED indicatesone status in one color. On the same type of component or assembly,the LED is always in the same position relative to its mates. While theequipment is powered, there is always one LED that is lit.

Isolating faults on the Multiservice Switch 15000Fault management for Multiservice Switch 15000 involves the followingtasks:

• Control processor test

• Control processor disk diagnostics

• Function processor test

• Function processor lock and unlock

• Port lock and unlock

• Function processor loopback port test

• Remote loopback port test

• Manual loopback port test

ATTENTION• Multiservice Switch 15000 fault management tasks can only be carried out

using MDM tools. Multiservice Switch 15000 fault management tasks cannotbe carried out through the OSS interface.

• For more information on these Multiservice Switch 15000 tests, seeMSS 15000, MG 15000 & MDM Basics (PT-AAL1/UA-AAL1/UA-IP)(NN10028-111).

Multiservice Switch 15000 Control processor testThis test is performed following a control processor fault or failure usingthe MDM Advisor Command Console. The most common problems with acontrol processor are software load failures and crashes. These problemscan affect both the active and the standby control processor. Problemsinclude the following errors:

• the control processor does not load

• the control processor crashes

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 109: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Multiservice Data Manager 109

• the control processor switchover

• standby control processor does not load

Multiservice Switch 15000 Control processor disk diagnosticsThese diagnostics are used following control processor disk fault orfailure, and use the MDM Advisor Command Console. Disk tests verify theintegrity of the disk. Test a disk only when a fault is suspected in the diskhardware. The disk cannot be tested on the active control processor. Ifthe disk to be tested is currently active, switch control between the activeand the standby control processors.

Multiservice Switch 15000 function processor testPerform this test following a function processor fault or failure, using theMDM Advisor Command Console. The 16-port OC-3 ATM IP functionprocessor must be tested

Multiservice Switch 15000 function processor lock and unlockPerform this test prior to replacing a function processor using the MDMAdvisor Command Console.

Multiservice Switch 15000 loopback port lock unlockPerform port lock prior to running diagnostics or deleting a port, andperform port unlock after completing diagnostics. Use the MDM AdvisorCommand Console to perform these operations. The following ports canbe locked and unlocked: SONET, Ethernet, Ethernet OAM&P, BITS timing.

Multiservice Switch 15000 function processor loopback porttestPerform this test following a port fault or failure using the MDM AdvisorCommand Console. The function processor loopback test verifies theinternal working of the function processor and the line interface. This testtransmits a test pattern through the internal circuits and processors. Datais transmitted back from the link interface of the port.

Multiservice Switch 15000 remote loopback port testPerform this test following a port fault or failure using the MDM AdvisorCommand Console. Ports that can be tested are OAM&P Ethernet andSONET ports. Use remte tests to verify the full length of the transmissionfacility.

Manual loopback port testPerform this test following a port fault or failure using the MDM AdvisorCommand Console. Most port tests set up their own loopbacks. Themanual test requires the operator to arrange for a loopback to be insertedat some point along the connection. A loopback loops the received data

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 110: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

110 Fault management fundamentals

back to the function processor under test. Loopbacks do not produce testresults for the local node because they loop received information back tothe switch under test.

Additional informationFor additional details on fault management tools and utilities for MDM, seethe following component documentation:

• MDM Fault Management User Guide (241-6001-011)

Device Manager for Ethernet Routing Switch 8600The Device Manager uses Simple Network Management Protocol (SNMP)to provide alarm monitoring for the Ethernet Routing Switch 8600 (ERS8600) through a Graphical User Interface (GUI). Configurations on theERS 8600 can also be viewed through a Web interface.

Remote monitoring is available through Device Manager and provides fourmain functions to monitor devices on a port by port basis:

• setting alarms for user-defined events

• gathering real-time and historical Ethernet statistics

• logging events

• sending traps for events

The user can configure switching or routing configurations and enableother features such as RMON alarms and port mirroring to analyze trafficon a per-port basis. The RMON MIB is an interface between the RMONagent on the ERS 8600 Series switch and the Device Manager. It definesobjects that are suitable for the management of any type of network, butsome groups are targeted for Ethernet networks in particular. Remotemonitoring (RMON) is a management information base (MIB) or a groupof ’management objects’ that you use to ’get’ or ’set’ values. Using DeviceManager, you enable RMON globally for devices on the switch. WhenRMON is enabled globally, you then enable monitoring for individualdevices on a port-by-port basis.

Within Device Manager, RMON alarms can be set that relate to specificevents or variables simply by selecting these variables from a drop-downmenu. Specify events associated with alarms to be set to either trap orlog-and-trap. In turn, these alarms, when tripped, are trapped or logged.

The user can view the alarms and the status of hardware componentsthrough the Network Patch Manager which also supports the followingTroubleshooting tools and utilities:

• Testing the switch fabric and address resolution table

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 111: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Device Manager for Ethernet Routing Switch 8600 111

— Test the switch fabric and check the address resolution (AR) tablefor consistency. The Fabric test causes the CPU to generate trafficand send it through the switch fabric. Given the forwarding rateof ERS 8000 Series switches, the CPU does not generate muchtraffic, but it performs a simple test of the switch fabric memory.The AR table test performs a consistency check on addressresolution table entries.

• Monitoring how often a port goes down

— Monitor the number of times a link is going up or down rapidly (thatis, flapping) on a port. This action can be detrimental to networkstability because it could trigger spanning tree and routing tablerecalculation. If the number exceeds a given boundary during aspecified interval, the port is forced out of service.

• Configuring and monitoring port mirroring

— Use port mirroring to specify a destination port on which mirroredtraffic can be seen and specify the source ports from which trafficis mirrored. Any packets entering or leaving the specified ports areforwarded normally and a copy of the packets is sent out the mirrorport. Also use the port mirroring feature to monitor traffic from MACaddresses where traffic with a given MAC source address (SA) orMAC destination address (DA) is copied to the mirror port. Thisfeature is enabled by setting the Monitor field to true for a MACaddress in the Forwarding dialog box.

• Trapping errors

— All errors detected are then sent to a log that you can view inDevice Manager.

• Viewing address resolution statistics

— The AR Stats tab shows statistics for the internal state of theaddress translation table. These statistics are debugging aids andcan only be used when consulting with Nortel support personnel.

• Enabling the system log

— Enable the system log feature globally to send messages to up to10 syslog hosts. By default, five hosts are supported.

• Receiving system log messages

— Use the system log-messaging feature of the ERS to manageswitch event messages on any UNIX-based managementplatform. The ERS syslog software supports this functionality bycommunicating with a counterpart software component namedsyslog on your management workstation. The UNIX daemon syslogis a software component that receives and locally logs, displays,

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 112: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

112 Fault management fundamentals

prints, and/or forwards messages that originate from sourcesinternal and external to the workstation.

• Changing the severity level mapping

• Checking the MIB status

• Checking the details of the MIB status

For more information on Device Manager fault reporting, see the EthernetRouting Switch 8600 component documentation.

Storage Management Manager (STORM Manager)ATTENTIONThe STORM Manager does not apply to the PT-AAL1 and UA-AAL1 solutions.

STORM Manager performs the following fault management functions:

• views current alarms

• sorts current alarms

• filters current alarms

• displays current log reports

For information on STORM Manager Alarm Screen, see theCommunication Server 2000 Storage Manager GUI in Figure 40 "STORMManager Alarm Screen" (page 112).

Figure 40STORM Manager Alarm Screen

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 113: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Carrier VoIP lines testing 113

For information on STORM Manager Log Screen, see the STORMManager GUI in Figure 41 "STORM Manager Log Screen" (page 113).

Figure 41STORM Manager Log Screen

Nuera ConfiguratorFor information on Nuera Configurator, see the ORCA BTX-SeriesSoftware Manual or the BTX System Overview.

NueraViewFor information on NueraView, see the ORCA BTX-Series SoftwareManual or the BTX System Overview.

HP OpenViewFor information on HP OpenView, see the ORCA BTX-Series SoftwareManual or the BTX System Overview.

Carrier VoIP lines testingThis document describes the Carrier VoIP lines testing solution. Thisinformation applies to the Carrier VoIP UA-AAL1 and UA-IP MG 9000 forNorth America.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 114: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

114 Fault management fundamentals

Overview of line testingThis document is divided into the following sections that describe the twomethods of supporting Carrier VoIP lines testing applications:

• MG 9000 lines testing - This section describes the MG 9000 line testsubsystem.

• Communication Server 2000-based lines testing - This sectiondescribes Communication Server 2000- based tests that use theMaintenance and Administration Position (MAP) commands.

Both sections show capabilities, provisioning rules, and user information.

Communication Server 2000-based lines testingCommunication Server 2000 MAP-based lines testing tests lines usingtraditional MAP terminal commands available from the LNS MAP level.This configuration requires a maintenance trunk module (MTM) or remotemaintenance module (RMM) be available.

MG 9000 lines testingMG 9000 lines testing tests lines that connect to line cards in the MG 9000and require the following:

• metallic access for loop and line test

• line circuit diagnostics

Line test systems generally use metallic access to test and troubleshootproblems with the twisted pair loop from the MG 9000 line card to thecustomer premise.

The MG 9000 provides metallic access for external or adjunct loop andline test systems, and for available cutover service (automatic board toboard testing [ABBT]). These can be connected to MG 9000s by eitherdirect network interface or no-test trunk (NTT).

Fault management trunk testingThe following section of fault management deals with trunk testing.

PT-AAL1 trunk testing capabilitiesDeveloped for the AAL1 hybrid switch, this configuration contains anENET and an Integrated Service Module (ISM). The ISM is specificallydesigned to accommodate a variety of test hardware which can performmaintenance testing on TDM trunks. This solution takes advantage ofthe TDM to IP bridge capabilities of the IW-SPM, to provide connectivitybetween the Gateway TDM trunk under test and the test hardware locatedin the ISM.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 115: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

PT-AAL1 trunk testing capabilities 115

The solution is fully integrated with the XA-Core. This means thattest execution, results, and logs are equivalent to the Legacy TDMenvironment.

The Hybrid Switch Solution supports the following trunk test functionality.Customers can test TDM trunks via the MAPCI TTP level and ATT levelinterfaces. The applicable Nortel documentation reference is provided foreach MAPCI level.

The following table describes the TTP level commands.

Table 24TTP level commands (297-8001-821, Volume 10)

Command Description

CktInfo Displays the name and state of the PM and thestate of the circuit in the trunk.

Ckt Connects a specified circuit to a trunk.

StkSdr Allows access to the Stuck Sender feature,which identifies trunks with outpulsing problems.

The following table describes the manual level commands.

Table 25Manual level commands (297-1001-821, Volume 7)

Command Description

LOSS Manually measures transmission loss. Foradditional information, see “TGEN, LOSS, andCKTMON Command additional information”(page 121)

TGEN Manually generates a tone of desired frequencyand level. For additional information, see“TGEN, LOSS, and CKTMON Commandadditional information” (page 121)

Noise Manually measures transmission noise.

OP Permits the ability to seize and outpulse a set ofdesired digits over a trunk.

TDet Manually measures the presence of anincoming tone.

HSet Connects an available headset to a trunk.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 116: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

116 Fault management fundamentals

Command Description

SGNL Executes a bit signaling test on a trunk.

TST Permits the user to run a number of supportedtests, including the following:• ISUP continuity test (ICOT)

• T100 (originating and terminating)

• T102 (originating and terminating)

• T104 (originating and terminating)

• T105 (originating and terminating)

The following table describes the monitor level commands.

Table 26Monitor level commands (297-8001-821, Volume 7)

Command Description

MonPost Provides the user the ability to monitor a trunkcircuit in a posted position.

MonLink Provides the user the ability to monitor a trunkcircuit in a linked position.

Montalk (L) Listen provides the user the ability to monitora trunk circuit in both directions in a listen onlymode.

Montalk (T) Talk provides the user the ability to monitor atrunk circuit in both directions while participatingin the call (listen and talk mode).

CktMon Displays the messages on the MAP from thedigital PM of the circuit without interruptingnormal call processing.

CPos Executes a test to a Centralized AutomaticMessage Accounting (CAMA) or RemoteOperator Number Identification (RONI) trunk.

Table 27C7TTP level commands (297-1001-821, Volume 2)

Command Description

CVTest Performs a Circuit Validation Test on a trunk.

TrkQry Displays the local or remote state of the trunk.

QrySig Queries the signaling availability and the stateof a trunk.

RouteSet Displays the route set CLLI of a CCS7 trunk.

CIC Displays the circuit identification code of a trunk.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 117: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

PT-AAL1 trunk testing capabilities 117

ATTENTIONAll commands at the Automatic Test Trunk (ATT) level are supported.

Supported TestsThe following tests are supported but are not supported by the MAPCI.For more information, see DMS-100 Family Maintenance and OperationsManual (297-8991-500), or the DMS-100 Family Trunks MaintenanceGuide (297-1001-595).

• T108 Terminating Test Line - a non-inverting digital loopbackconnection

• T101/C101 trunk call

• Digital Remote Office Test Line (ROTL) - 100/200 trunk types only

ATTENTION— Additional equipment is required.

— This is available for both TDM and DPT-SPM/MG 4000 trunk types.

• Automatic Trunk Routing (ATR) - 250 trunk types only.

ATTENTIONAdditional equipment is required.

Testing for DMs and SPM-based equipmentFor DMS and SPM-based equipment, the customer can test equipmentand the ATM layer using the test command that is supported on theCommunication Server 2000 Core Manager CLUIs. The equipmentconsists of hardware including the CEM and RMs. All other diagnostics,including testing of trunks and carriers, are tested at the carrier and trunktest position (TTP) levels of the MAP interface.

Note: For additional information on trunk testing, see Faultmanagement trunk testing in this Fault information module.

The customer cannot set parameters for testing. There are only two typesof service diagnostics:

• in-service (InSv) test that runs while the system is In Sv

• out-of-service (OOS) test that runs while the system is OOS

The InSv test can change states, but the OOS test cannot. If the InSv testfails, noncritical test failure causes the unit to become in-service trouble(ISTb). However, the system continues to generate the noncritical faultlog. Critical test failure results in trigger recovery and causes the unit tobecome system busy (SysB). The system discontinues fault reportingduring critical test failure.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 118: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

118 Fault management fundamentals

DPT trunk testing with the DPT-SPM and MG 4000

ATTENTIONThis section applies only to the North American PT-AAL1.

Digital Remote Office Test Line (ROTL) is a DMS feature to remotely testtrunks via a digital, four-wire, E&M trunk. This CAS trunk interface permitsa test head to be connected to the DS 2000 and, via a maintenance dialplan, select an outgoing trunk to be tested. A test head is connected tothe Communication Server 2000 by a T1, on which several channels areprovisioned with a ROTL trunk type. Test calls can then be outpulsed fromthe test head over the ROTL trunk members using a defined maintenancedial plan which instructs the Communication Server 2000 software toselect an outgoing trunk circuit and generate a test call to the far-endswitch. A connection is then established between the test equipment andthe trunk circuit being tested. Once all connections have been made, thetest head conducts the desired trunk test.

The (I)SN09 release enhances the Digital ROTL functionality to permittest connections over DPTs hosted by an DPT-SPM or MG 4000. Thecustomer can directly select a DPT group, node, and TID to test.

PT-IP trunk testing capabilitiesThere are currently two solutions for trunk troubleshooting in the PT-IPsolution:

• “Hybrid switch solution (AAL5 IP only)” (page 118)

• “Non-hybrid/Communication Server 2000-Compact solution (AAL2 andAAL5)” (page 118)

Hybrid switch solution (AAL5 IP only)Developed for the AAL5 hybrid switch configuration contains an ENET andan Integrated Service Module (ISM). The ISM is specifically designed toaccommodate a variety of test hardware which can perform maintenancetesting on TDM trunks. This solution takes advantage of the TDM to IPbridge capabilities of the IW-SPM, to provide connectivity between theGateway TDM trunk under the test and the test hardware located in theISM.

Non-hybrid/Communication Server 2000-Compact solution (AAL2 andAAL5)

Developed for the AAL2 and AAL5 switch configurations that do notcontain an ENET or ISM, this solution utilizes the 3rd party Media Server2000 Series as a limited replacement for the ISM. The MS provides thetrunk test functionality to the Gateway TDM trunk under test.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 119: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

PT-IP trunk testing capabilities 119

Both solutions are fully integrated with the XA-Core. This means thattest execution, results, and logs are equivalent to the Legacy TDMenvironment. However, since the Hybrid solution utilizes the ISM, thereare more trunk testing options available to the customer than the initialNon-hybrid/Communication Server 2000-Compact solution.

The Hybrid switch solution supports the following trunk test functionality.Customers can test TEM trunks via the MAPCI TTP level and ATT levelinterfaces. The applicable Nortel documentation reference is provided foreach MAPCI level.

Table 28TTP level commands (297-8001-821, Volume 10)

Command Description

CktInfo Displays the name and state of the PM and thestate of the circuit in the trunk.

Ckt Connects a specified circuit to a trunk.

StkSdr Allows access to the Stuck Sender feature,which identifies trunks with outpulsing problems.

Table 29Manual level commands (297-1001-821, Volume 7)

Command Description

LOSS Manually measures transmission loss. Foradditional information, see “TGEN, LOSS, andCKTMON Command additional information”(page 121)

TGEN Manually generates a tone of desired frequencyand level. For additional information, see“TGEN, LOSS, and CKTMON Commandadditional information” (page 121)

Noise Manually measures transmission noise.

OP Permits the ability to seize and outpulse a set ofdesired digits over a trunk.

TDet Manually measures the presence of anincoming tone.

HSet Executes a bit signaling test on a trunk.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 120: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

120 Fault management fundamentals

Command Description

SGNL Connects an available headset to a trunk.

TST Permits the user to run a number of supportedtests, including the following:• ISUP continuity test (ICOT)

• T100 (originating and terminating)

• T102 (originating and terminating)

• T104 (originating and terminating)

• T105 (originating and terminating)

Table 30Monitor level commands (297-8001-821, Volume 7)

Command Description

Montalk (L) Listen provides the user the ability to monitora trunk circuit in both directions in a listen onlymode.

Montalk (T) Talk provides the user the ability to monitor atrunk circuit in both directions while participatingin the call (listen and talk mode).

CktMon Displays the messages on the MAP from thedigital PM of the circuit without interruptingnormal call processing.

CPos Executes a test to a Centralized AutomaticMessage Accounting (CAMA) or RemoteOperator Number Identification (RONI) trunk.

Table 31DATATTP level commands (297-1001-821, Volume 3)

Command Description

Berttime Sets the length of time a BERT will run.

BTerm Sets the type of outpulse or digits for a BERT.

BERT Executes a Bit Error Rate Test on a trunk.

Table 32C7TTP level commands (297-1001-821, Volume 2)

Command Description

CVTest Performs a Circuit Validation Test on a trunk.

TrkQry Displays the local or remote state of the trunk.

QrySig Queries the signaling availability and the stateof a trunk.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 121: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

PT-IP trunk testing capabilities 121

Command Description

RouteSet Displays the route set CLLI of a CCS7 trunk.

CIC Displays the circuit identification code of a trunk.

Trunk Type Start Signaling Type Pulse Type Card Type

OP Wink MF DSISIG

ES Wink MF DSISIG

ATC Wink MF DSISIG

IT Wink MF DSISIG

CELL Wink MF DSISIG

IBNT2 Wink MF, DP DSISIG

IBNTO Wink MF, DP DSISIG

IBNTI Wink MF, DP DSISIG

The SGNL command support on MG15000 PTS trunks has the followingqualifications:

• No international trunk variants are supported.

• Delay Dial and Immediate start signaling types are not supported.

• Not supported in the IAC solution which employs PTS trunk variantsOP and ES hosted on TGCP protocol based trunk gateways.

• Current MG15000 based PTS trunk architecture does not supportonhook signal reporting during digit collection. If the user is executingthe SGNL command whose far-end trunk is a MG15000 based PTStrunk, the following procedure applies:

— User posts supported PTS trunk circuit and executes the SGNL’onoffhk’ command.

— Far-end MG15000 PTS trunk connects to an available digit receiverand outputs a Wink signal reply.

— Upon successful reporting of Wink signal, user releases postedPTS trunk circuit trunk via the available RLS release command.

— User waits for far-end trunk digit receiver timer to expire. Uponexpiration, far-end trunk returns to its idle state.

TGEN, LOSS, and CKTMON Command additional informationThe TGEN, LOSS, and CKTMON commands support the following NorthAmerican PTS trunk variants which use the standard /TDM PTS call modelfor the H.248 based protocol.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 122: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

122 Fault management fundamentals

Trunk Type Start Signaling Type Pulse Type Card Type

OP Wink MF DSISIG

ES Wink MF DSISIG

ATC Wink MF DSISIG

IT Wink MF DSISIG

CELL Wink MF DSISIG

IBNT2 Wink, DT, IM MF, DP, DD DSISIG

IBNTO Wink, DT, IM MF, DP, DD DSISIG

IBNTI Wink, DT, IM MF, DP, DD DSISIG

TGEN, LOSS, and CKTMON commands support on MG15000 PTS trunkshas the following qualifications:

• No international trunk variants are supported.

• Not supported in the IAC solution which employs PTS trunk variantsOP and ES hosted on TGCP protocol based trunk gateways.

• The LOSS command line option "E" for the echo test is not supported.

• The CKTMON command resides on the MAPCI TTP Monitor level.No physical hardware connections will be attempted upon entry ofMAPCI TTP Monitor level. Monitor level commands MonPost, MonLink,MonBoth, and MonTalk are not supported.

The following tests are supported but are not supported by the MAPCI.For more information, see DMS-100 Family Maintenance and OperationsManual (297-8991-500), or the DMS-100 Family Trunks MaintenanceGuide (297-1001-595).

• T108 Terminating Test Line - a non-inverting digital loopbackconnection.

• T101/C101 trunk call

• Digital Remote Office Test Line (ROTL) - 100/200 trunk types only

ATTENTIONAdditional equipment is required.

• Automatic Trunk Routing (ATR) - 250 trunk types only

ATTENTIONAdditional equipment is required.

The Non-Hybrid/Communication Server 2000-Compact Switch Solutionsupports the following trunk test functionality. Customers can test TDMtrunks via the MAPCI TTP level and ATT level interfaces*. Trunk test

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 123: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

PT-IP trunk testing capabilities 123

functionality is provided by the Media Server 2000 Series products, MS2010 (AAL5) and MS 2020 (AAL2). The applicable Nortel documentationreference is provided for each MAPCI level.

Table 33TTP level commands (297-8001-821, Volume 10)

Command Description

CktInfo Displays the name and state of the PM and thestate of the circuit in the trunk.

StkSdr Allows access to the Stuck Sender feature,which identifies trunks with outpulsing problems.

Table 34Manual level commands (297-1001-821, Volume 7)

Command Description

OP Permits the ability to seize and outpulse a set ofdesired digits over a trunk.

MWTSwap Permits the ability to send and receive a 1004Hz test tone and measure transmission loss onthe received tone.

SGNL Connects an available headset to a trunk.

TST Permits the user to run a number of supportedtests, including the following:• T100 (originating and terminating)

• T102 (originating and terminating)

• T104 (originating and terminating)

• T105 (originating and terminating)

ATTENTIONT105 termination on the MS2000 seriessupports only the LSC, L, NSC, RN, N, andNT subtests.

Table 35Monitor level commands (297-8001-821, Volume 7)

Command Description

CktMon Displays the messages on the MAP from thedigital PM of the circuit without interruptingnormal call processing.

Table 36C7TTP level commands (297-1001-821, Volume 2)

Command Description

CVTest Performs a Circuit Validation Test on a trunk.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 124: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

124 Fault management fundamentals

Command Description

TrkQry Displays the local or remote state of the trunk.

QrySig Queries the signaling availability and the stateof a trunk.

RouteSet Displays the route set CLLI of a CCS7 trunk.

CIC Displays the circuit identification code of a trunk.

ATTENTIONAll commands at the Automatic Test Trunk (ATT) level are supported.

TRKQRYThe TRKQRY test displays the local or remote status of the posted trunk.

Performing a TRKQRY testThe following procedure outlines the steps you must perform from the MAPto perform a TRKQRY on an ISUP trunk.

Running an TRKQRY test

Step Action

1 At the MAPCI;MTC;TRKS;TTP;C7TTP level of the MAP

Post the trunk you wish to test.

Examplepost g ATM01S7ITW2

LAST CKTN = 48POSTED CKT IDLEDSHORT CLLI IS: ATM01SOK, CKT POSTED

2 Run the TRKQRY test.

Example>trkqry

Local Trunk State: IDLE, LOCALLY AND REMOTELYBLOCKEDRemote Trunk State: IDLE, LOCALLY AND REM0TELYBLOCKEDStates are compatible - No (protocol) actionrequired

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 125: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Ethernet Routing Switch 8600 Ethernet port configuration, and 100Base-T cable replacement 125

Naming conventions for Media Gateway carrier endpoint namesThe first digit, in the carrier endpoint name, represents the logicalprocessor (LP) number on the Media Gateway, and is a number from 1 to15. If the carrier is associated with LP one, then you would use the digit 1,not 01 to indicate this LP. You are not permitted to use a leading zero. Forexample, DS1_101.1 is correct, but DS1_0101.1 is incorrect.

For information on the naming conventions for Media Gateway carrierendpoints, see OSSGate User’s Guide (NN10004-512).

Checking for Media Gateway logsMedia Gateway 15000/7400 alarm logs can be viewed in switch controlcenter 2 (SCC2) format at your operations support system (OSS). TheMDM software collects alarm and log data from the Media Gateway15000/7400. This data is, in turn, forwarded to the Communication Server2000 Core Manager. The Communication Server 2000 Core Manager LogStreamer application converts the Media Gateway logs into SCC2 format.The Communication Server 2000 Core Manager, then transmits the logsto the OSS.

Alternatively, you can view Media Gateway alarms by using applicationson the MDM. For information on viewing Media Gateway alarms onthe MDM, see Multiservice Data Manager Fault Management Tools(241-6001-011).

Media Gateway alarms can be viewed by using applications on the NortelMDM. For information on viewing Media Gateway alarms on the NortelMDM, see Nortel MDM Fault Management User Guide (241-6001-011).

Ethernet Routing Switch 8600 Ethernet port configuration, and100Base-T cable replacement

This document provides you with references to the Ethernet RoutingSwitch 8600 documents you need to consult for configuration of Ethernetports, and 100Base-T cabling.

Configuring Ethernet ports on Ethernet Routing Switch 8600For information on configuring a Ethernet Routing Switch 8600 Ethernetport, see Configuring Network Management (314723C), on this compactdisk. Refer to the chapter on port configuration and graphing.

100Base-T cablingFor information on verifying and replacing 100Base-T cabling thatconnects the Communication Server 2000 GWC to the ERS 8600,see Installing and Maintaining the 8010co Chassis and Components(312746C), on this compact disk.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 126: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

126 Fault management fundamentals

Add carriers to a GWCAfter associating a media gateway to a Gateway Controller (GWC) node,use this procedure to add carriers (endpoint groups) and their endpointsto the gateway.

Do not use this procedure to add V5.2 carriers. Instead, use procedure"Adding V5.2 interfaces" in Nortel Gateway Controller Configuration(NN10205-511).

CAUTIONPossible partial service disruptionIf you are adding carrier endpoints after having previouslyremoved them from the same gateway, you will have terminalidentifier (TID) mismatches. Contact your next level of supportfor instructions on how to avoid TID mismatches.

Table 37Carrier and endpoint names

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

AMBIT line gateways Endpoint names must conform to the following rules for small lineendpoints:

• The endpoint name must contain at least one forward slash ("/").

• The name cannot start with “SU”.

• The maximum length of endpoint name is 32 characters.

ASKEY_LINE

Integrated Access Devices(IAD) gateways

Endpoint names must conform to the following rules for small lineendpoints:

• The endpoint name must contain at least one forward slash ("/").

• The name cannot start with “SU”.

• The maximum length of endpoint name is 32 characters.

MG32LN p/<nnn>

where

p is lower case<nnnn> is an endpoint number: 1 to 384

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 127: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Add carriers to a GWC 127

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

MEDIA_SRVMG32003500

(Nortel Media Gateway3200 and 3500)

Carrier or gateway endpoints are provisioned in groups representingE1 and DS1 levels. Provisioning at this level gives access to alltimeslots for service provisioning.

Carrier names are case sensitive and must be entered in uppercase. The following descriptions include both endpoint groups andindividual endpoint descriptions. Services can be applied at theDS1/E1 timeslot level.

DS1

DS1/<h2>/<g> (H.248/Megaco)

where

<h2> is the DS1 (two digit) port or span number:01-04

<g> is the channel number: 1-24 (no leading 0),assigned by the system

Example:

Carrier name: DS1/03Carrier Endpoint name (timeslot): DS1/03/1

MEDIA_SRV_6310_TRUNK

(Nortel Media Gateway3500 using TP-6310 cardand configured as trunkgateway)

Carrier or gateway endpoints are provisioned in groups representingE1 and DS1 levels.

Carrier names are case sensitive and must be entered in uppercase. The following descriptions include both endpoint groups andindividual endpoint descriptions. Services can be applied at theDS1/E1 timeslot level.

DS1

DS1/<h2>/<g_optional> (H.248/Megaco)

where

<h2> is the DS1 (two digit) port or span number:01-84

<g> is optional PRI IID: 0 to 31 (32 to 63 range is notsupported). This value must be unique foreach gateway.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 128: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

128 Fault management fundamentals

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

Example:Carrier name: DS1/03

Each DS1 carrier supports 24 terminations and the appropriatenumber of terminations must be reserved in advance. A fullyequipped gateway with DS1 carriers requires 84 x 24=2016terminations.

E1

E1/<h2>/<g_optional> (H.248/Megaco)

where

<h2> is a number: 1 to 63<g> is optional PRI IID: 0 to 63

Example:Carrier name: E1/03

Each E1 carrier supports 31 terminations and the appropriatenumber of terminations must be reserved in advance. A fullyequipped gateway with E1 carriers requires 63 x 31=1953terminations.

MEDIA_SRV_MS2030

Media Server 2030

Endpoints are not specified during provisioning of audio servers.

CALIX_C7 tp/<TT><tt>

where

tp is lower case<TT> is a number: 00-10<tt> is a number: 00-99 (except <TT> is 10, in which

case <tt> is limited to the range 00-22); for amaximum of 1022 terminations allowed for eachgateway

Example: tp/1013

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 129: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Add carriers to a GWC 129

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

CICM tp/<vmg>/<nnnn>

where

tp is lower case<vmg> is a number: 0 - 2<nnnn> is a number: 0001 - 1022; always zero padded

Example: tp/1/0002

Each endpoint appears as a virtual media gateway (VMG). For moreinformation, see Nortel CICM Fundamentals ( (NN10044-111)) .

CISCO_2600

CISCO_3600

CISCO_AS5300

CISCO_H323_IOS

Supporting H.323 signalingprotocol

EPG_<n>

where

<n> is a number: 000 - 999

The underscore character "_" is required and the number mustinclude three digits (it must be zero-padded).

Duplicate carrier names on different media gateways are permitted.

Example: EPG_001

H323_PROXY EPG_<n>

where

<n> is a number: 000 - 999

The underscore character "_" is required and the number mustinclude three digits (it must be zero-padded).

Duplicate carrier names on different media gateways are permitted.

Example: EPG_001

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 130: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

130 Fault management fundamentals

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

KEYMILE_UMUX POTS table entry: p/nnn

where

p is lower case<nnnn> is an endpoint number: 1 to 480; not padded

(no leading zeros for padding to three digits)This value should match the terminal number.

Example: p/24

Third-party media gateways support flexible allocation of the LENcircuit number and the endpoint terminal number, independently ofthe endpoint name. The circuit number allocated for an endpointmust be unique and in the range of 1 to 480. This value identifiesthe LEN that the system creates. Nortel recommends that the circuitnumber is equal to: <nnn> - 1. For example, if the endpoint is p/1,the associated LEN is: <LGRP> 00 00. Nortel recommends that youcreate the endpoints as blocks of 30 contiguous endpoints to alignwith the configuration of the Keymile UMUX cards.

KEYMILE_UMUX BRI table entry: ba/nnn

where

ba is lower case<nnnn> is an endpoint number: 1 to 224; not padded

(no leading zeros for padding to three digits)This value should match the terminal number.

Example: ba/20

The circuit number must be restricted to the bottom half of theGW’s LGRP (for example, if the LGRPINV entry is KEYM 00 0,the valid range of circuit numbers allows for LENs in the rangeKEYM 00 0 00 00 to KEYM 00 0 05 10). For each BRI endpointthe secondary auto created endpoint is circuit number + 512. Thismeans that for a given BRI LEN, the associated LEN cannot be usedfor other purposes such as a POTS line (for example, KEYM 00 0 0000 provisioned for BRI blocks the use of KEYM 00 0 05 12 for otherpurposes). For ease of maintenance, Nortel recommends that yousplit each block of 30 endpoints into three blocks of eight, with sixunassigned endpoints at the end of the block.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 131: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Add carriers to a GWC 131

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

MEDIATRIX

Integrated Access Device(IAD) gateways

Endpoint names must conform to the following rules for small lineendpoints:

• The endpoint name must contain at least one forward slash ("/").

• The name cannot start with “SU”.

• The maximum length of endpoint name is 32 characters.

MG3600_4U Carrier or gateway endpoints are provisioned in groups representingE1 and DS1 levels.

DS1

DS1/<nn>

where

<nn> is a number: 01-80

Example:Carrier name: DS1/03

E1

E1/<nn>

where

<nn> is a number: 01 to 80

Example:Carrier name: E1/03

MG3600_8U Carrier or gateway endpoints are provisioned in groups representingE1 and DS1 levels.

DS1

DS1/<nnn>

where

<nnn> is a number: 001-128

Example:Carrier name: DS1/003

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 132: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

132 Fault management fundamentals

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

E1

E1/<nnn>

where

<nnn> is a number: 001-128

Example:Carrier name: E1/003

MGCP line gatewayMGCP_IAD_40 linegateway

Endpoint names must conform to the following rules for small lineendpoints:

• The endpoint name must contain at least one forward slash ("/").

• The name cannot start with “SU”.

• The maximum length of endpoint name is 32 characters.

MILEGATE POTS table entry: p/nnn

where

p is lower case<nnnn> is an endpoint number: 1 to 608; not padded

(no leading zeros for padding to three digits)This value should match the terminal number.

Example: p/24

Third-party media gateways support flexible allocation of the LENcircuit number and the endpoint terminal number, independently ofthe endpoint name. The circuit number allocated for an endpointmust be unique and in the range of 1 to 608. This value identifiesthe LEN that the system creates. Nortel recommends that the circuitnumber is equal to: <nnn> - 1. For example, if the endpoint is p/1,the associated LEN is: <LGRP> 00 00. Nortel recommends that youcreate the endpoints as blocks of 32 contiguous endpoints to alignwith the configuration of the MILEGATE cards.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 133: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Add carriers to a GWC 133

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

MOTOROLA MTA

line gateways

Endpoint names must conform to the following rules for small lineendpoints:

• The endpoint name must contain at least one forward slash ("/").

• The name cannot start with “SU”.

• The maximum length of endpoint name is 32 characters.

NORTEL_BCM

Supporting H.323 signalingprotocol

EPG_<n>

where

<n> is a number: 000 - 999

The underscore character "_" is required and the number mustinclude three digits (it must be zero-padded).

Duplicate carrier names on different media gateways are permitted.

Example: EPG_001

NUERA_BTX4K The gateway profile name NUERA_BTX4K supports the NueraBTX4K gateway, which allows provisioning of six DS3 or two opticalOC3 interfaces.

There are 24 DS0 channels for each DS1 interface for a total of4032 DS0 channels for each BTX4K gateway.

OC3 (with DS1 level)

DS/OC3-<u1>/DS1-<u2>/<c>

where

<u1> is a decimal value referring to the particularinstance of oc3: 1-2

<u2> is a decimal value referring to the particularinstance of ds1: 1-84

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: DS/OC3-2/DS1-1Carrier Endpoint name (timeslot): DS/OC3-2/DS1-1/1

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 134: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

134 Fault management fundamentals

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

DS3 (with a DS1 level within a logical processor or slot)

DS/S-<u1>/DS3-<u2>/DS1-<u3>/<c>

where

<u1> is a decimal value referring to the particularinstance of s (slot): 1-2

<u2> is a decimal value referring to the particularinstance of ds3: 1-3

<u3> is a decimal value referring to the particularinstance of ds1: 1-28

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: DS/s-2/DS3-3/DS1-1Carrier Endpoint name (timeslot): ds/s-2/ds3-3/ds1-1/1

DS3 (without DS1 framing)

DS/S-<u1>/DS3-<u2>/<c>

where

<u1> is a decimal value referring to the particularinstance of s (slot): 1-2

<u2> is a decimal value referring to the particularinstance of ds3: 1-3

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: DS/S-2/DS3-3/Carrier Endpoint name (timeslot): DS/S-2/DS3-3/1

DS3 (with channelized DS1 levels)

DS/DS3-<u1>/DS1-<u2>/<c>

where

<u1> is a decimal value referring to the particularinstance of ds3: 1-3

<u2> is a decimal value referring to the particularinstance of ds1: 1-28

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 135: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Add carriers to a GWC 135

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

Example:Carrier name: DS/DS3-2/DS1-3Carrier Endpoint name (timeslot): DS/DS3-2/DS1-3/1

DS1 (within a logical processor or slot)

DS/S-<u1>/DS1-<u2>/<c>

where

<u1> is a decimal value referring to the particularinstance of s (slot): 1-2

<u2> is a decimal value referring to the particularinstance of ds1: 1-84

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: DS/S-2/DS1-1Carrier Endpoint name (timeslot): DS/S-2/DS1-1/1

DS1

DS/DS1-<u>/<c>

where

<u> is a decimal value referring to the particularinstance of ds1: 1-84

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: DS/DS1-1Carrier Endpoint name (timeslot): DS/DS1-1/1

E1

DS/E1-<u1>/<c>

where

<u1> is a decimal value referring to the particularinstance of e1: 1-63

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-31

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 136: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

136 Fault management fundamentals

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

Example:Carrier name: DS/E1-1Carrier Endpoint name (timeslot): DS/DE1-1/1

NUERA_GX Carrier or gateway endpoints are provisioned in groups representingE1 and DS3 levels. Provisioning at this level gives access to alltimeslots for service provisioning. Supported services include ISUPtrunking and PRI trunking.

Carrier names are case sensitive and must be entered in uppercase. The following descriptions include both endpoint groups andindividual endpoint descriptions. Services can be applied at theDS1/E3 timeslot level.

E1/<h1>/<h2>/<g> (H.248/Megaco)

where

<h1> is the two-digit LP (logical processor) number(or slot) of the E1: 1-15

<h2> is the E1 (two digit) port number: 01-32<g> is the channel number: 1-31 (no leading 0),

assigned by the system

H.248/Megaco example:Carrier name: E1/03/05Carrier Endpoint name (timeslot): E1/05/05/1

DS3/DS1

DS1 carriers are provisioned using the same endpoint naming asDS3.

DS3/<h1>/<h2>/<g> (H.248/Megaco)

where

<h1> is the LP (logical processor) number or slot ofthe DS3: 1-15 (2-5 recommended)

<h2> is the DS3 port number<g> is the channel number in the DS3: 1-24, assigned

by the system

H.248/Megaco example:Carrier name: DS3/03/05Carrier Endpoint name (timeslot): DS3/05/05/1

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 137: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Add carriers to a GWC 137

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

PVG7K,PVG15KPVG15K_1000PVG15K_PARTIAL

Carrier or gateway endpoints are provisioned in groups representingE1 and DS3 levels. Provisioning at this level gives access to alltimeslots for service provisioning. Supported services include ISUPtrunking and PRI trunking.

Carrier names are case sensitive and must be entered in uppercase. The following descriptions include both endpoint groups andindividual endpoint descriptions. Services can be applied at theDS3/E1 timeslot level.

E1

e1/<h1>/<h2>/<g> (H.248/Megaco)

where

<h1> is the two-digit LP (logical processor) numberor slot of the E1: 1-15 (no leading 0)

<h2> is the E1 (two digit) port number: 01-32<g> is the channel number: 1-31 (no leading 0),

assigned by the system

H.248/Megaco example:Carrier name: e1/03/05Carrier Endpoint name (timeslot): e1/03/05/1

DS3

DS3/<h1>/<h2>/<g> (H.248/Megaco)

where

<b1> is the LP (logical processor) number or slot ofthe DS3: 1-15 (2-5 recommended)

<b2> is the DS3 (single digit) port number: 0-1<c> is the DS3 number: 1-28<d> is the channel number in the DS3: 1-24, assigned

by the system

H.248/Megaco example: Carrier name: DS3/03/05Carrier Endpoint name (timeslot): DS3/05/05/1

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 138: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

138 Fault management fundamentals

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

STM-1

STM/<lp>/<p>/1/VC4VC12/1/<k>/<l>/<m>/<e>(H.248/Megaco)

where

<lp> is the LP (logical processor) number or slotof the STM-1 interface: 2-15 (slots 2-5 arerecommended)

<pp> is the two-digit port number: 00-03<p> is the one-digit port number: 0-3<k> is the one-digit TUG-3 number within a VC4: 1-3<l> is the one-digit TUG-2 number within a TUG-3:

1-7<m> is the one-digit TU number within a TU: 1-3<e> is the VC12 channel/timeslot: 1-31 (no leading

0)

Hard-coded values 1/VC4VC12/1 (H.248) indicate the STM carriertype, multiplexing within the STM-1 frame, and the AUG within theSTM-1 frame.

H.248/Megaco example:Carrier name: STM/2/1/1/VC4VC12/1/3/6/1Carrier Endpoint name (timeslot): STM/2/1/1/VC4VC12/1/3/6/1/1

PVG15K_MEGACO, PVG15K_PARTIAL_MEGAGO, andPVG_VSP3_MEGACO gateways also support the V5.2 service typeand this STM-1 (Megaco) carrier name format in V5.2 services.

OC-3

STS/<lp>/<p>/3/VT15/<t>/<l>/<m>/<e>(H.248/Megaco)

where

<lp> is the LP (logical processor) number or slot ofthe OC-3 interface: 1-15 (recommended slots2-5)

<pp> is the (two digit) port number: 00-03<p> is the (one digit) port number: 0-3<jj> is the (two digit) STS-1 number within the

STS-3: 01-03

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 139: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Add carriers to a GWC 139

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

<l> is the (one digit) VT group number within STS-1:1-7

<m> is the (one digit) VT number within a VT: 1-4<t> is the (one digit) STS-1 number within the STS-3:

0-3<e> is the VT1.5 channel/timeslot: 1-24 (no leading

0)

Hardcoded value /3/VT15/ (H.248) indicates the STS carrier typeand the multiplexing within the OC-3 carrier.

H.248/Megaco example:Carrier name: STS/2/1/3/VT15/1/6/1Carrier Endpoint name (timeslot): STS/2/1/3/VT15/1/6/1/1

PVG_VSP4E STM/<l>/<p>/1/VC4VC12/1/<t>/<u>/<v>

where

<l> is the LP (logical processor) number or slot ofthe STM interface: 2-15

<p> is the one-digit port number: 0-3<t> is the one-digit TUG-3 number within a VC4: 1-3<u> is the one-digit TUG-2 number within a TUG-3:

1-7<v> is the one-digit TU number within a TU: 1-3

The maximum capacity is 31 endpoints.

Hard-coded value of 1/VC4VC12/1 indicates the STM carrier type,multiplexing within the STM-1 frame, and the AUG within the STM-1frame.

Example:Carrier name: STM/2/1/1/VC4VC12/1/3/6/1Carrier Endpoint name (timeslot): STM/2/1/1/VC4VC12/1/3/6/1

STS/<l>/<p>/3/VT15/<t>/<v>/<w>

where

<l> is the LP (logical processor) number or slot ofthe STS interface: 1-15

<p> is the port number: 0-3<t> is the STS-1 number: 1-3<v> is the VT group number: 1-7<w> is the VT number: 1-4

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 140: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

140 Fault management fundamentals

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

The maximum capacity is 24 endpoints.

Hard-coded value of 3/VT15 indicates the STS carrier type.

Example: Carrier name: STS/2/1/3/VT15/1/3/4Carrier Endpoint name (timeslot):STS/2/1/3/VT15/1/3/4

SIPVOICE /<SITE>/<FFF>/<G>/<TT tt>

where

<SITE> is a name defined in table SITE configured onthe Core

<FFF> is a frame number: 0 - 511<G> is a group number: 0 - 9<TT tt> is a terminal number: 00 00 - 1023

Example: /ABCD/10/9/01 20

SUCCESSION_1000

Supporting H.323 signalingprotocol

EPG_<n>

where

<n> is a three-digit number: 000 - 999

The underscore character "_" is required and the number mustinclude three digits (it must be zero-padded).

Duplicate carrier names on different media gateways are permitted.

Example: EPG_001

TGCP GWCs support TGCP media gateways with

• DS1 interfaces

• DS1 interfaces within a logical processor or slot

• OC3 interfaces with channelized DS3 and DS1 levels

• OC3 interfaces with channelized DS3 and DS1 levels within alogical processor or slot

• DS3 interfaces with a channelized DS1 level within a logicalprocessor or slot DS3 interfaces without DS1 framing

• DS3 interfaces with a channelized DS1 level

• E1 interfaces

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 141: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Add carriers to a GWC 141

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

The following abbreviations are used in TGCP carrier and endpointformats:

• u = unit number

• c = channel number

The gateway profile name TGCP supports third party mediagateways using TGCP signaling protocol.

DS1

ds/ds1-<u>/<c>

where

<u> is a decimal value referring to the particularinstance of ds1: 1-68

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: ds/ds1-1Carrier Endpoint name (timeslot): ds/ds1-1/1

DS1(within a logical processor or slot)

ds/s-<u1>/ds1-<u2>/<c>

where

<u1> is a decimal value referring to the particularinstance of s (slot): 1-28

<u2> is a decimal value referring to the particularinstance of ds1: 1-28

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: ds/s-2/ds1-1Carrier Endpoint name (timeslot): ds/s-2/ds1-1/1

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 142: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

142 Fault management fundamentals

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

OC3 (with DS3 and DS1 levels)

ds/oc3-<u1>/ds3-<u2>/ds1-<u3>/<c>

where

<u1> is a decimal value referring to the particularinstance of oc3: 1-28

<u2> is a decimal value referring to the particularinstance of ds3: 1-28

<u3> is a decimal value referring to the particularinstance of ds1: 1-28

<channel #> is a decimal value indicating the channelnumber at the lowest level in the hierarchy:1-24

Example:Carrier name: ds/oc3-2/ds3-3/ds1-1Carrier Endpoint name (timeslot): ds/oc3-2/ds3-3/ds1-1/1

OC3(with DS3 and DS1 levels within a logical processor or slot)

ds/s-<u1>/oc3-<u2>/ds3-<u3>/ds1-<u4>/<c>

where

<u1> is a decimal value referring to the particularinstance of s (slot): 1-28

<u2> is a decimal value referring to the particularinstance of oc3: 1-28

<u3> is a decimal value referring to the particularinstance of ds3: 1-28

<u4> is a decimal value referring to the particularinstance of ds1: 1-28

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: ds/s-2/oc3-2/ds3-3/ds1-1Carrier Endpoint name (timeslot): ds/s-2/oc3-2/ds3-3/ds1-1/1

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 143: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Add carriers to a GWC 143

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

DS3(with a DS1 level within a logical processor or slot)

ds/s-<u1>/ds3-<u2>/ds1-<u3>/<c>

where

<u1> is a decimal value referring to the particularinstance of s (slot): 1-28

<u2> is a decimal value referring to the particularinstance of ds3: 1-28

<u3> is a decimal value referring to the particularinstance of ds1: 1-28

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy:1-24

Example:Carrier name: ds/s-2/ds3-3/ds1-1Carrier Endpoint name (timeslot): ds/s-2/ds3-3/ds1-1/1

DS3(without DS1 framing)

ds/s-<u1>/ds3-<u2>/<c>

where

<u1> is a decimal value referring to the particularinstance of s (slot): 1-28

<u2> is a decimal value referring to the particularinstance of ds3: 1-28

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: ds/s-2/ds3-3/Carrier Endpoint name (timeslot): ds/s-2/ds3-3/1

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 144: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

144 Fault management fundamentals

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

DS3(with channelized DS1 levels)

ds/ds3-<u1>/ds1-<u2>/<c>

where

<u1> is a decimal value referring to the particularinstance of ds3: 1-28

<u2> is a decimal value referring to the particularinstance of ds1: 1-28

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy:1-24

Example:Carrier name: ds/ds3-2/ds1-3Carrier Endpoint name (timeslot): ds/ds3-2/ds1-3/1

TGCP

(continued)

E1

ds/e1-<u1> /<c>

where

<u1> is a decimal value referring to the particularinstance of e1: 1-68

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-31

Example:Carrier name: ds/e1-1Carrier Endpoint name (timeslot): ds/e1-1/1

TOUCHTONE_NN

line gateways

Endpoint names must conform to the following rules for small lineendpoints:

• The endpoint name must contain at least one forward slash ("/").

• The name cannot start with “SU”.

• The maximum length of endpoint name is 32 characters.

UAS

Audio servers (includingNortel Media Server 2000Series)

Endpoints are not specified during provisioning of audio servers.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 145: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Geographic survivability 145

Table 37Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

UE9000MG

(Media Gateway 9000)

For non-ABI (access bridge interface) lines:

tp/<slot>/<circuit>

where

tp is lower case<slot> is a number: 2-9<circuit> is a number: 14-21

Example: tp/2/14

Each endpoint appears as a virtual media gateway (VMG).

The <slot> and <circuit> values constitute the SLOT and theCIRCUIT part of the line equipment number (LEN) for the gateway.

For MG 9000 ABI lines:

tp/channel/<channelID>

where

tp is lower case<channelID> is a number between 000 and 511

WESTELL

Supporting H.323 signalingprotocol

EPG_<n>

where

<n> is a three-digit number: 000 - 999

The underscore character "_" is required and the number mustinclude three digits (it must be zero-padded).

Duplicate carrier names on different media gateways are permitted.

Example: EPG_001

Geographic survivabilityGeographic survivability is the distribution of components across ageographic area to ensure services continue in the event of a disaster. Adisaster can include fire, flood, tropical storm, or act of terrorism.

The following figure shows the topology for Communication Server 2000- Compact support for geographic survivability.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 146: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

146 Fault management fundamentals

Figure 42Geographic survivability topology

The Communication Server 2000 - Compact supports geographicsurvivability through the failover and sparing functionality of individualnodes. The configuration for the Communication Server 2000Management Tools (CMT) and Integrated Element Management System(IEMS) servers and the Core and Billing Manager (CBM) 850 is as follows:

• Site A is configured with the CMT and IEMS high-availability (HA)server pairs

• Site B is configured with the CBM 850 HA server pairs (referred to asCBM 850 HA u0 and u1)

• Site A is configured with an additional CBM in to be used in the eventof disaster at Site B (referred to as CBM 850 standby u0)

• Site B is configured with an additional server for the CMT and IEMSin the event of disaster at Site A

For a communication server, geographic survivability requires thatredundant components (other than OAM components) reside at differentsites (referred to as Site A and Site B).

For OAM components, the Automatic Backup and Accelerated restorefeature (known as remote backup) remotely backs up all data on the targetunit. This provides a standby backup system ready to provide serviceshould the primary system or cluster be unavailable for an extended periodof time (for example, catastrophic site loss). The remote backup can

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 147: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Geographic survivability 147

assume the identity of the target system with data and files accurate tothe last sync and will be located at a different site from the target system.Remote backup performs the backup with a TCP/IP connection, and storesan exact copy on the standby server which can be quickly and remotelyactivated. This remote backup copies all files in each file system markedfor backup using the same behavior as a full system backup.

A remote backup configuration tool is provided to set the necessaryparameters for backup to automatically occur from once to four times aday. This tool also provides a facility for manually initiating a backup andmonitoring its progress. The standby server has an identical copy of filesfrom the last backup, so it can become the primary system via changingthe boot pointer and rebooting. When the primary site becomes available,the remote backup feature can be reused to transfer current systemconfiguration back to the primary site and system.

CAUTIONIf configuration, provisioning, patching or other “write”-typeoperations occurred since the last remote backup, the remotebackup system can be out of sync compared to the data innetwork elements and/or the primary OAM system.

When initiating a switchover to a remote backup OAM server,do not execute configuration, provisioning, patching or other“write”-type operations through the remote backup OAM systemuntil out-of-sync conditions are cleared.

Take actions before initiating the switchover to a remotebackup OAM server (that is, response to a geographic or otherprolonged outage of the primary OAM system) to halt or prevent“write”-type operations by OSSs and operations personnel untilan in-sync status is achieved.

If a site outage is imminent (for example, threatening severeweather conditions), consider precautionary preparations todiscontinue “write”-type operations. Make manual backups tothe remote backup OAM servers to ensure data synchronizationof the remote OAM server before performing a switchover.

Apply similar precautions when initiating a recovery back to theprimary cluster. In this more controlled scenario, discontinue“write”-type operations before initiating this procedure

Site failure in a geographic survivable configurationThere are two scenarios for recovery of a geographic survivable networkconfiguration:

• “Site A failure” (page 148)

• “Site B failure” (page 148)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 148: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

148 Fault management fundamentals

Site A failureA loss of Site A would include a loss of the CMT and IEMS servers. Thisloss would result in the following losses of functionality:

• all OAM&P functions of the CMT and IEMS, including GUI access,element management, alarms, and non-Core logs

• access to the Communication Server 2000-Compact Core Manageron the CBM 850

The responsive action in this scenario is to initiate a switchover to thestandby CMT and IEMS servers at Site B.

Site B failureA loss of Site B would include a loss of the CBM 850 servers. This losswould result in the following losses of functionality:

• transfer of billing records from the Core to the CBM 850

• billing records resident in the CBM 850 that had not been offloadedto an OSS

• ability to receive scheduled Core OMs and logs (resulting in thembeing discarded)

• access to the Communication Server 2000-Compact Core Manager

• backup bootp load repository

The responsive action in this scenario is to initiate a switchover to thestandby CMT and IEMS servers at Site A.

Fresh pre-install of CBM 850 cold u0To reduce the duration of downtime during an outage, it is recommendedthat the CBM 850 cold u0 be pre-installed. Contact your next level ofsupport for this task.

Maintaining sites in a geographic survivable configurationThe following procedures contain information for OA&M auto backup andaccelerated restore capability:

Geographical Survivability impacts to Communication Server2000-Compact

The Geographic Survivability feature allows services to continue in theevent of a natural or man-made disaster. A sub-panel to the Call AgentCard View Provisioning panel (tab) allows you to enable or disablethe feature. The following figure shows an example configuration withGeographic Survivability enabled.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 149: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Geographic survivability 149

Figure 43Call Agent Card View Provisioning panel: Geographic Survivability enabled

Failure scenariosThe following table provides summaries of system responses duringvarious failure scenarios when Geographical Survivability is enabled. Thescenarios assume redundant configurations are located in two separatesites.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 150: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

150 Fault management fundamentals

Table 38Failure system response summaries

Scenario Response

Ethernet Routing Switch8600 failure at one site

At the site with the failed routing switch:• All nodes lose mate connectivity via Ethernet.

• The Call Agent loses WAN backup connectivity.

• FC or GigE call data sync link remains up until the Call Agentresets.

• The Call Agent resets itself approximately 1 minute after losingnetwork connectivity and tries to boot from the network. The CallAgent keeps trying to boot until the network recovers and theboot is successful.

• The Call Agent detects IST loss, but cannot disable EthernetRouting Switch 8600 routing.

• USPc detects isolation and takes down SS7 links at the site withthe Ethernet Routing Switch 8600 failure.

• USP mated pair configurations:

— Mated pairs connected to pair of gateway POP EthernetRouting Switch 8600s:No impact to both USPs from a failure.

— Mated pairs split across a Communication Server-LAN site(each connected to the Ethernet Routing Switch 8600 of localCommunication Server 2000 CS-LAN):For Communication Server 2000 - Compacts that share thesame Communication Server-LAN, the USP detects isolationand takes down its SS7 links. An Ethernet Routing Switch8600 failure does not impact USP mated pair behavior inother Communication Server 2000 - Compact configurations.

— Mated pairs split across a Communication Server-LAN site(each connected to one Ethernet Routing Switch 8600connected to an edge site):No impact from a failure.

At the site with the in-service routing switch:• The Call Agent remains active if it is already active. No outage

occurs. If the Call Agent was not active, it takes activity within2 seconds.

• SOS goes through a warm or restart (approximately 20 secondsfor the NTRX51HZ card with the MCPN905 processor and 30seconds for the NTRX51GZ card with the MPCN765 processor).

• Other nodes go active and follow the Call Agent example afterlosing mate connectivity.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 151: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Geographic survivability 151

Scenario Response

• If Site B has the in-service router, the standby CMT servermust be brought into service. (See the following ATTENTIONstatement.)

• If Site A has the in-service router, the standby CBM must bebrought into service. (See the following ATTENTION statement.)

ATTENTIONConsider using this option based on the estimated time to recoverthe fault or failure of the Communication Server LAN versus timeand effort to bring up the cold standby system and recover the HApair afterward.

For example, if the time to recover the Communication Server LANis estimated to be 10 hours, consider if it is worth activating thecold standby CMT server or CBM.

See procedure Initiating a switchover to the remote backup server.

Optical frame failure atone site

At the site with the failed optical frame:• All nodes lose mate connectivity via Ethernet.

• Call Agent FC or GigE call data link connectivity is lost.

• WAN backup remains up.

• The active Call Agent remains active, but without sync. Nooutage occurs.The remainder of this scenario assumes the active Call Agentis at this site.

• If not already active, expect other nodes co-located with theactive Call Agent to become active after losing mate connectivity.If not already inactive, expect other nodes co-located withthe inactive Call Agent to become inactive after losing mateconnectivity. (Done without mate connectivity.)

• USP mated pair configurations:

— Mated pairs connected to pair of gateway POP EthernetRouting Switch 8600s:No impact to both USPs from a failure.

— Mated pairs split across a Communication Server-LAN site(each connected to the Ethernet Routing Switch 8600 of localCommunication Server 2000 Communication Server-LAN):For CS2K-1, only the USP at the active side providesservice. For CS2K-2, both USPs provide service, butover a packet network. There are no impacts to otherCommunication Server 2000s.

— Mated pairs split across a Communication Server-LAN site(each connected to one Ethernet Routing Switch 8600connected to an edge site):No impact from a failure.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 152: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

152 Fault management fundamentals

Scenario Response

At the site with the in-service optical frame:• The inactive Call Agent detects IST loss. For an Ethernet

Routing Switch 8600 Communication Server LAN using OSPF,the Call Agent disables OSPF. For an Ethernet Routing Switch8600 Communication Server LAN using Border GatewayProtocol (BGP) or for a third-party Communication Server LANusing either OSPF or BGP, execute manual action to disablerouting at the site with the inactive Call Agent.See section Enabling protocols BGP and OSPF.

• Other nodes become inactive and follow the Call Agent exampleafter losing mate connectivity. (Done without mate connectivity.)

• USPc detects isolation from the active Call Agent, and takesdown SS7 links at the site with the inactive USP.

• If Site B has the failed optical frame, the standby CBM must bebrought into service at Site A. (See the following ATTENTIONstatement.)

• If Site A has the failed optical frame, the standby CMT must bebrought into service at Site B. (See the following ATTENTIONstatement.)

ATTENTIONConsider using this option based on the estimated time to recoverthe fault or failure of the Communication Server LAN versus timeand effort to bring up the cold standby system and recover the HApair afterward.

For example, if the time to recover the Communication Server LANis estimated to be 10 hours, consider if it is worth activating thecold standby CMT server or CBM.

See procedure Initiating a switchover to the remote backup server.

One site is destroyed in acatastrophic event

At the site that is destroyed:• There is no activity.

• The SS7 network takes down the links to the destroyed building.

• USP mated pair configurations:

— Mated pair connected to pair of gateway POP EthernetRouting Switch 8600s:No impact occurs to either USP as they are located indifferent sites from a failure.

— Mated pair split across a Communication Server-LAN site(each connected to the Ethernet Routing Switch 8600 of localCommunication Server 2000 Communication Server-LAN):

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 153: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Geographic survivability 153

Scenario Response

The SS7 network takes down all links to the destroyed site.

— Mated pair split across a Communication Server-LAN site(each connected to one Ethernet Routing Switch 8600connected to an edge site):If the USPs are located at the same site where the failureoccurred, the SS7 network takes down all links at thedestroyed site. Otherwise, no impact occurs to either USPfrom the failure.

At the site that is not destroyed:• All nodes lose mate connectivity.

• The Call Agent loses all mate connectivity, including the backupWAN link, and drops sync.

• If the Call Agent is active, it remains active. No outage occurs. Ifthe Call Agent is not active, it takes activity within 2 seconds.

• If the Call Agent is inactive, it takes activity within 2 secondsfollowed by a warm or SOS restart (approximately 20 secondsfor the NTRX51HZ card with the MCPN905 processor, and 30seconds for the NTRX51GZ card with the MCPN765 processor).

• Other nodes go active, and follow the Call Agent example afterlosing mate connectivity.

• If Site B was destroyed, initiate a switchover to the standby CBMat Site A. (See procedure Initiating a switch over to the remotebackup server.

• If Site A was destroyed, initiate a switchover to the standby CMTat Site B. (See procedure Initiating a switch over to the remotebackup server.

Active Call Agent cardfails at one site

At the site with the failed Call Agent:• All nodes at the site can communicate with their mates.

At the site with the mate Call Agent:• All nodes in the site can communicate with their mates.

• The (inactive) Call Agent detects loss of connectivity with themate, detects local and WAN connectivity, takes activity, andrestarts the SOS.

• Other nodes experience disconnection from the SOS forapproximately 20 seconds for the NTRX51HZ card with theMCPN905 processor, and 30 seconds for the NTRX51GZ cardwith the MCPN765 processor (normal restart behavior)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 154: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

154 Fault management fundamentals

Scenario Response

Inactive Call Agent cardfails at one site

At the site with the failed inactive Call Agent:• All nodes in the site can communicate with their mates.

At the site with the mate Call Agent:• All nodes in the site can communicate with their mates.

• The (active) Call Agent detects loss of connectivity with the mate,detects local and WAN connectivity, and stays active.

• No SWACT or restart is required.

Recovery fromisolation split brain(Active/Inactive)

At both sites:• Once the Call Agents can communicate with their mates, they

recognize that both are active. The Call Agent that was inactivebacks down, leaving the other Call Agent active. The fallout is toforce Unit 0 active and Unit 1 inactive.

• Other nodes can communicate with their mates, negotiateactivity, and resume normal operations.

System impact of failures Failures could cause the following systemimpacts:

• When negotiating activity, the Call Agent preference is always toremain on the same side, if that side supports activity.

• If failover of the Call Agent is necessary, the Call Agent switchesactivity in less than 2 seconds. When the Callp Application performs arestart, call processing is interrupted for approximately 20 seconds forthe NTRX51HZ card with the MCPN905 processor, and 30 secondsfor the NTRX51GZ card with the MCPN765 processor (normal restartbehavior). Failovers of other Callp nodes follow the Call Agent failoverwithin 2 seconds.

• In Enterprise-only configurations where the solution contains MessageController (MC) cards connected to Message Switches (MS), ENETand TDM peripherals, the MC and MS cards are co-located in one ofthe main geographically redundant sites.

• During site isolation, when determining the appropriate master site,preference is given to the site that contains the MC cards. Thisassumes that the site is able to take activity. If necessary, activity isswitched to this side during the activity negotiation.

Recovery scenariosThe following table provides a summary of system responses duringrecovery.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 155: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Geographic survivability 155

Table 39Recovery system response summary

Scenario Response

Recovery fromisolation split brain(Active/Inactive)

• When Call Agents can communicate with their mates, theyrecognize that both are active. The Call Agent that was inactivebefore the failure backs down, leaving the other Call Agentactive. The fallout is to force Unit 0 active, and Unit 1 inactive.

• Other nodes can communicate with their mates, negotiate activityand resume normal operations.

General recoverybehavior

All elements:• continually monitor connections with their mates, and with

other network elements with which they normally communicate.When connectivity is not present, they continue to monitor theconnections for restored connectivity. (The elements continuemonitoring regardless of their activity state.)

• negotiate activity and services when connectivity recovers, andresume normal operations.

• USP mated pair configurations:

— Mated pair connected to pair of gateway POP EthernetRouting Switch 8600s:No impact to USP mated pair.

— Mated pair split across a Communication Server-LAN site(each connected to the Ethernet Routing Switch 8600of local Communication Server 2000 CommunicationServer-LAN):USP mated pair does not negotiate activity during recovery.This is a result of USP mated pair operating in a load sharingmode; both are active during normal conditions.

— Mated pair split across a Communication Server-LAN site(each connected to one Ethernet Routing Switch 8600connected to an edge site):No impact from a failure.

System impact of recovery Recovery could cause the following systemimpacts:

• When negotiating activity, the Call Agent preference is always toleave activity on the same side. When recovering to a full systemconfiguration, activity remains on the same unit, without impact.

• During recovery from a split system (caused by incorrect messagerouting), node activity resolves in a few seconds. Call processing couldrequire up to 15 minutes to recover completely.

• If an inactive call agent is not in service when the IST goes down,OSPF on the Ethernet Routing Switch 8600 is not disabled on the

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 156: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

156 Fault management fundamentals

site of the inactive call agent, potentially resulting in failed calls due tomisdirected messages. This condition can occur on site recovery afterthe Ethernet Routing Switch 8600 returns to service and the opticalring is still recovering. In the event that this outage occurs, manuallydisable OSPF on the Ethernet Routing Switch 8600 or disconnect thelinks to the WAN on the site of the inactive call agent until it returns toits operational state.

Enabling protocols BGP and OSPFThis section applies only to Communication Server-LAN routers configuredwith Ethernet Routing Switch 8600s in which BGP is the default routingprotocol.

Protocols OSPF and BGP serve as an interdomain protocols to distributerouting information between endpoints in an Geographic Survivabilityconfiguration.

Recall that the Communication Server-LAN consists of dual EthernetRouting Switch 8600s running as layer 2/3 switches. For redundancy, apair of upstream routers must be deployed on the Core network edge asthe entrance to the Communication Server-LAN.

Nortel supports a square Communication Server-LAN topologyconfiguration, in which each Ethernet Routing Switch 8600 is connected toone of two upstream routers. The following figure shows an example ofthe Ethernet Routing Switch 8600s configured for Geographic Survivabilityin a square topology.

Figure 44Ethernet Routing Switch 8600

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 157: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Geographic survivability 157

In this example, each Ethernet Routing Switch 8600 includes two BGPneighbors. One BGP neighbor is the eBGP peer to the wide area network(WAN) Core router. The other BGP neighbor is the iBGP peer to itsEthernet Routing Switch 8600 mate.

The BGP functionality also supports the following features:

• can be used with both MCPN 905 and MCPN 765 type Call Agentcards

• selection of either BGP or OSFP as the protocol during GeographicSurvivability installation

• script setgeoconfig supports either BGP or OSPF input

• Call Control Agent (CCA) sends control commands (Disable/EnableBGP or OSPF) to the Ethernet Routing Switch 8600 based on protocolselection during installation

• Inactive CCA triggers command BGP disable command to the EthernetRouting Switch 8600 when optical connectivity is lost between twosites. When recovering optical connectivity, the inactive CCA triggerscommand BGP enable to the Ethernet Routing Switch 8600.

• Existing routing protocol disable or enable event log is used with thecorresponding OSPF or BGP protocol.

USP mated pairsTo support Geographic Survivability, a mated pairs of USPs can beconfigured with up to six Communication Server 2000 - Compact callservers. Nortel supports the following USP mated pair configurations in aGeographic Survivability deployment:

• mated pair of USPs connect to a pair of gateway POP EthernetRouting Switch 8600s, which is co-located with the CommunicationServer-LAN or in a remote site from the Communication Server 2000or Communication Server 2000 - Compact

• mated pair of USPs are split across a Communication Server-LAN site,whereby each USP connects to an Ethernet Routing Switch 8600 in alocal Communication Server-LAN configuration

• mated pair of USPs are split across a Communication Server-LAN siteand each connects to a single Ethernet Routing Switch 8600, whichalso is connected to an edge site

For details on mated USP pairs, see USP Configuration Management(NN10093-511).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 158: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

158 Fault management fundamentals

Mated pair connected to a pair of gateway POP EthernetRouting Switch 8600sWhile configuration also applies to a non-Geographic Survivabilityenvironment, in a Geographic Survivability environment, the Cores canshare the same optical ring providing the distance is close. The matedUSP pair is co-located and connect to two Ethernet Routing Switch 8600s,which also connect to a POP gateway.

The following figure shows an example of this configuration.

Figure 45Mated pairs connected to a pair of gateway POP Ethernet Routing Switch8600s

The following conditions apply to this configuration:

• Two RTC cards cross-connect to two Ethernet Routing Switch 8600son each USP.

• Dedicated pairs of IPS7 cards serving as inter-USP mated linkscross-connect to two Ethernet Routing Switch 8600s on each USP.The mated link transfers SSNM messages. (The mated link transferstraffic during some failure conditions.)

• Other IPS7 card pairs dedicated to Communication Server 2000 Coresconnect to local Ethernet Routing Switch 8600 on each USP.

• All the Communication Server 2000s share SS7 cards or havededicated SS7 cards.

• Only ABS stand-alone servers can be used.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 159: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Geographic survivability 159

• Two ABS servers are co-located with two USP pairs and connected totwo Ethernet Routing Switch 8600s.

• Both USP pairs are connected to only one IEMS server.

Mated pair of USPs split across a CS-LAN site (each connectedto the Ethernet Routing Switch 8600 in a local CommunicationServer 2000 CS-LAN)In this configuration, a mated pair of USPs are split across aCommunication Server-LAN site, whereby each USP connects to anEthernet Routing Switch 8600 in a local Communication Server-LANconfiguration.

The following figure shows an example of this configuration.

Figure 46Mated USP pair split across a CS-LAN site (each connected to the EthernetRouting Switch 8600 of local Communication Server 2000 CS-LAN)

The following conditions apply to this configuration:

• Two RTC cards connect to different Ethernet switch boards on eachmated USP pair.

• Dedicated pairs of IPS7 cards serving as inter-USP mated linkscross-connect to two Ethernet Routing Switch 8600s on each USP.The mated link transfers SSNM messages. (The mated link transferstraffic during some failure conditions.)

• Other IPS7 cards dedicated to Communication Server 2000 Coresconnect to local Ethernet Routing Switch 8600 on each USP.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 160: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

160 Fault management fundamentals

• All the Communication Server 2000s share SS7 cards or havededicated SS7 cards.

• The CBM and CMT can be used as an ABS server for mated USPpairs.

• An ABS server is co-located with one half of CS2Kc-1.

• Two UPS pairs connect to one IEMS server.

Mated pair of USPs split across a CS-LAN site (each connectedto a single Ethernet Routing Switch 8600)In this configuration, a mated pair of USPs are split across aCommunication Server-LAN site. Each pair connects to a single EthernetRouting Switch 8600, which also could be connected to an edge site.

The following figures show examples of this configuration. Both figuresreflect connections for RTC and IPS7 cards.

Figure 47Mated USP pair split across a CS-LAN site (each connected to a singleEthernet Routing Switch 8600)

The following figure shows the same configuration but with the EthernetRouting Switch 8600 connected to an edge site.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 161: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Geographic survivability 161

Figure 48Mated USP pair split across a CS-LAN site (each connected to a singleEthernet Routing Switch 8600, which connects to an edge site)

VLAN assignmentAll USP mated pair configurations support the following VLANs:

• Both RTC cards of USP A are assigned to OAM_USP_A_VLAN.

• Both RTC cards of USP B are assigned to OAM_USP_B_VLAN.

• IPS7 cards serving as mated links on USP A are assigned toCallP_USP_A_Mate VLAN.

• IPS7 cards serving as mated links on USP B are assigned toCallP_USP_B_Mate VLAN.

• IPS7 cards dedicated to CS2Kc-1 on USP A are assigned toCallP_USP_A_CS2K1.

• IPS7 cards dedicated to CS2Kc-1 on USP B are assigned toCallP_USP_B_CS2K1.

• IPS7 cards dedicated to CS2Kc-2 on USP A are assigned toCallP_USP_A_CS2K2.

• IPS7 cards dedicated to CS2Kc-2 on USP B are assigned toCallP_USP_B_CS2K2.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 162: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

162 Fault management fundamentals

Limitations and restrictionsThe following limitations and restrictions apply to this USP configuration.

• Customers must guarantee a secure connection between the USP andthe remote Communication Server 2000 - Compact.

• For USPs to communicate with remote Communication Server 2000 -Compacts over the Core network, the CCA, 3PC or XA-Core HIOPsmust be advertised in the carrier’s network, although they have beenconfigured in the call processing private VLAN. To reach the USPs, theVLAN must be advertised outside of the Ethernet Routing Switch 8600Communication Server-LAN. Other devices that have been provisionedin the same VLAN (for example, STORM-IA) can be exposed.

• To reduce the risk of DoS attack, provide new filtering rules andnew traffic limitation rules on the GW-POP. For example, the newfiltering rules on the GW-POP must accept traffic to the USP only if itemanated from the GWC.

• If IPsec is enabled, apply the following filtering rules to implement theGW-POP firewall.

— Discard all traffic to the CCA except traffic from the GWC subnets.

— Discard all non-ESP or non-IKE traffic (UDP traffic to and from port500).

• Ensure that USP mated pair-specific VLANs are not disabled duringa ring outage since the USPs are not split across the ring as in othergeographic-configured VLANs.

• Nortel recommends that mated USPs installations not exceed 120 kmsfrom the Communication Server-LAN.

SSL Geographic SurvivabilityBefore CVM11, SSL components supported network interface failure,whereby they could detect when all network interfaces connecting themto the LAN had failed. When Session Server Lines (SSL) componentsdetected a network isolation condition, they ensured that an isolatedcomponent did not remove activity from its mate when the isolationcondition was repaired. This behavior mirrors fault tolerant behaviorfor a component that detects itself to be in the non-surviving site of aGeographic Survivable configuration. The affected components arecompletely isolated from the rest of the network while call processingcontinues.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 163: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Geographic survivability 163

SSL components detect themselves network isolated under the followingconditions:

• the Geo mode is enabled in the Communication Server 2000 CallAgent window

• no active Communication Server 2000 Compact Call Agent existswhen an SSL System Manager or Session Manager starts

The SSL component should detect the isolation before it attemptsactivation. The component shuts itself down and waits until there is anactive Communication Server 2000 Compact Call Agent present beforeit restarts itself.

Non-surviving site network interface isolationSSL Geographic Survivability supports a geographic distribution ofredundant SSL components across a second site to preserve callprocessing, billing and element management functions in case of a failureisolating one of the sites. In addition, call capacities do not degradefrom the failure of one of the sites. Network requirements and facilitiesinterconnecting the two sites is efficient and cost effective.

SSL components that detect themselves to be in a non-surviving site of aGeographically Survivable configuration behave as if they do when theydetect a network interface failure. That is, they consider themselves to benetwork isolated.

This feature introduces a mechanism to detect the condition of anon-surviving site of a Geographically Survivable configuration. Detectingnetwork isolation of a non-surviving site of a Geographically Survivableconfiguration is based on the reachability of the active IP address ofthe Communication Server 2000 Call Agent. Internet Control MessageProtocol (ICMP) echo request (ping) messages are sent to the active IPaddress of the Communication Server 2000 Compact Call Agent. SSLcomponents that detect this IP address "unreachable" consider themselvesto be in the non-surviving site (that is, network isolated).

See NTP SSL Configuration (NN10023-511) for details on how to enableor disable pinging on the active IP address of the Communication Server2000 Call Agent.

Network isolation detection for a non-active network elementinstance (NEI)When a non-active NEI detects itself to be network isolated, it shuts itselfdown and waits for the network isolation condition to clear before restartingitself.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 164: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

164 Fault management fundamentals

This action avoids a dual-active scenario when a stand-by NEI detectsitself to be network isolated (whereby its mate is active) before falselydetecting its active mate as failed. In this case, it also shuts itself down.The NEI restarts itself when the network isolation condition is cleared. Thismeasure prevents the active mate from having to decide if it should cedeactivity.

Network isolation detection for an active NEIAn active NEI does not deactivate itself unless it detects that its mate isalso active. Thus, an active NEI remains active when it detects itself tobe network isolated.

Geographic Survivability limitations and restrictionsThe following limitations and restrictions apply to the GeographicSurvivability feature:

• The physical distance between active and standby sites is limited to120 KM/75 miles.

• Each site must have a single Ethernet Routing Switch 8600 with dualswitch fabric and CPU blades or third-party equivalent.

• Synchronization of data between two CCA blades over GigE linksapplies to the Communication Server 2100 market with reducedcapabilities. This configuration applies just to Enterprise solutions.

• In Enterprise hybrid configurations (with TDM equipment homed atone site), the TDM equipment is not geographically redundant. Indetermining the master site, preference is given to the TDM side onlywhen either side can support Callp. If necessary, perform a SWACT tothe TDM side to allow Callp on that side.

• Because the Call Agent interacts with the Ethernet Routing Switch8600, the feature requires that each site have only one routing switchand IST links configured between sites. Dual Ethernet Routing Switch8600s at each site and SMLT links between sites are not supported.Interactions between the Call Agent and the Communication ServerLAN are supported to prevent split brain scenarios (by disablingOSPF) when the Ethernet Routing Switch 8600s are used for theCommunication Server LAN. Upgrades from previous releases inan Enterprise geographic survivable configuration (which have dualEthernet Routing Switch 8600’s at each site) require that the dualEthernet Routing Switch 8600s be migrated to a single EthernetRouting Switch 8600 site.

• When a total loss of communication between sites occurs (that is, allthree master links are down), the two Call Agents cannot negotiateactivity decision. The decision is based on connectivity check fromeach site the WAN network.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 165: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Communication Server 2000 Management Tools geographic survivability 165

— While unlikely, it could be possible to have an active/active (splitbrain) scenario, or an inactive/inactive scenario (no processing).

— The WAN backup path mitigates the risk of optical ring failure. TheWAN connection check helps resolve activity when the backup pathis down.

• Communication Server 2000 - Compact supports only a single timezone setting. If the two physical sites are in different time zones, Nortelrecommends that the time zone be set to either GMT or the time zoneof one of the sites.

• Both Session Server units of a pair are located at the same site. Foroffices with Message Controllers, Nortel recommends that the SessionServers be located at the same site as the TDM components.

• For maximum redundancy, the WAN backup path must be configuredseparately from the optical network, as follows:

— special vlans configured on Ethernet Routing Switch 8600 forbackup path use only

— vlans route over the WAN network instead of over the optical ring

— vlans are not disabled with OSPF disable

— alarm generated for lack of connectivity

• Gateways and services node components of Communication Server2000 - Compact are single units, and are not geographically redundant.Where the nodes are located and how they are connected to thenetwork affects whether they survive a failure. Although the samenodes are supported in configurations with and without GeographicSurvivability, there is no change in configuration or connection in theconfiguration with the Geographic Survivability configuration.

CAUTIONLine data integrity audit impacts the maintenance actions suchas GWC status and GWC swact. At times, the bearer networkcannot be retrieved from the core due to the bearer networktimeout, as it is design intent.

Communication Server 2000 Management Tools geographicsurvivability

The following procedures contain information for OA&M auto backup andaccelerated restore capability:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 166: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

166 Fault management fundamentals

Understanding Link StatesThe information displayed in the l3-link-status, activation-state,remote-block-status, local-inhibit-state, remote-inhibit-state,congestion-level, discard-level and level-2-status report information onyour links. The system updates all SS7 states once per second.

l3-link-statusThe l3-link-status box shows the availability of the link: available orunavailable

activation-stateThe activation-state box shows you if the link is active. Possiblestates are inactive, act-restoring, active, failed, suspended-t17,suspended-card-out-of-service, and initializing.

remote-inhibit-statusThe remote-inhibit-status refers to links that are inhibited by the far-endnode of this link. The remote-inhibit-status box displays the remote inhibitstatus: remote-inhibit or uninhibited.

remote-block-stateThe local-block-state refers to links that are inhibited locally. Thelocal-block-state box displays the local inhibit state: remote-blocked orunblocked.

local-inhibit-stateThe local-inhibit-state refers to links that are inhibited by the near-endnode of this link. The local-inhibit-state box displays the local inhibit state:local-inhibit or uninhibited.

discard-levelThe discard-level ranges from 0 (lowest) through 3 (highest). All messagesare assigned a discard priority level. Any messages with a priority levelless than the currently displayed discard level are discarded.

congestion-levelMeasurement of link congestion differs, depending on the protocol usedby your system. If your system identity is ANSI-based, link congestion ismeasured in four levels: 0 (lowest) through 3 (highest). If your systemidentity is ITU 14-bit based, link congestion is measured in two levels: 0(lowest) and 1 (highest).

The congestion levels provide a way for the USP to manage messagesduring times of elevated congestion. Each SS7 message is assigned acongestion priority level. Messages with high priority levels are more likelyto be sent, even when congestion is high.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 167: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Communication Server 2000 Management Tools fault management overview 167

Any messages with a priority level lower than the currently displayedcongestion level result in the following actions:

• The system generates a signaling network management (SNM)transfer control message (TFC) to notify the senders of the messagesin the network of the congestion status.

• The system checks the discard level.

If the congestion level continues to remain above 0 for an extended periodof time, you may need to add a link.

level-2-statusThe level 2 status for a link appears in the level-2-status box.Possible states are idle, in-service, out-of-service, initial-alignment,aligned-not-ready, aligned-ready, processor-outage, not-aligned, proving,aligned, monitoring, local-processor-outage, remote-processor-outage,both-processor-outage, l2-congestion and unknown.

Communication Server 2000 Management Tools fault managementoverview

The Server Platform Foundation Software (SPFS) on the CommunicationServer 2000 Management Tools server consists of several commoncomponents, including a fault management sub-system. The networkelements forward their alarms to the fault management sub-system usingstandard simple network management protocol (SNMP) traps. The alarmsub-system consolidates the alarms from the network elements andprovides the user with the ability to monitor both active and historicalalarms.

The SPFS platform also has a fault detection and reporting mechanism fora power supply failure, disk failure, file system over usage, memory overusage, CPU load over usage, swap space over usage, fan failure, networkconnectivity failure, and high temperature. The SPFS fault detection andreporting mechanism is integrated with the alarm utility, which has thefollowing capabilities:

• keeps track of alarms on the platform

• lights a light when an alarm is raised and extinguishes the light whenthe alarm is cleared (if no other alarms are present)

• writes a customer log that corresponds to the state of an alarm

• stores information about all the alarms on the system in its database,which can be queried by a user at any time

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 168: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

168 Fault management fundamentals

User interfaceA user can perform fault management using one of the followinginterfaces:

• A user can view and filter NE alarms through the CommunicationServer 2000 Management Tools GUI.

• A user can use the CDMA Network Manager (CNM) IntegratedElement Management System (IEMS) GUI, which provides aconsolidated historical and a real-time view of the events thathave occurred in a Communication Server 2000 central office. Fordetails about Fault Management of IEMSCNM, see Nortel IEMSFundamentals (NN10329-111).

Communication Server 2000 Management Tools GUIThe Communication Server 2000 Management Tools GUI provides bothan Alarm Manager and an Alarm History window for viewing and filteringalarms. Active alarms are typically viewed with the Alarm Manager, whilethe Alarm History window provides both active and inactive alarms. Bothprovide information on alarms in a tabular format. There are methods forfiltering alarms to show only alarms for a particular network element, of aparticular severity or alarm category.

Alarm severity color codesBased on alarm severity, each alarm has an associated color code asshown in the following figure.

• Critical and major - red

• Minor - amber

• Warning - yellow

Figure 49Alarm Severity Color Codes

Alarm totalsThe Alarm Manager provides an alarm summary of the total number ofactive alarms, as well as the specific number of alarms for each severity(for example, Critical, Major, Minor, and Warning). This summary islocated in the lower left-hand corner of the alarm manager below the"Raw Alarm List". The Alarm History browser provides the total number

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 169: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

User interface 169

of alarms, but does not show the number of alarms for any severity. Inaddition, both the Alarm Manager and the Alarm History windows includethe time the display was last updated.

Figure 50Raw Alarm List

Syslog traplogger interfaceThe SESM server application on the Communication Server 2000Management Tools server uses UNIX syslog to store SNMP traps thatarrive from network elements into the Communication Server 2000Management Tools server. All traps managed by SESM are stored in palog files in the /opt/nortel/NTsesm/admin/logs directory. The number of palog files is determined manually using the CLI and the SESM logging anddebug log configuration options available from the configuration script.

The syslog traplogger interface is automatically set up by scripts and isstarted when the SESM server application is started or restarted. Thereare currently three configuration parameters for the syslog traploggerinterface:

• the maximum file size

• the hour when file rotation backup occurs

• the minute when file rotation backup occurs

The preceding parameters can be configured using the "configure" toolon the Communication Server 2000 Management Tools server. For moreinformation, see “Configuring SESM log reporting” (page 529).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 170: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

170 Fault management fundamentals

SESM server application debug logsThese are debug log files residing on the Communication Server 2000Management Tools server that are produced by the SESM serverapplication. Optionally, the maximum size, maximum number of files,default debug levels, log file name, and other options can be configuredusing the "configure" tool on the Communication Server 2000 ManagementTools server. For more information, see “Configuring SESM log reporting”(page 529).

There are three debug log file name extensions: mi2, misc, and pa.The "mi2" log files (for example, ptmdebuglog1.mi2) contains logs fromthe SESM server application. This is typically the largest and mostsignificant log file. The "pa" logs files (for example, ptmdebuglog1.pa) areproduced by the Proxy Agent which is a different application that is alsopart of the SESM server application. The "misc" log files (for example,ptmdebuglog1.misc) are produced by miscellaneous parts of the SESMserver application. The "misc" file could contain critical errors such asstartup errors in the SESM server application processes or OutOfMemoryerrors.

The logs are located in the /opt/nortel/NTsesm/admin/logs directory andshould not be erased. They rotate when necessary, such that the file sizeor number of log files remains manageable. Display the log files usingthe UNIX command "ls -alt" in this directory. The newest logs will have a"1" in the title (assuming a default rotation) such as ptmdebuglog1.mi2.Older files that have been rotated may be named ptmdebuglog2.mi2 orptmdebuglog3.mi2. The oldest file, for example ptmdebulog7.mi2, isdeleted during the next rotation.

The /opt/nortel/NTsesm/admin/logs directory contains thecreateDB_<date>.log file created during the installation of the SESMserver application. This file contains debug information from the initialcreation of the SESM server application data base schema. This log filedoes not persist across upgrades of the SESM software.

Communication Server 2000 Management Tools logsThe default northbound alarm feed for the SESM-based applications onthe Communication Server 2000 Management Tools server is Corbanotifications to the Integrated Element Management System (IEMS) CDMANetwork Manager (CNM).

The SESM syslog alarm feed is disabled by default to avoid duplication ofthe alarm logs. The “Configuring SESM log reporting” (page 529) in thisdocument, is available to enable or disable the SESM syslog alarm feedwhen required.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 171: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

User interface 171

The Communication Server 2000 SAM21 Manager, QoS CollectorApplication (QCA), and Network Patch Manager (NPM) applications on theCommunication Server 2000 Management Tools server, use the SPFSlogging Application Program Interface (API) to record their logs. This APIalso provides the customer log feed to the OSS Fault Collector. The APIsupports five severity levels of custlogs: critical, major, minor, warning,and none.

The SPFS logging API includes the following types of log files located in"/var/log":

• customerlog (alarms and other customer visible elements)

• auditlog (user actions taken)

• authlog (authentication events)

• securitylog (security events)

• debuglog (software or hardware errors)

• SPFSlog (platform events)

• ptmlog (SNMP traps received by the system)

Syslog has built-in capabilities to forward logs to another IP address. Thisfunctionality is used to send the logs in the SPFS custlog feed to the OSSFault Collector.

The logs in "/var/log"rotate when necessary, such that the file size ornumber of log files remains manageable. After rotation, the following logfiles are compressed by the gzip compression program: customerlog,auditlog, securitylog, debuglog, and SPFSlog. Log files fs.log and lv.log in"/var/log/filesys", are also compressed.

Syslog forwarding is configured using the Syslog Configuration optionat the SPFS command line interface (CLI). This option modifies thesyslog.conf text file (a standard syslog file) which dictates how syslogoperates. It can define whether to record the logs on the local machine,send them to another machine (such as the OSS Fault Collector), or both.Syslogs are controlled on a log-level basis, such that critical logs can behandled one way and warning logs another way.

In general, the only logs that will be forwarded are customer logs whichcontain alarm logs. Although, this can be customized using SyslogConfiguration.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 172: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

172 Fault management fundamentals

Perform an auditUse this procedure to manually perform a trunk or Communication Server2000 (CS2K) data audit.Use this procedure to manually perform a line,trunk, V5.2 interface, or Communication Server 2000 (CS2K) data audit.

The V5.2 audit is only available in the international version of the softwareand not in the North American.

You can set the audit to run automatically at a specific time on a daily or aweekly basis. For more information about configuring an audit schedule,see (NN10409-500).

You can set the audit to run automatically at a specific time on a daily or aweekly basis. For more information about configuring an audit schedule,see Packet MSC Configuration (NN20000-213).

Media gateway endpoint namesThis section contains information on the format of the media endpointgateway name used in the OUT command that deletes certain lines.

This information is not relevant to SIP lines, and is not relevant to lineson CICM gateways. For those types of lines, you do not use the mediaendpoint gateway name in the OUT command; instead, you use the lineequipment number (LEN) to identify the line.

Names for media gateway endpoints on cable MTA gatewaysThe format for media gateway endpoint names is as follows:

<media-gateway-name> <endpoint-name>

where

• <media-gateway-name> is in the form of a fully qualified domain nameincluding the hostname of the device and suitable for lookup usingDirectory Name Service (DNS). The name must contain a period (.).

Examplecust34671.rdu.attcable.net

ATTENTIONIf a gateway is associated with a GWC for which a default gateway domainname has been provisioned, do not use the default gateway domain name aspart of the <media-gateway-name> in any SERVORD+ command. SERVORD+commands are the subset of SERVORD commands that can be entered usingthe OSSGate interface.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 173: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Media gateway endpoint names 173

• <endpoint-name> has the following format: aaln/<n> where n is aninteger in the range 1 to n, where n is the number of the voice port onthe MTA.

ATTENTIONIn the format, there is a space between <media-gateway-name> and<endpoint-name>.

Names for media gateway endpoints on MG 9000 H.248 gatewaysThe format for media gateway endpoint names is as follows:

<media-gateway-name> <endpoint-name>

where

• <media-gateway-name> has the following format:<site><frame>-<logical-frame>-<shelf>

where

— <site> is the value previously datafilled in table SITE. The value isone to four alphanumeric characters.

— <frame> is a three-digit integer in the range 000 to 511, indicatingthe frame number within the office.

— <logical-frame> is an integer in the range 0 to 7, indicating thelogical frame number within the physical MG.

— <shelf> is an integer in the range 0 to 3 indicating the shelf numberwithin the frame.

• <endpoint-name> has the following format: tp/<card>/<circuit>

where

— <card> is a two-digit integer in the range 02 to 09 or in the range14 to 21, indicating the card number.

— <circuit> is a two-digit integer in the range 00 to 31, indicating thecircuit number.

ATTENTIONIn the format, there is a space between <media-gateway-name> and<endpoint-name>.

For example, the following name conforms to the naming format for mediagateway endpoints on an MG 9000 H.248 gateway:

ABCD511-7-3 tp/21/31

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 174: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

174 Fault management fundamentals

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 175: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

175.

Fault management tasksThis section lists the tasks associated with monitoring and analyzing faultdata across the Carrier Voice over IP (CVoIP) solutions. Tasks associatedwith alarm clearing and interpretation, as well as network restoration, arealso listed here.

Navigation• “View and search log files” (page 177)

• “Communication Server 2000 Management Tools software downgrade” (page 185)

• “Breaker module replacement ” (page 221)

• “Maintaining Communication Server 2000 GWC alarms and logs ”(page 231)

• “Deleting a line ” (page 305)

• “Maintaining GWC alarms and logs ” (page 313)

• “I/O module replacement in an ERS 8600 in a carrier CS-LAN ” (page343)

• “Powering up the XA-Core” (page 437)

• “Powering up the network” (page 459)

• “Server shut down and backup” (page 515)

• “Communication Server 2000 Management Tools fault management ”(page 527)

• “Replacing a failed SPFS-based server” (page 632)

• “CORBA configuration issue correction between the CommunicationServer 2000 server and the MG 9000 Manager server” (page 651)

• “Communication Server 2000 troubleshooting” (page 673)

• “GWC troubleshooting ” (page 689)

• “Trunk testing” (page 697)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 176: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

176 Fault management tasks

• “Communication Server 2000 management tools troubleshooting”(page 717)

• “Communication Server 2000 Management Tools geographicsurvivability reference” (page 751)

• “Preventative maintenance for CVoIP components” (page 757)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 177: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

177.

View and search log filesThe log viewer utility provides a way to view and search log files. It has thefollowing three components:

• CopyLogs

• LogExtractor

• LogBrowser

View and search log files navigation

• “Running the CopyLogs log viewer utility ” (page 178)

• “Running the LogExtractor log viewer utility” (page 180)

• “Running the LogBrowser log viewer utility” (page 182)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 178: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

178 View and search log files

Running the CopyLogs log viewer utility

The log viewer utility provides a way to view and search log files. It is splitinto three components:

• CopyLogs. The CopyLogs utility captures log files for later analysis. Itgrabs a set of predefined logs from the IEMS system and compressesthem using tar and bzip. There is an option to specify the age of thelogs to be copied.

• LogExtractor. The LogExtractor takes the compressed snapshotfile created by the CopyLogs utility and extracts them in a specifieddirectory. For full details of the LogExtractor, refer to the procedure“Running the LogExtractor log viewer utility” (page 180).

• LogBrowser. The LogBrowser utility offers a way of displaying andsearching the uncompressed log text files. The files can be filtered byspecifying a date/time range. For full details of the LogBrowser, refer tothe procedure “Running the LogBrowser log viewer utility” (page 182).

You invoke CopyLogs through the CLUI. The tool does not require anyparameters and, by default, it collects all supported logs. You can specifyoptional parameters to collect only specific files which reduces the memoryusage of the tool and the time it takes to run.

Usage:

copylogs.ksh [-a <days>] [-conf [<logconf> ... <logconf>]

where

-a: Is the option to specify the file aging days. <days> is the number ofdays. The default is 0 (no aging).

-conf: Is the option to collect other logs specified in the logconf file under/opt/sspfs/logconf. If no <logconf> is specified, the tool collects all configfiles under /opt/sspfs/logconf.

PrerequisitesThis procedure requires you to know the root userID and password.

Procedure stepsATTENTIONThe CopyLogs utility can use a large amount of system resources. Run theutility cautiously if the system is low on resources.

Perform the following steps to complete this procedure.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 179: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 179

Step Action

1 At your workstation establish a connection to the applicationclient box through telnet or SSH and log in using the root userID and password. For more information about logging in to anSPFS-based server, see Nortel ATM/IP Solution-level Securityand Administration (NN10402-600).

2 Access the command line interface by typing:

# cli

and pressing the Enter key.

3 At the Command Line Interface menu, enter the number next to"Other" and press the Enter key.

4 At the Other menu, enter the number next to "copylogs (CaptureVarious Logs For Debugging Purposes)", and press the Enterkey.

5 When prompted to enter the aging option, enter the number ofprevious days to be included in the capture, or enter 0 to captureall files, and press the Enter key.

Select which config file to use for CopyLogs:1) all.conf2) capt_files

Config File?

6 Enter the number next to the type of logs you wish to capture,and press the Enter key. The utility copies the log files andstores the resulting compressed file.

To extract the logs, refer to procedure “Running the LogExtractorlog viewer utility” (page 180).

To browse previously extracted logs, refer to procedure “Runningthe LogBrowser log viewer utility” (page 182).

7 Exit each menu level of the command line interface to eventuallyexit the command line interface, by typing

select - x

and pressing the Enter key.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 180: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

180 View and search log files

Running the LogExtractor log viewer utility

The log viewer utility provides a way to view and search log files. It is splitinto three components:

• CopyLogs. The CopyLogs utility captures log files for later analysis. Itgrabs a set of predefined logs from the IEMS system and compressesthem using tar and bzip. There is an option to specify the age of thelogs to be copied. For full details of CopyLogs, refer to the procedure“Running the CopyLogs log viewer utility ” (page 178).

• LogExtractor. The LogExtractor takes the compressed snapshotfile created by the CopyLogs utility and extracts them in a specifieddirectory.

• LogBrowser. The LogBrowser utility offers a way of displaying andsearching the uncompressed log text files. The files can be filtered byspecifying a date/time range. For full details of the LogBrowser, refer tothe procedure “Running the LogBrowser log viewer utility” (page 182).

Usage:

LogExtractor [-c <compressedFile>]

Where

-c is the full path of the compressed file.

PrerequisitesThis procedure requires you to know the root userID and password.

Procedure stepsATTENTIONThe LogExtractor utility can use a large amount of system resources. Run theutility cautiously if the system is low on resources.

Perform the following steps to complete this procedure.

Step Action

1 At your workstation, establish a connection to the applicationclient box through telnet or SSH and log in using the root userID and password. For more information about logging in to anSPFS-based server, see Nortel ATM/IP Solution-level Securityand Administration (NN10402-600).

2 Access the command line interface by typing:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 181: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 181

# cli

and pressing the Enter key.

3 At the Command Line Interface menu, enter the number next to"Other" and press the Enter key.

4 At the Other menu, enter the number next to "logextractor(Extracting compressed log files captured by copylogs)", andpress the Enter key.

Example response

=== Executing "logextractor"Which compressed file to extract:

1: all.tar.bz2

5 At the prompt, select 1, and press the Enter key.

The system extracts the logs and stores in the /var/log/logviewer/data/ directory.

To browse the extracted logs, refer to the procedure “Runningthe LogBrowser log viewer utility” (page 182).

6 Exit each menu level of the command line interface to eventuallyexit the command line interface, by typing

select - x

and pressing the Enter key.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 182: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

182 View and search log files

Running the LogBrowser log viewer utility

The log viewer utility provides a way to view and search log files, and issplit into three components:

• CopyLogs. The CopyLogs utility captures log files for later analysis. Itgrabs a set of predefined logs from the IEMS system and compressesthem using tar and bzip. There is an option to specify the age of thelogs to be copied. For full details of CopyLogs, refer to the procedure“Running the CopyLogs log viewer utility ” (page 178).

• LogExtractor. The LogExtractor takes the compressed snapshotfile created by the CopyLogs utility and extracts them in a specifieddirectory. For full details of the LogExtractor, refer to the procedure“Running the LogExtractor log viewer utility” (page 180).

• LogBrowser. The LogBrowser utility offers a way of displaying andsearching the uncompressed log text files. The files can be filtered byspecifying a date/time range.

Usage:

logbrowser [-conf[<logconf> ... <logconf>]]

Where

-conf is the option to collect other logs specified in the logconf file under/opt/sspfs/logconf. If no <logconf> is specified, the system uses all configfiles under /opt/sspfs/logconf.

PrerequisitesThis procedure requires you to know the root userID and password.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation establish a connection to the applicationclient box through telnet or SSH and log in using the root userID and password. For more information about logging in to anSPFS-based server, see Nortel ATM/IP Solution-level Securityand Administration (NN10402-600).

2 Access the command line interface by typing:

# cli

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 183: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 183

and pressing the Enter key.

3 At the Command Line Interface menu, enter the number next to"Other" and press the Enter key.

4 At the Other menu, enter the number next to "logbrowser(Browse logs on the system)", and press the Enter key.

Example response

=== Executing "logbrowser"

Where do you want to browse the logs:1: System.

2: Extracted Logs.

5 Select whether you wish to browse logs that have previouslybeen extracted, or are currently on the system. Select 2 tobrowse previously extracted logs.

6 Please choose a set of logs to capture:

1: all.ext

7 Select 1 and enter the time and date of the logs as follows:

Starting time to collect log (dd/mm/yyyy mm:hh:ss or press Enterfor none):

Ending time to collect log (dd/mm/yyy mm:hh:ss or press Enterfor none):

The system responds as follows:

Processing Config file /var/log/logviewer/ext/all.ext ...

8 Exit each menu level of the command line interface to eventuallyexit the command line interface, by typing

select - x

and pressing the Enter key.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 184: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

184 View and search log files

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 185: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

185.

Communication Server 2000Management Tools softwaredowngrade

Downgrade the software on the Communication Server 2000 managementtools when you want to rollback the running software to a previous version.

Navigation

• “Cloning the image of one node in a cluster to the other node” (page186)

• “Migrating the SAM21 network elements back to the CS 2000 SAM21Manager on the core manager” (page 193)

• “Launching the Communication Server 2000 SAM21 Manager Clientthat resides on the Communication Server 2000 Management ToolsServer” (page 197)

• “Rolling back the Shelf Controller software” (page 199)

• “Performing a rollback of the CS 2000 SAM21 Manager” (page 202)

• “Performing a full system restore on an SPFS-based server” (page204)

• “Restoring the Oracle/MySQL data on an SPFS-based server” (page209)

• “Clearing the JWS Cache on a Client Workstation” (page 215)

• “Routing log streams from an SPFS-based server to a remote host”(page 218)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 186: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

186 Communication Server 2000 Management Tools software downgrade

Cloning the image of one node in a cluster to theother node

Use this procedure to clone the image of the active node in a cluster tothe inactive node.

Prerequisites• You need the root user ID and password.

• It is recommended that you have console access before starting thisprocedure. Console access is required if the Ethernet address of theinactive unit has changed or this is the first time this procedure isexecuted. For more information, see procedure "Configuring SPFSconsole access" in Nortel Carrier Voice over IP Network Upgrades andPatches, NN10440-450.

ATTENTIONEnsure that no provisioning activities are in progress, or are scheduled to takeplace during this procedure.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, establish a login session to the Active nodeas a root user. See procedure "Logging in to an SPFS-basedserver".

2 Verify the status of replicated disk volumes by typing

# udstat

If all file systems are Then

ACTIVE the next step

not ACTIVE contact your next level of support

3 Verify that there are no major or critical alarms on the system byentering the following command on the active unit:

queryflt

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 187: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 187

If Then

there is no major (M) or critical(C) alarm

go to the next step

there are major (M) or critical (C)alarms

refer to Carrier Voice overIP Fault Management LogsReference (NN10275-909)

4 Verify that all applications on the server are running by typing

# servquery -status all

The expected status under normal operation are: running, offline.

Example response:

APP NAME STATUS================================DATABASE...............RUNNINGCINOTIFIER.............OFFLINEBACKUP_MANAGER.........RUNNINGBOOTP..................RUNNINGWEBSERVER..............RUNNINGCORBA..................RUNNINGOMPUSH.................RUNNINGSESMService............RUNNINGWEBSERVICES............RUNNINGORA_AUTO_BACKUP........RUNNINGDDMSPROXY..............RUNNINGORA_ARCHIVE_ROTATOR....RUNNINGNPM....................RUNNINGPROP_SRV...............RUNNINGSAM21EM................RUNNINGSNMP_POLLER............OFFLINEQCA....................RUNNING

5 Use the following table to determine your next step.

If Then

the status of the applications arerunning or offline

go to step 8

one or more applications are notrunning

go to the next step

6 Start each application that is not running by typing

# servstart <app_name>

7 Use the following table to determine your next step.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 188: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

188 Communication Server 2000 Management Tools software downgrade

If Then

one or more applications do notstart

contact your next level of support

the status of the applications arerunning or offline

go to the next step

8 Verify the Patching Server Element (PSE) server application isrunning by typing

# pse status

If PSE is Then

running go to step 10

not running go to the next step

9 Start the PSE server application by typing

# pse start

If PSE Then

does not start contact your next level of support

starts the next step

10 Select your next step

If this server is Then

running the CS 2000Management Tools software

go to the next step

not running the CS 2000Management Tools software

step 13

11 Verify that the SESMservice application is fully functional bytyping

# ptmctl status

Example response:

SESM STATUS-------------------------COMPONENT STATUS--------- ------Proxy Agent RUNNINGRMI Registry RUNNINGSnmpfactory RUNNINGMI2 Server RUNNINGCurrent number of SESM processes running: 4 (of 4)

SESM APPLICATION STATUS: All Applications ready

12 Select your next step

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 189: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 189

If the SESMService is Then

not fully functional contact your next level of support

fully functional the next step

13 At your workstation (login session to the Active node), start thecloning process. Select your next step.

If Then

the inactive unit has a differentEthernet address than one givenat the last clone attempt or thisis the first time a clone has beenperformed on the active unit

To obtain the value of theEthernet address continue to thestep 15.

else enter the following command:# startb

and go to the next step.

14 Select your next step.

If Then

the system prompts you for theEthernet address

go to the next step

the system prompts you to entercommand "boot net - image"

go to step 19

the system does not prompt youfor the Ethernet address or "bootnet - image"

go to step 23

15 At the console connected to the inactive node, determine theEthernet address of the Inactive node. Log in to the inactivenode through the console (port A) using the root user IDand password. For more information about logging in to anSPFS-based server, see Nortel ATM/IP Solution-level Securityand Administration (NN10402-600).

Ensure you are on the Inactive server by typing ubmstat.If ClusterIndicatorACT is displayed in the response, whichindicates you are on the Active server, log out of that serverand log in to the other server. The response must displayClusterIndicatorSTBY, which indicates you are on the Inactiveserver.

16 Bring the system to the OK prompt by typing

# init 0

17 At the OK prompt, display the Ethernet address of the inactivenode by typing

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 190: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

190 Communication Server 2000 Management Tools software downgrade

OK banner

Example response:

Netra 240, No keyboardCopyright 1998-2004 Sun Microsystems, Inc. All rightsreserved.OpenBoot 4.16.2, 4096 MB memory installed, Serial#65143495.Ethernet address 0:3:ba:e2:2:c7, Host ID: 83e202c7.

Ethernet address formats can vary from machine to machine.The fields in an Ethernet address may contain one alphanumericcharacter or two alphanumeric characters, both are acceptableformats.

18 Record the Ethernet address that is displayed. On the activeunit, enter the following command::

# startb <Ethernet address>

Go to step 22.

19 Login to the active unit as root and enter the following command:

# startb <Ethernet address>.

20 Log in to the inactive node through the console (port A) using theroot user ID and password. For more information about loggingin to an SPFS-based server, see Nortel ATM/IP Solution-levelSecurity and Administration (NN10402-600).

Ensure you are on the Inactive server by typing ubmstat.If ClusterIndicatorACT is displayed in the response, whichindicates you are on the Active server, log out of that serverand log in to the other server. The response must displayClusterIndicatorSTBY, which indicates you are on the Inactiveserver.

21 Select your next step.

At the console connectedto the inactive node, if theconsole displays

Then go to

the OK prompt step 22

the login prompt the next step

22 Bring the system to the OK prompt by typing

# init 0

23 When the active system prompts you to enter the command"boot net - image", boot the inactive node from the image of theactive node by typing

OK boot net - image

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 191: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 191

There must be a space between the "-" and "image."

Example response:

SC Alert: Host System has ResetSC Alert: CRITICAL ALARM is setProbing system devicesProbing memoryProbing I/O busesNetra 240, No KeyboardCopyright 1998-2004 Sun Microsystems, Inc. All rightsreserved.OpenBoot 4.16.2, 4096 MB memory installed, Serial#65143495.Ethernet address 0:3:ba:28:2b:23, Host ID: 83282b23.

Rebooting with command: boot net - imageBoot device: /pci@1f,700000/network@2 File and args: -image

24 At your workstation (login session to the Active node), monitorthe progress of the cloning from the active node. Cloning theinactive node takes approximately 1 hour to complete.

Example response:

Waiting for network response from unit1-priv0...Feb 6 13:19:32 wcars2yb unix: NOTICE: bge0: link downFeb 6 13:19:34 wcars2yb unix: uplink1: Standby linkfailure - not receiving heartbeats (A)Feb 6 13:19:35 wcars2yb unix: NOTICE: bge0: link up1000Mbps Full-Duplex

INFO: Waiting for response from unit1-priv0.Feb 6 13:20:33 wcars2yb unix: NOTICE: bge2: link downFeb 6 13:20:46 wcars2yb unix: NOTICE: bge2: link up1000Mbps Full-DuplexFeb 6 13:21:06 wcars2yb unix: uplink1: Standby linkfailure - not receiving heartbeats (B)INFO: Received network response from unit1-priv0.INFO: Waiting for unit1-priv0 to complete clone.INFO: Waiting. Current count: 1INFO: Waiting. Current count: 2INFO: unit1-priv0 is cloning: /export/d2INFO: unit1-priv0 is cloning: /export/d2INFO: unit1-priv0 is cloning: /export/d2

Clone successful

25 If you are using SPFS10 or earlier, at your workstation (loginsession to the Active node), verify the status of replicated diskvolumes by typing

# udstat

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 192: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

192 Communication Server 2000 Management Tools software downgrade

If all file systems are not ACTIVE, contact your next level ofsupport.

If you are using SPFS11, verify that the clone completed withthe message "Clone successful". If not, contact your next levelof support.

--End--

Variable definitionsUse the information in the following table to determine the value for thevariables used in this procedure.

Variable Value

<app_name> is the name of the application that isnot in a RUNNING state; for example,SAM21EM

<Ethernet address> is the MAC address of inactive unit asper the banner output

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 193: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 193

Migrating the SAM21 network elements back to theCS 2000 SAM21 Manager on the core manager

Use this procedure to migrate the SAM21network element back to the CS2000 SAM21 Manager on the core manager.

PrerequisitesNone

Procedure stepsLaunch the CS 2000 SAM21 Manager client that resides on the core manager

Step Action

1 At the client workstation, log on the client workstation using thecorrect user ID and password. Do not log on as the root user.

2 Launch the SAM21 Manager client application that resides onthe core manager by typing

# /sdm/bin/sam21gui

and pressing the Enter key.

The user authentication window appears.Figure 51User authentication window

3 Enter a valid user name and password, and click the "Login"button.

4 Reprovision each SAM21 network element (NE) by modifying aSAM21 NE from the configuration menu.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 194: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

194 Communication Server 2000 Management Tools software downgrade

The Reprovision window for the SAM21NE you selected, opens.

5 Replace the existing IP address with the IP address of the coremanager, and click Save.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 195: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 195

An SNMP message is sent to both SAM21 NEs with the IPand port of the core manager. Within approximately 1 minute,the SAM21 NE recovers from isolation on the CS 2000 SAM21Manager that resides on the core manager. Within approximately2 minutes, the SAM21 NE is isolated on the CS 2000 SAM21Manager that resides on the CS 2000 Management Tools server.

--End--

Launch the CS 2000 SAM21 Manager client that resides on the CS 2000Management Tools server

Step Action

1 At your workstation, launch the SAM21 Manager clientapplication that resides on the CS 2000 Management Toolsserver. For more information, see ATM/IP Solution-levelAdministration and Security (NN10402-600).

2 Reprovision each SAM21 network element (NE) as follows:

• Use the Configuration menu to modify a SAM21 NE.

The Reprovision window for the SAM21NE you selected,opens.

• Replace the existing IP address with the IP address of thecore manager, and click Save.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 196: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

196 Communication Server 2000 Management Tools software downgrade

All non-isolated shelves are communicating with the CS 2000SAM21 Manager on the core manager.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 197: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 197

Launching the Communication Server 2000 SAM21Manager Client that resides on the CommunicationServer 2000 Management Tools Server

Use this procedure to launch the Communication Server 2000 SAM21Manager client that resides on the Communication Server 2000Management Tools Server.

Procedure steps

Step Action

1 At your workstation

Launch the SAM21 Manager client application that resides onthe Communication Server 2000 Management Tools server.

2 Reprovision each SAM21 network element (NE).

3 Use the Configuration menu to modify a SAM21 NE.

The Reprovision window for the SAM21NE you selected,appears.

4 Replace the existing IP address with the IP address of the coremanager, and click Save.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 198: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

198 Communication Server 2000 Management Tools software downgrade

All non isolated shelves are communicating with theCommunication Server 2000 SAM21 Manager on the coremanager.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 199: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 199

Rolling back the Shelf Controller software

Use this procedure to roll back the shelf controller software to a previousversion.

ATTENTIONIf the office supports Communication Server 2000-Compact, roll back the CallAgent software before rolling back the SAM21 Shelf Controller software. If youdo not roll back the Call Agent software first, a reset loop may occur on the nextunlock or RExTst of the Call Agent.

This procedure enables you to restore the previous Shelf Controllersoftware load following an unsuccessful upgrade.

The following figure summarizes the upgrade procedure. Rollback isavailable until the second SAM21 Shelf Controller is unlocked andupgraded to the new software load. This point is indicated with the star.

Use this procedure if problems occurred during the upgrade (automated ormanual), and the upgrade was terminated.

ATTENTIONRollback is available until the second SAM21 Shelf Controller is unlocked andupgraded to the new software load. This point is indicated with the star.

Procedure steps

Step Action

1 At the Communication Server 2000 SAM21 Manager client(Java Web Start client)

From the Subnet View, select Configuration, Modify, ExistingSAM21 Network Element and the SAM21 shelf with the SAM21Shelf Controller software you want to rollback.

2 In the Reprovision window, enter the name of the old softwareload in the Server Load field.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 200: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

200 Communication Server 2000 Management Tools software downgrade

If you upgraded from 9.0.4.0311070959 to 10.0.0.0301120523,enter 9.0.4.0311070959 as the name of the old software load.

3 Click Save.

4 Open the Shelf View of the SAM21 Shelf with the SAM21 ShelfControllers that need a software rollback.

5 Double-click the card icon of a SAM21 Shelf Controller thatneeds a software rollback.

6 In the Card View window, click the States tab and check theStatus of the SAM21 Shelf Controller.

7 If the SAM21 Shelf Controller is active go to Step 10.

8 If the SAM21 Shelf Controller is inactive go to Step 15.

9 Close the Card View window.

10 Right-click on the card icon of the active SAM21 Shelf Controllerand select Swact from the menu.

11 Wait for completion of the switch of activity.

12 Double-click the card icon of the SAM21 Shelf Controller.

13 In the Card View window, click the States tab and make sure theStatus is Inactive.

14 Close the Card View window.

15 Right click on the card icon of inactive SAM21 Shelf Controllerand select Lock from the menu.

16 Wait for the lock icon to appear on the inactive SAM21 ShelfController.

17 Right-click on the card icon of the inactive SAM21 ShelfController and select Unlock from the menu.

18 Wait for the hashed outline to disappear from the inactiveSAM21 Shelf Controller.

19 Right-click on the card icon of the active SAM21 Shelf Controllerand select Swact from the menu.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 201: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 201

20 Wait for completion of the switch of activity.

21 If you need to rollback software on the other SAM21 ShelfController go to Step 15.

22 If you do not need to rollback software on the other SAM21 ShelfController go to Step 23.

23 Return to the higher level task flow or procedure that directedyou to this procedure.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 202: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

202 Communication Server 2000 Management Tools software downgrade

Performing a rollback of the CS 2000 SAM21Manager

Use this procedure to rollback the CS 2000 SAM21 Manager from the CS2000 Management Tools server to the core manager.

ATTENTIONThis procedure deletes all provisioned data for the CS 2000 SAM21 Managerfrom the database on the CS 2000 Management Tools server.

PrerequisitesNone

Procedure steps

Step Action

1 At the CS 2000 Management Tools server, telnet to the CS 2000Management Tools server by typing

> telnet <server>

and pressing the Enter key.

where

server is the IP address or hostname of the CS 2000Management Tools server

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su - root

and pressing the Enter key.

4 When prompted, enter the root password.

5 Stop the SAM21 Manager server application by typing

# servstop SAM21EM

and pressing the Enter key.

6 Change directory by typing

# cd /opt/nortel/sam21em/bin/migration

and pressing the Enter key.

7 Run the SAM21 Manager rollback script by typing

# ./sam21emRollback.sh

and pressing the Enter key.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 203: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 203

Example response:

SAM21 Element Manager Server Persistent Data Migration Rollback==============================================This script should only be executed to rollback fromthe SPFS SAM21 EM to the CS2E SAM21 EM.WARNING: This script deletes all provisioned data from the SAM21 EM database.

Do you wish to proceed?Please confirm ("YES", "Y", "NO", "N")

8 Confirm that you want to proceed with the rollback by typing

# y

and pressing the Enter key.

Example response:

Retrieving Oracle SAM21 EM Password...

Clearing the SAM21EM Database Tables...SAM21 Element Manager Persistent data migration rollback successful.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 204: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

204 Communication Server 2000 Management Tools software downgrade

Performing a full system restore on an SPFS-basedserver

Use this procedure to perform a full system restore from backup media ona Server Platform Foundation Software (SPFS)-based server.

A full system restore consists of reverting to the previous release of SPFS,restoring the file systems, and restoring the system data.

ATTENTIONAfter performing a full system restore on IEMS you must also perform asubsequent central server restore. Refer to Nortel IEMS Administration andSecurity (NN10336-611).

The server can be hosting one or more of the following components:

• CS 2000 Management Tools

• Integrated Element Management System (IEMS)

• Audio Provisioning Server (APS)

• Media Gateway (MG) 9000 Manager

• CS 2000 SAM21 Manager

• Network Patch Manager

• Core Billing Manager (CBM)

• Multiservice Data Manager (MDM)

ATTENTIONRestoring the oracle data is not applicable to the CBM as it does not use anoracle database.

ATTENTIONSystem logs indicating application and database errors will be generated untilthe file systems and oracle data are restored on the system using this procedureand procedure Restoring the Oracle/MySQL data on an SPFS-based server.No database errors will be generated on the CBM as it does not use an oracledatabase.

PrerequisitesThis procedure has the following prerequisites:

• you need SPFS CD disk #1 or SPFS DVD for the release you arereverting to

• you need the DVD on which you backed up the file systems

• you need the DVD on which you backed up the system data

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 205: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 205

ATTENTIONThe recovery must be performed on the system where the backup was created.Recovery cannot be performed on a different system.

Procedure stepsUse one of the following methods according to your office configuration.

• “Simplex configuration (one server)” (page 205)

• “High-availability configuration (two servers)” (page 206)

ATTENTIONFor MDM and other components that use a simplex configuration with no oracledatabase, complete ONLY steps 1-7. If the additional steps are executed, errorswill occur.

Simplex configuration (one server)

Step Action

1 At the server console, log in to the server through the console(port A) using the root user ID and password. For moreinformation, see Logging in to an SPFS-based server.

2 Bring the system to the OK prompt by typing

# init 0

3 Insert SPFS CD disk#1 or SPFS DVD into the drive.

4 At the OK prompt, restore the system by typing

boot cdrom - restore

5 When prompted, accept the software license restrictions byentering

ok

The system reboots.

If restoring from DVD, you will be prompted to insert Volume1 of the backup DVD into the drive. Insert the DVD on whichyou backed up the file systems. During the restore process, thesystem will prompt you for additional Volumes if more than oneDVD was used during the backup of file systems.

The restore process can take several hours to completedepending on the number and size of the files that are beingrestored.

Although it can appear as if the system is hanging at times, donot interrupt the restore process. If you suspect an issue with therestore process, please contact your next level of support.

6 Eject the backup DVD from the drive as follows:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 206: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

206 Communication Server 2000 Management Tools software downgrade

• Ensure you are at the root directory level by typing# cd /

• Eject the DVD by typing# eject cdrom

If the DVD drive tray will not open after you have determinedthat the DVD drive is not busy and is not being read from orwritten to, enter the following commands:# /etc/init.d/volmgt stop

# /etc/init.d/volmgt start

Then press the eject button located on the front of the DVDdrive.

• Remove the backup DVD from the drive.

7 Determine your next step.

• If the server is hosting a CBM, then do step 9.

• If the server is hosting other applications then go to the nextstep.

8 Restore the oracle data using procedure Restoring theOracle/MySQL data on an SPFS-based server.

9 Reboot the server by typing

# init 6

10 Restore the central security server. For more information,see Restoring the Central Security Server in Nortel IEMSAdministration and Security (NN10336-611) .

--End--

High-availability configuration (two servers)

ATTENTIONThe recovery must be performed on the same unit where the backup wascreated.

Step Action

1 At the console connected to the inactive node, log in to theinactive node through the console (port A) using the root userID and password. For more information, see Logging in to anSPFS-based server.

2 Bring the system to the OK prompt by typing

# init 0

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 207: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 207

3 At the console connected to the active node, log in to theactive node through the console (port A) using the root userID and password. For more information, see Logging in to anSPFS-based server.

4 Bring the system to the OK prompt by typing

# init 0

5 At the console connected to the node where the backup wascreated, insert SPFS CD disk#1 or SPFS DVD into the drive.

6 At the OK prompt, restore the system by typing

boot cdrom - restore

7 When prompted, accept the software license restrictions bytyping

ok

The system reboots.

8 When prompted, insert Volume 1 of the DVD on which youbacked up the file systems into the drive.

During the restore process, the system will prompt you foradditional Volumes if more than one DVD was used during thebackup of file systems.

The restore process can take several hours to completedepending on the number and size of the files that are beingrestored.

Although it can appear as if the system is hanging at times,please do not interrupt the restore process. If you suspect anissue with the restore process, contact your next level of support.

9 To eject the backup DVD from the drive ensure you are at theroot directory level by typing:

# cd /

10 Eject the DVD by typing:

# eject cdrom

If the DVD drive tray will not open after you have determined thatthe DVD drive is not busy and is not being read from or writtento, enter the following commands:

# /etc/init.d/volmgt stop

# /etc/init.d/volmgt start

Then press the eject button located on the front of the DVDdrive.

11 Remove the backup CD or DVD from the drive.

12 Verify the status of replicated disk volumes by entering:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 208: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

208 Communication Server 2000 Management Tools software downgrade

udstat

If all file systems are Then

ACTIVE the next step

not ACTIVE contact your next level of support

13 Determine your next step.

• If the server is hosting a CBM, then do step 16.

• If the server is hosting other applications then go to the nextstep.

14 Restore the data using procedure Restoring the Oracle/MySQLdata on an SPFS-based server.

15 Once the data restore is complete, go to the next step.

16 Reboot the server by typing

# shutdown -i 6 -y

17 Restore the central security server. For more information, seeNortel IEMS Administration and Security (NN10336-611).

18 Reimage the inactive node using the active node’s image.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 209: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 209

Restoring the Oracle/MySQL data on an SPFS-basedserver

Use this procedure to restore the Oracle/MySQL data from a backupDVD on a Server Platform Foundation Software (SPFS)-based server(Sun Netra 240 ). Also, use this procedure to restore the data that wasautomatically backed up to a file on the server by the backup restoremanager.

The server can be hosting one or more of the following components:

• Audio Provisioning Server (APS)

• Core Billing Manager (CBM)

• CS 2000 Management Tools

• CS 2000 SAM21 Manager

• Integrated Element Management System (IEMS)

• Media Gateway (MG) 9000 Manager

• Multiservice Data Manager (MDM)

• Network Patch Manager

Although the MDM, MG 9000 Manager and CBM do not host a database,this restore procedure still applies to these components as it can be usedto recover other critical files.

PrerequisitesYou need the DVD on which you backed up the Oracle/MySQL data. If thedata was backed up by the backup restore manager, you need the nameof the file located in directory /backup/bkresmgr/backup.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At the server, insert the DVD on which you backed up theOracle/MySQL data, into the drive. If restoring the data from afile, go to the next step.

2 At your workstation, log in to the server.

> telnet <server>

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 210: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

210 Communication Server 2000 Management Tools software downgrade

Press the Enter key.

where

server is the IP address or host name of theactive SPFS-based server on which you areperforming the data restore

3 When prompted, enter your user ID and password.

4 Change to the root user.

$ su -

and press the Enter key.

5 When prompted, enter the root password.

6 Verify the permissions on the restore log directory (bkslog).

# ls -alrt /var/opt/nortel

and press the Enter key.

Example response

total 22lrwxrwxrwx 1 root succssn 28 Dec 22 2003 gwc ->/net/47.141.126.131//swd/gwc

drwxrwxr-x 4 root other 512 Dec 22 2003 .drwxr-xr-x 7 root sys 512 Sep 10 12:41 ..drwxr-xr-x 2 oracle oinstall 512 Dec 8 15:58 dbdrwxrwxrwt 2 root other 6656 Dec 15 19:56 bkslog

If the permissions of bkslog Go to

are drwxrwxrwt step 9

are not drwxrwxrwt the next step

7 Change the permissions of bkslog.

# chmod 1777 /var/opt/nortel/bkslog

and press the Enter key.

8 Determine if server applications have been stopped.

# servquery -status all

and press the Enter key.

9 If not already done, stop the server applications that run on theserver.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 211: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 211

For Refer to

CS 2000 Management Toolsserver applications

Stopping the SESM serverapplication.

Stopping the SAM21 Managerserver application

Stopping the APS server application

Nortel ATM/IP Solution-levelAdministration and Security(NN10402-600) Nortel PacketMSC Security and Administration(NN20000-216)

MG 9000 Manager andmid-tier server applications

Nortel MG 9000 Fault Management(NN10074-911)

Nortel MG 9000 Administration andSecurity (NN10162-611)

Also, if required, Nortel MG 9000Administration and Security(NN10402-600)

IEMS server application Stopping the NPM server application

Nortel IEMS Administration andSecurity (NN10336-611)

10 Verify the permissions, owner and group of /data/oradata bytyping:

#ls -l /data

and pressing the Enter key.

Example response:

drwxr-xr-x 7 oracle oinstall 512 Oct 16 18:13 oradata

11 Restore the database from backup DVD.

$ /opt/nortel/sspfs/bks/rsdata

and press the Enter key.

Example response

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 212: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

212 Communication Server 2000 Management Tools software downgrade

Done patching databaseDATABASE re-started successfullyRestoring critical files from opt/critdata.cpio2384 blocks!! WARNING !! Restoring /etc/passwd, /etc/shadow or/etc/group files could cause problems with respect topasswords, file’s ownership. Do you want to restorethese files (y/n):[y]n

12 Remove the DVD from the drive as follows:

• Ensure you are at the root directory level.# cd /

and press the Enter key.

• Eject the DVD.# eject cdrom

and press the enter key.If the DVD drive tray does not open after you havedetermined that the DVD drive is not busy and is not beingread from or written to, enter the following commands:# /etc/init.d/volmgt stop# /etc/init.d/volmgt startPress the eject button located on the front of the DVD drive.

• Remove the DVD from the drive and proceed to step 14.

13 Restore the database from the backup file.

$ /opt/nortel/sspfs/bks/rsdata -f /backup/bkresmgr/backup/<filename>

and press the Enter key.

Variable Value

<filename> is the name of the backup filecreated by the backup restoremanager

14 Verify that the database restored properly.

# queryAllFaults

and press the Enter key.

15 Use the information in the following table to determine the nextstep.

If Then

no alarm appears and you needto restore the certificate

the next step

no alarm appears and you do notneed to restore the certificate

step 19

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 213: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 213

If Then

an alarm appears contact Nortel

the database did not restoreproperly

contact Nortel

16 Determine if WebPKProxy is configured.

# ls /data/pkclient/certificates

and press the Enter key.

17 Use the following table to determine your next step.

If Then

the following files are associated withthis machine:• CA1_Cert.pem

• CA2_Cert.pem

• TRUSTED_Cert.pem

the next step

the above files do not exist step 19

18 Reconfigure WebPKProxy as follows:

• From the CLI, select Configuration ---> Security ServicesConfiguration ---> PKManager Certificate Installation --->install_certs (Install Certificates supplied by PKManager).Wait for the script to complete. Verify that no errorsdisplayed.

• From the CLI, select Configuration ---> Security ServicesConfiguration ---> WebPKProxy/PKClient Configuration--->register_pkclient.

• Enter the host name of the Certificate Manager.The Certificate Manager box must be fully functional for thisstep to succeed.

Wait for the script to complete. Verify that no errors displayed.

19 Start the server applications that run on the server.

For Refer to

CS 2000 Management Toolsserver applications

Starting the SESM serverapplication

Starting the SAM21 Managerserver application

Starting the APS serverapplication

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 214: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

214 Communication Server 2000 Management Tools software downgrade

For Refer to

Nortel ATM/IP Solution-levelAdministration and Security(NN10402-600)Nortel PacketMSC Security and Administration(NN20000-216)

MG 9000 Manager and mid-tierserver applications

Nortel MG 9000 FaultManagement ( (NN10074-911))

Nortel MG 9000 Administrationand Security (NN10162-611):

Also, if required, Nortel MG 9000Administration and Security(NN10162-611)

IEMS server application Starting the NTP serverapplication

Nortel IEMS Administration andSecurity (NN10336-611):Starting the IEMS server orchanging the IEMS server modeto standby

If one or more applications do not start, contact Nortel forassistance.

If the files /etc/passwd, /etc/group were backed-up previously,the process prompts the following warning message:

WARNING !! Restoring /etc/passwd, /etc/shadow or/etc/group files could cause problems with respect topasswords, file’s ownership. Do you want to restorethese files (y/n):[y]

Select:n - No, if you have changed some passwords, added/changednew users/groups since the last backup.y - Yes, otherwise.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 215: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 215

Clearing the JWS Cache on a Client Workstation

Use this procedure to clear the Java™ Web Start (JWS) cache on a clientworkstation.

The JWS cache on a client workstation needs to be cleared after anHTTPS certificate is installed on an existing Sun server that was notpreviously using a certificate. Clearing the cache allows you to properlylaunch the CS 2000 Management Tools client applications from yourworkstation.

PrerequisitesNone

Procedure stepsPerform the following steps to complete this procedure.

Clear JWS cache

Step Action

1 At your workstation, access the Java Web Start ApplicationManager.

2 Click File->Preferences to access the Preferences window.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 216: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

216 Communication Server 2000 Management Tools software downgrade

3 Click the Advanced tab to access the Advanced window.

4 Click Clear Folder to clear the cache.

5 Click Yes to confirm you want to clear the cache (remove alldownloaded resources).

6 Click OK to close the Preferences window.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 217: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 217

7 Exit the Java Web Start Application Manager.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 218: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

218 Communication Server 2000 Management Tools software downgrade

Routing log streams from an SPFS-based server toa remote host

Use this procedure to forward one or more logs streams from anSPFS-based server to a remote host such as the core manager or IEMS.

PrerequisitesYou must have root user privileges as well as the IP address of the remotehost.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, log in to the server by typing

> telnet <IP address>

and pressing the Enter key.

where

IP address is the IP address of the SPFS-basedserver

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su -

and pressing the Enter key.

4 When prompted, enter the root password.

5 Access the command line interface by typing

# cli

and pressing the Enter key.

6 Enter the number next to the "Configuration" option in the menu.

7 Enter the number next to the "Syslog Configuration" option in themenu.

8 Enter the number next to the "route_syslog_on" option in themenu.

9 When prompted, enter the facility to be routed, for example,"local0.notice".

10 When prompted, enter the IP address of the remote host.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 219: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 219

11 Repeat steps 8 through 10 to for each facility you want to haverouted.

12 Exit each menu level of the command line interface to eventuallyreturn to the command prompt, by typing

select - x

and pressing the Enter key.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 220: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

220 Communication Server 2000 Management Tools software downgrade

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 221: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

221.

Breaker module replacementThis chapter describes the installation, removing, and replacing of aBreaker Interface Panel (BIP) alarm module and a breaker module, as wellas the replacing of a breaker interface module.

The following frames contain an alarm module:

• Services Application module Frame (SAMF) for Communication Server2000 and Communication Server 2000 Compact

• Cabinetized Operations Administration and Maintenance (COAM)frame for Communication Server 2000 and Communication Server2000 Compact

• Call Control Frame (CCF) for Communication Server 2000 Compactonly

• Ethernet Routing Switch (ERS) 8600 Carrier Voice over IP Frame,which is also referred to as the CS LAN Frame, for CommunicationServer 2000 and Communication Server 2000 Compact

• Universal Signaling Point (USP), which can be one of USP CompactCCF, USP Frame with a Control Application Module (CAM) shelf, orUSP in a COAM

Navigation

• “Removing a breaker module” (page 222)

• “Installing a BIP alarm module” (page 224)

• “Removing a BIP alarm module” (page 226)

• “Replacing a BIP alarm module” (page 228)

• “Replacing a (BIM) Breaker Interface Module” (page 229)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 222: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

222 Breaker module replacement

Removing a breaker module

Remove a breaker module to replace it when the module fails or to ensuresafe removal of the Breaker Interface Panel (BIP).

WARNINGRisk of static electricity damageMake sure that you have protection against electrostaticdischarge (ESD). Wear a wrist strap connected to the wrist-strapgrounding point of the frame supervisory panel (FSP).

CAUTIONLoss of serviceRead and make sure you thoroughly understand the instructionsin this procedure before removing the BIP alarm module.While the module is removed, alarm reporting is temporarilysuspended.

Prerequisites• You must be familiar with the safety considerations and best practices

for your location.

• You must wear an electrostatic discharge wrist strap connected to thewrist-strap grounding point of the FSP.

Procedure steps

Step Action

1 At the front of the BIP

Grasp the BIP screw cover and pull it out of the BIP chassis.

2 Switch all breakers to 0.

3 Unfasten the large captive screw of the breaker module.

4 While pinching the lower front lip of the module with your thumband finger tips, pull gently but firmly straight out until the alarmdisengages. Stop pulling when the front of the module is pastthe front of the BIP.

DANGERRisk of electric shockAllow 15 seconds before removing the breakermodule. Capacitors on the breaker module must beallowed to discharge.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 223: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 223

5 Pull the module straight out of the BIP.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 224: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

224 Breaker module replacement

Installing a BIP alarm module

Install a BIP alarm module as part of a replacement or initial installationtask.

The BIP alarm module indicates hardware alarms, drives the alarmlight-emitting diode (LED) board and monitors the state of the powerbreakers.

CAUTIONRisk of static electricity damageMake sure that you have protection against electrostaticdischarge (ESD). Wear a wrist strap connected to the wrist-strapgrounding point of the frame supervisory panel (FSP).

CAUTIONLoss of serviceRead and make sure you thoroughly understand the instructionsin this procedure before removing the BIP alarm module. Tominimize the effect of removing an alarm module, insert thereplacement immediately. Once the alarm module is installed,software alarms stop.

Prerequisites• You must be familiar with the safety considerations and best practices

for your location.

• You must wear an electrostatic discharge wrist strap connected to theground point of the FSP.

Procedure steps

Step Action

1 At the front of the BIP

Get a BIP alarm module PEC NTRX51HC.

2 Grip the replacement alarm module with the fingers and thumb ofone hand on the lower front lip (the screw end) and use the otherhand to align the rear of the module with the opening in the BIP.

3 Push the module gently but firmly straight in until it engages.Stop pushing when the front of the module is flush with the frontof the BIP.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 225: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 225

4 Engage and fasten the large slotted captive screw of the alarmmodule. Do not strip the threads or the hole because the modulecannot seat properly.

5 Check the alarm module status LEDs.

If the color is green, go to step 7.

If the color is other than green, go to step 6.

6 Have the network operator confirm the status in the software.

If the color is green, go to step 7.

If the color is other than green, perform troubleshootingprocedures to clear the problem, and continue to step 7.

7 Replace the BIP screw cover by aligning the screw cover withthe BIP chassis and pushing it into place. The BIP screw coversnaps into the snap-in posts.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 226: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

226 Breaker module replacement

Removing a BIP alarm module

Remove a BIP alarm module to replace it when the module fails or toensure safe removal of the BIP.

CAUTIONRisk of static electricity damageMake sure that you have protection against electrostaticdischarge (ESD). Wear a wrist strap connected to the wrist-strapgrounding point of the frame supervisory panel (FSP).

CAUTIONLoss of serviceRead and make sure you thoroughly understand the instructionsin this procedure before removing the BIP alarm module.While the module is removed, alarm reporting is temporarilysuspended.

Prerequisites• You must be familiar with the safety considerations and best practices

for your location.

• You must wear an electrostatic discharge wrist strap connected to thewrist-strap grounding point of the FSP.

Procedure steps

Step Action

1 At the front of the BIP

Grasp the BIP screw cover and pull it out of the snap-in postson the BIP chassis.

2 Unfasten the large captive screw of the BIP alarm module.

DANGERRisk of electric shockAllow 15 seconds before removing the breakermodule. Capacitors on the breaker module must beallowed to discharge.

3 While pinching the lower front lip of the module with your thumband finger tips, pull gently but firmly straight out until the alarmdisengages. Stop pulling when the front of the module is pastthe front of the BIP.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 227: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 227

4 Pull the module straight out of the BIP.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 228: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

228 Breaker module replacement

Replacing a BIP alarm module

Replace a BIP alarm module when the module fails.

CAUTIONRisk of static electricity damageMake sure that you have protection against electrostaticdischarge (ESD). Wear a wrist strap connected to the wrist-strapgrounding point of the frame supervisory panel (FSP).

CAUTIONLoss of serviceRead and make sure you thoroughly understand the instructionsin this procedure before replacing the BIP alarm module.While the module is removed, alarm reporting is temporarilysuspended.

Prerequisites• You must be familiar with the safety considerations and best practices

for your location.

• You must wear an electrostatic discharge wrist strap connected to theground point on the FSP.

Procedure steps

Step Action

1 At the front of the BIP

Remove the failed BIP alarm module. To remove the module,use the procedure “Removing a BIP alarm module” (page 226).When the module has been removed, proceed to step 2 of thisprocedure.

2 Install the replacement BIP alarm module. To replace themodule, use the procedure “Installing a BIP alarm module” (page224).

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 229: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 229

Replacing a (BIM) Breaker Interface Module

Use this procedure to replace an A or B feed Breaker Interface Module(BIM) in a Call Control frame (CCF), Service Application Module frame(SAMF) or Cabinetized Operations Administration and Maintanence(COAM) frame.

This procedure is executed on an in-service frame and will not result in aservice outage.

PrerequisitesIt is recommended to perform backups on the equipment within the frameprior to performing this procedure. Power will be running in simplex modeduring the replacement of the module.

Procedure stepsProcedure 1At the CC, SAM or COAM frame

Step Action

1 On the BIP, turn the feed breakers to the OFF position. The OFFposition is indicated with a O.

If you are replacing the Then

A feed BIM turn A feed breakers to OFF

B feed BIM turn B feed breakers to OFF

2 Verify alarms are generated to show that the power feed you justturned off (A or B) is down.

3 Remove the front cover from the Breaker Interface Panel (BIP).

4 Using a flat-head screwdriver, loosen the screw securing the Aor B feed breaker module to the BIM.

Do not remove the breaker module at this time.

5 Unseat, but do not remove, the A or B feed breaker module fromthe BIP. Wait 15 seconds for the breaker module to discharge.

6 Remove the breaker module from the BIP.

7 Install the correct breaker module in the A or B position (asappropriate).

8 Tighten the screw to hold the A or B feed breaker module inplace.

9 If all BIM replacement is complete, install the Breaker InterfacePanel (BIP) front cover.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 230: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

230 Breaker module replacement

10 On the BIP, turn the feed breakers to the ON position. The ONposition is indicated with a |.

If you replaced the Then

A feed BIM turn A feed breakers to ON

B feed BIM turn B feed breakers to ON

11 Verify the alarms are cleared on the equipment within the frame.

12 Verify that both the A and B feeds are ON, and that there are noalarms.

13 Verify alarm lights are OFF as shown on the right of the figurethat follows.

You have completed this procedure.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 231: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

231.

Maintaining Communication Server2000 GWC alarms and logs

Maintain Communication Server 2000 Gateway Controller (GWC) alarmsand logs to assist you in analyzing and resolving issues.

For information on gaining access to customer log files, see GWC FaultManagement (NN10202-911).

Navigation

• “Pinging the Communication Server 2000 Core Manager ” (page 233)

• “Pinging the core and billing manager” (page 236)

• “Ensuring that the OSS Comms Svcs and OSS and Application Svcsare in service” (page 238)

• “Checking the status of the DDMS proxy” (page 240)

• “Adding the Communication Server 2000 Management Tools Server asa DDMS client of the CBM” (page 241)

• “Adding the Communication Server 2000 Management Tools Server asa DDMS client of the Communication Server 2000 Core Manager Core”(page 245)

• “Restarting the Apache Web server” (page 248)

• “Checking the occupancy of the /opt/ directory on the CommunicationServer 2000 Management Tools server” (page 249)

• “Adding LEN information back into the XA-Core” (page 251)

• “Checking for MSC Server 1000 XACoreCS 2000 or CS 2000-Compactlogs” (page 254)

• “Deleting carriers from a GWC” (page 255)

• “Viewing carrier provisioning data for a GWC node” (page 257)

• “Adding carriers to a GWC” (page 261)

• “Viewing media proxy associations” (page 284)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 232: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

232 Maintaining Communication Server 2000 GWC alarms and logs

• “Viewing media proxies associated with a GWC node” (page 286)

• “Taking a manual GWC software image” (page 287)

• “Reconfiguring NTP service” (page 290)

• “Using the Table Editor to Delete a Tuple from a Table” (page 293)

• “Changing the APS Oracle account password” (page 295)

• “Viewing GWC platform hardware alarms” (page 297)

• “Viewing GWC PM logs” (page 299)

• “Viewing GWC logs in syslog files” (page 300)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 233: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 233

Pinging the Communication Server 2000 CoreManager

Ping the Communication Server 2000 Core Manager to determine if thereis communication between the Communication Server 2000 ManagementTools server and the Core and Billing Manager (CBM).

Procedure steps

Step Action

1 Telnet to the Communication Server 2000 Management Toolsserver, by typing

> Telnet <server>

and pressing the Enter key.

2 When prompted, enter your user ID, and password.

3 To determine the IP address of the Communication Server 2000Core ManagerCommunication Server 2000 Core Manager, firstnote the IP address of the Communication Server 2000 CoreManagerCommunication Server 2000 Core Manager.

4 Exit from the SESM Server Application configuration utility, bytyping

select x

and press the Enter key.

5 Change to the root user, by typing

$ su root

and pressing the Enter key.

6 When prompted, enter the root password.

7 Change directory by typing

# cd /bin

and pressing the Enter key.

8 Execute the configuration script, by typing

# ./configure

and pressing the Enter key.

System response

SESM configuration1 SESM common configuration (IP addresses, Market,CM CLLI)2 SESM database tools

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 234: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

234 Maintaining Communication Server 2000 GWC alarms and logs

3 SESM related applications configuration (MG9K, LMM)4 SESM provisioning configuration5 SESM logging configuration 9syslog, sesm debug log)6 view sesm configuration settingsx exit

select

9 Select option six (view sesm configuration settings), by typing

select 6

and pressing the Enter key.

Example system response

10 Ping the Communication Server 2000 Core ManagerCommunication Server 2000 Core Manager, by typing

# ping <ip_address>

and press the Enter key.

Example system response

172.17.40.250 is alive

If the result is alive, there is communication between theCommunication Server 2000 Management Tools server and the

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 235: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 235

Communication Server 2000 Core Manager. A result of deadindicates that there is no communication.

--End--

Variable definitionsVariable Value

<ip_address> The IP address of Communication Server 2000 Core Manager.

<server> The Internet Protocol (IP) address, or host name of theCommunication Server 2000 Management Tools server

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 236: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

236 Maintaining Communication Server 2000 GWC alarms and logs

Pinging the core and billing manager

Ping the Core and Billing Manager (CBM) to determine if there iscommunication between the Communication Server 2000 ManagementTools server and the Core and Billing Manager (CBM).

Procedure steps

Step Action

1 Telnet to the Communication Server 2000 Management Toolsserver, by typing

> Telnet <server>

and pressing the Enter key.

2 When prompted, enter your user ID and password.

3 To determine the IP address of the CBM, change to the rootuser.

$ su root

and press the Enter key.

4 When prompted, enter the root password.

5 Change directory, by typing

# cd /bin

and pressing the Enter key.

6 Execute the configuration script, by typing

# ./configure

and pressing the Enter key.

System response

SESM configuration1 SESM common configuration (IP addresses, Market,CM CLLI)2 SESM database tools3 SESM related applications configuration (MG9K, LMM)4 SESM provisioning configuration5 SESM logging configuration 9syslog, sesm debug log)6 view sesm configuration settingsx exit

select

7 Select option six (view sesm configuration settings), by typing

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 237: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 237

select 6

and pressing the Enter key.

Example system response

8 Note the IP address of the Core and Billing Manager (CBM).

9 Exit from the SESM Server Application configuration utility, bytyping

select x

and press the Enter key.

10 Ping the Core and Billing Manager (CBM), by typing

# ping <ip_address>

and press the Enter key.

Example system response

172.17.40.250 is alive

If the result is alive, there is communication between theCommunication Server 2000 Management Tools server and theCore and Billing Manager (CBM). A result of dead indicates thatthere is no communication.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 238: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

238 Maintaining Communication Server 2000 GWC alarms and logs

Ensuring that the OSS Comms Svcs and OSS andApplication Svcs are in service

Use this procedure to determine if the applications OSS Comms Svcs andthe OSS Application Svcs are in-service on the Communication Server2000 Core ManagerCommunication Server 2000 Core Manager. If theseapplications are not in-service, this procedure provides the steps forreturning the applications to service.

Procedure steps

Step Action

1 Log on to the Communication Server 2000 Core Manager.

2 Access the Application level by typing

# sdmmtc appl

and press the Enter key.

Use the up and down commands to scroll through the list ofapplications.

Example response:

3 Determine the state of the two applications: OSS Comms Svcsand OSS and Application Svcs.

If both applications are in-service, you have completed thisprocedure; otherwise, go to the next step.

4 Busy the out-of-service application by typing

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 239: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 239

> bsy <application_number>

and press the Enter key.

If you did not successfully busy the application, contact your nextlevel of support; otherwise, go to the next step.

5 Return to service the out-of-service application by typing

> rts <application_number>

and press the Enter key.

If you successfully returned to service the application, proceed tothe next step; otherwise, contact your next level of support.

6 Repeat steps 4 and 5 for the second out-of-service application ifnecessary.

--End--

Variable definitionsVariable Value

<application_number> Is the number used to identify the specific application in theapplication list.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 240: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

240 Maintaining Communication Server 2000 GWC alarms and logs

Checking the status of the DDMS proxy

Use this procedure to check the status of the DMS Data ManagementSystem (DDMS) proxy that runs on the Communication Server 2000Management Tools server.

Procedure steps

Step Action

1 At your workstation

Telnet to the Communication Server 2000 Management Toolsserver by typing

> Telnet <ip_address>

and press the Enter key.

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su - root

and press the Enter key.

4 When prompted, enter the root password.

5 Determine if the DDMS proxy is running, by typing

/opt/ddmsproxy/bin/ddmsproxy status

# servquery -status all

6 Note whether the DDMS proxy is running or not running. Lookfor DDMSPROXY in the resulting list of applications and note if itis running or not running.

--End--

Variable definitionsVariable value

<ip> Is the Internet Protocol (IP) address of theCommunication Server 2000 ManagementTools server.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 241: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 241

Adding the Communication Server 2000Management Tools Server as a DDMS client of theCBM

Use this procedure to add the Communication Server 2000 ManagementTools Server as a DDMS client of the Communication Server 2000 CoreManagerCore and Billing Manager (CBM).

Procedure steps

Step Action

1 Log onto the Core and Billing Manager (CBM) as a maint classuser or root user and access the maintenance interface by typing

# cbmmtc

and press the Enter key.

2 Access the Config level by typing

# config

and press the Enter key.

Example of a screen showing the Config level on the Core andBilling Manager (CBM)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 242: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

242 Maintaining Communication Server 2000 GWC alarms and logs

3 In the list of filesets, find OSS Comms Svcs and note the filesetnumber to the left of the Fileset Description.

4 Select OSS Comms Svcs by typing

# select <fileset number>

and pressing the Enter key.

5 Start to configure the OSS Comms Svcs fileset by typing

# config

and pressing the Enter key.

Example of a screen showing system output

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 243: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 243

6 Advance to the CM User Setup screen by pressing the Enterkey.

Example of the CM User Setup screen

7 Advance to the DDMS Clients Configuration screen by typing:

# 0

and pressing the Enter key.

Example of the DDMS Clients Configuration screen

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 244: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

244 Maintaining Communication Server 2000 GWC alarms and logs

8 From the DDMS Clients Configuration list choose to add newclients, by typing

# 1

and pressing the Enter key.

9 Enter the Internet Protocol (IP) address of the CommunicationServer 2000 Management Tools Server by typing

# <CS2K_tool_server_ip_address>

and pressing the Enter key.

10 Exit from the DDMS Clients Configuration screen by typing

# done

and pressing the Enter key.

11 If the addition fails, contact your next level of support.

--End--

Variable definitionsVariable Value

<CS2K_man_server_ip_address> The IP address of the Communication Server 2000Management Tools server.

<fileset number> The number shown to the left of the Fileset Description.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 245: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 245

Adding the Communication Server 2000Management Tools Server as a DDMS client of theCommunication Server 2000 Core Manager Core

Use this procedure to add the Communication Server 2000 ManagementTools Server as a DDMS client of the Communication Server 2000 CoreManager.

Procedure steps

Step Action

1 Log onto the Communication Server 2000 Core ManagerCommunication Server 2000 Core Manager as a maint class user or rootuser and access the maintenance interface by typing

# sdmmtc

# sdmmtc (if this is for an SDM) or

# cbmmtc ( if this is for a CBM)

and press the Enter key.

2 Access the Config level by typing

# config

and press the Enter key.

Example of a screen showing the Config level on theCommunication Server 2000 Core Manager

3 In the list of filesets, find OSS Comms Svcs and note the filesetnumber to the left of the Fileset Description.

4 Select OSS Comms Svcs by typing

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 246: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

246 Maintaining Communication Server 2000 GWC alarms and logs

# select <fileset number>

and pressing the Enter key.

5 Start to configure the OSS Comms Svcs fileset by typing

# config

and pressing the Enter key.

Example of a screen showing system output

6 Advance to the CM User Setup screen by pressing the Enterkey.

Example of the CM User Setup screen

7 Advance to the DDMS Clients Configuration screen by typing:

# 0

and pressing the Enter key.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 247: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 247

Example of the DDMS Clients Configuration screen

8 From the DDMS Clients Configuration list choose to add newclients, by typing

# 1

and pressing the Enter key.

9 Enter the Internet Protocol (IP) address of the CommunicationServer 2000 Management Tools Server by typing

# <CS2K_man_server_ip_address>

and pressing the Enter key.

10 Exit from the DDMS Clients Configuration screen by typing

# done

and pressing the Enter key.

11 If the addition fails, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 248: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

248 Maintaining Communication Server 2000 GWC alarms and logs

Restarting the Apache Web server

Restart the Apache Web server if you are unable to launch CommunicationServer 2000 SAM21 Manager, Communication Server 2000 ManagementTools GUI, or Network Patch Manager. The Apache Web server is one ofthe servers included in the Server Platform Foundation Software (SPFS)which in turn is a software package included in the Communication Server2000 Management Tools suite of software.

Procedure steps

Step Action

1 Telnet to the Communication Server 2000 Management Toolsserver by typing

> Telnet <server>

and pressing the Enter key.

2 When prompted, enter your user ID and password.

3 Change to the root user, by typing

$ su - root

and pressing the Enter key.

4 When prompted, enter your root password.

5 Restart the Apache Web server by typing

# /opt/apache/rcscripts/spfs_apache restart

and pressing the Enter key.

6 If the restart fails, contact your next level of support.

--End--

Variable definitionsVariable Value

<server> The Internet Protocol (IP) address, or host name of theCommunication Server 2000 Management Tools server.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 249: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 249

Checking the occupancy of the /opt/ directory onthe Communication Server 2000 Management Toolsserver

Check the occupancy of the /opt/ directory on the Communication Server2000 Management Tools server to verify that the /opt/ directory is full.

Procedure steps

Step Action

1 Telnet to the Communication Server 2000 Management Toolsserver by typing

> Telnet

<server>

and pressing the Enter key.

2 When prompted, enter your user ID and password.

3 Determine the occupancy of the /opt/ directory by typing

$ df -k

Example of a screen showing occupancy of the /opt/ directoryat 98%

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 250: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

250 Maintaining Communication Server 2000 GWC alarms and logs

Variable definitionsVariable Value

<server> The Internet Protocol (IP) address or host name of the CommunicationServer 2000 Management Tools server.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 251: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 251

Adding LEN information back into the XA-Core

Add line equipment number (LEN) information back into the XA-Core byusing the NEW command in OSSGate.

Use this procedure only if you have attempted to delete a line in OSSGateusing the OUT command and you have incorrectly specified the LENinstead of the endpoint. As a result, there is now a data mismatch(relating to the deleted line) that exists between the XA-Core and theCommunication Server 2000 Management Tools data. In the XA-Core theline has been deleted, but in the GWC Manager client, the line endpointsstill exist. After you have added the LEN information back into theXA-Core (using this procedure), you can then correctly delete the line inOSSGate using the OUT command by specifying the endpoint information(termination point name and media gateway name).

ATTENTIONThis procedure requires that you Telnet to the OSSGate application that resideson the Communication Server 2000 Management Tools server.

Procedure steps

Step Action

1 Initiate a Telnet session to OSSGate by typing

Telnet <ossgate_server_name> <port_number>

and pressing the Enter key.

Example of system response:

Trying 47.142.94.80...Connected to wcn0s5jk.Escape character is ’^]’.

Enter username and password

2 Enter your user name and password by typing

<username> <password>

and pressing the Enter key.

Example of a system response

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 252: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

252 Maintaining Communication Server 2000 GWC alarms and logs

************************************************ **** OSS Gateway **** **** This is a PRIVATE Database. **** **** All activity is subject to monitoring. **** ***Any UNAUTHORIZED access or use is PROHIBITED*** and may result in PROSECUTION. **** ************************************************

You have connected to OSSGate, and the system is displaying aprompt (the ’>’ character) in the Telnet window on your screen.

3 Put the Telnet session into CI mode by holding down the controlkey and type B.

4 At the question-mark prompt enter the command interpretermode by typing

mode ci

and press the Enter key.

5 Add the LEN information back into the XA-Core by enteringthe NEW command at the ">" prompt. NEW is the SERVORDcommand that you use to provision a line.Figure 52Example of the NEW command with the LEN instead of endpoint

6 You have completed this procedure, but you may wish to inputother commands in OSSGate before disconnecting to OSSGate.If so, return to the procedure that directed you here originally. If

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 253: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 253

you wish to disconnect from OSSGate now, follow the remainingsteps in this procedure.

7 To disconnect from OSSGate, hold down the control key andtype B.

8 At the question-mark prompt, type

logout

and press the Enter key.

9 To terminate the Telnet session, hold down the control key andtype B.

10 At the question-mark prompt, type

clearconv

and press the Enter key.

--End--

Variable definitions

Variable Value

<ossgate_server_name> The hostname or the IP address of the server on whichOSSGate is running.

<password> The password used to log onto the OSSGate.

<port_number> The server port used by OSSGate (the default is 10023).

<username> The user name used to log onto the OSSGate.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 254: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

254 Maintaining Communication Server 2000 GWC alarms and logs

Checking for MSC Server 1000 XACoreCS 2000 orCS 2000-Compact logs

Check for MSC Server 1000 XACoreCommunication Server 2000 logs oftype XAC, MS, NET, IOD, EXT, and APPL. using LOGUTIL (at the MAP).You can also use this procedure to check for Communication Server2000-Compact call processing logs of type IOD, EXT, or APPL.

The XAC, MS, and NET logs do not apply to the Communication Server2000-Compact.

Procedure steps

Step Action

1 Access the log utility (LOGUTIL), by typing

> logutil

and press the Enter key.

2 View the most recent MSC Server 1000 XACoreCommunicationServer 2000(or Communication Server 2000-Compact) logs of aspecific type, by typing

> open <log_name>

and press the Enter key.

The switch responds with the most recent XAC log.

3 View the entire log buffer of logs of a specific type, by typing

> dumplogs <log_name>

and press the Enter key.

The switch dumps the entire log buffer of XAC logs.

--End--

Variable definitionsVariable Value

<log_name> The name of a group of logs, for example: XAC, IOD, MS, NET,EXT, or APPL.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 255: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 255

Deleting carriers from a GWC

Delete (remove) carriers from a Gateway Controller (GWC) node.Line endpoints are removed automatically when lines are removed(de-provisioned) from a GWC. Carriers for all other gateway types mustalso be removed manually.

Do not remove V5.2 carriers using this procedure.

Prerequisites• All carriers on a gateway must be in one of the following states before

you can remove them:

— INB (Installation Busy)

— UNKNOWN (Core datafill is missing)

Additional Core table datafill may need to be removed in order toremove all trunk and line endpoint data from all databases.

• Make sure that you are deleting one carrier at a time.

Procedure steps

Step Action

1 At the Communication Server 2000 Management Tools window,from the Device Types menu, click the Gateway Controllerfolder.

2 From the Contents of Gateway Controller frame, select the GWCnode that you wish to access.

3 Click the Provisioning tab, then the Carriers tab.

4 Click the Retrieve All button to retrieve all carriers on thespecified GWC node.

5 From the Carrier List, select one carrier you wish to delete. Youcan delete only one carrier at a time.

Your selection is highlighted.

6 Click the Delete button at the bottom of the screen.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 256: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

256 Maintaining Communication Server 2000 GWC alarms and logs

7

CAUTIONPossible partial service disruptionIf the endpoints associated with a carrier are deletedfrom the GWC and not from the Core, call processingproblems occur. See CS 2000 Configurationdocument applicable to your solution to removecorresponding trunk data from the Core tables.

At the confirmation box, click Yes to confirm that you wish todelete the carrier.

The carrier list is automatically updated following a successfuldeletion.

8 Repeat this procedure for other carriers you wish to delete onthis GWC node.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 257: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 257

Viewing carrier provisioning data for a GWC node

This procedure describes how to view carrier (endpoint group) provisioningdata for a selected Gateway Controller (GWC) node.

Use this procedure when you require specific provisioning informationabout carriers associated with a specific GWC node.

Procedure steps

Step Action

1 At the Communication Server 2000 GWC Manager client

At the Communication Server 2000 Management Tools window,from the Device Types menu, click the Gateway Controllerfolder.

2 From the Contents of: Gateway Controller frame, select theGWC node that you wish to view.

3 Click the Provisioning tab, then the Controller tab to viewgeneral provisioning information for the GWC node selected.

4 Click the Carriers tab to view information related to carriers(endpoint groups) belonging to gateways associated with theGWC node selected previously.

Select the edge of any tab to adjust the display.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 258: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

258 Maintaining Communication Server 2000 GWC alarms and logs

5 Click the Retrieve All button to view carrier endpoint data.

Table 40 "Description of carrier endpoint search criteriafunctions" (page 258) contains information about the searchfunctions (refer to numbers in the preceding figure).

6 If you wish to view the trunk endpoints and terminal numbersassociated with a specific carrier, click the carrier entry and clickthe Display button in the lower right corner of the screen.

--End--

Job aidTable 40Description of carrier endpoint search criteria functions

Step Menu component Description

1 Retrieval criteria: selectionbox and drop-down list

Enter a search criteria string in this field. Up to 20 searchstrings are saved during the session. Previous searchstrings can be recalled by selecting from the drop-downlist. For more information about using search string, seeTable 41 "Examples of carrier search criteria" (page 259).

2 Limit results: selection boxand drop-down list

Select a value from the drop-down list. This value limitsthe number of matching entries returned in response tothe query. Valid values are as follows: 25, 50, 100, 250,500, 1000, and "no limit".

The "no limit" value is still itself limited by the maximumnumber of entries currently downloadable.

3 Replace List radio button Deselect this option if you do not want the query resultsto replace previously existing data displayed in the table.The default selection is to replace the existing data.

Append to List radio buttonSelect this option if you want the query results to beappended to the existing data displayed in the table.

4 Retrieve button This selection retrieves data from the CommunicationServer 2000 GWC Manager server database matchingthe specified search retrieval criteria. The data presentedis a "snapshot" of the Communication Server 2000 GWCManager database based on the currently provisioneddata. The view is not updated in real time.

Newly added or deleted data will not be added to ordeleted from the displayed table view except when there isa deletion of endpoints. If an carrier endpoint is selected

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 259: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 259

Table 40Description of carrier endpoint search criteria functions (cont’d.)

Step Menu component Description

from the table and deleted using the Delete button, thenthe endpoint will be removed from the table if deletion wassuccessful.

Endpoint deletion using this interface is limited to carrierendpoints only. Line endpoints are added automaticallywhen you provision lines to a GWC. For information abouthow to add or delete lines, see the Communication Server2000 Configuration document applicable to your solution.

Retrieve All button

This selection retrieves all the gateway or endpoint data.Values in the "Retrieval criteria" and "Limit results" areignored.

5 Carrier List table This is the tables where the search results are displayed.By clicking on a column header, you can sort the rows inascending order, based on the values in that column. Byclicking on the column header a second time, you can sortthe rows in descending order.

The data is not preserved once another GWC node isselected; however, the retrieval criteria are maintainedin the drop-down list so the same search can bere-executed.

6 Display Highlight a carrier name and click this button if you wish toview the endpoints trunk and terminal numbers associatedwith a specific carrier.

Table 41Examples of carrier search criteria

Examples of carrier search criteria

The gateway and endpoint panes use the same search functionality. The functionality is similar tothat available in web search engines.

• One or more strings can be entered in the Retrieval Criteria: box, separated by space(s).

• Each string in the criteria is compared with the value in each column. If a match is found in anycolumn, the record will be returned.

• Each string in the criteria is automatically wildcard at the beginning and end of the string. (Forexample, the string "gate" will match "MYGATEWAY" and "GATEWAY1".)

• Two special characters are supported as modifiers: the plus "+" and minus "-" signs.The "+" plus character prefixed to the string (for example, "+gate") means the value mustmatch. The "-" character prefixed to the string (for example, "-gate") means the value mustnot match.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 260: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

260 Maintaining Communication Server 2000 GWC alarms and logs

Table 41Examples of carrier search criteria (cont’d.)

Examples of carrier search criteria

The "+" character is assumed, and therefore is not typically used.

• The search is not case sensitive. A string using "gate" will return records where the followingstrings are found: "MYGATEWAY", "MyGateway", or "mygateway".

Example Retrieval Criteria: "DS3"

Expected Results: Returns all records where the string is found in any column.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 261: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 261

Adding carriers to a GWC

Add carriers (endpoint groups) and their endpoints to the gateway afteryou associate a media gateway to a Gateway Controller (GWC) node.

Do not use this procedure to add V5.2 carriers.

CAUTIONPossible partial service disruptionIf you are adding carrier endpoints after having previouslyremoved them from the same gateway, you will have terminalidentifier (TID) mismatches. Contact your next level of supportfor instructions on how to avoid TID mismatches.

Prerequisites• You must first associate a media gateway with a GWC node before

you can add carriers (endpoint groups) to the gateway.

• You need to know the names of the media gateways associated withthe GWC node.

To determine the naming convention for your gateway profile type, seeTable 42 "Carrier and endpoint names" (page 264).

CAUTIONPRI carrier limitationSince SN10, for Trunk GWC, the PRI carrier (D-Channels)number can not exceed 130 due to current GWC capacitylimitation, any attempt to add more that 130 D-Channels will fail.

Procedure steps

Step Action

1 At the Communication Server 2000 GWC Manager client

At the Communication Server 2000 Management Tools window,from the Device Types menu, click the Gateway Controllerfolder.

2 From the Contents of: Gateway Controller frame, select theGWC node that you wish to access.

3 Click the Provisioning tab, then the Carriers tab.

4 Click the Add button at the bottom of the screen to display theAdd Carrier dialog box.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 262: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

262 Maintaining Communication Server 2000 GWC alarms and logs

5 In the Carrier name box, type the name (alphanumeric characterstring) of the carrier that you want to add.

Determine if your system uses strict naming conventions forassigning some types of gateway endpoints.

The carrier name is based on the media gateway type (thegateway profile name). For naming conventions applicable tospecific media gateways, see Table 42 "Carrier and endpointnames" (page 264).

For DPNSS carriers, this name must match the name configuredfor the Physical Name Pattern field when configuring the MG3200 gateway using the Web Server Interface. For moreinformation, see Media Gateway 3200 H.248 User’s Manual(LTRT-72704).

6 In the Gateway name box, type the name (alphanumericcharacter string) of the gateway to which you are adding carriers.

7 This step is optional.

In the Start terminal number, type a number representing thestarting point of a contiguous block of endpoints or terminalidentifiers (TID). If you do not type a number in this field, thesystem automatically defines which TIDs are used and how theymap to the carrier.

For any gateway other than an H.323 gateway, the contiguousblock of endpoints will be either 24 (North American market)or 32 (International market). Note that 31 endpoints will beadded for International PRI. The system automatically identifiesthe number of endpoints included in the block based on theconfiguration of the Communication Server 2000 ManagementTools server.

Ensure that a contiguous range of endpoints are available onyour gateway to fulfill your requirements when you assign a

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 263: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 263

value in the Start terminal number: field and change the size ofyour carrier block.

8 This step applies to H.323 gateways only.

If you are adding carriers to an H.323 gateway, select the H.323check box. You must select the H.323 check box to add carriersto an H.323 gateway.

The Number of ports box becomes active.

9 This step applies to H.323 gateways only and is optional.

In the Number of ports box, type a number defining the size ofthe H.323 virtual carrier block you are adding. If you do not typea number in this box, the system allocates the default block ofendpoints for your system, either 24 (NA) or 32 (International).

A range of 4 to 672 endpoints (inclusive) is supported.

The total sum of values entered in the Number of port box forall carriers assigned to an H.323 gateway must be less than, orequal to, the value entered in the Reserved terminations box ofthe Associate Media Gateway dialog box, when associating theH.323 gateway to a GWC node.

10 This step applies only if you are adding carriers to a non-H.323gateway and PRI trunks are provisioned in the carrier groups.

Select the PRI/DPNSS check box and type a PRI Interface ID(IID) value within the range of 0 to 31 (inclusive).

The IID value is not used in international PRI. Therefore, foroffices using international PRI, select the PRI check box and setthe value to 0.

For PRI in the North American market, the IID value is used inthe non-facility associated signaling (NFAS) configuration inwhich multiple DS1 trunks are controlled by one D-channel. Inthis case, the IID value must match the IID value provisioned atthe far-end switch.

11 This step applies only to DPNSS carriers (international markets).

For DPNSS carriers, select the PRI/DPNSS check box and enteran integer Interface ID (IID:) value within the range of 0 to 63.

This IID must be unique for each gateway and must match thevalue specified for this particular carrier when configuring the MG3200 gateway. For more information, see the Media Gateway3200 H.248 User’s Manual () LTRT-72704.

12 Click OK to apply the input.

If the add carriers operation is successful, the system displays adialog box. Click OK to continue.

If the add carrier operation fails, the system displays an errormessage. Click the Show Details button for more information.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 264: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

264 Maintaining Communication Server 2000 GWC alarms and logs

An error message also appears in the status bar at the bottomof the screen.

13 To verify your changes, click the Retrieve All or the Retrievebutton to update the Carrier List.

14 Provision the trunks that correspond to the carriers by adding atuple to table TRKMEM. For information, see the CommunicationServer 2000 Configuration document applicable to your solution.

--End--

Job aidTable 42Carrier and endpoint names

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

AMBIT line gateways Endpoint names must conform to the following rules for small lineendpoints:

• The endpoint name must contain at least one forward slash ("/").

• The name cannot start with “SU”.

• The maximum length of endpoint name is 32 characters.

ASKEY_LINE

Integrated Access Devices(IAD) gateways

Endpoint names must conform to the following rules for small lineendpoints:

• The endpoint name must contain at least one forward slash ("/").

• The name cannot start with “SU”.

• The maximum length of endpoint name is 32 characters.

MG32LN p/<nnn>

where

p is lower case<nnnn> is an endpoint number: 1 to 384

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 265: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 265

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

MEDIA_SRVMG32003500

(Nortel Media Gateway3200 and 3500)

Carrier or gateway endpoints are provisioned in groups representingE1 and DS1 levels. Provisioning at this level gives access to alltimeslots for service provisioning.

Carrier names are case sensitive and must be entered in uppercase. The following descriptions include both endpoint groups andindividual endpoint descriptions. Services can be applied at theDS1/E1 timeslot level.

DS1

DS1/<h2>/<g> (H.248/Megaco)

where

<h2> is the DS1 (two digit) port or span number:01-04

<g> is the channel number: 1-24 (no leading 0),assigned by the system

Example:

Carrier name: DS1/03Carrier Endpoint name (timeslot): DS1/03/1

AUDIOCODES_6310_TRUNK

MEDIA_SRV_6310_TRUNK

(Nortel Media Gateway3500 using TP-6310 cardand configured as trunkgateway)

Carrier or gateway endpoints are provisioned in groups representingE1 and DS1 levels.

Carrier names are case sensitive and must be entered in uppercase. The following descriptions include both endpoint groups andindividual endpoint descriptions. Services can be applied at theDS1/E1 timeslot level.

DS1

DS1/<h2>/<g_optional> (H.248/Megaco)

where

<h2> is the DS1 (two digit) port or span number:01-84

<g> is optional PRI IID: 0 to 31 (32 to 63 range is notsupported). This value must be unique foreach gateway.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 266: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

266 Maintaining Communication Server 2000 GWC alarms and logs

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

Example:Carrier name: DS1/03

Each DS1 carrier supports 24 terminations and the appropriatenumber of terminations must be reserved in advance. A fullyequipped gateway with DS1 carriers requires 84 x 24=2016terminations.

E1

E1/<h2>/<g_optional> (H.248/Megaco)

where

<h2> is a number: 1 to 63<g> is optional PRI IID: 0 to 63

Example:Carrier name: E1/03

Each E1 carrier supports 31 terminations and the appropriatenumber of terminations must be reserved in advance. A fullyequipped gateway with E1 carriers requires 63 x 31=1953terminations.

MEDIA_SRV_MS2030

Media Server 2030

Endpoints are not specified during provisioning of audio servers.

CALIX_C7 tp/<TT><tt>

where

tp is lower case<TT> is a number: 00-10<tt> is a number: 00-99 (except <TT> is 10, in which

case <tt> is limited to the range 00-22); for amaximum of 1022 terminations allowed for eachgateway

Example: tp/1013

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 267: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 267

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

CICM tp/<vmg>/<nnnn>

where

tp is lower case<vmg> is a number: 0 - 2<nnnn> is a number: 0001 - 1022; always zero padded

Example: tp/1/0002

Each endpoint appears as a virtual media gateway (VMG). For moreinformation, see Nortel CICM Fundamentals ( (NN10044-111)) .

CISCO_2600

CISCO_3600

CISCO_AS5300

CISCO_H323_IOS

Supporting H.323 signalingprotocol

EPG_<n>

where

<n> is a number: 000 - 999

The underscore character "_" is required and the number mustinclude three digits (it must be zero-padded).

Duplicate carrier names on different media gateways are permitted.

Example: EPG_001

H323_PROXY EPG_<n>

where

<n> is a number: 000 - 999

The underscore character "_" is required and the number mustinclude three digits (it must be zero-padded).

Duplicate carrier names on different media gateways are permitted.

Example: EPG_001

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 268: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

268 Maintaining Communication Server 2000 GWC alarms and logs

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

KEYMILE_UMUX POTS table entry: p/nnn

where

p is lower case<nnnn> is an endpoint number: 1 to 480; not padded

(no leading zeros for padding to three digits)This value should match the terminal number.

Example: p/24

Third-party media gateways support flexible allocation of the LENcircuit number and the endpoint terminal number, independently ofthe endpoint name. The circuit number allocated for an endpointmust be unique and in the range of 1 to 480. This value identifiesthe LEN that the system creates. Nortel recommends that the circuitnumber is equal to: <nnn> - 1. For example, if the endpoint is p/1,the associated LEN is: <LGRP> 00 00. Nortel recommends that youcreate the endpoints as blocks of 30 contiguous endpoints to alignwith the configuration of the Keymile UMUX cards.

KEYMILE_UMUX BRI table entry: ba/nnn

where

ba is lower case<nnnn> is an endpoint number: 1 to 224; not padded

(no leading zeros for padding to three digits)This value should match the terminal number.

Example: ba/20

The circuit number must be restricted to the bottom half of theGW’s LGRP (for example, if the LGRPINV entry is KEYM 00 0,the valid range of circuit numbers allows for LENs in the rangeKEYM 00 0 00 00 to KEYM 00 0 05 10). For each BRI endpointthe secondary auto created endpoint is circuit number + 512. Thismeans that for a given BRI LEN, the associated LEN cannot be usedfor other purposes such as a POTS line (for example, KEYM 00 0 0000 provisioned for BRI blocks the use of KEYM 00 0 05 12 for otherpurposes). For ease of maintenance, Nortel recommends that yousplit each block of 30 endpoints into three blocks of eight, with sixunassigned endpoints at the end of the block.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 269: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 269

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

MEDIATRIX

Integrated Access Device(IAD) gateways

Endpoint names must conform to the following rules for small lineendpoints:

• The endpoint name must contain at least one forward slash ("/").

• The name cannot start with “SU”.

• The maximum length of endpoint name is 32 characters.

MG3600_4U Carrier or gateway endpoints are provisioned in groups representingE1 and DS1 levels.

DS1

DS1/<nn>

where

<nn> is a number: 01-80

Example:Carrier name: DS1/03

E1

E1/<nn>

where

<nn> is a number: 01 to 80

Example:Carrier name: E1/03

MG3600_8U Carrier or gateway endpoints are provisioned in groups representingE1 and DS1 levels.

DS1

DS1/<nnn>

where

<nnn> is a number: 001-128

Example:Carrier name: DS1/003

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 270: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

270 Maintaining Communication Server 2000 GWC alarms and logs

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

E1

E1/<nnn>

where

<nnn> is a number: 001-128

Example:Carrier name: E1/003

MGCP line gatewayMGCP_IAD_40 linegateway

Endpoint names must conform to the following rules for small lineendpoints:

• The endpoint name must contain at least one forward slash ("/").

• The name cannot start with “SU”.

• The maximum length of endpoint name is 32 characters.

MILEGATE POTS table entry: p/nnn

where

p is lower case<nnnn> is an endpoint number: 1 to 608; not padded

(no leading zeros for padding to three digits)This value should match the terminal number.

Example: p/24

Third-party media gateways support flexible allocation of the LENcircuit number and the endpoint terminal number, independently ofthe endpoint name. The circuit number allocated for an endpointmust be unique and in the range of 1 to 608. This value identifiesthe LEN that the system creates. Nortel recommends that the circuitnumber is equal to: <nnn> - 1. For example, if the endpoint is p/1,the associated LEN is: <LGRP> 00 00. Nortel recommends that youcreate the endpoints as blocks of 32 contiguous endpoints to alignwith the configuration of the MILEGATE cards.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 271: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 271

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

MOTOROLA MTA

line gateways

Endpoint names must conform to the following rules for small lineendpoints:

• The endpoint name must contain at least one forward slash ("/").

• The name cannot start with “SU”.

• The maximum length of endpoint name is 32 characters.

NORTEL_BCM

Supporting H.323 signalingprotocol

EPG_<n>

where

<n> is a number: 000 - 999

The underscore character "_" is required and the number mustinclude three digits (it must be zero-padded).

Duplicate carrier names on different media gateways are permitted.

Example: EPG_001

NUERA_BTX4K The gateway profile name NUERA_BTX4K supports the NueraBTX4K gateway, which allows provisioning of six DS3 or two opticalOC3 interfaces.

There are 24 DS0 channels for each DS1 interface for a total of4032 DS0 channels for each BTX4K gateway.

OC3 (with DS1 level)

DS/OC3-<u1>/DS1-<u2>/<c>

where

<u1> is a decimal value referring to the particularinstance of oc3: 1-2

<u2> is a decimal value referring to the particularinstance of ds1: 1-84

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: DS/OC3-2/DS1-1Carrier Endpoint name (timeslot): DS/OC3-2/DS1-1/1

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 272: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

272 Maintaining Communication Server 2000 GWC alarms and logs

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

DS3 (with a DS1 level within a logical processor or slot)

DS/S-<u1>/DS3-<u2>/DS1-<u3>/<c>

where

<u1> is a decimal value referring to the particularinstance of s (slot): 1-2

<u2> is a decimal value referring to the particularinstance of ds3: 1-3

<u3> is a decimal value referring to the particularinstance of ds1: 1-28

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: DS/s-2/DS3-3/DS1-1Carrier Endpoint name (timeslot): ds/s-2/ds3-3/ds1-1/1

DS3 (without DS1 framing)

DS/S-<u1>/DS3-<u2>/<c>

where

<u1> is a decimal value referring to the particularinstance of s (slot): 1-2

<u2> is a decimal value referring to the particularinstance of ds3: 1-3

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: DS/S-2/DS3-3/Carrier Endpoint name (timeslot): DS/S-2/DS3-3/1

DS3 (with channelized DS1 levels)

DS/DS3-<u1>/DS1-<u2>/<c>

where

<u1> is a decimal value referring to the particularinstance of ds3: 1-3

<u2> is a decimal value referring to the particularinstance of ds1: 1-28

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 273: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 273

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

Example:Carrier name: DS/DS3-2/DS1-3Carrier Endpoint name (timeslot): DS/DS3-2/DS1-3/1

DS1 (within a logical processor or slot)

DS/S-<u1>/DS1-<u2>/<c>

where

<u1> is a decimal value referring to the particularinstance of s (slot): 1-2

<u2> is a decimal value referring to the particularinstance of ds1: 1-84

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: DS/S-2/DS1-1Carrier Endpoint name (timeslot): DS/S-2/DS1-1/1

DS1

DS/DS1-<u>/<c>

where

<u> is a decimal value referring to the particularinstance of ds1: 1-84

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: DS/DS1-1Carrier Endpoint name (timeslot): DS/DS1-1/1

E1

DS/E1-<u1>/<c>

where

<u1> is a decimal value referring to the particularinstance of e1: 1-63

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-31

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 274: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

274 Maintaining Communication Server 2000 GWC alarms and logs

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

Example:Carrier name: DS/E1-1Carrier Endpoint name (timeslot): DS/DE1-1/1

NUERA_GX Carrier or gateway endpoints are provisioned in groups representingE1 and DS3 levels. Provisioning at this level gives access to alltimeslots for service provisioning. Supported services include ISUPtrunking and PRI trunking.

Carrier names are case sensitive and must be entered in uppercase. The following descriptions include both endpoint groups andindividual endpoint descriptions. Services can be applied at theDS1/E3 timeslot level.

E1/<h1>/<h2>/<g> (H.248/Megaco)

where

<h1> is the two-digit LP (logical processor) number(or slot) of the E1: 1-15

<h2> is the E1 (two digit) port number: 01-32<g> is the channel number: 1-31 (no leading 0),

assigned by the system

H.248/Megaco example:Carrier name: E1/03/05Carrier Endpoint name (timeslot): E1/05/05/1

DS3/DS1

DS1 carriers are provisioned using the same endpoint naming asDS3.

DS3/<h1>/<h2>/<g> (H.248/Megaco)

where

<h1> is the LP (logical processor) number or slot ofthe DS3: 1-15 (2-5 recommended)

<h2> is the DS3 port number<g> is the channel number in the DS3: 1-24, assigned

by the system

H.248/Megaco example:Carrier name: DS3/03/05Carrier Endpoint name (timeslot): DS3/05/05/1

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 275: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 275

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

PVG7K,PVG15KPVG15K_1000PVG15K_PARTIAL

Carrier or gateway endpoints are provisioned in groups representingE1 and DS3 levels. Provisioning at this level gives access to alltimeslots for service provisioning. Supported services include ISUPtrunking and PRI trunking.

Carrier names are case sensitive and must be entered in uppercase. The following descriptions include both endpoint groups andindividual endpoint descriptions. Services can be applied at theDS3/E1 timeslot level.

E1

e1/<h1>/<h2>/<g> (H.248/Megaco)

where

<h1> is the two-digit LP (logical processor) numberor slot of the E1: 1-15 (no leading 0)

<h2> is the E1 (two digit) port number: 01-32<g> is the channel number: 1-31 (no leading 0),

assigned by the system

H.248/Megaco example:Carrier name: e1/03/05Carrier Endpoint name (timeslot): e1/03/05/1

DS3

DS3/<h1>/<h2>/<g> (H.248/Megaco)

where

<b1> is the LP (logical processor) number or slot ofthe DS3: 1-15 (2-5 recommended)

<b2> is the DS3 (single digit) port number: 0-1<c> is the DS3 number: 1-28<d> is the channel number in the DS3: 1-24, assigned

by the system

H.248/Megaco example: Carrier name: DS3/03/05Carrier Endpoint name (timeslot): DS3/05/05/1

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 276: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

276 Maintaining Communication Server 2000 GWC alarms and logs

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

STM-1

STM/<lp>/<p>/1/VC4VC12/1/<k>/<l>/<m>/<e>(H.248/Megaco)

where

<lp> is the LP (logical processor) number or slotof the STM-1 interface: 2-15 (slots 2-5 arerecommended)

<pp> is the two-digit port number: 00-03<p> is the one-digit port number: 0-3<k> is the one-digit TUG-3 number within a VC4: 1-3<l> is the one-digit TUG-2 number within a TUG-3:

1-7<m> is the one-digit TU number within a TU: 1-3<e> is the VC12 channel/timeslot: 1-31 (no leading

0)

Hard-coded values 1/VC4VC12/1 (H.248) indicate the STM carriertype, multiplexing within the STM-1 frame, and the AUG within theSTM-1 frame.

H.248/Megaco example:Carrier name: STM/2/1/1/VC4VC12/1/3/6/1Carrier Endpoint name (timeslot): STM/2/1/1/VC4VC12/1/3/6/1/1

PVG15K_MEGACO, PVG15K_PARTIAL_MEGAGO, andPVG_VSP3_MEGACO gateways also support the V5.2 service typeand this STM-1 (Megaco) carrier name format in V5.2 services.

OC-3

STS/<lp>/<p>/3/VT15/<t>/<l>/<m>/<e>(H.248/Megaco)

where

<lp> is the LP (logical processor) number or slot ofthe OC-3 interface: 1-15 (recommended slots2-5)

<pp> is the (two digit) port number: 00-03<p> is the (one digit) port number: 0-3<jj> is the (two digit) STS-1 number within the

STS-3: 01-03

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 277: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 277

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

<l> is the (one digit) VT group number within STS-1:1-7

<m> is the (one digit) VT number within a VT: 1-4<t> is the (one digit) STS-1 number within the STS-3:

0-3<e> is the VT1.5 channel/timeslot: 1-24 (no leading

0)

Hardcoded value /3/VT15/ (H.248) indicates the STS carrier typeand the multiplexing within the OC-3 carrier.

H.248/Megaco example:Carrier name: STS/2/1/3/VT15/1/6/1Carrier Endpoint name (timeslot): STS/2/1/3/VT15/1/6/1/1

PVG_VSP4E STM/<l>/<p>/1/VC4VC12/1/<t>/<u>/<v>

where

<l> is the LP (logical processor) number or slot ofthe STM interface: 2-15

<p> is the one-digit port number: 0-3<t> is the one-digit TUG-3 number within a VC4: 1-3<u> is the one-digit TUG-2 number within a TUG-3:

1-7<v> is the one-digit TU number within a TU: 1-3

The maximum capacity is 31 endpoints.

Hard-coded value of 1/VC4VC12/1 indicates the STM carrier type,multiplexing within the STM-1 frame, and the AUG within the STM-1frame.

Example:Carrier name: STM/2/1/1/VC4VC12/1/3/6/1Carrier Endpoint name (timeslot): STM/2/1/1/VC4VC12/1/3/6/1

STS/<l>/<p>/3/VT15/<t>/<v>/<w>

where

<l> is the LP (logical processor) number or slot ofthe STS interface: 1-15

<p> is the port number: 0-3<t> is the STS-1 number: 1-3<v> is the VT group number: 1-7<w> is the VT number: 1-4

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 278: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

278 Maintaining Communication Server 2000 GWC alarms and logs

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

The maximum capacity is 24 endpoints.

Hard-coded value of 3/VT15 indicates the STS carrier type.

Example: Carrier name: STS/2/1/3/VT15/1/3/4Carrier Endpoint name (timeslot):STS/2/1/3/VT15/1/3/4

SIPVOICE /<SITE>/<FFF>/<G>/<TT tt>

where

<SITE> is a name defined in table SITE configured onthe Core

<FFF> is a frame number: 0 - 511<G> is a group number: 0 - 9<TT tt> is a terminal number: 00 00 - 1023

Example: /ABCD/10/9/01 20

SUCCESSION_1000

Supporting H.323 signalingprotocol

EPG_<n>

where

<n> is a three-digit number: 000 - 999

The underscore character "_" is required and the number mustinclude three digits (it must be zero-padded).

Duplicate carrier names on different media gateways are permitted.

Example: EPG_001

TGCP GWCs support TGCP media gateways with

• DS1 interfaces

• DS1 interfaces within a logical processor or slot

• OC3 interfaces with channelized DS3 and DS1 levels

• OC3 interfaces with channelized DS3 and DS1 levels within alogical processor or slot

• DS3 interfaces with a channelized DS1 level within a logicalprocessor or slot DS3 interfaces without DS1 framing

• DS3 interfaces with a channelized DS1 level

• E1 interfaces

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 279: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 279

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

The following abbreviations are used in TGCP carrier and endpointformats:

• u = unit number

• c = channel number

The gateway profile name TGCP supports third party mediagateways using TGCP signaling protocol.

DS1

ds/ds1-<u>/<c>

where

<u> is a decimal value referring to the particularinstance of ds1: 1-68

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: ds/ds1-1Carrier Endpoint name (timeslot): ds/ds1-1/1

DS1(within a logical processor or slot)

ds/s-<u1>/ds1-<u2>/<c>

where

<u1> is a decimal value referring to the particularinstance of s (slot): 1-28

<u2> is a decimal value referring to the particularinstance of ds1: 1-28

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: ds/s-2/ds1-1Carrier Endpoint name (timeslot): ds/s-2/ds1-1/1

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 280: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

280 Maintaining Communication Server 2000 GWC alarms and logs

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

OC3 (with DS3 and DS1 levels)

ds/oc3-<u1>/ds3-<u2>/ds1-<u3>/<c>

where

<u1> is a decimal value referring to the particularinstance of oc3: 1-28

<u2> is a decimal value referring to the particularinstance of ds3: 1-28

<u3> is a decimal value referring to the particularinstance of ds1: 1-28

<channel #> is a decimal value indicating the channelnumber at the lowest level in the hierarchy:1-24

Example:Carrier name: ds/oc3-2/ds3-3/ds1-1Carrier Endpoint name (timeslot): ds/oc3-2/ds3-3/ds1-1/1

OC3(with DS3 and DS1 levels within a logical processor or slot)

ds/s-<u1>/oc3-<u2>/ds3-<u3>/ds1-<u4>/<c>

where

<u1> is a decimal value referring to the particularinstance of s (slot): 1-28

<u2> is a decimal value referring to the particularinstance of oc3: 1-28

<u3> is a decimal value referring to the particularinstance of ds3: 1-28

<u4> is a decimal value referring to the particularinstance of ds1: 1-28

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: ds/s-2/oc3-2/ds3-3/ds1-1Carrier Endpoint name (timeslot): ds/s-2/oc3-2/ds3-3/ds1-1/1

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 281: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 281

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

DS3(with a DS1 level within a logical processor or slot)

ds/s-<u1>/ds3-<u2>/ds1-<u3>/<c>

where

<u1> is a decimal value referring to the particularinstance of s (slot): 1-28

<u2> is a decimal value referring to the particularinstance of ds3: 1-28

<u3> is a decimal value referring to the particularinstance of ds1: 1-28

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy:1-24

Example:Carrier name: ds/s-2/ds3-3/ds1-1Carrier Endpoint name (timeslot): ds/s-2/ds3-3/ds1-1/1

DS3(without DS1 framing)

ds/s-<u1>/ds3-<u2>/<c>

where

<u1> is a decimal value referring to the particularinstance of s (slot): 1-28

<u2> is a decimal value referring to the particularinstance of ds3: 1-28

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-24

Example:Carrier name: ds/s-2/ds3-3/Carrier Endpoint name (timeslot): ds/s-2/ds3-3/1

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 282: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

282 Maintaining Communication Server 2000 GWC alarms and logs

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

DS3(with channelized DS1 levels)

ds/ds3-<u1>/ds1-<u2>/<c>

where

<u1> is a decimal value referring to the particularinstance of ds3: 1-28

<u2> is a decimal value referring to the particularinstance of ds1: 1-28

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy:1-24

Example:Carrier name: ds/ds3-2/ds1-3Carrier Endpoint name (timeslot): ds/ds3-2/ds1-3/1

TGCP

(continued)

E1

ds/e1-<u1> /<c>

where

<u1> is a decimal value referring to the particularinstance of e1: 1-68

<c> is a decimal value indicating the channel numberat the lowest level in the hierarchy: 1-31

Example:Carrier name: ds/e1-1Carrier Endpoint name (timeslot): ds/e1-1/1

TOUCHTONE_NN

line gateways

Endpoint names must conform to the following rules for small lineendpoints:

• The endpoint name must contain at least one forward slash ("/").

• The name cannot start with “SU”.

• The maximum length of endpoint name is 32 characters.

UAS

Audio servers (includingNortel Media Server 2000Series)

Endpoints are not specified during provisioning of audio servers.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 283: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 283

Table 42Carrier and endpoint names (cont’d.)

Media gateway type(based on profile) Carrier name and endpoint names or identifiers

UE9000MG

(Media Gateway 9000)

For non-ABI (access bridge interface) lines:

tp/<slot>/<circuit>

where

tp is lower case<slot> is a number: 2-9<circuit> is a number: 14-21

Example: tp/2/14

Each endpoint appears as a virtual media gateway (VMG).

The <slot> and <circuit> values constitute the SLOT and theCIRCUIT part of the line equipment number (LEN) for the gateway.

For MG 9000 ABI lines:

tp/channel/<channelID>

where

tp is lower case<channelID> is a number between 000 and 511

WESTELL

Supporting H.323 signalingprotocol

EPG_<n>

where

<n> is a three-digit number: 000 - 999

The underscore character "_" is required and the number mustinclude three digits (it must be zero-padded).

Duplicate carrier names on different media gateways are permitted.

Example: EPG_001

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 284: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

284 Maintaining Communication Server 2000 GWC alarms and logs

Viewing media proxy associations

View all current associations of the selected media proxy. A media proxycan be associated as a default media proxy with one or more GatewayControllers (GWC) or it can belong to one or more preferred media proxygroups, or both.

Use this procedure when you wish to view the list of all GWCs and allpreferred media proxy groups with which the selected media proxy iscurrently associated.

Procedure steps

Step Action

1 At the Communication Server 2000 GWC Manager client

From the Device Types menu, select the Gateway Controllerfolder.

2 In the Network Devices section, click the Media Proxies tab,then click the newly displayed Media Proxies tab.

3 Select the proxy you wish to view. Your selection is highlighted.

4 Click the Properties button.

The Media Proxy Description dialog box displays all thepreferred media proxy groups to which the selected media proxybelongs and all GWCs with which the proxy is associated.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 285: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 285

5 Click OK to close the dialog box.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 286: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

286 Maintaining Communication Server 2000 GWC alarms and logs

Viewing media proxies associated with a GWC node

Use this procedure to view the configuration data for media proxiescurrently associated with a selected Gateway Controller (GWC) node.

The same media proxies can also belong to preferred media proxy groups.Do not use this procedure to view media proxy groups or to view the list ofGWCs with which a selected media proxy is associated.

Procedure steps

Step Action

At the Communication Server 2000 GWC Manager client1 At the Communication Server 2000 Management Tools window,

click the Gateway Controller folder from the Device Types menu.

2 From the Contents of: Gateway Controller frame, select theGWC node that has a media proxy you wish to view.

3 Click the Provisioning tab in the GWC node view.

4 Click the Media Proxies tab. The displayed table lists thename, IP address, protocol and protocol version and the groupsassociated with the selected GWC to which the selected mediaproxy belongs. The <default> entry means that the selectedmedia proxy is one of the default media proxies for the selectedGWC information about media proxies currently associated withthe selected GWC node.

If you wish to view all GWCs and all media proxy groups withwhich a particular media proxy is currently associated, seeprocedure “Viewing media proxy associations” (page 284).

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 287: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 287

Taking a manual GWC software image

Use this procedure to save a software image to the Communication Server2000 Core Manager or Core and Billing Manager (CBM).

Use this procedure as a part of upgrading GWC software for an office oras a part of maintenance activity.

Take an image after all patches have been applied to GWC software.Refer to your site operating procedures for information about soak timeand how many patches to apply before taking an image. In the absence ofthis information, Nortel recommends taking an image immediately afterapplying R or P status patches.

ATTENTIONDo not invoke the Save Image function during patching activities. Doing so cancause an invalid or incomplete image to be taken.

ATTENTIONA procedure also exists to enable the auto-imaging of a GWC software loadonce daily. For more information about enable or disable GWC softwareauto-imaging, see Upgrading the Gateway Controller (NN10196-461).

Procedure steps

Step Action

1 Log on to the Communication Server 2000 Core Manager orCBM as the root user.

2 Change directory to the GWC software directory.

# cd /swd/gwc

3 Optionally, you can change the permissions for the GWCdirectory and its file contents by typing

# chmod 777 gwc/*

and pressing the Enter key.

4 Copy the existing GWC software load file to a backup.

# cp pgc06as.imag pgc06as.imag.bak

5 At the Communication Server 2000 Management Tools window,click on the Gateway Controller folder from the Device Typesdirectory tree in the far left frame.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 288: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

288 Maintaining Communication Server 2000 GWC alarms and logs

6 From the Contents of: Gateway Controller frame, select theappropriate GWC node from which you want to take an image.

7 Select the Maintenance tab to view the Maintenance panel.

8 In the Maintenance panel, identify the GWC card in the nodethat has an upgraded load already installed. Click the SaveImage button for that card to save the software image back tothe Communication Server 2000 Core Manager or CBM.

The Save Image command overwrites the existing GWCsoftware load file on the Communication Server 2000 CoreManager or CBM.

ATTENTIONDo not invoke the Save Image function during patching activities.Doing so can cause an invalid or or corrupt image to be saved.

9 At the following warning message, click OK to continue withsaving the GWC card’s software image on the CommunicationServer 2000 Core Manager. To abort the operation, click Cancel.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 289: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 289

ATTENTIONIf the load file name is a link on the file system, the link is replacedwith a file of the same name. Nortel does not support using links toload files.

--End--

Variable definitionsVariable Value

<sw_load> The software load to be copied for backup (for example, pgc06as.imag pgc06as.imag.bak). Note that you can use any name for the backup filename.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 290: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

290 Maintaining Communication Server 2000 GWC alarms and logs

Reconfiguring NTP service

Use this procedure to change the network time protocol (NTP) source forall the cards in the SAM21 shelf to a new NTP server. Reconfiguring theNTP server does not affect timing for call processing or signaling, theintent is to synchronize the timestamps on log reports.

CAUTIONPossible service interruptionReconfiguring the NTP service requires locking and unlockingthe cards in the shelf. Locking and unlocking the ShelfControllers does not affect service, but locking and unlockingnon-system slot (NSS) cards can disrupt service.

Gateway Controllers (GWC) do not require locking andunlocking to update the NTP server information. GWCs requirea Warm Swact.

For information about updating the NTP service on the CallAgents, see Nortel Call Agent Configuration (NN10109-511).

For offices with a Communication Server 2000-Compact, this proceduredoes not change the NTP source for STORage Management (STORM)units that are on the SAM-XTS hardware platform (STORM-IA).

Procedure steps

Step Action

1 At the Communication Server 2000 SAM21 Manager client

From the subnet view, click the Configuration menu and selectModify, Existing SAM21 Network Element, then select theSAM21 shelf to modify.

2 In the displayed Reprovision <SAM21> dialog box, replace thevalue in the Primary NTP and Secondary NTP boxes with thenew NTP server addresses.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 291: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 291

3 Lock and unlock the inactive Shelf Controller.

Two critical alarms are raised after the lock operation. Thealarms clear after the unlock operation.

If the Shelf Controllers are provisioned with ATM interfaces,verify that the inactive Shelf Controller does not carry the activeATM link. If necessary, switch link activity at the far end node.

4 Switch activity by selecting Swact from the card context menu.If the Shelf Controllers are provisioned with ATM interfaces,perform the Swact during a low traffic period.

5 Lock and unlock the newly inactive Shelf Controller.

6 Before locking any card, refer to the documentation for the cardtype and determine any precautions to take before locking thecard.

Lock and unlock each of the cards in the SAM21 shelf.

For GWCs, open the GWC Manager. For more information, see“Job aid” (page 291).

--End--

Job aidSome network components require different maintenance activities toupdate the NTP server information.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 292: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

292 Maintaining Communication Server 2000 GWC alarms and logs

Gateway ControllerTo update the NTP server information in a GWC, warm swact one unit ofthe GWC node and then warm SWACT again to update the second unit inthe node. Open the GWC Manager by selecting Show Node View fromthe Shelf View.

When the GWC Manager opens, click the WarmSwact button on theMaintenance pane to update the NTP information. Warm Swact the GWCnode twice so that both GWC units are updated. Perform these WarmSwacts for all GWC nodes in the SAM21 shelf.

After any unlock, return to service (RTS), or Swact to a Gateway Controller(GWC), the GWC makes its first NTP requests to the primary NTP server.If the service is unavailable, the GWC makes a final NTP service requestto the Communication Server 2000 Management Tools server. For theSN06 release, the secondary NTP server is ignored.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 293: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 293

Using the Table Editor to Delete a Tuple from a Table

To delete certain provisioned entities in the XA-Core component of theCommunication Server 2000, you must use the table editor to delete tuplesfrom tables.

Procedure steps

Step Action

1 At the MAP terminal, start the table editor. At the user interfaceprompt on any MAP screen type.

>TABLE <table-name>

and press the Enter key

2 Use the POS command to move to the tuple that you want todelete. Type

>POS <key-value>

and press the Enter key

For example, if you wanted to delete the INCOMING_TRUNKtuple from table CLLI, you would enter

>POS INCOMING_TRUNK

Example of system response, continuing the example:

INCOMING_TRUNK 250 10 INCOMING_IT

250, 10, and INCOMING_IT are the values of the fields thatcompose a tuple in the CLLI table: ADNUM, TRKGRSIZ, andADMININF.

3 Delete the tuple.

>DEL

and press the Enter key.

Example of system response:

TUPLE TO BE DELETED:INCOMING_TRUNK 250 10 INCOMING_ITENTER Y TO CONFIRM, N TO REJECT OR E TO EDIT.

4 Confirm the deletion.

>Y

and press the Enter key.

Example of system response:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 294: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

294 Maintaining Communication Server 2000 GWC alarms and logs

TUPLE DELETED:WRITTEN TO JOURNAL FILE AS JF NUMBER 567

5 Exit from the table editor.

>QUIT

and press the Enter key.

--End--

Variable definitionsVariable Value

<key-value> Is the key value identifying the tuple.

<table-name> Is the name of the table from which you want to delete a tuple.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 295: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 295

Changing the APS Oracle account password

Use this procedure to change the default password assigned to the Oracleaccount, to increase system security.

Procedure steps

Step Action

1 In a Telnet connection to the APS server open an Xterm windowand log on using the "root" logon and password.

2 If you are unsure whether the password has been changed,obtain the current Oracle account password by entering thefollowing command:

getNTDBpasswd.ksh

3 Become the "Oracle" user by entering the following command:

su - oracle

4 Enter the following command to run the script that enables youto change the password:

/usr/ntdb/uas/scripts/setNTDBpasswd.ksh

5 At the prompt, enter the current APS Oracle account password.

This is the password that you displayed in 2.

6 At the prompt, enter the new APS Oracle account password.

7 At the prompt, reenter the new APS Oracle account password.

The system changes the password in UNIX and in the Oracledatabase.

8 Enter the following command to exit from the Oracle useraccount:

exit

This causes you to become the "root" user again.

9 Restart the APS processes:

/opt/uas/aps/scripts/killDbServer.shservrestart APS

A message eventually displays indicating that APS restartedsuccessfully.

10 Enter the following commands to complete the passwordchange:

cd /

. /etc/profile

. ./.profile

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 296: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

296 Maintaining Communication Server 2000 GWC alarms and logs

11 You can now check the password change you have made byentering the following command:

getNTDBpasswd.ksh

The system displays the current Oracle account password.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 297: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 297

Viewing GWC platform hardware alarms

View GWC platform hardware alarms as a part of alarm clearing activityat the Communication Server 2000 SAM21 Manager or as a secondarysource of diagnostic information for GWC service-related alarms.

This procedure provides access to platform-related alarms, such ascommunication over Ethernet, operating system resource availability, andhardware faults.

Procedure steps

Step Action

1 At the Communication Server 2000 SAM21 Manager client

Right-click the card in an alarm condition and select Card Viewfrom the menu.

The Card View window opens.

2 Click the Alarms tab to display the alarm details.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 298: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

298 Maintaining Communication Server 2000 GWC alarms and logs

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 299: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 299

Viewing GWC PM logs

View GWC PM logs as a part of scheduled maintenance and as asecondary source of diagnostic information.

This procedure describes how to access service-related peripheral module(PM) logs generated by the GWC and forwarded to the Core.

Procedure steps

Step Action

1 At the MAPCI interface, enter the following command:

logutil

2 Retrieve the latest PM log:

open pm

3 For more information about commands available in logutil, enter:

print logutildir

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 300: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

300 Maintaining Communication Server 2000 GWC alarms and logs

Viewing GWC logs in syslog files

This procedure describes how to access GWC logs stored in the syslogfiles on the CS 2000 Management Tools server by logging into the CS2000 Management Tools server. You can also access syslog files usingthe Integrated Element Management System (IEMS) GUI. For moreinformation about viewing the audit and the security logs, see Nortel IEMSAdministration and Security (NN10336-611).

This procedure also describes how to search for specific entries in thesyslog files.

Use this procedure as a part of scheduled maintenance and as asecondary source of diagnostic information.

Prerequisites• You need the root user ID and password to log into the CS 2000

Management Tools server.

Procedure steps

Step Action

1 At your workstation, establish a login session to the CS 2000Management Tools server using one of the following methods:

• telnet (unsecure)For instructions, go to step 2.

• ssh (secure)

ATTENTIONUse this option only if your workstation platform supports ssh.Otherwise, your attempt to log in fails.

For instructions, go to step 7.

2 Enter:

telnet <server>

3 When prompted, enter you user ID and password.

The system prompt changes to a dollar sign ($).

4 Change to the root user. Enter:

su -

5 When prompted, enter the root password.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 301: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 301

6 Go to step 9 to continue the procedure.

7 Enter:

ssh -1 root <server>

8 When prompted, enter the root password.

The system prompt changes to a dollar sign ($).

9 At the CS 2000 Management Tools client, access the directorylevel where the syslog files reside. Enter:

cd /var/log

10 List the directory content. Enter:

ls

The system displays a list of different log files, such as,customerlog, securitylog, and so on. These files are appendedwith numbers, for example "customerlog.0". The files with thelower numbers are the newer files.

For a list of GWC syslog logs, see “Job aid” (page 302).

11 If the file that you want to view is zipped (has an extension .gz),unzip it using the following command. Otherwise, go to the nextstep.

To unzip the file, enter:

gunzip <log_filename.gz>

Example

$ gunzip securitylog.1.gz

The file changes to a readable file securitylog.1.

12 Enter this command to display the entire content of a log file:

cat <log_filename> |more

Example$ cat customerlog.0 |more

Press the space bar to scroll through the file if it is larger thanthe screen can display.

OR

Enter this command to search and display specific content of alog file:

cat <log_filename> |grep <search_string>

Examples

cat customerlog.0 |grep GWC309

cat securitylog.1 |grep KRB_LOG

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 302: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

302 Maintaining Communication Server 2000 GWC alarms and logs

13 If you want to print the file, contact your site system administratorfor assistance with using UNIX print commands and with locatinga printer connected to your network.

--End--

Variable definitionsVariable Value

<log_filename> The name of the log file that you wanto display.

<log_filename.gz> The name of the log file that you wantto unzip.

<search_string> The text that you want to search for.For example, KRB (to search forlogs associated with the Kerberosapplication)

<server> The hostname or IP address of theserver.

Job aidThe following tables describe the syslog logs in the /var/log directory thatcontain entries relevant to the GWC.

Table 43Syslog logs containing GWC entries

Log type DescriptionExamples of log filenames

Audit log Records the actions taken byusers on the system, includingsome of the parameters theyused.

Records all the user logins andlogouts from any source usingftp, proftp, telnet, sftp, scp, orssh. The successful logins andlogouts are logged to auditlog(/var/log/auditlog).

auditlog

auditlog.0

auditlog.1

Customer log Records all alarms the system hasreceived.

customerlog

customerlog.0

customerlog.1

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 303: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 303

Table 43Syslog logs containing GWC entries (cont’d.)

Log type DescriptionExamples of log filenames

Debug log Records debug information for CS2000 Management Tools networkcomponents to help detect anunderlying problem.

debuglog

debuglog.0

PTM log Contains a record of all the SNMPtraps received by the system.

ptmlog

ptmlog.1

ptmlog.2

Security log Records failed actions taken byusers on the system. Securitylogfile also includes fault-relatedlogs for Kerberos and IKE/IPSecsecurity.

Records all the user logins andlogouts from any source using ftp,proftp, telnet, sftp, scp, or ssh.The login failures are logged tosecuritylog (/var/log/securitylog).

securitylogsecuritylog.0

Use the following table to interpret the syslog application logs on theredirecting media gateway controller (RMGC).

Table 44GWC syslog application logs

Application logdescription Cause or condition Action

RMGC: SuccessfulCount: xFailed Count: y

The RMGC application produces asyslog performance report once anhour. (See the debug log in /var/log.)It contains the counts of the numberof RSIPs processed successfully (x)and the number failed (y). The countsare cumulative, so that to calculate thenumber of successful/failed RSIPs,it is not necessary to parse eachand every log but just to subtract thecounts from the previous log to derivethe counts between the current logand the previous.

No actionrequired

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 304: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

304 Maintaining Communication Server 2000 GWC alarms and logs

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 305: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

305.

Deleting a lineThis section contains a procedure you use to delete a line on theCommunication Server 2000.

Navigation

• “Removing a line” (page 306)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 306: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

306 Deleting a line

Removing a line

Delete a line on the Communication Server 2000, when necessary.

Use one of these methods to delete a line:

• Establish a Telnet connection to OSSGate and enter a SERVORD+command.

ATTENTIONSERVORD+ commands are the subset of SERVORD commands that canbe entered using the OSSGate interface.

For detailed information about SERVORD commands, see NADMS-100 SERVORD Reference Manual (297-8021-808) in the NorthAmerican market or Nortel DMS-100 MMP SERVORD ReferenceManual Volume 1 (297-9051-8081) in the international market.

• Use an operations support system (OSS), which must be purchasedseparately. The OSS has its own user interface. The OSS acts as theintermediary between you and OSSGate

ATTENTIONAn OSS must be purchased separately.

This procedure explains how to delete a line through a Telnet session toOSSGate and entering SERVORD+ commands one by one. If you havean OSS, see the documentation provided with the OSS for information onprovisioning lines.

Prerequisites• You must have the Telnet application.

• You must know either the host name or the IP address of the serveron which OSSGate is running. If in doubt, ask the local networkadministrator.

• You must know the number of the port used by OSSGate. If in doubt,ask the local network administrator.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 307: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 307

Procedure stepsCAUTIONLoss of serviceWhen deleting lines, use only the user interfaces referred to inthis procedure. Do not use the MAP interface to modify any ofthe data schema tables directly. Modifying these tables directlyvia the MAP will result in service-impacting corruption that willrequire Nortel assistance to resolve.

Step Action

1 At the PC connected to the CS LAN

If you intend to delete the line through Telnet to OSSGate andentering a SERVORD command go tostep 3.

2 If you intend to delete the line by using an operations supportsystem (OSS) go to step 22.

3 Initiate a Telnet session to OSSGate. Type the followingcommand at the system prompt

Telnet <OSSGate-server-name> <port-number>

and press the Enter key

ATTENTIONThe default port number is 10023.

For example, type

Telnet wcn0s5jk 10023

and press the Enter key.

Example of system response:

Trying 47.142.94.80...Connected to wcn0s5jk.

Escape character is ’^]’.

Enter username and password

4 Enter your user name and password. Type

<username> <password>

and press the Enter key.

For example, type

user1 passwd

and press the Enter key.

Example of system response:

user1 logged in on 2002/2/2 at 11:51:55.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 308: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

308 Deleting a line

************************************************ **** OSS Gateway **** **** This is a PRIVATE Database. **** **** All activity is subject to monitoring. **** ***Any UNAUTHORIZED access or use is PROHIBITED*** and may result in PROSECUTION. **** ************************************************

>

5 You have connected to OSSGate, so the system is displaying aprompt, the ">" character, in the Telnet window on your screen.Now you put the Telnet session into CI mode.

6 Hold down the control key and type B.

Example of system response:

?

7 At the question-mark prompt, type

mode ci

and press the Enter key.

Example of system response:

Mode is CI.

>

8 If you are deleting a SIP lin go to step 11.

9 If you are deleting a line on a CICM gateway go to step 13.

10 If you are deleting a line other than a SIP line or a line on aCICM gateway go to step 14.

11 Delete the SIP line by entering the OUT command at the ">"prompt.

The following figure shows an example of the OUT command todelete a SIP line.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 309: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 309

Figure 53Example of the OUT command to delete a SIP line

For detailed information on the format of the LEN identifying aSIP line, see the module that explains how to provision SIP lines.That module will be available if your solution supports SIP lines.

12 Delete the line on the CICM gateway by entering the OUTcommand at the ">" prompt.

The following figure shows an example of the OUT command todelete a line on a CICM gateway.Figure 54Example of the OUT command to delete a line on a CICM gateway

For detailed information on the format of the LEN identifying aline on a CICM gateway, see the module that explains how toprovision lines on CICM gateways. That module will be availableif your solution supports lines on CICM gateways.

13 Go to step 16.

14 Delete the line by entering the OUT command at the ">" prompt.

The following figure shows an example of the OUT command.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 310: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

310 Deleting a line

Figure 55Example of the OUT command

ATTENTIONFor detailed information on the format of the media gateway endpointname in the OUT command, see “Media gateway endpoint names”(page 172).

When the system processes the OUT command, it deletes theline from table LNINV, and cancels service on the line equipmentnumber (LEN). It also deletes the endpoint from the gatewaycontroller.

15 Disconnect from OSSGate.

16 Hold down the control key and type B.

Example of system response:

?

17 At the question-mark prompt, type

logout

and press the Enter key.

Example of system response:

user1 logged out.

>

18 Terminate the Telnet session.

19 Hold down the control key and type B.

Example of system response:

?

20 At the question-mark prompt, type

clearconv

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 311: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 311

and press the Enter key.

Example of system response:

SESSION TERMINATED.

Connection closed by foreign host.

21 If you are going to use an operations support system (OSS) todelete a line, ask your system administrator for the instructionsfor accessing the OSS and for the information about the OSSinterface

22 Access the OSS following the instructions of your systemadministrator.

23 Delete the line using the OSS interface.

--End--

Variable definitionsVariable value

<OSSGate-server-name> The host name of the server on which OSSGateis running.- Is the IP address of the server on whichOSSGate is running.

<port-number> The server port used by OSSGate

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 312: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

312 Deleting a line

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 313: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

313.

Maintaining GWC alarms and logsThis section lists the procedures you use to view Gateway Controller(GWC) alarms and logs. The information you gather from the alarms andlogs allows you to monitor and resolve issues.

Maintaining GWC alarms and logs Navigation

• “Viewing GWC service alarm history” (page 314)

• “Viewing GWC service alarms” (page 316)

• “Viewing and interpreting the operational status of a GWC node” (page318)

• “Viewing switch-wide GWC status” (page 322)

• “Monitoring operational and activity status of GWCs” (page 329)

• “Restarting or rebooting a GWC card” (page 331)

• “Restarting GWC card services” (page 333)

• “Performing a Communication Server 2000 data integrity audit” (page335)

• “Recovery of the GWC320 certificate expiry alarm” (page 338)

• “Increasing the size of a logical volume” (page 340)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 314: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

314 Maintaining GWC alarms and logs

Viewing GWC service alarm history

This procedure allows you to query the GWC service-related alarms thatalready occurred, and permits alarm display filtering based on GWC unit,alarm severity, alarm category and date/time.

Use this procedure as a part of scheduled maintenance and as a primarysource of fault diagnostic information for GWC services.

Procedure steps

Step Action

At the Communication Server 2000 GWC Manager client1 From the Communication Server 2000 Management Tools

window menu, select the Fault and Alarm History.

2 Review the alarms displayed.

The colors to the left of the alarm display provide a visualindication of alarm severity:

• yellow - warning

• orange - minor

• red - major and critical

3 Click Refresh to update the alarm list.

4 Click the Next Page button (if applicable) to view more alarms.

5 Select the Advanced Filters: Enable check box to open theAdvanced History Filters dialog box and filter alarms based onselected criteria.

If the check box is already selected, click the Settings button toopen the Advanced History Filters dialog box.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 315: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 315

6 In the View list, select the GWC units to be excluded (filtered).You can press and hold the <Shift> key to select multiple GWCunits.

7 Click the Remove > button to place the selected GWC units inthe Exclude (filtered) list. Click the Remove All >> button toplace all GWC units in the Exclude (filtered) list.

If necessary, select GWC units in the Exclude list. Then, clickthe < Add button to place the selected GWC units in the Viewlist. Click the << Add All button to place all GWC units in theView list.

8 De-select the Severity check boxes to exclude an alarm type.Any alarm severities that remain selected will not be filtered. Ifrequired, click the Select All button to select all alarm severitycheck boxes.

9 De-select the alarm Category check boxes to exclude an alarmtype. Any alarm categories that remain selected will not befiltered. If required, click the Select All button to select all alarmcategory check boxes.

10 If you wish to filter according to a specific range of dates, typethe date range in the format, yyyy/mm/dd.

11 If you wish to filter according to a specific time frame, type thetime frame in the format, hh:mm.

12 After selecting filter criteria, click the Apply button to displayselected alarms and keep the Advanced History Filters dialogbox open. Click OK to display selected alarms and close theAdvanced History Filters dialog box.

13 To exit the Alarm History, click the File menu at the top of thescreen and select Close.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 316: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

316 Maintaining GWC alarms and logs

Viewing GWC service alarms

Use this procedure to access service-related alarms that are currentlyactive on the GWC application. The Alarm Manager displays alarms asthey occur (in real time). This option also permits alarm display filteringbased on GWC unit and alarm category.

Use this procedure as a primary source for fault diagnostic informationrelated to GWC services.

Procedure steps

Step Action

At the Communication Server 2000 GWC Manager client1 At the Communication Server 2000 Management Tools window,

click the Fault menu and select Alarm Manager to open theAlarm Manager window.

2 From the Alarm Manager window, review the alarms displayed.

The colors to the left of the alarm display provide a visualindication of alarm severity:

• yellow - warning

• orange - minor

• red - major or critical

See section “Viewing GWC service alarms” (page 316) for detailsabout the alarm types displayed, including appropriate actions todiagnose and resolve the alarm condition.

3 Click Refresh List to update the alarm list.

4 Click the Details button to review specific details about an alarm.

5 Click the Advanced Filters: Enable check box to open theAdvanced Filters dialog box and filter alarms based on selectedcriteria.

If the check box is already selected, click the Settings button toopen the Advanced Filters dialog box.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 317: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 317

6 In the view list, select the GWC units to be excluded (filtered).You can press and hold the <Shift> key to select multiple GWCunits.

7 Click the Remove > button to place the selected GWC units inthe Exclude (filtered) list.

Click the Remove All >> button to place all GWC units in theExclude (filtered) list.

If required, select GWC units in the Exclude list. Then, clickthe < Add button to place the selected GWC units in the View(unfiltered) list.

Click the << Add All button to place all GWC units in the View(unfiltered) list.

8 De-select the Alarm Category check boxes to exclude (filter) analarm type for the GWC units listed in the Exclude box.

Any alarm categories that remain selected will be included (willnot be filtered) for the GWC units in the Exclude list.

9 Click the Apply button to display selected alarms and keep theAdvanced Filters dialog box open. Click OK to display selectedalarms and close the Advanced Filters dialog box.

10 To exit the Alarm Manager, click the File menu at the top of thescreen and select Close.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 318: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

318 Maintaining GWC alarms and logs

Viewing and interpreting the operational status of aGWC node

Use this procedure to determine the operational status of a selectedGateway Controller (GWC) node using the Communication Server 2000GWC Manager. Use this procedure as a primary source of informationabout the operational status of a GWC card or GWC node.

Procedure steps

Step Action

At the Communication Server 2000 GWC Manager client1 At the Communication Server 2000 Management Tools window,

click the Gateway Controller folder from the Device Types menu.

2 From the Contents of: Gateway Controller frame, select theGWC node that you wish to view.

3 Click the Maintenance tab.

The GUI displays the Maintenance panel with two independentstatus views, one for each of the GWC cards in the node.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 319: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 319

4 See Table 45 "Communication Server 2000 GWC Managerstatus fields" (page 319) following this procedure to interpret theGWC card (unit) status fields.

If the selected GWC loses communication with the GWCManager, the client does not provide an accurate status of theGWC node. You can verify the call processing status of a GWCnode using the MAPCI interface.

--End--

Job aidThe following table describes the GWC card (unit) status fields.

Table 45Communication Server 2000 GWC Manager status fields

Status field Possible values Meaning

Administrativestate:

locked The unit is prohibited, administratively,from providing service to users.

A status of "locked" on the Communication Server 2000 GWC Managerindicates that the software application on the card is no longer performingits primary call processing function, but the card is still running. (The callprocessing function has been "busied", but underlying maintenance andcommunications activities are still functioning.)

A status of "locked" on the Communication Server 2000 SAM21 Managerindicates that the hardware is locked to ROM level, and the softwareapplication is no longer running.

unlocked The unit is permitted, administratively, toprovide service to users.

Operational state: enabled The unit is partially or fully providingservice to users.

disabled The unit is not operating or providingservice to users. If the Administrative statefor this unit is "locked", then the unit hasbeen manually busied. If the Administrativestate for this unit is "unlocked", then theunit has been busied by the system.

Activity state: active The unit is currently providing end userservices. This is the state of the node asseen by other network elements.

standby The unit is not providing end user servicesbut can be switched to Active at any time ifthe active (mate) unit fails.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 320: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

320 Maintaining GWC alarms and logs

Table 45Communication Server 2000 GWC Manager status fields (cont’d.)

Status field Possible values Meaning

Isolation state: isolated The unit is not communicating with theCore.

notisolated The unit is communicating with the Core.

Available state: offLine(3) The unit has not received its configurationdata from the Communication Server 2000GWC Manager. The unit cannot provideservice until it is booted and receivesconfiguration data.

degraded(6) The unit does not have heartbeatcommunication with its mate and it isoperating without fault-tolerant redundancy.

offLine(3), degraded(6) The unit has both: offline and degradedconditions.

00 00 00 00 The unit does not have either of thepreceding conditions.

Loadname: <string_of_alphanumeric_characters>

This is the name of the load file that theunit currently boots from. The file is locatedon the Communication Server 2000 CoreManager or Core and Billing Manager(CBM) disk drive.

Usage state: idle The GWC maintenance system is notcurrently working on a request, such asa Return to Service (RTS). The unit isavailable for maintenance requests.

busy Maintenance is in progress on this unit andno further requests are accepted.

Stand by state: providingService The unit is the active unit and is providingservice.

hotStandby The unit is the standby unit - ready toprovide service.

coldStandby The unit is synchronizing with the activeunit (not providing redundancy). Aftercompletion of synchronization, thestatus changes to hotStandby when theOperational state is enabled.

Swact state: This field indicates the last switch ofactivity for the unit.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 321: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 321

Table 45Communication Server 2000 GWC Manager status fields (cont’d.)

Status field Possible values Meaning

manualSwActWarm Last switch of activity was due to a manualwarm SwAct. Requested by a user, awarm SwAct causes no service interruptionto stable calls, but calls in the setupprocesses can be lost.

manualSwActCold Last switch of activity was due to a manualcold SwAct. Requested by a user, a coldSwAct temporarily takes both units out ofservice and takes down all calls.

autonomousSwActWarm Last switch of activity was due to asystem warm SwAct. These SwActs areautomatically performed by the device inresponse to faults or failures. Establishedcalls are preserved. Calls in setup are lost.

autonomousSwActCold Last switch of activity was due to asystem cold SwAct. These SwActs areautomatically performed by the device inresponse to faults or failures. All calls arelost.

noSwAct No switch of activity has occurred.

Alarm state: This field indicates the severity of thecurrently raised alarms.

00 00 00 00 There are no alarms raised on the GWCcard unit.

critical(1) If present, indicates that one or morecritical alarms have been raised.

major(2) If present, indicates that one or more majoralarms have been raised.

minor(3) If present, indicates that one or more minoralarms have been raised.

alarmOutstanding(4) If present, indicates that at least one or acombination of different alarms has beenraised.

Fault state: none(0) This field is not used.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 322: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

322 Maintaining GWC alarms and logs

Viewing switch-wide GWC status

Use this procedure to query switch-wide Gateway Controller (GWC) statusinformation. You can view the status of a specified GWC or all GWCs inyour network. If required, you can also store the information in a file on aspecified device. Use this procedure when needed, for example, whenresponding to a service degradation or an outage.

Commands used in this procedure are not case-sensitive.

Procedure steps

Step Action

At the CI interface1 Access the GWC status level. Enter:

GWCSTAT

System response:

GWCSTAT:

2 This step is optional.

• If you wish to query all subcommands available from theGWCSTAT level, enter:help gwcstat

• If you wish to query the usage of a specific subcommand,enter:help <subcommand>

3 Use the appropriate subcommand to display the requiredinformation associated with a specific GWC or with all GWCs inthe network. For each subcommand,

• [ ] (square brackets) represent optional parameters;

• the default output is console, but you can direct the output toa specified file on a specified device, using the optional "file"parameter.

For detailed description of each command, see Table 46"GWCSTAT commands" (page 326).

Enter one of the following commands:

config <ALL or gwc#> [file <device_name> <file_name>]

OR

status <ALL or gwc#> [file <device_name> <file_name>]

OR

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 323: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 323

alarms <ALL or gwc#> [file <device_name> <file_name>]

OR

gwlist <ALL or gwc#> [file <device_name> <file_name>]

OR

all <ALL or gwc#> [file <device_name> <file_name>]

OR

gwctrkquery gwc <ALL or gwc#> [file <device_name><file_name>]

OR

gwctrkquery clli <clli_name> <member_id_start>[<member_id_end>] [file <device_name> <file_name>]

Example commands and system responses

> config 1

Example system response:

> status all

Example system response:

> gwlist all

Example system response:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 324: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

324 Maintaining GWC alarms and logs

> alarms 0

Example system response:

> all all

This command displays the combined output of thefollowing four commands: config, status, gwlist, andalarms.

> gwctrkquery gwc 1

Example system response:

> gwctrkquery clli 0 1

Example system response:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 325: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 325

4 If you wish to return to the CI level, enter

QUIT

--End--

Variable definitionsVariable Value

<ALL or gwc#> ALL — use this parameter if you wish to display status informationfor all GWCs in the network.ORA number between 0 and 255 — indicates which specific GWC youwish to query. For example, for GWC-8, enter 8.

[file <device_name><file_name>]

Optional parameter. If you wish to direct the output to a file on aspecified device, enter file, followed by a device name and a filename.

<clli_name> Common language location identifier (CLLI) code of the trunkgroup that you wish to query.

<member_id_start>[member_id_end>]

The trunk member numbers that together define the rangeof trunks within a trunk group that you wish to query. Themember_id_start is a mandatory parameter, but member_id_endis optional.

ExamplesIf you enter 0 for <member_id_start> and 10 for<member_id_end>, the system displays trunks whosemember numbers are between 0 and 10.If you enter the same number for both parameters (forexample, 5 5), the system displays the trunk with membernumber 5).If you enter only the <member_id_start> number, thesystem displays trunks whose member numbers are equalor greater than that specified number. For example, ifyou specify 5, the system displays trunks whose membernumber is equal or greater than 5.

<subcommand> One of the following subcommands available from the GWCSTATlevel: config, status, gwlist, alarms, all, or gwctrkquery.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 326: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

326 Maintaining GWC alarms and logs

Job aidTable 46GWCSTAT commands

Command Description

For all commands listed in this table, if the queried GWC does not respond, the system displaysthe Not Responding message next to the affected GWC.

config Use this command to view the following configuration information for aspecified GWC or for all GWCs in the network:• element manager (EM) IP address

• IP addresses for CORE.

• GWC number

• node number

• active IP address

• profile name

Active IP, Inactive IP, Unit0 IP and Unit1 IP are sequential, increase by 1,so Inactive IP, Unit0 IP and Unit1 IP are not listed in the output. You canderive them from the Active IP.

status Use this command to view the general status information of a specifiedGWC or all GWCs in the network. This command displays the GWC loadname and the following states:• AVAILST (available)

• FAULT

• ADMIN (administrative)

• USAGE

• OPER (operational)

• STANDBY

• ACT (activity)

• SWACTST (Swact)

• ISOLST (isolation)

You can also view this information using the GWC Manager. For moreinformation about these states and their possible values, see “Viewingand interpreting the operational status of a GWC node” (page 318).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 327: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 327

Table 46GWCSTAT commands (cont’d.)

Command Description

gwlist Use this command to display the following information about all gatewaysassociated with a specified GWC or with all GWCs in the network:• gateway name

• gateway IP address

• gateway type (for example, LARGE_GW)

• gateway state (DISABLED or ENABLED)

• protocol name and version (for example, MEGACO 1.0)

• protocol port number

ATTENTIONFor wireline solutions, the following queries are not supported:

• gateways associated with a small line GWCsSystem displays the following message: Small line notsupported

• GWCs with associated RMGC or small line gatewaysSystem displays the following message: Gateway type notsupported

alarms Use this command to list the number of critical, major, minor, and warningalarms for a specified GWC or all GWCs in the network.

all Use this command to display the combined output of the following fourcommands: config, status, gwlist, and alarms.

gwctrkquery gwc Use this command to display the information about trunks associated witha specified GWC or all GWCs in the network. This command lists thefollowing information:• GWC number

• external terminal number (EXTTERMNO)

• trunk group identifier (CLLI)

• trunk member number (MEMBERNO)

• state of the trunk

• subgroup type (SGRPTYPE)

gwctrkquery clli Use this command to display GWC trunks information for a specified trunkgroup. This command lists the following information:• trunk group clli and trunk member number (Trunk:)

• subgroup type (Sgrp Type:)

• state of the trunk

• GWC number

• external terminal number (ExtTermNo:)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 328: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

328 Maintaining GWC alarms and logs

Table 46GWCSTAT commands (cont’d.)

Command Description

• gateway name (GW name:)

• gateway IP address (GW IP:)

• endpoint

• Core status (SSC_State:)

Because there can be thousands of endpoints associated to a GWC,the system displays only ten members at a time, then prompts to youcontinue.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 329: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 329

Monitoring operational and activity status of GWCs

Use this procedure to access the GWCALL command underMAPCI;MTC;PM level, which allows you to continuously monitoroperational and activity status of all Gateway Controllers (GWC) in thenetwork. The system updates the output whenever the status of any GWCchanges.

Procedure steps

Step Action

At the CI interface1 Access the GWC directory: Enter:

MAPCI;MTC;PM;POST GWC

2 Display the operational and activity status of all GWCs. Enter:

GWCALL

The system displays all GWCs sorted by their number, eachrow represents a unit of GWCs. The system updates the outputwhenever the status of any GWC changes.

For the description of the symbols used in the display, see Table47 "GWCALL status symbols" (page 330).

Example system response:

3 If you wish to return to the CI level, enter

QUIT ALL

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 330: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

330 Maintaining GWC alarms and logs

Job aidTable 47GWCALL status symbols

Symbol Meaning

. (dot) The GWC unit is active and its operational state is InService.

* (star) The GWC unit is inactive and its operational status is InService.

- (dash) The GWC unit is unequipped.

M The GWC unit is active and its operational status is ManB (manuallybusy).

m The GWC unit is inactive and its operational status is ManB.

S The GWC unit is active and its operational status is SysB (system busy).

s The GWC unit is inactive and its operational status is SysB.

O The GWC unit is active and its operational status is Offl (off line).

o The GWC unit is inactive and its operational status is Offl

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 331: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 331

Restarting or rebooting a GWC card

Restart a GWC card to stop all software processes on the GWC card,perform a hardware reset, and reload the GWC card software from theCore and Billing Manager (CBM). Reboot a GWC card and force a GWCto download and execute a software load from the Communication Server2000 Core Manager or CBM.

Prerequisites• To reduce the risk of service interruption, first busy the GWC

applications on specific GWC nodes using the Communication Server2000 GWC Manager.

Procedure steps

Step Action

At the Communication Server 2000 SAM21 Manager client1 From the Shelf View, right-click the GWC card you want to

reboot and select Card View.

2 At the Card View, select the States tab.

3 Click the Lock button to lock the card.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 332: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

332 Maintaining GWC alarms and logs

4 Wait until the Card Status is Locked and the History windowindicates that: Application locked successfully.

5 Click the Unlock button.

6 Monitor the reboot process. Wait until the Card Status changesto Unlocked and the History window indicates Bootloadedsuccessfully.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 333: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 333

Restarting GWC card services

Restart GWC card services to stop and restart call processing and networkservices on a standby GWC card. Use this procedure on a GWC cardto determine if a known fault is temporary or persistent and if the fault islimited to this particular GWC card.

Prerequisites• To reduce the risk of service interruption, ensure that the card you

are about to restart is in standby status. If it is not, perform a callprocessing switch activity (SWACT) to change the card state fromactive to standby.

Procedure steps

Step Action

At the Communication Server 2000 GWC Manager client1 At the main Communication Server 2000 Management Tools

window, click the Gateway Controller folder from the DeviceTypes directory tree in the far left frame.

2 From the Contents of: Gateway Controller frame, select theappropriate GWC node you wish to restart.

3 Click the Maintenance tab.

4 Locate the Activity State field for the GWC unit and determinewhich unit (card) is active (either unit 0 or unit 1) and which isin standby mode.

5 Click the Busy (Disable) button for the GWC unit in a standbymode.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 334: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

334 Maintaining GWC alarms and logs

6 At the confirmation box, click OK.

7 Wait for the Communication Server 2000 GWC Manager screento update the Operational state of the card to disabled(2), whichindicates that the card is busied

8 Click the RTS (Enable) button to return the card to service.

After 30 seconds, the Administrative state field changes tounlocked and the Operational state field changes to enabled.

9 The state change is automatic. However, if necessary, you canrefresh the screen. At the top of the Communication Server 2000GWC Manager screen, click the Windows menu and selectRefresh GWC Status.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 335: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 335

Performing a Communication Server 2000 dataintegrity audit

This procedure describes how to perform an on-demand data integrityaudit of the Communication Server 2000 Gateway Controller (GWC)Manager, Communication Server 2000 Core, and Session Server Managerdatabases.

Use this procedure when you are receiving logs or alarms on theCommunication Server 2000 Core or at the Communication Server 2000SAM21 Manager client indicating a possible provisioning errors or datainconsistencies.

When the audit is running, suitable locks are in place that disableprovisioning operations.

Prerequisites• Review and be familiar with concepts, restrictions and limitations

related to a Communication Server 2000 data integrity audit. Forinformation, see “Performing a Communication Server 2000 dataintegrity audit” (page 335).

• Ensure that no provisioning activities are scheduled to take placeduring the audit.

• Ensure that there is no other line audit running or scheduled torun. Only one line audit can be in progress at any given time. Anin-progress line audit blocks all attempts to run any subsequent lineaudit requests. If you run an on-demand line audit, and if that audit isstill in progress at the start time of a scheduled audit, the scheduledaudit will not occur.

Procedure steps

Step Action

At the Communication Server 2000 GWC Manager client1 At the Communication Server 2000 Management Tools window,

select Maintenance, then Audit System.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 336: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

336 Maintaining GWC alarms and logs

The Audit System dialog box appears.

2 Click the Audit box and from the list of audits, select CS2K DataIntegrity Audit.

3 At the Audit System dialog box, click the Run Audit button todisplay the CS2K Data Integrity Audit Configuration dialog box.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 337: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 337

4 Select the Communication Server 2000 component that you wishto audit:

• Select the CS2KSS EM Data Integrity Audit check box —if you wish to verify that the network zones configurationdata in the GWC Manager and the Session Server Managerdatabases match.

• Select the CS2K Call Server Data Integrity Audit checkbox — if you wish to verify that the data held by the GWCManager database and the Communication Server 2000 CallServer Core match.

• Select both check boxes — if you wish to audit bothcomponents.

If there is no Session Server Manager configured on theCommunication Server 2000, the CS2KSS EM Data IntegrityAudit check box is disabled.

5 Click the Run Audit button to start the audit.

If you wish to cancel the process, click the Close button.

During a Communication Server 2000 audit, the system displaysa progress window with a progress bar.

6 The audit may take a few minutes to complete. When the auditends successfully, the system displays the following message:Audit Completed. Please see report files.

If the audit fails, the system displays the CS2K Data IntegrityAudit Failed to Complete message, with an error messageindicating the reason. Contact your next level of support toresolve the problem.

7 Click the Close button to close the Audit Status window.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 338: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

338 Maintaining GWC alarms and logs

Recovery of the GWC320 certificate expiry alarm

Use this procedure to clear a GWC 320 alarm indicating that a GWCcertificate is expiring or has expired.

Prerequisites• A GWC320 alarm must be raised. The alarm can be of any severity

and must indicate that a certificate is expiring or has expired.

Procedure steps

Step Action

1 At the Certificate manager

If there is a root CA replacement in progress STOP and WAITfor the root CA replacement procedure to complete. MonitorPKM logs.

2 Carry out the following:

3 Verify that the default for the GWC Minor alarm is set to 840hours and the GWC Major alarm is set to 420 hours. Verify thatthe Cert.Expiry days is set to a value of 365 days.

4 Wait 30 minutes for the GWC320 alarm to clear.

5 If the GWC320 alarm clears within 30 minutes go to step 11.

6 If the GWC320 alarm does not clear within 30 minutes go to thenext step.

7 At the GWC manager

Navigate to the GWC where the alarm is raised:

8 Open the:

Provisioning tab -> IPSec tab -> IKE Certificates tab

For more information about how to view IKE certificates on theGWC Manager, see Gateway Controller Fault Management(NN10202-911).

9 Are there two sets of certificates on the GWC?

If there are two sets of certificates on the GWC, then it isnecessary to verify which set is being used by the system, and toverify whether it is safe to delete the unused older set.

10 From the alarm, record the set number of the certificate that isexpiring or expired.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 339: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 339

11 Record the device certificate serial number of the set indicated inthe above step and go to the next procedure.

12 At the Certificate manager

View all GWC Certificates. For more information about how toview the certificates using Certificate Manager, see GatewayController Fault Management (NN10202-911).

13 If the serial number recorded in the above procedure is notlisted, then this set of certificates is not used. It is safe to deletethis set.

14 If the serial number recorded in above procedure is listed, thenthis set of certificates is used.

If It is not safe to delete this set.

15 At the GWC manager

Carry out the following:

16 Perform the procedure "Delete IKE certificates on theGWC Manager" in Gateway Controller Fault Management.(NN10202-911) and delete this set of certificates

17 Wait for the GWC320 alarm to clear

18 If the GWC320 alarm clears immediately go to step 22.

19 If the GWC320 alarm does not clear immediately STOP thisprocedure and contact your next level of support.

20 Carry out the following:

21 Perform the recovery procedure for the CMT304 alarm. For moreinformation on CMT304 Gateway Controller Fault Management(NN10202-911).

22 Wait 30 minutes for the GWC320 alarm to clear

23 If the GWC320 alarm clears within 30 minutes you havecompleted this procedure.

24 If the GWC320 alarm does not clear within 30 minutes STOPthis procedure and contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 340: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

340 Maintaining GWC alarms and logs

Increasing the size of a logical volume

Use this procedure to allocate more disk space to a logical volume.

DANGERIncreasing the size of a logical volume can limit futuresoftware upgrade capabilitycore manager logical volumes are pre-engineered to sizes thatare adequate for Nortel customers. Do not increase the size of alogical volume unless absolutely necessary.

If you need to change the size of a logical volume, do so only with theassistance of Nortel Technical Assistance and Support. Failure to followthis warning may jeopardize future software upgrade capability.

Prerequisites• You must be a user authorized to perform config-admin actions.

Procedure steps

Step Action

1 At the local VT100 console

Log onto the core manager as a user authorized to performconfig-admin actions.

2 Access the top menu level of the remote maintenance interface(RMI):

# sdmmtc

3 Access the system (Sys) menu level of the RMI:

> sys

4 Access the storage menu level of the RMI:

> storage

Example response:

Volume Group Status Free (MB)rootvg mirrored 1932datavg mirrored 7760

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 341: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 341

Logical Volume Location Size (MB) % full/threshold 1 / rootvg 88 11/ 802 /usr rootvg 600 28/ 903 /var rootvg 200 7/ 704 /tmp rootvg 24 5/ 905 /home rootvg 304 11/ 906 /sdm rootvg 504 23/ 907 /data datavg 208 6/ 80

Logical volumes showing: 1 to 7 of 7

ATTENTIONThe example response only shows part of the information displayedat the storage menu level of the RMI.

5 Determine if there is un-allocated disk space that can be used toincrease a logical volume.

6 If there is enough disk space then go to step 8.

7 If there is not enough disk space then you have completed thisprocedure.

8 Identify the logical volume to increase in size. Record the volumename of the logical volume on the left of the System menu of theRMI.

9

ATTENTIONA logical volume on the core manager must never reach 100% full.System behavior cannot be predicted when a logical volume reaches100% full.

Change the size of the logical volume:

> change lv / <logical_vol> <Mbyte>

Example input:

> change lv /home 48

Example response:

Expanding Volume /homeExpanding Volume /home - Command complete

ATTENTIONThe core manager can round the new size to the nearest 8-, or16-Mbyte increment.

10 For a 4 Gbyte disk, add 8- or 16-Mbyte multiples. When thelogical volume is created, the operating system determines themultiple that has to be used.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 342: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

342 Maintaining GWC alarms and logs

11 If the occupancy level of the specified logical volume hasexceeded its alarm threshold, contact your system administratorto assess the current condition of the logical volume.

--End--

Variable definitionsVariable value

<logical_vol> Is the name of the logical volume.

<Mbyte> Is the size in Mbytes to be added to the logicalvolume. The size must be less than the amountof un-allocated disk space.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 343: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

343.

I/O module replacement in an ERS8600 in a carrier CS-LAN

This section lists the procedures associated with replacing I/O modules inan ERS 8600 within a carrier CS-LAN.

I/O module replacement in an ERS 8600 in a carrier CS-LAN Navigation

• “Configuring the SESM Server Application” (page 345)

• “Starting the SESM server application” (page 347)

• “Stopping the SESM server application” (page 349)

• “Viewing SESM configuration settings” (page 351)

• “Stopping and starting the DDMS proxy” (page 354)

• “Stopping the SAM21 Manager server application” (page 356)

• “Stopping the APS server application” (page 358)

• “Stopping the NPM server application” (page 360)

• “Starting the SAM21 Manager server application” (page 362)

• “Starting the APS server application” (page 364)

• “Starting the NPM server application” (page 367)

• “Starting and stopping CINotifier” (page 369)

• “Changing the Oracle user password on an SPFS-based server” (page372)

• “Disabling Ethernet Routing Switch 8600 WAN Edge routing” (page374)

• “Disabling and reenabling the router ports on the Ethernet RoutingSwitch 8600” (page 376)

• “Installing the remote backup server” (page 378)

• “Viewing configuration information for remote server backups” (page383)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 344: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

344 I/O module replacement in an ERS 8600 in a carrier CS-LAN

• “Replacing an I/O module in an ERS 8600 in a carrier CS-LAN” (page385)

• “Locking the USP-Compact” (page 400)

• “Taking down the paths to the ERS 8600 chassis X” (page 408)

• “Determining if STORM devices require rotating, demounting andbusying” (page 410)

• “Locking the inactive Shelf Controller (SC) blade connected to chassisX” (page 415)

• “Unlocking the Border Control Point 7100” (page 419)

• “Unlocking the inactive SC” (page 422)

• “Unlocking the USP-Compact” (page 424)

• “Returning the paths to the ERS 8600 chassis X to service” (page 432)

• “Enabling the Ethernet links between the XA-Core and the ERS 8600”(page 434)

• “Accessing the Network Patch Manager CLUI” (page 436)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 345: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 345

Configuring the SESM Server Application

Use this procedure to configure the Succession Element and Sub-elementManager (SESM) server application.

ATTENTIONOnly perform this procedure if you installed an HTTPS certificate after the CS2Msoftware was installed or upgraded.

PrerequisitesPrior to performing this procedure, obtain the following information:

• the IP address of the CS 2000 Management Tools server

• the market for which you are configuring this application (NorthAmerica or International)

• the CLLI name of the office (CM CLLI), and the IP address of the SDM(CS 2000 Core Manager)Packet SDMX (XA-Core manager) associatedwith the CLLI

• the IP address of the Media Gateway 9000 Manager if present in thenetwork

ATTENTIONOnly the root user can perform this procedure.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, telnet to the server by typing

> telnet <server>

and pressing the Enter key.

where

server is the IP address or host name of the CS2000 Management Tools server

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su - root

and pressing the Enter key.

4 When prompted, enter the root password.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 346: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

346 I/O module replacement in an ERS 8600 in a carrier CS-LAN

5 Execute the configuration script by typing

# configure

and pressing the Enter key.

6 Enter the number next to the "SESM common configuration"option in the menu.

7 When prompted, enter the IP address of the CS 2000Management Tools server, or press the Enter key to accept thedefault if one is specified.

8 When prompted, enter the number next to the market for whichyou are configuring the SESM server application.

9 When prompted, enter the number next to the profile that appliesto your network configuration, or press the Enter key to acceptthe default if one is specified.

The system displays the information you entered forconfirmation.

10 When prompted, confirm the information by typing

y

and pressing the Enter key.

The system executes the command, and returns you to theSESM configuration main menu.

11 Exit "SESM configuration" by typing

select - x

and pressing the Enter key.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 347: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 347

Starting the SESM server application

Use this procedure to start the Succession Element and SubelementManager (SESM) server application on the SPFS-based server.

PrerequisitesNone

Procedure stepsPerform the steps that follow to complete this procedure.

Step Action

1 Establish a connection to the server that is hosting the CS 2000Management Tools through telnet or SSH, and log in using theroot user ID and password.

In a two-server configuration, log in to the active server using thephysical IP address of the active server, and ensure you are onthe active server using the ubmstat command.

For more information about logging in to an SPFS-based server,see Nortel ATM/IP Solution-level Security and Administration(NN10402-600).

2 Verify the status of the SESM server application by typing

# servman query -status -group SESMService

and pressing the Enter key.

If the response indicates Then

not running go to step 3

running go to step 6

3 Start the SESM server application by typing

# servstart SESMService

and pressing the Enter key.

4 Wait approximately 3 to 5 minutes before you proceed to thenext step to allow the SESM server application to start.

5 Verify the SESM server application started as follows:

• Verify the SESM services started by typing# servman query -status -group SESMService

and pressing the Enter key.Example response:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 348: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

348 I/O module replacement in an ERS 8600 in a carrier CS-LAN

CS2K Management Tools are running

• Verify the SESM applications started by typing# ptmctl status

and pressing the Enter key.Example response:SESM STATUS -------------------------

COMPONENT STATUS ---------------Proxy Agent RUNNINGRMI Regisry RUNNINGSnmpfactory RUNNINGMI2 Server RUNNINGCurrent number of SESM processes running: 4 (of 4)SESM APPLICATION STATUS: All Applications Ready

6 You have completed this procedure. If applicable, return to thehigh level task or procedure that directed you to this procedure.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 349: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 349

Stopping the SESM server application

Use this procedure to stop the Succession Element and SubelementManager (SESM) server application on the CS 2000 Management Toolsserver.

Prerequisites• You must be logged in to the active server as the root user using telnet

(unsecure) or ssh (secure).

Procedure stepsATTENTIONIn a two-server configuration, you must perform this procedure on the activeserver.

Step Action

1 Ensure you are logged in to the active server. Enter

ubmstat

If the server response is ClusterIndicatorACT then you areon the active server.

If the server response is ClusterIndicatorSTBY then youare on the inactive server. You must be logged in to the activeserver.

2 Verify the status of the SESM server application by typing

# servman query -status -group SESMService

If SESMService is Then

running go to step 5

not running go to step 6

3 Stop the SESM server application by typing

# servstop SESMService

and pressing the Enter key.

4 Wait approximately 3 to 5 minutes before you proceed to thenext step to allow the SESM server application to stop.

5 Verify the SESM server application stopped as follows:

• Verify the SESM services stopped by typing# servman query -status -group SESMService

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 350: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

350 I/O module replacement in an ERS 8600 in a carrier CS-LAN

and pressing the Enter key.Example response:CS2K Management Tools are not running

• Verify the SESM applications stopped by typing:# ptmctl status

and pressing the Enter keyExample response:SESM STATUS -------------------------COMPONENTSTATUS --------- ------ Proxy Agent NOT RUNNING RMIRegisry NOT RUNNING Snmpfactory NOT RUNNING MI2 ServerNOT RUNNINGCurrent number of SESM processes running: 0 (of 4)SESM APPLICATION STATUS: No Applications are ready

6 You have completed this procedure. If applicable, return to thehigh level task or procedure that directed you to this procedure.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 351: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 351

Viewing SESM configuration settings

Use this procedure to view the configuration settings for the SuccessionElement and Subelement Manager (SESM) server application anddetermine what the current setting for the CM CLLI.

Prerequisites• You need the root user ID and password to log on to the server where

the Communication Server 2000 Management Tools reside.

Procedure steps

Step Action

1 Telnet to the server where the Communication Server 2000Management Tools reside by typing

> Telnet <server>

and press the Enter key.

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su - root

and press the Enter key.

4 When prompted, enter the root password.

5 Change directory by typing

# cd /opt/nortel/NTsesm/admin/bin

and press the Enter key.

6 Access the command line interface by typing

# cli

and press the Enter key.

Sample system response

Command Line Interface1 - View2 - Configuration3 - Other

X - exit

select -

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 352: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

352 I/O module replacement in an ERS 8600 in a carrier CS-LAN

7 Enter the number next to the "Configuration" option in the menu(the number 2 in the preceding sample response).

Sample system response

Configuration1 - NTP Configuration2 - Apache Proxy Configuration

X - exit

select -

8 Enter the number next to the Succession Element Configuration"option in the menu (the number 18 in the preceding sampleresponse).

Sample system response

Succession Element Configuration1 - SESM Application Configuration2 - SAM21EM Application Configuration3 - NPM Application Configuration4 - PSE Application Configuration5 - DDMSProxy Application Configuration6 - OMPUSH Application Configuration7 - RESMON Application Configuration

X - exit

select -

9 Enter the number next to the "SESM Application Configuration"option in the menu (the number 1 in the preceding sampleresponse).

Sample system response

SESM Application Configuration1 - SESM_configuration (SESM Application Configuration)

X - exit

select -

10 Enter the number next to the "SESM_configuration" option in themenu (the number 1 in the preceding sample response).

Sample system response

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 353: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 353

SESM_configuration1 - SESM common configuration (IP addresses, market,

CM CLLI2 - SESM database tools3 - SESM related applications configuration4 - SESM provisioning configuration5 - SESM logging configuration (syslog, sesm debug

log)6 - view sesm configuration settings7 - SESM refresh properties

X - exit

select -

11 Select the "view sesm configuration settings" option by typing

select - 6

and pressing the Enter key.

12 Take note of the CM CLLI. If you need to change it andyou are using a Sun server, and for more information aboutsetting the Communication Server 2000 CLLI on the Sunserver, see ATM/IP solution-level Configuration Management(NN100409-500).

If you are using an MSC server, then for more information aboutsetting the MSC server 1000 Communication Server 2000 CLLIon the Sun server.

13 Exit SESM configuration by typing

select - x

and press the Enter key.

--End--

Variable definitionsVariable Value

<server> Is the IP address or host name of the serverwhere the Communication Server 2000Management Tools reside.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 354: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

354 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Stopping and starting the DDMS proxy

Use this procedure to stop and start the DDMS proxy on the CS 2000Management Tools server. The DDMS proxy enables communicationbetween the CS 2000 Management Tools server and the core.

PrerequisitesThere are no prerequisites for this procedure.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, establish a login session to the inactiveserver, using one of the following methods:

If using Then

telnet (unsecure) step 2

ssh (secure) step 3

2 Log in to the server using telnet (unsecure) as follows:

• Log in to the server by typing> telnet <server>and pressing the Enter key.

• When prompted, enter your user ID and password.

• Change to the root user by typing$ su - rootand pressing the Enter key.

• When prompted, enter the root password.Proceed to step 4.

3 Log in using ssh (secure) as follows:

• Log in to the server by typing> ssh -l root <server>and pressing the Enter key.If this is the first time you are logging in using ssh, the systemwill request that you confirm to continue connecting. Enteryes at the prompt.

• When prompted, enter the root password.

4 Access the command line interface by typing

# cli

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 355: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 355

and pressing the Enter key.

5 Enter the number next to the "Configuration" option in the menu.

6 Enter the number next to the "Succession ElementConfiguration" option in the menu.

7 Enter the number next to the "DDMSProxy ApplicationConfiguration" option in the menu.

8 Enter the number next to the "ddmsproxy_stop" option in themenu.

9 Enter the number next to the "ddmsproxy_start" option in themenu.

10 You have completed this procedure. If applicable, return to thehigh level task or procedure that directed you to this procedure.

--End--

Variable definitionsVariable Value

server is the physical IP address of the inactive server

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 356: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

356 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Stopping the SAM21 Manager server application

Use this procedure to stop the SAM21 Manager server application on theSPFS-based server that is hosting the CS 2000 Management Tools.

PrerequisitesNone

Procedure stepsPerform the steps that follow to complete this procedure.

Step Action

1 Establish a login session to the server, using one of the followingmethods.

In a two-server configuration, log in to the active server using thephysical IP address of the active server, and ensure you are onthe active server using the ubmstat command.

For more information about logging in to an SPFS-based server,see Nortel ATM/IP Solution-level Security and Administration(NN10402-600).

2 Verify the status of the SAM21 Manager server application bytyping

# servman query -status -group SAM21EM

and pressing the Enter key.

If SAM21EM is Then

running go to the next step

not running you have completed thisprocedure

3 Stop the SAM21EM server application by typing

# servstop SAM21EM

and pressing the Enter key.

4 Verify the SAM21EM server application stopped by typing

# servman query -status -group SAM21EM

and pressing the Enter key.

Example response:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 357: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 357

# servman query -status -group SAM21EMExecuting: /opt/servman/bin/servquery -status -groupSAM21EMSAM21EM===============================

Executing : /opt/nortel/sam21em/bin/healthSuccession SAM21 Element Manager version: SAM21EM_10_039_0Current status of the Succession SAM21 Element Manager:SNMP Access Gateway is not running.SAM21 Element Manager Server is not running.

5 You have completed this procedure. If applicable, return to thehigh level task or procedure that directed you to this procedure.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 358: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

358 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Stopping the APS server application

Use this procedure to stop the APS server application on the ServerPlatform Foundation Software (SPFS)-based server.

The APS resides on the same server as the CS 2000 Management Toolssoftware (CS2M).

Prerequisites• Before performing this procedure, verify that the APS server application

is running. Enter:# servman query -status -group APS

• You must have root user privileges to perform this procedure.

Procedure stepsPerform the steps that follow to complete this procedure.

ATTENTIONIn a two-server configuration, perform the steps that follow on the Active server.

Step Action

1 At your workstation, log in to the server by typing

> telnet <server>

and pressing the Enter key.

where

server is the IP address or host name of theSPFS-based server where the APS resides

In a two-server configuration, enter the physical IP address ofthe Active server (unit 0 or unit 1).

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su - root

and pressing the Enter key.

4 When prompted, enter the root password.

In a two-server configuration, ensure you are on the Activeserver by typing ubmstat. If ClusterIndicatorSTBY is displayedin the response, which indicates you are on the Inactive server,log out of that server and log in to the other server through telnet

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 359: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 359

using the physical IP address of the other unit. The responsemust display ClusterIndicatorACT, which indicates you are onthe Active server.

5 Stop the APS server application by typing

# servstop APS

and pressing the Enter key.

6 Verify the APS server application stopped by typing

# servman query -status -group APS

and pressing the Enter key.

Example response

Executing : /opt/servman/bin/servquery -status-group APSAPS======================================Executing : /usr/ntdb/uas/scripts/apsHealth.kshOracle Instance is DOWNOracle Listener is DOWN.PROBLEM: Oracle is not running.

The APS requires ORACLE to be up.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 360: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

360 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Stopping the NPM server application

Use this procedure to stop the Network Patch Manager (NPM) serverapplication on a Server Platform Foundation Software (SPFS)-basedserver.

ATTENTIONIn a two-server configuration, perform the steps that follow on the Active server.

Prerequisites• You need root user privileges to perform this procedure.

• It is recommended that all users exit the NPM CLUI and GUI beforestopping the NPM server application.

Procedure steps

Step Action

1 Log on to the server by typing

> Telnet <server>

and press the Enter key.

In a two-server configuration, enter the physical IP address ofthe Active server (unit 0 or unit 1).

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su - root

and press the Enter key.

4 When prompted, enter the root password.

In a two-server configuration, ensure you are on the Activeserver by typing ubmstat. If ClusterIndicatorSTBY is displayed inthe response, which indicates you are on the Inactive server, logout of that server and log on to the other server through Telnetusing the physical IP address of the other unit. The responsemust display ClusterIndicatorACT, which indicates you are onthe Active server.

5 Verify the status of the NPM server application by typing

# servman query -status -group NPM

and press the Enter key.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 361: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 361

6 If the NPM server application is running then continue step7 and if the NPM server application is not running then you havecompleted this procedure.

7 Stop the NPM server application by typing

# servstop NPM

and press the Enter key.

8 Verify the NPM server application is no longer running by typing

# servman query -status -group NPM

and press the Enter key.

Example response

NPM===========================Executing: /op/NTnpm/.HA_NPM_status

NPMServer is not running

--End--

Variable definitionsVariable Value

<server> IP address or host name of the SPFS-basedserver where the NPM server applicationresides.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 362: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

362 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Starting the SAM21 Manager server application

Use this procedure to start the SAM21 Manager server application on theSPFS-based server that is hosting the CS 2000 Management Tools.

PrerequisitesNone

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 Establish a connection to the server that is hosting the CS 2000Management Tools through telnet or SSH, and log in using theroot user ID and password.

In a two-server configuration, log in to the active server using thephysical IP address of the active server, and ensure you are onthe active server using the ubmstat command.

For more information about logging in to an SPFS-based server,see Nortel ATM/IP Solution-level Security and Administration(NN10402-600).

2 Verify the status of the SAM21 Manager server application bytyping

# servman query -status -group SAM21EM

and pressing the Enter key.

If SAM21EM is Then

not running go to the next step

running you have completed thisprocedure

3 Start the SAM21EM server application by typing

# servstart SAM21EM

and pressing the Enter key.

4 Verify the SAM21EM server application started by typing

# servman query -status -group SAM21EM

and pressing the Enter key.

Example response:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 363: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 363

Executing: /opt/servman/bin/servquery -status -groupSAM21EMSAM21EM===============================Executing : /opt/nortel/sam21em/bin/healthSuccession SAM21 Element Manager version: SAM21EM_9_020_0

Current status of the Succession SAM21 Element Manager:SNMP Access Gateway is running.SAM21 Element Manager Server is running.

5 You have completed this procedure. If applicable, return to thehigh level task or procedure that directed you to this procedure.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 364: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

364 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Starting the APS server application

Use this procedure to start the APS server application on the ServerPlatform Foundation Software (SPFS)-based server where the APSresides.

The APS resides on the same server as the CS 2000 Management Toolssoftware (CS2M).

PrerequisitesNone

Procedure stepsPerform the steps that follow to complete this procedure.

ATTENTIONIn a two-server configuration, perform the steps that follow on the Active server.

Step Action

1 At your workstation, log in to the server by typing

> telnet <server>

and pressing the Enter key.

where

server is the IP address or host name of theSPFS-based server where the APS resides

In a two-server configuration, enter the physical IP address ofthe Active server (unit 0 or unit 1).

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su - root

and pressing the Enter key.

4 When prompted, enter the root password.

In a two-server configuration, ensure you are on the Activeserver by typing ubmstat. If ClusterIndicatorSTBY is displayedin the response, which indicates you are on the Inactive server,log out of that server and log in to the other server through telnetusing the physical IP address of the other unit. The responsemust display ClusterIndicatorACT, which indicates you are onthe Active server.

5 Verify the status of the APS server application by typing

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 365: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 365

# servman query -status -group APS

and pressing the Enter key.

If APS server is Then

not running go to the next step

running you have completed thisprocedure.

6 Start the APS server application by typing

# servstart APS

and pressing the Enter key.

7 Verify the APS server application started by typing

# servman query -status -group APS

and pressing the Enter key.

Example response

APS===============================Executing : /usr/ntdb/uas/scripts/apsHealth.kshOracle Listener and DB appear are up.APS server processes:APS PID files are on the filesystem.-rw-rw-rw- 1 root other 6 Jan 25 06:00 /opt/uas/aps/scripts/runAdmanager.pid-rw-rw-rw- 1 root other 6 Jan 27 03:06 /opt/uas/aps/scripts/runAlertHandler.pid-rw-rw-rw- 1 root other 6 Jan 24 22:56 /opt/uas/aps/scripts/runApsDbServer.pid-rw-rw-rw- 1 root other 6 Jan 24 22:56 /opt/uas/aps/scripts/runDbServer.pid-rw-rw-rw- 1 root other 6 Jan 24 22:56 /opt/uas/aps/scripts/runFileUpload.pid-rw-rw-rw- 1 root other 6 Jan 24 22:56 /opt/uas/aps/scripts/runHeartbeat.pid-rw-rw-rw- 1 root other 6 Jan 24 22:56 /opt/uas/aps/scripts/SnmpAgent.pidChecking process: /opt/uas/aps/scripts/runApsDbServer.pidThe ’runApsDBServer’ process, PID: 15012 is ACTIVE andRUNNING.Checking process: /opt/uas/aps/scripts/runDbServer.pid The ’runDBServer’ process, PID: 14456 is ACTIVE andRUNNING.Checking process: /opt/uas/aps/scripts/runFileUpload.pid The ’runFileUpload’ process, PID: 14826 is ACTIVEand RUNNING.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 366: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

366 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Checking process: /opt/uas/aps/scripts/runAdmanager.pid The ’ADmanager’ process, PID: 23018 is ACTIVE andRUNNING.Checking process: /opt/uas/aps/scripts/runHeartbeat.pid The ’Heartbeatmon’ process, PID: 15403 is ACTIVE andRUNNING.Checking process: /opt/uas/aps/scripts/SnmpAgent.pidThe ’SnmpAgent’ process, PID: 14271 is ACTIVE andRUNNING.

APS is up and running.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 367: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 367

Starting the NPM server application

Start the Network Patch Manager (NPM) server application to beginthe NPM server application on a Server Platform Foundation Software(SPFS)-based server.

Prerequisites• You need root user privileges to perform this procedure, and CORBA

must be running in order for the NPM to come up.

ATTENTIONIn a two-server configuration, perform the steps that follow on the active server.

Procedure steps

Step Action

1 Log on to the server by typing

> Telnet <server>

and press the Enter key.

In a two-server configuration, enter the physical IP address ofthe Active server (unit 0 or unit 1).

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su - root

and press the Enter key.

4 When prompted, enter the root password.

In a two-server configuration, ensure you are on the Activeserver by typing ubmstat. If ClusterIndicatorSTBY is displayed inthe response, which indicates you are on the Inactive server, logout of that server and log on to the other server through Telnetusing the physical IP address of the other unit. The responsemust display ClusterIndicatorACT, which indicates you are onthe active server.

5 Verify the status of the NPM server application by typing

# servman query -status -group NPM

and press the Enter key.

6 If the NPM server application is not running then continue thenext step and if the NPM server application is running then youhave completed this procedure.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 368: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

368 I/O module replacement in an ERS 8600 in a carrier CS-LAN

7 Start the NPM server application by typing

# servstart NPM

and press the Enter key.

8 Verify the NPM server application is running by typing

# servman query -status -group NPM

and press the Enter key.

Example response

Executing: /opt/servman/bin/servquery -status -groupNPMNPM===============================Executing : /opt/NTnpm/.HA_NPM_status

The NpmServer is running

--End--

Variable definitionsVariable Value

<server> IP address or host name of the SPFS-basedserver where the NPM server applicationresides.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 369: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps to start CINotifier 369

Starting and stopping CINotifier

Use the CINotifier application to notify network users that remoteconnectivity is running.

ATTENTIONWhen you start CINotifier and then delete the GWC manager node, all clientGWCs auto-refresh their GWCEM lists. However, when you stop CINotifier theGWCEM lists do not refresh.

PrerequisitesThese procedures have the following prerequisites:

• You must be logged in to your work station

• You need root user privileges to perform this procedure.

Procedure steps to start CINotifierPerform the following steps to complete this procedure.

ATTENTIONIn a two-server configuration, perform the steps that follow on the Active server.

Step Action

1 Log in to the server by typing

> telnet <server>

and pressing the Enter key

where

where server

ATTENTIONIn a two-server configuration, enter the physical IP address of theActive server (unit 0 or unit 1).

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su - root

and pressing the Enter key.

4 When prompted, enter the root password.

In a two-server configuration, ensure you are on the Activeserver by typing ubmstat.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 370: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

370 I/O module replacement in an ERS 8600 in a carrier CS-LAN

If ClusterIndicatorSTBY is displayed in the response,which indicates you are on the Inactive server, log out of thatserver and log in to the other server through telnet using thephysical IP address of the other unit. The response must displayClusterIndicatorACT, which indicates you are on the Activeserver.

5 Verify the status of the CINotifier application by typing

# servman query -status -group CINOTIFIER

and pressing the Enter key.

If the CINOTIFIER application is not running, do Step 6.

If the CINOTIFIER application is running, you have completedthis procedure.

6 Start the CINOTIFIER application by typing

# servstart CINOTIFIER

and pressing the Enter key.

7 Verify the CINotifier application is running by entering:

# servman query -status -group CINOTIFIER

--End--

Procedure steps to stop CINotifierUse this procedure to stop the CINotifier application on a Server PlatformFoundation Software (SPFS)-based server..

ATTENTIONIn a two-server configuration, perform the steps that follow on the Active server.

Step Action

1 Log in to the server by typing

> telnet <server>

and pressing the Enter key

where

where server

ATTENTIONIn a two-server configuration, enter the physical IP address of theActive server (unit 0 or unit 1).

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 371: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps to stop CINotifier 371

$ su - root

and pressing the Enter key.

4 When prompted, enter the root password.

In a two-server configuration, ensure you are on the Activeserver by typing ubmstat.

If ClusterIndicatorSTBY is displayed in the response,which indicates you are on the Inactive server, log out of thatserver and log in to the other server through telnet using thephysical IP address of the other unit. The response must displayClusterIndicatorACT, which indicates you are on the Activeserver.

5 Verify the status of the CINotifier application by typing

# servman query -status -group CINOTIFIER

and pressing the Enter key.

If the CINOTIFIER application is running, do step 6.

If the CINOTIFIER application is not running, you havecompleted this procedure.

6 Stop the CINOTIFIER application by typing

# servstop CINOTIFIER

and pressing the Enter key.

7 Verify the CINotifier application is no longer running by typing

# servman query -status -group CINOTIFIER

and pressing the Enter key.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 372: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

372 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Changing the Oracle user password on anSPFS-based server

Use this procedure to change the default Oracle passwords on a ServerPlatform Foundation Software (SPFS)-based server.

Refer to procedure, “Changing the Oracle user password on anSPFS-based server” (page 372) to change the APS Oracle account defaultpassword.

ATTENTIONUser accounts and passwords are automatically propagated from the activeserver to the inactive server in a high-availability (two-server) configuration toallow users to log in to either server. However, user files are not propagated tothe other server.

PrerequisitesNone

Procedure stepsPerform the following steps to complete this procedure.

ATTENTIONIn a two-server configuration, perform the steps that follow on the active server.

Step Action

1 At your workstation, log in to the Active server by typing

> telnet <server>

and pressing the Enter key.

where

server is the IP address or host name of theSPFS-based server

In a two-server configuration, log in to the active server using itsphysical IP address.

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su -

and pressing the Enter key.

4 When prompted, enter the root password.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 373: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 373

5 Change to the Oracle user by typing

# su - oracle

and pressing the Enter key.

6 Change the password by typing

$ /opt/nortel/sspfs/db/pfsora_set_pwd <userID>

and pressing the Enter key.

where

userID is the user ID of the Oracle user

Example response:

Enter new password for user SYSTEM:

7 When prompted, enter the new password for the system.

Example response

Re-enter new password:

8 When prompted, enter the password a second time to confirmthe password.

Example response

Please wait...Changing user password in the Oracle database...Changing user password in the Properties Server...

Successfully changed user password.

The command takes approximately 15 to 20 seconds to execute.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 374: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

374 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Disabling Ethernet Routing Switch 8600 WAN Edgerouting

ATTENTIONPerform this procedure immediately after a site failure occurs to limit effects toCall Processing.

This procedure is required to workaround a deficiency in the architecturewhen the active in-service and SYSB (out of service) GWC sendmessages to a gateway provisioned off the GWC causing mismatches inthe gateway resulting in loss of call processing.

This procedure will impact every element sitting off the ERS 8600.

There are two options for this procedure. The first option will disable alldynamic routing on the chassis and no traffic will be able to leave or enterthe VLANs configured on that chassis when using Open Shortest PathFirst (OSPF). Static routes and Local communication is still possible.

The second option will disable the WAN Edge Interface. All traffic, whetherthrough dynamic or static routes will not be able to leave or enter thechassis.

Procedure steps

Step Action

1 Determine which option you will use.

2 If you will use option 1 then do Step 4.

3 If you will use option 2 then do Step 6.

4 Disable OSPF by typing

#config ip ospf disable

and press the Enter key.

5 Go to Step 7.

6 Identify the WAN Edge VLAN ID by typing

#show vlan info basic

and press the Enter key.

Example response

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 375: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 375

7 Identify the ports used for WAN Edge connectivity by typing.

#show vlan info ports <vlan_id>

and press the Enter key.

Example response

8 Disable the ports used for WAN Edge connectivity by typing.

#config ethernet <port_range_1> state disable

#config ethernet <port_range_2> state disable

...

#config ethernet <port_range_n> state disable

and press the Enter key.

--End--

Variable definitionsVariable Value

<vlan_id> The VLAN ID obtained in step 4.

<port_range_1> Range of ports used for WAN Edge.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 376: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

376 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Disabling and reenabling the router ports on theEthernet Routing Switch 8600

Use this procedure when a failure occurs, such that Sites A and B areisolated from each other, to disable and re-enable the Ethernet RoutingSwitch 8600 router ports that are connected to the GWC in the inactivesite.

Prerequisites• Know the list of router ports connected to the GWCs.

Procedure steps

Step Action

1 Confirm which site has the active Call Agent.

2 Confirm that the GWCs in the active site are active.

3 Disable the ports to the GWC units on the inactive site by typing

#config ethernet <port_range_1> state disable

#config ethernet <port_range_2> state disable

...

#config ethernet <port_range_n> state disable

and press the Enter key.

4 If you disabled OSPF in the procedure Disabling EthernetRouting Switch 8600 WAN Edge routing on page 205, re-enableit by typing

#config ip ospf enable

and press the Enter key.

5 If you disabled the ports used for WAN Edge connectivity,reenable those ports by typing

#config ethernet <port_range_1> state enable

#config ethernet <port_range_2> state enable

...

#config ethernet <port_range_n> state enable

and press the Enter key.

6 Check the time by typing

#date

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 377: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 377

and press the Enter key.

Example response

local time: THU MAR 31 17:32:57 2005 EST

hardware time: THU MAR 31 22:32:57 2005 UTC

7 Allow at least one minute to pass to ensure that internal GWCaudits set the units to inactive.

8 Check the time by typing

#date

and press the Enter key.

Example response

local time: THU MAR 31 17:33:59 2005 EST

hardware time: THU MAR 31 22:33:59 2005 UTC

9 If the inactive buliding is accessible, verify the inactive GWCunits have gone inactive.

10 Connect the terminal to the console port on the GWC.

11 Enter PMDEBUG and perform the T,N, and A commands.

12 Confirm that the state is inactive. if not, repeat step 3 throughstep 8 on those ports.

13 Enable the ports to the GWC units in the inactive site by typing

#config ethernet <port_range_1> state enable

#config ethernet <port_range_2> state enable

...

#config ethernet <port_range_n> state enable

and press the Enter key.

--End--

Variable definitionsVariable Value

<port_range> Range of ports connected to the GWC(example: 3/1-3/48).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 378: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

378 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Installing the remote backup server

TargetUse this procedure to install the remote backup server for GeographicSurvivability. Backing up the remote server provides a standby backupsystem which is ready to provide service if the primary system isunavailable for an extended period of time. The remote server can assumethe identity of the primary server with data and files accurate to the lastsynchronization.

This procedure applies only to DVD installations.

PrerequisitesCollect the following information beforehand to ensure that you have theinformation ready for input for this procedure. Also acquire the root userID and password.

ATTENTIONAlso collect this information prior to an upgrade.

SystemVariable Description Actual value

Hostname

IP address Remote backup server

Netmask

Router Default gateway IP

DNS Yes/No

Unit 0 IPaddress

IP of primary cluster unit 0

Daily backup(up to four)

Format =HH : MM

where

HH = hours (00-23)MM = minute (00 -

59)

DNS domain

IP address DNS server(s)

DNS searchdomain(s)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 379: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Action 379

SystemVariable Description Actual value

Timezone Timezone =Timezone (# echo $TZ)

SPFS serialnumber

Located on label on backof SPFS

DNS variables apply only when a DNS server has been configured.

ActionInstalling the remote backup server on a Geographic Survivabilitystandby server

At your workstation or the remote server console

Step Action

1 Log on to the server that is installed as the remote backupserver. For more information about logging in to an SPFS-basedserver, see Nortel ATM/IP Solution-level Security andAdministration (NN10402-600).

2 Log on to the server through the console (port A) and whenprompted, enter the root or emsadm user ID and password.

3 Bring the system to the {0} OK prompt by typing:

# init 0

4 Enter the following command to verify that the auto-boot optionis set to true.

{0} ok printenv auto-boot?

If it is set to false, enter the following command.

{0} ok setenv auto-boot?=true

5 Insert the SPFS0** DVD into the DVD drive on the remotebackup unit.

where

** refers to the release (for example, 091 forSN09.1)

6 Install the remote backup server for Geographic Survivability.

{0} OK boot cdrom - rbackup

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 380: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

380 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Example responses

{0} ok boot cdrom - rbackupBoot device: /pci@1e,600000/ide@d/cdrom@0,0:f Fileand args: -rSunOS Release 5.8 Version Generic_108528-18 64-bitCopyright 1983-2001 Sun Microsystems, Inc. Allrights reserved.Hardware watchdog enabledConfiguring /dev and /devicesUsing RPC Bootparams for network configurationinformation.Skipping interface bge3Skipping interface bge2Skipping interface bge1Skipping interface bge0/This CD contains software that under the terms of yourlicense agreement with Nortel Networks is licensedfor use only on Nortel Networks OAM&P servers.

Type "ok" to acknowledge this restriction on your useof the software.

Any other input will halt the installation of thissoftware.

7 Type OK and press Enter to acknowledge restriction on your useof the software.

OK

Searching for configuration file(s)...

8 Enter the serial number located in the rear view of the SPFS andpress Enter.

*** REMOTE Backup INSTALL ***Begin entering installation informationPlease select a server profile for this systemrbackup: Remote Automatic Backup standby systemEnter the server profile for this system:

( rbackup )

9 Select the rbackup server profile for the system.

Installing with server profile "Remote Automatic Backupstandby system"

Install can proceed with factory default values fornetwork configuration or you may specify the networkconfiguration now.

Use factory default network configuration: [yes] no

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 381: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Action 381

10 Enter No to not select the default settings. You must connectthe N240 server to the network and you must have access tothe default gateway. This allows you to enter the settings on theserver for the installation.

No

11 Enter site-specific information in response to the followingprompts. (See the information entered in the table at thebeginning of this procedure.)

Enter the hostname for this system.Enter the IP address for <hostname>.Enter the subnet mask for this network.Enter the IP address for this network’s router.Enter the timezone for this system [US/Eastern].

The default timezone is US/Eastern. Enter ? for a list ofsupported time zones.

Will this system use DNS? [yes]

Enter yes or no. If you answer yes, you are prompted for theDNS domain name, name server IP addresses, and the searchdomains. You can enter several name servers and searchdomains. Enter a blank line to stop the entry.

12 Log on to the server using the root user ID and password. (Atthis point, the root password has been reset to "root.")

13 Remove the SPFS from the server. If necessary, enter

eject cdrom

14 Enter the command line interface (CLI) tool.

cli

15 Enter the number next to the Configuration option in the menu.

16 Enter the number next to the Succession Element Configurationoption in the menu.

17 Enter the number next to the PSE Application Configurationoption in the menu.

18 Enter the number next to the Configure PSE option in the menu.

19 Enter the IP address of the machine on which the NPM isconfigured.

20 Enter Y to confirm the IP address.

Ignore the following error message if it appears.

Can’t configure PSE on remote backup unit to enable NPM

21 Enter X to exit each level until you have exited from the CLI tool.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 382: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

382 I/O module replacement in an ERS 8600 in a carrier CS-LAN

22 Start the PSE server.

# pse start

23 Verify the server has started. If the server does not start, contactyour next level of support.

# pse status

24 If an SPFS MNCL CD is to be installed, see the documentationincluded with the CD for complete installation instructions.

25 Enter NPM on the Management Tools server and follow patchingprocedures to apply all relevant patches.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 383: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 383

Viewing configuration information for remote serverbackups

View configuration information for remote server backups. The systemdisplays the IP address of the target system and the times in whichautomatic backups of the target system will occur.

ATTENTIONThis procedure is for use with Geographic Survivability only.

Procedure steps

Step Action

1 Log on to the Sun Netra server that is installed as the remotebackup server.

2 If you want to log on using ssh, enter:

ssh -l

and press the Enter key.

3 If you want to log on using Telnet, enter:

Telnet

and press the Enter key.

4 If you want to log on to the remote server console go to step 5.

5 Start the command line interface tool by entering:

cli

The system responds by displaying a menu.

6 Select the Configuration menu.

The system displays the Configuration menu.

7 Select the Remote Backup option.

Response:

Remote Backup Configuration1-rbackup_display (Display Remote Backup Configuration)2-rbackup_config (Remote Backup Configuration)3-rbackup_exec (Execute Remote Backup Now)4 - rbackup_cancel (Cancel Running Remote Backup)

X-exit

8 Select

1-rbackup_display (Display Remote Backup Configuration)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 384: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

384 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Response:Current settings:Target system is: <nnn.nnn.nnn.nnn>

Back up times are: <Time 1>...<Time n>

9 Exit the Remote Backup Configuration level by typing:

x

--End--

Variable definitionsVariable Value

<server> Name of the SPFS-based server.

<nnn.nnn.nnn.nnn> IP address of the remote server.

<Time 1>...<Time n> Set of times at which automated backups occur.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 385: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Prerequisites 385

Replacing an I/O module in an ERS 8600 in a carrierCS-LAN

The following procedure provides the steps you need to replace an I/Omodule in an Ethernet Routing Switch (ERS) 8600 that supports theCommunication Server LAN (CS-LAN). When applicable, a reference thatcontains the detailed steps is provided.

In a CS-LAN environment most devices are cabled across two ERS 8600s.In some instances, it may be required to groom the traffic off of the I/Omodule to another I/O module located within the same chassis or on thealternate ERS 8600.

The following document is referenced in this procedure: Installing EthernetRouting Switch 8600 Modules (312749-K).

Prerequisites• Ensure you have a replacement I/O module.

• Make sure you use READ/WRITE/ALL (RWA) logon privileges,password privileges or both when performing this procedure. For moreinformation about RWA privileges, refer to Passport Documentation inthe Nortel website.

• Access the ERS 8600s using the serial (console) port. You can use adirect serial connection and a terminal program from an attached PCor a Telnet connection to an appropriately configured IP address on aterminal server connected to the console port.

• If any of the following devices are connected to the I/O module beingreplaced, ensure that it has a unit subtending to the other ERS 8600switch:

— Border Control Point (BCP) 7100

— Media Gateway (MG) 7000 or MG 15000

ATTENTIONOnly if the I/O card is an 8672

— Gateway Controller (GWC)

— High performance Input/Output Processor (HIOP)

— USP/USP Compact

— STORM-IA

— Shelf Controller (SC)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 386: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

386 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Procedure steps

Step Action

1 Enter the following command to identify connections to the I/Omodule being replaced:

show port info name

In the following sample output, it is clear that this I/O module hasconnections to the following devices: Communication Server2000 management tools, CBM, USP, MDM, SDM, MG 3500 EMserver, MG 9000 EM server, XA-Core HIOP, SSLM, and SSLI.

This module also has two GBIC ports. Port 1/33 as an MLT portand port 1/34 is used to connect to ERS-03

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 387: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 387

2 Verify that console access is available to the ERS 8600 switchwhere the I/O module is being replaced. This is to ensure thatout-of-band access to the switch is available in the event thatin-band access is lost when the module is removed.

3 If using a Compact Call Agent (CCA) Core, ensure there are noalarms on the call agent.

4 Save the original boot and runtime configuration files to flashmemory and copy to pcmcia using the following commands.Enter y when prompted.

save config backup /pcmcia/config.cfg

save bootconfig backup /pcmcia/boot.cfg

5 Ensure that direct connectivity has been established according tothe instructions in bulletin SEB-08-01-009.

6 From the System tree, click the plus sign next to NetworkElements to expand the view of the folder.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 388: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

388 I/O module replacement in an ERS 8600 in a carrier CS-LAN

7 Next, click the plus sign next to RTP Portals to expand the viewof the folder. The contents of this folder lists all BCP 7100s andBCP 7200s configured for the site.

8 Continue to expand the view of folders until the target BCP 7100is selected. Do not modify the state of any BCP 7200s.

9 Click the NE Maintenance option. A new window opens.

10 Click the Stop button to initiate shut down.

The Stop button shuts down the system. The Kill button locksthe BCP 7100, and the Restart button stops and starts thesystem.

11 A confirmation window appears. Click Yes to continue.

12 The BCP 7100 component shuts down gracefully and eventuallygoes into an inactive state when the last active media sessionends (as seen in the General Information Area of the SystemManagement Console).

13 Check the provisioning information for the VSPs, NSTA, andVGS subcomponents as follows:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 389: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 389

Enter the following command to display media and signalinggateway IP addresses.

d -p nsta/* vgs ctrl/*

The following list provides sample MG and SG IP addresses fora single VSP card provisioned in slot 7.

ATTENTIONThe "nsta/* vgs ctrl/*" portion of this command causes IPaddresses for all provisioned VSP cards to be displayed. Theexample only shows MG and SG IP addresses for a single VSP cardprovisioned in slot 7.

14 Enter the following command to display the IPmcon IP address.

d -p nsta/* vgs ipmcon

The following list provides a sample IPmcon IP address for asingle VSP card provisioned in slot 7.

ATTENTIONThe "nsta/* vgs ipmcon" portion of this command causes IPaddresses for all provisioned VSP cards to be displayed. Theexample only shows IPmcon IP address for a single VSP cardprovisioned in slot 7.

15 Enter the following command on each ERS 8600 chassis tocheck the VLAN configuration file.

sho config module vlan

This command outputs all VLANs provisioned on the ERS 8600chassis.

The example that follows shows the output of this command foronly one VSP blade.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 390: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

390 I/O module replacement in an ERS 8600 in a carrier CS-LAN

ATTENTIONBoth ERS 8600s display the identical PVG VLAN configuration.The "vlan <#> ports add" entry identifies each VSP ATM Sonetconnection. This information is required in step 25.

16 Check the routing table information for each ERS 8600 chassisas follows:

Enter the following command to display IP route information.

sho ip route info ip x.y.z

ATTENTIONSpecifying the x.y.z octets limits the command output to those nodesserved by the specified IP Net.

Enter the following command to display IP route information.

sho ip route info ip x.y.z

ATTENTIONSpecifying the x.y.z octets limits the command output to those nodesserved by the specified IP Net.

17 Repeat for each VSP VLAN on the ERS 8600 chassis X.

18 Repeat for ERS 8600 chassis Y.

19 Check the SPVCAP subcomponent of each VSP VGSCtrl/<mg.sg> and IPmcon as follows:

Enter the following command to display the provisioninginformation for each VSP.

d -p -no nsta/* vgs ctrl/* spvcap

The following list shows VSP provisioning information for slot 7.The actual command output lists all VSP information.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 391: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 391

20 Confirm that the VSP VGS "addressToCall" (ATMIF UNIaddress) for Ctrl/mediaGateway, Ctrl/signaling Gateway, andIpMcon are identical.

ATTENTIONIf they are not identical, contact your next level of support.

21 Enter the following command to check the ATMIF UNI addressesprovisioned as primary and alternate.

l atmif/* uni addr/*

A sample of the command output for the ATMIFs present on thePVG shelf follows.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 392: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

392 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Make sure the ATMIF UNI provisioning reflects a Primary andAlternate configuration for the same address for each VSP.(Refer to your engineering guidelines for more information.) Ifthey do not match, contact your next level of support.

22 Determine which ATMIF the VSP VGS Ctrl/<mg,sg> and IPmconATMCON subcomponents point to as follows:

a

d nsta/* vgs ctrl/* atmconn

A sample of the command output for a VSP provisioned inslot 7 follows. The actual command output lists all connectionstates and the ATMIFs they use.

ATTENTIONMake sure the AtmIf information for each VSP is identical to theprimary interface information listed in step 21. If they do notmatch, contact your next level of support.

23 Enter the following command to display the NSTA VGS IPmconATMCON subcomponent for the VSPs.

d nsta/* vgs ipmcon atmcon

A sample of the command output for a VSP provisioned in slot 7follows. The actual command output lists all connection statesand the ATMIFs they use.

ATTENTIONMake sure the AtmIf information for each VSP is identical to theprimary interface information listed in step 21. If they do not match,contact your next level of support.

24 Enter the following command to display the VGS Ctrl/<mg,sg>ATMCON subcomponent for all VSPs.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 393: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 393

25 Check the physical connections between each PVG ATMinterface and their related ERS 8600.

ATTENTIONSee step 15 for the ERS 8600 local route tables for the VSPs tobe offloaded and verify the fiber connectivity between the PVGsand ERS 8600s. Refer to your engineering guidelines for additionalinformation.

26

ATTENTIONOSPF routing does not require manual maintenance to reduce therisk of call processing outage, as the link redundancy on PVGs failsover nearly immediately.

27 Disable the ATM interface for each VSP as follows:

28 Enter the following command to identify the ATM interfaces andSonet layer.

d -p -no lp/* sonet/* sts/0

A sample of the command output for the first Sonet interface oneach card follows. The actual command output lists all Sonetinterfaces on the PVG.

29 Enter the following command to lock all SONET layerconnections related to the ATM ports connected to ERS 8600chassis X.

lock -force lp/<slot> sonet/<sonetif>

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 394: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

394 I/O module replacement in an ERS 8600 in a carrier CS-LAN

ATTENTIONThis command results in Sonet layer alarms.

30 Record all locked Sonet interfaces.

31 Confirm the ATM (software) connection switchover is successfulby entering the following command to display the VGSCtrl/<mg,sg>ATMCON subcomponent for the VSPs.

d nsta/* vgs ctrl/* atmconn

A sample of the command output for the switched ATMCON forthe Ctrl/* to the ATMIF in slot 7 follows. This example shows aswitchover from AtmIf/141 (see step 22) to AtmIf151.

32 Enter the following command to display the routing table for bothERS 8600s.

sho ip route info ip x.y.z

ATTENTIONThe routing information under the PROT column heading mustindicate OSPF for all VSP VLANs on ERS 8600 chassis X.

33 Select the next step as follows:

34 Enter the following command to unlock all Sonet ports locked atstep 27.

unlock lp/<slot> sonet/<sonetif>

35 Enter the following command to make sure all VSP VLANsremain OSPF on ERS 8600 chassis X, and LOC on ERS 8600chassis Y.

sho ip route info ip 172.16.201

The routing information under the PROT column heading mustindicate OSPF for all VSP VLANs on ERS 8600 chassis X. If itdoes not, contact your next level of support.

36 Launch Communication Server 2000 Management Tools fromyour web browser.

37 Access the Maintenance information for the GWCs as follows:

38 Select GatewayController from the Device Types field of theCommunication Server 2000 Management Tools GUI.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 395: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 395

The GatewayController folder in the Device Types field ishighlighted.

39 Select the Directory tab under Contents of: GatewayController.

40 Select the first GWC in the list.

The GWC is highlighted and the related Maintenance orProvisioning information (depending on the active tab) appears inthe large rightmost field.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 396: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

396 I/O module replacement in an ERS 8600 in a carrier CS-LAN

41 Select the Maintenance tab to make sure the Maintenanceinformation is displayed.

42 Make all GWCs inactive on ERS 8600 chassis X.

43 Check the Activity State field for GWC Unit N to determine whichunit is active and which is standby

44 Click WarmSwact.

A warning window appears

Click OK to continue or Cancel to abort.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 397: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 397

45 Click OK, and wait for the activity change to complete.

The selected GWC undergoes a change of activity. The changeof activity can take approximately 2 minutes.

Select the next GWC in the list, and repeat these substeps untilthe Activity State field for Unit: N Status is standby for allGWCs.

46 Enter the following command to access the ETHR level of theMAP display.

>MAPCI;MTC;XAC;ETHR

The following is a sample MAP display for the ETHR level.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 398: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

398 I/O module replacement in an ERS 8600 in a carrier CS-LAN

47 Refer to the figure and table that follow. Enter the followingcommand to disable the XA-Core to ERS 8600 chassis X(chassis containing I/O module to be replaced) Ethernet link.

bsy <slot> r link force

Accept the state change by typing Y

Executing this command produces an alarm indicating that theport is down. Ignore this alarm.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 399: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 399

ATTENTIONIn the previous configuration, one HIOP is connected to each ERS8600.

--End--

Variable definitionsVariable Value

chassis X Chassis containing module to be replaced.

chassis Y Chassis not containing module to be replaced.

M = 0 or 1, depending on which volume isconnected to the ERS 8600

** Refers to the release (for example, 091 forSN09.1)

x.y.z Specifies the IP Net address provided.

sd0M The STORM volume defined.

<subsystem> The list of available subsystems executed.

<slot> The slot number of the ATMIF blade.

<sonetif> Sonet interface on the ATM blade.

<STORM_address> The IP address of the STORM device.

<volume> The volume on the STORM determined.

<variable> The regular or parallel.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 400: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

400 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Locking the USP-Compact

Lock the USP-Compact to secure the USP-Compact.

Prerequisites• Ensure you have a replacement I/O module.

• Make sure you use READ/WRITE/ALL (RWA) logon privileges,password privileges or both when performing this procedure. For moreinformation about RWA privileges, refer to Passport Documentation inthe Nortel website.

• Access the ERS 8600s using the serial (console) port. You can use adirect serial connection and a terminal program from an attached PCor a Telnet connection to an appropriately configured IP address on aterminal server connected to the console port.

• If any of the following devices are connected to the I/O module beingreplaced, ensure that it has a unit subtending to the other ERS 8600switch:

— Border Control Point (BCP) 7100

— Media Gateway (MG) 7000 or MG 15000

ATTENTIONOnly if the I/O card is an 8672

— Gateway Controller (GWC)

— High performance Input/Output Processor (HIOP)

— USP/USP Compact

— STORM-IA

— Shelf Controller (SC)

Procedure stepsATTENTIONPrior to performing step 2 through step 17, ensure that the slave USPc blade isbeing addressed.

Determine the USPc blade-to-ERS 8600 connectivity. The slave USPc blademust connect to ERS 8600 chassis X (chassis containing I/O module to bereplaced); master USPc blade to chassis Y (chassis not containing I/O moduleto be replaced).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 401: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 401

Step Action

1 Preparing the slave USPc blade for locking consists of thefollowing tasks:

2 Inhibiting all SS7 links on the slave USPc blade.

ATTENTIONIf only one link is available on a linkset, it cannot be inhibited. Ifthere are other links on the linkset, ensure at least one of the links isavailable before attempting to inhibit the link.

3 Deactivating all SS7 links on the slave USPc blade.

4 Bringing down all IPS7 paths on the slave USPc blade.

5 Locking the slave USPc blade from the USPc GUI.

ATTENTIONBoth USPc blades carry traffic and are in service.

6 Launch the USPc GUI.

7 Select Configuration --> platform --> Node.

The following window displays.

8 Select the Search tab and click Retrieve.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 402: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

402 I/O module replacement in an ERS 8600 in a carrier CS-LAN

9 From the list of USP-Compact cards, select the rowcorresponding to the card to be locked and click Post.

10 Verify the master and slave USPc blade-to-ERS 8600connectivity by selecting the Graphical View tab.

The following figure displays the Graphical View of the USPcblades.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 403: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 403

The icon with a checkmark identifies the master USPc blade.Slot 12 contains the master USPc blade. Slot 15 contains theslave USPc blade.

11 Select the next step as follows.

12 In the Administration tab, click Swact to perform a SWACT onthe two USPc blades.

13 Turn down the paths and the links associated with the slaveblade to be locked.

Select Configuration --> MTP --> Link from the pull-downmenu.

Select Search->Retrieve.

The following screen appears.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 404: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

404 I/O module replacement in an ERS 8600 in a carrier CS-LAN

14 Refer to the Slot column to verify the SS7 links against the USPcblades. Select any SS7 links that are against the slave USPcblade.

Post the link.

Click Inhibit to inhibit the links that apply to the slave USPcblade.

The following prompt displays.

The inhibit command will be executed for all posted records. Are you sure you want to continue?

15 Click Yes to execute the command on the selected links.

16 Using the same screen, reselect the links, and click Deactivate.

The following window displays.

17 Click Posted to execute the command on the selected links.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 405: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 405

18 Post the IPS7 paths that use the slave USPc blade.

Select Configuration --> ips7 --> application-server-process-path.

Select Search->Retrieve.

The following screen appears.

19 Post the IPS7 paths against the slave USPc blade.

Select all the IPS7 paths against the slave USPc blade, and clickDown.

The down command will be executed for all posted records. Are you sure you want to continue?

20 Click Yes to execute the command on the IPS7 paths on theslave USPc blade.

The IPS7 paths and SS7 links against the slave USPc bladehave been deactivated.

21 Lock the slave USPc blade.

Select Configuration --> platform --> Node.

The Administration tab redisplays.

22 Click Lock.

The following prompt displays:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 406: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

406 I/O module replacement in an ERS 8600 in a carrier CS-LAN

The node lock command will impact the last in-servicelink in at least 1 linkset and cause at least 1 routeset to become inaccessible which may impact networktraffic. Do you want to continue?

23 Click Yes to lock the USPc blade.

Click the Graphical View tab.

The Graphical View tab shows that the USPc blade has a lockicon next to it, indicating it has been locked.

24 Access the SAM21 Element Manager.

25 Select the SAM21 shelf by double-clicking on its icon.

The following figure shows an example of a shelf which containsa slave USPc blade residing on slot 14.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 407: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 407

26 Right-click on the icon that corresponds to where the slave USPcblade resides.

A sub-window appears with the Card View and Lock selectionsenabled.

27 Click Lock.

The slave USPc blade is locked.

ATTENTIONThis prevents an automatic change of activity from occurring. Anautomatic SWACT at this point can cause a system outage.

ATTENTIONAn alternative method to lock the blade from the SAM21 Manager isto first open the card by double-clicking on it. Then click Lock in theStates tab.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 408: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

408 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Taking down the paths to the ERS 8600 chassis X

Use this procedure to take down the paths to the ERS 8600 chassis X

Prerequisites• Ensure you have a replacement I/O module.

• Make sure you use READ/WRITE/ALL (RWA) logon privileges,password privileges or both when performing this procedure. For moreinformation about RWA privileges, refer to Passport Documentation inthe Nortel website.

• Access the ERS 8600s using the serial (console) port. You can use adirect serial connection and a terminal program from an attached PCor a Telnet connection to an appropriately configured IP address on aterminal server connected to the console port.

• If any of the following devices are connected to the I/O module beingreplaced, ensure that it has a unit subtending to the other ERS 8600switch:

— Border Control Point (BCP) 7100

— Media Gateway (MG) 7000 or MG 15000

ATTENTIONOnly if the I/O card is an 8672

— Gateway Controller (GWC)

— High performance Input/Output Processor (HIOP)

— USP/USP Compact

— STORM-IA

— Shelf Controller (SC)

Procedure steps

Step Action

1 Launch the USP GUI.

Taking down the paths to the ERS 8600 chassis X consists ofthe following tasks:

2 Identifying all IP cards in the USP, and noting which EthernetRouting Switch chassis they are connected .

3 Ensuring that for each path, there is an alternate in-service paththrough chassis Y

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 409: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 409

4 Taking down all IPS7 paths using chassis X.

5 Select Configuration->platform->node->Search.

6 Click retrieve, and look for all records that have a slot-type ofiplink.

7 Note the following information for each IP link card:

• Shelf

• Slot

• IP Address

• ERS8600 unit it is connected to

8 Select Configuration->ips7->application-server-process-path->Search.

9 Click retrieve, and look for all paths on the cards noted in step 7,that are connected to chassis X.

10 Click on the asp-name column to sort the table, and confirm thatfor each path connected to a card using chassis X, there is analternate path using chassis Y that has a path-state of up. Referto the information you noted in step 7.

CAUTIONPotential service interruptionIf the alternate path to any destination is not inservice, performing the step that follows will result ina service interruption.

11 Select each path connected to chassis X, and click Post, thenconfirm that only the paths associated with the cards connectedto chassis X, are selected. Refer to the information you noted instep 7.

12 Click Down to take the posted paths out of service.

13 Click Yes to confirm the command execution on the IPS7 pathsassociated with the cards connected to chassis X.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 410: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

410 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Determining if STORM devices require rotating,demounting and busying

Determine if STORM devices require rotating, demounting and busying toestablish the requirement.

Prerequisites• Ensure you have a replacement I/O module.

• Make sure you use READ/WRITE/ALL (RWA) logon privileges,password privileges or both when performing this procedure. For moreinformation about RWA privileges, refer to Passport Documentation inthe Nortel website.

• Access the ERS 8600s using the serial (console) port. You can use adirect serial connection and a terminal program from an attached PCor a Telnet connection to an appropriately configured IP address on aterminal server connected to the console port.

• If any of the following devices are connected to the I/O module beingreplaced, ensure that it has a unit subtending to the other ERS 8600switch:

— Border Control Point (BCP) 7100

— Media Gateway (MG) 7000 or MG 15000

ATTENTIONOnly if the I/O card is an 8672

— Gateway Controller (GWC)

— High performance Input/Output Processor (HIOP)

— USP/USP Compact

— STORM-IA

— Shelf Controller (SC)

Procedure stepsATTENTIONBefore proceeding with the following STORM-related steps, consider if otherapplications will be affected by the loss of STORM disk volumes.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 411: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 411

Step Action

1 If STORM uses a Dothill device then go to step 2 and if STORMIA (SAM-XTS) device1 then you have completed this procedure.

1. The SAM-XTS configuration uses dual-homed split multilink trunks (SMLT). Both STORM

volumes are recognized by both ERS 8600 routers. Therefore, taking down one router in an

SAM-XTS configuration does not affect the STORM volume being addressed.

2 If the volumes on any STORM disk (for example, sd01) areactively mounted in Device Independent Recording Package(DIRP) 1 then go to step 3 and if the volumes on the STORMdisk are not actively mounted DIRP2 then you have completedthis procedure.

1. Loss of network connectivity between SOS, the call processing application, and the active

STORM disk can cause some or all DIRP volumes to be marked “INERROR,” resulting in

permanent loss of data.

2. Regardless of whether DIRP volumes are mounted, the procedure can result in an

unavailable STORM disk and associated volumes.

3 If needed, enter the following command to check which STORMdevice is connected to the ERS 8600. If configured properly,XTS0 should have a lower IP address than XTS1.

sho ip arp info <STORM_address>

4 Logon to the Call Agent as a maintenance user.

5 Enter the DF command to map the STORM volumes to XTS IPaddresses.

$ df

Example response

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 412: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

412 I/O module replacement in an ERS 8600 in a carrier CS-LAN

6 Based on the responses of step 3 and step 5, determine whichXTS volume (SD00 or SD01) corresponds with the STORM.

7 Logout of the Call Agent.

8 From the Core MAP command line, enter the following commandto list all available DIRP subsystems (field SYSSNAME in tableDIRPSSYS) for the Core.

>table dirpssys; lis all

9 Enter the following command to access the DIRP level in SOS.

>mapci nodisp;mtc;iod;dirp

10 Enter the following command to query all volumes associatedwith the subsystem.

>query <subsystem> all

The following figure shows an example of this command for thedlog subsystem with an active file highlighted.

11 Enter the following command to rotate the active volume for eachsubsystem if it is located on a STORM blade connected to theERS 8600. Manually rotate all subsystems until the active files

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 413: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 413

reside on STORM volumes connected to the other ERS 8600chassis.

>rotate <subsystem> <variable>

ATTENTIONParallel is intentionally misspelled.

ATTENTIONThe volume associated with the slave ERS 8600 must not be active.Ensure that no volume is active in each pass which is prefixed withsd0M.

(Rotate each subsystem with volumes prefixed with sd0M thatare active.)

ATTENTIONTo minimize risk, rotate and demount the AMA subsystem last. Thenrestore the AMA subsystem first.

The following figure shows example of input for variable regular.

12 Repeat step 11 to ensure that all volumes needing rotating havebeen rotated successfully.

13 Enter the following command to demount all DIRP volumesconnected to the ERS 8600 chassis that contains the CPU to bereplaced.

>dmnt <subsystem> <volume> <variable>

ATTENTIONThe <variable> parameter is not required when mounting anddemounting regular DIRP volumes

ATTENTIONNortel recommends recording all subsystems, volume, and variableinformation for each volume being demounted in this step.

14 Enter the following command to busy all STORM volumesconnected to the ERS 8600 chassis that contains the CPU to bereplaced

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 414: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

414 I/O module replacement in an ERS 8600 in a carrier CS-LAN

>diskadm sd0M; bsy all

ATTENTIONNortel does not recommend using option bsy all force.

ATTENTIONIssuing a bsy all command affects all volumes serviced by thatSTORM blade, not just those used by DIRP.

--End--

Variable definitionsVariable Value

<STORM_address> The IP address of the STORM device.

<subsystem> The list of available subsystems executed.

<variable> The regular or parallel.

<volume> The volume on the STORM determined.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 415: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 415

Locking the inactive Shelf Controller (SC) bladeconnected to chassis X

Lock the inactive Shelf Controller (SC) to secure the inactive SC bladeconnected to chassis X.

Prerequisites• Ensure you have a replacement I/O module.

• Make sure you use READ/WRITE/ALL (RWA) logon privileges,password privileges or both when performing this procedure. For moreinformation about RWA privileges, refer to Passport Documentation inthe Nortel website.

• Access the ERS 8600s using the serial (console) port. You can use adirect serial connection and a terminal program from an attached PCor a Telnet connection to an appropriately configured IP address on aterminal server connected to the console port.

• If any of the following devices are connected to the I/O module beingreplaced, ensure that it has a unit subtending to the other ERS 8600switch:

— Border Control Point (BCP) 7100

— Media Gateway (MG) 7000 or MG 15000

ATTENTIONOnly if the I/O card is an 8672

— Gateway Controller (GWC)

— High performance Input/Output Processor (HIOP)

— USP/USP Compact

— STORM-IA

— Shelf Controller (SC)

Procedure stepsATTENTIONDuring a SWACT on the SCs, flapping (that is, rapid enabling and disabling ofthe router port connected to an SC) can result in temporary loss of Ethernetconnectivity. This can cause ongoing failures on the newly active SC followingthe SWACT.

To prevent the possibility of an outage from this condition, monitor ongoing orrecent logs for Ethernet instability on the router ports that are associated withthe SCs.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 416: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

416 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Do not SWACT the SCs until all network problems involving flapping areaddressed

Step Action

1 Confirm that SC unit 0s (in slot 7) subtend ERS 8600 chassis0, and SC unit 1s (in slot 9) subtend ERS 8600 chassis 1for all SAM21 shelves. If the SAM21 shelf varies from thisrecommendation, adjust this procedure accordingly.

2 Lock the inactive SAM21 SC blade for each SAM21 shelf in turnas follows:

3 Launch the SAM21 Manager.

The SAM21 Manager GUI appears with the related shelf iconsdisplayed.

4 Use the cursor to select the shelf icon for the first SAM21 shelfdisplayed.

Example

The first shelf in the preceding figure is SAM21-1 CSAM00-01

5 Double-click the selected shelf icon.

The SAM21 Manager shelf view for the selected shelf appears.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 417: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 417

6 Right-click on the icon for the SC blade associated with theinactive ERS 8600 chassis.

A sub-window appears with the Card View and Lock selectionsenabled.

7 Click Card View.

The SC Card View appears.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 418: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

418 I/O module replacement in an ERS 8600 in a carrier CS-LAN

8 Select the States tab and make sure the SC is inactive.

If the SC state is Active, click Swact to change the activitystatus to Inactive.

9 Close the Card View.

The Shelf View reappears.

10 Right-click on the icon for the inactive SC blade.

A sub-window appears with the Card View and Lock selectionsenabled.

11 Click Lock.

The inactive SC blade is locked.

ATTENTIONThis prevents an automatic change of activity from occurring. Anautomatic SWACT at this point can cause a system outage.

12 Close the Shelf View.

The SAM21 Manager view reappears.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 419: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 419

Unlocking the Border Control Point 7100

Unlock the Border Control Point 7100 to release or open the BCP 1700.

Prerequisites• Ensure you have a replacement I/O module.

• Make sure you use READ/WRITE/ALL (RWA) logon privileges,password privileges or both when performing this procedure. For moreinformation about RWA privileges, refer to Passport Documentation inthe Nortel website.

• Access the ERS 8600s using the serial (console) port. You can use adirect serial connection and a terminal program from an attached PCor a Telnet connection to an appropriately configured IP address on aterminal server connected to the console port.

• If any of the following devices are connected to the I/O module beingreplaced, ensure that it has a unit subtending to the other ERS 8600switch:

— Border Control Point (BCP) 7100

— Media Gateway (MG) 7000 or MG 15000

ATTENTIONOnly if the I/O card is an 8672

— Gateway Controller (GWC)

— High performance Input/Output Processor (HIOP)

— USP/USP Compact

— STORM-IA

— Shelf Controller (SC)

Procedure steps

Step Action

1 From the System tree, click the plus sign next to NetworkElements to expand the view of the folder.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 420: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

420 I/O module replacement in an ERS 8600 in a carrier CS-LAN

2 Next, click the plus sign next to RTP Portals to expand the viewof the folder. The contents of this folder lists all BCP 7100s andBCP 7200s configured for the site.

3 Continue to expand the view of folders until the target BCP 7100is selected. Do not modify the state of any BCP 7200s.

4 Click the NE Maintenance option. A new window opens.

5 Click the Start button to restart the previously shut down BCP7100.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 421: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 421

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 422: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

422 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Unlocking the inactive SC

Unlock the inactive SC to release or open the inactive SC.

Prerequisites• Ensure you have a replacement I/O module.

• Make sure you use READ/WRITE/ALL (RWA) logon privileges,password privileges or both when performing this procedure. For moreinformation about RWA privileges, refer to Passport Documentation inthe Nortel website.

• Access the ERS 8600s using the serial (console) port. You can use adirect serial connection and a terminal program from an attached PCor a Telnet connection to an appropriately configured IP address on aterminal server connected to the console port.

• If any of the following devices are connected to the I/O module beingreplaced, ensure that it has a unit subtending to the other ERS 8600switch:

— Border Control Point (BCP) 7100

— Media Gateway (MG) 7000 or MG 15000

ATTENTIONOnly if the I/O card is an 8672

— Gateway Controller (GWC)

— High performance Input/Output Processor (HIOP)

— USP/USP Compact

— STORM-IA

— Shelf Controller (SC)

Procedure steps

Step Action

1 Launch the SAM21EM client.

The SAM21 Manager GUI appears.

2 Unlock each locked SC blade as follows:

3 Right-click on each locked SC blade.

4 Select Unlock from the drop-down menu that appears.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 423: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 423

ATTENTIONA padlock icon identifies locked SC blades.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 424: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

424 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Unlocking the USP-Compact

Unlock the USP-Compact to release or to open the USP-Compact.

Prerequisites• Ensure you have a replacement I/O module.

• Make sure you use READ/WRITE/ALL (RWA) logon privileges,password privileges or both when performing this procedure. For moreinformation about RWA privileges, see the Passport Documentation inthe Nortel website.

• Access the ERS 8600s using the serial (console) port. You can use adirect serial connection and a terminal program from an attached PCor a Telnet connection to an appropriately configured IP address on aterminal server connected to the console port.

• If any of the following devices are connected to the I/O module beingreplaced, ensure that it has a unit subtending to the other ERS 8600switch:

— Border Control Point (BCP) 7100

— Media Gateway (MG) 7000 or MG 15000

ATTENTIONOnly if the I/O card is an 8672

— Gateway Controller (GWC)

— High performance Input/Output Processor (HIOP)

— USP/USP Compact

— STORM-IA

— Shelf Controller (SC)

Procedure steps

Step Action

1 Access the SAM21 Manager. Unlock the locked slave USPcblade.

ATTENTIONUnlocking the slave USPc blade from the SAM21 Manager enablesthe USPc GUI.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 425: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 425

ATTENTIONAn alternative method to unlock the blade from the SAM21 Manageris to open the card by double-clicking on it. Click Unlock in theStates tab

2 Access the USPc GUI, waiting to ensure that the inactive bladeis enabled and locked.

ATTENTIONRe-accessing the USPc GUI can take several minutes.

3 Select Configuration --> platform --> Node.

The window opens as follows:

4 Select the Search tab and click Retrieve.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 426: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

426 I/O module replacement in an ERS 8600 in a carrier CS-LAN

5 From the list of USP-Compact cards, select the rowcorresponding to the card to be unlocked and click Post.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 427: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 427

6 Select the Administration tab.

The Administration tab displays.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 428: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

428 I/O module replacement in an ERS 8600 in a carrier CS-LAN

7 Click Unlock to unlock the slave USPc blade.

8 Reactivate the SS7 links and the paths associated with the slaveUSPc blade.

From the Administration tab, select Configuration --> MTP -->Link from the pull-down menu.

The following screen appears.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 429: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 429

9 Select any SS7 links displayed in the form that are against theslave blade, and click Activate.

10 Select Yes to execute the command on the selected links.

11 Using the same screen, reselect the links, and click Uninhibit.

12 Select Yes to execute the command on the selected links.

13 Post the IPS7 paths that use the slave USPc blade.

From the menu, select Configuration --> ips7 --> application-server-process-path from the pull-down menu.

The following screen appears.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 430: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

430 I/O module replacement in an ERS 8600 in a carrier CS-LAN

14 Select all the IPS7 paths against the slave USPc blade, and clickUp.

The following prompt displays:

execute ips7 application-serverprocess-path up command ?

15 Select Yes to execute the command on the IPS7 paths on theslave USPc blade.

16 If desired, perform an optional SWACT on the USPc blades torestore initial mastership states.

To perform a SWACT, redisplay the Administration tab, asshown in step 3. Press the Swact button in the upper-rightportion of the screen.

17 Select the next step as follows:

18 Launch the SAM21EM client.

The SAM21 Manager GUI appears.

19 Unlock each locked USPc blade as follows:

Right-click on each locked USPc blade.

20 Select Unlock from the drop-down menu that appears.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 431: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 431

ATTENTIONA padlock icon identifies locked USPc blades.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 432: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

432 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Returning the paths to the ERS 8600 chassis X toservice

Return the paths to service the ERS 8600 chassis X.

Prerequisites• Ensure you have a replacement I/O module.

• Make sure you use READ/WRITE/ALL (RWA) logon privileges,password privileges or both when performing this procedure. For moreinformation about RWA privileges, see the Passport Documentation inthe Nortel website.

• Access the ERS 8600s using the serial (console) port. You can use adirect serial connection and a terminal program from an attached PCor a Telnet connection to an appropriately configured IP address on aterminal server connected to the console port.

• If any of the following devices are connected to the I/O module beingreplaced, ensure that it has a unit subtending to the other ERS 8600switch:

— Border Control Point (BCP) 7100

— Media Gateway (MG) 7000 or MG 15000

ATTENTIONOnly if the I/O card is an 8672

— Gateway Controller (GWC)

— High performance Input/Output Processor (HIOP)

— USP/USP Compact

— STORM-IA

— Shelf Controller (SC)

Procedure steps

Step Action

1 At the USP GUI

Post the IPS7 paths that use nodes connected to chassis X byselecting Configuration->ips7->application-server-process-path

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 433: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 433

2 Select each path connected to chassis X and click Post, thenconfirm that only the paths associated with the cards connectedto chassis X, are selected.

3 Click Up to bring the posted paths back into service.

4 Click Yes to confirm the command execution on the IPS7 pathsassociated with the cards connected to chassis X.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 434: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

434 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Enabling the Ethernet links between the XA-Coreand the ERS 8600

Enable the Ethernet links between the XA-Core and the ERS 8600 to allowthe Ethernet links.

Prerequisites• Ensure you have a replacement I/O module.

• Make sure you use READ/WRITE/ALL (RWA) logon privileges,password privileges or both when performing this procedure. For moreinformation about RWA privileges, see the Passport Documentation inthe Nortel website.

• Access the ERS 8600s using the serial (console) port. You can use adirect serial connection and a terminal program from an attached PCor a Telnet connection to an appropriately configured IP address on aterminal server connected to the console port.

• If any of the following devices are connected to the I/O module beingreplaced, ensure that it has a unit subtending to the other ERS 8600switch:

— Border Control Point (BCP) 7100

— Media Gateway (MG) 7000 or MG 15000

ATTENTIONOnly if the I/O card is an 8672

— Gateway Controller (GWC)

— High performance Input/Output Processor (HIOP)

— USP/USP Compact

— STORM-IA

— Shelf Controller (SC)

Procedure steps

Step Action

1 Enter the following command to access the ETHR level of theMAP display.

>MAPCI;MTC;XAC;ETHR

2 Refer to the following tables and enable the Ethernet linkbetween the XA-Core and ERS 8600 chassis X, as appropriate.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 435: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 435

3 If the XA-Core uses HIOPs or HCMICs,

rts <slot> r link

4 If the XA-Core uses IOPs with ETHR packlets, enter thefollowing command to enable the link:

rts <slot1> r l link

Wait for the command to complete, then enter the followingcommand:

rts <slot2> r l link

5

--End--

Variable definitionsVariable Value

<slot> The slot number of the ATMIF blade.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 436: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

436 I/O module replacement in an ERS 8600 in a carrier CS-LAN

Accessing the Network Patch Manager CLUI

Access the Network Patch Manager (NPM) command line user interface(CLUI) to get into the NPM CLUI.

ATTENTIONYou can also access the NPM CLUI from the Integrated Element ManagementSystem (IEMS) when the IEMS is present in the office.

ATTENTIONThe Network Patch Manager also has a graphical user interface (GUI). See“Launching the Communication Server 2000 SAM21 Manager Client that resideson the Communication Server 2000 Management Tools Server” (page 197).

Prerequisites• You must have a valid user ID and password to access the NPM

interface.

• You must be assigned to the user group emsadm to perform patchingactivities using the NPM.

• You must establish a logon session with the SPFS-based server usingTelnet (unsecure) or ssh (secure).

Procedure steps

Step Action

1 Access the NPM CLUI by typing

$ npm

and pressing the Enter key.

2 When prompted, enter your user ID and password.

Example response:

Entering shell mode: Enter ’npm’ commands, help or quit to exit.npm>

3 If applicable, return to the high level task or procedure thatdirected you to this procedure.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 437: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

437.

Powering up the XA-CorePower up a dead switch that has an eXtended Architecture Core(XA-Core). The switch can be a DMS SuperNode or SuperNode SEswitch. The switch is dead if the complete switch is without power. Thepower loss results from a loss or interruption of A and B dc power feeds tothe power distribution center (PDC).

Powering up the XA-Core navigation

• “XA-Core power up procedures ” (page 438)

• “Powering up from the power distribution center” (page 440)

• “Powering up from the XA-Core cabinet and terminal” (page 443)

• “Powering up from the MAP terminal” (page 445)

• “Powering up at the IOC/PDC” (page 449)

• “Powering up at the PM frames” (page 452)

• “Completing the power up of the XA-Core” (page 454)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 438: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

438 Powering up the XA-Core

XA-Core power up procedures

This task flow shows you the sequence of procedures you perform topower up the XA-Core.

To link to any procedure, click on “XA-Core power up procedurenavigation” (page 438).

XA-Core power up procedure navigation• “Powering up from the power distribution center” (page 440)

• “Powering up from the XA-Core cabinet and terminal” (page 443)

• “Powering up from the MAP terminal” (page 445)

• “Powering up at the IOC/PDC” (page 449)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 439: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

XA-Core power up procedure navigation 439

• “Powering up at the PM frames” (page 452)

• “Completing the power up of the XA-Core” (page 454)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 440: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

440 Powering up the XA-Core

Powering up from the power distribution center

Power up from the power distribution center (PDC).

Procedure stepsCAUTIONCall ETAS or your next level of supportIn the event of a dead system, call the Emergency TechnicalAssistance Services (ETAS) of Nortel. Also call your next levelof support before you perform this procedure.

DANGERRisk of electrocutionDo not touch the cabinet wiring. Connections with unshieldedcabinet wiring can result in electric shock. Only qualified powermaintenance personnel can perform the voltage measurementsin step 3.

Step Action

1 At the PDC

When possible after detection of the power outage, remove allthe fuse holders for the line concentrating equipment (LCE) talkbatteries, trunk module equipment (TME) talk batteries and PDCfilter fuses from the correct PDCs

ATTENTIONThe location of the fuse holders in the fuse panel can vary, dependingon your office configuration. For help in locating the fuse holders,refer to the fuse assignment diagram for your office. Also call yournext level of support to help you locate the fuses.

2 The next step depends on if the switch power is a -48 V dc feedor by a -60 V dc feed.

3 If the switch power is -48 V dc then go to step 5.

4 If the switch power is -60 V dc then go to step 8.

5 Continue when you know of restored power at the power plantfor your office. Power maintenance personnel must check forrestored power at each PDC. At the rear of each PDC, measurethe dc voltage across the A feed bus and the battery return plate.Repeat the dc voltage measurement for the B feed bus. Power iscorrect when the voltage on each feed is -48 V dc.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 441: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 441

ATTENTIONPower can be at a nominal potential of -48 V dc. Under conditionsthat are not normal, the operating voltage can range from -43.75 V dcto -55.8 V dc. A not normal condition is a commercial power failure

6 If the switch has power retrieval then go to step 11.

7 If the switch has no power retrieval then go to step 12.

8 Continue when you know of restored power at the power plantfor your office. Request power maintenance personnel to checkfor restored power at each PDC. At the rear of each PDC,measure the dc voltage across the A feed bus and the batteryreturn plate. Repeat the dc voltage measurement for the B feedbus. Power is correct when the voltage on each feed is -60 V dc.

ATTENTIONPower can be at a nominal potential of -60 V dc. Under conditionsthat are not normal, the operating voltage can range from -57.4 V dcto -67.7 V dc. A not normal condition is a commercial power failure.

9 If power has power retrieval then go to step 11.

10 If power has no power retrieval then go to step 12.

11 At the power room

For help in restoring power to the PDC, call the personnelresponsible for maintenance of power at your site.

When power restores to the PDC, return to this point.

12 At the PDC

Inspect the alarm indication fuses for the XA-Core and thenetwork cabinets.

13 If fuses that blew are present then go to step 15.

14 If fuses that blew are not present then go to step 23.

15 Replace the blown cartridge fuse in the rear of the fuse holder.Make sure that the amperage of the replacement cartridge fusematches the amperage marked on the PDC.

16 Remove the blown alarm-indication fuse from the front of thefuse holder.

17 Insert the fuse holder, with the alarm-indication fuse removed,into the PDC.

18 Get an alarm-indication fuse for replacement.

19 Insert the alarm-indication fuse that is for replacement into thefuse holder.

20 If the replacement fuse is not successful and blows repeatedlythen go to step 22.

21 If the replacement fuse is successful then go to step 23.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 442: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

442 Powering up the XA-Core

22 Call your next level of support for help.

Continue when you complete replacement of all blown fuses andrestored power to the XA-Core and network cabinets.

23 If a second person is available to help in the recovery, continuethis procedure with two sets of tasks. Request the secondperson to restore power from the pdc to the peripheral moduleframes. The second person restores power by use of stepsspecified in “Powering up at the IOC/PDC” (page 449). While thesecond person restores power, you recover the core and networkby use of steps specified in “Powering up from the XA-Corecabinet and terminal” (page 443)and “Powering up from the MAPterminal” (page 445). If one person is available, recover the coreand network first.

24 Determine if the switch has a remote oscillator shelf.

25 If the switch has a remote oscillator shelf then go to step 16.

26 If the switch has no remote oscillator shelf then do step 1 in“Powering up from the XA-Core cabinet and terminal” (page443).

27 At the remote oscillator shelf

Turn on the power converters for the shelf.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 443: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 443

Powering up from the XA-Core cabinet and terminal

Power up from the XA-Core cabinet and terminal.

Procedure steps

Step Action

1 At the XA-Core cabinet

Determine if the switch is a SuperNode switch or a SuperNodeSE switch.

2 If the switch is a SuperNode switch then go to step 4.

3 If the switch is a SuperNode SE switch then go to step 7.

4 Turn on circuit breakers for shelf interface module (SIM) A. Turnon the three switches for circuit breakers A1, A2, and A3. TheSIM A card is in slot 3R of the XA-Core shelf.

5 Turn on circuit breakers for shelf interface module (SIM) B. Turnon the three switches for circuit breakers B1, B2, and B3. TheSIM B card is in slot 16R of the XA-Core shelf.

6 Go to step 9.

7 Turn on circuit breakers for shelf interface module (SIM) A. Turnon the two switches for circuit breakers A1 and A2. The SIM Acard is in slot 3R of the XA-Core shelf.

8 Turn on circuit breakers for shelf interface module (SIM) B. Turnon the two switches for circuit breakers B1 and B2. The SIM Bcard is in slot 16R of the XA-Core shelf.

9 Determine if all the power converters have power. You know thatall the power converters have power when all the Converter Offlights go off.

10 If all the power converters have power then go to step 14.

11 If all the power converters have not power then go to step 12.

12 To power up the frame perform the procedure, "Clearing anExt FSP DPCC cabinet major alarm" in Alarm Clearing andPerformance Monitoring Procedures () , 297-8001-543 (NorthAmerican market) or 297-9051-543 (International market).

When you completed the procedure, return to this point.

13 Go to step 1 in “Powering up from the MAP terminal” (page 445).

14 At the XA-Core reset terminal

Monitor the XA-Core reset terminal to determine if the switch hasbooted.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 444: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

444 Powering up the XA-Core

When the switch boots, the XA-Core reset terminal displaysa response to indicate a boot in progress. The response alsodisplays different diagnostic messages and alphanumericaddresses. When the switch has completely booted, an A1appears on the RTIF display.

15 If the response has an A1 then go to step 1 in “Powering up fromthe MAP terminal” (page 445).

16 If the response has no A1 after approximately 15 min thencontact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 445: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 445

Powering up from the MAP terminal

Power up from the MAP terminal.

Procedure steps

Step Action

1 At the MAP terminal

CAUTIONExtended service interruptionThe exact log on procedure can vary, depending onyour office configuration. If you need additional help,call the personnel responsible for the next level ofsupport.

Determine if you have to log on.

ATTENTIONThe log on message indicates that you have to manually log on. Anautomatic log on can occur if the office parameters have automaticlog on.

Example of a MAP response

Please Login.

2 If the log on is not automatic then go to step 4.

3 If the log on is automatic then go to step 8.

4 Press the break key.

Example of a MAP response

?

5 Log on to the MAP terminal.

>LOGIN

Example of a MAP response

Enter User Name

6 Enter the user name.

>user_name

Example of a MAP response

Enter Password

7 Enter the password.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 446: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

446 Powering up the XA-Core

>password

Example of a MAP response

SuperNode1 Logged in on 1997/01/15 at 20:37:17

8

CAUTIONAll customers must follow the sequence of stepsset out in this procedure.

Do not interrupt this procedure at this point to clearan alarm. If a TOD critical alarm appears under theAPPL level in the alarm banner, and if the systemuses Network Time Protocol (NTP), you mustcomplete all steps in the sequence shown. You willclear the TOD critical alarm by completing step 39.(For information on NTP, see step 39.)

Turn on priority.

>PRIORITY ON

Example of a MAP response

Pref>

9 Determine if the system time is correct.

>TIME

Example of a MAP response

Time is 14:55:50

10 If the system time is correct then go to step 14.

11 If the system time is not correct then go to step 12.

12 Enter the correct time (by use of the 24 hour clock).

>SETTIME hh mm

Example of a MAP response

Warning:There is an automated TOD clock changerequest scheduled on:1997/10/15 at 1:00 (see table DSTTABLE).Do you want to proceed with this request?Please confirm ("YES", "Y", "NO", or "N")

13 Confirm the command.

>YES

Example of a MAP response

Time is 20:40:00 on WED 1997/10/15.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 447: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 447

14 Determine if the system date is correct.

15 If the system date is correct then go to step 20.

16 If the system date is not correct then go to step 17.

17 Enter the correct date.

>SETDATE dd mm yyyy

Example of a MAP response

Warning: There is an automated TOD clock changerequestscheduled on:1997/10/15 at 1:00 (see table DSTTABLE).Do you want to proceed with this request?Please confirm ("YES", "Y", "NO", or "N")

18 Confirm the command.

>YES

Example of a MAP response

Date is WED. 15/OCT/1997 00:00:00

19 If the system uses Network Time Protocol (NTP) as the timingreference, and if a TOD critical alarm is displayed under theAPPL level in the alarm banner.

ATTENTIONIn the German market only, switches can use Network Time Protocol(NTP) as the timing reference for the time-of-day clock. The systemuses Network Time Protocol if the value of the SNTP_CLIENT officeparameter in table OFCENG has been set to Y.

20 At the MAP terminal

Access the NET level of the MAP display.

>NET

Example of a MAP response

NET

11111 11111 2222222222 33Plane 01234 56789 01234 56789 0123456789 010 00001 0000JNET

21 Manually busy the network module for return to service.

>BSY plane_no pair_no

22 Return the network module to service.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 448: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

448 Powering up the XA-Core

>RTS plane_no pair_no

23 Repeat step 21 through step 22 for each JNET shelf.

When all JNET shelves recover, continue this procedure at step24.

24 Determine if there are additional input output controller (IOC)and maintenance and administration position (MAP) terminalsto recover.

25 If recover of additional IOCs and MAP terminals is then go tostep 24.

26 If recover of additional IOCs and MAP terminals is then go tostep 13 in “Powering up at the IOC/PDC” (page 449).

27 Restore power to all remaining power inverters in the office.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 449: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 449

Powering up at the IOC/PDC

Power up at the IOC/PDC.

Procedure steps

Step Action

1 At the IOC

Locate the IOC for recovery.

2 If recovery is for an IOC then go to step 4.

3 If recovery is for an IOM then go to step 8.

4 Turn on the power converters on the IOC.

ATTENTIONThe version of IOC determines if the IOC has one or two powerconverters.

5 While you press the reset button on one of the IOC powerconverters, lift the related circuit breaker to turn on the FSP.

6 Release the reset button.

7 Repeat step 1through step 6for each IOC for recovery, thencontinue this procedure at step 8.

8 Access the input output device (IOD) level of the MAP display.

>IOD

9 Access the IOC level of the MAP display for the IOC forrecovery.

>BSY ioc_no

10 Return the IOC or IOM to service.

>RTS ioc_no

11 Repeat step 9 and step 10 for each IOC or IOM for recovery,then continue this procedure at step 12.

12 Log on to additional MAP terminals as required.

ATTENTIONstep 4 through step 8 in “Powering up from the MAP terminal” (page445) describe how to log on to the MAP terminal.

13 At the PDC

step 16 through step 38 describe how to restore power from thePDC to Series I and Series II peripheral module frames.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 450: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

450 Powering up the XA-Core

14 If the PDC power to the PM frames restores correctly then go tostep 1 in “Powering up at the PM frames” (page 452).

15 If the PDC power to the PM frames restores not correctly thengo to step 16.

16 Get a 100-W, 120-V light bulb installed into a socket thathas pigtail leads or tool number T000655 (CPC numberNTA0600512), that has a fuse holder-style connector instead ofpigtail leads for easier insertion

17

WARNINGPossible equipment damage or extendedservice interruptionUse correct fuses. When you return fuses to thePDC, make sure that the amperage of the fusesis correct. The fuse amperage must match theamperage marked on the PDC.

At the first empty fuse slot in the PDC, connect the leads of thecapacitor charging tool. Connect the leads across the contactsfor the fuse holder until the lamp decreases brightness. If youuse a charging tool with a fuse holder-style connector, insertthe connector into the slot. Insert the connector until the lampdecreases brightness.

18 Remove the capacitor charging tool and immediately insert againthe correct fuse holder into the slot.

19 Repeat step 58 and step 59 for all the LCE talk battery, TMEtalk battery, and PDC filter fuse holders you removed in >>step1<<. When all fuses restore to the PDCs, continue with thisprocedure.

20 Determine if any alarm-indicating fuses blew.

ATTENTIONThe fuse alarm-indicator lamp lights when an alarm-indicating fuseblows.

21 If any alarm-indicating fuses have blown then contact your nextlevel of support.

22 If any alarm-indicating fuses have not blown then go to step 1 in“Powering up at the PM frames” (page 452).

23 Locate a fuse holder with a blown alarm-indicating fuse.

ATTENTIONYou can replace blown fuses in any order.

24 The cartridge fuse in the fuse holder has blown. Remove thefuse holder from the PDC.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 451: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 451

25 Replace the blown cartridge fuse in the rear of the fuse holder.Make sure that the amperage of the replacement cartridge fusematches the amperage marked on the PDC.

26 Remove the blown alarm-indicating fuse from the front of thefuse holder.

27 Insert again the fuse holder, with the alarm-indicating fuseremoved, into the PDC.

28 Get a replacement alarm-indicating fuse.

29 Insert the replacement alarm-indicating fuse into the fuse holder.

30 Determine if the alarm-indicating fuse blows.

ATTENTIONThe fuse alarm indicator lamp lights when an alarm-indicating fuseblows.

31 If the alarm-indicating fuse is blown then go to step 23.

32 If the alarm-indicating fuse is not blown then go to step 33.

33 Determine if you replaced all the blown alarm-indicating fuses.

34 If you have replaced all the blown alarm-indicating fuses thengo to step 36.

35 If you have not replaced all the blown alarm-indicating fuses thengo to step 23.

36 Determine if the fuse alarm indicator lamp lit.

37 If the fuse alarm indicator lamp lit yes then contact your nextlevel of support.

38 If the fuse alarm indicator lamp lit no then go to step 1 in“Powering up at the PM frames” (page 452).

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 452: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

452 Powering up the XA-Core

Powering up at the PM frames

Power up at the PM frames.

Procedure steps

Step Action

1 At the PM frames

Select a peripheral module (PM) frame to power up.

ATTENTIONThe PM frames can power up in any order.

2 Locate the frame supervisory panel (FSP) and the powerconverters on the frame.

3 Determine if the FSP for the frame has fuses or circuit breakers.

4 If the FSP has fuses then go to step 6.

5 If the FSP has circuit breakers then go to step 17.

6 Determine if the power converters have Power Reset buttons orPower Reset switches.

7 If the power converters have power Reset buttons then go tostep 9.

8 If the power converters have power Reset switches then go tostep 13.

9 To turn on each power converter press and hold its Power Resetbutton for 2s.

ATTENTIONThe Converter Fail light goes off when the power converter turns on.

10 Determine if all the power converters turn on correctly, indicatedby all the Converter Fail lights going off.

11 If all the Converter Fail lights are off then go to step 23.

12 If all the Converter Fail lights are not off then go to step 26.

13 To turn on each power converter pull out the power switch andtoggle it to the Power Reset position.

ATTENTIONThe Converter Fail light goes off when the power converter turns on.

14 Determine if all the power converters turns on correctly, indicatedby all the Converter Fail lights are off.

15 If all the Converter Fail lights are off then go to step 23.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 453: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 453

16 If all the Converter Fail lights are not off then go to step 26.

17 Turn on each power converter by toggle the circuit breaker to theON position.

18 Press and hold the Power Reset button for 2s.

19 Release the circuit breaker and the Power Reset button.

ATTENTIONThe Converter Fail light goes off when the power converter turns on.

20 Determine if all the power converters turn on correctly, indicatedby all the CONVERTER FAIL lights are off.

21 If all the Converter Fail lights are off then go to step 23.

22 If all the Converter Fail lights are not off then go to step 26.

23 Determine if all PM frames turn on.

24 If all the PM frames turn on then go to step 1 in “Completing thepower up of the XA-Core” (page 454).

25 If all the PM frames turn off then go to step 26.

26 Determine if a try made to power up the remaining PM frames.

27 If power up has not tried then go to step 29.

28 If power up has tried and failed then go to the next step.

29 Power up the next PM frame.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 454: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

454 Powering up the XA-Core

Completing the power up of the XA-Core

Complete the power up of the XA-Core.

Procedure steps

Step Action

1 At the MAP terminal

Access the SRSTATUS level of the MAP display.

>MAPCI;MTC;SRSTATUS

Example of a MAP response

SRSTATUS0 Quit OVERALL STATUS Pend: 0% Inprg: 0%Comp: 100% Fail: 0%2 View_ View: SYSTEM14:08:303 List_ Pend InPrg Comp Fail PendInPrg Comp Fail4 MS 0 0 2 0 IOD5 5 30 25 NET 0 0 6 0 Other21 3 13 36 SER1 0 41 0 07 SER2 0 39 0 08 SER3 0 0 37 0910 MTC:11 STATUS:

2 From the MAP display, determine the recovery status of theSeries I and II PMs.

ATTENTIONSeries I PM recovery status displays to the right of the word "SER1"in the MAP display. Series II PM recovery status displays to the rightof the word "SER2" in the MAP display. Recovery status can be oneof pending, in progress, complete, or failed.

3 If the recovery status is zero then go to step 7.

4 If the recovery status is not zero then go to step 5.

5 Determine from office records or other office personnel whichPMs to manually recover first.

6 To manually recover the PMs in the required order, performthe correct alarm clearing procedures in Alarm Clearing and

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 455: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 455

Performance Monitoring Procedures, 297-8001-543 (NorthAmerican market) or 297-9051-543 (International market).

7

CAUTIONLoss of billing dataDifferent billing systems than automatic messageaccounting (AMA) or additional billing system, canbe in your office configuration. Call your next level ofsupport to determine if other billing systems are inyour office, and if you require recovery action.

Access the device independent recording package (DIRP) levelof the MAP.

>IOD;DIRP

8 Determine the state of the recording volumes for the billingsystem.

>QUERY subsystem ALL

Example of a MAP response

SSNAME SSNO SEQNO ROTATES POOLNO PARLPOOL EMERGENCYAMA 0 1 6 9 62 ***YES***REGULARFILE(S) STATE VOLUME RECCOUNT BLOCK E V V_B VLID FNUM FRN#ACTIVE NONEgSTANDBY1 NONEPARALLELFILE STATE VOLUME BLOCK E V V_B VLIDNUM FRN#NONEREGULAR VOLUME(S)VOL# VOLNAME STATE IOC CARD VOL FSEG ROOMVLID FILESREGULAR SPACE

9 If the state of the recording volumes for the billing system hasno volumes allocated, as indicated by the word NONE under thestate header on the MAP display then go to step 15.

10 If the state of the recording volumes for the billing system hasany volume is IN ERROR, as indicated under the REGULARVOLUME(S) header on the MAP display then go to step 12.

11 If the state of the recording volumes for the billing system hasall volumes are READY, as indicated under the REGULARVOLUME(S) header on the MAP display then go to step 16.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 456: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

456 Powering up the XA-Core

ATTENTIONDifferent billing systems than automatic message accounting (AMA)or additional billing system, can be in your office configuration. Callyour next level of support to determine if other billing systems are inyour office, and if you require recovery action.

12 Reset any volumes that are IN ERROR.

>RSETVOL vol_name

13 If the volumes reset has passed then go to step 16.

14 If the volumes reset has failed then contact your next level ofsupport.

15 Perform the procedure, "Allocating recording volumes in theDIRP utility" in Routine Maintenance Procedures, 297-8001-546(North American market) or 297-9051-546 (International market).When you complete the procedure, return to this point.

16 Determine the state of the DLOG recording volumes.

>QUERY DLOG ALL

Example of a MAP response

SSNAME SSNO SEQNO ROTATES POOLNO PARLPOOL EMERGENCYDLOG 2 1 102 10 NONE ****YES***.REGULARFILE(S) STATE VOLUME RECCOUNT BLOCK E V V_B VLID FNUM FRN#ACTIVE AVAIL S01DDLOG 6 6 0 22 NO 8447 0013 204DSTANDBY1 AVAIL S00DDLOG 0 0 0 23 NO 84080014 309B.REGULAR VOLUME(S)VOL# VOLNAME STATE IOC CARD VOL FSEG ROOMVLID FILES22 S01DDLOG READY N/A N/A 7 5 188447 A23 S00DDLOG READY N/A N/A 8 4 188408 S1REGULAR SPACEVOL# VOLNAME STATE SEGS EXP UNEXP TOTAL22 S01DDLOG READY 5 13 0 1823 S00DDLOG READY 4 14 0 18

17 If the state of DLOG recording volumes is no allocated volumes,as indicated by the word NONE under the state header on theMAP display then go to step 23.

18 If the state of DLOG recording volumes is any volume is INERROR, as indicated under the REGULAR VOLUME(S) headeron the MAP display then go to step 20.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 457: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 457

19 If the state of DLOG recording volumes is all volumes areREADY, as indicated under the REGULAR VOLUME(S) headeron the MAP display then go to step 24.

ATTENTIONDifferent billing systems than automatic message accounting (AMA)or additional billing system, can be in your office configuration. Callyour next level of support to determine if other billing systems are inyour office, and if you require recovery action.

20 Reset any volumes that are IN ERROR.

>RSETVOL vol_name

21 If the volumes reset has passed then go to step 24.

22 If the volumes reset has failed then contact your next level ofsupport.

23 Perform the procedure, "Allocating recording volumes in theDIRP utility" in Routine Maintenance Procedures, 297-8001-546(North American market) or 297-9051-546 (International market).When you complete the procedure, return to this point.

24 Determine from your next level of support if you need to recoverother DIRP systems (for example, JF, OM).

25 If you need to recover other DIRP systems then go to step 27.

26 If you need to recover other DIRP systems no then go to step33.

27 Perform the correct procedures in Alarm Clearing andPerformance Monitoring Procedures, 297-8001-543 (NorthAmerican market) or 297-9051-543 (International market). Whenyou complete the procedures, return to this point.

28 Determine if DIRP logs generated.

>LOGUTIL;OPEN DIRP

29 If DIRP log generated yes then go to step 31.

30 If DIRP log generated no then go to step 33.

31 See Log Report Reference Manual 297-8001-840 (NorthAmerican market) or 297-9051-840 (International market), andtake the correct action.

When you complete the log report activities, return to this point.

32 Turn off priority.

>PRIORITY OFF

33 Access the SRSTATUS level of the MAP display.

>MAPCI;MTC;SRSTATUS

Example of a MAP response

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 458: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

458 Powering up the XA-Core

SRSTATUS0 Quit OVERALL STATUS Pend: 0% Inprg: 0%Comp: 100% Fail: 0%2 View_ View: SYSTEM14:08:303 List_ Pend InPrg Comp Fail PendInPrg Comp Fail4 MS 0 0 2 0 IOD5 5 30 25 NET 0 0 6 0 Other21 3 13 36 SER1 0 0 41 07 SER2 0 0 39 08 SER3 0 0 37 0910 MTC:11 STATUS:

34 Determine the status of the switch recovery.

35 If the status of the switch recovery is any Series III PMs thatfailed recovery then go to step 42.

36 If the status of the switch recovery is any Series I or II PMs thatfailed recovery then go to step 40.

37 If the status of the switch recovery is any IODs or other devicesand services that failed recovery then go to step 42.

38 If the status of the switch recovery is that the system hascompletely recovered then you have completed this procedure.

39 Go to step 33.

40 Determine from office records or other office personnel whichPMs you can recover first.

41 Go to step 33.

42 To manually recover IODs and other devices and services,perform the correct procedure in this document. Also you canrefer to your site-related operating procedures.

43 For additional help, call your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 459: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

459.

Powering up the networkThis section lists the procedures you use to power up variousCommunication Server 2000 components within the network.

Powering up the network navigation

• “Powering up the Message Switch” (page 461)

• “Powering up a Ethernet Routing Switch 8600” (page 464)

• “Powering up the Enhanced Network” (page 465)

• “Powering up the Link Peripheral Processors” (page 467)

• “Powering up the SDM” (page 470)

• “Powering up a Sun Netra server” (page 471)

• “Powering up the Border Control Point Manager” (page 472)

• “Powering up the SAMF frame” (page 473)

• “Power-On and boot a Policy Controller unit” (page 475)

• “Powering on and booting an SST unit” (page 476)

• “Viewing the operational status of the SIP Gateway application” (page477)

• “Powering up the Call Control Frame” (page 481)

• “Unlocking the Call Agent” (page 483)

• “Unlocking a GWC card” (page 484)

• “Locking a GWC card” (page 486)

• “Manually re-provisioning GWC cards” (page 488)

• “Powering up a USP” (page 492)

• “Powering up a Media Gateway/Multiservice Switch 15000” (page 494)

• “Powering up a Media Gateway/Multiservice Switch 7400” (page 495)

• “Powering up the MDM workstation” (page 497)

• “Powering up an SPM device” (page 498)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 460: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

460 Powering up the network

• “Powering up the MCS servers” (page 500)

• “Powering up the IPmedia 2000 shelves” (page 505)

• “Powering up the SAM16 shelves” (page 506)

• “Powering up an MG 9000 device” (page 507)

• “Viewing logs from a remote backup” (page 512)

• “Setting date and time” (page 513)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 461: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 461

Powering up the Message Switch

Power up the Message Switch (MS) when recovering from a poweroutage.

Procedure steps

Step Action

1 At the XA-Core frame

Locate the NT9X31 and NT9X30 power converters for messageswitch 0 (MS 0) in slots 33F and 36F on the MS 0 shelf.

2 Turn on the NT9X31 and NT9X30 power converters in slots 33Fand 36F of the MS 0 shelf at the same time. Lift and release thepower switches located on the faceplates of the converters.

3 Locate the NT9X31 and NT9X30 power converters for MS 0 inslots 1F and 4F on the MS 0 shelf.

4 Turn on the NT9X31 and NT9X30 power converters in slots 1Fand 4F of the MS 0 shelf at the same time. Lift and release thepower switches located on the faceplates of the converters.

5 Locate the NT9X31 and NT9X30 power converters for messageswitch 1 (MS 1) in slots 33F and 36F on the MS 1 shelf.

6 Turn on the NT9X31 and NT9X30 power converters in slots 33Fand 36F of the MS 1 shelf at the same time. Lift and release thepower switches located on the faceplates of the converters.

7 Locate the NT9X31 and NT9X30 power converters for MS 1 inslots 1F and 4F on the MS 01shelf.

8 Turn on the NT9X31 and NT9X30 power converters in slots 1Fand 4F of the MS 1 shelf at the same time. Lift and release thepower switches located on the faceplates of the converters.

9 At the MAP terminal

Access the SRSTATUS level of the MAP.

>MAPCI;MTC;SRSTATUS

Example of a MAP response

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 462: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

462 Powering up the network

SRSTATUS0 Quit OVERALL STATUS Pend: 0% Inprg: 0%Comp: 100% Fail: 0%2 View_ View: SYSTEM14:08:303 List_ Pend InPrg Comp Fail PendInPrg Comp Fail4 MS 0 0 2 0 IOD5 5 30 25 NET 0 0 6 0 Other21 3 13 36 SER1 0 41 0 07 SER2 0 39 0 08 SER3 0 37 0 0910 MTC:11 STATUS:

10 Determine the recovery status of the MSs.

ATTENTIONMS recovery status displays to the right of the word "MS" in the MAPdisplay. Recovery status for each MS can be one of pending, inprogress, complete, or failed.

11 If the recovery status is either MS failed recovery then you havecompleted this procedure.

12 If the recovery status is either MS continues to have pendingrecovery then go to step 14.

13 If the recovery status is another status different from failed orpending recovery then go to step 15.

14 Until both MSs are either in recovery development or havecompleted recovery.

When neither MS continues to have pending recovery, go to step10.

15 Access the MS Clock level of the MAP display.

>MAPCI;MTC;MS;CLOCK

16 Synchronize the clocks.

>SYNC

17 If the SYNC command is successful then you have completedthis procedure.

18 If the SYNC command is failed then go to step 19.

.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 463: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 463

19 Record the reason for synchronization failure, as shown in theMAP response. Repeat the try to synchronize the MS clockslater, after networks and PMs are in service.

20 For additional help, call the personnel responsible for the nextlevel of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 464: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

464 Powering up the network

Powering up a Ethernet Routing Switch 8600

Power up a Ethernet Routing Switch 8600.

Procedure steps

Step Action

1 At the Ethernet Routing Switch 8600 frame

Turn the DC input power source on or reset the power sourcecircuit breaker to provide power to the power supply.

2 Turn the power supply switch for all three DC power supplies tothe on position.

ATTENTIONYou must turn on two of the power supply units within 2 seconds ofeach other. If you wait longer to turn on the second power supply,one of the power supplies could shut down. To correct this condition,turn off both power supplies, wait at least 30 seconds, and then turnon both power supplies again within 2 seconds.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 465: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 465

Powering up the Enhanced Network

Manually power up the enhanced network (ENET) in the event of a powerfailure.

Procedure steps

Step Action

1 At the ENET frames

Locate the NT9X31 power converters in slots 1F and 33F on theENET shelves.

2 To turn on the NT9X31 power converters lift and release thepower switches located on the faceplates of the converters.

3 Locate the NT9X30 power converters in slots 4F and 36F on theENET shelves.

4 Turn on the NT9X30 power converters lift and release the powerswitches located on the faceplates of the converters.

5 Determine if all the converters have power. Power is indicated byall the Converter Off lights going off.

6 If all the power converters have power then go to step 9.

7 If all the power converters have not power then go to step 8.

8 Power up the ENET frame. Perform procedure, "Clearing an ExtFSP DPCC cabinet major alarm" in the document, Alarm andPerformance Monitoring Procedures, 297-8021-543.

When you complete the procedure, return to this point.

9 At the MAP terminal

Access the SRSTATUS level of the MAP display:

>MAPCI;MTC;SRSTATUS

Example of a MAP response

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 466: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

466 Powering up the network

SRSTATUS0 Quit OVERALL STATUS Pend: 0% Inprg: 0%Comp: 100% Fail: 0%2 View_ View: SYSTEM14:08:303 List_ Pend InPrg Comp Fail PendInPrg Comp Fail4 MS 0 0 2 0 IOD5 5 30 25 NET 0 0 6 0 Other21 3 13 36 SER1 0 41 0 07 SER2 0 39 0 08 SER3 0 37 0 0910 MTC:11 STATUS:

10 From the MAP display, determine the recovery status of thenetwork.

ATTENTIONNetwork recovery status displays to the right of the word "NET" in theMAP display. Recovery status can be one of pending, in progress,complete, or failed.

11 If the status of any network element is failed then go to step 16.

12 If the status of any network element is pending then go to step14.

13 If the status of any network element is another status then youhave completed this procedure.

14 Continue when there are no network elements that continue tobe pending recovery.

15 Go to step 10.

16 Manually recover the ENET.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 467: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 467

Powering up the Link Peripheral Processors

Power up the Link Peripheral Processors (LPP) or fiberized LPPs (FLPPs)in the event of a power failure.

ATTENTIONThroughout this procedure, LPP is used to refer to both the LPP and FLPP.

Procedure steps

Step Action

1 At the PDC

Determine if PDC has power restored to the LPP(s).

2 If PDC has power restored to the LPPs then go to step 9.

3 If PDC has not power restored to the LPPs then go to step 4.

4 Check the PDC fuses that supply the LPP.

5 If there are blown fuses then go to step 7.

6 If there are no blown fuses visible then go to step 8.

7 Replace the blown fuses.

ATTENTIONIf fuses blow repeatedly, call your next level of support for help.

When PDC power restores to the LPPs, continue this procedureat step 9.

8 Call the personnel responsible for maintaining power at your site,or refer to your next level of support for help.

When PDC power restores to the LPPs, continue this procedureat step 9.

9 Locate the LPPs for recovery.

10 At the LPP cabinet

WARNINGStatic electricity damageWear a wrist strap connected to the wrist-strapgrounding point of a frame supervisory panel(FSP). Also a wrist strap can connect to a modularsupervisory panel (MSP). Wear a wrist strap whenyou handle circuit cards. A wrist strap protects thecards against damage caused by static electricity.

Locate the NT9X74 cards in all link interface shelves (LIS).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 468: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

468 Powering up the network

ATTENTIONNT9X74 cards are in shelf position 7F and 32F on all LISs.

11 To unseat each NT9X74 card, release the locking levers andcarefully pull the card towards you about 25 mm (1 in.).

12 Locate the NT9X30 power converters in slots 4F and 36F of thelink interface module (LIM) unit shelf.

13 To turn on the power converters of the LIM unit shelf, toggle theswitch each NT9X30 card.

14 Locate the NT9X30 or NTDX16 power converters for each LIS.

ATTENTIONNT9X30 power converters are in slots 4F and 36F for each LIS.NTDX16 power converters can be in slots 1F, 4F, 33F, and 36F foreach LIS.

15 To turn on the LIS power converters, toggle the switch on eachNT9X30 or NTDX16 card.

16 Determine if all the power converters turned on power correctly.All the CONVERTER OFF lights go off when power turns oncorrectly.

17 If all the CONVERTER OFF lights are off then go to step 21.

18 If all the CONVERTER OFF lights are not off then go to step 19.

19 Power up the frame.

20 Go to step 12.

21 Put in position all NT9X74 cards carefully slide each NT9X74card back into the LIS.

22 Push on the upper and lower edges of each faceplate. Makesure that the card is completely in the slot of the shelf.

23 Close the locking levers on each card.

24 Repeat step 5 through step 15 for each LPP in your office.

When power restores to all LPPs, continue this procedure at step17.

25 To access the SRSTATUS level of the MAP display, type:

>MAPCI;MTC;SRSTATUS

and pressing the Enter key.

Example of a MAP response

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 469: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 469

SRSTATUS0 Quit OVERALL STATUS Pend: 0% Inprg: 0% Comp: 100%Fail: 0%2 View_ View: SYSTEM14:08:303 List_ Pend InPrg Comp Fail PendInPrg Comp Fail4 MS 0 0 2 0 IOD5 5 30 25 NET 0 0 6 0 Other21 3 13 36 SER1 0 41 0 07 SER2 0 39 0 08 SER3 0 37 0 0910 MTC:11 STATUS:

26 From the MAP display, determine the recovery status of theSeries III PMs.

ATTENTIONSeries III PM recovery status displays to the right of the word "SER3"in the MAP display. Recovery status can be one of pending, inprogress, complete, or failed.

27 If the Series 3 PMs are zero then you have completed thisprocedure.

28 If the Series 3 PMs are not zero then go to step 29.

29 To manually recover the PMs, see Recovery Procedures(297-8021-545).

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 470: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

470 Powering up the network

Powering up the SDM

Power up the SuperNode Data Manager (SDM) that houses theCommunication Server 2000 Core Manager.

Procedure steps

Step Action

1 At the SDM frame

Turn on the two top breakers at the top of the SDM frame.

2 Wait a few moments until the green lights on each card aresteady.

3 At the VT100 terminal

Log on as root and type

# sdmmtc

4 Verify that all hardware is in service.

5 Verify that rootvg and datavg logical volumes are mirrored at thestorage level.

ATTENTIONThere will likely be alarms at the connection and application levels.The general status of the SDM will be in-service trouble (IsTb).

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 471: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 471

Powering up a Sun Netra server

Power up a Netra server. Powering up the server supplies power for thefollowing applications:

• IEMS

• Core and Billing Manager

• Communication Server 2000 Management Tools applications

• MG 9000 Manager

Procedure steps

Step Action

1 To bring the duplex SPFS-based servers into service, turn on thecircuit breakers. To help with recovery, turn the power on to whatwas the active server first. If there is no record of this, turn onthe Unit 0 server before the Unit 1 server.

Once power is restored, the servers recover on their own.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 472: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

472 Powering up the network

Powering up the Border Control Point Manager

Power up the Border Control Point Manager. The Border Control PointManager is installed on a Sun Netra™ 240 server.

Procedure steps

Step Action

1 At the frame housing the Border Control Point Manager

For DC powered shelves, switch the circuit breakers at the EBIPthat provide power to the servers being powered to the ONposition. The breaker will have the ’|’ side (top) of the breakerdepressed.

ATTENTIONThe DC powered SPFS-based server does not have a PowerON/OFF switch and can only be powered ON/OFF from the breakerat the EBIP.

2 For AC powered shelves, push the rocker located on the topof the AC outlet to apply power to the servers. It should beswitched to the "|" position.

3 Once the breakers are switched ON at the power source, powerup the servers by switching the breakers on the back of thechassis to the ON position.

4 Verify the system is powered up correctly by observing the LEDson the front of the SPFS-based server.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 473: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 473

Powering up the SAMF frame

Power up the SAMF frame. Powering up the SAMF frame allows theSAM21 hardware cards, the Policy Controller unit, and the Session Serverunit to return to service.

Procedure steps

Step Action

1 At the SAMF frame

Power up the SAMF BIP breakers. At the top of the cabinet, turnon the breakers that supply power to the two SAM21 shelvesand the Session Server (if equipped).

2 At the Communication Server 2000 SAM21 Manager

If necessary, unlock the previously active Shelf Controller (SC).

Refer to Unlocking a SAM21 Shelf Controller.

3 If necessary, unlock the previously inactive SC.

Refer to Unlocking a SAM21 Shelf Controller.

4 If necessary, unlock any remaining locked cards on the SAM21other than the GWC cards.

5 At the Communication Server 2000 GWC Manager

If necessary, unlock all previously active GWC cards.

Refer to Unlock a GWC card.

6 If necessary, unlock all previously inactive GWC cards.

Refer to Unlock a GWC card.

7 At the Policy Controller unit

Determine which physical or logical unit you want to becomeactive and power that unit up first by executing procedurePower-On and boot a Policy Controller unit.

8 Once the active unit has begun call processing, power up themate unit by executing procedure Power-On and boot a PolicyController unit.

9 At the Session Server unit

Determine which physical or logical unit you want to becomeactive and power that unit up first by executing procedurePower-On and boot a Session Server unit..

10 Log onto the active unit to monitor the status of the SIP Gatewayapplication using procedure View the operational status of theSIP Gateway application.. Verify that the Administrative state

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 474: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

474 Powering up the network

of the SIP Gateway application becomes Unlocked and theOperational state becomes Enabled.

If the active unit comes up in any state other thanUnlocked:Enabled, see Session Server Security andAdministration (NN10346-611), and complete one or both of thefollowing procedures.

11 Complete procedure Unsuspend the SIP gateway application.

12 Complete procedure Unlock the SIP gateway application.

13 Once the active unit has begun call processing, power up themate unit by executing procedure Power-On and boot a SessionServer unit..

14 From the active unit, verify that the SIP Gateway applicationdatabases on the both units have synchronized using procedureVerify synchronization status of Session Server units.

15 Monitor the system for an appropriate period per your siteguidelines.

If you experience problems with call processing, see SessionServer Security and Administration (NN10346-611), and performthe following tasks in order.

16 Complete procedure Invoke a maintenance SwAct of the SessionServer platform.

17 Complete procedure Inhibit a system SwAct (Jam).

18 Contact your next level of support or Nortel GNPS.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 475: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 475

Power-On and boot a Policy Controller unit

This procedure is used to power on a Policy Controller unit that has beeninstalled as a replacement, or was shutdown for any other reason.

This procedure may be used as a standalone task or as part of a higherlevel activity such as part of a dead office recovery activity or softwareupgrade activity.

Prerequisites• If the unit was a replacement unit recently installed, ensure that all

power cabling connections have been properly installed and securedat the rear of the chassis and SAM-F frame.

Procedure steps

Step Action

1 At the front panel of the Policy Controller unit

If necessary, power on the Policy Controller using the mainpower switch located on the front panel.

2 If desired, at the Policy Controller console, monitor the bootprogress of the unit.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 476: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

476 Powering up the network

Powering on and booting an SST unit

Power on and boot the SST unit to replace or for other reasons.

This procedure may be used as a standalone task or as part of a higherlevel activity such as part of a dead office recovery activity or softwareupgrade activity.

Prerequisites• If the unit was a replacement unit recently installed, ensure that all

power cabling connections have been properly installed and securedat the rear of the chassis and SAM-F frame.

Procedure steps

Step Action

1 At the front panel of the SST unit

If necessary, power on the SST using the main power switchlocated on the front panel.

2 At the SST console, monitor the boot progress of the unit.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 477: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 477

Viewing the operational status of the SIP Gatewayapplication

View the operational status of the SIP Gateway application to know theservice status.

Procedure steps

Step Action

1 At the Communication Server 2000 Session Server Manageror IEMS client

Select Carrier VoIP Communication Server 2000 SessionServer Manager.

2 Click to open the Session Server folder.

3 Click to open the Maintenance folder.

4 Click to open the Application folder.

5 Click the SIP Gateway link.

6 Monitor the status of the SIP Gateway application.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 478: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

478 Powering up the network

This view is refreshed according to the value shown in the dropdown box at the bottom of the status panel.

7 To increase or decrease the refresh rate, select a different valueand click Refresh Rate.

8 Click Refresh to manually refresh the page.

9 The following service affecting actions are available:

• Lock the SIP Gateway application

• Unlock the SIP Gateway application

• Suspend the SIP Gateway application

• Unsuspend the SIP Gateway application

• Cold SwAct the SIP Gateway application

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 479: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 479

10 To view the number of active calls currently handled by theapplication, and the synchronization status of the units, clickQueryInfo.

--End--

Job aidTable 48Node status field descriptions

Field Description

Unit Connection Status Bar Indicates which unit in the node the CommunicationServer 2000 Session Server Manager is connectedto.

Unit Number Identifies the two units in the node, labeled 0 and 1.

Activity State Indicates which unit is active and which isinactive (standby). Acts as an indirect indicatorof fault-tolerant status; when both units havean Operational status of enabled, the node isfault-tolerant.

Operational State Indicates the service status of each unit, enabled ordisabled.

Table 49SIP Gateway application Status field descriptions

Field Indication

Administrative State Locked, Unlocked, or ShuttingDown

Operational State Enabled or Disabled

Procedural Status Terminating or -

Control Status Suspended or -

Use the following table to interpret the SIP Gateway area’s CCITTX.731-style and related DMS-style status indicators.

Table 50SIP Gateway Maintenance field descriptions and interpretation of service states

AdministrativeState

OperationalState

ProceduralStatus

Control Status DMS styleService States

Locked Disabled - Suspended Offline (OFFL)

Locked Enabled - - Manual Busy(MANB)

(-) indicates a status of in-service

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 480: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

480 Powering up the network

Table 50SIP Gateway Maintenance field descriptions and interpretation of service states (cont’d.)

AdministrativeState

OperationalState

ProceduralStatus

Control Status DMS styleService States

Locked Enabled Terminating - Manual BusyTransitioning(MANBP)

Unlocked Enabled - - In Service(INSV)

Unlocked Disabled - - System Busy(SYSB)

Shutting Down Enabled - - Going out ofservice(INSVD)

(-) indicates a status of in-service

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 481: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 481

Powering up the Call Control Frame

Power up the Call Control Frame (CCF). Powering up the CCF allows theSAM21 hardware cards and the STORM disk array to return to service.

Procedure steps

Step Action

1 At the CCF

Determine the configuration of your CCF.

2 If the CCF configuration includes a STORM disk array and twoSTORM cards in the Call Agent shelf then go to step 4.

3 If the CCF configuration includes two STORM SAM-XTS unitsthen go to step 6.

4 Power up the CCF BIP breakers. At the top of the cabinet, turnon the breakers that supply power to the STORM disk array andthe two SAM21 shelves.

5 Go to step 9.

6 Power up the CCF BIP breakers. At the top of the cabinet, turnon the breakers that supply power to the two SAM21 shelvesand the STORM SAM-XTS units.

7 At the front panel of the STORM SAM-XTS units

If necessary, power on the STORM SAM-XTS units using themain power switch located on the front panel.

8 At the console, monitor the boot progress of the unit until bootingis complete.

9 At the Communication Server 2000 SAM21 Manager

If necessary, unlock the previously active Shelf Controller (SC).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 482: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

482 Powering up the network

Refer to Unlocking a SAM21 Shelf Controller.

10 If necessary, unlock the previously inactive SC.

Refer to Unlocking a SAM21 Shelf Controller.

11 If necessary, unlock the previously active Compact Call Agent(CCA) card.

Refer to “Unlocking the Call Agent” (page 483).

12 If necessary, unlock the previously inactive CCA card.

Refer to “Unlocking the Call Agent” (page 483).

13 If necessary, unlock the previously active USP-lite card.

14 If necessary, unlock the previously inactive USP-lite card.

15 If necessary, unlock any remaining locked cards on the SAM21other than the GWC cards.

16 At the Communication Server 2000 GWC Manager

If necessary, unlock all previously active GWC cards.

Refer to “Unlocking a GWC card” (page 484).

17 If necessary, unlock all previously inactive GWC cards.

Refer to “Unlocking a GWC card” (page 484).

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 483: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 483

Unlocking the Call Agent

Unlock the Call Agent to reboot the call agent.

Procedure steps

Step Action

1 At the Communication Server 2000 SAM21 Manager client

From the Shelf View, right click on the card and select Unlockfrom the context menu.

Unlock is also available from the States tab of the Card Viewwindow.

The card resets, downloads software, and reboots.

2 Wait for the lock icon to disappear.

Do not perform any patching activities on the Call Agent until tenminutes have passed.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 484: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

484 Powering up the network

Unlocking a GWC card

Unlock a GWC card to initiate a reboot of the GWC card causing the cardto download its software from the CS 2000 Core Manager and to restart itscall processing services and applications software.

Use this procedure

• after replacing a GWC card.

• as part of a fault clearing activity.

• when a new software load is available.

• when you complete re-provisioning a GWC card or GWC node and youwant the card or node to use the new provisioning values.

• when you applied or removed a patch to the GWC software using theNetwork Patch Manager (NPM) and saved a new GWC software imageto the CS 2000 Core Manager.

Prerequisites• The card must be locked.

• If IP addresses for the card that you want to unlock and its mate arenot contiguous, you cannot unlock the card.

• If the GWC hardware does not support the service profile assigned tothe GWC node, the unlock process fails and the system generates theCMT305 critical alarm.

Procedure steps

Step Action

At the Communication Server 2000 GWC Manager client1 At the Communication Server 2000 Management Tools window,

from the Device Types directory tree in the far left frame, clickthe Gateway Controller folder.

2 From the Contents of: GatewayController frame, select the GWCnode that contains the card you want to unlock.

3 Click the Maintenance tab to display maintenance informationabout the node.

4 Click the Card View button for the card you want to unlock. TheCS 2000 SAM21 Manager opens.

If the card is locked, all fields display the value <unknown>.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 485: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 485

5 At the Communication Server 2000 SAM21 Manager

In the card view, select the States tab, then click the Unlockbutton.

If you want to display the status of all cards in the shelf, from theView menu, select Shelf View.

6 If the IP addresses for the selected card and its mate are notcontiguous, the system displays the following error message:

If the GWC service profile and the GWC hardware are notcompatible, the unlock operation fails and the system generatesthe CMT305 critical alarm.

7 Observe the system response in the History window.

The card is unlocked when you see the Application unlockedsuccessfully message.

8 If required, repeat this procedure for the next GWC card untilall the GWC cards are unlocked and brought into service. EachGWC node has two cards.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 486: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

486 Powering up the network

Locking a GWC card

Lock a GWC card to stop the services, applications, and platform softwarerunning on that card.

Use this procedure

• when you are removing the card from service.

• along with procedure “Unlocking a GWC card” (page 484) to reboot aGWC and force a software download.

• as part of fault clearing activity to determine if a problem is temporaryor persistent.

• when you apply or remove a patch to the GWC software using theNetwork Patch Manager (NPM) and create a new GWC softwareimage on the Communication Server 2000 Core Manager.

• when you are removing a GWC node from the Communication Server2000 GWC Manager database.

• when you are replacing or upgrading hardware.

Prerequisites• If the card that you want to lock is active, switch call processing to

its mate card in the node. The switch places the card in the standbymode.

• When the card is in the standby mode, disable (busy) services on thecard.

Procedure steps

Step Action

At the Communication Server 2000 GWC Manager client1 At the Communication Server 2000 Management Tools window,

from the Device Types directory tree in the far left frame, clickthe Gateway Controller folder.

2 From the Contents of: GatewayController frame, select the GWCnode that contains the card you want to lock.

3 Click the Maintenance tab to display maintenance informationabout the node.

4 Click the Card View button for the card you want to lock.

5 At the Communication Server 2000 SAM21 Manager client

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 487: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 487

In the card view, select the States tab, then click the Lockbutton.

6 Observe the system response in the History window.

The card is locked when you see the text Application lockedsuccessfully in the History window. The lock icon (circled in thefollowing figure) should also be present on the card graphic atthe left of the screen:

7 If necessary, repeat this procedure for the next GWC card in thenode.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 488: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

488 Powering up the network

Manually re-provisioning GWC cards

This procedure provides instructions on how to change GWC card-relatedprovisioning information in the CS 2000 SAM21 Manager database. Thisprocedure does not provide instructions on how to make provisioningchanges to GWC services in the CS 2000 GWC Manager database (suchas changing a service profile for a GWC node).

Use this procedure to manually change the basic provisioning informationfor a set of Gateway Controller (GWC) cards, including IP addresses, portaddresses, gateway addresses, and load paths.

When re-provisioning IP addresses for GWC cards, you must use a blockof four contiguous IP addresses for each card pair (node).

CAUTIONService disruptionChanges to the IP addresses of a GWC card will be localto SAM21 Element Manager. These changes will not getpropagated to the rest of the network. If the IP address of afully configured GWC node has to be changed, then multipleelements of the network will require updating, and this activity isservice impacting. Please contact Nortel Support to assist withIP address changes to fully configured GWC nodes.

Procedure steps

Step Action

1 At the Communication Server 2000 GWC Manager client

At the Communication Server 2000 Management Tools window,from the Device Types menu, click the Gateway Controllerfolder.

2 Select or type the name of the GWC node that you wish tore-provision.

3 Click the Card View button for the card you busied in theprevious step. This action opens the CS 2000 SAM21 Manager.

4 At the Communication Server 2000 SAM21 Manager

Select the States tab to view the states window.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 489: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 489

5 Determine whether the card is locked by looking at the cardgraphic at the left of the screen. If the lock icon is present, thecard is locked. If the lock icon is not present, the card is notlocked. The Card Status also indicates whether the card islocked.

If the card is already locked, go to the next step.

If the card is not locked, click the Lock button.

System response:

Application locked Successfully.

If you are denied your first attempt to lock the card, you canoverride the lock request denial and force the card to lock.

A card has accepted a lock request if it has the following states:

• Card Status: locked

• Operational state: disabled

6 Click the Provisioning tab, then the Modify button.

7 Enter the new or changed provisioning data as described in thefollowing list.

In most cases, when re-provisioning a GWC, use the defaultvalues indicated. Otherwise, obtain and enter the followingvalues:

• IP: <GWC_unit_0_IP_address> orIP: <GWC_unit_1_IP_address>A contiguous block of four IP addresses is required. If the IPaddresses for the GWC card and its mate are not contiguousand you save your settings, the system displays a warningmessage. The message explains that you must correct the IPaddresses; otherwise, you will not be able to unlock the card.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 490: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

490 Powering up the network

If necessary, repeat this procedure to make sure that the IPaddresses are contiguous.If any of the four IP addresses for the new card is alreadyused by another card, the system displays an error message.Contact your site system administrator to identify the IPaddresses you must use.

• Gateway IP: default_router or gateway_IP_address

• Subnet Mask: subnet_mask

• FW Version: firmware version of the GWC load

• Host IP: Communication Server 2000 ManagementTools_Server_IP_Address

• Server IP: IP address of the Communication Server 2000Core Manager or Core and Billing Manager (CBM)

• Path: /swd/gwc/ - on the Communication Server 2000 CoreManager or CBM

• Load: name of the GWC load image file;example: pgc08bg.imagClick the Get Load Files button and select the required loadfrom the drop-down list.

• If available, select the FW Flash Enable check box if youwish to flash the card firmware with a new firmware version

• Primary Domain Server: server_IP_address

• 1st Alternate Domain Server: server_IP_address

• 2nd Alternate Domain Server: server_IP_address

8 When you are finished entering changes, click the Save button.

9 Re-provisioning does not take effect until the card is unlockedand rebooted.

ATTENTIONIf you modify Load Info, NTP server or Domain server, unlockingthe card is enough. If you modify GWC-EM or GWC-IP Address,you need to unlock and then reboot the card to make re-provisioningeffective.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 491: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 491

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 492: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

492 Powering up the network

Powering up a USP

Power up a Universal Signaling Processor (USP) when it is necessary topower up the USP. USP RTC-12 will initially be used to get the shelf backin service.

Prerequisites• The Communication Server 2000 Core and USP ABS server must be

in-service.

Procedure steps

Step Action

1 At the USP shelf

Ensure all cards are seated in the USP shelf with the exceptionof the RTC-15 (slot 15) front panel card on the Control shelf.

2 Power up both A/B buttons on the control and extension shelves.

3 RTC-12 should SCSI boot and it should come into service within1-2 minutes of being powered up.

4 At the USP Manager

Once RTC-12 is up, use the USP Manager to uninhibit andactivate all links, where applicable. Refer to Configuring Linkson page 111.

5 Verify logs and alarms to ensure no unexpected problems andperform test calls.

6 Verify that all cards on all shelves are up and running with theexception of RTC-15.

7 Ensure that the System Date/Time is set to the current localtime, if not using SNTP. Refer to “Setting date and time” (page513).

8 At the USP frame

Once all cards are up and properly running, reseat RTC-15.RTC-15 should SCSI boot and perform a databasesynchronization with RTC-12. RTC-15 should come into servicein approximately 5-7 minutes.

9 At the USP Manager

Verify that RTC-15 is in-service and inactive.

10 Verify logs and alarms again to ensure no unexpected orundesired events and perform test calls.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 493: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 493

11 Verify that the USP can communicate with the ABS. From theUSP Manager main menu, click on the Administration button andselect ABS Settings.

12 Select the Test button to initiate a series of tests to ensure theUSP can properly communicate with the ABS.

13 Check the USP logs for the test results. Troubleshoot ABSconnectivity in the event that the tests fail.

ATTENTIONTo assist with troubleshooting, the Logs message includes the natureof the test failure.

14 Take a snapshot of the USP, and copy the snapshot to the ABSusing the File Manager.

15 Using the ABS CM GUI, modify the site information to point tothe new snapshot.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 494: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

494 Powering up the network

Powering up a Media Gateway/Multiservice Switch15000

Power up a Media Gateway/Multiservice Switch 15000 when it isnecessary to power up a Media Gateway/Multiservice Switch 15000.

Procedure steps

Step Action

1 At the chassis

Breaker interface panels (BIPs) are located at the top of thechassis. Switch on the breakers on each BIP. The system shouldcome up on its own.

ATTENTIONIf the CP card or its disk becomes unserviceable, the backup-upprovisioning data and software are downloaded from the MDMbackup copy and SDS site, respectively.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 495: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 495

Powering up a Media Gateway/Multiservice Switch7400

This procedure is used to power up a Media Gateway/Multiservice Switch7400.

Prerequisites• All power supply switches must be in the standby position.

Procedure steps

Step Action

1 At the chassis

Disengage all processor cards from the backplane of the shelf.

2 Engage the minimum number of processor cards required, basedon the number of power supplies in the shelf.

3 If you have one power supply, engage a CP in slot 0 and at leastone FP.

Do not engage more than seven FPs. Ensure that all otherprocessors are disengaged.

4 If you have two power supplies, engage at least one CP and twoFPs. Or, engage two CPs and at least one FP.

5 Turn on the circuit breakers for the outlets that supply power toyour switch.

6 Verify that the LEDs on all power supplies are red.

7 Apply power to one of the power supplies by setting its powerswitch to the on position. Verify that the power supply LED isgreen.

Normal operation is not guaranteed if there are no processorcards in the shelf. If you have not installed any processor cardsbefore you switch on a power supply, the system does notsupply power to the shelf and the power supply LED remainsred.

8 Verify that the cooling unit LED is green and is operational. Youshould be able to hear the fans start to rotate.

9 If necessary, apply power to a second and third power supplyby setting their power switches to the on position. Verify that theLED for each power supply is green.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 496: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

496 Powering up the network

If you install eight or more FPs, you must use a second powersupply. If your system contains fewer than eight FPs, the systemuses the second power supply for redundancy.

10 If necessary, engage the remaining processor cards.

11 Verify the LED color on each FP in appropriate.

12 Verify that the appropriate LEDs are illuminated on terminationpanels.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 497: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 497

Powering up the MDM workstation

This procedure is used to Power up the Multiservice Data Manager (MDM)Sun FireTM V480 workstation.

ATTENTIONIf your MDM is an N240 server, see “Powering up a Sun Netra server” (page471).

Prerequisites• An MDM user account is required.

Procedure steps

Step Action

1 At the workstation

Switch the workstation on.

2 Once the workstation has started, logon with an MDM useraccount and ensure that the MDM toolset is available.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 498: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

498 Powering up the network

Powering up an SPM device

This procedure details the sequence of steps necessary to power up aSpectrum Peripheral Module (SPM), Media Gateway 4000 (MG 4000),Interworking Spectrum Peripheral Module (IW-SPM), or Dynamic PacketTrunking Spectrum Peripheral Module (DPT-SPM).

Procedure steps

Step Action

1 At the MAP terminal

Restore power to the frame and re-insert the fuses.

2 Reseat the SIM cards if they were unseated and turn them onone at a time using the switches on the front.

3 Reseat any unseated cards in the shelves.

4 Post the SPM/MG 4000/IW-SPM/DPT-SPM.

>MAPCI;MTC;PM;POST SPM <spm_no>

5 Select and RTS the CEMs.

>SELECT CEM ALL;BSY ALL; LOADMOD ALL;RTS ALL

6 Select and RTS the SRM, OC3 RMs, ATM RMs, and/or GEMRMs.

>QUIT;SELECT <rm_type> ALL;BSY ALL;LOADMOD ALL;RTS ALL

ATTENTIONTo RTS the ATM, use the BSY FORCE and RTS FORCE commands.

7 Bring the carriers into service.

>QUIT ALL;MAPCI;MTC;TRKS;CARRIER;POST SPM <spm_no>;BSYALL;RTS ALL

8 Post and recover the trunks on the SPM at the TTP level. (Pressthe Enter key after each of the following commands.)

>QUIT ALL;MAPCI;MTC;TRKS;TTP;PRADCH;POST AD SPM<spm_no>;BSY ALL;RTS ALL

>QUIT ALL;MAPCI;MTC;TRKS;TTP;POST D SPM <spm_no>;BSYALL;RTS ALL

9 Return to the PM level and post the SPM.

>QUIT ALL;MAPCI;MTC;PM;POST SPM <spm_no>

10 Select and RTS the remaining RMs.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 499: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 499

>SELECT <rm_type> ALL;BSY ALL;LOADMOD ALL;RTS ALL

--End--

Variable definitionsVariable Value

<spm_no> The number of the SPM/MG 4000/IW-SPM/DPT-SPM.

<rm_type> Ether SRM, OC3, ATM, or GEM.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 500: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

500 Powering up the network

Powering up the MCS servers

This procedure details the steps needed to power up the eight MCSservers. The database server should be brought into service before anyother servers. If other servers come up first, restart the managementserver.

Procedure steps

Step Action

1 At the frame housing the MCS servers

Power on the database server by turning on the circuit breakersthat provide power to the servers. It takes from 5 to 8 minutes forthe server to boot and load Oracle.

ATTENTIONThe following shows the preferred order, which is not the way thesystem comes up if all servers are powered on at the same time.

2 After the DBSvr powers on, Telnet to the box and run a spotcheck to make sure it is ready.

3 Make sure Oracle is loaded. The best way is to Telnet tothe database. Type: sqlplus and fill in the username andpassword (in other words, log on as user, not root). If Oracle isrunning, you should receive the SQL> command prompt. Typequit to exit from sqlplus.

4 Verify that the snmp service is running on the database box.Type: ps -ef | grep snmp. You should see three snmpprocesses:

/usr/local/sbin/snmpd -f udp:161/bin/sh/opt/app/oracle/product/9.2.0/bin/dbsnmpwd

/opt/app/oracle/product/9.2.0/bin/dbsnmp

The snmp processes are not critical to getting the system up.However, if any of the snmp processes are not running, you willnot be able to see accurate reporting of this box on the SystemManagement Console.

5 To stop or start oracle, Telnet to the database as sysadmin andthen su to root. Go to the following directory: /etc/init.d.The stop command is ./dbora stop. The start command is./dbora start. These commands will severely affect service.

6 Verify that the SysMgr processes on the MgmtSvr/AcctMgr boxare running in order to control the MCS components from the

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 501: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 501

System Management Console. Telnet to the MgmtSvr/AcctMgrbox as nortel.

7 Make sure that the SysMgr processes are running by typing:meinit -p. You should see three processes running. If any ofthese processes are not running, you will not be able to launchthe System Management Console:

Rel2.0 NTme_pids mgmtsvr mgmtsvr.3Rel2.0 NTme_pids tsscma tsscma.5

Rel2.0 NTme_pids tssfpma tssfpma.6

8 To stop or start the SysMgr, go to the following directory: IMS/mgmtsvr/bin/mgmtsvr/ and type: ./MgmtSvrShutdown.pl.This will kill any (or all) of the SysMgr processes.

9 To start the SysMgr, stay in the same directory and type:MgmtSvrConfigSetup.pl. This will try to start all threeprocesses. You will see a messages on the Telnet sessionthat say something like "starting tsscma waiting 30seconds." The tssfpma process will start next, followed by themgmtsvr processes.

10 If you want to watch the SysMgr startup logs, afterperforming step 3b above, go to the following directory:/var/Rel2.0/mgmtsvr. Then type: tail -fmgmtsvr.3.log.

11 Verify that the snmp process is running on the MgmtSvr/AcctMgrbox by typing: ps -ef | grep snmp.

You should see one process running:/usr/local/sbin/snmpd -f udp:161. If this process isnot running or you get an alarm from the System ManagementConsole that there is a problem with the MgmtSvr/AcctMgr’ssnmp, then kill the process. The process will automaticallyrestart in about 30 seconds.

12 Restart the AcctMgr process. If any of the components sendan alarm in the System Management Console indicating thatthe component cannot communicate with the primary or backupCAM, then again restart the AcctMgr process. Right click on theAccounting component in the System Management Console andselect Restart. Confirm the request. A progress box will popup and then disappear when the restart has begun. While theAccounting module is rebooting, the other MCS component willthrow alarms to the System Management Console, but those willgo away when the Acct component is fully operational.

13 Now verify the state of the other machines and the MCScomponents on them. Launch the System Management Consoleas you normally would. Expand the navigation tree fully so thateach component is fully exposed.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 502: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

502 Powering up the network

14 Restart the AppSvr. If the clients are having troublecommunicating with the AppSvr, then restart the AppSvr. Rightclick on the AppSvr component in the System ManagementConsole and select Restart. Confirm the request. You shouldsee a progress window and it disappears after the restart hasbegun. To view AppSvr startup logs, Telnet to the AppSvrbox (we recommend that you Telnet as nortel) and go to thefollowing directory: /var/Rel2.0/appsvr. Type: tail -fappsvr.0.log.

15 Verify that the AppSvr’s snmp process is running. This step isthe same as step 3e above.

16 Restart the IPCM. If the i2004 phones are not responding toclient registers, the hollow blinking icons in the i2004 display,then restart the IPCM component. Right click on the IPCMcomponent in the System Management Console and selectRestart. Confirm the request. You see a progress window thatwill disappear after the restart has begun. To view IPCM startuplogs, Telnet to the IPCM box (recommend as nortel) and go tothe following directory: /var/Rel2.0/esm. Type: tail -fesm.1.log.

17 Verify that the IPCM’s snmp process is running. This step is thesame as step 3e above. Most likely, the Provisioning Moduleand the WebClient Module are deployed on the same box as theIPCM. Therefore, you only need to verify the snmp service onthe box once.

18 Restart the Provisioning component. The best way to see ifthe Provisioning Module is behaving properly is to log on tothe ProvClient. Launch a web browser to the IP address ofthe box where the Provisioning Module is deployed, such ashttp:192.168.0.10/prov. Log on as admin/admin and attemptany of the List options in the navigation tree, for example, ListDevices or List Users. If the browser doesn’t respond properly,then restart the Provisioning Module by right clicking on theProvisioning component in the System Management Consoleand select Restart. Confirm the request.

19 Restart the WebClientMgr component. Log on to the PersonalAgent through your browser, such as http://192.168.0.10/pa.Log on as a subscriber, such as [email protected],and enter the subscriber’s password. The Personal Agentshould appear for that subscriber. If it does not, or the interfacedoes not seem to respond correctly, restart the ProvisioningModule described in step 4e above. (If you’ve already restartedit once and the ProvClient interface seems ok, but the PersonalAgent interface is still not working correctly, it is probably aconfiguration issue.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 503: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 503

20 Launch the WebClient GUI from inside a subscriber’s PersonalAgent page. Attempt to log on to the WebClient when theSystem Management Console completes loading.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 504: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

504 Powering up the network

Powering up a Border Control Point

The Border Control Point shelves are powered from either a PDU or EBIPlocated at the top of the frame depending on if the frame is AC or DCpowered.

Procedure steps

Step Action

1 At the frame housing the Border Control Point

For DC powered shelves, switch the circuit breakers at the EBIPthat provide power to the shelf being powered to the ON position.The breaker will have the ’1’ side (top) of the breaker depressed.

2 For AC powered shelves, push the rocker located on the topof the AC outlet to apply power to the chassis. It should beswitched to the "|" position.

3 Once the breaker is switched ON at the power source, power upthe shelf by switching the breaker on the back of the chassis tothe ON position.

4 Observe the System, Power/Fan, Slot (filled) and Host CardLEDs as the unit is being powered. Once power on is complete,they should all have a steady green color.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 505: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 505

Powering up the IPmedia 2000 shelves

Power up the IPmedia 2000 shelves that house the MS 2000 seriesdevices and the Packet Media Anchor. The IPmedia 2000 shelves residein either a SAMF frame or CCF, depending on whether or not yoursolution uses the Communication Server 2000 or Communication Server2000-Compact.

Procedure steps

Step Action

1 At the CCF or SAMF frame

Open the front door of the CCF/SAMF frame.

2 At the EBIP at the top of the frame, turn the breaker associatedthe Media Server to the ON position.

3 Ensure that the Green Power LED on the front of the NortelMedia Server 2000 is on (right hand side of the card).

After 1 to 2 minutes, the Green LED on the left hand side of thecard lights.

4 Verify the status of the LEDs on the front of the Nortel MediaServer 2000

5 Red Fail LED is off (left hand side of the board).

6 ACT LED is on (left hand side of the board).

ATTENTIONIt may take up to 60 seconds for the ACT LED to light up.

7 Blue LED does not remain on (right hand side of the board).

8 PWR LED is on (right hand side of the board).

9 Both Ethernet Link LEDs are on (middle of the board).

ATTENTIONThe Ethernet LEDs are Green when indicating a connection, andOrange when indicating activity.

ATTENTIONCheck the Media Server 2000 to make sure it is set to an unlockedstate.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 506: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

506 Powering up the network

Powering up the SAM16 shelves

Power up the SAM16 shelves that house the UAS devices. The SAM16shelves reside in either a SAMF frame or CCF, depending on whether ornot your solution uses the Communication Server 2000 or CommunicationServer 2000-Compact.

Procedure steps

Step Action

1 At the CCF or SAMF frame

Open the front door of the CCF/SAMF frame.

2 At the EBIP at the top of the frame, turn the breakers associatedthe SAM16 to the ON position.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 507: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 507

Powering up an MG 9000 device

This procedure is used to power up MG 9000 when power to the MG 9000has been lost.

Procedure steps

Step Action

1 At the MG 9000 Manager

At the Subnet View of the MG 9000 Manager, select the MG9000 to be re-commissioned. A warning window indicating thatyour connection to the gateway is down may appear. This willnot block the saving of services information.

2 Save a copy of the MG 9000 Manager SLoA provisioning data asfollows (you’ll be re-entering this data later):

3 To Save the SLoA services provisioning data, at the Frame(Element) view, select the "Save SLOA services" option from theActions menu.

An acknowledgment window appears in response to this request.The acknowledgment window contains the name of the two files(with .html and .text suffixes) where your data had been storedon the MG 9000 Manager.

4 Record the file name provided in the Save acknowledgementwindow for later use.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 508: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

508 Powering up the network

5 Save a copy of the MG 9000 Manager PLoA provisioning data asfollows (you’ll be re-entering this data later):

6 To save the PLoA services provisioning data, at the Frame(Element) view, select the "Save PLOA services" option from theActions menu.

An acknowledgment window appears in response to this request.The acknowledgment window contains the name of the fileswhere your data had been stored on the MG 9000 Manager.

7 Record the file name provided in the Save acknowledgementwindow for later use.

8 Obtain the IP address of the MG 9000 Manager server(not the mid-tier or client). At the Subnet View, selectConfiguration->View/Modify NE Properties from the menu bar.

9 Record the IP address in the MG 9000 Manager IP Addressfield.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 509: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 509

10 FTP to the address recorded in step 5 and copy the files in the/tmp directory to a secure location, such as your desktop orpermanent server before proceeding.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 510: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

510 Powering up the network

11 At the MG 9000 frame

Power up the shelves in the frame by reinstalling the fusemodules on the IBIP. Restore all fuses left to right in the frame.

12 Connect a laptop PC to the active DCC card and launch the localcraft interface (LCI).

13 Wait for the restart to complete and for all cards initialize,typically takes 5 minutes. The Administrative State of allcommon cards will be Locked.

14 From the LCI, Unlock the active DCC OC-3/DS1 IMA card byselecting the Admin. State Unlock radio button. Wait for thestatus at the LCI to change to Enabled, Online, and Unlocked.

15 From the LCI continue to unlock the common cards in thefollowing order until all are unlocked and online:

active ITX

active ITP

OC-3 carriers on the active OC-3 DCC card

inactive DCC

inactive ITX

inactive ITP

OC-3 carriers on the inactive OC-3 DCC card

DS1 cards (if provisioned)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 511: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 511

ABI cards (if provisioned)

Initiate rediscovery by sending a cold start trap from the LCI.

16 At the MG 9000 Manager

Monitor the MG 9000 network element and wait until thediscovery process is complete.

17 Wait for the network element to show it is operational, thencheck for dial tone.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 512: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

512 Powering up the network

Viewing logs from a remote backup

View logs from a remote backup created during automatic and manualbackups of the remote server.

Procedure steps

Step Action

1 Log on to the Sun Netra server that is installed as the remotebackup server.

2 If you want to log on using ssh, enter

ssh -l

and press the Enter key.

3 If you want to log on using Telnet, enter

Telnet

and press the Enter key.

4 If you want to log on to the remote server console then go tostep 5.

5 Enter

less /var/adm/messages

The system responds by displaying the contents of the log file.

--End--

Variable definitionsVariable Value

<server> Name of the SPFS-based server.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 513: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 513

Setting date and time

Set date and time to reset the date and time information for a system.

If your user account has administrative privileges, you can set the dateand time information for a system. The date and time information includesmonth-day-year, hour-minute-second, and time zone data. In addition, youcan configure the system to automatically adjust the clock settings whendaylight savings time begins and ends.

You need to manually reset the date/time information if both of the RTCsystem nodes should go offline at the same time for any length of time.This usually occurs for one of the following reasons:

• if you perform a restore operation

• if you perform a complete office recovery (COR) on your system

• if your system is booted from an alternate boot snapshot

You can set your USP to use the simple network time protocol (SNTP)to query a network time protocol server. Each time the USP queries thenetwork time protocol server, the USP resets its time accordingly. TheSNTP protocol enables the USP to obtain the accurate time of day andkeep the same time as other elements in the network.

ATTENTIONIf simple network management protocol (SNMP) is not enabled for your system,verify the time settings for your system weekly to ensure that the system time issynchronized with the time on your OAMP workstation.

Procedure steps

Step Action

1 At the OAMP workstation

Click Administration>Date/Time window. The current date andtime settings for the system appear.

2 Determine if you want your USP to use SNTP.

3 If you do not want your USP to use SNTP then proceed to step5.

4 If you want to use your USP to use SNTP then check EnableSNTP Time Sync (User defined) in the SNTP portion of thewindow. Add an IP address for the SNTP server in the SNTPServer Address field. The information in the Time portion of thewindow is greyed out. Proceed to step 7.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 514: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

514 Powering up the network

5 To set the date, click the calendar icon and highlight the daycorresponding to today’s date.

6 To set the time of day, click the clock icon. Use the spin boxes toset hour, minutes and seconds.

7 To set the local time zone, click the Time Zone drop-downmenu. A world-wide list of time zones appears. Select the timezone listing that matches the local time zone.

The daylight savings time boxes are unavailable when the timezone you select does not use daylight savings time.

8 To enable or disable the automatic system clock adjustmentfor daylight savings time, click the Adjust clock for daylightsavings changes box.

If you are enabling the automatic system clock adjustment fordaylight savings time for the first time, make sure you performstep 9 to step 15.

9 Set the starting month for daylight savings time, clickdst-start-month. Select the month in which daylight savingstime starts.

10 Set the starting day in the dst-start-day spinbox.

11 Set the starting hour in the dst-start-hour spinbox.

12 Set the ending month for daylight savings time, clickdst-end-month. Select the month in which daylight savings timeends.

13 Set the ending day in the dst-end-day spinbox.

14 Set the ending hour in the dst-end-hour spinbox.

15 Click OK.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 515: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

515.

Server shut down and backupThis section lists the procedures you use to prepare for a shut down of theactive server, and a switchover to a remote backup server.

Navigation

• “Scheduling automatic backups on the remote server” (page 516)

• “Initiating a recovery back to the cluster” (page 519)

• “Initiating a switch over to the remote backup server” (page 520)

• “Shutting down an SPFS-based server” (page 522)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 516: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

516 Server shut down and backup

Scheduling automatic backups on the remote server

Schedule automatic backups on the remote sever. Backing up the primaryserver provides a standby backup system which is ready to provide serviceif the primary system is unavailable for an extended period of time. Theremote server can assume the identity of the primary server with data andfiles accurate to the last synchronization.

ATTENTIONThis procedure is for use with Geographic Survivability only.

Procedure steps

Step Action

1 Log on to the Sun Netra server that is installed as the remotebackup server. If you want to log on using ssh, enter:

ssh -l

and press the Enter key.

OR

If you want to log on using Telnet, enter:

Telnet

and press the Enter key.

2 When prompted, enter the root password.

3 Start the command line interface tool by entering:

cli

4 Select the Configuration menu.

The system displays the Configuration menu.

5 Select the Remote Backup option.

Response:

Remote Backup Configuration1-rbackup_display (Display Remote Backup Configuration)2-rbackup_config (Remote Backup Configuration)3-rbackup_exec (Execute Remote Backup Now)4-rbackup_cancel (Cancel Running Remote Backup)

X-exit

6 Select

2-rbackup_config (Remote Backup Configuration)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 517: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 517

The system responds with the IP address of the primary serverthat is currently configured as the remote server, and the timesthat are currently configured for automatic backups.

7 Enter the unit 0 IP address of the primary server to be backedup.

<nnn.nnn.nnn.nnn> is alive

8 If the system prompts you to accept the ssh key Enter yes. Goto Step 6.

9 If the system does not prompt you to accept the ssh key then goto Step 6.

10 Enter the first time for a daily backup to occur.

You can configure up to four times for daily backup to occur.

Response:

Enter a time for a daily backup to occur (HH:MM):

Response:

Enter a second time for a daily backup to occur (HH:MM) orenter "x" to stop provisioning backup times:

11 If you want to enter another time for a remote backup to occurenter a second time for a daily backup to occur. Go to Step 8.

12 If you do not want to enter another time for a remote backup tooccur enter x. Go to Step 10.

13 If you want to enter another time for a remote backup enter athird time for a daily backup.

14 If you do not want to enter another time for a remote backup tooccur enter x.

15 If you want to enter another time for remote backup to occurenter a fourth time for a daily backup to occur.

16 If you do not want to enter another time for a remote backup tooccur enter x.

17 To commit changers go to Step 11.

18 To exit enter quit.

19 To re-enter setting enter anything other than ok or quit.

20 Enter

ok

=== "rbackup_config"completed successfully

21 Exit the Remote Backup Configuration level.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 518: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

518 Server shut down and backup

x

--End--

Variable definitionsVariable Value

<server> The name of the SPFS-based server.

<nnn.nnn.nnn.nnn> The IP address that you entered.

<HH> The hours. Valid values are 00 to 23.

<MM> The minutes. Valid values are 00 to 59.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 519: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 519

Initiating a recovery back to the cluster

Initiate a recovery back to the cluster.

Prerequisites• If the server was previously a CBM and contains billing files not already

sent to a downstream billing server, using the cluster server, copythese files to a downstream server before performing this procedure.Otherwise, these files and the billing records will be lost. Contact nextlevel of support for assistance.

• Both cluster units 0 and 1 must be in the firmware mode. (Entercommand init 0 to perform this action.)

• In this procedure, unit0 server of the cluster is used as a remotebackup server. Use the same host name and IP address used toconfigure the remote backup server.

WARNINGIf both the remote backup unit and the high-availability (HA)pair are in service after a switch command, both units vie fordominance. This conflict results in a loss of communication tothe core.

Procedure steps

Step Action

1 Use only one automated schedule and make sure to select atime that will not be invoked shortly.

2 Bring down the machine currently active.

3 Follow procedure “Initiating a switch over to the remote backupserver” (page 520) to bring the services back to unit0 of thecluster.

4 If the server was previously a CBM and contains billing files notalready sent to a down-stream billing server, use the remotebackup server to copy these to a downstream server beforeperforming “Initiating a switch over to the remote backup server”(page 520). Otherwise these files and the billing records are lost.Contact next level of support for assistance.

5 Reinstall the backup server.

6 Reconfigure the backup server.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 520: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

520 Server shut down and backup

Initiating a switch over to the remote backup server

Initiate a switch over to the remote backup server.

ATTENTIONWhen the procedure is complete, the remote backup server reboots as the unit0of the cluster.

Prerequisites• Shut down the Cluster machine.

• You must be able to log on as the root or emsadm user in order toinitiate the switch command.

CAUTIONIf configuration, provisioning, patching or other “write”-typeoperations occurred since the last remote backup, the remotebackup system can be out of sync compared to the data innetwork elements and/or the primary OAM system.

Take actions before initiating the switchover to a remotebackup OAM server (that is, response to a geographic or otherprolonged outage of the primary OAM system) to halt or prevent“write”-type operations by OSSs and operations personnel untilan in-sync status is achieved.

When initiating a switchover to a remote backup OAM server,do not execute configuration, provisioning, patching or other“write”-type operations through the remote backup OAM systemuntil out-of-sync conditions are cleared.

Procedure steps

Step Action

1 Log on to the server by typing

> Telnet <server>

and pressing the Enter key.

2 When prompted, enter your user ID and password.

3 Change to the root user.

su - root

4 When prompted, enter the root password.

5 Invoke the switch by typing:

$ /opt/sspfs/rbks/switch

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 521: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 521

6 When ready, indicate you want to proceed by typing:

OK

--End--

Variable definitionsVariable Value

<server> the IP address or host name of the SPFS-basedremote backup server.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 522: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

522 Server shut down and backup

Shutting down an SPFS-based server

Use this procedure to shut down a Server Platform Foundation Software(SPFS)-based server, which may be hosting one or more of the followingcomponents:

• CS 2000 Management Tools

• Integrated Element Management System (IEMS)

• Audio Provisioning Server (APS)

• Media Gateway (MG) 9000 Manager

• CS 2000 SAM21 Manager

• Network Patch Manager (NPM)

• Core and Billing Manager (CBM)

• Multiservice Data Manager (MDM)

MDM when installed on SPFS-based servers is not configured as atwo-server cluster but as two distinct one-server configurations.

ATTENTIONThe SPFS-based server may be hosting more than one of the precedingcomponents, therefore, ensure it is acceptable to shut down the server.

PrerequisitesYou must have root user privileges.

Perform this procedure from a console only.

Prerequisites for Core and Billing Manager 850In order to perform this procedure, you must have the followingauthorization and access:

• You must be a user in a role group authorized to perform fault-adminactions. Therefore, Step 3 and Step 9 are not required.

• You must obtain non-restricted shell access.

For more information about how to log in to the CBM as an authorizeduser, how to request a non-restricted shell access, or how to displayactions a role group is authorized to perform, review the procedures in thefollowing table.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 523: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 523

Table 51Related procedures

Procedure Document

Logging in to the CBM Nortel Core and Billing Manager 850Administration and Security ( (NN10358-611))Core and Billing Manager 850 Security andAdministration for CDMA ((NN20000-320))Core and Billing Manager 850 Security andAdministration for GSM/UMTS ((NN20000-321))

Requesting non-restricted shell access Nortel Core and Billing Manager 850 Administration and Security ( (NN10358-611)) Core andBilling Manager 850 Security and Administrationfor CDMA () (NN20000-320)Core and BillingManager 850 Security and Administration forGSM/UMTS () (NN20000-321)

Displaying actions a role group is authorized toperform

Nortel Core and Billing Manager 850 Administration and Security ( (NN10358-611)) Core andBilling Manager 850 Security and Administrationfor CDMA () (NN20000-320)Core and BillingManager 850 Security and Administration forGSM/UMTS () (NN20000-321)

Procedure stepsUse one of the following procedures according to your office configuration:

• “One-server configuration” (page 523)

• “Two-server (cluster) configuration” (page 524)

One-server configuration

Step Action

1 At your workstation, log in to the server by typing

> telnet <IP address>

and pressing the Enter key.

where

IP address is the IP address of the SPFS-basedserver you want to power down

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su -

and pressing the Enter key.

4 When prompted, enter the root password.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 524: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

524 Server shut down and backup

5 Shut down the server by typing

# init 0

and pressing the Enter key.

The server shuts down gracefully, and the telnet connection isclosed.

6 If required, turn off the power to the server at the circuit breakerpanel of the frame.

To bring the server back up, turn on the power to the server atthe circuit breaker panel of the frame. The server recovers onits own once power is restored.

--End--

Two-server (cluster) configuration

Step Action

1 At your workstation, log in to the inactive server by typing

> telnet <IP address>

and pressing the Enter key.

where

IP address is the physical IP address of theInactive SPFS-based server in the clusteryou want to power down (unit 0 or unit 1)

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su - root

and pressing the Enter key.

4 When prompted, enter the root password.

Ensure you are on the Inactive server by typing ubmstat.If ClusterIndicatorACT is displayed in the response, whichindicates you are on the Active server, log out of that serverand log in to the other server through telnet using the physicalIP address of the other unit. The response must displayClusterIndicatorSTBY, which indicates you are on the Inactiveserver.

5 Shut down the Inactive server by typing

# init 0

and pressing the Enter key.

The server shuts down gracefully, and the telnet connection isclosed.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 525: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Variable definitions 525

6 If required, turn off the power to the Inactive server at the circuitbreaker panel of the frame. You have completed a partial powerdown (one server).

If you want to perform a full power down (both servers), proceedto step 7, otherwise, you have completed this procedure.

7

ATTENTIONOnly perform the remaining steps if you want to perform a full powerdown, which involves powering down both servers in the cluster.

Telnet to the Active server by typing

> telnet <IP address>

and pressing the Enter key.

8 When prompted, enter your user ID and password.

9 Change to the root user by typing

$ su - root

and pressing the Enter key.

10 When prompted, enter the root password.

11 Shut down the Active server by typing

# init 0

and pressing the Enter key.

The server shuts down gracefully, and the telnet connection isclosed.

12 If required, turn off the power to the servers at the circuit breakerpanel of the frame. You have completed a full power down (twoservers).

To bring the servers back up, turn on the power to the servers atthe circuit breaker panel of the frame. The servers recover ontheir own once power is restored.

--End--

Variable definitionsVariable Value

<IP address> is the physical IP address of the Active SPFS-basedserver in the cluster you want to power down

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 526: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

526 Server shut down and backup

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 527: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

527.

Communication Server 2000Management Tools fault management

This section provides procedural information on managing and analyzingfaults on the Communication Server 2000.

Navigation

• “Configuring SESM log reporting” (page 529)

• “Viewing debug logs” (page 532)

• “Viewing OMPUSH logs” (page 533)

• “Viewing customer logs on an SPFS-based server” (page 535)

• “Posting a line by directory number” (page 537)

• “Posting a line by gateway” (page 539)

• “Busying a line” (page 542)

• “Installation busying a line” (page 544)

• “Force releasing a line” (page 546)

• “Returning a line to service” (page 548)

• “Clearing one or more posted lines from the display” (page 550)

• “Retrieving line properties” (page 552)

• “Querying line gateways in a trouble state ” (page 554)

• “Performing trunk maintenance using the Trunk Maintenance Manager”(page 559)

• “Posting PRI Group D-channels” (page 573)

• “Performing an ISUP Continuity Test” (page 577)

• “Accessing the Alarm Manager” (page 579)

• “Retrieving details about an alarm” (page 586)

• “Filtering alarms” (page 589)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 528: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

528 Communication Server 2000 Management Tools fault management

• “Resetting the filters for alarms” (page 592)

• “Defining alarms using the NPM CLUI” (page 594)

• “Defining alarms using the NPM GUI” (page 596)

• “Performing an audit” (page 600)

• “Replacing one or more failed disk drives on an SPFS-based server”(page 621)

• “Replacing a DVD drive on an SPFS-based server” (page 629)

• “Replacing a failed SPFS-based server” (page 632)

• “Replacing failed Ethernet interfaces” (page 635)

• “Re-establishing the connection between the NPM GUI and the server”(page 637)

• “Restoring SSH communication between the CS 2000 ManagementTools server and the core manager” (page 639)

• “Enabling X11-forwarding” (page 642)

• “Disabling X11-forwarding” (page 645)

• “Displaying X11-forwarding status” (page 648)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 529: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 529

Configuring SESM log reporting

Use this procedure to configure SESM syslog and SESM debug log, andenable or disble SESM syslog alarm logging.

As of the SN07 release, the default northbound alarm feed is Corbanotifications to the Integrated Element Management System (IEMS). TheSESM syslog alarm feed is disabled by default to avoid duplication of thealarm logs. Step 19 in this procedure allows you to enable or disable theSESM syslog alarm feed when required.

PrerequisitesYou need root user privileges.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, telnet to the CS 2000 Management Toolsserver by typing

> telnet <IP address>

and pressing the Enter key.

where

IP address is the IP address of the CS 2000Management Tools server

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su -

and pressing the Enter key.

4 When prompted, enter the root password.

5 Change directory to where the configuration script is located bytyping

# cd /opt/nortel/NTsesm/admin/bin

and pressing the Enter key.

6 Execute the configuration script by typing

# ./configure

and pressing the Enter key.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 530: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

530 Communication Server 2000 Management Tools fault management

7 Enter the number next to the "SESM logging configuration"option in the menu.

8 To configure SESM syslog, enter the number next to the "SESMSyslog configuration" option in the menu.

9 When prompted, enter the maximum file size (in megabytes) ofSESM Syslog, or press the Enter key to accept the default valueof "3 meg".

10 When prompted, enter the SESM Syslog Backup Hour (for dailyfile rotation, or press the Enter key to accept the default value of"3" hours.

11 When prompted, enter the SESM Syslog Backup Minute, orpress the Enter key to accept the default value of "0" minutes.

The system sets the values you selected and returns you to theSESM Logging Configuration menu

12 To configure debug log, enter the number next to the "SESMDebug Log configuration" option in the menu.

13 When prompted, enter the maximum file size of SESM DebugLog, or press the Enter key to accept the default value of "20560KB".

14 When prompted, enter the maximum number of SESM DebugLog files, or press the Enter key to accept the default value of"7".

15 When prompted, enter the MI2 default log level, or press theEnter key to accept the default value of "MNR".

16 When prompted, enter the Proxy Agent default log level, or pressthe Enter key to accept the default value of "CRT".

17 When prompted, enter the SESM log file name, or press theEnter key to accept the default value of "ptmdebuglog".

18 When prompted, enter the log file rotation method, or press theEnter key to accept the default value of "Size based".

The system sets the values you selected and returns you to theSESM Logging Configuration menu.

19 To enable or disable syslog alarm logging, Enter the numbernext to the "SESM Alarm Logging configuration" option in themenu.

The status of the SESM syslog fault feed is determined by thevalue of the “Alarm Syslog Enabled” attribute. The default valuefor this setting is FALSE, meaning the SESM syslog alarmlogging is disabled.

20 When prompted, enter Y to enable syslog alarm logging or N todisable syslog alarm logging.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 531: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 531

The system returns you to the SESM Logging Configurationmenu

21 Exit SESM Logging Configuration by typing

select - x

and pressing the Enter key.

22 Exit SESM configuration by typing

select - x

and pressing the Enter key.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 532: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

532 Communication Server 2000 Management Tools fault management

Viewing debug logs

Use this procedure to view debug logs.

PrerequisitesNone

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, telnet to the CS 2000 Management Toolsserver by typing

> telnet <IP address>

and pressing the Enter key.

where

IP address is the IP address of the CS 2000Management Tools server

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su - root

and pressing the Enter key.

4 When prompted, enter the root password.

5 Display the debug logs by typing

# ptmctl display <log_type>

and pressing the Enter key.

where

log_type is MI2, PA, or MISC

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 533: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 533

Viewing OMPUSH logs

Use this procedure to view OMPUSH logs.

PrerequisitesNone

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, telnet to the Sun server by typing

> telnet <IP address>

and pressing the Enter key.

where

IP address is the IP address of the Sun serverwhere OMPUSH resides

2 When prompted, enter your user ID and password.

3 Use the following table to determine your next step.

If you want to Do

monitor the OMPUSH syslogstream

step 4

view the OMPUSH history log file step 5

save the OMPUSH history log fileto a file

step 6

4 Monitor the OMPUSH syslog stream by typing

# tail -f /var/adm/messages |grep -i ompush

and pressing the Enter key.

5 View the OMPUSH history log file by typing

# cat /var/adm/messages |grep -i ompush

and pressing the Enter key.

6 Save the OMPUSH history log file to a file by typing

# cat /var/adm/messages |grep -i ompush ><filename>

and pressing the Enter key.

where

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 534: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

534 Communication Server 2000 Management Tools fault management

filename is the name you want to give to the OMPUSHhistory log file you are saving

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 535: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 535

Viewing customer logs on an SPFS-based server

Use this procedure to view customer logs for the following components:

• Succession Element and Sub-element Manager (SESM)

• Gateway Controller (GWC)

• Server Platform Foundation Software (SPFS)

Use this procedure to view customer logs for the CS 2000 ManagementTool server.

Customer logs reside in directory /var/log/var/adm/custlog on the server.For details on customer logs, refer to Nortel Carrier Voice over IP FaultManagement Logs Reference Volume 1 ( (NN10275-909v1)) .

PrerequisitesNone

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, telnet to the server by typing

> telnet <IP address>

and pressing the Enter key.

2 When prompted, enter your user ID and password.

3 Access the directory where the customer log files reside by tying

$ cd/var/logcd/var/adm/custlog

and pressing the Enter key.

4 List the directory content by typing

$ ls

and pressing the Enter key.

The customer log files are appended with numbers, for example"customerlog.0". The files with the lower numbers are the newerfiles.

5 Use the following table to determine your next step.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 536: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

536 Communication Server 2000 Management Tools fault management

If you want to Then

view the entire content of a logfile

step 6 only

view specific content of a log file step 7 only

6 View the entire content of a log file by typing

$ cat <log_filename> |more

and pressing the Enter key.

Example$ cat custlog.0 |more

7 View specific content of the log file by typing

# cat <log_filename> |grep <search_string>

and pressing the Enter key.

Example$ cat custlog.0 |grep SPFS350

8 To print the contents of this file, contact your site systemadministrator for assistance with using UNIX print commandsand with locating a printer connected to your network.

--End--

Variable definitionsVariable Value

<IP address> is the IP address of the SPFS-based server

<log_filename> is the name of the customer log file you want to view

<search_string> is the text you want to search for.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 537: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 537

Posting a line by directory number

Use this procedure to post a line by its directory number (DN) so you canperform maintenance on the line.

RestrictionsV5.2 line maintenance is not supported using the Line MaintenanceManager (LMM). V5.2 line maintenance is performed using the CS 2000XA-Core MAPCI interface.

Since the CS 2000 supports both legacy DMS TDM lines as well as CVoIPlines, TDM lines can be posted on LMM and maintenance operationslike BSY/RTS/FRLS/INB can be performed; however, the CS Line Statecolumn will not be displayed accurately. The Endpoint State column willshow "Legacy Line" and the GW Profile column and all properties notapplicable to TDM lines will display as "Not Available".

When using hunt groups, you can only post the primary DN using the LineMaintenance Manager. You cannot perform line maintenance operationson all members of a hunt group using the LMM. If you need to perform linemaintenance on all members of a hunt group, use the CS 2000 XA-CoreMAPCI interface. For details, refer to the DMS-100 Family National ISDNBRI Service Implementation Guide () (297-2401-201).

PrerequisitesAll Line Maintenance Manager (LMM) status fields must be Green (OK)in order to post lines.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, launch the Line Maintenance Manager(LMM) GUI. For more information, see ATM/IP Solution-levelAdministration and Security (NN10402-600).

2 At the LMM GUI, select Post DN, enter the directory number,and click Post.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 538: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

538 Communication Server 2000 Management Tools fault management

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 539: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 539

Posting a line by gateway

Use this procedure to post one or more lines by their associated gatewayso you can perform maintenance on the lines.

Use this procedure after adding a gateway and before performingmaintenance on that gateway.

Restrictions• V5.2 line maintenance is not supported using Line Maintenance

Manager (LMM)

• V5.2 line maintenance is performed using the CS 2000 XA-CoreMAPCI interface.

• Only small line and MG9K gateways can be posted.

• You can only get Call information about MG9K gateways; you cannotget Endpoint state information.

• Gateways with a profile of SIP, CICM or third-party are not supported.

PrerequisitesThe following prerequisites apply to posting one or more lines by gateway:

• The associated gateway must be provisioned with endpoints in the CS2000 GWC Manager database

• All LMM status fields must be Green (OK) in order to post lines.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, launch the Line Maintenance Manager(LMM) GUI. For more information about how to launch CS 2000management tools and NPM client applications, see ATM/IPSolution-level Administration and Security (NN10402-600).

2 At the LLM GUI, select Post by Gateway, enter the name of thegateway and its path, then click Post.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 540: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

540 Communication Server 2000 Management Tools fault management

--End--

Additional informationWhen post-by-gateway is selected on the LMM GUI, the line state at theCM is displayed. A on-demand audit to identify the state of the endpointon the gateway is also done through the CS 2000 GWC Manager andthe endpoint state is displayed at the LMM GUI Endpoint State field. Thisaudit is done in a protocol independent manner by the LMM. Any errorsproduced during the posting activity will be displayed on the status panelshown below.

A list of currently supported LMM GUI error codes with descriptions isshown in the following table.

ErrorCode

LMM client text displayed instatus panel Description

1000 OK Audit response received fromendpoint

1010 Wait_GW_response GWC waiting for gateway auditresponse

1020 GW_Protocol_Unsupported Unsupported gateway protocol

1100 GWC_Unavailable GWC not available for endpoint query

1200 IP_Unavailable Unable to retrieve IP address forgateway

2000 SNMP_exception SNMP error between LMM and GWC

2010 EP_Not_Found Endpoint status expired in GWCbefore retrieval

2020 Invalid_EP Endpoint does not exist in GWC

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 541: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Additional information 541

ErrorCode

LMM client text displayed instatus panel Description

3000 GWC_LoadName_error Incompatible GWC load versiondetected

3010 Unsupported_GWC_Load GWC is running an unknown orunsupported load

4000 Decode_Failure Error decoding Gateway IP address

4010 Unknown_Endpoint LMM could not get endpoint state

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 542: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

542 Communication Server 2000 Management Tools fault management

Busying a line

Use this procedure to busy (BSY) a line and put it into a maintenance busy(MB) state. Once busied, the line cannot perform call processing.

Use this procedure during maintenance or alarm clearing activities.

RestrictionsV5.2 line maintenance is not supported using Line Maintenance Manager.V5.2 line maintenance is performed using the CS 2000 XA-Core MAPCIinterface

PrerequisitesThe line must be posted. Refer to procedures “Posting a line by gateway”(page 539) or “Posting a line by directory number” (page 537) in thisdocument, if required.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, launch the Line Maintenance Manager(LMM) GUI. For more information about how to launch CS 2000management tools and NPM client applications, see ATM/IPSolution-level Administration and Security (NN10402-600).

2 At the LMM GUI, select the DN you want to busy, then on theActions menu, click BSY, or right-click on the selected DN thenclick BSY.

You can perform this operation on one or more posted linesat the same time. You can use the Shift key to select multiplelines consecutively, or the Ctrl key to select multiple linesnon-consecutively

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 543: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 543

3 Observe that the selected DN(s) are put into a maintenance busy(MB) state as shown below.

The line state will be CPD if the original line state was CPB priorto Busy.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 544: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

544 Communication Server 2000 Management Tools fault management

Installation busying a line

Use this procedure to set a line to the installation busy (INB) state.

Use this procedure during maintenance and fault clearing activities.

RestrictionsV5.2 line maintenance is not supported using Line Maintenance Manager.V5.2 line maintenance is performed using the CS 2000 XA-Core MAPCIinterface

PrerequisitesThe line must be posted. Refer to procedures “Posting a line by gateway”(page 539) or “Posting a line by directory number” (page 537) in thisdocument, if required.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, launch the Line Maintenance Manager(LMM) GUI. For more information about how to launch CS 2000management tools and NPM client applications, see ATM/IPSolution-level Administration and Security (NN10402-600).

2 At the LMM GUI, select the DN you want to installation busy,then on the Actions menu, click INB, or right-click on theselected DN then click INB.

You can perform this operation on one or more posted linesat the same time. You can use the Shift key to select multiplelines consecutively, or the Ctrl key to select multiple linesnon-consecutively.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 545: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 545

3 Observe that the selected DN(s) are put into an installation busy(INB) state as shown below.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 546: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

546 Communication Server 2000 Management Tools fault management

Force releasing a line

Use this procedure to force release (FRLS) a line, putting it into amaintenance busy (MB) state. Use this procedure if recommended byNortel support personnel.

Restrictions• V5.2 line maintenance is not supported using Line Maintenance

Manager (LMM).

• V5.2 line maintenance is performed using the CS 2000 XA-CoreMAPCI interface.

• Only line gateways can be posted.

PrerequisitesThe line must be posted. Refer to procedures “Posting a line by gateway”(page 539) or “Posting a line by directory number” (page 537) in thisdocument, if required.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, launch the Line Maintenance Manager(LMM) GUI. For more information about how to launch CS 2000management tools and NPM client applications, see ATM/IPSolution-level Administration and Security (NN10402-600).

2 At the LMM GUI, select the DN you want to force a release on,then on the Actions menu, click FRLS, or right-click on theselected DN then click FRLS.

You can perform this operation on one or more posted linesat the same time. You can use the Shift key to select multiplelines consecutively, or the Ctrl key to select multiple linesnon-consecutively.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 547: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 547

3 Click OK to confirm the force release of the line(s).

The call will be dropped if there was a call on the selected DN.

4 Observe that the selected DN(s) are put into an maintenancebusy (MB) state as indicated in the following figure.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 548: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

548 Communication Server 2000 Management Tools fault management

Returning a line to service

Use this procedure to return a busied line to service (RTS). Once returnedto service, the line can then perform call processing.

Use this procedure after maintenance has been performed on a busiedline.

RestrictionsV5.2 line maintenance is not supported using Line Maintenance Manager.V5.2 line maintenance is performed using the CS 2000 XA-Core MAPCIinterface

PrerequisitesThe line must be in a maintenance busy (MB) state. Refer to procedure“Busying a line” (page 542) in this document, if required.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, launch the Line Maintenance Manager(LMM) GUI. For more information about how to launch CS 2000management tools and NPM client applications, see ATM/IPSolution-level Administration and Security (NN10402-600).

2 At the LMM GUI, select the DN you want to return to service,then on the Actions menu, click RTS, or right-click on theselected DN then click RTS.

You can perform this operation on one or more posted linesat the same time. You can use the Shift key to select multiplelines consecutively, or the Ctrl key to select multiple linesnon-consecutively.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 549: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 549

3 Observe that the state of the selected DN(s) is changed frommaintenance busy (MB) to idle (IDL) as shown below.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 550: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

550 Communication Server 2000 Management Tools fault management

Clearing one or more posted lines from the display

Use this procedure to remove a directory number (DN) from the list ofposted DNs.

Use this procedure before performing maintenance on a posted set of linesor when the posted lines no longer need to be monitored.

RestrictionsV5.2 line maintenance is not supported using Line Maintenance Manager.V5.2 line maintenance is performed using the CS 2000 XA-Core MAPCIinterface

PrerequisitesThe line must be posted. Refer to procedures “Posting a line by gateway”(page 539) or “Posting a line by directory number” (page 537) in thisdocument, if required.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, launch the Line Maintenance Manager(LMM) GUI. For more information about how to launch CS 2000management tools and NPM client applications, see ATM/IPSolution-level Administration and Security (NN10402-600).

2 At the LMM GUI, select the DN you want to remove from thedisplay of posted lines, then on the Actions menu, click Clear,or right-click on the selected DN then click Clear.

You can perform this operation on one or more posted linesat the same time. You can use the Shift key to select multiplelines consecutively, or the Ctrl key to select multiple linesnon-consecutively.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 551: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 551

3 Observe that the selected DN(s) are cleared from the list asshown below.

To clear all posted DN, click Clear All.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 552: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

552 Communication Server 2000 Management Tools fault management

Retrieving line properties

Use this procedure to retrieve and display the line properties for a selecteddirectory number (DN) or group of DNs.

Use this procedure as a part of fault clearing activities.

RestrictionsV5.2 line maintenance is not supported using Line Maintenance Manager.V5.2 line maintenance is performed using the CS 2000 XA-Core MAPCIinterface

PrerequisitesThe line must be posted. Refer to procedures “Posting a line by gateway”(page 539) or “Posting a line by directory number” (page 537) in thisdocument, if required.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, launch the Line Maintenance Manager(LMM) GUI. For more information about how to launch CS 2000management tools and NPM client applications, see ATM/IPSolution-level Administration and Security (NN10402-600).

2 At the LMM GUI, select the DN you want to view the propertiesfor, then on the Actions menu, click Properties, or right-click onthe selected DN then click Properties.

You can perform this operation on one or more posted linesat the same time. You can use the Shift key to select multiplelines consecutively, or the Ctrl key to select multiple linesnon-consecutively.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 553: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 553

The Properties window, similar to the following, opens.

If a gateway was provisioned with IP address 0.0.0.0.0(Discovery), the IP address shown in the Properties windowwill indicate the current Dynamic IP address of the gateway asreflected in the Gateway Controller.

3 Click OK to close the Properties window.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 554: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

554 Communication Server 2000 Management Tools fault management

Querying line gateways in a trouble state

This procedure describes how to perform a query on line gateways in atrouble state, and view reports.

You can configure a query to run on a daily or weekly basis at a specifictime. For more information about how to configure a query for linegateways in a trouble state, see Nortel ATM/IP Solution-level Configuration(NN10409-500).

Use this procedure to troubleshoot line gateways.

PrerequisitesAll Line Maintenance Manager (LMM) status fields must be Green (OK)in order to perform a query.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, launch the Line Maintenance Manager(LMM) GUI. For more information about how to launch CS 2000management tools and NPM client applications, see ATM/IPSolution-level Administration and Security (NN10402-600).

2 At the LMM GUI, Diagnostics menu, click Query gateways introuble state.

3 Use the following table to determine your next step.

If you want to Then

perform a query step 4

view reports step 8

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 555: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 555

4 Click Query to query all line gateways associated with allgateway controllers for the switch, or click the Select linegateway controllers check box to query all line gatewaysassociated with specific gateway controllers .

5 Use the following table to determine your next step.

If you are querying Then

specific gateway controllers step 6

all gateways step 7

6 Select the reachable gateway controller(s) of your choice, pressCtrl+a, or click the Select all reachable gateway controllerscheck box, then click Start.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 556: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

556 Communication Server 2000 Management Tools fault management

The system starts the query.

Click Cancel at any time to cancel the query.

7 Click OK once the query completes.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 557: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 557

8 Select a report from the list and click View Report. If youjust performed a query, the report is the one dispayed in theReports: box.

The system maintains a maximum number of seven reports foreach gateway controller.

9 Click on the gateway controller of your choice to display theassociated gateways and their trouble state information.

The gateway can be in any one of the following states:

• GW OK

• GW introuble

• GW inservice trouble

• GW disabledThe "GW disabled" state has security details appended toit to quickly determine if security is causing the gatewaydisabled condition. The security details are as follows:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 558: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

558 Communication Server 2000 Management Tools fault management

State Meaning

GW DISABLED; SECURE:200 (0,0) gateway has secure policy(policy ID = 200), but no SAs

GW DISABLED; FLEX-secure:400(0,0)

gateway has flex policy(policy ID = 400) in securemode, but no SAs

GW DISABLED; FLEX-non-secure:400

gateway has flex policy(policy ID = 400) innon-secure mode

GW DISABLED; FLEX-unknown:400 gateway has flex policy(policy ID = 400), IP known,but security mode not yetdiscovered

GW DISABLED; BYPASS:default gateway has bypass policy

GW DISABLED; BYPASS:1000 gateway has bypass policy(policy ID = 1000)

GW DISABLED; DISCARD:2000 gateway has discard policy(policy ID = 2000)

The policy ID numbers are only examples

If the security details indicate a SECURE or FLEX-securepolicy without Security Associations (SAs), that is "0,0", whichrepresents the count of incoming and outgoing SAs respectively,then security is likely involved and needs to be verified. If adiscard policy is provisioned, this will cause all messaging to orfrom the gateway to be discarded, and will cause the disabledstate. If the gateway uses Dynamic Host Configuration Protocol(DHCP), and it has not yet discovered its IP address, the state ofthe gateway will be disabled with no security details.

10 As required, you can select other reports from the Query Reportwindow. You can scroll through the list of gateways using thePrev (previous) and Next buttons.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 559: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 559

Performing trunk maintenance using the TrunkMaintenance Manager

Use this procedure to perform maintenance operations on trunks using theTrunk Maintenance Manager (TMM). This procedure provides the steps toperform maintenance operations on trunks using any one of the followingmethods:

• “Maintenance by carrier” (page 560)

• “Maintenance by Gateway Name” (page 564)

• “Maintenance by trunk CLLI” (page 568)

The maintenance operations available, are as follows:

• post - posts trunk for maintenance activities

• busy - removes trunk from service and makes it unavailable for callprocessing

• installation busy - busies trunk from the MB (manual busy) or NEQ (notequipped) state for provisioning or test purposes

• return to service - returns trunk to service and makes it available forcall processing

• force release - forces trunk already involved in call processing, out ofservice, and makes it unavailable for call processing

You can also query carrier and endpoint states using methods“Maintenance by carrier” (page 560) or “Maintenance by Gateway Name”(page 564) respectively.

Maintenance operations using the Trunk Maintenance Manager (TMM) aresupported on ISUP, PRI (Carrier Voice over IP only), and PTS trunks.

PrerequisitesNone

Procedure stepsPerform the steps under one of the following methods to complete thisprocedure:

• “Maintenance by carrier” (page 560)

• “Maintenance by Gateway Name” (page 564)

• “Maintenance by trunk CLLI” (page 568)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 560: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

560 Communication Server 2000 Management Tools fault management

Maintenance by carrier

Step Action

1 At your workstation, launch the Trunk Maintenance Manager(TMM) GUI. For more information about how to launch CS 2000management tools and NPM client applications, see ATM/IPSolution-level Administration and Security (NN10402-600).

2 At the TMM GUI, click Maintenance By Carrier.

A window similar to the following appears:

3 Enter the request criteria as follows:

• Select a gateway from the Gateway Name list.

• Select a gateway from the Gateway Name list.The Force Release Endpoints (FRLS) maintenance actiondoes not support a range of endpoints. Individual endpointsmust be specified, or listed with a comma between them formultiple endpoints (eg, 1,3,5,10).

• Optionally select the state for the endpoints you wantdisplayed, or use the default (All States) value to displayendpoints in all states.

• Optionally select one or more carriers from the CarrierNames list, and click Add to add the selected carriers to therequest input, or use the default (All Carriers) value for allcarriers.

• Optionally click the Show Details check box if you want todisplay additional details about the carriers, otherwise, only asummary of the carriers will be displayed.

4 Use the following table to determine your next step:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 561: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 561

If you want to Then

Query carrier state step 4 only

Post carrier step 5 only

Busy carrier step 6 only

Return carrier to service step 7 only

Force release carrier step 8 only

Installation busy carrier step 9 only

5 Select Query Carrier State from the Maintenance Action list,and click Go.

Following is an example of the response with Show Detailschecked:

Proceed to step 10.

6 Select Post Carrier from the Maintenance Action list, and clickGo.

Following is an example the response with Show Detailschecked:

Proceed to step 10.

7 Select Busy Carrier (BSY) from the Maintenance Action list,and click Go.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 562: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

562 Communication Server 2000 Management Tools fault management

If you attempted to busy more than one carrier and the NeedConfirmation check box is checked, you need to confirm thebusy action as shown in the following figure.

If it is acceptable to busy more than one carrier, click OK.

Following is an example of the response:

Observe that the selected carriers are manual busy (MB).

Proceed to step 10.

8 Select Return Carrier to Service (RTS) from the MaintenanceAction list, and click Go.

Following is an example of the response:

Observe that the selected carriers are idle (IDL).

Proceed to step 10.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 563: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 563

9 Select Force Release Carrier (FRLS) from the MaintenanceAction list, and click Go.

The Force Release Endpoints (FRLS) maintenance action doesnot support a range of endpoints. Individual endpoints mustbe specified, or listed with a comma between them for multipleendpoints (eg, 1,3,5,10).

Following is an example of the response:

Observe that the selected carriers are manual busy (MB).

Proceed to step 10.

10 Select Installation Busy Carrier (BSY INB) from theMaintenance Action list, and click Go.

Following is an example of the response:

Observe that the selected carriers are installation busy (INB).

11 Use the following table to determine your next step.

If you Then

want to perform another action step 4

do not want to perform anotheraction

you have completed thisprocedure

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 564: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

564 Communication Server 2000 Management Tools fault management

Maintenance by Gateway Name

Step Action

1 At your workstation, launch the Trunk Maintenance Manager(TMM) GUI. For more information about how to launch CS 2000management tools and NPM client applications, see ATM/IPSolution-level Administration and Security (NN10402-600).

2 At the TMM GUI, click Maintenance By Gateway Name.

A window similar to the following, appears.

3 Enter the request criteria as follows:

• Select a gateway from the Gateway Name list.

• Optionally enter an endpoint range value, or use the default(0-) value to display all endpoints.

• Optionally click the Show Details check box if you want todisplay additional details about the endpoints, otherwise, onlya summary of the endpoints will be displayed.

• Optionally select the state for the endpoints you wantdisplayed, or use the default (All States) value to displayendpoints in all states.

4 Use the following table to determine your next step.

If you want to Then

Query endpoint state step 5 only

Post endpoints step 6 only

Busy endpoints step 7 only

Return endpoints to service step 8 only

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 565: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 565

If you want to Then

Force release endpoints step 9 only

Installation busy endpoints step 10 only

5 Select Query Endpoint State from the Maintenance Action list,and click Go.

Following is an example of the response with Show Detailschecked:

Proceed to step 11.

6 Select Post Endpoints from the Maintenance Action list, andclick Go.

Following is an example of the response with Show Detailsunchecked:

Proceed to step 11.

7 Select Busy Endpoints (BSY) from the Maintenance Actionlist, and click Go.

If you attempted to busy more than one endpoint and the NeedConfirmation check box is checked, you need to confirm thebusy action as shown in the following figure.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 566: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

566 Communication Server 2000 Management Tools fault management

If it is acceptable to busy more than one endpoint, click OK.

Following is an example of the response with Show Detailschecked:

Observe that the selected trunk endpoints are manual busy(MB).

Proceed to step 11.

8 Select Return Endpoints to Service (RTS) from theMaintenance Action list, and click Go.

Following is an example of the response with Show Detailschecked:

Observe that the selected trunk endpoints are idle (IDL).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 567: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 567

Proceed to step 11.

9

CAUTIONCall lossIf you force release trunk endpoints already involvedin call processing, the endpoints will transition tomanual busy (MB) and the calls in progress will bedropped.

Select Force Release Endpoints (FRLS) from the MaintenanceAction list, and click Go.

The Force Release Endpoints (FRLS) maintenance action doesnot support a range of endpoints. Individual endpoints mustbe specified, or listed with a comma between them for multipleendpoints (eg, 1,3,5,10).

Following is an example of the response with Show Detailschecked:

Observe that the selected trunk endpoints are manual busy(MB).

Proceed to step 11.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 568: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

568 Communication Server 2000 Management Tools fault management

10 Select Installation Busy Endpoints (BSY INB) from theMaintenance Action list, and click Go.

Following is an example of the response with Show Detailschecked:

Observe that the selected trunk endpoints are installation busy(INB).

11 Use the following table to determine your next step.

If you Then

want to perform another action step 4

do not want to perform anotheraction

you have completed thisprocedure

--End--

Maintenance by trunk CLLI

Step Action

1 At your workstation, launch the Trunk Maintenance Manager(TMM) GUI. For more information about how to launch CS 2000management tools and NPM client applications, see ATM/IPSolution-level Administration and Security (NN10402-600).

2 At the TMM GUI, click Maintenance By Trunk CLLI.

A window similar to the following, appears.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 569: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 569

3 Enter the request criteria as follows:

• Enter the trunk CLLI name under Trunk CLLI.

• Optionally enter a trunk range value, or use the default (0-)value to display all trunk members.

4 Use the following table to determine your next step.

If you want to Then

Post trunks step 5 only

Busy trunks step 6 only

Return trunks to service step 7 only

Force release trunks step 8 only

Installation busy trunks step 9 only

5 Select Post Trunks from the Maintenance Action list, and clickGo.

Following is an example of the response:

Proceed to step 10.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 570: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

570 Communication Server 2000 Management Tools fault management

6 Select Busy Trunks (BSY) from the Maintenance Action list,and click Go.

If you attempted to busy more than one trunk member and theNeed Confirmation check box is checked, you need to confirmthe busy action as shown in the following figure.

If it is acceptable to busy more than one trunk member, click OK.

Following is an example of the response:

Observe that the selected range of trunk members are manualbusy (MB).

Proceed to step 10.

7 Select Return Trunks to Service (RTS) from the MaintenanceAction list, and click Go.

Following is an example of the response:

Observe that the selected trunk members are idle (IDL).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 571: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 571

Proceed to step 10.

8

CAUTIONCall lossIf you force release a trunk already involved in callprocessing, the trunk will transition to manual busy(MB) and the calls in progress will be dropped.

Select Force Release Trunks (FRLS) from the MaintenanceAction list, and click Go.

The Force Release Endpoints (FRLS) maintenance action doesnot support a range of endpoints. Individual endpoints mustbe specified, or listed with a comma between them for multipleendpoints (eg, 1,3,5,10).

Following is an example of the response:

Observe that the selected range of trunk members are manualbusy (MB).

Proceed to step 10.

9 Select Installation Busy Trunks (BSY INB) from theMaintenance Action list, and click Go.

Following is an example of the response:

Observe that the selected range of trunk members areinstallation busy (INB).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 572: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

572 Communication Server 2000 Management Tools fault management

10 Use the following table to determine your next step.

If you Then

want to perform another action step 4

do not want to perform anotheraction

you have completed thisprocedure

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 573: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 573

Posting PRI Group D-channels

Use this procedure to display statistics about the D-channels for a selectedPRI trunk and perform maintenance operations.

This procedure can only be performed on Carrier Voice over IP PRI trunks.

PrerequisitesNone

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstationLaunch the Trunk Maintenance Manager(TMM) GUI. For more information about how to launch CS 2000management tools and NPM client applications, see ATM/IPSolution-level Administration and Security (NN10402-600).

2 At the TMM GUI, click D-Channel Maintenance.

A window similar to the following, appears.

3 Enter the trunk CLLI name, then select Post Group D-Channelfrom the Maintenance Action drop down menu, and click Go.

4 Observe that the selected range of PRI trunk D-channel appearsas shown in the following example.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 574: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

574 Communication Server 2000 Management Tools fault management

5 Select the desired maintenance operation as shown in thefollowing example.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 575: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 575

Displaying trunk CLLI codes by gateway

Use this procedure to retrieve a list of trunk CLLI codes associated with aspecific trunk gateway.

PrerequisitesNone

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, launch the Trunk Maintenance Manager(TMM) GUI. For more information about how to launch CS 2000management tools and NPM client applications, see ATM/IPSolution-level Administration and Security (NN10402-600).

2 At the TMM GUI

Click Get TrkCllis by GW Name.

A window similar to the following, appears.

3 From the Gateway Name list, select a gateway, then click Go.

The trunk CLLI codes associated with the trunk gateway aredisplayed as shown in the following example.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 576: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

576 Communication Server 2000 Management Tools fault management

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 577: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 577

Performing an ISUP Continuity Test

Use this procedure to perform an ISUP (Integrated Services DigitalNetwork User Part) continuity test (ICOT) on a single trunk member fora specific CLLI.

This test can only be performed on outgoing or 2-way ISUP trunks.

PrerequisitesThe trunk must be in a maintenance busy (MB) state. Refer to procedure“Performing trunk maintenance using the Trunk Maintenance Manager”(page 559) in this document, if required.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, launch the Trunk Maintenance Manager(TMM) GUI.

2 At the TMM GUI

Click ICOT Test.

A window similar to the following, appears.

3 Enter the trunk CLLI name and member number.

4 Select Perform ICOT Test from the Maintenance Action list,and click Go.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 578: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

578 Communication Server 2000 Management Tools fault management

5 Observe that the test results and continuity conditions for thetrunk number are displayed as shown in the following example.

If you encounter any unexpected errors such as "test failed",review any details and determine a solution, or contact your nextlevel of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 579: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 579

Accessing the Alarm Manager

This procedure provides access to service-related alarms on the hardwareapplication. The Alarm Manager provides information on the fieldsdescribed in the following table:

Table 52Alarm Manager field details

Field Name Description

Network Elementname

Provides the name of the Network Element for whichthe selected alarm is being raised.

Category Represents the alarm problem category values (ITU-TX.733):

"Communications"

"Quality of Service"

"Processing Error"

"Equipment Error" "Environment"

Alarm Time Represents the time that the alarm was raised in thefollowing format:

HH:MM:SS day-Month-year

Severity Provides the associated severity of the alarm beingraised. The range of values is as follows:

"Critical"

"Major"

"Minor"

"Warning"

Probable Cause Provides the probable cause of the particular alarm(X.733 Probable Cause):

"Adapter Error"

"Non-Repudiation Failure"

"Bandwidth Reduced"

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 580: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

580 Communication Server 2000 Management Tools fault management

Table 52Alarm Manager field details (cont’d.)

Field Name Description

"Call Establishment Error"

"Communications Protocol Error"

"Communications Subsystem Error"

"Configuration or Customization Error"

"Congestion"

"Corrupt Data"

"CPU Cycles Limit Exceeded"

"Data Set or Modem Error"

"Degraded Signal"

"DTE DCE Interface Error"

"Enclosure Door Open"

"Equipment Malfunction"

"Excessive Vibration"

"File Error"

"Fire Detected"

"Flood Detected"

"Framing Error"

"Heating, Ventilation, or Cooling System Problem"

"Humidity Unacceptable"

"Input Output Device Error"

"Input Device Error"

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 581: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 581

Table 52Alarm Manager field details (cont’d.)

Field Name Description

"LAN Error"

"Leak Detected"

"Local Node Transmission Error" "Loss of Frame"

"Loss of Signal"

"Material Supply Exhausted"

"Multiplexer Problem"

"Out of Memory"

"Output Device Error"

"Performance Degraded"

"Power Problem"

"Pressure Unacceptable"

"Pump Failure"

"Queue Size Exceeded"

"Receive Failure"

"Remote Node Transmission Error"

"Resource At or Nearing Capacity"

"Response Time Excessive"

"Retransmission Rate Excessive"

"Software Error"

"Software Program Abnormally Terminated"

"Software Program Error"

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 582: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

582 Communication Server 2000 Management Tools fault management

Table 52Alarm Manager field details (cont’d.)

Field Name Description

"Storage Capacity Problem"

"Temperature Unacceptable"

"Threshold Crossed"

"Timing Problem"

"Toxic Leak Detected"

"Transmit Failure"

"Transmitter Failure"

"Underlying Resources Unavailable"

"Version Mismatch"

"Authentication Failure"

"Breach of Confidentiality"

"Cable Tamper"

"Delayed Information"

"Denial of Service" "Duplicate Information"

"Information Modification Detected"

"Information Missing"

"Information Out of Sequence"

"Intrusion Detection"

"Key Expired"

"Non-Repudiation Failure"

"Out of Hours Activity"

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 583: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 583

Table 52Alarm Manager field details (cont’d.)

Field Name Description

"Out of Service"

"Procedural Error"

"Unauthorized Access Attempt"

"Unexpected Information"

"Unspecified Reason"

"Probable Cause Conversion Failed"

To avoid using too much memory to display the alarms, only 1000 alarmsare displayed at any given time. Alarms that are no longer displayed onthe Alarm Manager can be viewed using the Alarm History window, whichuses paging to minimize the size to the GUI.

PrerequisitesNone

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, launch the CS 2000 Management ToolsGUI. For more information about how to launch CS 2000management tools and NPM client applications, see ATM/IPSolution-level Administration and Security (NN10402-600).

2 At the CS2000 Management Tools GUI, on the Fault menu, clickAlarm Manager.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 584: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

584 Communication Server 2000 Management Tools fault management

The Alarm Manager window, similar to following, appears.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 585: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 585

Figure 56Alarm Manager window

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 586: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

586 Communication Server 2000 Management Tools fault management

Retrieving details about an alarm

Use this procedure to display detailed information about service-relatedalarms on the managed network elements. The detailed view providesinformation on the items in the following table:

Table 53Alarm details fields

Field Name Description

NE Name Provides the name of the Network Element for which theselected alarm is being raised.

Alarm Level Provides the associated severity of the alarm being raised.The range of values is as follows:

"Critical"

"Major"

"Minor"

"Warning"

Category Represents the alarm problem category values (ITU-TX.733):

"Communications"

"Quality of Service"

"Processing Error"

"Equipment Error" "Environment"

Alarm Time Represents the time that the alarm was raised in thefollowing format:

HH:MM:SS day-Month-year

Probable Cause Provides the probable cause of the particular alarm (X.733Probable Cause).

System uptime Represents the time since the network element was lastre-initialized. Here is an example of how system uptime isdisplayed:

2 hours, 10 minutes, 30 seconds

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 587: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 587

Table 53Alarm details fields (cont’d.)

Field Name Description

Component ID Represents the distinguished name (DN) (refer X.720) ofthe component object against which the particular alarmis raised.

Example:

GWC=GWC-3-UNIT-1;Version=PGC91AFZ;Unit=unit_1;Software=NODEMTC

Alarm Description Provides the description text for the particular alarm.(X.733 Additional text)

Example:

Element Manager communication failure.

Specific Problem Provides the specific problem data (ITU-T X.733) to furtherqualify the probable cause of the particular alarm. (X.733Specific Problem)

Example:

EM not responding, provisioned data loaded from localFlash.

PrerequisitesThis procedure has no prerequisites.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, launch the CS2000 Management Tools GUI.For more information about how to launch CS 2000 managementtools and NPM client applications, see ATM/IP Solution-levelAdministration and Security (NN10402-600).

2 At the CS2000 Management Tools application GUI, on the Faultmenu, click Alarm Manager.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 588: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

588 Communication Server 2000 Management Tools fault management

The Alarm Manager window opens.

3 Select an alarm in the Alarm Manager window.

4 Click Details.

Detailed information about the alarm appears.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 589: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 589

Filtering alarms

Use this procedure to filter the alarms you want to view. You can filter byalarm severity, network element and type of alarm (alarm category).

This procedure provides the steps to filter alarms in the Alarm Manager,but similar filtering capabilities are also available in the Alarm History.

PrerequisitesNone

Procedure steps

Step Action

1 At your workstation, launch the CS2000 Management Tools GUI.For more information about how to launch CS 2000 managementtools and NPM client applications, see ATM/IP Solution-levelAdministration and Security (NN10402-600).

2 At the CS2000 Management Tools GUI, on the Fault menu, clickAlarm Manager.

The Alarm Manager window opens.

3 Use the following table to determine your next step.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 590: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

590 Communication Server 2000 Management Tools fault management

If you want to filter by Then

network element or alarmcategory

step 4

severity step 5

4 Filter by network element or alarm category as follows:

• In the Alarm Manager window, click Advanced Filters.The Advanced Filters window opens.

• Select the network elements you want to exclude from theviewable alarms in the View column, then click Remove.The network elements you have selected move to theExclude column.If you move all Network Elements to the Exclude column,the filters based on Network Element node names will bedeactivated. As a result, the alarm manager will displayalarms from all managed devices and alarms from internalCall server applications (for example, Call Server Data Audit).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 591: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 591

• Under Alarm Category, check the box next to the type ofalarms you want to view for the network elements, then clickApply filters.

5 Filter by severity as follows:

• If checked, uncheck the severity type(s) you want to filter.

• Click Refresh List to refresh the alarm list.The alarms of the unchecked severity type(s) no longerappear in the Alarm Manager window.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 592: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

592 Communication Server 2000 Management Tools fault management

Resetting the filters for alarms

Use this procedure to reset the filters for service-related alarms at theAlarm Manager. This applies only for alarm severity.

PrerequisitesNone

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, launch the CS2000 Management Tools GUI.For more information about how to launch CS 2000 managementtools and NPM client applications, see ATM/IP Solution-levelAdministration and Security (NN10402-600).

2 On the Fault menu, click Alarm Manager.

The Alarm Manager window opens.

3 On the Actions menu, click Reset Filters.

4 Click Refresh List to refresh the alarm list.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 593: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 593

The alarms of the previous default severity types appear in theAlarm Manager window.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 594: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

594 Communication Server 2000 Management Tools fault management

Defining alarms using the NPM CLUI

Define your own alarms to match your specific criteria using the NetworkPatch Manager (NPM) CLUI.

Procedure steps

Step Action

1 Access the NPM CLUI at your workstation.

2 Define an alarm by typing

npm> addalarm <alarm_name> <alarm_enable><alarm_severity> "<alarm_pd> <alarm_criteria>"’<alarm_desc>’

and pressing the Enter key.

Example

addalarm PATCHUNAVAIL Y CRITICAL "PATCH where PATCH.FILEAVAILABLE=’FALSE’" ’No patch file available’

--End--

Variable definitionsVariable Value

<alarm_criteria> The SQL statement that defines the alarm condition.

<alarm_desc> A brief description of the alarm.

<alarm_enable> Identifies whether the alarm will be enabled initially (Y, N).

<alarm_name> The name of the alarm being defined.

<alarm_pd> Identifies whether the alarm is from a patch or device (PATCH,DEVICE).

<alarm_severity> Identifies the alarm severity (NONE, MINOR, MAJOR, orCRITICAL).

Job aidThe NPM is initially configured with the following system-defined alarms:

• PFRSGETPATCH: PFRS GETPATCH task has failed

• DEVICE_ONLHOLD: Devices on hold

• PFRSGENREPORT: PFRS GENREPORT task has failed

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 595: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 595

• DNR_NOT_APP: Do Not Remove (DNR)-type patches not applied

• ACT_NOT_APP: Activatable patches not applied

• OBE_NOT_REMOVED: Obsolete Emergency (OBE)-type patches notremoved

• OBS_NOT_REMOVED: Obsolete (OBS)-type patches not removed

• DEBUG_APP: Debug patches applied

• DEVICE_AUDIFAIL: Devices which have failed or not executed auditssince registration

• ACT_NOT_ACT: Activatable patches not activated

• PATCH_ONLHOLD: Patches on hold

• ENABLED_REMOVED: Patches that are removed but enabled

• REMOVED_PATCHES: Removed patches which are not categoryOBS, OBE, or DBG

• EMG_NOT_APP: Emergency (EMG)-type patches not applied

• GEN_NOT_APP: General (GEN)-type patches at V (verification) status,not applied

• REL_GEN_NOT_APP: General (GEN)-type patches at R (released)status, not applied

• DISABLED_APPLIED: Patches that are applied but disabled

• LTD_NOT_APP: Limited (LTD) patches not applied

You can enable or disable system-defined alarms, and you can view, butnot modify, the definition of system-defined alarms.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 596: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

596 Communication Server 2000 Management Tools fault management

Defining alarms using the NPM GUI

Define your own alarms to match your specific criteria using the NetworkPatch Manager (NPM) command line interface (CLI).

Procedure steps

Step Action

1 Access the NPM GUI at your workstation.

2 On the System menu, click Alarms....

The Alarms window opens.

3 Select an alarm type from the Alarm Type list.

4 Select a severity level from the Severity list.

5 Click the Enabled checkbox to enable the alarm.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 597: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 597

6 In the Alarm Criteria panel, enter the criteria for the alarm usingthe boxes. Go to the next step.

OR

In the Alarm Criteria panel, enter the criteria for the alarm usinga command string.

Go to step 12.

7 Select the field from the Field list.

8 Select the operator from the Operator list.

The table Table 54 "Supported operators" (page 599) lists thesupported operators and their meaning.

9 Select the value from the Value list, or enter the value.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 598: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

598 Communication Server 2000 Management Tools fault management

The data type forValue changes depending on the data type ofthe field. For alphanumeric data, enter the value. For booleandata, select the value.

10 To combine multiple criteria statements, select the AND or theOR options from the And/Or box.

11 Go to step 13.

12 Using a command string, specify the alarm criteria in the textarea.

Parenthesis "()" may be inserted to define precedence formultiple criteria statements.

13 Enter a unique name for the alarm in the Alarm Name box.

14 Optionally enter a description for the alarm in the Alarmdescription box.

15 Click Save to save the alarm.

The new alarm will be displayed in the Alarm List once the NPMServer has saved the alarm.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 599: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 599

--End--

Job aidTable 54Supported operators

Operator Meaning

= Equal

<> Not equal

> Greater than

>= Greater than or equal

< Less than

<= Less than or equal

LIKE Matches string with wildcard (%)

NOT LIKE Does not match string with wildcard (%)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 600: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

600 Communication Server 2000 Management Tools fault management

Performing an audit

Use this procedure to manually perform a trunk or CS 2000 (CS2K) dataaudit.Use this procedure to manually perform a line, trunk, V5.2 interface,or CS 2000 (CS2K) data audit.

The V5.2 audit is only available in the international version of the softwareand not in the North American.

You can set the audit to run automatically at a specific time on a daily ora weekly basis. Refer to procedure "Configuring an audit schedule" inNN10409-500.

You can set the audit to run automatically at a specific time on a daily ora weekly basis. Refer to procedure "Configuring an audit schedule" in thePacket MSC Configuration () document, NN20000-213.

CS 2000 data auditFor a CS2K data audit, the system compares the GWC element manager(EM) database (part of the CS 2000 GWC Manager database) with theCS 2000 XA-Core database and flags any mismatches between the twodatabases. You need to take action in order to correct a data mismatch.The XA-Core is considered to hold the ’master’ database. The systemalso audits IP-VPN (NAT) datafill internal to Succession Element andSub-element Manager (SESM) and its CS2KSS EM equivalent. This auditattempts to automatically correct the datafill on CS2K SS EM when aproblem is detected. Whether the audit succeeded or failed to correct theproblem is indicated in the problem report.

Remedial actions offered are likely to involve deletion of inconsistent data.However, where possible, the option to repair data inconsistencies will begiven.

Line auditFor a line audit, the system compares the ENDPOINTENTRY area in theCS 2000 GWC Manager database with the following tables in the CS 2000XA-Core database:

• DNINV

• LGRPINV

• LNINV

• LNENDPT (if DQOS gateway has been provisioned)

• HUNTMEM (if hunt groups have been provisioned)

• MDNMEM (if MADN groups have been provisioned)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 601: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Aborting audits 601

The system writes the results of the audit into two files, one containing alist of valid data and the other containing a list of problem data. The filesare stored on the CS 2000 Management Tools server.

Trunk auditFor a trunk audit, the system compares the ENDPOINTENTRY area inthe CS 2000 GWC Manager database with the following tables in the CS2000 XA-Core database:

• SERVRINV

• TRKMEM

• LTMAP

• TRKSGRP

The system writes the results of the audit into two files, one containing alist of valid data and the other containing a list of problem data. The filesare stored on the CS 2000 Management Tools server.

V5.2 auditFor a V5.2 data integrity audit, the system compares data in the followingdatabases and flags any mismatches:

• V5.2 interface data stored in the Network View database

• V5.2 endpoint data stored in the CS 2000 GWC Manger database

• V5.2 interface data stored in the table GPPTRNSL in the CS 2000XA-Core database

Remedial actions offered are likely to involve deletion of inconsistent data.However, where possible, the option to repair data inconsistencies will beoffered.

Aborting auditsAll four types of audits can be aborted when running by selecting theView Current Audits option from the Maintenance menu of the CS 2000Management Tools GUI.

Select the audit you want from the Audit View window and click Abort inorder to end an audit currently in progress.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 602: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

602 Communication Server 2000 Management Tools fault management

PrerequisitesIt is recommended to run only one audit at a time, therefore, ensure yourmanual audit does not conflict with another scheduled audit. Runningmultiple audits at the same time may cause some unexpected errors.

You must be assigned to user group "mgcadm" to run an audit. If required,refer to procedure "Setting up local user accounts on an SPFS-basedserver" in NN10402-600.

You must be assigned to user group "mgcadm" to run an audit. If required,refer to procedure "Setting up local user accounts on an SPFS-basedserver" in the Packet MSC Security and Administration () document,NN20000-216.

CAUTIONPossible service disruptionBefore scheduling any data integrity audeit, consider the impactof the audit on the provisioning operations. Use the followingguidelines:

• CS2K Data Integrity Audit blocks the provisioning operations(ADD/DELETE/CHANGE) for

— Gateway Controller (GWC) provisioning

— small line, large line and trunk media gatewaysprovisioning

— third-party media gateway provisioning

— third-party lines provisioning

• Line Data Integrity Audit blocks the directory numbers (DN)provisioning.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 603: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 603

• V5.2 Data Integrity Audit blocks the provisioning operations(ADD/DELETE/CHANGE) for the V5.2 interface.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, launch the CS2000 Management Tools GUI.For more information about how to launch CS 2000 managementtools and NPM client applications, see ATM/IP Solution-levelAdministration and Security (NN10402-600).

2 At the CS2000 Management Tools GUI, on the Maintenancemenu, click Audit System.

The Audit System window opens.

3 In the Audit list, select the type of audit you want to perform.

In addition to the audit types shown in the preceding example,the international version of the software also contains the V5.2audit.

The Line Data Integrity Audit is not supported in PMSC.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 604: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

604 Communication Server 2000 Management Tools fault management

4 Use the following table to determine your next step.

If you selected the Then

CS2K Data Integrity Audit step 5

Line Data Integrity Audit step 9

Trunk Data Integrity Audit step 13

V5.2 Data Integrity Audit step 13

5 Click Run Audit.

6 Select the type of CS2K audit to be run by clicking one or both ofthe check boxes.

The CS2KSS EM Data Integrity Audit option is only availablewhen the CS2KSS EM is provisioned with SESM.

Choosing the first option runs an audit that compares theGWC manager with the CS 2000 XA-Core database and flagsmismatches. Choosing the second option runs an audit thatcompares the IP-VPN (NAT) datafill internal to SESM and itsCS2KSS EM equivalent. Both options can be selected at thesame time as well.

7 Click Run Audit.

During the audit, an Audit Status window appears to indicate theaudit is in progress. The Audit Status window indicates when theaudit is complete.

If the audit does not execute successfully, the Audit Statuswindow indicates that the audit failed and provides the reason.Contact your next level of support to resolve the problem ifrequired.

8 Proceed to step 14.

9 Click Run Audit.

10 Select the elements you want audited and click Add, or click theSelect ALL checkbox to audit the entire call server.

You cannot select GWCs and LGRPs for the same audit.

The ALL audit (by checking the "Select ALL" checkbox) is notcomprised of granular audits (by selecting some GWCs orLGRPs separately). The ALL audit checks line-related datamismatches between XA-Core and the Element Manager

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 605: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 605

database bidirectionally. The granular audit only comparesthe line-related data in the Element Manager database withXA-Core and not vice versa. The ALL audit and granular auditare implemented in different ways, so there may be some auditresult discrepancies between them.

The Line Data Integrity Audit list on the left displays all the smalland large line GWCs and the associated gateways, as well asthe logical line groups of these small line or large line gateways.

11 Click Run Audit.

During the audit, an Audit Status window appears to indicate theaudit is in progress. The Audit Status window indicates when theaudit is complete.

If the audit does not execute successfully, the Audit Statuswindow indicates that the audit failed and provides the reason.Contact your next level of support to resolve the problem ifrequired.

12 Proceed to step 14.

13 Click Run Audit.

14 Click Close to close the Audit Status window once the auditcompletes.

15 Use the following table to determine your next step.

If you performed a Then

CS2K Data Integrity Audit step 16

Line Data Integrity Audit step 20

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 606: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

606 Communication Server 2000 Management Tools fault management

If you performed a Then

Trunk Data Integrity Audit step 23

V5.2 Data Integrity Audit step 26

16 Proceed as follows for a CS2K Data Integrity Audit:

• Ensure that you have selected CS2K Data Integrity Auditfrom the Audit field pull-down menu at the top of the AuditSystem dialog box.

• Select Report <date> from the pull-down menu in the Reportfield at the bottom of the dialog box.The file name has the following format:Report- <date>

where

<date> is the date in yyyy-mm-dd format, forexample, 2005-07-14.

• Click View Report.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 607: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 607

The system displays the selected report. If no problems werediscovered, the report will be empty. Here is an example of areport containing problems:

This example report is the result of the user running boththe CS2K Call Server Data Integrity Audit and CS2KSS EMData Integrity Audit. The problems labelled with the status of’Problem Corrected’ were automatically corrected as part ofthe CS2KSS EM Data Integrity Audit.The CS 2000 Management Tools server retains the mostrecent CS 2000 audit report. When a new audit occurs, theserver deletes the previous report.The system places the audit report in the followingdirectory on the CS 2000 Management Tools server:/opt/nortel/ptm/current/MI2/apps/Audit.The CS 2000 GWC Manager does not provide an option tosave a CS 2000 data audit report to local disk.

17 Review the results of the audit and click on a problem with thestatus of ’Problem Exists’ that needs to be resolved.

If necessary, resize the entire window to completely view theProblem Description field.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 608: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

608 Communication Server 2000 Management Tools fault management

18 Evaluate actions to resolve a problem and take action.

• Click and hold on the Action pull-down menu near the bottomof the screen to assess any possible actions.

• If appropriate, select an action. Read the description of theaction and ensure that you observe any recommended stepsor cautions.

• Click Take Action.

If you see the message "Correction Failed", please contact yournext level of support.

19 Return to step 17 to review another problem.

20 Proceed as follows for a Line Data Integrity Audit:

If you want to view Then

the line valid-data report step 9

the line problem-data report step 22

21 View the line valid-data report as follows:

• Ensure that you have selected Line Data Integrity Auditfrom the Audit field pull-down menu at the top of the AuditSystem dialog box.

• Select ValidLineData from the pull-down menu in the Reportfield at the bottom of the dialog box. If there is more than oneValidLineData report, assess the date and time informationin the report names to guide you in selecting the report youwant to view.The file name has the following format:ValidLineData-<date>-<time>.log

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 609: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 609

where

<date> is the date in yyyy.mm.dd format, forexample, 2003.02.15.

<time> is the time in hh.mm format, for example17.30.

• Click View Report.

The system displays the selected report. Here is an exampleof a "ValidLineData" report.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 610: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

610 Communication Server 2000 Management Tools fault management

The CS 2000 Management Tools server retains the six mostrecent "ValidLineData" reports. When a new line audit occurs,the server deletes the oldest report.The system places valid data audit reports in the followingdirectory on the CS 2000 Management Tools server:/opt/nortel/ptm/current/www/Audit/LineDataIntegrityAudit/.

• If you want to retain one of these reports for a longer time, orif you want to print a report, click the Save as button at thebottom of the screen. Then, save the report under a file nameof your choice.

• To print a report you have saved, open the file using a texteditor and print the file.

• After viewing the valid-data report, click the Exit button at thebottom of the screen.

22 View the line problem-data report as follows:

• Ensure that you have selected Line Data Integrity Auditfrom the Audit field pull-down menu at the top of the AuditSystem dialog box.

• Select ProblemLineData from the pull-down menu in theReport field at the bottom of the dialog box. If there is morethan one ProblemLineData report, assess the date and timeinformation in the report names to guide you in selecting thereport you want to view.The file name has the following format:ProblemLineData-<date>-<time>.log

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 611: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 611

where

<date> is the date in yyyy.mm.dd format, forexample, 2003.02.15.

<time> is the time in hh.mm format, for example17.30.

• Click the View Report button.

The system displays the selected report.If the audit found no problems, the "Problem" report containsa message stating that no problems were found.The "Problem" report produced by a line audit can containmessages in the following formats:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 612: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

612 Communication Server 2000 Management Tools fault management

— DN <DN> for LEN <len> has no associated Endpoint on<GWC ID>.

— Endpoint <endpointname> on gateway <gatewayname>on GWC <GWCname> has no associated DN/LEN onCM.

Following is an example of a "ProblemLineData" report:

The CS 2000 Management Tools server retains the six mostrecent "ProblemLineData" reports. When a new audit occurs,the server deletes the oldest report.The system places problem data audit reports in the followingdirectory on the CS 2000 Management Tools server:/opt/nortel/ptm/current/www/Audit/LineDataIntegrityAudit/.

• If you want to retain one of these reports for a longer time, orprint a report, click the Save as button at the bottom of thescreen. Then, save the report under a new file name.

• To print a report you have saved, open the file using a texteditor and print the file.

• To correct the problems, refer to the printed copy of thereport. You will need to delete and then reprovision the listedlines.

• After viewing the problem-data report, click the Exit button atthe bottom of the viewer screen.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 613: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 613

23 Proceed as follows for a Trunk Data Integrity Audit:

If you want to view Then

the trunk valid-data report step 24

the trunk problem-data report step 25

24 View the trunk valid-data report as follows:

• Ensure that you have selected Trunk Data Integrity Auditfrom the Audit field pull-down menu at the top of the AuditSystem dialog box.

• Select ValidTrunkData from the pull-down menu in theReport field at the bottom of the dialog box. If there is morethan one ValidTrunkData report, assess the date and timeinformation in the report names to guide you in selecting thereport you want to view.The file name has the following format:ValidTrunkData-<date>-<time>.log

where

<date> is the date in yyyy.mm.dd format, forexample, 2003.02.15.

<time> is the time in hh.mm format, for example17.30.

• Click View Report.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 614: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

614 Communication Server 2000 Management Tools fault management

The system displays the selected report. Here is an exampleof a "ValidTrunkData" report.

The CS 2000 Management Tools server retains the six mostrecent "ValidTrunkData" reports. When a new trunk auditoccurs, the server deletes the oldest report.The system places trunk audit reports in the followingdirectory on the CS 2000 Management Tools server:/opt/nortel/ptm/current/www/Audit/TrunkDataIntegrityAudit/.

• If you want to retain one of these reports for a longer time, orif you want to print a report, click Save as at the bottom ofthe screen. Then, save the report under a file name of yourchoice.

• To print a report you have saved, open the file using a texteditor and print the file.

• After viewing the valid-data report, click Exit at the bottom ofthe screen.

25 View the trunk problem-data report as follows:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 615: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 615

• Ensure that you have selected Trunk Data Integrity Auditfrom the Audit field pull-down menu at the top of the Auditsystem dialog box.

• Select ProblemTrunkData from the pull-down menu in theReport field at the bottom of the dialog box. If there is morethan one ProblemTrunkData report, assess the date and timeinformation in the report names to guide you in selecting thereport you want to view.The file name has the following format:ProblemTrunkData-<date>-<time>.log

where

<date> is the date in yyyy.mm.dd format, forexample, 2003.02.15.

<time> is the time in hh.mm format, for example17.30.

• Click the View Report button.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 616: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

616 Communication Server 2000 Management Tools fault management

The system displays the selected report.If the audit found no problems, the "Problem" report containsa message stating that no problems were found.The "Problem" report produced by a trunk audit can containmessages in the following formats:

— Trunk <trunk name> (node number = <NODE>, terminalnumber = <TID>) has no associated Endpoint on GWC<GWC ID>.

— Endpoint <EP NAME> on gateway <GW NAME>(terminal number = <TID>) on GWC <GWC ID> has noassociated trunk member datafilled on the CM.

Here is an example of a "ProblemTrunkData" report:

The CS 2000 Management Tools server retains the sixmost recent "ProblemTrunkData" reports. When a new auditoccurs, the server deletes the oldest report.The system places trunk audit reports in the followingdirectory on the CS 2000 Management Tools server:/opt/nortel/ptm/current/www/Audit/TrunkDataIntegrityAudit/.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 617: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 617

• If you want to retain one of these reports for a longer time,or print a report, click Save as at the bottom of the screen.Then, save the report under a file name of your choice.

• To print a report you have saved, open the file using a texteditor and print the file.

• To correct the problems, refer to the printed copy of thereport. You will need to delete and then reprovision the listedtrunks.

• After viewing the problem-data report, click Exit at the bottomof the viewer screen.

26 View a V5.2 audit report as follows:

• Ensure that you have selected V5.2 Data Integrity Auditfrom the Audit field pull-down menu at the top of the AuditSystem dialog box.

• Select Report <date> from the pull-down menu in the Reportfield at the bottom of the dialog box.The file name has the following format:Report-<date>

where

<date> is the date in yyyy-mm-dd format, forexample, 2003-02-15.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 618: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

618 Communication Server 2000 Management Tools fault management

• Click View Report.

The system displays the selected report. If no problems werediscovered, the report will be empty. Here is an example of areport in which two problems were discovered:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 619: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 619

The CS 2000 Management Tools server retains the mostrecent V5.2 audit report. When a new audit occurs, the serverdeletes the previous report.The system places the audit report in the followingdirectory on the CS 2000 Management Tools server:/opt/nortel/ptm/current/MI2/apps/Audit.The CS 2000 GWC Manager does not provide an option tosave a V5.2 audit report to local disk.

27 Review the results of the audit and click on a problem to resolve.

If necessary, resize the entire window to completely view theProblem Description field.

28 Evaluate actions to resolve a problem and take action.

• Click and hold on the Action pull-down menu near the bottomof the screen to assess any possible actions.

• If appropriate, select an action. Read the description of theaction and ensure that you observe any recommended stepsor cautions.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 620: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

620 Communication Server 2000 Management Tools fault management

• Click Take Action.

If you see the message "Correction Failed", please contactyour next level of support.

29 Return to step 9 to review another problem.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 621: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Netra 240 621

Replacing one or more failed disk drives on anSPFS-based server

Use this procedure to replace one or more failed disk drives on a ServerPlatform Foundation Software (SPFS)-based server (a Netra 240 server).Also use this procedure if a disk drive was pulled out by mistake. Simplyreinserting the disk is not sufficient to recover.

Disk failures will appear as IO errors or SCSI errors from the Solariskernel. These messages will appear in the system log and on the consoleterminal. To indicate a disk failure, log SPFS310 is generated, and analarm light is illuminated on the front panel. After the disk is replaced, thealarm light will go off within a few minutes.

Systems installed with SPFS use disk mirroring. With mirrored hot-swapdisks, a single failed disk can be replaced without interrupting theapplications running on the server. Thus, a single disk can be replacedwhile the system is in service. Follow this link for a view of disks on a“Netra 240 ” (page 621).

The steps to replace a failed drive are to identify the failed drive, replace itphysically, and replace it logically.

ATTENTIONThe steps contained within these procedures change depending on whetherthe SPFS platform has previously been upgraded. If you are working on anupgraded platform, be sure to pay special attention to the associated notesduring the disk replacement procedures.

Netra 240Each Netra 240 is equipped with two hot-swap drives: "c1t0d0", and"c1t1d0".

The following figure identifies the disk drives of the Netra 240.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 622: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

622 Communication Server 2000 Management Tools fault management

Figure 57Netra 240 disk drives

PrerequisitesThis procedure has the following prerequisites:

• You may log into the server as root or emsadmin.

• Physical access to the server.

• A replacement hard drive of the of the same size as the hard drivebeing replaced.

• No other maintenance procedure, such as upgrade, or patchapplication are currently being executed.

Prerequisites for Core and Billing Manager 850In order to perform this procedure, you must have the followingauthorization and access:

• You must be a user in a role group authorized to perform fault-adminactions.

• You must obtain non-restricted shell access.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 623: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 623

For more information about how to log in to the CBM as an authorizeduser, how to request a non-restricted shell access, or how to displayactions a role group is authorized to perform, review the procedures in thefollowing table.

Table 55Related procedures

Procedure Document

Logging in to the CBM Nortel Core and Billing Manager850 Administration and Security( (NN10358-611)) Core andBilling Manager 850 Securityand Administration for CDMA((NN20000-320)) Core andBilling Manager 850 Security andAdministration for GSM/UMTS((NN20000-321))

Requesting non-restricted shell access Nortel Core and Billing Manager850 Administration and Security( (NN10358-611)) Core andBilling Manager 850 Securityand Administration for CDMA((NN20000-320)) Core andBilling Manager 850 Security andAdministration for GSM/UMTS((NN20000-321))

Displaying actions a role group isauthorized to perform

Nortel Core and Billing Manager850 Administration and Security( (NN10358-611)) Core andBilling Manager 850 Securityand Administration for CDMA((NN20000-320)) Core andBilling Manager 850 Security andAdministration for GSM/UMTS((NN20000-321))

Procedure stepsPerform the following steps to complete this procedure.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 624: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

624 Communication Server 2000 Management Tools fault management

ATTENTIONThe CLI responses that you get when performing this procedure aresystem-dependant and may be different from the example responses that areshown.

Replacing failed disks on a Netra 240

Step Action

1 At your workstation, log in to the system that is reporting disk ordisk mirroring errors as the root user using the system physicalIP address.

2 Start the CLI tool by typing "cli" at the shell console.

# cli

and press the Enter key.

3 Enter the number next to the "Configuration" option in the menu.

4 Enter the number next to the "Disk Drive Replacement" option inthe menu.

5 Select the number next to the "disk_drive_replace" option in themenu.

Example response

=== Executing "disk_drive_replace" ===

INFO: Once the disk replacement is complete, it couldtake several hours until full disk redundancy is restored since all file systems on the new drive must resync. Are you sure you want to continue [y,n]?:

6 Enter y if you want to continue with the disk replacementprocedure.

The output provided by the disk replacement utility afterspecifying y will depend on the current status of the system’sdisks and their configuration which, in turn, depends on manyfactors. Therefore, the response will probably look different fromthe following example.

Example response

INFO: Initializing disk data: Success.INFO: Determining this system’s disk configuration variant: Success.INFO: Configuration status for disks: Results:- Status for "disk" on "c1t0d0", on "wcars2yh-unit0",at level "INFO": - Disk size = 73GB.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 625: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 625

- Status for "disk" on "c1t1d0", on "wcars2yh-unit0",at level "INFO": - Disk size = 73GB.- Status for "metadisk" on "c1t1d0", on "wcars2yh-unit0", at level "WARNING":- Missing metadb replicas on slice "c1t1d0s7".- Slice "c1t1d0s5" of submirror "d102" of mirror "d100" is in "Maintenance" state.- Slice "c1t1d0s1" of submirror "d1" of mirror "d2" is in "Maintenance" state.- Slice "c1t1d0s3" of submirror "d7" of mirror "d8" is in "Maintenance" state.- Submirror "d4" is not attached to mirror "d5".- Submirror "d10" is not attached to mirror "d11".- Possible orphaned submirror "d10".

Which disk would you like to replace ("c1t0d0", "c1t1d0", "exit"):

7 If you would like to abort the disk replacement procedure, enterexit otherwise, enter the disk you would like to replace.

Example response

Analyzing system state to determine if disk replacement can proceed:INFO: Disk replacement can proceed.Details:Warnings exist, but they are recoverable.

Do you want to continue with the disk replacement procedure [y,n]?:

8 If the disk analysis detects:

• A situation that prevents the disk replacement fromproceeding, then it will automatically abort this procedure.Call second level support.

• That disk replacement can continue, then confirm that youwould like to continue by entering y and pressing the Enterkey. The disk replacement utility will take a couple minutesto finish this step.

Example response

INFO: Ensuring the mate disk configuration is sane.Results:- Status for "metadisk" on "c1t0d0", on "wcars2yh-unit0", at level "INFO":- Configured the MetaDisk for target disk: c1t0d0

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 626: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

626 Communication Server 2000 Management Tools fault management

INFO: Deconfiguring the target disk "c1t1d0". Results:- Status for "disk" on "c1t1d0", on "wcars2yh-unit0", at level "INFO":- Deconfigured disk "c1t1d0".- Status for "metadisk" on "c1t1d0", on "wcars2yh-unit0", at level "INFO":- Removed metadb replicas from slice "c1t1d0s7", on target disk: "c1t1d0".- Deleted the concatenation or stripe "d10" defined on disk "c1t1d0".- Deleted the concatenation or stripe "d1" defined ondisk "c1t1d0".- Deleted the concatenation or stripe "d4" defined ondisk "c1t1d0".- Deleted the concatenation or stripe "d102" defined on disk "c1t1d0".Deleted the concatenation or stripe "d7" defined ondisk "c1t1d0".

Go to the next step in the procedure "Replacing oneor more failed disk drives on an SSPFS-based server" in the document "NN10408-900" for the necessary steps to physically replace the faulty hard drive withthe new hard drive.

Enter "y" when you have completed physically replacing the faulty hard drive to continue with the configuration of the new hard drive:

Generally, if any status is provided at "ERROR" level, then thereis a disk configuration or health issue that the disk replacementutility cannot resolve, and the disk replacement procedure mustbe halted.

If the "ERROR" is that a file system is currently in "Resync"state then you must wait until the file system has finishedresynchronizing. This may take a couple hours and dependson the size of the file system being resynchronized as well assystem load. You can retry this disk replacement procedure afterthe file systems are resynchronized.

If any status is provided at "WARNING" level, then there is a diskconfiguration or health issue but most likely the disk replacementutility can resolve the issue(s), depending on the combinationof warnings that it has detected. This analysis step will let youknow if the disk replacement can continue.

If there is one warning that states that a file system is in"Maintenance" state, and another warning that states that a filesystem is in "Last Erred" state, then the disk with the file systemin "Maintenance" state must be replaced first and its file systems

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 627: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 627

resynchronized before replacing the hard drive with the filesystem in "Last Erred" state.

If status is only provided at "INFO" level or if there is no statusinformation, then both disk configuration and health are in goodcondition.

9 Physically replace the failed disk.

Messages similar to the following might be displayed at theconsole or in the system logs while replacing the hard drive:

Message 1.Apr 21 11:39:47 wcars2yh unix: MAJOR ALARM is setSC Alert: MAJOR ALARM is set

Message 2.SC Alert: DISK @ HDD0 has been removed.Apr 27 16:24:44 wcars2yh unix: DISK @ HDD0 has beeninserted.SC Alert: DISK @ HDD0 has been inserted.

Message 3.Apr 21 11:39:47 wcars2yh root:/opt/resmon/Hardware/disk_check.ksh: Disk or disk mirroringerrors: See system alarms for details

Procedure for Netra 240 is as follows:

1. Ensure that you are properly grounded before removing thedrive.

2. Open the front panel of the Netra 240 .

3. Unlatch the hard drive with the blue light lit to its left.

4. Use the latch as a lever to start removing the hard drive.

5. Pull the drive the rest of the way out of the SCSI bay.

6. Ensure the latch on the replacement hard drive is open.

7. Insert the replacement hard drive completely into the vacantSCSI bay as far as it will go.

8. Close the latch on the replacement hard drive.

9. Close the front panel of the Netra 240 .

10 Complete the configuration of the new drive by entering y at theprompt. It will take the disk replacement utility a couple minutesto complete this step.

INFO: Configuring new hard drive.Results:- Status for "disk" on "c1t1d0", on "wcars2yh-unit0", at level "INFO":Configured target disk "c1t1d0".

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 628: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

628 Communication Server 2000 Management Tools fault management

- Status for "metadisk" on "c1t1d0", on "wcars2yh-unit0", at level "INFO":- Created metadb replicas on target disk: c1t1d0- Configured concatenation or stripe "d102" definedon target disk "c1t1d0".- Configured mirror "d100".- Configured concatenation or stripe "d1" defined ontarget disk "c1t1d0".- Configured mirror "d2".- Configured concatenation or stripe "d7" defined ontarget disk "c1t1d0".- Configured mirror "d8".- Configured concatenation or stripe "d4" defined ontarget disk "c1t1d0".- Configured mirror "d5".- Configured concatenation or stripe "d10" definedon target disk "c1t1d0".- Configured mirror "d11".- Configured the MetaDisk for target disk: c1t1d0

INFO: The disk replacement procedure has completed successfully.

=== "disk_drive_replace" completed successfully

11 Exit each menu level of the command line interface to eventuallyreturn to the command prompt, by typing

select - x

and pressing the Enter key.

It will take several hours for the file systems on the new harddrive to synchronize. The "SPFS310" alarm will clear in about aminute.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 629: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 629

Replacing a DVD drive on an SPFS-based server

Use this procedure to replace a DVD drive on a Netra 240 server. Thisprocedure applies to simplex and high-availability (HA) systems. An HAsystem refers to a Sun Netra 240 server pair.

ATTENTIONThe DVD drive is not hot-swappable. The server must be powered down.Therefore, ensure the server can be powered down before you proceed withthe procedure.

The following figure shows the location of the DVD drive on the Netra 240 .

Figure 58Sun Netra 240

Use one of the following methods according to your office configuration:

• “Simplex configuration (one server)” (page 630)

• “High-availability configuration (two servers)” (page 630)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 630: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

630 Communication Server 2000 Management Tools fault management

PrerequisitesNone.

You may log into the server as root or emsadmin.

Procedure stepsSimplex configuration (one server)

Step Action

1 At your workstation, power down the server. Refer to procedure"Shutting down an SPFS-based server”, if required.

2 Physically replace the DVD drive using the documentation forthe Netra 240.

• Go to www.sun.com.

• In the Search field, enter the following. Include the quotationmarks to refine your search."Netra 240 Server Service Manual".

• Click the Submit search button.A list of links related to the Netra Server Service Manual isdisplayed.

• Find and select Netra 240 Server Service Manual,817-2699-13.pdf, from the list of results.

• Refer to the Netra Server Service Manual for procedures onhow to remove and install a DVD drive.

ATTENTIONDo NOT issue the software commands given in the Sun procedureto unconfigure/configure the drive during removal or insertion. Thehard drives are hot-swappable in the SPFS file system, so the Suncommands are not necessary. Follow only the physical steps detailedin the Sun procedure to remove/insert the hard drives.

3 Once the new DVD drive is in place, restore power to the serverby turning on the power at the circuit breaker panel of the frame.The server recovers on its own once power is restored.

--End--

High-availability configuration (two servers)

Step Action

1 At your workstation, use the following table to determine yourfirst step.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 631: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 631

If you are replacing theDVD drive on the Then

active server step 2

inactive server step 3

2 Initiate a manual failover. Refer to procedure “Initiating a manualfailover on a Sun Netra 240 server pair”, if required.

3 Once the active server acquires the status of standby (inactive),power down the server. Refer to procedure “Shutting down anSPFS-based server”, if required.

4 Physically replace the DVD drive using the Sun documentationfor the Netra 240.

• Go to www.sun.com.

• In the Search field, enter the following. Include the quotationmarks to refine your search."Netra 240 Server Service Manual"

• Click the Submit search button.A list of links related to the Netra Server Service Manual isdisplayed.

• Find and select Netra 240 Server Service Manual,817-2699-13.pdf, from the list of results.

• Refer to the Netra Server Service Manual for procedures onhow to remove and install a DVD drive.

ATTENTIONDo NOT issue the software commands given in the Sun procedureto unconfigure/configure the drive during removal/insertion. Thehard drives are hot-swappable in the SPFS file system, so the Suncommands are not necessary. Follow only the physical steps detailedin the Sun procedure to remove/insert the hard drives.

5 Once the new DVD drive is in place, restore power to the serverby turning on the power at the circuit breaker panel of the frame.The server recovers on its own once power is restored.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 632: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

632 Communication Server 2000 Management Tools fault management

Replacing a failed SPFS-based server

Use the following procedure when an SPFS-based server has failed andyou need to replace it. This procedure provides the instructions for aone-server configuration or a two server configuration.

The server can be hosting one or more of the following components:

• CS 2000 Management Tools

• Integrated Element Management System (IEMS)

• Audio Provisioning Server (APS)

• Media Gateway 9000 Manager

• CS 2000 SAM21 Manager

• Network Patch Manager (NPM)

• Core Billing Manager (CBM)

PrerequisitesYou must have a replacement server.

ATTENTIONEnsure that no provisioning activities are in progress, or scheduled to take placeduring this procedure.

You must have the root user ID and password to log into the system.

Prerequisites for Nortel Core and Billing Manager 850In order to perform this procedure, you must have the followingauthorization and access:

• You must be a user in a role group authorized to perform fault-adminactions.

• You must obtain non-restricted shell access.

For more information about how to log in to the CBM as an authorizeduser, how to request a non-restricted shell access, or how to displayactions a role group is authorized to perform, review the procedures in thefollowing table.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 633: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 633

Table 56Related procedures

Procedure Document

Logging in to the CBM Nortel Core and Billing Manager850 Administration and Security( (NN10358-611)) Nortel Coreand Billing Manager 850 Securityand Administration for CDMA ()(NN20000-320) Nortel Core andBilling Manager 850 Security andAdministration for GSM/UMTS ()(NN20000-321)

Requesting non-restricted shell access Nortel Core and Billing Manager850 Administration and Security( (NN10358-611)) Nortel Coreand Billing Manager 850 Securityand Administration for CDMA ()(NN20000-320) Nortel Core andBilling Manager 850 Security andAdministration for GSM/UMTS ()(NN20000-321)

Displaying actions a role group isauthorized to perform

Nortel Core and Billing Manager850 Administration and Security( (NN10358-611)) Nortel Coreand Billing Manager 850 Securityand Administration for CDMA ()(NN20000-320) Nortel Core andBilling Manager 850 Security andAdministration for GSM/UMTS ()(NN20000-321)

Procedure stepsPerform the steps under one of the headings that follow to complete thisprocedure.

• Replacing an SPFS-based server (one-server configuration)

• Replacing one server in an HA configuration

• Replacing both servers in an HA configuration

Replacing an SPFS-based server (one-server configuration)

Step Action

1 At the COAM frame, disconnect and remove the failed server.

2 Connect and power up the replacement server.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 634: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

634 Communication Server 2000 Management Tools fault management

3 Restore the file systems and oracle data from backup media.If required, refer to procedure Routing log streams from anSPFS-based server in Nortel ATM/IP Solution-level FaultManagement (NN10408-900).

Restoring the oracle data does not apply to the CBM as it doesnot use an oracle database.

--End--

Replacing one server in an HA configuration

Step Action

1 At the COAM frame, disconnect and remove the failed server.

2 Connect and power up the replacement server.

3 Clone the image of the active server onto the server you justreplaced. If required, refer to procedure Routing log streamsfrom an SPFS-based server in Nortel ATM/IP Solution-level FaultManagement (NN10408-900).

--End--

Replacing both servers in an HA configuration

Step Action

1 At the COAM frame, disconnect and remove one failed server.

2 Connect and power up the replacement server.

3 Restore the file systems and oracle data from backup mediaon the server you just replaced. For more information on howto route log streams from an SPFS-based server, see NortelATM/IP Solution-level Fault Management (NN10408-900).

Restoring the oracle data does not apply to the CBM as it doesnot use an oracle database.

4 Disconnect and remove the other failed server.

5 Connect and power up the replacement server.

6 Clone the image of the active server on to the server you justreplaced. For more information on how to route log streams froman SPFS-based server, see Nortel ATM/IP Solution-level FaultManagement (NN10408-900).

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 635: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Prerequisites 635

Replacing failed Ethernet interfaces

PurposeUse the following procedures to replace a failed Ethernet interface.

ApplicationThe Ethernet interface is not a field replaceable unit. The server must beput out of service and powered down before hardware can be removedfrom the shelf.

Instructions for entering commands in the following procedure do not showthe prompting symbol, such as #, >, or $, displayed by the system througha GUI or on a command line.

PrerequisitesYou must have the root user ID and password to log into the server.

You must have a replacement N240 server with the same PEC code asthat which is being replaced.

Prerequisites for Nortel Core and Billing Manager 850In order to perform this procedure, you must have the followingauthorization and access:

• You must be a user in a role group authorized to perform fault-adminactions.

• You must obtain non-restricted shell access.

ATTENTIONYou must have root user access privileges to perform a swact in step 3 if thefailed ethernet interface is on the ACTIVE node of the CBM 850 clustered pair.

For more information about how to log in to the CBM as an authorizeduser, how to request a non-restricted shell access, or how to displayactions a role group is authorized to perform, review the procedures in thefollowing table.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 636: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

636 Communication Server 2000 Management Tools fault management

Table 57Related procedures

Procedure Document

Logging in to the CBM Nortel CS 2000 Core Manager Security andAdministration () (NN1070-611) Nortel Core andBilling Manager 850 Security and Administrationfor CDMA () (NN20000-320) Nortel Core andBilling Manager 850 Security and Administrationfor GSM/UMTS () (NN20000-321)

Requesting non-restricted shell access Nortel CS 2000 Core Manager Security andAdministration () (NN1070-611) Nortel Core andBilling Manager 850 Security and Administrationfor CDMA () (NN20000-320) Nortel Core andBilling Manager 850 Security and Administrationfor GSM/UMTS () (NN20000-321)

Displaying actions a role group is authorized toperform

Nortel CS 2000 Core Manager Security andAdministration () (NN1070-611) Nortel Core andBilling Manager 850 Security and Administrationfor CDMA () (NN20000-320) Nortel Core andBilling Manager 850 Security and Administrationfor GSM/UMTS () (NN20000-321)

ActionReplacing failed Ethernet interfaces

Step Action

1 At the CBM, log in to the failed unit using the console port. Formore information about logging in to an SPFS-based server,see Nortel ATM/IP Solution-level Security and Administration(NN10402-600).

2 Determine the cluster activity of the failed unit by typing:

ubmstat

3 If the activity indicates ClusterIndicatorACT, make this unit theinactive unit by typing:

swact

4 Shut down the failed unit by typing:

init 0

5 Perform the procedure “Replacing a failed SPFS-based server”(page 632).

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 637: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 637

Re-establishing the connection between the NPMGUI and the server

Use this procedure to re-establish the connection between the NetworkPatch Manager (NPM) GUI and the Sun server where the Network PatchManager (NPM) software resides. The NPM GUI loses server connectivitywhen the server goes out of service. Once the server returns to serviceyou need to reconnect.

PrerequisitesThe server where the NPM software resides must be in service.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At the NPM GUI, System menu, click Re-Connect to Server.

When the connection is successful, a response similar to thefollowing one appears.

When the connection fails, a response similar to the followingone appears.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 638: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

638 Communication Server 2000 Management Tools fault management

When the connection fails, try to reconnect at a later time. If theconnection failure persists, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 639: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 639

Restoring SSH communication between the CS 2000Management Tools server and the core manager

Use this procedure to re-establish SSH communication between theCS 2000 Management Tools server and the core manager if either oneundergoes a fresh install after the CS 2000 SAM21 Manager has beenmigrated from the core manager to the CS 2000 Managemen Tools server.

PrerequisitesNone

Procedure stepsPerform the steps in one of the following procedures depending on whichplatform underwent a fresh install:

• “Re-establishing SSH communication after a core manager freshinstall” (page 639)

• “Re-establishing SSH communication after an SPFS fresh install”(page 641)

Re-establishing SSH communication after a core manager fresh install

Step Action

1 , log in to the core manager.

2 Access the SWIM level of the maintenance interface by typing

#

and pressing the Enter key.

3 Access the DetailsDetails level by typing

> details

and pressing the Enter key.

4 Ensure the following filesets are applied.

• OpenSSH

• Bootp Loading Service

Use the up/down commands to scroll through the list ofapplications if required.

5 At the CS 2000 Management Tools server (SPFS), log in to theCS 2000 Management Tools server by typing

> telnet <server>

and pressing the Enter key.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 640: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

640 Communication Server 2000 Management Tools fault management

where

server is the IP address or host name of the CS2000 Management Tools server

6 When prompted, enter your user ID and password.

7 Change to the root user by typing

$ su - root

and pressing the Enter key.

8 When prompted, enter the root password.

9 Change to the CS 2000 SAM21 Manager user by typing

# su - sam21em

and pressing the Enter key.

10 Remove the existing entry for the SDMSDM in file"/export/home/sam21em/.ssh/known_hosts".

11 Secure-append the public key to the authorized key’s list thatresides on the core manager by typing

$

and pressing the Enter key.

Example response:

The authenticity of host Ô7.142.128.16(47.142.128.16)’ can’t be established. RSA key fingerprint is21:cb:c6:7f:df:05:f8:4a:2f:23:e9:09:c8:37 bc:le sam21em@znc0s0j6Are you sure you want to continue connecting (yes/no)?

12 When prompted, confirm you want to continue connecting bytyping

# yes

and pressing the Enter key.

Example response:

Warning: Permanently added Ô7.142.128.16’ (RSA) to the list of known [email protected]’s password:

13 When prompted, enter the password for the SDMSDM root user.

14 Ensure the "authorized_keys2" file has the proper ownership onthe SDMSDM by typing

$

and pressing the Enter key.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 641: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 641

Example response:

[email protected]’s password:

15 When prompted, enter the password for the SDMSDM root user.

16 Propagate the SDMSDM’s bootptab file with the entries fromSPFS by typing

$

and pressing the Enter key.

When prompted, enter the password for the SDMSDM root user.

$

and pressing the Enter key.

--End--

Re-establishing SSH communication after an SPFS fresh install

Step Action

1 , log in to the core manager.

2 Access the SWIM level of the maintenance interface by typing

#

and pressing the Enter key.

3 Access the DetailsDetails level by typing

> detailsdetails

and pressing the Enter key.

4 Ensure the OpenSSH fileset is applied.

Use the up/down commands to scroll through the list ofapplications if required.

5 Perform procedure "Setting up the BootP file on SPFS" in theVoice over IP Network Upgrade document, NN10440-450.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 642: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

642 Communication Server 2000 Management Tools fault management

Enabling X11-forwarding

Enable the status of the Secure Shell (ssh) X11-forwarding functionalitythrough the Command Line Interface (CLI) tool. This procedure allows theend user to launch the X-applications in the remote machine and accessthe display on the local machine.

By default, X11-Forwarding is disabled and must be enabled by the user.

Prerequisites• The system must have a Netra 240 box.

• You must have root user privileges.

• The NTSSH package must be included as a part of the SPFSinstallation on the machine.

Procedure steps

Step Action

1 At a secure workstation

Log on to the server using ssh (secure) by typing:

> ssh -l root <server>

and pressing the Enter key.

2 If this is a first-time ssh log on, the system prompts you toconfirm the connection. Enter yes at the prompt.

3 When prompted, enter the root password.

In a two-server configuration, ensure you are on the activeserver by typing ubmstat. If ClusterIndicatorSTBY is displayed inthe response, which indicates you are on the inactive server, logout of that server and log on to the other server through Telnetusing the physical IP address of the other unit. The responsemust display ClusterIndicatorACT, which indicates you are onthe active server.

4 Access the command line interface. Select the following options:

Configuration

Security services Configuration

X11-Forwarding

Enable_X11Forwarding

5 Refer to the following example.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 643: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 643

# cliCommand Line Interface

1 - View2 - Configuration3 - OtherX - exit

select - 2

Example responseConfiguration

1 - NTP Configuration2 - Apache Proxy Configuration3 - OAMP Application Configuration4 - CORBA Configuration5 - IP Configuration6 - DNS Configuration7 - Syslog Configuration8 - Remote Backup Configuration9 - Database Configuration

10 - NFS Configuration11 - Bootp Configuration12 - Restricted Shell Configuration13 - Security Services Configuration14 - Disk Drive Upgrade15 - Disk Drive Replacement16 - Login Session17 - Location Configuration18 - Cluster Configuration19 - Succession Element Configuration20 - snmp_poller (SNMP Poller Configuration)21 - backup_config (Backup Configuration)

X - exitselect -13

Example responseSecurity Services Configuration1 - Socks Configuration2 - Security Server Location Configuration3 - PAM Configuration4 - Common Inet Services (ftp, tftp, Telnet, snmp

and nfs)5 - Tools Inet Services (time, daytime)6 - Other Inet Services7 - proftpd User Configuration8 - proftpd Authentication Configuration9 - PKManager Certificate Installation

10 - WebPKProxy/PKClient Configuration11 - Password Monitor Configuration12 - Certificate Monitor Configuration13 - Register Certificate for Monitoring14 - X11-Forwarding

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 644: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

644 Communication Server 2000 Management Tools fault management

15 - query_registry (Query Network Services Package Registration)16 - change password encryption key length (change password encryption key l...)

X - exitselect -14

Example responseX11-Forwarding1 - Enable_X11Forwarding2 – Disable_X11Forwarding3 – Display_X11ForwardingX - exit

Select -1

Example response

=== Executing "Enable_X11Forwarding"Enabling X11Forwarding.........X11Forwarding enabled successfully.Please log out of the machine and establish a new SSHconnection to get the changes effected=== "Enable_X11Forwarding" execution completed

6 After you enable X11-Forwarding, log out.

7 Log on and establish a new ssh connection to putX11-Forwarding into effect.

8 Check the log entry in the /var/log/sspfslog log file to confirm thatX11-Frowarding is enabled.

Example

Jan 14 23:02:13 wcars35d x11forwarding.ksh[3120]: X11-Forwarding enabled successfully

--End--

Variable definitionsVariable Value

<server> The IP address or host name of theSPFS-based server, or the physical IPaddress of the active server in a two-serverconfiguration.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 645: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 645

Disabling X11-forwarding

Disable X11-Forwarding on an SPFS system, blocking X-traffic via SSHonly. The new SSH connection blocks the X traffic via the SSH channel.

The system prompts you if you attempt to disable X11-Forwarding while itis disabled.

Prerequisites• The system must have a Netra 240 box.

• You must have root user privileges.

• The NTSSH package must be included as a part of the SPFSinstallation on the machine.

Procedure steps

Step Action

1 At a secure workstation, using CLI

Access the command line interface. Select the following options:

Configuration

Security services Configuration

X11-Forwarding

Disable_X11Forwarding

2 Refer to the following example.

wcar35d% cliCommand Line Interface

1 - View2 - Configuration3 - OtherX - exit

select - 2

Example responseConfiguration

1 - NTP Configuration2 - Apache Proxy Configuration3 - OAMP Application Configuration4 - CORBA Configuration5 - IP Configuration6 - DNS Configuration7 - Syslog Configuration

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 646: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

646 Communication Server 2000 Management Tools fault management

8 - Remote Backup Configuration9 - Database Configuration

10 - NFS Configuration11 - Bootp Configuration12 - Restricted Shell Configuration13 - Security Services Configuration14 - Disk Drive Upgrade15 - Disk Drive Replacement16 - Login Session17 - Location Configuration18 - Cluster Configuration19 - Succession Element Configuration20 - snmp_poller (SNMP Poller Configuration)21 - backup_config (Backup Configuration)

X - exitselect -13

Example responseSecurity Services Configuration1 - Socks Configuration2 - Security Server Location Configuration3 - PAM Configuration4 - Common Inet Services (ftp, tftp, Telnet, snmp

and nfs)5 - Tools Inet Services (time, daytime)6 - Other Inet Services7 - proftpd User Configuration8 - proftpd Authentication Configuration9 - PKManager Certificate Installation

10 - WebPKProxy/PKClient Configuration11 - Password Monitor Configuration12 - Certificate Monitor Configuration13 - Register Certificate for Monitoring14 - X11-Forwarding15 - query_registry (Query Network Services Package Registration)16 - change password encryption key length (change password encryption key l...)

X - exitselect -14

Example responseX11-Forwarding1 - Enable_X11Forwarding2 – Disable_X11Forwarding3 – Display_X11ForwardingX - exit

Select -2

Example response=== Executing "Disable_X11Forwarding"

Disabling X11Forwarding ......

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 647: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 647

X11Forwarding is disabled successfullyPlease log out of the machine and establish a new SS

H connection to get the changes effected

=== "Disable_X11Forwarding" execution completed

ATTENTIONAfter disabling X11-Forwarding, for the change to go in to effect,you must log out of the machine and then establish a new SSHconnection to the machine.

3 You may check the log entry in the /var/log/sspfslog log file toconfirm.

Example

Jan 14 23:02:13 wcars98d x11forwarding.ksh[3120]: X11-Forwarding disabled successfully

4 After you disable X11-Forwarding, log out.

5 Log on and establish a new ssh connection to effect the changeto X11-Forwarding.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 648: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

648 Communication Server 2000 Management Tools fault management

Displaying X11-forwarding status

Display the status of the Secure Shell (SSH) X11-Forwarding through theCommand Line Interface (CLI) tool. This procedure allows the end user tolaunch the X-applications in the remote machine and view the display onthe local machine.

Prerequisites• The system must have a Netra 240 box.

• You must have root user privileges.

• The NTSSH package must be included as a part of the SPFSinstallation on the machine.

Procedure steps

Step Action

1 At a secure workstation, using CLI

Access the command line interface. Select the following options:

Configuration

Security services Configuration

X11-Forwarding

Display_X11Forwarding

2 Refer to the following example.

(Identical to the previous examples until you get to theX11-Forwarding submenu. Only that section is representedhere.)

X11-Forwarding1 - Enable_X11Forwarding2 – Disable_X11Forwarding3 – Display_X11ForwardingX - exit

Select -3

Example response for Display if enabled=== Executing "Display_X11Forwarding"

X11Forwarding is currently enabled

=== "Display_X11Forwarding" execution completed

3 Check the log entry in the /var/log/sspfslog log file to confirm theaction.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 649: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 649

Examples

Jan 14 22:56:25 wcars98d x11forwarding.ksh[20577]: X11-forwarding is currently enabled

or

Jan 14 22:59:39 wcars98d x11forwarding.ksh[27328]: X11-forwarding is currently disabled

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 650: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

650 Communication Server 2000 Management Tools fault management

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 651: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

651.

CORBA configuration issue correctionbetween the Communication Server2000 server and the MG 9000 Managerserver

Use this procedure when you are unable to provision MG 9000 linesor asynchronous digital subscriber lines (ADSLs) through OSSGate, orprovision virtual media gateways (VMGs) from the MG 9000 Managergraphical user interface (GUI).

This procedure provides the steps to configure the naming servicebetween the Communication Server 2000 Management Tools server andthe server where the MG 9000 Manager resides.

CORBA configuration issue correction between the Communication Server2000 server and the MG 9000 Manager server navigation

• “Correcting a CORBA configuration issue between the CS 2000Management Tools server and the MG 9000 Manager server” (page652)

• “Recovering from having two inactive servers in a cluster” (page 657)

• “Performing a restore of APS audio files from DVD” (page 660)

• “Performing an APS-only Database restore from a CD” (page 661)

• “Verifying the state of a cluster” (page 663)

• “Initiating a manual failover on a Sun Netra server pair” (page 667)

• “Querying the state of the SPFS platform” (page 669)

• “Rebooting an SPFS-based server” (page 671)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 652: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

652 CORBA configuration issue correction between the Communication Server 2000 server andthe MG 9000 Manager server

Correcting a CORBA configuration issue betweenthe CS 2000 Management Tools server and the MG9000 Manager server

Use this procedure when you are unable to provision MG 9000 linesor asynchronous digital subscriber lines (ADSLs) through OSSGate, orprovision virtual media gateways (VMGs) from the MG 9000 Managergraphical user interface (GUI).

This procedure provides the steps to configure the naming servicebetween the CS 2000 Management Tools server and the server where theMG 9000 Manager resides.

You must configure the naming service on both servers using the stepsunder

• “Configuring the naming service on the CS 2000 Management Toolsserver” (page 652)

• “Configuring the naming service on the server where the MG 9000Manager resides” (page 654)

Once you have configured the naming service on both servers, youcan verify the naming service is set up correctly using the steps under“Verifying the naming service is configured correctly” (page 656).

PrerequisitesNone

Procedure stepsPerform the following steps to complete this procedure.

Configuring the naming service on the CS 2000 Management Tools server

Step Action

1 At your workstation, telnet to the CS 2000 Management Toolsserver by typing

> telnet <server>

and pressing the Enter key.

where

server is the IP address or host name of the CS2000 Management Tools server

2 When prompted, enter your user ID and password.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 653: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 653

3 Change to the root user by typing

$ su - root

and pressing the Enter key.

4 When prompted, enter the root password.

5 Access the command line interface by typing

# cli

and pressing the Enter key.

6 Enter the number that corresponds to the "Configuration" optionin the menu.

7 Enter the number that corresponds to the "CORBAConfiguration" option in the menu.

8 Enter the number that corresponds to the "nsmirror" option in themenu.

9 Enter the number that corresponds to the "Add CORBA NamingContext Mirror" option in the menu.

10 When prompted, enter the Application Name for the SuccessionElement and Sub-Network Manager by typing

sesm

and pressing the Enter key.

11 When prompted, enter the Host Name or IP address of theserver where the MG 9000 resides.

12 When prompted, enter the Context Path by typing

NameService/Subnet_ <release>

and pressing the Enter key.

where

release is the current release of the software,for example 07 for (I)SN07

The command is case sensitive.

13 When prompted, enter the Context Name by typing

Subnet_ <release>

and pressing the Enter key.

where

release is the current release of the software,for example 07 for (I)SN07

The command is case sensitive.

14 When prompted, press the Enter key to accept the default value(2001) for the Local Port.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 654: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

654 CORBA configuration issue correction between the Communication Server 2000 server andthe MG 9000 Manager server

15 When prompted, press the Enter key to accept the default value(2001) for the Remote Port.

Example response:

You have selected to bind the remote naming context as follow:

Application Name :SESMHost Name :47.142.89.70Context Path :NameService/Subnet_07Context Name :Subnet_07Local Port :2001Remote Port :2001

Continue with configuration? (default:Y[Y/N/Q])

16 When prompted, confirm the configuration by typing.

y

and pressing the Enter key.

17 Exit each menu level of the command line interface to eventuallyexit the command line interface, by typing

select - x

and pressing the Enter key.

18 You have completed this procedure. Proceed to “Configuringthe naming service on the server where the MG 9000 Managerresides” (page 654).

--End--

Configuring the naming service on the server where the MG 9000 Managerresides

Step Action

1 At your workstation, telnet to the server where the MG 9000Manager resides by typing

> telnet <server>

and pressing the Enter key.

where

server is the IP address or host name of the serverwhere the MG 9000 Manager resides

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su - root

and pressing the Enter key.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 655: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 655

4 When prompted, enter the root password.

5 Access the command line interface by typing

# cli

and pressing the Enter key.

6 Enter the number that corresponds to the "Configuration" optionin the menu.

7 Enter the number that corresponds to the "CORBAConfiguration" option in the menu.

8 Enter the number that corresponds to the "nsmirror" option in themenu.

9 Enter the number that corresponds to the "Add CORBA NamingContext Mirror" option in the menu.

10 When prompted, enter the Application Name for the MG 9000Manager by typing

mgems

and pressing the Enter key.

11 When prompted, enter the Host Name or IP address of the CS2000 Management Tools server where the SESM resides.

12 When prompted, enter the Context Path by typing

NameService/Nortel

and pressing the Enter key.

The command is case sensitive.

13 When prompted, enter the Context Name by typing

Nortel

and pressing the Enter key.

The command is case sensitive.

14 When prompted, press the Enter key to accept the default value(2001) for the Local Port.

15 When prompted, press the Enter key to accept the default value(2001) for the Remote Port.

Example response:

You have selected to bind the remote naming context as follow:

Application Name :MGEMSHost Name :47.142.85.56Context Path :NameService/NortelContext Name :NortelLocal Port :2001Remote Port :2001

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 656: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

656 CORBA configuration issue correction between the Communication Server 2000 server andthe MG 9000 Manager server

Continue with configuration? (default:Y[Y/N/Q])

16 When prompted, confirm the configuration by typing.

y

and pressing the Enter key.

17 Exit each menu level of the command line interface to eventuallyexit the command line interface, by typing

select - x

and pressing the Enter key.

18 You have completed this procedure. Proceed to “Verifying thenaming service is configured correctly” (page 656).

--End--

Verifying the naming service is configured correctly

Step Action

1 At your workstation, close all GUIs and OSSGate telnet sessionsif any.

2 Restart the SESM server application. For more information, see“Starting the SESM server application” (page 347).

3 Restart the MG 9000 Manager server application. For moreinformation, see Nortel MG 9000 Administration and Security(NN10162-611).

4 Start OSSgate client and provision one or more lines or ADSLs.Refer to the OSSgate User Guide, if required.

5 Access the MG 9000 Manager and provision a VMG. For moreinformation, see Nortel MG 9000 Administration and Security(NN10162-611), if required.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 657: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 657

Recovering from having two inactive servers in acluster

Use this procedure to make one server active when both servers in thecluster are inactive.

When there is no active server, log SPFS330 is generated.

For more information on logs, refer to the Carrier Voice over IP NetworksFault Management Logs Reference document (NN10275-909) .

PrerequisitesYou need the root user ID and password.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, determine which server has the most recentapplication data as follows:

• Telnet to one server of the cluster by typing> telnet <IP address>and pressing the Enter key.

where

IP address is the physical IP address of theserver

• When prompted, enter your user ID and password.

• Access the "/data" directory by typing> cd /dataand pressing the Enter key.

• List the files in the "/data" directory with their detailedinformation by typing> ls -land pressing the Enter key.

• Note the timestamp on each of the files.

• Exit the "/data" directory by typing> cdand pressing the Enter key.

• Telnet to the other server of the cluster by typing> telnet <IP address>and pressing the Enter key.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 658: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

658 CORBA configuration issue correction between the Communication Server 2000 server andthe MG 9000 Manager server

where

IP address is the physical IP address of theserver

• When prompted, enter your user ID and password.

• Access the "/data" directory by typing> cd /dataand pressing the Enter key.

• List the files in the "/data" directory with their detailedinformation by typing> ls -land pressing the Enter key.

• Note the timestamp on each of the files.

• Exit the "/data" directory by typing>cdand pressing the Enter key.

2 On the server that has the most recent application data, changeto the root user by typing

$ su - root

and pressing the Enter key.

3 When prompted, enter the root password.

4 Make this server the Active server by typing

# ActivateNode

and pressing the Enter key.

5 When prompted, confirm you want to make this server the Activeserver by typing

# yes

and pressing the Enter key.

6 Verify the status of file systems on this server by typing

# udstat

and pressing the Enter key.

If the file systems are Then

ACTIVE normal UP clean step 7

not ACTIVE normal UP clean contact your next level of support

7 Verify the cluster indicator by typing

# ubmstat

and pressing the Enter key.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 659: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 659

If the system response is Then

"ClusterIndicatorACT" step 8

not "ClusterIndicatorACT" contact your next level of support

8 Telnet to the other server in the cluster by typing

> telnet <IP address>

and pressing the Enter key.

where

IP address is the physical IP address of the server

9 When prompted, enter your user ID and password.

10 Change to the root user by typing

$ su - root

and pressing the Enter key.

11 When prompted, enter the root password.

12 Verify the status of file systems on this server by typing

# udstat

and pressing the Enter key.

If the file systems are Then

STANDBY normal UP clean step 13

not STANBY normal UP clean contact your next level of support

13 Verify the cluster indicator by typing

# ubmstat

and pressing the Enter key.

If the system response is Then

"ClusterIndicatorSTBY" the cluster is in a good stateand you have completed thisprocedure

not "ClusterIndicatorSTBY" contact your next level of support

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 660: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

660 CORBA configuration issue correction between the Communication Server 2000 server andthe MG 9000 Manager server

Performing a restore of APS audio files from DVD

Use this procedure to perform a restore of APS audio files from a backupDVD on a Server Platform Foundation Software (SPFS)-based server (SunNetra 240 ) running the SN06.2 or greater release of the SPFS.

ATTENTIONThis procedure only applies to a Sun Netra 240 type server with a DVD drive.

PrerequisitesYou need the DVD on which you backed up the APS audio files.

Step Action

1 At the server, insert the DVD on which you backed up the APSaudio files into the drive on the server where the APS resides.

2 At the server, log in to the server by typing

> telnet <server>

and pressing the Enter key.

where

server is the IP address or host name of theSPFS-based server on which you are restoringthe APS audio files

3 When prompted, enter your user ID and password.

4 Change to the root user by typing

$ su - root

and pressing the Enter key.

5 When prompted, enter the root password.

6 Change directory to the root file system by typing

# cd /

and pressing the Enter key.

7 Restore the audio files to the file system by typing

# cp /cdrom/cdrom0/* /

and pressing the Enter key.

The files are read from the DVD and placed in the following filesystems: /audio_files and /user_audio_files.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 661: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Prerequisites 661

Performing an APS-only Database restore from a CD

Use this procedure on a SUN v240 server with a CD drive to restore theAPS-only database. Perform this procedure at the APS server.

This procedure is for sites with a SUN v240 and the CD-DVD drive.

This procedure is not for sites with a SUN t1400 server equipped with a4mm DAT drive.

Procedure steps

Step Action

1 Become the root user by entering the following command andthen entering the "root" password when prompted.

su - root

2 Load the backup CD into the CD drive on the server.

3 Change to the aps backup directory on the server by typing thefollowing command.

cd /audio_files/aps_db_backup

4 Copy the dmp files on the CD to this directory by typing thefollowing command.

$ cp /cdrom/cdrom0/* ./

5 Verify the dmp files were copied from CD to the destinationdirectory by typing the following command and then viewing thecommand output.

$ ls -l *

-rwxrwsrwx 1 root other 54272 May 13 11:11 ips_db_audit.dmp-rwxrwsrwx 1 root other 4096 May 13 11:11 ips_node_config.dmp-rwxrwsrwx 1 root other 2048 May 13 11:11 ips_prov_prg_grp.dmp-rwxrwsrwx 1 root other 2048 May 13 11:11 ips_prov_set.dmp-rwxrwsrwx 1 root other 3072 May 13 11:11 ips_user.dmp-rwxrwsrwx 1 root other 3072 May 13 11:11 ips_user_perm_group.dmp-rwxrwsrwx 1 root other 71 May 13 11:11 README-rwxrwsrwx 1 root other 4096 May 13 11:11 sys_parms.dmp. . . . . .-rwxrwsrwx 1 root other 44032 May 13 11:11 uas_segments.dmp

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 662: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

662 CORBA configuration issue correction between the Communication Server 2000 server andthe MG 9000 Manager server

-rwxrwsrwx 1 root other 6144 May 13 11:11 uas_selector_relation.dmp-rwxrwsrwx 1 root other 3072 May 13 11:11 uas_selector_type.dmp-rwxrwsrwx 1 root other 7168 May 13 11:11 uas_selector_value.dmp-rwxrwsrwx 1 root other 5120 May 13 11:11 uas_sequences.dmp-rwxrwsrwx 1 root other 3072 May 13 11:11 uas_set_content.dmp-rwxrwsrwx 1 root other 3072 May 13 11:11 uas_set_definition.dmp-rwxrwsrwx 1 root other 3072 May 13 11:11 uas_user_perms.dmp-rwxrwsrwx 1 root other 3072 May 13 11:11 uas_var_start_seg_id.dmp-rwxrwsrwx 1 root other 3072 May 13 11:11 uas_variables.dmp

#

6 Start the DB restore from the disk files by typing the followingcommand.

$ . ips_export_db.sh -diskonly -restore

This command restores the APS DB from the disk files to thedirectory shown below.

/audio_files/aps_db_backup/*.dmp

7 Restore the APS audio files to the file system if not alreadyperformed.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 663: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Prerequisites for Core and Billing Manager 850 663

Verifying the state of a cluster

Use this procedure to verify the state of a cluster, which involves verifyingthe status of replicated disk volumes, the cluster indicator, and the clusterconfiguration. A cluster refers to a Sun Netra 240 server pair.

ATTENTIONPerform this procedure to determine the state of the cluster rather thanrely on the presence of the ClusterOutOfSync file in the root directory. TheClusterOutOfSync file does not give the definitive state of the cluster.

When a cluster node is out of sync, log SPFS310 is generated.

For more information on logs, see Nortel Carrier Voice over IP FaultManagement Logs Reference Volume 1 (NN10275-909v1). Forinformation on how to configure log reporting, route customer logs to aremote host and view customer logs, see Nortel ATM/IP Solution-levelFault Management (NN10408-900).

For more information on logs, see Nortel Carrier Voice over IP FaultManagement Logs Reference Volume 1 (NN10275-909v1). Forinformation on how to configure log reporting, route customer logs to aremote host and view customer logs, see Packet MSC Fault Management(NN20000-212).

PrerequisitesYou need the root user ID and password for the inactive server.

Prerequisites for Core and Billing Manager 850In order to perform this procedure, you must have the followingauthorization and access.

• You must be a user in the role group authorized to performconfig-admin functions.

• You must obtain non-restricted shell access

For CBM 850, root user ID and password are not required. If logging inusing ssh you can use your own user ID in step 3 of this procedure.

For more information on how to log in to the CBM as an authorized user,how to request a non-restricted shell access, or how to display actions arole group is authorized to perform, review the procedures in the followingtable.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 664: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

664 CORBA configuration issue correction between the Communication Server 2000 server andthe MG 9000 Manager server

Table 58Related procedures

Procedure Document

Logging in to the CBM

Requesting non-restricted shell access

Displaying actions a role group isauthorized to perform

This is only applicable to the CBM and not any other SPFS-based server.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, establish a login session to the inactiveserver, using one of the following methods:

If using Then

telnet (unsecure) step 2

ssh (secure) step 3

2 Log in to the server using telnet (unsecure) as follows:

• Log in to the server by typing> telnet <server>and pressing the Enter key.

where

server is the physical IP address of theinactive server

• When prompted, enter your user ID and password.

• Change to the root user by typing$ su -and pressing the Enter key.

• When prompted, enter the root password.Ensure you are on the right server by typing ubmstat. Youmust be on the inactive server and the response must beClusterIndicatorSTBY.

3 Log in using ssh (secure) as follows:

• Log in to the server by typing> ssh -l root <server>and pressing the Enter key.

where

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 665: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 665

server is the physical IP address of theinactive server

If this is the first time you are logging in using ssh, the systemwill request that you confirm to continue connecting. Enteryes at the prompt.

• When prompted, enter the root password.Ensure you are on the right server by typing ubmstat. Youmust be on the inactive server and the response must beClusterIndicatorSTBY.

4 Determine your next action based on the system response to theubmstat command.

If Then

the system response isClusterIndicatorSTBY

step 6

otherwise step 5

5 Log out of the active server and return to step 1 to log in to theinactive server.

6 Verify the status of replicated disk volumes by typing

# udstat

and pressing the Enter key.

If Then

the system response for eachfilesystem is STANDBY normalUP clean

step 7

otherwise step 8

7 Verify the status of the cluster configuration by typing

# CheckConfiguration

and pressing the Enter key.

If Then

the system responseis Checking localconfiguration againstunit0-priv0 #

the cluster is in a good state andyou can proceed to step 9

otherwise step 8

There will be some discrepancy between the (I)SN07/(I)SN08MTX13 response to the CheckConfiguration command and the(I)SN09MTX14 response. In (I)SN09MTX14, the system returnsan additional response if the verification passes and the cluster

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 666: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

666 CORBA configuration issue correction between the Communication Server 2000 server andthe MG 9000 Manager server

nodes are in sync. This response is not returned in (I)SN07 or(I)SN08MTX13. This discrepancy is normal.

8 Reboot the inactive server by typing

# init 6

and pressing the Enter key.

If Then

you identified a problem Contact your next level ofsupport.

otherwise the cluster is in a good state andyou can proceed to step 9

9 You have completed this procedure. If applicable, return to thehigh level task or procedure that directed you to this procedure.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 667: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 667

Initiating a manual failover on a Sun Netra serverpair

Use this procedure to initiate a manual failover on a Sun Netra 240 serverpair. The need to initiate a manual failover can be required in the followingsituations:

• general maintenance

• software update without a data schema or configuration change

The failover causes the standby (inactive) server to take over and startproviding OAM&P services as the new active server.

ATTENTIONDuring an automatic or manual failover, the high-availability (HA) cluster takesapproximately 5 minutes to failover and bring up the standby node to the Activestate.

PrerequisitesYou must perform this procedure on the active node.

Procedure stepsATTENTIONPerform the steps that follow on the Active server.

Step Action

1 At the active node console, log on to the active node throughthe console (port A) using the root or emsadmin user ID andpassword. For more information, see Logging on to an SPFS-based server.

Ensure you are on the Active server by typing ubmstat. IfClusterIndicatorSTBY is displayed in the response, whichindicates you are on the Inactive server, log off of that serverand log on to the other server in the pair. The response mustdisplay ClusterIndicatorACT, which indicates you are on theActive server.

2 To initiate the manual failover, enter:

# swact

and press the Enter key.

Example response:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 668: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

668 CORBA configuration issue correction between the Communication Server 2000 server andthe MG 9000 Manager server

Are you sure you want to initiate a cluster failover? [Y/N]

3 If it is acceptable to initiate a manual failover, to indicate youwant the failover to occur, enter:

y

and press the Enter key.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 669: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 669

Querying the state of the SPFS platform

Use this procedure to query the state of the system (all monitors registeredwith AlarmD or just those that have alarms against them).

PrerequisitesYou must have root user privileges.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation, telnet to the Sun server by typing

> telnet <IP address>

and pressing the Enter key.

where

IP address is the IP address of the Sun server youwant to query

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su - root

and pressing the Enter key.

4 When prompted, enter the root password.

5 Access the command line interface by typing

# cli

and pressing the Enter key.

6 Enter the number next to the "Configuration" option in the menu.

7 Enter the number next to the "Succession ElementConfiguration" option in the menu.

8 Enter the number next to the "RESMON ApplicationConfiguration" option in the menu.

9 Enter the number next to the "queryFaults" option in the menu.

Example response

===Executing "queryFaults"The Log Information is

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 670: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

670 CORBA configuration issue correction between the Communication Server 2000 server andthe MG 9000 Manager server

*SPFS350 Current filesystem usage = 77%Filesystem is filling upiemscs2k=wxrpz0xy;NODE=wxrpz0xy;CLASS=SYSSYSTYP=FSMon;FSMonName=FSUsage;FSName=/Wed Jun 30 14:54:36 2004

==="queryFaults" completed successfullyRESMON Application Configuration1 - deleteTrapDest (Delete location for IEMS traps)2 - settrapdest (Set location for IEMS traps)3 - queryFaultsMate (Query all faults on the mate)4 - queryFaults (Query all faults on the box)5 - configureCPULoadThresholds (Configure CPULoadThresholds)

X - exit

select -

10 If log information appears, refer to the Nortel Carrier Voiceover IP Fault Management Logs Reference Volume 1 ((NN10275-909v1)) , for more information.

11 Exit each menu level of the command line interface to eventuallyexit the command line interface, by typing

select - x

and pressing the Enter key.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 671: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 671

Rebooting an SPFS-based server

Use this procedure to reboot a Server Platform Foundation Software(SPFS)-based server, which may be hosting one or more of the followingcomponents:

• CS 2000 Management Tools

• Integrated Element Management System (IEMS)

• Audio Provisioning Server (APS)

• Media Gateway (MG) 9000 Manager

• CS 2000 SAM21 Manager

• Network Patch Manager

• Core and Billing Manager (CBM)

• Multiservice Data Manager (MDM)

MDM when installed on SPFS-based servers is not configured as atwo-server cluster but as two distinct one-server configurations.

ATTENTIONThe SPFS-based server may be hosting more than one of the precedingcomponents, therefore, ensure it is acceptable to reboot the server.

PrerequisitesYou must have root user privileges.

Procedure stepsPerform the following steps to complete this procedure.

Step Action

1 At your workstation , telnet to the Sun server by typing

> telnet <IP address>

and pressing the Enter key.

where

IP address is the IP address of the SPFS-basedserver you want to reboot

2 When prompted, enter your user ID and password.

3 Change to the root user by typing

$ su - root

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 672: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

672 CORBA configuration issue correction between the Communication Server 2000 server andthe MG 9000 Manager server

and pressing the Enter key.

4 When prompted, enter the root password.

5 Reboot the server by typing

# shutdown -i 6 -y

and pressing the Enter key.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 673: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

673.

Communication Server 2000troubleshooting

This section contains procedures you can use to troubleshoot issues onthe Communication Server 2000.

Navigation

• “Troubleshooting multiple DS1 carriers in a Carrier Fail Loss (CFL)state” (page 674)

• “Troubleshooting DS1 carriers in an intermittent Carrier Fail Loss (CFL)state” (page 676)

• “Troubleshooting DS0s trunks that are in a Carrier Fail Loss (CFL)state” (page 678)

• “Troubleshooting DS0s trunks that are in a peripheral manual busy(PMB) state” (page 680)

• “Troubleshooting DS0s trunks that are in a System Busy (SB) state”(page 681)

• “Troubleshooting DS0s trunks that are in a remote manual busy (RMB)state” (page 683)

• “Troubleshooting line(s) in a permanent lock out (PLO) state” (page684)

• “Troubleshooting lines in a manual busy (MB) state” (page 686)

• “Troubleshooting lines in a system busy (SB) state” (page 687)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 674: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

674 Communication Server 2000 troubleshooting

Troubleshooting multiple DS1 carriers in a CarrierFail Loss (CFL) state

Troubleshoot multiple DS1 carriers in a Carrier Fail Loss (CFL) state.

Procedure steps

Step Action

1 Use the MAPCI and Communication Server 2000 ManagementTools applications to correlate the DS1 CFL alarms to theassociated GWC and Media Gateway.

2 Determine if the DS1 carriers (in the CFL state) span more thanone GWC.

3 If the DS1 carriers span more than one GWC, go to step 5.

4 If the DS1 carriers do not span more than one GWC, go to step6.

5 There is a problem with the GWC. To fix the problem use theappropriate alarm clearing procedure in GWC Fault Management(NN10202-911).

6 Determine if all the DS1 carriers in the CFL state are on a singleGWC.

7 If all the DS1 carriers in the CFL state are on a single GWC, goto step 9.

8 If all the DS1 carriers in the CFL state are not on a single GWC,go to step 13.

9 To ensure that the GWC is up and running, use the proceduresin GWC Fault Management (NN10202-911). Check the statusof the GWC and check for alarms. If there are alarms, use theappropriate alarm clearing procedure in GWC Fault Management(NN10202-911), to clear the alarms.

10 Determine if you have resolved the problem of DS1 carriers inthe CFL state.

11 If the problem is fixed, then you have completed the procedure.

12 If the problem is not fixed, go to step 13.

13 Determine if the DS1 carriers in the CFL state span all VSPcards in a single Media Gateway.

14 If the DS1 carriers in the CFL state span all VSP cards, go tostep 16.

15 If the DS1 carriers in the CFL state do not span all VSP cards,then contact your next level of support.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 675: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 675

16 From the GWC, ping the host name that is associated with eachVSP card on the Media Gateway that terminates DS1 carriersthat are CFL.

17 Determine if all the VSP cards respond to the ping attempts.

18 If all VSP cards respond, go to step 21.

19 If some, but not all VSP cards respond, see NortelMedia Gateway 7480/15000 Technology Fundamentals(NN10600-780).

20 There is a physical problem between the public switchedtelephone network DS1 carriers and the time division multiplexfunction processor (FP) on the Media Gateway.

Take the necessary steps to isolate the problem.

21 Reset the VSP cards on the Media Gateway that did not respondto the ping attempts.

22 Determine if the problem has been resolved.

23 If the problem has been resolved, then you have completed theprocedure.

24 If the problem has not been resolved, then contact your nextlevel of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 676: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

676 Communication Server 2000 troubleshooting

Troubleshooting DS1 carriers in an intermittentCarrier Fail Loss (CFL) state

Troubleshoot DS1 carriers that are intermittently in a Carrier Fail Loss(CFL) state.

Procedure steps

Step Action

1 Use the MAPCI and Communication Server 2000 ManagementTools applications to correlate the DS1 CFL intermittent alarmsto the associated GWC and Media Gateway.

2 Determine if there are any GWC PM logs being generated atthe Communication Server 2000 or Communication Server2000-Compact.

3 If there are GWC PM logs being generated, go to step 7.

4 If there are no GWC PM logs being generated, there is aproblem with the GWC. For more information to determine ifthere is any action necessary to fix the problem, see GWC FaultManagement (NN10202-911).

5 Determine if the original DS1 problem has been resolved.

6 If the problem has been resolved, then you have completed thisprocedure.

7 Determine if the Media Gateway that terminates the DS1 carrierwith the CFL alarms is generating any logs.

8 If the Media Gateway is generating logs, go to step 10.

9 If the Media Gateway is not generating logs, go to step 14.

10 There is a problem with the Media Gateway. For moreinformation about how to resolve the problems, see NortelMedia Gateway 7480/15000 Technology Fundamentals(NN10600-780).

11 Determine if the original DS1 problem has been resolved.

12 If the problem has been resolved, then you have completed thisprocedure.

13 If the problem has not been resolved, go to the next step.

14 Determine if there are logs being generated by the routingequipment in the IP packet network.

15 If there are logs being generated, go to step 17.

16 If there are no logs being generated, go to step 21.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 677: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 677

17 There is a problem with the router equipment in the IP packetnetwork. Use the documentation provided by your 3rd partyvendor to fix the problem.

18 Determine if the original DS1 problem has been resolved.

19 If the problem has been resolved, then you have completed thisprocedure.

20 Determine if there are any Communication Server 2000 orCommunication Server 2000-Compact logs being generated.

21 If there are logs being generated, go to step 24.

22 If there are no logs being generated, go to step 28.

23 There is a problem with the Communication Server 2000 orCommunication Server 2000-Compact. Use the log informationto determine what action (if any) is necessary to fix the problem.For more information on Communication Server 2000 orCommunication Server 2000-Compact logs, see .For informationon Call Agent hardware and software logs, and SAM21 logs, seeCVoIP Fault Management Logs Reference (NN10275-909).

24 Determine if the original DS1 problem has been resolved.

25 If the problem has been resolved, then you have completed thisprocedure.

26 If the problem has not been resolved, go to the next step.

27 Determine if there are any GWC service or hardware alarms.

28 If there are any GWC service or hardware alarms, go to step 31.

29 If there are no GWC service or hardware alarms, then contactyour next level of support.

30 To fix the problem, see GWC Fault Management(NN10202-911).

For details on individual alarms related to NSS cards in theSAM21 shelf, see the appropriate information in this document.

31 Determine if the original DS1 problem has been resolved.

32 If the problem has been resolved, then you have completed thisprocedure.

33 If the problem has not been resolved, then contact your nextlevel of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 678: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

678 Communication Server 2000 troubleshooting

Troubleshooting DS0s trunks that are in a CarrierFail Loss (CFL) state

Troubleshoot one or more DS0 trunks that are in a Carrier Fail Loss (CFL)state.

The CFL status may be the result of the Media Gateway sending ChangeStatus Notification messages to the Communication Server 2000 informingthe system that the DS0 trunk is unequipped.

Procedure steps

Step Action

1 Use the MAPCI and Communication Server 2000 ManagementTools applications to correlate the DS0 CFL alarms to theassociated GWC and Media Gateway.

2 Determine if there are any GWC PM logs being generated atthe Communication Server 2000 or Communication Server2000-Compact.

.

3 If there are no GWC PM logs being generated, go to step 8.

4 Check to see if there is a GWC provisioning error.

See “Troubleshooting a provisioning error with a GWC card ornode” (page 725).

5 Determine if the original DS0 problem has been resolved.

6 If the problem has been resolved, then you have completed thisprocedure.

7 If the problem has not been resolved, go to the next step.

8 Attempt to return the DS0 trunk to service.

For more information, see Nortel Carrier Voice over IP Upgradeand Patches (NN10440-450).

At the same time, use the Sniffer Network Analyzer to collectsignalling information or perform a GWC trace.

9 If you were able to return the DS0 trunk(s) to service, go to step11.

10 If you were unable to return the DS0 trunk(s) to service, contactyour next level of support.

11 Check the Communication Server 2000 and GWC history logs todetermine the cause of the failure.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 679: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 679

Also check the results of the Sniffer Network Analyzer and theGWC trace to determine the cause of the problem with the MediaGateway equipment.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 680: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

680 Communication Server 2000 troubleshooting

Troubleshooting DS0s trunks that are in a peripheralmanual busy (PMB) state

Troubleshoot one or more DS0 trunks that are in a peripheral manual busy(PMB) state.

Procedure steps

Step Action

1 A GWC is not accessible, with the result that one or more DS0sare in a PMB state. Check XA-Core logs for the occurrence ofa PM181 log. To fix the problem, see GWC Fault Management(NN10202-911).

2 If there are PM181 logs, see GWC Fault Management(NN10202-911), to fix the problem.

3 If there are no PM181 logs, then see GWC Fault Management(NN10202-911).

4 Determine if you have fixed the problem.

5 If the problem has been corrected, then you have completed thisprocedure.

6 If the problem has not been corrected, then contact your nextlevel of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 681: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 681

Troubleshooting DS0s trunks that are in a SystemBusy (SB) state

Troubleshoot one or more DS0 trunks that are in a System Busy (SB)state.

Procedure steps

Step Action

1 There is a status mismatch between the MSC Server 1000XA-Core MSC Server 2000 Call Agent and the GWC. To fix theproblem, check logs and alarms for the Communication Server2000 MSC Server 1000 XA-Core and GWC, and follow theappropriate procedures for clearing them. See the followingdocuments:

• MSC Server 1000 Fault Management (NN20000-222).

• XA-Core Reference Manual (297-8991-810).

• CVoIP Fault Management Logs Reference (NN10275-909).

• GWC Fault Management (NN10202-911).

• DMS-MTX Logs Reference Manual (411-2131-510).

2 Determine if you have fixed the problem and are able to returnthe DS0 links to service.

3 If the problem has been resolved, then you have completed thisprocedure.

4 If the problem persists, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 682: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

682 Communication Server 2000 troubleshooting

Troubleshooting DS0s trunks that are in a lock out(LO) state

Troubleshoot one or more DS0 trunks that are in a lock out (LO) state

Procedure steps

Step Action

1 There is an SS7 problem. Check for logs and alarms to isolatethe problem. See the following documents:

• Communication Server 2000 Fault Management(NN10083-911).

• XA-Core Reference Manual (297-8991-810).

• CVoIP Fault Management Logs Reference (NN10275-909).

• USP Fault Management (NN10071-911).

2 Determine if you have fixed the SS7 problem and are able toreturn the DS0 trunks to service.

3 If the problem has been fixed, then you have completed thisprocedure.

4 If the problem persists, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 683: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 683

Troubleshooting DS0s trunks that are in a remotemanual busy (RMB) state

Troubleshoot one or more DS0 trunks that are in a remote manual busy(RMB) state.

Procedure steps

Step Action

1 The far end public switched telephone network (PSTN) hasplaced one or more DS0 trunks in a manually busy state. TheseDS0 trunks must be returned to service at the PSTN site beforethe DS0 trunks can resume carrying traffic.

2 Return the DS0 trunks to service using the appropriateprocedure.

3 If the problem has been fixed, then you have completed thisprocedure.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 684: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

684 Communication Server 2000 troubleshooting

Troubleshooting line(s) in a permanent lock out(PLO) state

Troubleshoot one or more lines in a permanent lock out (PLO) state.

This procedure applies to the UA-IP and IAC solutions. In the UA-IPsolution, the line-terminating gateway is the MG 9000. In the IAC solutionthe line-terminating gateway is the Media Terminal Adapter (MTA).

Procedure steps

Step Action

1 Use the MAPCI and Communication Server 2000 ManagementTools applications to correlate the lines alarms to the associatedGWC and gateway that terminates the line.

2 From the GWC, ping the MG 9000 (for a UA-IP solution) or MTA(for an IAC solution) to determine if the device can be reached.

3 If the device can be reached, go to step 16.

4 If the device cannot be reached, go to step 5.

5 From the GWC, perform an IP trace route (a ping with the IPtrace route option) to the network element (the MG 9000 or MTA)to determine if the device can be reached.

6 If the device can be reached, go to step 16.

7 If the device cannot be reached, go to step 8.

8 Use the IP trace route software to determine if you can reach theedge router that serves the MG 9000 or MTA.

9 If the edge router can be reached, go to step 16.

10 If the edge router cannot be reached, go to step 5.

11 Use the IP trace route software to determine if you are able toreach the edge router that serves the Communication Server2000.

12 If the edge router can be reached, go to step 14.

13 If the edge router cannot be reached, go to step 15.

14 There is a problem with the IP core packet network. Use theappropriate third party management system and documentationto fix the problem.

15 There is a problem with the Ethernet Routing Switch 8600.Use the Device Manager and the appropriate Ethernet RoutingSwitch 8600 documentation to fix the problem.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 685: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 685

16 If you are working in a UA-IP solution, then there is a problemwith the MG 9000. Use the MG 9000 Manager and for moreinformation about correcting the problem see, MG 9000 FaultManagement (NN10074-911).

If you are working with an IAC solution, then there is a problemwith the third-party MTA equipment. Use the third-partyMTA management system and the appropriate third-partydocumentation to correct the problem.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 686: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

686 Communication Server 2000 troubleshooting

Troubleshooting lines in a manual busy (MB) state

Troubleshoot one or more lines in a manual busy (MB) state.

Procedure steps

Step Action

1 The lines have been manually removed from service. To clearthe alarm, manually return the lines to service.

2 If the lines fail to return to service, use maintenance procedureto correct the problem.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 687: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 687

Troubleshooting lines in a system busy (SB) state

Troubleshoot one or more lines in a system busy (SB) state.

Procedure steps

Step Action

1 If you are working in a UA-IP solution, then there is a problemwith the MG 9000. To correct the problem, use the MG 9000Manager and the appropriate procedures in MG 9000 FaultManagement (NN10074-911).

2 If you are working on an IAC solution, then there is a problemwith the third-party MTA equipment. Use the third-partyMTA management system and the appropriate third-partydocumentation to correct the problem.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 688: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

688 Communication Server 2000 troubleshooting

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 689: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

689.

GWC troubleshootingThis section contains procedure to troubleshoot the Gateway Controller(GWC) associated with Fault Management.

Navigation

• “Troubleshooting a GWC that continuously initializes and fails to returnto service (RTS)” (page 690)

• “Troubleshooting a GWC that fails to boot” (page 691)

• “Troubleshooting a GWC that has a poor call completion rate” (page692)

• “Troubleshooting PRI trunks that remain in a lockout state” (page 693)

• “Troubleshooting a failed attempt to create a GWC image” (page 694)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 690: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

690 GWC troubleshooting

Troubleshooting a GWC that continuously initializesand fails to return to service (RTS)

Use this procedure to troubleshoot a GWC that continuously initializes andfails to return to service (RTS).

Procedure steps

Step Action

1 Determine the setting of the Ethernet port on the EthernetRouting Switch 8600.

2 Ensure the Ethernet port settings on the Ethernet Routing Switch8600 have Auto Negotiation set to True.

3 Determine that you are able to successfully return the GWC toservice.

4 Determine if the IP addressing scheme for the GWC is correct.

5 Assign four consecutive IP addresses to the GWC. The first IPaddress must be divisible by 4. For more information, see GWCConfiguration Management (NN10205-511), in the introductorysection. This contains information on how to modify the baseconfiguration of a GWC.

6 Determine if you are able to successfully return the GWC toservice.

7 Determine if the cabling between the GWC and the EthernetRouting Switch 8600 router is correctly installed and fullyfunctional.

8 Replace the cabling between the GWC and the EthernetRouting Switch 8600 router with cabling that supports 100Mb/sbandwidth.

9 Determine if you are able to successfully return the GWC toservice.

10 If the trouble persists, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 691: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 691

Troubleshooting a GWC that fails to boot

Use this procedure to troubleshoot a GWC that fails to boot.

Procedure steps

Step Action

1 There is a problem with the GWC. Diagnose the problems thatprevent the GWC from booting.

2 Determine if you can boot the GWC successfully.

3 If you cannot boot the GWC, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 692: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

692 GWC troubleshooting

Troubleshooting a GWC that has a poor callcompletion rate

Use this procedure to troubleshoot a GWC that has a poor call-completionrate.

Procedure steps

Step Action

1 Determine the setting of the ethernet port on the EthernetRouting Switch 8600.

2 Change the Ethernet port settings on the Ethernet RoutingSwitch 8600 to100Base-T Half Duplex Auto Negotiate = N.

3 Determine if the GWC call completion rate is acceptable.

4 Contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 693: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 693

Troubleshooting PRI trunks that remain in a lockoutstate

Use this procedure to troubleshoot primary rate interface (PRI) trunks thatremain in a lockout state.

Procedure steps

Step Action

1 Determine if the Media Gateway host name matches thegateway name that is datafilled in the Communication Server2000 Management Tools software.

2 Determine if the Media Gateway carrier endpoint name containsa leading 0 (zero), for example DS1_02031.1

3 There is a problem with the Media Gateway. For information,see Nortel Networks Media Gateway 7480/15000 TechnologyFundamentals (NN10600-780) to fix the problem.

4 Determine if you have eliminated the PRI trunk lockout state.

5 Change the Media Gateway host name to match the gatewayname in the Communication Server 2000 ManagementTools datafill. For more information, seeNortel NetworksMedia Gateway 7480/15000 Switched Service ConfigurationManagement (NN10600-782).

6 Determine if you have eliminated the PRI trunk lockout state.

7 If the problem persists, contact your next level of support

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 694: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

694 GWC troubleshooting

Troubleshooting a failed attempt to create a GWCimage

Use this procedure to troubleshoot a failed attempt to create a GWCimage.

Procedure steps

Step Action

1 In the response dialog at the GWC Manager client window, aftera failure to take the image, click Show Details .

2 Examine the Return Code (RC) value in the dialog box.

3 If the RC value equals -4, go to Step 6.

4 If the RC value equals -9, go to step 8.

5 If the RC value has any other value, go to step 10.

6 The directory permissions for the directory /swd/gwc/ on theCommunication Server 2000 Core ManagerCommunicationServer 2000 Core Manager are incorrect. Change the directorypermissions to 777 using the chmod command.

7 Determine if you are able to correct the problem and successfullycreate a GWC image.

8 There is not enough space allocated on the CommunicationServer 2000 Core ManagerCommunication Server 2000Core Manager for directory /swd/gwc/. Increase the diskspace allocated on the Communication Server 2000 CoreManagerCommunication Server 2000 Core Manager forthe /swd/gwc/ directory Communication Server 2000 CoreManagerCommunication Server 2000 Core Managerusing theSDM RMI sdmmtc->storage->change Iv commandusing the SDMRMI sdmmtc->storage->change Iv command.

9 Determine if you are able to correct the problem and successfullycreate a GWC image.

10 Use the following table to correct the FTP problem.

RC value Reason for the FTP error Explanation

-1 BAD_HOST_IP incorrect Host IP

-2 BAD_USER incorrect UserID

-3 BAD_DIR incorrect directory

-5 ABORT_SERV A service has aborted.

-6 ABORT_USER A user has aborted.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 695: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 695

RC value Reason for the FTP error Explanation

-7 ERROR error

-8 ERROR_DATA_CONN data connection error

-10 ERROR_CMD incorrect command

-11 FILE_NOTFOUND The file was not found.

-12 BUFFER_TOO_SMALL The buffer is too small.

-13 TRANSFER_INCOMPLETE Transfer is incomplete

-14 RENAME_ERROR An error in renaming

-15 IMAGE_FILE_FAIL Failure to create an image file.

11 Determine if you are able to correct the problem and successfullycreate a GWC image.

12 Contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 696: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

696 GWC troubleshooting

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 697: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

697.

Trunk testingThis section provides information on the trunk tests you can perform in anATM solution for TDM (time division multiplex) trunks that terminate on theMG 4000. You must perform all the trunk tests described in this sectionfrom the trunk maintenance level of the MAP (which you access throughthe Communication Server 2000 Core Manager).

For more detailed information on trunk testing, see DMS-100 FamilyTrunks Maintenance Guide (297-1001-595).

Navigation

• “Performing a TL 100 trunk test ” (page 698)

• “Performing a TL 101 trunk test” (page 700)

• “Performing a TL 102 trunk test” (page 701)

• “Performing a TL 104 trunk test” (page 703)

• “Performing a TL 105 trunk test” (page 705)

• “Performing a TL 108 trunk test” (page 711)

• “Performing a ISUP continuity test (ICOT)” (page 713)

• “Performing a CVTEST” (page 714)

• “Performing a QRYSIG” (page 715)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 698: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

698 Trunk testing

Performing a TL 100 trunk test

The 100 Test line trunk test, also known as a quiet or balancedtermination, provides noise and loss measurements. Three versions ofthe TL100 are present: T100, S100, and N100. Use the T100 when theequipment at the terminating office is unknown. When the T100 test linetest is complete, a two second time-out attempts to detect a milliwatt tone.If the time-out does not detect a tone, the S100 version of the test runs.The S100 version provides a quiet termination for noise measurementsonly. The N100 includes a milliwatt test. You can use the N100 forfar-to-near loss measurements. If you know the version of the distant officetest line, you can run that version of the test directly. A two second delayfor each trunk is not required.

Procedure steps

Step Action

1 At the system level

The originating office transmits the test code to the terminatingoffice. The terminating office responds with an off-hook signalwhen the test code establishes a connection.

2 The originating office connects the Transmission Test Trunk(TTT) to the trunk under test.

3 The terminating office delays the off-hook signal and the start ofthe milliwatt test tone by 300 ms.

4 The originating office detects the milliwatt test tone and a levelmeter measures the far-to-near loss. The milliwatt tone ispresent for a maximum of six seconds. The terminating officeprovides a quiet termination.

5 The terminating office maintains the quiet termination for as longas the originating office holds the incoming trunk.

6 The originating office measures noise on the trunk.

7 The originating office sends an on-hook signal to the terminatingoffice to indicate the end of the test, and releases the incomingtrunk.

--End--

Job aidThe following MAPCI output demonstrates the activation of the TL100Trunk test.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 699: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 699

>MTC;TRKS;TTPMTC:TRKS:SHORT CLLI IS: ATM01SPOSITION RESTARTEDTTP:>post g ATM01S7ITW2LAST CKTN = 48POSTED CKT IDLEDSHORT CLLI IS: ATM01SOK, CKT POSTED>tst t100MSH3_TM03BM ***+ TRK127 NOV 13 13:53:29 8900 PASSTL100 PASSEDCKT ATM01S7ITW2TTT = TTT 1 EML = 0.0 DBF_N DEV = 0.0 DB NOISE = 1.2 DBRNNML = 50 DBRN NIAL = DBRN

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 700: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

700 Trunk testing

Performing a TL 101 trunk test

Use the 101 Test Line trunk test to establish two-way talk between a testposition and a trunk (incoming or outgoing). The trunk must associate withthe switch (DMS-200). The number of test lines depends on the officesize. The operating company assigns the 101 Test Lines in data tableTRKGRP.

Procedure steps

Step Action

1 At the system level

The XA-Core or SN70EM receives an incoming 101 test callfrom a distant office, and a TL101 test trunk.

2 The XA-Core or SN70EM applies ringing to all telephone setsconnected to that test trunk and returns audible ringing tone tothe calling far-end.

3 When any of the connected telephone sets go off-hook, theXA-Core or SN70EM removes ringing and audible ringing toneand enters the talking phase.

4 If the receiving telephone line is on hold, ring forward appliesringing until the telephone line answers. A flash-hook from thereceiving telephone set sends ringing to the calling end.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 701: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 701

Performing a TL 102 trunk test

The 102 Test line trunk test, also called a milliwatt test line, providesfar-to-near transmission loss measurements. The far-end sends a ninesecond off-hook signal, and applies the milliwatt test tone. An on-hooksignal and quiet termination follow the tone. If the far-end is a local office,the test tone repeats in ten second cycles: nine seconds on, and onesecond off. For a toll office, quiet termination continues until the near-endreleases the connection or requests another test cycle. The toll officemakes the test cycle request with a ring-forward signal. The milliwattsource levels in use depend on the office type and the trunk group.

Procedure steps

Step Action

1 At the system level

The originating office transmits the test code to the terminatingoffice.

2 When the test code establishes a connection, the terminatingoffice returns an off-hook signal.

3 The originating office connects the DTU (digital test unit) to thetrunk under test.

4 The terminating office introduces a 300 ms delay, followed by a1004 Hz milliwatt tone.

5 The originating office detects the tone and a level metermeasures far-to-near loss.

6 The terminating office maintains the tone for nine seconds. Afternine seconds the terminating office sends an on-hook signalfollowed by quiet termination.

7 The originating office releases the connection or requestsanother test sequence through a ring forward signal.

--End--

Job aidThe following MAPCI output demonstrates the activation of the TL102Trunk test.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 702: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

702 Trunk testing

>MTC;TRKS;TTPMTC:TRKS:SHORT CLLI IS: ATM01SPOSITION RESTARTEDTTP:>post g ATM01S7ITW2LAST CKTN = 48POSTED CKT IDLEDSHORT CLLI IS: ATM01SOK, CKT POSTED>tst t102TEST OKMSH3_TM03BM ***+ TRK125 NOV13 13:55:43 0800 PASSTL102 PASSEDCKT ATM01S7ITW2EML = 0.0 DB F_N DEV = 0.0 DB

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 703: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 703

Performing a TL 104 trunk test

The 104 Test line provides:

• measurement of two-way transmission loss

• measurement of far-to-near noise

• a check of near-to-far noise

Procedure steps

Step Action

1 At the system level

The originating office transmits the test code, and when theterminating circuit connects, the terminating office responds withan off-hook signal.

2 The originating office connects the DTU (digital test unit) to thetrunk under test.

3 The originating office sends a milliwatt tone for three seconds.The terminating office measures the near-to-far loss.

4 The terminating office sends a milliwatt tone for three seconds.The originating office measures the far-to-near loss.

5 A delay occurs. The terminating office sends a milliwatt tone forthree seconds attenuated by the near-to-far loss measured instep 3. This tone enables the originating office to measure thenear-to-far and far-to-near losses.

6 Immediately following the attenuated milliwatt tone, theoriginating office measures the far-to-near noise level. Theterminating office measures the near-to-far noise level after twoseconds.

7 If the near-to-far level exceeds 41 dBm, the terminating officesends a tone at 30 or 120 IPM (impulses per minute).

8 The originating office sends on-hook supervision to theterminating office to stop the test.

--End--

Job aidThe following MAPCI output demonstrates the activation of the TL104Trunk test.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 704: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

704 Trunk testing

>MTC;TRKS;TTPMTC:TRKS:SHORT CLLI IS: ATM01SPOSITION RESTARTEDTTP:>post g ATM01S7ITW2LAST CKTN = 48POSTED CKT IDLEDSHORT CLLI IS: ATM01SOK, CKT POSTED>tst t104TEST OKMSH3_TM03BM ***+ TRK134 NOV13 13:58:09 7300 PASSTL104 PASSEDCKT ATM01S7ITW2 1EML = 0.0 DB F_N DEV = 0.0 DBN_F DEV = 0.0 DB F_N NOISE = 1 DBRNNML = 50 DBRN NIAL 50 DBRN

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 705: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 705

Performing a TL 105 trunk test

The 105 Test line trunk test provides two-way testing from the originatingoffice. The types of measurement include:

• loss

• noise

• echo return loss

• transmission loss

• singing point return

• high and low frequency measurements

The TL105 tests run automatically. Each designation code causes thesystem to run a different group of tests. The following table lists thedesignation codes and associated tests.

Table 59TL105 Test combination description

Code Tests Description

T105 LSC, L, NSC, RN, N Loss measurementwith self-check at 0dBm and 1004 Hz.Noise measurementwith self-check (quiettermination; C-messagefilter).

T105 LSC, L, NSC, RN, N Loss measurementwith self-check at 0dBm and 1004 Hz.Noise measurementwith self-check (quiettermination; C-messagefilter).

TL0N L, RN, N Loss measurement at0 dBm and 1004 Hz.Noise measurement (quiettermination; C-messagefilter).

TL0S LSC, L Loss measurement withself-check at 0 dBmand 1004 Hz. Noisemeasurement (quiettermination, C-messagefilter).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 706: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

706 Trunk testing

Table 59TL105 Test combination description (cont’d.)

Code Tests Description

TL05 L Loss measurement at 0dBm and 1004 Hz.

TL165 LSC4, LSC10, LSC28, L4,L10, L28, NTSC, NT

Loss measurement at –16dBm and 404, 1004 2804Hz. Noise measurementwith self-check (toneat –16 dB, 1004 Hz.C-notched filter).

TL6N L4, L10, L28, NT Loss measurement at –16dBm and 404, 1004 2804Hz. Noise measurement(tone, C-notched filter).

TL6S LSC4, LSC10, LSC28, L4,L10, L28

Loss measurement withself-check –16 dBm and404, 1004 2804 Hz.

TL65 L4, L10, L28 Loss measurement at –16dBm and 404, 1004 2804Hz.

TS05 LSC, NSC Far-end loss self-checkat 1004 Hz, and 0 dBm.Far-end noise self-check(quiet termination,C-message filter).

TS65 LSC4, LSC10, LSC28,NTSC

Far-end loss self-check at–16 dBm and 404, 1004,2804 Hz.

T5SB ERLSC, SRLSC, SHISC Return loss measurementand self-check.

T5BS ERL, ERLSC, SRLLO,SRLSC, SHI, SHISC

Return loss measurementand self-check.

TERL ERL Echo return lossmeasurement.

T5LH SRL, SHI Singing return loss (lowand high) measurements.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 707: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 707

Table 59TL105 Test combination description (cont’d.)

Code Tests Description

T5AS LSC, L, NSC, RN, N,ERL, ERLSC, SRLLO,SRLSC, SHI, SHISC

Loss measurementand self-check at 0 dBand 1004 Hz. Noisemeasurement andself-check (C-messagefilter). Return lossmeasurement andself-check.

T5AT L, RN, N, ERL, SRLLO,SHI

Loss measurement at0 dB and 1004 Hz.Noise measurement(C-message filter). Returnloss measurement.

T50L L, ERL, SRLLO, SHI Loss measurement at 0dB and 1004 Hz. Returnloss measurement.

T56N L4, L10, L28, NT, ERL,SRLLO, SHI

Loss measurement at –16dB and 404, 1004 2804Hz. Noise measurement(C-notched filter). Returnloss measurement.

T5LB L4,10, L28, ERL, SRLLO,SHI

Loss measurement at–16 dB and 404, 10042804 Hz. Return lossmeasurement.

TSBS LSC, NTSC, SRLSC,SHISC

Loss self-check at –16dB and 404, 1004 2804Hz. Noise self-check(C-notched filter). Returnloss self-check.

TSBT ERL, SRLLO, SHI Return loss measurement.

The system groups tests in layers, named layer 2 or layer 3. Each test hasan associated Multi-Frequency (MF) digit, but two tests in different layerscan share the same MF digit. The MF transmits with you request a test.Another MF digit, known as a "layer change request" digit (1100/1700 Hz)goes before a request for a test in layer 3. If this digit is not present, thesystem selects the associated test in layer 2 by default.

A Noise Metering Set (NMS) reads noise levels. Noise passes through aC-message filter before measurement. The filter reads the noise as C-levelnoise. For trunks on a carrier that compresses and expands signals, a1004 Hz tone is transmitted over the trunk with the NMS measurements.A C-notch filter filters out the tone, and reads the noise as C-notch noise.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 708: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

708 Trunk testing

See Table 59 "TL105 Test combination description" (page 705) and Table60 "Test and layer format description" (page 709) for tests that use thesefilters.

Procedure steps

Step Action

1 At the system level

The originating office transmits the number, that is assigned tothe TL105 test line in the terminating office.

2 After the terminating office receives this assigned number, theterminating TL105 test line connects to the incoming trunk. Theterminating TL105 test line sends a 2225 Hz Test Progress Tone(TPT) to the originating end.

3 The originating office connects the DTU (digital test unit) to thetrunk under test.

4 When the terminating office is ready to start the test, theterminating office removes the Test Progress Tone (TPT).

5 The originating office detects the removal on the TPT and startswith the requested group of tests. The tests run in the followingorder.

• loss self-checks

• loss measurements

• noise self-checks

• noise measurements

• Echo-return losses

ATTENTIONThe specified group of tests does not affect this sequence. A requestsent to the far end starts each test. The request is one or two MFdigits.

--End--

Job aidThe following MAPCI output demonstrates the activation of the TL105Trunk test.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 709: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 709

>MTC;TRKS;TTPMTC:TRKS:SHORT CLLI IS: ATM01SPOSITION RESTARTEDTTP:>post g ATM01S7ITW2LAST CKTN = 48POSTED CKT IDLEDSHORT CLLI IS: ATM01SOK, CKT POSTED>tst t105TEST OKMSH3_TM03BM ***+ TRK172 NOV13 13:59:29 7300 PASSTL105 PASSEDCKT ATM01S7ITW2 1LOSS MEAS AT 0 DB 1003 HZ EML = 0.0FN_DEV = 0.0 NF_DEV = 0.–1 DBNOISE MEAS C-MESSAGE. NML = 50 NIAL = 50 DBRNFN_NSE = 1 NF_NSE = 16 DBRN

Table 60Test and layer format description

Test Layer MF digitMFFrequency Test description

ERL 2 C 700/1700 Echo return loss measurement with quiettermination

ERLSC 3 A, C 700/1700 Echo return loss self-check with quiettermination

L 2 2 700/1100 Two-way loss measurement at 0 dBmand 1004 Hz.

LA 2 6 1100/1300 Two-way loss measurement at –16 dBmand 404 Hz.

L10 2 C 1500/1700 Two-way loss measurement at –16 dBmand 1004 Hz.

L28 2 E 1300/1700 Two-way loss measurement at –16 dBmand 2804 Hz.

LSC 2 1 700/900 Far-end loss self-check at 0 dBm and1004 Hz.

LSC4 3 6 1100/1300 Far-end loss self-check at –16 dBm and404 Hz.

LSC10 3 c 1500/1700 Far-end loss self-check at –16 dBm and1004 Hz.

LSC28 3 E 1300/1700 Far-en loss self-check at –-16 dBm and2804 Hz.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 710: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

710 Trunk testing

Table 60Test and layer format description (cont’d.)

Test Layer MF digitMFFrequency Test description

N 2 3 900/1100 Far-end noise measurement withC-message filter.

NSC 2 4 700/1300 Far-end noise self-check withC-message filter.

NT 2 F 700/1700 Two-way noise tone measurement withC-notched filter.

NTSC 3 F 700/1700 Far-end noise/tone self check withC-notched filter at –16 dBm and 1004Hz.

RN 2 A 1300/1500 Near-end noise measurement withC-message filter. Filter at –16 dBm and1004 Hz tone.

SHI 3 2 700/1100 High frequency singing return lossmeasurement with quiet termination.

SHISC 3 A, 1 700/900 High frequency singing return lossself-check with quiet termination.

SRLHI 3 A, 2 2200/3400 High frequency singing return loss withquiet termination.

SRLLO 2 8 900/1500 Low frequency singing return lossmeasurement with quiet termination.

SRLSC 3 A, 8 900/1500 Low frequency singing return lossself-check with quiet termination.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 711: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 711

Performing a TL 108 trunk test

The T108 Terminating Test Line test is a method for accessing the dialednon-inverting loopback on trunks features known as TRKLPBK. T108 canbe used to isolate trunk troubles and to measure net loss, noise, and runBERT (Bit Error Rate Test) for trunks at the DS0 rate.

The T108 Terminating Test Line test is supported but not associated withthe MAPCI. For more information, see DMS-100 Family Maintenance andOperations Manual (297-8991-500).

Procedure steps

Step Action

1 At the system level

The originating toll office connects the ESMS (Echo SuppressorMeasuring Set) and seizes an outgoing intertoll trunk.

2 After the originating office seizes the trunk, the originating officedials a T108 Test Line code. The seized trunk is the trunk undertest.

3 The terminating office translates the T108 Test Line code andconnects the incoming trunk to test Port-0 of the T108 Test Line.The terminating office uses a 1 kHz tone to calibrate the ESMS.The terminating office sends the tone to the originating office.

4 The originating office seizes another outgoing trunk, called theauxiliary trunk and dials the T108 Test Line code again.

5 The terminating office translates the TI08 code and connectsthe auxiliary trunk to test Port-1 of the T108 Test Line. Theterminating office cuts off the 1 kHz tone and establishes aconnection to Port-0.

6 When the terminating office establishes a connection, theoriginating office transmits a 1 kHz tone on the auxiliary trunk.This tone passes through the T108 Test Line, back over thetrunk under test. The tone completes the calibration of theESMS. The office uses the Auxiliary trunk as a reference trunkfor later tests of echo suppressors.

7 When tests on the trunk under test complete, the originatingoffice releases the trunk under test. The originating office seizesthe next trunk for test, dials the T108 Test Line code over thistrunk. When all tests are complete, the originating office goeson-hook on both the trunk under test, and the auxiliary trunk.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 712: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

712 Trunk testing

The terminating office releases (idles) the T108 Test Linetermination set.

--End--

Job aidThe following MAPCI output demonstrates the activation of the TL108Trunk test.

>MTC;TRKS;TTPMTC:TRKS:SHORT CLLI IS: ATM57SPOSITION RESTARTEDTTP:>datattpDATATTP:>bterm dig 2349002Information registeredBERT termination : DIGITS : 2349002It will be used on next BERT test request>bert start 56 p511BERT test time : No time specified - foreverBERT termination : DIGITS : 2349002BERT test started>bert stopBERT test stopped

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 713: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 713

Performing a ISUP continuity test (ICOT)

The ISUP continuity test validates the datafill and speech path for trunksthat use CCS7 signaling.

Procedure steps

Step Action

1 At the MAPCI;MTC;TRKS;TTP level of the MAP

Post the trunk you wish to test.

Examplepost g ATM01S7ITW2

LAST CKTN = 48POSTED CKT IDLEDSHORT CLLI IS: ATM01SOK, CKT POSTED

2 Test the trunk for continuity.

Exampletst icot

TEST OK

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 714: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

714 Trunk testing

Performing a CVTEST

The CVTest verifies trunk datafill parameters including glare. The CVTestcommand is found at the C7TTP MAP level. The CVTest performs a circuitvalidation test on the posted ISUP trunk. The test verifies that the circuitgroup characteristics pertaining to the trunk are datafilled consistently atthe two connected offices. The CVTest checks the compatibility of thedatafill for the glare resolution methods, found in the GLARETYP field oftable TRKSGRP.

Procedure steps

Step Action

1 At the MAPCI;MTC;TRKS;TTP;C7TTP level of the MAP

Post the trunk you wish to test.

Examplepost g ATM01S7ITW2

LAST CKTN = 48POSTED CKT IDLEDSHORT CLLI IS: ATM01SOK, CKT POSTED

2 Run the CVTest.

Examplecvtest

SUCCEEDED, GLARE control data compatible with remote office.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 715: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 715

Performing a QRYSIG

The QRYSIG test displays the signaling status of the posted CCS7 trunk.

Procedure steps

Step Action

1 At the MAPCI;MTC;TRKS;TTP;C7TTP level of the MAP

Post the trunk you wish to test.

Examplepost g ATM01S7ITW2

LAST CKTN = 48POSTED CKT IDLEDSHORT CLLI IS: ATM01SOK, CKT POSTED

2 Run the QRYSIG test.

Exampleqrysig

Signaling Available

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 716: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

716 Trunk testing

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 717: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

717.

Communication Server 2000management tools troubleshooting

This section contains procedures for troubleshooting problems withCommunication Server 2000 management tools.

Navigation

• “Troubleshooting a failed attempt to add and provision a GWC or toassociate a gateway with a GWC” (page 719)

• “Troubleshooting a failure to add or delete a GWC” (page 720)

• “Troubleshooting when an OSSGate session fails to launch from a PC”(page 721)

• “Troubleshooting a connection failure when adding a GWC node”(page 722)

• “Troubleshooting a failure to start the SESM server application” (page724)

• “Troubleshooting a provisioning error with a GWC card or node” (page725)

• “Troubleshooting a failure of the Communication Server 2000Management Tools alarm manager to properly set alarm date andtime” (page 726)

• “Troubleshooting small gateway IP addresses that are reset to 0.0.0.0in the Communication Server 2000 Management Tools server” (page728)

• “Troubleshooting a failure to RTS a DMS Maintenance Application thatis IsTb” (page 729)

• “Troubleshooting a failure to add a fourth Media Gateway 15000 to aGWC” (page 730)

• “Troubleshooting a failure to perform maintenance actions while usingthe Communication Server 2000 Management Tools” (page 731)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 718: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

718 Communication Server 2000 management tools troubleshooting

• “Troubleshooting a failure to launch Communication Server 2000SAM21 Manager, Communication Server 2000 Management ToolsGUI, or Network Patch Manager” (page 732)

• “Diagnosing problems with a GWC card that cannot be booted” (page734)

• “Troubleshooting problems arising when an SSL certificate is changedand the CM hostname is new or changed” (page 738)

• “Troubleshooting combined Communication Server 2000 ManagementTools failures” (page 739)

• “Troubleshooting a failure to delete a line from a media gateway in theIAC or Intl IAC solution” (page 740)

• “Troubleshooting a failed attempt to input the SDNA command inOSSGate” (page 742)

• “Troubleshooting a failure to add an endpoint to a GWC during lineprovisioning of the IAC or Intl IAC solution” (page 743)

• “Troubleshooting a failed attempt to create a GWC image from theCommunication Server 2000 Management Tools GUI” (page 744)

• “Troubleshooting APS logon problems” (page 745)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 719: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 719

Troubleshooting a failed attempt to add andprovision a GWC or to associate a gateway with aGWC

Troubleshoot a failed attempt to add and provision a Gateway Controller(GWC). In addition, use this task to troubleshoot a failed attempt toassociate a gateway (such as line, or trunk media gateway) with a GWC.

Procedure steps

Step Action

1 The computing module common language location identifier(CM CLLI) may be configured incorrectly on the CommunicationServer 2000 Management Tools server. The CM CLLI is thename of the office and is case sensitive. However, there aremany other reasons why this problem might occur.

2 Determine if the CM CLLI is correctly configured on theCommunication Server 2000 Management Tools server.

See “Viewing SESM configuration settings” (page 351).

3 If the CM CLLI is configured correctly, go to step 5; otherwise,go to the next step.

4 Correct the CM CLLI on the Communication Server 2000Management Tools server. If you are using a Sun server, and formore information about setting the Communication Server 2000CLLI on the Sun server see, ATM/IP solution-level ConfigurationManagement (NN100409-500).

If you are using an MSC server, and for more informationabout setting the Communication Server 2000 CLLI on the Sunserver, see ATM/IP solution-level Configuration Management(NN100409-500).

5 If you were able to correct the problem, and can now add andprovision GWCs or associate gateways successfully, you havecompleted this procedure.

6 If problems persist, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 720: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

720 Communication Server 2000 management tools troubleshooting

Troubleshooting a failure to add or delete a GWC

Use this procedure to troubleshoot a failed attempt to add or delete aGWC.

ATTENTIONThis problem only occurs during initial provisioning of the office.

Procedure steps

Step Action

1 There is a failed attempt to add or delete a GWC. The reasonfor this failure is that Table SITE does not have an entry with akey of ’LG’.

2 Using the appropriate procedure, add an entry to table SITE witha key of ’LG’ using the Table Editor. Then reattempt to add ordelete a GWC.

3 If you are now able to add or delete a GWC successfully, youhave completed this procedure.

4 If you are unable to add or delete a GWC successfully, contactyour next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 721: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 721

Troubleshooting when an OSSGate session fails tolaunch from a PC

Use this procedure to troubleshoot a failure to launch an OSSGate sessionfrom an IBM type of personal computer (PC).

Procedure steps

Step Action

1 The Windows Telnet client sends the username and passwordcharacter-by-character to the OSSGate application; however, theOSSGate application expects the username and password tobe sent in an entire line. As a result, the OSSGate applicationdisplays the following error message.

’Incomplete login information supplied’

To fix this problem, launch the OSSGate session from a UNIXTelnet client instead of from a PC Telnet client.

2 If you are now able to correctly launch OSSgate, you havecompleted this procedure.

3 If the trouble persists, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 722: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

722 Communication Server 2000 management tools troubleshooting

Troubleshooting a connection failure when adding aGWC node

Use this procedure to troubleshoot a connection failure when addinga GWC node, and you see this error message: An error has beendetected while trying to communicate to the Server. Pleaseclose the current session and restart the client."

Procedure steps

Step Action

1 Determine if you received the following message whenattempting to add the GWC node:

’An error has been detected while trying to connect to the Server. Please close the current session and restart the client.’

If you received this message, proceed to the next step;otherwise, you have completed this procedure.

2 There is a communication failure between the SESM serverapplication and the OSSComms system on the CommunicationServer 2000 Core Manager.

This may be a network-wide problem. Attempt to pingthe Communication Server 2000 Core Manager from theCommunication Server 2000 Management Tools Server.

3 If you successfully ping the Communication Server 2000 CoreManagerCommunication Server 2000 Core Manager , go to step5 or if you were unable to ping the Communication Server 2000Core ManagerCommunication Server 2000 Core Manager, go tothe next step.

4 There is a network wide problem. Check for alarms and logsusing the appropriate procedures to fix the network problem.

5 Attempt to add the GWC node.

6 If you were able to add the GWC node, you have completed thisprocedure; otherwise, proceed to the next step.

7 The OSS Comms and the OSS and Applications servicesapplications may be in a failed state on the CommunicationServer 2000 Core ManagerCommunication Server 2000 CoreManager.

8 Log on to the Communication Server 2000 CoreManagerCommunication Server 2000 Core Manager

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 723: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 723

console and check the state of the OSS Comms Svcs and OSSand Applications Svcs applications.

9 If the two applications are in-service, go to step 13.

OR

If the two applications are not in-service, go to the next step.

10 Manually busy, then return to service the OSS Comms Svcsapplication and the OSS and Application Svcs application.

11 Attempt again to add the GWC node.

12 If you were able to add the GWC node, you have completed thisprocedure; otherwise, go to the next step.

13 Check the status of the DDMS proxy utility that runs on theCommunication Server 2000 Management Tools server.

14 If the DDMS proxy utility is running, you have completed thisprocedure; otherwise, go to the next step.

15 Restart the DDMS proxy utility.

16 Restart the SESM server application.

17 If the communication problem is fixed and you are able to add aGWC, you have completed this procedure.

OR

18 If the problem persists, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 724: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

724 Communication Server 2000 management tools troubleshooting

Troubleshooting a failure to start the SESM serverapplication

Use this procedure to troubleshoot the SESM server application when theapplication fails to start, and there is an mi2 log that states:ORA-01034 ORACLE not available;ORA-27101 shared memory realm does not exist,SVRA Error No such file or directory.

Procedure steps

Step Action

1 If the SESM server fails to start, determine if you have receivedthe following Communication Server 2000 Management Toolsserver database error message:

java.sql.SQLEexception:ORA-01034:ORACLE not availableORA-27101:shared memory realm does not exist

SVR4 Error 2: No such file or directory

If you did not receive the message, you have completed thisprocedure; otherwise, go to the next step.

2 At the APS server, determine if the Oracle server is running.

See “Troubleshooting APS logon problems” (page 745).

If the Oracle server is running, you have completed thisprocedure; otherwise, go to the next step.

3 Start the Oracle database.

See “Troubleshooting APS logon problems” (page 745).

4 If you are able to start the SESM server application, you havecompleted this procedure.

OR

5 If the trouble persists, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 725: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 725

Troubleshooting a provisioning error with a GWCcard or node

Use this procedure to troubleshoot a provisioning error with a GatewayController (GWC) card or node.

Procedure steps

Step Action

1 Determine if there are logs or alarms generated indicating apossible GWC card or node provisioning problem.

If there are no alarms or logs generated, you have completedthis procedure; otherwise, go to the next step.

2 Perform a data integrity audit using the procedure “Performinga Communication Server 2000 data integrity audit” (page 335).Then perform the actions necessary to fix the problem.

3 If the provisioning error associated with the GWC card iscorrected you have completed the procedure.

OR

4 If the trouble persists contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 726: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

726 Communication Server 2000 management tools troubleshooting

Troubleshooting a failure of the CommunicationServer 2000 Management Tools alarm manager toproperly set alarm date and time

Use this procedure to troubleshoot a failure of the CommunicationServer 2000 Management Tools alarm manager to properly set alarmdate and time. While this problem is occurring the date and time onthe Communication Server 2000 Core Manager, the CommunicationServer 2000 Management Tools, and the Communication Server2000Communication Server 2000 Core Manager, the CommunicationServer 2000 Management Tools, and the Communication Server 2000 areall set correctly. When this problem occurs, the inaccurate date is alsoshown in the details sub-window at the bottom of the main window.

Procedure steps

Step Action

1 Determine if you received the following problem indicators for theCommunication Server 2000 Management Tools alarm manager.

• The alarms are being output with a date and time such as21:37:103-Mar-2036

• The inaccurate date is also shown in the details sub-windowat the bottom of the main window.

If you did not receive the problem indicators, you havecompleted this procedure; otherwise, go to the next step.

2 The source for the Network Time Protocol (NTP) server for theSAM21 shelf is incorrectly set to the Communication Server2000 Core ManagerCommunication Server 2000 Core Manager.This is an error because the Communication Server 2000 CoreManagerCommunication Server 2000 Core Manager does notprovide NTP.

3 Check the configuration of the NTP service.For configuring a timing provider on an SPFS-based sever, seeCarrier VoIP Nortel MSS 15000, MG 15000 and MDM in VoIPNetworks Attribute Reference (PT-AAL1/UA-AAL1/UA-IP/PTIP/PT-AAL2) (NN10225-512).

4 Reconfigure the NTP service.

When you come to the steps for changing the Primary andSecondary NTP server addresses, proceed as follows:

5 Change the primary IP to the IP address of the master NTPserver (master DCE server).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 727: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 727

6 Change the secondary NTP to 0.0.0.0.

7 If you are able to correct the problem, you have completed thisprocedure.

OR

8 If the trouble persists, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 728: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

728 Communication Server 2000 management tools troubleshooting

Troubleshooting small gateway IP addresses thatare reset to 0.0.0.0 in the Communication Server2000 Management Tools server

Use this procedure to troubleshoot the Internet Protocol (IP) addresses ofsmall gateways that are reset to 0.0.0.0 in the Communication Server 2000Management Tools server.

Procedure steps

Step Action

1 Determine if the IP addresses of all six small gateways are resetto 0.0.0.0 in the Communication Server 2000 Management Toolsserver and that there is no call processing on the associatedGWCs after reloading.

If both cases are not true, you have completed this procedure;otherwise, go to the next step.

2 The Domain Name Server (DNS) capability is not set up for theIP addresses of the small gateways.

3 If you do not have a backup of the file systems of theCommunication Server 2000 Management Tools server, youhave completed this procedure; otherwise, go to the next step.

4 Restore the latest Communication Server 2000 ManagementTools backup. See ATM/IP Solution-level Security andAdministration (NN10402-600) and Packet MSC Security andAdministration (NN20000-216).

5 If you were able to correct the original problem, you havecompleted the procedure.

OR

6 If the trouble persists, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 729: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 729

Troubleshooting a failure to RTS a DMSMaintenance Application that is IsTb

Use this procedure to troubleshoot a failure to return to service (RTS) adigital multiplex system (DMS) Maintenance Application (DMA). After thereturn-to-service (RTS) fails, the DMA remains in an in-service-trouble(IsTb) state.

Procedure steps

Step Action

1 If the DMS is IsTb and fails to RTS, ensure that the followingDMA500 and SDM303 logs have been generated.

• DMA500 NONE TBL Process StatusIn Service Trouble. DMA is unable to register with DMI.

• SDM303 MINOR TBL SDM Base Maintenance Package:SDM_DMA.dmaProcess: mtcappTrouble condition assertedReason: DMA is unable to register with BMI

You access the DMA500 log on the directory /var/adm/custlog on the Communication Server 2000 CoreManagerCommunication Server 2000 Core Manager console.

You see the SDM303 log by logging in to the CommunicationServer 2000 Core ManagerCommunication Server 2000Core Manager as a maint or root user, accessing thesdmmtc;mtcsdmmtc;mtc level, and inputting the querysdmfltquerysdm flt command.

2 Contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 730: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

730 Communication Server 2000 management tools troubleshooting

Troubleshooting a failure to add a fourth MediaGateway 15000 to a GWC

Use this procedure to troubleshoot a failure to add a fourth Media Gateway15000 to a GWC.

Procedure steps

Step Action

1 The typical reason for a failure to add a fourth Media Gateway15000 is that the first three Media Gateway 15000s were addedas media gateways of type PVG15K. The PVG15K mediagateway type reserves 1120 endpoints for each Media Gateway15000. A GWC can only support a total of 4096 endpoints forall of the gateways with which the GWC is associated (4096 - (3x 1120) = 736). Since a PVG15K media gateway type requires1120 endpoints, and only 736 are available, the fourth MediaGateway 15000 could not be added.

You have the option of choosing from three Media Gateway15000 types:

• PVG15K

• PVG15K_1000

• PVG15K_PARTIAL

PVG15K requires 1120 available endpoints.

PVG15K_100 requires 1000 available endpoints.

PVG15K_PARTIAL requires 624 available endpoints.

2 To fix the problem, add the fourth Media Gateway 15000 as aPVG15K_PARTIAL since there are only 736 endpoints available.

3 If you are able to add the fourth Media Gateway 15000, you havecompleted this procedure.

4 If you are unable to add the fourth Media Gateway 15000,contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 731: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 731

Troubleshooting a failure to perform maintenanceactions while using the Communication Server 2000Management Tools

Use this procedure to troubleshoot a failure to perform maintenanceactions while using the Communication Server 2000 Management Toolswhen the XA-Core does not respond.

Procedure steps

Step Action

1 Determine if there is any response from the XA-Core. If theXA-Core responds, you have completed this procedure;otherwise, go to the next step.

2 You are experiencing these problems because the commonlanguage identifier (CLLI) name of the CM (the host name ofthe Communication Server 2000 Core ManagerCommunicationServer 2000 Core Manager) is incorrect in the configuration dataof the SESM server application.

For more information about Setting the MSC server 1000Communication Server 2000 CLLI on the Sun server to correctthe CLLI name, see ATM/IP solution-level ConfigurationManagement (NN100409-500).

3 Additionally, you need to add the Communication Server2000 Management Tools server as a DDMS client of theCommunication Server 2000 Core ManagerCommunicationServer 2000 Core Manager.

4 If you are now able to perform maintenance activities using theCommunication Server 2000 Management Tools and that theXA-Core responds, you have completed this procedure.

5 If you are unable to perform maintenance activities or theXA-Core does not respond, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 732: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

732 Communication Server 2000 management tools troubleshooting

Troubleshooting a failure to launch CommunicationServer 2000 SAM21 Manager, CommunicationServer 2000 Management Tools GUI, or NetworkPatch Manager

Use this procedure to troubleshoot a failure to launch the CommunicationServer 2000 SAM21 Manager, the Communication Server 2000Management Tools GUI, or the Network Patch Manager from theApplication Launch Point applet screen.

After failing to launch one of these GUIs, you may receive one of thefollowing error messages:

• SAM21 client must exit

• SESM is down, unreachable or incorrect version

• Check Apache server status (servquery -status -groupWEBSERVER).Check Apache server status (/etc/init.d/apache status).Please correct the problem before logging in through SESM

Procedure steps

Step Action

1 Determine if you received one of the following error messages:

• SAM21EM client must exit

• SESM is down, unreachable or incorrect version

• Check Apache server status (/etc/init.d/apache status).Please correct the problem before logging in through SESM

• Check Apache server status (servquery -status -groupWEBSERVER).

If you did not receive any of these messages, you havecompleted the procedure; otherwise, go to the next step.

2 Restart the Apache Web Server.

See “Restarting the Apache Web server” (page 248).

3 Wait a minimum of 10 minutes, and then attempt to re-launchto GUIs.

If you are able to launch the GUIs, you have completed theprocedure.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 733: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 733

4 If you are unable to launch the GUIs, contact your next level ofsupport.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 734: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

734 Communication Server 2000 management tools troubleshooting

Diagnosing problems with a GWC card that cannotbe booted

Use this procedure to diagnose problems with a GWC card that isinstalled in the SAM21 shelf but does not appear on the CommunicationServer 2000 SAM21 Manager shelf view and it cannot be booted.

Prerequisites• You must have root user access to the Communication Server 2000

Core Manager or Communication Server 2000 Core Manager or CBMCore and Billing Manager (CBM) console.

Procedure steps

Step Action

1 At the Communication Server 2000 Core Manager orCBMCommunication Server 2000 Core Manager or CBMconsole

Logon to the Communication Server 2000 Core Manager orCBMCommunication Server 2000 Core Manager or CBM as theroot user.

2 Start the Communication Server 2000 Core Manageror Communication Server 2000 Core Manager or CBMmaintenance interface. Enter:

sdmmtcsdmmtc

OR

cbmmtc

The system prompt changes to >.

3 Access the applications (APPL) level of the CommunicationServer 2000 Core Manager or CBM maintenance interface andverify that the Bootp Loading Service and File Transfer Serviceapplications are in service (.). Enter:

Access the applications (APPL) level of the CommunicationServer 2000 Core Manager or CBM maintenance interface.Enter:

appl

Example system response:

Example system response:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 735: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 735

In SDM, the application "Bootpd and tftpd" is an optionalapplication and may not be installed on your system. In this caseit would not be visible in the APPL level menu. In CBM, "Bootpdand tftpd" does not have a presence on the maintenanceinterface APPL level.

In SDM, the application "Bootpd and tftpd" is an optionalapplication and may not be installed on your system. In this caseit would not be visible in the APPL level menu. In CBM, "Bootpdand tftpd" does not have a presence on the maintenanceinterface APPL level.

4 Verify that the Bootp Loading Service and File Transfer Serviceapplications are in service (.).

5 If these applications are not in service, busy (Bsy) then returnto services (RTS) the applications. For information aboutbusying applications and returning them to service, see NortelCommunication Server 2000 Core Manager Administration and

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 736: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

736 Communication Server 2000 management tools troubleshooting

Security (NN10170-611) or Nortel Core and Billing Manager 800Security and Administration (NN10362-611).

If these applications are in service (.), check for bootpd and tftpdmessages in the /var/adm/syslog and /var/adm/daemon.log.

If these applications are in service (.), check for bootpd and tftpdmessages in the /var/adm/syslog and /var/adm/daemon.log.

Unless log entries relating to application problems weregenerated, no log file exists for daemon.log.

6 Verify that the GWC card has power by looking for the lightedyellow or green LEDs on its faceplate.

7 Use a VT100 terminal or a PC with terminal application softwareto connect to the DB9 serial port on the faceplate of the GWCcard.

Use a standard straight through serial cable, rather than a nullmodem cable.

8 Configure the PC software to set the PC serial port to 9600baud, 8 bits, no parity, 1 stop bit.

9 Start the terminal application and select a direct connection fromCOM1.

10 Press and hold the reset button on the faceplate of the GWCcard for 5 seconds.

11 Monitor the boot process on the terminal.

If the boot fails, check for the error number and reference it toTable 61 "Possible boot errors" (page 736).

--End--

Job aidTable 61Possible boot errors

Error ID Reason text

0500 TFTP retry time out.

The following problems could exist:

• network has too much traffic

• the Communication Server 2000 Core Manager orCBMCommunication Server 2000 Core Manager orCBMis busy

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 737: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Job aid 737

Error ID Reason text

• the tftp daemon is not running

• the load name was entered incorrectly

0600 BOOTP retry time out.

The following problems could exist:

• network has too much traffic

• the Communication Server 2000 Core Manager orCBMCommunication Server 2000 Core Manager orCBM is busy

• the bootp daemon is not running

• the /etc/bootptab file is incorrectly configured

8100 The load file on the Communication Server 2000Core Manager or CBMCommunication Server 2000Core Manager or CBM has the wrong path, the wrongpermissions, or the wrong load name.

0020 Message CRC errors. The network could be busy andcausing traffic errors.

0017 10baseT link failure. Verify that the Ethernet cable isfully seated in the faceplate and the router.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 738: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

738 Communication Server 2000 management tools troubleshooting

Troubleshooting problems arising when an SSLcertificate is changed and the CM hostname is newor changed

Use this procedure to troubleshoot problems arising when a secure socketlayer (SSL) certificate is changed and the computing module (CM) hostname (CM CLLI) is new or changed.

Use this troubleshooting procedure if:

• A temporary SSL certificate is installed with an Internet Protocol (IP)address instead of a CM host name (CLLI).

• An SSL certificate is installed using the CM host name, but afterwards,the host name is changed.

Procedure steps

Step Action

1 This situation may arise when a temporary SSL certificate isinstalled with an IP address instead of a CM host name, but theCM host name is subsequently changed.

2 Configure the SESM server application with the correct CMhostname.

See “Configuring the SESM Server Application” (page 345).

3 If the problem has been solved, you have completed thisprocedure.

4 If the problem persists, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 739: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 739

Troubleshooting combined Communication Server2000 Management Tools failures

Use this procedure to troubleshoot combined failures

• to start the Succession Element and Sub-Network Manager (SESM)server application

• to apply a patch to SESM

• to associate gateways and add endpoints using Communication Server2000 Management Tools GUIs

Procedure steps

Step Action

1 Determine if the /opt/ directory on the Communication Server2000 Management Tools server is full.

See “Checking the occupancy of the /opt/ directory on theCommunication Server 2000 Management Tools server” (page249).

2 If the /opt/directory is not full, you have completed thisprocedure; otherwise, go to the next step.

3 The /opt/ directory is full because the Oracle system passwordhas expired. Correct the problem by setting a new Oracle systempassword.

See “Changing the Oracle user password on an SPFS-basedserver” (page 372).

4 If the original problem has been corrected, you have completedthis procedure.

5 If the problem persists, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 740: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

740 Communication Server 2000 management tools troubleshooting

Troubleshooting a failure to delete a line from amedia gateway in the IAC or Intl IAC solution

Use this procedure to troubleshoot a failure to delete a line from a MediaTerminal Adapter (MTA0 media gateway in the Integrated Access-Cable(IAC) or International (Intl) IAC solution.

This problem occurs when you attempt to delete a line from an IACor Intl IAC MTA (with the OUT command) in OSSGate with enhancedSERVORD commands, using the line equipment number (LEN) instead ofthe endpoint information. In OSSGate, with enhanced SERVORD (alsoreferred to as SERVORD+) commands the media gateway name andthe termination point name, together define the endpoint. In OSSGate,the endpoint has replaced the LEN. Note, that the terminal point namealone may not be unique in the Communication Server 2000, but themedia gateway name and terminal point pair together uniquely describe asubscriber line. In OSSGate, when provisioning lines using SERVORD+commands, you must always use the endpoint information (terminal pointname, and media gateway name) rather than the LEN.

If you attempt to delete a line in OSSGate with SERVORD+ commandsusing a LEN instead of an endpoint, you create a data mismatch betweenthe XA-Core and the Communication Server 2000 Management Toolsdata. In the XA-Core, the system deletes the line from table LNINV andcancels service on the LEN. However, in the Communication Server 2000Management Tools, the endpoint data for the line will still exist.

After this data mismatch occurs, you receive an error message statingthat the ÔMG has provisioned endpoints. In addition, using the GatewayController (GWC) Manager client graphical user interface (GUI) you willsee that the deleted line is still associated with the media gateway.

Procedure steps

Step Action

1 Determine the data mismatch between the XA-Core and theCommunication Server 2000 Management Tools data. In theXA-Core, the line is elected, but in the Communication Server2000 Management Tools data, the endpoint data for the line stillexists.

2 To correct the problem, first add the LEN information backinto the XA-Core. For more information about adding LENinformation back into the XA-core, see Nortel ATM/IPSolution-Level Fault Management (NN10408-900).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 741: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 741

3 In OSSGate, delete the line using endpoint information.

For more information about deleting a line, see Nortel ATM/IPSolution-Level Fault Management (NN10408-900).

4 If you have corrected the original problem, you have completedthis procedure.

5 If the trouble persists, contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 742: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

742 Communication Server 2000 management tools troubleshooting

Troubleshooting a failed attempt to input the SDNAcommand in OSSGate

Use this procedure to troubleshoot a failed attempt to input a SERVORD+command to set up directory number (DN) attributes (SDNA).

ATTENTIONThe term SERVORD+ means enhanced SERVORD. You input enhancedSERVORD commands in OSSGATE.

Use this task if you receive the following error messages after the SDNAcommand fails:

• Unable to allocate new store

• Update to table DNGRPS failed

• Journal File Record Not Created

Procedure steps

Step Action

1 The reason the SDNA command failed is because tableDNGRPS in the XA-Core is full.

Correct the problem by removing unused entries in tableDNGRPS.

For more information about using the table editor to deletea tuple from a table, see Nortel ATM/IP Solution-Level FaultManagement (NN10408-900).

2 If you are now able to input the SDNA command, you havecompleted this procedure.

3 If you are unable to input the command, contact your next levelof support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 743: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 743

Troubleshooting a failure to add an endpoint to aGWC during line provisioning of the IAC or Intl IACsolution

Use this procedure to troubleshoot a failure to add an endpoint to aGateway Controller (GWC) during line provisioning of the IAC or Intl IACsolution.

This failure to add a line endpoint occurs when you attempt to input theNew command in OSSGate. Use this task, if after your failed attempt, thesystem displays the error message: "System: LineProv: EndPoint can notbe added to GWC."

In addition, log PM181 is generated with the information that ÔStatic dataupdate Failed–Reason: Failed to Send Static Data.’

OSSGate provides the interface through which you input enhancedSERVORD commands. The term SERVORD+ means enhancedSERVORD. When you successfully provision a line in OSSGate using theNew command, the system datafills a tuple in table LNINV for the logicalgroup node (LGRP) line.

Procedure steps

Step Action

1 The boolean xpm_supports_dynamic_sd in the XA-Core iscorrupted so that it now contains a value of false. This valuecauses the system to block downloads of static data.

2 Contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 744: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

744 Communication Server 2000 management tools troubleshooting

Troubleshooting a failed attempt to create a GWCimage from the Communication Server 2000Management Tools GUI

Troubleshoot a failed attempt to create a GWC image from theCommunication Server 2000 Management Tools GUI.

Procedure steps

Step Action

1 Follow the procedure “Troubleshooting a failed attempt to createa GWC image” (page 694) and try again to create the image.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 745: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 745

Troubleshooting APS logon problems

This procedure enables you to identify and solve the following commonproblems that prevent you from logging in to the APS:

• URL in the browser address window is incorrect

• keyboard "Caps Lock" is on

• Oracle database is down

• web server is not running

• APS application software was removed or is not installed

Procedure steps

Step Action

At your console1 Verify that the URL in your browser address window is correct.

The URL will be of the format: http://<hostname or IP addressof the APS>:8080/aps/

If the URL is correct, go to step step 3.

OR

If the URL is incorrect, correct the URL entry in the browser. Goto the next step.

2 Try to log on to the APS.

If you are able to log on to the APS, you have completed thisprocedure.

If you are unable to log on to the APS, go to the next step.

3 Ensure that "Caps Lock" is not enabled on your keyboard. Ifthe "Caps Lock" is enabled, Press the "Caps Lock" key on yourkeyboard, and go to the next step.

OR

If "Caps Lock" is not enabled, go to step step 5.

4 Try to log on to the APS.

If you are able to log on, you have completed this procedure.

If you are unable to log on to the APS, go to the next step.

5 In a Telnet connection to the APS server

Open an xterm window and log on using the "maint" logon andpassword.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 746: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

746 Communication Server 2000 management tools troubleshooting

6 Become the "root" user by entering:

su - root

7 If the APS logon page is accessible, go to the next step.

OR

If the APS logon page is not accessible, go to step step 20.

8 Ensure that the Oracle database is online by entering thefollowing command:

/opt/servman/bin/servman query -status -g DATABASE -v

servquery -status -g DATABASE -v

The display indicates that the Oracle processes, listed at theend of the display (that is, entries in the display that begin with"oracle <pid>"), are running.

If the displayed Oracle processes are not running, go to the nextstep.

OR

If the displayed Oracle processes are running, go to step step14.

9 Restart the Oracle database by entering the following command:

/opt/servman/bin/servstart DATABASE

servstart DATABASE

10 Kill the APS server process and let the server restartautomatically, by entering the following command:

/opt/uas/aps/scripts/killDbServer.sh

servrestart APS

A message appears indicating that the server is restarting.

11 Enter the following command to check the status of thedatabase:

/opt/servman/bin/servman query -status -g DATABASE -v

servquery -status -g APS

The display indicates that the Oracle processes, listed at theend of the display (that is, entries in the display that begin with"oracle <pid>"), are running.

12 If the displayed Oracle processes are running, go to the nextstep.If the APS is running, go to the next step.

OR

If the displayed Oracle processes are not running, contact yournext level of support.If the APS is not running, contact your nextlevel of support.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 747: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 747

13 Try to log on to the APS.

If you are able to log on, you have completed this procedure.

If you are unable to log on to the APS, go to the next step.

14 Ensure that you can connect to the Oracle database by enteringthe following command:

sql

An "sql" prompt appears.

If the sql prompt does display, go to step step 27; otherwise, goto the next step.

15 Determine if you already restarted the Oracle database onceduring this procedure. If you have already restarted the databaseonce before, contact your next level of support. If you have notalready restarted the database once before, go to the next step.

16 Restart the Oracle database by entering the following command:

/opt/servman/bin/servstart DATABASE

17 Kill the APS server process and let the server restartautomatically, by entering the following command:

/opt/uas/aps/scripts/killDbServer.sh

A message appears to inform that the server is restarting. Thismay take from 2 to 5 minutes.

18 Enter the following command to check the status of thedatabase:

/opt/servman/bin/servman query -status -g DATABASE -v

The display indicates that the Oracle processes, listed at theend of the display (that is, entries in the display that begin with"oracle <pid>"), are running.

If the displayed Oracle processes are not running, contact yournext level of support; otherwise, go to the next step.

19 Try to log on to the APS. If you are able to log on, you havecompleted this procedure.

If you are unable to log on to the APS, contact your next levelof support.

20 If the correct IP address of the APS server is entered in thebrowser address window, but the logon page is not accessible,an Application Launch Point page appears.

If the Application Launch Point page appears, go to step step 22.

If the Application Launch Point page does not display, go to thenext step.

21 Enter the following command to start the Apache server:

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 748: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

748 Communication Server 2000 management tools troubleshooting

/opt/servman/bin/servstart WEBSERVICES

Messages that indicate the Apache server has started display.

22 Verify that the APS software packages have been installed byentering the following command:

pkginfo | grep aps

If a list of the required APS software packages displays, go tothe next step.

OR

If a list of the required APS software packages does not display,you will need to install the required APS packages. Contact yournext level of support.

23 Enter the following command to check the status of thedatabase:

/opt/servman/bin/servman query -status -g DATABASE -v

The display indicates that the Oracle processes, listed at theend of the display (that is, entries in the display that begin with"oracle <pid>"), are running.

If the displayed Oracle processes are running, go to step step27.

OR

If the displayed Oracle processes are not running, go to the nextstep.

24 Restart the Oracle database by entering the following command:

/opt/servman/bin/servstart DATABASE

25 Kill the APS server process and let the server restartautomatically, by entering the following command:

/opt/uas/aps/scripts/killDbServer.sh

A message appears to inform you that the server is restarting.This may take from 2 to 5 minutes.

26 Enter the following command to check the status of thedatabase:

/opt/servman/bin/servman query -status -g DATABASE -v

The display indicates that the Oracle processes, listed at theend of the display (that is, entries in the display that begin with"oracle <pid>"), are running.

If the displayed Oracle processes are running, go to the nextstep.

OR

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 749: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Procedure steps 749

If the displayed Oracle processes are not running, contact yournext level of support.

27 Try to log on to the APS. If you are able to log on to the APS,you have completed this procedure.

28 If you are unable to log on to the APS, it may be necessary toreboot the server. Contact your next level of support.

--End--

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 750: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

750 Communication Server 2000 management tools troubleshooting

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 751: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

751.

Communication Server 2000Management Tools geographicsurvivability reference

This section contains information about the geographic survivability of thecommunication Server 2000 Management Tools

Navigation

• “Logs and operational measurements supported by CEM” (page 752)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 752: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

752 Communication Server 2000 Management Tools geographic survivability reference

Logs and operational measurements supported byCEM

LogsFor logs generated by the core and sent to the CEM and then to the IEMS,see the following table.

Table 62Core logs sent to CEm and IEMS

Log numbersDocumentationreference

ESA120, ESA121 North American DMS-100Log Report Reference Manual(297-8021-840)

ATM300, ATM301, ATM501, ATM600,ATM605

North American DMS-100Log Report Reference Manual(297-8021-840)

XPKT806, XPKT807 Nortel Carrier Voice overIP Fault ManagementLogs Reference Volume 2(NN10275-909v2)

BITS300, BITS310, BITS500, BITS600,BITS601, BITS610

Nortel Carrier Voice overIP Fault ManagementLogs Reference Volume 2(NN10275-909v2)

BOOT101 North American DMS-100Log Report Reference Manual(297-8021-840)

C7UP104, C7UP109, C7UP113, C7UP114,C7UP120, C7UP130, C7UP310

CDMA/TDMA Logs ReferenceManual, (411-2131-510)

CARR300, CARR310, CARR320, CARR330,CARR331, CARR340, CARR341, CARR500,CARR501, CARR510, CARR511, CARR512,CARR800, CARR801, CARR811

CDMA/TDMA Logs ReferenceManual (411-2131-510)

CCMT301, CCMT501, CCMT502, CCMT601 North American DMS-100Log Report Reference Manual(297-8021-840)

DPTM500, DPTM501, DPTM502, DPTM503,DPTM504, DPTM550, DPTM560, DPTM700,DPTM701

North American DMS-100Log Report Reference Manual(297-8021-840)

ENET308, ENET311 North American DMS-100Log Report Reference Manual(297-8021-840)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 753: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Logs 753

Table 62Core logs sent to CEm and IEMS (cont’d.)

Log numbersDocumentationreference

EXT102, EXT108 North American DMS-100Log Report Reference Manual(297-8021-840)

IWBM500, IWBM501, IWBM603, IWBM800,IWBM801, IWBM802, IWBM803

Nortel Carrier Voice overIP Fault ManagementLogs Reference Volume 2(NN10275-909v2)

LINK300 Nortel Carrier Voice overIP Fault ManagementLogs Reference Volume 2(NN10275-909v2)

NODE300, NODE303, NODE323, NODE326,NODE450, NODE500, NODE600

GSM NSS/UMTS VCNOAM Reference Manual,(411-8111-511)

Nortel Carrier Voice overIP Fault ManagementLogs Reference Volume 3(NN10275-909v3)

QCA201, QCA202, QCA203, QCA300,QCA301, QCA302, QCA305, QCA310,QCA315, QCA322, QCA399

Nortel Carrier Voice overIP Fault ManagementLogs Reference Volume 3(NN10275-909v3)

SDM300, SDM301, SDM302, SDM303,SDM306, SDM308, SDM309, SDM314,SDM315, SDM317, SDM321, SDM327,SDM500, SDM501, SDM502, SDM503,SDM504, SDM505, SDM550, SDM600,SDM601, SDM602, SDM603, SDM604,SDM608, SDM609, SDM617, SDM621,SDM627, SDM630, SDM650, SDM700

Nortel Carrier Voice overIP Fault ManagementLogs Reference Volume 2(NN10275-909v2)

SDMB300, SDMB310, SDMB315, SDMB316,SDMB320, SDMB321, SDMB350, SDMB355,SDMB360, SDMB365, SDMB367, SDMB375,SDMB380, SDMB390, SDMB400, SDMB530,SDMB531, SDMB550, SDMB600, SDMB610,SDMB615, SDMB620, SDMB621, SDMB625,SDMB650, SDMB655, SDMB660, SDMB675,SDMB680, SDMB820

Nortel Carrier Voice overIP Fault ManagementLogs Reference Volume 2(NN10275-909v2)

SOS701, SOS702, SOS703 Nortel XA-Core ReferenceManual (297-8991-810)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 754: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

754 Communication Server 2000 Management Tools geographic survivability reference

Table 62Core logs sent to CEm and IEMS (cont’d.)

Log numbersDocumentationreference

SPM301, SPM310, SPM311, SPM312,SPM313, SPM314, SPM330, SPM331,SPM332, SPM334, SPM335, SPM336,SPM337, SPM338, SPM339, SPM340,SPM341, SPM342, SPM344, SPM350,SPM352, SPM353, SPM352, SPM355,SPM356, SPM357, SPM370, SPM370,SPM399, SPM500, SPM501, SPM510,SPM600, SPM619, SPM630, SPM637,SPM638, SPM641, SPM642, SPM644,SPM650, SPM651, SPM652, SPM653,SPM654, SPM655, SPM656, SPM657,SPM658, SPM660, SPM661, SPM625,SPM670, SPM700, SPM701, SPM702,SPM703, SPM704, SPM705, SPM706,SPM707, SPM708, SPM709, SPM710

Nortel Carrier Voice overIP Fault ManagementLogs Reference Volume 4(NN10275-909v4)

TMN301, TMN302, TMN304, TMN303,TMN304, TMN309, TMN311, TMN600,TMN601, TMN604, TMN605

Nortel Carrier Voice overIP Fault ManagementLogs Reference Volume 3(NN10275-909v3)

TRK882 North American DMS-100Log Report Reference Manual(297-8021-840)

XAC300, XAC302, XAC303, XAC304,XAC305, XAC306, XAC307, XAC308,XAC309, XAC310, XAC312, XAC320,XAC321, XAC322, XAC323, XAC324,XAC325, XAC326, XAC327, XAC329,XAC330, XAC333, XAC400, XAC413,XAC415, XAC418, XAC420, XAC600,XAC601, XAC602, XAC603, XAC604,XAC605, XAC606, XAC607, XAC608,XAC609, XAC610, XAC612, XAC613,XAC614, XAC615, XAC618, XAC619,XAC620, XAC622, XAC623, XAC624,XAC625, XAC626, XAC627, XAC628,XAC629, XAC630, XAC631, XAC632,XAC633, XAC634, XAC635, XAC640,XAC641, XAC801, XAC814

Nortel XA-Core ReferenceManual (297-8991-810)

XACP300, XACP500, XACP600 Nortel XA-Core ReferenceManual (297-8991-810)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 755: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Operational Measurements 755

Log reports CSEM300 and CSEM600Log reports CSEM300 and CSEM600 address a change of format fornorthbound events on the log feeds of IEMS when used in conjunctionwith CEM. Log reports CSEM300 and CSEM600 act as an envelope tocontain Communication Server 2000 (Communication Server 2000) andSuperNode Data Manager (SDM) logs. The logs in IEMS are encapsulatedinside the log reports CSEM300 and CSEM600 in the northbound NT STDand SCC2 feeds. Log report CSEM300 indicates alarm sets and alarmclears for these logs. Log report CSEM600 indicates INFO and unmappedlogs for these logs.

Log numbers Documentation reference

CSEM300, CSEM600 Nortel Carrier Voice over IP FaultManagement Logs Reference Volume3 (NN10275-909v3)

Suppressed logsWhen the Store and Forward service is enabled, the logs in the followingtable are disabled at the Core for the SDM/CBM log stream. These logsare not available through the Log Delivery application. For the procedureto unsuppress these logs, see Specifying the logs delivered from the CMto the core manager in Nortel Communication Server 2000 Core ManagerConfiguration (NN10104-511).

Logs that are DELREPed by the SAF service at the Core

AUTH, CAFT, CALM, CDPD, CHIP, CLFL, CPER, CPRT, CTID, DEPS, DFA,DIM, DROP, ERV, ESNF, FCEL, FRAU, FRS, GAME, HMT, IOGA, IS41, LAM,MBPG, MCD2, MCD3, MCDR, MIWF, MLNP, MTX, NCAS, NOPT, OCC, OM2,OMGA, OTAF, PCO, PVAL, RAD, RFES, RFP, RFPE, RPC, RRIF, RSSI, SMO,SMS, SMT, SPRF, SSR, TIER, TUPL, X75

Operational MeasurementsFor performance management functionality, all of Communication Server2000 operational measurements (OM) are supported. OM groups can beselected and exported in tab delimited format or switch format.

For details of Communication Server 2000 OMs that are available throughthe CEM interface, see the following documents:

• Nortel XA-Core Reference Manual (297-8991-810)

• DMS-SPM Feature Description Reference Manual (297-1771-330)

• DMS-100 Family Extended Peripheral Module OperationalMeasurements Reference Manual (297-8321-814)

• DMS-GSP Operational Measurements Reference Manual(297-2651-814)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 756: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

756 Communication Server 2000 Management Tools geographic survivability reference

• DMS SuperNode Signaling Transfer Point Operational MeasurementsReference Manual (297-8101-814)

• DMS SuperNode STP/SSP Integrated Node OperationalMeasurements Reference Manual (297-8083-814)

• Nortel Carrier Voice over IP Fault Management Logs ReferenceVolume 2 (NN10275-909v2)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 757: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

757.

Preventative maintenance for CVoIPcomponents

This section provides a list of tables used to perform preventativemaintenance for CVoIP components. Preventative maintenance is requiredon components to prevent service-impacting fault conditions.

Navigation

• “Preventative maintenance tasks” (page 758)

• “Downgrading the CS 2000 Management Tools software” (page 763)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 758: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

758 Preventative maintenance for CVoIP components

Preventative maintenance tasks

The tables are divided into the following categories:

• Table 63 "Tasks to be performed daily" (page 758)

• Table 64 "Tasks to be performed weekly" (page 759)

• Table 65 "Tasks to be performed monthly" (page 759)

• Table 66 "Tasks to be performed per office schedule or as required"(page 760)

• “Backup and restore” (page 762)

These tables list preventive maintenance tasks.

Table 63Tasks to be performed daily

Component Task Document Notes

GWC Monitor for alarms andcheck that functionscontinue withoutinterruptions

GWC Fault Management(NN10202-911)

Media Server 2000series

Monitor alarms Media Server 2000Series Fault Management(NN10328-911)

Ethernet RoutingSwitch 8600

Monitor alarms andLEDs

Ethernet Routing Switch8010co Chassis hardwarecomponents (312746-*)

MultiserviceSwitch/Media Gateway7400/15000

Monitor alarms Nortel Multiservice Switch7400/15000/20000 Faultand Performance Management: Troubleshooting(NN10600-520)

MultiserviceSwitch/Media Gateway15000

Monitoring equipmentstatus LEDs

Nortel MultiserviceSwitch 15000/20000Hardware Installation,Maintenance, and Upgrade(NN10600-130)

Multiservice Switch7400/15000

Monitoring ATMservices

Nortel Multiservice Switch7400/15000/20000ATM Fault andPerformance Management(NN10600-715)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 759: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Operational Measurements 759

Table 63Tasks to be performed daily (cont’d.)

Component Task Document Notes

MultiserviceSwitch/Media Gateway15000 or 7400

Using status indicatorLEDs and sounds

Nortel Multiservice Switch15000/20000 HardwareDescription (NN10600-120)

SAM21 SC Monitor alarms andcheck that functionscontinue withoutinterruptions

SAM21 Shelf ControllerFault Management(NN10089-911)

Session Server Monitor alarms andlogs

Session ServerFault Management(NN10332-911)

UAS Monitor alarms UAS Fault Management(NN10073-911)

XA-Core Monitor alarms Communication Server2000 Fault Management(NN10083-911)

Table 64Tasks to be performed weekly

Component Task Document Notes

GWC Inspect the LEDs onall GWC cards in yoursystem to ensure thereare no faults and thatall cards appear to befunctioning properly.

GWC Fault Management(NN10202-911)

Session Server Inspect the LEDs frontpanel of both units andensure there are nofaults indicated.

HP Carrier-GradeServer cc3310 ProductGuide, HP part number:cc3310_Product

Acquire HP guidefrom HP.comwebsite.

USP Visually inspect theLEDs

USP Security andAdministration(NN10159-611)

Table 65Tasks to be performed monthly

Component Task Document Notes

XA-Core/Call Agent Perform LEDmaintenance

XA-Core MaintenanceManual (297-8991-510)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 760: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

760 Preventative maintenance for CVoIP components

Table 66Tasks to be performed per office schedule or as required

Component Task Document Notes

Contivity VPN Switch Saving current systemconfiguration

Reference for theContivity VPN Switch(311643-*)

Perform immediately after youcomplete theconfiguration, orwhen changes aremade to the configuration/firmware.

Contivity VPN Switch Performing filemaintenance

Reference for theContivity VPN Switch(311643-*)

Perform immediately after youcomplete theconfiguration, orwhen changes aremade to the configuration/firmware.

Contivity VPN Switch Creating a recoverydisk

Reference for theContivity VPN Switch(311643-*)

Perform immediately after youcomplete theconfiguration, orwhen changes aremade to the configuration/firmware.

Core Billing Manager Monitor log files inthe /var/adm directoryfor system or securityabnormalities

no formal procedurerequired

Perform as partof scheduledmaintenance.

Communication Server2000 Core Manager

Monitor log files inthe /var/adm directoryfor system or securityabnormalities

no formal procedurerequired

Perform as partof scheduledmaintenance.

GWC Configure a recurringdata integrity audit

GWC ConfigurationManagement(NN10205-511)

Includes Line,Trunk, V5.2, andCommunicationServer 2000 dataintegrity audits.

GWC Take a manualsoftware image

GWC Upgrades(NN10196-461)

Perform afterpatches areapplied.

GWC Retrieve GWCservices alarm history

GWC Fault Management(NN10202-911)

GWC Retrieve and correlateGWC PM logs

GWC Fault Management(NN10202-911)

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 761: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Operational Measurements 761

Table 66Tasks to be performed per office schedule or as required (cont’d.)

Component Task Document Notes

GWC Retrieve and correlateSyslog logs

GWC Fault Management(NN10202-911)

MDM Clean up accumulatedlog files

Multiservice DataManager Customizationand Server Administration(241-6001-303)

This procedureis not normallynecessary becausethere is one logfile for each dayof the week andinformation can onlyaccumulate in a logfile for a one weekperiod, then the fileis automaticallyoverwritten.Should it becomenecessary, thereare two ways toclean up currentlog files: manuallyby entering acommand, orautomaticallyby running thecommand as a cronjob

Session Server Monitor GWC logsto verify connectivityhas not been lost withthe Session Serverand to identify any callprocessing problems.

GWC Fault Management(NN10202-911)

Schedule canbe adjusted tocorrespond withGWC log monitoringschedule.

Session Server Monitor Core Logs toverify that DPT trunksdo not unexpectedlygo out of service.Each outage shouldhave correspondinglogs in the SessionServer to explainwhy. The lack ofcorresponding SessionServer logs indicates aproblem.

Communication Server2000 Fault Management(NN10083-911)

Schedule canbe adjustedto correspondto XA-Corelog monitoringschedule.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 762: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

762 Preventative maintenance for CVoIP components

Table 66Tasks to be performed per office schedule or as required (cont’d.)

Component Task Document Notes

Session Server Monitor alarms andlogs

Session ServerFault Management(NN10332-911)

XA-Core/Call Agent Create a test volumeon disks

XA-Core MaintenanceManual (297-8991-510)

perform afterinstallation of newdisk in disk drivepacklet in the IOPcard

XA-Core/Call Agent Create volumes ondisks

XA-Core MaintenanceManual (297-8991-510)

perform on anunformatted orearlier formatteddisk, or performafter installation ofthe disk in the diskdrive packlet in theIOP card

Backup and restoreFor component-level backup and restore procedures, see ATM/IPSolution-level Security and Administration (NN10402-600).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 763: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Downgrading to (I)SN07 763

Downgrading the CS 2000 Management Toolssoftware

This section provides the information required to roll back a softwareupgrade and revert to the previous release of the software. Follow the linkassociated with the release you are downgrading to.

• “Downgrading to (I)SN07” (page 763)

• “Downgrading to (I)SN08” (page 765)

It is highly recommended that all console and telnet sessions used duringrollbacks, be logged to a file to assist in any necessary troubleshooting.

Downgrading to (I)SN07Follow the link according to your office configuration.

• “Rollback on Sun Netra 240 one-sever configuration” (page 763)

• “Rollback on Sun Netra 240 two-server configuration” (page 764)

Rollback on Sun Netra 240 one-sever configurationUse the table below as a checklist during the rollback of the SPFS and CS2000 Management Tools software to (I)SN07. Place a check (√) in the √column as you complete each procedure.

Table 67Downgrading a Sun Netra 240 one-server configuration to (I)SN07

Activities √ Procedures

1 Restore the file systems andOracle data from backup media.

Refer to procedure “Routing logstreams from an SPFS-basedserver to a remote host” (page218).

2 Clear the Java ™ Web Start(JWS) cache on the clientworkstation.

Refer to procedure “Clearingthe JWS Cache on a ClientWorkstation” (page 215).

Restoring the file systems and Oracle data from backup media, restoresyour system as it was before the upgrade to (I)SN09.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 764: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

764 Preventative maintenance for CVoIP components

Rollback on Sun Netra 240 two-server configurationFollow the link according to whether only one server in the cluster wasupgraded to (I)SN09, or both servers in the cluster were upgraded to(I)SN09.

• “Downgrading the only upgraded server in the cluster” (page 764)

• “Downgrading both upgraded servers in the cluster” (page 764)

Downgrading the only upgraded server in the clusterUse the table below as a checklist during the rollback of the SPFS and CS2000 Management Tools software to (I)SN07 on the only upgraded server.Place a check (√) in the √ column as you complete each procedure.

Table 68Downgrading the only upgraded server to (I)SN07

Activities √ Procedures

1 Clone the image of the serverwith the (I)SN07 software onto theother (upgraded) server.

Refer to “Cloning the image ofone node in a cluster to the othernode” (page 186).

Cloning the image of the (I)SN07 server onto the server that wasupgraded, restores the server as it was before the upgrade to (I)SN09.

Downgrading both upgraded servers in the clusterUse the table below as a checklist during the rollback of the SPFS and CS2000 Management Tools software to (I)SN07 on both upgraded servers.Place a check (√) in the √ column as you complete each procedure.

Table 69Downgrading both upgraded servers to (I)SN07

Activities √ Procedures

1 Restore the file systems andOracle data from backup media onone of the servers.

Refer to procedure “Routing logstreams from an SPFS-basedserver to a remote host” (page218).

2 Clone the image of the server youjust downgraded to (I)SN07, ontothe other server.

Refer to procedure “Cloning theimage of one node in a cluster tothe other node” (page 186).

3 Clear the Java ™ Web Start(JWS) cache on the clientworkstation.

Refer to procedure “Clearingthe JWS Cache on a ClientWorkstation” (page 215).

Restoring the file systems and Oracle data from backup media, restoresyour system as it was before the upgrade to (I)SN09.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 765: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Downgrading to (I)SN08 765

Downgrading to (I)SN08Follow the link according to your office configuration.

• “Rollback on Sun Netra 240 one-server configuration” (page 765)

• “Rollback on Sun Netra 240 two-server configuration” (page 765)

Rollback on Sun Netra 240 one-server configurationUse the table below as a checklist during the rollback of the SPFS and CS2000 Management Tools software to (I)SN08. Place a check (√) in the √column as you complete each procedure.

Table 70Downgrading a Sun Netra 240 (one-server configuration) to (I)SN08

Activities √ Procedures

1 Restore the file systems andOracle data from backup media.

Refer to procedure “Routing logstreams from an SPFS-basedserver to a remote host” (page218).

2 Clear the Java ™ Web Start(JWS) cache on the clientworkstation.

Refer to procedure “Clearingthe JWS Cache on a ClientWorkstation” (page 215).

Restoring the file systems and Oracle data from backup media, restoresyour system as it was before the upgrade to (I)SN09.

Rollback on Sun Netra 240 two-server configurationFollow the link according to whether only one server in the cluster wasupgraded to (I)SN09, or both servers in the cluster were upgraded to(I)SN09.

• “Downgrading the only upgraded server in the cluster” (page 765)

• “Downgrading both upgraded servers in the cluster” (page 766)

Downgrading the only upgraded server in the clusterUse the table below as a checklist during the rollback of the SPFS and CS2000 Management Tools software to (I)SN08 on the only upgraded server.Place a check (√) in the √ column as you complete each procedure.

Table 71Downgrading the only upgraded server to (I)SN08

Activities √ Procedures

1 Clone the image of the serverwith the (I)SN08 software onto theother (upgraded) server.

Refer to procedure “Cloning theimage of one node in a cluster tothe other node” (page 186).

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 766: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

766 Preventative maintenance for CVoIP components

Cloning the image of the (I)SN08 server onto the server that wasupgraded, restores the server as it was before the upgrade to (I)SN09.

Downgrading both upgraded servers in the clusterUse the table below as a checklist during the rollback of the SPFS and CS2000 Management Tools software to (I)SN08 on both upgraded servers.Place a check (√) in the √ column as you complete each procedure.

Table 72Downgrading both upgraded servers to (I)SN08

Activities √ Procedures

1 Restore the file systems andOracle data from backup media onone of the servers.

Refer to procedure “Routing logstreams from an SPFS-basedserver to a remote host” (page218).

2 Clone the image of the server youjust downgraded to (I)SN08, ontothe other server.

Refer to procedure “Cloning theimage of one node in a cluster tothe other node” (page 186).

3 Clear the Java ™ Web Start(JWS) cache on the clientworkstation.

Refer to procedure “Clearingthe JWS Cache on a ClientWorkstation” (page 215).

Restoring the file systems and Oracle data from backup media, restoresyour system as it was before the upgrade to (I)SN09.

Carrier VoIPNortel ATM/IP Solution-Level Fault Management

NN10408-900 07.0430 April 2009

Copyright © 2008-2009 Nortel Networks

.

Page 767: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04
Page 768: ATM IP Solution Level Fault Management CVM12 NN10408 900_07.04

Carrier VoIP

Nortel ATM/IP Solution-Level Fault ManagementCopyright © 2008-2009 Nortel NetworksAll Rights Reserved.

Printed in Canada, the United States of America, and the United KingdomRelease: (I)CVM12Publication: NN10408-900Document revision: 07.04Document release date: 30 April 2009

To provide feedback or to report a problem in this document, go to www.nortel.com/documentfeedback.

www.nortel.comLEGAL NOTICE

While the information in this document is believed to be accurate and reliable, except as otherwise expressly agreed to in writingNORTEL PROVIDES THIS DOCUMENT "AS IS" WITHOUT WARRANTY OR CONDITION OF ANY KIND, EITHER EXPRESSOR IMPLIED. The information and/or products described in this document are subject to change without notice.

Nortel, the Nortel logo, Business Made Simple and the Globemark are trademarks of Nortel.

All other trademarks are the property of their respective owners.