tdc telecom’s experience with hp network node manager (nnmi)
DESCRIPTION
TDC Telecom, a Nordic Operator, shares their experience with Network Node Manager (NNMi) for fault and performance management.TRANSCRIPT
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Discover 2013 Barcelona | December 10-12, 2013
Network Node Manager i (NNMi)success story at TDC Sweden Lars Niklasson
Tyler Peterson
Tapan Shah
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 2
Who are we?
Lars Niklasson • Senior IT Consultant, TDC
• Working with HP NNM and other network management tools for over 12 years
• Beta tester of NNMi + iSPIs
• Practicing Aikido for over 20 years
• Living together with Karin, five kids and three king pythons
Tyler Peterson • Senior Product Manager, Network Management Center (NMC), Hewlett-Packard Software
• Based in Fort Collins, Colorado, worked with HP and NNM / NNMi for 14 years
Tapan Shah • Expert Product Consultant, Hewlett-Packard (based in Bangalore, India)
• Working with HP and NNM product for almost 11 years now in different capacities and roles
• Taking care of customer deployments in EMEA and APJ regions for NNMi and iSPIs
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3
Agenda
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
TDC background
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5
Company background
TDC in Nordics
• Parent company in Denmark
• Approx. 9,300 employees
• Turnover: 26.1 billion DKK (2012)
• A Nordic network
• Coordinated Nordic SLA
• Coordinated Nordic support
TDC in Sweden
• Offices nationwide
• 830 employees
• Turnover: 3.1 billion SEK (2012)
• 100% owned by TDC Group (Denmark)
• The Nordic Region's most modern IP network
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 6
TDC's product and service portfolio
Networking
• IP VPN
• Ethernet VPN
• Internet access
• Security services
• Local networks
• Data Centre
• Support for IPv4/IPv6
• Mobile data access
Unified Communications
• Telephony access
• Number services
• Meeting services
• Company switchboards
• Videoconferencing
• Contact Centre
• Telephony applications
• Mobile applications
• Mobile telephony access
TDC
background Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 7
TDC's product and service portfolio
Operating model • Public cloud
• Private cloud
• Hybrid
• Customer managed
• Hosting (data centre)
Service management • Mobile integration
• Consultancy and integration
• Service concept
• Operating services
• Installation services
Integrated communication services • We can help you take further steps in your development by taking on overall responsibility for your
communication and helping you with other aspects such as financing.
Service Management • TDC Service Online (TSO)
• Device management
• Product support
• Energy efficiency
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 8
Company background
Products for medium-sized enterprises • TDC Scale – a modern IP Centrex service located in the cloud which supports both fixed and mobile
connections.
• TDC Mobile – a comprehensive mobile service with subscription packages for every type of corporate user.
• TDC Internet Access – tailored to both the small and medium-sized enterprise.
• IP VPN – Connects your business, scalable, high availability, short wait speeds.
• TDC Service Online – our customer portal for optimal overview and administration of your services.
• Direct sales products
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9
Customers
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 10
Partners
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 11
Example on SOC/NOC services
• IT Management Network - NNMi
• IT Management Server – NNMi, Symantec
• IT Management Telephony – NNMi + IPT iSPI
• IT Report – NNMi and iSPIs
• TDC KST Services (KST stands for Communication As Service)
– KST Telco and KST LAN/WAN
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Network topology
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 13
TDC Operator Nordic—network topology
TDC background
Network topology
HP solution Integrations Tips and
tricks
TDC Operator • Monitor TDC’s own devices and LAN As A Services
• 60 000 network devices in the Nordic countries
• Performance metrics, IP SLA and NetFlow in CPEs
• Auto-creation of trouble-tickets
• One of the largest telecom networks in the Nordic countries
• Customer support centres in all four Nordic countries
• The only major operator with a fully integrated pan-Nordic IP network
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 14
TDC Operator Nordic—network topology
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 15
TDC Integrator SE - Customer Access Network
TDC Integrator SE • Monitor customer devices and
servers/network equpiment used for deliver Integrator services
• 4 000 network devices
• 1 500 telco switches
• 500 servers which delivers TDC services to customer
• Auto-creation of trouble-tickets TDC core
MPLS-VPN
Customer 2.
Help desk Technicians, Specialist, Tekniker
Firewall
Firewall
Router/Firewall
Authentication
Logging
Dial Manager
The Function House
NOC
Customer 1.
Firewall
Router/Firewall
ISDN
PSTN Internet
Router/Firewall VPN
ISDN dial out
IDS
Back end servers, Terminal servers
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HP solution
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 17
Network Management Center
What is the solution?
HP applications used by TDC
• NNMi
• iSPI Performance for Metrics
• iSPI Performance for Traffic
• iSPI Performance for Quality Assurance
• iSPI IP Telephony
• ArcSight Logger/Express
Network Operation Center
• Fault availability performance change • Configuration compliance diagnostics • Advanced network services trending reporting
Automated
Secure Efficient
TDC background
Network topology
HP solution Integrations Tips and
tricks
HP ArcSight Tickets
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 18
Current solution deployment – Nordic (SE/NO/FI)
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 19
Current solution deployment – Nordic (SE/NO/FI)
TDC Integrator Swedish
• NNMi Unlimited
• iSPI Performance for Metrics
• iSPI Traffic
• iSPI Quality Assurance
• iSPI IP Telephony
• Global Network Manager (RMs are SE Operator and one customer NNMi)
TDC Operator Swedish
• NNMi Unlimited
• iSPI Performance for Metrics
• iSPI Traffic
• iSPI Quality Assurance
• Regional Manager (GM is Swedish Integrator NNM)
TDC Operator Norwegian
• NNMi Unlimited
• iSPI Performance for Metrics
• iSPI Traffic
• iSPI Quality Assurance
• Standalone
TDC Operator Finnish
• NNMi Unlimited
• iSPI Performance for Metrics
• iSPI Traffic
• iSPI Quality Assurance
• Standalone
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 20
Current solution deployment
Hardware sizing TDC Norway setup (the biggest)
• NNMi + iSPI QA, Metrics, Traffic Ext
– Physical HW 16 cores, 48 GB RAM, 500 GB DISK
• NPS
– Physical HW 64 cores, 96 GB RAM, 7 TB DISK
• Traffic Master + Leaf
– Physical HW 12 cores, 42 GB RAM, 100 GB DISK
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 21
Objects
TDC Operator Norway
• Nodes 18274
• SNMP Agents 17637
• Interfaces 1011753
• IP Addresses 116250
• L2Connections 19966
• Incidents 24926
TDC Operator Norway
• IP Subnets 54180
• VLAN5 36523
• Node Components 186009
• Cards 32401
• Ports 143458
• Router Redundancy Groups 1421
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 22
Polls
TDC Operator Norway
State Poller Health
• Status: Normal Collections requested in last 5 minutes: 71778 Collections completed in last 5 minutes: 71759 Collections in process: 123 Time to execute skips in last 5 minutes: O Stale collections in last 5 minutes: O Poller result queue length 5 min. avg.: 0.0 State mapper queue duration 5 min. avg.: 0 State updater queue duration 5 min. avg.: O State updater exceptions in last 5 minutes: O
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 23
Relations between systems—TDC Integrator SE
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 24
Other applications used
• EasySoft EasyCall - Proxy for email and SMS
• CheckPoint Mobile Access - Remote access for customers
• Op5 Monitor - Monitoring the NOC environment and TDC internal IT environment
• HP ArcSight Logger/Express
• Cisco JouleX Energy Manager
• Cisco Prime Infrastructure
• Symantec - Antivirus, backup, server monitoring
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 25
Services addressed by this setup
Performance analysis
• Configure NNMi to monitor performance metrics
• Monitoring and/or schedule reports weekly and monthly
• Remote access via CheckPoint Mobile Access, now working on TSO - NNMi integration for reports
Different needs – different reports
• Troubleshooting – utilization, errors, discards, CPU loads etc
• Planning – utilization, forecasts
• Burst control TDC
background Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 26
Services addressed by this setup
Traffic analysis
• Configure customers’ devices to send NetFlow to our Traffic Leaf (either placed at customer or at TDC). Depends on DNS lookup or not
• Monitoring and/or schedule reports weekly and monthly
• Remote access via CheckPoint Mobile Access
QA analysis
• Configure customers devices with IP SLA. Either via NNMi or manually
• Monitoring and/or schedule reports weekly and monthly
• Remote access via CheckPoint Mobile Access
New feature in latest NNMi
• iResponse Agent–end-to-end and capability to offer QA services to non-Cisco customers
TDC
background Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 27
Services addressed by this setup
What to think about
Pricing is relatively easy for Performance and QA but not for Traffic
• Traffic needs more analysis for correct pricing or start with POC
Remote access to the customer
• Shorter contracts use VPN over Internet
• More long term contracts use IPVPN
Security
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Integrations and other solutions
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 29
NNMi customization / automation utilities
• Homemade auto-discovery of monitoring • SNMP polling and setting CNA which Node groups definition uses and Custom poll uses
• End customers access to NNMi • CheckPoint Mobile Access to secure the access, NNMi security groups and tenant divide the environment
• Automated NNMi tenant and security modeling • Scripting the security configuration of NNMi, eliminating human errors
• Making automatic actions flexible • Using external script accessing NNMi DB collecting more data and also do different actions based on
source node
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 30
NNMi customization / automation utilities
• Trap and incident management • Regular trimming based on top contributors of traps
• Reporting on incidents in iSPIs using some db queries
• Script check top 20 contributors to traps and incidents
• Incident reports without iSPIs • Script creating and emailing report of RC incidents including duration time
• Nodes Out-of-service / Not Managed report • Script check nodes reachability with ICMP on Out-of-service/Not-Managed nodes
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Tips and tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 32
QA iSPI tuning
• Beware of bugs in RTT-MIB and CBQoS in Cisco IOS when enabling QA SPI. Some IOSs reboot when SNMP is polled in these MIBs
• Disable CBQoS and create filters for IP SLA probes if you know you have lots of implementations of these configurations. Otherwise it can create a big load on the system
• Use CNA. Create node groups base on CNA. We integrate with other system to set CNA SLA for different devices automatically. It helps the NOC to prioritize issues if lots of problems happen at the same time
• We also integrate with other systems to add CNAs to show which services this node delivers
• Or use SNMP location for grouping
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 33
Traffic SPI tuning
• Figure the flow records/minute received by each of the leaf collectors
• Recommended: One logical leaf instance per physical leaf collector
• Maximum of two logical leaf instances per physical collector
• Based on the flow records/minute, find out uniqueness of records
• Tune the flow record object pools based on the unique records/minute
• Tune the Java Heap accordingly
• Disable auto-tuner in a high-scale traffic environment
• Adjust the traffic flow batch processing size for high-scale traffic deployment
• Talk to us in case of any further details on “how-to”
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 34
NPS tuning
• Sybase main and temporary cache tuning
• Dealing with dbsize.ovpl
• ETL tuning
• Importance of I/O
• Follow NPS performance tuning whitepaper for large scale deployments
TDC background
Network topology
HP solution Integrations Tips and
tricks
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 35
Get started now ! https://hpln.hp.com/group/bsm-community-content
Check your BSM community for content!
• Free content, add value to BSM products
• Visit Expert Blogs for tips, tricks, hints, & workshops on BSM products and the solution in general
• Share your integrations, snippets, scripts, articles and everything else the community can benefit from
• Find answers, interact with community members
• Contact HP experts on BSM products and the solution in general
How can you benefit?
© Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
For more on NNM hp.com/go/nnm