Download - Flexi CO-IB Troubleshooting Guide
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
1/35
Flexi Content Optimizer 6.0
Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Approval date 2014-01-13
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
2/35
Flexi CO-IB Troubleshooting Guide Error! Use the Home tab to apply berschrift 1 to thetext that you want to appear here.
The information in this document is subject to change without notice and describes only theproduct defined in the introduction of this documentation. This documentation is intended for theuse of Nokia Solutions and Networks customers only for the purposes of the agreement under whichthe document is submitted, and no part of it may be used, reproduced, modified or transmittedin any form or means without the prior written permission of Nokia Solutions and Networks. The
documentation has been prepared to be used by professional and properly trained personnel,and the customer assumes full responsibility when using it. Nokia Solutions and Networks welcomescustomer comments as part of the process of continuous development and improvement of thedocumentation.The information or statements given in this documentation concerning the suitability, capacity,or performance of the mentioned hardware or software products are given "as is" and all liabilityarising in connection with such hardware or software products shall be defined conclusively andfinally in a separate agreement between Nokia Solutions and Networks and the customer. However,Nokia Solutions and Networks has made all reasonable efforts to ensure that the instructionscontained in the document are adequate and free of material errors and omissions. Nokia Solutionsand Networks will, if deemed necessary Nokia Solutions and Networks, explain issues whichmay not be covered by the document.Nokia Solutions and Networks will correct errors in this documentation as soon as possible. IN NOEVENT WILL NOKIA SOLUTIONS AND NETWORKS BE LIABLE FOR ERRORS IN THISDOCUMENTATION
OR FOR ANY DAMAGES, INCLUDING BUT NOT LIMITED TO SPECIAL, DIRECT, INDIRECT,INCIDENTAL OR CONSEQUENTIAL OR ANY LOSSES, SUCH AS BUT NOT LIMITEDTO LOSS OF PROFIT, REVENUE, BUSINESS INTERRUPTION, BUSINESS OPPORTUNITYOR DATA, THAT MAY ARISE FROM THE USE OF THIS DOCUMENT OR THE INFORMATIONIN IT.
NSN is a trademark of Nokia Solutions and Networks. Nokia is a registered trademark of NokiaCorporation. Other product names mentioned in this document may be trademarks of their respectiveowners, and they are mentioned for identification purposes only.
Copyright Nokia Solutions and Networks 2014. All rights reserved.
Nokia Solutions and Networks are continually striving to reduce the adverse environmental effects ofits products and services. We would like to encourage you as our customers and users to join us inworking towards a cleaner, safer environment. Please recycle product packaging and follow therecommendations for power use and proper disposal of our products and their components.
If you should have questions regarding our Environmental Policy or any of the environmental serviceswe offer, please contact us at Nokia Solutions and Networks for additional information.
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
3/35
About this document Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
3/35
Table of Contents
1 About this document ......................................................................................... 5
1.1 Scope ................................................................................................................ 5
1.2 Audience ........................................................................................................... 5
1.3 Related documents ........................................................................................... 5
2 IB Connection ................................................................................................... 6
2.1 Connection for integration mode ....................................................................... 7
2.2 Connection for standalone mode without PDN network ................................... 8
2.3 Connection for standalone mode with PDN network ........................................ 9
2.3.1 Gi and PDN network independent .................................................................... 9
2.3.2 Gi and PDN network correlating ..................................................................... 10
3 Troubleshooting .............................................................................................. 12
3.1 SCLI instruction .............................................................................................. 12
3.2 Checking the IB status .................................................................................... 13
3.3 Shutting down and starting up IB .................................................................... 13
3.4 Checking the MAC table ................................................................................. 15
3.5 Checking the arp table .................................................................................... 15
3.6 Checking the VRF configuration ..................................................................... 16
3.7 Checking the status of VRF and network-filter for network-filter function ....... 16
3.8 Checking the VLAN configuration ................................................................... 17
3.9 Checking the interface status ......................................................................... 18
3.9.1 Interface type .................................................................................................. 19
3.9.2 Interface list .................................................................................................... 19
3.10 Checking the IP address ................................................................................. 20
3.11 Checking the virtual IP configuration .............................................................. 21
3.12 Checking the SNAT IP configuration .............................................................. 21
3.13 Checking the VRF translation rule configuration ............................................ 22
3.13.1 VRF translation for virtual IP ........................................................................... 22
3.13.2 VRF translation for snat IP .............................................................................. 22
3.14 Checking the gateway .................................................................................... 23
3.15 Checking the gateway pool ............................................................................. 24
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
4/35
About this document Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
4/35
3.16 Checking the network-filter rule ...................................................................... 24
3.17 Checking the MTU for VLAN .......................................................................... 25
3.18 Checking the route table ................................................................................. 26
3.18.1 Showing the route table for internal traffic ...................................................... 26
3.18.2 Showing the route table for VRF ..................................................................... 26
3.19 Using host-ping to check the network connection .......................................... 27
3.20 Using host-traceroute to check the network connection ................................. 27
3.21 Using tcpdump to capture the packet ............................................................. 28
3.22 Checking the logs for IB .................................................................................. 28
3.22.1 nginfo command ............................................................................................. 28
3.22.2 Checking the active CLA server ..................................................................... 29
3.23 Checking the statistics for IB feature .............................................................. 29
3.23.1 Checking the CPU usage for IB instance ....................................................... 29 3.23.2 Checking the statistics for network-filtering .................................................... 30
4 Cluster log ....................................................................................................... 31
5 System log ...................................................................................................... 32
6 Alarm log ......................................................................................................... 33
6.1.1 Active alarm .................................................................................................... 33
6.1.2 History alarm ................................................................................................... 34
7 Appendix ......................................................................................................... 35
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
5/35
About this document Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
5/35
1 About this document1.1 Scope
This document contains the information about the command introduction for IB in Flexi NG2.1.
1.2 Audience
This document is for the reader to do the troubleshooting for IB configuration in Flexi NG2.1. The audience must be familiar with Flexi NG.
1.3 Related documents
Flexi NG User Guide
Flexi NG Alarms
Flexi NG Troubleshooting
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
6/35
IB Connection Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
6/35
2 IB ConnectionIn order to eliminate the traffic drop when the uplink failed, every VRF needs to set up twoVLANs for HA and load-balance.
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
7/35
IB Connection Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
7/35
2.1 Connection for integration mode
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
8/35
IB Connection Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
8/35
2.2 Connection for standalone mode without PDN
network
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
9/35
IB Connection Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
9/35
2.3 Connection for standalone mode with PDN network
2.3.1 Gi and PDN network independent
According to operators network, if Flexi CO access the Gi and PDN network through thedifferent network device, you can use the below deployment.
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
10/35
IB Connection Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
10/35
2.3.2 Gi and PDN network correlating
According to operators network, if Flexi CO can access the Gi and PDN network through thesame network device, you can use the below deployment.
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
11/35
IB Connection Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
11/35
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
12/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
12/35
3 Troubleshooting3.1 SCLI instruction
Flexi NG uses a structured command line interface (SCLI) for configuration,administration, and monitoring tasks. The SCLI shell of Flexi NG, fsclish, is an extensionof the command line interface that makes it easier to find and browse commands. WithSCLI, the user can browse available commands in a command syntax tree view usingcommand autocompletion, and accessing context sensitive help.
[root@CLA-0(Cappuccino) /root]
# f scl i sh
root@CLA-0 [Cappuccino] >
Tab displays all available parameters.
root@CLA-0 [Cappuccino] >
[X] add - add - family of commands
[X] cmdtree - show the command syntax tree
[X] commit - commit - family of commands
[X] delete - delete - family of commands
[X] Select any one parameter
[OX] Option parameter
[MX] Mandatory parameter
For the details, refer to section 4: Introduction to the structured command line interface(SCLI) in the Flexi NG Operating Documentation.
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
13/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
13/35
3.2 Checking the IB status
On CLA server, run the command of ngi nf o - s to check the output.
[root@CLA-0(Cappuccino) /root]# ngi nf o - s
Status check started: Wed May 22 17:46:21 CST 2013
* * * * * * * * * * * * * * * * * * * * * * * * *
Cluster status check:
Status Admin Oper Usage Alarm
* * * * * * * * * * * * * * * * * * * * * * * * *
/ OK
* * * * * * * * * * * * * * * * * * * * * * * * *
Node status check:
Status Admin Oper Usage Alarm
* * * * * * * * * * * * * * * * * * * * * * * * *
/CLA-0 OK
/CLA-1 OK
/IB7-0 OK
/IB7-1 OK
/IB10-0 OK
/IB10-1 OK
The OK status means the IB is running, and the NOK status means the IB has someproblem or is shut down.
3.3 Shutting down and starting up IB
On CLA server, the fshascli command can power off, power on, shut down, or start up
IB instance.# f shascl i - h
fshascli [COMMAND [OPTION...]] [MOName...]
COMMANDs:
-h, --help Print this page
Detailed description with: fshascli -h COMMAND.
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
14/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
14/35
-l, --lock Lock Managed Objects.
-u, --unlock Unlock Managed Objects.
-w, --switchover Switchover Recovery Groups and Recovery Units.
-r, --restart Restart Managed Object.
-p, --power {ON|OFF} Power on or off Nodes.
-B, --heartbeat {ON|OFF} Set heartbeating on or off for Processes.
-X, --shutdown Shutdown gracefully Managed Objects.
-i, --isolate Set a faulty node that cannot be reset through IPMI as isolated.
-v, --view View system model.
-s, --state View state and status of Managed Objects.
--bare List MONames.
--parent List MOName of parent.
--children List MONames of children.-I, --inert-mode {ON|OFF} Prevent recovery actions for nodes or the cluster.
-S, --set = Set dynamic attributes for Managed Objects.
-C, --reconfigure Request HAS to re-read and update configuration in all nodes.
--force-active Force a recovery unit that has a resource controller to become active.
--upgrade Request HAS to upgrade the specified node agent.
--list-local-processes List running processes in the local node.
--list-dependencies List RG dependencies
OPTIONs:
-n, --nowarning -a, --administrative -A, --assignments
-F, --force -o, --operational --dn
-N, --noblock -U, --usage --logerrors
-E, --noerror -P, --procedural -e, --regex
-t, --timeout -k, --unknown -V, --availability
-m, --alarm --filter -R, --role
Example
# f shascl i p of f / I B7- 0
Request will power off /IB7-0
Are you sure you want to proceed? [y/n] y
/IB7-0 is powered OFF successfully
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
15/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
15/35
3.4 Checking the MAC table
Log in to the IB instance from CLA server, and run i p maddr show command to displaythe mac table.
[root@CLA-0(Cappuccino) /root]# ssh I B7- 0FlexiPlatform Release 5[root@IB7-0(Cappuccino) /root]# ip maddr show1: lo
inet 224.0.0.1inet6 ff02::1
5: fpn0link 33:33:00:00:00:01inet6 ff02::1
19: eth2link 33:33:ff:a7:0c:ddlink 33:33:00:00:00:01inet6 ff02::1:ffa7:cdd
3.5 Checking the arp table
Log in to every IB instance from CLA server, and run ar p command to check the arp
table.Example:
[root@CLA-0(Mocha) /root]
# ssh I B7- 0
FlexiPlatform Release 5
[root@IB7-0(Mocha) /root]
# arp
Address HWtype HWaddress Flags Mask Iface
169.254.0.35 ether 00:A0:A5:74:FF:C8 C bond0
192.168.1.2 ether 00:00:50:98:52:F3 C vlan523_intGi
169.254.0.4 ether 00:A0:A5:74:FF:C8 C bond0
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
16/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
16/35
3.6 Checking the VRF configuration
The command below can be used to check how the network is configured for VRF.
[root@CLA-0(Cappuccino) /root]
# f scl i sh
root@CLA-0 [Cappuccino] > show networking vrf
default
VRFid : 0
vrfgi
VRFid : 1
vrfnbg
VRFid : 2
vrfpdn
VRFid : 3
3.7 Checking the status of VRF and network-fil ter for
network-filter function
The command below can be used to check how the network-filter can be configured forVRF.
root@CLA-0 [Cappuccino] > show ng net wor k- f i l t er vr f s
vrf = vrfgi (enabled)
vrf = default (disabled)
vrf = vrfpdn (enabled)
vrf = vrfnbg (enabled)
root@CLA-0 [Mocha] > show ng f eat ur e net wor k- f i l t er i ng
root@CLA-0 [Cappuccino] > show net worki ng vr f
[X] - press to execute the command
[OX] id - Virtual Routing Instance Id
[OX] name - Virtual Routing Instance name
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
17/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
17/35
root@CLA-0 [Cappuccino] > show net worki ng vl an
[X] - press to execute the command
[OX] admin - Interface admin state [ { up | down } ]
[OX] iface - VLAN Interface name [ IFACE-NAME ]
[OX] ipv4forwarding - IP forwarding [ { yes | no } ]
[OX] ipv4rpfilter - Reverse path filter [ { yes | no } ]
[OX] ipv6forwarding - IP forwarding [ { yes | no } ]
[OX] ipv6rpfilter - Reverse path filter [ { yes | no } ]
[OX] mapping - Existing VLAN priority map name [ { VLAN-Mapping-NAME } ]
[OX] mtu - Interface Maximum Transmission Unit [1280..9000 ]
[OX] owner - Name of the existing HAS managed object
[OX] realiface - Real interface name
[OX] vid - VLAN identifier [ 1..4094 ]
network-filter = enable
After adding the VRF, the default status for network-filter is disabled, when the networkfiltering function is related with the VRF, enabling the VRF for network-filter is necessary.
3.8 Checking the VLAN configuration
The command below can be used to check how the vlan is configured for IB part.
Command usage
root@CLA-0 [Cappuccino] > show net worki ng vl an
vlan instance vrfgi interfaces:
vlan317_gi
owner : /IB7-0
realiface : ethfront1
vid : 317
MTU : 1500
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
18/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
18/35
adminstate : up
IPv4 forwarding : yes
IPv4 rpfilter : no
IPv6 forwarding : yes
IPv6 rpfilter : no
vlan317_gi
owner : /IB7-1
realiface : ethfront4
vid : 317
MTU : 1500
adminstate : up
IPv4 forwarding : yesIPv4 rpfilter : no
IPv6 forwarding : yes
IPv6 rpfilter : no
3.9 Checking the interface status
The command below can be used to check the status of each interface.
root@CLA-0 [Mocha] > show net worki ng i nter f ace r unt i me node I B7- 0
Showing runtime status of interfaces
bond0
index : 27
node : IB7-0
type : Bond
flags : UP BROADCAST RUNNING MULTICAST INFRAMAC : 00:00:50:98:52:d8
MTU : 9000
admin state : up
oper state : up
Rx packets :
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
19/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
19/35
total : 8001903
bytes : 1016497369
error : 0
Tx packets :
total : 6066001
bytes : 1086737355
error : 0
slave : eth0:32768
eth1:49152
..
The status of oper state shows if the interface is activated, up means the interface isactivated and down means the interface is deactivated.
3.9.1 Interface type
interface type
lo1/2/ loopback interface
eth0/1/ internal interface
ethfront1/2/ external interface
bond0/1 logical inteface
vlanxxx_gi vlan interface
3.9.2 Interface lis t
[root@CLA-0(Cappuccino) /root]
# ssh I B7- 0
FlexiPlatform Release 5
[root@IB7-0(Cappuccino) /root]
# i p addr
1: lo: mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
IPv4 forwarding: on IPv6 forwarding: on
IPv4 force reassembly: no IPv6 force reassembly: no
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
20/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
20/35
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: tunl0: mtu 1480 ttl 0 tos 0x0 qdisc noop
link/ipip 0.0.0.0 brd 0.0.0.0
IPv4 forwarding: on IPv6 forwarding: on
IPv4 force reassembly: no IPv6 force reassembly: no
3.10 Checking the IP address
This command can show the IP configuration for the interface on the specific node.
root@CLA-0 [Cappuccino] > show net worki ng address
address instance default interfaces:
eth4
type : dedicated
address : 10.56.135.150/25
owner : /CLA-0
user : /PAP
/RuimReplicator
/SSH
eth4
type : dedicated
address : 10.56.135.151/25
owner : /CLA-1
user : /PAP
/RuimReplicator/SSH
address instance vrfgi interfaces:
vlan317_gi
type : dedicated
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
21/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
21/35
address : 10.56.137.44/25
owner : /IB7-0
vlan317_gi
type : dedicated
address : 10.56.137.45/25
owner : /IB7-1
3.11 Checking the virtual IP configuration
The virtual IP is used as the proxy IP address for UE. Showing the configuration of the
specific rule can list the virtual IP and related information.root@CLA-0 [Cappuccino] > show ng net wor k- f i l t er r ul e wap
*** MATCH PARAMETERS ***
id = 2
nf-rule-name = wap
action = gateway
dst-addrv4 = 10.56.137.43
dst-port = 8080
dst-port = 9200-9203precedence = 10
verdict = stop
vrf = vrfgi
3.12 Checking the SNAT IP configuration
The SNAT IP is used as the source IP when the packets sent from Flexi CO towebserver. Showing the configuration of the snat-pool can list the SNAT IP. It can be aindividual IP or IP pool.
root@CLA-0 [Cappuccino] > show ng net wor k- f i l t er snat - pool snat _pdn
id = 1
snat-pool-name = snat_pdn
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
22/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
22/35
entry = [addrv4 = 10.56.226.192/32; port-range = undefined; subnet-mask = undefined]
3.13 Checking the VRF translation rule configuration
The static translation can modify packet L3/L4 information and virtual routing andforwarding instance (VRF) based on a statically defined configuration.
3.13.1 VRF translation for virtual IP
As the below configuration shown, when the packets match parameters, the action willbe done.
root@CLA-0 [Cappuccino] > show ng net wor k- f i l t er r ul e vr f _t o_nbg
*** MATCH PARAMETERS ***id = 1
nf-rule-name = vrf_to_nbg
action = static-translate
dst-addrv4 = 10.56.137.43/32
precedence = 5
src-addrv4 = 0.0.0.0/0
verdict = next
vrf = vrfgi
*** ACTION ***
black-hole = disabled
create-flow = enabled
vrf = vrfnbg
3.13.2 VRF translation for snat IP
root@CLA-0 [Cappuccino] > show ng net wor k- f i l t er r ul e vr f _t r ansl at e
*** MATCH PARAMETERS ***
id = 10
nf-rule-name = vrf_translate
action = static-translate
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
23/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
23/35
precedence = 40
src-addrv4 = 198.18.0.0/16
verdict = next
vrf = vrfnbg
*** ACTION ***
black-hole = disabled
create-flow = enabled
vrf = vrfpdn
3.14 Checking the gateway
Using the gateway, Flexi NG can redirect packet by IPv4 destination address based onload balancing algorithms.
root@CLA-0 [Cappuccino] > show ng net wor k- f i l t er gateways
app1 (guid = 1)
app2 (guid = 2)
app3 (guid = 3)
app4 (guid = 4)
root@CLA-0 [Cappuccino] > show ng net work- f i l t er gateway app1id = 1
gateway-name = app1
addrv4 = 198.18.16.1
health-check-icmp-echo = enabled
health-check-vrf = vrfnbg
gateway-pool = radius (capacity = 1)
gateway-pool = wap (capacity = 1)
gateway-pool = ppg (capacity = 1)
gateway-pool = accounting (capacity = 1)
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
24/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
24/35
3.15 Checking the gateway pool
Check how the gateway pool is configured and applied with rules, so the traffic can beredirected to gateway according to the configuration.
root@CLA-0 [Cappuccino] > show ng net wor k- f i l t er gateway- pool s
radius (guid = 1)
wap (guid = 2)
ppg (guid = 3)
accounting (guid = 4)
root@CLA-0 [Cappuccino] > show ng net wor k- f i l t er gateway- pool wap
id = 2
gw-pool-name = wap
health-check-string = Traffic
load-balancing-mode = w-con
pool-type = ipv4
gateway = app1 (capacity = 1)
gateway = app2 (capacity = 1)
gateway = app3 (capacity = 1)
gateway = app4 (capacity = 1)
3.16 Checking the network-fil ter rule
The Rules can define the behavior of the network filtering functionality by providingfiltering parameters which should match attached actions to be applied on incomingpackets.
root@CLA-0 [Cappuccino] > show ng net wor k- f i l t er r ul e ppg
*** MATCH PARAMETERS ***
id = 14nf-rule-name = ppg
action = gateway
dst-addrv4 = 10.56.137.43
dst-port = 5080
precedence = 70
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
25/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
25/35
protocol = tcp
verdict = stop
vrf = vrfgi
*** ACTION ***
gateway-pool = ppg
send-original-destination = disabled
session-persistence = disabled
3.17 Checking the MTU for VLAN
In interface blade deployment, MTU values of internal access and packet data networkinterfaces must match the MTU values of the corresponding external interfaces.
root@CLA-0 [Cappuccino] > show net worki ng vl an owner / I B7- 0
vlan instance vrfgi interfaces:
vlan317_gi
owner : /IB7-0
realiface : ethfront1
vid : 317
MTU : 1500adminstate : up
IPv4 forwarding : yes
IPv4 rpfilter : no
IPv6 forwarding : yes
IPv6 rpfilter : no
vlan instance vrfnbg interfaces:
vlan4003_nbg
owner : /IB7-0
realiface : bond1
vid : 4003
MTU : 9000
adminstate : up
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
26/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
26/35
IPv4 forwarding : yes
IPv4 rpfilter : no
IPv6 forwarding : yes
IPv6 rpfilter : no
3.18 Checking the route table
Routing is a basic gateway functionality that allows selecting paths in a network to senddata.
3.18.1 Showing the route table for internal traffic
root@CLA-0 [Cappuccino] > show r out i ng node I B7- 0 r out e
Codes: C - Connected, S - Static, I - IGRP, R - RIP, B - BGP, O - OSPF
E - OSPF external, A - Aggregate, K - Kernel Remnant, H - Hidden
P - Suppressed
C 169.254/24 is directly connected bond0
C 169.254.1/24 is directly connected bond1
3.18.2 Showing the route table for VRF
root@CLA-0 [Cappuccino] > show r out i ng i nst ance vr f gi node I B7- 0route
Codes: C - Connected, S - Static, I - IGRP, R - RIP, B - BGP, O - OSPF
E - OSPF external, A - Aggregate, K - Kernel Remnant, H - Hidden
P - Suppressed
O E 0.0.0.0/0 via 10.56.137.1 vlan317_gi cost 1 age 239140
O E 10.0.0.172/32 via 10.56.137.2 vlan317_gi cost 20 age 239140
O E 10.0.0.182/32 via 10.56.137.2 vlan317_gi cost 20 age 239140
O E 10.50/24 via 10.56.137.2 vlan317_gi cost 20 age 239140
O E 10.56.133.184/29 via 10.56.137.2 vlan317_gi cost 20 age 239140
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
27/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
27/35
O E 10.56.134/24 via 10.56.137.2 vlan317_gi cost 20 age 239140
O E 10.56.134.3/32 via 10.56.137.2 vlan317_gi cost 20 age 239140
O E 10.56.134.50/32 via 10.56.137.2 vlan317_gi cost 20 age 239140
O E 10.56.134.201/32 via 10.56.137.2 vlan317_gi cost 20 age 239140
O E 10.56.134.202/32 via 10.56.137.2 vlan317_gi cost 20 age 239140
..
3.19 Using host-ping to check the network connection
Host-ping command can show the IP connectivity status between Flexi NG and aNetwork Host.
root@CLA-0 [Mocha] > host - pi ng node- name I B7- 0 sour ce- vr f 3 sour ce-i nt er f ace198. 18. 19. 243 dest i nat i on- addr ess 198. 18. 16. 1
PING 198.18.16.1 (198.18.16.1) from 198.18.19.243 : 56(84) bytes of data.
64 bytes from 198.18.16.1: icmp_seq=1 vrfid=3 ttl=64 time=0.241 ms
64 bytes from 198.18.16.1: icmp_seq=2 vrfid=3 ttl=64 time=0.219 ms
3.20 Using host-traceroute to check the network
connection
Display the route that the IP packets take from Flexi NG to a network host.
root@CLA-0 [Mocha] > host - t r acer out e node- name I B7- 0 sour ce- vr f 2sour ce- i nt er f ace vl an317_Gi dest i nat i on- addr ess 1. 1. 1. 1
traceroute to 1.1.1.1 (1.1.1.1), 30 hops max, 38 byte packets
1 10.56.137.2 (10.56.137.2) 0.747 ms 1.391 ms 0.391 ms
2 10.56.137.1 (10.56.137.1) 0.719 ms 0.771 ms 0.373 ms
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
28/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
28/35
3.21 Using tcpdump to capture the packet
Log in to all the IB instances, and the t cpdumpcommand to capture the packet withspecific parameter.
t cpdump - n - i - s 9000 por t host - w/ l ocat i on/ *. cap
Example: capture the http packets for vrfgi on IB7-0
# t cpdump n - i vl an317_Gi - s0 por t 8080
listening on vlan317_Gi, link-type EN10MB (Ethernet), capture size 65535 bytes
To enable the tcpdump in fastpath, enter the following command:
f sf ast pat h set - t ap- enabl e on
This command enables user to view fastpath traffic in tcpdump. However, it degrades
the system performance and must be disabled after completing the debugging session.To disable the tcpdump in fastpath, enter the following command:
f sf ast pat h set - t ap- enabl e of f
3.22 Checking the logs for IB
3.22.1 nginfo command
You can use the symptom data collection tool (nginfo) to collect log files, core files,miscellaneous system information (for example configurations, alarms,software/hardware information), and node-local runtime information that providesymptom data for Customer Interaction Team.Usage:
- h: print command options- v: print delivery information- s: perform status check- c: exclude core files
- l : exclude log files- f: exclude tar and leave the result files into directory- x: exclude platform information- b: delete core files after collection- d: yyyy-mm-dd :get only files changed after given date- t : hh:mm :additional parameter for -d option
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
29/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
29/35
The default directory of the collection information is /var / l og, the result file format:ngi nf o_ddmmyy_hmmss. t gz.
3.22.2 Checking the active CLA server
The CLA server works on Host active/standby redundancy model. If you want to check thelogs, you should judge which CLA server is active. The below command can help you tocheck it.
root@CLA-0 [Mocha] > show has st at e managed- obj ect / CLA-0/ FSCl ust er St at eSer ver /CLA-0/FSClusterStateServer:administrative(UNLOCKED)operational(ENABLED)usage(ACTIVE)procedural()
availability()unknown(FALSE)alarm()role(ACTIVE)
root@CLA-0 [Mocha] > show has st at e managed- obj ect / CLA-1/ FSCl ust er St at eSer ver /CLA-1/FSClusterStateServer:administrative(UNLOCKED)operational(ENABLED)usage(ACTIVE)procedural()
availability()unknown(FALSE)alarm()role(HOTSTANDBY)
3.23 Checking the statist ics for IB feature
3.23.1 Checking the CPU usage for IB instance
root@CLA-0 [Mocha] > show st at s dat a cur r ent omes- i d 2002 nameFPNODE- I B7- 0/ FPCPU- SUMOverall CPU Usage(local_m02002c0001) : 9CPU Usage Cumulative elapsed time(local_m02002c0002) : 5760Minimum CPU Usage(m02002c0003) : 1
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
30/35
Troubleshooting Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
30/35
Maximum CPU Usage(m02002c0004) : 25Average CPU usage(m02002c0005) : 5CPU Usage Cumulative idle time(local_m02002c0006) : 5488
3.23.2 Checking the statistics for network-filtering
root@CLA-0 [Mocha] > show st at s dat a cur r ent omes- i d 3081 nameFPNODE- SUM/ NETWORK_FI LTERI NG- SUMNumber of active flows(m3081c0001) : 10526Number of active sessions(m3081c0002) : 6516Number of persistence records(m3081c0003) : 18195Number of SNAT table entries(m3081c0004) : 773Number of bytes sent(m3081c0005) : 12544213493959Number of packets sent(m3081c0006) : 12908853026Number of flow activations(m3081c0007) : 98311388Number of session activations(m3081c0008) : 3528277Number of memory allocation failures(m3081c0009) : 0Number of packets passed through flows without translation(m3081c0010) : 190661Number of packets passed through flows with translation(m3081c0011) : 12908662587Number of packets dropped by flows(m3081c0012) : 917Number of packets passed through rules without translation or flows(m3081c0013) : 0Number of packets passed through rules with translation and no flowcreation(m3081c0014) : 0Number of packets dropped by rules and no flow creation(m3081c0015) : 0Number of packets matched by default action rule(m3081c0016) : 12109Number of packets matched by fallback action rule(m3081c0017) : 0Number of packets matched by static translation rule(m3081c0018) : 31563304Number of packets matched by session tracker rule(m3081c0019) : 0
Number of packets matched by gateway rule(m3081c0020) : 5294101Number of packets matched by SNAT rule(m3081c0021) : 1928502Number of SNAT table entry collisions(m3081c0022) : 0Number of packets matched by client SNAT rule(m3081c0023) : 24507108Number of packets dropped due to errors(m3081c0024) : 335
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
31/35
Cluster log Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
31/35
4 Cluster logYou should check the logs on the active CLA server.
Log file Purpose
/srv/Log/log/syslog the cluster (master) log files.
/srv/Log/log/fsaudit/auth.log
security relevant cluster(master) log
files.
/srv/Log/log/fsaudit/alarms the cluster alarm system.
/srv/Log/log/debug
debug level logs and application specific
error level logs.
/var/log/syslog-hostname.log System log for every node
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
32/35
System log Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
32/35
5 System logYou can check the logs on each node.
Log file Purpose
/var/log/tallylog failed login attempts for all users
/var/log/lastlog login details for each user.
/var/log/ntp directory for ntp daemon logs
/var/log/wtmp
login and logout information for the
system
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
33/35
Alarm log Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
33/35
6 Alarm log6.1.1 Active alarm
To view the active alarm data in the system, log in to the CLA server, and access the SCLImodel by f scl i sh command.
Run show al ar m act i ve, the first line displays the parameter meaning, for example:
Alarm ID | Specific Problem | Alarm Text | Probable Cause |Event Type Name | Alarm Time | Perceived Severity | Acknowledged| Acknowledged Time | Acknowledged User ID | Cleared | ManagedObject ID | Application ID | Identifying Application AdditionalInfo | Application Additional Info70007 | 70164 | ETHERNET LINK FAILURE | 517 | Equipment | Thu, 21Jan 2010 17:22:42.425 | Minor | True | Fri, 22 Jan 201015:04:17.571 | | False | fsipHostName=CLA-1,fsFragmentId=Nodes,fsFragmentId=HA,fsClusterId=ClusterRoot |fshaProcessInstanceName=NetworkManager,fshaRecoveryUnitName=FSNetworkManagerServer,fsipHostName=CLA-
1,fsFragmentId=Nodes,fsFragmentId=HA,fsClusterId=ClusterRoot |eth5 | Link down
You also can filter the output by parameter:
root@CLA-0 [Cappuccino] > show al ar m act i ve[X] - press to execute the command[OX] from-index - It means, starting from which record onwards, the"Active Alarms" in the Alarm System should be fetched.[OX] how-many - It means, how many of the "Active Alarms" in the"Alarm System" should be fetched.[X] application-ids - Shows the "Application ID's" of "Active Alarms".
[X] filter-by - Shows the list of all filtered active alarms based on thespecified filtering criteria parameters.[X] managed-object-ids - Shows the "Managed Object ID's" of "ActiveAlarms".[X] probable-causes - Shows the "Probable Causes" of "ActiveAlarms".[X] severities - Shows the "Severities" of "Active Alarms".
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
34/35
Alarm log Flexi CO-IB Troubleshooting Guide
D509477015 Issue en 1-0
Copyright 2013 Nokia Solutions and Networks. Allrights reserved.
34/35
6.1.2 History alarm
To view the active alarm data in the system, log in to the CLA server, access the SCLImodel by f scl i sh command.
Run show al ar m hi st or y, the first line displays the parameter meaning, for example:Notification id | Specific Problem | Alarm Text | ProbableCause | Event Type Name | Alarm Time | Perceived Severity |Acknowledged | Acknowledged Time | Acknowledged User Id |Cleared | Managed Object ID | Application ID | IdentifyingApplication Additional Info | Application Additional Info |Alarm ID | Extended Event Type Name | Control Indicator75186 | 70246 | ALARM SYSTEM HEARTBEAT | 546 | Processing Error| Fri, 15 Jan 2010 02:07:48.555 | Cleared | False | 0 | | False|fshaProcessInstanceName=AlarmProcessor,fshaRecoveryUnitName=FSAlarmSystemServer,fsipHostName=CLA-0,fsFragmentId=Nodes,fsFragmentId=HA,fsClusterId=ClusterRoot |
fshaProcessInstanceName=AlarmProcessor,fshaRecoveryUnitName=FSAlarmSystemServer,fsipHostName=CLA-0,fsFragmentId=Nodes,fsFragmentId=HA,fsClusterId=ClusterRoot |
| 300 | 70022 | Clear Alarm | 0 |
For the details description, refer to Flexi NGAlarmsdocuments.
-
7/26/2019 Flexi CO-IB Troubleshooting Guide
35/35
Appendix Flexi CO-IB Troubleshooting Guide
7 Appendix