brkcol-2602 - collaboration edge troubleshooting (2015 san diego).pdf
TRANSCRIPT
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
1/215
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
2/215
Collaboration EdgeTroubleshooting
Philip SmeuninxTechnical Leader Services
BRKCOL-2602
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
3/215
Introduction
Mobile and Remote Access
XMPP Federation
B2B
Takeaways
Agenda
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
4/215
Before we start
For your reference
Tool bookmark
Questions
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
5/215
Mobile and Remote Acces
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
6/215
Topology
Expressway x8.5
CUCM/CUP 10.5(2) Jabber for Windows 10.5(2)
CUCMCUP
Expressway-C Expressway-E
Internet
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
7/215
ExpressWay Configuration and Troubleshoo
System configuration
Firewall configuration
Certificate configuration and deployment
Traversal zone configuration
UC server discovery
DNS and domain configuration/deployment
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
8/215
Mobile and Remote aSystem Configu
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
9/215
System Configuration
Set Unified Communications mode to Mobile and remote access
Configuration > Unified Communications > Configuration
Check the Administrator guide for more help on system configurati
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
10/215
System configuration - NTP
Each system must be synched with NTP server> System > Time
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
11/215
System Configuration - NTP
If NTP is not configured and synchronized on ExpressWa
ExpressWay-E Jabber Telephony registration to CUCMsucceed.
Security mechanism based on SIP SERVICE messages.
Expressway-E time-stamps a SERVICE message
Expressway-E sends the SERVICE message to Expres Expressway-C verifies the SERVICE is received within
error margin
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
12/215
Mobile and Remote aFirewall Configu
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
13/215
Firewall Configuration
What traffic does the firewall need to pass? HTTPS proxy for secure provisioning of endpoints
SIP/TLS, RTP/SRTP for audio/video media XCP/XMPP for IM&P for Jabber
HTTPS Services
Traversal Connection between ExpressWay-C and E
ClusterDB change notifications (ssh tunnel)
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
14/215
Firewall ConfigurationTo which ports does this translate?
Port usage: ExpressWay C to Expressway E
CUCM-UDS ExpressWay C ExpressWay E
InternetDMZ
ExpressWay C
Source Port
ExpressWay E
Listening Port
Management Control Inbound and outbound calls
Open Firewall Private to DMZ
IP AddressIP address of- ExpressWay C
IP address of- ExpressWay E
XMPP (IM and Presence)TCP Ue
30000 to 35999 *TCP 7400
SSH(HTTP/S tunnels)
TCP Ue30000 to 35999 *
TCP 2222
SIPsignalingTCP & TLSA
25000 to 29999TCP & TLSB
7001
SIP mediaUDPYC
36000 to 59999 **UDPYE
36000 to 36011 **
IM&P
TCP & TLSA = Configurable TCP Outbound p
TCP & TLSB = Configurable traversal port forbetween Expressway C and Expressway E (
etc.)
Ue = Configurable TCP ephemeral port range
YC = Configurable traversal media ports rangC)
YE = Configurable multiplexed media ports raExpressway E)
IPPorts
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
15/215
ExpressWay C
Source Port
ExpressWay E
Listening Port
Management Control Inbound and outbound calls
Open Firewall Private to DMZ
IP AddressIP address of- ExpressWayC
IP address of- ExpressWay E
XMPP (IM and Presence)TCP Ue
30000 to 35999 *TCP 7400
SSH(HTTP/S tunnels)
TCP Ue30000 to 35999 *
TCP 2222
SIP signalingTCP & TLSA
25000 to 29999TCP & TLSB
7001
SIP mediaUDPYC
36000 to 59999 **UDPYE
36000 to 36011 **
Firewall ConfigurationWhere to configure these ports?
ExpressWay C > System > Administration
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
16/215
ExpressWay C
Source Port
ExpressWay E
Listening Port
Management Control Inbound and outbound calls
Open Firewall Private to DMZ
IP AddressIP address of- ExpressWay C
IP address of- ExpressWay E
XMPP (IM and Presence) TCP 7400TCP Ue
30000 to 35999 *
SSH(HTTP/S tunnels)
TCP Ue30000 to 35999 *
TCP 2222
SIP signalingTCP & TLSA
25000 to 29999TCP & TLSB
7001
SIP mediaUDPYC
36000 to 59999 **UDPYE
36000 to 36011 **
Firewall ConfigurationWhere to configure these ports?
ExpressWay C > Protocols > SIP
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
17/215
ExpressWay C
Source Port
ExpressWayE
Listening Port
Management Control Inbound and outbound calls
Open Firewall Private to DMZ
IP AddressIP address of- ExpressWayC
IP address of- ExpressWayE
XMPP (IM and Presence) TCP 7400TCP Ue
30000 to 35999 *
SSH(HTTP/S tunnels)
TCP Ue30000 to 35999 *
TCP 2222
SIP signalingTCP & TLSA
25000 to 29999TCP & TLSB
7001
SIP mediaUDPYC
36000 to 59999 **UDPYE
36000 to 36011 **
Firewall ConfigurationWhere to configure these ports?
ExpressWay C > Configuration > Traver
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
18/215
ExpressWay C
Source Port
ExpressWay E
Listening Port
Management Control Inbound and outbound calls
Open Firewall Private to DMZ
IP AddressIP address of- ExpressWay C
IP address of- ExpressWayE
XMPP (IM and Presence)TCP Ue
30000 to 35999 *TCP 7400
SSH(HTTP/S tunnels)
TCP Ue30000 to 35999 *
TCP 2222
SIP signalingTCP & TLSA
25000 to 29999TCP & TLSB
7001
SIP mediaUDPYC
36000 to 59999 **UDPYE
36000 to 36011 **
Firewall ConfigurationWhere to configure these ports?
ExpressWay E > Configuration > Zone > Tr
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
19/215
Expressway E Demultiplexing media ports
Small/medium deployment
->Configured Media Demultiplexing portsDefault : 2776 (RTP) 2777 (RTCP)or->First 2 ports from Traversal Media port rangeDefault : 36000 (RTP) 36001 (RTCP)
ExpressWay C ExpressWay E
36000-36001or2776-277736000-59999
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
20/215
For large systems new install
-> First 12 ports from Traversal Media port rangeDefault : 36000 (RTP) 36011 (RTCP)
ExpressWay C ExpressWay E
36000-3601136000-59999
Expressway E Demultiplexing media ports
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
21/215
Firewall configurationDemultiplex port range after upgrades
Upgrade from x7 to x8.1 -> 50000 50001System uses port pair from Traversal Media port range
Upgrade from x8.1 (upgraded from x7) to x8.2 -> 50000 50001Demultiplex port range = retained from previous version andUse configured demultiplexing ports is set to Yes
Upgrade from x7 to x8.2 -> 2776 2777Demultiplex port range = retained from previous version andUse configured demultiplexing ports is set to Yes
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
22/215
Firewall ConfigurationTo which ports does this translate?
Port usage: Expressway E to/from Public Internet
Expressway E
Source Port
Internet SIP UA
Listening Port
Management Control Outbound to SIP UA in the Internet
Open Firewall DMZ to Internet
IP AddressPublic IP address of- ExpressWay E
IP address of- Any (or specific IP)
XMPP (IM and Presence)) Client/Server N/A N/A/5269
UDS(Provisioning and Phonebook)
N/A N/A
TURN Server Control N/A N/A
SIP signalingTLS
25000 to 29999TLS S
>= 1024
MediaUDPYE
36000 to 59999 **UDP N>= 1024
IPPorts
N = ExpressWay wait unit it receives memedia to the IP port from which media w
port of the media from the far end non S
S = Source port, typically >=1024
YE = Configurable traversal media ports E)
CUCM-UDS ExpressWay C Expressway E
InternetDMZ
IM&P
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
23/215
Firewall ConfigurationTo which ports does this translate?
Port usage: Expressway E to/from Public Internet
Expressway C
Listening Port
Internet SIP UA
Source Port
Management Control Inbound from SIP UA in the Internet
Open Firewall Internet to DMZ
IP Address IP address of- VCS Expressway IP address of- Any (or specific IP)
XMPP (IM and Presence)) Client/Server TCP 5222/5269TCP S
>= 1024
UDS(Provisioning)
TCP 8443TCP S
>= 1024
TURN Server Control UDP 3478UDP S
>= 1024
SIP signaling TLS 5061TLS S
>= 1024
MediaUDPYE
36000 to 59999 **UDP N>= 1024
IPPorts
CUCM-UDS ExpressWay C ExpressWay E
InternetDMZ
IM&P
N = ExpressWay wait unit it receives medmedia to the IP port from which media wa
port of the media from the far end non SIP
S = Source port, typically >=1024
YE = Configurable traversal media ports raExpressway/E)
** Default media ports range (X8.1) is 360configurable
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
24/215
Firewall ConfigurationTo which ports does this translate?
Port usage: ExpressWay C to Unified CM and IM&P
Internet
CUCM&CUP System
Listening Port
ExpressWay C
Source Port
Management Control Private Network
Open Firewall N/A
IP AddressIP address of- Unified CM- IM & Presence Server
IP address of- ExpressWayC
XMPP (IM and Presence)TCP 7400
(IM&P Server)TCP Ue
30000 to 35999 *
UDS CUCMSOAP IM&P
TCP 8443(CUCM Server, IM&P Server)
TCP Ue30000 to 35999 *
TFTPTCP 6970
(TFTPServer)TCP Ue
30000 to 35999 *
CUC (Voicemail)TCP 443
(CUC server)TCP Ue
30000 to 35999 *
CUCM-UDS ExpressWay C ExpressWay E
DMZ
IM&P
Ue = Configurable TCP ephemera
* Default ephemeral ports range (X
which configurable
IPPorts
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
25/215
Dual NIC consideration (advanced networkin
If option key is addedit will add a second LAN (LAN 2)
This will result in followingdefault configuration
With following port assignment
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
26/215
Dual-NIC enabled but not used/connected (only for static NAT) Expwill not be able to connect to 7400 for XMPP
ExpressWay C diagnostic logsxwayc XCP_JABBERD[23843]: UTCTime="2015-03-25 17:19:45,843" ThreadID="13974721257651
Level="INFO " CodeLocation="mio.c:1109" Detail="Connecting on fd 28 to host '10.48.55.99', port 74xwayc XCP_JABBERD[23843]: UTCTime="2015-03-25 17:19:45,847" ThreadID="13974721257651Level="ERROR" CodeLocation="mio.c:1121" Detail="Unable to connect to host '10.48.55.99', porConnection refusedxwayc XCP_JABBERD[23843]: UTCTime="2015-03-25 17:19:45,847" ThreadID="13974740693580Level="ERROR" CodeLocation="base_connection.cpp:104" Detail="Failed to connect to compone1.xwayc-coluc-com
Solution : Disable LAN 2 (internal) or connect it physically
Dual NIC consideration (advanced networkin
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
27/215
Firewall SetupPort Status and Configuration
Maintenance > Tools > Port Usage
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
28/215
HTTP Server Allow list
> Configuration > Unified Communications > Configuration
The hostname or IP address of an on-prem HTTP server that a Jabblocated outside of the enterprise is allowed to access.
Access is granted when server portion of the client-supplied URI matname entered here or resolves via DNS lookup to configured IP.
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
29/215
Mobile and Remote ACertif
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
30/215
Certificates
> Maintenance> Security Certificate> Server Certificate
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
31/215
Certificates
> Maintenance > Security Certificate > Trusted CA Certificate
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
32/215
ExpressWay C Server Certificate
Used with ExpressWay E for traversal zone connection
Used with CUCM when endpoint security mode is Authenticatedor Encrypted (TLS transport used)
Must be CA Signed -> Enterprise CA or Public CA
CA Root which issued the certificate must be appended to Trustedcertificate on both ExpressWays
CA Root must be uploaded to Callmanager-trust store on every nocluster
Troubleshooting
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
33/215
TroubleshootingCA Root not uploaded on ExpressWay E
Traversal Zone State Failed
Expressway-C Diagnostics logs (traversal client)
xwayc tvcs: Event="Outbound TLS Negotiation Error" Service="SIP" Src-ip="10.48.5port="25016" Dst-ip="10.48.55.99" Dst-port="7001" Detail="tlsv1 alert unknown ca"Common-name="xwaye.coluc.com" Level="1" UTCTime="2014-03-24 17:33:30,872
Expressway Event logs
Troubleshooting
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
34/215
TroubleshootingCA Root not uploaded on CUCM
Softphone Registration fails (other will work) when endpoint securiauthenticated or encrypted
Troubleshooting
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
35/215
TroubleshootingCA Root not uploaded on CUCM
ExpressWay-C diagnostic logs
2014-03-24T18:57:37+00:00 xwayc tvcs: Event="Outbound TLS Negotiation EService="SIP" Src-ip="10.48.55.98" Src-port="25264" Dst-ip="10.48.55.96" Dst-Detail="tlsv1 alert unknown ca" Protocol="TLS" Common-name="COLCM9PULevel="1" UTCTime="2014-03-24 18:57:37,777
Expressway-C event logs
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
36/215
ExpressWay C Certificate Requirements
Extended Key Usage- TLS Web Server Authentication
- TLS Web Client Authentication
SAN elements configured with :- FQDN Expressway C- IM and Presence chat node alias- Unified CM Security Profile names
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
37/215
ExpressWay C Certificate Requirements
Expressway C CUP
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
38/215
ExpressWay C Certificate Requirements
Expressway C CUCM
Troubleshooting
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
39/215
TroubleshootingSecurity Profile added as SAN (CUCM trace
SIPTcp - Connection Indication - Listen Port = 5061, Peer Port = 25002
SIPTcp - wait_SdlReadRsp: Incoming SIP TCP message from 10.48.55.98 on port 25002 index 10 w
bytes:[53,NET]REGISTER sip:COLCM9PUB SIP/2.0//SIP/SIPHandler/ccbId=0/scbId=0/wait_SIPCertificateInd: could not find a trunk device using addrex509SubjectName calling findSIPStationInit//SIP/SIPHandler/ccbId=0/scbId=0/findDeviceByX509Subject:x509Subject:xwayc.coluc.com, port:5//SIP/SIPHandler/ccbId=25/scbId=0/findDevicePID: Routed to SIPStationInitSIPStationInit: connId=10, CSFEWAYJ, 10.48.55.98:5061, Incoming register request received over TSubject=[/C=BE/ST=BRABANT/L=DIEGEM/O=CISCO/OU=TAC/CN=xwayc.coluc.com]SIPStationD(9) - validTLSConnection:TLS InvalidX509NameInCertificate, Rcvd=xwayc.coluc.comExpected=CSFEWAYJ. Will check SAN the nextSIPStationD(9) - validTLSConnection: Found matching SAN, SAN Rcvd=xwayc.coluc.com;conferenecup9.coluc.com;csf-secure, Expected=csf-secure
Troubleshooting
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
40/215
TroubleshootingSecurity Profile not added as SAN (CUCM tr
SIPTcp - Connection Indication - Listen Port = 5061, Peer Port = 25004
SIPTcp - wait_SdlReadRsp: Incoming SIP TCP message from 10.48.55.98 on port 25004 index 10 w
bytes:[53,NET]REGISTER sip:COLCM9PUB SIP/2.0//SIP/SIPHandler/ccbId=0/scbId=0/wait_SIPCertificateInd: could not find a trunk device using addressx509SubjectName calling findSIPStationInit//SIP/SIPHandler/ccbId=0/scbId=0/findDeviceByX509Subject: x509Subject:xwayc.coluc.com, port:506//SIP/SIPHandler/ccbId=25/scbId=0/findDevicePID: Routed to SIPStationInitSIPStationInit: connId=10, CSFEWAYJ, 10.48.55.98:5061, Incoming register request received over TSubject=[/C=BE/ST=BRABANT/L=DIEGEM/O=CISCO/OU=TAC/CN=xwayc.coluc.com]SIPStationD(3) - validTLSConnection:TLS InvalidX509NameInCertificate, Rcvd=xwayc.coluc.comExpected=CSFEWAYJ. Will check SAN the next
SIPStationD(3) - validTLSConnection:TLS InvalidX509NameInCertificate Error , did not find matchinRcvd=xwayc.coluc.com;conference-2-ecup9.coluc.com,Expected=csf-secure
Troubleshooting
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
41/215
TroubleshootingSecurity Profile not added as SAN (CUCM tr
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
42/215
ExpressWay E Server Certificate
Used with ExpressWay C for traversal zone connection
Used with foreign domains for XMPP Federation Must be CA Signed
Public CA
CA Root which issued the certificate must be appended toTrusted CA certificate on both ExpressWays
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
43/215
ExpressWay E Certificate Requirements
Extended Key Usage- TLS Web Server Authentication
- TLS Web Client Authentication
SAN elements configured with :- Unified CM Registration domains (incl. voiceservices domains)- IM and Presence chat node alias- XMPP Domain
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
44/215
ExpressWay E Certificate Requirements
Expressway E Express
T bl h ti
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
45/215
TroubleshootingCA root not uploaded to ExpressWay C Traversal Zone State
ExpressWay E diagnostic logsxwaye tvcs: Event="Inbound TLS Negotiation Error" Service="SIP" Src-ip="10.48.55port="25006" Dst-ip="10.48.55.99" Dst-port="7001" Detail="tlsv1 alert unknown caLevel="1" UTCTime="2014-03-25 09:52:36,680
ExpressWay E event logs
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
46/215
Bookmark X8.5 Tool
Secure traversal test Expressway C
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
47/215
Mobile and Remote Unified Communications Travers
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
48/215
Unified Communications Traversal Zone
Expressway-E is traversal server in DMZ
Expressway-C is traversal client inside the network Establish traversal link between both using traversal zone configur
CUCM
Enterprise Network DMZ Outside Network
Expressway-CTraversal Client
Expressway-ETraversal Server
Endpoint
Internet
Endpoint A
Traversal Link ManagementSignal
Media Payload
UC Traversal Zone
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
49/215
UC Traversal ZoneExpressWay E Traversal Server
Select Type : Unified Comm
traversal
Configure username to be uClient to authenticate with se
Port is default 7001, listenintraversal client connection
Must match CN or SAN frompresented by Traversal Clie(ExpressWay C)
UC Traversal Zone
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
50/215
UC Traversal ZoneExpressWay E Traversal Server
Traversal Zone Status
Connection status with Traversal Client
UC Traversal Zone
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
51/215
UC Traversal ZoneExpressWay C Traversal Client
Select Unified CommunicTraversal as Type
Configure same usernamepassword as added on theServer (Expressway E)
Destination port Traversallistening on
UC Traversal Zone
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
52/215
UC Traversal ZoneExpressWay C Traversal Client
Must be FQDN (*)
Must match CN or SANCertificate presented b
Expressway E
Must resolve to Public Expressway E whensingle NIC deploymen
Troubleshooting
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
53/215
TroubleshootingPeer Address not matching CN
Peer Address configured as IP address
ExpressWay C diagnostic logs2014-03-25T14:08:16+00:00 xwayc tvcs: Event="Outbound TLS Negotiation ErService="SIP" Src-ip="10.48.55.98" Src-port="25697" Dst-ip="10.48.55.99" Dst-Detail="Peer's TLS certificate identity was unacceptable" Protocol="TLS" Comm
name="10.48.55.99" Level="1" UTCTime="2014-03-25 14:08:16,699 ExpressWay C Event logs
Troubleshooting
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
54/215
TroubleshootingPeer Address not matching CN
Peer Address/FQDN not matching CN
ExpressWay C diagnostic logs
2014-03-25T14:16:36+00:00 xwayc tvcs: Event="Outbound TLS Negotiation ErService="SIP" Src-ip="10.48.55.98" Src-port="25714" Dst-ip="10.48.55.99" Dst-
Detail="Peer's TLS certificate identity was unacceptable" Protocol="TLS" Commname="xwy.coluc.com" Level="1" UTCTime="2014-03-25 14:16:36,699"
ExpressWay C Event logs
Troubleshooting
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
55/215
TroubleshootingPassword incorrect
Traversal Client will show for this zone
ExpressWay C diagnostic logs
Module="network.dns" Level="DEBUG": Detail="Sending DNS query" Name="xwaye.coand AAAAModule="network.dns" Level="DEBUG": Detail="Resolved hostname to: ['IPv4''TCP''10(A/AAAA) Number of relevant records retrieved: 1Module="network.tcp" Level="DEBUG": Src-ip="10.48.55.98" Src-port="25723" Dst-ip="port="7001" Detail="TCP ConnectingModule="network.tcp" Level="DEBUG": Src-ip="10.48.55.98" Src-port="25723" Dst-ip="port="7001" Detail="TCP Connection Established
Password incorrect (contd )
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
56/215
Password incorrect (contd.) ExpressWay C diagnostics logs
Module="network.sip" Level="DEBUG": Action="Sent" Local-ip="10.48.55.98" Local-port="25723" Dst-ip="10.48.55.9SIPMSG:|OPTIONS sip:10.48.55.99:7001;transport=tls SIP/2.0.Module="network.sip" Level="DEBUG": Action="Received" Local-ip="10.48.55.98" Local-port="25723" Src-ip="10.48.SIPMSG:|SIP/2.0 401 UnauthorisedWWW-Authenticate: Digest realm="TraversalZone", nonce="527e7f2a24ff1c54e3e4cd5025f674967e81d2aa9b214fda98opaque="AQAAAPet..Module="network.sip" Level="DEBUG": Action="Sent" Local-ip="10.48.55.98" Local-port="25723" Dst-ip="10.48.55.9SIPMSG:|OPTIONS sip:10.48.55.99:7001;transport=tls SIP/2.0.
Authorization: Digest nonce="527e7f2a24ff1c54e3e4cd5025f674967e81d2aa9b214fda98cef27f3f82", realm="TraversalZopaque="AQAAAPet+0JJTq4cyuB34opHePwV7bkk", algorithm=MD5, uri="sip:10.48.55.99:7001;transport=tls", usernam...2014-03-25T14:19:56+00:00 xwayc tvcs: UTCTime="2014-03-25 14:19:56,705" Module="network.sip" Level="DEBUG":SIPMSG:|SIP/2.0 401 Unauthorised.Event="External Server Communications Failure" Reason="gatekeeper timed out" Service="NeighbourGatekeeper" Dst-port="7001" Detail="name:xwaye.coluc.com" Protocol="TCP" Level="1" UTCTime="2014-03-25 14:19:56,705"
Troubleshooting
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
57/215
TroubleshootingPassword incorrect
ExpressWay E diagnostic logs
Module="network.ldap" Level="INFO": Detail="Authentication credential found in directory for identModule="developer.nomodule" Level="WARN" CodeLocation="ppcmains/sip/sipproxy/SipProxyAuthMethod="SipProxyAuthentication::checkDigestSAResponse" Thread="0x7f2485cb0700": calculatedmatch supplied response, calculatedResponse=769c8f488f71eebdf28b61ab1dc9f5e9,response=319a0bb365decf98c1bb7b3ce350f6ecEvent="Authentication Failed" Service="SIP" Src-ip="10.48.55.98" Src-port="25723" Detail="Incorrecredential for user" Protocol="TLS" Method="OPTIONS" Level="1
Troubleshooting
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
58/215
ExpressWay C event log
ExpressWay E event log
TroubleshootingPassword incorrect
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
59/215
Mobile and Remote AUC Server Disc
UC Server Discovery
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
60/215
UC Server Discovery
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
61/215
CUCM Server Discovery
Discovers hostname (processnodetable)
Discovers version
Discovers Cluster Security mode (Transport Protocols)
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
62/215
CUCM Server Discovery
expwayC.domain1.com
Expressway C TOMCAT UDS/8443
colcm10pub.coluc.com
HTTPS
Q: What do I entA: Depends on T
C C S S f
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
63/215
TLS verify mode = On
CUCM Server Discovery TLS verify mode
Publisher address = FQDN, MUST match CN TOMCAT Certifica
CUCM S Di TLS if d
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
64/215
TLS verify mode = On
CUCM Server Discovery TLS verify mode
OR (*)Publisher address = FQDN MUST match SAN TOMCAT Certifica
(*) Only valid sta
CUCM S Di TLS if d
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
65/215
TLS verify mode = On
CUCM Server Discovery TLS verify mode
CA Certificate must be uploadedTrusted CA certificate list Expressway C
CUCM Server Discovery TLS verify mode
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
66/215
TLS verify mode = Off
No requirements forTOMCAT Certificate Publisher
CUCM Server Discovery TLS verify mode
CUCM S Di Z C fi ti
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
67/215
Auto-Zone Configuration per node and per transport protocol
Syntax : CEtcp- and CEtls-
CUCM Server Discovery Zone Configuration
CUCM Server Discovery Zone Configuration
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
68/215
TLS verify mode Zone
TLS verify mode Discovery
TLS verify mode = On
CUCM Server Discovery Zone Configuration
CUCM Server Discovery Zone Configuration
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
69/215
CEtls- Zo- TLS Verify mode = On- Peer Address must mafrom Callmanager certif
TLS ve
CUCM Server Discovery Zone Configuration
CUCM Server Discovery Zone Configuration
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
70/215
TLS verify mode configuration Zone
TLS verify mode configuration Discovery
TLS verify mode = Off
CUCM Server Discovery Zone Configuration
CUCM Server Discovery Zone Configuration
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
71/215
y g
CUCM Server Discovery Search Rule Configu
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
72/215
CUCM Server Discovery Search Rule Configu
1 Search Rule per node per transport protocol
Pattern matching for header
T bl h ti Diff t D i
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
73/215
Troubleshooting - Different server Domain
expwayC.edge1.com
Expressway C Internal DNS CUCM
colcm9pub.coluc.com
How does Server configuration on CUCM impact the discovery?
Troubleshooting
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
74/215
Status is Active @ discovery will succeed
When CCM cert is uploaded first -> discovery will fail
TLS verify + Self Signed CCM/Tomcat certificate + Encryption
Either discovery will fail or TLS connections with CUCM will fail
With self-signed certificates use TLS verify mode = Oand only upload the CUCM cert
Troubleshooting - Self Signed Certificates
Troubleshooting - Single Server Certificate
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
80/215
Expressway disregard CN for identity verification when SAN attribupresent
RFC 6125 Move from CN-ID to DNS-ID, SRV-ID or URI-ID
With TLS Verify mode for HTTPS (discovery) and SIP TLS (edgeCCM and TOMCAT Certificates MUST FQDN SAN = DNS-ID
g g(CCM & TOMCAT)
Troubleshooting - Multi-Server Certificates for
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
81/215
Servers
Multi-Server certificates for CUCM/CUPhave -ms appended to the CN
Certificate will have SAN populatedwith all server nodes
Expressway X8.2 + supportsmulti-server certificates
Troubleshooting - Search Rule matching for
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
82/215
Edge/MRA calls|INVITE sip:[email protected];user=phone SIP/2.0Via: SIP/2.0/TLS 10.48.55.93:7001;egress-zone=TraversalUC;branch=Via: SIP/2.0/TLS 10.48.55.106:52008;branch=z9hG4bK000073dc;received=10.48.55.106;ingress-zone=CollaborationEdgeZoneCall-ID: [email protected]: 101 INVITERemote-Party-ID: "5445" ;party=calling;id-type=subscriber;privacy=off;screen=yes
Contact: ;video;bfcpFrom: "5445" ;tag=0050568a003a000800006fdd-00006fe8To: Max-Forwards: 10
Route: Record-Route: Record-Route: Allow: ACK,BYE,CANCEL,INVITE,NOTIFY,OPTIONS,REFER,REGISTER,UPDATE,SUBSCRIBE,INFOUser-Agent: Cisco-CSF.
Set by clie
Device
Device
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
83/215
Mobile and RemoteDNS and
Domain Configuration
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
84/215
gExpressWay C & E DNS Configuration
System > DNS
Domain Configuration
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
85/215
gExpressWay C Domain Configuration
> Configurations > Domains
Client Service Discovery
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
86/215
Client Service Discovery
Service discovery enables clients and endpoints to automatically dlocate service.
The client/endpoint does query DNS servers to retrieve service (SRthat provide the location of servers.
Clients/endpoints outside internal network must be able to resolve_collab-edge._tls. with target Expressway E server
Clients/endpoints & ExpressWay C inside the internal network mus
resolve _cisco-uds._tcp. SRV record with target CUCM
The external DNS may not resolve _cisco-uds._tcp SRV records
The internal DNS may not resolve _collab-edge._tls SRV records
ExpressWay Mobile and Remote Access
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
87/215
p yDomain and DNS configuration
Scenario 1- Flat domain structure
- ExpressWay Servers : domain1.com- UC servers : domain1.com- IM&P domain : domain1.com
xwayC.domain1.com
Jabber Client Expressway C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain1 com cucm.domain1.c
ExpressWay Scenario 1
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
88/215
xwayC.domain1.com
Jabber Client Expressway C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain1 com cucm.domain1.co
Question : How do I login?
Answer : With @domain1.com
Domain and DNS configuration
ExpressWay Scenario 1
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
89/215
expwyC.domain1.com
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
expwyE.domain1 com cucm.domain1.co
Question: How is my external DNS configured?Answer:Entry Resolves to
SRV record _collab-edge._tls.domain1.com expwyE.domain1.com port 8443
A record xwayE.domain1.com External IP address ExpressWay E
Domain and DNS configuration
ExpressWay Scenario 1
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
90/215
xwayC.domain1.com
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
cucm.domain1.co
Question: How is my ExpressWay E configured?Answer:> System > DNS >- System host name xwayE- Domain name domain1.com
Domain and DNS configuration
ExpressWay Scenario 1
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
91/215
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain1 com cucm.domain1.co
Question: How is my ExpressWay C configured?Answer:
> System > DNS >- System host name xwayE- Domain name domain1.com
> Configuration > Domains >- Domain domain1.com enabled for:UCM registrations and IM and Presence
Domain and DNS configuration
ExpressWay Scenario 1
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
92/215
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain1 com cucm.domain1.co
Question: How is my Internal DNS configured?Answer:
xwayC.domain1 com
Entry Resolves to
SRV record _cisco-uds._tcp.domain1.com cucm.domain1.com port 84
A record cucm.domain1.com IP address CUCM
Domain and DNS configuration
ExpressWay Scenario 1
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
93/215
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain1 com
Question: How is my CUCM configured?Answer:> CCMADMIN > System > Server
- Server with hostname cucm> CLI set network domain domain1.com
xwayC.domain1 com
Domain and DNS configuration
ExpressWay Scenario 1
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
94/215
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain1 com
Question: How is my CUP configured?Answer:> CUPAdmin > Clustertopology
- Node configuration with cup.doma- IM and Presence Domain with do
xwayC.domain1 com cucm.domain1.co
Domain and DNS configuration
ExpressWay Mobile and Remote Access
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
95/215
Domain and DNS configuration
Scenario 2- Mixed domain structure- Expressway servers : domain2.com- UC and CUP servers : domain1.com- IM&P domain : domain1.com
xwayC.domain2.com
Jabber Client Expressway C Internal DNS CUCM Home UDExpressway EExternal DNS
xwayE.domain2 com cucm.domain1.
ExpressWay Scenario 2D i d DNS fi i
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
96/215
xwayC.domain2.com
Jabber Client Expressway C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain2 com cucm.domain1.co
Question : How do I login?
Answer :- With @domain1.com (*)- jabber-config.xml has voiceservicesdomain set to domain2.com
Domain and DNS configuration
ExpressWay Scenario 2D i d DNS fi ti
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
97/215
xwayC.domain2.com
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain2 com cucm.domain1.co
Question: How is my external DNS configured?
Answer:Entry Resolves to
SRV record _collab-edge._tls.domain2.com xwayE.domain2.com port 8443
A record xwayE.domain2.com External IP address ExpressWay E
Domain and DNS configuration
ExpressWay Scenario 2D i d DNS fi ti
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
98/215
xwayC.domain1.com
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
cucm.domain1.co
Question: How is my ExpressWay E configured?
Answer:> System > DNS >- System host name xwayE- Domain name domain2.com
Domain and DNS configuration
ExpressWay Scenario 2D i d DNS fi ti
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
99/215
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain2 com cucm.domain1.co
Question: How is my ExpressWay C configured?
Answer:
> System > DNS >- System host name xwayC- Domain name domain2.com
> Configuration > Domains >- Domain domain1.com enabled for UCM registrations and IM- Domain domain2.com enabled for UCM registrations and IM
Domain and DNS configuration
ExpressWay Scenario 2D i d DNS fi ti
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
100/215
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain2.com cucm.domain1.co
Question: How is my Internal DNS configured?
Answer:
xwayC.domain2.com
Entry Resolves to
SRV record _cisco-uds._tcp.domain2.com cucm.domain1.com port 84
A record cucm.domain1.com IP address CUCM
Domain and DNS configuration
ExpressWay Scenario 2D i d DNS fi ti
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
101/215
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain1 com
Question: How is my CUCM configured?Answer:> CCMADMIN > System > Server
- Server with hostname cucm> CLI set network domain domain1.com
xwayC.domain1 com
Domain and DNS configuration
ExpressWay Scenario 2D i d DNS fi ti
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
102/215
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain1 com
Question: How is my CUP configu
Answer:> CUPAdmin > Clustertopology
- Node configuration with cup- IM and Presence Domain wi
xwayC.domain1 com cucm.domain1.co
Domain and DNS configuration
ExpressWay Mobile and Remote AccessD i d DNS fi ti
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
103/215
Domain and DNS configuration
Scenario 3- Mixed domain structure- Expressway servers : domain3.com- UC and CUP servers : domain2.com- IM&P domain : domain1.com
xwayC.domain3.com
Jabber Client Expressway C Internal DNS CUCM Home UDExpressway EExternal DNS
xwayE.domain3 com cucm.domain2.
ExpressWay Scenario 3D i d DNS fi ti
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
104/215
xwayC.domain3.com
Jabber Client Expressway C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain3 com cucm.domain2.co
Question : How do I login?
Answer :- With @domain1.com- jabber-config.xml has voice voiceservicesdomain set to domain3.co
Domain and DNS configuration
ExpressWay Scenario 3D i d DNS fi ti
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
105/215
xwayC.domain3.com
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain3 com cucm.domain2.co
Question: How is my external DNS configured?
Answer:Entry Resolves to
SRV record _collab-edge._tls.domain3.com xwayE.domain3.com port 8443
A record xwayE.domain3.com External IP address ExpressWay E
Domain and DNS configuration
ExpressWay Scenario 3Domain and DNS config ration
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
106/215
xwayC.domain3.com
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
cucm.domain2.co
Question: How is my ExpressWay E configured?
Answer:> System > DNS >- System host name xwayE- Domain name domain3.com
Domain and DNS configuration
ExpressWay Scenario 3Domain and DNS configuration
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
107/215
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain3.com cucm.domain2.co
Question: How is my ExpressWay C configured?
Answer:> System > DNS >
- System host name xwayC- Domain name domain3.com> Configuration > Domains >
- Domain domain1.com enabled for UCM registrations and IM and- Domain domain2.com enabled for UCM registrations and IM and- Domain domain3.com enabled for UCM registrations and IM and
Domain and DNS configuration
ExpressWay Scenario 3Domain and DNS configuration
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
108/215
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain3 com cucm.domain2.co
Question: How is my Internal DNS configured?
Answer:
xwayC.domain3 com
Entry Resolves to
SRV record _cisco-uds._tcp.domain3.com cucm.domain2.com port 84
A record cucm.domain2.com IP address CUCM
Domain and DNS configuration
ExpressWay Scenario 3Domain and DNS configuration
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
109/215
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain3 com
Question: How is my CUCM configured?Answer:> CCMADMIN > System > Server
- Server with hostname cucm> CLI set network domain domain2.com
xwayC.domain3 com
Domain and DNS configuration
ExpressWay Scenario 3Domain and DNS configuration
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
110/215
Jabber Client ExpressWay C Internal DNS CUCM Home UDSExpressway EExternal DNS
xwayE.domain1 com
Question: How is my CUP configu
Answer:> CUPAdmin > Clustertopology
- Node configuration with cup- IM and Presence Domain wi
xwayC.domain1 com cucm.domain2.co
Domain and DNS configuration
Troubleshooting - CNAME Considerations
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
111/215
g
Target URL Jabber can be subdomain of domain returned by HTTP(Expressway E)
-> Cookie domain : cisco.com-> Target URL : expressway.internal.cisco.com
Cookie is returned by server in get_edge_config responds
Cookie is save and re-used for subsequent HTTP requests
With correct domain/DNS/Alias configuration Jabber will sho
-> Cookies size = 1With incorrect domain/DNS/Alias configuration Jabber will sh-> Cookies size = 0
Jabber does not save the cookie and discovery will fail
Troubleshooting - CNAME Considerations
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
112/215
g
[csf.httpclient] [http::executeImpl] - *-----* HTTP response from:https://expway.cisco.com:8443/dmFyZGUuZGs/get_edge_config?service_name=_cisco-uds&service_name=_cupl
[csf.edge] [netutils::adapters::httpResponseToEdgeResponse] - Cookie : .cisco.com TRUE/TRUE 1421787961 X
4583-a433-5d56ed2671be
[csf.edge] [netutils::adapters::httpResponseToEdgeResponse] - Cookies size = 1
[csf.netutils.adapters] [netutils::adapters::EdgeUtilsAdapter::transformRequest] - TransformedUrls:https://expway.cisco.com:8443/dmFyZGUuZGsvaHR0cHMvMTAuMTg0LjEuNTIvODQ0Mw/cucm-uds/user/930
[edge::EdgeUtilsImpl::transformHttpCookies] - Transforming 0 Http Cookies for each transformedUrl -size: 2[csf.edge] [edge::EdgeUtilsImpl::getHttpCookies] - checking if http cookies can be returned from cached edge config
[csf.httpclient] [http::CurlHttpUtils::setCookies] - setting cookie : X-Auth
Jabber for each HTTP request will search for cached cookiesIf found and domain/target is matched will be used in subsequent requests
Troubleshooting - CNAME Considerations
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
113/215
[csf.httpclient] [http::executeImpl] - *-----* HTTP response from:https://expway.cisco.com:8443/dmFyZGUuZGs/get_edge_config?service_name=_cisco-uds&service_name=_cupl
[csf.edge] [netutils::adapters::httpResponseToEdgeResponse] - Cookie : .internal.com TRUE/TRUE 1421787961
e978-4583-a433-5d56ed2671be
[csf.edge] [netutils::adapters::httpResponseToEdgeResponse] - Cookies size = 0
** Discovery has failed. Calling Callback! **
Cookie domain does not match HTTP target domain
TroubleshootingExpressWay or UC Server Domain not configured
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
114/215
ExpressWay or UC Server Domain not configured
ExpressWay or UC server domain not addedor not enabled for Unified Communications
Jabber login will fail Cannot communicate with the server
Diagnostic logs will showHTTPMSG:|GEThttps:///Y29sdWMuY29t/get_edge_config?service_name=_cisco-uds&service_HTTP/1.1Authorization: xxxxxHost: xwaye.coluc.com:8443Accept: */*User-Agent: Jabber-Win-345
HTTPMSG:|HTTP/1.1 403 ForbiddenDate:Mon, 17 Mar 2014 16:07:20 GMTConnection: closeServer:CE_EContent-Length: 0|
Decodes to
TroubleshootingIM&P Domain not configured (UC Domain)
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
115/215
IM&P Domain not configured (UC Domain)
IM&P domain not added or not enabled for IM&P
Jabber login will fail Cannot communicate with the server
Diagnostic logs will showxwaye XCP_JABBERD[12144]: UTCTime="2014-03-14 14:30:25,310"ThreadID="140582990952192" Module="Jabber" Level="INFO Detail="bouncing a packet to 'domain3.com from 'cm-1_jsmcp-1.xwaye-dom
xwaye XCP_CM[12513]: UTCTime="2014-03-14 14:30:25,310" ThreadID="140Module="cm-1.xwaye-domain1.com" Level="INFO " CodeLocation="SASLManaDetail="Failed to query auth component for SASL mechanisms"
Tool bookmark
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
116/215
Service record lookups
https://mxtoolbox.com/NetworkTools.aspx
Tool bookmark
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
117/215
Tool bookmark
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
118/215
Base64 decoding/encoding
https://www.base64decode.org
Tool Bookmark - Jabber URL transform
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
119/215
Tool Bookmark - Jabber URL transform
- Jabber transforms original Url: http://colcm9pub:6970/CSFxwayj.cn
- Base Url with appended Edge domain: coluc.com/
- Base Url with appended protocol: coluc.com/http/
- Base Url with appended host: coluc.com/http/colcm9pub
- Base Url before encoding: coluc.com/http/colcm9pub/6970
- Encoded Base64 Url:Y29sdWMuY29tL2h0dHAvY29sY205cHViLzY
- Transformed Url:
https://xwaye.coluc.com:8443/Y29sdWMuY29tL2h0dHAvY29sY205=/CSFxwayj.cnf.xml
Tool bookmark Jabber get edge config
http://colcm9pub:6970/CSFxwayj.cnf.xmlhttp://colcm9pub:6970/CSFxwayj.cnf.xml -
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
120/215
Tool bookmark Jabber get_edge_config
A good way to verify that the basic MRA components are in place is tHTTP request Jabber would do.
To do this verification, open a browser and enter the following URL toHTTP Reverse proxy is working, and that the ExpressWay-C can dischttps://xwaye.coluc.com:8443/Y29sdWMuY29/get_edge_config?servsco-uds&service_name=_cuplogin
Use a CUCM User credentials when prompted by the browser
COLUC
Tool bookmark Jabber get edge config
https://xwaye.coluc.com:8443/Y29sdWMuY29/get_edge_config?service_name=_cisco-uds&service_name=_cuploginhttps://xwaye.coluc.com:8443/Y29sdWMuY29/get_edge_config?service_name=_cisco-uds&service_name=_cuploginhttps://xwaye.coluc.com:8443/Y29sdWMuY29/get_edge_config?service_name=_cisco-uds&service_name=_cuploginhttps://xwaye.coluc.com:8443/Y29sdWMuY29/get_edge_config?service_name=_cisco-uds&service_name=_cuplogin -
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
121/215
g _ g _ g
Service Config
Expressway Diagnostic Logs
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
122/215
Expressway Diagnostic Logs
Diagnostics logs
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
123/215
XMPP Federation
XMPP Federation Support
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
124/215
XMPP Federation Support
XMPP Federation on CUP
XMPP Federation Support
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
125/215
XMPP Federation Support
XMPP Federation on Expressway E
XMPP Federation Configuration Tasks
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
126/215
XMPP Federation Support
DNS vs Static
Dialback Secret Security mode
Privacy mode
Serviceability
XMPP Federation Support
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
127/215
Disable XMPP Federation on CUPCisco Unified CM IM and Presence Administration > Presence > In
Federation > XMPP Federation > Settings
XMPP Federation Support
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
128/215
Expressway CEnable Domain for XMPP Federation
Expressway EEnable XMPP Federation feature
XMPP Federation Support
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
129/215
Verify Notifications on CUP for restart XCP router
XMPP Federation Support
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
130/215
Verify Notifications on CUP for restart XCP router
XMPP Federation Support Expressway
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
131/215
Event="System Configuration Changed" Node="[email protected]="xconfiguration xcpS2SStatus uuid 9896d611-5603-408e-bec4-6remote_address: 10.48.55.113:7001 remote_address: 10.48.55.113:700
Event="System Configuration Changed" Node="[email protected]="xconfiguration xcpS2SStatus uuid 9896d611-5603-408e-bec4-6remote_address: 10.48.55.113:7001 s2s_realm: cm-2_s2scp-1.eft-xwye
Module="network.axl" Level="INFO" Action="Send"URL="https://ecup10.coluc.com:8443/axl/" Function="executeSQLQuery
admin:run sql select * from xmpps2snodespkid cp_id==================================================055c13d9-943d-459d-a3c6-af1d1176936d cm-2_s2scp-1.ef
CUP shows
XMPP Federation
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
132/215
XMPP Federation Support
DNS vs Static
Dialback Secret Security mode
Privacy mode
Serviceability
DNS vs Static Routes
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
133/215
DMZ
IM/P
5269
CM
XCP employee1@v
7400IM/P
XCP
5222
DOMAIN - COLUC.COM
UC IM&P Serv
Expressway-C
IM/P
StaticRouteXCP
Expressway-E
7400
DNS
SRV
lookup
S2S
5269
DNS vs Static Routes
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
134/215
Static routes = Off
Queries for SRV records
_XMPP-SERVER._TCP._XMPP-SERVER._TCP.
DNS vs Static Routes
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
135/215
Static routes = On
Queries static routes configured with failover to DNS
DNS vs Static Routes
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
136/215
Scenario - XMPP Federation with DNS
XCP_CM2[1382]:..Level="INFO " Detail="Starting resolver lookup for 'vngtp.lab:puny=vngtp.lab:service=_xmpp-s
XCP_CM2[1382]:..Level="DEBUGDetail="_lookup: look for static route for info->host=vngtp.lab:info->service=_xmp>socktype=1'
XCP_CM2[1382]:..Level="INFO " Detail="_lookupSRV: static routes not found, proceed to SRV lookup '
XCP_CM2[1382]:..Level="INFO " Detail="(54fe6aa8-687d-40d6-8954-8d9bac710652, coluc.com:vngtp.lab, OUT)resolved outbound address for host=vngtp.lab method=SRV _xmpp-server._tcp addrs=10.48.36.171:5269 ...
DNS vs Static
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
137/215
Scenario - XMPP Federation with Static Routes
XCP_CM2[20104]:..Level="INFO "..Detail="Starting resolver lookup for 'vngtp.lab:puny=vngtp.lab:service=_xmpp-servXCP_CM2[20104]:..Level="DEBUG"..Detail="_lookup:look for static route for info->host=vngtp.lab:info->service=_xm
>socktype=1'"
XCP_CM2[20104]:..Level="DEBUG"..Detail="_lookup:static route match static_route.GetDomain()=vngtp.lab'"
XCP_CM2[20104]:..Level="DEBUG"..Detail="_lookup:static route add host=10.48.36.171, port=5269'"
XCP_CM2[20104]:..Level="INFO "..Detail="_lookupSRV: static routes found'"
DNS vs Static
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
138/215
Scenario No Matching Configured Static Route, DNS Failover
XCP_CM2[24046]: ..Level="INFO "..Detail="Starting resolver lookup for 'vngtp.lab:puny=vngtp.lab:service=_xmpp-ser
XCP_CM2[24046]: ..Level="DEBUG"..Detail="_lookup: look for static route for info->host=vngtp.lab:info->service=_xmp>socktype=1'"
XCP_CM2[24046]: ..Level="1" Subject="cm-2.eft-xwye-a-coluc-com" Event="Static route did not match domain:[vngModule="XMPPFederation"
XCP_CM2[24046]: ..Level="INFO "..Detail="_lookupSRV: static routes not found, proceed to SRV lookup'"
XCP_CM2[24046]: ..Level="INFO "..Detail="Finished resolver lookup for 'vngtp.lab:puny=vngtp.lab:service=_xmpp-serve0.000652s"
DNS vs Static
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
139/215
When no static routes defined for a federated domain or chat nthe system will use DNS instead
If static routes are defined for the federated domain or chat nodbut the remote system cannot be contacted over those routes,the system will not fall back to DNS.
If Privacy mode is set to Allow list and Use static routes is Oany domains (or chat node aliases) that are configured as statiare included automatically in the allow list
XMPP Federation
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
140/215
XMPP Federation Support
DNS vs Static
Dialback Secret Security mode
Privacy mode
Serviceability
XMPP Federation Server Dialback
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
141/215
XMPP Federation Server Dialback
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
142/215
What is Dialback server?Identity verification with federated domain
Initiating ServerReceiving Server Authoritative Server
Send dialback key
Send verify request
Send verify responds
Report dialback result
DOMAIN1 DOMAIN2 DOMAIN1
XMPP Federation Server DialbackI iti ti S
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
143/215
Initiating Server
XCP_CM2[12122]:.. Level="INFO " CodeLocation="stanza.component.out" Detail="xcoder=34A9B60C8 sending::d780f198ac34a6dbd795fcdaf8762eaf52ea9b03"
XCP_CM2[12122]:.. Level="DEBUG" CodeLocation="stream.out" Detail="(00000000-0000-0000-0000-000000000000, c
xcoder=34A9B60C8 Scheduling dialback timeout in 30 secs."
XCP_CM2[12122]:.. Level="INFO " CodeLocation="ConnInfoHistory" Detail="Connection state change: PENDING->C
XMPP Federation Server Dialback
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
144/215
Receiving Server
XCP_CM2[22992]:.. Level="VBOSE" CodeLocation="stanza.component.in" Detail="xcoder=05E295A2B received::d780f198ac34a6dbd795fcdaf8762eaf52ea9b03
..XCP_CM2[22992]:.. Level="INFO " CodeLocation="Resolver.cpp:128" Detail="Starting resolver lookup for 'coluc.com:puny=coluc.com:service=_xmpp-server._tcp:defport=0'..XCP_CM2[22992]:.. Level="INFO " CodeLocation="debug" Detail="(e5b18d01-fe24-4290-bba1-a57788a76468, vngtpresolved dialback address for host=coluc.com method=SRV dns-timings=(TOTAL:0.003157 SRV:0.002885)..XCP_CM2[22992]:.. Level="INFO " CodeLocation="DBVerify.cpp:270" Detail="(e5b18d01-fe24-4290-bba1-a57788a76IN)DBVerify stream is open. Sending db:verify packet: d780f198ac34a6dbd795fcdaf8762eaf52ea9b03
..XCP_CM2[22992]:.. Level="INFO " CodeLocation="DBVerify.cpp:282" Detail="(e5b18d01-fe24-4290-bba1-a57788a76IN)DBVerify Packet Received d780f198ac34a6dbd795fcdaf8762eaf52ea9b03
XMPP Federation Server Dialback
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
145/215
Receiving Server
XMPP Federation Server Dialback
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
146/215
Authoritative Server
XCP_CM2[5164]:..Level="INFO " CodeLocation="debug" Detail="xcoder=94A9B60C8 onStreamOpen::
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
147/215
Tool Bookmark - Wireshark
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
148/215
XMPP Federation Server Dialback
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
149/215
Scenario - DNS Problem on ReceivingInitiator shows
XMPP Federation Server Dialback
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
150/215
Scenario - DNS problem on Receiving ServerReceiving Server event log show
XMPP Federation Server Dialback
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
151/215
After timeout XMPP traffic will fail Domain pair blocked for 30min
XCP_CM2[21104]: CodeLocation="stanza.router.in" Detail="cm-2_s2scp-1.eft-xwye-a-coluc-com onPacket::
XCP_CM2[21104]: CodeLocation="debug" Detail="Bouncing packet because domain pair (453d2518-9894-4bb2-ae7coluc.com:vngtp.lab, OUT) is marked as failed:
Result is that Jabber user continues to receiveMessage to user could not be delivered
Correct problem and restart XCP (Expressway)
After 30min domain pair state is cleared again
XMPP Federation
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
152/215
XMPP Federation Support
DNS vs Static
Dialback Secret
Security mode
Privacy mode
Serviceability
XMPP Federation Security Mode
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
153/215
TLS RequiredAlways TLS
TLS OptionalAttempts TLSfalls back to TCP
No TLSAlways TCP
XMPP Federation Security Mode
C tifi t i t
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
154/215
Certificate requirements:
Must contain SAN with XMPP domain
(Optional) contain SAN with XMPP Chat node alias
TroubleshootingReceiver required TLS, initiator Req or optio
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
155/215
TroubleshootingReceiver required TLS, initiator Req or optio
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
156/215
TroubleshootingReceiver required TLS, initiator no TLS
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
157/215
TroubleshootingReceiver required TLS, initiator no TLS
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
158/215
TroubleshootingReceiver optional TLS, initiator TLS optional or
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
159/215
Initiator TLS
TroubleshootingReceiver no TLS, initiator required
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
160/215
TroubleshootingReceiver no TLS, initiator required
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
161/215
TroubleshootingDomain not contained in server certificate
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
162/215
Initiating Server logs
XCP_CM2[21722]:..Level="VBOSE" CodeLocation="stanza.router.in" Detail="cm-2_s2scp-1.ExpresswayE-vngtp-labfrom='[email protected]/jabber_6705' to='[email protected]
type='groupchat' xml:lang='en'>
XCP_CM2[21722]:..Level="INFO " CodeLocation="ConnInfoHistory" Detail="Connection state change: IDLE_TIMEO(f8d3c3d4-27df-4cf2-88d2-625090104543, vngtp.lab:conference-4-standaloneclusterf1fa2.coluc.com, OUT) state=PE
XCP_CM2[21722]:..Level="INFO " CodeLocation="Resolver.cpp:143" Detail="Finished resolver lookup for 'conferstandaloneclusterf1fa2.coluc.com:puny=conference-4-standaloneclusterf1fa2.coluc.com:service=_xmpp-serve0.001163s" XCP_CM2[21722]:..Level="DEBUG" CodeLocation="stream.out" Detail="xcoder=2783DD838 ne
XCP_CM2[21722]:..Level="INFO " CodeLocation="XMPPStream.cpp:2395" Detail="The hostname conference-4-standaloneclusterf1fa2.coluc.com was not found on the SSL certificate: 'eft-xwye-a.coluc.com' ... Disconnec
Security Mode TLS Required/OptionalRequire client-side security certificates
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
163/215
Verifies CA/Issuer from certificate presented by foreign domain
TLS negotiation will fail when CA root is not uploaded to ExpresswaCA root list
Falls back to TCP when TLS is optional
Fails when TLS is required
TroubleshootingCA not uploaded to initiator trust store
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
164/215
TroubleshootingCA not uploaded to initiator trust store
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
165/215
XMPP Federation
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
166/215
XMPP Federation Support
DNS vs Static
Dialback Secret
Security mode
Privacy mode
Serviceability
XMPP Federation - Privacy
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
167/215
Allow list white list
Contains domains and chat node aliases with which federation is Deny list black list
Contains domains and chat node aliases with which federation is
XMPP Federation - Privacy
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
168/215
Scenario Initiating Server Allow list does not contain foreign domain
XCP_CM2[5366]:..Level="INFO " CodeLocation="Resolver.cpp:128" Detail="Starting resolver lookup for 'vngtp.lab:puny=vnserver._tcp:defport=0'
XCP_CM2[5366]:..Level="INFO " CodeLocation="stream.out" Detail="(3b86cdb2-af61-4d5e-a50b-e9875ebb8d4a, coluc.cohost:vngtp.lab using addrs:10.48.36.171:5269
XCP_CM2[5366]:..Level="DEBUG" CodeLocation="debug" Detail="authorizeOutToAddr is returning false for: 10.48.36.
XCP_CM2[5366]:..Level="INFO " CodeLocation="stream.out" Detail="(3b86cdb2-af61-4d5e-a50b-e9875ebb8d4a, coluc.coresolved address is on blacklist host:vngtp.lab ip:10.48.36.171:5269"
Troubleshooting PrivacyReceiving Server Allow list does not contain sXCP CM2[8002]: Level="INFO " CodeLocation="debug" Detail="xcoder=21107F2AE onStreamOpen::
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
169/215
XCP_CM2[8002]:..Level= INFO CodeLocation= debug Detail= xcoder=21107F2AE onStreamOpen::
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
170/215
XMPP Federation
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
171/215
XMPP Federation Support
DNS vs Static
Dialback Secret
Security mode
Privacy mode
Serviceability
XMPP Federation Serviceability
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
172/215
XMPP Federation Serviceability
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
173/215
Domain pair blocked for 30min
Connection State from None to Pending to Connected or Fai
10 Retries
XMPP Federation Serviceability
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
174/215
XMPP Federation Serviceability
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
175/215
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
176/215
Business to Businesscalls
Business to Business calls
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
177/215
Enterprise Network DMZ
CUCM
Expressway-CCollab Gateway
Expressway-ECollab Gateway
Internet
Traversal Link MaSignal
Media Payload
Business to Business calls - Configuration
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
178/215
CUCM
Expressway-C Expressway-E
Internet
DNS ZoneTraversalZone Server
TraversalZone Client
SIP Trunk
URI Dialing
NeighborZone
Dialplan(Search Rules,Transforms ..)
Bussiness to Business SIP Trunk
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
179/215
Edge traffic Device registration
B2B traffic Trunk Calls
Bussiness to Business SIP Trunk
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
180/215
None Secure SIP Trunk
Bussiness to Business SIP Trunk
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
181/215
Secure SIP Trunk
FQDN Expressway C Server
Business to Business Traversal Zone
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
182/215
UC Traversal
B2B Traversal
Business to Business Traversal Zone
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
183/215
Business to Business DNS Zone
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
184/215
DNS lookup for SRV
_sip._tcp.domain_sips._tls.domain_h323cs._tcp.domain_h323ls._udp.domain
TroubleshootingINVITE send to wrong IP address
E E i f ll i INVITE f CUCM
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
185/215
Expressway E receives following INVITE from CUCM
Module="network.sip" Level="DEBUG": Src-ip="10.48.79.189" Src-port="2501
SIPMSG:|INVITE sip:[email protected]:5060 SIP/2.0
When port information is included in URI Expressway E will useresult from DNS A record lookup for domain and not SRV for SIP s
This results in INVITE send to wrong IP address
Solution : Configure Transform rule which strips port from URI
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
186/215
Key TakeAways
Key TakeAways
Re ie Fire all ports
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
187/215
Review Firewall ports
Review Certificate Requirements
Review UC Domains on Expressway C
Review Services on the UC domain
SRV records for the different services must exist in DNS with Split
Trunk vs Line
Continue the Conversation using Cisco Spar
Sign up free for Cisco Spark at http://www ciscospark com/
http://www.ciscospark.com/http://www.ciscospark.com/ -
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
188/215
Sign up free for Cisco Spark at http://www.ciscospark.com/
Download the application from iOS App Store, Google Play Store,
http://download.ciscospark.com/ Visit the World of Solutions Cisco Spark area for demos
Use Cisco Spark to continue the conversation or ask any additionawith the speaker for this session. The room name is BRKCOL-260
How to get added to the Cisco Spark room for this session
To opt in, send an email to [email protected] with the messagadd me to the BRKCOL-2602 room
Participate in the My Favorite Speaker ConPromote Your Favorite Speaker and You Could Be a Winner
http://www.ciscospark.com/http://download.ciscospark.com/mailto:[email protected]:[email protected]://download.ciscospark.com/http://www.ciscospark.com/ -
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
189/215
Promote your favorite speaker through Twitter and you could win $Press products (@CiscoPress)
Send a tweet and include Your favorite speakers Twitter handle @PhilipSmeuninx
Two hashtags: #CLUS #MyFavoriteSpeaker
You can submit an entry for more than one of your favorite speak
Dont forget to follow @CiscoLive and @CiscoPress
View the official rules at http://bit.ly/CLUSwin
Complete Your Online Session Evaluation
Give us your feedback to be
http://bit.ly/CLUSwinhttp://bit.ly/CLUSwin -
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
190/215
Dont forget: Cisco Live sessionfor viewing on-demand after theCiscoLive.com/Online
Give us your feedback to beentered into a Daily SurveyDrawing. A daily winnerwill receive a $750 Amazongift card.
Complete your session surveysthough the Cisco Live mobileapp or your computer on
Cisco Live Connect.
Continue Your Education
Demos in the Cisco campus
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
191/215
Demos in the Cisco campus
Walk-in Self-Paced Labs
Table Topics
Meet the Engineer 1:1 meetings
Related sessions
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
192/215
Thank you
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
193/215
Appendix
Other useful HTTP query to run XCP route
To verify XCP router status run following :
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
194/215
To verify XCP router status run following :https://getxml?location=/Status/XMPP
Other useful HTTP query to run XCP route
Enter Expressway credentials (administrator login)
https://10.53.52.167/getxml?location=/Status/XMPPhttps://10.53.52.167/getxml?location=/Status/XMPP -
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
195/215
Enter Expressway credentials (administrator login)
Other useful HTTP query to run XCP route
ExpressWay E
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
196/215
ExpressWay E
Other useful HTTP query to run XCP route
ExpressWay C
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
197/215
ExpressWay C
Register Jabber client on UCM via MRA
Jabber Registration Walk Trough
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
198/215
Expected signaling flow for Jabber Client logon and registration on simple IM&P deployment
Jabber login [email protected]
Jabber Client ExpressWay C Internal DNS CUCM HomeUDS
TS
Expressway EExternal DNS
Jabber Registration Walk Trough
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
199/215
Jabber Client ExpressWay C Internal DNS CUCM HomeUDS
TS
Expressway EExternal DNS
DNS Query
SRV _cisco-uds._tcp.coluc.com
Query Response
DNS Query
SRV _cuplogin._tcp.coluc.com
Not Found
Query Response
Not Found
Jabber Registration Walk Trough
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
200/215
ExpressWay C Internal DNS CUCM HomeUDS
TS
DNS Query
SRV _collab-edge._tls.coluc.com
Query Response
(Contain Answers including SRV and A/AAAA record)
Service: collab-edgeProtocol: tlsName: coluc.comType: SRVPort: 8443Target: xwaye.coluc.com
SRV coluc.com
DNS Query
A xwaye.coluc.com
Query Response
(Contain Answers including A/AAAA record)
Name: xwaye.coluc.comType: A
Addr: 122.208.118.4
Jabber Client Expressway EExternal DNS
Jabber Registration Walk Trough
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
201/215
Expressway C Internal DNS CUCM HomeUDS
TS
SSL: Client Hello
SSL: Server Hello
SSL: Certificate, Server Hello Done
HTTPS
HTTPS: GET /get_edge_config
HTTPMSG:GET https:///Y2lzY290cC5jb20/get_edge_config HTTP/1.1
Authorization: xxxxx
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
202/215
ExpressWay C Internal DNS CUCM HomeUDS
TS
DNS Query
SRV _cisco-uds._tcp.coluc.com
Query Response
(Target: colcm9pub.coluc.com)
DNS Query
A colcm9pub.coluc.com
Query Response
(Addr: 172.16.1.36
Jabber Client Expressway EExternal DNS
When DNS record is not cached ExpressWay C will send out following DNS queries
SRV _cisco-phone-tftp._tcp.coluc.com
Query Response
(Target: colcm9pub.coluc.com)
Jabber Registration Walk Trough
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
203/215
ExpressWay C Internal DNS CUCM HomeUDS TS
DNS Query
SRV _cuplogin._tcp.coluc.com
Query Response
(Target: colcup.coluc.com)
DNS Query
A colcup.coluc.com
Query Response
(Addr: 172.16.1.33)
Jabber Client Expressway EExternal DNS
Jabber Registration Walk Trough
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
204/215
Expressway C Internal DNS CUCM HomeUDS TS
HTTP(S)
HTTPS: GET ///cucm-uds/clusterUser?
HTTPMSG:GET //colcm9pub:8443/cucm-uds/clusterUser?username=xwayj HTTP/1.1
Jabber Client Expressway EExternal DNS
HTTP(S) 200 OK
HTTPMSG:HTTP/1.1 200 OKContent-Type: application/xmlServer:172.16.1.36
Requesting CUCM home node information
Should see Found user cluster and Found UDS server internal status log this point in diagnostic log
===========================================================Module="developer.edgeconfigprovisioning.server" Level="DEBUG"CodeLocation="edgeconfigprovisioningserver(655)"Detail="Found user cluster" Username=xwayj"Cluster="172.16.1.36
Module="developer.edgeconfigprovisioning.server" Level="DEBUG"CodeLocation="edgeconfigprovisioningserver(682)" Detail="Found UDS server" Cluster="172.16.1.36"UdsServer="colcm9pub===========================================================
Jabber Registration Walk Trough
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
205/215
ExpressWay C Internal DNS CUCM HomeUDS TS
HTTP(S)
HTTPS: GET ///cucm-uds/user//devices
HTTPMSG:GET //colcm9pub:8443/cucm-uds/user/xwayj/devices HTTP/1.1
Authorization:
Jabber Client Expressway EExternal DNS
HTTP(S) 200 OK
HTTPMSG:HTTP/1.1 200 OKSet-Cookie: JSESSIONIDSSO=xxxxx, Path=/; Secure; HttpOnlySet-Cookie: JSESSIONID=xxxxx; Path=/cucm-uds/; Secure; HttpOnly
Content-Type: application/xml663e40ed-b3bd-3b6721d04c32eCSFxwayjCisco Unified Client Services Fram |
Get Devices
Jabber Registration Walk Trough
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
206/215
ExpressWay C Internal DNS CUCM HomeUDS TS
HTTPS 200 OK
HTTPMSG:HTTP/1.1 200 OKServer: CE_C ECSSet-Cookie: X-Auth=; Expires=xxxxx; Domain=.coluc.com; Path=/; Secure _cisco-phone-tftp0069colcm9pub.coluc.com_cuplogin008443imp33.coluc.com .. |
Jabber Client Expressway EExternal DNS
HTTPS 200 OK
HTTPMSG:HTTP/1.1 200 OKServer: CE_C ECSSet-Cookie: X-Auth=; Expires=xxxxx; Domain=.coluc.com; Path=/; Secure _cisco-phone-tftp0069colcm9pub.coluc.com_cuplogin008443imp33.coluc.com .. |
Returned configuration:1) IMP, CUCM, TFTP SRV2) SIP edge3) Randomized list of UDS4) XMPP edge5) HTTP edgeetc.
Jabber Registration Walk Trough
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
207/215
ExpressWay C Internal DNS CUCM HomeUDS TS
HTTPS
Jabber Client Expressway EExternal DNS
HTTPS: GET /jabber-config.xml
HTTPMSG:GET https:///...../jabber-config.xml HTTP/1.1Host: xwaye.coluc.com:8443Cookie: X-Auth=User-Agent: Jabber-Win-746
HTTPS: POST /EPASSoap/service/ login
HTTPMSG:POST https:///...../EPASSoap/service/v80 HTTP/1.1Host: xwaye.coluc.com:8443User-Agent: gSOAP/2.8User-Agent: Jabber-Win-746Cookie: $Version=1;X-Auth=;$Path="/";$Domain=".coluc.comSOAPAction: "urn:cisco:epas:soap/EpasSoapServiceInterface/login"
.
Jabber Registration Walk Trough
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
208/215
ExpressWay C Internal DNS CUCM HomeUDS TS
HTTPS
Jabber Client Expressway EExternal DNS
HTTPS: GET /EPASSoap/service / CTLSEP.tlv
HTTPMSG:GET https:///...../CTLSEPCSFxwayj.tlv HTTP/1.1
Authorization: xxxxxHost: xwaye.coluc.com:8443Cookie: X-Auth=User-Agent: Jabber-Win-746
HTTPS: GET /EPASSoap/service / CTLSEP.cnf.xml
HTTPMSG:GET https:///....../CSFxwayj.cnf.xml HTTP/1.1
Authorization: xxxxx
Host: xwaye.coluc.com:8443Cookie: X-Auth=User-Agent: Jabber-Win-746
Jabber Registration Walk Trough
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
209/215
ExpressWay C Internal DNS CUCM HomeUDS TSJabber Client Expressway EExternal DNS
SIP - REFER
REFER sip:colcm9pub SIP/2.0Via: SIP/2.0/TLS 10.71.50.153:50036;branch=z9hG4bK00007a0dCall-ID: [email protected]: 1000 REFERFrom: ;tag=081196545e6500020000428b-00005ddfTo: Route: ,,
SIP 407 Proxy
Authentication Required
Client includes the route set received atstartup negotiation
Jabber Registration Walk Trough
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
210/215
ExpressWay C Internal DNS CUCM HomeUDS TS
SIP - REFER
Jabber Client Expressway EExternal DNS
REFER sip:colcm9pub SIP/2.0Via: SIP/2.0/TLS 10.71.50.153:50036;branch=z9hG4bK00007a0dCall-ID: [email protected]: 1001 REFERFrom: ;tag=081196545e6500020000428b-00005ddfTo: Route: ,,Proxy-Authorization: Digest username="xwayj", realm="xwaye.coluc.com", uri="sip:colcm9pub",response="4900cdfe65c4a4551f1129903c9ed98d", nonce=xxxxx", opaque=xxxxx", cnonce="000030a0", qop=auth,nc=00000001, algorithm=MD5
Jabber Registration Walk Trough
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
211/215
ExpressWay C Internal DNS CUCM HomeUDS TS
SIP - REFER
Jabber Client Expressway EExternal DNS
REFER sip:colcm9pub SIP/2.0Via: SIP/2.0/TLS 10.71.50.153:50036;branch=z9hG4bK00007a0dCall-ID: [email protected]: 1001 REFERRefer-To: Referred-By: From: ;tag=081196545e6500020000428b-00005ddfTo: Route: P-Asserted-Identity:
SIP - REFER
REFER sip:colcm9pub SIP/2.0Via: SIP/2.0/TLS 10.71.50.153:50036;branch=z9hG4bK00007a0dCall-ID: [email protected]: 1001 REFERRefer-To: Referred-By: From: ;tag=081196545e6500020000428b-00005ddfTo: Route: P-Asserted-Identity:
Jabber Registration Walk Trough
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
212/215
ExpressWay C Internal DNS CUCM HomeUDS TSJabber Client Expressway EExternal DNS
SIP
202 Accepted
SIP
202 Accepted
SIP
202 Accepted
Registration request including Contact andall Route information
SIP - REGISTER
REGISTER sip:colcm9pub SIP/2.0Via: SIP/2.0/TLS 10.71.50.153:50036;branch=z9hG4bK00007a0dCall-ID: [email protected]: 101 REGISTERContact: ;+sip.instance="";+sip.instance="";+u.sip!devicename.ccm.cisco.com="CSFxwayj";+u.sip!model.ccm.cisco.com="503";videoFrom: ;tag=081196545e6500020000428b-00005ddfTo: Route: ,,
SIP 407 Proxy
Authentication Required
Jabber Registration Walk Trough
-
7/24/2019 BRKCOL-2602 - Collaboration Edge Troubleshooting (2015 San Diego).pdf
213/215
ExpressWay C Internal DNS CUCM HomeUDS TS
SIP - REGISTER
Jabber Client Expressway EExternal DNS
REGISTER sip:colcm9pub SIP/2.0Via: SIP/2.0/TLS 10.71.50.153:50036;branch=..
CSeq: 102 REGISTERContact: ..+u.sip!devicename.ccm.cisco.com="CSFxwayj";+u.sip!model.ccm.cisco.com="503"From: ;tag=081196545e6500020000428b-00005ddfTo: Proxy-Authorization: Digest username="xwayj", realm="xwaye.coluc.com", uri="sip:colcm9pub",response="4900c