accelerate end-to-end troubleshooting in mobile … · introduction what does ericsson do? testing...
TRANSCRIPT
Accelerate end-to-end troubleshooting in mobile networks
Pa
trik
Pa
ks
y
Be
ne
de
k K
ová
cs
What does Ericsson do?Introduction
Testing
challenge
Proposed
idea
Case studies
Take away
Network of Softwares
› Network elements = separate software instances– Called network functions or nodes
› Network– Connection of network elements
– Communication according to standards
› Users’ view: „I would like to call somebody”– Through the complex network many errors could happen
– Focus on the end-to-end view of the network
Introduction
Testing
challenge
Proposed
idea
Case studies
Take away
Receives signaling message
Forwards signaling messageResponse signaling message
Initiates signaling message
A B
CState machine
User Data Management domain2G / 3G network domain
MSC
RNCNodeB
BSCBTS
2G/3GUser
HLR/VLR
SGSN
GGSN
Internet
4G network domain
eNodeB
4GUser
MME PCRF
SGW PGW
HSS/SLF
IP Multimedia Subsystem (IMS) domain
P-CSCF
I/S-CSCF
E-CSCF/E
ATF
MMTel AS
SCC AS
MGWMRFP
MGCF
BGCF
MRFC
TrGW
IBCF
BGF
Presence Server
Complexity growth
AAA
1991-2010
2G to 4G
Number of network
functions doubled
2014
Voice over 4G
2x complexity
2014-2015
Voice over WiFi
+ few functions
Increasing complexity leads to
higher risk & difficulty in maintainability
ePDGInternet
Internet
Introduction
Testing
challenge
Proposed
idea
Case studies
Take away
Test levels
› Focus on end-to-end view
› V-model
› Solution test
Solution
test
Domain test
Product test
Software logic test
Introduction
Testing
challenge
Proposed
idea
Case studies
Take away
Test levels
Ag
ile t
est m
eth
od:
end-t
o-e
nd t
esting
?
Introduction
Testing
challenge
Proposed
idea
Case studies
Take away
Solution
test
Domain test
Product test
Software logic test
In operationSystem / Solution Test
Mobile
networkLoad generator
Logging or
traffic capturingUser equipment
Test & operation
framework
Introduction
Testing
challenge
Proposed
idea
Case studies
Take awaySignaling
messages
Challenges ideas
Challenges- Network complexity growth
- Solution level testing is a key for user experience
- End-to-end traffic is large huge dataset
- Details are must
- Offline to real-time
How-to+ Utilize probing technology and analytics
to verify performance
+ Scale-up test processes
+ Search new methods
+ Improve environment & tools
+ Big data handling & filtering
Introduction
Testing
challenge
Proposed
idea
Case studies
Take away
Operational support
system requirementMonitor system behavior + Fast troubleshooting
In service test
› Apply the same technology
–For testing
–For monitoring and
troubleshooting
Introduction
Testing
challenge
Proposed
idea
Case studies
Take away
Solution
test
Domain test
Product test
Software logic test
Testing is not enough
Testing +
Monitoring
Troubleshooting
End-to-end
On network level
With Big Data handling
=
Continous
live testing
Fast
feedback
+
Introduction
Testing
challenge
Proposed
idea
Case studies
Take away
Network observability
Hard to analyze problems
in live networks with high load
Logging MB to GB / day
- Difficult to analyze
- No well-defined format
- It may differ node to node
/ vendor to vendor
Traffic capturing More TBs / day
- Difficult to capture
- Many different sulutions
- Multiple capture points
in a large network
- Not all the data is relevant
Node reports
performance data
Observe communication
between nodes
Introduction
Testing
challenge
Proposed
idea
Case studies
Take away
Case Study - CATHow to deal with these challenges
›Call-flow Analyzer Tool–Local innovation project in Hungary
–Purpose
› Speed-up troubleshooting
› Point out error cases
› Help in emergency situation
–Use cases
› Offline solution for post-analysis
› Automated test case evaluation
Not only repeat manual
steps, but provide useful
information which is
hard to get manually
Introduction
Testing
challenge
Proposed
idea
Case studies
Take away
Case Study - CATHow to deal with these challenges
›STEPS
–Understand the manual process
› Build the knowledge into the tool
–Understand and filter data set
› Define error scenarios
› Describe system states and rules
› Define data set to be collected
Introduction
Testing
challenge
Proposed
idea
Case studies
Take away
Details needed
for troubleshooting
All the details NOT needed
for monitoring
Collect data
separately
from monitoring
Reference
to the stored data
Filt
erin
gSto
ring
Network traffic
Case Study - CATManual way to automatic
Introduction
Testing
challenge
Proposed
idea
Case studies
Take away
List of messages(different flows might overlap)
Assembled message flows
+Various filtering possibility
Point out errors
Additional information
Visualization
U
E
Case Study - CATProvided data for troubleshooting
Introduction
Testing
challenge
Proposed
idea
Case studies
Take away
Call
Attach
Registration
Call type 1
(success)
Call type 2
(success)
Call type 3
(failed)
...Key Performance Indicators & statistics
Service / user level information
CAT, automation
Case Study - CATResult
1 GB trace analysis
in offline mode, after data collection
Automation + additional data
10-20mins
1-2hours
Manual flow
Introduction
Testing
challenge
Proposed
idea
Case studies
Take away
Take away...Focus on end-to-end system
behavior and drill down to
the details
Not only reproduce a
manual flow, but add
new value with heuristics
Handle data storing
& analysis separately,
use reference to the raw data
Find similarities between
testing and operation
in your system
Introduction
Testing
challenge
Proposed
idea
Case studies
Take away
› Contact
– Benedek Kovács (System Engineer @ Ericsson)
– Patrik Paksy (Product Owner @ Ericsson)
Q&A
