[email protected]© Omnitele Ltd. 2005 1
Analysis of Service Quality in 3G Mobile Networks
Heidi Lagerström
Supervisor: Professor Heikki Hämmäinen
Instructors: M.Sc. Sami Vesala & M.Sc. Katja Koivu
[email protected]© Omnitele Ltd. 2005 2
Contents
1. Introduction to the study• Background, research problem, research methods
2. Quality of Service (QoS) in UMTS Networks
3. Measuring service quality• Defining Key Performance Indicators (KPI)
4. Case study
[email protected]© Omnitele Ltd. 2005 3
Background
• UMTS introduces new real time services to mobile networks, such as video telephony.
• These real time services require QoS guarantees to function properly.
• For operators to maintain satisfactory service quality constant network monitoring is needed.
• Network measurements are based on correctly defined KPIs for each service.
Operators’ possibilities to utilise QoS in practice have not been widely researched.
Key Performance Indicators have not been defined for the new services from the end-user perspective.
[email protected]© Omnitele Ltd. 2005 4
Research problem
Objectives:
1. What are the KPIs that measure service quality, from end user perspective, in 3G networks for the key services (AMR voice, video telephony, video streaming, web browsing and e-mail)?
2. What are the QoS mechanisms in Release 99 and how can they be used to improve service quality?
How should service quality be measured in 3G networks and how the QoS mechanisms can be used to affect the
service quality perceived by subscribers.
[email protected]© Omnitele Ltd. 2005 5
Research methods
• Literature study– 3GPP, ETSI, ITU specifications– Several books and publications
• Interviews– Network equipment vendors: Ericsson, Nokia– Operators: Elisa– Several other radio network experts
• Case study– Field measurements for two
operators in live networks
[email protected]© Omnitele Ltd. 2005 6
Contents
1. Introduction to the study• Background, research problem, research methods
2. Quality of Service (QoS) in UMTS Networks
3. Measuring service quality• Defining Key Performance Indicators (KPI)
4. Case study
[email protected]© Omnitele Ltd. 2005 7
Why do we need QoS?
• UMTS networks support services with very different performance requirements
– Real-time services require performance guarantees
– Customer acceptance closely tied to service quality
• Optimal usage of network resources
– Radio resources scarce
– Cost-effectiveness
– Return of investment
• Service and user differentiation
– Meet different needs of customers (e.g. business vs. consumer)
– Support different services (real-time vs. best effort)
• Competitive advantage!
Delay Jitter Loss
Video call High High High Med
Streaming High Med Med Med
Web browsing Med Med Low High
E-mail Low Low Low High
Application BandwidthSensitivity
Performance Requirements
[email protected]© Omnitele Ltd. 2005 8
QoS Traffic Classes
Traffic class Characteristics Example application
Conversational Preserve time relation between information entities of the stream. Conversational pattern (stringent and low delay)
Speech
Video calls
Streaming Preserve time relation between information entities of the stream.
Real-time streaming video
Interactive Request-response pattern. Preserve payload content.
Web browsing
Background Destination is not expecting the data within a certain time. Preserve payload content.
File downloading
Demanding• Delay• Jitter
Demanding• Bit rate• Jitter
Tolerant• Delay and bit rate
can vary• Integrity
Easiest• Delay and bit rate
can vary• Integrity
[email protected]© Omnitele Ltd. 2005 9
QoS Profile Attributes
R99 QoS attribute Example valueResidual BER 10 –5
SDU error ratio 10 –4
Delivery of erraneous SDUs No
Maximum SDU size (octets) 1500
Delivery order No
Transfer delay 100 ms (conversational)
280 ms (streaming)
ARP 1, 2 or 3
Traffic Class Conversational, streaming, interactive, background
THP 1, 2 or 3 (same as ARP)
Maximum allowed bit rate e.g. 64, 128 or 384 kbps
Maximum guaranteed bit rate e.g. 64, 128 or 384 kbps
Depends on operator’s
QoS strategy
Depends on the QoS strategy
and UE/RNC capabilities
[email protected]© Omnitele Ltd. 2005 10
QoS Differentiation
Conversational RAB
Streaming RAB
Interactive RAB, THP/ARP = 1
Interactive RAB, THP/ARP = 3
Background RAB
MMS
Web browsing
Push-to-talk
Streaming
Videotelephony
• Each service gets the treatment it requires according to the QoS profile• Network resources are shared according to the service needs• Network resources can be used more efficiently
[email protected]© Omnitele Ltd. 2005 11
QoS Mechanisms
External IP (Internet)
External IP (Internet)
UE Node B
TE
Firewall
RNC 3G-SGSN 3G-GGSNUTRAN
Iu Gn Gn
IP
PS Domain
Inter-PLMN Backbone NT
Inter-PLMN Backbone NT
DiffServ on transport level IP(ATM QoS for CS)
Diffserv on transport level IP
Diffserv in Gi
Different channel
types
ATM QoS
• Different QoS techniques in different parts of the network• Appropriate QoS must be provided in every network so that the user can experience good service quality
PDP context with the requested QoS capabilities
RRM
[email protected]© Omnitele Ltd. 2005 12
Operators’ QoS Strategy
Conversational RAB
Streaming RAB
Interactive RAB, THP/ARP = 1
Interactive RAB, THP/ARP = 3
Background RAB
RNC3G
GGSN3G
SGSN
Applicationserver
Node B
HLR
User profiles are stored in HLR.Each user can have several user
profiles, which correspond to different services and are mapped
to different bearers according to the operator’s strategy.
• Operators can practise user differentiation by giving each user set of QoS profiles, which he/she is entitled to use• Operators can practise service differentiation by mapping each service to the bearer that meets its requirements
• Meet the needs of different customers• Offer each service the quality it requires• Optimise network resource usage
[email protected]© Omnitele Ltd. 2005 13
Contents
1. Introduction to the study• Background, research problem, research methods
2. Quality of Service (QoS) in UMTS Networks
3. Measuring service quality• Defining Key Performance Indicators (KPI)
4. Case study
[email protected]© Omnitele Ltd. 2005 14
Measuring network performance
RNC3G
GGSN3G
SGSN
Applicationserver
Node B
UTRAN Core nw External nw
E2E service quality,
QoE
Performance statistics from
application serverNetwork statistics from different counters and interfaces
Customerfeedback
Network Performance Monitoring Optimisation
[email protected]© Omnitele Ltd. 2005 15
Defining the right KPIs
• Different services have different quality requirements– KPIs must be defined separately for each of the key services
• KPI categories– Service Accessibility– Service Integrity– Service Retainability
• With inadequate performance indicators and monitoring– Hidden problems in network performance and user perceived
quality of service– Poorly defined indicators may show better quality than in the reality
• Incorrect formulas and counters• Unreasonable measurement periods
(too much averaging etc.)
[email protected]© Omnitele Ltd. 2005 16
Example: Voice Services - CS
Customer demand
Indicator Measure
Service accessability
Availability & Coverage
Call setup success rate
Call setup delay
Ec/No, RSCP
Admission control
RAB assignment
Service integrity Voice quality Noisy frames (FER), MOS
Service retainability
Call drop rate Handover failure
No coverage
Interference
[email protected]© Omnitele Ltd. 2005 17
Example: Data Services - PS
Customer demand Indicators Measures
Service accessability
Availability & Coverage
Access success rate
Service access delay
Ec/No, RSCP
Admission control
Attach, PDP context activation, IP service setup
Service integrity Video quality
Audio quality
Web page download time
E-mail sending time, etc.
BLER, FER, throughput, delay, jitter
Service retainability
Dropped data connection
Connection timeouts
Dropped PDP context/attach
No coverage etc.
Handover failure
[email protected]© Omnitele Ltd. 2005 18
Contents
1. Introduction to the study• Background, research problem, research methods
2. Quality of Service (QoS) in UMTS Networks
3. Measuring service quality• Defining Key Performance Indicators (KPI)
4. Case study
[email protected]© Omnitele Ltd. 2005 19
Measurement plan
Operator 1 Operator 2
Drive test: AMR speech X X
Drive test: FTP download X X
Video telephony X N/A
Streaming X X
Web page download X X
E-mail X X
Data connection: attach, PDP context activation, RTT, FTP DL & UL
X X
Tools: Nemo Outdoor, Optimi x-AppMonitor, Ethereal
[email protected]© Omnitele Ltd. 2005 20
AMR voice – Drive test statistics
Good < 21dBm
Good > -92dBm
Good > -10dB
Shoud be ~100%
Reasons for call failure:• Ec/N0 was not at adequate level• Call setup was unsuccessful (unsuccessful RACH procedure)• Look at L3 signalling
Operator 1 Operator 2Call setup time (s) 4.649 2.494Call setup success rate (%) 70 100Call completion rate (%) 100 100
Soft handovers per call 10.33 10.38Soft handover interval (s) ave. 7.901 7.953Soft handover success rate (%) 100 100
Best active Ec/N0 (dB) ave. -4.03 -3.97Best active RSCP (dBm) ave. -79.3 -66.9Tx Power (dBm) ave. -15.5 -29.5
BLER 0.254 0.221Pilot BER 1.917 2.363
[email protected]© Omnitele Ltd. 2005 21
CPICH coverage – Ec/N0
According to Ec/N0 values both operators have good coverage. Couple of RED areas, which need to be
further investigated!
Operator 1
Operator 2
If large interference areas are generated, the problem could be minimised later by adjusting the
antenna direction or height, or by down tilting the antenna or by slightly tuning the pilot power
levels.
[email protected]© Omnitele Ltd. 2005 22
Data Connection
5.227
1.716
8.715
7.188
0.264 0.234
0
1
2
3
4
5
6
7
8
9
Operator 1 Operator 2 Operator 1 Operator 2 Operator 1 Operator 2
Attach time PDP context activation time RTT
Typical RTT in UMTS network is ~200ms, which enables good
quality conversational PS services, such as
VoIP.
[email protected]© Omnitele Ltd. 2005 23
Video streaming
In mobile phone display ~60 kbps streaming bit rate produces good video quality.
[email protected]© Omnitele Ltd. 2005 24
Web browsing
Operator 1 Operator 2Service accessibility (%) 100 % 100 %Service access time (s) 0.25 0.26Web page download time (s) 22.79 16.59Service retainability (%) 100 % 100 %
22.79
16.59
0
5
10
15
20
25
Tim
e (
s)
Operator 1 Operator 2
Web page download time
0.25
0.26
0.245
0.25
0.255
0.26
Tim
e (s
)
Operator 1 Operator 2
Service access time
Sample web page 319 kB
Throughput
0
50
100
150
200
1 2 3 4 5 6 7 8 9 10 11 12
Time
Th
rou
gh
pu
t (k
bp
s)
Instantaneous
Average
[email protected]© Omnitele Ltd. 2005 25
Conclusions
• In 3G networks QoS management is required– Real-time services require QoS guarantees– Need to support different kinds of services– With QoS mechanisms operators can use their network resources more
efficiently and gain competitive advantage
• To maintain and improve the network performance and user experienced service quality constant monitoring and performance follow-up is needed
– Successful network measurements are based on correct KPI definitions– A combination of end-to-end field measurements, interface probes, network
element counter statistics and customer feedback is required
• The measurement results show that there are big differences in the performance of operators’ UMTS networks
– Currently UMTS networks are not fully optimised there is a clear need for optimisation!
– Majority of 3G measuring equipment and terminals are still quite immature
© Omnitele Ltd. 2004 26© Omnitele Ltd. 2005 26
For more information about Omnitele,please visit our web site
www.omnitele.fi
© Omnitele Ltd. 2004 27© Omnitele Ltd. 2005 27
KPI Definitions
[email protected]© Omnitele Ltd. 2005 28
AMR Speech KPIs
Parameters
Trigger points
Place a call Alerting message Speech interchangeIntentional
termination of session
Service accessibility
Service access time
Service coverage
Service retainability
T0 T1 T2 T3
Speech quality
Channel request ALERTING Start of audio stream RELEASE
Codec usage
[email protected]© Omnitele Ltd. 2005 29
Video Telephony KPIs
Parameters
Trigger points
Video callRequest
Alerting message Audio/video outputIntentional
termination of session
Service accessibility
Service access time
Service coverage
Video call setup time
Video call setup success ratio
Service retainability
Video quality
Speech quality
Audio/video synchronisation
T0 T1 T2 T3
Channel request ALERTING / Call accepted Audio/video outputstarts
Audio/video output ends
RELEASE
[email protected]© Omnitele Ltd. 2005 30
Video Streaming KPIs
Parameters
Trigger points
StreamRequest
Buffering message appears on player
Stream reproductionIntentional
termination of session
Service accessibility
Service access time
Service coverage
Streaming reproduction
start delay
Streaming reproduction start failure
Streaming reproduction cut-off ratio
Video quality
Audio quality
Audio/video synchronisation
T0 T1 T2 T3
RTSP: SETUP RTP: payload1st data packetBUFFERING
Streaming reproductionstarts – picture appearsPLAY
RTSPTEARDOWN
Video/audio stream ends
[email protected]© Omnitele Ltd. 2005 31
Web Browsing KPIs
Parameters
Trigger points
Service access Data transferIntentional
termination of session
Service accessibility
Service access time
Service coverage
Service retainability
Web page download time
T0 T1 T2 T3
1st TCP [SYN] 1st HTTP: GET HTTP: FIN/ACKReception of last data packetDisplay data
[email protected]© Omnitele Ltd. 2005 32
E-mail KPIs
Parameters
Trigger points
Service access E-mail sending E-mail download
Service accessibility
Service access time
Service coverage
Sending time Receiving time
T0 T1 T2 T4
Service retainability
1st TCP [SYN] SMTP: 250ACK (HELO)
Last data packet sendTCP [FIN/ACK]
Last data packet receivedTCP [FIN/ACK]
T3
IMAP: FETCH Body