analysis of service quality in 3g mobile networks

[email protected]© Omnitele Ltd. 2005 1

Analysis of Service Quality in 3G Mobile Networks

Heidi Lagerström

Supervisor: Professor Heikki Hämmäinen

Instructors: M.Sc. Sami Vesala & M.Sc. Katja Koivu


Contents

1. Introduction to the study• Background, research problem, research methods

2. Quality of Service (QoS) in UMTS Networks

3. Measuring service quality• Defining Key Performance Indicators (KPI)

4. Case study


Background

• UMTS introduces new real time services to mobile networks, such as video telephony.

• These real time services require QoS guarantees to function properly.

• For operators to maintain satisfactory service quality constant network monitoring is needed.

• Network measurements are based on correctly defined KPIs for each service.

Operators’ possibilities to utilise QoS in practice have not been widely researched.

Key Performance Indicators have not been defined for the new services from the end-user perspective.


Research problem

Objectives:

1. What are the KPIs that measure service quality, from end user perspective, in 3G networks for the key services (AMR voice, video telephony, video streaming, web browsing and e-mail)?

2. What are the QoS mechanisms in Release 99 and how can they be used to improve service quality?

How should service quality be measured in 3G networks and how the QoS mechanisms can be used to affect the

service quality perceived by subscribers.


Research methods

• Literature study– 3GPP, ETSI, ITU specifications– Several books and publications

• Interviews– Network equipment vendors: Ericsson, Nokia– Operators: Elisa– Several other radio network experts

• Case study– Field measurements for two

operators in live networks


Contents




4. Case study


Why do we need QoS?

• UMTS networks support services with very different performance requirements

– Real-time services require performance guarantees

– Customer acceptance closely tied to service quality

• Optimal usage of network resources

– Radio resources scarce

– Cost-effectiveness

– Return of investment

• Service and user differentiation

– Meet different needs of customers (e.g. business vs. consumer)

– Support different services (real-time vs. best effort)

• Competitive advantage!

Delay Jitter Loss

Video call High High High Med

Streaming High Med Med Med

Web browsing Med Med Low High

E-mail Low Low Low High

Application BandwidthSensitivity

Performance Requirements


QoS Traffic Classes

Traffic class Characteristics Example application

Conversational Preserve time relation between information entities of the stream. Conversational pattern (stringent and low delay)

Speech

Video calls

Streaming Preserve time relation between information entities of the stream.

Real-time streaming video

Interactive Request-response pattern. Preserve payload content.

Web browsing

Background Destination is not expecting the data within a certain time. Preserve payload content.

E-mail

File downloading

Demanding• Delay• Jitter

Demanding• Bit rate• Jitter

Tolerant• Delay and bit rate

can vary• Integrity

Easiest• Delay and bit rate

can vary• Integrity


QoS Profile Attributes

R99 QoS attribute Example valueResidual BER 10 –5

SDU error ratio 10 –4

Delivery of erraneous SDUs No

Maximum SDU size (octets) 1500

Delivery order No

Transfer delay 100 ms (conversational)

280 ms (streaming)

ARP 1, 2 or 3

Traffic Class Conversational, streaming, interactive, background

THP 1, 2 or 3 (same as ARP)

Maximum allowed bit rate e.g. 64, 128 or 384 kbps

Maximum guaranteed bit rate e.g. 64, 128 or 384 kbps

Depends on operator’s

QoS strategy

Depends on the QoS strategy

and UE/RNC capabilities


QoS Differentiation

Conversational RAB

Streaming RAB

Interactive RAB, THP/ARP = 1


Background RAB

MMS

Web browsing

Push-to-talk

Streaming

Videotelephony

• Each service gets the treatment it requires according to the QoS profile• Network resources are shared according to the service needs• Network resources can be used more efficiently


QoS Mechanisms

External IP (Internet)

External IP (Internet)

UE Node B

TE

Firewall

RNC 3G-SGSN 3G-GGSNUTRAN

Iu Gn Gn

IP

PS Domain

Inter-PLMN Backbone NT

Inter-PLMN Backbone NT

DiffServ on transport level IP(ATM QoS for CS)

Diffserv on transport level IP

Diffserv in Gi

Different channel

types

ATM QoS

• Different QoS techniques in different parts of the network• Appropriate QoS must be provided in every network so that the user can experience good service quality

PDP context with the requested QoS capabilities

RRM


Operators’ QoS Strategy

Conversational RAB

Streaming RAB



Background RAB

RNC3G

GGSN3G

SGSN

Applicationserver

Node B

HLR

User profiles are stored in HLR.Each user can have several user

profiles, which correspond to different services and are mapped

to different bearers according to the operator’s strategy.

• Operators can practise user differentiation by giving each user set of QoS profiles, which he/she is entitled to use• Operators can practise service differentiation by mapping each service to the bearer that meets its requirements

• Meet the needs of different customers• Offer each service the quality it requires• Optimise network resource usage


Contents




4. Case study


Measuring network performance

RNC3G

GGSN3G

SGSN

Applicationserver

Node B

UTRAN Core nw External nw

E2E service quality,

QoE

Performance statistics from

application serverNetwork statistics from different counters and interfaces

Customerfeedback

Network Performance Monitoring Optimisation


Defining the right KPIs

• Different services have different quality requirements– KPIs must be defined separately for each of the key services

• KPI categories– Service Accessibility– Service Integrity– Service Retainability

• With inadequate performance indicators and monitoring– Hidden problems in network performance and user perceived

quality of service– Poorly defined indicators may show better quality than in the reality

• Incorrect formulas and counters• Unreasonable measurement periods

(too much averaging etc.)


Example: Voice Services - CS

Customer demand

Indicator Measure

Service accessability

Availability & Coverage

Call setup success rate

Call setup delay

Ec/No, RSCP

Admission control

RAB assignment

Service integrity Voice quality Noisy frames (FER), MOS

Service retainability

Call drop rate Handover failure

No coverage

Interference


Example: Data Services - PS

Customer demand Indicators Measures

Service accessability

Availability & Coverage

Access success rate

Service access delay

Ec/No, RSCP

Admission control

Attach, PDP context activation, IP service setup

Service integrity Video quality

Audio quality

Web page download time

E-mail sending time, etc.

BLER, FER, throughput, delay, jitter


Dropped data connection

Connection timeouts

Dropped PDP context/attach

No coverage etc.

Handover failure


Contents




4. Case study


Measurement plan

Operator 1 Operator 2

Drive test: AMR speech X X

Drive test: FTP download X X

Video telephony X N/A

Streaming X X

Web page download X X

E-mail X X

Data connection: attach, PDP context activation, RTT, FTP DL & UL

X X

Tools: Nemo Outdoor, Optimi x-AppMonitor, Ethereal


AMR voice – Drive test statistics

Good < 21dBm

Good > -92dBm

Good > -10dB

Shoud be ~100%

Reasons for call failure:• Ec/N0 was not at adequate level• Call setup was unsuccessful (unsuccessful RACH procedure)• Look at L3 signalling

Operator 1 Operator 2Call setup time (s) 4.649 2.494Call setup success rate (%) 70 100Call completion rate (%) 100 100

Soft handovers per call 10.33 10.38Soft handover interval (s) ave. 7.901 7.953Soft handover success rate (%) 100 100

Best active Ec/N0 (dB) ave. -4.03 -3.97Best active RSCP (dBm) ave. -79.3 -66.9Tx Power (dBm) ave. -15.5 -29.5

BLER 0.254 0.221Pilot BER 1.917 2.363


CPICH coverage – Ec/N0

According to Ec/N0 values both operators have good coverage. Couple of RED areas, which need to be

further investigated!

Operator 1

Operator 2

If large interference areas are generated, the problem could be minimised later by adjusting the

antenna direction or height, or by down tilting the antenna or by slightly tuning the pilot power

levels.


Data Connection

5.227

1.716

8.715

7.188

0.264 0.234

0

1

2

3

4

5

6

7

8

9

Operator 1 Operator 2 Operator 1 Operator 2 Operator 1 Operator 2

Attach time PDP context activation time RTT

Typical RTT in UMTS network is ~200ms, which enables good

quality conversational PS services, such as

VoIP.


Video streaming

In mobile phone display ~60 kbps streaming bit rate produces good video quality.


Web browsing

Operator 1 Operator 2Service accessibility (%) 100 % 100 %Service access time (s) 0.25 0.26Web page download time (s) 22.79 16.59Service retainability (%) 100 % 100 %

22.79

16.59

0

5

10

15

20

25

Tim

e (

s)



0.25

0.26

0.245

0.25

0.255

0.26

Tim

e (s

)


Service access time

Sample web page 319 kB

Throughput

0

50

100

150

200

1 2 3 4 5 6 7 8 9 10 11 12

Time

Th

rou

gh

pu

t (k

bp

s)

Instantaneous

Average


Conclusions

• In 3G networks QoS management is required– Real-time services require QoS guarantees– Need to support different kinds of services– With QoS mechanisms operators can use their network resources more

efficiently and gain competitive advantage

• To maintain and improve the network performance and user experienced service quality constant monitoring and performance follow-up is needed

– Successful network measurements are based on correct KPI definitions– A combination of end-to-end field measurements, interface probes, network

element counter statistics and customer feedback is required

• The measurement results show that there are big differences in the performance of operators’ UMTS networks

– Currently UMTS networks are not fully optimised there is a clear need for optimisation!

– Majority of 3G measuring equipment and terminals are still quite immature

© Omnitele Ltd. 2004 26© Omnitele Ltd. 2005 26

For more information about Omnitele,please visit our web site

www.omnitele.fi


AMR Speech KPIs

Parameters

Trigger points

Place a call Alerting message Speech interchangeIntentional

termination of session

Service accessibility

Service access time

Service coverage


T0 T1 T2 T3

Speech quality

Channel request ALERTING Start of audio stream RELEASE

Codec usage


Video Telephony KPIs

Parameters

Trigger points

Video callRequest

Alerting message Audio/video outputIntentional



Service access time

Service coverage

Video call setup time

Video call setup success ratio


Video quality

Speech quality

Audio/video synchronisation

T0 T1 T2 T3

Channel request ALERTING / Call accepted Audio/video outputstarts

Audio/video output ends

RELEASE


Video Streaming KPIs

Parameters

Trigger points

StreamRequest

Buffering message appears on player

Stream reproductionIntentional



Service access time

Service coverage

Streaming reproduction

start delay

Streaming reproduction start failure

Streaming reproduction cut-off ratio

Video quality

Audio quality

Audio/video synchronisation

T0 T1 T2 T3

RTSP: SETUP RTP: payload1st data packetBUFFERING

Streaming reproductionstarts – picture appearsPLAY

RTSPTEARDOWN

Video/audio stream ends


Web Browsing KPIs

Parameters

Trigger points

Service access Data transferIntentional



Service access time

Service coverage



T0 T1 T2 T3

1st TCP [SYN] 1st HTTP: GET HTTP: FIN/ACKReception of last data packetDisplay data


E-mail KPIs

Parameters

Trigger points

Service access E-mail sending E-mail download


Service access time

Service coverage

Sending time Receiving time

T0 T1 T2 T4


1st TCP [SYN] SMTP: 250ACK (HELO)

Last data packet sendTCP [FIN/ACK]

Last data packet receivedTCP [FIN/ACK]

T3

IMAP: FETCH Body

analysis of service quality in 3g mobile networks

Documents