country example of qos measurement of voice service case of morocco

Cotonou, Benin, 16-17 July 2012

Country example of QoS measurement of voice serviceCase of Morocco

Hassan TALIB,Vice-Chair ITU-T SG 12,

[email protected] // [email protected]

ITU Workshop on “Workshop on Practical measurement of QoS/QoE

Parameters for Regulatory Compliance”

(Cotonou, Benin, 16-17 July 2012)

mailto:[email protected]



Presentation Scope

The QoS for Morocco telecommunications A protocol for voice QoS measurements:

Choice of a subjective method,List and definitions of indicators,Measurements Methodology:

Definition of voice QoS measurement,Sample and Statistic considerations,

Organization and Operation,Process of Outcomes.

Recommendations to regulators/Op.


Every operators are responsible for QoS

The QoS for Morocco Telecommunications

VSATOperators

VSATOperators

GMPCS Operators

GMPCS Operators

Radio Trunk Operators

Radio Trunk Operators ISPISP Fixed and mobile

Operators Fixed and mobile

Operators Fixed Operator with

narrow mobility Fixed Operator with

narrow mobility

Fixed and mobile: oligopoly

SPACECOM

GULFSAT

CIMECOM

SPACECOM

GULFSAT

CIMECOM

MAROC TELECOM

MEDITELECOM

MAROC TELECOM

MEDITELECOM EUROPEAN DATACOM

GLOBALSTAR

ORBCOMM

SOREMAR

THURAYA

EUROPEAN DATACOM

GLOBALSTAR

ORBCOMM

SOREMAR

THURAYA

INQUAM MORATELINQUAM MORATEL

DuopolyOligopolyoligopoly Open

Competition

MAROC TELECOM

MAROC CONNECT(WANA)

MEDITELECOM

ISPs

MAROC TELECOM

MAROC CONNECT(WANA)

MEDITELECOM

ISPs

MAROC CONNECT(WANA)

MAROC CONNECT(WANA)


The QoS for telecommunications in Morocco

The ANRT regularly

monitors the QoS

A national regulatory Framework:-Allocations -QoS Obligations for operators

International regulatory framework (standards) :-IUT-T : series E, G, P, Y, QoS handbook, ...-Regional : ETSI (series EG), IEEE,…

A benchmark of best practices and technological monitoring

Use feedback :-Complaints,-Press,-End Users Associations,-…

Analysis of data received from the operators in terms of performances, KPI and QoE measurements

Ground Measurements (campaigns) by

ANRT


Development of voice mobile telephony service in Morocco

The QoS for telecommunications in Morocco


Presentation Scope







The quality of service sets a premium on models for reference (IUT-T standards : E, I, G, P, Y… ou ETSI : ETSI TS 102 250-x), on the levels of performance for service provided and the measurement methods. The models and methods are supposed to be coherent with the users expectations.

For voice service, QoS requirements could be related to many aspects sumed up in its availability, accessibility and continuity and reliability.

Protocol for voice QoS measurements

Choice of a subjective method


Protocol for voice QoS measurements Choice of a subjective method

Operator’s vision Regulator’s Vision

Motivation Economic

Competitive

Rules and Regulations

Center of interest (users)

Users loyalty (churn) and attraction of newcomers

(Concept SLA)Protection for users

Measurement Local (various components of the network)

KPI

External (end to end: service as daily

understood by users)

Measurement Tools

- Local machines meters in the network (objectives methods)

- DTS

(objectives methods)

- Measures and investigations by subjective methods

- DTS

(Subjective Methods)Necessity to harmonize the definitions of indicators between operators and the regulator.


Presentation Scope







Protocol for Voice QoS measurements List and definitions of indicators

Rate Indicator Definition

TR Rate of successful communications and maintained during 2 minutes

A communication is considered as successful if the call launched succeeds at the first attempt and if the communication is maintained 2 minutes without being cut off. The

rate (TR) is related to the number of successful communications and the total number of call attempts made.

TR

Perfect Rate of successful communications and maintained during 2 minutes and quality of perfect audio

A communication is considered as successful and the perfect quality if it is succeeded at the first indicator and the audio quality considered by both parties are

perfect (compared to the quality of direct communication).

Acceptable

Rate of successful communications and maintained during 2 minutes and quality of acceptable audio

A communication is considered as successful and the acceptable quality if it is succeeded at the first indicator and if audio quality considered by both parties is

slightly disrupted without interferring the conversation.

Poor Rate of successful communications and maintained during 2 minutes and quality of poor audio

A communication is considered as successful and poor quality if it is succeeded at the first indicator and if the audio quality considered by both parties is frequently

disrupted in the receiving channel by numerous hit on the line, but it is possible again to hear one another.

Bad Rate of successful communications and maintained during 2 minutes and quality of bad audio

A communication is considered as successful and the quality degenerated if it is successful at the first indicator and if it is very difficult to hear one another, the

conversation is impossible.

RC Rate of cut off communications

A communication is considered as cut off if at the first attempt, it is established and maintained more than 5 seconds, but cut off before 2 mins. The rate (TC) is the ratio

between the number of cut off communications and the total number of call attempts made.

RF Rate of failed communications

A communication is considered as failed if at the first attempt, it is not established or has not been maintained, more than 5 seconds. The Rate (TE) is the ratio between

the number of failed communications and the total of call attempts made.

TR + TE + TC = 100% and TNR = TE + TC


Protocol for voice QoS measurements List and definitions of indicators

Evaluation subjective de la qualité auditive : définitions des défauts de communication vocale

Default Definition

Low Voice Level received more and less very low than the normal level

Echo Return of its transmission distinctive from when it’s simultaneous

Metallic Noise Chopping Noises to be compared to metallic one

Frying noise Interferences

Micro-defaults Small cut-offs of sound giving an impression of chopping

Silence Blank enough long

Distorsion Degradation of voice distant without other noise


Presentation Scope







Protocol for voice QoS measurements Measurements Methodology

A QoS measurement shall necessarily be done within the coverage area by all concerned mobile networks. The QoS measurement is different from the coverage measurement.

A voice QoS measurement consists in trying to establish a communication from the mobile investigator to the fixed investigator to verify the reception of the call (no failure) and the continuation of the communication for two minutes (no cut off) and to assess the audio quality of the established communication.

Each network is tested by a “couple” of investigators, on mobile and the other fixed.



A communication is considered cut off if, at the first attempt:

it is established and maintained more than 5 seconds, but cut off before 2 minutes.

A communication is considered failed if:

at the first attempt (maximum delay time is about 15 seconds), it is not established, or has not been maintained more than 5 seconds.



Precautions to be taken:Outdoor : avoid measuring under trees.

Indoor : very inside buildings and till 8th floor.

Incar : use terminals without additional antenna and drive at a speed less than 80km/h.

Force terminals on 2G in case of use of bi-mode posts.

Every points for measurements shall be covered.



Sample and statistic considerations:

Average duration of a measurement: 7 mn,Average duration for measurements takes place everyday: 8:00 to 10:00 (Example : 09:00-13:00 and 14:00 to 20:00),Everyday (per team) : 85 measurements x Number of operators; For 3 operators : 255 measurements per day,



Sample and statistic considerations:

Breakdown Element: Per operator, Per site (cities, motorways, national highways, railways), per configuration (Indoor, Outdoor, Incar), per traffic destination (OnNet and OffNet).Important numbers to provide a significant sample (statistic error about 2%) : elementary values after any breakdown shall be higher than 33.


Presentation Scope








Organization and Operation:

Decider to achieve based on own resources or controlled subcontracting,Select effectively investigators (mobiles and Fixed) and conduct a targeted training,Consult with operators on the methodology but not the sample and the planning of each campaign,


Protocol for voice QoS measurements

Measurements Methodology

Organization and operation:

Develop a guide for the investigator with some templates to be used by the investigator: Ref., delay, GPS Information, recipient, outcomes,…

Use the principle for codificationprincipe de codification : opérateur, site, configuration, trafic,…



Organization and operation:

Guarantee any conditions so that each measurement is exactly conducted in the same conditions for every operators: delay, space, method,…

Proceed to a number of important blank tests calibrate any test system.


Presentation Scope








Outcomes Processing: (Scheduled to be Delivered)

Trend in indicator “positive” : successful rate.Trend in indicator “negative” : unsuccessful rate = TE+TC.Reports:

Global Report: reference,Specific Report to each operator (request for

actions plan for corrections),Report for publication.


Presentation Scope







Conclusions and Recommendations

Recommendation n°1Permanent Consultation with the upstream operators upon the methodology: financing of campaigns. Adoption of an agreed protocol of references for measurements.

Recommendation n°2Positive use of outcomes by the operators (consider sanctions as last resort).


Conclusions and Recommendations

Recommendation n°3Publication of measurements outcomes,Proposal of a scenario :

1st voice campaign on based upon a large sample (thirty cities, highways, national motorways and railways).1st quarter of the year

2nd similar campaign practically based on the first upon the same sample.4th quarter of the year(Publication of results)

Communication of operators’ results

Delay of 5 to 6 months for correction of discrepancies

identified by the ANRT


Conclusions and Recommendations Recommendation n°3 (bis)

Publication of outcome measurements

Taux de réussite global (On-Net et Off-Net) par site de mesures et par opérateur

Site for measurement

sTMR Operator 1 Operator 2 Operator 3)

All cities 95,32% = = =

Cities 96,78% = = =

Highways 95,68% = - - +

National Motorways 96,03%

+ = =

Railways 87,00%= = =

TMR

- - - - - - = = + + + + + +

TMR×0,95 TMR×0,98 TMR×0,99 TMR×1,01 TMR×1,02 TMR×1,05

The TMR is equal to the successfuk average rates (TR) recorded by each of the three operators:

TMR


Thanks for your attentionQuestions/Answers

[email protected] // [email protected]



country example of qos measurement of voice service case of morocco

Documents

voice qos measurements

voice qos measurements

moroccothe qos

qos handbook

qosthe qos

qos requirements

presentation scopethe

voice service