lte kpi guide

LTE KPI GuideSlide title minimum 48 pt Slide subtitle minimum 30 pt
LTE KEY PERFORMANCE INDICATORS
Slide title minimum 32 pt (32 pt makes 2 rows Text and bullet level 1 minimum 24 pt Bullets level 2-5 minimum 20 pt !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~¡¢£¤¥¦§¨©ª«¬®¯°±²³´¶·¸¹º»¼½ÀÁÂÃÄÅÆÇÈËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿŒœŠšŸƒˆ˜–—‘’‚“”„†‡•…‰‹›⁄€™−≤≥ ΑΒΓΕΖΗΘΙΚΛΜΝΞΟΠΡΣΤΥΦΧΨαβγδεζηθικλνξορςΣΤΥΦΧΨΩ Do not add objects or text in the footer area
LTE KPI Guide | Ericsson Internal | EAB-09:023983 Uen, Rev PA9 | 2009-10-22 | Page * (50)
Revision Information
EPA Greg Harman
EPA Greg Harman
EPA Greg Harman
EPA Greg Harman
Changes following development of recommended KPI values, added CP Latency
EPA Greg Harman
LTE KPI Guide
Content
How to measure LTE KPI
Recommended LTE KPI values for acceptance
LTE KPI Guide
LTE Technology
Reduced cost per bit
Increased service provisioning – more services at lower cost with better user experience
Flexibility of use of existing and new frequency bands
Simplified architecture, Open interfaces
A main focus area of the standardization process
has been system simplification and reduced complexity
LTE KPI Guide
Flexible bandwidth
Uplink: SC-FDMA with dynamic bandwidth
Higher power efficiency, reduced interference
Downlink: Adaptive OFDMA
Multi-Antennas, both RBS and terminal
MIMO, beamforming, TX and RX diversity
Both FDD and TDD supported
LTE Radio Interface
LTE KPI Guide
Q1
Q2
Q3
Q4
Q3
Q4
Q1
Q2
Q3
Q4
Increasing system
High data rates with low latency
Directed fallback to legacy technology
Candidate
Released
Indicative
IRAT Handover (WCDMA & GSM)
LTE Radio Bearers
Evolution of Radio Access Network Capabilities
Initial LTE system will provide single and multiple RAB capabilities to user terminals
Over time, the capabilities of the system will evolve to provide
End user bit rate shaping
QoS aware scheduling with Guaranteed Bit Rate priority support
Subscriber prioritisation
VoIP bearer support (3GPP Rel 9 feature)
The initial LTE network will provide a basic high rate bit pipe
LTE KPI Guide
Content
LTE KPI Guide
LTE Requirements
11.1 Complexity requirements for overall system
E-UTRA and E-UTRAN shall satisfy the required performance. Additionally, system complexity shall be minimized in order to stabilize the system & inter-operability in earlier stage and decrease the cost of terminal & UTRAN. For these requirements, the following shall be taken into account:
a) Minimize the number of options
b) No redundant mandatory features
c) Reduce the number of necessary test cases, e.g. Reduce the number of states of protocols, minimize the number of procedures, appropriate parameter range and granularity
Basic requirement for LTE - Systemic Simplification
LTE KPI Guide
Operator Requirements
NGMN White Paper
“…. performance management in NGMN shall be done so that Key Performance Indicators (KPIs) can be agreed and:
KPIs data generated by different vendors can be analysed without additional processing
KPIs shall have same semantics and be at the same level of granularity”
3GPP TS 32.450 Key Performance Indicators (KPI) for E-UTRAN: Definitions
3GPP TS 32.451 Key Performance Indicators (KPI) for E-UTRAN: Requirements
Ericsson’s implementation of KPIs and PIs consistent with 3GPP
LTE KPI Guide
KPI for E-UTRAN
Content
LTE KPI Guide
Ericsson KPI Strategy
GSM, WCDMA, LTE.
KPI Strategy: Few KEY QoS KPI
LTE KPI Guide
KPI Fundamentals
KPIs show the RBS contribution to End-to-End performance
A black box approach to system observability
Primarily measures properties that it is worth money to the operator to improve
Measures properties that the operator can control by means of dimensioning and optimisation
The KPI framework supports operators in expansion decisions
and in RAN tuning and optimization tasks.
LTE KPI Guide
KPI level
Top-down approach for KPIs and PIs
Verification at the KPI level ensures compliance at the PI level
KPI
PI
PI
PI
PI
PI
Events
Events
Events
KPI Criteria
* System utilization is an Ericsson observability area provided to assist Operators in network capacity and configuration management
Accessibility
The ability of a service to be obtained, within specified tolerances and other given conditions, when requested by the user.
Retainability
The ability of a service, once obtained, to continue to be provided under given conditions for a requested duration.
Integrity
The degree to which a service is provided without excessive impairments, once obtained.
Mobility
System utilisation *
Hardware usage, Licenses, Traffic and Load management and Admission and congestion management
Availability
WHAT : Ericsson KPI strategy is based on having a FEW KEY Contractual QoS KPIs
Recommended KPI scope (4 KPI)
Optional KPI scope (6 KPI) – space for negotiations
WHERE: Define Measurement areas and acceptance split
Controlled Environment (Optional KPI)
Other Clusters (Recommended mostly and some Optional KPI)
HOW: Measurements methods, tools, duration and exclusions
Drive tests ; Static spots; Statistics
Define statistical confidence intervals
Understanding KPIs
Content
LTE KPI Guide
Measurement Processes
Controlled Environment Testing
Why to
Measures requiring dedicated test equipment
Cell loading
Controlled environment allows repeatable testing
Avoids external influences such as interference
Network and technology capabilities can be accurately demonstrated
Can start early in a network rollout, allowing configuration verification
Can use a trial site or lab environment depending on what is available
Measurements use commercial RBS and terminal equipment
Shielded enclosures and attenuators can be used to simulate edge of cell conditions
LTE KPI Guide
Golden Cluster Measurements
Why to
Handover performance – eg handover success rate
Allows early testing of important network features and functionality
Network and technology capabilities can be accurately demonstrated
Can start early in a network rollout, allowing configuration verification
Measurements use commercial RBS and terminal equipment
Drive tests can be conducted
Measurements can be taken in multicell and interference limited environments
LTE KPI Guide
Network Drive Test
Why to
Network performance – eg. throughput, latency
Provides an indication of network wide performance following a network build or optimisation project
The only meaningful way to measure network performance when customer numbers are low
Drive tests use commercial terminal equipment connected to data logging equipment
Test locations (static) and drive routes (mobile) should be selected to be indicative of network usage patterns (eg. 80% static and 20% mobile)
Specific areas can be targeted, eg. shopping centres and train subways
LTE KPI Guide
Statistics
Why to
Availability metrics
Indicates performance experienced by end users
Easy to implement, minimal resources required
Some metrics can only be measured through statistics
Record network statistics on O&M platform and post process if required
Uses network performance information from real users
Requires a large number of users on the network to be statistically significant – confidence intervals required for new networks
LTE KPI Guide
Progress & Cluster KPI Achieved Payments
Progress and Cluster/Network KPI Achieved Payments
Mobile System Rollout Project Model
Services View
2) Which ever comes first
3) Delayed deployed sites or expansion sites
Launch
Readiness
Post Launch Optimisation
Content
LTE KPI Guide
Recommended KPI values are not included in the current release of this document
KPI values will be included at a later date when performance metrics are available from live network deployments
Contact BNET Sales Support and the BUGS KPI team for assistance with KPI values for inclusion in contract documents
KPI Values
LTE KPI Guide
There are 4 KPI recommended for use in LTE acceptance
Session Setup Success Rate
Session Abnormal Release Rate
or
Used for stats based KPI & drive tests with long calls
DL User Throughput
UL User Throughput
1
2
3
4
1
2
3
1
2
4
3
2
Recommended KPI
Notes:
2a – Session Abnormal Release rate is ONLY to be used for drive test results using the “short call” test method
2b – Minutes per Abnormal Release is to be used for statistical counter and “long call” test
No
Recommended KPI
Notes:
2a – Session Abnormal Release rate is ONLY to be used for drive test results using the “short call” test method
2b – Minutes per Abnormal Release is to be used for statistical counter and “long call” test
No
Recommended KPI - 1
Definition :
Formula :
The Session Setup Success Rate (SSSR) is defined as the ratio of successful session setups, divided by the total number of session setup attempts.
A combined metric which includes RRC, S1 and E-RAB establishment success.
The SSSR can be measured in controlled environments, golden clusters, for general cluster acceptance and via network stats.
LTE is designed as an “always on” data technology so UEs are expected to maintain active data sessions for long periods.
SSSR is to be calculated as follows using drive test results
SSSR is to be calculated as follows using statistic counters
LTE KPI Guide
Recommended KPI – 2a
Definition :
Formula :
The Session Abnormal Release Rate (SARR) is defined as the ratio of abnormal session disconnections, divided by the total number of successful session setups.
It is the probability of successfully holding and then terminating a data session from an end user perspective.
The SARR is ONLY to be used for acceptance involving network drive tests using short data sessions.
Testing will involve drive test equipment setting up a session, conducting a short data transfer and then terminating the session.
LTE is designed as an “always on” data technology so UEs are expected to maintain active data sessions for long periods hence this metric is not reliable for statistical and long call measurements (use Minutes per Abnormal Release instead).
SARR is to be calculated as follows using drive test results
LTE KPI Guide
Recommended KPI – 2b
Definition :
Formula :
The Minutes per Abnormal Release (MPAR) is defined as the ratio of the total end user time where a data session is maintained after being successfully set up divided by the abnormal call disconnections.
It represents the probable length of time a data session call can be held by the end user before it is abnormally terminated.
The MPAR is only to be used for acceptance involving network statistical data or drive tests using long data sessions.
Drive testing will involve drive test equipment setting up a session and maintaining a continuous data transfer until the session abnormally released. Failed sessions are immediately re-established.
Network drive test acceptance is normally conducted using short calls so for drive test acceptance Session Abnormal Release Rate is a more appropriate metric.
MPAR is to be calculated as follows from statistics
MPAR is to be calculated as follows from long call drive test data
LTE KPI Guide
Recommended KPI - 3
Formula :
The DL User Throughput (DLUT) is the average user throughput for data in the Downlink on a Radio Bearer.
DLUT includes user data but excluding retransmissions, padding bits, data PDU headers and RLC control PDU's.
Radio bandwidth will impact this metric with lower throughput on smaller bandwidths
DLUT can be measure during drive tests or via network stats.
DLUT can be measured in controlled environments, golden clusters, for general cluster acceptance and via network stats.
Drive test and stats values may differ due to environmental and recording level differences
DLUT is to be calculated as follows using drive test results
DLUT is to be calculated as follows using statistic counters (recorded on PDCP level)
LTE KPI Guide
Recommended KPI - 4
Formula :
The UL User Throughput (ULUT) is the average user throughput for data in the Uplink on a Radio Bearer.
ULUT includes user data but excluding retransmissions, padding bits, data PDU headers and RLC control PDU's.
Radio bandwidth will impact this metric with lower throughput on smaller bandwidths
ULUT can be measure during drive tests or via network stats.
ULUT can be measured in controlled environments, golden clusters, for general cluster acceptance and via network stats.
Drive test and stats values may differ due to environmental and recording level differences
ULUT is to be calculated as follows using drive test results
ULUT is to be calculated as follows using statistic counters (recorded on PDCP level)
LTE KPI Guide
Time (ms)
First data is transmitted to the UE
The send buffer is again empty
The last TTI with data shall always be removed. This is because the coding can be selected based on size, not radio conditions, hence not end-user impacting.
Data_Throughput_Time
Time_Lat
Failed transmission (”Block error”)
Successful transmission, buffer empty
Σ
Optional KPI
For use in LTE Acceptance
There are 6 optional KPI for possible use in LTE acceptance
Simultaneous Connected Users per RBS
Intra LTE HO Success rate
Round Trip Time (RTT)
Control Plane Latency (Idle to Active)
Cell Availability
Optional KPI
No
Medium
6
Medium
7
Medium
8
Medium
10
Optional KPI
No
X
6
X
X
7
X
X
X
8
X
10
Optional KPI - 5
Definition :
Formula :
The Simultaneous Connected Users per RBS (SCUPR) is defined as the number of simultaneous connected user that can be supported by an RBS.
From an operator perspective, it is the number of subscribers that can be simultaneously served by a RBS.
The SCUPR is best measured in controlled environments. It can also be verified by network stats which record capacity as a % of maximum licensed level.
Operators often demand this type of testing as part of “Proof of Concept” style testing.
SCUPR can be calculated as follows from controlled environment testing
SCUPR verified using the following statistic counter
Recorded as a PDF (Probability Distribution Function)
pmLicConnectedUsersDistr
Optional KPI - 6
Definition :
Formula :
The Intra LTE HO Success rate (HOSR) is defined as the number of successful intra LTE handover attempts divided by the total number of intra LTE handover attempts.
It is the probability of successfully initiating and completing a handover to another cell in the LTE network which is centred on the same carrier frequency.
The HOSR is best measured in controlled environments. It can also be verified by network stats which record the success rate of handover attempts.
HOSR can be calculated as follows from controlled environment testing
HOSR verified using the following statistic counter
LTE KPI Guide
Optional KPI - 7
Formula :
The Round Trip Time (RTT) is defined as the time taken for an IP packet to travel from a UE through to a server located in the core network and back to the UE.
For the test to accurately reflect LTE network performance the test server should be connected to the S-GW. Servers on the internet should never be used for this test.
The RTT is measured in controlled environments or via drive tests. RTT can not be easily measured using statistical counters.
The best tool for measuring RTT is the PING program. Multiple PING test results can be averaged in order to get an overall measure of network performance.
RTT is best measured in a controlled environment or drive test by averaging PING test results
RTT can not be verified by counters
LTE KPI Guide
Optional KPI - 8
Definition :
Formula :
The Terminating Session Success Rate (Paging) (TSSR) is defined as the number of successful terminating session setup attempts divided by the total number terminating session attempts.
It is the probability of successfully initiating and completing terminating session.
The TSSR is best measured in controlled environments.
L10A will focus on LTE data terminals so terminating data sessions are not anticipated to be a common occurrence.
TSSR can be calculated as follows from controlled environment & drive testing
TSSR verified using the following statistic counter
LTE KPI Guide
Optional KPI - 9
Definition :
Formula :
The Control Plane Latency (Idle to Active) (CPL) is defined as the time taken for a UE to transition from an idle state to an active state.
It is the time to initiate a new session when the UE starts in an idle mode.
The CPL is best measured in controlled environments.
CPL can be calculated as follows from controlled environment testing
CPL can not be verified using statistical counters
LTE KPI Guide
Optional KPI - 10
Cell Availability (CA)
Formula :
The Cell Availability (CA) is defined as the percentage of time, within a given measurement period, that a cell or measurement area is available to accept traffic.
The CA is measured using statistical counters.
The CA metric can be further broken down to reflect cell outage times for planned (manual) or unplanned (auto) events.
CA verified using the following statistic counter
LTE KPI Guide
Not Recommended KPI
Not for use in LTE Acceptance
There are 2 KPI which are not recommended for use in LTE Acceptance
Definition :
How & Where to measure :
The DownLink Latency (DLLat) is defined as the time between reception of IP packet in an empty DL buffer of the RBS and the time when the RBS transmits the first block on the air interface to the UE.
The UpLink Latency (ULLat) is defined as the time between reception of IP packet in an empty UL buffer of the UE and the time when the UE transmits the first block on the air interface to the RBS.
The DLLat is measured using statistical counters.
The ULLat should never be used in LTE Acceptance as it is a metric which only measures the performance of the UE.
Round Trip Time (RTT) is a better metric for use in LTE Acceptance.
LTE KPI Guide
Confidence Intervals
Confidence intervals are linked with the number of samples
Confidence intervals are mathematical calculation and are not related to network performance!!
When setting targets always use confidence intervals
Example: Call Setup Success Rate : 98%
With 200 samples (60 sec call or 3.5 hours of Drive Test) – Interval is 98% +/-2%
With 700 samples (60 sec call or 12 hours of Drive Test) – Interval is 98%+/-1%
Result: With 200 samples, and a result of 96% - it’s in the target range, KPI is met!
Acceptance Method:
Drive test based acceptance : easier to achieve KPI level, but more effort to measure
Statistics based acceptance : harder to achieve KPI levels, but very easy to measure
LTE KPI Guide
Number of Samples
129.2
96.77%
99.23%
inputs
calculated
Sheet2
Sheet3
Slide title minimum 32 pt (32 pt makes 2 rows Text and bullet level 1 minimum 24 pt Bullets level 2-5 minimum 20 pt !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~¡¢£¤¥¦§¨©ª«¬®¯°±²³´¶·¸¹º»¼½ÀÁÂÃÄÅÆÇÈËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿŒ&o

lte kpi guide

Documents