Teemu Rovanperä

RADIO TIMING MEASUREMENT AUTOMATION

RADIO TIMING MEASUREMENT AUTOMATION

Teemu Rovanperä Bachelor’s Thesis Fall 2021 Information Technology Oulu University of Applied Sciences

3

ABSTRACT

Oulu University of Applied Sciences Degree Programme in information Technology, Option of Software Development

Author: Teemu Rovanperä

Title of thesis: Timing Measurement Automation

Supervisor: Kari Jyrkkä

Term and year when the thesis was submitted: November, 2021

Number of pages: 29

The aim of this thesis work was to automate the Common Public Radio Interface

timing measurements for radio products, reduce the need to do this type of testing

manually and to remove a human error from the equation. Previously the testing

had been done manually with the help of a Python-based tool that takes the

measurements and provides the results. The Radio unit and most of the

measurement device configurations had to be done manually.

The automation was to be done with Robot Framework, using already existing

libraries, and creating new ones if needed. The tool to be automated would also

need some modifications to be robot compatible. The aim was to create a fully

automated testing process for the tester but to also add this automation to be a

part of the current CI framework.

As a result, the testing process was fully automated utilizing Robot Framework.

Adding more radio units to the test environment and further development of the

automation continues after the thesis.

Keywords: 5G, Robot Framework, Python, Automation

4

CONTENTS

ABSTRACT 3

CONTENTS 4

VOCABULARY 5

1 INTRODUCTION 6

2 COMMON PUBLIC RADIO INTERFACE 7

2.1 Common Public Radio Interface 8

2.2 Enhanced Common Public Radio Interface 8

2.3 Timing 9

3 CURRENT STATE AND FUTURE OF THE ECPRI TIMING MEASUREMENT

10

3.1 Current State of The Timing Measurement 10

3.2 Test Setup 12

3.3 Requirements for The Automation 14

3.4 How 14

4 ROBOTIZATION 16

4.1 Making Current Code Robot Compatible 16

4.2 Creating a Robot Keyword File 18

4.3 Packaging 19

5 AUTOMATION & ROBOT STRUCTURE 20

5.1 Jenkins 20

5.2 Jenkins Automation Structure 20

5.2.1 Trigger 21

5.2.2 Test Suite Pipeline 22

5.2.3 Testing Pipeline 24

5.3 Robot 26

6 MODULE TESTING 28

7 CONCLUSION 29

REFERENCES 30

5

VOCABULARY

4G .................................................................................................. 4th Generation

5G .................................................................................................. 5th Generation

BB ....................................................................................................... Base Band

BBU .............................................................................................. Base Band Unit

BW ....................................................................................................... Bandwidth

CC ....................................................................................... Carrier Configuration

CPRI ................................................................... Common Public Radio Interface

DL ........................................................................................................... Downlink

eCPRI ................................................ Enhanced Common Public Radio Interface

NR ....................................................................................................... New Radio

RE ............................................................................................. Radio Equipment

REC .............................................................................. Radio Equipment Control

RFSW ........................................................................................... Radio Software

RRU ....................................................................................... Remote Radio Unit

RU ........................................................................................................ Radio Unit

SSU ...................................................................................... Signal switching unit

UL ............................................................................................................... Uplink

6

1 INTRODUCTION

Test automation is the practice of running automatically tests that would be

performed otherwise manually. Software testing is an essential part of software

development, but when done manually, software testing can be a slow process

and there is always a possibility of human error. The major aim of automating

software testing is to reduce the time spent on testing to free resources from the

basic testing tasks.

Continuous Integration is a practice in software development where the members

of the team continuously merge commits to a shared trunk. Each of those

commits are verified by an automated qualitative testing to find faults in the

software as quickly as possible [1].

Robot Framework is a generic cross-platform open-source automation framework

developed originally by Nokia Networks. Being open-source, it has a growing

ecosystem of tools and libraries. Robot Framework’s code is intended to be

readable even to the untrained eye so someone with no programming expertise

can use it effectively. Robot Framework’s language is tabular which makes it easy

to read. Tasks can be created with simple keywords and phrases. Keywords and

libraries can be built-in or external. Also, custom keywords and libraries created

with Python or Java can be used. Robot Framework is a widely used tool for test

automation and test-driven development for various applications. It is also used

Robotic Process Automation, a technology that is used to create, deploy, and

manage software robot emulating human interactions with digital systems [2].

In this thesis the main objective was to create a fully automated testing process

for radio unit air interface timing utilizing Robot Framework to make the

automated process compatible with Nokia’s current continuous integration

framework.

7

2 COMMON PUBLIC RADIO INTERFACE

CPRI’s (Common Public Radio Interface) aim was to remove copper and coaxial

cables from between the RRU (Remote Radio Unit) and the BBU (Base Band

Unit) and replace them with an optical fiber. The RRU is located on the top of the

tower and the BBU is usually located at the base of the tower. The RRU receives

analog signal from mobile device and converts it to a digital form according to the

CPRI specification and sends it along the fiber to the BBU, vice versa to the other

data direction. (Figure 1.)

FIGURE 1. Radio base station

8

2.1 Common Public Radio Interface

The CPRI specification is defined in cooperation with such vendors as Nokia,

Ericsson, Huawei, and NEC [3]. The CPRI interface defines the key internal

interface specification between the RE (Radio Equipment Control) and the RE

(Radio Equipment) of radio base stations [4]. In other words, CPRI is the

fronthaul data transfer protocol that is used between RE and REC [5]. (Figure 2.)

FIGURE 2. CPRI

2.2 Enhanced Common Public Radio Interface

With the introduction of 5G and ever-growing data demands, a new fronthaul

protocol was needed to replace CPRI. To meet that demand eCPRI (Enhanced

Common Public Radio Interface) was introduced [6]. It provides a flexible packet

based fronthaul network. eCPRI can monitor, analyze, manage, and optimize the

data flow between the eRE and eREC. It provides an increased efficiency,

capability, and lower latencies in the network. eCPRI reduces the needed

bandwidth up to ten times when compared to CPRI [7]. The bandwidth is scalable

according to traffic to ensure a reliable and sustainable operation.

9

2.3 Timing

Like In all communication technology, 4G (4th Generation) and 5G (5th

Generation), there are certain specifications that manufactures need to meet for

their product to work correctly. In eCPRI the network elements require accurate

timing and synchronization to ensure that the data going over these devices is

placed on the RF air interface at an exact time. If the timing is not correct, the

throughput at the user end cannot be maintained when switching between

different radio units.

This thesis concentrates on DL (downlink) and UL (uplink) air interface timing

measurements. The air interface is the frequency portion between the base

station and the mobile device. The DL direction measurement measures when

the radio starts sending the data over the air interface relative to the RU’s slave

clock. The purpose of the measurement is to verify that the downlink data is sent

at the correct time. The uplink direction the measurement measures if the radio

has the correct timing for the incoming data over the air interface relative to the

RU’s slave clock. The purpose of the measurement is to verify that the RU

processes the UL data at the correct time.

10

3 CURRENT STATE AND FUTURE OF THE ECPRI TIMING

MEASUREMENT

3.1 Current State of The Timing Measurement

Currently, to perform eCPRI air interface timing measurements and setting up the

RU and the BBU emulator is done manually. The measurements are also

performed manually with the aid of the software. The software automates

measurement device configuration, takes the measurements, and calculates the

results taking every variable into account. Measurements in the DL direction are

in this scope fully automatic, requiring no user input during the measurement

process. However, during the measurements in the UL direction, the software

requires the user to take a capture of the UL data and place it into a specific folder

so that it can complete the measurement. When changing from DL to UL or vice

versa, or to a different antenna, the user is required to change antenna routing

from the SSU (Signal Switching Unit). In the figure below (Figure 3.), manual

steps are marked with an asterisk. In the automation these manual steps will be

automatized.

11

FIGURE 3. Current state diagram

12

3.2 Test Setup

The testing setup consist of a RU, a BB emulator, an SSU, an Oscilloscope, a

Signal generator, an RF circulator, and a test PC (Figure 4.). The PC is used to

control the devices in the test environment over Ethernet and also to run the

automation codes. RU and BB emulator are connected with an optical fiber.

RU’s every antenna is connected to the SSU with RF cables. With the SSU the

antenna to be measured can be routed to the wanted output without touching

the RF cables. The signal generator is connected to the SSU’s output 1 through

an RF circulator with RF cables. Using a circulator prevents the power from

RU’s antenna going to the signal generator. The oscilloscope is connected to

the SSU’s output 1 through the RF circulator and directly to the output 2. When

measuring DL timing, the output 2 is used and when measuring UL timing, the

output 1 is used. The Basic idea of the measurement in both data directions is

to measure at what time the data arrives at the oscilloscope in reference to the

time it should have arrived.

13

FIGURE 4. Test setup

14

3.3 Requirements for The Automation

Requirements for the 1st stage of automation are to use already existing Robot

Framework libraries to configure the RU and BBU emulator and, also to control

the SSU. The program used to perform the measurements needs to be modified

to be able to run it with Robot Framework. Keywords need to be created. They

will be used to run the program. The new Robot Framework compatible version

of the measurement software needs to be packaged as a Python module for easy

installation and useability. Jenkins must be used to trigger and run the

automation. Some sort of unit testing or at least testing the package installation

needs to be implemented and run when there is a commit or merge request in

Git.

3.4 How

The automation to be created, aims to create a fully automated testing process

requiring no user input to start the measurement, or to require any actions from

the user during the measurements. Automating the RU and BBU emulator

configuration and controlling the SSU would be done using existing automation

libraries. The measurement software needs to be fully automatic, needing no user

input during the measurements, i.e., taking the data capture of the UL data with

the BBU emulator needs to be automated. The software also needs modifications

to be able to run it with Robot Framework. Custom keywords need to be created.

They will be used to execute the program using Robot Framework.

15

At first, the test cases are simple, and they will be manually defined in a robot file.

In the future the test cases will be defined in a json file and the robot suite will

consist of loops that are utilized based on the JSON file provided. For example,

when there are multiple test cases to run for multi carrier configuration, i.e.,

different frequencies. It does not make sense to manually create a robot suite for

every possible test variation. It is more feasible to create one test suite that uses

a JSON file where the test cases and frequencies are defined. And based on it,

the test suite goes through all the frequencies in a loop and runs the test cases

for all the frequencies in the JSON file.

The automation will be run in Jenkins, so the trigger and the test case running

jobs will be created using pipelines. Module testing will also be done in Jenkins

pipeline job which will be triggered from GitLab using a webhook.

16

4 ROBOTIZATION

Robot Framework uses a keyword based tabular syntax where the test cases

are defined with keywords. Robot Framework has a lot of keyword libraries but

to utilize it for this specific purpose, custom keywords need to be created to run

custom Python programs. With Robot Framework, Python scripts should not be

executed using the script’s main function. If a script is run using main, it needs

to be modified to be run as a function or as a class method. Keywords needs to

be created to execute those functions or methods. The custom keyword file

needs to be created where those keywords are defined.

4.1 Making Current Code Robot Compatible

Since Robot Framework is a keyword driven framework, some changes were

needed to make the program run with Robot Framework.

The current program consists of multiple Python classes and “main.py”. The

timing measurement program was run using the “main.py” with the environment

and measurement settings and test cases needed to be defined in various

configuration files. The first step was to study the program and find out what

needed to be done to be able to run the program using Robot framework.

A new Python file and class were needed. They would be used to run the program

through methods instead of the previous “main.py”. The contents of the previous

main.py were modified to contain a class and only a method of that class. (Figure

5.) The program and all the test cases are run through a single method in the

class. That method was modified to accept the test case name as an argument

instead of the previous way of reading it from a file. This change was made so

that a keyword could be created for every test case.

17

FIGURE 5. Code Changes

18

4.2 Creating a Robot Keyword File

A robot keyword file was created to run the program with Robot framework. The

keywords were setup to create the Python class and run the test running method

with the wanted test case name as an argument and to return the result of the

measurement to a variable. (Figure 6.)

FIGURE 6. Keyword file

When forming the keyword file, the Python file must be imported as a Library in

the setting section. When creating the keywords, the keywords need to be named

according to their functions. For example, in the above picture (Picture 6.) the

keyword is formed by naming the keyword first, “Test dl air frame radio pps”.

Below the keyword name, the Python class and the method used are defined,

“Timing.Run”.

19

After the defining of the Python class and method, the test case name is written,

“dl_air_frame_radio_pps” to be supplied as a string argument running the “Run”

method. The method returns the measured delay value to the variable named

“delay_value”. Then the keyword is made to return the value to the robot suite

with the line “[return] ${delay_value}”.

4.3 Packaging

The new Robot Framework compatible version of the timing program was

packaged as a Python module which could be installed from a local copy with

Python’s PIP installation tool. When installed as a Python module, the keyword

file can be imported to the robot suite as a resource just like Python modules

are imported to Python scripts with the import command. This also ensures that

the program would always be installed correctly with all the Python libraries it

needs to work properly.

20

5 AUTOMATION & ROBOT STRUCTURE

5.1 Jenkins

Jenkins is an open-source automation server. It is generally used to automate

testing, building, and implementing CI (Continuous Integration). In this thesis it is

used to chain the automation components together to create an automation

framework [8]. Scripted pipeline jobs were chosen to run the robot scripts in

Jenkins. Scripted pipelines are written to a Jenkinsfile with the Groovy

programming language. The scripted pipeline contains the instructions for the job,

for example, running Python or robot scripts and other various tasks that are done

with those jobs.

5.2 Jenkins Automation Structure

In the automation framework created, Jenkins contains the trigger, test suite,

RFSW (Radio Software) and testing pipeline jobs. Jenkins takes care of triggering

the testing, managing the automation structure, and collecting artifacts from the

jobs. (Figure 7.). In an example case, the trigger is set to trigger the automation

for every hour. When the trigger is run, first it checks if the testing setup is free.

After that, the trigger checks the latest RFSW version available. If the setup is

free, the test suite is started with the latest RFSW version. The test suite launches

the RFSW update job and after the update has been done, the testing is run.

Lastly it collects results from the test pipeline’s run within the test suite. The test

suite’s main purpose is to manage and link together the jobs run from within it to

form an automation framework.

21

FIGURE 7. Automation structure

5.2.1 Trigger

Separate trigger pipelines were created to start the test suite pipelines since there

would be test suite pipelines for every radio unit tested. With the trigger, triggering

the testing can be set to specific times or time intervals. In this case the trigger

was to be set to run the automation during nighttime and weekends, when the

test environment was not in use.

Every time the trigger is run, it checks the latest software build available and if

the test environment is available. When the test environment is available, the

trigger starts the test suite and supplies it with information about the latest

software build available and what test cases to run.

22

5.2.2 Test Suite Pipeline

With the test suite pipeline, multiple different testing pipelines can be run with

different test cases without making the testing pipeline or the robot script itself too

long or complex. (Figure 7.) Test suite also keeps track of the latest RFSW build

that has passed the timing CI. The main reason for keeping track of the latest

working RFSW build is to quickly find the first RFSW build that has caused an

issue with the RU’s timing. The test suite runs a limited number of testing

pipelines and collects the results from the pipelines. Depending on the outcomes

of the pipelines, if all the tests are passed, it sets the tested RFSW as the latest

RFSW that has passed the timing testing, a.k.a. the “Latest passable”. If one or

more timing test cases fail in the test suite, the “Latest passable” is not updated.

The “Latest passable” means that the RFSW has passed the testing for a specific

configuration that is chosen for the CI. More testing would be needed to validate

that RFSW works with all the different configurations that the RU it is capable of.

Outcomes of the tests are also collected and set in the test suites job description.

Also, a hyperlink to the executed testing pipeline is added. (Figure 8.)

FIGURE 8. Test suite run with one 1xNR100 test case with passed results.

23

The trigger can be used to select what testing pipeline or pipelines are going to

be run with the test suite pipeline (Figure 9.). Also, the trigger can be used to

configure the test suite pipelines by passing down parameters from the trigger to

the test suite and from the test suite to the testing pipeline. For example, the

trigger can tell the test suite to test different test cases depending on the time of

day. For example, more extensive testing can be run during the night on the

RFSW build that the CI has marked as the latest working software.

FIGURE 9. Test suites

24

5.2.3 Testing Pipeline

Testing pipeline is where the timing test cases are run. (Figure 10.)

FIGURE 10. Content of the testing pipeline.

25

First, the pipeline parameters are defined. They are inherited from the test suite.

The pipeline accepts the following parameters: Node, CC, BW and Tech. Node

is the PC where the tests are run from. CC is the number of cell carriers. BW is

the carrier bandwidth and lastly, Tech is the mobile network technology, which in

this case is 5G NR. After the parameters are defined some more environment

variables are defined, for example, Git repository branches which to clone the Git

repositories.

After the parameters and environment variables are defined, repositories

containing the Python scripts and robot suites used in the testing are cloned from

Git.

Before launching the robot testing suite, the radio needs to be powered on. After

the radio is powered up, the robot testing suite is launched, radio and BBU

emulator are configured and then the tests are performed. After the

measurements are done, the results are visible in the pipelines job description.

(Figure 11.)

FIGURE 11. Measurement results.

Depending on the measurement results, passed or failed, the pipeline outcome

status is set as “success” or “failure”, which tells the testing suite if the

measurements have passed or failed in the testing pipeline.

26

5.3 Robot

Robot Framework portion of the automation consists of robot scripts and various

custom robot libraries that are used in these scripts. The first part of the robot

script is used to configure the radio unit, base band emulator and test

environment.

The second part of the robot script is where the robot compatible version of the

timing measurement program is used. In this robot script, the signal switching unit

is also controlled to select the desired antenna from the radio unit. (Figure 12.)

FIGURE 12. Example robot test case.

In the “*** Settings ***” tab the timing measurement program module is imported

as a resource. The SSU controller is imported as a Python script as it contains

only a class with methods which can be used as keywords by the Robot.

27

In the “*** Test Case ***”, first the IP address of the SSU is given. Next, the

antenna route is set for the downlink measurement. Then the timing

measurement is started with the keyword “Test dl air frame radio pps” which tests

radio’s downlink air frame timing. The keyword returns the result of the

measurement as nanoseconds, a csv result file is created with the Keyword

“Create Downlink Result Files”. The keyword accepts the RU name, result value,

measured antenna and frequency as input parameters. Then the uplink

measurements are run with the same principles.

28

6 MODULE TESTING

To guarantee that there is always a working version of the Robot framework

compatible timing program on the master branch of the Git repository, the

validity of the changes pushed or merged to master must be checked for errors.

The basic CI was setup using a GitLab webhook and Jenkins. The Jenkins

pipeline is triggered with a webhook from GitLab. (Figure 13.). At the time of

writing this thesis, only testing the installation of the timing program as a Python

module was implemented in the CI. In the future the plan is to add unit testing to

the CI to test the methods in the timing program.

FIGURE 13. Module testing flow

29

7 CONCLUSION

The aim of this thesis was to create a fully automated testing process for RU air

interface timing using Robot Framework to add the created automation to

Nokia’s existing continuous integration framework.

The automation created worked as planned, though there are still issues that

prevent taking the automation into use with the RU this automation was created

with. The main issue is configuring the RU and BB emulator correctly with the

automation libraries since the libraries were not developed directly to the RU

that was used in the development of the automation. Adding the support for the

RU was not in the scope of this thesis but development also continues on this

after the thesis. In the future, the plan is also to add to the test environment

more radios that are already supported by the automation libraries.

Most of the automation developed was done without the test setup shown in

figure 4. It slowed down the development when there was no possibility to test

the work done to the timing program. Later during the project, a test setup was

built for the purpose of developing the automation and it sped up the

development and issues with the code were debugged quickly.

30

REFERENCES

1. Fowler, Martin 2006. Continuous Integration. Date of retrieval: 1.9.2021

https://www.martinfowler.com/articles/continuousIntegration.html

2. Robot Framework 2021. Robot Framework. Date of retrieval: 18.8.2021.

https://www.robotframework.org

3. CPRI Info 2021. Common Public Radio Interface. Date of retrieval:

2.8.2021.

http://www.cpri.info/

4. Common Public Radio Interface Specification V7.0 2015. Common

Public Radio Interface (CPRI); Interface Specification.Date of retrieval:

2.8.2021.

http://www.cpri.info/downloads/CPRI_v_7_0_2015-10-09.pdf

5. Enhanced Common Public Radio Interface Specification V2.0 2019.

Common Public Radio Interface: eCPRI Interface Specification. Date of

retrieval: 2.8.2021.

http://www.cpri.info/downloads/eCPRI_v_2.0_2019_05_10c.pdf

6. RF Wireless world 2017. Difference between CPRI and eCPRI used in

5G. Date of retrieval: 3.8.2021.

https://www.rfwireless-world.com/Terminology/CPRI-vs-eCPRI.html

7. Carrier Tech 2020. eCPRI. Date of retrieval: 3.8.2021.

https://www.carritech.com/news/ecpri/

8. Jenkins 2021. Jenkins. Date of retrieval: 1.9.2021.

https://jenkins.io