coordinated control and spectrum management for 5g ... · andsf access network discovery and...

Coordinated Control and Spectrum Management for 5G Heterogeneous Radio Access Networks Grant Agreement No. : 671639 Call: H2020-ICT-2014-2

Deliverable D6.1 Draft report on technical validation

The project is co-funded by

Version: 1.0

Due date: 31.01.2017

Delivered date: 24.03.2016

Dissemination level: PU

D6.1 Draft report on technical validation

H2020 5G-PPP COHERENT Deliverable 6.1 2

Authors Kostas Katsalis (EUR), Navid Nikaien (EUR), George Agapiou (OTE), Antonio Cipriano (TCS), Laurent Louf (TCS), Fang-Chun Kuo (TP), Kostas Pentikousis (TP), Janne Seppänen (VTT), Teemu Rautio (VTT), Tao Chen (VTT), Supreeth Herle (CNET), Roberto Riggio (CNET), Dimitri Marandin (CMA), Rebecca Steinert (SICS), Heikki Kokkinen (FS), Jaako Ojaniemi (FS), Daniel Peethala (UDE), Karol Kowalik (INEA) Editors Kostas Katsalis (EUR), Navid Nikaein (EUR) Coordinator Dr. Tao Chen

VTT Technical Research Centre of Finland Ltd

Tietotie 3

02150, Espoo

Finland

Email: [email protected]

Disclaimer The information in this document is provided ‘as is’, and no guarantee or warranty is given that the information is fit for any particular purpose. The above referenced consortium members shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials subject to any liability which is mandatory due to applicable law. Acknowledgement This report is funded under the EC H2020 5G-PPP project COHERENT, Grant Agreement No. 671639. © 2015-2017 COHERENT Consortium



Executive summary

The aim of this deliverable is to create a test plan in order to successfully carry out the assessment of the COHERENT platform. This test plan uses as the input of the results of the technical work packages, describes the infrastructure deployment, and provides a first detailed description of the integration process of the different components that will compose the final COHERENT testbed. Moreover, this deliverable defines the detailed scenarios and test cases that are considered to test and validate the COHERENT solution.

The COHERENT integration methodology is described with the requirements towards integration, while a specific use case format is defined for every use case. The sequential steps that will lead to the final COHERENT testbed include the outline of the infrastructure topology, the specification of the test case scenarios and finally the integration of all COHERENT components.

This deliverable also describes the COHERENT testbeds used for the proof of concept (PoC) demonstrations, by means of hardware installation and configuration as well as a high level description of the main software components. This deliverable also describes the extensions planned towards integration on per testbed basis.

The technical approach and COHERENT testbed architecture in relation to the design-level structure and organization for the entire COHERENT system are presented. The basic principles throughout the design of every aspect of the testbed are provided. Specifically, we analyze the following aspects with attention on implementation regarding the COHERENT KPIs, the software architecture and the network graph Infrastructure.

The project will employ several use cases carrying out the assessment of the COHERENT concepts. For every selected use case, a description of the use case is provided, together with a note on the added value, the technical approach, expected or early results and the technical impacts.



Table of contents

EXECUTIVE SUMMARY ....................................................................................................................................................... 3 TABLE OF CONTENTS .......................................................................................................................................................... 4 LIST OF ABBREVIATIONS ................................................................................................................................................... 5 LIST OF FIGURES ................................................................................................................................................................. 11 LIST OF TABLES .................................................................................................................................................................. 12 1. INTRODUCTION ......................................................................................................................................................... 13 2. METHODOLOGY ........................................................................................................................................................ 15

2.1 COHERENT INTEGRATION METHODOLOGY ....................................................................................... 15 2.2 TEST CASE FORMAT ............................................................................................................................ 18 2.3 EXPECTED RESULTS ............................................................................................................................. 19

3. COHERENT TESTBEDS DESCRIPTION................................................................................................................... 20 3.1 OPENAIRINTERFACE (OAI) LTE TESTBED........................................................................................... 20

3.1.1 Testbed Description ........................................................................................................................ 20 3.1.2 Hardware Platform .......................................................................................................................... 22

3.2 5G-EMPOWER TESTBED .................................................................................................................... 22 3.2.1 Testbed Description ........................................................................................................................ 22 3.2.2 Architecture, services and testbed resources................................................................................... 23

3.3 COMMAGILITY LTE TESTBED ............................................................................................................. 24 3.3.1 Testbed description ......................................................................................................................... 24 3.3.2 eNodeB hardware and software description ................................................................................... 24 3.3.3 IPC Mechanisms ............................................................................................................................. 25

3.4 UDE DAS PHY TESTBED .................................................................................................................... 26 3.4.1 Testbed description ......................................................................................................................... 26 3.4.2 Scenarios and Software description ................................................................................................ 27

3.5 CEP TESTBED FOR SPECTRUM SHARING: THE CORE SOLUTION ........................................................... 28 3.5.1 Testbed Description ........................................................................................................................ 28

4. DESIGN AND IMPLEMENTATION OF C3 ............................................................................................................... 30 4.1 KPIS .................................................................................................................................................... 30 4.2 C3 SOFTWARE ARCHITECTURE ............................................................................................................ 31 4.3 COHERENT NETWORK GRAPHS AND INTEGRATION TO C3 ............................................................... 34

4.3.1 Design and Implementation of COHERENT network graphs ........................................................ 35 4.3.2 Extreme Complexity and Data Events in COHERENT .................................................................. 36 4.3.3 Implementation of COHERENT Network Graphs Infrastructure ................................................... 37

5. EXTENSIONS ON EXISTING TESTBEDS TOWARDS INTEGRATION ................................................................ 38 5.1 TESTBED EXTENSIONS ......................................................................................................................... 38

5.1.1 OAI Testbed: Extensions and new APIs ......................................................................................... 38 5.1.2 5G-EmPower Testbed: Extension and new API ............................................................................. 39 5.1.3 CommAgility (CMA) Testbed: Extensions and new APIs ............................................................. 39 5.1.4 CEP Testbed: Extensions and new APIs ........................................................................................ 40 5.1.5 UDE DAS PHY Testbed ................................................................................................................ 40

5.2 INTEGRATION ISSUES AND CHALLENGES ............................................................................................. 41 6. DESIGN, SETUP AND EXECUTION OF PROOF OF CONCEPT DEMONSTRATIONS ....................................... 43

6.1 VALIDATION SCENARIOS SPECIFICATIONS ........................................................................................... 43 6.2 THE COHERENT USE CASES TOWARDS INTEGRATION ........................................................................ 44

6.2.1 Spectrum sharing use cases............................................................................................................. 44 6.2.2 RAN Sharing and Network Slicing Use Case ................................................................................. 54 6.2.3 Load balancing Use Cases .............................................................................................................. 61

7. CONCLUSIONS ........................................................................................................................................................... 68



List of abbreviations

2G 2nd Generation Mobile Networks

3D Three-Dimensional

3GPP Third Generation Partnership Project

4G 4th Generation Mobile Networks

5G 5th Generation Mobile Networks

5GPoAs 5G Point of Access

5G PPP 5G Infrastructure Public Private Partnership

AI Air Interface

ANDSF Access Network Discovery and Selection Function

API Application Programming Interface

ARPU Average Revenue Per User

AS Access Stratum

ASA Authorised Spectrum Access

BBU Baseband Unit

BM-SC Broadcast Multicast Service Centre

BNetzA Federal Network Agency for Electricity, Gas, Telecommunications, Post and

Railway in Germany

BS Base Station

BSS Base Station System

C3 Central Controller and Coordinator

CA Carrier Aggregation

CAL CHARISMA Aggregation Level

CAPEX Capital Expenditure

CBRS Citizens Broadband Radio Service

CC Component Carrier

CDN Content Delivery Network

CEPT European Conference of Postal and Telecommunications Administrations

CESC Cloud-Enabled Small Cell

CESCM Cloud Enabled Small Cell Manager

CMOS Complementary Metal Oxide Semiconductor

CN Core Network

CoMP Coordinated Multipoint

CP Control Plane

CPE Customer Premises Equipment

C-RAN Cloud Radio Access Network

CriC Critical Communications



D2D Device-to-Device

DC Data Centre

DAS Distributed Antenna System

DMO Direct Mode Operation

DPI Deep Packet Inspection

D-RAN Distributed Radio Access Network

ECC Edge Cloud Computing

eICIC enhanced Inter-Cell Interference Coordination

eNB Evolved Node B

eNodeB Evolved Node B

eMBB enhanced Mobile Broadband

eMBMS enhanced Multimedia Broadcast Multicast Service

EMS Element Management System

EPC Evolved Packet Core

E-UTRAN Universal Terrestrial Radio Access Network

eV2X enhanced Vehicle-to-X Communications

EPC Evolved Packet Core

ESO Essential Service Operator

ETP European Technology Platform

ETSI European Telecommunications Standards Institute

EU European Union

FDD Frequency-Division Duplex

FICORA Finnish Communications Regulatory Authority

ForCES Forwarding and Control Element Separation

GPP General Purpose Processors

GSM Global System for Mobile Communications

GW GateWay

GWCN Gateway Core Network

H2H Human-to-Human

HAN Heterogeneous Access Network

HMN Heterogeneous Mobile Network

HSS Home Subscriber Server

ICIC Inter-Cell Interference Coordination

IEA International Energy Agency

IEEE Institute of Electrical and Electronics Engineers

IETF Internet Engineering Task Force

IMT International Mobile Telecommunications

IMU Intelligent Management Unit



IoT Internet of Things

IP Internet Protocol

IRTF Internet Research Task Force

ISI Integral SatCom Initiative

ITRS International Technology Roadmap for Semiconductors

ITU International Telecommunication Union

ITU-R ITU – Radio Communication Sector

IWF Interworking Function

KPI Key Performance Indicator

L1 Layer 1

L2 Layer 2

LSA Licensed Shared Access

LSN Last Sequence Number

LTE Long-Term Evolution

LTE-A Long-Term Evolution Advanced

M2M Machine-to-Machine

MAC Media Access Control

MANO MANagement and Orchestration

MBMS Multimedia Broadcast Multicast Service

MCS Modulation and Coding Schemes

MEC Mobile Edge Computing

MIMO Multiple-Input Multiple-Output

MIoT Massive Internet of Things

MMC Massive Machine Communication

MME Mobility Management Entity

MN Mobile Network

MOCN Multi-operator Core Network

MTC Machine-Type Communication

MTD Machine-Type Device

NEO Network Operation

mmWave Millimeter Wave

MNO Mobile Network Operator

MOCN Multi-operator Core Network

MoI Ministry of Interior

MVNO Mobile Virtual Network Operator

Naas Network as a service

NAS Non-Access Stratum

NEO Network Operation



NFV Network Function Virtualisation

NFVO Network Functions Virtualisation Orchestrator

NFVI Network Function Virtualisation Infrastructure

NGMN Next Generation Mobile Networks Alliance

NRA National Regulation Agency

NVS Network Virtualisation Substrate

Ofcom Office of Communications

OFDM Orthogonal Frequency Division Multiplexing

OFDMA Orthogonal Frequency Division Multiple Access

ONF Open Networking Foundation

OPEX Operational Expenditure

OSI Open Systems Interconnection

OSS Operations support systems

PCC Primary Component Carrier

P-GW PDN gateway

PHY Physical Layer

PLMN Public Land Mobile Network

PMR Private (or Professional) Mobile Radio

PNF Physical Network Function

PON Passive Optical Network

PoP Point of presence

PPDR Public Protection and Disaster Relief

PRACH Physical Random Access Channel

ProSe Proximity Services

PTT Push-To-Talk

PTS Swedish Post and Telecom Agency

QoE Quality of Experience

QoS Quality of Service

RAN Radio Access Network

RANaaS Radio Access Network as a Service

RAT Radio Access Technology

RAU Radio Access Unit

RB Resource Block

RF Radio Frequency

RN Relay Node

RT Radio Transceiver

R-TP Radio Transmission Point

RRH Remote Radio Head



RRM Radio Resource Management

RRU Radio Remote Unit

RTC Real-Time Controller

SA Service and System Aspects

SAS Spectrum Access System

SC Small Cell

SCC Secondary Component Carrier

SCS Service Capability Server

SDK Software Development Kit

SDN Software Defined Network

SD-RAN Software-Defined Radio Access Network

S-GW Serving gateway

SID Service Integration Driver

SLA Service Level Agreement

SLP SUPL Location Platform

SME Small and Medium Enterprises

SON Self Organizing Network

SONAC Service-Oriented Virtual Network Auto-Creation

SUPL Secure User Plane Location

TCO Total Cost of Ownership

TDD Time-Division Duplex

TEDS TETRA Enhanced Data Service

TETRA Terrestrial Trunked Radio

TN Transport Node

TR Technical Report

TDM Time Division Multiplexing

TR Technical Report

UAV Unmanned Aerial Vehicles

UE User Equipment

UM Unacknowledged Mode

UMTS Universal Mobile Telecommunications System

UP User Plane

V2X Vehicle-to-X Communications

VIM Virtualised Infrastructure Manager

VM Virtual Machine

VNF Virtual Network Function

VNFD VNF Descriptors

VNFM VNF Manager



vBBU virtual Baseband Unit

vBSC virtual Base Station Controller

VIM Virtualised Infrastructure Manager

VNF Virtual Network Function

VNO Virtual Network Operator

VPL Vehicular Penetration Loss

WiFi Wireless Fidelity

WiMAX Worldwide Interoperability for Microwave Access

WLAN Wireless Local Area Network



List of figures

Figure 1: The requirements pyramid following the Zielczynski approach. .......................................... 17 Figure 2: COHERENT testbed development methodology ................................................................. 18 Figure 3: OAI LTE Software Stack ...................................................................................................... 21 Figure 4: OAI ExpressMIMO2 hardware platform .............................................................................. 22 Figure 5: The 5G-EmPOWER high-level architecture ......................................................................... 23 Figure 6: Network slicing in 5G-EmPOWER ...................................................................................... 24 Figure 7: AMC-K2L-RF2 card ............................................................................................................. 25 Figure 8: eNodeB software architecture ............................................................................................... 25 Figure 9: Message posting .................................................................................................................... 26 Figure 10: Inter-processor message passing mechanism ...................................................................... 26 Figure 11: UDE - LTE + DAS PHY testbed ........................................................................................ 26 Figure 12: IAF Dual C6670 AMC module layout ................................................................................ 27 Figure 13: DL PHY modules layout for DAS+LTE ............................................................................. 27 Figure 14: The main components of the CORE trial environment. ...................................................... 28 Figure 15: Realization of the spectrum sharing testbed utilizing the CORE trial environment. .......... 29 Figure 16: COHERENT Testbed .......................................................................................................... 32 Figure 17: C3 and supporting database ................................................................................................ 34 Figure 18: From Original network to Network graph, that represents the physical connectivity formed at the abstraction layer by extracting the radio access layer parameters. ............................................. 36 Figure 19: SINR (in dB) of network graphs from underlying network. ............................................... 37 Figure 20: Enabling programmability in the OAI eNodeB using SDN-based agent-controller solution. .............................................................................................................................................................. 38 Figure 21: Coordinating heterogeneous radio access networks using the 5G-EmPOWER Mobile Network Operating System. ................................................................................................................. 39 Figure 22: Realization of the spectrum sharing testbed ....................................................................... 44 Figure 23: The spectrum operation of TD-LTE is managed by CEP Testbed. .................................... 45 Figure 24: The spectrum operation of two commercial networks (WiMAX and TD-LTE) is managed by CEP Testbed with FS module. ......................................................................................................... 47 Figure 25: The LSA Spectrum Sharing: CORE solution testbed ......................................................... 50 Figure 26: Coverage of INEA WiMAX network around Poznan area ................................................. 51 Figure 27: Changes of spectrum allocation depending on the traffic patterns ..................................... 52 Figure 28: RAN Sharing Use Case ....................................................................................................... 55 Figure 29: For this use case the testbed used is OAI. Both the RTC and non-RTC features are exploited. .............................................................................................................................................. 55 Figure 30: RAN Sharing OAI Scheduling for RTC ............................................................................. 59 Figure 31: Structure of a policy reconfiguration message. ................................................................... 60 Figure 32: Policy reconfiguration for MVNO management: Dynamic allocation of resources ........... 60 Figure 33: Policy reconfiguration for MVNO management: CDF of UE throughput when different scheduling types are used for different slices. ...................................................................................... 60 Figure 34: For this use case the 5G-Empower and the SICS testbeds will be exploited. ..................... 61 Figure 35: Topological overview of the load balancing use case infrastructure. ................................. 64 Figure 36: Sequence diagram for load balancing in LTE ..................................................................... 65 Figure 37: Sequence diagram for load balancing in Wi-Fi .................................................................. 66



List of tables

Table 1: COHERENT Architecture Entities ........................................................................................ 15 Table 2: COHERENT testbeds ............................................................................................................. 16 Table 3: COHERENT use case template .............................................................................................. 18 Table 4: OAI configuration options ..................................................................................................... 21 Table 5: Technical KPIs used for system evaluation ........................................................................... 30 Table 6: Parameters to consider for LTE evaluations .......................................................................... 31 Table 7: LTE parameters for multi-connectivity .................................................................................. 36 Table 8: Scenarios specification according to D2.1 ............................................................................. 43 Table 9: Time measured for different phases of the LSA band evacuation ......................................... 51 Table 10: Total delay of research and commercial part of LSA trial ................................................... 51



1. Introduction This deliverable describes the integrated COHERENT platform and defines the test scenarios in order to facilitate the delivery of functional demonstrators. The COHERENT project focuses on developing a next generation unified control and coordination framework for heterogeneous radio access networks. COHERENT adopts the concept of resource and service virtualization across technology domains, while the key innovation of the project is the development of a unified programmable control framework used to coordinate the underlying heterogeneous mobile networks (LTE and Wi-Fi) as a whole.

The COHERENT coupling of network virtualization with SDN control facilitates efficient operation using abstract network views (which we refer to as network graphs in the project) that can be used for a) network monitoring and optimization and b) for sharing of physical resources among various virtual operators, introducing new business models and enabling new exploitation opportunities.

The goal of D6.1 is twofold. On one hand, this deliverable defines the steps required in order to build an integrated COHERENT testbed. On the other hand, it defines the COHERENT use case scenarios that will fully demonstrate the COHERENT concept. Previous project deliverables [COHERENT-D2.1, COHERENT-D2.2] define the technical and service requirements, the COHERENT architecture, and identified the use cases relevant to the COHERENT approach.

This deliverable is related to all Tasks in WP6, namely Testbed Integration and Setup (T6.1), Execution of the Experiments (T6.2), Analysis of the Results and Technical Impact (T6.3) with more effort associated with T6.1.

COHERENT Testbed Integration: COHERENT testbed integration is related to both the data plane and control plane procedures that are required to carry out the demonstrators, evaluations and assessments of the technical approach. The data plane and physical resources (LTE and Wi-Fi networks) provide transport services to the modules built on top. An abstract network view (or network graph as defined in [COHERENT-D2.2]) exposes information of the physical and virtualized networks to the control plane. In the control plane the logical operations such as routing and traffic engineering are carried over the converged multi-RAT network. In COHERENT, a logically centralized entity called Central Controller and Coordinator (C3) is in charge of logically centralized network-wide control and coordination among entities in RAN, based on centralized network view (see [COHERENT-D2.2] for more details). In this deliverable we also analyze the COHERENT approach where information related to both operational and configuration data is exposed as network graphs. Furthermore, the actual implementation of a system that supports the network graphs is provided and the interaction with the C3 is described. Four testbed facilities will be used to support the integrated COHERENT testbed- namely, a LTE eNodeB testbed- provided by CommAgility; OpenAirInterface (OAI) platform provided by EURECOM that supports both the LTE EPC and eNodeB sub-systems; the 5G-EmPOWER testbed supporting both LTE and Wi-Fi access technologies provided by CREATE-NET; and Complex Event Processing (CEP) system provided by the Converging Network Laboratory (CNL) of VTT. In this document, a technical description of each testbed is given by means of hardware installation and a high level description of the main software components.

COHERENT use cases and KPIs: The use cases in this deliverable reflect the different perspectives of the COHERENT actors and highlight the benefits and new opportunities that could be derived from the overall COHERENT framework. They try also to capture the technical innovations that are carried out in the project. All the use cases are defined and described using a common template which has been designed to cover the technical aspects, associated to each one. D6.1 also introduces the fundamental (Key Performance Indicators) KPIs that will drive the testing procedures.

In more detail using a standard template we describe the necessary steps for the successful and thorough execution of the following use cases:

• Spectrum sharing: Due to the potentially large area to be considered, we scope and analyze the spectrum sharing use cases in the following sub use cases:

o Spectrum sharing within the COHERENT-controlled network, o Spectrum sharing within the non-COHERENT-controlled networks



between microwave links and WiMAX networks, and between two TD-LTE networks.

• RAN sharing: In this use case we exploit the unique programmability features of OAI and we demonstrate scenarios where the eNodeB is virtualized and supports concurrent operation of multiple Mobile Virtual Network Operators (MVNOs) with specific scheduling principles per operator.

• Load balancing: In this use case we consider an open programmable platform used for LTE and Wi-Fi load balancing scenarios.

These use cases have been carefully selected by the project partners due to their importance towards integrated 5G communications. A detailed analysis for each use case is provided in Section 6, where preliminary results are also presented for each use case. A discussion is also made for per-user-throughput improvement using distributed antennas designs (DAS): Deployment of RRH, UE pairing based on a selection transmission technique for improving/maintaining per user throughput and coverage extension.

Each use case demonstrator is a collaboration action between groups of partners, who complement each other as they tackle a common problem. As per the project plan, WP6 partners share resources and knowledge and mutually benefit from working together as they accomplish the COHERENT goals. For the software solution aspects the concrete results of the collaborative actions are related to the integrated C3 and network graphs infrastructure. This approach will also be followed for the whole project duration and will facilitate convergence on all the software and hardware aspects of the integrated demonstrators that will be delivered in D6.2 [COHERENT-6.2].

The testbed components map to the relevant components of the COHERENT architecture that was described in deliverable D2.2 [COHERENT-D2.2]. The concept of COHERENT network abstraction with Network Graphs that maps horizontally and vertically at different levels of the proposed architecture is part of the WP3 activities. WP4 activities are related to spectrum sharing, while in WP5 the partners combine their expertise in lower layers for mobile networks and software defined networking (SDN) to explore the programmable control methods and algorithm implementations for inter-cell coordination, resource allocation, and control plane resiliency. The work delivered in the context of these WPs will serve as input for the technical base of the integrated testbed in WP6. The activities in this deliverable will serve as input for D6.2 [COHERENT-6.2] for the final technical validation of the COHERENET system.

Structure of the document

• Section 2 describes the methodology applied to document the test scenarios and define the requirements towards integration of the final COHERENT testbed.

• Section 3 provides a description of the COHERENT infrastructure, referring to its four core testbeds from which the final COHERENT testbed will be derived.

• Section 4 examines the implementation perspective for the integrated C3 and Network Graphs infrastructure.

• Section 5 presents extensions of the individual testbeds towards integration. • Section 6 defines the use cases to be considered in end-to-end network integration testing. In

short, the section describes four use cases with the aim of testing and validating particular COHERENT components and functions while providing the proof of concept (PoC) of the COHERENT architecture as an integrated solution. It also provides preliminary results for each use case.

• Section 7 concludes the deliverable and describes the next steps.



2. Methodology This section defines the methodology adopted to address the scenarios which will showcase system integration. The sequential steps that will lead to the final COHERENT testbed include the outline of the infrastructure topology, the specification of the test case scenarios and finally the integration of all COHERENT components.

The upcoming COHERENT evaluation carried out in D6.2 will be based on both the technical and business aspects of the proposed solution defined in previous deliverables provided by WP2-WP5 through the work delivered in the respective technical WPs.

2.1 COHERENT Integration methodology In the COHERENT layered architecture defined in the deliverable D2.2 [COHERENT-D2.2] a heterogeneous physical infrastructure layer includes a hybrid wireless domain composed by LTE/Wi-Fi access networks. LTE and Wi-Fi technologies were selected as they are expected to play an important role in the next generation converged wireless access networks. 5G will be composed by evolved LTE for below 6 GHz, New Radio for above 6 GHz and Wi-Fi that are expected to converge [MT14]. Due to the increased complexity, the project focuses on the evolved LTE network and Wi-Fi, nevertheless the effort will be given to provide generic solutions for the heterogeneous wireless domain whenever possible. According to the definitions provided in D2.2 [COHERENT-D2.2] a number of entities are part of the overall COHERENT architecture. We summarize the description of each entity in Table 1:

Table 1: COHERENT Architecture Entities

Entity Acronym Description

Radio Transmission

Point R-TP R-TP is a radio access point implementing full or partial RAN node functions. An

R-TP may include control plane functions.

Virtual Radio Processing

vRP vRP is a computing platform enabling centralized processing of full or partial RAN node functions. vRP is offloaded from one or multiple R-TPs and may include control plane functions.

Radio Transceiver

RT RT is a logical radio access entity with full RAN node functions, which is the flexible combination of R-TP, vRP and RTC functions. A set of RTs is forming a radio access network which is coordinated and controlled by C3. There are multiple physical and virtual resources and components per RT. An RT could comprise one vRP (virtual device) and one or more R-TPs (physical devices). For example, in the Cloud-RAN architecture the R-TP coincides with the RRH, while the vRP coincides with the BBU Pool. In some particular case, e.g., D2D, RT could be an UE, operating as a relay node.

Transport Node

TN TN is the entity located between RTs and the core network. A set of TNs is forming a backhaul/fronthaul network whose data plane can be configured by the C3. A network switch is an example of Transport Node.

Real-Time Controller

RTC RTC is a logical entity in charge of local or region-wide control, targeting real-time control operations, e.g., MAC scheduling. RTC maintains a local network view and could run directly on one RT or on a virtualized platform and, accordingly, receives monitoring information gathered from one RT or multiple RTs.

Central Controller

and Coordinator

C3 C3 is a logically centralized entity in charge of logically centralized network-wide control and coordination among entities in RAN based on a centralized network view. C3 could be implemented with distributed physical control instances sharing network information with each other. Sharing network information among C3 instances creates the logically centralized network view and therefore achieves logically centralized control and coordination.

Some examples of physical RTs include LTE eNodeBs in cellular networks or Wi-Fi APs in the WLANs, where C3 is the most critical architectural entity that allows the operation of all network



infrastructures (physical or virtual) at runtime. C3 provides all functionalities for the management and dynamic provisioning of multi-layer network connections.

The integrated COHERENT testbed will be based on the integration of the following five testbeds provided by individual project partners. A summary of each testbed offering is provided in Table 2.

Table 2: COHERENT testbeds

Testbed Technology Technology support summary

CommAgility Provider: CommAgility

LTE The CommAgility testbed currently consists of one LTE eNodeB including a LTE PHY and LTE protocol stack connected to one commercial LTE UE.

OAI Provider: EURECOM1

LTE OAI includes a remotely accessible large scale system validation platform for software-defined radio access networks. It is based on existing hardware components, namely OAI EXMIMOII, USRP B210, as well as software components, namely OAI LTE eNodeB and UE spanning all layers. The goal of this testbed is to validate the innovations in MAC/PHY abstractions and programmability developed in WP3 and WP5.

5G-EmPOWER Provider: CreateNET2

Wi-Fi/LTE 5G-EmPOWER is an open Mobile Network Operating System for Wi-Fi and LTE networks. The testbed consists of 20 Wi-Fi Access Points, 6 open small cells based on the USRP B210 platform and running a modified version of the OAI stack allowing integration with the 5G-EmPOWER Mobile Network Operating System. The testbed also includes a software-defined backhaul network including 6 Micro Servers and 2 OpenFlow Switches.

CEP Provider: VTT3

CEP CEP provides generalized interfaces to input data events, makes decisions based on them and output control and new data events for further processing. Control events are used to change configuration such as spectrum, power and other parameters of target systems. The CEP testbed is targeted in this context mainly for spectrum sharing tests/demonstrations.

UDE DAS PHY Provider: UDE

LTE UDE implements Distributed Antenna System (DAS) functionality with maximum 2 (2x2 MIMO) RRHs based on LTE PHY. This includes identification of software PHY processing blocks needed to be modified to accommodate DAS and necessary testing. The testbed targets improvements on coverage, minimization of transmit power and increased per-user throughput.

Software and hardware component testing will be applied at each testbed independently prior to integration. The proof of principle activities will follow a common process based on the requirements analysis and the detailed design of the services and the work carried out in WP2, WP3, WP4 and WP5. Requirements towards integration

A set of requirements towards 5G communications dictates and shapes the major functionalities and features that the COHERENT platform should support. In general, depending on the format, source and common characteristics, the requirements can be classified into several different types. Typical examples include requirements raised from stakeholders, use case and test case requirements, etc. In 1 http://www.openairinterface.org/ 2 http://empower.create-net.org/ 3 http://core.willab.fi/?q=node/66

http://core.willab.fi/?q=node/66



D2.1 a set of requirements was defined for the overall COHERENT system. See D2.1 [COHERENT-D2.1] for a detailed analysis on the requirements that drive the whole project activities.

In this document, based on the rationale described in D2.1 [COHERENT-D2.1], the requirements can be used to drive the integration process. We adopt a top-down approach and from High Level Business Requirements first, Service Requirements are derived that in turn lead to integrated Network Service Requirements that finally translate to Physical Infrastructure Requirements. To graphically represent the logical relationship between these requirements, the pyramid chart approach [Zielczynski] has been adopted as shown in Figure 1.

Figure 1: The requirements pyramid following the Zielczynski approach.

The High-Level Business Requirements of the stakeholders reside at the very top of the pyramid. In COHERENT an important innovation is that Network Slicing is the main component around which the Business Requirements are described. The Service Requirements, the integrated Network Service Requirements and the Physical Infrastructure Requirements appear at the lower levels of the pyramid chart. • Based on the level of each requirement, different degree of detail is provided. More specifically,

the lower the level, the more detailed the requirements are. • It is clear that this pyramid graph also depicts the relationship between requirements at different

levels and hence enables their traceability in a top down fashion. An example for the exploitation of the requirements pyramid is presented in the right hand side of Figure 1, for the RAN Sharing use case, where we also present the way the COHERENT C3 is used to satisfy the requirements set by the use case. Example: RAN Sharing use case: a High Level Business Requirement could be the resource block (RB) allocation required in the long run for a virtual operator (e.g., 40% of RB allocation), while the Service Requirement requires for the necessary northbound API available (including lifecycle management of the service), the Network Service Requirement would be the scheduling mechanism available and the Physical Infrastructure Requirement would be a set of integrated programmable RAN elements in the eNodeB. In principle, the C3 Northbound API exposes network agnostic services and orchestrates the necessary procedures that will drive the actual service deployment and the actual service operation. Depending on the service requested and the requirements set, RTC or/and non-RTC are exploited. These requirements will drive the integration procedure. We note that specific demonstrators will utilize specific parts of the system, on per testbed basis using the services exposed by the C3 control plane, through a novel northbound API.



Figure 2: COHERENT testbed development methodology

The integration of all COHERENT components will lead to the final COHERENT testbed. Because of the increased complexity, we will follow a step-by-step process, presented in Figure 2. Each individual testbed will be extended towards integration with the necessary southbound and northbound APIs, while each API and each data-plane and control-plane operation will be extensively tested in order to demonstrate the efficiency of the software/hardware and the ability to satisfy both the COHERENT’s system and user level requirements described in D2.1 [COHERENT-D2.1]. These activities are also related to the publication of the COHERENT SDK made in the context of WP2 activities and will be presented in D2.3 [COHERENT-D2.3] and D2.4 [COHERENT-D2.4].

2.2 Test Case Format

The COHERENT test cases define the necessary interactions between components (or the whole system) under consideration in order to accomplish the project goals. Section 3 of D2.1 [COHERENT-D2.1] presents a wide range of scenarios and use cases that are within the capabilities of the COHERENT solution. More specifically, we categorized a number of use cases organized around the following 6 scenarios:

• Scenario 1: Network co-operation and inter-operability • Scenario 2: Spectrum management • Scenario 3: Critical communications • Scenario 4: Network slicing • Scenario 5: Massive IoT • Scenario 6: Enhanced broadband communications in public transportation

Several use cases were described per scenario, out of which a subset will be considered for the Proof of Concept (PoC) demonstrations. For every use case, we will use the following format presented in Table 3 . This format was carefully designed to address and specify the integrated test cases.

Table 3: COHERENT use case template

Test Description A brief description of the test case.

Architecture Layers Specification of the architecture layers involved in the specific test case. Components Specification of the components involved in the specific test case.

Pre-Conditions Specification of what needs to be valid before the test case execution. Dependencies Definition of any dependencies on other test cases or test requirements.

Test Steps Step-by-step instructions on how to carry out the test. Expected Results Specification of the expected results that describe system output or how the system

must react. Post Conditions Specification of the state of the system after the execution of this test case.

KPIs Key Performance Indicators to be used in the technical performance evaluation of the system.

Final COHERENT Tetsbed

Tetsbed IntegrationData Plane Aspects Control Plane Aspects Service Plane Aspects

TestingAPIs API Extensions

Individual TestbedsComAgility (LTE) OAI (LTE) 5G-EmPOWER CEP testbed



2.3 Expected Results Throughout this deliverable the test scenarios and test cases are specified and together with the proposed integration plan, indicate how the COHERENT individual software and hardware components will form the full COHERENT testbed platform. It is expected that following the platform integration, the testing process will verify and validate that the platform meets the system requirements and specifications as specified in the previous COHERENT deliverables.



3. COHERENT Testbeds description This section describes the COHERENT testbeds used for the proof of concept (PoC) demonstrations, by means of hardware installation and configuration as well as a high level description of the main software components. As shown in Table 2, three LTE testbeds will be used (OAI, CommAgility, UDE DAS PHY), one integration testbed of Wi-Fi and LTE ( 5G-EMPOWER) and one spectrum sharing testbed (CEP). The remainder of this section presents the main technologies provided by each testbed. A note on the direction of needed extensions is provided in Section 4.

3.1 OpenAirInterface (OAI) LTE testbed 3.1.1 Testbed Description

The OAI platform offers an open-source software-based implementation of the LTE system spanning the full protocol stack of 3GPP standards both in E-UTRAN and EPC [OAI, NN14, NN14b]. It can be used to build and customize an LTE base station and a core network on a PC and connect a commercial User Equipment (UE) to test different configurations and network setups and monitor the network and mobile devices in real-time. OAI is based on a PC-hosted software radio frontend architecture. With OAI, the transceiver functionality is realized via a software radio front-end connected to a host computer for processing. This approach is similar to other software-defined radio (SDR) prototyping platforms in the wireless networking research community such as SORA [KT11].

Other similar approaches combining PCs and FPGA-based processing make use of NI LabVIEW software or the WARP [KA07] architecture. To our best knowledge, OAI is the only full x86-based SDR open-source solution, providing UE, eNodeB, and core-network functionality. A similar closed-source development commercialized by Amarisoft (LTE 100) which targets several USRP platforms provides eNodeB and core network functionality on standard Linux-based PCs. OAI is written in standard C language for several real-time Linux variants optimized for x86 and released as free software under the terms of version 3 of the GNU General Public License (GPLv3). OAI provides a rich development environment with a range of built-in tools, such as highly realistic emulation modes, soft monitoring and debugging tools, protocol analyzer, performance profiler, and configurable logging system for all layers and channels.

Currently, the OAI platform includes a full software implementation of the 4th generation mobile cellular systems compliant with 3GPP LTE standards written in C language under real-time Linux optimized for x86. At the Physical layer, OAI provides the following features:

• LTE release 10 compliant, with a subset of release 12; • FDD and TDD configurations in 5, 10, and 20 MHz bandwidth; • Transmission mode: 1 (SISO), and 2, 4, 5, and 6 (MIMO 2x2); • CQI/PMI reporting; • All DL channels are supported: PSS, SSS, PBCH, PCFICH, PHICH, PDCCH, PDSCH,

PMCH; • All UL channels are supported: PRACH, PUSCH, PUCCH, SRS, DRS; • HARQ support (uplink, UL and downlink, DL); • Highly optimized base band processing (including turbo decoder). • With AVX2 optimization, a full software solution would fit with an average of 1x86 core per

eNodeB instance (64QAM in downlink, 16QAM in uplink, 20MHz, SISO). For the E-UTRAN protocol stack, it provides:

• LTE release 10 compliant, with a subset of release 12; • Implements the MAC, RLC, PDCP and RRC layers; • Protocol service for all Rel 8 Channels and Rel 10 eMBMS (MCH, zCCH, MTCH); • Channel-aware proportional fair scheduling; • Fully reconfigurable protocol stack; • Integrity check and encryption using the AES and Sonw3G algorithms;



• Support of RRC measurement with measurement gap; • Standard S1AP and GTP-U interfaces to the Core Network; • IPv4 and IPv6 support.

Evolved Packet Core network features:

• MME, SGW, PGW and HSS implementations. OAI reuses standards compliant stacks of GTPv1u and GTPv2c application protocols from the open-source software implementation of EPC called nwEPC;

• NAS integrity and encryption using the AES and Snow3G algorithms; • UE procedures handling: attach, authentication, service access, radio bearer establishment; • Transparent access to the IP network (no external SGW nor PGW are necessary).

Configurable access point name (APN), IP range, DNS and E-RAB QoS; • IPv4 and IPv6 support.

Figure 3: OAI LTE Software Stack

Figure 3 shows a schematic of the implemented LTE protocol stack in OAI. OAI can be used in the context of a rich software development environment including Aeroflex-Geisler LEON /GRLIB, RTOS either RTAI or RT-PREEMPT, Linux, GNU, Wireshark, control and monitoring tools, message and time analyser, low-level logging system, traffic generator, profiling tools and soft scope. It also provides tools for protocol validation, performance evaluation and pre-deployment system test. Several interoperability tests have been successfully performed with commercial LTE-enabled mobile devices, namely Huawei E392, E398u-1, Bandrich 500 as well as with commercial 3rd party EPC prototypes. Table 4 lists several different OAI platform configurations involving support for commercial components to varying degrees.

Table 4: OAI configuration options

OAI configuration options UE eNodeB EPC

Case 1 OAI OAI OAI Case 2 OAI OAI Commercial Case 3

(under development) OAI Commercial OAI

Case 4 (under development)

OAI Commercial Commercial

Case 5 Commercial OAI OAI Case 6 Commercial OAI Commercial Case 7

(under development) Commercial Commercial OAI



3.1.2 Hardware Platform

For real-world experimentation and validation, the default software radio frontend for OAI is ExpressMIMO2 PCI Express (PCIe) board. This board features a LEON3 embedded system based on Spartan 6 LX150T FPGA as well as 4 high-quality RF chipsets from Lime Micro Systems (LMS6002), which are LTE-grade MIMO RF front-ends for small cell eNodeBs. It supports standalone operation at low-power levels (maximum 0 dBm transmit power per channel) simply by connecting an antenna to the board. External RF for high-power and TDD/FDD duplexing can be connected to ExpressMIMO2 depending on the deployment scenario. RF equipment can be configured for both TDD and FDD operations with channel bandwidth up to 20 MHz covering a very large part of the available RF spectrum (250 MHz-3.8 GHz) and a subset of LTE MIMO transmission modes. ExpressMIMO2 boards are reasonably-priced and completely open (GNU GPL), both at the hardware and software level. Figure 4 shows the ExpressMIMO2 hardware platform.

Figure 4: OAI ExpressMIMO2 hardware platform

The embedded software for the FPGA is booted via the PC or can reside entirely in the boot ROM which is part of the FPGA design. In the current design, the embedded software is dynamically booted by PCI Express under control of the PC device driver. The basic design does not include any on-FPGA signal processing and consumes approximately 10-15% of the FPGA resources. There is significant room left for additional processing on the FPGA, for instance Xilinx FFT processors to offload some processing from the host PC if required.

To enhance the current design in the FPGA, every newly added block should have an AHB bus interface. The LEON3 processor is easily configured to interact with every block connected to the AHB bus. The PCI Express controller block has been optimized in order to support the transfer of samples between the ExpressMIMO2 card and the PC at a rate that can go up to 30.72 Msamples/s in both directions, while using only a one-lane PCI Express interface. More details on the technical aspects of the ExpressMIMO can be found in [NN14].

Besides ExpressMIMO2, OAI now supports the UHD interface on recent USRP PC-hosted software radio platforms which are widely used in the research community. Specifically, Agilent China has recently succeeded in interconnecting the OAI softmodem software with a USRP B210 platform. This development is now delivered as part of the publicly-available software package from the OAI website and SVN server. EURECOM will continue to maintain this development and extend to X300 (USRP-Rio) family products. This achievement illustrates the validity of the standard PC plus generic SDR frontend approach taken in OAI since the code has been independently ported successfully on a totally different hardware target.

3.2 5G-EmPOWER testbed 3.2.1 Testbed Description

5G-EmPOWER is an open Mobile Network Operating System for SDN and Network Function Virtualization (NFV) research and experimentation in mobile networks. Its flexible architecture and the high-level programming APIs allow for fast prototyping of novel services and applications. 5G-EmPOWER blurs the line between radio and core network introducing the concept of Programmable Data Plane which abstracts the radio and packet processing resources available in a network and exposes them to network service developers via a high level intent driven framework application framework.



5G-EmPOWER natively supports multi-tenancy allowing Mobile Virtual Network Operators (MVNOs) to specify radio resource management policies (e.g. scheduling; modulation and coding scheme, MCS selection; or HARQ) separately for each slice while ensuring performance isolation between slices and efficient spectrum utilization.

The 5G-EmPOWER platform consists of three components:

• Data-plane, Multi-technology Programmable Data Plane: in 5G-EmPOWER everything is a Virtual Network Function (including radio access). The current supported technologies are 802.11a/b/g/n and LTE/LTE-A. Future technologies will include LPWAN technologies such as LORA and 802.11ah.

• Management and Service plane. Complex network services can be implemented by chaining different packet processing functions. A domain specific language can be used to specify custom packet processing function over a specific portion of the flow space.

• Software Development Kit (SDK). Data and management plane abstractions are exposed as high-level programming primitives by the 5G-EmPOWER SDK runtime.

3.2.2 Architecture, services and testbed resources

The 5G-EmPOWER architecture loosely follows the ETSI NFV MANO specifications. We name Programmable Data Plane, or PDP, the set of all virtualized network resources. The Wireless Termination Points, (RTs using the COHERENT terminology) are the physical points of attachment in the RAN (e.g. the Wi-Fi Access Points or the LTE eNodeBs). The Click Packet Processors, or CPPs, are Micro Servers combining forwarding with packet processing capabilities. The 5G-EmPOWER high-level architecture is depicted in Figure 5. In this figure the virtual base station represents an abstracted eNodeB.

Figure 5: The 5G-EmPOWER high-level architecture

Service Orchestration In 5G-EmPOWER the physical network is abstracted into network slices. Each slice defines a particular connectivity service and consists of a set of VNFs with the associated SDN control logic (see Figure 6). Each network slice can be fully customized to support the requirements of the service in terms of coverage, capacity, latency, and security.



Figure 6: Network slicing in 5G-EmPOWER

Testbed

5G-EmPOWER builds upon a single platform consisting of general purpose hardware (x86) and operating system (Linux) to deliver three types of virtualized network resources: forwarding nodes (OpenFlow switches), packet processing nodes (Micro Servers), and radio processing nodes (Wi-Fi Access Points or LTE eNodeBs).

The 5G-EmPOWER experimental facility consists of the following elements:

• 14 Wi-Fi Access Points running OpenWRT and implementing the 5G-EmPOWER programmable Wi-Fi data plane. These access points are deployed in a realistic office environment at CREATE-NET premises. Remote Access to COHERENT partners is provided through VPN;

• 6 Wi-Fi Access Points running OpenWRT and implementing the 5G-EmPOWER programmable Wi-Fi data plane. These access points are part of the mobile testbed which can be used to stage demo at conferences and project reviews.

• 6 LTE eNodeBs based on the USRP b210 SDR platform and running the OAI implementation of the LTE stack;

• 2 OpenFlow Switches with 48 Gigabit Ethernet ports and 4 10G uplink ports.; • 6 Mobile Edge Computing (MEC) Servers based on the Soekris 6501 platform (Intel Atom

CPU, 2GB RAM, variable storage) and equipped with 12 gigabit Ethernet ports; • 1 Blade server. This is used mainly as controller.

3.3 CommAgility LTE Testbed 3.3.1 Testbed description

The LTE radio testbed consists of two CommAgility eNodeBs, EPC, and a commercial UE. As UEs, commercial LTE dongles or smartphones will be used. In order to use fully LTE-compliant stacks at base station and UE sides, an EPC emulator tool will be used to emulate the LTE core network part entities MME, S-GW and protocols like S1AP, NAS. The integration with COHERENT C3 will be done by integrating the COHERENT RTC and non-RTC services at the CommAgility eNodeB.

3.3.2 eNodeB hardware and software description

Hardware Description

The CommAgility eNodeB will be based on the high-performance ARM and DSP based processing card AMC-K2L-RF2 that represents a complete baseband and RF small cell solution. The AMC-K2L-RF2 is designed to support wireless baseband processing and a 2x2 MIMO air interface in eNodeBs, for standard or specialised LTE and LTE-Advanced systems up to and beyond Release 10.



Figure 7: AMC-K2L-RF2 card

The main processor is the TCI6630K2L, part of Texas Instruments’ new KeyStone II generation of DSP/ARM SoCs. It includes four C66x DSP cores and two ARM® Cortex®-A15 MPCore™ processors, all operating at up to 1.2 GHz. The TCI6630K2L also contains wireless base station coprocessors which offload layer 1 and layer 2 processing demands to keep the cores free for receiver algorithms and other functions. A range of I/O is provided to the TCI6630K2L, including Gigabit Ethernet, PCIe and CPRI connectivity, giving connectivity to networks, host processors and additional RF. Using external cavity filters, the card can be directly deployed as an LTE small cell. eNodeB Software description

AMC-K2L-RF2 provides a comprehensive DSP Board Support Library along with a full SMP Linux implementation for the ARM cores. The physical layer software is highly integrated with the Texas Instruments TCI6630K2L SoC for maximum performance. Above the physical layer software, LTE protocol stack compliant with Small Cell Forum MAC-PHY API is fully integrated. For the testbed following eNodeB software architecture is used:

Processor 1 (DSP 1)Processor 1 (ARM)

L2 Layer

PDCP

RLC

MAC

L3 Layer

RRC

PDCP_TX

RLC_TX

MAC_TX MAC_RX

RLC_RX

PDCP_RX

Scheduler

S1-X2

X2-AP S1-AP

Provisioning Applications

RRM

OCM

UI (User Interface) Abstraction Layer

GTP

GTP-U

IPC

DSP 2 & DSP 3

LTE PHY

IPC

SCTP AL

IP-Sec

L2

L1

GTP AL

IP-Sec

L2

L1

COHERENT AgentOAM

Figure 8: eNodeB software architecture

The protocol stack and the physical layers will run on different processors. LTE control plane is on ARM ARM/Linux processor and LTE data plane in on DSP core 1 (SYSBIOS). This allows to free DSP core 1 from control mechanisms of LTE stack (like measurement, handover procedure and algorithms). Other DSP cores will run PHY.

Inter-process communication (IPC) between processors is achieved through use of TI Msgcom API in this case. The communication framework facilitates exchange by the inter component messages (ICMs) between the eNodeB stack components. Each ICM is a structured C type having a fixed header as the first field. COHERENT control services on the eNodeB will be implemented to run on the ARM processor.

3.3.3 IPC Mechanisms

In the project two different IPC mechanisms will be used:

If source component X and destination component Y run on the same processor but on different threads, message posting (see Figure 9) to destination component (non-blocking for source component) is used.).



Figure 9: Message posting

If source and destination components run on different processors (implies different threads), then the inter-processor message passing mechanism is used (see Figure 10). This mechanism would be dependent on the inter-processor communication method (DMA, Message queue with pull/push mechanism, etc.). Then the communication framework at the destination processor posts a message to destination component (non-blocking for source component). This communication is done using the board support package (BSP) for AMC-K2L-RF2.

Figure 10: Inter-processor message passing mechanism

3.4 UDE DAS PHY Testbed UDE implements Distributed Antenna System (DAS) functionality with at least 2 (2x2 MIMO) Remote Radio Heads (RRHs) based on LTE PHY, which includes identification of software PHY processing blocks needed to be modified to accommodate DAS and necessary testing.

3.4.1 Testbed description

Hardware description

• DAS PHY testbed consists of • DSP - IAF Dual 6670 AMC in a micro-TCA chassis, • Two MIMO RRHs from HHI, • Keysight’s MXA • LTE tester/s.

Figure 11: UDE - LTE + DAS PHY testbed



Figure 12: IAF Dual C6670 AMC module layout

3.4.2 Scenarios and Software description

Figure 13 provides an overview of the LTE PHY DL processing chain with DAS. The processing blocks that need DAS modifications will be analyzed and the required modifications in the blocks will be implemented.

Figure 13: DL PHY modules layout for DAS+LTE

DAS Engine is a processing block which deals with the type of DAS technique implemented. “Selection transmission” DAS technique is widely used in the literature. UDE targets to implement “Selection transmission” based on LTE in DL to achieve same resource allocation for all the UEs (min. 2) in DL transmission. This technique will be verified in the DL using this real-time testbed by observing the impact of distances and antenna placement. As UDE DAS PHY is a testbed focussed mainly on PHY layer implementation and procedures, a simple MAC emulator is used to send and receive PHY level configuration messages to the PHY. UDE targets to formulate the relevant MAC and higher layer procedures for the deployment of selection transmission.



3.5 CEP testbed for spectrum sharing: the CORE solution 3.5.1 Testbed Description

VTT’s software based spectrum sharing testbed is implemented with VTT’s Complex Event Processing (CEP) system (http://core.willab.fi/?q=node/66), namely CORE Trial environment, located in Converging Networks Laboratory (CNL). The CEP platform provides generalized interfaces to input data events, makes decisions based on them and outputs control and new data events for further processing. Control events are used to change configurations such as spectrum, power and other parameters of target systems.

The purpose of the trial environment is to allow researchers to build a new system by combining different systems together and carry out experiments on top of it using the CORE tools for data collection, decision making and adjustments. The experiments and system integrations can be realized by developing new CORE clients for interfacing the different systems for data collection and configuration. The system’s configurations can be changed based on different conditions by using the CORE cognitive engine and its decision-making system via a browser-based CORE trial user interface. The controlling decision rules can be written online.

The VTT CORE trial environment, illustrated in Figure 14, consists of components that provide data collection, decision making, and reconfiguration of the external systems as well as controlling of the trial environment itself.

Figure 14: The main components of the CORE trial environment.

The external system, which will be interfaced by a CORE client, could be a computer, service and network equipment, anything that can be monitored and optionally interfaced for configuration and control. A CORE client is developed to monitor and control the external system4. The CORE clients collect information from the external system, describe the system status using the data format supported by the cognitive engine and deliver it to the cognitive engine for decision-making. The cognitive engine processes each event and calculates the outcome using its decision-making algorithms as well as cached information. If a decision rule calculation results a positive outcome the cognitive engine is able to produce a new event that contains information or a configuration event. These events are routed to CORE clients based on their interest on different event types. The CORE cognitive manager keeps track of a group of core clients and transports information efficiently between core clients and the cognitive engine acting as a proxy which hides the network complexities introduced by the external systems.

4 http://core.willab.fi/sites/default/files/CORE-using_cognitive_engine.pdf

http://core.willab.fi/sites/default/files/CORE-using_cognitive_engine.pdf



Figure 15: Realization of the spectrum sharing testbed utilizing the CORE trial environment.

The realization of the spectrum sharing testbed (see more detailed information on the past Licensed Spectrum Access (LSA) work e.g. in [Palola], [Matinmikko] and [LSAVideo]) with the CORE trial environment is sketched in Figure 15. As shown in figure, the LSA repository, as an external system, is connected with the LSA controller by means of a dedicated CoreClient (the leftmost CoreClient). The purpose of this connection is to feed information on the incumbent spectrum usage to the LSA licensee and report back to the incumbent whether the spectrum was cleared after an evacuation request. The intelligent decision logic is implemented by an additional CoreClient (in the middle) and Cognitive engine combination, which form the LSA Controller entity. Note that this is not related to the C3 but it is a local control entity to the LSA/CORE solution. The rightmost CoreClient is responsible of collecting essential information from the network operating on the LSA band.

In the CORE trial environment, this CoreClient was used for monitoring the status of LSA base stations (on air, off air, starting, shutting down) and the current centre frequency. In addition to status reporting, the CoreClient commands base stations based on decisions made by the LSA Controller. In the CORE project tests, the used management system commands were Administrative lock/unlock, graceful shutdown, and change of a centre frequency, which each had their own internal CORE event. The LSA Controller can monitor and control the network through OAM or eNodeBs directly, and example realizations exist for products of a single vendor. Additionally, VTT’s testbed consists of network emulator/simulator behaving like a real OAM (with respect to LSA operation), and the emulator can be also utilized and further developed at initial development phases. The VTT’s spectrum sharing testbed has been integrated with the spectrum management repository provided by Fair Spectrum during past LSA testbed work.

LSARepository

LSAController

Incumbent manager

Network

• Web interface• Mobile application• Sensor

CoreClient CoreClientCoreClient

CognitiveEngine

Commands:• OAM• eNBs• Emulator/

Simulator

VTT’s spectrum management testbedFS tools/products



4. Design and Implementation of C3 This section presents the technical approach and COHERENT testbed architecture in relation to the design-level structure and organization for the entire COHERENT C3 system. The basic principles throughout the design of every aspect of the testbed are provided. Specifically, we analyze the following aspects with attention on implementation regarding the COHERENT KPIs, the software architecture and the network graphs Infrastructure.

4.1 KPIs COHERENT Key Performance Indicators (KPIs) act as quantification for the technical performance of the systems developed. To this end, a number of KPIs will be used to evaluate the performance of the corresponding components and functions tested, for every experiment and any functionality demonstrated. The KPIs that will be used to assess the efficiency of the integrated system are technology-specific KPIs but also KPIs defined by the project in order to evaluate the COHERENT system as a whole.

In Table 5 we present an example list with the important KPIs that will be part of the evaluation process. A mapping of these KPIs to the specific COHERENT use cases is presented in section 6. The requirements and KPIs for 4G and 5G systems are described in [Cacheda2007][NGMN][3gpp-rel14].

Table 5: Technical KPIs used for system evaluation

KPI ID

KPI Name Target Values

High-level Application level & System KPI

1.1 Ability for application to request bitrate on testbed Binary value = Yes 1.2 RAN scheduling on testbed Binary value = Yes 1.3 Handover functional to small cell or Wi-Fi load on

testbed Binary value = Yes

1.4 Run slices on top of testbeds. Measures the ability of the COHERENT framework to operate with concurrent vertical slices.

Binary value = Yes

1.5 Multi-domain network provisioning Binary value = Yes 1.6 RTC requirements satisfied. Related also to protocol

messaging exchange between RTC and C3. Binary value = Yes

1.7 integrated audio/video application level delay on testbed <150ms limit 400ms

RAN KPI

2.1 RAN latency, can be improved with better scheduling techniques

30-100 ms (depending on the channel quality and traffic load)

2.2 RAN jitter, delay variation introduced by the radio access networks, it will be useful when exploring scheduling techniques.

<10 ms

2.3 Ability to dynamically change MAC scheduling . Binary value = Yes

EPC KPI

3.1 Ability of EPC and scheduling to support a range of service profiles

Up to 5 different profiles

3.2 EPS Attach Success Rate (EASR), which can be used to determine how COHERENT can improve connectivity.

>99%

3.3 Dedicated EPS Bearer Creation Success Rate.. >90% 3.4 Dedicated Bearer Set-up Time by MME (DBSTM) < 1sec 3.5 Service Request Success Rate (SRSR), it determines the

success rate of the times that the UE goes from idle to connected (for instance after paging occasions)

>90%

3.6 Inter-RAT Handover Success Rate (IRATHOSR), to characterize the performance of seamless handover.

>90%

Traffic KPI

4.1 End To End delay at transport level (UE to S-GW) <150 ms 4.2 Jitter for Audio Channel < 5 ms 4.3 Packet Loss rate for audio channel (UE to server

endpoint) <0.8 %

4.4 Data rate: Network Layer 15 Mb/s (for 5MHz bandwidth, single user) fairness: 0.3-0.4



In the light of Network Slicing, the COHERENT C3 also aims to maximize the resource utilization efficiency. This will be achieved by provisioning resources from C3, from the other Slice providers so that they can serve the request using unused resources.

For the purpose of evaluation a set of observable system parameters need to be identified. In the case of LTE networks a number of parameters are considered, presented in Table 6.

Table 6: Parameters to consider for LTE evaluations

Type Parameters A. Packet-level parameters Packets are the data units that constitute the logical channels. Every packet has a set of pre-defined parameters.

• Average packet size (APS) • Packets inter-arrival time (PIT) • Packet arrival time (PAT) • Packet maximum-allowable delay (PMD) • Packet remaining time (PRT)

B. Logical channel-level parameters These parameters characterize the logical channel/service of a user.

• Number of protocol data units (NPDU) • Head-of-line delay (HOD) • Buffer size of a logical channel (BS) • Guaranteed bit-rate (GBR) • Level of traffic (TL)

C. User-level parameters The parameters associated with user • Total buffer size (TBS)

• Number of logical channels (NLC) • Channel quality indicator (CQI) • Subscription type (ST)

D. System-level parameters These are the global parameters of a system.

• Number of active users (NU) • Maximum allowed scheduled users (MAU) • Frame configuration type (CT) • Transmission time interval (TTI) • Minimum resource allocation unit (MRU)

These are very important for examining the LTE eNodeB operation when designing for example RAN sharing and spectrum sharing policies with the ability to support the Network Slicing concept, where multiple providers utilize the same underlay network infrastructure.

4.2 C3 Software Architecture The entire available tested infrastructure will be used to support the operation of the COHERENT Control Plane, realized by the C3 software component. The integration approach we consider allows each testbed to utilize its own control and data plane infrastructures, as long as they are compliant with the KPIs specified for the project needs towards 5G. C3 will be the set of methods from the control planes of all testbed exposing a northbound API, while the exposed API will depend on the targeted application.

Following the component-based software engineering approach in the COHERENT software design, all system processes are placed into separate components. Inside each component, all of the data and functions are semantically related.



Figure 16: COHERENT Testbed

Figure 16, a visual representation of the approach is presented. The figure also illustrates the relationship between the components of testbed-specific control and data planes with the COHERENT architectural components.

In the data plane and the related technologies, the project focus is on heterogeneous RAT. The environment we study is mainly related to LTE and Wi-Fi technologies, while we also consider for application of techniques on spectrum management and spectrum sharing that also consider for WiMAX. Two LTE testbeds will be exploited namely the OAI and CMA, while when these two testbeds are not used, LTE experimentation will be performed exploiting commercial LTE solutions.

Core Software Components in the Control Plane:

• OAI Control Plane: the control plane for the OAI infrastructure. Flex-RAN with different functional splits will be demonstrated alongside enhanced programmability features. The OAI control plane and the Flex-RAN protocol support for both real-time and non-real-time applications. This is the control plane exploited by the Eurecom’s OAI testbed.

• 5G-EmPOWER Control Plane: used for the programmability of both Wi-Fi and LTE networks. The CMA control plane will also exploit the EmPOWER control plane to expose functionalities to C3. This is the control plane exploited by the 5G-EmPOWER testbed.

• CMA Control Plane: is used for the control of the LTE network. UDE DAS exploits the CMA control plane in order to perform distributed antennas scenarios.

• Software Toolset and functions: Includes software tools and functions that are open for exploitation from both the OAI and 5G-EmPOWER control sub-systems, which can also communicate with multiple radio technologies in heterogeneous radio environment: o a CEP software solution provided by VTT mainly used for spectrum sharing related

information and event handling.



o a network information function provided by the SICS testbed which runs on an eNodeB/AP and implements a local estimator of attainable throughput. The output of the NIF can be used as input to network graphs.

o a software testbed solution provided by SICS, used for data modelling and information gathering.

• Network Graphs Infrastructure: used for any type of information generated from any test-bed and using any available data gathering tool. The network graphs infrastructure will be built in the context of WP6 activities with interaction with all the technical WPs (WP3, WP4 and WP5).

Below we highlight the core concepts around the integrated software solutions:

• Seamless interaction is achieved via the Network Graphs Infrastructure and C3. All information generated by the system must be available to every component through the Network Graph API exposed through C3 to each testbed.

• The C3 northbound interface exposes the functionalities provided by the LTE OAI control plane and the LTE-Wi-Fi interfaces provided by 5G-EmPOWER control plane, using Northbound Abstractions. This integration step will occur through suitable use case execution in order to precisely identify the necessary method calls.

• In the case of the OAI testbed, the OAI control plane exposes both the RTC and non-RTC functionalities. Note that depending on the function required, specific software elements are utilized in an OAI controller or/and the OAI agent located at the eNodeB.

• The COHERENT testbed integration will follow an incremental approach, with software components, major functions and features added progressively to the test scenarios. Suitable software stubs, emulating functions and interfaces of specific modules or devices, will be developed when needed. This allows for simplified will allow to a) simplify the integration and testing of process for different subsets of the entire system, and b) validate the software performance on additional and more complex physical topologies.

The main aspects that will be evaluated during the software integration process are the following:

• Compliance of the messages exchanged at the external interfaces of the software modules with the specified protocols and message formats.

• APIs between user plane and local view controllers, e.g., OAI agent and OAI controller (Flex protocol; see also section 4.2.1 for more details for the OAI agent –controller architecture).

• APIs between OAI, 5G-EmPOWER, and the other testbeds and with C3 (“protocol mappers”). For example, in the case of OAI a new API must be designed and implemented in order to facilitate RTC and non-RTC functionalities and expose them through the integrated C3 control plane. A new API was designed and implemented for the OAI agent controller communication; see [Foukas16].

• APIs between C3 controller and Application Layer (e.g., Network Slices).

The results will be used for improvements at the software coding level (SDK development) and for architectural considerations in terms of adopted protocols and deployment aspects. The main software elements provided by the integrated C3 are related to function and service selection and interaction with testbed-specific control planes, routing functions for multi-RAT environments, data control and interaction with the Network Graphs Infrastructure and topology management.

Note that a programmable RAN underlay is absolutely essential in order to facilitate the scheduling and resource allocation procedures and for the proliferation of functionalities that can be exposed to the integrated Control Plane.

For the case of LTE, note that in the eNodeB real-time constraints impose extreme challenges since towards 5G communications, system responsiveness must be in ms level; available southbound APIs and protocols (e.g., NETCONF, OpenFlow, REST, etc.) will not work in that case (see [He15] for an analysis on SDN latency). A possible interplay between C3 and the RT also depends on the design we consider (e.g. a cloud-RAN design with functional splitting, with remote virtual RRH elements). Lower level time-critical protocols are used in the PDCP, RLC and MAC layers, while higher layer



protocols (RRC, X2, S1-AP) mainly deal with mobility management, security and non-time-critical resource management operations.

Furthermore, the integration of the CEP platform for spectrum sharing is also challenging. The CEP testbed in practice provides a way to introduce intelligent logic that decides if e.g. some of the base stations should be powered off based on the network (base stations and incumbent information) parameters. Then the testbed forwards this configuration change to be performed within the network (i.e. to C3). It is also related to non-RTC type of control and can be also utilized by any testbed (e.g. OAI, 5G-EmPower) that supports the relevant interfaces.

4.3 COHERENT Network Graphs and Integration to C3 One of the main innovations in COHERENT is the way to aggregate abstracted information from radio network entities and to represent it as different types of annotated network graphs. Information stored in the network graphs is exposed to the C3 control layers for high-level resource allocation and spectrum sharing. Another source of information is related to the control plane internal state (configuration and operational data). A detailed description of the Network Graphs and the information stored there is made in the context of the WP3 activities and presented in deliverable [COHERENT-D3.1]. For example, information related to the way user and base station transmissions affect neighbouring transmissions are stored in an Interference Graph, while the base stations deployment and user locations are stored in a Topology Graph. Due to the very high level of LTE system complexity, these graphs are then used to facilitate efficient resource allocation, devise algorithms in support of the control resilience or for making handover decisions and promote agility in decision making by the RTC/C3.

Figure 17: C3 and supporting database

While there are advantages and disadvantages to each database platform, we stress the importance of selecting a system that best meets the project needs. Either the design choice goes for SQL or noSQL, a specific interface will be provided for the interaction of the Network Graphs Database with the C3 control plane. In our approach the RTs will talk with the RTC/C3 through southbound interface (SBi) and the C3 will talk with the Network Graph Database, as it is depicted in Figure 17. A direct access to the network elements will be also possible (without the interleaving of the C3), for actions that require fast responsiveness.



4.3.1 Design and Implementation of COHERENT network graphs

From an implementation point of view, the Network Graphs will be supported by a database infrastructure. Best practices in database design are adopted to meet the challenges imposed by the use cases which the COHERENT system aims to support.

The COHERENT Database Design Phases

• Requirements Analysis: Each use case in the COHERENT ecosystem generates a rich set of data. The requirements and the collection analysis phase can be different since different components of the system require for different approach and set different levels of accuracy and speed (e.g, real-time eNodeB operations vs Wi-Fi control plane operations or operations in the switch fabric). In principle and for all the types of activities we will perform, we consider the following sets of requirements:

o Data requirements: The data requirements are used as a source of database design. o Functional requirements. These are the functional requirements of the use case we

are interested each time. These consist of operations that will be applied to the database like retrievals and update. The functional requirements will be used as the main source of the COHERENT network graphs database software design.

• Conceptual Design: Once all requirements have been collected and analyzed, for each use case we create a high level conceptual data model. During or after the conceptual schema design, the basic data model operations are used to specify the high-level user operations identified during the functional analysis.

• Logical Design and Normalization: Normalization is a design approach that minimizes data redundancy and optimizes data structures by systematically and properly placing data elements into the appropriate groupings.

• Physical Design: The goal of this last phase of database design is to implement the database and define the database management system (DBMS) to be used.

Note that there are different types of databases which are categorized according to their functions. For our analysis we consider:

• Structured Query Language (SQL) databases • NoSQL or, relational databases and non-relational databases. • Graph databases

Their difference is about the way they are built, the type of information they store, and how they store it. Relational databases are structured, while non-relational databases are document-oriented and distributed. Open-source options for SQL databases include MySQL, PostgreSQL and SQLite, while for noSQL candidate solutions are MongoDB and Redis. While many documents describe the characteristics of each type, in COHERENT we are particularly interested in the noSQL type and the Graph Database type. While there is well-established documentation and several software solutions for the SQL and noSQL database types, Network Graphs databases are relatively new and in the following we provide a note for this type of database.

While traditional databases compute relationships expensively at query time, a graph database stores connections, readily available for any “join-like” navigation operation. Accessing those already persistent connections is an efficient, constant-time operation and allows to quickly traversing millions of connections per second per core. A graph database uses graph structures for semantic queries with nodes, edges and properties to represent and store data. Examples of Network Graph Database include Neo4J5 , AllegroGraph6 , ArangoDB7 , OrientDB8 , Titan9 , and Apache Giraph10 .

5 www.neo4j.com 6 franz.com/agraph/allegrograph/ 7 https://www.arangodb.com/ 8 orientdb.com 9 titan.thinkaurelius.com 10 giraph.apache.org



4.3.2 Extreme Complexity and Data Events in COHERENT

In the COHERENT project, we are targeting extremely complex environments with millions of network elements and billions of end users that are connected through multi-RAT infrastructures. For such types of environments, the runtime statistics for the heterogeneous RAN systems, while also for the end users, are exposed as network information. These can be represented in network graphs, while being exploited by various use cases and applications and can be used to define complex system relationships.

In this context, we consider two types of events that generate data that we save/retrieve in the COHERENT Network Graph Infrastructure. The Atomic events are related to atomic parameters (e.g. atomic parameter is the SINR of a specific link) and Complex events are derived from compositions of atomic events or other complex events.

• Atomic Events: An event that is generated corresponding to any change (i.e., create, delete or update) of an atomic parameter.

Complex Event: An event that is generated corresponding to any change of complex parameter. These changes are resulted from any changes of the input atomic or complex parameters of the corresponding function. The process of complex events could be treated as the way to abstract the network parameters provided by atomic events. Accordingly, the results of the complex events, namely complex parameters, become network abstractions for generic and RAT agnostic control, e.g. throughput per network slice or per allocated resources. In the following, we illustrate the use of network graphs in the case of multi-connectivity and SINR observations. In multi-connectivity conditions, UE can exploit resources from different eNodeBs for UL and DL transmissions. In this case, we establish network graphs of the underlying LTE RAN and formulate an optimization problem for throughput maximization running as a control application that utilizes the network graph as an input under multi-connectivity use-case. In Figure 18 a representation from the original network to Network Graph is depicted, that represents the physical connectivity formed at the abstraction layer by extracting the radio access layer parameters.

UE1

UE3

BS1

BS2

BS3

UE2

u1 u2 u3

b1 b2 b3

fb3,u3

fu3,b3

f b1,

u1

f u1,

b1

fu2,b1fb1,u2

f b3,u1

f u1, b2

f b2,u

1 f u1,b3

f u2,

b2

f b2,

u2

u1

Figure 18: From Original network to Network graph, that represents the physical connectivity formed at the abstraction layer by extracting the radio access layer parameters.

A set of atomic events essentially handle the corresponding parameters. In the multi-connectivity use case, we are interested in the following parameters of interest depicted in Table 7 : Table 7: LTE parameters for multi-connectivity

Parameter Layer Direction Reference Signal Received Power L1/L3 DL/UL Total number of PRBs per basestation L1/L2 DL/UL Maximum number of PRBs of UE L2 DL/UL Bandwidth per PRB L1 DL/UL Thermal noise power L1 DL/UL UE Aggregated maximum bit rate L3 DL/UL Non-GBR bearers number L3 DL/UL



A visualization of the corresponding Network Graph is presented in Figure 19.

Figure 19: SINR (in dB) of network graphs from underlying network.

4.3.3 Implementation of COHERENT Network Graphs Infrastructure

Our high-level goal is to provide a database infrastructure that seamlessly supports multi-RAT. In this case, schema dependencies in traditional SQL-based RDBMS will be a major source of constrains. For the database infrastructure that will support the COHERENT network graphs, we will exploit a hybrid solution with noSQL and Graph databases. For the noSQL database solution the main features we are interested in are the following:

• Leverage on non-relational and schema-less data models • Low latency and high performance • Scalability

Relationships are very important in a graph database, which is not always the case in other database management systems. In our initial proposal we consider the following two options:

• Option 1: OrientDB it is a high-performance, operational database. OrientDB incorporates a search engine, Key-Value and Object-Oriented concepts along with a reactive model (with Live Queries) and geospatial awareness, making it the first and only true Multi-Model Database

• Option 2: MongoDB (noSQL) together with Neo4J (Graph Database): Neo4j implements the Property Graph Model efficiently down to the storage level. As opposed to graph processing or in-memory libraries, Neo4j provides full database characteristics including ACID transaction compliance, cluster support, and runtime failover, making it suitable to use graph data in production scenarios.

Although there are benchmarking tests available that compare between OrientDB and Neo4j (while also between other network graph databases and noSQL schemes), the following period extensive tests over the COHERENT use cases, will drive the actual implementation of the network graph databases.



5. Extensions on existing Testbeds towards Integration In this section we describe the extensions planned towards integration on per testbed basis. We also describe a number of integration challenges.

5.1 Testbed Extensions 5.1.1 OAI Testbed: Extensions and new APIs

In the context of COHERENT, the OAI testbed described in section 3.1, will be extended with the integrated network programmability capabilities that are realized by an SDN-based agent-controller solution that allows new applications and services to be deployed as network applications. A design of the OAI SDN-based agent-controller is depicted in Figure 20. The OAI controller will operate as a plugin inside the C3, to offer the required services through the C3 Northbound interface (NBi).

Message Handler Control and Mnitoring

Control Delegation Policy managerEvent Manager

Communication APIAgent

Message Handler

Task Manager

Radio Info Base

Event Manager

North APIController

Communication API

Soft-Real Time Application (e.g., Mobility Manager)

C3 Control Plane

Prog

ram

mab

le L

TE e

NO

DEBs

VNFs

FlexRAN Protocol

VNFs

• Radio Link Control (RLC) • Packet Data Convergence Protocol (PDCP)• Radio Resource Control (RRC)• GPRS Tunneling Protocol (GTP)• ...

VNFs Related to

PHY Control Module

MAC Control Module

RRC Control Module

PDCP Control Module

...

VNFs VNFs ...

Hard Real Time Application (e.g., UE Scheduling)

RAN API

North API

API

Figure 20: Enabling programmability in the OAI eNodeB using SDN-based agent-controller

solution.

On the agent side who is actually performing the operations instructed by the RTC components, the OAI solution provides a control abstraction that can be used to support mobility management schemes (in T3.3 activities), spectrum sharing (T4.2) in the heterogeneous mobile network (HMN), and RAN sharing (T5.3). For example, in order to support the RAN sharing scenario a three-layer scheduler implementation (MAC-layer scheduling) proposed in [AB14] will be utilized and extended to provide a more dynamic approach to satisfy the operator-driven, technology-driven and service-driven requirements through reconfigurable APIs. This design will allow the provisioning of the required QoS to every client, while differentiating services and prioritizing traffic between MVNOs. In our design, programmability of the scheduler will be supported by a real-time agent function. The framework is applied to downlink but it can be easily extended to uplink. Another dimension of the extensions required for COHERENT is the design and implementation of new APIs for the interaction of the OAI system with the C3 and the Network Graphs Infrastructure. Interaction with the 5G-EmPOWER system through C3 is also investigated in the context of the project, while also extensions for new measurements, exposed from the agent to the C3 control plane. Further explanations on the extensions are provided in Section 6 under the RAN Sharing Use Case.



5.1.2 5G-EmPower Testbed: Extension and new API

The 5G-EmPOWER testbed described in section 3.2, already fully supports Wi-Fi based radio access networks as well as OpenFlow backhauls. 5G-EmPOWER also already provides preliminary support for LTE small cells. This includes Open Small Cells based on software tools, such as the OAI and the srsLTE platform. Moreover, support for CMA commercial small cells is currently being added. The architecture of the 5G-EmPOWER is depicted in Fig. 21.

Figure 21: Coordinating heterogeneous radio access networks using the 5G-EmPOWER Mobile

Network Operating System.

The 5G-EmPOWER platform provides a set of abstractions for network programmability that will be used to support load balancing schemes and joint mobility management and multicast rate adaptation schemes (WP3), spectrum management and sharing (WP4), and RAN sharing (WP5). The envisioned applications will span both the Wi-Fi and the LTE segments of the network. Support for traffic steering over a packet, OpenFlow-controller backhaul is also provided trough an intent-driven interface. The 5G-EmPOWER platform and API will be extended to properly support the LTE-related use cases. This includes modifying the current 5G-EmPOWER southbound abstraction layer to accommodate the needs of LTE-based small cells; particular priority will be given to CMA small cells. Moreover we will also extend 5G-EmPOWER with new APIs for the interaction of the 5G-EmPOWER platform with the Network Graphs Infrastructure.

5.1.3 CommAgility (CMA) Testbed: Extensions and new APIs

To integrate CommAgility eNodeB with COHERENT C3, the existing testbed described in section 3.3, will be extended with implementation of COHERENT non-RTC and RTC components at the CommAgility eNodeB. The COHERENT Agent will be implemented with two threads. One thread will implement the message handling the loop for reception of messages from the other components of the protocol stack (e.g. CQI reports, RSRP, RSRQ) through inter-process communication (IPC), providing them to the agent installed and sending them in an aggregated form to the COHERENT C3 via socket. The second thread will implement the message handling loop for reception of messages from the COHERENT C3 over the socket, and passing them to the corresponding components of the protocol stack through IPC mechanism. The intended functionalities to be implemented include reporting cell and UE measurements to COHERENT C3 (e.g. RRC measurements) and performing a load-based handover based on the C3 decision.



5.1.4 CEP Testbed: Extensions and new APIs

The main goal of the testbed is to trial spectrum sharing with the COHERENT architecture in a laboratory setting, potentially involving live trials in later development phases. The VTT’s CEP testbed has been integrated with the spectrum management repository of Fair Spectrum during previous LSA testbed work. The testbed consists of emulated core network, partially emulated RAN, and a real eNodeB. The testbed can be used with a commercial LTE device such as a smartphone with a real SIM card. In practice, in order to integrate the spectrum management testbed (used for spectrum sharing applications in COHERENT architecture) to the remaining COHERENT architecture implementation, a new implementation of CoreClient (i.e. adaptor) is required in order to talk with the respective interface of COHERENT architecture realization. Most natural way is to build CoreClient on top of HTTP, and information represented using a specified JSON structure. In addition, the interface controlling and monitoring the spectrum related parameters in the network, as well as the underlying functionalities in the COHERENT architecture need to be implemented. The information which the C3 API should provide to the CEP testbed, based on the previous experience, includes at minimum:

• Base station(s) geographical location (latitude, longitude, height, …) • Sectors and their directions • Transmitting power • Centre frequency • Air interface status (on air, off air, starting, shutting down)

The configurations which command the C3 API should be exposed: • Ability to turn on and off the base station air interface (administrative unlock/lock and

graceful shutdown - the latter is optional) • Means to change the base station centre frequency (optional)

Still, several options exist for realizing the COHERENT controlled LTE network, in spectrum sharing demo/tests :

• Alternative 1: A real LTE network from a project partner (adapter to OAI should be implemented, spectrum sharing application (the testbed) talks through OAI)

• Alternative 2: In phase 1, an emulator/simulator behaving the same way like a real OAM would do (through OAI or directly) directly (if appropriate scenario is being defined for available frequencies, i.e. 2,6 GHz in Oulu).

5.1.5 UDE DAS PHY Testbed

At UDE the available LTE testbed described in Section 3.5 is used, which consists of a DSP based hardware (HW) platform and the corresponding LTE eNodeB PHY software (SW). This HW and SW versions are predecessors of the current CommAgility LTE product platform (HW/SW). As DAS is purely a PHY level functionality, the major technical focus would be at the PHY level. The LTE based PHY level DAS functionality can be verified at the UDE site using a very simple MAC emulator and PHY level LTE analyzer.

The initial targets are: identification of modifications required in the DL standard LTE PHY layer for the deployment of selection transmission DAS technique, implementation and verification of the modifications on UDE DAS PHY testbed. Various measurement trials will be performed to understand the impact of DAS at different UE locations in order to understand the requirements of power, distances, interference, per-user throughput, spectral efficiency and resource allocation. The measurements will start with Single-Input, Single-Output (SISO) configurations at least for 2 UE DL transmissions. Based on the feasibility of software changes, the implementation and verification will proceed towards MIMO DL transmissions to 2 UEs.

Furthermore, UDE will describe the DAS relevant MAC and higher layer procedures. For efficient control and coordination of MAC and PHY resources in DAS by C3, a consolidated list of parameters for the formulation of abstracted network graphs will be provided



5.2 Integration Issues and Challenges A number of challenges must be addressed towards building the integrated COHERENT solution. With respect to the Network Slicing concept and considering that the physical infrastructure is shared among several operators, one operator cannot interfere with actions from another operator using the same physical infrastructure. In order to address this challenge, C3 mechanisms must automate and guarantee the following aspects:

• Isolation between different virtual infrastructure operators (i.e. MVNOs) • Privacy at the virtualization level in both terms of operation/control and management

operations; • Consistency / integrity of the network at any given instant (e.g. several virtual network

operators may lead to inconsistencies of the network due to malicious configurations by one of them).

Please note that the aforementioned challenges, with the addition of security, are even more important when the MVNO is running applications for public safety scenarios, like in use case 4. MV “Enhanced Mobile Virtual Network Operator (MVNO) for PMR services” in [D2.1]. In those applications, the system must guarantee the possibility to enforce strict priority policies among the services of the same MVNO and between services of different MVNOs. In the COHERENT integrated testbed, all these aspects must be guaranteed without affecting performance of the provisioning and control mechanisms in C3. Another dimension is related to the propagation delay between the control plane (CP) and data plane (DP) of the C3. Note that C3 is a logically-centralized physically-distributed entity. In that sense scalability constraints must be considered. These are associated with the C3 centralized nature and the need for collection and management of control information, needed to efficiently and optimally support the service requests. Furthermore, future heterogeneous 5G wireless networks with multi-RAT (e.g. Wi-Fi, LTE) must be able to meet the following challenges:

• What happens when users switch networks in mid-sessions? • How the selection of wireless interfaces is performed? • How to perform seamless handoffs? • How to automatically authenticate for seamless handoffs? • How to interact with the mobile core for policies and charging?

See also [Chowdury09] for a detailed analysis of the main challenges associated to network virtualization like signalling and bootstrapping, resource and topology discovery, resource allocation issues, mobility management, monitoring, configuration, and failure handling or security and privacy. We highlight that a number of challenges are associated with the complex LTE control architectures in order to allow seamless inter-domain network functionality. The basic challenges that we consider during the COHERENT integration:

• Network reliability: An issue that might be encountered during the integration is the robustness of the components.

• Software level integration: type of programming languages, software architecture, software maintainability and management of versions, etc.

• Dependencies: The COHERENT solution is based on the integration of next generation wireless access network technologies. These technologies involve external component dependencies which need to be available for the integration phase and participate in the integrated testbed.

• Management challenges: Although every distinct domain may use a solid network management framework, the necessary integration mechanisms and the necessary protocols must be used in harmony to provide the COHERENT integrated management solution.

Technical challenges: Regarding the massive MIMO functionality described in DoA, UDE will demonstrate the massive MIMO concept using Distributed Antennas approach in the UDE DAS PHY testbed with at most 2 MIMO RRHs. UDE will be able to demonstrate the DAS at PHY layer using the UDE DAS PHY testbed, as opposed to initial plan of having DAS demonstrated on CommAgility Testbed. The UDE PHY implementation of DAS cannot be ported to the CommAgility Testbed due



to significant changes in HW/SW architecture. UDE intends to demonstrate DAS using distributed RRHs connected to the UDE DAS PHY testbed via CPRI links. This is supported in the older generations of Texas Instruments SoC, while the CommAgility testbed is based on the latest generation of TI SoC where the antenna interface (in HW and SW) was significantly changed. It is no more compliant with the antenna interface of the older generation of Texas Instruments SoCs used by UDE.

Concluding Remark The final COHERENT testbed will serve as the foundation for part of research efforts of the project and also of possible future projects. Convergence in the control plane will be achieved using the C3 architecture and the components specified in D2.2. Together with testbed and technology specific enhancements made in the user plane, a rich set of use cases towards 5G communications will be supported. Proof of Concept (PoC) prototyping will enable the key technologies of the proposed COHERENT solution and its sub-elements to be tested for viability and performance. In the following section, we provide technical specifications of the use cases that will be used to demonstrate the efficiency of the proposed solution.



6. Design, Setup and Execution of Proof of Concept Demonstrations 6.1 Validation scenarios Specifications As mentioned earlier, the project will employ several use cases carry out the assessment of the COHERENT concepts. Furthermore, software prototyping and hardware installation and configuration together with the execution of test cases are required to determine that:

• The appropriate design directions have been selected during the project. • Technical aspects of the development have been successfully addressed. • Technology integration is within the proposed envelope. • The design requirements are successfully met and demonstrated through PoCs.

In the following we describe the scenarios that will be considered in system integration. COHERENT Validation Scenarios categorization. In [COHERENT-D2.1] a detailed view of COHERENT’s reference scenarios were presented. We summarize these scenarios in Table 8. Every scenario includes one or more supporting use cases.

Table 8: Scenarios specification according to D2.1

Scenario Use Cases Scenario 1

Network co-operation and inter-operability

• use case 1. RS: RAN sharing among heterogeneous mobile networks • use case 1.SR: Supporting RAT sharing • use case 1.CO: Cooperation among multi-operators

Scenario 2

Spectrum management

• use case 2. MM: Massive MIMO / distributed antenna system in dense small cell deployments

• use case 2. FSA: Flexible spectrum access Scenario 3 Critical

communications • use case 3. MN: Coordination of rapidly deployable mesh networks • use case 3. FS: Flexible resource sharing for broadband PMR networks • use case 3. CE: Coverage extension and support of out-of-coverage

communications (D2D communications) Scenario 4 Network Slicing • use case 4. MV: Enhanced MVNO for PMR services

• use case 4. eM: Dynamic eMBMS for public safety applications • use case 4.GD: User groups’ differentiation in multimedia service

provision • use case 4. SD: Service differentiation

Scenario 5 Massive IoT • use case 5.WoPL: Wireless over power line Scenario 6

Broadband communications in public transportation

• use case 6. AG: Air-to-ground communications • use case 6. MR: Delivery of services in public

A connection of these use cases was also presented in D2.2 where the way COHERENT architecture can support different use cases was described. In this section, we focus on three use cases that will serve as the main demonstrators, namely spectrum sharing and management, RAN sharing and load balancing. We focus on these demonstrators since they are directly linked to the work delivered in the technical WPs; network abstractions and load balancing in WP3, Spectrum Sharing mechanisms in WP4 and RAN Sharing algorithms in WP5. A detailed description is provided, where for each use case the template defined in Table 3 is used to provide a description of the use case.



6.2 The COHERENT use cases towards integration 6.2.1 Spectrum sharing use cases

Traditional spectrum allocations cannot guarantee the availability of wireless bandwidth for 5G communication systems and beyond, especially when applying network slicing for supporting multi-tenancy. Thus, there is a need for more dynamic spectrum sharing mechanism for LSA. In this use case, we demonstrate the feasibility of spectrum sharing in both homogenous and heterogeneous radio environment. Because of that this use case consists of two separate sub use cases:

Spectrum sharing with the COHERENT architecture: this sub use case operates with homogenous 5G environment controlled by COHERENT with network slicing for multi-tenancy in LTE networks.

Spectrum sharing between heterogeneous radio technologies: this sub use case operates in heterogeneous radio environment with heterogeneous radio technologies (i.e. WiMAX networks).

6.2.1.1 Spectrum Sharing use cases description

Use case :

Spectrum sharing with

the COHERENT

architecture

Figure 22: Realization of the spectrum sharing testbed

Incumbent

Incumbent tools

LSA Repository

LSA Controller

LicenseeLTE Network

Separation is possible in three dimensions:

Time

Frequency

Space

IncumbentLicensee Licensee

Sharedspectrum

C3



Figure 23: The spectrum operation of TD-LTE is managed by CEP Testbed.

The spectrum usage of the Incumbent user is managed by the Incumbent tools which may rely on informing incumbent or it may include a sensing function. The LTE mobile network of VTT with commercial components is Licensee. Spectrum sharing is implemented by FS LSA Controller and FS LSA Repository. The system demonstrates LSA spectrum sharing with the COHERENT architecture. The spectrum is sliced in a way that whenever the spectrum at a certain location is not in use by the incumbent, it can be utilized by MNO for mobile broadband. Automatic deployment and evacuation of such a secondary network is covered by this test case. Separation is possible in one (or more) of the three dimensions, i.e. in time, space and frequency domain.

Involved Partners

VTT and FS

Architecture Layers

- C3 - LTE control plane (specific for spectrum sharing functions) - LTE user plane - Spectrum sharing layer (LSA-like components)

Components - Spectrum sharing application (limited functionality based on LSA controller of FS) - Knowledge Base of the spectrum sharing application (based on FS spectrum sharing

repository and incumbent tools) - C3 with the developments for spectrums sharing and respective APIs from VTT - LTE network control plane from VTT - User devices accessing the LTE broadband service from VTT - Management tool for control and testing

Pre-Conditions

- Necessary functions for spectrum sharing developed in C3 (e.g. capability to switch on/off base stations and to query base station status information)

- Real or simulated LTE control plane available and integrated with C3 from VTT. A standard –commercial LTE control plane can control eNodeB directly or through network management system or its simulator.

- In a case of a real LTE network, the primary network that will be used in a case of shared network not being available should be defined and deployed.



Dependencies

- No dependencies with other use cases

Test Steps - The spectrum sharing application is frequently informed about eNodeBs operating on shared spectrum and about their state

- Spectrum band becomes available for eNodeBs, and this is reported to spectrum sharing application by the repository

- The spectrum sharing application commands C3 to switch on the air interfaces of the eNodeBs on the shared spectrum

- Some user(s) begin(s) to use the additional network capacity - Incumbent reclaims its spectrum back, and the spectrum sharing application is

informed accordingly - The spectrum sharing application commands C3 to switch off the affected eNodeBs - User(s) are moved back from the shared band eNodeBs

Expected Results

- Required integrations are done - eNodeB(s) on the shared spectrum will be switched on and off upon the incumbent

needs - Users can utilize the available extra bandwidth - Users are automatically moved to another network if eNodeBs on shared spectrum

become unavailable Post

Conditions - After the round of this test case, the shared spectrum should be available for

incumbents own use - Possible users should be connected with some other network providing the basic

connectivity KPIs - Time it takes to turn on an eNodeB on shared spectrum upon request

- Time it takes to switch off an eNodeB on shared spectrum upon request. Delay of a handover between an eNodeB on shared spectrum an eNodeB on exclusive spectrum band

The above use case deals with spectrum sharing in 5G environments, while following use cases will extend the one above in order address heterogeneous network environment. In tasks T4.1 and 4.2 we have proposed to analyse various coexistence scenarios. The following use case demonstrates spectrum sharing aspects in heterogeneous network environments.



Use case : Spectrum

sharing between heterogeneous

radio technologies

An important aspect which needs to be addressed when sharing spectrum is how to deal with legacy radio technologies. Legacy hardware may expose only limited functionality making it difficult to control in order to facilitate spectrum sharing. In this use case we demonstrate limitations and feasibility of spectrum sharing schemes when dealing with legacy hardware.

For this use case we exploit two commercial networks:

• INEA WiMAX and microwave network (located in Poland) • OTE TD-LTE network (located in Greece)

Because of the geographical distance between these two networks the use case in fact consists of two parts: one for WiMAX network and another for the TD-LTE network.

Figure 24: The spectrum operation of two commercial networks (WiMAX and TD-LTE) is managed by CEP Testbed with FS module.

Involved Partners

INEA, FS and PUT

Architecture Layers

- Management plane.

Components - INEA WiMAX network on 3.4-3.8 GHz - INEA microwave links on 3.4-3.8 GHz. - OTE TD-LTE network on 3.4-3.6 GHz - OTE TD-LTE network on 3.4-3.6 GHz - by CEP Testbed (FS module).

Pre-Conditions - INEA WiMAX network is controlled by CEP Testbed (FS module)

- 3.5 links are controlled by CEP Testbed (FS module). - Sharing agreement is executed between TD-LTE network 1 and 2 - OTE TD-LTE network 1 reports its status to FS spectrum sharing application

through NETCONF protocol. - OTE TD-LTE network 1 is controlled by FS spectrum sharing application. - OTE TD-LTE network 2 reports its status to FS spectrum sharing application. - OTE TD-LTE network 2 is controlled by FS spectrum sharing application.



Dependencies - No dependencies with other use cases.

Test Steps WiMAX network: - WiMAX and microwave link operators make an agreement on joint use: time of

day when WiMAX has more capacity than microwave links and vice versa, based on usage patterns

- Both networks are activated. - Capacity is reallocated according to the agreement. - The agreement parameters are changed.

TD-LTE network: - OTE TD-LTE network 1 operates normally. - OTE TD-LTE network 2 is activated. - OTE TD-LTE network 1 adapts to spectrum sharing. - OTE TD-LTE network 1 is deactivated. - OTE TD-LTE network 2 operates normally. - OTE TD-LTE network 1 is activated. - OTE TD-LTE network 2 adapts to spectrum sharing.

Expected

Results - Controlled interference between legacy systems

WiMAX network: - Resource reallocation based on the demand and usage patterns.

TD-LTE network: - RSSI levels of TD-LTE network 1 and 2 are on same level in the cases when

network 1 is activated first and last.

Post Conditions - All the networks return to the normal state, i.e. WiMAX network operates without sharing, and TD-LTE networks operate according to the sharing agreement.

KPIs WiMAX network: - Interference experienced by the WiMAX network. - Interference experienced by Microwave links. - Integrated RSSI of the WiMAX network during sharing. - Integrated RSSI of microwave links during sharing.

TD-LTE network: - Interference experienced by TD-LTE network 1. - Interference experienced by TD-LTE network 2. - Integrated RSSI of TD-LTE network 1. - Integrated RSSI of TD-LTE network 2. - In equal sharing agreement, RSSI and interference difference between cases when

network 1 is activated first and last.

6.2.1.2 Added value to the related work The spectrum sharing use cases demonstrate the COHERENT architecture and spectrum sharing application in three different sharing arrangements. VTT demonstration is located in Finland and it shows wireless cameras as primary user and mobile network as a secondary user. INEA demonstration is located in Poland, where residential WiMAX and corporate microwave links share a co-primary allocation. OTE demonstration is located in Greece and shows a co-primary allocation between two independent and non-synchronized mobile networks.

Use Case: Spectrum sharing with the COHERENT architecture



The VTT demonstration setup is a standard LSA system. The novelty is that the LSA Repository is for the first time connected to a commercial wireless camera incumbent management system (in the Netherlands), which also is new and unique.

Use Case: Spectrum sharing between heterogeneous radio technologies

WiMAX network: The WiMAX demonstration shows how a frequency band can be shared between corporate users, who require more capacity during the day time and residential users, who require more capacity outside office hours. The residential and corporate users are collocated on the same geographical area and the two radio systems are active simultaneously. However, both radio systems can be configured with various channel bandwidth. Therefore, LSA system can direct the two radio system to modify their channel bandwidth (allowing also some overlapping) and transmit power in order to shift capacity but at the same time maintaining limited interference levels. It is assumed that each radio system maintains a small amount of capacity at all times. When the communication systems become more dynamic in increasing and decreasing bandwidth it could monitor gradient increase in the demand for bandwidth in both systems. Then the setup can be used so that the small capacity (limited by legacy hardware configuration capabilities) is shifted between the radio systems when needed. In this setup, we use commercially available devices and we will also evaluate their delay to change channel bandwidth i.e. dynamics of such a setup.

TD-LTE network: The TD-LTE demonstration shows a situation of regional licenses. The same spectrum band is allocated to several operators or even to micro-operators, who deploy their networks regionally. The networks are not synchronized. At the borders of the network, there is a possibility for interference. In case of national level mobile operators, the borders can be negotiated, managed, and controlled by mutual agreements, but in the case of large number of small operators, mutual agreements become complicated and an external management system can utilize the spectrum more efficiently than with static licenses.

6.2.1.3 Technical approach In both use cases spectrum sharing job is carried by the CEP Testbed and in particular the FS module in order to demonstrate various flavors of spectrum sharing.

Use Case: Spectrum sharing with the COHERENT architecture

Spectrum sharing been demonstrated in the past, and the same approach can be used also in this context. This live LSA system demonstration was carried out in CORE+ project [CORE], in which LSA band evacuation times are measured. Spectrum use of commercial LTE system is managed at the 2.3-2.4 GHz band according to incumbent’s PMSE service activity. Demonstration shows that LSA band can be managed with existing network equipment with a minimum number of additional components, LSA Repository and LSA Controller.

The testbed is illustrated Figure 25. For clarity, the Complex Event Processing (CEP) occurs in LSA controller, similar to Figure 15.



The Internet

Base StationsVTT – Visualization VTT – LSA Controller

Fair Spectrum

LSA ControllerMNO User Interface

Network StatusVisualisation

eNB Controller

eNodeB

eNodeB

LSA Repository

LSA IncumbentManager

LSA Controller

Figure 25: The LSA Spectrum Sharing: CORE solution testbed

Results from CORE+ test:

We present the results gathered from CORE+ demonstration in here, as they are relevant to the work to be executed in COHERENT. For this demonstration, VTT provided LSA Controller, co-ordinated and presented LSA demos, took LSA measurements. WISE2 project provided LSA repository and Incumbent Manager. Centria handled configurations, operations, etc. of LTE terminals and networks. CWC calculated small and macro cell exclusion zones and implemented demo visualisation. Nokia provided LTE equipment, NetAct, and radio plans.

Demonstration procedure:

• LTE network evacuation: Incumbent defines a spectrum use and MNO frees the LSA spectrum by shutting down interfering TD-LTE sectors or base stations

• LTE network activation: if no incumbents in the licensed area -> the TD-LTE on LSA band is used for MNO.

• Emergency evacuation: LSA License is cancelled temporarily, all LSA base stations are shut down

Table 9 and Table 10 show the results for this demonstration.



Table 9: Time measured for different phases of the LSA band evacuation

Table 10: Total delay of research and commercial part of LSA trial

Use Case: Spectrum sharing between legacy radio technologies

The WiMAX subcase tests the coexistence of the existing WiMAX 802.16e working in the 3.4-3.8 GHz band together with microwave radio-links dedicated for corporate clients. The following diagram demonstrated coverage of INEA WiMAX network around Poznan area together with example locations of microwave links.

Figure 26: Coverage of INEA WiMAX network around Poznan area



The goal will be to utilize the daily traffic patterns of the WiMAX clients and offer the unused fragments of band to the business clients via radio-lines. The following diagram demonstrates changes of spectrum allocation depending on the traffic patterns.

Figure 27: Changes of spectrum allocation depending on the traffic patterns

The amount of spectrum occupied by WiMAX and the microwave radio-links as well as the transmit power will be managed by the CEP Testbed and FS databases using SNMP and CBRS-oriented protocols. This scheme can be interpreted as the co-primary sharing (if WiMAX and microwave links are treated with the same priority), where two incumbent operators coexist in the certain geographical area and share the spectrum but it rather should be treated as LSA sharing but the priorities between WiMAX and microwave links would change depending on the time of a day. This scenario will be further complicated by adding so called third tier users (as GAA - General Authorised Access, in SAS). The GAA users will implement cognitive radio in order to look for access opportunities when the priority WiMAX and microwave systems will not utilise the spectrum fully. These GAA third tier users will be provided by PUT and realized by the means of URSP, which either will sense the spectrum or will utilize the data stored in the database to decide if the spectrum is vacant or occupied. If the spectrum is vacant, it will start some data transmission in such a way that the primary users will not be affected.

From COHERENT project perspective, the motivation behind these demonstrations is the following: first, within tasks T4.1 and 4.2 it was proposed to analyze scenarios with coexistence of various heterogeneous radios, so such a demonstration will be a good match. Moreover, there will be an exercise to define the NBi, SBi and EWi messages. The functionality of C3 will be also realized although it will be rather limited due to functionality of the legacy hardware exposed through interfaces such as SNMP

Radio access equipment to be used in the WiMAX subcase:

WiMAX 802.16e production network consists of around 80 base stations and around 5000 CPEs. It operates in 3.6-3.8GHz band. The equipment is manufactured by Cambium Networks and consists of the following elements:

• PMP 320 Access Point



• PMP 320 CPE

• CMM4 (which provides GPS synchronisation signal and power supply for attached Access Points)

Microwave radio-links used in the demonstration utilise RW-2230-4H25 (RADWIN 2000 A-Plus Series) manufactured by RADWIN Ltd.

The GAA (third tier) user will be implemented using USRP - Universal Software Radio Peripheral platform with GNU-Radio Kit.

Spectrum Sharing Application Operations

During the operation of spectrum sharing application, it could coordinate policies in the network and insert them dynamically into the network based on the agreed scenario. Then based on the interference graph it could modify the parameters of base stations (such as TX Power, Channel Bandwidth) in order to minimize the interference.

Generic time plan – key phases:

1. Definition of the setup, including necessary interfaces, parameters, protocols etc.

2. The measurement of coexistence of WiMAX system and microwave radio-lines will be conducted.

3. Once the first initial test will be finished, the interfaces between the GAA device and the system will be defined. Also the transmit module based on USRP will be created.

4. Final field trial will be conducted.

TD-LTE network:

The OTE TD-LTE network consists of a full system that contains the UE, eNodeB and EPC. 4x4 MIMO is used with 30dBm per port and channel bandwidth of 10MHz with up to 64QAM modulation (MCS 8-28). The UE terminal is the Telrad 7000 that supports LAN ports, Wi-Fi, VoIP ports and web based configuration. It uses OFDMA in DL and single carrier FDMA in UL.

Configurations

In all these demonstrations, we use Protection Zone method for expressing the incumbent protection requirement.

Spectrum sharing with the COHERENT architecture use case:

The main configurations for this demonstration are:

– Protection Zone area, Protection Zone maximum interference, Protection Zone reference receiver and antenna parameters.

Spectrum sharing between legacy radio technologies use case:

The main configurations for the WiMAX network are:

– Minimum bandwidth, which is always available for microwave links and for WiMAX



- Times of change

– Protection Zone area, Protection Zone maximum interference, Protection Zone reference receiver and antenna parameters for microwave links and for WiMAX

The main configurations for LTE network demonstration are:

– Protection Zone area, Protection Zone maximum interference, Protection Zone reference receiver and antenna parameters for both mobile operators.

6.2.1.4 Expected results The main Key Performance Indicators in the spectrum sharing are related to the questions how much interference the parties experience due to sharing agreement and how efficiently the spectrum is used. In sharing agreements, interference and efficiency often work against each other: decreasing interference, decreases also efficiency and increased efficiency causes more interference. Due to this phenomenon, the incumbent and efficiency should be studied simultaneously. For interference the main studied factor is signal to interference and noise ratio (SINR), and for efficiency Received Signal Strength Indicator (RSSI).

6.2.1.5 Analysis and technical Impact All the demonstration related to spectrum sharing allow to verify feasibility of spectrum sharing in 5G environment also in a presence of legacy radio technologies. Such a broad application of spectrum sharing will demonstrate completeness of the CEP Testbed and FS module and prove that it is not only ready for research purposes but may also be implemented in commercial networks serving thousands of customers.

6.2.2 RAN Sharing and Network Slicing Use Case

6.2.2.1 RAN sharing use cases description In the evolved LTE, one observation that cannot be skipped is that Network Slicing is closely related to the concept of RAN Sharing. In principle, there exist different kinds of active infrastructure sharing depending on the degree of sharing. 3GPP has defined and ratified different kinds of architecture with varying degrees of sharing [3GPP-TS23.251]:

• Multi-Operator RAN (MORAN): only equipment is shared;

• Multi-Operator Core Network (MOCN): both spectrum and equipment are shared;

• Gateway Core Network (GWCN): both the RAN and some elements of the core network are shared.



Use Case: Service

Differentiation and

RAN Sharing

Different RAN sharing schemes and scenarios exist. Our focus stays on the way base station (eNodeB in LTE) resources are shared, like Resource Blocks (RB) in frequency/time/space domain.

Figure 28: RAN Sharing Use Case

For this use case, we exploit the OAI testbed extended with advanced programmability features and a new scheduling framework. This will offer the enabling technologies to the C3, in order to support complex network slicing concepts.

Figure 29: For this use case the testbed used is OAI. Both the RTC and non-RTC features are exploited.

Involved Partners

EUR, OTE, TSC

Architecture Layers

• C3 • LTE User Plane • Slice Layer

Components • OAI EPC

• OAI eNodeB

• OAI Agent

• OAI non-RTC and RTC functions



• OAI Flex protocol (for the agent –controller communication)

• New policy based Resource Block (RB) Scheduler

Pre-Conditions

• The physical LTE resources have an available management interface (remotely accessible) or a complete management system controlling them with an available remote interface.

• The LTE control plane is integrated to the C3 software framework.

• There is one instance of C3 deployed and controlling the whole infrastructure. C3 holds one management interface available.

• The C3 northbound and southbound interfaces are integrated and are functional.

• There are enough resources to serve at least two different MVNOs.

• A policy-based scheduler can operate in the OAI eNodeB

Dependencies • No dependencies with other use cases.

Test Steps • Each MVNO defines an service level agreement (SLA) agreement with the physical infrastructure provider by means of QoS guarantees for its users.

• The request is sent to C3 through the Northbound API for the provisioning of resources.

• The SLA request is sent to the OAI controller through the corresponding C3 service interface.

• The OAI Controller sends the right message to the OAI Agent that processes the request and triggers the planning process.

• If no Policy-based scheduling is in effect, the OAI controller-agent updates the scheduler.

• The planning phase is finished. Each configuration is stored in the Network Graphs database.

• The virtual resources provisioning or instantiation starts: the different virtual instances for each slice are created by the scheduler to allocate resources and the interfaces are attached to them.

• Operation: The virtual resources through feedback from the controller become available to the virtual operator.

• The virtual resources (RB allocations) are operated to validate the RAN Sharing process.

• The virtualized RAN infrastructures are finally decommissioned and all the policy-based scheduler may be removed.

Expected Results

• OAI – C3 successful integration

• Request processed at C3 and the OAI controller-agent.

• Activation and execution of procedures for provisioning of policy-based scheduling in the eNodeB.

• RAN sharing efficiency demonstration

Post Conditions

• MVNOs network operation with SLAs guarantees under multi-tenancy



KPIs The KPIs we investigate in this use case are related to the following KPIs categories as defined in Table 5:

• High-level Application level & System KPI o RAN scheduling in testbed. o Real-time operation of the SDN-enabled RT o QoS requirements per MVNO satisfied.

• RAN KPI o E.g., RTC requirements satisfied.

• Traffic KPI o E.g., latency.

6.2.2.2 Added value to the related work works The main objective of this use case is to analyze the necessary enhancements and protocol configurations available for an LTE eNodeB (release v12) that is able to consider for traffic characteristics and meet RAN sharing application requirements.

Some interesting works related to RAN sharing coming from the industrial research community are [kokku10,kokku13] and [Guo13]. They present different solutions for RAN sharing which will be used as a basis (together with other references and work in COHERENT Task 5.3 about RAN sharing) for deciding the next steps in the evolution of the COHERENT RAN sharing demonstrator. For this reason, they are briefly described in the following. For instance, [Kokku10] presents a solution for network slicing isolation coupled with the MAC schedulers of the base stations, while [Kokku13] presents a solution implemented at the base stations gateways, with minimal impact on the bases stations.

In [Kokku10], Network Virtualization Substrate is one of NEC's attempts to provide a virtualization substrate for WiMAX. It virtualizes the base station resources (UL and DL) into slices and allows for bandwidth-based and resource-based reservations, where slices can support multiple flows, each having its own QoS requirements. It operates at MAC-frame granularity by choosing the slices that maximize a utility function at each instant. flow scheduling is a separate aspect from the slice scheduling: each slice can manage its flows, but NVS provides a flow scheduling framework that provides the possibility to either choose a pre-programmed scheduler or use custom algorithms.

In [Kokku13] CellSlice is a system whose aim is to provide a solution for RAN sharing and focuses on the gateway level. The paper shows an implementation of RAN sharing for WiMAX, but the authors stress the technology agnostic nature of their solution and propose to apply it also in the context of LTE and LTE-A. CellSlice only needs from the BS the resource utilization and average per-flow MCS, and sets a single shaping parameter per flow: the maximum sustained rate. For the UL, it works by cleverly and progressively increasing this rate for each flow and by respecting its reserved rate. For DL, a shaper (utilization maximization) and a slice scheduler (intra-slice convergence, inter-slice isolation) are used. The shaper works by estimating the number of slots available at the BS (and sending the corresponding number of slots) and refining this estimate thanks to the resource slot utilization feedback. The slice scheduler will only send this estimated number of available slots to the BS, while satisfying the different reserved rates for each slice and implementing the specified flow scheduling policy within each slice.

In [Guo13] the partial resource reservation approach is straightforward: operators have a guaranteed minimum share of resource while the remaining part can be shared among all operators. It is an extension of NVS with the addition of a two-step Admission Control (AC) mechanism to guarantee QoS for active bearers. Each slice has a weight which is used at each sub-frame by the slice scheduler to determine which slice to select. An additional slice is added in order to support for partial resource reservation. If the shared slice is selected, the MAC scheduler considers all active bearers without regard to slice priority. To admit a new bearer request, the two-step AC will first verify if the addition



of the load (or estimated load) of the new bearer to the reserved part does not exceed a threshold value. If it does, it then verifies the same thing but for the shared part, with another threshold.

In COHERENT we exploit state of the art approached like the ones described, however for the RAN sharing problem our technical analysis considers programmable RAN technologies using the SDN design paradigm on the OAI eNodeB. In addition, a new three tier architecture is proposed in order to facilitate the per MVNO/Slice service provisioning by means of resource block (RB) allocation in the frequency and time domains. Using the C3 approach, the data plane is decoupled from the control plane and an RTC controller is communicating with a local agent residing in the base station. Real time decisions will be made regarding a) Policy enforcement on per MVNO basis and b) MAC scheduling and RB allocation to facilitate real time prioritization.

The KPI of interests are jitter, latency, data rate, and loss rate as well as fairness. Our work is related to the design and implementation of a framework that is able to effectively decouple the policy enforcement from the RBs scheduling of multiple MVNOs/slices. Moreover, QoS objectives, isolation between MVNOs and isolation guarantees between the different slices and type of traffic are considered. To the best of our knowledge this is the first open-source solution available that enables for a programmable RAN sharing framework.

6.2.2.3 Technical approach The following demonstration is performed using the OAI platform. In summary, the following approach is applied:

• Use SDN principles to address the challenges in the RAN

• Separation of the control and the data plane

• Centralized view of the RAN for improved scheduling decisions.

• Re-programmability of the data plane on the fly by the centralized controller

The controller can be connected to a number of eNodeBs whose states will be able to directly control and modify. A standard protocol is required for communication between the local Agent deployed in the eNodeB and the Controller. For the SDN-enabled eNodeB programmability approach we will exploit the FlexProtocol [Foukas2016] developed in the context of the COHERENT project.

In order to support the RAN sharing concept, we exploit a three-layer scheduler implementation (MAC-layer scheduling) to provide a more dynamic approach to satisfy the operator-driven, technology-driven and service-driven requirements through reconfigurable APIs. The framework is applied to downlink but we will extend it also for the uplink. This design will allow the provisioning of the required QoS to every client, while differentiating services and prioritizing traffic between MVNO users and other stakeholders. This RAN sharing framework can be also used to enable the concept of Network Slices.



Figure 30: RAN Sharing OAI Scheduling for RTC

In the devised approach presented in Figure 30 the OAI scheduling component has been extended in order to support multiple Network Slices (e.g. on per MVNO basis) while the scheduling decisions can be either Service Driven or User Driven:

• Service Driven: Allocation based on service prioritization, where service-level scheduling is done based on logical channel priority. At the end of this stage, total number of resources of every service is calculated.

• User Driven: Allocation based on dynamic selection of scheduler policy. User-level scheduling is done based on a dispatching policy reconfigurable through API. Different scheduling policies differentiate users based on the overall strategy, e.g. PF, MLWDF, SRPT. At the end of this stage, the RBs will be distributed among users for a given service.

Policy reconfiguration: A policy reconfiguration message is used to indicate to the OAI agent that the underlying virtual subsystem function (VSF) needs to be modified using YAML. A sample message is depicted in Figure 31. At the top level, the control module name is indicated, followed by a sequence of VSFs to be modified, each composed of two (optional) sections: behaviour and parameters. The behavior section is used for swapping VSFs. The parameters section indicates a list of parameters that can be modified for the specific VSF. These parameters act as a public API that the controller can modify and can either refer to a single value or a sequence of values (e.g., an array).



Figure 31: Structure of a policy reconfiguration message.

6.2.2.4 Early results For our first experiment, each operator was assigned 5 UEs for which uniform UDP downlink traffic was generated, while the percentage of radio resource blocks allocated to each operator was dynamically adjusted based on their requirements. The results in terms of the total throughput per operator are illustrated in Figure?. Initially, the MNO was allocated 70% of the radio resources and the MVNO was allocated the remaining 30%. At 10s, the master controller application sent a policy reconfiguration message, setting the available resources of the MNO to 40% and of the MVNO to 60%, simulating a brief requirement for additional resources for the MVNO. Then, at 140s, the application sent a second policy reconfiguration message that re-adjusted the radio resources so that 80% were allocated to the MNO.

Figure 32: Policy reconfiguration for MVNO management: Dynamic allocation of resources

As a second experiment, we implemented an agent-side scheduler supporting a fair scheduling policy and a group-based policy of premium and secondary users, with 70% of the resources allocated to the premium users and the rest to the secondary. One MNO and one MVNO were employed, where the MNO was assigned the fair policy and the MVNO the group-based one. Each operator was allocated half of the available radio resources and was assigned 15 UEs. In the group-based MVNO case, nine UEs belonged to the premium group and the remaining six acted as secondary users. We generated uniform UDP downlink traffic for all the UEs and measured the throughput of each UE per operator.

Figure 33: Policy reconfiguration for MVNO management: CDF of UE throughput when different scheduling types are used for different slices.



The results are illustrated in Figure 33. In the case of the MNO, all the UEs had a throughput of about 380Kb/s due to their fair scheduling policy. On the other hand, in the case of the MVNO, UEs assigned to the premium group had a throughout of about 450Kb/s, while the UEs of the secondary group achieved a throughput of less than 200Kb/s.

6.2.2.5 Analysis and technical Impact On the technical impact enabling RAN Sharing capabilities in the OAI platform is expected to improve the technical base and technology readiness in support of Network Slices. In addition, the platform is open for any type of scheduling principle to be tested and evaluated (like Proportional Fair, Max Weight etc), thus research around LTE resource allocation problems under multi-tenancy is expected to be boosted.

6.2.3 Load balancing Use Cases

6.2.3.1 Load balancing use cases description

Use Case: Load

balancing

Load balancing in LTE and Wi-Fi networks.

EPC

UE

eNodeB

eNodeB

COHERENT Controller(C3)

UE

WiFi AP

WiFi AP

Figure 34: For this use case the 5G-Empower and the SICS testbeds will be exploited.

Involved Partners

CMA, CREATE-NET, SICS, TP

Architecture • 5G-EmPOWER Control Plane



Layers • Wi-Fi User Plane

• LTE User Plane

• LTE Control Plane

Components • CommAgilty EPC

• CommAgilty eNodeB

• Wi-Fi AP

• OAI EPC

• OAI eNodeB

• 5G-EmPOWER Agent (Wi-Fi and LTE)

• CommAgilty RT-Controller (MAC Scheduler)

• SICS Network information function (NIF) providing RAT agnostic probabilistic models on attainable bandwidth for Wi-Fi and/or LTE (if applicable)

• TP erGW, an open source implementation of GGSN/PGW functionality, which can be used as part of the EPC configuration of the use case.

Pre-Conditions

• UE has interfaces for LTE, Wi-Fi or both.

• All eNodeBs or APs nodes operate in the same band (LTE) or channel (Wi-Fi) respectively, but in different bands from each other.

• SICS network information functions (NIF) LTE-specific monitoring requirements, if applicable:

o NIFs running on eNodeBs require a UE to be able to connect to multiple neighboring eNodeBs such as the scenario of CoMP in 3GPP Rel. 11, where multiple links can be evaluated to and from a UE.

o eNodeBs and UEs should be given access to measure channel quality metrics on neighbor links and allowed access of block error ratio (BLER) to estimate throughput on neighbor links

• SICS NIF Wi-Fi-specific monitoring requirements, if applicable:

o APs have 2 wireless interfaces of which one is dedicated to channel quality monitoring and the other is used to serve UEs

o APs and UEs should be 802.11k compliant such that APs can receive reports about downlink channel quality in the Wi-Fi channel

• SICS NIF (LTE/Wi-Fi): For simplicity of evaluation, link/rate adaptation by modifying MCS is disabled in both LTE and/or Wi-Fi.

• Setup radio conditions in such a way so that UEs will have the strongest signal from the first eNodeB and/or the first AP.

Dependencies • No dependencies with other use cases.

Test Steps • Start eNodeBs and/or APs



• Infrastructure setup options

o LTE-specific:

2 eNodeBs initialized and connected with S1 interface to EPC

eNodeBs are connected with the 5G-EmPOWER control plane.

2 UE that will be connected to the first eNodeB based on radio condition (no UEs are connected to the second eNodeB).

o Wi-Fi-specific infrastructure setup:

2 APs are initialized and connected with 5G-EmPOWER coordinator

2 more UEs that will be connected to the first AP based on radio condition (no UEs are connected to the second AP)

• 5G-EmPOWER coordinator receives information of the resource utilization (i.e. load) from two eNodeBs (containing RSRP/RSRQ measurements of neighboring cells) and/or two APs and UE performance reports from eNodeBs and/or APs (radio link performance information, such as attainable bandwidth).

• Intra-RAT scenario: The 5G-EmPOWER coordinator commands handover (while taking the radio link performance of the UE into account) of either:

o 1 UE from one eNodeB to the second eNodeB

o 1 UE from one AP to the second AP.

• Inter-RAT scenario (optional): The 5G-EmPOWER coordinator commands handover of 1 UE from one eNodeB to an AP, initiating an inter-RAT handover (requires additional mechanisms).

Expected Results

• Demonstration of COHERENT coordinated resource management based on the monitored network state. By distributing UEs among available radio nodes based on COHERENT coordinator decision taking into account not only radio conditions, but also load contributes to efficient resource management.

KPIs • Network throughput

• Spectral efficiency

• User QoS.

6.2.3.2 Added value to the related work Within the scope of this use case we will demonstrate load balancing based on unified and technology-agnostic network graph information handled by the COHERENT C3 decisions. For this purpose, both LTE and Wi-Fi testbed infrastructures are considered. The prototype will implement relevant components and concepts from the COHERENT architecture, such as NIFs for building up network graphs [COHERENT-D2.2]. In this case, load is dynamically balanced by the COHERENT C3 using the learned attainable throughput, the available node capacity [COHERENT-M3.3] and other relevant metrics. This way of modelling of the network state in combination with resource



coordination enables handover decisions and admission control that precisely match the service requirements of the individual UE relative to the varying network conditions and user concentrations.

6.2.3.3 Technical approach This use case consists of two scenarios:

• Intra-RAT load balancing: In this case the LTE and Wi-Fi RAT are addressed separately, i.e. the controller performs the load-balancing of the wireless stations treating each station as two different entities: the Wi-Fi station and the LTE UE.

• Inter-RAT load balancing: In this case the controller addresses the LTE and the Wi-Fi RAT jointly, i.e. the controllers can handover a wireless station from LTE to Wi-Fi and vice-versa. This case is optional and will be considered, provided that a suitable mobility management technique for handover between LTE-Wi-Fi can be identified and that measurements needed for creating comparative models of channel quality can be accessed in both testbeds.

In the following we focus on the experimental setup for the intra-RAT scenarios.

Experimental Setup

The network setup is composed of two LTE eNodeBs, two Wi-Fi APs, and the C3. Two dual stack (Wi-Fi and LTE) terminals are also used. The C3 is implemented by capitalizing on the EmPOWER coordinator and agents running on the LTE/Wi-Fi serving nodes. Network graphs are maintained through the EmPOWER agents by the use of NIFs [D2.2] (implemented in SICS testbed) monitoring and processing MAC-layer information locally at the serving nodes for the purpose of representing the channel quality [D3.1, M3.3].

Below is an example of the topology considered for load balancing in LTE and/or Wi-Fi, composed of serving nodes (eNodeBs or APs) and two UEs (smartphones) and the COHERENT C3 (Figure 35).

EPC

UE

eNodeB

eNodeB

COHERENT Controller(C3)

UE

WiFi AP

WiFi AP

Figure 35: Topological overview of the load balancing use case infrastructure.

Intra-RAT Scenario: LTE At the beginning of the use case, 2 UEs are connected to one eNodeB based on radio condition. The second eNodeB will have no UEs attached (the UEs have a weaker channel to it comparing to the first eNodeB). The EmPOWER Agent at eNodeBs provides the 'slave' counterpart in the controller-agent communication of EmPOWER coordinator that act as a part of COHERENT coordinator.



The resource utilization of two eNodeBs and UE reports (containing RSRP/RSRQ measurements of neighboring cells) will be passed to COHERENT coordinator where they will be abstracted for technology-agnostic graph construction. The coordinator will note that the second eNodeB is not loaded at all and from the channel point of view is acceptable for UEs. Then, COHERENT coordinator will request the first eNodeB to do handover of one UE to the second eNodeB and then both UEs can extend their throughputs. Thus, by distributing UEs among eNodeBs based on COHERENT coordinator decision taking into account not only radio conditions, but also cell load contributes for efficient spectrum management and sharing.

Legend

RSRP, RSRQ of serving and

neighbouring cells

SINR

UE RAN Node with EmPOWER Agent

Abstraction module

LTE Handover execution

Technology-spesific

Technology -agnostic

Measurement Configuration

EmPOWER Coordinator

RSRP, RSRQ

Available Node Capoacity

Load balancing decision

(technology-agnostic)

Load balancing RequestHandover request

(Reasion: Load balancing)

Resource blocks (RB) utilisation

Figure 36: Sequence diagram for load balancing in LTE

Intra-RAT Scenario: Wi-Fi At the beginning of the use case, 2 Wi-Fi stations are connected to one Wi-Fi AP, while the second AP is serving no wireless station. The EmPOWER Agent at Wi-Fi AP provides the Controller with measurements about the channel condition and resource utilization (e.g. attainable bandwidth) based on RSSI, PDR and CBT/CBR. This monitoring functionality is implemented in terms of a NIF instance running on the Wi-Fi AP. In this case, the Wi-Fi NIF implements and outputs the attainable bandwidth which is a RAT-agnostic abstraction that allows for load balancing relative to probabilistic guarantees on the link performance [COHERENT-D3.1,COHERENT-D2.2]. The output from the NIF is communicated to the coordinator via the EmPOWER Agent of the AP. An example of a load balancing process is illustrated in Figure 36.



Figure 37: Sequence diagram for load balancing in Wi-Fi

Two cases can here be demonstrated:

• Load-balancing: The coordinator notices that the second AP is not loaded and that it can serve (from the channel quality point of view) any of the two Wi-Fi stations. The coordinator then performs the handover of the one Wi-Fi station to the other AP.

• Policy-based handoff: The coordinator detects a channel quality degradation between the second AP-UE pair and migrates the UE back to the first AP.

Monitoring models providing high-level abstractions based on processed low-level measurements (e.g. RSSI) enable robust mobility management operations conditioned on both channel quality and user specific service requirements.

6.2.3.4 Expected results The benefits are load-balancing handover in LTE have been shown in many studies. For example, in a simple lab setup consisting of two eNodsBs and two UEs we have been able to demonstrate that by enforcing a handover it is possible to bring the network to an operating point that would not have been reached with standard 3GPP procedures. Likewise, in the Wi-Fi case a centrally coordinated load-balancing can both improve the aggregated network throughput and can avoid oscillations in the station association behaviour (we remind the reader that in most Wi-Fi deployment association is driven purely by the station). By leveraging on the COHERENT C3 it is possible to enable new level of programmability across BOTH Wi-Fi and LTE network implementing sophisticated load-balancing scheme which leverage on a global network view to implement their decision logic. Early implementation and integration of Wi-Fi NIFs providing probabilistic models on low-level metrics [COHERENT-D3.1] into an EmPOWER testbed has been successfully tested. More sophisticated RAT-agnostic models providing the attainable bandwidth are currently developed in WP3 and will be considered for integrated prototyping.

6.2.3.5 Analysis and technical Impact For the LTE scenario, performance evaluation will be carried out using the key indicators: cell throughput, UE throughput, RB resource usage. During initial study in the practical setup, it was detected that for a load-based handover in LTE of a specific UE to neighbouring cell, it is very critical



to fine tune the UE-specific handover offset. Especially, large gains were observed when the UE to be handed over is located at the cell edge of eNodeB. The fine-tuning of the algorithm and integration with the COHERENT coordinator is in progress. The performance evaluation for the Wi-Fi will orient development on system throughput as a key indicator. The expected impact of the prototype is to show a complete control loop where the infrastructure resources are coordinated conditioned on channel quality and service policies based on abstracted network information in line with the COHERENT concept. Further implementation and integration efforts are currently in progress.



7. Conclusions The design, deployment and evaluation procedures in WP6 are aimed to provide a proof of principle for the COHERENT architecture and evaluate the practical feasibility and performance of the technical solution proposed. As it was stated in the introduction the aim of this deliverable is to create a testing ground as close as possible to a real scenario in order to validate the integrated COHERENT platform.

The experimental evaluation of the COHERENT platform will rely on the implementation of the test case specifications, the development of the COHERENT solution and the COHERENT testbed deployment. The test cases which have been specified in this deliverable, namely Spectrum Sharing, RAN Sharing and Load balancing are specified to indicate the expected capabilities of the overall COHERENT platform. These have been designed taking into consideration the service requirements specified in D2.1 [COHERENT-D2.1] and the functionalities which were proposed by the system architecture in D2.2 [COHERENT-D2.2].

The underlying physical infrastructures that will support the overall COHERENT testbed, are three LTE testbeds (provided by Eurecom, CommAgility, UDE), one Wi-Fi testbed (by CreateNET) and one CEP (from VTT).

A number of challenges must be addressed towards building the integrated COHERENT solution. With respect to the Network Slicing concept and considering that the physical infrastructure is shared among several operators, one operator cannot interfere with actions from another operator using the same physical infrastructure. In order to address this challenge, C3 mechanisms must automate and guarantee isolation between different virtual infrastructure operators (i.e. MVNOs) and consistency / integrity of the network at any given instant (e.g. several virtual network operators may lead to inconsistencies of the network due to malicious configurations by one of them). Another dimension is related to the propagation delay between the CP/DP parts of the C3. The following deliverable, D6.2 [COHERENT-D6.2], will specify the execution principles of the networking scenarios as outlined in this deliverable. It will also describe in detail the execution, results and validation of the use cases. The goal in D6.2 [COHERENT-D6.2] is twofold: in one hand will prove system integration, while in the other hand will prove the COHERENT solution efficiency.



REFERENCES

[COHERENT-D2.1] Deliverable D2.1, "Use cases and architecture", delivered M4

[COHERENT-D2.2] Deliverable D2.2, "System architecture and abstractions for mobile networks", delivered M12

[COHERENT-D2.3] Deliverable D2.3, "Initial release of the SDK", delivered M12

[COHERENT-D2.3] Deliverable D2.4, "Final release of the SDK", to be delivered M24

[COHERENT-D6.2] Deliverable D6.2, "Final report on technical validation", to be delivered M30 [Zielczynski] Zielczynski, P., “Requirements management using IBM Rational RequisitePro”, Boston, MA, USA, IBM Press, 2008

[MT14] Osseiran, Afif, et al. "Scenarios for 5G mobile and wireless communications: the vision of the METIS project." IEEE Communications Magazine 52.5 (2014)

[Palola] M. Palola et al., "Live field trial of Licensed Shared Access (LSA) concept using LTE network in 2.3 GHz band," IEEE DySPAN, McLean, VA, 1-4 April 2014.

[Matinmikko] Kliks, A., Holland, O., Basaure, A., & Matinmikko, M. (2015). Spectrum and license flexibility for 5G networks. IEEE Communications Magazine, 53(7), 42-49.

[LSAVideo] “Licensed Shared Access (LSA) trial demonstration using real LTE network.” Online: https://www.youtube.com/watch?v=7AxJIzv4I9o (accessed at August 2016)

[OAI] The OpenAirInterface Initiative. http://www.openairinterface.org/.

[NN14] Navid Nikaein, Raymond Knopp, Florian Kaltenberger, Lionel Gauthier, Christian Bonnet, Dominique Nussbaum, and Riadh Ghaddab. Demo: OpenAirInterface: an open LTE network in a PC. In MOBICOM 2014, 20th Annual International Conference on Mobile Computing and Networking, 2014.

[NN14b] N. Nikaein, M. Marina, S. Manickam, A. Dawson, R. Knopp, and C. Bonnet. Openairinterface: A flexible platform for 5G research. ACM Sigcomm Computer Communication Review, 2014.

[KT11] K. Tan et al. Sora: High-Performance Software Radio Using General-Purpose Multi-Core Processors. Communications of the ACM, 54(1):99–107, Jan 2011.

[KA07] K. Amiri et al. WARP, a Unified Wireless Network Testbed for Education and Research. In Proceedings of IEEE MSE, 2007.

[AB14] A. Bhamri, N. Nikaein, F. Kaltenberger, J. Hamalainen and R. Knopp, "Three-Step Iterative Scheduler for QoS Provisioning to Users Running Multiple Services in Parallel," 2014 IEEE 79th Vehicular Technology Conference (VTC Spring), Seoul, 2014, pp. 1-5.

[Chowdury09] Chowdhury, N.M.M.K.; Boutaba, R., "Network virtualization: state of the art and research challenges," Communications Magazine, IEEE , vol.47, no.7, pp.20,26, July 2009 doi: 10.1109/MCOM.2009.5183468

[Foukas16] X. Foukas, N.Nikaein et al, “FlexRAN: A Flexible and Programmable Platform for Software-Defined Radio Access Networks”, ACM Conext, 2017

[Kokku10] R. Kokku, R. Mahindra, H. Zhang, and S. Rangarajan (NEC Laboratories America), “NVS: A virtualization substrate for WiMAX networks,” ACM Mobicom, 2010

[Kokku13]R. Kokku, R. Mahindra, H. Zhang, S. Rangarajan (NEC Laboratories America), "CellSlice: Cellular Wireless Resource Slicing for Active RAN Sharing", 5th International Conference on Communication Systems and Networks (COMSNETS), January 2013

Tao Guo, Rob Arnott (NEC Telecom Modus Ltd.), “Active LTE RAN Sharing with Partial Resource Reservation”, submitted to IEEE Vehicular Technology Conference, September 2013



[Cacheda2007] Cacheda, R. A., García, D. C., Cuevas, A., Castano, F. J. G., Sánchez, J. H., Koltsidas, G., Pantò, A. (2007). QoS requirements for multimedia services. In Resource Management in Satellite Networks (pp. 67-94). Springer US.

[NGMN] Alliance, N. G. M. N. (2015). 5G white paper. Next generation mobile networks, white paper.

[3gpp-rel14] http://www.3gpp.org/news-events/3gpp-news/1768-ran_rel14

[He15] Keqiang He, Junaid Khalid, Sourav Das, Aaron Gember-Jacobson, Chaithan Prakash, Aditya Akella, Li Erran Li, and Marina Thottan. 2015. Latency in Software Defined Networks: Measurements and Mitigation Techniques. SIGMETRICS Perform. Eval. Rev. 43, 1 (June 2015)

[3GPP-TS23.251] 3GPP TS 23.251 specification, Network sharing; Architecture and functional description.

[Foukas16] Xenofon Foukas, Navid Nikaein, Mohamed M. Kassem, Mahesh K. Marina, Kimon P. Kontovasilis: FlexRAN: A Flexible and Programmable Platform for Software-Defined Radio Access Networks. CoNEXT 2016: 427-441

[CORE] www.core.willab.fi/

http://dblp.uni-trier.de/pers/hd/n/Nikaein:Navid

http://dblp.uni-trier.de/pers/hd/k/Kassem:Mohamed_M=

http://dblp.uni-trier.de/pers/hd/m/Marina:Mahesh_K=

http://dblp.uni-trier.de/pers/hd/k/Kontovasilis:Kimon_P=

http://dblp.uni-trier.de/pers/hd/k/Kontovasilis:Kimon_P=

http://dblp.uni-trier.de/db/conf/conext/conext2016.html#FoukasNKMK16

coordinated control and spectrum management for 5g ... · andsf access network discovery and...

Documents