A Survey of Deployment Solutions and OptimizationStrategies for Hybrid SDN Networks

Xinli Huang, Shang Cheng, Kun Cao, Peijin Cong, Tongquan Wei, Shiyan Hu

Abstract—A hybrid SDN network contains both traditionaland SDN network, which combines the robustness of traditionalprotocols with the flexibility of SDN while avoiding their limita-tions and incompatibility. However, a hybrid SDN network comeswith its own set of challenges, including error-prone deploy-ment processes, risks of inconsistency, and complex incrementaldeployment strategies. In this paper, we present a survey ofthe deployment solutions and optimization strategies for hybridSDN networks. We systematically review solutions to controlplane and data plane deployments, and describe typical usecases of hybrid SDN networks. We discuss and compare variousoptimization strategies from perspectives of traffic engineering,resource saving, network control capacity, and network security.This paper aims to provide insights to researchers into the futuredevelopment of hybrid SDN networks and inspire more effortsin this area.

Index Terms—hybrid SDN, control plane deployment, dataplane deployment, optimization strategy, traffic engineering

I. INTRODUCTION

The distributed nature of traditional networks has multi-ple advantages such as scalability, reasonable convergence,resiliency and stability. Traditional equipment, however, hasproblems such as being vendor specific, offering a fixed setof features, requiring per-box management, cumbersome plan-ning and deployment phases, and high chances of human error.As a result, it is difficult to implement innovative conceptssuch as network virtualization and on-demand provisioningof Network as a Service (NaaS) in traditional networks.Moreover, a fine-grained level of control is not offered intraditional networks [1].

Software defined networks (SDN) is an emerging networkarchitecture which separates the control plane and the dataplane [2]. Because of the centralized network control withglobal view, SDN provides flexible and reliable network man-agement, support smart flow scheduling for the improvementof link utilization and network throughput. SDN is widelyused in different scenarios such as Google’s backbone network[3], Microsoft’s public cloud [4], NTT’s edge gateway [5],and optical (IP/WDM) network [6]. In addition, SDN hasbeen proven successful in Network Function Virtualization

X. Huang, S. Cheng, K. Cao, P. Cong and T. Wei are with the Departmentof Computer Science and Technology at East China Normal University,S. Hu is with the Department of Electrical and Computer Engineering atMichigan Technological University. T. Wei is the corresponding author:tqwei@cs.ecnu.edu.cn.

This work is partially supported by Shanghai Municipal Natural ScienceFoundation under Grant 16ZR1409000 and the Key Project of ShanghaiScience and Technology Innovation Action Plan under Grant 18511103302.

service (NFV) which has made significant progress from trialevaluation to production deployment [7] [8]. Because of thesesignificant benefits, enterprises and governments have strongmotivations to deploy SDN.

There are several options for helping convert traditionalnetworks directly into SDN networks [9] [10]. A ship in thenight strategy is a straightforward approach to new networkingarchitectures. In reality, however, the transition from the legacynetwork to an SDN network does not happen overnight. First,the transition requires significant deployment costs. In additionto the high price of SDN switches, companies also have tohire additional SDN programmers because an easy-to-use SDNconfiguration and management frameworks are still evolving[11] [12]. Second, SDN comes with its own limitations, theOpenFlow protocol [13] is not mature enough and commercialSDN switches and controllers are not completely stable andreliable [1]. Those factors slow down the SDN deploymentstep, thus promoting the birth of the hybrid SDN network[14] [15].

Different definitions about hybrid SDN networks are givenin the literature. Generally speaking, hybrid SDN is a logicalstep in the process of transitioning from a traditional networkto SDN. It combines the programmability of the centralizedcontrol with the robustness of the distributed routing. Op-erators can balance the load and manage the network moreeasily. Controllers in hybrid SDN networks can only focus onuseful flow or traffic, while most packets are managed by therobustness traditional protocols. In this case, when deployinga network protector or a network optimizer, operators onlyneed to migrate legacy access devices to SDN switches.However, SDN switches in hybrid SDN networks may beimproperly deployed or an inefficient optimization strategymay be adopted, resulting in network performance degradationor inconsistency. Forwarding decisions made by the controllermay conflict with traditional routing protocols due to theisolated control domains, which may lead to forwarding loopsor black-holes. Therefore, it is of great significance to carryout the investigation into the deployment and optimization ofhybrid SDN networks.

There are many surveys about SDN networks, however,few of them review the state of the art of hybrid SDNnetworks. Kreutz et al. [16] and Thyagaturu et al. [6] brieflydescribe several representative hybrid SDN solutions in theirSDN survey. They do not consider the investigation of hybridSDN networks as a specific research field, but just regardit as a special case of SDN. Vissicchio et al. [1] discussthe opportunities and research challenges of hybrid SDN

Use Cases

（Section Ⅵ）

Control Plane Deployment （Section Ⅲ）

Data Plane Deployment （Section Ⅳ）

HAL, Overlay,

SDN Shim, Middlebox

（Section Ⅳ）

Optimization Strategy

（Section Ⅴ）

• Restful API (Hybnet, RouteFlow ...)

• The interworking and inter-domain routing

• The management of traditional devices

• The experimental platform and simulation tools

Intra-domain

Deployment

Inter-domain

Deployment

• Legacy & SDN Switches

• Only legacy Switches

• Seamless interworking

• Interdomain management

• RouteFlow framework

• Performance analysis

• Hardware Abstraction Layer

• HAL extension

• Hybrid southbound interface

• Hybrid IP/SDN node （ OSHI）

• Hybrid SDN in mobile networks

• The optimization of link utilization

• The economization of network resources

• The optimization of network control capacity

• The protection of network security

Ⅵ.A

Ⅵ.B

Ⅵ.C

Ⅳ.A

Ⅳ.A

Ⅳ.B

Ⅳ.C

Ⅳ.D

Ⅴ.A

Ⅴ.A

Ⅴ.B

Ⅴ.C

Ⅲ.A

Ⅲ.A

Ⅲ.B

Ⅲ.B

Ⅲ.B

Ⅲ.BController

Switch/Router

• Topology discovery Ⅲ.C

Fig. 1. The deployment solutions and optimization strategies for hybrid SDN networks.

networks, and describe different use cases in each hybrid SDNmodel. However, no concrete techniques regarding to hybridSDN networks are summarized. Rathee et al. [17] review somedeployment solutions in hybrid SDN networks, and discussthe coexistence of traditional networks with SDN networksin detail. However, they do not consider the optimizationstrategies in hybrid SDN networks.

In this paper, we present a comprehensive survey of theresearch relating to deployment and optimization solutions inhybrid SDN networks that have been carried out to date. Forthe deployment techniques, we analyze different hybrid SDNmodels, summarize control plane and data plane solutions,and discuss several use cases. For the optimization solutions,we discuss and compare different mathematical models andcorresponding optimization algorithms.

The structure and organization of the paper are shown inFig. 1. Section II discusses the model and background ofhybrid SDN networks. Section III and Section IV describe howto seamlessly unify a traditional network with an SDN networkfrom perspectives of the control plane and the data plane,respectively. We classify related deployment solutions from theperspective of the control plane and the data plane, which issimilar to the pure SDN network. In addition, several commondesign principles are summarized, including the underlyingprotocols, topology discovery issues and the choice of hybridnetwork models, which can be used to reduce repetitive worksin the deployment process. Section V compares some typicaloptimization strategies in hybrid SDN networks, includingmathematical models, core algorithms, the evaluation and

the use case of each strategy. Section VI summarizes someapplication scenarios where hybrid SDN networks have uniqueadvantages.. We summarize this survey and discuss the futureresearch and development trend in Section VII. Table I liststhe main abbreviations used in this paper.

II. WHY HYBRID SDNA. The concept of pure SDN

Before discussing the hybrid SDN network, we need tohave a brief review of the pure SDN network. The basicelements of a network are nodes (e.g., switches, routers,load-balancers), and interconnections (both physical links andprotocol-dependent logical adjacencies) between nodes. InSDN, network nodes implement only the data-plane, while aseparated architectural element, called SDN controller, realizesthe control-plane. The SDN controller is a logically-centralizedcustom software, possibly corresponding to a distributed sys-tem. The independence between the controller and the nodessimplifies the development of a high-level management inter-face [18]. The SDN-based architecture is vertically dividedinto three layers, that is, SDN data plane, control plane andapplication layer, as shown in Fig. 2.

SDN data plane: The data plane (i.e. the forwardingplane) consists of distributed forwarding network elements thatforward data packets. In order to separate control and dataplane, the data plane need to be remotely accessed throughan open vendor-independent southbound interface. OpenFlowand ForCES [19] are well-known protocols for the southboundinterface. They are responsible for splitting the forwarding

Application

layerRouting

Load balancer

Network Application

Control layerNetwork

modelNetworkservice

Controller Platform

Northbound

InterfacesREST API

Data layerOpenFlowSwitches

Southbound

InterfacesForCESOpenFlow

Non-OpenFlowSwitches

OtherControllers

Traffic engineering

Fig. 2. A three-layer SDN architecture [18].

TABLE IABBREVIATIONS

Abbreviation ExplanationAS Autonomous SystemACL Access Control ListBDDP Broadcast Domain Discovery ProtocolCDN Content Distribution NetworkCLI Command-line interfaceFPTAS Full Polynomial Time Approximation Algo-

rithmGMPLS Generalized Multi-Protocol Label Switch-

ingHAL Hardware Abstraction LayerIETF Internet Engineering Task ForceILP Integer Linear ProgrammingISP Internet service providerLFA Link Flooding AttackMPLS Multi-protocol Label SwitchingNaaS Network as a ServiceNFV Network Function VirtualizationNOS Network Operating SystemOFELIA OpenFlow in Europe Linking Infrastructure

and ApplicationsOSHI Open Source Hybrid IP/SDNOVS Open Virtual SwitchSDN Software Defined NetworksSDX Software Defined Network Switching Cen-

terSNMP Simple Network Management ProtocolSTP Spanning Tree ProtocolTCAM Ternary Content Addressable MemoryVLAN Virtual Local Area Network

plane and control plane in the network, and regulate thecommunication between the two planes.

SDN control plane: The SDN controller in the controlplane is mainly responsible for two tasks. One is to translateSDN application layer requests to SDN datapath, and the otheris to provide an abstract model of the underlying networkfor the SDN application layer. One SDN controller includesthree parts: northbound interface agent, SDN control logic, andcontrol plane interface driver. The SDN control layer is oftenreferred to as the network operating system (NOS) becauseit supports network control logic and provides the applicationlayer with an abstract view of the global network. The SDNcontrol layer contains enough information to specify policieswhile hiding all implementation details.

SDN application plane: The application plane is composedof SDN applications. An SDN application can submit arequest to the controller in a programmable manner. The SDNapplication includes a great amount of northbound interfacedrivers, which are responsible for driving the open northboundAPI provided by SDN controller. At the same time, the SDNapplication can abstract and encapsulate its own functions toprovide a northbound proxy interface.

B. The challenges of pure SDN

This subsection discuss the challenges in pure SDN that canbe mitigated in hybrid SDNs.

Deployment challenges: Although SDN has such greatadvantages as improving network traffic control capability,reducing network resource consumption, and balancing linkloads, it indeed has its own limitations. Deploying SDN inexisting networks will incur economic, technical, and organi-

Fig. 3. The overview and comparison of legacy network, hybrid SDN, and pure SDN.

zational challenges. First of all, SDN network can not ignorethe initialization costs in the equipment transformation andprofessional training. Considering the scenario where SDNneeds to make a huge change to the network model especiallywhen it comes to architectural updates on the operators side,network managers need to learn how to design, update, debugand operate SDN networks, and companies need to hireexperienced SDN network engineers [20]. Secondly, it takes acertain amount of time to produce SDN controllers that can beused at the production level. In order to ease the inconsistentrisks among nodes in the network, the controller must makequick decisions in various complicated situations. This willinevitably increase the complexity of network design anddeployment, especially in large scale and high performancereal-time networks. Thirdly, SDN technology has completelychanged the original processing flow of the network chip,which is equivalent to the development of a new forwarding ta-ble. Finally, despite the successes so far, SDN implementationis still in an early stage of development, and various researchinstitutes have proposed different design schemes, leadingto the lack of uniform standards. The discrepancy amongSDN standardization organizations also limits the developmentprospects of SDN to some extent.

Reliability and security challenges: First, new demandsfor mobility, server virtualization and cloud computing leadto increasing real-time requirements of applications such asvideo conferencing or web browsing in a very short timerange, especially when it comes to quality of service or

security issues. To ensure reliability in pure SDN, a fastand expensive out-of-band wide area network between SDNcontrollers and switches will be needed in large networks[1]. For example, updating information about failures or newinput streams would double overheads of network design andmanagement. Second, SDN controllers are software that runson Windows or Linux operating systems, which leaves thecontroller and operating system at the risk of being attacked.As long as the attacker can gain control over the SDNcontroller through continuous attacks, the entire network maybe easily attacked. Even if there is no attack, the controlleris extremely computationally intensive. Once the controllerfails, the entire data center network is paralyzed and becomesuncontrollable.

In fact, the traditional network can naturally avoid the abovedeployment costs, and the reliability and security issues. Thisis because its forwarding behavior is realized in complicatedhardware devices with no control planes. If there is alreadya solution to shortcomings of pure SDN in the traditionalnetwork, the architecture of the corresponding legacy networkremains unchanged. Given this, we only need to focus on howto build a flexible and efficient network model that combinesadvantages of legacy and pure SDN networks. In other words,the hybrid SDN network can effectively alleviate the abovechallenges. Therefore, managing heterogeneous paradigms andensuring profitable interaction between the two types of net-works are of particular importance. Fig. 3. gives an overviewand comparison of different network architectures.

C. The models of hybrid SDN

Brief definition: Hybrid SDN refers to a networking archi-tecture where both centralized and decentralized paradigmscoexist and communicate together to different degrees toconfigure, control, change, and manage network behavior foroptimizing network performance and user experience. For ex-ample, traditionally switches with their distributed algorithmstry to control overall traffic routing whereas, in SDN, thecontroller routes traffic based on the global view. If these arecombined, say a part of traffic is under traditional control andthe remaining under the SDN controller, we get a hybrid SDNarchitecture.

In order to deploy hybrid SDN networks correctly andeffectively, a suitable hybrid SDN network model is needed. Inthis section, we classify the modeling of hybrid SDN networksbased on the functional division of networks, the combinationof switches, and the network service and usage scenario.

Modeling hybrid SDN networks based on the combina-tion of switches: Based on the combination of switches, wedivide the hybrid network into the three categories, including

• The network integrates SDN switches and traditionalswitches.

• Traditional switches utilize SDN framework for central-ized control.

• SDN switches focus on the interconnection between SDNautonomous system (AS) and non-SDN AS.

For this scenario, deployment strategies can be summarized as• SDN switch as a middlebox to send information and

configuration to the entire network.• The controller manages traditional switches indirectly by

sending seed packets.• Expanding the controller to control both SDN switches

and legacy switches.• Achieving the hardware abstraction layer or extending

southbound interfaces without making any changes to thecontroller.

Modeling hybrid SDN networks based on the networkservice and usage scenario: According to the network serviceand usage scenario, Vissicchio et al. [1] classify the hybridSDN network as Topology-Based Hybrid SDN (TB hSDN),Service-Based hybrid SDN (SB hSDN), Class-Based HybridSDN (CB hSDN) and Integrated Hybrid SDN (IntegratedhSDN). In the TB hSDN model, the network is partitionedinto different zones, and each node or switch is within onezone. In this model, a zone is defined as a collection ofinterconnected nodes which are controlled by either SDNcontrollers or traditional protocols. It is required to select theappropriate locations to deploy SDN devices, or to divide theappropriate area as SDN deployment area. In the SB hSDNmodel, legacy and SDN framework provide different services.For the network-wide forwarding service, the two paradigmscan control a different portion of the FIB of each node. In theCB hSDN model, the traffic is divided into two paradigms, oneis CN-controlled (legacy), the other is SDN-controlled. Legacyand SDN framework typically span all the network devices,

controlling a disjoint set of FIB entries on each switch. In theIntegrated hSDN model, SDN has full control of the entirenetwork, and the role of traditional protocol is to forward thecontrol message to the forwarding table in all legacy switches.

After the identification of different types of hybrid SDNmodels, operators can consider concrete deployment plans.Building a hybrid SDN network requires deploying a portof the SDN switches in a traditional network. However, ifno efforts are taken, traditional switches and SDN switchescannot communicate with each other. To solve this problem,many deployment solutions have been proposed, which canbe divided into control plane deployment and data planedeployment solutions. In the control plane solutions, changesto the data plane are reduced as much as possible, so that thecontroller is responsible for unifying these complex underlyingswitches. In the data plane solutions, it is better to makethe underlying network transparent to the controller, therebyminimizing changes in the control plane.

The control plane deployment and data plane deploymentsolutions are suited for different scenarios. For example, ifa new hybrid SDN network is built from scratch, a controlplane deployment scheme can be adotped to improve thenetwork operating efficiency. If the original network is a pureSDN network, and the network has been running for a while,or some SDN switches are added to a traditional networkby using existing SDN controllers (e.g., OpenDayLight [21],NOX [22]) with no further modifications, it is preferable touse a data plane deployment scheme. However, some dataplane deployment schemes introduce an additional hardwareabstraction device that may degrade the performance of thenetwork [23].

When these deployment problems are solved, it is necessaryto find optimization strategies to make better use of these fewSDN switches in hybrid SDN networks. These optimizationstrategies can not only be implemented in SDN controllers(control plane), but also can affect the deployment order ofSDN switches (data plane).

D. The standardization of hybrid SDNs

In order to promote the standardization process of hybridSDN networks, some organizations and research groups havededicated extensive research effort to hybrid SDN networksand published numerous technical documents and reports. Un-doubtedly, the biggest beneficiary of the hybrid SDN networkare network equipment enterprises. Therefore, several equip-ment manufacturers have developed some industry standardsand technical guidance of hybrid SDN networks.

Standardization efforts: NFV addresses the topic about“traditional networking coexistence”, and discusses possiblescenarios of hybrid SDN networks [24]. These scenarios arei) one set of ports (physical interfaces) being assigned to atraditionally controlled datapath whereas other ports (physicalinterfaces) are assigned to an SDN controlled datapath; ii)forwarding is controlled by traditional mechanisms, whichcan be used to carry the traffic for an SDN managed over-lay network; iii) SDN managed classification operations and

actions are used to implement value added processing (e.g.classification into categories for QoS purposes or firewalling)while traditional mechanisms continue to be used for forward-ing; and iv) SDN performs major traffic classification anddelegates partial forwarding to traditional forwarding elements.The Internet Engineering Task Force (IETF) [25] defines a setof southbound interfaces, which can be used in data planedeployment solutions (Section III-B).

Industry efforts: NEC white paper (2014) [26] tries toavoid pitfalls of SDN by gradually introducing SDN frame-work to the area of an existing network where fine-grainedcontrol is required. The proposed hybrid models can becategorized into three types: add-on type, partial replacementtype, and overlay type. In the white paper, NEC also shares thepotential commercial value of hybrid SDN networks, includingsecurity gateway, DoS/DDoS attack countermeasures, opti-mization of inter-data center connections, virtualization of theserver network, the migration of intra-data center network, andaggregating network management for multiple departments.

HP SDN white paper [27] describes how HP SDN solutionuses SDN hybrid mode to achieve scalable, low-risk networkdeployments. The controller delegates some portion of thedata plane forwarding decisions to the controlled switches.OpenFlow-hybrid switches are introduced in their frameworkto process all kinds of packets and receive/send instructions.Similar solution includes Huawei Enterprise Network Proces-sor, Huawei Smart Network OpenFlow Controller [28], andOpenDayLight [21].

III. DEPLOYMENT SOLUTIONS IN CONTROL PLANE

In order to properly deploy a hybrid SDN network, it isnecessary to solve two key problems. The first is how tomake the SDN network and the traditional network unifiedin the view of the controller, and the second is how to get thecorrect status of the network so that the controller can make theforwarding decision. In this section, we focus on deploymentmethods in control plane, which are responsible for unifyingdifferent kinds of switches by adding extra components andmodules. On this basis, depending on the scope of the deploy-ment, these solutions are divided into Intra-domain and Inter-domain deployment methods, respectively. As complementary,we discuss the topology discovery issues and related protocolsthat need to be considered when implementing a real hybridSDN network.

A. Intra-domain deployment

The intra-domain deployment is designed for the deploy-ment of a hybrid SDN network within an AS. Controllersin these solutions need to find viable paths in the networkfor the seamless connection. The key to deployment is howto integrate the features of the SDN switches into the entirenetwork. According to whether there is an SDN switch in thenetwork or not, we divide the research into two categories. Thefirst category is legacy switches and SDN switches coexist,and the second category is only legacy switches in the hybridnetwork.

1) Legacy switches coexist with SDN switches:The coexistence of SDN switches and traditional switches

is the most common situation in hybrid SDN networks. Inthis section, we summarize three types of deployment meth-ods, including i) the management of legacy devices, ii) thewaypoint enforcement of traffic, and iii) the hybrid extensionof controllers.

The management of legacy devices: The most intuitiveway is to force legacy switches to send packets to the con-troller without considering other strategies. Once the controllerreceives these packets, it will compute the network operationsand announce the updates that need to be performed in thehybrid underlying network.

Cheng et al. [29] implement a hybrid network controller,called Telekinesis, that provides SDN-enabled routing throughlegacy paths. The key is to forward the packet to the controlleras soon as possible. For the purpose of updating routing entriesin legacy switches, LegacyFlowMod integrates the concep-t of OpenFlow, legacy switch, traditional switch port andMAC address. Telekinesis calls LegacyFlowMod to instructthe SDN switches to send seed packets with the specificsource MAC address to legacy switches. When the traditionalswitch receives the packet, it is believed that the node of theMAC address can be reached, hence, it modifies its IP-MACmapping table. When a packet enters the network, the legacyswitch will forward it to the nearest Openflow switch based onthe IP-MAC mapping table. As shown in Fig. 4, the forwardingpath s-LE2-LE5-d in Telekinesis includes an OpenFlow switch(OF6).

S

D

LE1

LE2 LE3

LE4LE5

OF6

SDN

Controller

Legacy Switch

OpenFlow Switch

Fig. 4. Telekinesis [29].

The Telekinesis has two disadvantages: i) the controlleronly provides coarser-grained and destination-based controlof legacy path, and ii) the new installed SDN-enabled pathis vulnerable. In an SDN network, The controller will installmore fine-grained paths because it can match packets based onboth source and destination MAC addresses while the layer-2routing in a legacy network is only destination-based. Besides,regardless of whether these incoming packets are seed packets,the MAC learning function in the traditional switch will react

to all of them. When the switch relays a packet from a specificMAC address, a forwarding entry for this address may change,which may lead to the unstable path update. In order to solvethe above restrictions, Cheng et al. [30] further refine thecontroller, and introduce the concept of magnet address. Bysending gratuitous ARP messages, the end hosts will updateits IP to magnet address mapping table. The magnet addressdoes not correspond to any real host in the network, which is afake MAC address to obtain the network visibility and managethe forwarding behavior of end nodes and legacy switches. Inthis way, the packets from the same destination from differentsource hosts will go through different paths. According to thedestination IP address, the last SDN switch on the new pathwill rewrite the magnet address to the real MAC address. Theresult shows that when only 20% of network switches areSDN-enabled, it can achieve full control over routing in thehybrid network.

Network virtualization is the process of combining hardwareand software network functionality into a single software-based administrative entity. In order to properly deploy vir-tualization services, it is necessary to provide the networkoperator the centralized control over the whole hybrid network.Lu et al. [31] propose a hybrid controller, named HybNET. Inaddition to the centralized control capability in the hybrid SDNnetwork, a common API is supplied for operators to processtransactions and configure hybrid network infrastructure acrossboundaries. As for the configuration of two types of switches,HybNET does not have the special needs of the network topol-ogy like Panopticon [32] and Fabric [33]. The topology andnetwork status are entered manually by the administrator oracquired by a dynamic link discovery protocol (i.e., Link LayerDiscovery Protocol (LLDP)). With respect to the manipulationof underlying devices, the framework requires that all switchesin the network establish a connection with the controller.These features make it easy to control a traditional device asan SDN switch, and can be further utilized in the networkvirtualization deployment. For example, when an operatormanages the network, the controller analyzes the specificconfiguration of the underlying network, divides the overallrules into OpenFlow rules and traditional configurations, andsends rules to the OpenFlow controller and traditional switchesby SNMP and/or Network configuration protocol (NETCONF)[34].

The waypoint enforcement of traffic: Levin et al. [32]propose the Panopticon framework to abstract the transitionalnetwork into a logical SDN network. Panopticon is an incre-mental implementation method on the principle that the packetthat traverses at least one SDN switch can obtain the end toend network control (e.g., access control). In this framework,some legacy ports are defined as SDN controlled (SDNc) portsand the traffic between any two SDNc ports must go throughat least one OpenFlow switch. For each pair of ports thatinclude at least one SDNc port, the controller will choose oneSDN switch as the waypoint and compute the shortest end-to-end path that includes this waypoint. A traditional switchcluster (the set of connected components) is treated as a cell

block, and the OpenFlow switch directly connected to the cellblock is treated as the boundary node (frontier) of the cellblock. In this solution, a per-VLAN spanning tree protocolis configured in each legacy switch to generate a secure pathand independent VLAN ID is assigned to each spanning treeto restrict forwarding and guarantee waypoint enforcement,which is available at legacy switches. Legacy switches willforward the packet to the frontier based on MAC-learningand SDN switches act as VLAN gateways. Under specialcircumstances, when the path between two legacy ports onlytraverses the legacy switches, the forwarding is performedaccording to the traditional mechanisms and is unaffectedby the partial SDN deployment. Panopticon is a commonand high-efficiency deployment mechanism, which can deeplyextend SDN capabilities into existing legacy networks.

Considering that the behavior of the edge devices andcentral devices are different, if the edge devices and centraldevices are treated equally, it may bring unnecessary complex-ity to the whole network. An acceptable solution in an SDNnetwork is that the boundary devices are controlled by a fine-grained and service-oriented controller, while the remainingdevices are controlled by a coarse-grained controller thatfocuses on high-speed forwarding. While in a traditional net-work, in order to save Ternary Content Addressable Memory(TCAM) resources, destination-based solution is responsiblefor the routing service if the packet belongs to the majority ofnode pairs, and the other traffic is routed by the complementaryexplicit routing [35]. Casado et al. [33] extend the idea tothe hybrid SDN network. The authors suggest that boundaryswitches can be controlled by SDN controller, providing theadvanced and innovative services. Non-boundary switches arecontrolled by traditional network protocols that provide basicpacket transport function. This solution is suitable for networkvirtualization and SB hSDN model, which can be summarizedas waypoint enforcement in edge switches. However, at theedge of an enterprise network, introducing the OpenFlowframework that is not accommodated by existing hardwareinvolves replacing thousands of access switches. Furthermore,the solution limits the ability to apply forwarding policieswithin the network core, while Panopticon [32] focuses ontraditional enterprise networks and the overall performance ofthe networks.

Caria et al. [36] adopt a partitioned solution that divides theentire network into separate subdomains and SDN switches areused to connect these subdomains. LSA altering received bythe legacy router triggers a recomputation of the routing andforwarding table. In this solution, node and rule which updatewithin each domain are learned by the OpenFlow switchesat the boundary so that these messages can be passed tothe SDN controller. When the controller receives the LSAfrom SDN switches, it will simply forward it to the proposedhybrid network manager, and vice versa. Furthermore, themanager will optimize the internal routing configurations forload balancing by computing the OSPF link metrics basedon the partition. Finally, through the SDN switches in thecorresponding boundary of sub-domains, these configurations

will be flooded as LSAs and injected into the network. Thebest advantage of this method is that the impact of networkfluctuations will be limited to the original area, because theSDN switches can isolate these changes. As for the specificpartition strategies, the authors formulate the network partitionproblem as an ILP model and try to balance the size of eachdomain as much as possible.

Based on the concept of divide and conquer, the “opticalbypass” framework is proposed in [37]. Similarly, traditionalnetwork domains are separated by SDN switches, and long-distance high-speed transmission between SDN switches isachieved over optical networks. In this case, heavy traffic isoffloaded from the high-load link in the original OSPF domainwhile has no impact on the stability of the traditional networkdomain. The paper shows the use case in EU countries. Thetraffic between countries is forwarded through the opticalnetwork managed by the SDN controller, while internal trafficis still forwarded by traditional switches.

The hybrid extension of controllers: Modularization isone of the common ways to relief the complexity in softwaredevelopment, researchers can utilize this pattern to design anextended hybrid control plane and make it scalable. Honget al. [38] implement a typical hybrid network controller bycombining the optimized deployment scheme within the con-troller. The controller contains deployment planning module,global view module, traffic engineering module and failurerepairment module. When the new budget is approved, thedeployment planning module adopts the deployment algorithmto select appropriate switches for incremental deployment. Theglobal view module can obtain the link state of the entirenetwork and pass the information to the traffic engineeringmodule that implements the traffic engineering and fault-tolerance algorithms. The failover module is responsible foralleviating link congestion when failure happens and ensuringfast failure recovery.

OpenDaylight SAL [39] adds support for multiple south-bound protocols. For example, Off-the-shelf commodity eth-ernet switches are commonly allowed to be configured bySNMP, so that an ethernet switch can actively report its statusto the administrative computer (e.g., OpenDaylight controller)using SNMP trap. Therefore, an SNMP southbound plugin(SNMP4SDN) is proposed to control and unify underlyingdevices supporting SNMP by using off-the-shelf commodityethernet switch. This plugin provides capabilities to manageconfigurations that can only be accessed via CLI.

2) Only legacy switches in the network:In extreme cases, the network only contains controllers and

legacy switches. Under this circumstance, the controller canexert a certain degree of control over these legacy switches.

Centralized control over distributed routing: Unlike theway that the OpenFlow messages are converted to legacyswitch configurations through a hardware abstraction layer[40], Vissicchio et al. [41] implement a “Lie Definition Net-work” model, called Fibbing. The authors generate a falseaugmented topology in the data plane by passing false LSAmessages to the legacy switches, causing the switch to mis-

takenly identify some false switches and links. For the reasonthat real forwarding decisions are all determined by legacyswitches via the “tried and true” link state routing protocols.The key point is that the topology these switches find mayincludes fake nodes and fake links, so legacy switches can becontrolled by these fake destination addresses and fake linkweights. The fake routing generation process in the controlleris regarded as a mathematical function. Specifically, the inputparameters are these routing messages, the function is therouting protocols and algorithms in the network, and the outputis target FIB entries on legacy switches. Output and functionare given, the controller should automatically compute theinput parameters. The main process of the program is shown inFig. 5. After the generation of augmented topology, researchersadded optimization steps to reduce the size of the topology.Fibbing is similar conceptually to Telekinesis [29]. The biggestdifference between Telekinesis and Fibbing is that Telekinesisfocuses on hybrid SDN networks where legacy switches andOpenFlow switches coexist, while Fibbing is defined in alayer-3 legacy network.

Fig. 5. The four-staged Fibbing workflow [41].

Vissicchio et al. [42] further increase the availability andscalability, add support for back-up links, and define anefficient demand expression language that supports high-levelforwarding requirements. After operators enter the networkdemand, the controller automatically generates (or manuallyentered) the desired forwarding map. Then, the augmentedpath calculation module will design the extended topologybased on the input parameters within milliseconds. Under thepremise of retaining the forwarding paths, the module willfurther reduce the augmented topology because the originalaugmented topology can be very large. Finally, leveragingthe Forwarding Address field of OSPF messages, the con-troller turns fake configuration into actual routing messagesand injects them into the hybrid network. The framework isimplemented in the Cisco and Juniper routers.

The direct control of legacy switches: The controller canachieve the direct control of the legacy switches to somedegree. The establishment of a control system that basicallymeets the OpenFlow standard requires at least four basicattributes: i) the connection between the data plane and thecontrol plane, ii) the controller can discover the underlyingtopology, iii) the controller can send instructions to the switch,and iv) the switches are able to send packet-in messages. Handet al. [43] propose a hybrid framework, named ClosedFlow,that targets to meet corresponding requirements by: i) estab-lishing independent VLANs to establish a channel betweenthe two planes; ii) sending remote log records from switchesto the controller, which enables the controller to receive and

store the topology status; iii) achieving an SDN-like controlwith routeMaps or access-control lists; iv) if a packet doesnot match any rules, the switch will send a metadata tothe controller or just forward entire packet to controller.This solution is relatively simple and intuitive compared toFIBBING’s “spoofing” method, but it requires the switch tosupport layer 3 protocols and be preconfigured. ClosedFlowleads to a relatively low forwarding efficiency, which is notsuitable for large-scale networks.

B. Inter-domain deployment

Inter-domain hybrid SDN deployment solutions enhancethe cooperation capability of SDN domains in the large-scale network, and ensure the connectivity between differentdomains that based on different protocols. These solutions notonly indirectly improve the efficiency of inter-domain routing,but also realize the innovation of routing services runningacross domains, most of which can be considered as long-term use cases.

1) Deployment and management solutions:In this section, we focus on the interworking and manage-

ment between SDN and traditional BGP domains, and discussRouteFlow [44] [45] framework that enables remote IP routingservices in a centralized way.

Seamless interworking between SDN and BGP domains:Lin et al. [46] implement SDN-IP, which adopts the new SDNdevice to realize the interconnection between SDN domainsand traditional BGP domains. In the SDN domain, there areno legacy routers while some specific SDN switches act asBGP peers. To achieve peering, a BGP process is integratedin the network operating system (NOS) for the SDN AS,it exchanges routing updates and establishes the connectionwith BGP peers on the external IP network. The result isthat the SDN AS can be regarded as a single router in thewhole network. More specifically, the BGP process modulegenerates BGP route updates, the BGP route module willsynchronize these updates and store them in a local routeinformation base. The proactive flow installer in the frameworkis responsible for calculating and installing the flow entries forInter-domain traffic by using routes learned through BGP. Theinterconnection mechanisms between different domains can befurther used in Software Defined Network Switching Center(SDX) [47], which allows the operators of participating ASesto deploy novel applications.

In a real scenario, border routers usually have a goodperformance with high price, so operators may be reluctant todiscard them directly. Thus, Lin et al. [48] propose a practicalframework, called BTSDN, which retains the border BGProuters. As shown in Fig. 6, in BTSDN, the BGP still worksthe same as in current Internet. The only difference is thatthe controller also runs IBGP protocol and acts as an IBGProuter to learn the global inter-domain routing information. Inorder to synchronize routing information and set up a full meshnetwork topology, border routers in inter-domain run ExternalBorder Gateway Protocol (EBGP) and border routers in intra-domain run Internal Border Gateway Protocol (IBGP). Quagga

[49], which is a routing software package that can provideTCP/IP routing services, talks to all the border routers. Theremaining features are similar to those of SDN-IP [46].

AS A

AS B

AS C

Router 4

Router 1

Router 2 Router 3

Controller

Quagga

Router 5

EBGP

EBGP EBGP

IBGP

Fig. 6. A hybrid SDN network with BT-SDN framework [48].

The hybrid interdomain management layer: The slice ofSDN is usually considered as vertical slicing, and the SDNnetwork slices only isolate the network traffic between con-trollers while the hypervisor can see all nodes. When hosts andswitches belonging to separate entities are integrated together,it becomes a security and privacy problem because all elementsare visible to and controlled by the unknown hypervisor. Tosolve the problem, Thai et al. [50] propose an InterdomainManagement Layer (IML) that is compatible with hybrid SDNnetworks. It allows network elements to be integrated togetherwithout revealing their exact topology and devices attributes.Resources are shared and AS boundaries are managed by thehypervisor through the tools provided by the hybrid policycontrol controller. IML has the following features: i) thesupport for flexible centralized policy configuration interface,ii) the preserve of end-to-end packet control of the SDNarchitecture, and iii) the support for hybrid SDN networks. InIML, SDN ASes are compatible with BGP-dependent ASes.Fig. 7 shows the basic components of the IML architecturein a hybrid SDN network which includes two SDN ASes andone traditional BGP AS. The proxy and bridge work togetherto allow an SDN AS to setup flows in peering ASes, andthe policy controller is designed to avoid the interdomainpolicy misconfiguration. BGP aggregator intercepts any eBGPmessages being sent to a controller from a border SDN switch.

FlowVisor

Other

ControllersBridge

Routing

Controller

Policy

Controller

BGP

AggregatorProxy

FlowVisor

Other

ControllersBridge

Routing

Controller

Policy

Controller

BGP

AggregatorProxy

AS1（SDN） AS2（SDN）

AS3（BGP）

Fig. 7. Interdomain Management Layer [50].

Virtualized IP routing services on OpenFlow-enabled

hardware: In some cases, operators may expect thatOpenFlow-enabled nodes are aggregated as one single virtualrouter and exchange routes with traditional network devices.Nascimento et al. [44] [45] propose RouteFlow frameworkthat is ideal for combining SDN switches with virtualizationservice. The RouteFlow framework is designed to providevirtualized IP routing services on OpenFlow-enabled devices,which builds a virtual L3 topology by mapping all OpenFlow-enabled switches to a Virtual Machine (VM) with a routingengine. The architecture consists of a slave daemon RF-slaved running on each VM, a routing engine RF-server anda controller which runs the route-flow daemon. The RF-server communicates with the VM through the RF-protocoland calculates the corresponding flow-mod commands. Thecontroller uses these flow commands to configure the physicalforwarding plane through OpenFlow. When the virtual routerinteracts with a traditional Layer 3 switch, messages initializedin the VM are passed to the physical OpenFlow data plane bythe RF-server and the connected controller, respectively. Onthe contrary, routing messages from physical OpenFlow dataplane is converted and transmitted to the VM by the controllerand the RF-server.

RouteFlow creates a simulation framework that duplicatesthe physical network to the controller. Based on the work,Stringer et al. [51] create a distributed virtual router, namedCARDIGAN, which adopts distributed protocols for intercon-nection. On the one hand, the simplified network structureprovides shorter maintenance time and reduces the likelihoodof misconfiguration. On the other hand, due to the tightcoupling in the whole system, it will inevitably bring delaysand corruption when installing a large number of rules. Futureresearch could map more features in the traditional networkto the underlying SDN architecture, which has good prospectsin QoS, load balancing, and failover.

2) Convergence time and performance analysis:Routing between domains is usually realized in a distributed

way through BGP. Despite its global adoption, BGP hasseveral shortcomings, such as slow convergence after routingchanges, which may cause packet losses and even interruptthe communication. In this section, we review some worksthat focus on the inter-domain routing performance.

Caesar et al. [52] implement a Routing Control Platform(RCP) in traditional networks. RCP collects information aboutthe external devices and internal topology, which can beutilized to select the BGP routes for each router in an AS.The centralized idea is considered as a modification of thetraditional network. Integrating SDN to inter-domain routingcan indeed improve the performance of BGP.

In order to solve the problems about: i) how much conver-gence time can be reduced by using the inter-domain hybridmethod; ii) how many SDN domains are needed, and iii)how these domains need to be arranged in order to achievemaximum revenue, Sermpezis et al. [53] establish an inter-domain SDN model and propose a probabilistic approach thattakes various parameters (i.e., topology, path, number of SDNswitches) in the network as input. Based on the hybrid model,

the authors derive upper and lower bounds for the time neededto achieve data-plane connectivity between two ASes, andexact expressions and approximations for the time till control-plane convergence over the entire network. For the purposeof minimizing the convergence time of the entire network,the model can be further utilized to evaluate the deploymentof inter-domain hybrid SDN network or as an evaluationparameter for selecting the switches to join the SDN domain.

To show the difference intuitively, Gmperli et al. [54] con-struct a hybrid network simulation tool with multi-autonomousdomains to study the running state of the hybrid SDN network.A multi-AS routing controller is implemented to outsourcenetwork functionality to external service providers, which isdesigned to address the slow convergence of BGP. POX [55]is used for the interaction with the OpenFlow switch cluster,and ExaBGP [56] is used to interface with external BGProuters. The controller maintains Switch Graph and AS graphfor representing the core state. The results show that evenat a very low penetration level of SDN switches, the inter-domain routing centralization can also dramatically shortenthe convergence time. However, within a small-scale network,churn rates may slightly worse than the pure BGP network.

C. Topology discovery

Topology discovery is a critical service provided by thecontroller, it is the basis for the normal operation of thenetwork [57]. As for layer 2 discovery protocols, LLDP isalways used in a pure OpenFlow network. In SDN network,the controller periodically commands SDN switches to floodLLDP messages, and SDN switches will forward them backto the controller as soon as they receive these messages.While in a hybrid OpenFlow network with traditional switches,traditional switches will drop these LLDP packets floodedby SDN switches. In this case, LLDP-based links discoverymechanism is not applicable and LLDP+BDDP (BroadcastDomain Discovery Protocol) is a specific solution for discov-ering multi-hop links in a hybrid OpenFlow network [58].

The broadcast address in the destination field of BDDPmessages helps legacy switches to forward BDDP messages,which is adopted to find multi-hop links between OpenFlowswitches. First, by encapsulating BDDP in the packet-out mes-sage, the controller sends BDDP messages to each OpenFlowswitches. Then, if corresponding OpenFlow switch receivethe packet-out message, it sends the BDDP message to thedirectly attached switches. Moreover, if a traditional switchreceives the BDDP message, it matches the destination MACaddress and floods this message to other active ports. Finally,an OpenFlow switch will receive the BDDP message, andforward it as a packet-in message to the controller.

As for layer 3 discovery protocols, IGP can be used todiscover the interconnection between different devices. Forexample, SDN switches can intercept the OSPF link-stateadvertisement messages flooded by legacy routing protocoland forward it to the controller via a packet-in message.The controller should be extended to parse LSAs to topologyinformation and detect links between legacy devices. Based on

this principle, Hong et al. [38] implement a global topologymodule in HP Virtual Application Networks SDN controller[59]. The authors utilize OSPF Hello message to detect linksbetween SDN switches and legacy switches.

As for application layer protocols, OpenDaylight adds sup-port for SNMP [39], and we have discussed some BGP-basedsolutions in section III-B.

IV. DEPLOYMENT SOLUTIONS IN DATA PLANE

In general, operators expect the controller just focuses onthe original services while trying to avoid perceiving changesin the underlying network. Through the implementation ofthe hybrid data plane, we avoid the extra complexity in thecontrol plane and give the controller complete control over theunderlying network. In this section, we mainly concentrate on:i) hardware abstraction layer, ii) hybrid southbound interface,iii) hybrid IP/SDN node, and iv) hybrid SDN networks inmobile networks.

A. Hardware abstraction layer

When discussing the deployment solutions in the data plane,researchers aim to make little modifications in the controlplane, while make it possible for the service running in thecontroller cannot feel the difference caused by the underlyingnetworks. By adding an “overlay” or a “hardware abstractionlayer” between the control plane and data plane, operatorscould extend their SDN-enabled services to legacy infrastruc-ture, or seamlessly convert non-OpenFlow capable devices intomodern OpenFlow switches.

1) HAL architecture and features:Hardware Abstraction Layer (HAL) defined in [40] is used

to adjust different programmable network platforms. HAL islocated between the OpenFlow controller and non-OpenFlowswitches, which can be further extended according to differentservices. Through the management of HAL, the controllercan manipulate all elements in the data plane. Belter etal. [60] implement HAL in different hardware groups (e.g.,programmable network processors and point to multi-pointequipment) to enable OpenFlow framework on different typesof network equipment.

HAL mainly includes two layers, which are the Cross-Hardware Platform Layer (CHPL) and the Hardware-SpecificLayer (HSL). CHPL is in charge of node abstraction, virtual-ization, and configuration. HSL is responsible for performingall required configurations for different hardware platformsvia the hardware-specific modules. According to the type ofnetwork devices, the abstract forwarding API and the hardwarepipeline API are used for the communication between thetwo sublayers. This approach is similar to some programminglanguage design ideas (e.g., Java), which is divided intoplatform-independent and platform-related levels.

Fig. 8 shows the HAL architecture. CHPL is mainly com-posed of OpenFlow components and virtualization compo-nents. The OpenFlow component establishes a connectionwith the upper layer controller and receives/sends OpenFlowmessages. Virtualization components provide platform support

Network Device(s)

Cross-Hardware Platform Layer

Network Control Network Management

Hardware Specific Layer

OpenFlow

Virtualization

Translation

Orchestration

Discovery

Fig. 8. Hardware Abstraction Layer (HAL) architecture [40].

for virtualization and streaming-based partitioning, and enabledifferent controllers to control different areas using differentversions of the OpenFlow protocol. The operation of CHPLis manipulated by the network management system (NMS)to which the CHPL component is connected. HSL is mainlycomposed of network discovery module, rule translation mod-ule, and orchestration module. The network discovery moduleobtains information about a series of underlying devices,including the number of devices, models, features, and the un-derlying topology. The rule translation module is responsiblefor converting the control actions from the upper layer to therules supported by the underlying device. The orchestrationmodule sends the underlying network configurations to thecontroller to help it complete the initialization work and fixthe underlying physical network failures.

2) HAL extension and comparison:In addition to HAL, some works have achieved the hybrid

SDN network by introducing an additional level of abstraction.Farias et al. [61] create a path to config OpenFlow operation

and forward the configuration from the controller to legacydevices. The LegacyFlow datapath is the main feature ofthe proposal, which establishes a path between the controllerand the legacy network. When the controller sends the ac-tion messages and configurations by secure channel usingOpenFlow protocol to the LegacyFlow datapath, the actionmessage is processed, and the information of the flow (i.e.,Port in, VLAN ID, Ethertype) are extracted from OpenFlowmessage and finally sent to the switch control server. Theserver will build a correct setting according to the vendor ofswitches. This implementation is scalable, when new brandsof switches join the network, operators only need to installadapters of corresponding switches. However, when a legacyswitch receives a packet without specific rules, the switch cannot pass necessary information to the controller. As a result,these packets can only be forwarded by legacy rules.

Casey et al. [23] create a plug-and-play and simple hardwaredevice, named “SDN Shim”, which realizes flow-level controlfrom a legacy switch. This shim device provides SDN-like fea-tures on legacy switches to enable pre-sales testing and cost-

effective infrastructure upgrade planning. The device presentsitself to the controller as a regular OpenFlow switch, and itmanages flows on the connected legacy switch in accordancewith messages from the controller. In this solution, legacyswitches should support VLAN tags, so each port can reside onits own VLAN. These legacy switches will pass the receivedmessage to the shim device without forwarding it immediately.After the “shim” device receives the unsolved packet, it sendsback the result to the legacy switch via VLAN according tothe already stored rules or uploads it to the controller via anOpenFlow packet-in message. The design is implemented ona commercial development board. Because of the constraintthat a single shim device is hard to handle traffic on all theother ports at the same time, this solution is applicable forlow-traffic environments.

Szalay et al. [62] present a VLAN-based hybrid framework,named HARMLESS. In the framework, each received packetwill be tagged by the legacy switches with a unique VLAN idto identify the access port. Then, packets will be forwarded tothe OpenFlow translator component which acts as a softwareswitch. After that, the software switch outputs the packetsto OpenFlow switch that is managed by the controller. Afterbeing processed by the controller, these packets tagged with adifferent VLAN id will be sent back to these legacy switches.

HAL framework could be extended to study the differentspeed of reconfiguration between legacy and SDN devices, aswell as how those differences constrain the reality reconfig-urability of the network. Sieber et al. [63] first present themeasurement result that the difference in reconfiguration timebetween SDN and legacy devices can be a hundredfold. Then,the authors utilize the queuing theory to quantify and comparethe maximum reconfiguration rate of the whole network basedon the ratio of SDN and legacy devices. Finally, an intuitivemetric is proposed to compare different network topologiesin terms of their suitability for SDN deployment. In thiswork, the orchestrator adopts Network Services AbstractionLayer (NSAL) [64] to query the network topology and trig-ger reconfigurations. NSAL is a vendor and device-neutralabstraction layer, the reconfigurations are translated to device-specific configuration commands and placed in the queues ofthe devices. The simulation results proved that even a smallnumber of inflexible legacy devices can severely reduce themaximum reconfiguration rate of the entire network.

Feng et al. [65] deploy an intra-AS Source Address Val-idation (SAV) protocol which is used to prevent spoofingattack in the China education and research network 2. Basedon a common commercial router, the authors design andimplement an OpenFlow-enable open router, called Open-Router. OpenRouter adds lightweight control layer modulesto set up a datapath and send sampling packets to an externalOpenFlow controller. In this solution, the OpenFlow protocolis extended for routing notification and packets sampling. Afterprocessing, the OpenRouter receives the extended OpenFlowmessage from the controller. Finally, the control layer moduleanalyzes this message and converts it into a command thatsupported by the underlying forwarding module.

B. Hybrid southbound interface

SDN southbound interface has a variety of standards[16], typically OpenFlow. OpenFlow1.1 defines OpenFlowhybrid switches with both OpenFlow and normal Layer 2switch functionality. The OpenFlow specification support-s “OpenFlow-hybrid mode” since OpenFlow1.3. It definestwo classifications of switches, which are OpenFlow-onlyswitches and OpenFlow-hybrid switches. Forwarding decision-s on OpenFlow-only switches are completely determined bythe controller, while OpenFlow-hybrid switches support bothOpenFlow and legacy protocols (e.g., spanning-tree, OSPF)via a traditional networking pipeline. The OpenFlow1.3 spec-ification adds support for “OpenFlow-hybrid mode” via theproposed “NORMAL port” action, which instructs switchesto forward the packet based on the traditional networkingpipelines. The drawback of this approach is obvious, theseprotocols are still for switches that at least support OpenFlow.

For making use of existing hardware where possible, theInternet Engineering Task Force (IETF) defines another set ofsouthbound interface “I2RS” [67]. The purpose of I2RS is tointegrate routing based on traffic, policy, application, time cost,network status and external events, while taking full advantageof the existing software. In the existing technology, the mostcommon way to implement I2RS is to implement an agent inthe user space of the operating system that is installed by therouting device. I2RS agent can communicate with one or moreI2RS clients running on the application. It gathers informationabout the user and kernel space in the routing operating systemand then forwards it to the external SDN controller.

Due to the heterogeneity of a hybrid SDN network, whendeploying an SDN switch, manual configuration may introducethe risk of human errors and extra operational costs. Katiyar etal. [68] focus on automating SDN switch installation solutions.They propose a DHCP-SDN protocol as the extension ofDHCP to provide the configuration of new SDN switches.The authors first observe that in a hybrid network, it is notalways possible to ensure the reachability between the newSDN switches and the original controller. Then, the proposedSwitch Locator will locate SDN switch in the network and theintermediate switches will be configured by the IntermediateSwitch Configurator to connect with the new added SDNswitch. Finally, the extended DHCP server can react to theDHCP discover message and configure the newly introducedSDN switches by DHCP-SDN.

C. Hybrid IP/SDN node

Researchers can implement a SB hSDN network by addingsupport for legacy networks in some SDN switches.

Open Source Hybrid IP/SDN (OSHI) [69] node includesan SDN Capable Switch (Open vSwitch (OvS) [96]), an IPforwarding engine (Linux kernel IP networking) and an IProuting daemon (Quagga [49]). The IP forwarding engine isconnected to a set of virtual ports of the SDN Capable Switch(SCS), and the SCS connects to the physical network interfacethat belongs to a hybrid SDN network. The SCS and the IPforwarding engine are connected by the internal virtual ports

TABLE IICOMPARISON OF RELATED WORK BY DEPLOYMENT WAYS, PARADIGM MODEL [1], SWITCH, USE CASE, EXPERIMENTAL APPROACH, KEY IDEA

Related work Deployment Legacyswitch

Model Switch Use Case ExperimentalApproach

Key Idea

Cheng et al.[29] [30]

CP Intra C TB hSDN Legacy and SDN Internediate step, Long-term plan

Simulation,Testbed

Magnet address,MAC learning

Levin et al. [32] CP Intra U TB hSDN Legacy and SDN Internediate step, Long-term plan

Simulation VLAN

Casado et al.[33]

CP+DP U SB hSDN SDN or (Legacyand SDN)

Long-term plan Guideline

Lu et al. [31] CP Intra C TB hSDN Legacy and SDN NFV Simulation,Testbed

VLAN, Virtual Links

Hong et al. [38] CP Intra U TB hSDN Legacy and SDN Traffic engineering Simulation Extra algorithmCaria et al. [36][37]

CP Intra U TB hSDN Legacy and SDN Traffic engineering, In-ternediate step

Mathematicalsimulation

Partitioning

Vissicchio et al.[41] [42]

CP Intra C IntegratedhSDN

Legacy Traffic engineering, Long-term plan

Mathematicalsimulation

Augmenting topology

Hand et al. [43] CP Intra C IntegratedhSDN

Legacy Traffic engineering, In-ternediate step

Simulation Remote logging,VLAN

Lin et al [46][48]

CP Inter TB hSDN SDN(SDN-IP)/Legacy andSDN(BTSDN)

Traffic engineering, Long-term plan

Simulation OSPF, Converter

Thai et al. [50] CP Inter TB hSDN SDN Isolation Simulation Proxy, Horizontal s-licing

Vidal et al. [44][45]

CP+DP In-ter

SDN Virtualization,CARDIGAN [51]

Simulation Quagga, OpenFlow

Schlinker et al[66]

EmulationPlatform

TB hSDN Legacy and SDN Emulation low-level API Emulation Quagga, OpenFlow

Gmperli et al.[54]

EmulationPlatform

TB hSDN Legacy and SDN Emulation high-level API,Test convergence time

Emulation Quagga, OpenFlow,ExaBGP

Parniewicz etal. [40] [60]

DP C IntegratedhSDN

Legacy and SDN Europe Linking Infras-tructure and Applications(OFELIA)

Commercialuse case

Hardware AbstractionLayer

Farias et al.[61]

DP Intra C SB hSDN Legacy and SDN Traffic engineering Simulation VLAN

Casey et al.[23]

DP Intra C IntegratedhSDN

Legacy Traffic engineering Simulation VLAN, Shim device

Szalay et al.[62]

DP Intra C IntegratedhSDN

Legacy Traffic engineering Simulation VLAN,

Sieber et al.[63] [64]

DP Intra C IntegratedhSDN

Legacy and SDN Traffic engineering, QoS,monitor

Simulation Panoption, NSAL,Queuing theory

Tao et al. [65] CP+DP In-tra

C IntegratedhSDN

Legacy and SDN Intra-AS source addressvalidation (SAV) protocol

Simulation OpenRouter(Abstraction layer)

Hares et al. [67] DP C IntegratedhSDN

Legacy A new southbound inter-face

Routing System(I2RS) protocol

Katiyar et al.[68]

DP Intra U TB hSDN Legacy and SDN The configuration new S-DN switches

Simulation DHCP-SDN protocol

Salsano et al.[69]

DP Intra C CB hSDN OSHI node Flexibly configure, Backup, Traffic engineering

SimulationCommercial

Quagga, OvS

Sharma et al.[70]

CP+DP In-tra

U SDN SDN Network management andcontrol system

Simulation VLANs

Xu et al. [71] CP+DP In-tra

C CB hSDN Legacy and SDN Traffic engineering Numerical e-valuation

Hybrid path

Poularakis et al.[72]

DP C CB hSDN Smartphone Traffic engineering Prototypeimplementa-tion

OvS

DP: Control plane deployment solution CP: Data plane deployment solutionIntra: Intra-domain deployment Inter:Inter-domain deploymentC(Controllable): Legacy switch can be controlled by controller. U(Uncontrollable): Legacy switch cannot be controlled by controller.

, which are implemented through the virtual port module inOvS. For the purpose that the IP routing engine only needsto calculate the routes according to the virtual ports withoutconsidering the physical ports, each physical port is connectedto a corresponding virtual port of the network. The SCSclassifies regular IP packets and fine-grained SDN-controlledpackets. So that regular IP packets will be transmitted fromthe physical ports to the virtual ports, and these packets can beprocessed by the IP forwarding engine in IP routing daemon.In this solution, controllers and switches do not have totranslate the IP routing table into SDN rules. The drawback isthat the packet to be forwarded by IP routing will traverse theSCS switch twice, which may bring performance degradation.

Sharma et al. [70] propose an integrated network man-agement and control system (iNMCS), and validate it byimplementing four novel management use cases. iNMCS com-bines several legacy network management functions whichimplies that traditional network management tools and newSDN controllers can interact and operate on the same network.

In this architecture, policy manager provides the interfacefor specifying network requirements and the control decisionengine translates the policies specified by the network operatorto various OpenFlow-based actions. The flows that need toadhere to service requirements are forwarded to the controllerby the hybrid IP/SDN switches, whereas other flows are stillhandled by legacy protocols.

Hybrid switch that includes legacy functions (non-OpenFlow enabled switches) and an OpenFlow compatibledata plane is now commercially available, called Dual Switch.These switches are OpenFlow hybrid switches that can makedata plane forwarding decisions independently of the con-troller. For example, Huawei installs the Huawei EnterpriseNetwork Processor (ENP) chip on their switches to supportall the network protocols and SDNs and provide a large-sized OpenFlow table [28]. Accordingly, Huawei designs aSmart Network OpenFlow Controller (SOX) for controllinghybrid SDN networks. However, Dual Switch is a simplecombination of legacy switch and SDN switch, where SDN

TABLE IIICOMPARISON OF OPTIMIZATION AND DEPLOYMENT STRATEGY

Related work Target Selection ForwardingBehavior

Budget Key Idea(Solution)

Agarwal et al. [73] Minimize the maximum link utilization YES Only SDN NO Fully Polynomial Time ApproximationSchem (FPTAS)

Guo et al. [74] Minimize the maximum link utilization YES SDN andLegacy

NO Change the link weight of the currentnetwork

Wang et al. [75] Minimize the maximum link utilization and en-hance the controllability

YES SDN andLegacy

NO Fully Polynomial Time ApproximationSchem (FPTAS)

He et al. [76] Minimize the maximum link utilization NO Only SDN NO Linear programming (LP) problemWang et al. [77] Minimize the maximum link utilization NO Only SDN NO Linear programming (LP), Heuristic al-

gorithmDas et al. [78] Maximize the total number of alternative path YES YES Select the key nodeGuo et al. [79] [80] Minimize the maximum link utilization YES SDN and

LegacyNO Genetic search algorithm, K-means al-

gorithmXu et al. [81] [82] Maximize the throughput YES Only SDN YES Depth-first-search, Randomized round-

ingWang et al. [83] Find minimum-power network subsets in partial-

ly deployed SDN)NO Only SDN NO Create spanning trees for subsets of

nodesWei et al. [84] Turn off the idle links without traffic flow NO SDN and

LegacyNO Neighboring region search

Jia et al. [85] Turn off the idle links without traffic flow NO SDN andLegacy

NO MPLS, OvS

Hu et al. [86] Maximize controllable traffic YES Only SDN NO Fully Polynomial Time ApproximationSchem (FPTAS)

Cheng et al. [87] Select the waypoint for each flow and minimizethe maximum link utilization

YES SDN andLegacy

NO Mixed integer programming (MIP)model

Jia et al. [14] Maximize network control ability YES Only SDN YES Weighted Set Cover and MinimumWeighted Vertex Cover problem

Kar et al. [15] Bring more coverage benefit with minimum de-ployment cost

YES Only SDN YES Pcoverage and Hcoverage

Poularakis et al.[88]

Maximize controllable traffic and maximize thenumber of dynamically selectable routing paths

YES Only SDN YES Submodular and supermodular func-tions

Vissicchio et al.[89] [90]

Safe update of hybrid SDN networks NO Only SDN NO A generic control-plane model [91]

Amin et al. [92] Auto-Configuration of ACL Policy in Case ofTopology Change

NO SDN andLegacy

NO Fully Polynomial Time ApproximationSchema (FPTAS)

Chu et al. [93] Single link failure recovery YES SDN andLegacy

YES Heuristic algorithm

Markovitch et al.[94]

Fast failover YES Only SDN NO Spanning Tree Protocol (STP), BPDUs

Wang et al. [95] Mitigate Link Flooding Attack YES SDN NO TracerouteSelection: Whether select the appropriate location where the SDN switch should be deployed.Forwarding behavior: Forwarding behavior defines which type of switches the algorithm needs to control or adjust.

mode and legacy mode cannot take effect at the same time.Xu et al. [71] propose a way to implement a hybrid switch.When the switch receives a packet, it looks up the forwarding-table and switch/routing table in parallel. If a matching entrydoes not exists, the destination MAC address will be reportedto the controller. Most flows follow the traditional paths,while large flows may be redirected by the controller. Theframework decreases the overhead of TCAM resources byreducing the number of forwarding rules, which is moreefficient than adopting wildcard rules or redesigning a noveldatapath architecture in an SDN network [97].

D. Hybrid SDN in mobile networksExisting mobile network protocols emphasize on the dis-

tributed deployment of network resources, while SDN frame-work concentrates on centralized control. Therefore, it is achallenge to apply SDN design into mobile networks.

Poularakis et al. [72] implement a hybrid SDN prototype ina smartphone. The authors propose two methods to integrateSDN and distributed control planes. The first way is thedynamic migration of control protocol by using a distributedrouting protocol “as a backup”. When the network changessuch as link or node failures, mobile devices can automat-ically change their forwarding behavior from OpenFlow todistributed routing protocols. The second way is the cluster-based hierarchical control. By allowing a set of nodes to worktogether as one cluster and determining routes independentlyof other nodes, the SDN controller can only focus on theguidance of routing from one cluster to another. The proposedsmartphone framework includes an OvS software [96] anda local software agent. With OvS, a smartphone becomes avirtual switch that is similar to an OpenFlow switch. The agentis designed to maintain the distributed protocol, synchronizenetwork states with other nodes, and calculate routing paths.

In Table II, we provide a summary of all mentioned de-ployment methods and their characteristic proposed for eachresearch work in hybrid SDN networks.

V. HYBRID SDN NETWORK DEPLOYMENT ANDOPTIMIZATION STRATEGY

When considering traffic engineering solutions, a hybridSDN network can adopt similar approaches proposed in apure SDN network. In this section, we focus on optimiza-tion solutions and related deployment algorithms in hybridSDN networks. In Table III, we provide a summary of theproblem/goal and the solution proposed for each researchwork. Based on this table, we will discuss related worksfrom the following aspects: the traffic engineering in hybridSDN networks (i.e., the optimization of link utilization andload balancing, saving network resources), the optimizationof network control capacity, and the protection of networksecurity. Fig. 9. gives an overview of these optimizationstrategies.

A. The traffic engineering in hybrid SDN networksThe global centralized control and programmability of the

network behavior provide effective features to support traffic

engineering in a pure SDN network. In a hybrid SDN network,it is possible to apply some comprehensive traffic engineeringsolutions because the splitting ratio of the flows on SDNswitches is arbitrary. Researchers need to choose appropriatemigration sequence to maximize the profits of centralizedcontrol, which includes various optimization algorithms. Thesealgorithms can be applied not only to the deployment of hybridnetworks, but also to other network optimization strategies.For example, the migration solutions could be extended to i)find appropriate locations for the middlebox in the traditionalnetwork to achieve fine-grained control over the network [98];ii) solve the AP placement problem in the wireless networksto meet the energy requirements [99]; iii) improve the resourceutilization in data centers by solving the VM replacementproblem [100].

1) The optimization of link utilization:In B4 network [3], by splitting flows among multiple paths,

the centralized traffic engineering service brings the link upto nearly 100% utilization. The hybrid network is exceptedto approach this utilization, so we summarize optimizationalgorithms according to different network models and topologyparameters.

The minimization of the maximum link utilization:Agarwal et al. [73] establish a hybrid SDN network modeland propose a related full polynomial time approximationalgorithm (FPTAS). The purpose of the study is to developan SDN deployment framework that can be used to minimizethe maximum utilization of the links with different trafficpatterns. For simplicity, the weight setting of links is fixed,the network topology and the locations of SDN switcheshave been determined in advance. The full-polynomial timeapproximation algorithm is superior to the traditional standardlinear programming in time and space complexity. In orderto obtain a better “given network state”, the authors usedrandom selection and greedy solutions to further determinethe deployment location of the SDN switches.

Guo et al. [74] focus on how to optimize the OSPF weightsetting to balance the traffic coming out of the legacy switches.The authors advocate that the splitting ratio of the SDNswitches and the weight setting of the links can both beadjusted at the same time. The legacy switches that adoptOSPF protocol will forward the traffic from the specific portdue to the change of the link weight. Therefore, for thegoal of minimizing the maximum utilization of all links, theauthors propose the SDN/OSPF Traffic Engineering (SOTE)algorithm. The algorithm dynamically changes the weight ofeach link and then calls the SDN node optimization functionto calculate utilization in each iteration. At the end of alliterations, the link utilization, the weights of each link, andthe behavior of SDN switches can be derived.

Caria et al. [78] propose a similar solution about changinglink weights. In their work, SDN switches divide the initialOSPF domain into sub-domains, and LSA altering is adoptedto change the routing between sub-domains. The authors ob-served that there are some limitations to the goal of minimizingthe maximum link utilization. For example, in case of a heavy

Fig. 9. The objectives of optimization strategies.

loaded link at the beginning, the optimization method may notyield a feasible solution. In their work, each link is associatedwith a cost based on its real utilization, and the objective ofthe proposed Integer Linear Programming (ILP) model is tominimize the total cost in the hybrid network. Comparing withSOTE, this evaluation strategy is more flexible than just focuson minimize the maximum utilization.

Wang et al. [75] propose a generic traffic engineeringapproach that complies with the forwarding characteristics andcapabilities of SDN and distributed routing. They are the firstto take into account the differences between legacy switches inorder to minimize the maximum link utilization. The designedapproach supports both traditional single path and multipathrouting protocols in legacy switches. More specifically, thisapproach considers the flow-level and packet-level multipathforwarding on legacy switches, and describes their restrictionson forwarding. The traffic engineering algorithm in this paperis similar to [73] and [74], which is also based on FPTAS. Theevaluation results confirm that, with 70% deployment of SDNnodes, the hybrid forwarding with traffic engineering couldachieve as much throughput as a full SDN.

Existing hybrid traffic engineering approaches primarilyfocus on changing traffic splitting weights, Wang et al. [77]detect that the effectiveness of traffic engineering in hybridSDN networks strongly depends on both the next-hops andtraffic splitting ratios on SDN switches. They are innovativein considering the next-hops construction. In the research, theauthors first construct forwarding graphs with consistent andpotentially high throughput for effective traffic engineering,while maintaining forwarding consistency. Then, a heuristicforwarding graph algorithm that constructs forwarding graphsfor flows is proposed to reduce the traffic engineering over-head. Finally, they calculate the traffic distribution based on

the forwarding graphs with linear programming. Experimentalresults show that the algorithm achieves higher throughput andbetter load balancing than other forwarding graph constructionmethod, especially during the early SDN upgrade period withless than 40% SDN deployment.

The traffic engineering in barrier mode: He et al. [76]are the first to propose traffic engineering in barrier mode,which is adopted to forward traditional traffic and SDN trafficin distinct overlay networks. In order to pass SDN trafficthrough legacy switches, a destination based forwarding andtraffic aggregation routing protocol is defined in this paper, sothe routing table in legacy switches is modified to supportthe programmable flow splitting. For the same purpose ofminimizing the maximum link utilization, the problem isformulated as an ILP problem that can be solved by CPLEX orfast algorithms with approximate guarantee. Barrier mode is aconservative development method, because traditional networkwill not be influenced by new techniques, while the roughisolation may lead to the potential underutilization.

The selection of migration sequence: The above solutionsoptimize the utilization of the network when the hybrid net-work is basically deployed and the location of SDN switchesare determined. Intuitively, if SDN switches are located at theedge of the network or only adjacent to few devices, thisswitch may only control and distribute little traffic. Hence,it is relatively difficult to bring more benefits to the entirenetwork, and it is essential to decide the best location forSDN deployment.

Caria et al. [11] propose a novel two-stage migrationscheduling algorithm to decide which switches should be re-placed to increase as many alternative paths as possible. In thefirst stage, the algorithm first analyzes the network topologyto find all paths that can be used for traffic engineering. For

these paths, the algorithm then identifies the switches thatmust be SDN-enabled. In the second stage, for the purposeof maximizing the number of alternative path throughoutthe migration periods, the authors present an ILP model todetermine the migration schedule. However, the model doesnot consider the traffic issues in the network.

Guo et al. [79] [80] combine the node selection strategywith the topology optimization algorithm to find the nodeupdate sequence that minimizes the maximum link utilizationof the network. In order to accommodate uncertain traffic, theauthors combine the off-line weight optimization for legacyswitches with the on-line splitting ratio optimization for SDNswitches. When considering migration sequence, if N nodesneed to be updated, there are N! update sequences. The searchis exhaustive if the brute force algorithm is used. Therefore, theauthors choose heuristic algorithms, that is, genetic algorithmsearching (GAS) and greedy algorithm searching (GDS), foroptimizing migration sequence. After each iteration, SOTE[74] algorithm is used to calculate the optimal link utilizationof the network at this time. The result is used as the criterionto change the node update sequence to dynamically improvethe network performance until the number of iterations isreached or the expected target is satisfied. Furthermore, inthe pre-processing phase of the experimental data, the authorscluster historical traffic matrixs (TM) by using the k-meansalgorithm, and then computing the weight coefficient of everyrepresentative traffic matrix. When the algorithm optimizesthe link weights, a larger weight means that its correspondingTM is more important. As a result, an expected TM can beobtained, which can display the average traffic in differenttraffic modes. This solution enables intra-domain routing tobe robust to the changing traffic demands.

2) The economization of network resources:The flexible control capability of the SDN switches enables

multiple links that are previously not available for forward-ing. However, the increase of available links may cause thepressure in network costs.

The analysis of deployment cost: The economic andbudgetary implications need to be taken into account whenseeking the best deployment and optimization options. Asmentioned in [11], when a key-node node is deployed asan SDN switch, it provides some paths that will candidatefor traffic engineering while generating different deploymentcosts. It could be considered as a 0 − 1 knapsack problem.However, this model is too simple for large-scale and complexmodels, and more candidate links do not necessarily bring thebest performance [82].

The reduction of energy consumption: For the purposeof reducing the unnecessary energy consumption, Wang et al.[83] aim to find minimum-power network subsets in a hybridSDN network. They advocate that the power consumption oflegacy devices cannot be altered and the controller can onlymanage the SDN switches. The power consumption of SDNswitches plus the link connected to it is equal to the totalpower consumption. On this basis, the authors develop a newspanning tree algorithm to select the lowest-cost link subset,

ensuring that each link does not exceed the load and thenetwork reachability is not destroyed. However, when the loadis too heavy, too many overload links are generated duringthe calculation of the spanning tree. Hence, the process ofadjusting these links is complicated, which results in delaysand packet loss problems. Similar algorithms exist in the studyof CDN networks, that attempt to shut down idle devicesduring off-peak hours [101].

Wei et al. [84] propose the hybrid energy-aware trafficengineering (HEATE) algorithm, which aims to reduce powerconsumption by determining the optimal setting for the OSPFlink weight and the splitting ratio of SDN switches. Theauthors assume that operators expect they could aggregatetraffic flow onto partial links and then turn off underutilizedlinks to save energy. The solution is similar to the heuristicalgorithm in SOTE [74], which tries to delete the minimum-utilization link in each iteration, and then move traffic fromlow-utilization links to high-utilization links.

Jia et al. propose a more viable energy saving solution[85]. In their work, the SDN switches reroute packets basedon multiple MPLS labels which take the forwarding portsof switches. The latest OpenFlow protocol supports MPLStechnology, the Push MPLS header actions can push newMPLS headers onto the packet. When a new MPLS tag ispushed onto an IP packet, it becomes the outermost MPLStag, and is inserted as a shim header immediately beforeany MPLS tags or immediately before the IP header. SDNswitches encapsulate multiple MPLS labels for each packetthat indicate the forwarding port numbers of switches. Energysaving requires fine-grained flow scheduling to shut downswitches and links. An SDN switch achieves fine-grainedflow scheduling by encapsulating the MPLS labels of theforwarding information. Thus, it can save energy by reroutingthe flows to turn off the idle links and switches.

B. The optimization of network control capacity

When the traffic in the hybrid SDN network passes throughan SDN switch, the controller may obtain the meta-datathrough the packet-in message. In general, the route of trafficis jointly decided by traditional distributed routing proto-col running at non-SDN routers and the SDN controller.Considering the factors that may affect the percentage ofcontrollable traffic, researchers mainly focus on the numberof SDN switches, the location and forwarding behavior ofthese switches, the topology and the link weights of the entirenetwork.

The maximum of controllable flow: In the case that thenumber and the location of SDN switches are determined inadvance, Hu et al. [86] aim to find the maximum flow that canbe controlled by only tuning the forwarding behaviors of theSDN devices. The authors formulate it as a linear optimizationproblem, and a full-polynomial time approximation algorithmis proposed in this paper. To ensure that the link does notexceed the maximum load, the SDN switches should directtraffic to the lower-load link while avoiding these trafficthrough other SDN switches. In each iteration, the algorithm

computes the shortest controllable path between SDN switchesand other nodes. The simulation result shows that when thenetwork includes half of the SDN switches, all traffic iscontrollable.

Operators may expect the controller to be able to control alltraffic in a hybrid SDN network, Cheng et al. [87] advocatethat each end-to-end flow can be forced to traverse at leastone SDN switch. In their work, the forwarding process foreach flow is divided into two parts: i) from the source nodeto a selected SDN switch, and ii) from this selected switchto the destination node. Under this constraint, the authorsformulate the waypoint forwarding model and propose the flowrouting and splitting (FRS) algorithm to find the maximumlink utilization. FRS heuristically computes a most promisingsubset of all the available paths, and jointly determiningan appropriate SDN switch as the waypoint for every flow.However, when the percentage of SDN switches is low, ifeach flow is still forced to traverse at least one SDN switch,it may lead to a high price, and the network performance iseven worse than traditional networks. The result shows that theinitial performance is poor compared to the method proposedin [74] that does not have the waypoint enforcement, while theperformance is better when the proportion of migrated nodesexceeds 20%.

The migration sequence and budget analysis: Similarly,the migration sequence and the budget limitation of SDNswitches are also important factors that may affect the pro-portion of network controllable traffic. Jia et al. [14] aim tomaximize the network control ability with limited budgets, andminimize the cost of migration while achieving full control ofthe network. The authors formulate it as the Weighted SetCover problem and Minimum Weighted Vertex Cover prob-lem, respectively, and propose a unified heuristic algorithm.In each iteration, the algorithm greedily selects the optimallocation based on the number of flows and the cost of SDNswitches while ensuring that the deployment cost is lower thanthe budget.

Kar et al. [15] refine the network coverage model andpropose two evaluation parameters, Pcoverage and Hcoverage.The definition of the former is as long as there is an SDNswitch in a path, the path is P-covered, then Pcoverage isdefined by the proportion of these paths in the entire network.If 20% of the switches in this path are SDN switches, then thePcoverage is 20%. Hcoverage refers to the percentage of SDNswitches within the P-covered path. The purpose of the study isto maximize the coverage with budget constraint or minimizethe budget with the coverage constraint. Two correspondingheuristic solutions, maximum number of uncovered path first(MUcPF) and maximum number of minimum hop coveredpath first (MMHcPF), are proposed in this paper. Comparedwith other algorithms, MUcPF requires 5% to 15% less budgetto achieve the assigned Hcoverage target. MMHcPF is aconsistent algorithm, the difference between the maximumHcoverage and the minimum Hcoverage in MMHcPF is only20-30%.

Essentially, strategies in [14] [15] only take the budget as

a simple constraint and do not refine cost models. Poularakiset al. [88] propose a refined and complex cost compositionfunction. Based on the general cost model, the authors focuson maximizing the programmable traffic that passes throughat least one SDN switch and maximizing the flexibility byincreasing the number of alternative paths. The theory ofsubmodule and supermodule is used to design the algorithmwith provable approximation ratios. The authors observe thatthe interplay between the two objectives in the experienceis that if one objective is optimized, another objective willnaturally obtain a benefit.

The above works mainly focus on the arbitrarily splittableflow routing in a hybrid network, which means the SDN switchcan split the flow arbitrarily. However, when the size of theflow table is significantly less than the number of flows, thedifficulty of flow table management will be increased due tothe assumption of arbitrarily splittable flow routing. Xu etal. [81] add the h-splittable (h ≥ 1) restriction in each flowand introduce a novel incremental SDN deployment scheme,named duplicated deployment. Duplicated deployment refersthat new SDN equipment is placed “in addition” while not“instead of”, each SDN switches will be collocated with onelegacy router. In this solution, anomalies in the SDN switchesor SDN controller will not disrupt the basic connections inthe traditional network. As for deployment strategies, theheuristic algorithm (MAX-k-SFD) is proposed to determinethe location of SDN devices under the budget constraint. Ineach iteration, the algorithm selects the location to maximizethe number of new controllable flows. After the deployment ofSDN devices, the depth-first-search and randomized roundingbased algorithm, named MRHS, is proposed to maximize thethroughput, which has approximation ratio O

(1

logN

).

For the same purpose of throughput maximization in du-plicated deployment scheme, Xu et al. [82] consider thebudget constraint that the amount of additional bandwidthand the number of SDN switches should be limited. In orderto re-route flows at an SDN switch, additional bandwidth isrequired on certain links. The authors formulate the problem ofthroughput maximization under budget constraints as a jointduplicated deployment and routing (DDR) problem, and anapproximation algorithm based on the traffic mapping andrandomized rounding methods is proposed. The algorithmfirst solves the relaxed DDR problem and get the fractionalsolution, then rounds it to an integer solution. It is proved thatthe approximation factor is O(log n) in the worst case and O(1)under most practical situations for link capacity and flow-tablesize constraints (n is the number of all switches).

C. The protection of network security

In this section, we discuss some issues that may be en-countered when considering the deployment of a hybrid SDNnetwork, including: safe updates of hybrid SDN networks, thedetection of network failures, the implement of traffic matrixand the avoidance of link flooding attack.

Safe updates of hybrid SDN networks: Because of thepotential conflicts between different control planes, updating a

hybrid network may lead to numerous forwarding inconsisten-cies. To update hybrid networks without losing traffic or vio-lating security policies, Vissicchio et al. [89] develop provablycorrect techniques that enable consistency in: i) the update ofthe SDN-controlled or the IGP-controlled forwarding paths, ii)the update when IGP-controlled flows become SDN-controlledor the SDN-controlled flows become IGP-controlled. Duringthe hybrid network update, the authors first prove that anyend-to-end connection can be guaranteed (e.g., black-holes andforwarding loops can always be avoided), while it is not alwayspossible to guarantee the path consistency (i.e., violatingsome security policies). Then, the Generic Path InconsistencyAvoider (GPIA) algorithm is introduced to compute the longestconsistent sequence of FIB replacements with no overhead.

Vissicchio et al. [90] [91] further extend the above methodand theoretically prove that the new method can be used intopology-based hybrid networks. The router is defined with amodel that is general enough to capture all kinds of controlplanes. This model is independent of the path calculationalgorithm used for forwarding and the header field used formatching packets. On this basis, the control plane is dividedinto FIB-aware (FA) and FIB-unaware (FU) based on whetherthe forwarding entry is based on FIB content or not. Then,the algorithm [89] is proved that it can be adopted in the FU-only control plane, which may be unnecessarily complicatedfor a strongly consistent FU update and cannot be adoptedin the FA-Existence network. Finally, the authors proposethat safe updates for generic networks can be achieved bycombining the replacement and duplication of FIB entries.The replacement phase consists of the GPIA algorithm thatmentioned before. The duplication phase refers to duplicatethe FIB entries that cannot be replaced without creating pathinconsistencies.

The detection of network failures: Any network mayencounter the link failure problem. When a link failure occurs,the legacy switches will automatically select the standby linkfor transmission in a traditional OSPF network. The channelbetween a traditional IP router and an SDN switch can beestablished in hybrid SDN network, so that when a legacyswitch detects a link failure, it can immediately redirect trafficto SDN switches. After that, SDN switches will identify whichlink is failed according to original and tunneled IP headers,and controllers can help these flows bypass the failed node orlink. The selected recovery paths will be installed as rules inthe SDN switches that are on the corresponding paths. Hence,a single link failure problem can be resolved.

Operators can find out the number of SDN switches theyneed when considering the single link failure problem in ahybrid SDN network. Chu et al. [93] formulate it as a binarylinear programming problem and solve it through a heuristicalgorithm with polynomial time complexity. Given the affectedrouter and the destination hosts, the router should forward thepacket to an SDN switch without using the failed link. For eachfailed link, the authors first find out all candidate locations forSDN switches. Then, the algorithm will periodically selectthe location according to the number of failures that can be

covered in each iteration. The algorithm ends when each linkfailure is covered. Furthermore, because of the global view ofthe controller, given the affected routers and destination nodes,the algorithm is extended to minimize the max link utilizationof the post-recovery network by choosing the optimal recoverypath. Some related work [14] [88] try to find more selectablerouting paths, which also make it possible to dynamicallyrespond to link failures or link congestion.

Markovitch et al. [94] propose a network architecture,named SHEAR. In this architecture, the partially deployedOpenFlow switches divide the network into multiple loop-freedomains. These switches located in loop-breaking locations areregarded as monitor points, which help the SHEAR controllerto quickly detect and locate failures and provide traffic-engineering flexibilities. STP spanning trees are used by thecontroller so that no link failures are ignored. Specifically,SDN switches are responsible for each network domain andreceive network updates through MSTP messages (BPDUs) .These updates will be forwarded to the controller so that thecontroller can locate the failure link and compute the affectedtraffic. The principle of the solution is that link failures willchange the value of a root in the BPDU, and the controller canlocalize the failure between the expected root and the currentroot within the domain. Finally, the controller will reroutethe traffic according to the spanning trees or just notify thenetwork operator.

There are frequent link changes and new device additionsin the network, and the network policies configured at theinterfaces of switches may be violated. Amin et al. [92]propose an approach that automatically detects the networkpolicies that are affected because of the topology changes,called Auto-PDTC. This approach simulates the network-wideand local policy at forwarding devices by using a three-tupleand a six-tuple, so that the controller can obtain link statusinformation from all switches and then chart it. In the case oftopology changes, the graph difference algorithm is used toauto-detect the changes, it constructs the search tree to verifypolicy violation either exist or not.

The implementation of traffic matrix: Traffic matrix iswidely used to monitor network status and prevent networkanomalies. A traffic matrix requires a significant amount ofmonitoring equipment and network-wide configuration efforts,which is not readily available in legacy IP networks. Whilein an SDN network, SDN-enabled devices provide additionalbyte counters for all individual entries in their forwardingtables. Inspired by this feature, Medina et al. [102] aimto augment the SDN-based traffic statistics with SNMP-based throughput measurements, obtain and measure flows bytemporarily offloading them on IP backup links. A backuplink in addition to a regular IP link is easy to create andconfigure, allowing the measurement by regular SNMP linkbyte counters, which is vendor-independent and available inalmost every router. More specifically, a separate physical porton a pair of IP routers is configured as a backup to an IP link.In addition, the framework defines a set of ACLs such thatthe flow in question can be distinguished from the remaining

traffic. As complementary, to minimize the total cost, theauthors propose a linear optimization model and a greedyheuristic algorithm to determine the optimal measurementlocations for SDN switches and backup links.

The avoidance of link flooding attack: DDoS attack suchas Link flooding attack (LFA) may degrade or even blocknetwork connectivity in the target area. The legitimate andlow-density traffic in LFA can hardly be distinguished intraditional networks. Wang et al. [95] present a frameworkthat can effectively mitigate LFA in hybrid SDN networks,named Woodpecker. After the optimal selection of upgradingswitches based on the benefits (the amount of controllabletraffic) of upgrading a certain switch, the key is to find outthe congestion link and determine if LFA is happening. Thedetection module in Woodpecker is implemented to find thecongested link. When the SDN switch finds that the trafficexceeds the threshold, an alarm message will be sent to thecontroller. The controller will install two flow-mod rules thatmatch different ICMP messages to SDN switches. Then thecontroller will inject ICMP packet to the SDN switch via apacket-out message. The SDN switch will match and forwardthe ICMP packet based on the normal rules, while the legacyswitch will forward the packet and decrease the TTL value orreturn the ICMP reply message according to the current TTLstatus. As soon as an SDN switch receives the ICMP replypacket, the packet will be sent back to the controller. Basedon the received ICMP reply message from different SDNswitches, the controller will locate the congested links. Afterthat, a traffic engineering algorithm that aims to minimizethe maximum utilization of all links is enforced to mitigatethis attack. Finally, if some traffic is too heavy to handle, thecontroller will instruct switches to discard some packets if theIP address of these packets always appears on congested links.

D. Experiments and simulations

Most of the optimization strategies first find the problemsthat can be optimized or urgently needed to be solved in thehybrid SDN network, then formulate the network optimizationproblem, and design optimization algorithms or heuristics tosolve the problem. Due to the special structure of hybridSDN networks, there is no simple network simulation toolsuch as Mininet (a popular simulation tool for pure SDN)[103]. Therefore, researchers have adopted different methodsto validate the correctness of assumptions and verify theefficiency of algorithms, as is described below.

Simulation-based performance measurement: Mostworks use numerical verification methods to prove the effec-tiveness of their algorithms. The commonly used implementa-tion process are to i) extract the traffic data set of the traditionalnetwork, and ii) put the data set into the algorithms to verifythe performance using statistical methods [73]. The data setused in these experiments is generally from website topology-zoo [104] or rocketfuel [105]. In the assumption of manyschemes, the SDN switch allocate traffic according to customerdesigned algorithms, while the traditional switch performs datapacket forwarding according to traditional routing protocols,

which is difficult to implement in real-life network. This iswhy most research works adopt the static simulation approachfor verification.

Real-life traffic-based performance measurement: Someworks do not assume the arbitrary allocation of traffic inSDN switches, hence use real-time traffic in their experiments[106]. In the experiments, Mininet [103] and SDN controllerare the main components. In the Mininet, legacy switchesare materialized as host nodes that run the Quagga software[49], while Open vSwitches act as SDN switches. The SDNcontroller is able to parse and respond to OSPF hello packetsreceived and forwarded by the OvS switches [96] (throughadequate OpenFlow rules installed in the SDN switches) andensure the correct functioning of the adjacent OSPF routers.

VI. USE CASES IN HYBRID SDN NETWORKS

Vissicchio et al. [1] define four kinds of hybrid SDN models(i.e., TB hSDN, SB hSDN, CB hSDN and Integrated hSDN)according to the network service and usage scenario, eachmodel has its potential transition use case and long-term designuse case. As the extension to the deployment methods andoptimization strategies that mentioned above, in this section,we summarize and analyze several representative applicationsand business cases related to hybrid SDN networks. Fig. 10.gives an overview of this section.

A. The interworking and inter-domain routing

In the early stage of SDN, the most famous applicationshould be Google’s B4 network [3] and Microsoft’s S WAN[4]. Specifically, Google selected SDN to transform the in-terconnected WAN network (G-scale Network) between thedata centers, and has been fully transitioned to the OpenFlownetwork. Before the full deployment, B4 network experiencedtwo hybrid deployment process. The first phase was completedin the spring of 2010, the OpenFlow switch was added to thenetwork. However, the OpenFlow switch is the same as anyother non-OpenFlow devices in the network, except that thenetwork protocol is managed by the controller. The wholenetwork was still like the traditional network. The secondphase was completed by 2011, Google increased the size ofthe network and began to use the controller to manage thenetwork, allowing the network to evolve to the SDN network.In fact, Google promoted the idea of agile development and putboth SDN and traditional routing systems running in parallel.SDN has a higher priority than traditional routing, so that SDNcan be gradually deployed to various data centers, allowingmore and more traffic to be transferred from traditional routesto SDN framework. At the same time, if there is a problemwith the SDN, the SDN framework in B4 can be turned off andreturn to the traditional routing approach. In this phase, SDNswitches are allowed to interact with traditional routing proto-cols, and Google implements corresponding routing protocolsas an SDN application. Even if the pure SDN framework isfully implemented, the data center is inevitably required toexchange information with external traditional networks.

Fig. 10. Use cases in Hybrid SDN Networks.

Internet Exchange Point (IXP) is defined as a physicalnetwork access point where different ISP can connect theirnetwork and exchange BGP routes through this point. TheSDX-L3 [47] is the abbreviation for Software Defined InternetExchange Point - Layer 3. The traditional physical IXP routingand traffic forwarding is based on the IP prefix. The SDXsupports rules that match multiple header fields, and each ASis allowed to adopt remote control over the traffic. Besides,the SDX integrates the virtual switch abstraction to ensure thatASes are not able to see or control interdomain routing outsideof their purview. Furthermore, there are some optimizationmodels that update rules as soon as a policy or BGP routechanges. SDX makes IXP more flexible and reliable.

B. The management of traditional devices

The advantages of the HAL are especially reflected in theSoftware-Defined Optical Network (SDON) [6]. Due to thepresence of the abstraction layer, it is possible to make the op-tical switches that do not support the SDN framework becomethose switches that can be controlled by the SDN controller[107] [108]. OpenFlow in Europe Linking Infrastructure andApplications (OFELIA) is one of the important applicationsin optical networks. In order to allow optical switches toget rid of the shortcomings of not supporting OpenFlow,researchers assign agents for these optical switches. Similarto the deployment of HAL, the agent can make a connectionwith the controller, collect the state of the underlying layerand connect to the optical switches in the traditional way.Based on the OpenFlow agent, Hybrid GMPLS-OpenFlow[109] solution and Pure Extended OpenFlow [110] solution arepresented in OFELIA. In the first solution, the standardized

GMPLS control plane is reused to offload the OpenFlowcontroller from the complexity of circuit switching. In thesecond solution, the OpenFlow agent is used to exchange theconfiguration with the network elements and SDN controllersthrough the management interface and the extended OpenFlowprotocol, respectively.

Hybrid SDN could be further utilized in the VNF and cloudcomputing service. HybNET [31] is suitable for virtualizednetwork management service. If the operator needs to applyfor a new VM, the configuration requirements along withthe user information will be passed from the API to Hybnet.Researchers integrate HybNET with OpenStack. Specifically,it works in term with Neutron (the network service managerof OpenStack) to provide the hybrid network managementfunction. Hybnet provides the tenants to modify, add, anddelete virtual machines as well as achieve network isolation.

Choi et al. [111] implement a hybrid middlebox, namedSoftware-defined Unified Monitoring Agent (SUMA). SUMA,as an intelligent switch-side inline middlebox, is located be-tween OpenFlow switches and controllers. It provides manage-ment abstraction between SDN controllers, traditional NMS,and SDN switches by collecting traffic statistics in the back-ground, monitoring network events, filtering and aggregatingincoming packets. SUMA reduces the monitoring overhead ofthe controller, and the authors believe that it can be deployedas an important component of an efficient SDN deployment.

C. Edge computing in hybrid SDNs

Edge computing is a way to simplify traffic from IoTdevices and provide real-time local data analysis. SDN con-centrates the network intelligence at the controllers, thus

avoiding edge devices performing complex network activities.Therefore, the control mechanism provided by SDNs canreduce the complexity of the edge computing architectures bybringing a novel approach to utilizing the available resourcesin a more efficient manner [112]. Hybrid SDN network canaccelerate the process of the complexity reduction, becausesome edge computing service (i.e., video streaming, intensivecomputation) deployed in pure SDN can also be partiallyimplemented in hybrid SDN network(cite). For example, amobile user sends a service request to one of the cloudlets inthe vicinity. Before the request is accomplished by the server,the user is authenticated to another network by changingits location. In hybrid SDN, the controller can track thismovement with its ability to discover the topology and getthe necessary information about the new location of the user,such as its recently assigned IP address. This allows serviceresponses to be reached to the user by adding new flow rulesto the switches on the path. During this entire process, the useris not aware of the operations occurring within the network,and the user experience is not interrupted.

D. The experimental platform and simulation tools

With a reliable a simulation platform, network operators canclone their network architecture into an emulated environmentand then estimate the impact of changes in the networkto its existing architecture. By installing Quagga [49] andrunning the corresponding routing protocol, operators coulduse a common PC simulation to support existing mainstreamrouting. Mininet [103] can simulate the network host, andsupport OpenFlow switches, controllers, links, suitable forsimple network topology simulation. These two tools are themost commonly used simulation tools in traditional networksand SDN networks. MiniNext [66] combines Quagga [49]with Mininet to implement a tool that can build a simplehybrid SDN network simulation platform. This platform cansimulate a hybrid network that includes traditional IGP andSDN technologies. In this way, even a laptop can simulatea hybrid SDN network with hundreds of nodes, and thesenodes can be interconnected with real-world networks. Un-like the large-scale hybrid network simulation tool [54] thatspecializes in the creation of network graph, the measurementof convergence time and loss rates, and the visualization ofrouting changes, MiniNext focuses on simulating the operatingenvironment and provides low-level APIs.

VII. CONCLUSIONS AND DISCUSSIONS

The purpose of this survey is to provide researchers who areactive in or interested in the field of hybrid SDN issues with anoverview of the state-of-the-art, including hybrid SDN models,deployment solutions, optimization strategies and different usecases. We pay special attention to control plane and dataplane deployment solutions as well as optimization strategiesthat aim to improve the network performance and ensureconsistency. We also summarize some common issues in thehybrid SDN network, including underlying protocols, topologydiscovery, and hybrid SDN models.

According to our understanding, there are some gray areaswhich need to be identified and properly addressed beforehybrid SDN networks are commercially deployed, which con-stitute several future research directions, as presented below.

1) Security issues in deployment solutions:Security is not considered as part of the initial design

while it must be built as part of the long-term hybrid SDNnetwork architecture. Researchers could pay more attention tomigrate some security solutions to the hybrid SDN networks.For example, SDN data plane configuration checkers suchas Anteater [113] and Header Space Analysis [114] can beextended to hybrid SDN networks, increasing the scalabilityof these deployment solutions.

2) Optimization strategies for real-life traffic:As for optimization strategies, most of the current optimiza-

tion algorithms do not fully consider the real-life situations.For example, the selection of traffic data sets does not takeinto account the impact of different time periods (peak andoff-peak hours), and the assumption of switch deploymentcosts is too simple. In the future, researchers can investigatecomplex budget models, add special constraints (i.e., someswitches must migrate or can not migrate), adapt to multiplenetwork environments, and adopt the neural network, MarkovApproximation algorithms or data mining methods to solvecomplicated and real-time optimization problems [80].

Based on these optimization strategies, operators can pro-vide some practical services in the future. For example, CDNis used to bring the content closer to the user to decreaselatency and maximize throughput. During this process, CDNproviders have to optimize the assignment of end-users andsurrogates according to the load information in the network[115]. The hybrid SDN framework can accelerate the assign-ment for CDN providers by utilizing its global view andprogrammable interfaces of the whole network. This is becausein hybrid SDN, the controller can redirect some traffic betweenclient and server, that is, redirect the traffic of a given flow toan arbitrary node.

Using the traffic engineering strategies, researchers canstudy how the SDN controller guides more traffic and ensuresload balancing. The CDN hybrid SDN service might need tofocus on the correctness of TCP socket migration and theeffective transfer of HTTP session in the complex networkenvironments.

3) Virtualization services in hybrid SDN networks:NFV is the best platform to reflect the commercial value

of SDN, and there have been many NFV projects on pureSDN. For example, Flowvisor [116], a network slice servicein pure SDN, enhances transparency and isolation betweennetwork slices by checking, rewriting, and managing Open-Flow messages as they pass through virtual network slices.Obviously, the AS domain controlled by hybrid SDN frame-works can also be part of these network slices. However, thecombination of multiple switches may result in the degradationof network performance. Therefore, in addition to consideringthe unifying of different switches in the data plane, and theimplementation of the extended Flowvisor controller in the

control plane, researchers also need to consider the impact ofisolation services on overall network flexibility.

4) Practical simulation tools in hybrid SDN networks:We discuss some simulation tools [53] [54] in Section

III, but these simulation tools can only be used to evaluatenetwork convergence time. We also summarize simulationsolutions for optimization strategies in Section IV, however,these solutions are only suitable for specific experimentalenvironments. Section V describes a common simulation toolfor hybrid SDN (e.g., MiniNext [66]). However, it can only beused to test the network connectivity. In order to obtain typicalperformance metrics to verify if a hybrid SDN is successfullydeployed, more practical simulation and emulation tools areexpected. The challenges that need to be solved include i) thesetools should obtain full control over all traditional switchessuch as SDN switches in the Mininet, and ii) evaluationcriteria among deployment strategies are different, thus, newsimulation tools need to provide reliable and unified data setsto adapt to various experimental environments.

REFERENCES

[1] S. Vissicchio, L. Vanbever, and O. Bonaventure, “Opportunities andresearch challenges of hybrid software defined networks,” ACM SIG-COMM Computer Communication Review, vol. 44, no. 2, pp. 70–75,2014.

[2] W. Xia, Y. Wen, C. H. Foh, D. Niyato, and H. Xie, “A surveyon software-defined networking,” IEEE Communications Surveys &Tutorials, vol. 17, no. 1, pp. 27–51, 2015.

[3] S. Jain, A. Kumar, S. Mandal, J. Ong, L. Poutievski, A. Singh,S. Venkata, J. Wanderer, J. Zhou, M. Zhu, et al., “B4: Experiencewith a globally-deployed software defined WAN,” ACM SIGCOMMComputer Communication Review, vol. 43, no. 4, pp. 3–14, 2013.

[4] C.-Y. Hong, S. Kandula, R. Mahajan, M. Zhang, V. Gill, M. Nanduri,and R. Wattenhofer, “Achieving high utilization with software-drivenWAN,” ACM SIGCOMM Computer Communication Review, vol. 43,no. 4, pp. 15–26, 2013.

[5] S. Natarajan, A. Ramaiah, and M. Mathen, “A software definedcloud-gateway automation system using OpenFlow,” in Proc. IEEEInternational Conference on Cloud Networking (CloudNet), pp. 219–226, IEEE, 2013.

[6] A. S. Thyagaturu, A. Mercian, M. P. McGarry, M. Reisslein, andW. Kellerer, “Software defined optical networks (SDONs): A compre-hensive survey,” IEEE Communications Surveys & Tutorials, vol. 18,no. 4, pp. 2738–2786, 2016.

[7] R. Mijumbi, J. Serrat, J.-L. Gorricho, N. Bouten, F. De Turck, andR. Boutaba, “Network function virtualization: State-of-the-art and re-search challenges,” IEEE Communications Surveys & Tutorials, vol. 18,no. 1, pp. 236–262, 2016.

[8] S. Luo, K. Ota, M. Dong, J. Wu, J. Li, and B. Pei, “Toward highavailable SDN/NFV-based virtual network service in multi-providersscenario,” in Proc. World Automation Congress, pp. 1–5, 2016.

[9] T. Benson, A. Akella, and D. A. Maltz, “Mining policies from enter-prise network configuration,” in Proc. ACM SIGCOMM conference onInternet measurement conference, pp. 136–142, ACM, 2009.

[10] T. Nelson, A. D. Ferguson, D. Yu, R. Fonseca, and S. Krishnamurthi,“Exodus: Toward automatic migration of enterprise network configu-rations to SDNs,” in Proc. ACM SIGCOMM Symposium on SoftwareDefined Networking Research, p. 13, ACM, 2015.

[11] M. Caria, A. Jukan, and M. Hoffmann, “A performance study ofnetwork migration to SDN-enabled traffic engineering,” in Proc. IEEEGlobal Communications Conference (GLOBECOM), pp. 1391–1396,IEEE, 2013.

[12] B. Gu, M. Dong, C. Zhang, Z. Liu, and Y. Tanaka, “Real-time pric-ing for on-demand bandwidth reservation in SDN-enabled networks,”in Proc. Consumer Communications & NETWORKING Conference,pp. 696–699, 2017.

[13] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson,J. Rexford, S. Shenker, and J. Turner, “OpenFlow: enabling innovationin campus networks,” ACM SIGCOMM Computer CommunicationReview, vol. 38, no. 2, pp. 69–74, 2008.

[14] X. Jia, Y. Jiang, and Z. Guo, “Incremental switch deployment forhybrid software-defined networks,” in Proc. IEEE Conference on LocalComputer Networks (LCN), pp. 571–574, IEEE, 2016.

[15] B. Kar, E. H.-K. Wu, and Y.-D. Lin, “The budgeted maximumcoverage problem in partially deployed software defined networks,”IEEE Transactions on Network and Service Management, vol. 13, no. 3,pp. 394–406, 2016.

[16] D. Kreutz, F. M. Ramos, P. E. Verissimo, C. E. Rothenberg, S. Azodol-molky, and S. Uhlig, “Software-defined networking: A comprehensivesurvey,” Proceedings of the IEEE, vol. 103, no. 1, pp. 14–76, 2015.

[17] S. Rathee, Y. Sinha, and K. Haribabu, “A survey: Hybrid SDN,”Journal of Network and Computer Applications, 2017.

[18] F. Bannour, S. Souihi, and A. Mellouk, “Distributed SDN control:Survey, taxonomy and challenges,” IEEE Communications Surveys &Tutorials, 2017.

[19] G. Tarnaras, E. Haleplidis, and S. Denazis, “SDN and ForCES basedoptimal network topology discovery,” in Proc. IEEE Conference onNetwork Softwarization (NetSoft), pp. 1–6, IEEE, 2015.

[20] R. Horvath, D. Nedbal, and M. Stieninger, “A literature review onchallenges and effects of software defined networking,” ProcediaComputer Science, vol. 64, pp. 552–561, 2015.

[21] J. Medved, R. Varga, A. Tkacik, and K. Gray, “Opendaylight: To-wards a model-driven SDN controller architecture,” in Proc. IEEEInternational Symposium on World of Wireless, Mobile and MultimediaNetworks (WoWMoM), pp. 1–6, IEEE, 2014.

[22] N. Gude, T. Koponen, J. Pettit, B. Pfaff, M. Casado, N. McKeown,and S. Shenker, “NOX: Towards an operating system for networks,”ACM SIGCOMM Computer Communication Review, vol. 38, no. 3,pp. 105–110, 2008.

[23] D. J. Casey and B. E. Mullins, “SDN shim: Controlling legacy devices,”in Proc. IEEE Local Computer Networks (LCN), pp. 169–172, IEEE,2015.

[24] “Framework for SDN: Scope and requirements.” https://www.opennetworking.org/images/stories/downloads/sdn-resources/technical-reports/Framework for SDN -Scope and Requirements.pdf.

[25] “Internet Engineering Task Force.” https://www.ietf.org/.[26] “SDN component stack and hybrid introduction models.” https://www.

necam.com/docs/?id=c2e5a040-cdf1-4fd7-b63e-6eea4b1f7a7b.[27] “HP SDN hybrid network architecture.” https://community.

arubanetworks.com/aruba/attachments/aruba/SDN/43/1/4AA5-6738ENW.PDF.

[28] “SDN and dual control planes on one switch.” http://e.huawei.com/en/related-page/solutions/technical/agile-networking/technical-articles/Issue%205.

[29] C. Jin, C. Lumezanu, Q. Xu, Z.-L. Zhang, and G. Jiang, “Telekinesis:Controlling legacy switch routing with OpenFlow in hybrid networks,”in Proc. ACM SIGCOMM Symposium on Software Defined NetworkingResearch, p. 20, ACM, 2015.

[30] C. Jin, C. Lumezanu, Q. Xu, H. Mekky, Z.-L. Zhang, and G. Jiang,“Magneto: Unified fine-grained path control in legacy and OpenFlowhybrid networks,” in Proc. ACM Symposium on SDN Research, pp. 75–87, ACM, 2017.

[31] H. Lu, N. Arora, H. Zhang, C. Lumezanu, J. Rhee, and G. Jiang, “Hyb-net: Network manager for a hybrid network infrastructure,” in Proc.ACM Industrial Track of the 13th ACM/IFIP/USENIX InternationalMiddleware Conference, p. 6, ACM, 2013.

[32] D. Levin, M. Canini, S. Schmid, F. Schaffert, A. Feldmann, et al.,“Panopticon: Reaping the benefits of incremental SDN deployment inenterprise networks,” in Proc. USENIX Annual Technical Conference,pp. 333–345, 2014.

[33] M. Casado, T. Koponen, S. Shenker, and A. Tootoonchian, “Fabric: aretrospective on evolving SDN,” in Proc. ACM SIGCOMM Workshopon Hot Topics in Software Defined Networks, pp. 85–90, ACM, 2012.

[34] R. Enns, M. Bjorklund, and J. Schoenwaelder, “Network ConfigurationProtocol (NETCONF),” Network, 2011.

[35] J. Zhang, K. Xi, M. Luo, and H. J. Chao, “Load balancing for multipletraffic matrices using SDN hybrid routing,” in Proc. IEEE InternationalConference on High Performance Switching and Routing (HPSR),pp. 44–49, IEEE, 2014.

[36] M. Caria, A. Jukan, and M. Hoffmann, “SDN partitioning: A central-ized control plane for distributed routing protocols,” IEEE Transactionson Network and Service Management, vol. 13, no. 3, pp. 381–393,2016.

[37] M. Caria and A. Jukan, “The perfect match: Optical bypass and SDNpartitioning,” in Proc. IEEE 16th International Conference on HighPerformance Switching and Routing (HPSR), pp. 1–6, IEEE, 2015.

[38] D. K. Hong, Y. Ma, S. Banerjee, and Z. M. Mao, “Incrementaldeployment of SDN in hybrid enterprise and ISP networks,” in Proc.ACM Symposium on SDN Research, p. 1, ACM, 2016.

[39] “SAL plugin for supporting generic commodity ethernet switch.” https://wiki.opendaylight.org/view/Project Proposals:SNMP4SDN.

[40] D. Parniewicz, R. Doriguzzi Corin, L. Ogrodowczyk, M. Rashidi Fard,J. Matias, M. Gerola, V. Fuentes, U. Toseef, A. Zaalouk, B. Belter,et al., “Design and implementation of an OpenFlow hardware abstrac-tion layer,” in Proc. ACM SIGCOMM workshop on Distributed cloudcomputing, pp. 71–76, ACM, 2014.

[41] S. Vissicchio, L. Vanbever, and J. Rexford, “Sweet little lies: Faketopologies for flexible routing,” in Proc. ACM Workshop on Hot Topicsin Networks, p. 3, ACM, 2014.

[42] S. Vissicchio, O. Tilmans, L. Vanbever, and J. Rexford, “Central controlover distributed routing,” ACM SIGCOMM Computer CommunicationReview, vol. 45, no. 4, pp. 43–56, 2015.

[43] R. Hand and E. Keller, “ClosedFlow: OpenFlow-like control overproprietary devices,” in Proc. ACM Workshop on Hot topics in SoftwareDefined Networking, pp. 7–12, ACM, 2014.

[44] M. R. Nascimento, C. E. Rothenberg, M. R. Salvador, C. N. Correa,S. C. de Lucena, and M. F. Magalhaes, “Virtual routers as a service: theRouteFlow approach leveraging software-defined networks,” in Proc.International Conference on Future Internet Technologies, pp. 34–37,ACM, 2011.

[45] A. Vidal, F. Verdi, E. L. Fernandes, C. E. Rothenberg, and M. R.Salvador, “Building upon RouteFlow: a SDN development experience,”XXXI Simpsio Brasileiro de Redes de Computadores (SBRC), vol. 2013,2013.

[46] P. Lin, J. Hart, U. Krishnaswamy, T. Murakami, M. Kobayashi, A. Al-Shabibi, K.-C. Wang, and J. Bi, “Seamless interworking of SDN andIP,” ACM SIGCOMM computer communication review, vol. 43, no. 4,pp. 475–476, 2013.

[47] A. Gupta, L. Vanbever, M. Shahbaz, S. P. Donovan, B. Schlinker,N. Feamster, J. Rexford, S. Shenker, R. Clark, and E. Katz-Bassett,“SDX: A software defined internet exchange,” ACM SIGCOMM Com-puter Communication Review, vol. 44, no. 4, pp. 551–562, 2015.

[48] P. Lin, J. Bi, and H. Hu, “Btsdn: Bgp-based transition for the existingnetworks to SDN,” Wireless Personal Communications, vol. 86, no. 4,pp. 1829–1843, 2016.

[49] “Quagga Routing Suite.” http://www.nongnu.org/quagga/.[50] P. Thai and J. C. de Oliveira, “Decoupling policy from routing with

software defined interdomain management: Interdomain routing forSDN-based networks,” in Proc. IEEE Computer Communications andNetworks (ICCCN), pp. 1–6, IEEE, 2013.

[51] J. P. Stringer, Q. Fu, C. Lorier, R. Nelson, and C. E. Rothenberg,“Cardigan: Deploying a distributed routing fabric,” in Proc. ACMSIGCOMM workshop on Hot topics in Software Defined Networking,pp. 169–170, ACM, 2013.

[52] M. Caesar, D. Caldwell, N. Feamster, J. Rexford, A. Shaikh, andJ. van der Merwe, “Design and implementation of a routing controlplatform,” in Proc. USENIX Association Conference on Symposium onNetworked Systems Design & Implementation-Volume 2, pp. 15–28,USENIX Association, 2005.

[53] P. Sermpezis and X. Dimitropoulos, “Can SDN accelerate BGP con-vergence?,” arXiv preprint arXiv:1702.00188, 2017.

[54] A. Gamperli, V. Kotronis, and X. Dimitropoulos, “Evaluating the effectof centralization on routing convergence on a hybrid bgp-SDN emula-tion framework,” in Proc. ACM SIGCOMM Computer CommunicationReview, vol. 44, pp. 369–370, ACM, 2014.

[55] “The POX controller.” https://github.com/noxrepo/pox.[56] “The BGP swiss army knife of networking.” https://github.com/

Exa-Networks/exabgp.[57] S. Khan, A. Gani, A. W. A. Wahab, M. Guizani, and M. K. Khan,

“Topology discovery in software defined networks: Threats, taxono-my, and state-of-the-art,” IEEE Communications Surveys & Tutorials,vol. 19, no. 1, pp. 303–324, 2017.

[58] F. Pakzad, M. Portmann, W. L. Tan, and J. Indulska, “Efficient topologydiscovery in software defined networks,” in Proc. IEEE InternationalConference on Signal Processing and Communication Systems (ICSPC-S), pp. 1–8, IEEE, 2014.

[59] “Hp virtual application networks SDN controller.” http://h17007.www1.hp.com/docs/networking/solutions/sdn/4AA4-8807ENW.PDF.

[60] B. Belter, D. Parniewicz, L. Ogrodowczyk, A. Binczewski,M. Stroinski, V. Fuentes, J. Matias, M. Huarte, and E. Jacob, “Hard-ware abstraction layer as an SDN-enabler for non-OpenFlow networkequipment,” in Proc. IEEE European Workshop on Software DefinedNetworks (EWSDN), pp. 117–118, IEEE, 2014.

[61] F. Farias, J. Salvatti, P. Victor, and A. Abelem, “Integrating legacyforwarding environment to OpenFlow/SDN control plane,” in Proc.15th Asia-Pacific Network Operations and Management Symposium(APNOMS), pp. 1–3, IEEE, 2013.

[62] M. Szalay, L. Toka, G. Retvari, G. Pongracz, L. Csikor, and D. P.Pezaros, “Harmless: Cost-effective transitioning to SDN,” in Proc.ACM SIGCOMM Posters and Demos, pp. 91–93, ACM, 2017.

[63] C. Sieber, R. Durner, and W. Kellerer, “How fast can you reconfigureyour partially deployed SDN network?,” in Proc. IFIP Networking2017, 2017.

[64] C. Sieber, A. Blenk, A. Basta, D. Hock, and W. Kellerer, “Towards aprogrammable management plane for SDN and legacy networks,” inProc. IEEE NetSoft Conference and Workshops (NetSoft), pp. 319–327,IEEE, 2016.

[65] T. Feng, J. Bi, P. Xiao, and X. Zheng, “Hybrid SDN architecture tointegrate with legacy control and management plane: An experiences-based study,” in Proc. IFIP/IEEE International Symposium on Integrat-ed Network Management (IM), pp. 754–757, IEEE, 2015.

[66] B. Schlinker, K. Zarifis, I. Cunha, N. Feamster, E. Katz-Bassett, andM. Yu, “Try before you buy: SDN emulation with (real) interdomainrouting,” in Proc. ONS, 2014.

[67] S. Hares and R. White, “Software-defined networks and the interfaceto the routing system (I2RS),” IEEE Internet Computing, vol. 17, no. 4,pp. 84–88, 2013.

[68] R. Katiyar, P. Pawar, A. Gupta, and K. Kataoka, “Auto-configurationof SDN switches in SDN/non-SDN hybrid network,” in Proc. ACMAsian Internet Engineering Conference, pp. 48–53, ACM, 2015.

[69] S. Salsano, P. L. Ventre, L. Prete, G. Siracusano, M. Gerola, andE. Salvadori, “OSHI-Open source hybrid IP/SDN networking (and itsemulation on Mininet and on distributed SDN testbeds),” in Proc. IEEEEuropean Workshop on Software Defined Networks (EWSDN), pp. 13–18, IEEE, 2014.

[70] P. Sharma, S. Banerjee, S. Tandel, R. Aguiar, R. Amorim, and D. Pin-heiro, “Enhancing network management frameworks with SDN-likecontrol,” in Proc. IFIP/IEEE International Symposium on IntegratedNetwork Management (IM 2013), pp. 688–691, IEEE, 2013.

[71] H. Xu, H. Huang, S. Chen, and G. Zhao, “Scalable software-definednetworking through hybrid switching,” in Proc. IEEE Conference onComputer Communications, pp. 1–9, IEEE, 2017.

[72] K. Poularakis, Q. Qin, E. Nahum, M. Rio, and L. Tassiulas, “BringingSDN to the mobile edge,” arXiv preprint arXiv:1706.06001, 2017.

[73] S. Agarwal, M. Kodialam, and T. Lakshman, “Traffic engineering insoftware defined networks,” in Proc. IEEE INFOCOM, pp. 2211–2219,IEEE, 2013.

[74] Y. Guo, Z. Wang, X. Yin, X. Shi, and J. Wu, “Traffic engineering inSDN/OSPF hybrid network,” in Proc. IEEE International Conferenceon Network Protocols (ICNP), pp. 563–568, IEEE, 2014.

[75] W. Wang, W. He, and J. Su, “Boosting the benefits of hybrid SDN,”in Proc. IEEE International Conference on Distributed ComputingSystems (ICDCS), pp. 2165–2170, IEEE, 2017.

[76] J. He and W. Song, “Achieving near-optimal traffic engineering inhybrid software defined networks,” in Proc. IEEE IFIP NetworkingConference (IFIP Networking), pp. 1–9, IEEE, 2015.

[77] W. Wang, W. He, and J. Su, “Enhancing the effectiveness of trafficengineering in hybrid SDN,” in Proc. IEEE International Conferenceon Communications (ICC), pp. 1–6, IEEE, 2017.

[78] M. Caria, T. Das, and A. Jukan, “Divide and conquer: PartitioningOSPF networks with SDN,” in Proc. IFIP/IEEE International Sympo-sium on Integrated Network Management (IM), pp. 467–474, IEEE,2015.

[79] Y. Guo, Z. Wang, X. Yin, X. Shi, J. Wu, and H. Zhang, “Incrementaldeployment for traffic engineering in hybrid SDN network,” in Proc.

IEEE International Performance Computing and Communications Con-ference (IPCCC), pp. 1–8, IEEE, 2015.

[80] Y. Guo, Z. Wang, X. Yin, X. Shi, and J. Wu, “Traffic engineeringin hybrid SDN networks with multiple traffic matrices,” ComputerNetworks, vol. 126, pp. 187–199, 2017.

[81] H. Xu, X.-Y. Li, L. Huang, H. Deng, H. Huang, and H. Wang,“Incremental deployment and throughput maximization routing for ahybrid SDN,” IEEE/ACM Transactions on Networking, vol. 25, no. 3,pp. 1861–1875, 2017.

[82] H. Xu, J. Fan, J. Wu, C. Qiao, and L. Huang, “Joint deployment androuting in hybrid SDNs,” in Proc. IEEE/ACM International Symposiumon Quality of Service (IWQoS), pp. 1–10, IEEE, 2017.

[83] H. Wang, Y. Li, D. Jin, P. Hui, and J. Wu, “Saving energy inpartially deployed software defined networks,” IEEE Transactions onComputers, vol. 65, no. 5, pp. 1578–1592, 2016.

[84] Y. Wei, X. Zhang, L. Xie, and S. Leng, “Energy-aware traffic engineer-ing in hybrid SDN/IP backbone networks,” Journal of Communicationsand Networks, vol. 18, no. 4, pp. 559–566, 2016.

[85] X. Jia, Y. Jiang, Z. Guo, G. Shen, and L. Wang, “Intelligent path controlfor energy-saving in hybrid SDN networks,” Computer Networks,vol. 131, pp. 65–76, 2018.

[86] Y. Hu, W. Wang, X. Gong, X. Que, Y. Ma, and S. Cheng, “Maximizingnetwork utilization in hybrid software-defined networks,” in Proc. IEEEGlobal Communications Conference (GLOBECOM), pp. 1–6, IEEE,2015.

[87] C. Ren, S. Wang, J. Ren, X. Wang, T. Song, and D. Zhang, “Enhancingtraffic engineering performance and flow manageability in hybrid SD-N,” in Proc. IEEE Global Communications Conference (GLOBECOM),pp. 1–7, IEEE, 2016.

[88] K. Poularakis, G. Iosifidis, G. Smaragdakis, and L. Tassiulas, “Onestep at a time: Optimizing SDN upgrades in ISP networks,” in Proc.IEEE INFOCOM, 2017.

[89] S. Vissicchio, L. Vanbever, L. Cittadini, G. Xie, and O. Bonaventure,“Safe updates of hybrid SDN networks,” Universit catholique deLouvain, Tech. Rep, 2013.

[90] S. Vissicchio, L. Vanbever, L. Cittadini, G. G. Xie, and O. Bonaventure,“Safe update of hybrid SDN networks,” IEEE/ACM Transactions onNetworking (TON), vol. 25, no. 3, pp. 1649–1662, 2017.

[91] S. Vissicchio, L. Cittadini, O. Bonaventure, G. G. Xie, and L. Vanbever,“On the co-existence of distributed and centralized routing control-planes,” in Proc. IEEE Conference on Computer Communications(INFOCOM), pp. 469–477, IEEE, 2015.

[92] R. Amin, N. Shah, B. Shah, and O. Alfandi, “Auto-configuration ofACL policy in case of topology change in hybrid SDN,” IEEE Access,vol. 4, pp. 9437–9450, 2016.

[93] C.-Y. Chu, K. Xi, M. Luo, and H. J. Chao, “Congestion-aware singlelink failure recovery in hybrid SDN networks,” in Proc. IEEE Con-ference on Computer Communications (INFOCOM), pp. 1086–1094,IEEE, 2015.

[94] M. Markovitch and S. Schmid, “Shear: A highly available and flexiblenetwork architecture marrying distributed and logically centralizedcontrol planes,” in Proc. IEEE International Conference on NetworkProtocols (ICNP), pp. 78–89, IEEE, 2015.

[95] L. Wang, Q. Li, Y. Jiang, and J. Wu, “Towards mitigating link floodingattack via incremental SDN deployment,” in Proc. IEEE Symposiumon Computers and Communication (ISCC), pp. 397–402, IEEE, 2016.

[96] “OpenvSwitch.” http://openvswitch.org/.[97] H. Huang, S. Guo, J. Wu, and J. Li, “Green datapath for TCAM-based

software-defined networks,” IEEE Communications Magazine, vol. 54,no. 11, pp. 194–201, 2016.

[98] T. Lukovszki, M. Rost, and S. Schmid, “It’s a match!: Near-optimaland incremental middlebox deployment,” ACM SIGCOMM ComputerCommunication Review, vol. 46, no. 1, pp. 30–36, 2016.

[99] Z. Zheng, L. X. Cai, M. Dong, X. Shen, and H. V. Poor, “Constrainedenergy-aware AP placement with rate adaptation in WLAN meshnetworks,” in Proc. IEEE Global Telecommunications Conference(GLOBECOM 2011), pp. 1–5, IEEE, 2011.

[100] G. Luo, Z. Qian, M. Dong, K. Ota, and S. Lu, “Improving performanceby network-aware virtual machine clustering and consolidation,” TheJournal of Supercomputing, pp. 1–19, 2017.

[101] C. Ge, Z. Sun, N. Wang, K. Xu, and J. Wu, “Energy managementin cross-domain content delivery networks: A theoretical perspective,”IEEE Transactions on Network and Service Management, vol. 11, no. 3,pp. 264–277, 2014.

[102] A. Medina, N. Taft, K. Salamatian, S. Bhattacharyya, and C. Diot,“Traffic matrix estimation: Existing techniques and new directions,”ACM SIGCOMM Computer Communication Review, vol. 32, no. 4,pp. 161–174, 2002.

[103] “Mininet:an instant virtual network on your laptop.” http://mininet.org/.[104] S. Knight, H. X. Nguyen, N. Falkner, R. Bowden, and M. Roughan,

“The internet topology zoo,” IEEE Journal on Selected Areas inCommunications, vol. 29, no. 9, pp. 1765–1775, 2011.

[105] “Rocketfuel-topology.” http://research.cs.washington.edu/networking/rocketfuel/.

[106] N. Huin, M. Rifai, F. Giroire, D. L. Pacheco, G. Urvoy-Keller, andJ. Moulierac, “Bringing energy aware routing closer to reality withSDN hybrid networks,” IEEE Transactions on Green Communicationsand Networking, 2018.

[107] R. Alvizu and G. Maier, “Can OpenFlow make transport networkssmarter and dynamic? An overview on transport SDN,” in Proc.IEEE International Conference on Smart Communications in NetworkTechnologies (SaCoNeT), pp. 1–6, IEEE, 2014.

[108] M. Channegowda, R. Nejabati, and D. Simeonidou, “Software-definedoptical networks technology and infrastructure: Enabling software-defined optical network operations,” Journal of Optical Communica-tions and Networking, vol. 5, no. 10, pp. A274–A282, 2013.

[109] S. Azodolmolky, R. Nejabati, E. Escalona, R. Jayakumar, N. Efstathiou,and D. Simeonidou, “Integrated OpenFlow–GMPLS control plane: anoverlay model for software defined packet over optical networks,”Optics express, vol. 19, no. 26, pp. B421–B428, 2011.

[110] M. Channegowda, P. Kostecki, N. Efstathiou, S. Azodolmolky, R. Ne-jabati, P. Kaczmarek, A. Autenrieth, J. P. Elbers, and D. Simeonidou,“Experimental evaluation of extended OpenFlow deployment for high-performance optical networks,” in Proc. IEEE European Conferenceand Exhibition on Optical Communications (ECOC), pp. 1–3, IEEE,2012.

[111] T. Choi, S. Song, H. Park, S. Yoon, and S. Yang, “Suma: Software-defined unified monitoring agent for SDN,” in Proc. IEEE NetworkOperations and Management Symposium (NOMS), pp. 1–5, IEEE,2014.

[112] A. C. Baktir, A. Ozgovde, and C. Ersoy, “How can edge computingbenefit from software-defined networking: A survey, use cases, andfuture directions,” IEEE Communications Surveys & Tutorials, vol. 19,no. 4, pp. 2359–2391, 2017.

[113] H. Mai, A. Khurshid, R. Agarwal, M. Caesar, P. Godfrey, and S. T.King, “Debugging the data plane with anteater,” in Proc. ACM SIG-COMM Computer Communication Review, vol. 41, pp. 290–301, ACM,2011.

[114] P. Kazemian, G. Varghese, and N. McKeown, “Header space analysis:Static checking for networks,” in Proc. NSDI, vol. 12, pp. 113–126,2012.

[115] M. Wichtlhuber, R. Reinecke, and D. Hausheer, “An SDN-basedCDN/ISP collaboration architecture for managing high-volume flows,”IEEE Transactions on Network and Service Management, vol. 12, no. 1,pp. 48–60, 2015.

[116] R. Sherwood, G. Gibb, K.-K. Yap, G. Appenzeller, M. Casado, N. M-cKeown, and G. Parulkar, “Flowvisor: A network virtualization layer,”OpenFlow Switch Consortium, Tech. Rep, vol. 1, p. 132, 2009.

Xinli Huang received his Ph.D. degree in ComputerScience from Shanghai Jiao Tong University in2007. He is currently an Associate Professor with theDepartment of Computer Science and Technology atEast China Normal University. His research interestsare in the areas of Internetworking, Software DefinedNetworking, cloud data center networks, future net-works, and network security. He is a member of theIEEE.

Shang Cheng received his B.S. degree in computerscience from Harbin Engineering University, China,in 2015. He is currently a M.E. candidate at theDepartment of Computer Science and Technologyat the East China Normal University. His researchinterests include traffic engineering, routing opti-mization and Software Defined Networking.

Kun Cao is currently pursuing the Ph.D. degreein Computer Science from East China Normal U-niversity. His current research interests are in the ar-eas of high performance computing, multiprocessorsystems-on-chip and cyber physical systems.

Peijin Cong received the B.S. degree from the De-partment of Computer Science and Technology, EastChina Normal University, shanghai, China, in 2016.She is currently pursuing the master degree with theDepartment of Computer Science and Technology,East China Normal University, Shanghai, China. Hercurrent research interest is in the areas of powermanagement in mobile devices and edge computing.

Tongquan Wei received his Ph.D. degree in Elec-trical Engineering from Michigan Technological U-niversity in 2009. He is currently an Associate Pro-fessor in the Department of Computer Science andTechnology at the East China Normal University.His research interests are in the areas of Internetof Things, edge computing, cloud computing, anddesign automation of intelligent and CPS systems.He serves as a Regional Editor for Journal of Cir-cuits, Systems, and Computers since 2012. He is amember of the IEEE.

Shiyan Hu received his Ph.D. in Computer Engi-neering from Texas A&M University in 2008. He isan Associate Professor at Michigan Tech, and he wasa Visiting Associate Professor at Stanford Universityfrom 2015 to 2016. His research interests includeCyber-Physical Systems (CPS), CPS Security, DataAnalytics, and Computer-Aided Design of VLSI Cir-cuits, where he has published more than 100 refereedpapers. He is an ACM Distinguished Speaker, anIEEE Systems Council Distinguished Lecturer, anIEEE Computer Society Distinguished Visitor, and a

recipient of National Science Foundation (NSF) CAREER Award. Prof. Hu isthe Chair for IEEE Technical Committee on Cyber-Physical Systems. He is theEditor-In-Chief of IET Cyber-Physical Systems: Theory & Applications. He isan Associate Editor for IEEE Transactions on Computer-Aided Design, IEEETransactions on Industrial Informatics, and IEEE Transactions on Circuits andSystems. He is also a Guest Editor for a number of IEEE/ACM Journals suchas Proceedings of the IEEE and IEEE Transactions on Computers. He hasheld chair positions in numerous IEEE/ACM conferences. He is a Fellow ofIET.