green cloud provisioning throughout cooperation of a wdm ... · green cloud provisioning throughout...

J Grid Computing (2016) 14:127–151DOI 10.1007/s10723-015-9354-7

Green Cloud Provisioning Throughout Cooperationof a WDMWide Area Network and a Hybrid PowerIT InfrastructureA Study on Cooperation Models

Piotr Borylo ·Artur Lason · Jacek Rzasa ·Andrzej Szymanski ·Andrzej Jajszczyk

Received: 31 October 2014 / Accepted: 14 October 2015 / Published online: 24 October 2015© The Author(s) 2015. This article is published with open access at Springerlink.com

Abstract The paper focuses on cooperation betweencloud and network operators, as well as on fittingparticular routing strategies to various cloud services.Three cooperation models are presented, analyzed andcompared in the paper: the proposed model and twowidely used reference models. The main differencebetween the models is the set of information beingexchanged between the involved parties. Additionally,we analyze the applicability of four fitting schemas foreach considered model. It is shown that the proposedmodel, alongside with an appropriate fitting schema,is able to reduce the blocking probability of cloud ser-vices requests. At the same time, thanks to the useof green anycast strategies, it is able to significantlyreduce carbon dioxide emission.

P. Borylo (�) · A. Lason · J. Rzasa · A. Szymanski ·A. JajszczykAGH University of Science and Technology, Al.Mickiewicza 30, 30-059 Krakow, Polande-mail: [email protected]

A. Lasone-mail: [email protected]

J. Rzasae-mail: [email protected]

A. Szymanskie-mail: [email protected]

A. Jajszczyke-mail: [email protected]

Keywords Green cloud computing · Hybrid power ·Carbon footprint reduction · Cloud architectureintegration · All optical networks

1 Introduction

While considering the provisioning of cloud servicesone cannot neglect the importance of the networkinfrastructure which is necessary to ensure access tothe IT resources. Due to the global character of mod-ern cloud computing a great deal of effort is requirednot only to investigate networks within data cen-ters, but also access, metro and wide area networks.However, cloud and network infrastructures are usu-ally administered by different entities, with disparateoptimization objectives. What is more, the involvedparties are not usually willing to reveal the internalstructure of their assets and often treat this informationas confidential, which further complicates the case.As this behaviour may lead to suboptimal choices, weclaim that in order to meet customers’ needs and toeffectively follow the concept of greening the Internet[1] some models of cooperation between network andIT infrastructure operators need to be introduced andthoroughly investigated.

The issue of integration of network infrastructurewith IT resources was considered in numerous paperspublished in renowned journals and magazines. Forexample, authors in [2] provide a detailed definitionof Software Defined Networking (SDN), its interfaces

http://crossmark.crossref.org/dialog/?doi=10.1186/10.1007/s10723-015-9354-7-x&domain=pdf

mailto:[email protected]





128 P. Borylo et al.

as well as a list of its key attributes. One of themajor features pointed in the paper is the ability ofan SDN controller to cooperate with cloud orches-tration software in order to quickly provision cloudapplications and operate them in an automated man-ner. Thanks to the cooperation, networks may furthersupport virtualization, migration of virtual machinesand evolve towards application oriented networks.An additional effort is needed to bring those fea-tures to the Wide Area Networks (WANs). In [3],the authors present requirements, design alternativesand possible management architectures for a WideArea SDN (WA-SDN). Again, communication of anSDN controller with cloud applications was men-tioned as one of the most important use cases. Inthe wide area context cloud orchestration softwaremay have to operate on more than one operators’ ITinfrastructure. Numerous middleware solutions aimedat using multiple infrastructures cooperating underdifferent categories are available, e.g. [4]. Exhaus-tive survey about the existing solutions was providedin [5].

Not only scientific papers concern this issue. Forexample, B4 is an inter-datacenter private WAN con-necting 12 Google’s sites (data centers) across theplanet [6]. B4 WAN, as well as networks within thesites, are controlled by a set of SDN controllers.All of them exchange information in order to fullyutilize bandwidth available in the WAN and to con-trol applications and site networks. However, onlythe largest cloud providers are able to build theirown private WANs. Others must cooperate with var-ious network operators to improve service qualityand to reduce costs. This problem has been noticedby equipment vendors. For example, the concept ofSeamless Virtual Cloud presented by Cisco at numer-ous conferences (e.g. IEEE Network Operations andManagement Symposium 2014, Cisco Live - Milan2014) assumes that IT resources and WAN infrastruc-ture are seamlessly bounded together and are availablefor the tenants in an on-demand fashion. The pre-sented framework provides an abstraction of relevantnetworking technologies in a WAN and in data cen-ters. To sum up, it may be said that application drivenwide area networks are becoming a part of the cloudinstead of being just a way to access cloud applicationsand features.

The general idea of improving energy efficiencyand decreasing negative impact on the natural envi-

ronment is a hot topic and concerns almost everyaspect of our lives. However, there are variuos rea-sons for introducing green mechanisms into IT andnetwork infrastructure. The first, and the most naturalreason is the will to decrease the negative impact ofthe infrastructure on the natural environment. Follow-ing the idea of greening the cloud also affects publicrelations and marketing. A cloud provider advertisingitself as an environmental friendly one may improveits reputation and may seem to be more attractivefor selected customers. However, it seems that thestrongest driver is an economical one. Huge powerrequirements of the equipment directly translate tohigh operational costs related to electricity. On the onehand, the cost may be reduced by energy efficiencyimprovements. On the other hand, due to tax regula-tions introduced in numerous countries (e.g. UnitedKingdom, China), energy obtained from the renewableresources is cheaper. Thus, using renewable energymay also decrease the operational costs [7]. At thesame time, network operators attract the attention ofcloud providers by delivering green mechanisms andwilling to cooperate in the field of energy efficiency.Such an additional value is currently especially impor-tant, due to the fact that the revenues from simple datatransmission are decreasing. Thus, telecommunicationoperators must take advantage of emerging technolo-gies, e.g. SDN, to provide more profitable services,including the green ones.

In the context of cloud computing, a great deal ofprevious works was focused on improving energy effi-ciency of data centers (DCs), as DCs are one of themain contributors to the overall energy consumptionand carbon footprint of the ICT sector [8, 9]. The issueis of such importance, that the Green Grid Associa-tion, solely devoted to improve the efficiency of infor-mation technology and data centers throughout theWorld was founded [10]. The negative impact of DCson the natural environment and the amount of energythey consume may be significantly decreased usingsome of the numerous available solutions. Among theother, the most popular approaches are as follows [7]:

– Placing DCs in locations where electricity isobtained from the sources which generate lowerquantities of carbon dioxide and other greenhousegases (e.g. hydropower, solar and wind powerstations).

– Placing DCs in locations where Mother Naturemay effectively help to improve energy efficiency,

Green Cloud Provisioning 129

for instance, where there is cold enough to use airside economizers that use outside air to chill a DC.

– Reusing heat being generated by the IT infrastruc-ture and drawing upon recycled water rather thanfresh one for cooling purposes.

– Optimizing airflows in DCs by, for example, iso-lating hot and cold airflows.

For the purpose of our research we assume thatgreen DCs are those powered from renewable energysources. This approach is a common practice of thelargest IT players, like for example: Apple [11], Face-book [11, 12] or Google [11, 13]. However, our dis-cussion and solutions may be easily adopted to othervariants of greenDCs, including the ones listed above.

Usually, it is not possible to power all DCs usingrenewable energy. Green energy is available only inselected locations and transmitting it from distant sitesis not worthwhile due to high transmission loses.Thus, a hybrid power scenario, where a single cloudoperator may have DCs powered from renewableenergy mixed with those powered from conventionalenergy sources, is a very realistic one. In such a case itis reasonable to utilize those green DCs first, allowingthe equipment in other DCs to enter the low energyconsumption state, which reduces the overall carbondioxide emission [14].

In this paper we focus on provisioning of cloudservices throughout cooperation of the wide area alloptical network interconnecting the sites equippedwith the hybrid power IT infrastructure. We assume,that the network is controlled by the centralized con-troller consistent with the idea of the SDN. At thesame time, we assume that the cloud orchestrationsoftware controls the cloud infrastructure and oper-ates it in an automated manner. The architecture ofsuch a system is described and systematized in thispaper, followed by a discussion of different possi-ble cooperation models between the WAN operatorand the cloud provider. On the one hand, both enti-ties are not willing to reveal their internal information.On the other hand, they want to cooperate in order toimprove service quality and decrease carbon dioxideemission. Thus, we propose and evaluate a coopera-tion model allowing for improvement in performanceof the infrastructure. Additionally, by joining coop-eration models with green anycast strategies, carbondioxide emission can be significantly decreased byutilizing green energy in the first place. As a side

topic we investigate various anycast routing strategiesas well as schemas for fitting those strategies to cloudservices. To the best of our knowledge this work isthe first one that analyzes deeply the problem of coop-eration between network operator and cloud providerin order to provide cloud services with minimal car-bon footprint. The issue is important as the networkbecome an integral part of the cloud and, therefore,convergence of IT and telecommunication infrastruc-tures is needed to effectively provide services to endusers.

The rest of this paper is organized as follows. Thenext section provides a description and references tothe related work. Section 3 contains a description ofa cloud architecture being considered, including rela-tionships between the network and cloud controllers.Additionally, it provides the rationale for numerousassumptions taken in the paper, as well as explana-tion of resource and requests models. Section 4 pro-vides the formal problem statement, while Section 5presents anycast strategies and schemas for fittingthose strategies to different cloud services. Sec-tions 6 and 7 describe cooperation models and simula-tion environment. Simulation results are presented inSection 8. Finally, Section 9 concludes the paper.

2 Related Work

Although the problem of reduction of greenhousegases emission is a well known issue, only a limitedset of papers deals with it in the context of networkssupporting cloud architectures.

The problem of joint provisioning of network andIT resources was considered in [15, 16] and [17]. Theauthors of [15], apart from dimensioning issues, ana-lyze three strategies for joint resource provisioning.Two of those strategies are conceptually similar to ourreference models. The third one is an authors’ orig-inal proposition to decrease the blocking probabilityby utilizing load balancing features in the network andthe DCs, simultaneously. However, the paper focuseson performance issues only, while energy efficiencyand impact on the natural environment are not investi-gated.

In [16] the authors address the problem of jointdefragmentation of optical and computing resourcesin order to decrease the blocking probability. Refer-ence cooperation models being considered in our work


have their equivalents in the context of defragmen-tation. Differences between our models and modelsdescribed in [16] come from the fact that in the latterthe holding time of each request is known in advanceand the environmental aspect is not considered.

In [17], a control module is proposed to improvecooperation between cloud orchestration software andSDN controllers. The proposed idea of SoftwareDefined Infrastructure (SDI) provides management ofconverged computing and networking resources. TheSDI was implemented and assessed in a testbed. How-ever, two important issues were not addressed: precisedefinition of information being exchanged betweenentities and the impact on the natural environment.

The authors in [18] proposed anycast heuristics toreduce CO2 emission in IP-over-WDM networks con-necting DCs under a dynamic traffic scenario. Theirremarkable work was later extended in [19]. Theproposed heuristics uses information about the avail-ability of renewable energy, the amount of energyneeded to serve a request in DC (the authors assumeone specific service being provided) and network linksoccupancy. However, the authors assumed that net-work and cloud infrastructures are controlled jointlyand without consideration of different cooperationmodels.

Utilization of anycast traffic to serve DC requestswas also analyzed in [20]. Numerous modulationtypes and spectrum allocation strategies were inves-tigated in elastic optical networks with regard to theblocking probability. However, the impact on the nat-ural environment was not considered.

A strongly related issue was also addressed in [21].The paper investigates a multilayer IP over WDMtransport network. The most energy consuming partsof the infrastructure are network nodes switching thetraffic in the electrical domain. A power cappingtechnique, formerly used inside DCs, was adoptedto effectively move those power hungry tasks to thenetwork nodes powered from renewable resources.

Another approach was introduced in [22]. Theauthors proposed the migration of virtual machines(VMs) toward DCs powered from renewable energysources. The incoming requests are directed to VMsthrough the network, therefore, the placement of VMshas a direct impact on traffic distribution in the net-work. Full cooperation between entities was assumed,but the exact range of information being exchangedbetween entities was not directly specified. VMs

migration was also considered in [23]. VM alloca-tion supported by dynamic voltage scaling and CPUpinning techniques was proposed in order to maxi-mize resource utilization and energy efficiency whilesatisfying quality of service guarantees.

Numerous assumptions taken in this paper comefrom two of our previous works: [24] and [25]. In[24] we proposed three anycast strategies and com-pared them to three reference strategies with regardto network blocking probability and carbon diox-ide emission. The main difference between [24] andthis paper comes from the fact that in our previouswork DCs had infinite computing resources and wereassumed to offer only one type of cloud service. In[25] we introduced three types of cloud services andproposed schemas for fitting different anycast strate-gies to those services. The fitting was performed basedon network resource requirements and energy con-sumption. The aim was to decrease carbon footprintas much as possible without a significant deteriorationof network performance. But still, the only decisionfactor was associated with network resources utiliza-tion while the IT infrastructure had unlimited capacity.This assumption is no longer valid in this paper.Therefore, this paper is based on research conductedin both our previous works. However, it simultane-ously extends those works and introduces originalcooperation models between the entities.

3 Cloud and Network Architectures

One of the main paradigms of cloud computing isto offer numerous different services in an on-demandfashion. That is why incoming requests concerningdifferent services are unpredictable and the cloudarchitecture must be investigated under a dynamictraffic scenario [26]. Rapid CRUD (create, read,update, delete) of resources is necessary in seconds.Software to automate provisioning and operation of acloud is needed in order to meet the aforementionedrequirement. Numerous cloud orchestration frame-works are available under open source as well as com-mercial licenses. A common purpose of those frame-works is to provide an abstraction of IT resourcesavailable in the cloud and to CRUD those resourcesin an automated manner. Exemplary solutions are:EC2 [27], OpenStack [28], OpenNebula [29], Cloud-Stack [30]. Good scalability, independence from


underlaying hardware, versatility, modular architec-ture and user friendly interface are, among the others,the factors being considered while choosing a spe-cific product. Another paradigm of cloud computingis to offer the same services in different DCs simul-taneously. Thus, it is possible to choose one of manypossible DCs to provide the requested service [26].A corresponding routing scheme is called anycast.This assumption is realistic as numerous algorithmsand solutions for data replication have already beenproposed, e.g. [31].

Additionally, integration of a cloud with networkinfrastructure requires corresponding integration ofcloud orchestration with network control. The afore-mentioned idea of an SDN perfectly suits the require-ment as in the SDN architecture the control planeis separated from the switching plane. Such a con-trol plane is realized in software, denoted as an SDNcontroller and deployed as an entity separate fromthe network devices. The controller communicateswith hardware (OpenFlow enabled devices) and vir-tual devices through the OpenFlow Protocol in orderto deliver forwarding rules [32]. Therefore, communi-cation between the centralized controller and networkdevices must be ensured to allow proper operation ofthe Software Defined Network. Simultaneously, theSDN controller may be able to communicate throughthe Northbound-API with the aforementioned cloudorchestration software [2]. Examples of SDN con-trollers are: OpenDayLight [33], Floodlight [34], Bea-con [35], Big Cloud Fabric Controller [36]. The rela-tionships in the considered architecture are presentedin Fig. 1.

Modeling of IT resources within a DC is a separateproblem. It is common for a DC to have heteroge-neous physical machines, with different amounts ofRAM and different CPUs. However, this heteroge-nous infrastructure may be abstracted using cloudorchestration software responsible for resource allo-cation, virtualization and consolidation. Furthermore,numerous task schedulers may be used in order toeffectively allocate such DC resources, e.g. [37, 38]or [39]. Therefore, IT resources available in each DCmay be represented as a single value reflecting avail-able RAM and computing power. A specific amountof those resources is consumed while serving a DCrequest and is released afterwards. A third kind ofDC resources that needs to be considered is storagespace, which is commonly provided using dedicatedequipment. A shortage of storage space is generallyunacceptable in a DC. Additionally, storage resourcesare not usually released after tearing down a request,further complicating the case. Thus, we decided totreat the storage resources as unlimited and omit themfrom abstraction.

In addition to modelling IT resources within a DC,it is necessary to model resource consumption of aparticular request. Standardization of job submissionprocess in cloud infrastructure is an emerging andstill open issue (see [40]). A common practice is todivide all requests into a fixed number of classes. Suchan idea of categorization of cloud service requests isapplied, for example, by Google [41]. Three types ofcloud services are being considered in the paper: PaaS(Processing as a Service), StaaS (Storage as a Ser-vice) and SaaS (Software as a Service). Those types

Fig. 1 Consideredarchitecture (based on [33]),VTN: Virtual TenantNetwork, OVSDB: OpenvSwitch DataBase Protocol,LISP: Locator/IdentifierSeparation Protocol, BGP:Border Gateway Protocol,PCEP: Path ComputationElement CommunicationProtocol, SNMP: SimpleNetwork ManagementProtocol


of services have been described in detail in [42], butwe focus solely on their requirements for networkand data center resources as well as their energy con-sumption model. These three properties are crucial forappropriate fitting of anycast strategies to cloud ser-vice types and for proper evaluation of the proposedcooperation models. The presented assumptions areconsistent with our previous work [25].

PaaS Refers to a group of services in which a cloudcustomer is able to process computationally demand-ing tasks in a DC. It means that a PaaS requestoccupies network resources only for the time neededto send task related data to the DC, and consumeshuge amount of energy and IT resources inside theDC. According to the model proposed in [19], energyconsumption of a DC handling a PaaS task may beexpressed by the following equation: Eproc = 1.5 ·Tproc · Ps , where Ps is the power consumption of theserver (355 W), Tproc is the processing time (in hours)and the scaling factor represents the power consump-tion overhead (e.g., air conditioning). Thus, followingconsiderations presented in [19], an exemplary ser-vice of encoding a video file requires Eproc = 100Wh to process 10 Gb (1.25 GB) of data. The process-ing time usually exceeds the time of network resourceoccupancy, but if we assume that the whole task maybe computed in a parallel way in the time equal tothe network resource occupancy we get Eproc = 36kW/(Gb/s). It must be emphasized that this assump-tion does not affect the total amount of the consumedenergy, it only decreases the time in which the energywas consumed.

StaaS Represents cloud services that allow customersto store their data in the cloud. Thus, networkresources are occupied for the time needed to trans-fer the data. Energy and IT resources needed to handlea StaaS request are related only to the Input/Outputmemory operations (requiring some CPU and RAMresources) and are marginal. This assumption is validas, in general, power consumption of storage equip-ment is independent of the amount of currently storeddata. The average StaaS energy consumption sug-gested in [18] is equal to 0.141 W/(Gb/s) and thisvalue is reasonable from our point of view.

SaaS Spans numerous different cloud services, there-fore, properties of SaaS requests are very hard to

define. Some exemplary cloud services assigned tothe SaaS group are: dedicated web application, virtualdesktop, virtual machine with dedicated software, etc.As a result, a SaaS request may, for example, occupynetwork resources for a very long time while consum-ing only a little amount of IT resources and, subse-quently, energy in a DC (as in case of virtual desktopservice). However, at the same time, another SaaSrequest may also hold network resources for a similartime (required to make the use of dedicated software)and consume energy and IT resources comparable tothe PaaS service by running computationally intensivetasks using the aforementioned dedicated software.The authors in [18] proposed an estimate of the aver-age energy consumption in a DC for a SaaS requestequal to 2.7 W/(Gb/s). This value is proper for SaaSservices with very low energy requirements. Know-ing a variety of SaaS services it seems reasonable toanalyze different cases of energy consumption. There-fore, average energy consumption values assumed inour work are 1, 2.7, 5.4 and 10 kW/(Gb/s), i.e., muchhigher than the value proposed in [18], but still lowerthan for the PaaS services. However, the solution pro-posed in this paper is able to adjust even to the valuesuggested in [18].

A separate issue concerns the proportions in whichthe aforementioned services contribute to the total DCtraffic. We assume that those proportions may varyin a great deal. In modern clouds, PaaS is probablythe least popular as PaaS requires the cloud opera-tor to strongly adjust to user needs. Also, securityissues may decrease attractiveness of PaaS. On theother hand, SaaS appears to be the most frequentlyused as SaaS spans numerous services and offers themto clients in the most flexible way (for both cloudoperators and users).

4 Problem Statement

The problem investigated in this paper may be brieflydescribed as dynamic routing and wavelength assign-ment (RWA) merged with cloud services provisioning.A corresponding static problem that may be formu-lated is NP-hard. The complexity results from the factthat the RWA problem, which is a part of the consid-ered problem, was proved to be NP-complete (for theproof see [43]). As we focus on the dynamic prob-lem the static problem will not be formulated and


investigated in greater detail. The wide area networkis assumed to be an automatically switched all opticalnetwork interconnecting hybrid power IT infrastruc-ture. Optical data transmission is usually indicatedas the most appropriate to provide cloud comput-ing services. Elimination of electrical layer from thenetwork architecture ensures low energy consump-tion of the network nodes. DCs are associated withselected network nodes. Network and IT resources areadministrated by separate entities.

In the formal problem definition, graph G(V, E)

denotes the physical network, where V is the set ofnodes, and E is the set of links. Each fiber e ∈ E

carries a fixed number of wavelengths indexed usingw. Symbols ew = 1 and ew = 0 denote unoccupiedand occupied wavelength w on link e, respectively.VDC describes the set of nodes being associated withthe DCs. DCs powered from renewable energy sourcesare called green and network nodes associated withthe green DCs are denoted by VgDC. On the otherhand, DCs powered from traditional energy sourcesare called brown and nodes directly connected to themare denoted by VbDC. In all further equations g andb lower indices will always denote green and brown,respectively. Nodes that are not associated with anyDC are termed client nodes, and denoted by VC . Thefollowing relations are met:

VgDC ∩ VbDC = ∅ (1)

VgDC ∪ VbDC = VDC (2)

VDC ∩ VC = ∅ (3)

VDC ∪ VC = V (4)

For each data center d ∈ VDC, cd and rd denote totalamount of resources and resources currently avail-able, respectively. CDC and RDC describe the setscontaining cd and rd for all d ∈ VDC, respectively.

The centralized SDN controller has full knowledgeabout the network topology and its state, includinginformation about existing lightpaths, the location ofDCs (VDC) and their power source. It means that forany set of DCs d ∈ D ⊆ VDC the SDN controller isable to distinguish green ones, d ∈ Dg , from brownDCs d ∈ Db. At the same time, cloud orchestrationsoftware has knowledge about resources available ineach DC and cloud operator’s preferences.

Dynamic RWA algorithms operate on a lightpathrequest (LR). We assumed two types of LRs:

1. A unicast LR from a source node s ∈ V to adestination node d ∈ V .

2. An anycast LR from a source node s ∈ VC to onefrom the set of possible destinations d ∈ D ⊆VDC. Anycast LR are used to handle DC traffic.When a cloud service request (CR) arrives to thecloud orchestration software it is further passed tothe SDN controller as anycast LR.

From the SDN controller point of view, a path X

between s and d must be found to serve any LR. PathX is composed of adjacent links and each fiber x beingpart of the path X, x ∈ X, must meet the wavelengthcontinuity constraint (WCC):

∃w ∀x ∈ X xw = 1 (5)

where w is a wavelength that may be chosen to servethe LR over the path X.

Unicast LRs are served using alternate routing withthree link-disjoint, precomputed paths. The details ofthe algorithm for unicast LRs may be found in [44]. Inthis work unicast lightpath requests are used solely forhandling the background traffic which does not inter-act with the cloud orchestration software. As the cloudinfrastructure is not involved in serving unicast LRs,the path between s and d must only meet WCC (5).

As anycast scheme may be defined as one-to-one-of-many routing, for each anycast LR the RWA algo-rithm needs to choose a destination d ∈ D and thenset up a lightpath from the source node s to the cho-sen node d. The strategies for anycast LRs handlingare described in detail in Section 5. The presentedstrategies use the LAC(s, d) operation, which denoteslightpath availability check between the source nodes and the destination node d. This operation is alsobased on alternate routing and checks if any of thethree precomputed link-disjoint paths between s andd meets WCC. The result of LAC(s, d) operation isthe length (in the hop manner) of the shortest pathmeeting WCC. If WCC cannot be met by any of theprecomputed paths, LAC(s, d) returns infinity.

As mentioned earlier, each anycast LR is associ-ated with a cloud request, CR, which consumes ITresources within a DC associated with the chosen noded. Thus, the following constraint must be met:

rd ≥ LRr (6)


where, LRr is the amount of resources required toserve a cloud service request associated with the par-ticular LR. After tearing down the request, both net-work as well as IT resources are released. We assumethat cloud service requirements for IT resources areproportional to their energy consumption. Such anassumption seems to be reasonable in order to providea consistent model of cloud services.

A subset of DCs meeting the constraint (6) isdenoted as VrDC. Again, the VrDC subset may befurther divided into two subsets: VrgDC and VrbDC, cor-responding to green and brown DCs. The followingrelations exist:

VrgDC ∩ VrbDC = ∅ (7)

VrgDC ∪ VrbDC = VrDC (8)

VrDC ⊆ VDC (9)

VrgDC ⊆ VgDC (10)

VrbDC ⊆ VbDC (11)

Each CR and a corresponding anycast LR is used tohandle requests assigned to one of the three aforemen-tioned types of cloud services: PaaS, StaaS or SaaS.For the purpose of simplicity, anycast LRs handlingPaaS will be denoted as PaaS requests. The samenomenclature is applied to StaaS and SaaS. Each LRand CR carry information about the assignment. Thus,the cloud orchestration software and the SDN con-troller are able to use this information and performactions adjusted to different services.

In order to estimate the carbon footprint of theconsidered infrastructure we assume that brown DCscontribute to CO2 emission proportionally to the con-sumed power, while green DCs do not contribute toCO2 emission at all. Therefore, processing requests ingreen DCs creates opportunity to reduce CO2 emis-sion. Energy consumed by network nodes alwayscomes from non-renewable sources. However, itsamount is negligible in comparison to energy con-sumed by DCs (especially in the assumed case of alloptical networks) and, as such, was ignored in thesubsequent studies.

5 Anycast Strategies and Fitting Schemas

In this section we present detailed definitions of allanycast strategies used in subsequent studies. It isimportant to note that the algorithms of the strategiesmay be easily implemented in a network controller.They utilize only fundamental control information thatis to a great extent already present in the commonunicast network controller. The first strategy, clos-est, is a well known, reference anycast strategy. Theremaining two of them: closestGreen and closest-GreenWithPenalty are the strategies proposed in ourprevious work [24] and aimed at minimizing carbonfootprint.

5.1 Anycast Strategies

In the closest strategy the network controller performsLAC(s, d) for all d ∈ D. The closest (in the hopmanner) reachable destination d is chosen and thelightpath is established between s and d. If none ofd ∈ D is available then the LR is rejected. Pseudocode1 formalizes the description of the strategy.

Strategy 1 closest [24]

Input: (s, D)

1: d ← null

2: dist ← ∞3: for all i ∈ D do4: if LAC(s, i) < dist then5: d ← i

6: dist ← LAC(s, i)7: end if8: end for9: if d �= null then

10: Establish lighpath between s and d

11: return d

12: else13: return null14: end if

In the closestGreen strategy the network controllerperforms operations analogous to the closest strategyfor all d ∈ Dg , where Dg is a subset of green DCswithin D. If none of d ∈ Dg is available then thenetwork controller repeats the same operation for allbrown DCs within D, d ∈ Db. If none of d ∈ D


are available then the LR is rejected. Pseudocode 2formalizes the description of the strategy.

Strategy 2 closestGreen [24]

Input: (s, D) = ((s, {Dg, Db})1: d ← null

2: dist ← ∞3: for all i ∈ Dg do4: if LAC(s, i) < dist then5: d ← i

6: dist ← LAC(s, i)7: end if8: end for9: if d �= null then

10: Establish lighpath between s and d

11: return d

12: else13: for all i ∈ Db do14: if LAC(s, i) < dist then15: d ← i

16: dist ← LAC(s, i)17: end if18: end for19: if d �= null then20: Establish lighpath between s and d

21: return d

22: else23: return null24: end if25: end if

The closestGreenWithPenalty strategy works anal-ogously to the closest strategy but with one significantdifference. The distance from s to d ∈ Db is mul-tiplied by the penalty factor, where the penalty is anSDN controller internal parameter. The closestGreen-WithPenalty strategy with penalty = 1.0 is equivalentto the closest strategy. Pseudocode 3 formalizes thedescription of the strategy.

5.2 Fitting Schemas

In general, there are many possible choices regardingthe fitting of an anycast schema to a particular cloudservice type. However, based on properties of bothcloud service types and anycast strategies we proposethe following fitting schema which was thoroughlyinvestigated in [25]. PaaS requests, which have the

Strategy 3 closestGreenWithPenalty [24]

Input: (s, D) = (s, {Dg, Db})1: d ← null2: dist ← ∞3: for all i ∈ D do4: if i ∈ Db then5: if LAC(s, i) · Penalty < dist then6: d ← i

7: dist ← LAC(s, i) · Penalty8: end if9: else if i ∈ Dg then

10: if LAC(s, i) < dist then11: d ← i

12: dist ← LAC(s, i)13: end if14: end if15: end for16: if d �= null then17: Establish lighpath between s and d

18: return d

19: else20: return null21: end if

highest energy requirements, should be handled by theclosestGreen strategy, which strictly favorises greenDCs over brown DCs, at a cost of increased aver-age lightpath length and increased network resourceutilization. StaaS requests, which have the lowestenergy requirements, should be handled by the closeststrategy, which does not provide any green nodes pref-erence, but is expected to ensure the shortest averagelightpath length and thus, the lowest network resourceoccupancy. Finally, SaaS, as the most undefined ser-vice type, should be handled by the closestGreenWith-Penalty strategy and the penalty parameter should beused to adjust the aggressiveness of the strategy to thechanging properties of SaaS requests.

The rationale behind the proposed schema is thatthe energy intensive tasks should be performed ingreen DCs while for other requests it is better tobalance network resource use and carbon footprintreduction. In subsequent studies this solution will bedenoted as the compound fitting schema to reflect theusage of different anycast strategies.

The closest, closestGreen and closestGreenWith-Penalty fitting schemas are opposed and describecases when all types of cloud services are handled


using the same anycast strategy, closest, closestGreenand closestGreenWithPenalty, respectively.

6 Cooperation Models

Models presented in this section concern the cooper-ation between the cloud operator (cloud orchestrationsoftware) and the network provider (an SDN con-troller). The models are applicable to cloud servicerequests which are served by both entities.

Some basic rules concerning the cooperation arecommon for all the models. A cloud service request,CR, arrives to cloud orchestration software and is fur-ther passed to an SDN controller as an anycast LR. Insome cases, additional information is attached to theanycast LR being passed. Thus, cloud orchestrationsoftware provides some input for the SDN controller.The SDN controller chooses the destination d forthe anycast LR and sets up the lightpath between s

and d. Then, cloud orchestration software receivesinformation about the selected d and tries to reserveresources and provision the requested service in thatnode. If the pointed d does not have enough resources,the cloud service request is blocked and the associ-ated lightpath is automatically torn down by the SDNcontroller.

The difference between cooperation models con-cerns mainly the amount of information beingexchanged between the cloud provider and the net-work operator. The less information is exchanged themore both entities are comfortable with cooperation.However, the cooperation is needed to effectively pro-vide cloud services and reduce greenhouse gases emis-sion. Thus, exchange of information through clearlydefined interfaces is a compromise in this situa-tion. Precisely defined interfaces ensure full controlover the information being exchanged and therefore,improve the comfort of cooperation.

All the analyzed cooperation models have beenpresented in a formal way using pseudocodes. Eachaction described in them is performed by the cloudorchestration software with the exception of actionswithin SDN(arguments) blocks. The SDN blockincludes actions performed by the SDN controller,while arguments denotes the information passed tothe controller by the cloud orchestration software. TheSDN controller uses the arwa(s,D) operation whichdenotes invoking the anycast strategy fitted to the

cloud service associated with current LR. The s andD are the inputs for the anycast strategy: the sourcenode and the set of possible destinations, respectively.At the same time, function name arwa denotes theanycast strategy invoked by the arwa(s,D) operation.The chosen fitting schema associates each LR with ananycast strategy and LR and CR carry this informa-tion. Additionally, there are some operations that arespecific to a particular anycast strategy. They are apart of the model and are explained in the pseudocodecomments.

6.1 The Overlay Model

Pseudocode 4 formalizes the description of the model.In this model, the cloud orchestration software sim-ply passes an anycast LR to the SDN controller anddoes not attach any additional information (1st line).Therefore, the overlay model assumes no cooperationbetween the network operator and the cloud provider.As the SDN controller does not have any knowledgeabout resources available in DCs it may choose a des-tination d by simply running anycast strategy on VDC

set (2nd line), but the chosen destination might not beable to handle the request. Subsequently, cloud orches-tration software must examine the chosen d in termsof resource availability (conditional statement in 4thline). If the amount of available resources is sufficientthe cloud request is handled by destination d (lines 5and 6). The main drawback of this model is that theSDN controller may consequently choose destinationsd which do not have sufficient resources. In such acase request is rejected (8th line). It may increase theblocking probability of cloud service requests despitethe fact that it would be possible to handle rejectedrequests in a different DC.

Cooperation model 4 overlay

Input: CR1: SDN (LR)2: d ← arwa(s,VDC)

3: end SDN4: if d �= null AND rd ≥ LRr then5: Reserve resources in d

6: rd ← rd − LRr

7: else8: Reject request9: end if


6.2 The Augmented Model

The augmented cooperation model is described usingPseudocode 5. In this model the cloud providerinforms the SDN controller about the set of DCs thathave enough resources to serve the request. This VrDC

set is created in the for loop between 2nd and 6thline and is further passed to the SDN controller, whichuses it as an input for the anycast strategy (7th and8th lines, respectively). Such a basic cooperation elim-inates the main drawback of the overlay model, asonly DCs with enough available resources are consid-ered by the SDN controller. The augmented model isexpected to significantly decrease the total blockingprobability in comparison to the overlay model. Therequests are rejected only if controller is not able tofind a lightpath to any of d ∈ VrDC which is expressedby a conditional statement in lines from 10th to 15th.However, the model’s decisions are based mainly onthe network topology and state while the state of thecloud infrastructure has secondary impact on them.

Cooperation model 5 augmented

Input: CR1: VrDC ← ∅2: for all d ∈ VDC do3: if rd ≥ LRr then4: VrDC ← VrDC ∪ {d}5: end if6: end for7: SDN (LR, VrDC)8: d ← arwa(s,VrDC)

9: end SDN10: if d �= null then11: Reserve resources in d

12: rd ← rd − LRr


6.3 The Peer Model

In this cooperation model, the cloud orchestrationsoftware provides the following information to theSDN controller: LR, VrDC, preferenceMode, dcth, CDC

and RDC. Thus, the SDN controller has knowledgeabout DCs that have sufficient IT resources (VrDC),

the preference criteria (preferenceMode and dcth willbe further explained), as well as the total amountof resources in each DC (CDC) and the amount ofresources currently available in each DC (RDC). In thefirst step, the SDN controller applies the anycast strat-egy to the set of possible destinations D = VrDC,searching for the destination with enough resources toprovide the service. Thus, the dreal obtained this waydenotes the destination selected by the SDN controllerassuming no preference of any DC.

A general idea of further steps is to limit theincrease of the average lightpath length resulting fromcloud operator’s preferences. Therefore, the differ-ences in distances between each d ∈ VrDC and drealare computed and normalized by the G(V, E) diam-eter. Subsequently, the normalized differences arecompared with the netth threshold which expressesthe increase in the average lightpath length accept-able by the network operator. If netth is not exceededthen preferences of the cloud operator expressed bythree parameters: rd (already introduced), dcth andpreferenceMode are considered. The dcth is a thresh-old, expressed as a fraction of the total DC capac-ity (cd ). The preferenceMode may be assigned oneof the two values: mostOccupied and leastOccupied.The mostOccupied mode denotes preferring DCs withavailable resources less than the assumed thresholdwhich means that d ∈ VDC must met the rd ≤dcth · cd constraint to be preferred. Analogously,leastOccupied mode denotes preferring DCs withavailable resources greater than the assumed thresh-old, thus d ∈ VDC must meet the rd ≥ dcth · cd

constraint to be preferred.The cooperation between the cloud operator and the

network provider is quite tight and if only the netththreshold is not exceeded then cloud provider’s prefer-ences are more significant. In this case, the most or theleast occupied DC is selected for the mostOccupied orthe leastOccupied modes, respectively. However, thecloud operator has to reveal internal information aboutits preference criteria and resources available in eachof DCs. Pseudocode 6 formalizes the description ofthe model. The time complexity of the listed algorithmis considered to beO(N) as there are three independentloops over selected network nodes and the constant isneglected in the big O notation.

To sum up, the presented cooperation models differfrom each other with regard to data being exchangedbetween a cloud operator and a network provider.


Cooperation model 6 Peer

Input: CR1: VrDC ← ∅2: for all d ∈ VDC do3: if rd ≥ LRr then4: VrDC ← VrDC ∪ {d}5: end if6: end for7: SDN (LR, VrDC, preferenceMode, dcth, CDC, RDC)8: d ← null9: dreal ← arwa(s,VrDC) � Real case without preferences

10: if dreal �= null then11: distreal ← LAC(s, dreal)12: if arwa = closestGreenWithPenalty AND dreal ∈ VbDC then13: distreal ← distreal · Penalty � Distance to brown node must be penalized14: end if15: for all i ∈ VrDC do16: disttmp ← LAC(s, i)17: if arwa = closestGreenWithPenalty AND i ∈ VbDC then18: disttmp ← disttmp · Penalty � Distance to brown node must be penalized19: end if20: if (disttmp − distreal)/diameter ≤ netth then21: if arwa = closest OR � Only green DC may preempt green DC22: dreal ∈ VbDC OR23: (dreal ∈ VgDC AND i ∈ VgDC) then24: if preferenceMode = mostOccupied then25: if rd ≤ dcth · cd then26: if d = null OR ri ≤ rd then27: d ← i

28: end if29: end if30: end if31: if preferenceMode = leastOccupied then32: if rd ≥ dcth · cd then33: if d = null OR ri ≥ rd then34: d ← i35: end if36: end if37: end if38: end if39: end if40: end for41: if d = null then42: d ← dreal43: end if44: end if45: end SDN46: if d �= null then47: Reserve resources in d48: rd ← rd − LRr



Fig. 2 The topology of theNSF network 0

1

2

34

5

6 7

9

10 11

13

128

On the one hand, the cloud operator and the net-work provider prefer to reveal as little information aspossible. However, they also want to effectively pro-vide services and reduce CO2 emission. Therefore, akind of compromise is necessary. Well defined inter-faces for information exchange between those twoentities and a precisely specified range of exchangedinformation seem to be the solution.

The cooperation models described in this sectionare just initial solutions and not the complete frame-works. In order to create the latter, a well defined pro-tocol for communication between the cloud orchestra-tion software and an SDN controller is first needed.A Northbound-API seems to be able to perform thisfunction. However, according to [2] it is not stan-dardized. For the simulation purposes lack of thecommunication protocol between cloud orchestrationsoftware and SDN controller was not the case. Bothentities were implemented within a single applicationinstance and exchange the information through soft-ware variables within this instance. Furthermore, theimplemented centralized controller is authors’ originalimplementation designed to cooperate with simulatorenvironment and not the instance of any existing SDNcontroller. Any kind of Northbound-API will be indis-pensable in realistic distributed scenarios where SDNcontroller and cloud orchestration software are fullyseparated.

7 Simulation Environment

To asses the presented fitting schemas and cooper-ation models several simulations were performed in

two reference networks: the 14 node National ScienceFoundation (NSF) network shown in Fig. 2 and the21 node Italian Mesh Network (ItalyNet) shown inFig. 3. Detailed parameters of the topologies are pro-vided in Table 1. Each physical link is composed oftwo fibers, one in each direction. Each fiber carries80 wavelengths transporting data at a rate of 10 Gb/s.Additionally, there are no wavelength converters in the

16

2 3

4

5

6

7

9

10

11

13

12

8

15

14

17

18

19

20

21

1

Fig. 3 The topology of the ItalyNet


Table 1 Topology parameters for investigated networks

Parameter NSF ItalyNet

Nodes 14 21

Links 21 34

Connectivity index (links/nodes) 1.5 1.62

Average nodal degree 3 3.24

Diameter [Hop] 3 6

No. of green DCs 2 3

No. of brown DCs 3 3

network and the wavelength assignment is done usingthe first-fit scheme.

In the NSF network five DCs were deployed. Theirlocation was chosen based on results presented in[45] (VDC ∈ {2, 4, 7, 9, 11}). In consecutive simu-lation studies different pairs of DCs from the men-tioned set were assumed to be green. Analogously,six DCs were deployed in ItalyNet and were locatedfollowing the assumption made in [18] (VDC ∈{1, 7, 11, 13, 15, 20}). Numerous sets of three DCswere assumed to be green, in this assumption we arealso consistent with [18].

The offered traffic is composed of two traffic types:background and DC traffic. The background trafficconsists of unicast LRs and is generated based ona uniform traffic matrix. The DC traffic consists ofcloud service requests generated only by s ∈ VC (withthe same intensity for each s) to a single anycast groupD = VDC.

The request holding time (ht) is exponentially dis-tributed. For the background traffic the mean value ofht is always equal to 10 s. In case of DC traffic htdepends on service type: for PaaS and StaaS the meanvalue is equal to 10 s, while for SaaS, it is a variableparameter, where the considered values are 100, 360and 1800 s. For subsequent considerations about inter-arrival time (iat) let us assume that the mean value ofht for SaaS is equal to 360 s.

In the base case each request is a unidirectionalrequest with exponentially distributed iat and themean intensity of 0.3795 (background traffic), 1.0956(PaaS and StaaS) and 0.0297 (SaaS) requests persecond for the NSF network and 0.132 (backgroundtraffic), 1.1778 (PaaS and StaaS) and 0.0319 (SaaS)requests per second for the ItalyNet. For SaaS and htvalues other than 360 s, iat is scaled so that the total

amount of generated traffic (product of mean valuesof iat and ht) remains constant.

As a consequence, the total network load in the basecase is 14 ·13 ·0.3795 ·10+9 ·1 ·(1.0956 ·10+1.0956 ·10+0.0297·360) = 690.69 Erl+ 293.44 Erl= 984.13Erl for the NSF network and 21 ·20 ·0.132 ·10+15 ·1 ·(1.1778 ·10+1.1778 ·10+0.0319 ·360) = 554.40 Erl+ 525.60 Erl = 1080.00 Erl for the ItalyNet. The totaltraffic in each network is a sum of two components.In the following explanation we will use numbers cor-responding to the NSF network. The first componentreflects the amount of background traffic generated byeach node (14) to every other node (13). The trafficbetween each pair of nodes has a given mean inten-sity (0.3795) and a given mean holding time (10). Thesecond component describes the amount of DC trafficgenerated by each client node (9) to a single anycastgroup (1) with the assumed mean intensities (1.0956and 1.0956 and 0.0297) and, respectively, holdingtimes (10 and 10 and 360) for different cloud services.

In order to asses the assumed scenarios under vary-ing loads we multiplied the mean intensity of the DCtraffic by a scaling factor ranging from 1.0 to 2.35(NSF) and from 1.0 to 1.63 (ItalyNet). The mean iatfor background traffic and the mean ht of all traffictypes remained unchanged. The resulting contributionof DC traffic to the overall traffic varies from 30 to50 % in the NSF network and from 49 to 60 % inthe ItalyNet. In this way we try to adjust traffic con-ditions to the predictions presented in [46]. However,more traffic classes were provided in [46] and contri-bution of DC traffic to the overall traffic is constant.Therefore, we cannot directly apply traffic proportionsfound in this paper to our work.

For each traffic type (unicast, PaaS, StaaS andSaaS) separate pseudo-random number generatorswere used to generate interarrival times and hold-ing times. A Mersenne-Twister generator was used asthe one with good randomness in up to 623 dimen-sions and period length of 219937 − 1. The generatorimplementation was provided by the Boost Randomlibrary.

In the aforementioned considerations the threetypes of services contribute to the total DC traf-fic in 1:1:1 proportions (PaaS:StaaS:SaaS). How-ever, in subsequent studies also 1:2:4 and 1:5:25proportions were investigated in order to reflectconsiderations provided in Section 3. Changing theproportion between service types was achieved by


scaling iat, while ht for each service type remainedunchanged.

As already mentioned in Section 4, we assume thatrequirements for IT resources of cloud services areproportional to their energy consumption. We alsoassume that in greenDCs all IT resources are poweredfrom renewable energy sources. Therefore, a PaaSrequest consumes 250000 units of IT resources, aStaaS request consumes 1 unit of IT resources and aSaaS request consumes 7000, 18750, 37500 or 67500units of IT resources for different values of energyconsumption, respectively. The capacity of each DC(cd ) is assumed to be the same for all d ∈ VDC and isadjusted to the investigated scenario. Its value dependson the contribution of cloud services to the overall DCtraffic and the assumed consumption of IT resourcesby SaaS requests. The exact value was experimen-tally scaled to the network resources with the aimto make neither IT nor network resources a singlebottleneck of the architecture. Instead, we aimed toget a situation where both IT and network resourcesare approximately equally utilized and both get sat-urated simultaneously when DC traffic increases.Table 2 presents the capacity of a single DC for eachconsidered case.

To sum up, the architecture was investigated undera heavy traffic scenario. It is a realistic assump-tion only during unexpected traffic peaks. Duringlow traffic hours there is no need for any onlineoptimization technique since an overprovisioned

architecture should be able to efficiently handle all therequests.

8 Results

Numerous simulation scenarios were investigated asmany parameters are variable. However, only illus-trative and representative simulation results will bepresented in a graphical form. Additionally, during thepreliminary assessment of the results we found outthat applying different values of the ht parameter forthe SaaS service resulted in similar network behavior.Thus, we present results only for the mean value of theht parameter for the SaaS service equal to 360 s andomit the remaining ones. This conclusion is consistentwith our previous work [25].

To asses the fitting schemas and the cooperationmodels, two indicators were used. The first one esti-mates the carbon footprint and is the average ratioof the power consumed from non-renewable sourcesto the amount of DC traffic switched in all DCs(brownKiloWatts/(Gb/s)). Thanks to that normal-ization we obtain comparable results under differenttraffic loads. The second indicator is the blockingprobability, calculated as the ratio of rejected requeststo all requests. The simulation results were obtainedusing the OMNeT++ simulator (implemented inC++, for details see [47]) and evaluated at the 0.95confidence level using the batch means method. The

Table 2 IT resources in DCs

Service types’ SaaS energy SaaS resources DC resources

proportions consumption consumption

1:1:1 1 7000 10639289

2.6 18750 11125713

5.4 37500 11901923

10 67500 13143858

1:2:4 1 7000 4932293

2.6 18750 5766164

5.4 37500 7096810

10 67500 9225842

1:5:25 1 7000 1702673

2.6 18750 2879507

5.4 37500 4757434

10 67500 7762117


number of independent observations was chosen to be31 as this assumption satisfies the rule according towhich a sample is considered large. Therefore, basedon the Central Limit Theorem, distribution of the sam-ple mean is approximately normal. Additionally, inorder to dispose the data gathered during a transientphase the warm-up period was estimated and validatedbased on the Central Limit Theorem. All results werepresented with regard to the aforementioned scalingfactor of DC traffic as a measure of traffic intensity.

For the sake of clarity the following systematiza-tion of provided results was introduced. Section 8.1demonstrates the way in which values were assignedto the aforementioned input parameters. Section 8.2presents the results obtained by different fittingschemas assuming a specific cooperation model.Based on results presented in Section 8.2 we were ableto choose the best fitting schema for each coopera-tion model. Such a choice allows, finally, for a faircomparison of different cooperation models, which isprovided in Section 8.3.

8.1 Input Parameters Values

In order to use the closestGreenWithPenalty strategythe value of the penalty parameter needs to be deter-mined. Similarly, in order to use the peer cooperationmodel the values of preferenceMode, dcth and netthinput parameters need to be specified. In both caseswe assume that the operator is minded to slightly dete-riorate network performance in order to reduce carbonfootprint. Based on that assumption the required inputparameters were determined using the methodologydescribed below.

We assume that for the case where the closest-GreenWithPenalty strategy is applied to any of theservice types, the total blocking probability may notincrease by more than 0.3 of a percentage point incomparison to the closest fitting schema, which isexpected to utilize the least network resources. Thus,for each simulation scenario the total blocking proba-bility was measured with penalty values ranging from1.0 to 4.0 in 0.1 steps, and the penalty values thatmet the aforementioned limitation were denoted asapplicable. For the closestGreenWithPenalty fittingschema, the highest penalty from the applicable setwas chosen, as this prefers green DCs as much asadmissible. At the same time, for the compound fit-ting schema the penalty value that resulted in the

highest reduction of CO2 emission was chosen amongthe ones in the applicable set. The penalty was deter-mined separately for each cooperation model and foreach fitting schema that uses the closestGreenWith-Penalty strategy. Additionally, in the case of the peercooperation model, the penalty must also be deter-mined separately for each investigated combination ofinput parameters.

In order to determine the values of input param-eters for the peer cooperation model we run prelim-inary simulations spanning numerous combinationsof values. The first conclusion is related to the netthvalue. The obtained results suggest that cloud oper-ator preferences should be taken into account onlywhen considering DCs equally distant to the requestsource. Choosing more distant DCs in order to satisfycloud operator preferences results in an unacceptableincrease of the request blocking probability. There-fore, netth = 0 is a reasonable value as it forcescomparing only equally distant DCs.

Second, we concluded that links connected tohighly utilized DCs also have a great number ofwavelengths occupied. A resulting small number ofavailable wavelengths may quickly get occupied bybackground traffic, especially if the DC is associatedwith the node being traversed by numerous shortestpaths connecting other nodes. Therefore, it is reason-able to fully utilize those highly occupied DCs aslong as there are still network resources available toachieve that. As a result the mostOccupied value isassigned to the preferenceMode parameter. Thanks tothat, IT resources are utilized more effectively andanycast mechanisms do not have to direct traffic tomore distant DCs, which would result in the increaseof the average lightpath length and in higher networkoccupancy.

Intuitively, directing traffic to most utilized DCsinstead of distributing it throughout the whole DCinfrastructure may create congested network areas.However, one must note that for a particular requestonly DCs equally distant to the source node are con-sidered. Therefore, there is no globally preferred DCthat serves all the requests. Additionally, in case ofthe augmented cooperation model the SDN controlleralso has to make some arbitrary decisions when DCsare equally distant to the source node. In this caseit is more likely to always choose the same destina-tion when considering the same set of equally distantDCs. Thus, preferring most occupied DCs should not


result in creation of congestion regions. Finally, theconsideration of results leads to the conclusion thatall equally distant DCs should be compared withregard to available resources. Therefore, dcth = 1 wasassumed as it does not exclude any DC from the set ofequally distant ones.

Verification of the aforementioned assumptionswas performed experimentally. Numerous simulationswere run with different values of preferenceMode,dcth and netth parameters for different VgDC sets inNSF and ItalyNet networks. For both possible valuesof preferenceMode various combination of changingdcth and netth parameters was investigated. The dcthvaried between 0 and 1 with 0.2 step providing a rea-sonable density of mesurement points. The netth waschanged with a step that reflects the network diame-ter. For example, the diameter of the NSF network isequal to 3, therefore netth was varying from 0 to 1 with1/3 = 0.33 step.

From the presented set of results the combination ofparameters that led to the lowest total blocking proba-bility was found. The results obtained for the proposedset of parameter values were verified whether theydo not exceed the lowest obtained blocking proba-bility by more than 0.3 of a percentage point. Theverification was positive, and, what is more, in some

scenarios the proposed parameter values were foundas the ones providing the lowest blocking probability.The results also show that the proposed parameter val-ues provide one of the lowest carbon footprint amongall considered input parameter combinations. It is theresult of the fact that when greenDCs are even slightlypreferred, they become the most utilized ones.

Figure 4 presents blocking probability and car-bon dioxide emission as a function of dcth, netthand preferenceMode parameters. This auxiliary figureproves and visualizes the aforementioned considera-tions about selecting values for the parameters. Thepresented results were obtained in the NSF net-work under exemplary simulation scenario where thepenalty parameter and anycast traffic scaling fac-tor were assigned values of 1.2 and 1.75, respec-tively. Please note, that plots for preferenceModeset to mostOccupied (upper row) and leastOccupied(lower row) are viewed from different angles. Thisis necessary to ensure proper readability of the fig-ures, but it results in dcth and netth axis swappingplaces.

To sum up, it is reasonable that for all subsequentstudies the peer cooperation model is investigatedwith the following input parameters: preferenceModeis set to mostOccupied, dcth = 1 and netth = 0.

Fig. 4 Total blockingprobability of all traffictypes and brown kilowattsneeded to handle 1 Gb/s ofDC requests in the NSFnetwork withVgDC ∈ {9, 11}, servicetypes’ proportions 1:2:4 andassumed SaaS energyconsumption equal to 5.4kW/(Gb/s) and anycasttraffic scaling factor equalto 1.75, penalty = 1.2 andpreferenceMode set tomostOccupied (upper row)and leastOccupied (lowerrow)


Fig. 5 Brown kilowattsneeded to handle 1 Gb/s ofDC requests in the NSFnetwork withVgDC ∈ {9, 11}, servicetypes’ proportions 1:2:4,assumed SaaS energyconsumption equal to 5.4kW/(Gb/s) and augmentedcooperation model

However, changing those values introduces flexibil-ity to the peer cooperation model. Thanks to thatflexibility, the peer model may be adjusted to totallydifferent conditions, e.g. other scaling of network andIT resources as well as larger or much more densetopologies.

8.2 Fitting Schemas

To compare different cooperation models the mostprominent fitting schema must be selected for eachof them. In case of the overlay model only the clos-est fitting schema is reasonable as this model doesnot assume any cooperation between a cloud providerand a network operator. Therefore, from the SDN con-troller’s point of view the best approach is to directcloud service requests to the closest DC. The aug-mented and peer models assume cooperation, so allof the fitting schemas must be considered. Figures 5

and 6 present the carbon footprint, while Figs. 7and 8 show total blocking probabilities of the ana-lyzed fitting schemas under the augmented and peercooperation models, respectively.

The closest fitting schema has consistently thehighest carbon footprint among the analyzed fittingschemas, as it provides no direct preference of greennodes. In the same time, its blocking probability maybe considered as a baseline for other results, as it effec-tively uses the shortest available paths to route all DCtraffic in the network.

Applying the closestGreen fitting schema results instrict preference of green DCs for each service type.This results in much lower carbon footprint comparedto the closest fitting schema. However, this comesat a cost of an unacceptable deterioration of perfor-mance, as this solution blindly tries to direct eachkind of cloud service requests to green DCs, evenif those DCs are distant. Additionally, in some cases

Fig. 6 Brown kilowattsneeded to handle 1 Gb/s ofDC requests in the NSFnetwork withVgDC ∈ {9, 11}, servicetypes’ proportions 1:2:4,assumed SaaS energyconsumption equal to 5.4kW/(Gb/s) and peercooperation model


Fig. 7 Total blockingprobability of all traffictypes in the NSF networkwith VgDC ∈ {9, 11},service types’ proportions1:2:4, assumed SaaS energyconsumption equal to 5.4kW/(Gb/s) and augmentedcooperation model

requests that require little energy may occupy a greatdeal of network resources near green DCs, whichmay divert energy-hungry requests to other DCs. Asa consequence, this may result in worse green energyutilization than in the compound fitting schema, whichbalances the green DCs preference with SaaS requestenergy requirements and the network resource utiliza-tion.

The closestGreenWithPenalty fitting schema yieldsa higher carbon footprint than closestGreen, as greenDCs preference is less firm in this fitting schema. Onthe other hand, its blocking probability is compara-ble to the blocking probability of the closest fittingschema.

The compound fitting schema yields substantialimprovements in the carbon footprint and achievesa similar blocking probability compared to the clos-est fitting schema. The obtained balance is attributed

to two facts. First, in this fitting schema we do notblindly try to direct SaaS requests from distant clientsto green DC nodes and, second, we always directStaaS requests to the closest DC, saving networkresources for other requests. Moreover, in some casesthis schema provides the lowest carbon footprint andthe lowest blocking probability among all consideredones. The additional decrease in the blocking proba-bility seems to be a side effect of location of greenDCs in parts of the network where less backgroundtraffic is switched. As a consequence, DC traffic isload balanced more evenly and the overall networkcongestion is limited.

Based on the presented results, we decided to use insubsequent studies the closest fitting schema with theoverlay cooperation model and the compound fittingschema with the augmented and the peer cooperationmodels.

Fig. 8 Total blockingprobability of all traffictypes in the NSF networkwith VgDC ∈ {9, 11},service types’ proportions1:2:4, assumed SaaS energyconsumption equal to 5.4kW/(Gb/s) and peercooperation model


Fig. 9 Brown kilowattsneeded to handle 1 Gb/s ofDC requests in the NSFnetwork withVgDC ∈ {9, 11}, servicetypes’ proportions 1:2:4 andassumed SaaS energyconsumption equal to 5.4kW/(Gb/s)

Fig. 10 Brown kilowattsneeded to handle 1 Gb/s ofDC requests in the NSFnetwork withVgDC ∈ {2, 9}, servicetypes’ proportions 1:2:4 andassumed SaaS energyconsumption equal to 10kW/(Gb/s)

Fig. 11 Brown kilowattsneeded to handle 1 Gb/s ofDC requests in the ItalyNetwith VgDC ∈ {7, 15, 20},service types’ proportions1:2:4 and assumed SaaSenergy consumption equalto 5.4 kW/(Gb/s)


Fig. 12 Total blockingprobability of all traffictypes in the NSF networkwith VgDC ∈ {9, 11},service types’ proportions1:2:4 and assumed SaaSenergy consumption equalto 5.4 kW/(Gb/s)

Fig. 13 Total blockingprobability of all traffictypes in the NSF networkwith VgDC ∈ {2, 9}, servicetypes’ proportions 1:2:4 andassumed SaaS energyconsumption equal to 10kW/(Gb/s)

Fig. 14 Total blockingprobability of all traffictypes in the ItalyNet withVgDC ∈ {7, 15, 20}, servicetypes’ proportions 1:2:4 andassumed SaaS energyconsumption equal to 5.4kW/(Gb/s)


Fig. 15 Background trafficblocking probability in theNSF network withVgDC ∈ {9, 11}, servicetypes’ proportions 1:2:4 andassumed SaaS energyconsumption equal to 5.4kW/(Gb/s)

8.3 Cooperation Models

Figures 9, 10 and 11 present the carbon footprint ofthe analyzed cooperation models in different simula-tion scenarios. The aforementioned metric and energymodels were used. Figures 12, 13 and 14 show thecorresponding total blocking probabilities.

The overlay model has constantly the highest totalblocking probability among the analyzed coopera-tion models. This is a direct consequence of the factthat it assumes no cooperation between the networkcontroller and the orchestration software. A destina-tion DC is chosen solely based on network resourceavailability without any consideration for the avail-ability of IT resources. Thus, it may result in choos-ing a DC which cannot handle the request. On theother hand, other DCs may still be able to accept therequest, but they will not be taken into account dueto current network conditions. Thus, DC requests may

be blocked even if a great deal of IT and networkresources is still available.

Additionally, in most simulation scenarios theoverlay model resulted in the highest carbon foot-print, as using the closest fitting schema providesno direct preference of green nodes and use ofother schemas is impossible due to lack of neces-sary information about green DCs. However, in somecases the carbon footprint of the overlay coopera-tion model is lower than that provided by the peeror the augmented models. It happens when greenDCs are located statistically closer to client nodesthan brown DCs. Therefore, traffic is directed togreen DCs if only network resources are available,as the overlay model does not take into account DCresources availability. When a chosen green DC isfully utilized then the request is rejected withoutconsidering any brown DCs. Figure 11 presents thiscase.

Fig. 16 DC trafficblocking probability in theNSF network withVgDC ∈ {9, 11}, servicetypes’ proportions 1:2:4 andassumed SaaS energyconsumption equal to 5.4kW/(Gb/s)


Applying the augmented cooperation model resultsin a decrease of the total blocking probability in com-parison to the overlay model. It is a result of the factthat the SDN controller tries to establish lightpathsonly to DCs that have available IT resources. There-fore, the augmented model excludes situations wherea cloud service request is blocked despite reachabilityof a DC with sufficient IT resources. However, Fig. 13presents the scenario where the overlay model pro-vides lower blocking probability than the augmentedmodel for high traffic load. This phenomenon appearsbecause the total blocking probability is a superposi-tion of the background and DC traffic blocking prob-abilities. In the augmented model the controller triesto direct traffic to DCs with sufficient IT resourceseven if those DCs are distant. This results in the lowerblocking probability for anycast traffic but consumesmore network resources. As a result, the blockingprobability of the background traffic is much higher.On the other hand, the overlay model always tries todirect traffic to the closest reachable DC, no matter ifenough IT resources are available. This results in thehigher blocking probability for anycast requests butthe lower blocking probability of background traffic.This issue is addressed later in the text and illustratedin Figs. 15 and 16.

The peer, proposed cooperation model, constantlyprovides the best total blocking probability amongall the considered models. The additional informationprovided by the cloud orchestration software limitsthe number of situations where available IT resourcesare unreachable due to network congestion. Thus, themodel allows for efficient utilization of IT resources.Simultaneously, the carbon footprint is maintained atthe same level as in the augmented model.

In order to more precisely explain the differencesbetween the models additional results are provided.Figures 15 and 16 present the blocking probabilitiesof background and DC traffic, respectively. Anycast,compared to unicast traffic, has an additional degreeof freedom related to the choice of a destination node.Therefore, in the presence of network congestion any-cast requests have less likelihood of being affected.In both the augmented and the peer models, DC traf-fic is blocked with similar probability, lower thanin case of the overlay model. However, in the aug-mented model DC traffic is routed in a less efficientway. This results in a greater overall congestion of thenetwork and is reflected in the blocking probability

of background traffic, which is higher in the aug-mented model, compared to both the overlay and peermodels.

In the paper we limited the set of presented resultsto the most exemplary and illustrative ones. How-ever, the provided conclusions are valid for all ofthe obtained results. Additionally, we observed thatchanging the proportions between DC service typesfrom 1:1:1 through 1:2:4 to 1:5:25 results, in mostcases, in better reduction of the carbon footprint whilepreserving blocking probability at the same level.Increasing the assumed energy consumption of theSaaS service results in similar conclusions.

9 Conclusions

Current cloud architectures are composed of twoclearly defined entites: a network interconnecting datacenters and IT infrastructure within them. We state,that in order to effectively provide cloud services withminimal negative impact on the natural environmenta cooperation is necessary between those two enti-ties. However, it is necessary to remember, that closecooperation comes at the expense of some informa-tion disclosure between cooperating parties and is theadditional potential for security risks.

In the paper, we investigated models of coopera-tion between a network operator and a cloud serviceprovider. The models are conceptually simple and arewell placed in the control plane of modern cloud archi-tectures. Our aim was to analyze their impact on therequest blocking probability and carbon footprint.

The proposed peer model provides consistentlythe lowest blocking probability in comparison to theoverlay and augmented models. Additionally, its car-bon footprint remains at the level comparable to theaugmented model. Therefore, the peer model mayimprove service quality and, when joined with greenanycast strategies and an appropriate fitting schema,it may significantly decrease carbon dioxide emissionof the cloud. Numerous simulations performed in var-ious scenarios support this conclusion. Additionally,the results show that the compound fitting schema,introduced and analyzed in our previous work, is thebest one for this model.

The proposed cooperation model is flexible andadjustable to the network conditions thanks to pro-grammable thresholds and DC preference modes.


However, an appropriate choice of values for thoseparameters is not an easy task, and requires fur-ther investigation. For real deployments a methodto quickly estimate the values of those parametersis necessary in order to adapt the behaviour of themodel to the current state of IT resources and networkconditions. This subject is, however, left for furtherstudies.

Despite the fact that we focused on the all opti-cal WDM networks our solutions may be adoptedto other networking technologies that utilize theconcept of an end-to-end path, such as MultiprotocolLabel Switching (MPLS) or Elastic Optical Networks(EON). MPLS is very popular in contemporary WANswhile EON is an emerging technology that seems tobe the future of optical networks. A modification ofthe proposed solutions to such technologies is also aninteresting area for future work.

The investigated problem of cooperation between anetwork controller and cloud orchestration software isone of the most important issues that needs to be dealtwith for efficient and environmentally friendly cloudservices provisioning. We believe that the presentedarchitecture and the proposed solutions will be a stepforward in the process of solving this problem and thatthey will become a part of the green cloud computingconcept.

Acknowledgments This work was funded by the NCBiR andCHIST-ERA ERA-Net SwiTching And tRansmission project.

Open Access This article is distributed under the terms of theCreative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unre-stricted use, distribution, and reproduction in any medium,provided you give appropriate credit to the original author(s)and the source, provide a link to the Creative Commons license,and indicate if changes were made.

References

1. Gupta, M., Singh, S.: Greening of the internet. In: Proceed-ings of the ACM SIGCOMM 2003, pp. 19–26, Karlsruhe(2003)

2. Jarschel, M., Zinner, T., Hossfeld, T., Tran-Gia, P., Kellerer,W.: Interfaces, attributes, and use cases: a compass forSDN. IEEE Commun. Mag. 52(6), 210–217 (2014)

3. Ahmed, R., Boutaba, R.: Design considerations for manag-ing wide area software defined networks. IEEE Commun.Mag. 52(7), 116–123 (2014)

4. Yangui, S., Marshall, I.J., Laisne, J.P., Tata, S.: Compat-ibleone: the open source cloud broker. J. Grid Comput.12(1), 93–109 (2014)

5. Petcu, D.: Consuming resources and services from multipleclouds. J. Grid Comput. 12(2), 321–345 (2014)

6. Jain, S., Kumar, A., Mandal, S., Ong, J., Poutievski, L.,Singh, A., Venkata, S., Wanderer, J., Zhou, J., Zhu, M.,Zolla, J., Holzle, U., Stuart, S., Vahdat, A.: B4: expe-rience with a globally-deployed software defined Wan.In: Proceedings of the ACM SIGCOMM 2013, pp. 3–14(2013)

7. Alger, D.: Grow a greener data center. Cisco Press, Indi-anapolis (2010)

8. Lewis, A.: The four key environmental factors of ICT:Energy, carbon, e-waste and water. http://www.sustainability-perspectives.com/article/the-four-key-environmental-factors-of-ict-energy-carbon-e-waste-and-water/(2014).Cited 07 Oct 2014

9. Lannoo, B.: Energy consumption of ICT networks -TREND final workshop. http://www.fp7-trend.eu/system/files/content-public/502-final-trend-workshop-brussels-24-october-2013-presentations/energyconsumptionincentives-energy-efficient-networks.pdf (2013). Cited 07 Oct 2014

10. The Green Grid: Carbon,Water and Energy EfficiencyMet-rics, Measurements and Trends for Data Center Planning.http://www.thegreengrid.org/ (2014). Cited 07 Oct 2014

11. Greenpeace: How Companies are Creating the Green Inter-net. http://www.greenpeace.org/usa/Global/usa/planet3/PDFs/clickingclean.pdf (2014). Cited 07 Oct 2014

12. Facebook: Green On Facebook - Carbon and Energy.https://www.facebook.com/green/app 439663542812831?ref=page internal (2014). Cited 07 Oct 2014

13. Google: Data Centers Renewable Energy. www.google.pl/about/datacenters/renewable (2014). Cited 07 Oct 2014

14. Kliazovich, D., Arzo, S., Granelli, F., Bouvry, P., Khan, S.:Accounting for load variation in energy-efficient data cen-ters. In: Proceedings of the IEEE International Conferenceon Communications (ICC 2013), pp. 2561–2566, Budapest(2013)

15. Wang, X., Razo, M., Tacca, M., Fumagalli, A.: Planningand online resource allocation for the multi-resource cloudinfrastructure. In: Proceedings of the IEEE InternationalConference on Communications (ICC 2014), pp. 2944–2949, Sydney (2014)

16. Liu, X., Zhang, L., Zhang, M., Zhu, Z.: Joint defrag-mentation of spectrum and computing resources in inter-datacenter networks over elastic optical infrastructure. In:Proceedings of the IEEE International Conference on Com-munications (ICC 2014), pp. 3295–3300, Sydney (2014)

17. Lin, T., Kang, J.M., Bannazadeh, H., Leon-Garcia, A.:Enabling SDN applications on software-defined infrastruc-ture. In: Proceedings of the IEEE Network Operations andManagement Symposium (NOMS 2014), pp. 1–7, Krakow(2014)

18. Gattulli, M., Tornatore, M., Fiandra, R., Pattavina, A.:Low-Carbon routing algorithms for cloud computing ser-vices in IP-over-WDM networks. Proceedings of the IEEEInternational Conference on Communications (ICC 2012),pp. 2999–3003, Ottawa (2012)

19. Gattulli, M., Tornatore, M., Fiandra, R., Pattavina, A.: Low-emissions routing for cloud computing in IP-over-WDM

http://creativecommons.org/licenses/by/4.0/

http://creativecommons.org/licenses/by/4.0/

http://www.sustainability-perspectives.com/article/the-four-key-environmental-factors-of-ict-energy-carbon-e-waste-and-water/



http://www.fp7-trend.eu/system/files/content-public/502-final-trend-workshop-brussels-24-october-2013-presentations/energyconsumptionincentives-energy-efficient-networks.pdf




http://www.thegreengrid.org/

http://www.greenpeace.org/usa/Global/usa/planet3/PDFs/clickingclean.pdf

http://www.greenpeace.org/usa/Global/usa/planet3/PDFs/clickingclean.pdf

https://www.facebook.com/green/app_439663542812831?ref=page_internal

https://www.facebook.com/green/app_439663542812831?ref=page_internal

www.google.pl/about/datacenters/renewable

www.google.pl/about/datacenters/renewable


networks with data centers. IEEE J. Sel. Areas Commun.32(1), 28–38 (2014)

20. Walkowiak, K., Kasprzak, A., Klinkowski, M.: Dynamicrouting of anycast and unicast traffic in elastic opticalnetworks. In: Proceedings of the IEEE International Con-ference on Communications (ICC 2014), pp. 3313–3318,Sydney (2014)

21. Rzasa, J., Borylo, P., Lason, A., Szymanski, A., Jajszczyk,A.: Dynamic power capping for multilayer hybrid powernetworks. In: Proceedings of the IEEE Global Communica-tions Conference (GLOBECOM 2014). Austin (2014)

22. Mandal, U., Habib, M., Shuqiang, Z., Mukherjee, B., Tor-natore, M.: Greening the cloud using renewable-energy-aware service migration. IEEE Netw. 27(6), 36–43 (2013)

23. Rodero, I., Viswanathan, H., Lee, E., Gamell, M., Pom-pili, D., Parashar, M.: Energy-efficient thermal-aware auto-nomic management of virtualized HPC cloud infrastruc-ture. J. Grid Comput. 10(3), 447–473 (2012)

24. Borylo, P., Lason, A., Rzasa, J., Szymanski, A., Jajszczyk,A.: Anycast routing for carbon footprint reduction in WDMhybrid power networks with data centers. In: ProceedingsIEEE International Conference on Communications (ICC2014), pp. 3720–3726, Sydney (2014)

25. Borylo, P., Lason, A., Rzasa, J., Szymanski, A., Jajszczyk,A.: Fitting green anycast strategies to cloud services inWDM hybrid power networks. In: Proceedings of the IEEEGlobal Communications Conference (GLOBECOM 2014),Austin (2014)

26. Develder, C., Leenheer, M.D., Dhoedt, B., Pickavet, M.,Colle, D., Turck, F.D., Demeester, P.: Optical networks forgrid and cloud computing applications. Proc. IEEE 100(5),1149–1167 (2012)

27. Amazon: Amazon ec2. http://aws.amazon.com/ec2/ (2014).Cited 07 Oct 2014

28. Open Stack: Open Source Cloud Computing Software.https://www.openstack.org/ (2014). Cited 07 Oct 2014

29. Open Nebula: Flexible Enterprise Cloud Made Simple.http://opennebula.org/ (2014). Cited 07 Oct 2014

30. Cloud Stack: Open Source Cloud Computing. http://cloudstack.apache.org/ (2014). Cited 07 Oct 2014

31. Boru, D., Kliazovich, D., Granelli, F., Bouvry, P., Zomaya,A.: Energy-Efficient data replication in cloud computingdatacenters. In: Proceedings of the IEEE Global Commu-nications Conference Workshops (GLOBECOM Wkshps2013), pp. 446–451, Atlanta (2013)

32. McKeown, N., Anderson, T., Balakrishnan, H., Parulkar,G., Peterson, L., Rexford, J., Shenker, S., Turner, J.:Openflow: enabling innovation in campus networks. ACMSIGCOMM Comput. Commun. Rev. 38(2) (2008)

33. OpenDaylight: A Linux Foundation Collaborative Projects.http://www.opendaylight.org/ (2014). Cited 07 Oct 2014

34. Floodlight: Open Source Software for Building Soft-ware Defined Networks. http://www.projectfloodlight.org/(2014). Cited 07 Oct 2014

35. Stanford: Beacon. https://openflow.stanford.edu/display/Beacon/Home (2014). Cited 07 Oct 2014

36. Big Switch: Big Cloud Fabric. http://www.bigswitch.com/sdn-products/big-cloud-fabric (2014). Cited 07 Oct2014

37. Dorronsoro, B., Nesmachnow, S., Taheri, J., Zomaya,A.Y., Talbi, E.G., Bouvry, P.: A hierarchical approach forenergy-efficient scheduling of large workloads in multi-core distributed systems. Sustainable Computing: Infor-matics and Systems. doi:10.1016/j.suscom.2014.08.003(2014)

38. Nesmachnow, S., Dorronsoro, B., Pecero, J., Bouvry,P.: Energy-Aware Scheduling on multicore heterogeneousgrid computing systems. J. Grid Comput. 11(4), 653–680(2013)

39. Khajemohammadi, H., Fanian, A., Gulliver, T.: Efficientworkflow scheduling for grid computing using a leveledmulti-objective genetic algorithm. J. Grid Comput. 12(4),637–663 (2014)

40. Trger, P., Merzky, A.: Towards standardized job submissionand control in infrastructure clouds. J. Grid Comput. 12(1),111–125 (2014)

41. Google: Cluster Data Trace of Google Workloads. https://code.google.com/p/googleclusterdata/ (2014). Cited 07 Oct2014

42. Baliga, J., Ayre, R., Hinton, K., Tucker, R.S.: Green cloudcomputing: balancing energy in processing, storage, andtransport. Proc. IEEE 99(1), 149–167 (2011)

43. Chlamtac, I., Ganz, A., Karmi, G.: Lightpath communica-tions: an approach to high bandwidth optical WAN’s. IEEETrans. Commun. 40(7), 1171–1182 (1992)

44. Szymanski, A., Lason, A., Rzasa, J., Jajszczyk, A.: Per-formance evaluation of the grade-of-service-based routingstrategies for optical networks. In: Proceedings of the IEEEInternational Conference on Communications (ICC 2008),pp. 5252–5257, Beijing (2008)

45. Dong, X., El-Gorashi, T., Elmirghani, J.M.H.: Green IPover WDM networks with data centers. IEEE J. LightwaveTechnol. 29(12), 1861–1880 (2011)

46. Klinkowski, M., Walkowiak, K.: On the advantages of elas-tic optical networks for provisioning of cloud computingtraffic. IEEE Netw. 27(6), 44–51 (2013)

47. Varga, A.: Using the OMNeT++ discrete event simulationsystem in education. IEEE Trans. Educ. 42(4), 11 (1997)

http://aws.amazon.com/ec2/

https://www.openstack.org/

http://opennebula.org/

http://cloudstack.apache.org/

http://cloudstack.apache.org/

http://www.opendaylight.org/

http://www.projectfloodlight.org/

https://openflow.stanford.edu/display/Beacon/Home

https://openflow.stanford.edu/display/Beacon/Home

http://www.bigswitch.com/sdn-products/big-cloud-fabric

http://www.bigswitch.com/sdn-products/big-cloud-fabric

http://dx.doi.org/10.1016/j.suscom.2014.08.003

https://code.google.com/p/googleclusterdata/

https://code.google.com/p/googleclusterdata/

green cloud provisioning throughout cooperation of a wdm ... · green cloud provisioning throughout...

Documents