d2.32 - specification of the infrastructure virtualisation ... · orchestration and management of...

T-NOVA|DeliverableD2.32 SpecificationoftheInfrastructureVirtualisation,ManagementandOrchestration-Final

©T-NOVAConsortium

1

DeliverableD2.32

SpecificationoftheInfrastructureVirtualisation,ManagementandOrchestration-Final

Editor MichaelJ.McGrath(INTEL)

Contributors VincenzoRiccobene(INTEL),PedroNeves,JoséBonnet,JorgeCarapinha,AntonioGamelas(PTIN),DoraChristofi,GeorgiosDimosthenous(PTL),BeppeCoffano,LucaGalluppi,PierangeloMagli,MarcoDiGirolamo(HP),LetterioZuccaro,FedericoCimorelli,RobertoBaldoni(CRAT),GeorgeXilouris,EleniTrouva,AkisKourtis(NCSRD),ZdravkoBozakov,PanagiotisPapadimitriou(LUH),JordiFerrerRiera(i2CAT),PiyushHarsh,IrenaTrajkovska(ZHAW),KimonKarras(FINT),PaoloComi(ITALTEL)

Version 1.0

Date October28th,2015


©T-NOVAConsortium

2

ExecutiveSummaryThe specification presented in this document represents the final version for theInfrastructureVirtualisation,ManagementandOrchestration layersof theT-NOVAsystem.ThisspecificationutilisestherequirementsdescribedinpreviousdeliverablestogetherwiththelatestNetworkFunctionsVirtualisation(NFV)andvirtualisationrequirementsdefinedbyvariousindustrybodiesincludingtheETSIISGNFVandITU-T,aswellasexcerptsofrelevantpartsoftheETSIISGMANOWGarchitectureandassociatedFunctionalEntities(FEs).The concepts of virtualisation and NFV/SDN have a key influence on the design andimplementationoftheInfrastructureVirtualisation,ManagementandOrchestrationlayers.These approaches have respective pros and cons of both in the context of the T-NOVAsystem. TheOrchestration and Infrastructure Virtualisation andManagement (IVM) layersbuildtheseconceptstocreateafunctionalarchitecturethatformsakeyconstituentoftheoverallT-NOVAsystemarchitecture.TherelationshipbetweentheT-NOVAOrchestratorandtheNFVparadigmplaysakeyroleinitsdesign.TheOrchestratorneedstoaddressthechallengesassociatedwiththedeliveryofa suitable solution to support the deployment and management of virtualised networkservices (NS). The elucidation of associated functional requirements plays a key role inhelpingtoaddressesthesechallenges.Categorisingtherequirementsalsohelpsinensuringa suitable level of granularity which makes sure all aspects of system functionality areappropriately captured and documented. These requirements are then used to identifysystemcomponentssuchasinterfaces,cataloguesetc.andtheirassociatedfunctionalities.The IVM layer is responsible to the provisioning and management of the infrastructurewhich provides the execution environment for T-NOVA’s network services. The overalldesignofthelayerisdrivenbyavarietyofrequirementssuchasperformance,elasticityetc.The IVM layer is comprised of the Network Functions Virtualised Infrastructure (NFVI),Virtual InfrastructureManager(VIM)andWANInfrastructureConnectionManager(WICM)functional entities which are linked together with the other T-NOVA system componentsthrough various internal and external interfaces. The NFVI is comprised of compute,hypervisorandnetworkconstituentdomainswhoseobjectivesandrequirementsneedtobecarefullyconsideredinthecontextoftheoverallIVMdesignandimplementation.With any architectural design it is important to interrogate it in order todetermine if thedesignappropriately supports itsassociated requirementsanddesigngoals.The referencearchitectures for the IVM and Orchestration layers were interrogated and validated at afunctional level through the development of NS and virtualised network function (VNF)sequence diagrams. These workflow diagrams illustrate the key actions and interactionswithin the layers during standard operational activities related to the deployment andmanagementofVNF’sandNSs.In the design and implementation of any sophisticated system, gaps in availabletechnologieswillalmostcertainlyemerge. If significant thesegapsaffordopportunities fornew research and innovation. A focused gap analysis was carried out to determine thecurrent technology gapswhichaffect theT-NOVA system togetherwithpotential steps toaddressthem.


©T-NOVAConsortium

3

TableofContents

1.INTRODUCTION..............................................................................................................8

1.1.VIRTUALISATION.................................................................................................................81.1.1 TheVirtualisationConcept................................................................................81.1.2 TheProsandConsofNFVDeployments............................................................9

1.2 THET-NOVASOLUTION...............................................................................................101.2.1 TheT-NOVAOrchestrationPlatform(PTIN)....................................................111.2.2 TheT-NOVAIVMPlatform...............................................................................12

2.THET-NOVAORCHESTRATIONLAYER...........................................................................14

2.1.ORCHESTRATIONLAYEROVERVIEW......................................................................................142.2.ORCHESTRATORREQUIREMENTS.........................................................................................16

2.2.1.NFVORequirementsTypes....................................................................................172.2.1.1.NSLifecycle...................................................................................................................................182.2.1.2.VNFLifecycle.................................................................................................................................182.2.1.3.ResourceHandling........................................................................................................................192.2.1.4.MonitoringProcess.......................................................................................................................192.2.1.5.ConnectivityHandling...................................................................................................................202.2.1.6.PolicyManagement......................................................................................................................202.2.1.7.Marketplace-specificInteractions................................................................................................20

2.2.2.VNFMRequirementsTypes...................................................................................212.2.2.1.VNFLifecycle.................................................................................................................................212.2.2.2.MonitoringProcess.......................................................................................................................21

2.3.FUNCTIONALORCHESTRATORARCHITECTURE........................................................................222.3.1.ReferenceArchitecture..........................................................................................222.3.2.FunctionalEntities.................................................................................................23

2.3.2.1.NetworkFunctionVirtualisationOrchestrator(NFVO).................................................................232.3.2.2.VirtualNetworkFunctionManager(VNFM).................................................................................262.3.2.3.RepositoriesandCatalogues.........................................................................................................28

2.3.3.ExternalInterfaces.................................................................................................282.3.3.1.InterfacebetweentheOrchestratorandtheNetworkFunctionStore........................................292.3.3.2.InterfacebetweentheOrchestratorandtheMarketplace..........................................................292.3.3.3.InterfacebetweentheOrchestratorandtheVIM........................................................................302.3.3.4.InterfacebetweentheOrchestratorandtheTransportNetworkManagement.........................312.3.3.5.InterfacebetweentheOrchestratorandtheVNF........................................................................31

3.THET-NOVAIVMLAYER...............................................................................................33

3.1.INTRODUCTION............................................................................................................333.2.OBJECTIVESANDCHARACTERISTICSOFTHET-NOVAIVMLAYER.................................333.3.T-NOVAIVMLAYERREQUIREMENTS............................................................................343.4.VIRTUALINFRASTRUCTUREMANAGER(VIM)........................................................................35

3.4.1.WANInfrastructureConnectionManager.............................................................363.4.2.NFVICompute........................................................................................................363.4.3.NFVIHypervisor.....................................................................................................363.4.4.NFVIDCNetwork...................................................................................................37

3.5.T-NOVAIVMARCHITECTURE............................................................................................373.5.1.ExternalInterfaces.................................................................................................383.5.2.InternalIVMInterfaces..........................................................................................40

3.6.NFVIANDNFVI-POP........................................................................................................423.6.1.ITResources...........................................................................................................42

3.6.1.1.ComputeDomain..........................................................................................................................433.6.1.2.HypervisorDomain.......................................................................................................................47


©T-NOVAConsortium

4

3.6.2.InfrastructureNetworkDomain............................................................................483.7.VIRTUALISEDINFRASTRUCTUREMANAGEMENT......................................................................49

3.7.1.ITResourceManagementandMonitoring............................................................513.7.1.1.HypervisorManagement..............................................................................................................513.7.1.2.ComputingResourcesManagement............................................................................................52

3.7.2.InfrastructureNetworkResourcesManagementandMonitoring........................533.8.WANINFRASTRUCTURECONNECTIONMANAGER..................................................................543.9.WICM:CONNECTINGVNFSTOTHEWAN...........................................................................55

3.9.1.1.SDN-enabledNetworkElements..................................................................................................563.9.1.2.LegacyNetworkElements.............................................................................................................57

4 T-NOVAVNFSANDNSSPROCEDURES.......................................................................58

4.1 VNFRELATEDPROCEDURES...........................................................................................584.1.1 On-boarding....................................................................................................584.1.2 Instantiation....................................................................................................594.1.3 Monitoring.......................................................................................................634.1.4 Scale-out..........................................................................................................644.1.5 Termination.....................................................................................................65

4.2 NSRELATEDPROCEDURES.............................................................................................684.2.1 On-boarding....................................................................................................684.2.2 Instantiation....................................................................................................684.2.3 WANConnections............................................................................................714.2.4 Supervision......................................................................................................714.2.5 Scale-out..........................................................................................................734.2.6 Termination.....................................................................................................76

4.3 NS,VNFANDINFRASTRUCTUREMONITORING.................................................................76

5 GAPANALYSIS..........................................................................................................80

5.1 COMPUTE...................................................................................................................805.2 HYPERVISOR................................................................................................................805.3 SDNCONTROLLERS......................................................................................................815.4 CLOUDCONTROLLERS...................................................................................................825.5 NETWORKVIRTUALISATION...........................................................................................835.6 NFVORCHESTRATOR...................................................................................................85

6 CONCLUSIONS..........................................................................................................87

ANNEXA-ORCHESTRATORREQUIREMENTS....................................................................89

A.1 INTERNALREQUIREMENTS.............................................................................................90A.1.1 NFVORequirements........................................................................................90

A.1.1.1 NSLifecyclerequirements......................................................................................................90A.1.1.2 VNFLifecyclerequirements....................................................................................................91A.1.1.3 ResourceHandlingRequirements..........................................................................................92A.1.1.4 MonitoringProcessrequirements..........................................................................................92A.1.1.5 ConnectivityHandlingrequirements......................................................................................93A.1.1.7 Marketplace-specificinteractionsrequirements...................................................................94

A.1.2 VNFMRequirements........................................................................................94A.1.2.1 VNFLifecyclerequirements....................................................................................................94A.1.2.2 MonitoringProcessrequirements..........................................................................................95

A.2 INTERFACEREQUIREMENTS............................................................................................96A.2.1 InterfacewithVIM...........................................................................................96A.2.2 InterfacewithVNF...........................................................................................98A.2.3 InterfacewithMarketplace.............................................................................99

ANNEXB-VIRTUALISEDINFRASTRUCTUREMANAGEMENTREQUIREMENTS..................101


©T-NOVAConsortium

5

B.1 VIRTUALINFRASTRUCTUREMANAGEMENTREQUIREMENTS..............................................101B.2 WANINFRASTRUCTURECONNECTIONMANAGERREQUIREMENTS.....................................103B.3 NFVINFRASTRUCTUREREQUIREMENTS.........................................................................104

B.3.1 Computing.....................................................................................................104B.3.2 Hypervisor......................................................................................................107B.3.3 Networking....................................................................................................108

ANNEXC-TERMINOLOGY..............................................................................................111

C.1 GENERALTERMS........................................................................................................111C.2 ORCHESTRATIONDOMAIN...........................................................................................111C.3 IVMDOMAIN...........................................................................................................112

LISTOFACRONYMS........................................................................................................113

REFERENCES...................................................................................................................118


©T-NOVAConsortium

6

IndexofFigures

Figure1-1High-levelviewofoverallT-NOVASystemArchitecture........................................11Figure2-1:NSsandVNFsComplexOrchestrationOverview..................................................15Figure2-2:T-NOVAOrchestratorReferenceArchitecture......................................................22Figure2-3:NSOrchestrator(Internal&External)Interactions...............................................24Figure2-4:VirtualisedResourcesOrchestrator(Internal&External)Interactions................26Figure2-5:VNFManager(Internal&External)Interactions...................................................27Figure 3-1: T-NOVA infrastructure virtualisation and management (IVM) high levelarchitecture.............................................................................................................................38Figure3-2:ComputeDomainHighLevelArchitecture............................................................46Figure3-3:Hypervisordomainarchitecture...........................................................................48Figure3-4:HighlevelarchitectureoftheInfrastructureNetwork.........................................49Figure3-5:T-NOVAVIMhighlevelarchitecture.....................................................................50Figure3-6:VIMNetworkControlArchitecture.......................................................................53Figure3-7-End-to-endcustomerdatapathwithoutVNFsandwithVNFs............................54Figure3-8-WICMoverallvision.............................................................................................56Figure4-1:VNFOn-boardingProcedure.................................................................................59Figure4-2:VNFInstantiationProcedure(Orchestrator’sView).............................................60Figure4-3:VNFInstantiationProcedure(IVM’sView)............................................................61Figure4-4:VNFMonitoringProcedure(Orchestrator’sView)................................................63Figure4-5:VNFSupervisionProcedure(IVM’sView).............................................................64Figure4-6:ScalingoutaVNF...................................................................................................65Figure4-7VNFTerminationProcedure–Orchestrator’sView...............................................66Figure4-8:VNFTerminationProcedure–IVM’sView............................................................67Figure4-9:NSOn-boardingProcedure...................................................................................68Figure4-10:NSInstantiationProcedure(Orchestrator’sView)..............................................69Figure4-11:NSInstantiationProcedure(IVM’View).............................................................70Figure4-12:WICMsequencediagram....................................................................................71Figure4-13:NSMonitoringProcedure...................................................................................72Figure4-14:Scaling-outaNS..................................................................................................73Figure4-15:NSScale-out........................................................................................................75Figure4-16:NSTerminationProcedure..................................................................................76Figure4-17:CommunicationofmonitoringinformationacrosstheT-NOVAsystem............77


©T-NOVAConsortium

7

IndexofTables

Table3.1:ExternalInterfacesoftheT-NOVAIVM..................................................................39Table3.2:InternalinterfacesoftheIVM................................................................................41Table4.1:Monitoringmetricsperinfrastructuredomain......................................................77Table5.1:Gapanalysisinthecomputedomain.....................................................................80Table5.2:GapanalysisintheHypervisordomain..................................................................80Table5.3:GapanalysisregardingSDNControllers.................................................................81Table5.4:GapanalysisregardingCloudControllers...............................................................82Table5.5:GapanalysisregardingNetworkVirtualisation......................................................84Table5.6:GapanalysisregardingOrchestration....................................................................85Table6.1:OrchestratorRequirements–NFVO-NSLifecycle.................................................90Table6.2:OrchestratorRequirements–NFVO-VNFLifecycle...............................................91Table6.3:OrchestratorRequirements–NFVO-ResourceHandling......................................92Table6.4:OrchestratorRequirements–NFVO-MonitoringProcess.....................................92Table6.5:OrchestratorRequirements–NFVO-ConnectivityHandling.................................93Table6.6:OrchestratorRequirements–NFVO-Marketplacespecific...................................94Table6.7:OrchestratorRequirements–VNFM-VNFLifecycle..............................................94Table6.8:Orchestratorrequirements–VNFM-MonitoringProcess.....................................95Table6.9:RequirementsbetweentheOrchestratorandVIM................................................96Table6.10:RequirementsbetweentheOrchestratorandVNF..............................................98Table6.11:RequirementsbetweentheOrchestratorandtheMarketplace..........................99Table6.12:IVMRequirements–VIM...................................................................................101Table6.13:IVMRequirements–WICM................................................................................103Table6.14:IVMrequirements–Computing.........................................................................104Table6.15:IVMRequirements–Hypervisor........................................................................107Table6.16:IVMRequirements–Networking.......................................................................108Table6.17:GeneralTerms....................................................................................................111Table6.18:OrchestrationDomainTerminology...................................................................111Table6.19:IVMDomainTerminology...................................................................................112


©T-NOVAConsortium

8

1. INTRODUCTION

ThisdeliverableoutlinestheoutputsandtheresultsoftheactivitiescarriedoutinTasks2.3and 2.4 in Work Package 2 (WP2). These outputs and results are focused on theinfrastructure virtualisation layer as well as of the management and orchestration layerwithintheT-NOVAsystem.

1.1. Virtualisation

Virtualisation is a general term that can apply to a variety of different technologyapproachessuchashardware,operatingsystem,storage,memoryandnetwork.ItisthekeyenablertechnologythatallowstraditionalphysicalnetworkfunctionstobedecoupledfromfixedappliancesandtobedeployedontoindustrystandardserverslargeDataCentres(DCs).This approach is providing operators with key benefits such as greater flexibility, fasterdeliveryofnewservices,abroaderecosystemenhancinginnovationinthenetworketc.

1.1.1 TheVirtualisationConcept

Fromacomputingperspectivevirtualisationabstractsthecomputingplatformand,indoingso, hides its physical characteristics fromusers or applications.Datingback to the1960’s,theconceptofvirtualisationwasfirstintroducedwiththeAtlasComputerwiththeconceptof virtual memory, and paging techniques for system memory. IBM’s M44/44X projectbuildingontheseinnovationsdevelopedanarchitecturewhichfirstintroducedtheconceptof virtualmachines (VMs). Their approachwas based on a combination of hardware andsoftware allowing the logical slicing of one physical server into multiple isolated virtualenvironments [1].Virtualisationhasnowevolved from its initialmainframeorigins tonowbeingsupportedbytheX86architectureandbeingadoptedbyothernon-computingdomainsuchasstorageandnetworking.

The term Full Virtualisation describes the techniquewhere a complete simulation of theunderlyinghardwareisprovided.ThisapproachhasitsoriginsinIBM’scontrolprogramsfortheCP/CMSoperatingsystem.TodaythisapproachisusedtoemulateacompletehardwareenvironmentintheformofaVM,inwhichaguestOperatingSystem(OS)runsinisolation.Fullvirtualisationwasn’tcompletelypossiblewiththex86architectureuntiltheadditionofIntel’sVTandAMD-Vextensionsin2005-2006.Infact,fullx86virtualisationreliesonbinarytranslation to trap and virtualise the execution of certain sensitivity “non-virtualisable”instructions.Withthisapproach,criticalinstructionsarediscoveredandreplacedwithtrapsinto the Virtual Machine Manager (VMM), also called a hypervisor, to be emulated insoftware.

Virtualisation isnowfoundinapplicationsforotherdomainssuchasstorage,andnetworktodeliversimilarbenefitstothoserealisedinthecomputedomain.

• Storagevirtualisation refers toaprocessbywhichseveralphysicaldisksappear tobe a single unit. Virtualised storage is typically block-level rather than file-level,meaningthatitlookslikeanormalphysicaldrivetocomputers.Thekeyadvantagesof the approach are: (i) easier management of heterogeneous storageenvironments, (ii) better utilisation of resources, (iii) greater flexibility in theallocationofstoragetoVMs,


©T-NOVAConsortium

9

• Network virtualisation comes in many forms like Virtual Local Area Networks(VLANs),LogicalStorageAreaNetworks(LSANs)andVirtualStorageAreaNetworks(VSANs) that allow a single physical Local Area Networks (LAN) or Storage AreaNetworks (SAN) architecture to be carved up into separate networks withoutdependence on the physical connection. Virtual Routing and Forwarding (VRF)allowsseparateroutingtablestobeusedonasinglepieceofhardwaretosupportdifferentroutesfordifferentpurposeswhilevirtualswitchingsupportsL2EthernetconnectivitybetweenVMs.Thebenefitsofnetworkvirtualisationareverysimilartoservervirtualisation,namelyincreasedutilisationandflexibility.

These technologies in the form of cloud computing are now being rapidly adopted bynetwork operators in their carrier network domains in order to consolidate traditionalnetworkdevicesontostandardhighvolumex86servers,switchesandstorageintheformofVNFs.Indoingso,theyallowserviceproviderstotransformtheirnetworkfunctionsintoanelastic pool of resources while seeking compatibility with network and operationalmanagement tools. Building on cloud DCs allows operators to create an orchestrationenvironment for the management and control of their compute, network and storageresources.ForVNFstofunctionproperlytheconfigurationofthenetworkunderneaththemiscritical.ToprovisionoradaptVNFstochangingnetworkconditionsorcustomerrequestsrequirestheabilitytoconfigureoradaptnetworkroutesinahighlyexpeditiousmanner.

The advent of Software Defined Networking (SDN) with its support for programmaticprovisioningtransformsservicedeliveryfromweekstoamatterofminutesorevenseconds.SDN isbasedaroundanewnetworkingmodelwherecontrolof thenetwork isdecoupledfrom the physical hardware allowing a logically centralised software program (a networkcontroller) to control the behaviour of an entire network. The use of centralised networkcontrolandacommoncommunication layerprotocolacrosstheswitchingelements inthenetwork can enable increased network efficiency, centralised traffic engineering, improvetroubleshootingcapabilitiesandtheabilitytobuildmultiplevirtualnetworksrunningoveracommonphysicalnetworkfabric.InSDN,networkelementsareprimarilyfocusedonpacketforwarding,whereas switching and routing functions aremanagedby centralisednetworkcontroller which dynamically configures network elements using protocols such asOpenFlow or OVSDB. SDN is starting to be deployed in data centre and enterpriseenvironments e.g. Google. Virtual networks to support VNF deployment can be deleted,modified or restored in amatter of seconds inmuch the samemanner thatweprovisionvirtualmachinesincloudenvironments.

Virtualisationanditsadoptioninthekeyconstituentelementsofnetworksanddatacentreshascreatedanagilityforserviceprovidersthatwasnotpreviouslypossible.Virtualisationofinfrastructure,networksaswellas theapplicationsandservices that runon topwillallowserviceproviderstorapidlytransformtheirnetworksandtoembracenewinnovations.

1.1.2 TheProsandConsofNFVDeployments

AshighlightedbytheETSIISGNFVinitsfirstwhitepaper[2],thescenariowhichdefinesthesituationfacedbymostnetworkoperatorsnowadays,relatestothephysicalcomponentsoftheirnetworks,whicharecharacterisedbytheuseofawiderangeofproprietaryhardwareappliances. This problem of appliance and technology diversity continues to grow foroperatorsasnewequipmentisaddedtopreviousgenerationsofequipmentinthenetwork.

This leadstosignificantchallengesrelatedtothelaunchofnewservices, increasingenergycosts and capital investments coupledwith the difficulty of finding peoplewith themostappropriate skills to handle the design, integration and operation of increasingly complex


©T-NOVAConsortium

10

hardware-based appliances. In addition, the trend towards shorter operational lifespan ofhardwarealsoaffectsrevenues,leadingtosituationswherethereisnoreturnoninvestmentorwherethereisnotimeforinnovation.

As previously outlined in the T-NOVA project scope [3], Network Functions Virtualisation(NFV) will address these challenges by leveraging standard Information Technology (IT)virtualisation technologies to consolidate various network equipment types onto industrystandardhigh volume (SHV) servers, switches and storage located inDCs,NetworkNodesand in the end user premises. In this context, NFV refers to the virtualisation of networkfunctionscarriedoutbyspecialisedhardwaredevicesandtheirmigrationtosoftware-basedappliances,whicharedeployedontopofcommodityIT(includingCloud)infrastructures.

VirtualisingNetworkFunctionspotentiallyoffersmanybenefits,including:

• Reductioninbothequipmentcostsandpowerconsumption,• Reducedtimetomarket,• Availabilityofnetworkappliancesthatsupportmultiple-versionsandmulti-tenancy,

withtheabilitytoshareresourcesacrossservices,• Targetedserviceintroductionbasedongeographyorcustomertype,whereservices

canbequicklyscaledup/downasrequired,• Enablingawidevarietyofeco-systems,• Encouragingopennesswithintheecosystem.

One of the challenges in the deployment of NFV in the carrier domain is to leverage theadvantages of the IT ecosystem while minimising any of the associated disadvantages.Standardhighvolumeserversandsoftwaremustbemodifiedtomeetthespecificreliabilityrequirements in the telecoms environment, including 99.999 percent uptime availability.Thismissioncritical levelof reliability isakeyrequirementanddifferentiates traditional IT(just reboot the system!) and Telecom (where downtime or poor performance is notacceptable)environments.Tomeetdesigngoalswithoutsacrificingperformance,softwareapplications must be specifically designed or rewritten to run optimally in virtualisedTelecomenvironmentstomeetcarriergraderequirements.Otherwise,applicationsportedto virtualised environments may experience significant performance issues and may notscaleappropriatelytotherequirednetworkload.Anadditionalchallengeforvirtualisationina Telecomnetworkenvironment is the requirement todeliver low latency tohandle real-time applications such as voice and video traffic. In addition to performance, otheroperational characteristics that are crucial to successful deployments include:maturity ofthe hypervisor; Reliability, Availability, and Serviceability (RAS); scalability, security,managementandautomation;supportandmaintainability.

DeployingNFValsoincursotherwell-definedrisks,e.g.scalabilityinordertohandlecarriernetwork demands;management of both IT and network resources in support of networkconnectivity services andNetwork Functions (NFs) deployment; handling of network faultand management operations; Operations Supporting System (OSS) / Business SupportingSystem (BSS) backwards compatibility in migration situations; interoperability required toachieve end-to-end services offerings, including end-to-end Quality of Service (QoS). Inaddition, essential software appliances should achieve performance comparable to theirhardwarecounterpartswhichiscurrentlynotalwayspossibledueavarietyofreasonssuchastheperformanceofthevirtualisationtechnologies.

1.2 TheT-NOVASolution

TheT-NOVAprojectisfocusedonaddressingsomeofthekeychallengesofNFVdeploymentin Telco environments by designing and implementing an integrated architecture, which


©T-NOVAConsortium

11

includesanovelopen-sourceOrchestrationplatform.Thisplatformisexplicitlydedicatedtotheorchestrationof IT (i.e. CPU,memoryand storage) andend-to-endnetwork resourcesforNFVs,aswellastheautomatedprovisioning,management,monitoringandoptimisationof Network Functions-as-a-Service (NFaaS). The orchestration of the resources availableacross the NFV Infrastructure Points-of-Presence (NFVI-PoPs) is achieved through the T-NOVA Infrastructure Virtualisation and Management platform (IVM). IVM comprises of anumberof components thatoffer functionalities,whichcollectivelyprovide thevirtualisedcompute,storageandnetworkconnectivityrequiredtohostVNFs.

TheoverallT-NOVAsystemarchitectureisshowninFigure1-1whichincludestwoplatformsspecified in the present deliverable: the T-NOVA Orchestration and the T-NOVA IVMplatforms.

Figure1-1High-levelviewofoverallT-NOVASystemArchitecture

(Source:D2.21[4])

1.2.1 TheT-NOVAOrchestrationPlatform(PTIN)

The T-NOVA architecture has been conceived using a layering approach where theOrchestration layer ispositionedbetween theServiceand the InfrastructureVirtualisationand Management layers. This layering approach, together with the envisaged high-levelmoduleswithintheOrchestratorplatform,isillustratedinFigure1-1.


©T-NOVAConsortium

12

The functionalities provided by the T-NOVA Orchestrator are required to extend beyondtraditionalcloudmanagement,astheT-NOVAscopeisnotrestrictedtoasingleDataCentre(DC). The Orchestrator therefore needs to manage and monitor Wide-Area Networks(WANs) aswell as distributed cloud (compute/storage) services and resources in order tocouplebasicnetworkconnectivityserviceswithadded-valueNFs.

Orchestrationplatformcapabilities that could improve thedeploymentofVNFs inprivate,heterogeneouscloudincludeamongothers:

• Applicationassignmenttohardwareplatformscapableofimprovingitsperformancethough specific features, such as special purpose instructions or accelerators i.e.allocationofaSingleRootI/OVirtualisation(SR-IOV)virtualfunctiontoVMsrunningVNFsthatcanbenefitfromthecapability;

• ExploitationofEnhancedplatformawareness(EPA) information,which isextractedfromeachNFVI,inordertooptimisetheresourcemappingalgorithms.

• Support of live-migration (wherever applicable, taking into account the serviceprovidedbyeachVNF).

The Orchestrator’s requirements together with its detailed conception and description intermsofFunctionalElements(Fes)constitutetheoutputsofTaskT2.3whicharedescribedinSection3.

1.2.2 TheT-NOVAIVMPlatform

TheIVMlayerintheT-NOVAsystemisresponsibleforprovidingtheexecutionenvironmentfor VNFs. The IVM is comprised of a Network Function Virtualised Infrastructure (NFVI),Virtualised Infrastructure Manager (VIM) and a WAN Infrastructure ConnectionManagement(WICM).TheIVMprovidesfullabstractionoftheNFVIresourcestoVNFs.TheIVM achieves this by supporting separation of the software that defines the networkfunction(theVNF)fromthehardwareandgenericsoftwarethatconstitutetheNFVI.Controlandmanagement of the NFVI is carried out by the VIM in unison with the Orchestrator.While the IVMprovidesorchestrationof thevirtualisedresources in the formofcompute,storageandnetworking,responsibilityfortheorchestrationoftheVNFsissolelyafunctionoftheOrchestrationlayergivenitssystemwideviewoftheT-NOVAsystemandcentralisedcoordinationroleinthesystem.

A major challenge for vendors developing NFV-based solutions is achieving near-nativeperformance(i.e.,similartonon-virtualised)inavirtualisedenvironment.Onecriticalaspectis minimising the inherent overhead associated with virtualisation, and there has beensignificantprogressthankstoanumberofkeyinnovations.Anexampleishardware-assistedvirtualisation inCPUs, suchas Intel’sXeonmicroprocessorswith IntelVT-x,which reducesVMcontextswitchingtime,amongotherthings.

Another challenge is ensuring the orchestration layer fully exploits the capabilities of theserversitmanages.Typicalorchestrationlayerproductscanidentifyinfrastructuralfeatures(e.g.,CPUtype,RandomAccessMemory(RAM)sizeandhostoperatingsystem);however,some orchestrators are unaware of attached devices, like acceleration cards or networkinterface cards (NICs) with advanced capabilities. In such cases, they are unable toproactivelyloadanapplicationontoaplatformcapableofacceleratingitsperformance,asinassigning an IP security (IPsec) VPN appliance to a server with cryptographic algorithmaccelerationcapabilities.Other featuresof theplatformmaybeof interest, i.e. themodelandversionofCPU,thenumberofcores,andotherspecificfeatures.


©T-NOVAConsortium

13

The lackofplatformand infrastructuralawareness isamajordrawbacksincemanyvirtualappliances have intense I/O requirements and could benefit from access to high-performance instructions, accelerators and Network Interface Cards (NICs) for workloadssuchascompression,cryptographyandtranscoding.ThisisakeyfocusinWP3(Task3.2)andWP4 (Task 4.1). Undoubtedly, making the orchestration layer aware of the innatecapabilities of the devices attached to server platforms can help maximise networkperformance.

TheoutputsofTask2.4withrespecttotheoverallintegratedarchitectureoftheIVMlayerarepresentedinSection3.


©T-NOVAConsortium

14

2. THET-NOVAORCHESTRATIONLAYER

This section describes the Orchestration layer, starting with an overview of its maincharacteristics, challenges and framework (subsection 3.1); followed by a description ofrequirements associated with its FEs (subsection 3.2), and finally a description of itsfunctionalarchitecture(subsection3.3).

2.1. OrchestrationLayerOverview

NFVisanemergingconcept,whichreferstothemigrationofcertainnetworkfunctionalities,traditionally performed by dedicated hardware elements, to virtualised IT infrastructureswhere they are deployed as software components.NFV leverages commodity servers andstorage to enable rapid deployment, reconfiguration and elastic scaling of networkfunctionalities.

Decouplingthenetworkfunctionssoftwarefromthehardwarecreatesanewsetofentities,namely:

• VirtualNetworkFunctions(VNFs):software-basednetworkfunctionsdeployedovervirtualisedinfrastructure;

• Network Functions Virtualized Infrastructure (NFVI): virtualised hardware thatsupportsthedeploymentofnetworkfunctions;

• Network Service (NS): chain of VNFs and/or Physical Network Functions (PNFs)interconnectedthroughvirtualnetworklinks(VLs).

Since VNFs, NFVIs, NSs and the relationships between them did not exist before theNFVparadigm, handling them requires a new and different set of management orchestrationfunctions.

VNFs require more agile management procedures when compared with legacy PNFsdeployed over dedicated appliances. Besides the traditional management proceduresalready in place for PNFs, in charge of BSSs/OSSs, such as customer management,accountingmanagementandSLAmanagement,VNFsrequirenewmanagementprocedures,e.g.toautomaticallycreate,toupdateand/ortoterminateVNFsandNSs.Furthermore,theautomaticdeploymentandinstantiationproceduresassociatedwithaspecificVNFneedtobe inplaceaswell as themonitoringandautomatic scalingproceduresduring the serviceruntimephase.

AnotherchallengebroughtaboutbytheNFVparadigmisthemanagementofthevirtualisedinfrastructure. In fact, one of themain advantages of virtualising network functions is toenable the automatic adjustment of NFVI resources according to the network functiondemands.Toachievethis, theVNFspecificrequirements,accordingtothecontractedSLA,havetobemappedtotherequiredvirtualised infrastructureassets(compute–e.g.virtualand physical machines, storage and networking – e.g. networks, subnets, ports andaddresses). The mapping procedures should also consider the network topology,connectivity and network QoS constraints, as well as function characteristics (e.g. somefunctionsmayrequirelowdelay,lowlossorhighbandwidth).SincevirtualisedresourcescanbecentralisedinasingleNFVI-PoPordistributedacrossseveralNFVI-PoPs,themanagementandorchestrationentitieswillalsohavetodecidewhatisthemostappropriatedNFVI-PoPorNFVI-PoPstodeploythefunction.


©T-NOVAConsortium

15

Besides theVNFsandtheNFVI-PoPsmanagementchallenges,newandmorecomplexNSswillbeprovidedbasedonthecombination/chainingofseveralVNFs.Therefore,inadditionto the managing and orchestrating of each VNF and of the associated NFVI-PoP,orchestration and management procedures also have to be defined at the service level.ThesewillcoordinateseveralVNFs,aswellastheirassociationthroughVirtualnetworkLinks(VLs).Moreover,sincetheNSscanalsobecomposedbyPNFs,theinterconnectionsbetweenthese and theVNFs are also required. TheNS composition includes the constituentVNFs,PNFsandVLs,intheVNFForwardingGraph(VNFFG).EachNScanhaveoneormoreVNFFGs,ifthereareconditionstohavealternativesintermsofpathcreation,whichcanbeusedasbackups.

Figure 2-1 illustrates the entities introduced by the NFV paradigm, as well as theirrelationships.

Figure2-1:NSsandVNFsComplexOrchestrationOverview

TheNSpresentedinFigure2-1iscomposedbythefollowingentities:

• FourVNFs:A,B,CandD;

• OnePNF;

• FiveVLs:1(interconnectingVNFAandPNF),2(interconnectingVNFAandVNFB,3(interconnecting VNF A and VNF C), 4 (interconnecting VNF B and VNF D) and 5(interconnectingVNFCandVNFD).

TheVNFsaredeployedovertwodifferentNVFI-PoPs:

• NFVI-PoPI:supportsVNFAandDdeployments;


©T-NOVAConsortium

16

• NFVI-PoPII:supportsVNFBandCdeployments.

TwoVNFFGsareillustrated:

• VNFFGI:deliverstheNSthroughthePNF–VNFA–VNFC–VNFDnetworkingpath;

• VNFFGII:deliverstheNSthroughthePNF–VNFA–VNFB–VNFDnetworkingpath.

Internally,theVNFscanbecomposedbyoneormoresoftwarecomponents,alsoknownasVirtualNetworkFunctionComponents (VNFCs).EachVNFC is typicallydeployed inasingleVirtualMachine(VM),althoughotherdeploymentprocedurescanexist.AsVNFs,VNFCscanbe instantiated in a single NFVI-PoP or distributed across several NFVI-PoPs. The VNFCsinterconnections aremade through dedicatedVLs. Figure 2-1 illustrates the internals of aspecific(VNFD).Thelattersoftwarecomponents,namelywebserver(VNFCA),applicationserver(VNFCB)anddatabase(VNFCC),interconnectedthroughVLs(VL6andVL7).

Ontopofalltheseentities(e.g.NS,VNF,VNFC,VL,NFVI-PoP,etc.)standstheorchestrator,which has responsibility for managing the complexity associated with the NSs and VNFslifecycle management (e.g. on-boarding/deployment, instantiation, supervision, scaling,termination),includingtheinternalsoftheVNFs(notillustratedinthefigure).

In summary, the T-NOVAOrchestratorplatform is focusedon addressing twoof themostcriticalissuesinNFV:

1. AutomateddeploymentandconfigurationofNSs/VNFs;

2. Management and optimisation of networking and IT resources for VNFsaccommodation.

To address the complex management processes related with the NSs and VNFs, theOrchestratorissplitintwomainFEs:

1. NFV Orchestrator (NFVO): manages the virtualised NSs lifecycle procedures,includingthenetworkinglinksthatinterconnecttheVNFs;

2. VNFManager(VNFM):managestheVNFslifecycleprocedures.

TheT-NOVAOrchestratorwillalsobeabletodeployandmonitorT-NOVAservicesbyjointlymanagingWAN resources and cloud (compute/storage) assets (DCs). Indeed, the T-NOVAOrchestratorgoesbeyondtraditionalcloudmanagement,sinceitsscopeisnotrestrictedtoa single DC; it needs to jointlymanageWAN and distributed cloud resources in differentinterconnectedDCs inorder to couple thebasic network connectivity servicewith added-valueNFs.

Further details regarding these T-NOVA Orchestrator entities and functionalities areprovided in the following subsections. The VNF related concepts and architecturalcomponentsarediscussedextensivelyinDeliverableD2.41[5].

2.2. OrchestratorRequirements

As alreadyoutlined in subsection2.1, the T-NOVAOrchestrator is composedby twomainbuildingblocks:theNFVOandtheVNFM.

The NFVO orchestrates the subset of functions that are responsible for the lifecyclemanagementofNetworkServices (NSs). In addition, it is also responsible for the resourceorchestrationoftheNFVIresourcesacross:

• AsingleVIM,correspondingtoasingleNFVI-PoP,and/or


©T-NOVAConsortium

17

• Multiple VIMs, corresponding to multiple NFVI-PoPs, by using a specialized VIMdesignatedbyWICM.

The VNFM is the functional block that is responsible for the lifecyclemanagement of theVNFs.

Thedeploymentandoperationalbehaviourof theOrchestrator is captured indeploymenttemplates,wherethemostimportantforthissubsectionaretheNetworkServiceDescriptor(NSD)andtheVirtualNetworkFunctionDescriptor (VNFD).Othertemplatesarealsoused,e.g.,VirtualLinkDescriptor (VLD),andtheVNFForwardingGraph (VNFFGD),whichwillbefurtherdetailedinsubsection2.3.

This subsection details the Orchestrator requirements that have been identified after aresearch study involving several sources, e.g. use cases defined in D2.1 [6], ETSI ISG NFVrequirements[7],ITU-TrequirementsforNV[8],aswellasexcerptsofrelevantpartsoftheETSIISGMANOarchitectureandassociatedFEs[9].

ThelistofrequirementsforeachFEmaybefoundinAnnexA,where27requirementshavebeenidentifiedfortheNFVOand6requirementsfortheVNFM.However,itshouldbenotedthatnoneoftheserequirementsimposesanyspecificsolutionattheimplementationlevel,whichwillbeperformedinWP3/4.

Taking into account that the list of requirements is quite extensive, the entire set ofrequirementshasbeenclassifiedanddividedintotypesasindicatedintheremainingpartofthecurrentsubsection.

2.2.1. NFVORequirementsTypes

NetworkServicesundertheresponsibilityoftheNFVO,arecomposedbyVNFsand,assuch,are defined by their functional and behavioural specification. In this context, the NFVOcoordinates the lifecycle of VNFs that jointly realise a NS. This coordination includesmanaging the associations between the different VNFs thatmake-up part of the NS, andwhenapplicablebetweenVNFsandPNFs,thenetworktopologyoftheNS,andtheVNFFGsassociatedwiththeNS.

The operation of NSs defines the behaviour of the higher Orchestration layer, which ischaracterised by performance, dependability, and security specifications. The end-to-endnetworkservicebehaviouristheresultofcombiningindividualnetworkfunctionbehavioursaswell as the behaviours of the compositionmechanisms associatedwith the underlyingnetworkinfrastructurelayer,i.e.theIVMlayer.

Intermsofdeploymentandoperationalbehaviour,therequirementsofeachNSarecarriedinadeploymenttemplate,theNSD,andstoredduringtheNSon-boardingprocessintheNScatalogue, for future selection once the instantiation of the service takes place. The NSDfullydescribestheattributesandrequirementsnecessarytoimplementaNS, includingtheservicetopology,i.e.constituentVNFsandtherelationshipsbetweenthem,VLs,VNFFGs,aswellasNScharacteristics,e.g.intermsofSLAsandanyotherinformationnecessaryfortheNSon-boardingandlifecyclemanagementofitsinstances.

As the NS is the main responsibility of the NFVO, the NS lifecycle constitutes the mostrelevanttechnicalarearegardingtheNFVOclassificationintermsofrequirements.

As indicated below, the other requirement types are related with the VNF lifecyclemanagement with respect to the actions and procedures taken by the NFVO, which alsoincludes the secondFE that constitutespartof theOrchestrator, togetherwith theNFVO:


©T-NOVAConsortium

18

the VNFM. The actions and procedures associated with the VNFM’s behaviour, and inparticularthoserelatedtotheVNFlifecycle,willbefurtherdiscussedinsubsection2.2.2.

Regarding the remaining requirement types, it should be noted that there is one typerelatedto theNFVO,whichhandles themanagementof theresources located in theVIMand in the WICM; another one related with the policy management; and another one,specifictotheT-NOVAsystem,whichisconcernedwiththemostrelevantinteractionswiththeMarketplace,thelayerimmediatelyabovetheOrchestrationlayer.

Finally, therearestill two further types that relate toNS lifecycleoperations:connectivityhandling and themonitoringprocess. Adecisionwas taken to create separate groups forthese two (sub)types in order to emphasize the importance they play in the overalloperationoftheOrchestrator.

2.2.1.1. NSLifecycle

A Service Provider (SP) may choose one or more VNFs to compose a new NS, byparameterisingthoseVNFs,selectingaSLA,etc.within thecontextof theT-NOVAsystem.TheNFVOisthennotifiedofthecompositionofthisnewNS,bythereceptionofarequestthatincludesaNSD,whichisvalidatedintermsofdescription.

In a similar process, when a Customer subscribes to a NS, the Marketplace notifies theNFVO, which instantiates the NS according to its NSD description, agreed SLA and thecurrent status of the overall infrastructure usagemetrics.Upon a successful instantiation,the Orchestrator notifies the Marketplace, thus triggering the accounting process of thesubscribedNSaswellasofthecustomer.

After these steps the NFVO becomes responsible for NS lifecycle management, wherelifecycle management refers to a set of functions required to manage the instantiation,maintenanceandterminationofaNS.

2.2.1.2. VNFLifecycle

The NFVO performs its capabilities by using the VNFM operation in what concerns thehandlingoftheVNFlifecycle.AlthoughtheVNFMistheFEinchargeofthemanagementoftheVNFlifecycle,asdescribedinsubsection2.2.3,someoperationsrequiretheinterventionoftheNFVO.

Therequirement typespecified in thecurrentsubsectionrefersprecisely to thosepartsoftheVNFlifecyclemanagementthatareperformedbytheNFVO.

Inthiscontext,FunctionProviders(FPs)publicisetheirVNFsintheNetworkFunctionStore(NF Store). This implies the use of a VNFD describing the infrastructure (computation,storage,networkinfrastructureandconnection)neededfortheVNFtobeinstantiatedlateronbyarequestsenttotheOrchestrator.

After a validation of theVNFD, theNFVOpublicises theVNF to theMarketplace as beingreadytobepartofaNS.AssociatedwiththeVNFD, theremaybepotentiallyaVMimagethatwillmakepartofthedeploymentofsuchVNF.

As the FPprovidesnewer versions this process is repeated. If andwhen the FPwishes towithdraw the VNF, the reverse process is executed taking into consideration the currentstatusofNSexploitingtheunderdeletionVNF.


©T-NOVAConsortium

19

2.2.1.3. ResourceHandling

The NFVO is the Orchestrator FE that performs the resource handling of the subset ofOrchestratorfunctionsthatareresponsibleforglobalresourcemanagementgovernance.

In terms of scope, the following domains and associated IT virtualised resources aremanagedbytheNFVO:Compute, i.e.virtualprocessingCPUsandvirtualmemory;Storage,i.e.virtualstorage;andNetwork,i.e.virtuallinksintra/interconnectingVNFswithintheDataCentre Network (DCN). In T-NOVA, the NFVO also manages the resources of the WICMnetworkdomain.

ThegovernancedescribedaboveisperformedbymanagingthebetweentheVNFinstancesand theNFVI resourcesallocated to thoseVNF instancesandbyusing the InfraResourcescatalogueaswellasinformationreceivedfromtheVIMandfromtheWICM.

According to the characteristicsofeach service (agreedSLA)and the currentusageof theinfrastructure (computation, storage infrastructure and connectivity), there is an optimalallocationfortherequiredinfrastructure.

Thisoptimalinfrastructureallocationwillbetheresponsibilityofanallocationalgorithm(orasetofalgorithms)thatwillbedefined,inWP3.

2.2.1.4. MonitoringProcess

Oneof the key aspects of the T-NOVAproject is not only the ability to optimally allocateinfrastructures for a NS, but also to react, in real time, to the current performance of asubscribedNS,sothattheagreedSLAismaintained.Toaccomplishthesetwoaspects,it iscrucial that ameaningful set of infrastructure (computational, storage, infrastructure andconnectivity)usagemetricsbecollected.

NS metrics must be defined together with the SLA(s) to be provided with every NSinstantiation.

It is expected that the data to be collected will be significant with a high frequency ofchange,soadequatestrategieswillhavetobedesignedtosupportcollectinglargevolumesofdata.

Assuch,duringtheNS lifecycle, theNFVOmaymonitortheoveralloperationofaNSwithinformationprovidedbytheVIMand/orbytheWICM,ifsuchrequirementswerecapturedintheNSdeploymenttemplate.

SuchdatamaybeusedtoderiveusageinformationforNFVIresourcesbeingconsumedbyVNF instancesorgroupsofVNF instances.For instance,theprocessmay involvecollectingmeasurementsaboutthenumberofNFVIresourcesconsumedbyNFVIinterfaces,andthencorrelatingNFVIusagerecordstoVNFinstances.

BeyondtheinfrastructureusagemetricssetsofNSusagemetrics,needtobedefineduponservice composition, inorder to allow trackingof theagreed SLAand todetermine if it isbeingmaintainedornot.

Thesemetricsaremoreserviceorientedthaninfrastructureoriented,andarebuiltontopofinfrastructure usage metrics. For instance, a metric such as “the current number ofsimultaneous sessions” is something that the infrastructure cannot measure, but the“currentmaximumnetworklatency”issomethingavailableattheinfrastructurelevel,whichmightmakesenseattheservicelevelaswell.ThechoicebetweenwhichmetricstotrackismadebytheMarketplace,atservicecompositiontime.


©T-NOVAConsortium

20

Thecollectionofthesemeasurementmetricsmaybereportedtoexternalentities,e.g.theCustomer,theSPortheFP,viatheMarketplace,ifsuchrequirementswerecapturedintheNSdeploymenttemplate.

Inaddition, this informationmaybecomparedwithadditional information included in theon-boardedNSandVNFdeploymenttemplates,aswellaswithpoliciesapplicabletotheNSthat can be used to trigger automatic operational management of the NS instance, e.g.automaticscalingofVNFinstancesthatarepartoftheNS.

2.2.1.5. ConnectivityHandling

TheNFVOhasanabstractedviewofthenetworktopologyandinterfacestotheunderlyingVIMsandWICMsinordertohandleconnectivityservicesbyperformingthemanagementoftheNSinstances,e.g.,create,update,query,deleteVNFFGs.

Connectivitymanagementmustbehandledover the samedomainsas those indicated forresourcehandling.

2.2.1.6. PolicyManagement

Policies are defined by conditions and corresponding actions/procedures, e.g. a scalingpolicymay state execution of specific actions/procedures if the required conditions occurduring runtime. Different actions/procedures defined by the policy can be mutuallyexclusive, which implies a process of selection of a particular action/procedure (or set ofactions/procedures)tobeexecutedeitherautomaticallyormanually.

In the context of T-NOVA, once declared, a policy may be bound to one or more NSinstances, VNF instances, and NFVI resources. Policy management always implies somedegreeofevaluationfortheNSinstancesandVNFinstances,e.g.,intermofpoliciesrelatedwithaffinity/anti-affinity,lifecycleoperations,geography,regulatoryrules,NStopology,etc.

In addition, policy management also refers to the management of rules governing thebehaviour of Orchestrator functions, e.g., management of NS or VNF scaling operations,accesscontrol,resourcemanagement,faultmanagement,etc.

Associatedwiththepolicymanagementterminology is theconceptofpolicyenforcement,i.e.policesaredefinedbycertainentitiesandarethenenforcedinotherentities,whichmayintheirturnenforcetheminadditionalentities.

IntheT-NOVAcontext,policiesmaybedefinedbyexternalentities,e.g.theCustomer,theSP or the FP, and are then enforced into the NFVO, via theMarketplace. In its turn, theNFVOmayenforcethemintotheVNFM.

Policyenforcementmaybestaticoron-demand.

2.2.1.7. Marketplace-specificInteractions

TheMarketplaceistheT-NOVAlayerthatinterfaceswithexternalentities,e.g.,Customers,SPs and FPs. In the T-NOVA global architecture, it interacts with the Orchestration layerthroughaninterfacewhoserequirementsaredefinedinsubsection2.4.

The deployment and behaviour of the Marketplace imposes requirements that theOrchestration must fulfil in order to offer those external entities an entire set offunctionalities,whicharedefinedinD2.41[5].


©T-NOVAConsortium

21

TherequestmadebytheMarketplaceforthoserequirementsaswellasthecorrespondentresponsesfromtheOrchestratorare,mostofthetime,implicitinthecurrentdescriptionoftheOrchestratorrequirements.

However,forsomeofthoserequirementsthatarebasedwithinD2.1[6],e.g.publishingtheoutcomeof theNS instantiation,publishingNSmetrics,or reportingusagemetrics, itwasdecidedtocreateaseparategroupinordertohighlighttheirprocessingmechanisms.

Forinstance,thevariouskindsofmetricsdescribedabovemaybeusedbybusiness-orientedprocessesresiding intheMarketplace,namelytostartandstoptrackingoftheusageaNSforbillingpurposes.

2.2.2. VNFMRequirementsTypes

The deployment and operational behaviour requirements of each VNF is captured in adeploymenttemplate,theVNFD,andstoredduringtheVNFon-boardingprocessintheVNFcatalogueaspartofaVNFPackage,forfutureuse.Thedeploymenttemplatedescribestheattributes and requirements necessary to realise such the VNF and captures, in anabstractedmanner,therequirementstomanageitslifecycle.

TheVNFMperformsthelifecyclemanagementofaVNFbasedontherequirementsincludedinthistemplate.Assuch,theVNFlifecycleconstitutesthemostrelevanttypeintheVNFMclassificationofrequirementsinrelationtotheprocedurestakeninthisglobalprocess.

AsalsodecidedfortheNFVO,thereisstillafurthertypethatconstitutes,infact,anareaofoperationthatbelongstotheVNFlifecycle:themonitoringprocess.Onceagain,thereasonbehind the creation of a separate group for this (sub)type is related to emphasising theimportanceofitsruleintheOrchestrator’soperation.

2.2.2.1. VNFLifecycle

The VNFM is responsible for the VNF lifecyclemanagement,where lifecyclemanagementreferstoasetoffunctionsrequiredtomanagetheinstantiation,maintenance,scalingandterminationofaVNF.TheVNFlifecycleisdefinedindeliverableD2.42[10].

2.2.2.2. MonitoringProcess

Duringthe lifecycleofaVNF,theVNFManagementfunctionsmaymonitorKeyParameterIndicator (KPIs) of a VNF, if such KPIs were captured in the deployment template. Themanagementfunctionsmayusethisinformationforscalingoperations.Scalingmayincludechangingtheconfigurationofthevirtualisedresources(scaledown,e.g.,addCPU,orscaleup, e.g., remove CPU), adding new virtualised resources (scale out, e.g., add a new VM),shuttingdownandremovingVMinstances(scalein),orreleasingsomevirtualisedresources(scaledown).

So, every VNF will usually provide its own usage metrics to the VNFM, which will be, ingeneral, specific to the function the VNF provides, although theymight be based on theinfrastructureontopofwhichtheVNFhasbeendeployed.

ThetreatmentoftheinformationcollectedduringtheVNFmonitoringprocessisverysimilarto the one described for the NS process and may result in reports being sent externalentities, via theMarketplace, and/or to trigger automatic operationalmanagementof theVNFinstance,e.g.automaticscaling.


©T-NOVAConsortium

22

2.3. FunctionalOrchestratorArchitecture

This subsection describes the Orchestrator reference architecture, including its functionalentitiesaswellasexternalinterfaces.

2.3.1. ReferenceArchitecture

The Orchestrator reference architecture, as well as the interfaces with the externalFunctionalEntities (FEs) isdepicted inFigure2-2. Indetail, theorchestrator interactswiththeMarketplace,whichistheT-NOVAdomainresponsibleforaccounting,SLAmanagementandbusinessfunctionalities.BesidestheMarketplace,theOrchestratoralsointerfaceswiththe IVM, and in particular with the VIM, for managing the data centre network/ITinfrastructure resources, as well as with the WICM for WAN connectivity management.Finally,theOrchestratorinteractswiththeVNFitself,whichintheT-NOVAscopeislocatedintheIVMdomain,toensureitslifecyclemanagement.

Internally, the T-NOVA Orchestrator consists of two main components and a set ofrepositories. One of the core elements is the NFVO, acting as the front-end with theMarketplaceandorchestratingalltheincomingrequeststowardstheothercomponentsofthearchitecture.FurtherdetailsrelatingtotheNFVOandtheassociatedincomingrequestsare available in subsection 2.3.2.1. To support the NFVO operation procedures, a set ofrepositoriesisidentifiedinordertostorethedescriptionoftheavailableVNFsandNSs(VNFCatalogueandNSCatalogue),theinstantiatedVNFsandNSs(NS&VNFInstances),aswellastheavailableresourcesinthevirtualisedinfrastructure(InfrastructureResourcesCatalogue).Further details about the orchestrator repositories are provided in subsection 2.3.2.3.Finally, the NFVO also interacts with the other core element, the VNFManager (VNFM),responsible for the VNF-specific lifecycle management procedures, as described insubsection2.3.2.2.

Figure2-2:T-NOVAOrchestratorReferenceArchitecture


©T-NOVAConsortium

23

2.3.2. FunctionalEntities

ThissubsectiondescribesthefunctionalentitiesoftheOrchestratorarchitecture.

2.3.2.1. NetworkFunctionVirtualisationOrchestrator(NFVO)

ThemainfunctionoftheNFVOistomanagethevirtualisedNSslifecycleanditsprocedures.Since the NSs are composed by VNFs, (PNFs, VLs and VNFFGs, the NFVO is able todecompose each NS into these constituents. Nevertheless, although the NFVO has theknowledge of the VNFs that compose the NS, it delegates their lifecyclemanagement toanotherdedicatedFEoftheOrchestratordomain,designatedbyVNFM.

A description of the main deployment templates must be taken into account whendeterminingthebestconnectivitypathstodeliveraserviceisprovided:

• aVNFFGDisadeploymenttemplatethatdescribesatopologyoftheNSoraportionof the NS, by referencing VNFs and PNFs as well as VLs that used forinterconnection. In addition to the VLs, whose descriptor is described below, aVNFFGcanreferenceotherinformationelementsintheNSsuchasPNFsandVNFs.A VNFFG also contains a Network Forwarding Path (NFP), i.e. an ordered list ofConnectionPointsformingachainofNFs,alongwithpoliciesassociatedtothelist,

• aVLDisadeploymenttemplatewhichdescribestheresourcerequirementsthatareneededforestablishinga linkbetweenVNFs,PNFsandendpointsoftheNS,whichcouldbemetbychoosinganoptionbetweenvariouslinksthatareavailableintheNFVI.However, theNFVOmust first consult theVNFFG inorder todetermine theappropriate NFVI to be used based on functional (e.g., dual separate paths forresilience)andotherneeds(e.g.,geographyandregulatoryrequirements).

Inadditiontotheorchestrationofthevirtualisedserviceleveloperations,whichallowstheabstractionof service specificities fromthebusiness/operational level– in this case theT-NOVAMarketplace– theNFVOalsomanagesthevirtualised infrastructureresource leveloperations aswell as the configuration/allocation of transport connectionswhen twoormore distinct DCs are involved. Hence, it coordinates the resourcereservation/allocation/removaltospecificNSsandVNFsaccordingtotheavailabilityofthevirtualisedinfrastructures,alsoknownasdatacentres.

Toaddressthetwomainfunctionalitiesabovementioned,theNFVOisarchitecturallysplitintwo modules, namely the Network Services Orchestrator (NSO) and the VirtualisedResourcesOrchestrator(VRO),furtherdescribedbelow.

NetworkServiceOrchestrator

TheNSOisoneofthecomponentsoftheNFVOwiththeresponsibilityformanagingtheNSlifecycleanditsprocedures.Moreprecisely,thefollowingtasksfallundertheresponsibilityoftheNSO:

• NSs and VNFs on-boarding: management of Network Services deploymenttemplates, also known asNSDescriptors andVNFPackages, aswell as of theNSsinstancestopology(e.g.,create,update,query,deleteVNFForwardingGraphs).On-boarding of a NS includes the registration in the NS catalogue therefore ensuringthatall thetemplates(NSDs)arestored,seeNSon-boardingproceduredetailed insubsection5.2.1;

• NSinstantiation:triggerinstantiationofNSandVNFinstances,accordingtotriggersand actions captured in the on-boarded NS and VNF deployment templates. In


©T-NOVAConsortium

24

addition,managementofthe instantiationofVNFs, incoordinationwithVNFMsaswellasvalidationofNFVIresourcerequestsfromVNFMs,asthosemayimpactNSs,e.g.scalingprocess,seeNSinstantiationproceduredetailedinsubsection5.2.2;

• NSupdate:supportNSconfigurationchangesofvariouscomplexitysuchaschanginginter-VNFconnectivityortheconstituentVNFs;

• NS supervision: monitoring and measurement of the NS performance andcorrelationoftheacquiredmetricsforeachserviceinstance.Dataisobtainedfromthe IVM layer (performance metrics related with the virtual network linksinterconnecting the network functions) and from the VNFM (aggregatedperformancemetricsrelatedwiththeVNF,seeNSsupervisionproceduredetailedinsubsection5.2.3;

• NS scaling: increase or decrease of theNS capacity according to per-instance andper-service auto-scaling policies. The NS scaling can imply eitherincreasing/decreasing of a specific VNF capacity, create/terminate new/old VNFinstances and/or increase/decrease the number of connectivity links between thenetworkfunctions;

• NStermination: releaseofaspecificNS instancebyremovingtheassociatedVNFsandassociatedconnectivitylinks,aswellasthevirtualisedinfrastructureresources,(seeNSterminationproceduredetailedinsubsection4.2.5).

Inadditiontotheselifecyclerelatedprocedures,theNSOalsoperformspolicymanagementandevaluationfortheNSinstancesandVNFinstances,e.g.,policiesrelatedwithscaling.

Figure 2-3 provides an illustration about the NSO interactions within the T-NOVAOrchestratorandwiththeremainingT-NOVAexternalentities:

Figure2-3:NSOrchestrator(Internal&External)Interactions

From the external perspective, it interacts with the Marketplace for operational andbusinessmanagementpurposesasfollows:

• Exchange provisioning information (e.g., requests, modifications/updates,acknowledgements)abouttheNSs(throughtheT-Da-Orinterface);


©T-NOVAConsortium

25

• Providestheorchestratorwith informationoneachNS instanceSLAagreement. InturnheorchestratorsendsSLA-relatedmetricstotheMarketplace(throughtheT-Sl-Orinterface);

• DelivertotheMarketplaceusageaccounting informationwithrespecttoVNFsandNSs(throughtheT-Ac-Orinterface);

• Provides the orchestrator with information about the NSs composition. Theorchestrator delivers to the Marketplace information about the available VNFs(throughtheT-Br-Orinterface).

Internally,theNSOhasthefollowingcommunicationpoints:

• NS Catalogue: collects information about the NSs (NSD), including the set ofconstituent VNFs, interconnecting network links (VLD) and network topologyinformation(VNFFGD);

• VNFCatalogue:storestheVNFDduringtheon-boardingprocedures;

• NSandVNFInstances:storesinformationabouttheNSinstancesstatus;

• VirtualisedResourcesOrchestrator (VRO):exchangesmanagementactions relatedtovirtualisedresourcesand/orconnections,eitherwithinthedatacentrescope(e.g.compute,storageandnetwork)and/oronthetransportnetworksegment;

• Virtual Network Function Manager (VNFM): exchange lifecycle managementactionsrelatedwiththeVNFs.

VirtualisedResourcesOrchestrator

TheVirtualisedResourcesOrchestrator(VRO)istheresourcelayermanagementFunctionalEntityoftheNFVOmainblock.Itisresponsibleforthefollowingactions:

• Coordinateresourcereservation/allocation/removalandestablishtheplacementforeachVMthatcomposestheVNF(andtheNS);

• InteractwiththeWANelementsforconnectivitymanagementactions;

• Validate NFVI resource requests from VNFMs, as those may impact the way therequested resources are allocated within one NFVI-PoP or across multiple NFVI-PoPs.WhethertheresourcerelatedrequestscomesdirectlyfromtheVNFMorfromtheNFVOisimplementationdependent;

• Manage the relationship between the VNF instances and the NFVI resourcesallocatedtothoseVNFinstances;

• CollectusageinformationoftheNFVIresources;

• CollectperformanceinformationaboutthenetworklinksinterconnectingtheVNFs;

• CollectperformanceaboutthevirtualisedinfrastructureresourcessupportingNSs.

ThefollowingvirtualisedresourcesaremanagedbytheVRO:

• Compute:virtualprocessingCPUsandvirtualmemory;

• Storage:virtualstorage;

• Network:virtuallinksintra/interconnectingVNFswithintheDCN.

Figure2-4providesanillustrationwithfurtherdetailsabouttheVROinteractionswithintheT-NOVAOrchestratorandwiththeremainingT-NOVAexternalentities:


©T-NOVAConsortium

26

Figure2-4:VirtualisedResourcesOrchestrator(Internal&External)Interactions

From an external perspective, it interacts with the VIM and theWICM for the followingpurposes:

• Virtualised Infrastructure Manager: to enforce resourcereservation/allocations/removal and to collect monitoring information about thevirtuallinksinterconnectionsoftheVNFs,throughtheT-Or-Viinterface;

• Transport Network Manager: enforces resource/connectivity decisionsallocations/removals and to collect monitoring information about the transportnetworkelements,throughtheT-Or-Tminterface.

Internally,theVROinteractswiththefollowingblocks:

• NetworkServicesOrchestrator:exchangesresourcereservation/allocation/removalmanagementactionsrelatedwithaspecificNS,foralltheconstituentVNFs;

• Infrastructure Resources catalogue: queries and stores information about thevirtualisedandnon-virtualisedinfrastructureresources;

• Virtual Network Function Manager: exchanges resourcereservation/allocation/removal management actions, in the case the resourcemanagementishandledbytheVNFM.

2.3.2.2. VirtualNetworkFunctionManager(VNFM)

The VNFM is responsible for the lifecycle management of the VNF. Each VNF instance isassumedtohaveanassociatedVNFM.AVNFMmaybeassignedthemanagementofasingleVNFinstance,orthemanagementofmultipleVNFinstancesofthesametypeorofdifferenttypes.TheOrchestratorusestheVNFDtocreateinstancesoftheVNFitrepresents,andtomanage the lifecycle of those instances.AVNFDhas a one-to-one correspondencewith aVNFPackage,anditfullydescribestheattributesandrequirementsnecessarytorealizesucha VNF. NFVI resources are assigned to a VNF based on the requirements captured in theVNFD (containing resource allocation criteria, among others), but also taking intoconsiderationspecificrequirementsaccompanyingtherequestforinstantiation.

ThefollowingmanagementproceduresarewithinthescopeoftheVNFM:

• Instantiate:createaVNFonthevirtualisedinfrastructureusingtheVNFon-boardingdescriptor,aswellas theVNF feasibilitycheckingprocedure,seeVNF instantiationproceduredetailedinsubsection4.1.2;


©T-NOVAConsortium

27

• Configure:configuretheinstantiatedVNFwiththerequiredinformationtostarttheVNF. The request may already include some customer-specificattributes/parameters;

• Monitor:collectandcorrelatemonitoringinformationforeachinstanceoftheVNF.ThecollectedinformationisobtainedfromtheIVMlayer(virtualisedinfrastructureperformance information) and from the VNF (service specific performanceinformation),seeVNFmonitoringproceduredetailedinsection4.1.3;

• Scale: increase or decrease the VNF capacity by adding/removing VMs (out/inhorizontalscaling)seeVNFscale-outproceduredetailedinsubsection4.1.4;

• Update:modifyconfigurationparameters;

• Upgrade:changesoftwaresupportingtheVNF;

• Terminate: release infrastructure resources allocated for the VNFs, see VNFterminationproceduredetailedinsubsection4.1.5.

Figure2-5providesanillustrationwithfurtherdetailsontheVNFMinteractionswithintheT-NOVAOrchestratorandwiththeremainingT-NOVAexternalentities:

Figure2-5:VNFManager(Internal&External)Interactions

Fromtheexternalperspective,itinteractswiththeVNFandwiththeVIMwiththefollowingpurposes:

• VirtualNetworkFunction(VNF):configuresVNFspecificinformation(throughtheT-Ve-Vnfminterface)andreceivesVNFrelatedmonitoringinformation;

• Virtual Infrastructure Management (VIM): collects monitoring information aboutthevirtualisedinfrastructureresourcesallocatedtotheVNF(throughtheT-Vi-Vnfminterface).

Internally,theVNFMinteractswiththefollowingcomponents:

• Network Services Orchestrator (NSO): receive VNF instantiation requests for aspecificNSandprovideVNFmonitoringinformation;

• VNF Catalogue: collects information about the VNFs internal composition (VNFD),including the VNF Components (VNFCs), software images (VMs) andmanagementscripts;

• Virtualised Resources Orchestrator (VRO): exchanges resourcereservation/allocation/removal management actions, in cases where themanagementishandledbytheVNFM.


©T-NOVAConsortium

28

2.3.2.3. RepositoriesandCatalogues

To support the T-NOVA Orchestrator lifecycle management operations, the followingcataloguesaredefined:

• NSsCatalogue(NSCatalogue);

• VNFsCatalogue(VNFCatalogue);

• NSsandVNFsInstancesRepository;

• InfrastructureResourcesRepository.

NSCatalogue

Represents the repository of all the on-boarded NSs in order to support the NS lifecyclemanagement:

• NS Descriptor (NSD): contains the service description template, including SLAs,deploymentflavours,referencestothevirtuallinks(VLDs)andtheconstituentVNFs(VNFFG);

• Virtual LinkDescriptor (VLD): contains thedescriptionof the virtual network linksthatcomposetheservice(interconnectingtheVNFs);

• VNFForwardingGraphDescriptor(VNFFGD):containstheNSconstituentVNFs,aswellastheirdeploymentintermsofnetworkconnectivity.

VNFCatalogue

Represents the repository of all the on-boarded VNFs in order to support its lifecyclemanagement:

• VNFDescriptor(VNFD):containstheVNFdescriptiontemplate,includingitsinternaldecompositioninVNFCs,deploymentflavoursandreferencestotheVLDs;

• SoftwareimagesoftheVMslocatedintheIVMlayer.

NSandVNFInstances

Represents the repository of all the instantiated NSs and VNFs, which can becreated/updated/releasedduringthelifecyclemanagementoperations.

InfrastructureResources

Represents the repository of available, reserved and allocated NFVI-PoP resources, alsoincludingtheonesrelatedtotheWANsegment.

2.3.3. ExternalInterfaces

In this section the external interfaces of the Orchestrator are described. However it isimportantwithin theperspectiveof theT-NOVAarchitecture tounderstandthecontext inwhich the term interface is used as is its relationship to reference points a commonarchitecturallocususedwithinthenetworkingdomain

Innetworktermsareferencepoint isanabstractpoint inamodelofnetworkorprotocol.Thisreferencepointessentiallyservestopartitionfunctionsorconfigurationsandsoassistsin the description of a network model as well as serving as a point of interoperabilitybetweendifferentpartsof thenetwork [11]. Inanetworkingcontext,an interfacemayor


©T-NOVAConsortium

29

may not be associatedwith any given reference point. An interface typically represents aprotocollevelconnectionwhichmayormaynotbemappedtoareferencepoint.

This strict delimitation between the definition of reference points and interfaces in thecontextofNFVandSDNgiven thehybridisationofnetworkingand IT technologies canbechallenging,i.e.inthenetworkdomain,thisframeworkisstrictlydefined,anditisthereforenormalpracticetoretaintheuseofthetermreferencepoint,whileintheITdomain,thereisamoreflexibledemarcationbetweentechnologiesleadingtoadegreeofhybridisation.Asa consequence in the IT domain the term interface is used in amore flexiblemanner toencompassreferencepointsalso.

While strictly speaking the separation of the terms should be technically maintained theapproachadopted in thisdeliverable is toutiliseabroaderandmore flexibledefinitionofinterfacesandreferencepointsgiventheexpectedone-to-onemappingofreferencepointsandinterfacesinthecontextoftheproposedT-NOVAarchitecture.AdditionallyinterfacesintheT-NOVAsystemmaynotnecessarilybe tiedspecifically toaprotocolbut ratheractaspoint of information exchange through APIs. Hencewithin the context of this deliverableinterfaces areenvisioned toencompassboth thearchitectural characteristicsof interfacesandreferencespointsgivenfusionoftheITandnetworkingdomains.

Havingclarifiedtheuseoftheterm,thedescriptionoftheOrchestrator’sexternalinterfaceswillstarttobeprovidedbymeansofaveryshortreferenceonsecurity,whichisacommonarea that affects all the interfaces. It will be followed by an introduction to the interfacerequirementsthatarepresentedinAnnexA.2inatabularformat.

Regardingthecommonissue,andconsideringthemostgenericscenariosinwhichtherolesdescribed in D2.1 [6] are played by distinct entities all the external interfaces beingdescribedinthissectionmustsupportat leastaminimumdegreeofsecurity.Thedecisionon the exact degree of security for each implemented interfacewill be taken later in theproject’stimeline.

2.3.3.1. InterfacebetweentheOrchestratorandtheNetworkFunctionStore

The interfacebetweentheOrchestratorandtheNetworkFunctionStore (NFStore)servestwopurposes:

• For the NF Store, tonotify the Orchestrator about new, updated andwithdrawnVNFs;

• FortheOrchestrator,toretrievefromtheNFStoreandstoreintheVNFCataloguetheVNFDandVMsimagesthatneedtobeinstantiatedtosupportthatVNF.

This“two-phase”interactionbetweentheOrchestratorandtheNFStore,insteadofjustoneinwhichtheNFStorecouldpasstheOrchestratortheVNFDescriptorandVMimages,allowsfortheoptimisationofresourcesonbothsidesoftheinterface.OntheNFStoresidethisisjustanotificationtotheOrchestrator,andontheOrchestrator’sside,thedownloadoftheVMimages isonlycarriedoutwhentheVNF is instantiatedpreventingunnecessaryuseofresources.UploadingoftheVNFDtotheVNFcatalogueisexecutedatthestartoftheVNFon-boardingprocess.

2.3.3.2. InterfacebetweentheOrchestratorandtheMarketplace

TheinterfacebetweentheOrchestratorandtheMarketplaceservesthefollowingpurposes:


©T-NOVAConsortium

30

• Provide available VNFs: involves SP browsing and selection of VNFs, as well ascomposition ofmarket services by theMarketplace, followed by a request to theOrchestrator;

• Publishanewnetworkservice:relatedtotherequestforthestorageofinformationrelated toanewserviceby theMarketplace, included in theprovisionof theNSD(seesubsection2.2).ThisprocessisalsocalledNSon-boardingbytheOrchestratorFEs;

• RequestforaNetworkService:afteraCustomer’soraSPsubscriptionofaservice,asubsequentrequestfromtheMarketplaceisgenerated,whichincludesintheNSDalltheVNFstheNStobedeployedneeds,aswellasalltheVMsthoseVNFsneedintheVNFD,theavailableinfrastructureanditscurrentusage;

• Change configuration of a deployed network service upon a request from theMarketplace;

• Providenetworkservicestatetransitions:notificationprovidedbytheOrchestratortotheMarketplace;

• Providenetworkservicemonitoringdata:notificationprovidedbytheOrchestratortotheMarketplace;

• Terminate a provisioned network service: upon a request from theMarketplacewhenthereisanexplicitsolicitation.

2.3.3.3. InterfacebetweentheOrchestratorandtheVIM

TheinterfacebetweentheOrchestratorandtheVIMservesthefollowingpurposes:

• Allocate/release/update resources: upon a request from the Orchestrator to(re)instantiate,updatetheconfigurationof,orreleasearesource(VMorconnectionwithinthesameDC);

• Reserve/releaseresources:uponanexpectedfutureneedfromtheOrchestratortoinstantiateorreleaseareservedresource(VMorconnectioninthesameDC).Thisrequirement makes sense in scenarios where allocating resources from scratch iscomplexor too time consuming for thepurpose inmind.Reserved resourcesmayhavelowerpricesthaneffectiveallocatedonesandbecomefastertoallocatewhentimecomes;

• Add/update/deleteSWimage:wheneveranew,updateorremovalofaVMimageis needed in the process of allocating, updating or removing a new VNF/VNFCinstance;

• RetrieveinfrastructureusagedatatoNSO:informationprovidedbytheVIMtotheOrchestrator,NSOFE,so thatoptimalallocationofNS instances ispossibleandanadequate level of metrics can be reported to the Marketplace, if allowed byinformationincludedintheNSD;

• Retrieve infrastructure usage data toVNFM: information provided by theVIM totheOrchestrator,VNFMFE, so thatoptimalallocationofVNF instances ispossibleandanadequate levelofmetrics canbe reported, viaNSO, to theMarketplace, ifallowedbyinformationincludedintheVNFD;

• Retrieve infrastructure resourcesmetadata to VRO: information provided by theVIM to the Orchestrator, VRO FE, so that optimal allocation of NS instances ispossible,according to thecharacteristicsof the supporting infrastructure (e.g., the


©T-NOVAConsortium

31

availabilityofspecializedcomponents,suchasGraphicalProcessingUnits(GPUs)orDigitalSignalProcessors(DSPs),aswellasthemaximumnumberofvCPUsorGiga-bytesofRAM,whichwillinfluencetheallocatingalgorithmfordeterminingthemostappropriateresources);

• ManageVM’sstate:informationprovidedbytheVIMtotheOrchestrator,VNFMFE,so that atomicoperations, suchas redeployment/withdrawalof anentireVNF, ontheallocatedVMsare feasible (dependingonthe implementationapproach, theseoperationscanalsobedonebytheIVMlayer).

2.3.3.4. InterfacebetweentheOrchestratorandtheTransportNetworkManagement

TheinterfacebetweentheOrchestratorandtheTransportNetworkManagementservesthefollowingpurposes:

• Allocate/release/update transport connection: upon a request from theOrchestrator to (re)instantiate, update the configuration of or release a transportconnection(betweentwodistinctDCs);

• Reserve/release transport connection: upon an expected future need from theOrchestratortoinstantiateorreleaseatransportconnection(betweentwodistinctDCs).Thisrequirementmakessenseinscenarioswhereallocatingconnectionsfromscratch is complex or too time consuming. Reserved connectionsmay have lowerpricesthaneffectiveallocatedonesandbecomefastertoallocatewhentimecomes;

• Retrieve transport connection usage data to NSO: information provided by theWICM to the Orchestrator, NSO FE, so that optimal allocation of transportconnectionsbetweentwoormoredistinctDCsispossibleandanadequatelevelofmetricscanbereported,togetherwithVNFanNSlevelmetrics,totheMarketplace;

• Retrievetransportconnectionmetadata:informationprovidedbytheWICMtotheOrchestrator, VRO FE, such that the optimal allocation of transport connectionsbetweentwoormoredistinctDCsismadepossible,accordingtothecharacteristicsofthesupportinginfrastructuree.g.,maximumnumberofvLinksallowed,maximumbandwidth,etc.;

• Manage transport connection state: information provided by the WICM to theOrchestrator,NSOFE,sothatatomicoperations,suchasredeployment/withdrawalofanentireVNFwithindifferentDCsarefeasible(dependingontheimplementationapproach,theseoperationscanalsobeexecutedbytheIVMlayer).

2.3.3.5. InterfacebetweentheOrchestratorandtheVNF

TheinterfacebetweentheOrchestratorandtheVNFservesthefollowingpurposes:

• Instantiate/terminateVNF:sentbytheVNFMasarequest,wheneveraninstanceofaNSofwhichtheVNFisacomponentistobelaunchedorremoved.RemovalofaVNFinstancecanonlybedoneifthereisnoNSinstanceusingthatVNF.

• Retrieve VNF instance run-time information: sent by the VNFM, so that VNF SLAmetricscanbecheckedandtheSLAcanbefulfilled;

• ConfigureaVNF:sentbytheVNFM,sothatopenconfigurationparameterscanbefulfilledlaterorchangedaftertheVNFinstanceisalreadyrunning;


©T-NOVAConsortium

32

• ManageVNFstate:sentbytheVNFM,sothattheOrchestratorisabletostart,stop,suspendalreadyrunningVNFinstances;

• Scale VNF: sent by the VNFM, so that VNF scaling is feasible. All the VNF scalinginformationisavailableintheVNFD.VirtualisedresourcesareavailablethroughtheVRO.


©T-NOVAConsortium

33

3. THET-NOVAIVMLAYER

3.1. INTRODUCTION

T-NOVA’s InfrastructureVirtualisationandManagement (IVM) layerprovides the requisitehosting and execution environment for VNFs. The IVM incorporates a number of keyconcepts that influence theassociated requirements andarchitecture for the layer. Firstlythe IVM supports separation between control and data planes and networkprogrammability. The T-NOVAarchitecture leverages SDN for designing, dimensioning andoptimising control- and data-plane operations separately, allowing capabilities from theunderlyinghardware tobeexposed independently. Secondly the IVM isbasedaround theuse of clusters of SHV computing nodes in cloud computing configurations to supportinstantiationofsoftwarecomponentsintheformofVMsforNFVsupport,offeringresourceisolation, optimisation and elasticity. This configuration should support automateddeployment of VNFs from the T-NOVAmarketplace anddynamically expansion/resizing ofVMs as required by SLAs. Building on physical IT and network resource domains the IVMprovides full abstraction of these resources to VNFs. Finally the IVM must expose thenecessaryexternaland internal interfaces tosupportappropriate integration.Theexternalinterfaces provide connectivity with the T-NOVA Orchestration layer in order to executerequests from theOrchestrator and secondly toprovide information suchasperformancemetricsrelatingtotheinfrastructureandVNFsbeinghostedinorderfortheOrchestratortomakeeffectivemanagementdecisions.Theinternalinterfacesprovideconnectivitybetweenthe internal domains of the IVM to ensure the requests for the creation, deployment,management and termination of VNF services and their host VMs can be executedappropriatelyamongtheconstituentinfrastructureandcontrolcomponents.

For an architectural perspective the IVM is comprised byNFVI, VIM andWICM functionalentities.TheNFVI inturn iscomposedofCompute,HypervisorandNetworkDomains.TheVIMiscomprisedofcompute,hypervisorandnetworkcontrolandmanagementcapabilities,whiletheWICMworksasasingleFE.AlltheFE’swithintheIVMimplementnorthboundandsouthbound interfaces to providemanagement, control andmonitoring of the compositeinfrastructure, both physical and virtualised. Secondly these interfaces provide the keyintegrationcapabilitieswithintheoverallT-NOVAsystemarchitecture.

The following sections describe the key objectives and characteristics of the IVM and itsconstituent components, along with their requirements. These requirements where thenutilisedtogetherwithT-NOVAD2.1[6]andD2.22[4]todefinethearchitectureofIVMinamanner that addressed these requirements and the overall goals of T-NOVA. Thearchitecture for T-NOVA is presented as an overall integrated architecture together withdetaileddescriptionsofthearchitectureFEsandinterfacesoftheconstituentdomains.

3.2. OBJECTIVESANDCHARACTERISTICSOFTHET-NOVAIVMLAYER

TheT-NOVAIVMisconsideredtomanageamixtureofphysicalandvirtualnodesandwillbeused to develop, implement and showcase T-NOVA’s services. The IVM will be fullyintegratedwith theT-NOVAOrchestrator toensure that requirements for thedeploymentandlifecyclemanagementofT-NOVAVNFservicescanbecarriedoutinanappropriateandeffectivemanner. The IVMshouldbe sufficiently flexible to supporta varietyofuse casesbeyond those explicitly identified in T-NOVA (see D2.1 [6]). As mentioned previously,infrastructurevirtualisationplaysakeyroleinachievingthisvisioninT-NOVA.Virtualisation


©T-NOVAConsortium

34

andmanagementofthevirtualisedresourcesextendsbeyondthecomputeandstoragetoinclude network infrastructure in order to fully exploit the capabilities of the T-NOVAarchitecture.VirtualisationoftheDCnetworkinfrastructureallowsdecouplingofthecontrolfunctionsfromthephysicaldevicestheycontrol. InthisregardT-NOVAis implementinganSDN control plane for designing, dimensioning andoptimising the control- anddata-planeoperations separately, allowing capabilities from the underlying hardware to be exposedindependently.

InsummarythekeyobjectivesfortheT-NOVAIVMareasfollows:

• Supportfortheseparationofcontrolanddataplaneandnetworkprogrammabilityatleastatcriticallocationswithinthenetworksuchasthenetworkaccess/borders,

• Utilisation of commodity computing nodes in cloud configurations to support theinstantiationofsoftwarecomponents intheformofVMs,containersorunikernelsforNFVsupport,offeringresourceisolation,optimisationandelasticity,

• Use of L2 Ethernet switched networks (subnets) to provide physical networkconnectivitybetweenservers,

• Eachserver supportsvirtualisationandhostsanumberofVMs (virtualappliances)belonging to their respective vNets. Virtual switch instances or real physical SDN-capable switcheshandlenetwork connectivity among theVMseitheron the sameserveroramongtheserversco-locatedinthesameDC,

• InterconnectionofL2subnets insideandoutsideDC’sboundariesviaaL3network(IP routers). This interdata centre connectivity is provisioned throughappropriateWANingressandegresspoints.

• Virtualisation of compute and network resources allows the T-NOVA system todynamically scale out VMs. This accommodates sudden spikes inworkload traffic;the instantiation of network elements as VMs into clusters of nodes facilitateshorizontal scaling1 (hostingofmanyVM instances into thesamecluster)accordingto function requirements and traffic load) (see T-NOVA requirements/use cases –D2.1[6]).

3.3. T-NOVAIVMLAYERREQUIREMENTS

TherequirementscaptureprocessfocusedonidentifyingthedesiredbehavioursoftheIVM.The requirements identified focus on the entities within the IVM, the functions that areperformed to change states or object characteristics, monitoring of state and the keyinteractionswiththeT-NOVAOrchestrationlayer.Noneoftheserequirementsspecifieshowthesystemwillbeimplemented.ImplementationdetailsarelefttotheappropriatetasksinWP3/4astheimplementation-specificdescriptionsarenotconsideredtoberequirements.ThegoaloftherequirementswastodevelopanunderstandingofwhattheIVMneeds,howit interacts with Orchestration layer, its relationship to the overall T-NOVA architecturedescribedinD2.22[4].AdditionallytheusecasesincludedinD2.1[6]werealsoconsideredandcross-referencedwithIVMrequirementswhereappropriate.

Theinitialphaseofelicitingrequirementsincluded:• Reviewing available documentation including drafts and final versions ofD2.1 and

ETSI’s specification of the Network Functions Virtualisation (NFV); ArchitecturalFramework.

1TheT-NOVAinfrastructureincloudbasedwhichtypicallyonlysupportshorizontalscaling.


©T-NOVAConsortium

35

• Reviewing the high level T-NOVA architecture that was developed by Task 2.2 togather informationonhowtheusersandserviceproviderswillperformtheirtaskssuch as VNF deployment, scale out etc., and to better understand the keycharacteristics of the T-NOVA system that will be required to realise user goalsincludingthoseatasystemlevel.

Theadoptedapproachwas generally in-linewith the Instituteof Electrical andElectronicsEngineer(IEEE)guidelinesforrequirementsspecification.AsimilarprocesswasusedinTasks2.1 and2.2. Requirementswereprimarily anchored to theexisting T-NOVAuse cases andthe interactions with Orchestrator both in terms of the actions and requests that theOrchestrator would expect the IVM to execute. Additionally the data/information that isrequired by the Orchestrator to successful deploy and manage VNF services wereconsidered. Identified requirements were primarily functional in nature since they wererelated to the behaviour that the IVM is expected to exhibit under specific conditions. InadditionETSI’sNFVVirtualisationFrameworkrequirementswerealsoconsidered,inordertoensure approach scope and coverage for the requirements that have been specified. Thefollowingarethekeycategoriesofrequirementsthatwereconsidered:

• Portability• Performance• Elasticity• Resiliency

• Security• ServiceContinuity• ServiceAssurance• OperationsandManagement

UsingasystemengineeringapproachthehighlevelarchitecturefortheIVMwaspreviouslydescribedin[4].Eachcomponentoftheoverallsystemwasspecifiedintermsofhigh-levelfunctional block and the interactions between the functional blocks are specified asinterfaces.Thisapproachidentifiedthefollowingfunctionalblocks:

• VirtualisedInfrastructureManagement(VIM),• WANInfrastructureConnectionManager(WICM),• InfrastructureElements,consistingofComputing,HypervisorandNetworking.

Therequirementspresented inthefollowingsectionarerelatedtothesefunctionalblocksandweredevelopedusingthepreviouslydescribedmethodology.Theserequirementswereusedasa foundational input into thedevelopmentof theoverall IVMarchitectureand itsconstituentfunctionalblocks,whichispresentedinsubsection3.4.

A detailed specification of the requirements for each module within the scope of the T-NOVA IVM architecture can be found in Annex B. A total of 66 final requirements wereidentifiedanddocumentedrelatingtotheVIM,NFVI(compute,hypervisor,DCnetwork)andWICM.ItshouldbenotedthatrequirementsthatrelatetobasicexpectedbehavioursofthevariousdomainscomponentshavebeenexcludedinordertofocusonrequirementsthatarespecificallyneededbytheT-NOVAsystem.Analysisoftheserequirementshasidentifiedthefollowingconclusionsforeacharchitecturalmodule.

3.4. VirtualInfrastructureManager(VIM)

TheVIMisrequiredtomanageboththeIT(computeandhypervisordomains)andnetworkresources by controlling the abstractions provided by the Hypervisor and Infrastructurenetwork domains. It also implements mechanisms to efficiently utilise the availablehardwareresourcesinordertomeettheSLAsofNSs.TheVIMisalsorequiredtoplayakeyrole in the VNF lifecycle management. Additionally, the VIM is required to collectinfrastructure utilisation/performance data and to make this data available to theOrchestratorinordertogenerateusage/performancestatistics,aswellastriggeringscaling


©T-NOVAConsortium

36

ofVNFsifnecessary.ThespecificsofhowthemetricsareprovisionedandprocessedatboththeVIMandOrchestratorlayerscanvaryandwilltypicallybeimplementationspecific.ThedetailsoftheT-NOVAimplementationarebeingdeterminedinWP3/4.ToaccomplishthesegoalstheVIMneedsthefollowingcapabilities:

• TheNetworkControlcapabilityintheVIMexploitsthepresenceofSDNfeaturestomanagetheinfrastructurenetworkdomainwithinanNFVI-PoP;

• Hardware abstraction in the Compute domain for efficient management ofresources; however, the abstraction process should ensure that platform specificinformationrelevanttotheperformanceofVNFs isavailableforresourcemappingdecisions;

• Virtual resource management in the Hypervisor domain to provide appropriatecontrolandmanagementofVMs;

• Strong integration between the three sub-domains above through appropriateinterfaces;

• Integration with the Orchestrator via well-defined interfaces to provideinfrastructure related data to the Orchestrator and to receive management andcontrolrequestsfromOrchestratorforexecutionbytheVIM.

3.4.1. WANInfrastructureConnectionManager

TheWICMisexpectedtoprovidethelinkbetweenWANconnectivityservicesandtheNFVIhosting VNFs including connectivity to NSs allocated in more than one NFVI-PoP.Connectivity should take form of VLAN to WAN connections through ingress and egresspointsateachNFVI-PoPinvolvedintheNSservice.Thisconnectivityhastobeprovidedinaconfigurablemanner (i.e. supports a high level of customisation).Moreover, to setup thisconnectivity, cooperation between theWICMand theNFVINetwork domain is needed inordertoallocatethetrafficovertheinter-DCandWANnetworksinanappropriatemanner.

3.4.2. NFVICompute

TheNFVIComputedomainshouldbeabletoprovideanappropriatedperformancelevelfortheVNFsthatarebeendeployedintermsofperformancewhileensuringoptimalutilisationoftheavailablecomputeresource.Moreover,thecomputenodesandthehypervisorshouldworkinanintegratedandperformantmanner.ThecomputedomainshouldcollectmetricsontheperformanceofthephysicalresourcesandmakethemavailableattheVIMlevelforsubsequent exposure to the Orchestration layer. Finally, the T-NOVA Compute domainshould have the capability, if required by a network service, to support heterogeneouscompute resources, such as Graphical Processing Unit (GPUs), Field Programmable GateArray(FPGAs),Multi-IntegratedCores(MICs)etc.

3.4.3. NFVIHypervisor

TheNFVIHypervisordomain shouldbeable to implementhardware resourceabstraction,virtual resource lifecycle management mechanisms which are coordinated by theOrchestrator via theVIM, and toprovide to theVIMmonitoring informationwhilehavingminimalimpactontheVNFworkloadperformance.Additionaldetailsonthestepinvolvedinthecollection,processingandutilisationofmetrics in theT-NOVAsystemcanbe found insubsection4.1.3.


©T-NOVAConsortium

37

3.4.4. NFVIDCNetwork

The NFVI DC Network domain should implement an SDN approach to provide networkvirtualisationcapabilities insideaNFVI-PoP(creationofmultipledistinctdomainsoveronesingle physical network using VLANs), network programmability through the separationbetween Control Plane (CP) and Data Plane (DP). Moreover it should support transporttunnelling protocols of L2 packets over L3 networks, to assist theWICM in setting up thecommunication/betweendifferentNFVI-PoPs.ItshouldalsobeabletogatherperformancedataandsendthemtotheVIMNetworkControlmodule.

3.5. T-NOVAIVMArchitecture

As mentioned above, the T-NOVA IVM layer comprises of three key architecturalcomponentsnamelytheNFVI,theVIMandtheWICM.

Thehigh-levelarchitecture,whichhaspreviouslybeendescribedinD2.22[4],wasdesignedtoalignwiththeETSIMANOarchitecturefeaturingcorrespondingcomponentstotheNFVIandtheVIM.However,theadditionoftheWICMisaT-NOVAspecificfeature.

Theapproachadopted in thedesignandelucidationof the IVMfocusedon the functionalcharacteristics of the overall IVM architecture and its sub domains. Careful considerationwas given to decoupling the functional characteristics from implementation-orienteddesigns.ThisallowedustofocusonwhattheIVMneedstodorathertoavoidtheinclusionof implementation-orientated functionality. A good example of where this approachgenerated challenges was with the VIM architecture where there was a tendency togravitate towards technology solutionsas ameans toeasilyencapsulate functionalneeds.Howevercarefulconsiderationofthekeyinputswasimportantinfullydecouplingfunctionalneeds from implementation details to ensure that T-NOVA IVM architecture remainstechnology-agnostic but at same timeprovides appropriate guidance and structure to theactivitiesinWP3/4.

The key inputs that were considered during the architecture design process were thefollowing:

• D2.1(Usecaseandrequirements)[6],

• D2.22(OverallSystemArchitectureandInterfaces)[4],

• DGSNFV-INF001v1.1.1-InfrastructureOverview[12],

• DGSNFV-INF003v1.1.1-ArchitectureoftheComputeDomain[13],

• DGSNFV-INF004v1.1.1-ArchitectureoftheHypervisordomain[14],

• DGSNFV-INF005v1.1.1-Infrastructurenetworkdomain[15],

• DGSNFV-INF007v1.1.1-InterfacesandAbstractions[16],

• DGSNFV-MAN001v1.1.1-ManagementandOrchestration[9],

• DGSNFV-REL001v1.1.1-ResiliencyRequirements[17],

• DGSNFV-SWA001v1.1.1-VNFArchitecture[18].

TheIVMarchitecturehasbeendefinedinaccordancetoasystemsdesignprocesswhichwasused to identify the components,modules, interfaces, and data necessary for the IVM inordertosatisfytherequirementsoutlinedintheprevioussubsectionandthosedescribedin


©T-NOVAConsortium

38

D2.1[6]andD2.22[4].Figure3-1showstheoverallarchitectureoftheVIM,asdiscussedsofar.

3.5.1. ExternalInterfaces

ThekeyexternalinterfacesfortheIVMareoutlinedinTable3-1.TheseinterfacesprimarilydealwiththeconnectionbetweentheIVMandtheT-NOVAOrchestrator;howeverthereisalsoanexternalinterfacebetweentheWICMandthetransportnetwork.

TheinterfacesbetweentheOrchestratorandVIMsupportanumberofkeyfunctionswithinT-NOVA. The functions supported by the interfaces can be categorised as eithermanagement or control. As shown in the IVM architecture in Figure 3-1, two specificinterfaces have been identified, mapping to the interfaces identified in the ETSI MANOarchitecture.

Figure3-1:T-NOVAinfrastructurevirtualisationandmanagement(IVM)highlevelarchitecture

The first interface is the VNFM – VIM Interface (T-Vi-Vnfm) and is responsible for theexchange of infrastructure monitoring information either through explicit request by theOrchestrator or through periodic reporting initiated by the VIM. The types of dataexchanged over this interface include detail information on the status, performance andutilisation of infrastructural resources (such CPU, storage, memory, etc.). Data will alsoencompass networking information relating to a specific VNF such as NIC level networktraffic from thehostingVMor interVMnetwork traffic, if aVNF isdeployedacrossmorethanoneVM.

FinallyVNFperformancedatawillalsobeexchangedoverthisinterface.Collectivelythedatawill be used by the VNF Manager within the T-NOVA Orchestrator to track VNF serviceperformancebycomparisonwithspecificKPIsinordertoensureSLAcompliance.


©T-NOVAConsortium

39

The second interface identified is the NFV Orchestrator – VIM interface (T-Or-Vi). ThisinterfaceisresponsibleforhandlingrequestsfromtheNFVOrchestratorwithrespecttothefulllifecycleofaNS.Typicalexamplesofrequestssentoverthisinterfacewouldinclude,forexample, parts of the on-boarding, scaling and termination procedures of a NS. Thisinterface will be used by the NFV Orchestrator to send resource relatedcommands/informationtotheVIMsuchasresourcereservation/allocationorconfigurationdefinitions of VMs (e.g. HEAT templates in an OpenStack Cloud environment or networkrequirementssuchasthespecificationofthe interconnectionsbetweenVNF instances, i.e.networktopology).

Additionally, this interface will be utilised also to exchange specific types of monitoringinformation suchasdata related to thenetworkconnectionsbetweenNS instanceseitherwithinadata centreorwithin intradata centre connections that arephysicallydispersed.This interface will also include also detail information on the status, performance andutilisationof infrastructuralresources,networkinginformationrelatedtoaspecificVNF,aswellasVNFperformancedata.This interfaceisalsousedbytheVIMtoreportbacktotheNFVOrchestratortheoutcomeofallreceivedrequests.

TheWICMprovidesmanagement capabilitiesofnetwork connectivitybetweenNFVI-PoPs.The NFV Orchestrator – WAN Infrastructure Connection Manager (T-Or-TM) interfacesupport requests from the NFV Orchestrator to provide connectivity to either SDNControlled or non SDN control transport networks (such as IP or MPLS based networks)typicallyforinterDCs(MANorWAN).Thesenetworksarenon-virtualisedinnature.

TheWICMInterface–Externalnetworks(T-Tm-Tr)istheexplicitnetworkconnectiontothetransport network (WAN). The implementation of this interface will vary based on theprotocol the network is using. More than one interface may also be implemented ifconnectivitytodifferenttypesoftransportnetworksisrequired.

Table3.1:ExternalInterfacesoftheT-NOVAIVM

T-NovaName T-NOVAReference

ETSIISGNFVFrameworkReferencePoint

ReferencePointType

DescriptionandComment

VirtualNetworkFunctionManagement–VIMInterface

T-Vi-Vnfm Vi-Vnfm ManagementInterface

ThisinterfaceisresponsiblefortheexchangeofinfrastructuremonitoringinformationeitherthroughexplicitrequestbytheOrchestratororthroughperiodicreportinginitiatedbytheVIM.Thetypesofdataexchangedoverthisinterfaceincludestatus,performanceandutilisationofinfrastructuralresources


©T-NOVAConsortium

40

NFVOrchestrator–VIMInterface

T-Or-Vi Or-Vi OrchestrationInterface

ThisinterfaceallowstheNFVOrchestratortorequest/reserveandconfigureresourcestotheVIMandfortheVIMtoreportthecharacteristics,availability,andstatusofinfrastructureresources.

NFVOrchestrator–WideAreaNetworkInterface

T-Or-Tm - OrchestrationInterface

ThisinterfacestheWICMwiththeNFVOrchestratorandisusedtomanagetheset-up,teardownandmonitoringofconnectionsintransportnetworks.

WideAreaNetworkInterface–Externalnetworks

T-Tm-Tr Ex-Nf TrafficInterface

ThisinterfacestheWICMwithexistingWideAreaNetworks(SDN-enabledornon-SDN-enabled)andareusedtoimplementrequestsreceivedfromtheOrchestratorviatheOr-Tminterface.

3.5.2. InternalIVMInterfaces

The key internal interfaces of the IVM are outlined in Table 3-2. The interface for NFVImanagementincludingitsfunctionalentitiesisprovisionedviatheT-Nf-Viinterface.Itisthisinterface, which will be utilised to establish trust and compliance of the underlyinginfrastructure specifically the Hypervisor domain via the T-Nf-Vi/H implementation of theinterface,thecomputedomainviatheT-Nf-Vi/CinterfaceandthenetworkdomainviatheT-Nf-Vi/Ninterface.AfulldescriptionoftheseinterfacesispresentedinTable3-2.Apossibledeploymentconfigurationfor theVIMcouldbeprovidedrunning itwithinahostedVM(itcanbevirtualised).Forthisspecificconfiguration,theT-Nf-Vimanagement interfacemightbeabstractedasaSWA-5interface.However,evenifthisconfigurationispossible,itisnotdesirable,duetosecurityconcernsandFEresponsibilities.TherearealsoreliabilityconcernsregardingvirtualisingtheVIMonthesame infrastructurethat it ismanaging. Inascenariowherethehypervisordomainof theNFVI requiresarestart, theVIMwill lose itsability tooperate and continue to manage the NFVI therefore the VIM should run on separatedhardwareplatformseithervirtualisedornote.

OneoftheinterfacesthatareinternaltotheNFVIistheSWA-5interface2,whichisusedforresources,suchasavirtualNIC,avirtualdiskdrive,virtualCPU,etc.ExaminingFigure3-1,thisinterfaceinvolvesboththeT-Vn-Nf/VNandT-Vn-Nf/VM.Itisnotintendedforuseasamanagementinterface,insteaditisprimarilyintendedtologicallyfenceoffresponsibilities,2SWA-5correspondstoVNF-NFVIcontainerinterfaces:ThisisasetofinterfacesthatexistbetweeneachVNFandtheunderlyingNFVIthusSWA-5describestheexecutionenvironmentforadeployableinstanceofaVNF.


©T-NOVAConsortium

41

but isalso intendedforsecurityconsiderations. In fact, reasonablestepsmustbetakentopreventunauthorisedaccess,fromwithinaVM,fromattackingtheunderlyinginfrastructureand obtaining management privileges, thus being able to possibly shut down the entiredomain,includingallotheradjacentVMs,redirecttraffic,etc.

Table3.2:InternalinterfacesoftheIVM

T-NovaName

T-NOVAReference

ETSINFVFrameworkReferencePoint

INFReferencePoint

ReferencePointType

DescriptionandComment

VIM-NetworkInterface

T-Nf-Vi/N

Nf-Vi

[Nf-Vi]/N

Management,Orchestrationand MonitoringInterface

ThisinterfaceisusedforthemanagementofInfrastructureNetworkdomainresources.

VIM-HypervisorInterface

T-Nf-Vi/H [Nf-Vi]/H

Management,Orchestrationand MonitoringInterface

ThisinterfaceisusedforthemanagementoftheHypervisordomainresources.

VIM–ComputeInterface

T-Nf-Vi/C [Nf-Vi]/C

ManagementandOrchestrationInterface

ThisinterfaceisusedforthemanagementofComputedomainresources.

Hypervisor–NetworkInterface

T-Vl-Ha/Nr Vl-HA [Vl-Ha]/Nr ExecutionEnvironment

ThisinterfaceisusedtocarryinformationbetweentheHypervisorandtheInfrastructureNetworkdomains.

Hypervisor-ComputeInterface

T-Vl-Ha/CSr VI-Ha [Vl-Ha]/CSr Execution

Environment

ThisinterfaceisusedtocarryexecutioninformationbetweentheComputeandtheHypervisordomain.

Compute–NetworkInterface

T-Ha/CSr-Ha/Nr VI-Ha Ha/CSr-

Ha/Nr TrafficInterface

ThisinterfaceisusedtocarryexecutioninformationbetweentheComputeandtheNetworkdomain.

VirtualMachine–VNFC

T-Vn-Nf/VM Vn-Nf [Vn-Nf]/VM VNF

Execution

Thisinterfaceisusedtocarryexecution


©T-NOVAConsortium

42

Interface Environment environmentinformationforeachVNFCinstance.

VirtualNetwork–VirtualNetworkInterface

T-Vn-Nf/VN [Vn-Nf]/VN

VNF

ExecutionEnvironment

ThisinterfaceisusedtocarryexecutionenvironmentinformationbetweenVNFCinstances.

3.6. NFVIandNFVI-PoP

TheexecutionenvironmentforVNFsisprovidedbytheNFVIdeployedinvariousNFVI-PoPs.TheNFVI-PoPacts singlegeographic location i.e. aDCwhereanumberofNFVI-nodesarelocated.ANFVI-PoP is responsible for providing the infrastructural buildingblocks tohostandexecuteVNFservicesdeployedbytheT-NOVAsysteminaparticularlocation.TheNFVIcomprises of the IT resources in the form of the Compute and Hypervisor domains andnetwork resources in the formofNetwork domain, as shown in Figure 3-1. TheNFVI canutilise these domains in a manner that supports extension beyond a single NFVI-PoP tomultipleNFVI-PoPsasrequiredtosupporttheexecutionofagivenNS.

The NFVI-PoP is expected to support the deployment of VNFs in a number of potentialconfigurations.Thesedeploymentswill rangefromasingleVNFdeployedatasingleNFVI-PoP,tomultipleVNFsfromdifferentVNFprovidersinamulti-tenantmodelatoneNFVI-PoP.Additionally, theNFVImayneed to supportVNFsdeployedatmore thanoneNFVI-PoP toinstantiate the required NS. Interconnectivity between these PoPs is provisioned andmanaged in the case of T-NOVAby theWICMmodule,which is similar in function to theWANControllerintheETSIMANOarchitecture[9](seesubsection3.8).

ThenetworkaccesscapacityrequiredataparticularNFVI-PoPwilldependonthenetworkserviceworkloadtype,thenumberandcapacityoftheVNFs instantiatedontheNFVI.Themanagement and orchestration of virtualised resources should be able to handle NFVIresources in a single NFVI-PoP as well as when distributed across multiple NFVI-PoPs.Management of the NFVI is provided by the VIM through domain interfaces (T-Nf-Vi) asshown in Figure 3-1. The VIM also provides the intermediate interfaces between theOrchestratorandtheNFVI(T-Or-ViandT-Vi-VNFM).TheNFVIwillexecuterequestsfromtheOrchestratorviatheVIMrelatingtothelifecyclemanagementofVNFssuchasdeploymentscalein/outandtermination.

ThefollowingsectionsdescribethearchitectureandtherespectiveinternalcomponentsoftheNFVI,namelythecompute,hypervisorandnetworkdomains.TheinterfacesrequiredtoimplementanoverallfunctionalarchitecturefortheT-NOVANFVIsystemarealsodescribed.

3.6.1. ITResources

TheT-NOVA ITResourcesencompasses thecomputeandhypervisordomainsof theNFVI.These domains have their origins in traditional enterprise IT environments and morerecently in the deployment of cloud computing environments. In order to support thedevelopment ofNFV architectural approaches in carrier grade environments, IT resourcesandcapabilitieshavebeenembraced intheseenvironmentstosupportthedeploymentofVNFs. However the functionality, capabilities and how these IT resources are composed


©T-NOVAConsortium

43

within virtualised network architectures need to be carefully considered. The enterpriseoriginsofthesetechnologiesoftenhaveinherentgapsincapabilitysuchasline-ratepacketprocessing performance limitations. These gaps can influence architectural decisions andmayrequireinnovativesolutionstoaddressanyidentifiedgapsforVNFservicedeployment.The following sections discuss the compute and hypervisor domain architectureconsiderations and the proposed approach in the context of the T-NOVA systemarchitecture.

3.6.1.1. ComputeDomain

TheComputeDomainisoneofthreedomainsconstitutingtheNFVIandconsistsofservers,NICs, accelerators, storage, racks, and associated physical components within the rack(s)related to the NFVI, including the networking Top of Rack (ToR) switch. The Computedomainmaybedeployedasanumberofphysicalnodes(e.g.ComputeandStorageNodes)interconnectedbyNetworkNodes(devices)withinanNFVI-PoP.

Traditionally the compute environment within the telecoms domain has beenheterogeneousbasedaroundavarietyaroundmicroprocessorarchitectures suchasMIPS,PowerPC, SPARC, etc. with tight coupling between the microprocessor and the softwareimplementation.ManytraditionaltelecommunicationsystemsarebuiltinC/C++technologywith high interdependence on the underlying processing infrastructure and a specificinstructionset.

AsthefirstgenerationofcommercialVNFshavebecomeavailablebasedonadaptionofthesoftware from previous fixed appliances to a version that can run in virtualised X86environments some performance difficulties have been encountered. While virtualisationdecouplessoftwareandhardwarefromadeploymentpointofview,itdoesnotdoitfromadevelopment point of view. Selection of an appropriate cross compiler for the targetplatform (e.g. X86)may address someof the issues.However in order to achieveoptimalperformance,aproperredesignofthesoftwaremayberequiredtoensureappropriateuseofspecificcapabilities,forexamplehyperthreadinginX86processors.Theapplicationmayalso need to use certain software libraries to improve the performance of certain actionssuchaspacketprocessing.HoweverVNFsrunningonSHVserversgenerallywillhavesomeoftradetrade-offbetweenflexibility,easeofdevelopment&deployment,performanceetc.incomparisontocustomhardwareplatforms.

Thecomputedomainarchitectureshouldhavethecapability tosupportdistributedvirtualappliances that can be hosted across multiple compute platforms as required by specificSLAs of network services. Moreover storage technologies and management solutions areincluded in the domain and show a large degree of variability in terms of differenttechnologies; scalability and performance (see start of the art review). Depending on theworkloadsanduse-casesthechoiceofstoragetechnology is likelytobespecific tocertainworkloadtypes.

Anotherimportantobjectiveofthecomputedomainistoexposehardwarestatisticsofthecompute node with high temporal resolution. The VIM communicates directly to thecomputedomainandthroughthehypervisortoaccessallthehardwaremetrics,whichcanbestaticordynamicinnature.

Staticmetricsexposecomputenodecharacteristicswhichdonotchangeorchangeslowly(e.g.onceaday).Thesemetricsultimatelyactasafirstorderfilterforselecting/provisioninganode fordeployingaVNF. Staticmetrics areobtained from readingOSandACPI tables.The Advanced Configuration and Power Interface (ACPI) specification provides an openstandard for device configuration and power management by the operating system.


©T-NOVAConsortium

44

Additionalmetricsmaybestoredinlocalstructuresprovisionedbyservervendorsorsystemadministrators. For example compute node performance index, energy efficiency index,geographiclocation,specificfeaturesenabled/supported,securitylevel,etc.

AnOrchestratorcanidentifyacandidateplatformbasedonstaticmetrics,howeverinorderto actually instantiate aVNF additional dynamicmetrics are required, e.g. CPUutilisation,memory, I/Oheadroomcurrentlyavailableetc.Thesemetricscouldbeprovidedonaper-query basis or the compute node could proactively update hypervisor domain at regularintervals.

HeterogeneousComputeArchitectures

A VNF developed for a target compute architecture needs to be fully optimised andvalidated prior to rollout. This process will also need to be repeated on a per computearchitecturebasis.Whilethe initial focushasbeenonX86computearchitectures, recentlytherehasbeeninterestintheuseofco-processorssuchGPUsorFPGAstoacceleratecertainVNFworkloads: someworkloadscanexperienceperformancebenefitsbyhavingaccess todifferentprocessingsolutionsfromavarietyofsiliconvendorsincludingnetworkprocessorsand general-purpose co-processors. The move towards more heterogeneous cloudcomputingenvironmentsisstartingtogainattention,asstandardX86processorsmayhaveperformancelimitationsforcertainworkloadstaskse.g.highspeedpacketprocessing.Thishas ledDCequipmentvendors to investigate theuseofalternativecomputearchitecturesprovisioned in either in a standalonemanner or coupledwith X86 processors to enhancetheir offerings. In defining heterogeneous compute domain architectures for T-NOVA, wedivideddevicesintotwomaincategories:

• Deviceswhich can only operate in conjunctionwith a host CPU, like GPUs,multi-integratedcores(MIC)andMicron’sAutomataprocessor;

• Devices which can operate in a stand-alone fashion, like FPGAs and FPGA SoCs(althoughFPGAsandFPGASoCscanalsoactasdevicesattachedtoaCPU-controlledsystem).

Forthefirstclassofdeviceswecanderiveacomputenodearchitecture,wherethecomputenode is complemented by a co-processor, which can be any of the four technologiesmentioned above (in the FPGA and FPGA SoC cases, theywill, act as slave devices to theprocessor).Theextenttowhichtheacceleratorresourcesthemselvesarevirtualised is leftto each specific implementation, though such a solution is known to improve theperformance of the accelerator hardware. It must be noted that such a solution is onlyavailable forGPUs (e.g.nVidia’sGRID),butnot forFPGAsor theautomataprocessor.Thisgeneral architecture leaves a lot of the implementation choices open. For example theinterconnectionoftheCPUandtheco-processorcouldbeimplementedeitheroverPCIeoroveradirectlinklikeQuickPathInterconnect(QPI)ordivisionofthememorybetweentheCPU and the co-processor. In any scenario, the system must be able to adhere to therequirementsforthecomputenodesasoutlineddowninAnnexB.

ThesecondclassofdevicesisbasedaroundanFPGASoC,whichisanFPGAthatintegratesoneormoreprocessorcoresinthesamesilicondie.DevicesliketheseareavailablefromallmajorFPGAvendors.Inthiscase,theprocessingsystemontheFPGASoCrunsaHypervisoronwhichOS’andapplicationsareexecuted.Toalargeextentthesameconsiderationsasinthe previous scenario apply, both in terms of interconnection of components andvirtualisation of accelerator resources. The important difference here is the degree ofintegration, since the whole heterogeneous compute node resides within one physicaldevice.


©T-NOVAConsortium

45

KeyComponentsoftheComputeDomain

ThemaincomponentsoftheComputedomain’sarchitectureare:

• CPUandAccelerator:Ageneral-purposecomputearchitecture isconsideredbasedon commercial x86 server clusters. Additionally co-processors cards/FPGAs andGPUs are also considered for application specific workloads or functions such aspacket processing. As outlined in state of the art review the CPU nodes willincorporate technologies to support virtualisation of the actual CPU such as VT-x.ConnectionstoI/OdeviceswillusetechnologiessuchasVT-d.Specificco-processorsinclude acceleration chip for classification, Crypto, DPI/Regular Expression,Compression/Decompression, Buffer management, Queue management, Workscheduler,Timermanagement,Trafficmanagement,addresstranslation);

• NetworkInterfaces:ThenetworkinterfacecouldeitherbeaNICwhichconnectstotheprocessorviaPCIeorthenetworkinterfacecapabilitymayberesidenton-boardtheserver.ProvisioningofvirtualisednetworkconnectivityandaccelerationwillusetechnologiessuchasVT-c.

• Storage:Storageencompasseslarge-scalestorageandnon-volatilestorage,suchasharddisksandsolid-statedisk(SSD)whichcanbewithlocallyattachedornetworkedinconfigurationssuchasSAN.Forsomepurposes,itisnecessarytohavevisibilityofthe different storage hierarchy level (Cache Storage, Primary Storage, SecondaryStorage,ColdStorageorArchivedStorage)eachonecharacterisedbyspecificlevelsof latency, costs, security, resiliency and feature support. However, for manyapplications,thedifferentformsofstoragecanbeabstracted,especiallywhenoneformofstorageisusedtocacheanotherformofstorage.Thesecachescanalsobeautomatedtoformatieringfunctionforthestorageinfrastructure.

CollectivelythesetechnologiesenablethehypervisortoabstractthephysicalresourcesintovirtualresourceswhichcanbeassembledintoVMsforhostingVNFs.ItshouldbenotedthatasingleComputePlatformcansupportmultipleVNFs.


©T-NOVAConsortium

46

Figure3-2:ComputeDomainHighLevelArchitecture

ComputeDomainInterfaces

Thecomputedomainpresentsthreeexternalinterfaces:

• TheT-Nf-Vi/C-usedbytheVIMtomanagethecomputeandstorageportionoftheNFVI.Itisthereferencepointbetweenthemanagementandorchestrationfunctionsincomputedomainandthemanagementandorchestrationfunctionsinthevirtualinfrastructuremanagement(VIM);

• TheT-[Vi-Ha]/CSr interface is the interfacebetween thecomputedomainand thehypervisordomain.Itisprimarilyusedbythehypervisor/OStogaininsightintotheavailablephysicalresourcesofthecomputedomain;

• TheT-HA/CSr-Ha/Nr interface is used to carryexecution informationbetween theComputeandtheNetworkdomain.

Orchestration and management of the NFVI is strictly implemented via the T-Nf-Viinterfaces. The implementation of the interfacemustmatch the requirements outlined inAnnexBinadditiontohavingthegeneralcharacteristicsofbeingdedicatedandsecure.Thisinterface isutilisedtoestablishtrustandcompliancebetweentheVIMandtheunderlyingcomputeinfrastructure.Theinterfaceisexposedbymanagementfunctionsofthecomputenode and allows both control and monitoring of the compute domain. With regard tomonitoring,agentsareinstalledbothatthehostOS(tomeasurephysicalresources)aswellas(optionally)attheguestOSs(tomeasurevirtualisedresourcesassociatedwithaspecificVNFC).ThelattermetricsareofparticularinteresttoT-NOVAoperations,sincetheyprovideanindicationoftheresourcesconsumedbyaVNFCinstanceandaredirectlyusedforservicemonitoring. Heterogeneous compute resources need to provide additional methods toextracttheappropriatemetricsfromthedevicesandtosendthemviatheT-Nf-Viinterfaces.Application performance such as SLA compliance depends on a variety metrics such as


©T-NOVAConsortium

47

resource and system level metrics. The ability to measure application performance andconsumption of resources plays a critical role in operational activities such as customerbilling. Furthermore, statistical processing of VNFCmetrics will be exploited to indicate aparticular malfunction. Metrics to be collected at both physical compute node and VMinclude CPU utilisation,memory utilisation, network interface utilisation, processed trafficbandwidth,numberofprocesses,etc.

3.6.1.2. HypervisorDomain

Thehypervisordomain isthepartoftheT-NOVAarchitecturethatprovidesthevirtualisedcompute environment to VNFs. Since the hypervisor domain embraces different types ofhosts, with different Guest OSs and/or hypervisors, it is important to manageinteroperability issues appropriately. Issues relating to the virtualisation of VNFs ontechnologiesfromdifferentvendorsneedtobecarefullyconsidered.

The primary goal of the hypervisor domain is therefore to manage the heterogeneity oftechnologiesfromdifferentvendors,thusprovidinganinteroperablecloudenvironmenttotheVNFs. In that sense, thehypervisordomainprovidesanabstraction layerbetween theVIM (which controls,monitors and administrates the cloud) and the VNFs resources. ThehighlevelarchitectureofthehypervisordomainisshowninFigure3-3.

Looking at a single host, the hypervisor module provides virtual resources by emulatingdifferentcomponents,likeCPUs,NICs,memoryandstorage.ThisemulationcanbeextendedtoincludecompletetranslationofCPUinstructionssets;sothattheVMbelievesitisrunningonacompletelydifferenthardwarearchitecturewithrespecttotheoneitisactuallyrunningon. The environment provided by the hypervisor is functionally equivalent to the originalmachine environment. This emulation is carried out in cooperation with the Computedomain. The hypervisor module manages slicing and allocation of the local hardwareresourcestothehostedVMs.

It also provides the NICs to the VMs and connects them to a virtual switch in order tosupportboth internalandexternalVMcommunication.Asuitablememoryaccessdriver isintegratedintothevirtualswitchthatinterconnectsVMswitheachother.

Thevirtualswitches(vSwitches)arealsomanagedbythehypervisor,whichcan instantiateand configure one or more vSwitches for each host. They are logical switching fabricsreproducedinsoftware.

TheintegrationofvSwitchesandhypervisorsisanareaofspecificfocusduetoitssignificantinfluenceontheperformanceandonthereliabilityofVMs,especially inthecaseofVNFs.Thevariousaspectsandissuesrelatedtothe integrationofvSwitcheswithhypervisorsarediscussedinSOTAreviewinsection4.2.4.


©T-NOVAConsortium

48

Figure3-3:Hypervisordomainarchitecture

TheControl,AdministrationandMonitoringmoduleisessentiallyresponsibleforthecontrolfunctionalityforallthehostswithinthecloudenvironment,abstractingthewholecloud.Themaingoalofthismodule istoprovideacommonandstable layerthatwillbeusedbytheVIM to monitor, control and administer the VNFs over the T-NOVA cloud. This supportsvarious operations on the VMs (or VNFs) such as provisioning, creation, modification ofstate,monitoring,migrationandtermination.

Since different hypervisors provide different interfaces, the Control, Administration andMonitoring (CAM)module needs to support heterogeneous hypervisors,managing virtualresources on different vendors’ hypervisor at the same time. This particular task isaccomplishedbythehypervisormanager.

3.6.2. InfrastructureNetworkDomain

TheT-NOVAnetwork infrastructure comprehends thenetworkingdomainof theNFVI, i.e.thedifferentvirtualisednetworkresourcespopulatingtheNFVIasshowninFigure3-4.TheNetworkdomainwithintheNFVIconsidersvirtualresourcescomposingvirtualnetworksasthe functional entity. Those virtual networking resources are devoted to provideconnectivity to thedifferentvirtualisedcompute resources,whichhavebeenpresented inprevioussection.

The T-NOVA architecture, and thus the network domain controlled by the VIM, leveragesSDNforoptimisingnetworkoperations.Thenetworkdomainbuildsafullabstractionoftheactual resources included in the physical substrate and then creates vLinks between VMscomposingvNets,whichareprovisionedinordertosatisfytherequirementsofthedifferentVNFs.


©T-NOVAConsortium

49

ThemainobjectiveofthenetworkresourcesistoprovideconnectivitybetweenVMswhichcanrunonthesameserver,ondifferentserverswithinthesameDC,oronDC’sboundaries(inthelattercase,toprovideconnectivitywiththeWICMmoduleisrequired).WithintheT-NOVAIVM,thisisdirectlytranslatedintothecreationofvirtualnetworksthatinterconnectasetofcomputeresourceswhichareprovidingtheexecutionsubstratetoVNFs.

Virtual networks must be dynamically programmed in order to ensure network slicing,isolation, and ultimately connectivity in the T-NOVA multi-tenant scenario. Each vNet isdedicated toa specificVNFservice,andprovidesconnectivitybetween thedifferenthostsservingtheVNFservice.ElasticityofthevNetisstrictlyrequiredinordertoguaranteethatthecorrespondingVNFservicescanbeproperlyscaledinor–out.Avirtualisedcontrolplane(leveraging SDN concepts) will be responsible for controlling each one of these virtualnetworks.

The network domain exposes two basic interfaces: one to the VNF (T-Vn-Nf/VN), and theother to the basic VIM controller (T-Nf-Vi/N). A full description of the interfaces can befoundinTable3.2.

Figure3-4:HighlevelarchitectureoftheInfrastructureNetwork

3.7. VirtualisedInfrastructureManagement

WithintheT-NOVAIVMarchitecturetheVIMisthefunctionalentitythatisresponsibleforcontrolling and managing the NFVI compute, storage and network resources within theNFVI.TheVIMisgenerallyexpectedtooperateinoneoperator’sinfrastructuredomain(e.g.,NFVI-PoP).Howeverasmanyoperatorsnowhavedatacentresdistributedonaglobalbasis,scenarioswillarisewheretheVIMmayoperateinmorethanoneNFVI-PoPtosupporteitherthe architecture requirements for VNF services and/or operator business needs.AlternativelymultipleVIMsmayoperateacrossoperatordatacentresprovidingmultiNFVI-PoPs that canoperate independentlyor cooperativelyas requiredunder thecontrolofanOrchestrator.

While a VIM in general can potentially offer specialisation in handling certain NFVIresources, in the specific context of the T-NOVA system the VIM will handle multipleresourcestypesasshowninFigure3-5.TheVIMactsastheinterfacebetweentheT-NOVA


©T-NOVAConsortium

50

Orchestrator and the available IT-Infrastructure abstracted by the NFVI. The controlcomponentsareattheheartoftheVIM,encompassingthefollowingelements:

• Thealgorithmsand logic forcontrol,monitoringandconfigurationoftheirrelateddomain;

• An interface or API server to offer the implemented logic and collectedinformation’sinanabstractedwaytoothercomponents;

• Aninterfacetocontrolandaccessthevirtualisedinfrastructure.

Theinterfacesfromthecontrolcomponent intheVIMareaggregatedintheOrchestratorAgent and the VNFManager Agent functions to deliver a unified interface to the upperlayers of the T-NOVA components via the T-Vi-Vnfm and T-Or-Vi interfaces. Thisarchitecture allows interaction with the upper layers with a higher level of abstractiongiving theT-NOVAOrchestrator the layerswithahigher levelof abstractiongiving theT-NOVAorchestrator the flexibility, toconfigureaparticularpartof the infrastructureor tocollect infrastructurerelateddata.TheVIMalsoexposessouthboundinterfaces(asshownin Figure 3-5) to the infrastructure resources (Hypervisor/Compute/Network) of theNFVIwhich enable control andmanagement of these resources. A summary of the key northboundVIMinterfacescanbefoundinTable3.2.

Figure3-5:T-NOVAVIMhighlevelarchitecture

The following are the key set functions identified that must performed by T-NOVA VIMbasedongeneral requirements identifiedbyETSI foraVIMwithin theMANOarchitecture[9].

• Resourcemanagement,

• Orchestrating the allocation/upgrade/release/reclamation of NFVI resources, andmanaging the association of the virtualised resources to the physical compute,storage,networkingresources,


©T-NOVAConsortium

51

• Supporting the management of VNF Forwarding Graphs (create, query, update,delete), e.g., by creating andmaintaining Virtual Links, virtual networks, sub-nets,andports,

• Management of the NFVI capacity/inventory of virtualised hardware resources(compute,storage,networking)andsoftwareresources(e.g.,hypervisors),

• Management of VM software images (add, delete, update, query, copy) asrequestedbyotherT-NOVAfunctionalblocks(e.g.,NFVO),

• Collection and forwarding of performance measurements and faults/eventsinformation relative to virtualised resources via the northbound interface to theOrchestrator(T-Or-Vi),

• ManagementofcataloguesofvirtualisedresourcesthatcanbeconsumedfromtheNFVI. The elements in the catalogue may be in the form of virtualised resourceconfigurations (virtual CPU configurations, types of network connectivity (e.g., L2,L3),etc.),and/ortemplates(e.g.,avirtualmachinewith2virtualCPUsand2GBofvirtualmemory).

ThehighlevelarchitecturefortheT-NOVAVIMarchitecturereflectsthesekeyfunctions.

3.7.1. ITResourceManagementandMonitoring

In the T-NOVA system we distinguish between IT and virtualised network resources Alsofrom a management and control perspective this categorisation is clearly visible in thearchitectural design of the VIM. In fact, on the one hand, the VIM provides to theOrchestration layer a unified access point to all infrastructural resources at an abstractedlevel,but,ontheotherhand,internallythecontrolmodulesareexplicitlysplitintoComputeand Hypervisor Controlmanaging the IT resources, whereas the Network Controlmodulemanages the network resources. This architectural choice allows the T-NOVA system tomanagethemaccordingtodifferentrequirementsandneeds(intermsofperformance,timeconstraints,andsoforth)Thefollowingsubsectionsdiscusstherespectivemanagementandcontrol needs of the hypervisor and compute resources within the VIM and theirrelationshiptotheNFVI.

3.7.1.1. HypervisorManagement

The hypervisor control function is responsible for providing high level control andconfiguration capability that is independent of the technology implementationwithin thehypervisor domain. The interface to the hypervisor domain (T-Nf-Vi/H) provides thenecessary levelofabstraction to thespecificsof theunderlyinghypervisor.Thehypervisorcontrol component provides the basic commands like start, stop, reboot etc. and offersthese commands through an API server to the VIM. Specific commands related to aparticularhypervisorimplementationmayalsobesupportedoncasebycasebasisthroughthe same API server allowing finer performance tuning. Additionally the hypervisorcontrollercanimplementaqueryAPIthatcanbeusedtoprovidedetailedinformationsuchasconfigurationdetails,versionnumbersetc.

ThenetworkconfigurationofanewlycreatedVMisusuallyconfiguredbyascriptthatrunsat boot-time or can be donemanually via a CLI after the VMhas booted. The hypervisorcontrolcomponentwillofferthecapabilitytoimplementanetworkconfigurationduringVMinstantiation thus enabling a higher degree of automation which is important for serviceprovider operations. The hypervisor controller is able to configure the VM, provided thehypervisorsupportstheaction,thedesirednetworksettingsbeforethefirstbootoftheOS.


©T-NOVAConsortium

52

Thereliabilitycapabilitiesinsidethehypervisorcontrollerhaveanimportantroleascustomconfigurations can be requested by the T-NOVA Orchestrator. Their primary role is toprevent the user of the API from placing the hypervisor in an inconsistent or error stateleavingitunabletomanageorrespondtotheVMsunderitscontrol.Themodulemayalsoplay a role in managing issues such as misconfiguration, compromised commands orhypervisoroptionsthatdonotworkwellwiththeallocatedhardware.

Additionally,thehypervisorprovidesVMresourcemetricstobeusedforservicemonitoring,inadditiontothemetricscollectedatthecomputedomain(seenextsection).

3.7.1.2. ComputingResourcesManagement

A key functionality of the compute domain management will be the collection andprocessingofthemonitoringmetricscollectedbyboththephysicalcomputenodesaswellas the VMs (VNFCs). A dedicated monitoring manager is envisaged, to which monitoringagentswillconnecttocommunicatecomputedomainresourcemetrics’measuredvalues.Atthemonitoringmanager,thesemetricswillundergostatisticalprocessinginordertoextractadditional informationsuchasfluctuation,distributionandcorrelation.ThisprocessingwillprovideusefulinformationaboutthebehaviourofeachVNFcomponentandwillcontributetowardstheearlydetectionofpossibleVNFCmalfunctionsorperformanceissues.Thiswillbethecasee.g. ifsomemeasurements falloutsidethenormalVNF loadcurve(e.g. ifCPUutilisationrisesabnormallyeventhoughprocessednetworktrafficvolumedoesnot).

TheplacementofaVMonacomputeresourceisacriticaltaskandthecomputecontrollercarries out this function using a placement scheduler. This scheduler will, if no furtheroptions are specified, make a decision based on the requirements of the infrastructureprovider and place the VM in an automated fashion. To influence this decision makingprocess,theschedulerwillhaveafilterAPIthatcanbeused.DetailsrelatedtothedesiredSLAor specifichardwareandsoftware requirementscanbepassed to the filter.Themainfiltercategoriesthatinfluencethedecisionprocessbytheschedulerare:

• SpecifichardwarerequirementslikeCPUspeedortypeofdisk,

• SpecificsoftwarerequirementslikethehostOSorthehypervisor,

• Current load and performancemetrics of the compute node such as average CPUutilisationrateetc.,

• Financial considerations such as newest hardware or most energy efficienthardware.

AfurthercoretaskofthecomputecontrolleristoprovidespecificmanagementcapabilitiesoftheVMsfortheOrchestratorespeciallywheretheoperationsoverlapwithhypervisorandnetworkcontrolleractions.Suchtasksinclude:

• CreationanddeletionofaVM,

• Rebuilding,suspendandpauseaVM,

• MigratingaVMfromonecomputenodetoanothercomputenode,

• ResizingaVM.

CreationofaVMrequirescloseinteractionwiththebaseimagerepositorythatcontainsthebasicunmodifiedOSimages.

Thecapabilitytoenableinfrastructureadministratorstocarryoutroutinemaintenanceandadministration tasks on compute nodes is required. For example, the administratorsmay


©T-NOVAConsortium

53

needtomigrateVMsonaphysicalcomputenodetoanotheroneaspartofaninfrastructureupgrade activity. Other potential action may include interaction with the scheduler andfiltersinordertomodifytheconfigurationofaVMorsetofVMstomaintainanassociatedSLA for a VNF service running on the VMs. These types of functionalities can also beimplementedinanautomatedmanner inordertosupportnotonlymanagementactivitiesbutalsoautomaticSLAfulfilment.

3.7.2. InfrastructureNetworkResourcesManagementandMonitoring

The Network Control functional block within the VIM is responsible for configuring andmanaging the SDN-compatible network elements to provide an abstracted platform forrunningSDNapplications(i.e.networkvirtualisation,loadbalancing,accesscontroletc.).

In order tomeet specific VNF services’ requirements, the network elements, physical andvirtual, need to be properly programmed by the Network Control function to ensureappropriatenetworkslicing,isolationandconnectivityinamulti-tenantenvironment.Inthisway, the network will be properly partitioned and shared among several vNets, eachdedicatedtoaspecificVNFservice.IntheT-NOVAarchitecturethereisanexplicitdistinctionbetweenvirtualandphysicalnetworkcontroldomains:thevirtualnetworkcontroldomainismanaged by the VIM, whereas the physical network control domain is managed by theWICMentity(discussedinnextsection).

ThevirtualisationoftheNetworkControlisintendedtoaddressscalabilityandcentralisationissues affecting the SDN control plane in large network infrastructures. The proposedapproach considers a cluster of controllers to manage the network elements, using adistributed data store to maintain a global view of the network. By exploiting cloud-computing capabilities, the cluster can easily scale through the deployment of new CPinstancesonVMs,asthedemandsontheSDNcontrolplanegrow. Inthisregard,theSDNControlPlanecanofferelasticity,auto-scalingandcomputationalloadbalancing.

InordertoguaranteeefficientCPvirtualisation,aSDNControlPlaneCoordinatorisexpectedtomanageandmonitoreachCPinstance,balancingthedistributionoftheoverallworkloadtomultipleCPinstances.

Figure3-6:VIMNetworkControlArchitecture

TheNetworkControlcomponentinterfacesinternallywiththeVIMHypervisorandComputecomponents and externally with the Orchestrator layer, by accepting requests to deployvNetsbasedoncertaintopologyandQoSrequirements.


©T-NOVAConsortium

54

3.8. WANInfrastructureConnectionManager

Bydefinition,aVNFissupposedtoreplicatethebehaviorofanetworkfunctionhostedinaphysical appliance (e.g. firewall, DPI, etc.). In most practical scenarios, a VNF will be acomponentofawidernetworkenvironment,often includingaWideAreaNetwork (WAN)connectivity service, which may range from a simple Internet access to some form ofenterpriseconnectivity service, suchasan IPVPNor L2connectivity servicevariant,e.g. E-Line,orE-LAN3.Thus,theT-NOVAservicecanbedecomposedintwobasicparts:

• VNFas-a-Service(VNFaaS),asetofassociatedVNFshostedinoneormultipleNFVI-PoP.

• Aconnectivityservice,inmostcasesincludingoneormultipleWANdomains.

Withregardtothelattercomponent,althoughtheWANconnectivityserviceitselfisbeyondthescopeofT-NOVA,the integrationofWANconnectivityservicewiththeNFVI-PoPsthathosttheVNFsissomethingthatcannotbeneglectedandthereforemustbeanintegralpartof the T-NOVA architecture. TheWAN Infrastructure ConnectionManager (WICM4) is thearchitecturalcomponentthatissupposedtocopewiththisrequirement.

Figure3-7showsthetransitionfromapureWANconnectivityservicetoaVNFaaS-enabledscenario, inwhichthebasicconnectivityserviceisenrichedwithoneormultipleVNFsthatbecomeintegratedinthetrafficend-to-endpath.

Figure3-7-End-to-endcustomerdatapathwithoutVNFsandwithVNFs

AnimportantaspecttobetakenintoaccountinthiscontextistheconceptofVNFlocation.Infact,twotypesofVNFlocationcanbeconsidered5:

3InT-NOVA,theroleplayedbytheServiceProvider(SP)issupposedtoincludetheprovisionofVNFs,aswellastheunderlyingCloudandnetworkinfrastructure.Therefore,foragivencustomer,whenthefirstVNFisdeployed,theconnectivityservice(e.g.VPN)isexpectedtobealreadyupandrunning.InthecaseswheretheSProleisplayedbyanetworkoperatorprovidingVPNservices,VNFaaSmaybeseenasanadd-ontoatraditionalVPNserviceoffering.4 ETSINFVdefinesa fairly similar component calledWIM (WAN InfrastructureManager)but so farvery littlehasbeenspecifiedabout respective functionalityand interfaces.WhetherT-NOVAWICMandETSINFVWIMwillbefunctionallyequivalentisnotclearatthisstage.5 Inprinciple,physicalandlogical locationsshouldbeascloseaspossibletominimizetheimpactofVNF deployment on existingWAN network services – ideally, one NFVI-PoP close to each serviceprovideredgenode;howeverthismaynotbepossibleinpractice.Ultimately,therewillbeatrade-offbetweentheinstantiationofaNFVI-PoPateveryserviceprovideredgepoint(simplebutcostly)and


©T-NOVAConsortium

55

• ThelogicalVNFlocation,whichidentifiesthepointinthecustomernetworkwheretheserviceisinstalled(typically,correspondstoacustomerattachmentpointtotheserviceprovidernetwork);

• ThephysicalVNFlocation,whichidentifiestheactualplacementoftheVMsthatsupporttheVNF(NFVI-PoPIDandhosts).

AcomposedT-NOVAserviceinstanceincludesmultipleVNFsthatmayhaveoneormultiplephysicallocations(i.e.differentNFVI-PoPs),butshouldbeseenbythecustomerasasinglelogicallocation.

In Figure 3-8, the WAN connectivity service is represented by the central cloud and isdelimitedbyedgenodes.WithregardtotheconnectivitybetweencustomersitesandNFVI-PoPs,twocasesshouldbedistinguished:

• CaseA:TheconnectionbetweenVNFlogicalandphysicallocationsdoesnotcrossanyWANsegment–e.g.CustomersiteA-NFVIPoP1.ThismeansthattheinsertionoftheVNFintheend-to-endpathdoesnotimplyanyimpactontheexistingWANservice.

• CaseB:TheconnectionbetweenVNFlogicalandphysicallocationscrossesatleastoneWANsegment.Inthiscase,thecreationorremovalofaVNFmayrequirethemodificationofthetrafficpathintheWAN.

The NFVI GW represents the demarcation point between the NFVI-PoP and the externalnetwork infrastructure andplays a key role in this scenario – interworking betweenWANand internalNFVIprotocols.BoththeWANandtheNFVI-PoParemulti-tenantdomainsbydefinition, but aremanaged and controlled independently of each other, even if they areadministeredbythesameentity.Thismeans, forexample, thatmappingbetween internalandexternalVLANIDs(orwhateveraggregationmethodologyisusedoneithersideoftheNFVIGW) is needed to guaranteeend-to-end connectivity.At theNFVI-GW traffic comingfromtheWANisde-aggregatedandmappedtotherespectivevirtualnetworkintheNFVI-PoP.

3.9. WICM:ConnectingVNFstotheWAN

Figure 3-8 illustrates the positioning of the WICM in the T-NOVA architecture and therelationshipwithotherT-NOVAarchitecturalcomponents.TheWICMisresponsiblefortheintegrationofVNFsandWANconnectivityservicesandtakeanyactionsrequiredbyeventsrelated to the customer subscription status (e.g. new VNF subscribed by the customermodifiesend-to-endtrafficpath).

the centralization in a small number of NFVI-PoPs (economical but with strong impact on existingWANservices).ThisdiscussionisbeyondthescopeofT-NOVA.


©T-NOVAConsortium

56

Figure3-8-WICMoverallvision

TheroleplayedbytheWICMdependsonwhichofthecasesAorB,definedabove,apply.Incase A, the WICM is in charge of controlling network nodes at the access/aggregationnetworksegment,whereasincaseBtheWICMshouldalsocontrolthenetworknodes(e.g.MPLS PEs) located deeper in the service provider network. In T-NOVA, Case A should beconsideredaspreferredforimplementationanddemonstrationpurposes.

3.9.1.1. SDN-enabledNetworkElements

Today’sWANsarebecomingmorecomplexandtheintroductionofSDNalignedapproachesholds great potential formanaging andmonitoring network devices and the trafficwhichflowsoverthem.Thebenefitsaremainlyrelatedtoautomatednetworkprovisioningandtotheflexibilityinlinkdeploymentbetweendifferentdatacentres.

FromaT-NOVAperspective,theWICMhasresponsibilityforcontrollingandmonitoringthephysical network devices which are SDN-enabled, with the purpose of providing WANconnectivitybetweendifferentNFVI-PoPswhileconsideringbothSLAsrequiredbyVNFsandefficient management of cloud infrastructure resources. The WICM and VIM NetworkControlmoduleswillneedtobecoordinatedbytheOrchestratorinanappropriatemanner,inordertosetupVLANsamongdifferentvirtualandphysicalSDN-enableddevices.

Although for the intra-DC networking the deployment of SDN technologies is becomingmorecommonplace,theadoptionofSDNatthetransportlevelandespeciallyatLayer0to1, is still very much in its infancy [19] [20]. When considering the transport layers weencompasstechnologiesatlayer0(DWDM,photonics)andlayer1(SONET/SDHandOTN).AkeyreasonforthedelayinSDNadoptioninthetransportnetworkistheon-goingtransitionfrom the analog to digital domain. Themechanisms for dealing with analog attributes inopticalnetworksare vendor specific, and it isnotpossible for a generic controller todealwith thecurrentmyriadofvendorspecific implementations.Nor is itpossible fornetworkoperatorstoremovealloftheirtransportequipmentfromtheirnetworksandreplaceitwitha standardised optical hardware based around open standards. However, at higher layers(i.e.WAN)theadoptionpathispotentiallymoreexpeditious.CompanieslikeGoogleandtheCarrierEthernet(MEF)andcloudbusinessunitsofnetworkoperatorshavealreadyadoptedSDNsolutionsfortheirWANs[21].

ForthepurposeoftheT-NOVAproof-of-conceptdemonstrationtheWICMdevelopmentofSDNcompatibledevicesisoutofscopeasitisnotpartoftheobjectivesoftheprojectwhich


©T-NOVAConsortium

57

are mostly focusing on the NFVI-PoP network management and control. However, at anarchitecturallevelitisthoroughlysupportedtoensureappropriatefutureproofing.

3.9.1.2. LegacyNetworkElements

Managing and monitoring legacy network domains is a well understood and maturecapability.A varietyof standardisedandproprietary frameworkshavebeenproposedandimplementedintheformofcommercialandopensourcesolutions.

The WICM is the T-NOVA architectural component in charge of enabling interoperabilitybetweenNFVI-PoPandWANconnectivityservices, including legacyservicemodelssuchasorMPLS-basedIP-VPN,orCarrierEthernetE-LAN,E-Line.

Asnotedabove, the integrationofNFVI-PoPs in legacyWANservices canbemoreor lesscomplex,dependingonfactorssuchasthelogicallocationoftheNFVI-PoP,namelywhetherthe connection between logical and physical locations of the VNFs crosses any WANsegment.

As outlined in the SOTA review (section 4.3) the standard method for allowing networkoverlaysoverlegacynetworkdomainistheexploitationofL2toL3tunnellingmechanisms.Thosemechanisms introduce the use of tunnelling protocols that allow themanagementaspects of each tunnel on an end-to-end basis. Themost interesting tunnelling protocols,from a T-NOVA architecture perspective, are VxLAN, and Generic Routing Encapsulation(GRE). In short these protocols allow the interconnection of NFVI-PoPs over L3 legacynetwork by encapsulating the NFVI network traffic (actually DC traffic) end-to-end. Theyrequire the setup of an end-point for each connected DC, which is responsible forencapsulationanddecapsulationofpackets.

At a WAN level the architecture design of T-NOVA supports any type of legacy WANtechnology (i.e.MPLS,Optical,CarrierEthernetetc.)orSDNcompatible,providedthat theappropriateinterfacesaredeveloped.Inthiscontext,andforthesakeofdemonstratingtheUCasdiscussed inD2.1, T-NOVAwill exploit an IP/MPLS transportnetworkandprovideasimpleimplementationofWICMinordertosupportprovisioningofvNETsinanend-to-endmanner.

Fromamonitoringperspective,thereareanumberofframeworksrangingfrompassivetoactive and hybrid that allow the monitoring of legacy networks [22] [23]. T-NOVA willemploy such mechanisms in order to monitor adequately the status of the network andmore importantly the statusand resourceusageof theestablished tunnels.Monitoringofthe transport network resourceswill also be part of the consideredWICM functionalitiesthatwill be implemented.Moreover, themonitoring informationwill be conveyed to theNFVOasan input in themappingandpathcomputationmechanism.HistoricalmonitoringdatacollectionwillalsobecollectedtofacilitatetrackingofSLAbreaches.


©T-NOVAConsortium

58

4 T-NOVAVNFSANDNSSPROCEDURES

ThissectionillustratesthesetofmostcommonproceduresassociatedwiththedeploymentandmanagementofVNFsandNSs(e.g.on-boarding,instantiation,monitoring,scaling,etc.)to illustrate the interactions between the T-NOVAOrchestrator and IVM architecture FEsthathavebeendescribed inSections2and3.The flowdiagramspresented in this sectionserveasameanstovalidatethearchitecturesfortheT-NOVAOrchestratorandIVMlayersandtheirconstituentarchitecturalcomponents.Inaddition,theflowdiagramsalsoillustratethe interactionsat theFE level andvalidate thepurposeand capabilitiesof the interfacesthathavebeen identified insections2and3.Asstatedabove, thesequencediagramsareintendedtoillustratespecificallytheinteractionoftheentitiesoftheOrchestrationandtheIVM layers, however some of the details of the internal actions of each module are notillustratedastheseareimplementationspecificanddependonthetechnologiesutilised.

While a conceptual exercise, the description and illustration of the key VNF and NSdeployment and management workflows provide a means to stress tests regarding thetaken architectural decisions and capture necessary refinements prior to implementationrelated activities in WP3/4. Subsection 4.1 focuses on VNF related procedures, whereassubsection4.2iscentredonNSrelatedprocedures.

4.1 VNFrelatedprocedures

TodescribetheVNFrelatedprocedures,thefollowingassumptionsaremade:

• TheVNFiscomposedbyoneormoreVNFCs;

• EachVNFChasadedicatedVM;

• VNFCsareinterconnectedthroughVirtualNetworkLinks;

• TheVNF,aswellastheconstituentVNFCs,isinstantiatedwithinasingledatacentre.

Depending on which kind of network service the VNF provides (e.g., a firewall must beconnectedtospecificnetworkaddresses),thedeploymentofaVNFinstancemayalsoimplyaninteractionwiththeWICM.

The VNF details (e.g. deployment rules, scaling policies, and performance metrics) aredescribedintheVNFDescriptor.

4.1.1 On-boarding

VNF on-boarding (Figure 4-1) refers to the process of making the T-NOVA OrchestratorawarethatanewVNFisavailableontheNFStore.


©T-NOVAConsortium

59

Figure4-1:VNFOn-boardingProcedure

Steps:

1. UploadVNFmetadata

2. TheNFVOprocessestheVNFDtocheckifthemandatoryelementsareprovided.

3. TheNFVOuploadstheVNFDtotheVNFCatalogue.

4.1.2 Instantiation

VNF instantiation (Figure 4-2 and Figure 4-3) refers to the process of creating andprovisioning a VNF instance. Figure 4-2 refers to the instantiation process from theperspectiveoftheOrchestrationLayer,whereasFigure4-3showstheinstantiationprocessfromtheIVMlayerpointofview.


©T-NOVAConsortium

60

Figure4-2:VNFInstantiationProcedure(Orchestrator’sView)

StepsVNFInstantiation–Orchestrator’sView:

1. NFVOcallstheVNFMtoinstantiatetheVNF,withtheinstantiationdata.

2. TheVNFMvalidatestherequestandprocessesit.

3. TheNFVOperformsresourcemapping

4. TheNFVOrequeststheallocationofresourcesfromtheVIM(compute,storageandnetwork)neededfortheVNFinstance.

5. TheVIMrequeststheVNFimagefromtheNFStore

6. TheNFStorereturnstherequestedimagetotheVIM

7. The VIM instantiates the required compute and storage resources from theinfrastructure,forfurtherdetailsseeVNFInstantiationProcedure–IVMView.

8. The VIM instantiates the internal connectivity network – a VNF may requirededicatedvirtualnetworksto interconnect it’sVNFCs(networksthatareonlyusedinternallytotheVNFinstance),forfurtherdetailsseeVNFInstantiationProcedure–IVM’sView.

9. The VIM interconnects the instantiated internal connectivity network with theVNFCs,forfurtherdetailsseeVNFInstantiationProcedure–IVMView.


©T-NOVAConsortium

61

10. AcknowledgementofcompletionofresourceallocationbacktoNFVO.

11. TheNFVOacknowledges thecompletionof the resourceallocationback toVNFM,returningappropriateconfigurationinformation.

12. After theVNF is instantiated, theVNFMconfigures theVNFwith anyVNF specificlifecycleparameters(deploymentparameters).

13. TheVNFsendsanacknowledgementtotheVNFMthattheconfigurationprocessiscompleted.

14. TheVNFMacknowledgesthecompletionoftheVNFinstantiationbacktotheNFVO.

Thediagramcorrespondingtosteps8-10isindicatedbelow.

Figure4-3:VNFInstantiationProcedure(IVM’sView)

Thespecificsdetailsofsteps8-10areasfollows:

8.1. The VIM Orchestrator Service submits a request to the VIM Compute ControlmoduletocreatenewVMs,accordingtotheVNFrequirements.

8.2. TheVIMComputeControlmodule:


©T-NOVAConsortium

62

• Processestherequest;

• Analysestherequiredconfiguration;

• SelectsoneormoresuitablecomputenodestohosttheVMs;

• SendsrequesttoallocateVMstotheselectednodes.

8.3. TheselectedNFVIComputenode(s)allocatestheVMs.

8.4. The NFVI Compute nodes send back an acknowledgment to the VIM ComputeControlmodulewhentheysuccessfullyboot.

8.5. The VIM Compute Controlmodule sends back an acknowledgment to the VIMOrchestratorAgent.

9.1. The VIM Orchestrator Agent submits a request to the VIM Network ControlmodulefortheallocationofNetworkResources.

9.2. TheVIMNetworkControlmodule:

• Processestherequest;

• Analysestherequiredconfiguration;

• Sends a request for allocation of virtual network resources on the NFVINetwork.

9.3. TheNFVINetwork:

• Allocatestheresources;

• Sets up the virtual networks, by configuring the required interconnectivitybetweenvirtualswitches.

9.4. TheNFVINetworksendsbackanacknowledgment to theVIMNetworkControlmodule.

9.5. The VIM Network Control module sends back an acknowledgment to the VIMOrchestratorAgent.

10.1. The VIM Orchestrator Agent submits a request to the VIM Hypervisor ControlmoduletoattachthenewVMstotherequiredvirtualnetworks.

10.2. The VIM Hypervisor Control module sends the request to the hypervisorscontrollingthenewVMhostingnodes.

10.3. Thehypervisorsofthosenodes:

• Configure the vSwitches in order to manage the VLANs connectivitynecessarytosupporttheVNFrequirements;

• SetuptheinterconnectionsamongVMsandvSwitches.

10.4. The hypervisors send back acknowledgments to the VIM Hypervisor Controlmodule.

10.5. TheVIMHypervisorControlmodulesendsbackanacknowledgmenttotheVIMOrchestratorAgent.


©T-NOVAConsortium

63

4.1.3 Monitoring

VNFmonitoringfromtheOrchestrator’slayerpointofview(Figure4-4)referstotheprocessof checking if the values from a set of predefined parameters of the VNF are at withindefinedlimits,includingtheinfrastructurespecificparameterscollectedandreportedbytheIVM,aswellastheVNFapplication/service-specificparameters.

VNF monitoring from the IVM’s layer point of view (Figure 4-5) refers to the process ofmonitoringoftheVIM,includingtheVMperformancemetrics,theVLperformancemetrics,and the physicalmachines performancemetrics. From anOrchestration layer perspectivethesequenceisasfollows:

Figure4-4:VNFMonitoringProcedure(Orchestrator’sView)

Steps(VNFMonitoring–Orchestrator’sView):

1. VNFcollects/calculatesmetricsrelatedwiththeVNFapplication/service.

2. VNFnotifiestheVNFMwiththeVNFapplicationspecificmetrics

3. VIM collects performancemetrics relatedwith the infrastructure allocated for theVNF,forfurtherdetailsseeVNFSupervisionProcedure–IVMView.

4. VIMnotifiestheVNFMwiththeVNFinfrastructurerelatedperformancemetrics.

As far as themonitoring procedure from the IVM’s layer point of view is concerned, thesequenceisasfollows:


©T-NOVAConsortium

64

Figure4-5:VNFSupervisionProcedure(IVM’sView)

Steps(VNFSupervision–IVM’sView):

1. TheNFVIHypervisors:

a. CollectVMperformancemetricsdata;

b. SendthedatatotheVIMHypervisorControlmodule.

2. TheVIMHypervisorControlmodulesendsthedatatotheVIMVNFManagerAgent.

3. TheNFVINetworkdevices:

a. Collectperformancemetricsdatafromvirtualnetworklinks(VLs);

b. SendthedatatotheVIMNetworkControlmodule.

4. TheVIMNetworkControlmodulesendsthecollecteddatatotheVIMVNFManagerAgent.

5. TheNFVIComputenodes:

a. Collectthephysicalperformancemetricsdata;

b. SendthedatatotheVIMComputeControlmodule.

6. TheVIMComputeControlmodulesendsthedatatotheVIMManager.

4.1.4 Scale-out

VNFscale-outreferstotheprocessofcreatinganewVMtoincreasetheVNFcapacitywithadditionalcompute,storage,memoryandnetworkresources.

The scaling policies that indicate which/how/when VMs/VNFCs should be scaled areidentified in the ”VNF Deployment Flavour” attribute, which makes part of the VNFDescriptor(VNFD).

Figure 4-6 illustrates this case for a scaling-out: VNFC A is instantiated and a new VL iscreatedtoconnectthisnewinstancetotheexistingVNFCBinstance.VNFscalingisfurtherdetailedonanotherdeliverableofthisproject,D2.42[5].


©T-NOVAConsortium

65

Figure4-6:ScalingoutaVNF

Figure 4-6 illustrates the VNF scale-out process. Note that VNF scaling (out or in) mustalwaysbetriggeredattheNFVOlevel,sincetheneedtoscaleisdeterminedbymonitoringoftheNS,andnotonlyaparticularVNF.ScalingaNSisacomplexdecisionthatmayormaynotinvolvescalingtheVNFinstancesthatarepartofthatparticularNSinstancethatneedsto be scaled (e.g., the bottleneck might be in the network, and not in any of the VNFinstances)

4.1.5 Termination

VNFterminationFigure4-7andFigure4-8refertotheprocessofreleasingaVNFinstance,includingthenetworkandVMresourcesallocatedtoit.Figure4-7referstotheterminationprocess from the perspective of the Orchestrator’s layer, whereas Figure 4-8 shows theterminationprocessfromtheIVMinternalpointofview.


©T-NOVAConsortium

66

Figure4-7VNFTerminationProcedure–Orchestrator’sView

Steps(VNFTerminationOrchestrator’sView):

1. The NFVO calls the VNFM to terminate the VNF service. The VNF terminationprocedurecanbetriggered,forexample,bythefollowingactions:

• TerminationoftheNSinwhichtheVNFisinstantiated;

• Scale-inoftheNS,requestingaspecificVNFinstancetobeterminated;

• ExplicitrequestfromtheSPortheFPtoremovetheVNF.

2. TheVNFM gracefully shuts down the VNF, i.e.without interrupting theNS that isbeingdelivered, if necessary in coordinationwithothermanagemententities. TheVNFimage(s)willbemaintainedontheNFStore(inordertobeinstantiatedagaininthefuture).TheVNFcatalogueisnotaffectedbytheVNFtermination.

3. TheVNFMacknowledgesthecompletionoftheVNFterminationbacktotheNFVO.

4. TheNFVOrequestsdeletionoftheVNFresourcesbytheVIM.

5. Virtualnetworklinks(VLs)interconnectingtheVMsarereleased,forfurtherdetailsseeVNFInstantiationProcedure–IVM’sView.

6. VMs resources (compute, storageandmemory)usedby theVNFare released, forfurtherdetailsseeVNFInstantiationProcedure–IVM’sView.

7. An acknowledgement is sent indicating the success or failure of resource releasebacktoNFVO.

• Theinfrastructureresourcesrepositoryisupdatedautomatically.


©T-NOVAConsortium

67

Figure4-8:VNFTerminationProcedure–IVM’sView

Steps(VNFTermination–IVM’sView):

5.1. TheVIMOrchestrationServicesubmitsa request to theVIMHypervisorControllerfordetachingVM(s)fromthevirtualnetwork;

5.2. TheVIMHypervisorControlmodule forwards the request to theNFVIHypervisorsinvolved;

5.3. TheNFVIHypervisorsdetachVM(s)fromthenetwork;5.4. The NFVI Hypervisors send back an Acknowledgement to the VIM Hypervisor

Control;5.5. The VIM Hypervisor Control sends back an Acknowledgement to the VIM

OrchestratorAgent;6.1. TheVIMOrchestratorAgent submits a request to theVIMNetworkController for

releasingvirtualnetworkresources;6.2. The VIM Network Control module forwards the request to the NFVI Network

domain;6.3. TheNFVINetworkInfrastructurereleasesthenetworkresourcesandsendsbackan

Ack;6.4. TheVIMNetworkControlsendsbackanAcktotheVIMOrchestratorAgent;6.5. TheVIMOrchestratorAgent submitsa request to theVIMComputeController for

releasingvirtualcomputeresources;6.6. The VIM Compute Control module forwards the request to the involved NFVI

Computenodes;6.7. TheNFVIComputenodesreleasesthecomputeresourcesandsendsbackanAck;6.8. TheVIMComputeControlsendsbackanAcktotheVIMOrchestratorAgent.


©T-NOVAConsortium

68

4.2 NSrelatedprocedures

TodescribetheNSprocedures,thefollowingassumptionsaremade:

• TheNSiscomposedbyoneormoreVNFs(inthefollowingprocedures,twoVNFs–VNF1andVNF2–composetheNS);

• VNFscomposing theNSare interconnected throughVirtualNetworkLinks (ifVNFsrunon the sameDC) or throughNon-virtual/legacyNetwork Links (if VNFs runondifferent DCs) (in the following procedures, VNF1 runs on DC1 and VNF2 runs onDC2);

• TheNSconstituentVNFscanbeimplementedinasingleDCorspreadacrossseveralDCs;

• BesidesVNFs,PNFscanalsobepartoftheNS(inthefollowingprocedures,PNF1isinterconnectedwithVNF1).

The NS details (e.g. deployment rules, scaling policies, performance metrics, etc.) aredescribedintheNSD,e.g.VNFForwardingGraphfordetailingtheVNFsinterconnections.

4.2.1 On-boarding

NS on-boarding (Figure 4-9) refers to the process of submitting a NSD to the NFVOrchestratorinordertobeincludedinthecatalogue.

Figure4-9:NSOn-boardingProcedure

Steps:

1. TheMarketplacesubmitstheNSDtotheNFVOforon-boardingtheNS.

2. NFVOprocessestheNSDtocheckifthemandatoryelementsareprovided.

3. NFVOnotifiesthecatalogueforinsertionoftheNSD.

4. NFVOacknowledgestheNSon-boarding.

4.2.2 Instantiation

NS instantiation refers to the instantiation of a new NS, i.e. Figure 4-10 from theOrchestrator’sview,andFigure4-11fromtheIVM’sview.

Asstatedabove,thenextsequencediagramdepictsasituationwhereVNFscomposingtheNSrunondifferentDCsandareinterconnectedthroughNon-virtual/legacyNetworkLinks.


©T-NOVAConsortium

69

Figure4-10:NSInstantiationProcedure(Orchestrator’sView)

Steps(OrchestratorView):

1. TheNFVOreceivesarequesttoinstantiateanewNS.

2. TheNFVO validates the request, both validity of request (including validating thatthesenderisauthorisedtoissuethisrequest)andconfirmingtheparameterspassedaretechnicallycorrect.

3. TheOrchestratorcheckstheNScomposition(e.g.VNFFG)intheNScatalogue:

4. TheNFVOexecutesany requiredpre-allocationprocessingwork, e.g.VNF locationselection, Resource pool selection, Dependency checking. For further details seeVNFInstantiationProcedure–Orchestrator’sView.

5. The NFVO requests the WICM to setup the WAN resources required forinterconnectingtheVNFsacrosstheDCs(resourcephaseestablishment).

6. TheWICMconfigurestheWANresourcesbetweenDC1andDC2.

7. TheWICMsendsanacknowledgmenttotheNFVOreportingthattheWANhasbeenconfiguredasrequested.

8. TheNFVOsendsarequesttotheVIMto interconnecttheWANingressandegressrouterstotheDCVLANs(connectivityphaseestablishment).

9. TheVIM interconnects theconfiguredWANresourceswithVNF1andVNF2 inDC1andDC2,respectively.


©T-NOVAConsortium

70

10. TheVIMacknowledgescompletionoftheWAN/VLANsconfiguration:

• If necessary, NFVO requests Network Manager to connect VNF externalinterfacestoPNFsinterfaces:

1. TheNetworkManagercanbeanOSS,anNMSoranEM;

2. ConnectiontoPNFsisassumedtobedonebytheNFVO.

11. TheNFVOacknowledgescompletionoftheNSinstantiation.

Figure4-11:NSInstantiationProcedure(IVM’View)

Steps(IVMView):

1. TheWICMOrchestratorAgentsendstherequesttotheWICMNetworkControltoconfiguretheWANendpoints.

2. TheWICMNetworkControlconfigurestheendpoints.

3. TheWICM Network Control sends an acknowledgement indicating success orfailureoftheconfigurationsetuptotheWICMOrchestratorAgent.

4. WhentheacknowledgementofasuccessfulWANconfiguration isreceivedtheNFVOsendsarequesttotheVIMOrchestratorAgenttoconnectaVLANtoWANendpoints.TheVIMOrchestratorAgentsendstherequesttoconnectaVLANtoWANendpointstotheVIMNetworkController.

5. The VIM Network Controller sends the request to connect a VLAN to WANendpointstotheNFVINetwork.

6. TheNFVINetworkconnectstheVLANtotheWANendpoints.


©T-NOVAConsortium

71

7. The NFVI endpoint sends an acknowledgement of a successful or failedconnectionconfigurationtotheVIMNetworkController.

8. TheVIMNetworkControllersendsanacknowledgementofasuccessfulorfailedconnectiontotheVIMOrchestrationAgent.

4.2.3 WANConnections

Theset-upofaWANconnectiontriggeredbyacustomersubscribingtoanewservicethatrequirestheWICMinterventionisshowninFigure4-12.ThesequenceofstepsthatrequiresWICMinterventionareasfollows:

Steps:

1. TheCustomerpurchasesaVNFservicethroughtheMarketplacedashboard,possiblyindicatingthelogicallocationoftheservice(e.g.aspecificedgepointofhisnetwork)

2. The T-NOVA orchestrator forwards the request toWICM in order to allocate theWAN/NFVI-PoPnetworkconnectivity.

3. Ifstep2issuccessful,theWICMreturnstheVLANIDtobeusedinconfigurationoftheVNFinstances.

4. The VIM instantiates the resources required to support the new VNFs. configurestheNFVI-GWtoappropriatelymapexternalandinternalVLANIDs.

5. If step 4 is completed successfully, the Orchestrator notifies the WICM and theWICMenforcescustomertraffictobedivertedtowardsthenewlyinstantiatedVNFs.

Figure4-12:WANConnectionProcedure

4.2.4 Supervision

NS supervision (Figure 4-13) refers to the monitoring of the NS performance metrics,including:

• VNFinfrastructureandservicespecificinformation;• NetworklinksinterconnectingtheVNF(acrossmultipleDCs).

Again, as stated above, the next sequence diagram depicts a situation where VNFscomposingtheNSrunondifferentDCsandare interconnectedthroughNon-virtual/legacyNetworkLinks.


©T-NOVAConsortium

72

Figure4-13:NSMonitoringProcedure

Steps:

1. TheVNFMsendsperformancemetricsrelatedtoaVNFtotheNFVO.This includesboth metrics from the infrastructure supporting the VNF (VMs, Virtual Links) –obtaineddirectlyfromtheVIM-aswellasapplication/service-specificmetricsfromtheVNF-obtaineddirectlyfromtheVNForfromtheEM).Forfurtherdetailsaboutthe VNF performance metrics retrieval, please check the “VNF SupervisionProcedure”,wheretheoptiontoforwardaggregatedinformationtoNFVOhasbeentakenbytheVNFM.

2. TheWICMfetchesanddeliverstheWANsegmentmetricstotheNFVO.

3. TheVIMdeliversVM-relatedmetricstotheNFVO:

• BasedonthemetricsreceivedandonthedefinedscalingpoliciesincludedintheNSD,theNFVOmaydecidetotriggerascalingprocedure.Furthermore,


©T-NOVAConsortium

73

ifconfigured,theNFVOwillalsodeliveraggregatedNS-relatedperformancemetricstotheMarketplace.

4.2.5 Scale-out

NS scale-out refers to the process of increasing the capacity of the service in order toaccomplishaSLAthatischangingtoanewNSdeploymentflavour,ortomaintainanexistingSLA.

Thescale-outpoliciesaretriggeredbasedonthefollowinginformation:

• NSissues(retrievedfromVNFM);• VNFissues(retrievedfromVNFM);• WANsegment,i.e.connectingVNFsissues,retrievedfromWICM.

Figure4-14illustratesNSscalingcase,whereasecondinstanceofVNFBiscreatedandtheexistingVNFAinstanceisreconfiguredsothatitbecomesabletocommunicatewiththetwoVNFBinstances.

Figure4-14:Scaling-outaNS


©T-NOVAConsortium

74

Figure 4-15 refers to a situation where, according to the metrics received, the requiredperformanceoftheSLAcannotbeachieved.AssuchtheNFVOdecidestoscale-outtoanewdeployment flavour, takingalso intoaccount theauto-scalingpolicies included in theNSD.Thisimplies:

• Scale-outaspecificVNFtoanewdeploymentflavour(includedinthisworkflow);

• CreationofanewVNFinstance(includedinthisworkflow);

• ChangingtheVNFlocationtoanotherDC(notincludedinthisworkflow);

• IncreasingtheWANsegmentnetworklinkcapacity(includedinthisworkflow).

Note • ScalingofNSwithintheT-NOVAsystemisstillawork inprogress.ThespecificsofwhichwillbefinalisedduringthecourseofWP7wherethefinalT-NOVAsolutionincludingscalingwillbeimplemented


©T-NOVAConsortium

75

Figure4-15:NSScale-out

Steps:

1. TheNFVO collects and checksmonitoring information from theVNF,VIMand theWAN.


©T-NOVAConsortium

76

2. Basedontheretrievedperformancemetricsandontheauto-scalingpoliciesdefinedontheNSD,theNFVOdetectstheneedtoscale-outtheNS.

3. TheNFVO decides to change to anotherNS deployment flavour,which comprisestheVNF1scale-out,anewinstantiationofVNF2#2andanincreaseontheWANlinkcapacity:

• The other procedures (VNF scale-out, VNF instantiation and WANinterconnection)havealreadybeendescribedinthepreviousworkflows.

4.2.6 Termination

NStermination(Figure4-16)referstotheprocessofreleasingtheNSinstance,includingtheconstituentVNFs(VNF1,VNF2#1-instance1ofVNF2andVNF2#2–instance2ofVNF2),aswellastheWANsegment.

Figure4-16:NSTerminationProcedure

4.3 NS,VNFandInfrastructureMonitoring

Proper NS/VNF as well as infrastructure monitoring is crucial for the implementation ofmany of the use cases foreseen for the T-NOVA system, see D2.1 [6]. Especially UC2(ProvisionNFVservices),UC3(Reconfigure/RescaleNFVservices)andUC5(BillNFVservices)use themonitoringmetrics,which are collected duringUC4 (Monitoring). Specifically, thelatterpresentsanoverviewoftheT-NOVAmonitoringprocedures,focusingonboth:

1. VNFandServiceMonitoring-relatedtothestatusandresourcesoftheprovisionedservices,aswellas,

2. Infrastructure Monitoring - related to the status and resources of the physicalinfrastructure.

Monitoringmetricsaremostly collectedat thevariousdomainsof the Infrastructure layerandcommunicatedtotheupper layers.Figure4-17showsahigh-levelviewof the flowofthemonitoringinformationacrosstheT-NOVAsystem.


©T-NOVAConsortium

77

Figure4-17:CommunicationofmonitoringinformationacrosstheT-NOVAsystem

The first step is the collectionofboth InfrastructureandNS/VNFmonitoringmetrics fromdifferentdomainswithintheT-NOVAInfrastructurelayer.Thesemetricsaretypicallyofferedby a dedicated monitoring agent at each physical or virtual infrastructure element. Inaddition, some compute node and VNF/VM metrics can be collected directly via thehypervisor,whichalsoprovideshostandguestmachineresourceinformation.

Table 4.1 presents an indicative list of monitoring metrics to be collected at eachinfrastructure domain. It must be noted that this list is tentative and is expected to bemodified/updated throughout the project course, as the specific metrics which will beactuallyneededforeachstepoftheT-NOVAservicelifecycle,willbepreciselydefined.

Table4.1:Monitoringmetricsperinfrastructuredomain

Domain VNFApp/ServiceMetrics InfrastructureMetrics

VNF(VM,guestmachine) CPUutilisationCPUtimeusedNo.ofvCPUsRAMallocatedDiskread/writebitrateNetworkinterfacein/out

-

VNF

Marketplace

Orchestrator

Virtualised InfrastructureManagementTransportNetwork

Management

Compute Storage

Infrastru-cture

Network

TransportNetwork

Hypervisor

ServiceandInfrastructureMonitoring

(VNFM/NFVO)

Accounting SLAManagement Dashboard

VNF

NS/VNFInstancesRecords

InfrastructureResourcesRecords

WICM


©T-NOVAConsortium

78

bitrateNo.ofprocesses

Compute(percomputenode,hostmachineoftheDCdomain) -

CPUutilisationRAMallocatedDiskread/writebitrateNetworkinterfacein/outbitrate

Storage(objectorvolumestorageoftheDCdomain)

Read/writebitrateVolumeusage(volumestorage)No.ofobjects(objectstorage)Totalobjectssize(objectstorage)

TotalRead/writebitrateTotalVolumeusage(volumestorage)Totalno.ofobjects(objectstorage)Totalobjectssize(objectstorage)

InfrastructureNetwork(pernetworkelementoftheDCdomain)

Per-flowpacketscumulativeandpersecondPer-flowbytespacketscumulativeandpersecondFlowduration

Per-portpacketscumulativeandpersecondPer-portbytespacketscumulativeandpersecondPer-portreceivedropsPer-porttransmitdropsPer-portlinkstateandspeedCPUutilisation

TransportNetwork(pernetworkelement)

Per-flowpacketscumulativeandpersecondPer-flowbytespacketscumulativeandpersecondFlowduration

Per-portpacketscumulativeandpersecondPer-portbytespacketscumulativeandpersecondPer-portreceivedropsPer-porttransmitdropsPer-portlinkstateandspeedCPUutilisation

Infrastructuremetrics,aswellasevents/alertsareaggregatedbytheVIMandtheWICMandcommunicatedtotheOrchestratorFEs,NFVOandVNFM,whichareinchargeofassociatingeach group of metrics to specific VNFs and Network Service. Those parameters are thencompared against the NS and VNF templates composed by a. o. the NSD and the VNFD,whichdenotetheexpectedNSandVNFbehaviour.Ifanymismatchisdetected,appropriateactionsareselectedandexecuted,e.g.scaling.

ThisprocedureneedstobecarriedoutattheOrchestratorlevel,sinceonlythelatterhastheentireviewoftheend-to-endserviceaswellastheresourcesallocatedtoit.Inthiscontext,and in addition to the double check (NS and VNF) mentioned above, the monitoringprocessed at the Orchestrator, as part of the NFVO and VNFM, performs the followingoperations:

• Aggregate infrastructure-related metrics and update the Infrastructure Resourcesrecords,

• Associates metrics (e.g. VM and flow metrics) to specific services, produce anintegrated picture of the deployed service and updates the NS/VNF instancesrecords,

• ChecksmonitoredparametersagainsttheNSandVNFtemplates,composedbytheNSD and the VNFD, which denote the expected NS and VNF behaviour. If anymismatchisdetected,appropriateactionsareselectedandexecuted,e.g.scaling,

• Communicate servicemetrics to theMarketplacevia theOrchestratornorthboundinterface.

AttheMarketplacelevel,servicemetricsareexploitedforaccounting(especially inpay-as-you-gobillingmodels) aswell as SLAManagement, in order to compare the statusof theservice against the contracted one. They are also presented via the Dashboard to the


©T-NOVAConsortium

79

Customer,sothathe/shecanhaveaconsolidatedoverallviewofthestatusof theserviceandtheresourcesconsumed.


©T-NOVAConsortium

80

5 GAPANALYSIS

Considering both existing industry oriented initiatives and currently available technologiesthatarebeingusedcommercially(orareinadevelopmentstage)afocusedgapanalysiswascarriedout todeterminewhatstepsneedtobetaken inorder tomoveNFV/SDNfrom itscurrent state to a position that can fully realise the needs of key areas that need to beaddressedduringtheprojectactivities. Inthefollowing,keyresultsofthisgapanalysisaredescribed,forallthevariousdomainsrelatingtotheT-NOVAOrchestratorandT-NOVAIVMarchitectures.Wherepossible thegapshavebeenalignedwithT-NOVAtaskswhere thesecouldbeeitherelucidatedorprogressedtowardsaddressingthegapcouldbemade.

5.1 Compute

TheComputedomainoftheT-NOVAarchitectureprovidesbasicbuildingblocksforVNFs/NSexecution.Gapanalysisforthisdomainidentifiedtwokeyareasthatneedtobeaddressedduringtheprojectactivities:

Table5.1:Gapanalysisinthecomputedomain

Gap Description T-NovaTaskAlignment

Virtualisationinfrastructurefortelecommunicationworkloads

Featuresrequestedbyworkloadsneedtobeinvestigatedingreaterdetailandspecialpurposeprocessors(orco-processors)havetobeintegratedincomputedomaininfrastructures.

Thoseenhancedfeatureshavealsotobeexposedtotheupperlayerofthearchitecture,inordertomakethemavailablefromanorchestrationperspective

Task3.2

Task4.1

Interoperabilitybetweendifferenthardwarearchitectures

ComputearchitectureheterogeneityneedstobeimprovedincurrentcomputedomaininfrastructuretoprovideagreaterdiversityofoptionsforcertainNFVworkloadtypes.

Supportforheterogeneityshouldnotonlyapplyintermsofmulti-vendortechnologies,butalsointermsofdifferenthardwarearchitecturesrequiredbydifferentVNFs.

Task4.1

5.2 Hypervisor

The Hypervisor domain is responsible for abstracting VNFs from the underlying physicalresources.ThemaingapissuesthathavetobeaddressedbyT-NOVAsystemforthisdomainare:

Table5.2:GapanalysisintheHypervisordomain



©T-NOVAConsortium

81

IntegrationofvSwitcheswithhypervisorsandvNICs

ThehypervisorsareresponsiblefortheintegrationbetweenvSwitchesandvNICs.

Howeverthisintegrationcurrentlyneedsfurtherperformanceimprovements.InorderfortheT-NOVAplatformtoprovidetherequiredlevelofperformance,itisnecessarytoaddresstheseperformancefeatures.

Task4.1

PortabilityofVirtualResourcesacrossdifferentplatforms

InordertosupportthelivemigrationofVNFs,allthevSwitchsolutionsneedtobedescribedusingthesamesyntax,providingtotheT-NOVAsystemacommoninterfacetoallowportabilityondifferentplatformsandsupportlivemigrationbythehypervisor.

Task3.2

ProcessorPinning

SomeVNFvendorsuseadedicatedCPUplacementpolicy,whichstrictly‘pins’thevCPUstoasetofhostphysicalCPUs.

ThisisnormallydonetomaximiseperformancesuchL3cachehitrates.

Howeverthiscanmakemigration`bythehypervisorchallenginginordertoguaranteethatsamepinningallocationofvCPUstophysicalcoresonthedestinationsystem.

Task4.1

Lightweightvirtualisation

DeploymentofVNFsusingVMiscurrentlythedefaultapproachfordeployments.Whilethisapproachhasnumberofadvantagessuchassecurityetc.deploymentstimes,deploymentdensityandscalablearechallengingparticularlyforscalingactions.AlternativeapproachesbasedoncontainersandCloudOS’stodeliverlittleweightandfastdeployments(>1sec)neededtodemonstratedandfullyevaluatedasviablealternativeapproaches.SolutionstothemanagementofdeploymentsofcontainersorunikernelsneedtobeinvestigatedsuchasDCOS,Brooklynetc.

Task4.1

5.3 SDNControllers

The SDN Controller domain is responsible for the control of the SDN-enabled networkelementsregardingthedeploymentandthemanagementofthevNets.Themainissuesthatneedtobeaddressedinclude::

Table5.3:GapanalysisregardingSDNControllers


Control-planeworkloadbalance

ResearchactivitiesarerequiredtomitigatevariousscalabilityandavailabilityissuesduetoSDNControlplanecentralization,whereacluster-basedapproachwouldhavealsoto

Task3.3

Task4.2


©T-NOVAConsortium

82

beconsidered.

BottleneckAvoidance Researchactivitiesarerequiredtoaddresslarge-scale,highloaddeploymentscenariostoavoidbottleneckswhereacluster-basedapproachwouldnecessitateconsideration

Task3.3

Task4.2

UniformNorth-boundInterface CurrentlyauniforminterfaceforalltheSDNcontrollersdoesnotexist,whichincreasesboththecomplexityofdevelopmentprocess.Workisrequiredintheabstractionofnorthboundinterfacestosupportapplicationdevelopers.

Task4.3

5.4 CloudControllers

The Cloud Controller represents the central management system for cloud deployments.Ranging from basic to more advanced, the T-NOVA project will investigate gaps in thecurrentsolutionswithinthisdomain,whichfallshortwithrespecttothefollowingaspects:

Table5.4:GapanalysisregardingCloudControllers

Gap Description T-NOVA TaskAlignment

InteroperabilitybetweendifferentCloudComputingPlatforms

FromaT-NOVAOrchestratorperspective,aunifiedsouthboundinterfaceisneeded,inordertomaketheAPIofdifferentIaaSprovidersaccessibleandtoenablecommunicationswithdifferentCloudComputingPlatforms.Evenifmostcloudplatformshavewidelyadoptedopenstandards,therearestillinconsistencieswithrespecttothedifferentversionsandAPIs,inconsistenciesthatneedtobeconcealedunderasingleinterface.

Task3.1

ResourceAllocationandConfiguration

FromaT-NOVAVNFperspective,cloudcontrollersneedtosupporttheallocationandconfigurationofnetworkresourcesandallowenhancedresourcemanagement.

Existingcloudmanagementsolutionsneedtobeextendedtoprovideaninterfacethatwouldallow,forinstance,enhancedconfigurationofnetworkparameters.

Task3.2

Task3.3

Task3.4

Providinginfrastructureutilisationdata

MonitoringandcollectinginformationwithrespecttothecurrentloadandavailabilityofresourcesisacrucialcapabilityforT-NOVAsystem,asitwillbeacloudmanagementframeworkthatneedstoperforminreal-timeandsupportveryhighvolumetraffic.

Althoughcloudplatformsalreadyoffermonitoringcapabilities,thechallengewithrespecttoT-NOVAistocollecttheinformationthatisrelevantforVNFsandhowtorepresentitsuchthatthe

Task3.2

Task3.4

Task4.4


©T-NOVAConsortium

83

Orchestrationlayercanbestutiliseit.

PlatformAwareness

CloudenvironmentsneedtobecomemoreawareofthefeaturesandcapabilitiesoftheirconstituentresourcesandtoexposetheseenhancedplatformfeaturestotheOrchestrationlayertoimprovetheplacementdecisionsofworkloadssuchasVNFs.

Acommondatamodeltodescriberesourcesisneeded,whichcouldbeusedtoidentifyspecificfeatureslikeDPDKenablementorSR-IOVcapabledevices.

Task3.2

Task4.1

5.5 NetworkVirtualisation

NetworkvirtualisationintroducesseveralgapsandopenissuesthatneedtobeaddressedbytheT-NOVAproject:


©T-NOVAConsortium

84

Table5.5:GapanalysisregardingNetworkVirtualisation

Gap Description T-NOVATaskAlignment

Networkresourceisolation

VNsarebydefinitionbasedonsharedresourcesandthisbringsuptheisolationproblem,especiallywhenthenumberofVNssharingthesameinfrastructureisveryhigh.

Ontheotherhand,thestrictnessofisolationvariesaccordingtothespecificusecase.InaNetworkas-a-Servicescenarioisolationitwillobviouslybeafundamentalrequirement.

IsolationisrequiredbetweentheVNsrunningseparateTelcoservices.WithintheT-NOVAplatform,aVNmightrequirecompleteisolationwhileanotheronemightshareitsresourceswhenitisidle,dependingonthebusinessmodelandthetypeofthespecificservice.

Task4.2

MultipleDCinterconnection

LimitationsofsupportingdistributedcloudserviceprovisioningandtherequirementforVNstospanmultiplecomputingfarms;seamlessnetworkinghandovertechnologiesarestillimmatureorinefficient;potentiallycomplicatingservicedeliveryprocessesorbusinessmodels.

T-NOVAplatformneedstomanagethiscomplexity,sinceoneofitsmainlyfeaturesistosupportservicedeploymentoverdifferentDCs.

Task4.2

Task4.5

Reliabilityofavirtualnetwork(VN)

Reliabilityisultimatelydeterminedbythedependabilityoftheunderlyinginfrastructure.Virtualisationintroducesanadditionallevelofcomplexityandrepresentsapotentialextrasourceoffailure.

VNsmustbereliable,atleastasreliableasaphysicalnetworkcounterpart.Todaymostoftheavailableproductswithnetworkvirtualisationcapabilitiesaremainlytargetedatthehigh-endsegmentofthemarket.

Ontheotherhand,verypromising,flexibleandadaptabletechnologiessuchasOpenFlowareperceivedasresearchtoolsandhavenotyetreachedapointofmaturitytoenablelarge-scaledeployment.

Task4.2

Task4.5

Interoperabilitybetweendifferentheterogeneous

Standardisationactivitiesarerequired,especiallywithinterconnectionofnon-

Task4.5


©T-NOVAConsortium

85

domains contiguousnetworkdomains.

Interoperabilityisacrucialrequirementtoenablewidespreaddeploymentofnetworkvirtualisation.StandardisationwillberequiredtoenableinteroperabilitybetweenVNs,aswellasinteroperabilitybetweenvirtualisedandnon-virtualisednetworks.

Scalabilityanddynamicresourceallocation

Dealingwiththeincreasingnumberofservicesaswellassubscribersforeachservicewouldbechallenging.TheimportanceofscalabilityasanetworkvirtualisationrequirementisparticularlyrelevantwhenthenumberofVNsisexpectedtogrow.NewsolutionsarerequiredtohelptheT-NOVAsystemtoscalewithrespecttothenetworksize(forinstancereducingthesizeofOpenFlowtables).

Task4.1

5.6 NFVOrchestrator

TheNFVorchestratorisresponsiblefortheNSs&VNFslifecyclemanagement.Severalopenissuesarestilltobeaddressed:

Table5.6:GapanalysisregardingOrchestration

Gap Description T-NOVATaskAlignment

VNFsPlacement

StandardisationbodiesshouldaddressthedefinitionandimplementationofalgorithmsforoptimalinfrastructureallocationaccordingtothevirtualisedservicecharacteristicsandSLAagreements.

Task3.3

Interoperabilitybetweenheterogeneousvirtualiseddomains

Standardisationactivitiesarerequiredinordertodeliverend-to-endservicesthathavevirtualisedcomponentsdistributedacrossmultipledomains,ownedandoperatedbydifferentvirtualserviceand/orinfrastructureproviders.

Task3.1

VirtualandPhysicalNetworkFunctionsOrchestration

Enhancementsarerequiredinstandardisationbodies,suchasETSIMANO,toaddressdatacentreWANlinksinterconnectivityconfigurationandorchestrationissues.

Task3.2

Task3.4

NetworkServicesScaling

EnhancementsarerequiredintheNSDescriptor,toaddresstheScalingofNetworkServices(composedbymorethanoneinterconnectedVNFs).ThisisstillaworkinprogressinT-NOVA,thatwilleventuallygenerateoneormore

Task3.4

WP6


©T-NOVAConsortium

87

6 CONCLUSIONS

Virtualisation is a foundational technology for the T-NOVA system in particular for theinfrastructurevirtualisationandmanagementlayer.Originatinginthecomputedomain,useof this approach now finds application in a variety of domains including storage andnetworking.Theapproachisbasedonabstractingphysicalresourcesintoaformthathidesits physical characteristics fromeither users or applications. It brings a variety of benefitsincluding better utilisation of resources, greater flexibility, improved scalability, etc.Virtualisation encompasses a number of technology approaches, at both a hardware andsoftwarelevel.

In the courseof reviewing the various virtualisation technologies and considering them inboththecontextofthetelecomsserviceprovidersandthepotentialneedsoftheT-NOVAsystem a variety of gaps in the capabilities of the currently available technologies wereidentified (see Tables 5-1 to 5-6). While the use of virtualisation technologies in the ITdomain is well established, adoption of this approach in carrier grade environments tosupport NFV and SDN proliferation brings a unique set of challenges that do not exist inenterprise ITenvironments.A varietyof furtherdevelopmentswill be required toaddressspecific issues in the currently available compute, hypervisor, SDN Controller, Cloud OSs,networkvirtualisationandorchestrationrelatedtechnologies.Whereappropriate,wehavemappedT-NOVA taskswhoseactivitieswill be related to these technologies challengesorlimitations.ItisexpectedthatwewillfurtherrefinethesegapstospecificissuesidentifiedinimplementationoftheT-NOVAsystem,andhighlightprogressthathasbeenmadeinfurtherelucidating the characteristics of these problems, as well as the work that T-NOVA hascarriedoutinordertocontributetowardsasolution.

From an architectural point of view, the T-NOVA Orchestrator is composed by twomainbuildingblocks:theNFVOandtheVNFM.TheNFVOhastwomainresponsibilities,whichareaccomplished by its two FEs designated byNSO andVROwithin the T-NOVAproject. TheNSO orchestrates the subset of NFVO functions that are responsible for the lifecyclemanagementofNetworkServices,whiletheVROperformstheorchestration/managementof the virtualised infrastructure resources distributed acrossmultiple DCs. In particular, itperformsthemappingoftheincomingNSrequeststotheavailablevirtualisedinfrastructureresources, aswell as the coordinationof the resources allocation andplacement for eachVMthatcomposestheVNF(andtheNS).TheVROdoesthisbyinteractingwiththeVIM,aswellaswiththeWICMwhichinteractswiththeWANelementsforconnectivitymanagementpurposes.

Since the NSs are composed by VNFs, the NFVO is able to decompose each NS into theconstituentVNFs.AlthoughtheNFVOhastheknowledgeoftheVNFsthatcomposetheNS,itdelegatestheirlifecyclemanagementtoadedicatedFEdesignatedastheVNFM.

InSection2,thearchitectureforT-NOVAOrchestratorispresented,takingintoaccountthelistofOrchestratorrequirements identifiedafteraresearchingvarioussourcessuchastheuse cases defined in D2.1, ETSI ISG NFV requirements, ITU-T requirement for networkvirtualisation as well as excerpts from the relevant parts of the ETSI ISG MANO WGarchitecture. The reference architecture and its functionalities outlined in section 2.3 arenowbeingimplementedbythevarioustasksinWP3.

The T-NOVA IVM is responsible for providing the hosting and execution environment fornetworkservices intheformofvirtualisedresourcesthatareabstractedfromthephysicalresources in the compute and infrastructure network domains. A system engineering


©T-NOVAConsortium

88

approachwasadoptedtodefinethekeyfunctionalblocksoftheIVMandtheirinterfaces.InadditionthekeyobjectivesfortheIVMweredefined.UseofthisinformationandpreviousT-NOVA deliverables contributed to a requirement capture process that focused onidentifying thedesiredbehaviours for the IVM.Requirementswhere identified foreachofthefunctionalentitieswithintheIVM.TheserequirementswherethenusedinthedesignoftheIVM.

From an architectural perspective the T-NOVA IVM includes three key functional blocks,namely the NFVI, VIM and WICM, which are defined and discussed in Section 3. Thesefunctionalblocksarecomprisedofvariousdomainsthathavespecifictechnologycapabilitiesrequiredforthedeliveryandmanagementofvirtualisedresources.ForexampletheNFVIiscomprised of the Compute, Hypervisor and Network domains. A number of specificinterfaces provide both the internal and external connectivity that integrates the varioustechnologycomponents intoa functional IVM.FromaT-NOVAsystemperspectivethekeyexternal interfaces of the IVM are those to the T-NOVA Orchestrator, which areimplementedintheVIM.TheseinterfacesenabletheT-NOVAOrchestratortosendrequeststo theVIMtocreateandconnectVMs inorder tosupport thedeploymentofVNFserviceandtomanagethevirtual resourcesallocatedto theVNFs inorder toaccomplish toSLAs.Additionally, these interfaces allow the IVM to send infrastructuremetrics related to theutilisation and performance to the T-NOVA Orchestrator in order that this entity canperform placement decisions and management of existing deployed services. Anotherimportant interface istheoneprovidedbytheWICMtotheOrchestrator inorderfor ittomanage the network resources related to external networks i.e.WAN transport betweenDCs.

Collectively, these reference architectures and FEs instantiate the requirements thatwereidentifiedfortheT-NOVAOrchestratorandfortheT-NOVAIVMtogetherwithitsgoalsandobjectives.The referencearchitectureswere interrogatedandvalidatedat functional levelthrough thedevelopment sequencediagramsasoutlined in Section4,whichdescribe thekey actions and interactions takenwithin the T-NOVA systemduring standardoperationalactivitiesrelatedtothedeploymentandmanagementofNSandVNFservices.


©T-NOVAConsortium

89

ANNEXA-ORCHESTRATORREQUIREMENTS

The present annex contains a set of tables, which include the requirements identified insection2,i.e.OrchestratorinternalrequirementsandInterfacerequirements.

Eachrequirementhasassociatedasetofattributesrelatedtoitsidentification(Req.IDandReq.Name),toitstextsupport(Alignment)andtoitsdescription(RequirementDescriptionandcomplementaryComments).


©T-NOVAConsortium

90

A.1 Internalrequirements

A.1.1 NFVORequirements

A.1.1.1NSLifecyclerequirements

Table6.1:OrchestratorRequirements–NFVO-NSLifecycle

Req.ID Alignment Req.Name RequirementDescription Comments

NFVO.1 ETSIMANO§C.2.1

On-boardingNS TheNFVOSHALLbeabletoacceptneworupdatedNSstobeon-boarded,uponrequest.

TheNSlifecycleisinitiatedbyaNSon-boardingrequestbytheSP,andincludesprovidingtheNSDescriptor(NSD)totheNFVOandstoringitintheNSCatalogue.TheNSD,whichmustbevalidated,includestheNSdeploymentflavours,aswellasreferencestotheVNFsused,theVNFForwardingGraphs(VNFFGs)andtotheVirtualLinkDescriptors(VLDs)neededtoimplementit.

NFVO.2 ETSIMANO§C.3

NSInstantiationrequest

TheNFVOSHALLbeabletoinstantiateanalreadyon-boardedNS,uponrequest.

InordertostartaT-NOVAservice,anewinstancefromtheNSCataloguehastobedeployed,usuallybyrequestoftheSP(insomescenarios,aCustomerorevenanOSSmayalsotriggerthatrequest)throughtheMarketplace.Duringtheinstantiationprocess,theNFVOvalidatestherequest,e.g.byauthorizingtherequester,byvalidatingtechnicalcontent

NFVO.3 T-NOVA ExtractionofspecificdeploymentNSinformation

TheNFVOSHALLbeabletoextractfromtheincomingNSinstantiationrequestthesetofrequiredinformationtoproceedwiththeinstantiationofthatNS.

AfterreceivingtheNSinstantiationrequest,theNFVOextractstheinformationabouttheNSinstantiation(deploymentflavours,VNFFG,VLDs,etc.).

NFVO.4 T-NOVA ConfigureNS TheNFVOSHALLbeabletoconfigureorupdatetheconfigurationofaninstantiatedNS,uponrequest.

T-NOVANSsmustbeconfigurableuponexternalrequeste.g.CustomerorSP.AsaNSistheresultofthecompositionofatomicVNFinstancesandtheirinterconnections,theconfigurationofaNSmayimplytheconfigurationoftheentiresetofVNFs.

NFVO.5 UC1 NSTermination TheNFVOSHALLbeabletode-allocatealltheresourcesthathavebeenallocatedtoaspecificNSinstancewheneithertheNSSLAterminates,theNSinstanceisterminatedbyinternaltriggers,orwhentheNSinstanceisterminateduponrequest,e.g.bytheCustomerorbytheSP.

ThedurationoftheNSwillbespecifiedintheSLA.AlternativelytheSLAortheNSinstancecanbeterminatedon-demand,e.g.bytheCustomerorbytheSP.WhentheNSinstanceisnolongerneededthesystemshouldbeabletode-allocatealltheresourcesinitiallyallocatedtothatspecificNSinstanceandcanceltheSLA.UnlesstheSPaskstoremovetheNS,itremainson-boardedsothatothercustomerscanbuyandrequestnewinstancesofit.NSinstancescannotbepaused.


Scale-outNS TheNFVOSHALLbeabletoscaleouttheNS,eitheruponrequestorautomatically.

AutomaticscalingoutdependsonanalgorithmandalternativearchitectureordeploymentflavoursprovidedbytheSPwhencomposingtheNS.ScalingoutaNSmightimplyincreasingtheVMssupportingitsVNF(s).


©T-NOVAConsortium

91


Scale-inNS TheNFVOSHALLbeabletoscaleintheNS,eitheruponrequestorautomatically.

AutomaticscalingimpliestheuseofanalgorithmandalternativearchitectureordeploymentflavoursprovidedbytheSPwhencomposingtheNS.ScalinginaNSmightimplydecreasingtheVMssupportingitsVNF(s).

A.1.1.2VNFLifecyclerequirements

Table6.2:OrchestratorRequirements–NFVO-VNFLifecycle


NFVO.8 §B.2.1 On-boardingVNFPackageRequest

TheNFVOSHALLbeabletoreceiveneworupdatedversionsofexistingVNFpackagesfromtheNFStoreandstorethemintheVNFCatalogue.

VNFpackageincludestheVNFDescriptor(VNFD),theVNFsoftwareimage(s)andtheVNFversion.

NFVO.9 ETSIMANO§B.3.1.2

VNFInstantiationRequestbytheNVFO

TheNFVOSHALLbeabletoinstantiateaVNF,uponrequest. VNFinstantiationsoccurinthecontextofNSinstantiationrequests.WhenarequesttoinstantiateaVNFisreceived,theNFVOvalidatestherequest.TheNFVOacknowledgesthecompletionoftheVNFinstantiationafterconfiguringtheVNFthroughtheVNFM.

NFVO.10 UC2UC3

VNFConfigurationRequestbytheNFVO

TheNFVOSHALLbeabletorequesttheVNFMtoconfigureaninstantiatedVNF.

T-NOVAVNFsmustbeconfigurablebyexternalrequest,e.g.CustomerorSP,orautomaticallyuponthecompletionofaninstantiationprocess.Itincludesthenotificationofthesuccessfulconfiguration.

NFVO.11 UC3.2 VNFScaleOut BymonitoringtheNSdata,theNFVOSHALLrecogniseanSLAthatisgoingtobreachandactbyrequestingtheVNFMtoscale-outtheNS’sVNFs(allocatingmoreVMstothoseVNFs).

TheT-NOVAsystemmustprovidetheabilityforadditionalVMsrequestsinordertomeetbusinessneeds.

NFVO.12 UC3.2 VNFScaleIn BymonitoringtheNSdata,theNFVOSHALLrecogniseanSLAthatisgoingtobreachandactbyrequestingtheVNFMrequesttoscale-intheNS’sVNFs(releasingVMsusedbyinstancesoftheVNF).

T-NOVAsystemmustprovidetheabilityforareductionofVMsortocompletelyremoveaVNFasrequiredbytheirchangingbusinessneeds.

NFVO.13 UC3.1 VNFScaleUp BymonitoringtheNSdata,theNFVOSHALLrecognizeanSLAthatisgoingtobreachandactbyrequestingtheVNFMrequesttoscale-uptheNS’sVNFs(increasingthespecifiedamountsofVMresourcesfromVMsusedbyinstancesoftheVNF).

TheT-NOVAsystemmustprovidetheabilityforincreasinginaVNFinordertomeetbusinessneeds.

NFVO.14 UC3.1 VNFScaleDown BymonitoringtheNSdata,theNFVOSHALLrecognizeanSLAthatisgoingtobreachandbyrequestingtheVNFMrequesttoscale-downtheNS’sVNFs(decreasingthespecifiedamountofallocatedresourcesfromVMs,suchasmemoryandstorage,usedbyinstancesoftheVNF).

TheT-NOVAsystemmustprovidetheabilityfordecreasingresourcesinaVNFinordertomeetbusinessneeds.

NFVO.15

T_NOVA_24,T_NOVA_25,T_NOVA_35,T_NOVA_27

ManageVM’sstate

TheNFVOSHALLbeabletorequesttheVNFMtomanagetheVMsallocatedtoagivenVNF.

WecanassumeafiniteandsmallnumberofpossibleVMstates,e.g.,‘Beingconfigured’,‘Notrunning’,‘Running’,‘Beingre-scaled’,‘Beingstopped’.Itisassumedthatwhenina‘Running’statetheVMisreadytobe(re-)configured.


©T-NOVAConsortium

92

A.1.1.3ResourceHandlingRequirements

Table6.3:OrchestratorRequirements–NFVO-ResourceHandling


NFVO.16 ETSIMANO§C.3

Mappingofresources

TheNFVOSHALLbeabletooptimallymaptheVNFsthatarepartofaNStotheexistinginfrastructure,accordingtoanagreedNSSLA.

Baseduponthecurrentinfrastructurestatus,therequestedVNFandSLA,theNFVO,usingResourceMappingalgorithms,mustbeabletofindtheresourcesitshouldallocateintermsofPoPsandtheirconnectionsortodeclarethattheNScannotbeinstantiated.TheSLAstillneedstobedetailed.ResourceMappingcanconsiderlinklatencyorlinkdelayandlookforconnectionsamongPoP,whichguaranteetherespectoflatencyordelaythresholds.

NFVO.17 Resourcesinstantiation/releasing

TheNFVOSHALLbeabletorequesttheVIMfortheinstantiation/releaseoftheVMsthatcomposeeachVNFoftheNSandtheconnectionsbetweenthem

DuringtheNSinstantiation/scalingprocedures,foreachVNFandcorrespondingmappedPoP,theNFVOusestheOpenStackSchedulerfordecidingthebestlocationfortheVMs,anditrequeststheVIMtoallocatetherequiredvirtualisedresources(i.e..VMsandtheirinterconnections).

NFVO.18 ETSIMANO§5.4.1,§5.4.3

ManagementofVMimages

TheNFVOSHALLbeabletomanageVMimagesrelatedtotheVMssupportingagivenVNF.

TheNFVOistheFEinchargeofhandlingVMandVMresourcesintheT-NOVAsystem.Assuch,itmustbeabletomanageVMimages,e.g.byprovidingVIMwithVMimagesforVNFon-boardingorupdating,orbyremovingimagesforVNFremoval.

NFVO.19 UC3.1 ResourcesInventoryTracking

TheNFVOSHALLupdateitsinventoryofallocated/availableresourceswhenresourcesareallocated/released.

TheT-NOVASystemmustmaintaintheInfrastructureCatalogueandaccuratelytrackresourceconsumptionandthedetailsoftheservicesconsumingthoseresources.

A.1.1.4MonitoringProcessrequirements

Table6.4:OrchestratorRequirements–NFVO-MonitoringProcess


NFVO.20 UC2,UC3,UC4

NS-specificresourcemonitoringbytheNFVO

TheNFVOSHALLbeabletomonitorNS-relatedresourcesonarealtimebasis.

NS-specificmonitoringisrelatedwiththemonitoringofthevirtualnetworklinksthatinterconnecttheVNFs(retrievedfromtheVIM),aswellasmonitoringofVNF-specificdetailsthatcanbeusedtoassurethattheNSisfulfillingtheestablishedSLAwiththecustomer.

NFVO.21 UC4 MonitoringmetricsconsolidationbytheNFVO

TheNFVOSHALLbeabletoaggregateandconsolidateallmonitoringmetricsassociatedwithaservice.

A consolidated operational picture of the service via the dashboard is considered amandatory customer requirement. The gatheredmetrics should be presented to theCustomer,totheSP,ortotheFP,withanintegratedstatusoftheprovisionedservice,inoneoftheabovelayers.


©T-NOVAConsortium

93

A.1.1.5ConnectivityHandlingrequirements

Table6.5:OrchestratorRequirements–NFVO-ConnectivityHandling



Connectivitymanagement

TheNFVOSHALLbeabletorequesttheVIMforinstantiation/releaseoftheinter-connectionofrequiredVMsintheITcomputedomain,networkdomainandintheinfrastructurenetwork.

DuringtheNSinstantiation/scalingprocedures,afterinstallingtheVMsfortheVNF/NS,theNFVOrequeststheVIM/WICMtoallocate/releasetherequiredvirtualnetworkresources(i.e.,virtualnetworklinks).ThisrequirementincludesnotificationoftheVNFMoftheremovedVNF.VNFshavinginstancesparticipatinginNSinstancescannotberemoveduntiltheNSinstancestopsandisrequestedtoberemoved.Alsousedinre-instantiatingVNFinfrastructure(e.g.,forperformanceormal-functionreasons).


©T-NOVAConsortium

94

A.1.1.7Marketplace-specificinteractionsrequirements

Table6.6:OrchestratorRequirements–NFVO-Marketplacespecific


NFVO.23 T-NOVA PublishNSinstantiation

TheNFVOSHALLbeabletonotifythattherequested(new,updated)NSinstantiationisreadytobeused.

T-NOVAsystemmustnotifyrelevantexternalentitiesuponsuccessfulinstantiationofeveryVNFandconnectionsbetweenthemofarequestedNSinstance.InthecaseoftheMarketplace,itmayusethisnotificationforAccounting/Billingpurposes.

NFVO.24 T-NOVA PublishNSmetrics

TheNFVOSHALLbeabletopublishtheNSmetrics,ifconsideredintheSLA.

TheT-NOVAsystemmustprovidetoexternalentitiesNSmetricsinordertoenableservicecontrol.

NFVO.25 UC1.1 VNFmappingofSLAdata

TheNFVOSHALLbeabletomaptheVNFmonitoringparameterstoeachNSinstanceSLAattributes.

ResultsofselectionofferingsmaterializedinSLAsneedtobetranslatedintoVNFattributesinordertobeprocessedbytheT-NOVAsystem,accordingtothecontentsoftheVNFdescriptorwheretheamountofneededresourcesisindicated.

NFVO.26 T-NOVA SLAenforcementrequest

TheNFVOSHALLbeabletotaketherequiredactions(e.g.scaleout,withtheinstantiationofmoreVMsandconnectionsbetweenthem)uponrequesttoenforceaSLA.

ItisassumedthattheSLAprovidesalltheinformationaboutmetricsandthresholdstobecomparedwith,togetherwiththeNSdescriptorprovidingalternativearchitecturesordeploymentflavours,e.g.scalinginwhenmetricsshowunder-usedresourcesshouldbeautomatic.

NFVO.27 UC4,UC5

NSusageaccountingandbilling

TheNFVOSHALLstorealltheinformationaboutresourcesusageperservice,andSHALLprovideittoexternalentitiestobillonapay-per-usemode.

Pay-as-you-gomaybeconsideredattractiveforsomeCustomers,asanoption,asopposedtoflat-rate.TheNFVOshouldnotifyrelevantFEsinorderthattheT-NOVAsystembecomesabletodeploythistypeofbilling/charging.

A.1.2 VNFMRequirements

A.1.2.1VNFLifecyclerequirements

Table6.7:OrchestratorRequirements–VNFM-VNFLifecycle


VNFM.1 ETSIMANO§B.3.1.2

VNFInstantiationRequestbytheVNFM

TheVNFMSHALLbeabletoacceptarequestfromtheNFVOtoinstantiateaVNF.

AfterreceivingaVNFinstantiationrequestfromtheNFVO,theVNFMwillstartcoordinatingtheVNFinstantiationprocedure.Nevertheless,thevirtualizedresourcesallocationisunderthescopeoftheNFVOandthereforetheVNFMwillhavetorequestthelatertoallocatetherequiredresourcesfortheVNF.

VNFM.2 UC2UC3

VNFConfigurationRequestbytheVNFM

TheVNFMSHALLbeabletoacceptaNFVOrequesttoconfigureaninstantiatedVNF.

T-NOVAVNFsmustbeconfigureduponexternalrequestorfollowinganexternalinstantiationrequest.Itincludesthebackwardsnotificationofthesuccessfulconfiguration.


©T-NOVAConsortium

95

VNFM.3 ETSIMANO§C.3

CheckofVNFspartofaNSbytheVNFM

TheVNFMSHALLbeabletoacceptarequestfromtheNFVOtochecktheVNFsassociatedwithaNS,accordingtotheNSdescriptor.

ForeachVNFindicatedintheNSdescriptor,theVNFMchecksifaVNFmatchingexistsintheVNFCatalogue

VNFM.4 UC2 VNFlifecycleautomationbytheVNFM

TheVNFMSHALLbeabletoautomatetheinstantiationofVNFsandassociatedVMresourcesbytriggeringscalingprocedures.

AutomationofVNFlifecycleisanessentialcharacteristicoftheT-NOVAsystem.TriggeringofscalingproceduresisbasedonthemonitoringprocessmaintainedoverVNFs,aswellasonpolicymanagementandotherinternalalgorithmcriteria.TheexactresourcesontowhichtheVNFwillrunhavetoberequestedtotheNFVO.

A.1.2.2MonitoringProcessrequirements

Table6.8:Orchestratorrequirements–VNFM-MonitoringProcess


VNFM.05 UC2,UC3,UC4

VNF-specificresourcemonitoringbytheVNFM

TheVNFMSHALLbeabletomonitorVNF-relatedresourcesonarealtimebasis.

VNF-specificmonitoringisrelatedwiththemonitoringofinformationretrievedfromtheVIM,relatedtothevirtualizedinfrastructureresourcesallocatedtotheVNF(i.e.compute/storage/memoryoftheVMsandvirtualnetworklinksthatinterconnecttheVMs),aswellasmonitoringofVNF-specificmetricsthatcanbeusedtoassurethattheVNFisbehavingasitshould.

VNFM.06 UC4 MonitoringmetricsconsolidationbytheVNFM

TheVNFMSHALLbeabletoaggregateandconsolidateallmonitoringVNFmetricsassociatedwithaservice.

Aconsolidatedoperationalpictureoftheserviceviathedashboardisconsideredamandatorycustomerrequirement.ThecollectedmetricsshouldbepresentedbytheVNFMtotheNVFO,andfromthisFEtothedashboardwithanintegratedstatusoftheprovisionedservice.


©T-NOVAConsortium

96

A.2 Interfacerequirements

A.2.1 InterfacewithVIM

TherequirementsidentifiedfortheInterfacewiththeVIMmoduleareasfollows:

Table6.9:RequirementsbetweentheOrchestratorandVIM

Req.id Alignment Domain(s) RequirementName

RequirementDescription Comments

Or-Vi.01 Orchestrator,VIM Reserve/releaseresources

TheOrchestratorSHALLusethisinterfacetorequesttheVIMtoreserveorreleasetheentirerequiredsetofinfrastructureresourcesneededtoprovisionagivenVNF

Caremustbetakeninordernottohaveresourcesreservedandnotprovisionedforlongperiodsoftime,thusimpactingontheoptimisationofresourceusage.TheimplementationoftheresourcereservationwillnotbedoneduetothecurrentlylackofsupportfromthechosenVIM(OpenStack)

Or-Vi.02

T_NOVA_03,T_NOVA_21,T_NOVA_22,T_NOVA_26,T_NOVA_31,T_NOVA_33,T_NOVA_34,T_NOVA_36,T_NOVA_37,T_NOVA_38,T_NOVA_39,T_NOVA_40,T_NOVA_42,T_NOVA_43,T_NOVA_44,T_NOVA_45,T_NOVA_586

Orchestrator,VIMAllocate/release/updateresources

TheOrchestratorSHALLusethisinterfacetorequesttheVIMtoallocate,updateorreleasetherequiredinfrastructureandnetworkresourcesneededtoprovisionagivenVNF

Itisassumedthatconfigurationinformationisaresourceupdate.Resourceupdatemightimplystopandre-start,withamigrationinbetween.ExclusiveuseofthisinterfacegrantsconsistencebetweenactualresourcestatusandOrchestratorviewofit..ThisinterfaceisalsousedtoprovisionNFGsandinter-VNFCnetworks.

Or-Vi.03 Orchestrator,VIMAdd/update/deleteSWimage

TheOrchestratorSHALLusethisinterfacetoadd,updateordeleteaSWimage(usuallyforaVNFComponent)

PerformancewillprobablydemandhavingtheseimagesreadytobedeployedontheOrchestrator’sside.ThesesoftwareimageswillbeperVDU.

6ReferstoT-NOVArequirementsdescribedindeliverableD2.1[6] "UseSystemCasesandRequirements,"2014.Available:


©T-NOVAConsortium

97

Or-Vi.04 UC4,T_NOVA_46 Orchestrator,VIMRetrieveinfrastructureusagedata

TheOrchestratorSHALLuseaninterfacetocollect/receiveinfrastructureutilisationdata(network,computeandstorage)andanyrequiredVNFImetricfromtheVIM

Someofthisdataisusedtodeterminetheperformanceoftheinfrastructure(includingfailurenotifications)andtoinformdecisionsonwheretoprovisionnewlyrequestedservicesorwheretomigrateanalreadyprovisionedNSthatispredictedtobreakitsSLA.Thisinterfacewillverylikelyhavetosupportveryhighvolumetraffic.

Or-Vi.05 UC4,T_NOVA_20 Orchestrator,VIM

Retrieveinfrastructureresourcesmetadata

TheOrchestratorSHALLuseaninterfacetorequestinfrastructure'smetadatafromtheVIM

Duetohighperformanceneeds,thisdatawillmostprobablyhavetobecachedontheOrchestrator’sside.

Or-Vi.06 T_NOVA_02 Orchestrator,VIM SecureinterfacesTheinterfacesbetweentheOrchestratorandtheVIMSHALLbesecure,inordertoavoideavesdropping(andothersecuritythreats)

Weshouldkeepinmindthatencryptingallthecommunicationbetweenthesetwoentitieswillprobablymakeaperformingsolutiontoocostly

Or-Vi.07 Orchestrator,VIM VNFinitializationflows

TheinterfacesbetweentheOrchestratorandtheVIMSHALLallowtopassinitializationflowstotheVIMwhennewVNFsareinstantiated

TheinterfacesbetweentheOrchestratorandtheVIMmustallownotonlytoprovision/deprovisionresourcesaccordingtothe“static”VNFdescription,butalsotosendthroughtheinitializationcommandsfortheVNFwhichcanonlybedefinedatinstantiationtime(e.g.theIPaddresses).


©T-NOVAConsortium

98

A.2.2 InterfacewithVNF

TherequirementsidentifiedfortheInterfacewiththeVNFmoduleareasfollows:

Table6.10:RequirementsbetweentheOrchestratorandVNF

Req.id Alignment Domain(s) RequirementName

RequirementDescription JustificationofRequirement

Vnfm-Vnf.01 T_NOVA_02 VNFM,VNF SecureinterfacesAlltheinterfacesbetweentheVNFMandtheVNFSHALLbesecure,inordertoavoideavesdropping(andothersecuritythreats)

RequiredtoavoideavesdroppingtheconnectionbetweentheVNFMandeachVNF.Weshouldkeepinmindthatencryptingallthecommunicationbetweenthesetwoentitieswillprobablymakeahighperformancesolutiontoocostly

Vnfm-Vnf.02 VNFM,VNFInstantiate/terminateVNF

TheVNFMSHALLusethisinterfacetoinstantiateanewVNForterminateonethathasalreadybeeninstantiated Requiredtocreate/removeVNFsduringtheVNFlifecycle

Vnfm-Vnf.03T_NOVA_46,T_NOVA_48

VNFM,VNFAcceptVNFinstancerun-timeinformation

TheVNFMSHALLusethisinterfacetoaccepttheVNFinstancerun-timeinformation(includingperformancemetrics)

VNFinstancerun-timeinformationiscrucialbothforVNFscaling(requestedbytheNFVO)andforshowingNetworkServices’metricsintheMarketplace’sDashboard

Vnfm-Vnf.04T_NOVA_23T_NOVA_33

VNFM,VNF ConfigureaVNF TheVNFMSHALLusethisinterfaceto(re-)configureaVNFinstance

Inthegeneralcase,theCustomershouldbeableto(re-)configureaVNF(instance).Includesscaling.

Vnfm-Vnf.05T_NOVA_24,T_NOVA_35,T_NOVA_58

VNFM,VNF ManageVNFstateTheVNFMSHALLusethisinterfacetocollect/requestfromtheVNFsthestate/changeofagivenVNF(e.g.start,stop,etc.)

ThisinterfaceincludescollectingthestateoftheVNF(aswellaschangingit).TheVNFinstanceshouldincludeastatelike‘Readytobeused’whenitisregisteredintherepository.

Vnfm-Vnf.06

T_NOVA_36,T_NOVA_37,T_NOVA_38,T_NOVA_39,T_NOVA_42,T_NOVA_43,T_NOVA_44,T_NOVA_45

VNFM,VNF ScaleVNF TheVNFMSHALLusethisinterfacetorequesttheappropriatescaling(in/out/up/down)metadatatotheVNF

VNFscalingdependsonthe(mostlyarchitectural)optionstheFPprovidedwhenregisteringtheVNF.TheVNFscalingmetadataisthenusedbytheNFVOtorequesttheVIMtoallocatetherequiredinfrastructure


©T-NOVAConsortium

99

A.2.3 InterfacewithMarketplace

TherequirementsidentifiedfortheInterfacewiththeMarketplaceareasfollows:

Table6.11:RequirementsbetweentheOrchestratorandtheMarketplace

Req.id Alignment Domain RequirementName


NFVO-MKT.01

UC1,T_NOVA_10,T_NOVA_15

Orchestrator,Marketplace

ProvideavailableVNFs

TheMarketplaceSHALLusethisinterfacewiththeOrchestratortoprovidetheServiceProviderwithalistoftheVNFs,sothatitcanselectandparameterisethem,orusetheminthecompositionofanewnetworkservice.

ItisassumedthatthisVNFmetadataincludesaURL/repositorynamefromwhichtofetchtheactualVNFsoftwareandinstallitonthepreviouslyallocatedinfrastructure(seeNFVO.10below).Notethat,althoughthisinformationwillmostcertainlyhavetobecachedontheOrchestrator’ssideforperformancereasons,theavailableVNFswillbedynamic,soupdatestothiscachedinformationwillberatherfrequent.

NFVO-MKT.02

UC2,T_NOVA_04,T_NOVA_08,T_NOVA_20


ProvisionanewNS

TheMarketplaceSHALLusethisinterfacetorequesttheOrchestratortoacceptanewNS,aftertheSPhascomposeditbyusingtheVNFsavailableintheVNFCatalogue.

TheNSmusthaveaNSDescriptor,withthelistofcomposingVNFs,connectionsbetweenthem,etc.EachNScanbecomposedofoneormoreVNFs..Itisassumedthatinformationaboutscaling(up/down/in/out)isincludedintheNSD(oratleastreasonablevaluescanbeinferred).

NFVO-MKT.03

UC2,T_NOVA_04,T_NOVA_08,T_NOVA_20


ProvisionanewNSinstance

TheMarketplaceSHALLusethisinterfacetorequesttheOrchestratortoprovisionanewtheNSinstance,aftertheCustomerhasselectedandparameterisedthenetworkservice.

TheOrchestratorSHALLreadtheNSDandallocatestheneededresources.ThenotificationoftheservicereadinessmustbedonetotheMarketplace.

NFVO-MKT.04

UC3,T_NOVA_31,T_NOVA_32,T_NOVA_33,T_NOVA_36,T_NOVA_42,T_NOVA_44


ChangeconfigurationofaNSinstance

TheMarketplaceSHALLusethisinterfacetochangetheconfigurationofaNSinstanceontheOrchestrator.

NFVO-MKT.05

UC5,UC6,T_NOVA_03,T_NOVA_28,T_NOVA_29,T_NOVA_34,T_NOVA_41,


ProvideNSinstancestatetransitionsinformation

TheMarketplaceSHALLusethisinterfacetomakeagivenNSinstancetochangeitsstateanddeterminewhichstatetransitionsofagivenNSinstancehaveoccurred,e.g.tofacilitatestartingandstoppingbillingfortheservice.

ItisassumedthateachNShasapre-definedstate-diagram,like‘Readytorun’,‘Running’,‘Stopped’,etc.,thatisalsoknowntotheMarketplace.Itisassumedthattheimpactonthedependentmoduleslikebilling,aretakencarebytheMarketplace(seeNFVO.04).SLAManagementispartoftheMarketplace.Eitherafteracustomer’srequestorbythepre-definedendingdatehadbeenattained,theSLAManagement


©T-NOVAConsortium

100

T_NOVA_48,T_NOVA_56,T_NOVA_57,T_NOVA_58

notifiestheOrchestratoroftheendoftheSLA.

NFVO-MKT.06

UC4,T_NOVA_28,T_NOVA_29,T_NOVA_30,T_NOVA_46,T_NOVA_52


ProvideNSmonitoringdata

TheMarketplaceSHALLusethisinterfacetoshowtheCustomerhowthesubscribedNSinstanceisbehaving,howitcomparestotheagreedSLAandbilltheserviceusage.

Thisinterfacewillverylikelyhavetosupportveryhighvolumetraffic.

NFVO-MKT.07

T_NOVA_02 Orchestrator,Marketplace

Securecommunication

InterfacesbetweentheMarketplaceandtheOrchestratorSHOULDbesecured.

Encryptionshouldbeused,inordertopreventeavesdropping.EvenbetweentheMarketplaceandtheOrchestrator,sincetheMarketplaceisreallyasetofdistributedapps.


©T-NOVAConsortium

101

ANNEXB-VIRTUALISEDINFRASTRUCTUREMANAGEMENTREQUIREMENTS

B.1 VirtualInfrastructureManagementRequirements

Table6.12:IVMRequirements–VIM

Req.id T-NOVAUseCaseAlignment

Domain RequirementName


VIM.1 UC1,UC2.1 VIM Abilitytohandleheterogeneousphysicalresources

TheT-NOVAVIMSHALLhavetheabilitytohandleandcontrolbothITandnetworkheterogeneousphysicalinfrastructureresources.

BasicfunctionalrequirementoftheVIM.

VIM.2 UC1,UC2.1 VIM Abilitytoprovisionvirtualinstancesoftheinfrastructureresources

TheT-NOVAVIMSHALLbeabletocreatevirtualresourceinstancesfromphysicalinfrastructureresourcesuponrequestfromtheOrchestrator

RequiredtosupportVIMintegrationwiththeT-NOVAOrchestrator.

VIM.3 UC3/3.1/3.2 VIM APIExposure TheT-NOVAVIMSHALLprovideasetofAPI’stosupportintegrationofitscontrolfunctionswiththeT-NOVAOrchestrationlayer.

RequiredtosupportVIMintegrationwiththeT-NOVAOrchestrator

VIM.4 UC2.1 VIM Resourceabstraction

TheT-NOVAIVMlayerSHALLprovideresourceinformationattheVIMlevelfortherepresentationofphysicalresources.

RequiredtosupportVIMintegrationwiththeT-NOVAOrchestrator.

VIM.5 UC1.1UC1.3UC2.1UC4

VIM Abilitytosupportdifferentservicelevels

TheVIMcontrollerSHOULDprovidetheabilitytorequestdifferentservicelevelswithmeasurablereliabilityandavailabilitymetrics.

RequiredtosupportSLAsagreementswhenaserviceispurchasedintheT-NOVAMarketplace.

VIM.6 UC1 VIM Translationofreferencesbetweenvirtualandphysicalresourceidentifiers

TheVIMcontrollerSHOULDbeabletoassignIDstovirtualcomponents(e.g.,NFs,virtuallinks)andprovidethetranslationofreferencesbetweenlogicalandphysicalresourceidentifiers.

Needtotracktheuseofphysicalnetworkresources.


©T-NOVAConsortium

102

VIM.7 UC2

UC3

VIM Multi-tenancycompliance”

TheVIMcontrollerSHALLguaranteeisolationamongthedifferentvirtualnetworkresourcescreatedtoprovisiontherequestedservices.

T-NOVAsystemwillprovideservicesfordifferentcustomersthroughthecompositionanddeploymentofVNFs.Theseserviceswillsharethesamephysicalnetworkinfrastructure,atthesametimethus,isolationatthephysicallevelmustbeguaranteedforeachcustomer.

VIM.8 UC3.1,UC3.2,UC3.3UC4

VIM ControlandMonitoring

TheVIMSHALLbeabletoprovidetherequiredinformationtoallowthehigherlevelsystemlayerstovisualisethereal-timestatus,historicalperformanceandresourceutilisationofboththeunderlay(physical)andoverlay(virtual)networks.

T-NOVAsystemsmustbeabletovisualisethereal-timestatus,historicalstatusandresourceutilisationoftheunderlayandoverlaynetworkstosupportstandardoperationsactivitiessuchasbandwidthadjustments

VIM.9 UC3 VIM Scalability TheVIMcontrollerSHOULDscaleinaccordancetothenumberofvirtualresourceinstancesandphysicalnetworkdomains.

TheT-NOVAsystemshouldbeabletomanagealargenetworkinfrastructure.

VIM.10 UC1 VIM Networkserviceandresourcediscovery

TheVIMcontrollerSHOULDprovidemechanismstodiscoverphysicalnetworkresources.

TheOrchestratormustbeawareoftheavailablephysicalnetworkresources.

VIM.11 UC1.1UC2.1UC3.1UC3.2UC3.3UC4

VIM Specificationofperformanceparameters

TheVIMcontrollerSHOULDallowtheinfrastructureconnectivityservicestospecifyperformancerelatedparameters.

T-NOVAsystemshouldsupportahighlevelofcustomisationforthenetworkservice.

VIM.12 VIM Flowentrygeneration

TheVIMcontrollerSHOULDbeabletogenerateandinstallrequiredflowentriesforSDNswitchesforpacketforwardingandNFpolicyenforcement(i.e.,ensuretrafficwilltraverseasetofNFsinthecorrectorder).

RequiredtosupporttheNetworkServicedefinition.

VIM.13 VIM Virtualaddressspaceallocation

TheVIMcontrollerSHOULDbeabletoallocatevirtualaddressspaceforNFgraphs(virtualaddressesofNFsbelongingtodifferentgraphscouldoverlap).

Requiredtosupportisolationamongdifferentvirtualnetworkdomains.

VIM.14 UC4 VIM QoSsupport TheVIMcontrollerSHALLprovidemechanismstosupportQoScontrolovertheITinfrastructure.

Requiredtosupportthespecificperformanceneededbyanetworkservice.

VIM.15 VIM SDNControllerperformance

TheVIMcontrollerSHOULDminimisetheflowsetuptimemaximisingthenumberofflowspersecondthatitcansetup.

Requiredtoprovidearesponsiveconfigurationoftheunderlyinginfrastructure.

VIM.16 VIM VIMControllerRobustness

TheVIMcontrollerSHALLbeablecontendwithcontrolplanefailures(e.g.,viaredundancy)inarobustmanner,andinparticulartoensurepersistenceofnetworkconfigurationincaseofCPbreakdown/resumecycles.

RequiredforresiliencyintheT-NOVAsystem.


©T-NOVAConsortium

103

VIM.17 VIM HypervisorAbstractionandAPI

TheVIMControllerSHALLabstractabasicsubsetofhypervisorcommandsinaunifiedinterfaceandinaPlug-Infashion.Thisincludescommandslikestart,stop,rebootetc.

DifferentHypervisorshavevariousadvantagessuchasfullhardwareemulationorparavirtualisation.InordertogetthebestfunctionalityandthebestperformanceforaVM,differentHypervisorsmustbesupported.

VIM.18 VIM QueryAPIandMonitoring

TheVIMControllerSHALLhaveaQueryAPIthatallowsotherT-NOVAcomponentstoretrievemetrics,configurationandusedhypervisortechnologypercomputenode.

Theorchestratormustbeabletomakethebestdecisionregardingperformance,functionalityandaSLAforthecreationofVMs.Theorchestratorrequiresinformationfromthehypervisorandthecomputeinfrastructureunderitscontroltomakeplacementandmanagementdecisions.

VIM.19 VIM VMPlacementFilters

TheVIMControllerSHOULDofferasetofmechanismstoachieveappropriateplatformplacementdecisionsforVNFdeployments.

SomerequirementssetbytheorchestratordoneedamorespecificplacementoftheVM.E.g.aCPUcorefiltercanbeappliedtotheschedulersothattheVMisonlyplacedonacomputenode,ifmorethan3CPUcoresareavailable.

VIM.20 VIM Base-ImageRepositoryintegration

TheVIMControllerSHALLhaveanintegration-moduletointeractdirectlywiththenon-configuredVMimagesthatneedtobedeployed.

ThecomputecontrollermusthaveaccesstotherepositorywiththebasicVMimages.Thosebaseimageswillbedeployedonthedesiredflavourrequestedbytheorchestratorregardingconfiguration,diskspace,CPUandmemory.

VIM.21 UC4 VIM VirtualisedInfrastructureMetrics

TheVIMSHALLcollectperformanceandutilisationmetricsfromthevirtualisedresourcesintheNFVIandreportthisdatainraworprocessedformatsviaanorthboundinterfacetotheOrchestrator.

TheOrchestratorneedsdatatomakescalingdecisionsonVNFsmakingupagivenservice,basedonactingSLAcriteria.

B.2 WANInfrastructureConnectionManagerRequirements

Table6.13:IVMRequirements–WICM

Req.Id. T-NOVA UseCaseAlignment



WICM.1 UC2,UC3,UC6 WANManagement

Customer trafficpathsteering

The WICM SHOULD be able to steer the customer trafficacross the network infrastructure on an end-to-end basis,takingappropriateactionsasa resultof specificevents (e.g.subscription/ unsubscription of VNF services). Selectedmetrics and performance targets (e.g. latencyminimization,networkresourceconsumptionminimization,loadbalancing)may also be taken into account in the determination of the

TheconnectivitycomponentoftheT-NOVAservicemayhavetobe reconfiguredasa resultsofanevent related to theT-NOVA service. Events like subscription, unsubscription orreconfiguration of VNF services may imply modifications intheend-to-endpathofthecustomertraffic


©T-NOVAConsortium

104

end-to-endpath.

WICM.2 UC2,UC3,UC6 WANManagement

CompatibilitywithWANtechnologies

The WICM SHOULD support current WAN connectivityservices.

ItisexpectedthatcurrentWANconnectivityservicessuchasL2/L3VPNswilldominatefortheforeseeablefuture.

WICM.3 UC2,UC3 WANManagement

Rapid servicereconfiguration

The WICM SHOULD enable reconfiguration of WANconnectivity services in compliance with the expectedperformanceenvelopeoftheinfrastructure.

Subscription/unsubscription of services is expected to takeeffectinnearrealtime.


QoScontrol The WICM SHOULD control enforcement of QoS policiesassociatedwithagivencustomerprofile.

T-NOVA service is supposed to provide differentiated SLAlevels


Customer IDmapping

The WICM SHALL be able to map customer’s WAN virtualcircuit ID (e.g. VLAN ID,MPLS label) into the correspondingNFVI-PoPinternalvirtualnetworkID.

Consistentidentificationofcustomer/tenantsacrossdifferentnetwork domains is required to guarantee end-to-endconnectivity.


Northboundinterface

TheWICMSHOULDexposeanorthboundinterfacetotheT-NOVAorchestrationlayer.

Control of connectivity at WAN level should be consistentwithwholeNFVresourcecontrolprocess.

WICM.7 WANManagement

WANMetrics The WICM SHOULD collect metrics such as allocatedbandwidth,availablebandwidthetc.foreachWANlinkunderits control and SHALL expose these metric the T-NOVAOrchestrationlayerviaitsNorthboundinterface.

ThesemetricsarerequiredtosupportvisualizationNFVI-PoPinterconnects within Orchestrator Management UI. Thesemetrics are also required as an input into the resourcemappingalgorithm

B.3 NFVInfrastructureRequirements

B.3.1 Computing

Table6.14:IVMrequirements–Computing

Req.id T-NOVAUseCaseAlignment



C.1 UC2 Compute Nested/Extended HardwarepagevirtualisationSHALLbeutilisedtoimproveperformance.

PerformancebenefitsfromhardwarepagevirtualisationaretiedtotheprevalenceofVMexittransitions.CPUshouldhavelargeTLBs


©T-NOVAConsortium

105

C.2 UC2,UC3 Compute CentralStorage Acentralstoragesubsystem(SAN)SHALLbeavailableinordertoenableadvancedfunctionalitieslikeVMmigrationandserverclustering

Requiredtosupportusecases3/3.1/3.2.Alsorequiredforsystemresilience.

C.3 UC4 Compute NoSPOF AllhardwarecomponentsSHALLbedeployedwithproperredundancymechanisms(e.g.redundantSANswitchesandnetworkswitches)inordertoavoidsinglepointsoffailure

Requiredtosupportusecases3/3.1/3.2.Alsorequiredforsystemresilience.

C.4 UC4.1 Compute Performance AllhardwarecomponentsSHALLsatisfyspecifiedperformanceparameters(e.g.IOPSandR/Woperationratioincaseofstorageresources)inordertoproviderequiredperformancelevels

RequiredtoguaranteeproperSLAs

C.5 UC2 Compute Hypervisorcompatibility

ServersandstorageSHALLbecompatiblewiththechosenhypervisor(s)

Requiredtoensurebasicsystemfunctionalityandreliability.

C.6 UC2,UC3 Compute CentralStorage-efficiency

CentralstorageSHALLsupportfunctionalitieslikeAutomaticStorageTiering(AST),thinprovisioninganddeduplication,inordertoreducecosts,improveefficiencyandperformance

RequiredtosupportSLA’sassociatedwithVNFservices.

C.7 UC4 Compute ComputeDomainMetrics

ThecomputedomainSHALLprovidemetricsandstatisticsrelatingtothecapacity,capabilityandutilisationofhardwareresources:

• CPUcores• Memory• IO(includingaccelerators)• Storagesubsystem

Thesemetricshallincludebothstaticanddynamicmetrics

ThisinformationisrequiredattheOrchestrationlayertomakedecisionsabouttheplacementofnewVNF,services,tomanageexistingservicestoensureSLAcomplianceandtoensuresystemreliability.

C.8 UC3 Compute PowerManagement

ThecomputedomainSHOULDprovidepowermanagementfunctionsthatenabletheVIMtoremotelycontrolthepowerstateofthedomain

ThiscapabilitymayberequiredtomeetSLArequirementsonenergyutilisation,servicecostsortimeofdayservicesettings

(Non-functionalrequirement)

C.9 UC2,UC3 Compute HardwareAccelerators

ThecomputedomainSHALLsupportdiscoveryofhardware(HW)/functionalaccelerators

CertainVNFfunctionsmayrequireorexperienceperformancebenefitsfromtheavailabilityofco-processorcardssuchasFPGA’s,MIC(e.g.XEONPHI)orGPU’s(e.g.Nvida).TheOrchestratorshouldbeawareofthesecapabilitiestoensurecorrectplacementdecisions.

C.10 UC2,UC3 Compute HardwareAcceleratorVisibility

AllHWacceleratorsSHOULDbeabletoexposetheirresourcestotheVIMControllers.

TheOrchestratorshouldbeawareofacceleratorcapabilitiesavailablewithinanNFVI-PoPforplacementofVNF’sthatcanutilisethesecapabilitiestoimprovetheirperformance.


©T-NOVAConsortium

106

C.11 UC2,UC3 Compute HardwareAcceleratorVirtualisation

HWacceleratorresourcesMAYbevirtualisableandthisfeatureSHOULDbemadeavailabletothehostprocessor.

TypicallyacceleratorHWisnotvirtualisablewiththeexceptionofGPUs.Virtualisingtheacceleratorcanprovideallocationandschedulingflexibility

C.12 UC4 Compute HardwareAcceleratorMetrics

HWacceleratorsSHALLprovideperformancemetricstotheVIM.

Necessarytomeasureperformance,guaranteeSLAsanddeterminelimitsforscalingupanddowntheserviceifnecessary.


©T-NOVAConsortium

107

B.3.2 Hypervisor

Table6.15:IVMRequirements–Hypervisor

Req.id T-NOVA

UseCaseAlignment



H.1 UC3

UC4

Hypervisor ComputeDomainMetrics

TheHypervisorSHALLprovidemetricsincludingstatusinformationonvirtualcomputeresources.

TheOrchestratorrequirestheinformationfromthecomputedomaintomakedecisionsregardtheinstantiationofnewVNFsandtheplacementofnewvirtualisationcontainersfornewinstancesofVNFcomponents.MetricsarealsorequiredtoadjustingexistingservicesinordertomaintainSLA’s.MeasurementscollectionisalsoperformedwithinthePerformanceMeasurementfunctionprovidedbytheManagementandOrchestrationfunctionalblock.

H.2 UC3,

UC4

Hypervisor NetworkDomainMetrics

ThehypervisorSHALLprovidemetricsandstatisticsonnetworkingarrangedbyportforeachvirtualnetworkdeviceconfigured.

TheOrchestratorrequirestheinformationfromthenetworkdomaintomakedecisionsregardingtheplacementofnewVNFservicesandadjustexistingservicestomaintainSLA’s(seeabove)

H.3 UC3 Hypervisor VMPortability TheT-NOVAhypervisorSHALLbeabletounbindtheVMfromthehardwareinordertoallowtheVMtobemigratedtoadifferentphysicalresource.

RequiredtoensurethatVNFsarefullyportableintheT-NOVAsystemsforsupportingvariousconditionssuchasscaling,resilience,maintenanceetc.

H.4 UC3 Hypervisor VM–LifecycleManagement

TheT-NOVAhypervisorSHALLsupportinstructionstoprovision,rescale,suspend,terminateonVMscurrentlydeployedorreceivedviaaHypervisor-VIMinterface

ThisisafundamentalrequirementinordertoperformlifecyclemanagementofnetworkservicesVNFs.

H.5 UC2

UC3

Hypervisor PlatformFeaturesAwareness/Exposure

ThehypervisorSHOULDbeabletodiscovertheexistenceoffeaturesandfunctionalityprovidedbyresourcessuchasthecompute,accelerators,storageandnetworkingandtoexposethesefeaturestotheOrchestratorviatheVIM.

EnhancedplatformawarenessbytheHypervisorandmakingthisinformationavailabletotheOrchestratorwillallowtheOrchestratortomakeappropriateplacementdecisions,basedontherequiredcapabilitiesfortheinstantiationofVNFCservicesinstances.

H.6 UC2 Hypervisor VMLowPowerState

ThehypervisorSHALLhavetheabilitytoputresourcesintoalowerpowerstatebasedonutilisation/SLArequirementstoexposealowerpowerstatetotheOrchestrator.

ThiscapabilitymayberequiredtomeetSLArequirementsonenergyutilisation,servicecostsortimeofdayservicesettings.

H.7 UC2 Hypervisor RequestResultsInformation

ThehypervisorSHALLmakeavailabletotheVIMthestatusofexecutedrequests.

RequiredsotheVIMandOrchestratorcanmaintainaconsistentviewoftheinfrastructureresources.


©T-NOVAConsortium

108

UC6

H.8 UC2,

UC4

Hypervisor Performance–Resourceovercommit

ThehypervisorSHALLbeabletoprovidemechanismstoavoidresourceovercommit.

RequiredtoimproveperformanceandguaranteetherequestedSLAs.

H.9 UC4 Hypervisor Alarm/ErrorPublishing

ThehypervisorSHALLpublishalarmsorerroreventsviatheVIM.

TheOrchestratorrequiresthisinformationtoactonspecificevents.ThisispartoftheFaultManagementfunctionthatshallbeprovidedbytheManagementandOrchestrationfunctionalblock.

H.10 UC2

UC3

Hypervisor Security ThehypervisorSHALLbeabletoguaranteeresource(instruction,memory,deviceaccess,network,storage)isolationinordertoguaranteetheperformanceoftheVMsusingtheseresources.

Necessarytoensurethatnetworkservicesdoesn’tinterferewitheachotheranddoesn’timpactonperformanceorreliabilityofotherservices.

H.11 UC2

UC3

Hypervisor Network ThehypervisorSHALLbeabletocontrolnetworkresourceswithintheVMhostandprovidebasicinter-VMtrafficswitching.

ThisisrequiredtoallowproperVNFcommunicationforVNFsthatareinstantiatedwithinthesamecomputenode.

B.3.3 Networking

Table6.16:IVMRequirements–Networking

Req.id Alignment Domain RequirementName RequirementDescription JustificationofRequirement

N.1 UC1-UC6 Networking Switching ThephysicalandvirtualnetworkingcomponentsoftheT-NOVANFVIPoPSHALLsupportL2andL3connectivity Mandatoryrequirement

N.2 UC1-UC6 Networking Virtualisation NetworkingdevicesoftheT-NOVAIVMSHOULDhavetheabilitytosupportvirtualisednetworkoverlays RequiredtosupportscalabilitywithintheT-NOVAsystem.

N.3 UC1-UC6 Networking QoSconfigurationNetworkingcomponentsΜΑΥallowtheconfigurationofspecificqualityofservice(QoS)parameterssuchasthroughput,servicedifferentiationandpacketloss.

Allow the connectivity between VNFCwith specific networkQoSattributes

N.4 UC1-UC6 Networking TunnellingThenetworkingdevicesoftheT-NOVANFVIPoPΜΑΥsupportthecreationofmultipledistinctbroadcastdomains(VLANs)throughoneormoretunnellingprotocols(e.g.STT,

ThisisarequirementforthedeploymentoftenantnetworksacrossmultiplePoPsandtheisolation.


©T-NOVAConsortium

109

NVGRE,VxLAN)toallowthecreationofvirtualL2networksinterconnectedwithinL3networks(L2overL3).

N.5 UC1-UC6 Networking UsagemonitoringThenetworkingdevicesoftheT-NOVANFVIPoPSHOULDprovidemonitoringmechanismsthroughcommonlyusedAPIs(i.e.SNMP).

Required for auditing, monitoring, trouble-shooting,events/alarmsdetection,andliveoptimisation

N.6 UC1-UC6 Networking ConfigurationThenetworkingdevicesoftheT-NOVANFVIPoPSHOULDallowconfigurationthroughcommontechnologiesandprotocolssuchasNETCONF.

Requiredfor(remote)configurationaccess.

N.7 UC1-UC6 Networking SDNL2physicalandvirtualnetworkingdevicesoftheT-NOVANFVI-PoPSHOULDsupportthedynamicreconfigurationofthenetwork.

Required toallow thedynamicconfigurationof thenetworkat runtime. This requirement could potentially be fulfilledusinganSDNapproach.

N.8 UC1-UC6 Networking SDNManagement L2networkingdevicesoftheT-NOVANFVIPoPSHOULDbemanagedbytheVIM. Requiredtoensurethecontrolofthenetworkcomponents.

N.9 UC1-UC6 Networking Networkslicing

NetworkingdevicesintheT-NOVANFVIPoPSHOULDallowprogrammabilityoftheirforwardingtables.EachflowSHOULDbehandledandconfiguredseparatelytoenablenetworkslicing.

The VIM should be able to create network slices composedwithdifferentnetworkingdevices,whicharethenconfiguredindependently.

N.10 UC1-UC6 Networking ScalabilityTheNFVIPoPSHOULDbeabletosupportalargenumberofconnectedservers,whichinturn,SHOULDbeabletohostalargenumberofconcurrentVMs.

Requiredtosupportscalabilityandmulti-tenancy.

N.11 UC1-UC6 NetworkingL2Addressspaceandtrafficisolation

ForL2services,theinfrastructurenetworkoftheT-NOVANFVIPoPMUSTprovidetrafficandaddressspaceisolationbetweenvirtualnetworks.

Required to ensure the correct function ofmulti-tenancy intheT-NOVAsystem.

N.12 UC1-UC6 NetworkingL3Addressspaceandtrafficisolation

ForL3services,theinfrastructurenetworkoftheT-NOVANFVIPoPMUSTprovidetrafficisolationbetweenvirtualnetworks.Ifaddressisolationisalsorequireditcanbeachievedusingvarioustechniques:

• Anencapsulationmethodtoprovideoverlaynetworks(L2orL3service).

• Theuseofforwardingtablepartitioningmechanisms(L2service).

• Byapplyingpolicycontrolwithintheinfrastructurenetwork(L3service).

Required to ensure the correct function ofmulti-tenancy intheT-NOVAsystem.

N.13 UC1-6 Networking Trafficsteering ThenetworktrafficbetweentheVMsSHOULDbeforwardedthoughtherequiredpath, RequiredtosupportServiceFunctionChaininginT-NOVA


©T-NOVAConsortium

110

N.14 UC2,UC3 Networking Dataplaneencapsulation

TheNFVI-GWSHOULDhandletraffictransitingto/comingfromtheWAN,accordingtotransporttechnologiescommonlydeployedinWANdomains(e.g.IEEE802.1q,MPLS,Q-in-Q).

Encapsulationmethodsusedtoaggregatemulti-tenanttrafficshould be supported by the NFVI-GW to guaranteeinteroperabilitywithWAN.

N.15 UC2,UC3 Networking QoScontrol TheNFVI-GWshouldbeabletosupportQoSdifferentiationinthedataplane.

T-NOVAserviceshouldprovidedifferentSLAlevels.


©T-NOVAConsortium

111

ANNEXC-TERMINOLOGY

This annex contains general terms used throughout the deliverable in associationwith allmainT-NOVAarchitecturalentities.

Thetermsmarkedwithanasterisk(*)havebeenalignedwithETSINFVISGterminology[22].

C.1 GeneralTerms

Table6.17:GeneralTerms

Name Description

VirtualisedNetworkFunction(VNF)*

Avirtualised(puresoftware-based)versionofanetworkfunction.

VirtualisedNetworkFunctionComponent(VNFC)*

Anindependentlymanageableandvirtualisedcomponent(e.g.aseparateVM)oftheVNF.

T-NOVANetworkService(NS)

Anetworkconnectivityserviceenrichedwithin-networkVNFs,asprovidedbytheT-NOVAarchitecture.

NFVInfrastructure(NFVI)* ThetotalityofallhardwareandsoftwarecomponentswhichbuilduptheenvironmentinwhichVNFsaredeployed.

C.2 OrchestrationDomain

Table6.18:OrchestrationDomainTerminology

Name Description

Orchestrator*The highest-level infrastructure management entity whichorchestratesnetworkandITmanagemententitiesinordertocomposeandprovisionanend-to-endT-NOVAservice.

ResourcesOrchestrator*

The Orchestrator functional entity which interacts with theinfrastructure management plane in order to manage andmonitortheITandNetworkresourcesassignedtoaT-NOVAservice.

NSOrchestrator*

The Orchestrator functional entity in charge of the NSlifecyclemanagement(i.e.on-boarding,instantiation,scaling,update, termination)which coordinates all other entities inordertoestablishandmanageaT-NOVAservice.

VNFManager*TheOrchestrator functionalentity inchargeofVNF lifecyclemanagement (i.e. installation, instantiation, allocation andrelocationofresources,scaling,termination).

NSCatalogue* TheOrchestratorentitywhichprovidesarepositoryofallthedescriptorsrelatedtoavailableT-NOVAservices


©T-NOVAConsortium

112

VNFCatalogue* TheOrchestratorentitywhichprovidesarepositorywiththedescriptorsofallavailableVNFPackages.

NS&VNFInstancesRecord*

The Orchestrator entity which provides a repository withinformation on all established T-NOVA services in terms ofVNF instances (i.e. VNF records) and NS instances (i.e. NSrecords).

NFStore TheT-NOVArepositoryholdingtheimagesandthemetadataofallavailableVNFs/VNFCs.

C.3 IVMDomain

Table6.19:IVMDomainTerminology

Name Description

Virtualised InfrastructureManagement(VIM)*

The management entity which manages the virtualised(intra-NFVI-PoP) infrastructure based on instructionsreceivedfromtheOrchestrator.

WANInfrastructureConnectionManager(WICM)

The management entity which manages the transportnetwork for interconnecting service endpoints and NFVI-PoPs,e.g.geographicallydispersedDCs.

VNFManagerAgent* The VIM functional entity which interfaces with theOrchestratortoexposeVNFmanagementcapabilities

OrchestratorAgent* The VIM/WICM functional entity which interfaces with theOrchestratortoexposeresourcemanagementcapabilities.

HypervisorController* TheVIMfunctionalentitywhichcontrolstheVIMHypervisorsforVMinstantiationandmanagement.

ComputeController* The VIM functional entity which manages both physicalresourcesandvirtualisedcomputenodes.

NetworkControllerVIM* The VIM functional entity which instantiates and managesthe virtual networks within the NFVI-PoP, as well as trafficsteering.


©T-NOVAConsortium

113

LISTOFACRONYMS

Acronym Description

ACPI AdvancedConfigurationandPowerInterface

API ApplicationProgrammingInterface

AST AutomaticStorageTiering

BSS BusinessSupportingSystem

CAM Control,AdministrationandMonitoring

CAPEX CapitalExpenditure

CLC CloudController

CLI CommandLineInterface

CP ControlPlane

CPU ControlProcessingUnit

D2.1 DeliverableD2.1



DC DataCentre

DCN DataCentreNetwork

DMC DOVEManagementConsole

DOVE DistributedOverlayVirtualEthernet

DP DataPlane

DPDK DataPlaneDevelopmentKit

DPI DeepPacketInspection

E2E End-to-End

EG ExpertsGroup

EM ElementManager

EN EuropeanNorm

EPT ExtendedPageTables

ETSI EuropeanTelecommunicationsStandardsInstitute

EU EndUser

EVB EdgeVirtualBridge

FE FunctionalEntity

FN FutureNetworks


©T-NOVAConsortium

114

FP FunctionProvider

FPGA FieldProgrammableGateArray

GS GlobalStandard

GPU GraphicalProcessingUnit

GW Gateway

HG HomeGateway

HW Hardware

I/O Input/Output

IaaS InfrastructureasaService

IEEE InstituteofElectricalandElectronicsEngineer

IETF InternetEngineeringTaskForce

INF Infrastructure

IP InternetProtocol

IP InfrastructureProvider

IPAM IPAddressManagement

IPsec IPsecurity

ISG IndustrySpecificationGroup

ISO InternationalOrganisationforStandardisation

IT InformationTechnology

ITU InternationalTelecommunicationUnion

ITU-T ITUTelecommunicationStandardizationSector

IVM InfrastructureVirtualisationandManagement

KPI KeyParameterIndicator

L2 Layer2

L3 Layer3

LAN LocalAreaNetwork

LINP LogicallyIsolatedNetworkPartition

MAC MdediaAccessControl

MAN MetroAreaNetwork

MANO ManagementandOrchestration

MEF MetroEthernetForum

MIC Multi-IntegratedCores

MPLS MultiprotocolLabelSwitching


©T-NOVAConsortium

115

NaaS NetworkasaService

NC NetworkController

NETCONF NetworkConfigurationProtocol

NF NetworkFunction

NFaaS NetworkFunctions-as-a-Service

NFV NetworkFunctionsVirtualisation

NFVI NetworkFunctionsVirtualisationInfrastructure

NFVIaaS NetworkFunctionVirtualisationInfrastructureas-a-Service

NFVI-PoP NFVI-PointofPresence

NFVO NetworkFunctionVirtualisationOrchestrator

NG-OSS NextGenerationOperationsSupportingSystem

NIC NetworkInterfaceCards

NIP NetworkInfrastructureProvider

NOC NetworkOperatorsCouncil

NS NetworkService

NSD NetworkServiceDescriptor

NV NetworkVirtualisation

NVGRE NetworkVirtualisationusingGenericRoutingEncapsulation

OAN OpenAccessNetwork

OCCI OpenCloudComputingInterface

ONF OpenNetworkingFoundation

OPEX OperationalExpenditure

OPN OpenPlatformforNFV

OS OperatingSystem

OSI OpenSystemsInterconnection

OSS OperationsSupportingSystem

PF PhysicalFunction

PNF PhysicalNetworkFunction

PoC ProofofConcept

PC PersonalComputer

PER Performance&PortabilityBestPractices

QPI QuickPathInterconnect

QoS QualityofService


©T-NOVAConsortium

116

RAM RandomAccessMemory

RAS ReliabilityAvailabilityandServiceability

RESTAPI RepresentationStateTransferAPI

RFC RequestforComments

RPC RemoteProcedureCall

RTP RealTimeProtocol

SAN StorageAreaNetwork

SBC SessionBorderController

SDN Software-DefinedNetworking

SDO StandardsDevelopmentOrganisation

SG13 StudyGroup13

SLA ServiceLevelAgreement

SNMP SimpleNetworkManagementProtocol

SOTA State-Of-The-Art

SP ServiceProvider

SR-IOV SingleRootI/OVirtualisation

SSD Solid-state-disk

STP SpanningTreeProtocol

STT StatelessTransportTunnelling

SW Software

SWA SoftwareArchitecture

ToR TermsofReference

ToR TopofRack

TMF TeleManagementForum

WICM WANInfrastructureConnectionManager

TR TechnicalReport

TS TechnicalStandard

TSC TechnicalSteeringCommittee

T-NOVA NetworkFunctionsas-a-ServiceoverVirtualisedInfrastructures

UC UseCase

UML UnifiedModellingLanguage

VEB VirtualEdgeBridge

VEPA VirtualEthernetPortAggregator


©T-NOVAConsortium

117

VIM VirtualisedInfrastructureManager

VL VirtualLink

VLD VirtualLinkDescriptor

VLAN VirtualLocalAreaNetwork

VM VirtualMachine

VMM VirtualMachineManager

VMX VirtualMachineExtension

VN VirtualNetwork

VNF VirtualNetworkFunction

VNFC VirtualNetworkFunctionComponent

VFND VirtualNetworkFunctionDescriptor

VNFFG VirtualNetworkFunctionFowardingGraph

VNFFGD VirtualNetworkFunctionFowardingGraphDescriptor

VNFM VirtualNetworkFunctionManager

VRF VirtualRoutingandForwarding

VPN VirtualPrivateNetwork

VSAN VirtualStorageAreaNetwork

vNIC VirtualNetworkInterfaceCards

VPN VirtualPrivateNetwork

VTN VirtualTenantNetwork

WAN WideAreaNetwork

WG WorkingGroup

WP WorkPackage

WP WorkingProcedures

XML ExtendedMarkupLanguage


©T-NOVAConsortium

118

REFERENCES

[1] V.Travassos.(2012).VirtualizationTrendsTraceTheirOriginsBacktotheMainframe.Available:http://ibmsystemsmag.com/mainframe/administrator/Virtualization/history_virtualization/

[2] M.Chiosiet.al,"AnIntroduction,Benefits,Enablers,Challenges&CallforAction,"2012,Available:https://portal.etsi.org/nfv/nfv_white_paper.pdf

[3] T-NOVA,"NetworkFunctionsas-a-ServiceoverVirtualisedInfrastructures,"2014,Available:http://www.t-nova.eu/results/

[4] T-NOVA,"OverallSystemArchitectureandInterfaces,"2014.Available:http://www.t-nova.eu/results/

[5] T-NOVA,"SpecificationoftheNetworkFunctionFrameworkandT-NOVAMarketplace,"2014.Available:http://www.t-nova.eu/results/

[6] T-NOVA,"UseSystemCasesandRequirements,"2014.Available:http://www.t-nova.eu/results/

[7] ESTI,(2013).NetworkFunctionsVirtualisation(NFV);VirtualisationRequirements.Available:http://www.etsi.org/deliver/etsi_gs/nfv/001_099/004/01.01.01_60/gs_nfv004v010101p.pdf

[8] ITU-T,"Requirementsofnetworkvirtualizationforfuturenetworks,"ITU-T,2012,Available:https://www.itu.int/rec/T-REC-Y.3011-201201-I/en

[9] ETSI,(2014).NetworkFunctionsVirtualisation(NFV);ManagementandOrchestration.Available:http://www.etsi.org/deliver/etsi_gs/NFV-MAN/001_099/001/01.01.01_60/gs_nfv-man001v010101p.pdf

[10] T-NOVA,"SpecificationoftheNetworkFunctionFrameworkandMarketplace,"2015.Available:http://www.t-nova.eu/results/

[11] K.Ahmad,SourcebookofATMandIPInternetworking:IEEEPress.[12] ETSI,(2014).NetworkFunctionsVirtualisation(NFV);InfrastructureOverview.

Available:http://www.etsi.org/deliver/etsi_gs/NFV-INF/001_099/001/01.01.01_60/gs_nfv-inf001v010101p.pdf

[13] ETSI,(2014).NetworkFunctionsVirtualisation(NFV);Infrastructure;ComputeDomainAvailable:http://www.etsi.org/deliver/etsi_gs/NFV-INF/001_099/003/01.01.01_60/gs_NFV-INF003v010101p.pdf

[14] ETSI,(2015).NetworkFunctionsVirtualisation(NFV);Infrastructure;HypervisorDomain-ETSIGSNFV-INF004V1.1.1Available:http://www.etsi.org/deliver/etsi_gs/NFV-INF/001_099/004/01.01.01_60/gs_nfv-inf004v010101p.pdf

[15] ETSI,(2014).NetworkFunctionsVirtualisation(NFV);InfrastructureNetworkDomain.Available:http://www.etsi.org/deliver/etsi_gs/NFV-INF/001_099/005/01.01.01_60/gs_NFV-INF005v010101p.pdf

[16] ETSI,(2014).NetworkFunctionsVirtualisation(NFV);Infrastructure;MethodologytodescribeInterfacesandAbstractionsAvailable:http://www.etsi.org/deliver/etsi_gs/NFV-INF/001_099/007/01.01.01_60/gs_NFV-INF007v010101p.pdf

[17] ETSI,(2015).NetworkFunctionsVirtualisation(NFV);ResiliencyRequirements.Available:http://www.etsi.org/deliver/etsi_gs/NFV-REL/001_099/001/01.01.01_60/gs_nfv-rel001v010101p.pdf


©T-NOVAConsortium

119

[18] ETSI,(2014).NetworkFunctionsVirtualisation(NFV);VirtualNetworkFunctionsArchitecture.Available:http://www.etsi.org/deliver/etsi_gs/NFV-SWA/001_099/001/01.01.01_60/gs_nfv-swa001v010101p.pdf

[19] ONF,"OpenFlow-enabledTransportSDN,2014.Available:[20] T.Kourlas,(2014).SDNforIP/OpticalTranportNetworks.Available:

http://www.commtechshow.com/east/wp-content/uploads/ALU-SDN-for-IPOptical-Transport-Networks.pdf

[21] U.Hölzle,"OpenFlow@Google,"inOpenNetworingSummit,SantaClara,15-17April,2012.

[22] ETSI,(2013).NetworkFunctionsVirtualisation(NFV);TerminologyforMainConceptsinNFV.Available:http://www.etsi.org/deliver/etsi_gs/NFV/001_099/003/01.01.01_60/gs_nfv003v010101p.pdf

[23] B.Landfeldt,P.Sookavatana,andA.Seneviratne,"Thecaseforahybridpassive/activenetworkmonitoringschemeinthewirelessInternet,"presentedattheIEEEInternationalConferenceonNetworks(ICON2000),Singapore,Malaysia,2000.

d2.32 - specification of the infrastructure virtualisation ... · orchestration and management of...

Documents