extensionofresearchonsecurityasaserviceforvmsin...

Research ArticleExtension of Research on Security as a Service for VMs inIaaS Platform

Xueyuan Yin 1 Xingshu Chen 2 Lin Chen1 and Hui Li1

1College of Computer Science Sichuan University Chengdu 610065 China2Cybersecurity Research Institute Sichuan University Chengdu 610065 China

Correspondence should be addressed to Xingshu Chen chenxshscueducn

Received 31 October 2019 Revised 7 January 2020 Accepted 21 January 2020 Published 22 April 2020

Guest Editor Veljko Milutinovic

Copyright copy 2020 Xueyuan Yin et al is is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

To satisfy security concerns including infrastructure as a service (IaaS) security framework security service access networkanomaly detection and virtual machine (VM) monitoring a layered security framework is built which composes of a physicallayer a virtualization layer and a security management layer en two security service access methods are realized for varioussecurity tools from the perspective of whether security tools generate communication traffic One without generating trafficemploys the VM traffic redirection technology and the other leveraged the mechanism of multitasking process access Moreover astacked LSTM-based network anomaly detection agentless method is proposed which has advantages of a higher ratio ofprecision and recall Finally a Hypervisor-based agentless monitoring method for VMs based on dynamic code injection isproposed which has benefits of high security of the external monitoring method and good context analysis of the internalmonitoring mechanism e experimental results demonstrate the effectiveness of the proposed protection framework and thecorresponding security mechanisms respectively

1 Introduction

To satisfy partial security requirements including securityframework security service access network anomaly de-tection and monitoring for VMs in IaaS environment [1 2]various designs have been proposed

Primarily a comprehensive security framework for theIaaS platform needs to be considered CSA proposes atrusted cloud initiative reference architecture [2 3] whichcontains BOSS ITOS SRM and a hierarchical cloud serviceNIST proposes a role-based cloud architecture [4] whichdefines capabilities and responsibilities of cloud providerscloud users cloud auditors and cloud agents respectivelyENISA issues a security framework for the governmentcloud [5] which abstracts the role elements (including cloudusers cloud owners and cloud providers) and gives a cloudsecurity framework based on the PDCA (Plan Do CheckAct) lifecycle Amazon (AWS) builds a cloud model basedon shared security responsibilities that AWS takes respon-sibility for the common infrastructure and underlying ser-vices and users undertake responsibility for their own

applications and contents Alibaba Tencent and H3C buildcloud protection systems in manners similar to AWSOpenStack and OpenNebula rely on user authentication andcommunication access control to ensure VM security whichlacks flexible mechanisms to access security services Var-adharajan and Tupakula [6] analyse different threats fromtenant administrators tenants cloud provider administra-tors and the Internet based on Eucalyptus IaaS platform andbuilds a security service model by adding security compo-nents at different component levels Lin et al [7] points outthat cloud security needed to rely on trusted roots trustedlinks security management and security evaluation ele-ments Currently many research results focus on abstractingcloud security frameworks which are not conducive toimplement cloud security technologies rapidly

Secondly VM traffic cannot be monitored or blockedbecause the virtual communication channel is logical and itis invisibile of traditional security products To satisfy therequirements VMware NSX [8] implements a softwarefirewall implanted in the physical computing system kernelin a tightly coupled manner Sugon [9] proposes a

HindawiSecurity and Communication NetworksVolume 2020 Article ID 8538519 16 pageshttpsdoiorg10115520208538519

vulnerability scanning tool that encapsulated in a VMHowever a large amount of storage space is needed as thenumber of the VM increases and the characteristic ofmultitasking parallel processing of the vulnerability scan-ning engine cannot be brought into play A security groupfunction implemented in Openstack employs the NetfilterIptables to implement a software virtual firewall which isinconvenient to expand dynamically due to the tight cou-pling implementation Cisco and HP develop new protocolsof VN-TAG and Virtual Ethernet Port Aggregator (VEPA)respectively to pull the VM traffic to the physical access layerswitch to implement VM communication control [10 11]

irdly network anomaly detection is a key mean ofnetwork security protection which could analyse networktraffic agentless Currently network anomaly detectionmethods can be classified into four categories [12] (1)Classification method the method needs security expertwhich provides a known pattern of an attack and designeddetailed features Hamamoto et al [13] proposes a method ofcombining a genetic algorithm and fuzzy logic for networkanomaly detection which uses a genetic algorithm to generatemultiple features and decides whether an event is an anomalyby the fuzzy logic method e results achieve an accuracy of9653 and false positive rate of 056 e classificationmethod could make full use of the detailed characteristics ofnetwork traffic and obtain high accuracy (2) Statisticsmethod the basic idea of the method is to consider a largedeparture of events from normal as anomalies thus profilingnormal network behavior can detect unknown attacks the-oretically Fernandes et al [14] propose two anomaly de-tection mechanisms based on statistical procedure PCA andthe ant colony optimization which profiles normal networkbehaviours and compares with the real network traffic by amodification of the dynamic timewarpingmetric to recognizeanomalous events However statistics methods are notsuitable for some small-scale attacks due to loss of detailedfeatures (3) Information theory information-theoreticmeasures such as entropy could be used to create an anomalydetection model Callegari et al [15] proposes an intrusiondetection method that performs anomaly detection bystudying the variation in the entropy associated with thenetwork traffic e traffic is aggregated through three-di-mensional reversible sketches and computes the entropy ofdifferent traffic descriptors by using several definitions ofentropy Finally detecting network anomaly according to thedeviation of entropy information-theoretic ignores the de-tailed information of traffic which is hard to detect small-scale attacks (4) Clustering method the method does notrequire labelling data and could mine the pattern of dataautomatically An evolution of the Microclustering OutlierDetection (MCOD) machine learning algorithm is proposedby Flanagan et al [16] e method of distance-based outlierdetection and cluster density analysis are combined to detectanomaly in time series which do not need to label dataHowever it is difficult to design distance metrics

Finally monitoring is an essential part of the securityassurance mechanism [1 2] Currently monitoring tech-nologies for the state of a VM can be classified in two cat-egories (1) Internal monitoring method (a software-agent is

run inside a VM) which provides good context analysis forthe state of the VM but malicious code running inside theVM could damage the monitoring system (2) Externalmonitoring method (or Hypervisor-based monitoringmethod) [17] which is realized by employing a Hypervisor ora Virtual Machine Monitor (VMM)e state of a VM can bemonitored through some specific Hypervisor APIs while theretrieved information is limited to the inherent APIs severelywithout opportunities to expand or customize such asproducts of OpenStack Ceilometer andAWSCloudWatch Tomake up for that deficiency the methods based on VirtualMachine Introspection (VMI) have been proposed [18ndash20]e methods always rely on setting a specific hardwaretrapping for internal resource (eg CPU ormemory) of a VMfirstly When the VM accesses the specific resource theoperation will be captured by the VMM immediately enthe introspection mechanism is adopted to monitor the stateof the VM Compared with the internal monitoring methodHypervisor-based method has a higher safety and reliabilitybut it consumes more resources To solve the problem atriggering method based on VMFUNC is proposed [21]which minimizes overhead of VM exits and avoids the systempause caused by programs of VMI

To satisfy abovementioned security concerns a corre-sponding solution and techniques are designed and realizedin the following sections In Section 2 a cloud securityframework compiled the security model of P2DR and theideas of defense-in-depth of IATF and security as a servicewas constructede framework employed a physical layer avirtualization layer and a security management layer InSection 31 to satisfy the requirement of security serviceaccess two access methods were realized for various securitytools by leveraging NV NFV and SDN technologies fromthe perspective of whether the tools generate communica-tion traffic One without generating traffic employed the VMtraffic redirection technology with loose coupling and hadbetter flexibility e other one leveraged the mechanism ofmultitasking process access and the characteristic of mul-titasking parallel processing of partial security engine couldbe brought into play which was helpful to reduce resourceconsumption In Section 32 to ensure network security ofthe VM a LSTM-based network anomaly detection methodwas proposed Based on the periodicity of network traffic 11features were selected to describe traffic structure in a timewindow en multivariate time series consisting of 11features were input to LSTM-NAD for training Finally thefinely trained LSTM-NAD model predicted a binary valuewhich represents whether the current time window wasabnormal or not Compared with paper [1] the LSTM-NADmethod had the advantages of higher AUC score anddetecting anomaly more effectively In Section 33 based onthe characteristics of the internal monitoring method andexternal monitoring mechanism a Hypervisor-basedagentless monitoring method for VMs based on themechanism of dynamic code injection was proposed eproposed mechanism had benefits of high security of theexternal monitoring method that could not be destroyed bymalicious code running inside the VM and had the virtues ofgood context analysis for the state of the VM low resource

2 Security and Communication Networks

overhead and rapid response Compared to the virtuali-zation layer monitoring mechanism that completely reliedon semantic analysis the method had the advantages oflower performance overhead and faster response In Section4 the experimental campaign and discussion about theeffectiveness and performance costs of the proposedframework and the various supporting securities were givenMore complete technical discussions about the frameworkand concrete technologies (eg Encryption VPN VM es-cape detection data backup and Software update) will bediscussed in future work

2 IaaS Protection Framework

To satisfy security requirements of network access controlnetwork anomaly detection and VM monitoring for thephysical and virtual environment of IaaS environment anextensible security protective framework for the IaaS plat-form was built as shown in Figure 1

e framework complies the security model of P2DR(Policy Protection Detection Response) and ideas of de-fense-in-depth of information assurance technical frame-work (IATF) and security as a service Overall theframework consists of a physical layer a virtualization layerand a security management layer e physical layer pro-vides physical CPU memory storage network resources foran IaaS platform en building on top of it the virtuali-zation layer employs Xen KVM Hyper-V LVM SoftwareDefined Networking (SDN) Network Function Virtualiza-tion (NFV) and Network Virtualization (NV) to virtualizephysical resources Moreover both layers satisfy constraintsof the P2DR model

For the physical layer high levels of network isolationbetween different networks within IaaS environment areprovided in the horizontal direction ese segment net-works are composed of a Demilitarized Zone (DMZ) amanagement domain a VM migration domain a storagedomain a tenant business domain and a support domainereinto the DMZ is a secure communication zone be-tween an external network domain and an interior of theIaaS environment e management domain allows thecloud provider tomanage andmonitor resources within IaaSenvironment e reason for isolating both cloud migrationand storage domains are two-fold performance as bothtraffic need very fast data rates and security by fully iso-lating this network Tenant business domain hosts virtualresources encapsulated in a VM formate support domainis consisted of two subdomains one for the physical layerand the other for the virtualization layer For the one forphysical layer security measures (eg IPS IDS traditionalphysical firewalls and antivirus gateways) can be deployedin appropriate locations to protect the physical environment

e virtualization layer contains a tenant business domainand a security support subdomain for the virtualization en-vironment (1) For the tenant business domain from theoutside to the inside of the tenant VM a three-layered iso-lation network perimeter is organized (L1) Tenant domain(TD) perimeter (L2) Tenant subdomain perimeter (L3) VMperimeter (2) For the security support subdomain some

measures are orchestrated to eliminate partial security risksincluding resource access control communication controlnetwork abnormality detection and VM monitoring For L1and L2 capacities of managing tenant VMs communicationaccess control network abnormal detection and deep packetinspection can be provided For L3 services of VM moni-toring and file antivirus for VMs can be provided [1]

Based on the abovementioned discussion and combinedmechanisms of user identity management anomaly detec-tion vulnerability detection monitoring data backupemergency response etc a dynamic security protectionframework can be built as shown in Figure 2

3 Security Service Implementations

31 Security Service Access Currently most of security toolssuch as the firewall and vulnerability scanning can beroughly divided into two types from the perspective ofwhether security service tools generate communicationtraffic (1) Type 1 the tool does not generate traffic and onlyprocesses and forwards the received traffic (eg the firewallfilters the traffic passed through) (2) Type 2 the toolgenerates a communication flow and determines the securitystatus based on the reply information (eg vulnerabilityscanning and port scanning tool) To solve the problem ofthe security tool is difficult to access the tenant virtualnetwork dynamically the following concerns should beconsidered (1) the implementation form of security servicetools (2) e way to send a VM traffic to a security tool orthe way to deliver a communication flow from a securityservice tool to a tenant network

For concern (1) the NFV technology is employed torealize security appliances (eg choose a lightweightVM as a carrier for a service of network anomalydetection) Moreover by integrating multiple securityservice tools in one security appliance the securityservice property of ldquoEncapsulate multiples into onerdquocan be realizedFor concern (2)

(a) When a VM traffic needs to be sent to a securityappliance (definition in Type 1) the security ap-pliance should be instantiated in the tenant subnetthus the network of Layer 2 could be reachablebetween them After that a software switch sup-porting the OpenFlow protocol is deployed at theVM network interface which could redirect apacket from a specific VM to a specific securityappliance by modifying the native destinationMAC address in the packet to the MAC address ofthe security appliance according the security policyen the specific traffic can be processed or fil-tered as shown in Figure 3 Besides compared withthe way of instantiating a security appliance directlyinto a compute node in a tenant business domainby instantiating it into a separate support domainwhich has no effect on computing resources forVMs and is more conducive to improve the se-curity service capability

Security and Communication Networks 3

Detection

Protection

(1)(2)(3)(4)(5)(6)(7)(8)

Identity and access managementEncryptionNetwork isolationTenant domainVPNFirewallMonitoringData backup

Response

Response

Policy

Vulnerability detection Network anomaly detectionIntrusion detectionVirus detection

(1)(2)(3)(4)

Emergency responseSystem restorationData recovery

(1)(2)(3)

Protectio

n

Detection

Figure 2 Security protections for IaaS platform complied with the P2DR model

VM2

VM1

SDNcontroller

Securityappliances

(1) Add related flow entries into vSwitches to redirect specific traffic

(2) The traffic flows into vSwitch

(3) Redirect the matched traffic to security appliances

(5) Send the processed trafficto destination

helliphellipSDN vSwitch

SDN vSwitch

(4) Forward the processed traffic to destination

Figure 3 VM traffic redirection mechanism

Tenant business domain

SD for the virtualization layer

Tenant domain

Tenant domainSubdomain

Management domain

Supportdomain

(SD)

DMZ

SD for the physical layer

Storage domainMigration domain

IaaS environment

VMApp +dataTraffic flow

L1

Virtuali

zation lay

er

Physical

layer

L2 L3Policy

Protect

ion

Response

Detection

Figure 1 e layered IaaS protection framework model


(b) When a communication flow from a security ap-pliance (definition in Type 2) is needed to be de-livered to a tenant network actively existing partialapproaches based on the VM encapsulation un-derutilizes the capacity of multitasking parallelprocessing of the security engine and result inexcessive resource consumption

To solve the problem a security service access methodbased on multitask process and SDN is proposed as shownin Figure 4 By providing an independent security servicenode and constructing virtual network links dynamicallywith SDN technology a single security appliance could servemultiple tenants simultaneously and the security serviceproperty of ldquoone serves more than onerdquo can be realized

Take a security appliance as a vulnerability scanning asan example to explain the principles of the type 2 securityservice access method

(1) According to a security policy an operation ofvulnerability scanning for VM-A is launched Afterreceiving the specific instruction the security servicecreates a task running space for the security serviceengine and the security task process

(2) A communication link is established between thetask running space and the virtual network device bycreating virtual ports and virtual links Furthermorethe routing is set according to the IP address in-formation of the VM to be scanned and the com-munication isolation is implemented by combiningthe tenant layer network tag of the tenant domain(a) Virtual ports and links are implemented based onVETH devices One end of the virtual link is boundto the virtual switch and the other end is bound tothe task running space through the task interface Atthe same time the security task running space isreachable to the target VM network by configuringthe IP address of the task interface and the routinginformation (b) Communication isolation betweendifferent security task processes is implementedbased on SDN technology e network space of thetask process is connected to the virtual switch theLayer 2 network identifier of the tenant domain suchas VLAN TAG is combined to tag the securityservice traffic and delivered into the target VMnetwork At the same time it guarantees the Layer 2network isolation between different security taskprocesses (c) In a virtual network environmentVMs in different tenant domains may be configuredwith the same IP address To ensure that differentsecurity task traffic could be delivered to a correcttarget network virtual IP addresses are set for thesecurity task process and the target VMe detailedprocess is illustrated as follows

e key parameters are the real MAC and IP informationof VMA is (vmmac vmip) and the virtual address infor-mation mapped to the task running space is (virmac virip)e virtual object builds on the VETH device and bounds tothe task running space is named vport whose address

information is (vportmac vportip) e gateway MACaddress of the virtual network where VMA located isgwmac

(1) e security task process sends packets

S1 the security task process encapsulates the se-curity task traffic according to the address infor-mation configured by the local virtual networkinterface vport and the virtual address informationof the target VM Select a reserved address in thenetwork address as the virtual address resourcepool for vportip and virip At this point the basicinformation of the packet is

(SRC (vportmacvportip)DST (virmacvirip)TAG null)S2 after the data packet reaches at the virtualswitch the security task process data packet istagged according to the VMA VLAN TAG and thedestination address is modified back to the realtarget VM address At this point the basic infor-mation of the packet is(SRC (vportmacvportip) DST (vmmacvm

ip)TAG ID)S3 after receiving the security service traffic withthe destination address of the VMA and the VLANtagged equal to VMA VLAN TAG the VMA accesslayer virtual switch removes the tag and sends it tothe VMA At this point the basic information of thepacket is(SRC (vportmacvportip) DST (vmmacvm

ip)TAG null)

(2) e security task process receives packets

R1 the VMA encapsulates the reply informationaccording to the local address and the saved se-curity task process address Since the task processand the VM are in different networks the gatewayaddress is used to reply to the packet e basicinformation of the reply packet is(SRC (vmmacvmip)DST (gwmacvportip)

TAG null)R2 after the packet arrives at the virtual switch tagthe traffic that the destination IP address matchedto the security task process and modify the desti-nation MAC address to redirect to the security taskprocess interface vport At this point the basicinformation of the packet is(SRC (vmmacvmip) DST (vportmacvport

ip)TAG ID)R3 after the packet arrives at the virtual switch thatthe security task process is connected modify thesource IP and source MAC of the traffic flow thatdestination IP address is equal to the address of thesecurity task process back to the virtual IP addressand virtual MAC address At this point the basicinformation of the packet is(SRC (virmacvirip) DST (vportmacvport

ip)TAG null)


By employing the method the characteristic of multi-tasking parallel processing of partial security engine can bebrought into play which is helpful to reduce resourceconsumption that the storage required by the vulnerabilityscan engine and plugins which does not increase as thenumber of security service increased

32 Network Anomaly Detection

321 Network Traffic Features Network traffic features arecrucial to the mechanism of network anomaly detectionwhich describe both the normal and abnormal networktraffic patterns Normal network activity of the VM host inthe network causes periodic fluctuation of network trafficthus the static network feature is hard to profile the normalnetwork behavior in a network environment Many featuresare time varying and show strong periodic which are calledas the network traffic dynamic structure features

As shown in Table 1 the 1st to 4th features are the same aspaper [1] which are selected to describe the traffic structurein a particular window Besides the features 5th to 11th areproposed to describe the dynamic network traffic profile of aVMerefore instead of using the combined features in thetime window original features are adopted which com-pletely contain all the original information and are beneficialto the advanced characteristics of automatic learning of therecurrent neural network

e number of VMs in cloud environment is very largewhich means that the network traffic is huge and requiresthat the features we designed must be easy to collect theamount of calculation is small and the payload is invisible

due to encrypted traffic and user privacy restrictions As anexample the network traffic of a web server in a cloud datacenter is captured and 11 features in each time windows (whichset as 1 minute) are calculated e statistical results of theabovementioned 11 features in 7 days (total of 10080 timewindows) are shown in Figure 5 most of the features showstrong periodicity such as bps and pps It is worth noting thatsome features do not exhibit synchronization such as syn_rateand bytes_per_ip which are almost the opposite trend Addi-tionally some outliers are obvious such as an obvious anomalyin syn_count However due to the powerful predictive ability ofthe recurrent neural network these obvious outliers do not needto be processed and the network will learn those abnormalpatterns

322 LSTM-NAD LSTM-Based Network AnomalyDetection F is defined as a set of network behavior con-sisting of 11 features

F access_port ip_entro bps pps packet_per_ipip_count service_port_entro syn_rate syn_per_ip byte-s_per_ip syn_count

Given a fi isin F multivariate time series are assumed asX x(1) x(2) x(n)1113864 1113865 and x(t) was an m-dimensionalvector x

(t)1 x

(t)2 x(t)

m1113966 1113967 where each point x(t)i denotes

value of fi between t windowsLong short-term memory (LSTM) network can capture

the temporal structure of the sequence which is very suitablefor network traffic time series of the VM e LSTM is avariant of RNN that is capable of learning long term

VMA

SDN virtual switch

Physicalvirtualnetwork devices

Physicalvirtual network devices

Virtual network access component

Security task processVM

In the same subnet environment (layer 2 network logical channel with the TIDi tag)

Security task process

Security service node

In a samesubnet

Communication flow redirection control based

on OpenFlow protocol

Security service engine

SDN virtual switch

Task running spacevport

ARP table routing table

S1

S2

S3

R1

R2

R3

Tenantdomain

Tenant data network channel Security service network channel

Management channelSecurity service management

Figure 4 Type 2 security service access model


Table 1 Description of 11 traffic structure features

ID Feature name Description1 access_port e entropy of port count accessed by each client IP and corresponding probability of occurrence2 ip_entro Entropy with proportion of every communicated IP3 service_port_entro e entropy of access service port4 syn_rate e proportion of packets with SYN flag in all packets5 bps Bytes per seconds of VM6 pps Packets per seconds of VM7 packet_per_ip Average packet counts of every IP8 ip_count Communicate IP count between every time window9 syn_per_ip e count of packets with SYN flag per IP10 bytes_per_ip Average bytes per IP11 syn_count Count of packet with SYN flag

00020406081012

access_port

202530354045505560

ip_entro

00020406081012

bpstimes108

times103

times104

times104

times104

times107

0002040608101214

pps

00

02

04

06

08packet_per_ip

0010203040506070

ip_count

000000250050007501000125015001750200

service_port_entro

000002004006008010012014

syn_rate

075100125150175200225250

syn_per_ip

000510152025

syn_count

000025050075100125150175200

bytes_per_ip

Figure 5 Total 11 features of network traffic


dependencies which solves the vanishing gradients problemeffectively and is the most widely used type of RNN eformulas for a single LSTM block are given below [22]

it fi Wxixt + Whihtminus1 + bi( 1113857

zt fz Wxzxt + Whzhtminus1 + bz( 1113857

ct it ⊙ kt + zt ⊙ ctminus1

ot fo Wxoxt + Whohtminus1 + bo( 1113857

ht ot ⊙fh ct( 1113857

(1)

where kt is an input it is an input gate which defines a newlycomputed state for kt to be permitted pass through zt is aforget gate which defines the previous state that would bepermitted pass through ct is a cell state which combines theprevious memory ctminus1 and the new input it ⊙ kt ot denotesan output gate ht is a hidden state ⊙ is an elementwisemultiplication and f is an activation function which issigmoid or tanh function usually

erefore the LSTM network is introduced as a strongclassifier to decide whether the current time window isabnormal in the network traffic detection system Comparedwith traditional machine learning algorithms LSTM doesnot need to extract features elaborately which achieves end-to-end anomaly detection

In order to improve the ability of LSTM a stacked LSTMnetwork framework called ldquoLSTM-NADrdquo was designed andthe framework is shown in Figure 6 e input shape of thenetwork is an array which shapes like ltB T Fgt B representsthe batch size T represents the time step and F representsthe feature size en 2 stacked LSTM are applied to theinput layer and LSTM units in a hidden layer are fullyconnected through recurrent connections e first LSTMlayer had 64 hidden units which outputs the result of eachtime step and the second has 32 hidden units which onlyoutputs value of the last time step Finally 2 fully connectedlayers are followed by LSTM layers the first layer has 64units and the second has 32 units and the last layer in thenetwork is a fully connected layer in which the hidden unitsize is 1 and activation function is the sigmoid function

In multivariate time series the current time window isdependent on the previous windows so the sliding windowis used to generate training sequence data To improve thetraining effect we also shuffle the training data Since thetraining samples are extremely unbalanced the number ofabnormal points is very small compared to the number ofnormal samples Some data enhancement and oversamplingmethods are adopted to make the number of positive andnegative samples in the training samples roughly equalFinally some dropout layers are added to the fully connectedlayer in order to avoid overfitting

33 VM Monitoring A Hypervisor-based agentless moni-toring method for VMs based on the mechanism of dynamiccode injection is proposed as shown in Figure 7

Take KVM as a VMM to illustrate e framework isconstructed according to the following reasons

(1) ere is a semantic gap between the KVM and theVM e monitoring code injected into the VMthrough KVM runs correctly and the following re-quirements must be satisfied (a) obtain the semantics(the kernel symbol table) of the VM and resolve thekernel symbols used by the monitoring code (b)Allocate a kernel memory chunk of the VM (namedmem-A) to store the monitoring code (c) Afterloading the monitoring code into the memory in (b)an operation of relocation repairation is required toperform to make the monitoring code run correctly

(2) It is necessary to construct an execution opportunityfor the injected monitoring code (eg by means ofintercepting system calls of the VM transparently tocreate an executive opportunity the monitoringmechanismwill be activated when a system call eventoccurrence in the VM)

(3) A protection mechanism for the monitoring codealso should be considered Because the monitoringcode is hosted in the kernel space of the VM otherkernel modules of the VM may bypass the moni-toring mechanism (eg by means of (a) hook theSYSENTER_EIP_MSR (MSR) register of the VMthat saves the address of the system call entry (b)Hook the system call table of the VM

e framework contains (1) monitoring code injectionwhich is deployed in the KVM to perform an injection actionfor the monitoring code when a VM (named VM-A) ispowered on andmodifying the monitored system call entry tothe address of corresponding function of the monitoring codetransparently After that the system call execution process ofthe VM-A could be monitored (2) Monitoring protectionwhich is deployed in the KVM to accomplish read-onlyprotection for the injected code the MSR register the systemcall entry function and the system call table After that theycould not be tampered by the malicious code running insideof the VM-A (3) A shared memory (namedmem-S) which isallocated at the first run of the monitoring code and its baseaddress and size will be passed to the KVM by means ofVMCALL of the super system call us the monitoring codewill interact with the KVM through the mem-A to exchangedata (including delivering policies and retrieving results) (4)Monitoring policies which include IDs of the monitoredVMs system call numbers to be monitored and the systemcall response actions (eg record or block an operation ofplacing sensitive information in a file inside a VM)

331 Monitoring Code Injection KVM injects the moni-toring code into VM-A dynamically by the following stepsStep (1) When VM-A is powered on the kernel symbol tableof VM-A is resolved Step (2) A kernel memory area (namedmem-A) of VM-A is allocated Step (3) According to thekernel symbol table of VM-A and the base address of mem-A the symbol table of the monitoring code is resolved andthe relocation data is repaired Step (4) e reconstructedmonitoring code is injected into mem-A finally


(1) VM kernel Symbol Resolution Take CentOS 72 x86_64OS as an example to illustrate the mechanism Figure 8shows the distribution of the kernel symbol table of CentOS

From Figure 8 kallsyms_addresses is an array that storesthe symbol name and the corresponding kernel virtualaddress which is hosted in the read-only data segment andarranged in ascending order kallsyms_num_syms stores thenumber of kernel symbols which is equal to the entrynumber of kallsyms_addresses and also the first descendingvalue kallsyms_names is an array that stores all kernelsymbol strings after being ASCII encoded which includeskallsyms_num_syms entries in total kallsyms_markers is anarray that stores the kernel symbol offset after kallsyms_-names is grouped (every adjacent 256 kernel symbols aredivided into a group) kallsyms_token_table is an array thatstores the strings which are commonly used kallsym-s_token_index is an array that stores the offset of every entryof kallsyms_names in kallsyms_token_table

According to the distribution rule of Figure 8 thememory of code segment of VM-A was searched from thelow address to the high address in the KVM to analyze theaddresses of the key symbols and the kernel symbol table(named VM_SYMS) of VM-A

(2) VM Kernel Memory Allocation KVM needs to call thefunction of module_alloc (a kernel memory allocation func-tion) of VM-A actively to allocate a mem-A Based on thekernel symbol resolution method after the KVM gets theaddress of module_alloc of VM-A we employ a method ofbottom-up function call channel to allocate a mem-A from thehigh address of the kernel memory area of VM-Ae detailed

steps are as follows Step (1) Save the information of currentexecution environment of VM-A (eg values of instructionregisters) when an event occurrence with an exit of the VM-AStep (2) Construct parameters for themodule_alloc through theinstruction register Step (3)Modify the value of the instructionregister to the address of module_alloc Step (4) e exit oc-currence in Step (1) would reoccur after executing the mod-ule_alloc and restore the information saved in Step (1)

(3) Monitoring Code Symbol Resolution and Relocation DataReparation e principle of monitoring code symbol res-olution is shown in Algorithm 1 Every entry in the symboltable of the monitoring code will be traversed if the addressof sym[i] is NULL the address will be retrieved and reas-signed by looking up VM_SYMS of VM-A Else it will beamended according to the base address of kernel memory ofVM-A and its relative offset

In Algorithm 1 symsec is a pointer to a symbol tablesection in the ELF format file of the monitoring code sym isthe symbol table of the ELF format file Every entry in thetable contains information of an index (st_name) a type(st_shndx) and an address (st_value) of every kernel symbolname in strtab strtab is a pointer to a string tableSHN_UNDEF is an unresolved symbol guest_load_addr is abase address of VM-A kernel memory that the KVM hasallocated sechdrs is a pointer to the section head table of theELF format file Every entry in the table stored the infor-mation including offset (sh_offset) relative to sechdrs sectionsize (sh_size) and code segment address (sh_info) name is atemporary variable that stores a name of a kernel symbolget_address (name) is the function of according to namevariable to retrieve the address from VM_SYMS

e principle of relocation data repairation is shown inAlgorithm 2 which contains the following steps Step (1)Traverse the relocation table to retrieve every entry in-formation (including an index of the current entry and anaddressing type (eg R_X86_64_PC32) of the entry to berelocated) Step (2) Calculate the address of the entry to berelocated in memory of VM-A according to the relocationtype and the resolved symbol table and then write it intothe relocation table Step (3) Load the address of thereconstructed relocation table to the address ofguest_load_addr

Text

kallsyms_addresses[]

ASC

kallsyms_num_symskallsyms_names

byte lenbyte valbyte val


kallsyms_markerskallsyms_num_syms gtgt 8

kallsyms_token_tablestring1string2

kallsyms_token_index

ROdata

Low address

High address

Figure 8 Distribution of the system kernel symbol table ofCentOS

T step

Full

conn

ecte

d la

yer 1

LSTM layer 1

LSTM layer 2

Full

conn

ecte

d la

yer 2

Out

put

Figure 6 e framework of LSTM-NAD

MonitoredVM-A

Kernellayer

Application layer

KVM

Monitoring policiesMonitoring

code injection

Injecting dynamicly when powering on

Applications

Monitoringprotection

syscall tablesyscall entry func

Monitoring results

syscall monitor func (monitoring code)

VM

mem-A

Figure 7 e framework of the agentless monitoring system


In Algorithm 2 hdr is the base address of the mon-itoring code in ELF format file loaded into a KVMtemporary memory rel is a temporary variable a pointerto the relocation table section and every entry in the reltable stores a relative offset (r_offset) and offset type in-formation (r_info) of the entry (that to be relocated)relsec is an index of the relocation section in sechdrstarget_sec is a temporary variable that points to the codesegment of the relocation section loc is a temporaryvariable that points to the address of rel[i] in the KVMtemporary memory fake_loc is a temporary variable thatpoints to the address of rel[i] in the memory of the VM-Asymindex represents an index of the symbol table sectionin the section header table of the ELF format file rel_typeis a temporary variable that stores a value of the relocationtype val is a temporary variable that stores a target ad-dress after rel[i] had been relocated

(4) Dynamic Injection Monitoring Code e operation of theinjection monitoring code whose symbols have been re-solved and repaired can be accomplished by the KVM bywriting the monitoring code to mem-A en redirectcorresponding guest system call execution paths to ourmonitoring code transparently to make the monitoringmechanism take effect e steps are as follows Step (1)According to VM_SYMS retrieve the address of a moni-tored system call in the system call table of VM-A Step (2)Reassign the address value in Step (1) to the value of cor-responding monitoring function (func_name) in the mon-itoring code

332 Monitoring Function Protections e protectionmechanisms include an antitampering function for theaddress register (MSR) of system call entry and a read-only

Input hdr sechdrs relsec symindexstrtab guest_load_addrOutput Relocated resolved code in memoryrel⟵ hdr+ sechdrs[relsec]sh_offsetfor i⟵ 0 to sechdrs[relsec]sh_sizesizeof(lowastrel) dotarget_sec⟵ sechdrs[relsec]sh_infoloc⟵ hdr+ sechdrs[target_sec]sh_offset+ rel[i]r_offset

fake_loc⟵guest_load_addr +sech drs[target_sec]sh_offset+ rel[i]r_offset

sym⟵ hdr+ sechdrs[symindex]sh_offset +ELF_R_SYM(rel[i]r_info)

symname⟵ strtab+ sym-gtst_nameval⟵ sym-gtst_value+ rel[i]r_addendrel_type⟵ ELF_R_TYPE(rel[i]r_info)switch rel_type do

case rel_typeR_X86_64_64lowast(u64lowast)loc⟵ valcase rel_typeR_X86_64_32lowast(u32lowast)loc⟵ valcase rel_typeR_X86_64_32Slowast(s32lowast)loc⟵ valcase rel_typeR_X86_64_PC32 val⟵ val-(u64)fake_loc lowast(u32lowast)loc⟵ valotherwiseOutput Unknown

endswendsw

end

ALGORITHM 2 GuestCodeRelocResv

Input symsec sym strtab guest_load_addrOutput Symbol resolved code in memoryfor i⟵ 1 to symsec-gtsh_sizesizeof(Elf_Sym) doname⟵ strtab+ sym[i]st_nameif sym[i]st_shndx SHN_UNDEF thensym[i]st_value⟵ get_ address(name)

elsesym[i]st_value⟵ sym[i]st_value +guest_load_addr +sechdrs[sym[i]st_shndx]sh_offset

endend

ALGORITHM 1 GuestCodeSymResv


mechanism for the kernel code (including the system callentry function and the injected monitoring code) and thesystem call table

(1) Protection for MSR turn on the write operationtrapping of SYSENTER_EIP_MSR of VM-A bysetting MSR bitmaps of the KVM When VM-Aattempts to modify the value of SYSENTER_-EIP_MSR it will be captured by the KVM Becausethe KVM has higher privileges the KVM can in-validate the abovementioned modification operationdirectly Moreover the writing operation toSYSENTER_EIP_MSR only occur when VM-Akernel is initialized which bring little impact onperformance of VM-A

(2) Read-only protection for the kernel code and thesystem call table (A) When VM-A accesses data thepage table management mechanism of the OSconverts the virtual address to the physical address ofVM-A en the CPU traverses the ept page tableand converts the physical address of VM-A to thephysical address of the host and the data in memorycould be retrieved finally (B) e page table of VM-A is maintained by itself and the malicious moduleinside VM-A can tamper with the kernel (C)e eptpage table is maintained by the KVM which cannotbe tampered by the malicious code of VM-A eprotection module achieves security protection bymapping the kernel codes and the system call table ofVM-A into a read-only mode in the ept

333 Monitoring Policy Definition Monitoring policy P isdefined formally as a four-tuple P (IDNrConf Action)

(1) ID represents the ID of a VM to be monitoredID id1 idn1113864 1113865 if ine k then idi ne idk

(2) Nr represents a system call number set to be mon-itored Nr nr1 nrm1113864 1113865 if ine k then nri ne nrk

(3) Conf represents a system call configuration set (ega blacklist set of the monitored processes)Conf (s1 o1) (su ov)1113864 1113865 the subject S initiatesthe operation and the object O is being accessedS s1 sw1113864 1113865 O o1 ox1113864 1113865 if ine k thensi ne sk oi ne ok

(4) Action represents real time responses (eg recordand block the operation of placing sensitive infor-mation in a file on a VM) of the monitored systemcalls Action record block

4 Experimental Campaign

According to Figure 1 we employ several normal serversGigabit Ethernet switches and Gigabit NICs to build the IaaSplatforme key parameters are OpenStack Kilo as the IaaSmanagement toolkit KVM as the VMM and version is 230OpenvSwitch as the virtual SDN switch and Ryu as the SDNcontroller and CentOS 72 x64 release version as the host OSand its kernel version is 3100 and OpenVAS 7 as the

vulnerability scanning engine e key physical host pa-rameters are the CPU model is Intel (R) Xeon (R) CPU E5-2630 v3lowast 2 240GHz memory capacity is 128GB Otherexperimental parameters will be given in the correspondingsectione deployment architecture is as shown in Figure 9

e Tenant Domain is constructed by employing NVSDN and NFV technologies (1) Logical communicationchannel is constructed by employing the VxLAN technology(2) subnet configuration is relied on the virtual appliances ofDHCP router and switch implemented by NFV technology(3) tenant domain perimeter is built by a set of virtual routersand firewalls those deployed on the logical communicationchannel (4) communication access control that leveraging theACRs configured in virtual firewalls e isolation and in-terconnection between different tenant domains and sub-domains are realized by employing different virtual routerswhich configured different subnets and firewalls e firewallcould load iptablesebtables rules or SDN forward rules toimplement access control Moreover the support domain isconstructed by leveraging virtual firewall network anomalydetection VM monitoring and VM antivirus [1]

41 Security Service Access For the Type 1 access methodthe VM communication delay is increased due to the VMtraffic is redirected to the security appliance firstly estatistics of the PING operation delay test before and afterconfiguring the redirect policy for two VMs in the samesubnet in the IaaS environment is shown in Figure 10 eresults show that the communication delay after redirectionis twice that of in redirect manner which is as expected

For the Type 2 access method which compares with thecurrent method of providing security scanning with a VMgranularity we perform tests of the scanning performanceby employing OpenVAS 7 tool and the resource consumedby the multitasks method and the multi-VMs method inscanning different numbers of tenant network domains asshown in Table 2

e result shows that providing a vulnerability detectionservice at the VM granularity for different tenants leads to asharp increase in resource consumption Moreover becausethe method proposes in this paper involved virtual networkdevice creation network address translation etc statisti-cally which only increases 01ms delay compared with theway the VM provides services directly

42 Network Anomaly Detection To prove the effectivenessof the proposed method an experiment is designed tocompare the detection results of our algorithm and paper [1]method in a real network environment that is similar topaper [1] e network traffic of a web server running in aVM is captured and 11 features of network traffic aremonitored for a month which contains about 37439 timewindows and the first 21600 records are for training and therest for testing

enetwork traffic dataset contains about 28 billion packetsin train dataset and about 16 billion packets in test dataset Inorder to distinguish normal events and abnormal events weanalyze network traffic and system logs about the server and


mark abnormal and normal manually which contain SYNflood URL probe UDP flood port scan and partial unknownattacks Details of events are as shown in Table 3

After numerous experiments the LSTM-NAD networkusually converges in 50 epochs so the epoch is set to 50 andthe callback function is set to save the best model Since

Table 2 Performance test result ()

Test items Number of tenants Multi-VMs method Multitasking method

CPU1 223 1463 642 2275 998 355

Memory1 141 233 357 615 735 139

0

05

1

15

2

25

3

Dela

y tim

e (m

s)

PING (direct)PING (redirect)

Times of sending PING packet

1 14 27 40 53 66 79 92 105

118

131

144

157

170

183

196

209

222

235

248

261

274

287

300

313

326

339

352

365

378

391

404

417

430

443

456

469

482

495

508

521

534

547

560

573

586

599

Figure 10 Consumed time of PING operation withoutwith VM communication redirection

Physical resource

layer

Network interconnect

layer

Management network links

Tenant business network links

Cloud management center

Management domain

Storage network links

Storage domain

VM VM VM VM VM

Tenant business network

Sub-domain

DMZ

Network anomaly detection

Vulnerability scanning

Security services

Supporting domain

Security service network links

Tenant business domain ampamp support domain

Physical server

Physical server Physical serverPhysical server

Server

Network accesslayer

Network corelayer

Storage server KVM

Router Firewall

Physical server

KVM MonitoringMonitoring

Tenant domain

Figure 9 e deployment architecture of the IaaS platform


network anomaly detection is a binary classification prob-lem the loss function is set to binary cross entropy Finallythe LSTM-NAD is implemented by employing Keras andTensorflow and running on a server configured a NVIDIATesla P40 graphics card

Receiver-operating characteristic (ROC) and the areaunder curve (AUC) are two of the most important metricsfor evaluating any performance of classification ROC plotsdescribe the False Positive Rate (FPR) versus the TruePositive Rate (TPR) over all possible threshold values If theobserved curve in a ROC plot falls along the diagonal line xequals to y the model is assessed to perform no better thanrandom chance A perfect curve is the one that forms a rightangle at FPR 00 and TPR 10 which will indicate that allassigned probabilities to the positive class (here maliciousactivity) are greater than assigned probabilities to thenegative class (here benign activity) AUC is the valueattained by integrating over the ROC curve An AUC of 10denotes perfect classification and an AUC of 05 denotesrandom chance

us those two metrics are used to evaluate the anomalydetection performance of each method and the ROC-AUCfigure for two methods is as shown in Figure 11

As seen from Figure 11 AUC of LSTM-NAD is 094which is a pretty high score compared with [1] andsuggests that our LSTM-NAD methods have an effectiveperformance on network anomaly detection On the otherhand ROC of LSTM-NAD almost covered [1] whichmeans LSTM-NAD has a higher true positive rate andlower false positive rate regardless of the threshold In theexperiment [1] detects some unknown anomalies and theanomalies are also detected by LSTM-NAD which in-dicates that LSTM-NAD can detect unknown anomaliesthanks to the memory of the normal network trafficpattern

43 VM Monitoring

(1) Functional Validity Verifications In a VM the monitoredfile operation information is viewed from the log of the KVMwhen opening or editing the monitored files including thefile name file path operation process user ID (uid) andoperation type as shown in Figure 12 Furthermore becausethe monitoring code exists only in the kernel memory of theVM and does not depend on the OS driver and the process-related kernel data structures any information about themonitoring system cannot be detected by using detectiontools such as ldquolsmodrdquo or ldquopsrdquo running inside the VM

(2) Monitoring Function Protection Figure 13 shows thatwhen a module (named M) inside a VM attempts to modifythe protected code and data (eg perform a modification

operation to the system call table) an ept violation could betriggered into the KVM and the event will be captured by theprotection module

(3) Performance Testing According to the actual experiencethe ratio of random read and write operations inside anoperating system is usually 7 3 erefore use the Fio testtool in a VM to perform the following operations to simulatenormal read and write situations and parameters of the testVM are OS is CentOS 72 x86_64 configured 2 vCPUs andmemory capacity is 4GB

ldquofio -ioengine sync -bs 4k -direct 1 -thread-rw randrw -rwmixread 70 -size 9G -filename homeartest -name ldquotestrdquo -iodepth 32 -runtime 180rdquo

At the same time the following monitoring policy p isset for the VM where the system call number indications areas follows 0 indicates sys_read 1 indicates sys_write 2indicates sys_open and 3 indicates sys_close

Table 3 Details of events

Datasets Total events Normal event Abnormal eventTrain dataset 21600 21497 103Test dataset 15839 15721 118

00

02

04

06

08

10

00 02 04 06 08 10Tr

ue p

ositi

ve ra

teFalse positive rate

Network anomaly detection ROC

LSTM-NAD (AUC = 094)Paper [3] (AUC = 083)

Figure 11 ROC and AUC for two methods

[276230460696] FILE MONITOR INDEX[0] timestamp [1537193189]filename[test2] pathname [hometest2] operation [OPEN] process name [bash] user id[0][276230460698] FILE MONITOR INDEX[1] timestamp [1537193189]filename[test2] pathname [hometest2] operation [CLOSE] process name [bash] user id[0][276230460701] FILE MONITOR INDEX[2] timestamp [1537193190]filename[testc] pathname [hometest2testc] operation [OPEN] process name [cat] user id [0][276230460703] FILE MONITOR INDEX[3] timestamp [1537193190]filename[testc] pathname [hometest2testc] operation [READ] process name [cat] user id [0][276230460705] FILE MONITOR INDEX[4] timestamp [1537193190]filename[testc] pathname [hometest2testc] operation [CLOSE] process name [cat] user id [0]

Figure 12 e file operation monitoring result of the test VM


ldquop (c9b6bea3-a2db-42f7-a625-1fbc8d6012910123(vim etchosts) record)rdquo

In the case of random readwrite ratio of 7 3 the sta-tistical data of IOPS overhead brought by the monitoringmechanism as shown in Table 4

e results show that the monitoring system has littleimpact on VM IOPS

For CPU and memory overhead brought by the mon-itoring mechanism based on storage IOPS test case in theVM the ldquonmonrdquo test tool is used on the host to evaluateperformance impact of the monitoring mechanism on thehost systeme use cases and test data are shown in Tables 5and 6 e test data shows that the performance con-sumption of CPU and memory brought by the monitoringmechanism is basically negligible

Based on abovementioned discussion the method is (1)compared with the internal monitoring mode because thereis no need to install a visible agent inside the VM and themethod has higher reliability and security (2) comparedwith the agentless monitoring mode based on VMI thesemantic gap between a VM and a VMM can be effectivelyeliminated by the monitoring code injected transparentlyMoreover resource consumed by the method is lower thanthe VMI method (3) Compared with the method proposedin [1] the semantic analysis operation is performed everytime when the monitoring is performed and the operationneeds to be executed only once in the code injection phase byleveraging the proposed method which greatly reduces the

time-consuming switching of the context caused by thesemantic analysis operation

5 Conclusions and Future Work

In this paper the corresponding designs and techniques ofIaaS security framework security service access networkanomaly detection and VM monitoring are designed andrealized e current research situation of cloud securityframework security service access network anomalydetection and VM monitoring are discussed and ad-vantages and disadvantages are pointed out Many pub-licly available research results focus on abstracting thecloud security framework which are not conducive topromotion and implementation of cloud security tech-nologies rapidly Besides the detailed implementationsand experimental campaign and discussion about theeffectiveness and performance costs of the proposedframework and the various supporting securities aregiven More complete technical discussions about theframework and concrete technologies (eg EncryptionVPN VM escape detection Data backup and Softwareupdate) will be discussed in future work and a morecomprehensive IaaS security protection framework asshown in Figure 14 will be researched

sys_call_table address-gtffffffffbc603300module address-gt0xffffffffbc603300 is write protectedmodified virtual address-gtffffffffbc603598kvm vaddr-gtffffffff95a03300 is a large pagetry to splitekvm [26039] vcpu0 disabled perfctr wrmsr 0xc2 data 0xffffkvm convert gva-gtffffffff95a03300 to gpa-gt27c03300kvm marked gpa-gt0x27c03300 write protectedkvm guest try to modify read-only data gva-gt0xffffffff95a03598

Figure 13 e protection result of the system call table of the VM was tempered by M

Table 4e statistics of test cases of IOPS (R read W write N nomonitoring and Y monitoring enable)

IDIOPS Bandwidth (kBs)

R R W W R R W WN Y N Y N Y N Y

1 71 69 30 29 2856 2799 1207 11832 71 70 30 29 2849 2816 1204 11903 72 69 30 29 2888 2791 1224 11794 71 70 30 29 2867 2836 1212 11975 71 70 30 29 2879 2816 1217 11916 71 70 30 29 2844 2817 1202 11917 71 70 30 29 2846 2807 1202 11878 71 70 30 29 2864 2815 1211 11899 71 70 30 29 2871 2811 1214 118910 71 70 30 29 2861 2839 12109 1199Avg 711 698 30 29 2863 2815 1211 1189

Table 5 Test cases of CPU and memory performance

Test case Test method description

Test 1 CPU performanceoverhead test

e host uses nmon to collect theCPU load once every 10 s and

continuously collects 180 times for30 minutes

Test 2 memoryperformance overhead test

e host uses nmon to collect thememory load once every 10 s andcontinuously collects 180 times for

30 minutesPS each test perform 4 tests (2 tests before monitoring functiondeployment and 2 tests after monitoring function deployment)

Table 6 Test statistics of CPU and memory consumed

Testitems

No monitoring()

Monitoringenable ()

Resourceconsumed ()

CPU 267 27 003Memory 17477 18112 0635


Data Availability

e data used to support the findings of this study are in-cluded within the article

Conflicts of Interest

e authors declare that they have no conflicts of interest

Acknowledgments

is work was supported in part by the National NaturalScience Foundation of China under Grant nos 61802270and 61802271 Key Research and Development Project ofSichuan Province of China under Grant no 2018GZ0100and Fundamental Research Business Fee Basic ResearchProject of Central Universities under Grant no2017SCU11065

References

[1] X Y Yin X S Chen L Chen G L Shao H Li and S TaoldquoResearch of security as a service for VMs in IaaS platformrdquoIEEE Access vol 6 no 1 pp 29158ndash29172 2018

[2] G Brunette and R Mogull Security Guidance for CriticalAreas of Focus in Cloud Computing v40 Cloud SecurityAlliance Toronto Canada 2017

[3] CSA Trusted Cloud Initiative Reference Architecture v20Cloud Security Alliance Toronto Canada 2013

[4] NIST Cloud Computing Security Group ldquoNIST cloud com-puting security reference architecturerdquo NIST Special Publi-cation Gaithersburg MD USA 2011

[5] ENISA Security Framework for Governmental SecurityFramework for Governmental Clouds ENISA HeraklionGreece 2015

[6] V Varadharajan and U Tupakula ldquoSecurity as a servicemodel for cloud environmentrdquo IEEE Transactions on Networkand Service Management vol 11 no 1 pp 60ndash75 2014

[7] C Lin W B Su K Meng Q Liu and W D Liu ldquoCloudcomputing security architecture mechanism and modelingrdquoChinese Journal of Computers vol 36 no 9 pp 1765ndash17842013

[8] VMware VMware NSX Network Virtualization Design GuideVMware Palo Alto CA USA 2013

[9] Sugon ldquoSecurity vulnerability scanning system in cloudnetwork environmentrdquo Sugon Beijing China CN 103825891A 2014

[10] IEEE IEEE 8021BRmdashBridge Port Extension IEEE LANMANStandards Committee Piscataway NJ USA 2012

[11] IEEE IEEE 8021QbgmdashEdge Virtual Bridging IEEE LANMAN Standards Committee Piscataway NJ USA 2012

[12] M Ahmed A N Mahmood and J Hu ldquoA survey of networkanomaly detection techniquesrdquo Journal of Network andComputer Applications vol 60 pp 19ndash31 2016

[13] A H Hamamoto L F Carvalho L D H Sampaio T Abratildeoand M L Proenccedila ldquoNetwork anomaly detection system usinggenetic algorithm and fuzzy logicrdquo Expert Systems with Ap-plications vol 92 pp 390ndash402 2018

[14] G Fernandes Jr L F Carvalho J J P C Rodrigues andM L Proenccedila ldquoNetwork anomaly detection using IP flowswith principal component analysis and ant colony optimi-zationrdquo Journal of Network and Computer Applicationsvol 64 pp 1ndash11 2016

[15] C Callegari S Giordano and M Pagano ldquoEntropy-basednetwork anomaly detectionrdquo in Proceedings of the 2017 In-ternational Conference on Computing Net-working and

Iaas data center

Tenant domain

Virtual network perimeter security

Communication traffic flow

Userterminal perimeter

Data center perimeter security

Physical server

OSapplications +

dataHypervisor VM

VM perimeter security

Phone pad notebook PC etc InternetLANWAN

Physical server perimeter security

Userterminal

FirewallVPNAntivirusIdentity authentication and authorizationMonitoringVulnerability detectionSoware update

(i)(ii)

(iii)(iv)

(v)(vi)

(vii)

Cpu memory storage and network resources isolationHypervisor reinforcementMonitoringAntivirusFirewallVulnerability detectionEmergency response

(i)

(ii)(iii)(iv)(v)

(vi)(vii)

Access managementIdentity authentication and authorizationCommunication access controlCommunication encryptionVulnerability detectionEmergency response

(i)(ii)

(iii)

(iv)

(v)(vi)

MonitoringAntivirusFirewallVulnerability detectionSoware updateEmergency response

(i)(ii)

(iii)(iv)

(v)(vi)

Network perimeter access controlNetwork segment and isolationNetwork anomaly detectionVulnerability detectionEmergency response

(i)

(ii)

(iii)

(iv)(v)

Figure 14 A more comprehensive security protection framework for a virtualized IaaS environment


Communications (ICNC) pp 334ndash340 Santa Clara CA USAJanuary 2017

[16] K Flanagan E Fallon P Connolly and A Awad ldquoNetworkanomaly detection in time series using distance based outlierdetection with cluster density analysisrdquo in Proceedings of the2017 Internet Technologies and Applications (ITA) pp 116ndash121 Wrexham UK September 2017

[17] R Chandramouli ldquoSecurity recommendations for hypervisordeploymentrdquo NIST Special Publication Gaithersburg MDUSA 2015

[18] X D Ren and J Wang ldquoLinux system call hijacking detectionmethodrdquo Information Security and Technology vol 11 no 6pp 61-62 2015

[19] I Ahmed A Zoranic S Javaid and G G Richard ldquoMod-Checker kernel module integrity checking in the cloud en-vironmentrdquo in Proceedings of the IEEE InternationalConference on Parallel Processing Workshops pp 306ndash313Pittsburgh PA USA September 2012

[20] J Hizver and T C Chiueh ldquoReal-time deep virtual machineintrospection and its applicationsrdquo ACM Sigplan Noticesvol 49 no 7 pp 3ndash14 2014

[21] W J Liu L N Wang C Tan and X Lai ldquoA virtual machineintrospection triggering mechanism based on VMFUNCrdquoJournal of Computer Research and Development vol 54no 10 pp 2310ndash2320 2017

[22] P Malhotra L Vig G Shroff et al ldquoLong short termmemorynetworks for anomaly detection in time seriesrdquo in Proceedingsof the European Symposium on Artificial Neural NetworksComputational Intelligence and Machine Learning pp 89ndash94Bruges Belgium April 2015


vulnerability scanning tool that encapsulated in a VMHowever a large amount of storage space is needed as thenumber of the VM increases and the characteristic ofmultitasking parallel processing of the vulnerability scan-ning engine cannot be brought into play A security groupfunction implemented in Openstack employs the NetfilterIptables to implement a software virtual firewall which isinconvenient to expand dynamically due to the tight cou-pling implementation Cisco and HP develop new protocolsof VN-TAG and Virtual Ethernet Port Aggregator (VEPA)respectively to pull the VM traffic to the physical access layerswitch to implement VM communication control [10 11]

irdly network anomaly detection is a key mean ofnetwork security protection which could analyse networktraffic agentless Currently network anomaly detectionmethods can be classified into four categories [12] (1)Classification method the method needs security expertwhich provides a known pattern of an attack and designeddetailed features Hamamoto et al [13] proposes a method ofcombining a genetic algorithm and fuzzy logic for networkanomaly detection which uses a genetic algorithm to generatemultiple features and decides whether an event is an anomalyby the fuzzy logic method e results achieve an accuracy of9653 and false positive rate of 056 e classificationmethod could make full use of the detailed characteristics ofnetwork traffic and obtain high accuracy (2) Statisticsmethod the basic idea of the method is to consider a largedeparture of events from normal as anomalies thus profilingnormal network behavior can detect unknown attacks the-oretically Fernandes et al [14] propose two anomaly de-tection mechanisms based on statistical procedure PCA andthe ant colony optimization which profiles normal networkbehaviours and compares with the real network traffic by amodification of the dynamic timewarpingmetric to recognizeanomalous events However statistics methods are notsuitable for some small-scale attacks due to loss of detailedfeatures (3) Information theory information-theoreticmeasures such as entropy could be used to create an anomalydetection model Callegari et al [15] proposes an intrusiondetection method that performs anomaly detection bystudying the variation in the entropy associated with thenetwork traffic e traffic is aggregated through three-di-mensional reversible sketches and computes the entropy ofdifferent traffic descriptors by using several definitions ofentropy Finally detecting network anomaly according to thedeviation of entropy information-theoretic ignores the de-tailed information of traffic which is hard to detect small-scale attacks (4) Clustering method the method does notrequire labelling data and could mine the pattern of dataautomatically An evolution of the Microclustering OutlierDetection (MCOD) machine learning algorithm is proposedby Flanagan et al [16] e method of distance-based outlierdetection and cluster density analysis are combined to detectanomaly in time series which do not need to label dataHowever it is difficult to design distance metrics

Finally monitoring is an essential part of the securityassurance mechanism [1 2] Currently monitoring tech-nologies for the state of a VM can be classified in two cat-egories (1) Internal monitoring method (a software-agent is

run inside a VM) which provides good context analysis forthe state of the VM but malicious code running inside theVM could damage the monitoring system (2) Externalmonitoring method (or Hypervisor-based monitoringmethod) [17] which is realized by employing a Hypervisor ora Virtual Machine Monitor (VMM)e state of a VM can bemonitored through some specific Hypervisor APIs while theretrieved information is limited to the inherent APIs severelywithout opportunities to expand or customize such asproducts of OpenStack Ceilometer andAWSCloudWatch Tomake up for that deficiency the methods based on VirtualMachine Introspection (VMI) have been proposed [18ndash20]e methods always rely on setting a specific hardwaretrapping for internal resource (eg CPU ormemory) of a VMfirstly When the VM accesses the specific resource theoperation will be captured by the VMM immediately enthe introspection mechanism is adopted to monitor the stateof the VM Compared with the internal monitoring methodHypervisor-based method has a higher safety and reliabilitybut it consumes more resources To solve the problem atriggering method based on VMFUNC is proposed [21]which minimizes overhead of VM exits and avoids the systempause caused by programs of VMI

To satisfy abovementioned security concerns a corre-sponding solution and techniques are designed and realizedin the following sections In Section 2 a cloud securityframework compiled the security model of P2DR and theideas of defense-in-depth of IATF and security as a servicewas constructede framework employed a physical layer avirtualization layer and a security management layer InSection 31 to satisfy the requirement of security serviceaccess two access methods were realized for various securitytools by leveraging NV NFV and SDN technologies fromthe perspective of whether the tools generate communica-tion traffic One without generating traffic employed the VMtraffic redirection technology with loose coupling and hadbetter flexibility e other one leveraged the mechanism ofmultitasking process access and the characteristic of mul-titasking parallel processing of partial security engine couldbe brought into play which was helpful to reduce resourceconsumption In Section 32 to ensure network security ofthe VM a LSTM-based network anomaly detection methodwas proposed Based on the periodicity of network traffic 11features were selected to describe traffic structure in a timewindow en multivariate time series consisting of 11features were input to LSTM-NAD for training Finally thefinely trained LSTM-NAD model predicted a binary valuewhich represents whether the current time window wasabnormal or not Compared with paper [1] the LSTM-NADmethod had the advantages of higher AUC score anddetecting anomaly more effectively In Section 33 based onthe characteristics of the internal monitoring method andexternal monitoring mechanism a Hypervisor-basedagentless monitoring method for VMs based on themechanism of dynamic code injection was proposed eproposed mechanism had benefits of high security of theexternal monitoring method that could not be destroyed bymalicious code running inside the VM and had the virtues ofgood context analysis for the state of the VM low resource















Detection

Protection

(1)(2)(3)(4)(5)(6)(7)(8)


Response

Response

Policy


(1)(2)(3)(4)


(1)(2)(3)

Protectio

n

Detection


VM2

VM1

SDNcontroller

Securityappliances






SDN vSwitch





Tenant domain


Management domain

Supportdomain

(SD)

DMZ



IaaS environment


L1

Virtuali

zation lay

er

Physical

layer

L2 L3Policy

Protect

ion

Response

Detection














ip)TAG null)





ip)TAG null)











(t)1 x

(t)2 x(t)




VMA

SDN virtual switch








In a samesubnet




SDN virtual switch



S1

S2

S3

R1

R2

R3

Tenantdomain







00020406081012

access_port

202530354045505560

ip_entro

00020406081012

bpstimes108

times103

times104

times104

times104

times107

0002040608101214

pps

00

02

04

06

08packet_per_ip

0010203040506070

ip_count

000000250050007501000125015001750200

service_port_entro

000002004006008010012014

syn_rate

075100125150175200225250

syn_per_ip

000510152025

syn_count

000025050075100125150175200

bytes_per_ip








ht ot ⊙fh ct( 1113857

(1)





















Text


ASC







ROdata

Low address

High address


T step

Full

conn

ecte

d la

yer 1

LSTM layer 1

LSTM layer 2

Full

conn

ecte

d la

yer 2

Out

put


MonitoredVM-A

Kernellayer

Application layer

KVM


code injection


Applications



Monitoring results


VM

mem-A











endswendsw

end




endend

























CPU1 223 1463 642 2275 998 355

Memory1 141 233 357 615 735 139

0

05

1

15

2

25

3

Dela

y tim

e (m

s)



1 14 27 40 53 66 79 92 105

118

131

144

157

170

183

196

209

222

235

248

261

274

287

300

313

326

339

352

365

378

391

404

417

430

443

456

469

482

495

508

521

534

547

560

573

586

599


Physical resource

layer


layer




Management domain


Storage domain

VM VM VM VM VM


Sub-domain

DMZ



Security services

Supporting domain



Physical server


Server

Network accesslayer

Network corelayer

Storage server KVM

Router Firewall

Physical server


Tenant domain







43 VM Monitoring









00

02

04

06

08

10

00 02 04 06 08 10Tr

ue p

ositi

ve ra





















1 71 69 30 29 2856 2799 1207 11832 71 70 30 29 2849 2816 1204 11903 72 69 30 29 2888 2791 1224 11794 71 70 30 29 2867 2836 1212 11975 71 70 30 29 2879 2816 1217 11916 71 70 30 29 2844 2817 1202 11917 71 70 30 29 2846 2807 1202 11878 71 70 30 29 2864 2815 1211 11899 71 70 30 29 2871 2811 1214 118910 71 70 30 29 2861 2839 12109 1199Avg 711 698 30 29 2863 2815 1211 1189










Testitems

No monitoring()

Monitoringenable ()

Resourceconsumed ()

CPU 267 27 003Memory 17477 18112 0635


Data Availability




Acknowledgments


References
















Iaas data center

Tenant domain





Physical server

OSapplications +

dataHypervisor VM




Userterminal


(i)(ii)

(iii)(iv)

(v)(vi)

(vii)


(i)

(ii)(iii)(iv)(v)

(vi)(vii)


(i)(ii)

(iii)

(iv)

(v)(vi)


(i)(ii)

(iii)(iv)

(v)(vi)


(i)

(ii)

(iii)

(iv)(v)

























Detection

Protection

(1)(2)(3)(4)(5)(6)(7)(8)


Response

Response

Policy


(1)(2)(3)(4)


(1)(2)(3)

Protectio

n

Detection


VM2

VM1

SDNcontroller

Securityappliances






SDN vSwitch





Tenant domain


Management domain

Supportdomain

(SD)

DMZ



IaaS environment


L1

Virtuali

zation lay

er

Physical

layer

L2 L3Policy

Protect

ion

Response

Detection














ip)TAG null)





ip)TAG null)











(t)1 x

(t)2 x(t)




VMA

SDN virtual switch








In a samesubnet




SDN virtual switch



S1

S2

S3

R1

R2

R3

Tenantdomain







00020406081012

access_port

202530354045505560

ip_entro

00020406081012

bpstimes108

times103

times104

times104

times104

times107

0002040608101214

pps

00

02

04

06

08packet_per_ip

0010203040506070

ip_count

000000250050007501000125015001750200

service_port_entro

000002004006008010012014

syn_rate

075100125150175200225250

syn_per_ip

000510152025

syn_count

000025050075100125150175200

bytes_per_ip








ht ot ⊙fh ct( 1113857

(1)





















Text


ASC







ROdata

Low address

High address


T step

Full

conn

ecte

d la

yer 1

LSTM layer 1

LSTM layer 2

Full

conn

ecte

d la

yer 2

Out

put


MonitoredVM-A

Kernellayer

Application layer

KVM


code injection


Applications



Monitoring results


VM

mem-A











endswendsw

end




endend

























CPU1 223 1463 642 2275 998 355

Memory1 141 233 357 615 735 139

0

05

1

15

2

25

3

Dela

y tim

e (m

s)



1 14 27 40 53 66 79 92 105

118

131

144

157

170

183

196

209

222

235

248

261

274

287

300

313

326

339

352

365

378

391

404

417

430

443

456

469

482

495

508

521

534

547

560

573

586

599


Physical resource

layer


layer




Management domain


Storage domain

VM VM VM VM VM


Sub-domain

DMZ



Security services

Supporting domain



Physical server


Server

Network accesslayer

Network corelayer

Storage server KVM

Router Firewall

Physical server


Tenant domain







43 VM Monitoring









00

02

04

06

08

10

00 02 04 06 08 10Tr

ue p

ositi

ve ra





















1 71 69 30 29 2856 2799 1207 11832 71 70 30 29 2849 2816 1204 11903 72 69 30 29 2888 2791 1224 11794 71 70 30 29 2867 2836 1212 11975 71 70 30 29 2879 2816 1217 11916 71 70 30 29 2844 2817 1202 11917 71 70 30 29 2846 2807 1202 11878 71 70 30 29 2864 2815 1211 11899 71 70 30 29 2871 2811 1214 118910 71 70 30 29 2861 2839 12109 1199Avg 711 698 30 29 2863 2815 1211 1189










Testitems

No monitoring()

Monitoringenable ()

Resourceconsumed ()

CPU 267 27 003Memory 17477 18112 0635


Data Availability




Acknowledgments


References
















Iaas data center

Tenant domain





Physical server

OSapplications +

dataHypervisor VM




Userterminal


(i)(ii)

(iii)(iv)

(v)(vi)

(vii)


(i)

(ii)(iii)(iv)(v)

(vi)(vii)


(i)(ii)

(iii)

(iv)

(v)(vi)


(i)(ii)

(iii)(iv)

(v)(vi)


(i)

(ii)

(iii)

(iv)(v)























ip)TAG null)





ip)TAG null)











(t)1 x

(t)2 x(t)




VMA

SDN virtual switch








In a samesubnet




SDN virtual switch



S1

S2

S3

R1

R2

R3

Tenantdomain







00020406081012

access_port

202530354045505560

ip_entro

00020406081012

bpstimes108

times103

times104

times104

times104

times107

0002040608101214

pps

00

02

04

06

08packet_per_ip

0010203040506070

ip_count

000000250050007501000125015001750200

service_port_entro

000002004006008010012014

syn_rate

075100125150175200225250

syn_per_ip

000510152025

syn_count

000025050075100125150175200

bytes_per_ip








ht ot ⊙fh ct( 1113857

(1)





















Text


ASC







ROdata

Low address

High address


T step

Full

conn

ecte

d la

yer 1

LSTM layer 1

LSTM layer 2

Full

conn

ecte

d la

yer 2

Out

put


MonitoredVM-A

Kernellayer

Application layer

KVM


code injection


Applications



Monitoring results


VM

mem-A











endswendsw

end




endend

























CPU1 223 1463 642 2275 998 355

Memory1 141 233 357 615 735 139

0

05

1

15

2

25

3

Dela

y tim

e (m

s)



1 14 27 40 53 66 79 92 105

118

131

144

157

170

183

196

209

222

235

248

261

274

287

300

313

326

339

352

365

378

391

404

417

430

443

456

469

482

495

508

521

534

547

560

573

586

599


Physical resource

layer


layer




Management domain


Storage domain

VM VM VM VM VM


Sub-domain

DMZ



Security services

Supporting domain



Physical server


Server

Network accesslayer

Network corelayer

Storage server KVM

Router Firewall

Physical server


Tenant domain







43 VM Monitoring









00

02

04

06

08

10

00 02 04 06 08 10Tr

ue p

ositi

ve ra





















1 71 69 30 29 2856 2799 1207 11832 71 70 30 29 2849 2816 1204 11903 72 69 30 29 2888 2791 1224 11794 71 70 30 29 2867 2836 1212 11975 71 70 30 29 2879 2816 1217 11916 71 70 30 29 2844 2817 1202 11917 71 70 30 29 2846 2807 1202 11878 71 70 30 29 2864 2815 1211 11899 71 70 30 29 2871 2811 1214 118910 71 70 30 29 2861 2839 12109 1199Avg 711 698 30 29 2863 2815 1211 1189










Testitems

No monitoring()

Monitoringenable ()

Resourceconsumed ()

CPU 267 27 003Memory 17477 18112 0635


Data Availability




Acknowledgments


References
















Iaas data center

Tenant domain





Physical server

OSapplications +

dataHypervisor VM




Userterminal


(i)(ii)

(iii)(iv)

(v)(vi)

(vii)


(i)

(ii)(iii)(iv)(v)

(vi)(vii)


(i)(ii)

(iii)

(iv)

(v)(vi)


(i)(ii)

(iii)(iv)

(v)(vi)


(i)

(ii)

(iii)

(iv)(v)





















(t)1 x

(t)2 x(t)




VMA

SDN virtual switch








In a samesubnet




SDN virtual switch



S1

S2

S3

R1

R2

R3

Tenantdomain







00020406081012

access_port

202530354045505560

ip_entro

00020406081012

bpstimes108

times103

times104

times104

times104

times107

0002040608101214

pps

00

02

04

06

08packet_per_ip

0010203040506070

ip_count

000000250050007501000125015001750200

service_port_entro

000002004006008010012014

syn_rate

075100125150175200225250

syn_per_ip

000510152025

syn_count

000025050075100125150175200

bytes_per_ip








ht ot ⊙fh ct( 1113857

(1)





















Text


ASC







ROdata

Low address

High address


T step

Full

conn

ecte

d la

yer 1

LSTM layer 1

LSTM layer 2

Full

conn

ecte

d la

yer 2

Out

put


MonitoredVM-A

Kernellayer

Application layer

KVM


code injection


Applications



Monitoring results


VM

mem-A











endswendsw

end




endend

























CPU1 223 1463 642 2275 998 355

Memory1 141 233 357 615 735 139

0

05

1

15

2

25

3

Dela

y tim

e (m

s)



1 14 27 40 53 66 79 92 105

118

131

144

157

170

183

196

209

222

235

248

261

274

287

300

313

326

339

352

365

378

391

404

417

430

443

456

469

482

495

508

521

534

547

560

573

586

599


Physical resource

layer


layer




Management domain


Storage domain

VM VM VM VM VM


Sub-domain

DMZ



Security services

Supporting domain



Physical server


Server

Network accesslayer

Network corelayer

Storage server KVM

Router Firewall

Physical server


Tenant domain







43 VM Monitoring









00

02

04

06

08

10

00 02 04 06 08 10Tr

ue p

ositi

ve ra





















1 71 69 30 29 2856 2799 1207 11832 71 70 30 29 2849 2816 1204 11903 72 69 30 29 2888 2791 1224 11794 71 70 30 29 2867 2836 1212 11975 71 70 30 29 2879 2816 1217 11916 71 70 30 29 2844 2817 1202 11917 71 70 30 29 2846 2807 1202 11878 71 70 30 29 2864 2815 1211 11899 71 70 30 29 2871 2811 1214 118910 71 70 30 29 2861 2839 12109 1199Avg 711 698 30 29 2863 2815 1211 1189










Testitems

No monitoring()

Monitoringenable ()

Resourceconsumed ()

CPU 267 27 003Memory 17477 18112 0635


Data Availability




Acknowledgments


References
















Iaas data center

Tenant domain





Physical server

OSapplications +

dataHypervisor VM




Userterminal


(i)(ii)

(iii)(iv)

(v)(vi)

(vii)


(i)

(ii)(iii)(iv)(v)

(vi)(vii)


(i)(ii)

(iii)

(iv)

(v)(vi)


(i)(ii)

(iii)(iv)

(v)(vi)


(i)

(ii)

(iii)

(iv)(v)














00020406081012

access_port

202530354045505560

ip_entro

00020406081012

bpstimes108

times103

times104

times104

times104

times107

0002040608101214

pps

00

02

04

06

08packet_per_ip

0010203040506070

ip_count

000000250050007501000125015001750200

service_port_entro

000002004006008010012014

syn_rate

075100125150175200225250

syn_per_ip

000510152025

syn_count

000025050075100125150175200

bytes_per_ip








ht ot ⊙fh ct( 1113857

(1)





















Text


ASC







ROdata

Low address

High address


T step

Full

conn

ecte

d la

yer 1

LSTM layer 1

LSTM layer 2

Full

conn

ecte

d la

yer 2

Out

put


MonitoredVM-A

Kernellayer

Application layer

KVM


code injection


Applications



Monitoring results


VM

mem-A











endswendsw

end




endend

























CPU1 223 1463 642 2275 998 355

Memory1 141 233 357 615 735 139

0

05

1

15

2

25

3

Dela

y tim

e (m

s)



1 14 27 40 53 66 79 92 105

118

131

144

157

170

183

196

209

222

235

248

261

274

287

300

313

326

339

352

365

378

391

404

417

430

443

456

469

482

495

508

521

534

547

560

573

586

599


Physical resource

layer


layer




Management domain


Storage domain

VM VM VM VM VM


Sub-domain

DMZ



Security services

Supporting domain



Physical server


Server

Network accesslayer

Network corelayer

Storage server KVM

Router Firewall

Physical server


Tenant domain







43 VM Monitoring









00

02

04

06

08

10

00 02 04 06 08 10Tr

ue p

ositi

ve ra





















1 71 69 30 29 2856 2799 1207 11832 71 70 30 29 2849 2816 1204 11903 72 69 30 29 2888 2791 1224 11794 71 70 30 29 2867 2836 1212 11975 71 70 30 29 2879 2816 1217 11916 71 70 30 29 2844 2817 1202 11917 71 70 30 29 2846 2807 1202 11878 71 70 30 29 2864 2815 1211 11899 71 70 30 29 2871 2811 1214 118910 71 70 30 29 2861 2839 12109 1199Avg 711 698 30 29 2863 2815 1211 1189










Testitems

No monitoring()

Monitoringenable ()

Resourceconsumed ()

CPU 267 27 003Memory 17477 18112 0635


Data Availability




Acknowledgments


References
















Iaas data center

Tenant domain





Physical server

OSapplications +

dataHypervisor VM




Userterminal


(i)(ii)

(iii)(iv)

(v)(vi)

(vii)


(i)

(ii)(iii)(iv)(v)

(vi)(vii)


(i)(ii)

(iii)

(iv)

(v)(vi)


(i)(ii)

(iii)(iv)

(v)(vi)


(i)

(ii)

(iii)

(iv)(v)

















ht ot ⊙fh ct( 1113857

(1)





















Text


ASC







ROdata

Low address

High address


T step

Full

conn

ecte

d la

yer 1

LSTM layer 1

LSTM layer 2

Full

conn

ecte

d la

yer 2

Out

put


MonitoredVM-A

Kernellayer

Application layer

KVM


code injection


Applications



Monitoring results


VM

mem-A











endswendsw

end




endend

























CPU1 223 1463 642 2275 998 355

Memory1 141 233 357 615 735 139

0

05

1

15

2

25

3

Dela

y tim

e (m

s)



1 14 27 40 53 66 79 92 105

118

131

144

157

170

183

196

209

222

235

248

261

274

287

300

313

326

339

352

365

378

391

404

417

430

443

456

469

482

495

508

521

534

547

560

573

586

599


Physical resource

layer


layer




Management domain


Storage domain

VM VM VM VM VM


Sub-domain

DMZ



Security services

Supporting domain



Physical server


Server

Network accesslayer

Network corelayer

Storage server KVM

Router Firewall

Physical server


Tenant domain







43 VM Monitoring









00

02

04

06

08

10

00 02 04 06 08 10Tr

ue p

ositi

ve ra





















1 71 69 30 29 2856 2799 1207 11832 71 70 30 29 2849 2816 1204 11903 72 69 30 29 2888 2791 1224 11794 71 70 30 29 2867 2836 1212 11975 71 70 30 29 2879 2816 1217 11916 71 70 30 29 2844 2817 1202 11917 71 70 30 29 2846 2807 1202 11878 71 70 30 29 2864 2815 1211 11899 71 70 30 29 2871 2811 1214 118910 71 70 30 29 2861 2839 12109 1199Avg 711 698 30 29 2863 2815 1211 1189










Testitems

No monitoring()

Monitoringenable ()

Resourceconsumed ()

CPU 267 27 003Memory 17477 18112 0635


Data Availability




Acknowledgments


References
















Iaas data center

Tenant domain





Physical server

OSapplications +

dataHypervisor VM




Userterminal


(i)(ii)

(iii)(iv)

(v)(vi)

(vii)


(i)

(ii)(iii)(iv)(v)

(vi)(vii)


(i)(ii)

(iii)

(iv)

(v)(vi)


(i)(ii)

(iii)(iv)

(v)(vi)


(i)

(ii)

(iii)

(iv)(v)




















Text


ASC







ROdata

Low address

High address


T step

Full

conn

ecte

d la

yer 1

LSTM layer 1

LSTM layer 2

Full

conn

ecte

d la

yer 2

Out

put


MonitoredVM-A

Kernellayer

Application layer

KVM


code injection


Applications



Monitoring results


VM

mem-A











endswendsw

end




endend

























CPU1 223 1463 642 2275 998 355

Memory1 141 233 357 615 735 139

0

05

1

15

2

25

3

Dela

y tim

e (m

s)



1 14 27 40 53 66 79 92 105

118

131

144

157

170

183

196

209

222

235

248

261

274

287

300

313

326

339

352

365

378

391

404

417

430

443

456

469

482

495

508

521

534

547

560

573

586

599


Physical resource

layer


layer




Management domain


Storage domain

VM VM VM VM VM


Sub-domain

DMZ



Security services

Supporting domain



Physical server


Server

Network accesslayer

Network corelayer

Storage server KVM

Router Firewall

Physical server


Tenant domain







43 VM Monitoring









00

02

04

06

08

10

00 02 04 06 08 10Tr

ue p

ositi

ve ra





















1 71 69 30 29 2856 2799 1207 11832 71 70 30 29 2849 2816 1204 11903 72 69 30 29 2888 2791 1224 11794 71 70 30 29 2867 2836 1212 11975 71 70 30 29 2879 2816 1217 11916 71 70 30 29 2844 2817 1202 11917 71 70 30 29 2846 2807 1202 11878 71 70 30 29 2864 2815 1211 11899 71 70 30 29 2871 2811 1214 118910 71 70 30 29 2861 2839 12109 1199Avg 711 698 30 29 2863 2815 1211 1189










Testitems

No monitoring()

Monitoringenable ()

Resourceconsumed ()

CPU 267 27 003Memory 17477 18112 0635


Data Availability




Acknowledgments


References
















Iaas data center

Tenant domain





Physical server

OSapplications +

dataHypervisor VM




Userterminal


(i)(ii)

(iii)(iv)

(v)(vi)

(vii)


(i)

(ii)(iii)(iv)(v)

(vi)(vii)


(i)(ii)

(iii)

(iv)

(v)(vi)


(i)(ii)

(iii)(iv)

(v)(vi)


(i)

(ii)

(iii)

(iv)(v)




















endswendsw

end




endend

























CPU1 223 1463 642 2275 998 355

Memory1 141 233 357 615 735 139

0

05

1

15

2

25

3

Dela

y tim

e (m

s)



1 14 27 40 53 66 79 92 105

118

131

144

157

170

183

196

209

222

235

248

261

274

287

300

313

326

339

352

365

378

391

404

417

430

443

456

469

482

495

508

521

534

547

560

573

586

599


Physical resource

layer


layer




Management domain


Storage domain

VM VM VM VM VM


Sub-domain

DMZ



Security services

Supporting domain



Physical server


Server

Network accesslayer

Network corelayer

Storage server KVM

Router Firewall

Physical server


Tenant domain







43 VM Monitoring









00

02

04

06

08

10

00 02 04 06 08 10Tr

ue p

ositi

ve ra





















1 71 69 30 29 2856 2799 1207 11832 71 70 30 29 2849 2816 1204 11903 72 69 30 29 2888 2791 1224 11794 71 70 30 29 2867 2836 1212 11975 71 70 30 29 2879 2816 1217 11916 71 70 30 29 2844 2817 1202 11917 71 70 30 29 2846 2807 1202 11878 71 70 30 29 2864 2815 1211 11899 71 70 30 29 2871 2811 1214 118910 71 70 30 29 2861 2839 12109 1199Avg 711 698 30 29 2863 2815 1211 1189










Testitems

No monitoring()

Monitoringenable ()

Resourceconsumed ()

CPU 267 27 003Memory 17477 18112 0635


Data Availability




Acknowledgments


References
















Iaas data center

Tenant domain





Physical server

OSapplications +

dataHypervisor VM




Userterminal


(i)(ii)

(iii)(iv)

(v)(vi)

(vii)


(i)

(ii)(iii)(iv)(v)

(vi)(vii)


(i)(ii)

(iii)

(iv)

(v)(vi)


(i)(ii)

(iii)(iv)

(v)(vi)


(i)

(ii)

(iii)

(iv)(v)


































CPU1 223 1463 642 2275 998 355

Memory1 141 233 357 615 735 139

0

05

1

15

2

25

3

Dela

y tim

e (m

s)



1 14 27 40 53 66 79 92 105

118

131

144

157

170

183

196

209

222

235

248

261

274

287

300

313

326

339

352

365

378

391

404

417

430

443

456

469

482

495

508

521

534

547

560

573

586

599


Physical resource

layer


layer




Management domain


Storage domain

VM VM VM VM VM


Sub-domain

DMZ



Security services

Supporting domain



Physical server


Server

Network accesslayer

Network corelayer

Storage server KVM

Router Firewall

Physical server


Tenant domain







43 VM Monitoring









00

02

04

06

08

10

00 02 04 06 08 10Tr

ue p

ositi

ve ra





















1 71 69 30 29 2856 2799 1207 11832 71 70 30 29 2849 2816 1204 11903 72 69 30 29 2888 2791 1224 11794 71 70 30 29 2867 2836 1212 11975 71 70 30 29 2879 2816 1217 11916 71 70 30 29 2844 2817 1202 11917 71 70 30 29 2846 2807 1202 11878 71 70 30 29 2864 2815 1211 11899 71 70 30 29 2871 2811 1214 118910 71 70 30 29 2861 2839 12109 1199Avg 711 698 30 29 2863 2815 1211 1189










Testitems

No monitoring()

Monitoringenable ()

Resourceconsumed ()

CPU 267 27 003Memory 17477 18112 0635


Data Availability




Acknowledgments


References
















Iaas data center

Tenant domain





Physical server

OSapplications +

dataHypervisor VM




Userterminal


(i)(ii)

(iii)(iv)

(v)(vi)

(vii)


(i)

(ii)(iii)(iv)(v)

(vi)(vii)


(i)(ii)

(iii)

(iv)

(v)(vi)


(i)(ii)

(iii)(iv)

(v)(vi)


(i)

(ii)

(iii)

(iv)(v)
















43 VM Monitoring









00

02

04

06

08

10

00 02 04 06 08 10Tr

ue p

ositi

ve ra





















1 71 69 30 29 2856 2799 1207 11832 71 70 30 29 2849 2816 1204 11903 72 69 30 29 2888 2791 1224 11794 71 70 30 29 2867 2836 1212 11975 71 70 30 29 2879 2816 1217 11916 71 70 30 29 2844 2817 1202 11917 71 70 30 29 2846 2807 1202 11878 71 70 30 29 2864 2815 1211 11899 71 70 30 29 2871 2811 1214 118910 71 70 30 29 2861 2839 12109 1199Avg 711 698 30 29 2863 2815 1211 1189










Testitems

No monitoring()

Monitoringenable ()

Resourceconsumed ()

CPU 267 27 003Memory 17477 18112 0635


Data Availability




Acknowledgments


References
















Iaas data center

Tenant domain





Physical server

OSapplications +

dataHypervisor VM




Userterminal


(i)(ii)

(iii)(iv)

(v)(vi)

(vii)


(i)

(ii)(iii)(iv)(v)

(vi)(vii)


(i)(ii)

(iii)

(iv)

(v)(vi)


(i)(ii)

(iii)(iv)

(v)(vi)


(i)

(ii)

(iii)

(iv)(v)

























1 71 69 30 29 2856 2799 1207 11832 71 70 30 29 2849 2816 1204 11903 72 69 30 29 2888 2791 1224 11794 71 70 30 29 2867 2836 1212 11975 71 70 30 29 2879 2816 1217 11916 71 70 30 29 2844 2817 1202 11917 71 70 30 29 2846 2807 1202 11878 71 70 30 29 2864 2815 1211 11899 71 70 30 29 2871 2811 1214 118910 71 70 30 29 2861 2839 12109 1199Avg 711 698 30 29 2863 2815 1211 1189










Testitems

No monitoring()

Monitoringenable ()

Resourceconsumed ()

CPU 267 27 003Memory 17477 18112 0635


Data Availability




Acknowledgments


References
















Iaas data center

Tenant domain





Physical server

OSapplications +

dataHypervisor VM




Userterminal


(i)(ii)

(iii)(iv)

(v)(vi)

(vii)


(i)

(ii)(iii)(iv)(v)

(vi)(vii)


(i)(ii)

(iii)

(iv)

(v)(vi)


(i)(ii)

(iii)(iv)

(v)(vi)


(i)

(ii)

(iii)

(iv)(v)












Data Availability




Acknowledgments


References
















Iaas data center

Tenant domain





Physical server

OSapplications +

dataHypervisor VM




Userterminal


(i)(ii)

(iii)(iv)

(v)(vi)

(vii)


(i)

(ii)(iii)(iv)(v)

(vi)(vii)


(i)(ii)

(iii)

(iv)

(v)(vi)


(i)(ii)

(iii)(iv)

(v)(vi)


(i)

(ii)

(iii)

(iv)(v)












extensionofresearchonsecurityasaserviceforvmsin...

Documents