an investigative analysis of quality of service metrics on openstack · 2018-07-03 · an...

An Investigative Analysis of Qualityof Service Metrics on OpenStack

Edna Dias Canedo(B), Italo Paiva Batista, Emilie Trindade de Morais,and Aleteia Patricia Favacho de Araujo

University of Brasılia - UnB, P.O. Box 4466, Brasılia, DF 70910-900, Brazil{ednacanedo,aleteia}@unb.br, [email protected],

[email protected]

Abstract. Cloud computing is a paradigm that is becoming widespreadbecause of its benefits. The cloud services are structured in three mod-ules: Software as a Service (SaaS), Platform as a Service (PaaS) andInfrastructure as a Service (IaaS). This last one use the hardwareresources to provide services on virtual machines. A lot of tools have beencreated to support this type of service. OpenStack is an open-source toolcreated to provide cloud services on IaaS models through several inter-related projects. An important point of attention in cloud services isthe Quality of Service (QoS) and cloud computing tools should providemetrics to monitor and ensure the QoS. The purpose of this work wasto gather the QoS metrics provided the OpenStack and compare themto the QoS metrics for cloud computing services found in the literature.The metrics show that OpenStack provides approximately 24.25% of themetrics found in the literature.

Keywords: Metrics · Quality of service · Infrastructure as a serviceCloud computing · OpenStack

1 Introduction

The cloud computing frameworks and environments are able to address differentissues in current distributed and ubiquitous computing systems. The availabilityof infrastructure as a service and platform as a service environments provided afundamental base for building cloud computing based applications.

Cloud Computing is an on-demand resource delivering paradigm, provid-ing infrastructure, platforms and software. This paradigm has widely due to itsbenefits, like resource scalability, software flexibility, pay-per-use, consolidatedmanagement and maintenance system, reliability and reduction of carbon emis-sions [19]. With the spread of the use of cloud computing services, some studies[2,12,16,18,22,23] have been developed in order to help choose the more appro-priate service by analyzing intrinsic aspects of cloud computing services andQuality of Service (QoS). [24] states that the quality of service in cloud comput-ing is critical, but is hard to analyze.c© Springer International Publishing AG, part of Springer Nature 2018O. Gervasi et al. (Eds.): ICCSA 2018, LNCS 10960, pp. 337–352, 2018.https://doi.org/10.1007/978-3-319-95162-1_23

http://crossmark.crossref.org/dialog/?doi=10.1007/978-3-319-95162-1_23&domain=pdf

338 E. D. Canedo et al.

According to [14], measurements support management, processes and prod-uct improvement. It is a key discipline in quality assessment and, according to[17], cloud computing services measurement is one of the essential characteristics.

The work presented by [5] states that measurements allow transparency toboth provider and client, and that measuring cloud services helps to accomplishthe agreed service level agreements.

OpenStack [11] is an open source, cloud computing platform that allows themanagement and development of cloud computing infrastructure in a datacenter.It is currently maintained by the OpenStack Foundation and is used by severalcompanies [4,10]. In this context of cloud service selection and the increasinguse of OpenStack, the research question of this study is:

1. Which QoS metrics for cloud computing services proposed in theliterature are provided by OpenStack?

The goal of this paper was to identify QoS metrics, proposed in the literature,available natively in OpenStack.

The article is organized as follows. Section 2, Cloud Computing Servicespresents some cloud computing concepts and its quality are presented and ageneral view of the OpenStack tool is presented. In the Materials and Methods,Sect. 3 are presented the stages involved in this work accomplishment, as well asthe used methods. In Sect. 4, the work’s execution steps are presented. Finally,the conclusions are presented in Sect. 5.

2 Cloud Computing Services

According to [1] cloud computing refers to applications that deliver services usingthe Internet. These services are provided using the hardware and software of datacenters. [20] conceptualize cloud computing as services that offers mechanismsto provide virtual access to unlimited resources based on Pay-per-use model.

Cloud computing is structured in three modules: Software as a Service (SaaS),Platform as a Service (PaaS) and Infrastructure as a Service (IaaS), ordered byabstraction level, from high to low, respectively. SaaS provides all the functionsof a traditional application, but provides access to specific applications throughInternet. The SaaS mod-el reduces concerns with application servers, operatingsystems, storage, application development, etc. Hence, developers may focus oninnovation, and not on infrastructure, leading to faster software systems develop-ment. SaaS systems re-duce costs since no software licenses are required to accessthe applications. In-stead, users access services on demand. Since the software ismostly Web based, SaaS allows better integration among the business units of agiven organization or even among different software services. PaaS is the middlecomponent of the service layer in the cloud. It offers users software and servicesthat do not require downloads or installations. PaaS provides an infrastructurewith a high level of integration in order to implement and test cloud applications.The user does not manage the infra-structure (including network, servers, oper-ating systems and storage), but he controls deployed applications and, possibly,

An Investigative Analysis of Quality of Service Metrics on OpenStack 339

their configurations. PaaS provides an operating system, programming languagesand application programming environments [17].

IaaS is the portion of the architecture responsible for providing the infras-tructure necessary for PaaS and SaaS. Its main objective is to make resourcessuch as servers, network and storage more readily accessible by including appli-cations and operating systems. Thus, it offers basic infrastructure on-demandservices. IaaS has a unique interface for infrastructure management, an Appli-cation Programming Interface (API) for interactions with hosts, switches, androuters, and the capability of adding new equipment in a simple and transpar-ent manner. In general the, user does not manage the underlying hardware inthe cloud infra-structure, but he controls the operating systems, storage anddeployed applications. Eventually he can also select network components suchas firewalls. The term IaaS refers to a computing infrastructure, based on virtu-alization techniques that can scale dynamically, increasing or reducing resourcesaccording to the needs of applications. The main benefit provided by IaaS is thepay-per-use business model [17].

According to [5,17], cloud services has the following characteristics: self-service on demand, large network access, resource grouping, fast elasticity andservice measurement. The work presented by [20] characterizes the IaaS as ahardware resource provided on virtual machines. The client maintains the appli-cations, databases and servers while the server maintains the cloud virtualiza-tion, hardware, storage and networks.

To [3] the IaaS model is the service of delivering hardware (server, stor-age and network) and associated software (files system and systems virtual-ization). The authors established the following services as IaaS: virtual infras-tructure (server, storage and network); deployment of web-based applicationswith easy on-demand availability; load balancing; establishing service-level agree-ments the clients; CPU, data and network security; and management and accountprovisioning.

2.1 Quality of Service

Quality of Service (QoS) denotes the levels of performance, reliability, and avail-ability offered by an application and by the platform or infrastructure. QoS isfundamental for cloud users, who expect providers to deliver the advertised qual-ity characteristics, and for cloud providers, who need to find the right trade-offsbetween QoS levels and operational costs.

The QoS in cloud computing can be considered as the performance of theprovided service in a general mode [10]. The work presented by [5] characterizesthe QoS in two approaches: network and application. The network approachdeals with the requirements to ensure the QoS. The application approach dealswith the attributes that on the fulfill of the service level agreement. The QoScan be divided into two points of view: the client’s and the provider’s [3,10]. Thescope of this work is focused only on the metrics for the provider’s point of viewwhich, is more concerned with the cloud is overall performance.


2.2 OpenStack

OpenStack is a tool that provides cloud services an IaaS model. It consists ofa group of interrelated sub-projects that control a set of processing resources(Nova subproject), storage (Cinder and Swift sub-projects), and network (Neu-tron subproject). Each subproject is a module of OpenStack and forms a com-puting cloud infrastructure [4,10].

OpenStack also provides some shared services between the different modules,such as the authentication and authorization service (Keystone subproject), theimage service (Glance subproject) and the telemetry service (Ceilometer sub-project). These are just the main services provided by OpenStack, the completedescription of all its services can be seen in the tool’s documentation [10].

OpenStack is fairly robust. It is capable of a large number of computationalresources in a datacenter. This characteristic requires good hardware infrastruc-ture for full operation. The cloud infrastructure management and control can beperformed using a web application (Horizon subproject), via the command lineand/or RESTful APIs [4,10].

The internal module integration is performed by RESTful APIs and RemoteProcedure Call (RPC) via RabbitMQ (a tool that implements the inter-processcommunication following the Advanced Message Queue Protocol) [4]. Moredetails about the module integration can be seen in the study made by [4].Figure 1 shows the conceptual architecture of OpenStack, presenting the rela-tionship between the modules.

3 Materials and Methods

The methodology used in this study was organized into five steps as shown inFig. 2.

– Collection of existing metrics: This step consisted of researching the QoSmetrics for cloud services in the literature and documenting them, using aliterature review;

– Metrics filtering: From the metrics identified in the literature review, thisstep involved in filtering the results according to the previously defined cri-teria;

– Metrics comparison: To verify which metrics proposed in the literaturewere implemented natively in OpenStack, this step verified the existence ofthe filtered metrics in OpenStack, based on the documentation and the execu-tion of the tool. The way that the tool presents the metrics was also identified.In addition, the metrics provided by OpenStack that were not found in theliterature, but were related to the metrics found, were identified and docu-mented;

– Metrics exemplification: This step consisted of verifying if the comparedmetrics were in the tool and how they were provided. The verification wasperformed through the simple collection of the identified metrics present inOpenStack with the execution of the tool in a test scenario; This step alsoinvolved the following activities:


– Characterize scenario/context: In this activity the context of the exe-cution and the test scenario were described. The scenario was describedconsidering items as the environment for the execution, tool version,topology and available tools and resources.

– Collect data: In this activity the identified metrics in the previous stepwere collected and documented.

– Analysis and results report: In this step the results obtained in the pre-vious step were interpreted and then documented.

Fig. 1. Conceptual architecture of the OpenStack [10]

Fig. 2. Proposed methodology.


4 Execution

A literature review was performed in order to find existing QoS metrics proposedin other studies. Six papers that explicitly presented QoS metrics for cloud ser-vices were found. These papers were analyzed and 68 metrics identified, as canbe seen in the Table 1. The papers found are:

– Paper 1: QoS Metrics for Cloud Computing Services Evaluation [2];– Paper 2: On a Catalogue of Metrics for Evaluating Commercial Cloud Ser-vices [16];

– Paper 3: IaaS Cloud Service Selection using Case-Based Reasoning [22];– Paper 4: SMICloud: A Framework for Comparing and Ranking Cloud Ser-vices [12];

– Paper 5: Cloud Services Measures for Global Use [21];– Paper 6: An End-To-End QoS Mapping Approach for Cloud Service Selec-tion [15].

Four criteria were defined to filter the metrics based on the context and scopeof the work. The evaluation of the collected metrics can be seen in Table 2. Thecriteria explanation follows:

– Criterion 1 - Is the meaning known?– This criterion was used to assess if the meaning of the metric and its

calculation method were known.– Criterion 2 - Does it apply to the IaaS service model?

– This criterion was used to select metrics most applicable to the IaaSmodel, due to its singular characteristics.

– Criterion 3 - Can it be collected objectively?– This criterion was used to select the metrics that can be collected objec-

tively, because this kind of metric can be collected using automated tools.– Criterion 4 - Is it from the provider’s perspective?

– This criterion was used to select which metrics were related to the cloudcomputing service provider is point of view (Table 3).

4.1 Metrics Comparison

After the filtering step, 34 metrics remained to be compared against Open-Stack. In order to accomplish this, the metrics were divided into static anddynamic metrics. Static metrics do not require tool execution to be collected,while dynamic metrics do. Furthermore, the metrics were categorized based onthe study performed by [2].

The categories used were: Reliability, Economy, Memory, Performance, Secu-rity and General. It is worth mentioning that two equivalent metrics were identi-fied: Transfer Speed bit/byte TCP/UDP/IP and Bandwidth (download, upload).Considering this, only bandwidth was used in the comparison.


Table 1. Metrics found in the literature

Metric Category Paper

Cryptography algorithms Security Paper 6

Service learnability Usability

Access control Security

Data control Data control

Defects per million (DPM) -

Ease of use Usability

IaaS instalability Usability

Data privacy Security

Mean time between failures (MTBF) Reliability

Access control policies Security/privacy Paper 5

VM Cost Cost Paper 5Paper 4

Storage (HD) - Paper 3

NoSQL databases support -

Application layers -

CPU frequency -

I/O performance -

Bandwidth (download, upload) -

Application maximum latency -

Number of load balancers -

Number of instances -

Number of CPU cores -

Concurrent users maximum number -

Region -

Application servers -

Operational System (Type,Platform, Version)

-

Relational database support -

Application type -

Security (general metric) - Paper 3Paper 6

Memory (RAM required for theinstance)

Reliability/sustainability Paper 3Paper 4

Application response time Performance

Availability - Paper 3Paper 4Paper 6

(continued)


Table 1. (continued)

Metric Category Paper

Transfer latency TCP/UDP/IP Performance/communication Paper 2

Transfer speed bit/byteTCP/UDP/IP

Performance/communication

Confidentiality Security/authentication Paper 1

Effectiveness Security/authentication

Auditability Security/data security

Modularity -

Sensitivity Security/authentication

Boot time Economy/elasticity

Computation time Performance

Resources utilization Efficiency

Memory speed Performance/memory

CPU load Performance/computation Paper 1Paper 2

Component resources cost Economy/cost

FLOP cost Economy/cost

Fixed time cost Economy/cost

Total cost Economy/cost

Instance efficiency Performance/computation

Packet loss frequency Performance/communication

Latency over SLL Security/data security

Price/performance ratio Economy/cost

Is SSL applicable? Security/data security

RAM update rate Performance/memory

Connection errors rate Performance/communication

FLOPs (Float Point Operations PerSecond) rate

Performance/computation

Communication time Performance

Deletion time Economy/elasticity

Deploy time Economy/elasticity

Suspension time Economy/elasticity

Acquisition total time Economy/elasticity

Supported users on a fixed budget Economy/cost

Interoperability -

Memory reponse time Performance/memory

Accuracy - Paper 4

Adequacy -

Reliability (errors numbers) -

Elasticity -

Transparency -


Table 2. Metrics evaluation according to defined criteria (approved metrics).

Metric Criterion 1 Criterion 2 Criterion 3 Criterion 4

Transfer latency TCP/UDP/IP Yes Yes Yes Yes

Bandwidth (download, upload) Yes Yes Yes Yes

CPU load Yes Yes Yes Yes

Reliability (errors numbers) Yes Yes Yes Yes

Access control Yes Yes Yes Yes

CPU frequency Yes Yes Yes Yes

Total cost Yes Yes Yes Yes

Instance efficiency Yes Yes Yes Yes

Elasticity Yes Yes Yes Yes

Packet loss frequency Yes Yes Yes Yes

Auditability Yes Yes Yes Yes

Latency over SLL Yes Yes Yes Yes

Application maximum latency Yes Yes Yes Yes

Concurrent users maximumnumber

Yes Yes Yes Yes

Memory (RAM required for theinstance)

Yes Yes Yes Yes

NoSQL databases support Yes Yes Yes Yes

Number of CPU cores Yes Yes Yes Yes

Number of instances Yes Yes Yes Yes

Number of load balancers Yes Yes Yes Yes

Operational System (Type,Platform, Version)

Yes Yes Yes Yes

Relational database support Yes Yes Yes Yes

Memory response time Yes Yes Yes Yes

Is SSL applicable? Yes Yes Yes Yes

Storage (HD) Yes Yes Yes Yes

Boot time Yes Yes Yes Yes

Communication time Yes Yes Yes Yes

Deletion time Yes Yes Yes Yes

Deploy time Yes Yes Yes Yes

Application response time Yes Yes Yes Yes

Suspension time Yes Yes Yes Yes

Mean time between failures(MTBF)

Yes Yes Yes Yes

Acquisition total time Yes Yes Yes Yes

Transfer speed bit/byteTCP/UDP/IP

Yes Yes Yes Yes

I/O performance Yes Yes Yes Yes


Table 3. Metrics evaluation according to defined criteria (rejected metrics).

Metric Criterion 1 Criterion 2 Criterion 3 Criterion 4Accuracy Yes Yes Yes NoAvailability Yes Yes Yes NoAdequacy Yes No - -Service learnability Yes Yes No -Data Control Yes Yes No -Fixed time cost Yes Yes No -Defects per million Yes No - -Ease of use Yes Yes No -IaaS instalability Yes Yes No -Interoperability Yes Yes No -Access Control Policies Yes Yes No -Transparency Yes Yes No -VM Cost Yes Yes No -Cryptography algorithms No - - -Application servers No - - -Application layers No - - -Application type No - - -Confidentiality No - - -Component resources cost No - - -FLOP cost No - - -Effectiveness No - - -Modularity No - - -Data privacy No - - -Price/Performance ratio No - - -Region No - - -Security (general metric) No - - -Sensitivity No - - -RAM update rate No - - -FLOPs (Float Point OperationsPer Second) rate

No - - -

Computation time No - - -Supported Users on a Fixed Bud-get

No - - -

Resources utilization No - - -Memory speed No - - -Connection errors rate No - - -

The categorization and the metrics comparison is presented in Tables 4 and 5that represents the static metrics and dynamic metrics, respectively.

The existing metrics in OpenStack can be summarized in the follow items.Besides the identical metrics, some related metrics were also identified. All met-rics provided by the tool can be consulted in the documentation [10]. The metricspresented with the dot notation (‘some.metric.presented’), this is the name aspresented in OpenStack by its telemetry module.


Table 4. Evaluation of the static metrics in OpenStack.

Statics

Category Metrics Is it in OpenStack?

Economy Total cost -

General NoSQL databases support Yes

Operational System (Type, Platform, Version) Yes

Relational database support Yes

Access control policies Yes

Security Is SSL applicable? Yes

Auditability Yes

– Memory (RAM required for the instance)– Volume of RAM allocated to the instance (memory);– Volume of RAM used by the instance from the amount of its allocated

memory (memory.usage);– Volume of RAM used per project.

– Storage (HD);– Size of root disk (disk.root.size);– Permanent memory used per project.

– Application maximum latency– Average disk latency (disk.latency);– Average disk latency per device (disk.device.latency).

– Number of CPU cores– Number of virtual CPUs allocated to the instance (vcpus);– Number of virtual CPUs per project;– Average CPU utilization (cpu util).

– Number of instances– Number of active instances.

– Number of load balancers– Existence of a LB pool (network.services.lb.pool);– Existence of a LB VIP (network.services.lb.vip);– Existence of a LB member (network.services.lb.member);– Existence of a LB health probe (network.services.lb.health monitor);– Total connections on a LB (network.services.lb.total.connections);– Active connections on a LB (network.services.lb.active.connections).

– Bandwidth (download, upload)– Bytes through this l3 metering label (Network layer routers) (bandwidth).

– CPU load– CPU load in the past 1 min (hardware.cpu.load.1 min);– CPU load in the past 5 min (hardware.cpu.load.5 min);– CPU load in the past 10 min (hardware.cpu.load.10 min).

– CPU frequency– CPU frequency (compute.node.cpu.frequency).


Table 5. Evaluation of the dynamic metrics in OpenStack.

Dynamics

Category Metrics Is it in OpenStack?

Reliability Reliability (errors numbers) No

Mean time between failures (MTBF) No

Economy Elasticity No

Boot time No

Deletion time No

Deploy time No

Suspension Time No

Acquisition total time No

Memory Memory (RAM required for the instance) Yes

Storage (HD) Yes

Performance Application maximum latency No

Concurrent users maximum number No

Number of CPU cores No

Number of instances Yes

Number of load balancers No

Application response time No

Bandwidth (download, upload) Yes

CPU load Yes

CPU frequency Yes

Instance efficiency Yes

Packet loss frequency No

Security Transfer latency TCP/UDP/IP No

I/O Performance Yes

Memory response time No

Communication time No

Latency over SLL No

– Instance efficiency– CPU utilization (compute.node.cpu.percent).

– Packet loss frequency– Number of incoming packets (network.incoming.packets);– Average rate of incoming packets (network.incoming.packets.rate);– Number of outgoing packets (network.outgoing.packets);– Average rate of outgoing packets (network.outgoing.packets.rate).

– I/O Performance– Number of read requests (disk.read.requests);– Average rate of read requests (disk.read.requests.rate);


– Number of write requests (disk.write.requests);– Average rate of write requests (disk.write.requests.rate);– Volume of reads (disk.read.bytes);– Average rate of reads (disk.read.bytes.rate);– Volume of writes (disk.write.bytes);– Average rate of writes (disk.write.bytes.rate).

4.2 Metrics Exemplification

In order to verify how the presented metrics were presented in OpenStack, atool usage test scenario was performed to collect the metrics shown in Sect. 4.1.Subsect. 4.2 describes scenario used and Subsect. 4.2 discusses the obtainedresults.

Test Scenario. The data collection was performed in the following scenario/-context:

– Laboratory with machines that had simple hardware features;– Maximum of 4 GB of RAM.

– OpenStack Kilo version;– Cent OS 7 operational system;– Use of the topology of an OpenStack installation (Fig. 3);

– Two interconnected computers were used to distribute the processing loadbetween them so that a computer had the Controller node and the othercomputer had the Processing (Compute) and the Network nodes as shownin Fig. 3;

– The topology used was based on the topology proposed by AhmadImanudin in [13];

– Creation of two instances.

Fig. 3. Topology used in OpenStack installation. Source: Authors.

Two instances (using 512 MB of RAM and operational system CirrOS) werecreated in OpenStack as a test scenario to collect data. The topology used astest scenario can be seen in Fig. 4.


Fig. 4. Topology used as test scenario. Image taken from OpenStack.

Table 6. Collected metrics in test scenario.

OpenStack metric Max Min Average Unitcpu util 16.11 0 7.9 %memory 512 512 512 MBmemory.usage - - - MBdisk.root.size 1 1 1 GBvcpus 1 1 1 n vCPUsnetwork.incoming.packets 19 8 11.75 packets

network.incoming.packets.rate

0.0016 0.00011 0.00011 packets/s

network.outgoing.packets

9 9 9 packets

network.outgoing.packets.rate

0 0 0 packets/s

disk.read.requests 1122 877 1032.77 requestsdisk.read.requests.rate

0.23 0 0.015 requests/s

disk.write.requests 134 55 111.9 requestsdisk.write.requests.rate

0.091 0 0.001 requests/s

disk.read.bytes 20436992 18510848 20245043.2 bytesdisk.read.bytes.rate 3700.63 0 206.49 bytes/sdisk.write.bytes 397312 206848 348262.4 bytesdisk.write.bytes.rate 268.85 0 22.14 bytes/s


The Kilo [8] version of OpenStack was used because it is a stable and lighterversion, necessary to the limited resources available. It is worth mentioning thatalthough the support for the Kilo version has been terminated by the OpenStackmaintainers, its architecture is not far from its successor, OpenStack Liberty,which does not significantly impact or invalidate this work [9].

Results Obtained. The metrics were collected through the OpenStack datacollection service (Ceilometer module) and web application (Horizon module).With the test scenario fulfilled, the collected metrics can be seen in Table 6.

It is possible to see that OpenStack is a very powerful tool that provides agood measurement service (Ceilometer module). In the simple scenario (creationof two instances in the tool) several metrics are generated and can be monitoredto guarantee QoS.

5 Conclusions

With a literature review it was possible to collect 68 metrics of QoS into cloudservices. In OpenStack, 34 of these metrics were evaluated. In the evaluationit was found that OpenStack provides approximately 24.25% of the metrics, aswell as, providing additional related metrics.

The number of metrics can be related to the fact that OpenStack has aspecific module to collect metrics. The results obtained in this study can beused in other related studies and for tools that provide a cloud services selection,supporting the collection of possible metrics that can be found in OpenStack.

Beyond the telemetry module, OpenStack offers some mechanisms of qualityof service assurance, such as: establishing limits for files size, RAM memory,instances and virtual CPUs; creating security rules to limit specific traffic oninstances; and the creation of QoS rules to limit bandwidth on instances and toassure a minimum bandwidth [6,7].

An identified limitation of this study is that the metrics were identified usinga simple literature review. In this way, it was not possible to obtain the visibilityof a higher number of studies. The use of an old version of the tool was identifiedas another limitation.

Future work could involve a systematic review to find more metrics and thento apply them to a most current version of OpenStack.

References

1. Armbrust, M., Fox, A., Griffith, R., Joseph, A.D., Katz, R., Konwinski, A., Lee, G.,Patterson, D., Rabkin, A., Stoica, I., et al.: A view of cloud computing. Commun.ACM 53(4), 50–58 (2010)

2. Bardsiri, A.K., Hashemi, S.M.: QoS metrics for cloud computing services evalua-tion. Int. J. Intell. Syst. Appl. 6(12), 27 (2014)

3. Bhardwaj, S., Jain, L., Jain, S.: Cloud computing: a study of infrastructure as aservice (IaaS). Int. J. Eng. Inf. Technol. 2(1), 60–63 (2010)


4. Bui, T.A.: Cloud network performance analysis: an OpenStack case study. Master’sthesis, Universite Catholique de Louvain (2016)

5. Filho, L., Machado, D.: A influencia das informacoes de hardware e software nosservicos de IaaS: uma abordagem sobre desempenho de maquinas virtuais emnuvem. Ph.D. thesis, Universidade de Sao Paulo (2016)

6. O. Foundation. Neutron QoS API Models and Extension7. O. Foundation. OpenStack neutron QoS8. O. Foundation. OpenStack kilo (2015)9. O. Foundation. OpenStack Liberty Release Notes (2015)

10. O. Foundation. OpenStack Kilo Installation Guide, May 201611. O. Foundation. OpenStack Website (2016)12. Garg, S.K., Versteeg, S., Buyya, R.: SMICloud: a framework for comparing and

ranking cloud services. In: 2011 Fourth IEEE International Conference on Utilityand Cloud Computing (UCC), pp. 210–218. IEEE (2011)

13. Imanudin, A.: How to Install OpenStack Kilo Multi-node (2015)14. Systems and software engineering - Measurement process. Standard, International

Organization for Standardization, Switzerland, October 200715. Karim, R., Ding, C., Miri, A.: An end-to-end QoS mapping approach for cloud

service selection. In: 2013 IEEE Ninth World Congress on Services, pp. 341–348.IEEE (2013)

16. Li, Z., O’Brien, L., Zhang, H., Cai, R.: On a catalogue of metrics for evaluatingcommercial cloud services. In: Proceedings of the 2012 ACM/IEEE 13th Inter-national Conference on Grid Computing, pp. 164–173. IEEE Computer Society(2012)

17. Mell, P., Grance, T.: The NIST Definition of Cloud Computing (2011)18. Quarati, A., Clematis, A., D’Agostino, D.: Delivering cloud services with QoS

requirements: business opportunities, architectural solutions and energy-savingaspects. Future Gener. Comput. Syst. 55, 403–427 (2016)

19. Rehman, M.S., Sakr, M.F.: Teaching the cloud-experiences in designing and teach-ing an undergraduate-level course in cloud computing at the Carnegie MellonUniversity in Qatar. In: 2011 IEEE Global Engineering Education Conference(EDUCON), pp. 875–879. IEEE (2011)

20. Sefraoui, O., Aissaoui, M., Eleuldj, M.: OpenStack: toward an open-source solutionfor cloud computing. Int. J. Comput. Appl. 55(3), 260–276 (2012)

21. Siegel, J., Perdue, J.: Cloud services measures for global use: the service measure-ment index (SMI). In: 2012 Annual SRII Global Conference, pp. 411–415. IEEE(2012)

22. Soltani, S.: IaaS Cloud Service Selection Using Case-Based Reasoning (2016)23. Sun, L., Dong, H., Hussain, F.K., Hussain, O.K., Chang, E.: Cloud service selection:

state-of-the-art and future research directions. J. Netw. Comput. Appl. 45, 134–150 (2014)

24. Xia, Y., Zhou, M., Luo, X., Zhu, Q.: A comprehensive QoS determination modelfor infrastructure-as-a-service clouds. In: 2013 IEEE International Conference onAutomation Science and Engineering (CASE), pp. 122–127. IEEE (2013)

an investigative analysis of quality of service metrics on openstack · 2018-07-03 · an...

Documents