a market design for iaas cloud resources

A Market Design forIaaS Cloud Resources

Proefschrift voorgelegd op 30 mei 2011 tot het behalen vande graad van Master in de Wetenschappen,bij de faculteit Wetenschappen, aan de UniversiteitAntwerpen.

Promotoren:prof. Dr. Jan BroeckhoveDr. Kurt Vanmechelen

Joris Roovers

Research Group ComputationalModelling and Programming

Contents

List of Figures ii

List of Tables iv

Preface vi

Abstract 1

1 Introduction 21.1 Cloud Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 IaaS in Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.3 IaaS Cloud Markets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201.4 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271.5 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291.6 Thesis outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2 Market Design Principles 322.1 Introduction to Markets . . . . . . . . . . . . . . . . . . . . . . . . . . 322.2 Financial Markets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342.3 Auction Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402.4 Desirable Market Properties . . . . . . . . . . . . . . . . . . . . . . . . 442.5 Designing a Commodity Market . . . . . . . . . . . . . . . . . . . . . . 47

3 IaaS Cloud Computing Market: The Continuous Reverse Auction 603.1 Continuous Double Auction for IaaS Resources . . . . . . . . . . . . . 613.2 Issues with the CDA model for IaaS resources . . . . . . . . . . . . . . 783.3 The Continuous Reverse Auction for IaaS Resources . . . . . . . . . . . 85

4 Market Implementation 1014.1 Implementation of the Continuous Reverse Auction . . . . . . . . . . . 101

i

4.2 Implementing Tags and Constraints . . . . . . . . . . . . . . . . . . . . 1064.3 Implementation of the prototype . . . . . . . . . . . . . . . . . . . . . 113

5 Conclusion 1175.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1175.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

Appendices 121

Appendix A Cloud Computing in Practice: SaaS and PaaS 122A.1 Software-as-a-Service . . . . . . . . . . . . . . . . . . . . . . . . . . . 122A.2 Platform-as-a-Service . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

Appendix B Compound Requests 126

ii

List of Figures

1.1 Difficulty of accurately provisioning IT infrastructure . . . . . . . . . 41.2 Onion representation of the cloud computing hierarchy . . . . . . . . 71.3 Private, Public and Hybrid clouds . . . . . . . . . . . . . . . . . . . 91.4 Total costs of using a reserved small instance vs. using an on-demand

instance on Amazon’s EC2 platform . . . . . . . . . . . . . . . . . . 141.5 Evolution of the spot price of a small instance on Amazon’s EC2

platform in the EU-region . . . . . . . . . . . . . . . . . . . . . . . . 151.6 Retail competition model of the electricity market . . . . . . . . . . 231.7 Retail IaaS market based on the electricity retail market. . . . . . . 26

2.1 Evolution of the electricity spot price for 10th of February 2011 onthe European Energy Exchange. . . . . . . . . . . . . . . . . . . . . 38

2.2 Market actors in an commodity market . . . . . . . . . . . . . . . . 492.3 Market Settlement: On-demand Resources . . . . . . . . . . . . . . . 572.4 Market Settlement: Futures Resources . . . . . . . . . . . . . . . . . 572.5 Market Settlement: Reserved Resources . . . . . . . . . . . . . . . . 572.6 Market Settlement: Spot Resources . . . . . . . . . . . . . . . . . . . 58

3.1 Overview of the Continuous Double Auction for IaaS Resources . . . 733.2 Importance of taking into account the usage of additional costs com-

ponents (1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 803.3 Importance of taking into account the usage of additional costs com-

ponents (2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 813.4 Schematic overview of the workings of the Continuous Reverse Auction 863.5 Price discovery in a CRA-based market . . . . . . . . . . . . . . . . 923.6 Hierarchical market setup to reduce CRA communication bottleneck 99

4.1 Implementation of a CRA-based market using webservices . . . . . . 1024.2 Passing encrypted tickets between the PriceRequestService and the

ExecutionService via the consumer . . . . . . . . . . . . . . . . . . 106

iii

4.3 UML diagram of the webservices of a CRA-based IaaS market . . . 1074.4 UML diagrams of a Tag and TagSet . . . . . . . . . . . . . . . . . . 1074.5 UML diagrams of a ConstraintDescription and ContraintParameters

that specify a standard constraint . . . . . . . . . . . . . . . . . . . . 1084.6 Using a constraint lookup service to determine constraint semantics 1094.7 Implementing a standard constraint library using reflection . . . . . 1104.8 Standardized tags in the standardized constraint library . . . . . . . 1124.9 Implementation of custom constraints using webservices . . . . . . . 1124.10 Prototype deployment . . . . . . . . . . . . . . . . . . . . . . . . . . 1144.11 Screenshot of the resource specification user interface as offered by a

broker to a consumer . . . . . . . . . . . . . . . . . . . . . . . . . . . 1164.12 Screenshot of the user interface offered to administrators to monitor

and manage the market . . . . . . . . . . . . . . . . . . . . . . . . . 116

iv

List of Tables

1.1 Examples of Amazon EC2 instance types . . . . . . . . . . . . . . . 131.2 GoGrid Cloud Server sizes . . . . . . . . . . . . . . . . . . . . . . . . 181.3 Comparison of the electrical power grid with current IaaS clouds . . 22

2.1 Well-known stock exchanges . . . . . . . . . . . . . . . . . . . . . . . 342.2 Well-known stock market indices . . . . . . . . . . . . . . . . . . . . 352.3 Well-known commodity exchanges . . . . . . . . . . . . . . . . . . . 37

3.1 Examples of additional constraints that are regularly used when de-scribing IaaS resources. . . . . . . . . . . . . . . . . . . . . . . . . . 66

3.2 Bandwidth prices of different IaaS providers . . . . . . . . . . . . . . 783.3 Resources and services that are commonly charged separately by IaaS

providers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 793.4 Provider specified table that defines the relations between usage and

price tags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

4.1 Technologies used to implement the prototype . . . . . . . . . . . . . 115

v

Preface

The writing of a thesis is an essential part of the curriculum of the Master of Com-puter Science programme at the University of Antwerp, as it fulfills the role ofdemonstrating that the author is able to autonomously study and reason about acomplex scientific subject. The particular subject of this subject, “A Market De-sign for IaaS Cloud Resources”, was chosen by me as I have always been intriguedwith the potential capabilities of cloud computing, as well as by the fact that I havealways had a strong interest in financial and commodity markets. To add a goal-driven aspect to the thesis, I decided early on to focus on the design of market thathas concrete applicability in today’s IaaS ecosystem. In order to achieve this, anincremental approach has been used; starting from abstract and simple models anditeratively taking more complexity into account. The final result of this approach isthis thesis document and an accompanying prototype implementation.

When it comes to acknowledgements, I first and foremost wish to thank mypromoter Prof. Dr. Jan Broeckhove, co-promoter Dr. Kurt Vanmechelen andmentor Ruben Van den Bossche. In particular, I wish to express sincere gratitudeto Dr. Kurt Vanmechelen who helped me tremendously in shaping and orderingmy thoughts; I am certain his help made a huge difference on this thesis’ outcome.I would also like to thank my mother and father, and my two sisters Marjoleinand Annelies for their continuous moral and emotional support through all of thepast years. My gratitude goes also towards many close friends, in particular Ilseand Geert, for understanding my daily challenges and doubts as well as pulling methrough on more difficult moments.

Finally, I wish to express very sincere gratitude and appreciation to my fellowstudents, without whom I would not have been able to get through the past years.I wish to mention Kurt, Robbe and Nico in particular, as I’ve had the luck to workvery closely with them on numerous occasions, in which time they became closefriends of mine.

vi

Abstract

In today’s world computers and the Internet have become an integral part of every-day life and have introduced new ways of mass communication and collaborationamong people from across the globe. The latest iteration in the Internet ecosystemhas been the conception of Cloud Computing, a new service-oriented computingparadigm that tries to realize the long-lived dream of a global utility-like computingmodel by introducing the notion of on-demand, pay-what-you-use, elastic resources.

The so-called Infrastructure-as-a-Service (IaaS) providers materialize the cloudcomputing model by offering consumers access to (virtualized) hardware resources,and charging them based on how they use these resources, rather than charging aone time fee for the allocation of the hardware. Besides the ease of deployment andmanagement of applications on third party remote resources, the IaaS paradigm alsoincreases the possibilities for trading IaaS resources on a scale that moves beyondthe individual provider level. Such inter-provider trading is used in other commodi-ties markets such as electricity markets, to efficiently balance supply and demand onrelatively short timescales. In addition, an open market for IaaS resources can effi-ciently open up the cloud IaaS provider space. At present however, the possibilitiesfor creating such open markets for IaaS resources have been largely unexplored.

This thesis is an attempt to shed some light on this topic by investigating whetherit is feasible to design an open market for IaaS resources, as well as elaborate onthe structure of such an IaaS market by pinpointing its core components and mech-anisms. This thesis spends particular attention to the Continuous Double Auction(CDA) mechanism that is used in many commodity markets, and finds it unsuit-able as a market mechanism in the current IaaS setting due to the complexity anddiversity of price models used by providers. To overcome this, a new market mecha-nism called the Continous Reverse Auction (CRA) is introduced, which solves someof these issues. In order to validate its practical applicability, some implementationconsiderations about the CRA mechanism are made, and a prototype is implementedthat is able to cope with the heterogeneity of different real-world providers.

CHAPTER 1

Introduction

In today’s world computers and the Internet have become an integral part of every-day life as they are used by millions of people at any given time. The vast onlinenetwork of computers that is the Internet, has introduced new ways of mass com-munication and collaboration among people from across the globe. Although it isonly a few years ago that the Internet was just a collection of simple connectedwebpages, the Internet of today is increasingly being defined by complex webap-plications, social networks and online communities. From a user’s point of view,the advantages of using such online tools are apparent: they are immediate, easilyaccessible, location/time independent and social. Because of this, more and moreusers are leaving desktop applications for what they are, and have started usingtheir online counterparts.

As the online community continues to grow, it is only logical from a company’spoint of view to increasingly use the Internet as a platform on which to build ap-plications and software. Doing so, is however not without challenges as the Internetecosystem is a very demanding and rapidly changing environment which is new tomost traditional companies. In particular, in order to build and maintain onlineapplications, companies need a lot of powerful servers and skilled software develop-ers. As buying and maintaining these servers as well as composing and managingdeveloper teams are tedious undertakings, more and more companies are choosing tono longer (fully) develop online applications in house, but instead outsource them toexternal parties that are specialized in building such online applications and settingup online software environments.

Both the trend of building and using applications on the web as well as the idea tooutsource certain IT tasks to web companies are today more commonly referred to asCloud Computing, a new service-oriented computing paradigm that can potentially

1.1. CLOUD COMPUTING 3

disrupt the way the computer industry has worked the last few decades.

The following sections will give a more detailed view of what cloud computingis, by explaining some of the key concepts behind it and discuss some of its primarybenefits and challenges. Note that it is not the author’s intention to provide anacademic in-depth discussion about cloud computing, as this goes beyond the scopeof this thesis. Rather, the following section tries to outline the general ideas andprinciples behind it in simple terms. For a more detailed introduction to cloudcomputing, refer to [1] and [2]. Readers that are already familiar with the conceptsof cloud computing can safely skip to section 1.3.

1.1 Cloud Computing

1.1.1 Definition and core concepts

Cloud computing is considered by many as the next big thing in the evolution of thecomputer. Despite this, the term cloud computing still remains rather abstract toa lot of people. This vagueness that surrounds cloud computing is probably causedby the fact that term is predominantly used as a buzz-word by marketeers of largeIT companies. Every website that does more than provide static information iscurrently labeled a cloud application while a lot of those websites do not have anyof the specific characteristics that define cloud computing applications. In whatfollows, the author will try to take away some of this vagueness by giving a genericbut clear definition of cloud computing and some of its basic concepts.

In its most generic definition, cloud computing represents a new way of think-ing about computer resources in which privately owned infrastructure and softwareare replaced by publicly shared variants. In the cloud computing model, privatecompanies no longer own expensive servers, nor do they build complex proprietarysoftware. Instead of following this traditional model, companies now pay for specificservices that are offered by specialized IT vendors. This new service-oriented wayof handling computer resources is perhaps the most essential characteristic of cloudcomputing. Besides this service orientation, service delivery through the Internet isanother fundamental property of cloud computing. In other words, in order to usea cloud computing service, one must be connected to the Internet for as long as theservice is being used1. The term cloud computing expresses this inherit requirementby using the word cloud as a metaphor for the Internet.

Although the service-oriented delivery through the Internet is a core characteris-tic of cloud computing, there are some other traits to the paradigm that distinguish

1This is technically not entirely correct, as certain cloud computing services keep running whilethe user itself is not connected to the web. However, in order to manage or configure these services,as well as to see their output, the user has to be connected to the Internet.


it from other similar paradigms and general service oriented architectures. Thatis, the ideas behind cloud computing try to materialize John McCarthy’s idea2 ofutility-computing by exhibiting some of the same properties that are characteristicof existing utilities such as electricity, water, gas and telephony. More specifically,there are 3 key aspects that define cloud computing.

On-demand access to resourcesCloud computing resources share the on-demand features of classic utilities. Thatis, when using cloud computing solutions, users do not need plan far ahead for pro-visioning reasons. Instead, they can quickly and dynamically acquire the resourcesthey need, for the time that they need them. The term on-demand also refers tothe fact that cloud computing resources are accessible from anywhere at anytime,as the only real requirement to access resources is an internet connection.

ElasticityWhen using cloud services, users are given the illusion of having an infinite amountof computing resources at their disposal that can easily scale to meet their needs.By giving users the possibility to quickly scale resources up or down, they havethe flexibility to align IT infrastructure and resources with their business needs innear real-time. The main benefit of this is that companies no longer need to wastecapacity as they provision for peak loads, nor do they need to worry about losingcustomers caused by underprovisioning, as illustrated in figure 1.1.

(a) Provisioning for peak load (b) Underprovisioning

Figure 1.1: Difficulty of provisioning IT infrastructure. (a) Even if peak load can becorrectly anticipated, without elasticity, resources are wasted (shaded area) duringnonpeak times. (b) Potential revenue is sacrificed from users that are not served(shaded area). Source: [1]

Metered usageOne of the main parallels between cloud computing and existing public utilities isthe payment system. When using a public utility, customers only need to pay for theamount of resources they have used. Cloud computing suppliers try to adhere to thismodel by only letting customers pay for the amount of computer resources that theyhave used. This new payment methodology can potentially cause a dramatic shift

2In 1961, John McCarthy was the first to express the idea that “one day computation may beorganized as a public utility”.


in the way computing resources are dealt with as with such a system, companies nolonger have to make upfront investments in infrastructure. Indeed, a shift to the pay-what-you-use (or pay-per-use) model, incurs a shift from capital expenses (CAPEX)to operational expenses (OPEX), which can have far reaching consequences in theway companies look at and deal with information technology.

Using cloud computing solutions has some other interesting benefits in terms ofcost and flexibility. However, as there is no silver bullet, there are also some keychallenges that remain to be solved. Section 1.1.4 gives an overview of both theseadditional benefits and the key challenges.

1.1.2 Cloud computing, a layered model

When more thoroughly defining cloud computing, typically 3 different forms of cloudcomputing are recognized. These forms are called Infrastructure-as-a-Service (IaaS),Platform-as-a-Service (PaaS) and Software-as-a-Service (SaaS).

Infrastructure-as-a-Service captures the idea that companies no longer have theirown datacenters but instead rent bare-bone servers from an other, specialized, ITcompany. These rented servers are said to be ‘in the cloud’ as they are no longer aphysical part of a company’s computing infrastructure. In this context, the termson- and off-premise resource are commonly used to denote servers that are kepton the premise of a company itself and servers that are hosted by those other ITcompanies. Billing for the rented servers is typically done per used server minute orhour. That is, companies only pay for the amount of computing resources they haveactually used; they are not charged for the time that the servers are unallocated (i.e.pay-what-you-use).

IaaS providers typically make use of virtualization technology in order to con-solidate as much customer appliances as possible on a single physical machine. Tomake this possible, customers that want to make use of a certain IaaS, will need tochoose from a provider specified set of virtual machine images and types that definethe hardware characteristics, operating system and possibly software packages ofthe machine they will get access to. By using such virtualization technology, IaaSproviders are able to get much higher utilization rates on their servers, decreasingthe provider’s costs. Additionally, as IaaS vendors provide services to many cus-tomers at the same time, they can benefit from economies of scale that no singlecompany would be able to realize. These economies of scale are not only applicableto the purchasing of new server hardware - IaaS providers can potentially get sub-stantial discounts from hardware vendors as they buy servers in large volumes atonce - but also to operational costs such electricity and maintenance staff. A benefitof this is that, given sufficient competition among IaaS providers, it is interesting forproviders to pass these savings to their customers. Clearly, this makes IaaS solutionseven more appealing from a consumer’s financial point of view.

Besides fully operational virtualized servers, many IaaS providers also offer otherservices such as standalone storage services, firewall services, virtual network servicesand others. Although the price model for these specific service can be rather different


per provider, virtually all providers apply the pay-what-you-use model for theseservices, charging their customers per used storage, bandwidth or time unit.

Platform-as-a-Service (PaaS) builds upon the IaaS idea by adding a softwarelayer on top of the bare-bone servers. This provided software is not meant for end-users but rather for software developers, providing building blocks that can be usedto build other, more complex applications. In many cases the software stack providesa high level of abstraction by providing database managers, programming interfacesand (vendor specific) software libraries. In doing so, developers no longer needto worry about the hardware, operating system or low-level software that supportthe programming interfaces. Additionally, PaaS solutions typically take care ofthe technical issues of scaling up and down applications, parallelizing applicationexecution or supporting application security services.

When it comes to billing, PaaS products usually have a more complicated billingmethod than IaaS solutions. That is, users (developers) are not charged by the timethat they used the platform but rather for the number of database queries made,program interfaces used or webpages that were accessed.

The third form of cloud computing, Software-as-a-Service, is considered by manyas the most evolved form of cloud computing. With SaaS, complete software solu-tions are offered directly to end-users. Examples include webmail, office suites, cus-tomer relationship management (CRM) tools and many others. SaaS products havebecome increasingly popular over the last few years as they have rapidly started re-placing existing desktop applications. Because the size of the online market becomeslarger every day [3], it is becoming increasingly interesting for software vendors toprovide their services online. The result of this is that many software companies arecurrently investing heavily in the SaaS market. Of course, as is the case with everyinvestment, investments in cloud computing need to be profitable. When it comesto return-on-investment, SaaS vendors commonly use advertisements to monetizethe services that they offer for free to their users or ask additional fees when theirapplications are used in corporate environments.

Having discussed the different forms of cloud computing, it is interesting tobriefly compare the different forms in terms of the problems they each try to solve.(Note that making an exact separation is quite difficult since in many cases differentsolutions can be used to solve the same problem. For example, a certain companymay choose a PaaS solution to implement its supply chain management (SCM) sys-tem while another may choose a SaaS solution to do the same. The final choice isoften determined by the need for specific requirements, company policies or prefer-ences.) When choosing for IaaS, companies choose the most flexible form of cloudcomputing. That is, nearly every IT system can be deployed on the provided bare-bone server machines. This makes IaaS an excellent choice for companies requiringvery specific (e.g legacy) hard- or software, raw compute power or full control. Thisdoes mean however that building and maintaining solid IaaS solutions still requiresquite a lot of work compared to e.g. a SaaS solution. Such SaaS solutions typically


require only minimal or no development effort but usually offer very limited flexi-bility. This makes them almost ideal for more widespread applications that handlecommon tasks (e.g. email, accounting or human resource management) but lessattractive when more customization is desired. Integration with already existingsystems in particular can be very hard to achieve when using SaaS solutions. Notethat this does not necessarily mean that SaaS solutions are not mature enough, itonly means that SaaS products target a single, specific application. In between IaaSand SaaS, PaaS often provides a good solution for companies that do want tighterintegration with existing applications or for companies that wish to use specific soft-ware frameworks without wanting to worry about its deployment and maintenance.Also, when no (solid) SaaS solution already exists, and there are no specific softwareor hardware requirements, PaaS products are often the appropriate choice. Figure1.2 shows how the different levels of abstraction of IaaS, PaaS and SaaS relate in anonion model.

Figure 1.2: Onion representation of the cloud computing hierarchy. PaaS builds onIaaS, while SaaS builds on both PaaS and IaaS.

1.1.3 Deployment models

Public and private cloudsWhen the term cloud computing or cloud is mentioned, almost always the notionof a public cloud that is shared among different mutually untrusted consumers ismeant. However, in the context of IaaS, it is also possible to set up cloud-likeenvironments that can only be used by a specific group of users. The term privatecloud captures this idea, and is used to denote in-house (within a single company ororganization) datacenters that exhibit cloud-like characteristics. The most notableof these characteristics, is perhaps the use of the pay-what-you-use model to chargedifferent parties within a single organization. By letting different departments andteams pay for the amount of computing resources they use from their own budget,concepts of activity-based costing [4] can be applied to IT-related projects.

As virtualization technology provides interesting benefits - virtual machine iso-lation and increased server utilization - independent of the cloud deployment model,


it is also commonly used in private clouds. Tools such as Eucalyptus [5], Open-Nebula [6] and VMWare’s vCloud [7] allow system adminstrators to manage sucha private virtualized datacenter by providing installation, monitoring, migration,checkpointing and backup tools.

Although private clouds typically lack some of the advantages of public cloudsolutions (“infinite” scaling, economies of scale, etc.), they do give local users a flexi-ble and agile infrastructure to run service workloads within their own administrativedomains. For many applications this can have clear benefits, as e.g. response timescan be much lower compared to public clouds (due to the locality) and security isgenerally a lesser concern as the different parties on the cloud are from the sameorganization and thus typically trusted.

When some of the public cloud features are wanted, while also benefiting ofthese private cloud features, a so-called hybrid cloud can be set up that combinesthe benefits from both.

Hybrid cloudsA hybrid cloud is a cloud environment that contains computing resources from bothprivate and public clouds. This can for example be done by setting up a virtual LANbetween private and public clouds so that from the user’s perspective it seems as ifthe computing resources from the public cloud are part of the private cloud. Theuse-case of (almost) transparently expanding in-house computing power with off-premise capacity is certainly one of the core ideas behind hybrid clouds, as it allowscompanies to add extra computing capacity on-demand. Perhaps more importantlyhowever, is the fact that such a hybrid model allows companies to keep using existingin-house computing infrastructure, while also gaining some of the advantages ofpublic clouds. This effectively lets companies avoid having to lose existing (long-term) investments, when switching to cloud computing solutions. Figure 1.3 givesan overview of how hybrid clouds relate to private and public clouds.

The use of hybrid clouds can also has some interesting benefits in terms ofsecurity. That is, when certain information requires high protection and securitystandards, a company can choose to put the data and software that uses the data onthe private part of the hybrid cloud. By doing so, companies do not have to worryabout the additional security issues that public clouds introduce as they have fullcontrol over the software that runs on the machines that contain the sensitive data.

As a final note on hybrid clouds, it is important to note that although usinghybrid clouds is certainly very interesting, setting them up is not always trivial. Thisis partially caused by the lack of standardization between different virtualizationtechnologies, cloud providers and cloud management tools, but also by the fact thatsetting up such a WAN-based computing environment has some inherit difficultiesin terms of network and security configuration.


Figure 1.3: Private, Public and Hybrid clouds (Based on image by Sam Johnston,CC-BY-SA).

1.1.4 Challenges and Benefits

The main benefits of cloud computing are unmistakably derived from its core prin-ciples that were discussed in section 1.1.1. On-demand access, elasticity and me-tered usage are indeed the features that differentiate cloud computing from similarparadigms. Besides these, cloud computing has some additional benefits of whichthe most prominent are listed here.

General advantages of outsourcing Outsourcing activities allow companies tofocus on their core business. This leads to a reduction of capital investmentsas well as general increased flexibility. Also, as the contractors to which theactivities are outsourced are specialized in their respective field, they typicallyhave more expertise and experience about the specifics of the technology that isbeing used, leading to better performance and solutions. This is certainly truein the case of IT outsourcing where skilled software developers and techniciansare needed to develop and maintain software and infrastructure.

Security and certification One of the aspects cloud providers typically invest inheavily, is security and certification. While security breaches are a risk andhave far reaching consequences for any company, cloud providers risk evenmore as an exploit or bug in their software or infrastructure effectively makesall the companies that use their services vulnerable. Because such failuresof security can lead to customer lawsuits and serious damage to a brand’simage, cloud providers have a strong incentive to take security very seriously.Consequently, investing a lot of time and money in security related processesand certification (also non-security related certification) is a logical thing todo for many cloud providers. This directly benefits all customers, as makingsuch heavy investments in security and certification might normally not havebeen justifiable for all applications.


Green IT As building and maintaining server infrastructure is a core part of thebusiness of almost all cloud providers, they have a strong incentive to build thisinfrastructure as efficient as possible in order to minimize operating costs. As avery large part of these costs are attributed to power consumption, minimizingthat power consumption has a strong impact on the cost efficiency of anydatacenter. Reducing power consumption can be achieved by make heavy useof virtualization technology (as is the case with IaaS solutions) to increase theaverage server utilization, lowering the amount of power that is needed percompute cycle. Additionally, the use of efficient cooling technology can havea large influence on bringing on those costs further down.

In order to get an idea about the power efficiency of a datacenter the conceptof Power Usage Effectiveness (PUE) is often used. PUE is defined as the ratioof total amount of power used by a computer data center facility (includingall cooling, lightning, etc. equipment) to the power delivered to computingequipment. In other words, PUE expresses how much of the power is actuallyused by the computing equipment in contrast to cooling and other overhead.Ideally, a datacenter’s PUE value is 1.0, which corresponds to a data centerwhere all of the electrical grid power supplied is devoted to IT equipment andno power is used for cooling and other activities. Currently howevever, mostprivate datacenters are nowhere close to this ideal value, with most datacentershaving a PUE of around 1.9 [8]. However, by the consolidation of many serversinto large datacenters, cloud computing providers claim to achieve much bet-ter results, with PUE figures as low as 1.2 [9] and even 1.1 [10]. Besides theapparent financial benefits bringing down energy consumption incurs, mini-mizing energy consumption also has a substantial impact on the environment,hence the reference to “green IT”.

Although using cloud computing solutions has many advantages, there are stillchallenges that need to be solved before cloud computing will become a maturecomputing paradigm. A good overview of the current challenges that cloud comput-ing solutions face is given by [1]. In order to be concise, only an overview of thesechallenges is given here.

Availability of a Service Organizations may worry about whether cloud comput-ing services have adequate availability; many cloud providers have yet to provethat they are able to provide high levels of availability.

Data Lock-in The APIs of cloud computing services itself are still essentially pro-prietary, and have not been subject of active standardization (although someinitiatives are on the way [11], most notably [12]). In other words, customerscannot easily extract or export their data and programs from one provider torun on another. Customer lock-in may be attractive from a cloud provider’sviewpoint, but by locking users in, they are vulnerable to price increases, toreliability problems, or even to providers going out of business.


Data Confidentiality and Auditability Most companies are not eager to movetheir sensitive corporate data into the public cloud, as by doing so, they movetheir data into what is essentially a public network. Consequently, by usingpublic clouds, companies expose their system to more attacks.

Currently however, most cloud providers give very little guarantees about thesafety of the data of their users. Related to this is the issue of the fact thatmany nations set strict rules about moving customer data outside of nationalborders (as would typically happen when migrating to a public cloud), or abouthaving the right to access all stored data (i.e. USA PATRIOT act).

Data Transfer Bottlenecks As applications continue to become more data-inten-sive, they will require more and more bandwidth when running in the cloud.Bandwidth is however limited; sending large amounts of data over the Inter-net is both time-consuming and costly. Given this fact, it is only logical thatthese (time and financial) costs incurred by data transfers will be an importantfactor in the success or failure of cloud computing.

Performance Unpredictability Although sharing CPUs and main memory workvery well in virtualized environments, sharing I/O is a notable issue. Anexample in [1] shows that when multiple VM’s simultaneously write to thehard disk, the available disk bandwidth for each VM becomes unpredictable.

Scalable Storage As many I/O resources (most notable the harddisk) are typicallya lot slower than their computation counterparts, a possible concern can bewhether the storage resources of cloud providers are able to scale along with thecomputation resources. Without special purpose hard- and software, access tostorage systems can potentially become the system’s bottleneck.

Bugs in Large-Scale Distributed Systems A difficult challenge for cloud com-puting is removing errors in the very large distributed systems as used by cloudproviders. Finding and fixing these bugs can be notoriously difficult as manyof these bugs only occur at the scale of production datacenters, rendering theirreproduction in smaller setups impossible.

Scaling Quickly The speed with which cloud providers are able to scale can bevery important to customers as in many scenarios they require that their appli-cations can scale according to quickly changing real-time loads. The difficulty(and opportunity) for providers is to be able to respond quickly to scalingrequests in order to save their customers’ money and income opportunities,without violating service level agreements.

Reputation Fate Sharing Reputations do not virtualize well. One customers badbehavior can affect the reputation of the cloud as a whole. For instance, black-listing of a cloud provider’s IP address range by spam-prevention services maylimit other customers’ connectivity and thus which applications can effectivelybe hosted.

1.2. IAAS IN PRACTICE 12

Software Licensing Current software licenses commonly restrict the computers onwhich the software can run. Users pay for the software and then pay an annualmaintenance fee. As cloud computing services are based on the pay-what-you-use model, it is not entirely clear how these software licenses costs should beincurred to end-users. This has led to the fact that many cloud computingproviders make extensive use of (free) open-source software, although this isgradually changing now.

Consumer Choice In the current cloud computing ecosystem, consumers do nothave tools that can help them in the search for the cloud computing solu-tion that best fits their needs. Furthermore, consumers are often constrainedto the use of certain cloud providers because of accounting, administrative,legislative, policy or other reasons. Also, as there is no direct competition be-tween providers and consumers, the notion of free choice does not really existin the current cloud computing ecosystem. If cloud computing is to evolveinto a mature computing paradigm, there is a need for an open market plat-form that allows the trade of computing resources between all providers (andconsumers). The importance of the introduction of such a market for IaaSresources is discussed in detail in section 1.3.3.

1.2 IaaS in Practice

In this section, examples of real-world services will be given to give the reader a betterunderstanding of how the previously discussed theoretical concepts are applied inpractice. To this end, the product lines of 2 large IaaS providers are discussed inmore detail here. As SaaS and PaaS solutions are of lesser importance in the contextof this thesis, according product examples have been moved to Appendix A.

1.2.1 Amazon Web Services

While there are many IaaS providers, in the current IaaS market, Amazon - thecompany that everyone knows from their online book shop [13] - is considered to bethe absolute market leader. Although absolute numbers in terms of market share,number of customers or sales volume are not available, Amazon is widely regardedas the company that put IaaS on the map. Since its inception back in 2002, AmazonWeb Services (AWS) has added a whole range of products and services of which themost important are listed here.

Amazon Elastic Compute Cloud (EC2)The EC2 service [14] is the most complex solution offered by Amazon. The servicegives users access to a variety of virtual machines with specific hardware character-istics, which they can use for whatever purpose they want. In order to specify andmanage instances - virtual machines with a specific AMI (Amazon Machine Image)- Amazon provides users with a set APIs (webservices) that can be accessed directly,through one of the many provided software libraries, through commandline tools or


through the online AWS Management Console (webinterface). Once launched, userscan access their resources through SSH.

Before launching an instance, users need to make several choices that specifythe different characteristics of the instance. The most important choice perhaps, isthat of the instance type, as it specifies the hardware characteristics of the instancethat is chosen. Table 1.1 gives some examples of different instance types and theircharacteristics; a complete overview can be found in [15]. Note the usage of theEC2 Compute Unit (or ECU for short), which is a hardware agnostic indicator ofthe processor’s speed. More information about how the ECU corresponds to variousCPU benchmark results can be found online at [16].

Besides specifying the instance types, users also have to choose from one ofseveral availability regions, which determines in which of Amazon’s datacenters therequested resources will be allocated. Currently, Amazon allows users to choosefrom 5 different regions from across the globe (US-west, US-east, EU-west, APAC-Singapore, APAC-Tokyo).

Instance type Hardware characteristics

Small Instance 1.7 GB memory1 EC2 Compute Unit(1 virtual core with 1 EC2 Compute Unit)160 GB instance storage32-bit platformI/O Performance: Moderate

Large Instance 7.5 GB memory4 EC2 Compute Units(2 virtual cores with 2 EC2 Compute Units each)850 GB instance storage64-bit platformI/O Performance: High

High-MemoryDouble ExtraLarge Instance

34.2 GB of memory13 EC2 Compute Units(4 virtual cores with 3.25 EC2Compute Units each)850 GB of instance storage64-bit platformI/O Performance: High

Table 1.1: Examples of Amazon EC2 instance types

When it comes to pricing, Amazon allows customers to choose from three differ-ent price models. The most commonly used on-demand model, is the most flexibleprice model that adheres most to the pay-what-you-use model as it allows users tostart using instances, stop them at any given time and only pay for the amount oftime (in whole hours) they have actually used the instances. For example, at the


time of writing, the cost for a small instance in the EU-West region is $0.095/hour ifa Linux or UNIX based AMI is chosen and $0.12/hour if a Microsoft Windows-basedAMI is chosen. Similarly, the cost for a Double Extra Large High-Memory instanceis $1.14/hour for Linux/UNIX and $1.24 for Windows.

The reserved price model allows customers to get cheaper hourly prices withguaranteed instance availability by making up-front commitments. That is, if userschoose to pay an upfront fee to reserve a certain instance for a whole year at once,Amazon offers significant discounts on the hourly price for running that instance,while also guaranteeing the availability of that instance for that year. In reality, anupfront payment of $227.50 is required in case of a small instance. Afterwards thehourly cost of running a Linux- or UNIX-based image on that instance will onlybe $0.04, which is less than half of the hourly on-demand price. Similarly, whenreserving a Double Extra Large High-Memory reserved instance for whole year atonce (upfront cost: $2650), the hourly cost will effectively drop to $0.48 per hour,which is only 42% of the hourly on-demand price. Indeed, when it is known up frontthat instances will need to run for at long time at once, using reserved instancescan incur significant savings. Figure 1.4 shows that at the current prices a smallinstance should at least run 4137 hours a year before using a reserved instance ismore interesting than using an on-demand instance.

Figure 1.4: Total costs of using a reserved small instance vs. using an on-demandinstance on Amazon’s EC2 platform

Besides on-demand and reserved instances, Amazon’s Elastic Compute Cloud


(EC2) also allows the allocation of so-called spot instances [17]. These instancetypes are called so, because they adhere to the volatile pricing characteristics that aretypical of any spot market (see section 2.2.6 for more information on spot markets).That is, the spot price of the EC2 spot instance is changed on an hourly basis,according to the current load on EC2. When demand for a specific instance type islow in a certain availability region, the spot price for that instance type will also below for that availability region. Consequently, when demand for an instance typeis high in a certain region, the spot price for that instance type will also be highwithin that region.

Since querying the user after each spot price change (after each hour) to deter-mine whether the user still agrees with the new spot price could potentially generatea lot of unnecessary traffic, the user is asked to provide a maximum price for the in-stance before it is started. The spot instance will then keep running until it is eitherterminated by the user, or until the spot price exceeds the user specified maximumprice, at which point it is terminated by the EC2 service itself.

Since the spot price of an instance is usually much lower than its on-demandprice, users can potentially save a lot of money by using spot instances insteadof on-demand instances. However, there is always a certain level of uncertaintyabout the availability of the spot instances (since the spot price might unexpectedlypeak above the chosen maximum price), which makes using them less interesting forapplications that need guaranteed uptime.

Figure 1.5 shows the the evolution of the spot price for the small instance inEU-region between 18/01/11 and 23/01/11.

Figure 1.5: Amazon EC2 Spot price of the small instance in the EU-region between18/01/11 and 23/01/11. Source: [18]


Besides hourly costs associated with running instances, Amazon also chargescustomers to transfer data in and out of their datacenters. Incoming data is priced at$0.1/GB for all availability regions, while prices for outgoing datatransfer are similarput depend on the region and the amount of data transfer per month ($0.15/GB forthe first 10 TB in the EU region).

In order to build more complete solutions, Amazon also offers additional servicesthat provide extra functionality to the EC2 platform at an additional cost.

Amazon CloudWatch Allows customers to monitor instances (CPU usage, net-work traffic, etc).

Auto Scaling Allows customers to start or stop instances automatically, based ontriggers defined in Amazon Cloudwatch.

Elastic IP Addresses Allows customers to allocate fixed IP-addresses for instances.

Elastic Load Balancing Allows customers to balance the load of incoming requestover different instances.

Amazon Elastic Block Store Allows customers to work with block devices (hard-disks) that can be swapped between different instances.

Details about the extra EC2 services and pricing can be found online [14] [19].

Amazon Simple Storage Service (S3)The S3 service [20] is a storage service that allows users to store arbitrary data-objects in so-called buckets that are identified by global unique identifiers. Whenusing this service, the developer can choose in which of the different availabilityregions to create buckets and store data, giving him the freedom to optimize forlatency, minimize costs, or address regulatory requirements. Additionally, Amazonprovides authentication mechanisms that allows developers to ensure that data iskept secure from unauthorized access. More specifically, objects and buckets can bemade private or public, and rights can be granted to specific users.

Access to the resources is through HTTP or Bitorrent, while the managementof the resources is done through webservices (again, Amazon also provides softwarelibraries, commandline tools and an online webinterface to do this). Notable is thefact that Amazon guarantees an availability of 99.99% while guaranteeing a dura-bility of 99.999999999%, by duplicating all objects over 2 separate facilities. Storinga GB per month costs $0.14 in the EU-region (for the first 1000GB, afterwards theprice per GB slowly decreases), while $0.01 is paid per 1,000 management requests(PUT, COPY, POST, LIST) and $0.01 per 10000 access requests (GET).

Amazon Simple Queue Service (SQS)The SQS service [21] is a scalable hosted queue for storing messages up to 65KB oftext in any format. Such a message queue system can be used to build an automatedworkflows in which certain components submit messages (workloads) to the queuean other components process the messages (workload) by fetching them from the


queue. Access to the queue is done through webservice calls (software libraries arealso provided). Similar to the S3 service, Amazon allows granular access control byaccount, IP-address and time-of-day. A price of $0.01 is charged per 10000 queuerequests for all availibility regions. Additionally, $0.1 is charged per GB of inboundtraffic (into the queue) for all regions, while the prices for outbound traffic differdepending on region and volume (as with EC2, $0.15/GB for the first 10 TB in theEU region).

Amazon SimpleDBThe SimpleDB service [22] is a scalable and flexible non-relational data store inwhich developers can store and query data items. The idea behind SimpleDB isthat developers only need to worry about accessing the database, and not about themanagement of the database itself. In practice, this means that the service createsand manages multiple geographically distributed replicas of the data automaticallyto enable high availability and data durability. The service applies the pay-what-you-use model by charging customers only for the compute and storage resourcesactually consumed in serving their requests. In practice, customers pay $0.154 perused SimpleDB machine hour3 in the EU region, with an additional $0.1 per inboundGB and $0.15 per outbound GB for the first 10 TB. Also, $0.275 per GB per monthis charged for data that is actually stored in the database.

Other servicesBesides these core services, Amazon also offers a variety of additional products andservices that allows customers to build complete solutions on the Amazon Webser-vices platform more easily.

Amazon CloudFront A content delivery network (CDN) for distributing objectsstored in S3 to so-called “edge locations” near the requester.

Amazon Relational Database Service (RDS) Service providing a scalable MySQLcompatible database server.

Amazon Simple Email Service (SES) Service providing bulk and transactionalemail sending.

Amazon Virtual Private Cloud (VPC) Service that allows for the creation ofa logically isolated set of Amazon EC2 instances to be connected to an existingnetwork using a VPN or regular internet connection.

Amazon Elastic MapReduce Service that provides a hosted Hadoop framework[23] running on the web-scale infrastructure of EC2 and S3.

Note that some of the services that are provided by Amazon, such as SimpleDB,RDS, SQS, SES, MapReduce and others should arguably be labeled as PaaS solu-tions rather than IaaS solutions as those services provide significant abstractions

3From the SimpleDB website [22]: “Amazon SimpleDB measures the machine utilization of eachrequest and charges based on the amount of machine capacity used to complete the particularrequest (SELECT, GET, PUT, etc.), normalized to the hourly capacity of a circa 2007 1.7 GHzXeon processor”


of underlying hardware, operating systems and software libraries. In order to avoidconfusion, the author wishes to note that when the term IaaS is used throughout therest of this thesis, the more hardware-related solutions (that is, compute, storage,network and alike) are meant. More advanced services such as the ones mentionedhere will thus be considered PaaS solutions.

1.2.2 GoGrid

GoGrid [24] is an IaaS provider that offers services that are similar to that of AmazonWeb Services, albeit in a different form and under a slightly different price model.At the time of writing, GoGrid offers four different infrastructure components [25].

Cloud ServersThe Cloud Servers service is a service very similar to Amazon’s EC2 service andallows customers to allocate raw compute power by the specifying a server imageand size (similar to Amazon’s instance image and type). Additionally, the customerneeds to specify in which of GoGrid’s datacenters the server should be allocated(US-West or US-East). The sizes of the servers are characterized by the amountof memory, CPU cores and internal storage space they have. Table 1.2 shows thecurrently available server sizes. Both standard Microsoft Windows and Linux imagesare available, while images with more pre-installed software can be found on theGoGrid Exchange [26]. Access to the cloud servers is through SSH (Linux) or

CPU (cores) RAM (GB) Storage Space (GB)

0.5 0.5 251 1 502 2 1004 4 2008 8 400

8/16* 16 800

Table 1.2: GoGrid Cloud Server sizes. (*Windows Cloud Servers deployed with 16 GBRAM are assigned 8 cores, Linux cloud servers with 16 GB of RAM have 16 cores.)

Remote Desktop (Windows). Management of the servers is done through eithera web console or by using a RESTful API.

GoGrid tries to differentiate itself from other IaaS providers by guaranteeing100% uptime for all of its services, while promising a 10000% refund (on a minuteby minute basis) when downtime does occur. Compare this to Amazon, which onlygarantuees 99.95% uptime and uses a rather complex refund policy in which 10% ofthe monthly bill during the downtime is given as service-credit that can be spendon AWS services in the next month.

Billing is done per used RAM hour. A RAM hour is expressed as the numberof GBs of RAM that is used in a single hour. For example, using a server with0.5 GB of RAM incurs the cost of half a RAM hour for every hour the server isused, while a server with 4GB of RAM incurs a cost of 4 RAM hours per hour itis used. The price for a RAM hour $0.19 if on-demand access is preferred, but if


stronger up-front commitments are made (if customers pay for a whole month oryear at once), the effective RAM hour cost lays between $0.05 and $0.08. Note thatthe price for Windows or Linux-based server images is the same (although someadditional costs apply for the usage of Microsoft SQL Server products). Concerningbandwidth, $0.29 is paid per GB of outbound transfer if the on-demand model ischosen, while this can as low as $0.07 per GB when a large upfront commitment ismade. Inbound data-transfer is free.

Dedicated ServersBesides virtualized hardware, GoGrid also offers their customers access to dedicatedservers, if customers don’t want to run their applications on a multi-customer en-vironment. Dedicated servers are offered in 3 sizes (standard, advanced and ultra),offering 4 up to 8 CPU cores and 8 up to 24 GB of RAM. Each server also hasbetween 500GB and 1TB of RAID storage. Prices range from $300 up to $600 amonth, with extra licensing fees for Windows or Redhat based operating systems.If an up-front payment for a year is made, customers get a 2-month discount.

A special feature of the Dedicated Servers service is the possibility to setup ahybrid cloud infrastructure between dedicated servers and cloud servers, combiningthe isolation and performance guarantees given by the dedicated servers with theflexibility and scalability offered by on-demand cloud servers. A possible use-casecould be to run high-performance database servers on dedicated servers while usingcloud webservers that scale according to current demand on a website.

Cloud StorageThe GoGrid Cloud Storage service is a scalable and reliable file-level backup servicefor cloud servers running in the GoGrid cloud. Access to the storage is providedthrough a variety of protocols such as SCP, FTP, SAMBA/CIFS and RSYNC.

All cloud servers receive a free cloud storage tier that allows customers to saveup to 10GB a month at no additional charges. Once more than this 10GB of storageis used, $0.15 will be charged per extra GB.

Although as with the Cloud Service service, an 100% uptime (with 10,000% re-funds) guarantee is given by GoGrid, they do not provide any assurances concerningbackup, nor do they make further commitments in the case of permanent data loss.

Load BalancersThe GoGrid load balancer service is a service allows customers to balance incomingconnections to servers according to the Round Robin or Least Connect4 algorithms.The Load Balancers service is provided at no additional charge as part of the cloudserver and dedicated server services.

Besides these four core infrastructure components, GoGrid also offers firewalland content delivery services at additional charges. More information can be foundonline [25].

4The Least Connect algorithm sends incoming connections to the servers with the least amountof load on them.

1.3. IAAS CLOUD MARKETS 20

1.3 IaaS Cloud Markets

After having discussed the core concepts behind cloud computing, as well as givensome examples of some real world cloud computing products, it is now time to shiftfocus to the core subjects of this thesis. The following sections explain the analogiesand differences between IaaS clouds and the power industry and indicate how theseanalogies raise the idea of building an IaaS cloud market.

1.3.1 IaaS Clouds and the power industry

When pinpointing the core concepts behind cloud computing in section 1.1.1, it wasmentioned that many of the core principles behind cloud computing are the sameas those behind commoditized energy utilities. Although this is so for all publicutilities, it tends to be particularly true for electricity. The idea of being able toplug in a network cable to get access to an infinite amount of computing power inmuch the same way as power cords can be plugged in the wall to get access to aseemingly infinite amount of electricity is indeed very appealing. From this point ofview, it is also clear that the cloud utility model applies the most to IaaS clouds,and less to PaaS or SaaS as the former promises to deliver raw compute power wherethe latter two deliver more advanced services on top of that. If IaaS is analogous toraw electricity, then PaaS should be compared to the different electrical componentsthat make up household appliances, while SaaS products should be compared to theappliances themselves.

The idea of having an electricity-like global computing grid that anyone canaccess is however not new; it has been the core idea of what is known as Gridcomputing [27] for quite some time. As grid computing has been the subject ofextensive research and as explaining the core differences with cloud computing has nodirect value to this thesis, it will not be discussed further here. A good comparisonbetween cloud and grid computing is given by [28] while the introduction of [2]contains a good comparison between cloud, grid and cluster computing.

What is interesting in the context of this thesis however, is a more direct compar-ison between electricity and cloud computing. To that end, a work by Nicolas Carrprovides valuable insight. In [29], Carr tells the story of the birth of the modernelectricity industry and reasons about the similarities and differences between therevolution in the power industry and that of the computing industry. Specifically,Carr emphasizes the analogies between the different problems and stages the powerindustry had to go through in order to become a mature utility, and the difficultiesand stages the computing industry has gone through in the past and still has to gothrough in order to characterized be as a true utility.

Using Carr’s work, as well as the structured comparison between computationaland electrical grids from [30], a systematic overview of the similarities and differences


between IaaS clouds and electrical grids was constructed in Table 1.3.From this table, it can be learned that most of the similarities between IaaS

clouds and the electrical grid are based on their core idea to deliver a commoditizedutility (electricity vs. compute cycles) through a unified network. That is, leveraginglarge economies of scale in order to offer consumers a pay-what-you-use model thatallows for easy and virtually infinite scaling. By doing so, consumers no longerneed to worry about the underlying complexity of setting up and maintaining thehardware that generates the utility.

The main differences between the electrical grid and IaaS clouds are derivedfrom the observation that the electricity industry is a far more mature industrythan the cloud computing industry. It is important to note that this immaturityis not due to a lack of technological advancement, but rather due to the lack ofguaranteed high reliability and quality of service in the current cloud computingindustry. Other issues such as the absence of standardization, regulation or a marketon which providers and consumers can trade resources, also contribute to this. Infact, most of the differences between IaaS clouds and electrical grids are stronglyrelated to the current challenges cloud computing faces (see section 1.1.4). As such,if these challenges can be overcome, cloud computing will become more like a trueutility and consequently, the analogy between IaaS clouds and electrical grids willbecome stronger. Of course, it is important not to take the analogy between IaaSclouds and electrical grids to far as in the end they deliver utilities that address verydifferent use cases.

1.3.2 Power industry markets

In [31], a detailed analysis of the fundamentals of power system economics is given.Much of the information that is presented here reflects concepts and ideas explainedin that work.

For most of the twentieth century, when consumers wanted to buy electricalenergy they had no choice but to buy it from the organization that held the monopolyfor the supply of electricity in their region. However, by the 1980s, economistsstarted arguing that such a model was no longer sustainable. It was argued that themonopoly status of these organizations removed the incentive to operate efficientlyand encouraged unnecessary investments. Additionally, the economists argued thatthe cost of mistakes made by private organizations should be not be passed on toconsumers, as is almost always the case in monopoly economies. In order to solvethis, economists suggested that the supply of electricity should become the object ofmarket discipline rather than being regulated by a monopoly or government body.

After 20 years of lobbying and making changes in regulations and processes, it issafe to say that today the liberation of electricity market is a fact in most (western)countries. Because of this, the general structure of the electricity market has becomefar more complex than back in the days of electricity monopolies as ideas, principlesand mechanisms from existing commodity markets have been applied to the open


Feature Electrical power grid IaaS Clouds

Network Transmission lines, undergroundcables. Various sophisticatedschemes for line protection.

Internet

Ease of use Very simple: plug and play Complex: setup of applicationsand software

Reliability High: important lines are dupli-cated. Sophisticated protectionschemes exist for power stations.

Potentially high: Reliable anddurable technology exists. Ac-tual long-term reliability remainsto be proven.

Storage Only for low-power DC usingbatteries.

No storage of computation powerpossible; data can be stored.

Security Fuses, circuit breakers, . . . System isolation (virtualization).Firewalls, PKI, . . .

Payment model pay-what-you-use pay-what-you-use

Scaling model “infinite” “infinite”

Regulators In general, managed by au-tonomous bodies and govern-ment regulators.

Non-existent

Standardization Many widely used standards(sockets, voltage, DC/AC, trans-mission lines, . . . )

No standards. Efforts are on theway. Most notably: OpenStack[12].

Driver Economies of scale Economies of scale

Service LevelAgreement

Far reaching SLAs and penalties. Generally very little guarantees.

Resource hetero-geneity

Many different kind of genera-tors. Heterogeneity hidden fromconsumers.

Many different kind of computa-tional resources. Heterogeneityonly partially hidden by virtual-ization technology.

Market scope Exchange (spot) market tradeamong providers. No exchangemarket for consumers.

No exchange market forproviders nor consumers.

Time dependencyof demand/sup-ply

Demand and supply strongly cor-related to time of day.

Demand and supply correlated totime of day.

Table 1.3: Comparison of the electrical power grid with current IaaS clouds


electricity market. As a result, the structure of the electricity market now has severallayers and levels, involves many different actors, and stimulates active competitionamong its participants. To understand better how this market is exactly structured,consider figure 1.6 which depicts a schematic overview of the most evolved form ofan open electricity market5.

Figure 1.6: Retail competition model of electricity market based on model from [32].Source: [31]

The most prominent feature of the diagram is the fact that 2 different sub-markets are shown: a wholesale market between electricity generating companies(gencos) and electricity retailers, and a retail market between consumers and retail-ers. The existence of these separate markets is rather important as it indicates thatthe companies that generate power do not need to be the same companies that sellthe power to consumers (although they regularly are). Implicitly, this also meansthat it does not need to be case that the companies that distribute generated power(so-called discos; which can be different companies than the gencos) to the consumersare the same companies that are being paid for the delivered power. Indeed, in themarket model as shown here, it is possible that independent retailers lease powerlines from discos and sell electricity directly to consumers. Such a market situationis particularly interesting for consumers as it gives them choice between differentretailers, allowing them to buy electricity from the retailer that is the cheapest intheir region or best fits their needs. Note that figure 1.6 also indicates that largecustomers (such as companies that run factories) can buy their electricity directly on

5Several less competitive (evolved) forms of electricity markets also exist. The discussion of suchmarkets is however outside the scope of this thesis. For more detailed information, see [32].


the wholesale market, taking out the middle man and margin that retailers typicallytake on the energy they resell.

To get the required power for their customers, retailers thus have to buy electric-ity from the different generating companies through the wholesale market. Buyingof this electricity can be done through bilateral deals between the gencos and theretailers, or through a managed market in which a third party takes part in thedeal-making process.

When a bilateral deal is made between a genco and a retailer, they typicallyagree on a certain price per kilowatt-hour (KWh) that needs to be delivered by thegenco over a certain period of time. Retailers typically make several of such dealswith different gencos in order to acquire the power that they need in order to satisfytheir customers for a certain period. Note that these deals are entirely private; buyerand seller are in principle the only parties that know that a trade has occurred.

This is different in a managed market, where a third party helps buyers andsellers find each other more easily. In practice, this is typically done by letting bothbuyers (retailers) and sellers (gencos) submit offers with a price and an amount theywish to sell or buy to the third party which will then find (optimal) matches be-tween all submitted offers. Using such a managed market system has the potentialadvantages of being fair and optimal while guaranteeing anonymity among the mar-ket participants (besides the involved buyer and seller, only the market itself knowswhich deal occured).

As simple as the idea of retailers buying electricity from gencos and selling itto consumers may sound, there is a specific issue that comes with this way of trad-ing that deserves extra attention. That is, as the exact consumption behavior ofcustomers is typically very unpredictable, it is generally very difficult for retailersto know up-front how much electricity they will exactly need for a given period,making it thus impossible for them to trade the exact amount of electricity theywill need. Although heuristic-based techniques can help to make better consump-tion predictions (for example, statistical analysis may yield expected consumptionfor a given period), it is typically very hard to make accurate predictions on smalltimescales such as a single hour or less. However, making accurate provisioning isvital to retailers, as not being able to do so may lead to serious financial losses.More specifically, in case of overprovisioning, a serious financial loss for the retailercan be incurred as he will be unable to sell already paid for energy to consumers.Underprovisioning on the other hand, might mean that certain customers are un-serviced, risking loss of customers and potential penalties or law suites because ofbreaking certain service level agreements.

Related to this is the fact that in many cases, it is very difficult for gencos toquickly respond to requests for additional power as generating extra electricity mightrequire the startup of additional generators such as windturbines, hydroelectric- orfossil-fuel plants and so on, which can be both expensive and time-consuming. Sim-ilarly, shutting down generators might be time-consuming and may incur additional


future costs when they need to be restarted. Also, as gencos and retailers make upcontracts up front, gencos will typically claim penalties when retailers want down-scale contracts that were previously agreed upon.

In practice this problem is solved by the introduction of a so-called managedspot market (more about spot markets in section 2.2.6) in which retailers in thesame region can buy and sell electricity to each other on an hourly basis. Thisallows retailers to cope with over- or under-capacity on very small timescales. Suchtrading is possible due to the fact that the electricity distribution system is in facta single grid on which all gencos from which the retailers buy their electricity areinterconnected. Although regulation of such a market is needed to ensure that themarket is balanced - it is important to have demand meet supply up to a certainextend to ensure the stability of the grid - in practice the market tends to regulateitself due to the general principles of demand and supply. That is, if the demand forelectricity is high for a certain period, spot prices will automatically go up, making itmore interesting for retailers to buy more electricity from gencos instead of buyingit from other retailers, subsequently lowering demand and thus lowering the spotprice. If demand is low, prices will go down, making it more interesting for retailersto buy electricity from other retailers, affecting demand, making the prices go upagain.

As in reality, many gencos are also retailers, the provisioning problem becomesmore complex for them, as such retailers can both generate the electricity theyneed to fulfill the demand of their customers, as well as buy and sell extra capacityon the spot market. Producing and trading electricity can no longer be strictlyseparated in such scenarios, as the most cost-efficient and profitable way of deliveringelectricity becomes a combination of bilateral trading, spot market trading and in-house electricity production.

1.3.3 IaaS Cloud Market

As the IaaS market continues to grow and the number of IaaS players continues toincrease (as predicted by Gartner [33]), it is likely that in time the IaaS providerlandscape will evolve towards an ecosystem that is similar to that of the powerindustry. That is, monopoly positions of large companies such as Amazon will shrinkand direct competition among IaaS providers will increase. As a result, a marketmodel comparable to that of 1.6, in which different IaaS providers and retailers sellcomputing resources to each other and to consumers, is likely to emerge. Figure 1.7shows such a model.

In this IaaS market model, different IaaS providers sell IaaS resources to eachother and to retailers on the wholesale market, while retailers sell resources to con-sumers on the retail market. Note that where the single electricity grid made itpossible for providers and retailers to resell electricity to each other in the case ofpower markets, the Internet is now the main enabler of the IaaS market. The under-lying idea of this is that for a whole range of IaaS applications, the actual location


Figure 1.7: Retail IaaS market based on the electricity retail market.

of the physical server on which the resources are allocated is insignificant (and con-sequently the actual provider that provides these resources is too), as long as theconsumer can access them with a reasonable quality of service6.

Although the retail market and the wholesale market are depicted similarly inFigure 1.7, it is important to note that they are actually of a different nature. That is,as is the case in the electricity industry, the wholesale market should be considered asa (managed) market on which actual complex trading processes take place betweenproviders, retailers and large consumers, while the retail market should be observedas the ad-hoc market that is formed by the free choice that consumers have betweendifferent retailers.

In this context, the function of the retailer can effectively be seen as that ofa broker that hides the complexity of the wholesale market from the consumer.Consequently, it is clear that the existence of retailers (or retail market) is not anecessity for an IaaS market to exist, as it is also possible that end consumers takepart in the (wholesale) market directly. As such, the fact whether the wholesalemarkets trades only among providers and retailers (inter-provider trading) or alsoamong providers and (end) consumers is a question of scope that has only limitedinfluence on the design of the mechanisms that are used in that market. In a lesscomplex model, consumer and providers thus trade directly with each, without the

6Of course due to issues surrounding data locality - in many cases programs need to run on alocation (machine) where specific data is present - the statement that any program can run anywhereis not true in general. However, for a large number of applications that data can be transferred tothe destination along with the program itself. Applications that have more heavy data requirementscan use e.g. distributed file systems to share data both efficient and transparently.

1.4. PROBLEM STATEMENT 27

explicit notion of the retail market.

Although the successful introduction of an IaaS market system has the potentialto materialize the long-lived idea of a truly global computing utility, it is crucialthat this market will be an open market in which many providers and consumerscan participate. That is, only when a market exists that has a sufficiently low entrybarrier that brings together many different consumers and providers, will the priceof computing resources be subject to market discipline rather than being dictatedby hardware vendors or specific IaaS providers.

In order to come to such an open market, there is a need for a exchange marketsthat allow different parties to trade more easily by abiding to a predefined set of clearrules and mechanisms. Only by using such exchange markets will trading becomeaccessible, fair and transparent.

Actually designing and implementing such an exchange market for IaaS resourcesis however no easy feat. The main difficulty with designing an IaaS exchange marketis the fact that the good that is being traded is inherently more complex than inan other commodity markets such as the electricity market. More specifically, theproblem lies in the fact that the e.g. kilo-watt hour is a one-dimensional homoge-neous unit, while typical IaaS resources are heterogeneous and multi-dimensional(e.g. an instance types is characterized by the different hardware components it isbuilt of). As standardization of resources and APIs among IaaS providers is virtuallynon-existent, defining which goods are being traded is non-trivial.

Additionally, in reality many providers use different allocation and price models(all based on the pay-what-you-use model, but implemented differently) to chargetheir customers. Bringing together these different models into a single IaaS exchangemarket is not straightforward as it involves searching for a common denominatorbetween all these models.

The design of a trading language for IaaS resources,the IaaS exchange marketand particularly the exchange’s components and mechanisms are the core subjectsof this thesis, and are progressively discussed throughout the following chapters.

1.4 Problem Statement

As it is important to very clearly state the main problems this thesis tries to solvein order to outline which related issues are part of its actual subject and which arenot, this section specifies these problems explicitly. In particular, the core questionthat this thesis tries to answer is threefold and as follows.

(a) Whether it is theoretically feasible to design an open exchange market for IaaScloud resources?

1.4. PROBLEM STATEMENT 28

(b) What would such an IaaS exchange market look like, what are its core compo-nents and mechanisms?

(c) How can existing web service technology be used to implement such a market?

Of course, determining whether it is possible to design an open market can onlybe done by actually trying to design such a market. This effectively means that theanswer to the first question is given by the answer to the second question. However,to answer that second question, several sub-issues need to be addressed.

First of all, it is important to determine and define the different compo-nents and actors an IaaS market is made of, which can largely be done by havinga look at how existing (commodity) markets are designed. This involves determin-ing which actors can interact directly with the market that is being designed, andwhich have to use secondary channels to do so. This also involves indicating whichfunctionalities and responsibilities are part of the core of the market, andwhich ones can be taken care of by third parties.

An other important aspect of the design of the market will be the choice of themarket mechanism that determines which provider offers are matched with whichconsumer offers. As the matching mechanism effectively determines how consumersand providers interact with each other, specifying how it this mechanism worksexactly is obviously very important.

The choice of this matching mechanism language is one of the factors that hasa profound impact on the definition on the bidding language that determineshow different market participants should specify their offers. As IaaS resources aretypically very heterogeneous, this bidding language should be powerful enough toexpress that heterogeneity.

Additionally, some attention will be spend on the issue of price discoverywhich deals with how consumers and providers can get an idea of the value thatthe market currently attaches to certain IaaS resources. Price discovery tools areimportant as they shape market behavior by giving consumers and providers an ideaof how expensive a given resource is at a given moment, allowing them to act onthat.

Once these theoretical questions have been addressed, focus will be given to thethird question that asks how these proposed ideas can actually be implemented. Theanswer to this question will be given by the specification of a software design andarchitecture of the market. To verify that this design can be put into practice,a prototype implementation will also be build and briefly discussed.

As the goal of this thesis is to design an open market for IaaS resources in whichall providers and consumers can participate, the author believes that it is important

1.5. RELATED WORK 29

for the market design to be tailored to today’s IaaS cloud ecosystem to ensure itsreal-world applicability. This effectively means that a number of abstractions andgeneralizations of IaaS characteristics that are made in other work will not be appliedin this thesis. Doing so ensures that existing IaaS providers can easily take part inthe market. In other words, not forcing providers in a certain price, hardwareor allocation model is a key differentiator of this thesis from other work.As will become clear in the remainder of this thesis, this choice has a significantimpact on actual market design.

1.5 Related work

In order to give a broader context to the core questions and issues that have justbeen outlined, it is important to reference some work that is related to this thesis.

As already mentioned, the idea of introducing a global computing network thatanyone can access has already existed as part of the Grid Computing paradigmfor many years. As such, a lot of related work has already been done in the GridComputing context. Some insightful work in this domain includes [34], [35], [36] and[37]. A good overview of the recent evolution in the grid market area is given in [38].

Most of this work has however focused on grid economy management, utility-driven scheduling and user value-driven resource allocation, while mostly ignoringprovider profit. However, in the cloud computing context, providers and providerprofit are first citizens. As such, not all ideas and principles from this previous workare still applicable in the IaaS domain.

Some Grid Computing research that is closer related to the cloud computingparadigm is given in work by Nicolas Dube and by the GridEcon project.

In [39], Nicolas Dube designs a market for grid computing resources based on thecontinuous double auction mechanism. In his work, Dube develops a way to describeheterogeneous computer resources, and describes how using so-called resource setsin conjunction with a continuous double auction can be used to develop a marketmodel for grid computing. Dube also incorporates bidding strategies that lead to amarket equilibrium in his model by using an index-based price discovery mechanism.

The GridEcon project [40] [41] [42] was a “Sixth Framework Programme” Eu-ropean Community funded project, that “explored the perceived economic barriersto the adoption of grid and cloud computing”. In particular, the GridEcon projectworked out market matching mechanisms that determine which consumer can haveaccess to limited provider resources by using virtual machine scheduling algorithms.A prototype of the proposed market architecture including the matching algorithmscan be found on the GridEcon website.

1.6. THESIS OUTLINE 30

Unrelated to grid computing is the recent launch of the Spotcloud platform [43]by the US-based company Enomaly. Spotcloud is a spot market for cloud computingresources that is built on Google’s App Engine which offers consumers “a securecentral platform for buying or selling unused cloud capacity based on location, costand quality”. Specifically, SpotCloud allows consumers to choose from a set ofhardware profiles and VM images that have been submitted by providers. Pricingis determined by the providers and is fixed per appliance (hardware and VM) perhour; this price also contains all extra costs such as bandwidth or storage costs. Inorder to make profit, Enomaly takes a 30% cut from the profit of the providers.

1.6 Thesis outline

Chapter 1: IntroductionThis first chapter introduced the concept of Cloud Computing, discussing its coreprinciples and characteristics. After explaining cloud computing’s layered structure(IaaS, PaaS, SaaS) and deployment model (public, private and hybrid clouds), someadditional benefits and challenges were discussed. Some real-world examples of cloudcomputing products were subsequently given and discussed. After this, the parallelsbetween IaaS Clouds and the electricity industry were indicated by a summary ofthe main similarities and differences between both utilities. This lead to the topicof electricity markets, and subsequently to the idea of building a similar market forIaaS Clouds. The last sections clarified this idea by making the relevant researchquestions more explicit and discussing related work.

Chapter 2: Market Design PrinciplesThe second chapter provides some background information about markets in general,which is needed to understand what the important elements are when designing amarket for IaaS resources. To this end, different types of markets, auction modelsand the typical components of a commodity market are briefly covered and relatedto the design of an IaaS market. Also, a short outline of some desirable marketproperties is given. The goal of this chapter is not to provide an in-depth analysisof each of these topics as a lot of other work already does that, but to outline therelevant fundamental notions useful to computer scientists.

Chapter 3: IaaS Cloud Computing Market: The Continuous Reverse AuctionThe third chapter elaborates on the most critical parts of the IaaS market, namelyits bidding language and matching (auction) mechanism. To do so, it starts with aContinuous Double Auction based market for grid resources developed by NicolasDube. Once this market model has been properly introduced, some problems thismodel faces in the cloud computing setting will be pointed out. Working from theparticularities of these problems, a new bidding language and market mechanism(the Continuous Reverse Auction) that is able to cope with the specific issues of theIaaS ecosystem, will be iteratively constructed.

1.6. THESIS OUTLINE 31

Chapter 4: Market ImplementationThe design of market mechanisms is mainly theoretical in nature. Nevertheless,investigating how markets can be implemented in practice is valuable as this imple-mentation is a complex technological undertaking. In order to give an idea of theinvolved complexity, this chapter focuses on the concepts that are introduced in theprevious chapter, and discusses how they can be implemented concretely. Further-more, the prototype that was implemented as part of this thesis will also be brieflydiscussed at the end of this chapter.

Chapter 5: ConclusionThe final chapter of this thesis summarizes the previous chapters and puts forwardthe core concepts and ideas that were introduced. To finish, it puts forward somefuture work and formulates a final conclusion.

CHAPTER 2

Market Design Principles

This chapter provides some background information about markets in general, whichis needed to understand what the important aspects are when designing a marketfor IaaS resources. To this end, different types of markets, auction models and thetypical components of a commodity market are briefly covered and related to thedesign of an IaaS market. Also, a short outline of some desirable market propertiesis provided. The goal is not to provide an in-depth analysis of each of these topicsas a lot of other work (such as [44]) already does that, but to outline the relevantfundamental notions useful to computer scientists.

2.1 Introduction to Markets

When designing a market, a very large number of factors need to be taken intoaccount. Trying to find all of these factors by brainstorming and unstructuredreasoning is destined to lead to a flawed design in which certain important aspectshave not been taken into account. As such, it is imperative to start the marketdesign process by first taking a step back and having a look at how existing marketsoperate. This section starts by giving a generic (yet practical) definition of what amarket is, followed by a oversight of how markets can be categorized.

2.1.1 Market: a definition

In mainstream economics, a market(place) is any system that allows buyers and sell-ers to exchange goods, services or information. This exchange can be done directlybetween the different buyers and sellers or through intermediaries.

The notion of competition is essential in a market as it distinguishes a marketfrom regular trade. That is, two parties may trade goods, services or information,but it takes at least a third party to have a market so that there is competition

2.1. INTRODUCTION TO MARKETS 33

on at least one of its two sides. In other words, in a market, forces of demand andsupply are always present. Markets typically vary in size, location and type as wellas in the types of goods and services traded.

Many other formal and informal definitions of markets can be found in variousworks [45] [46].

2.1.2 Market Organization

There are many ways to categorize different kinds of markets more specifically. Oneway to do this is by looking at how competition among the different market par-ticipants is structured. Examples include markets of imperfect competition such asmonopolies (and others) and markets of perfect competition in which the behaviorof a single buyer or seller does not affect the market’s outcome. Although a detailedstudy each of these market structures (conducted in the field of micro-economics)can result in valuable insights, it has less added value in the context of this thesis asthe resulting conclusions are mainly of a theoretical nature. For a general overviewof market competition structures and micro-economics, [47] provides a good intro-ductory work. More in-depth analysis can be found in [48].

A practical way to classify markets, is by the way goods are traded. That is,in some markets there are intermediates that help buyers and sellers trade good toeach other, while in some markets buyers and sellers trade directly with each other.Markets in which intermediates have a prominent role are sometimes also referred toas managed markets or exchange markets. When there is no such party involved, theterm over-the-counter (OTC) or bilateral trading is often used. Where a core aspectof OTC trading is that no strict form-factor exists, exchanges are typically char-acterized by transparent pricing, clear basic trading regulations and mechanisms,and precise transaction costs and fees. Additionally, a typical exchange market in-cludes mechanisms to determine the price of the traded item, to communicate priceinformation to its participants and to facilitate deals and transactions.

When studying exchange markets in greater depth, the distinction between aso-called dealer market and auction market is usually made. In a dealer market,human market makers manually match deals from sellers and buyers, allowing themto make profit based on the difference between the bid and the ask price of theasset that is being sold. The theory is that competition between dealers will pro-vide the best possible price for investors. In an auction market, a certain type ofauction mechanism is used to facilitate trade among participants. Such an auctionmechanism has the clear advantage that it is unbiased, simple and deterministic,facilitating its implementation in electronic trading platforms. As auction mecha-nisms are especially important in commodity markets, they are discussed in moredetail in section 2.3.

A third way to classify goods is by the types of good or services the market

2.2. FINANCIAL MARKETS 34

trades. The most well-known of these are the physical retail markets, such as localfarmers’ markets which are held in town squares on an ongoing or occasional basis.These markets are characterized by their adhoc nature and the absence of strictregulation. More regulated and structured are the so-called financial markets, whichis a group of markets that focus on trading either financial securities (such as stocksand bonds), liquid assets or commodities through exchanges or OTC trading. Asthe financial markets are widely considered to be the most mature and sophisticatedtrading platforms, having a closer look at them can provide valuable insight. Inthe following section, an overview of the most important financial markets is giventogether with some considerations of how some ideas and principles of these marketsapply to the IaaS cloud market.

2.2 Financial Markets

2.2.1 Stock markets

A stock market (or equity market) is a public market that trades company sharesand derivatives (see 2.2.5). The stock market can be considered as one of the mostessential pieces of a market economy as it gives companies access to capital and allowsinvestors to acquire partial ownership of a company. The stock market consists oftwo sub markets, called the primary and secondary market. The primary marketdeals with issuing the shares of a company when it goes public. The trading of allother (already public) shares happens on the secondary market. Some of the mostwell-known and largest financial markets in the world are stock markets, usually(but not necessarily) named after their geographical location. Examples are givenin Table 2.1.

Stock Exchange Abbreviation

New York Stock Exchange NYSELondon Stock Exchange LSEFrankfurt Stock Exchange FSETokyo Stock Exchange TSEEuronext Paris -NASDAQ (New York) NASDAQ

Table 2.1: Well-known stock exchanges

As can be seen in the table, stock markets are almost exclusively exchange mar-kets; OTC trading is almost never used for stocks. The type of exchange mechanismthat is used is typically chosen by the exchange itself, and can thus be different foreach exchange. For example, the NYSE is an auction market that uses an open-outcry double auction mechanism (see 2.3.5), while the NASDAQ is a dealer market.

As the value of stock market enlisted companies is effectively determined on that


market, and as in reality almost all large companies are enlisted on such a stockmarket, it is essentially possible to extract aggregated information about a largerset of companies at once. Doing so can give buyers and sellers a better idea of howan industry or even an entire economy is doing. The means of doing this is througha so-called market index, which shows how a set of publicly owned companies havetraded during a standard trading session in the stock market. By comparing thevalue of the index against previous values, economists can get an idea how markets,industries or economies are forming. Some well-known indices are shown in Table2.2.

Index Aggregation

Dow Jones 30 large US-based companiesNASDAQ Composite Over 3000 component, mainly technology-

oriented companiesS&P 500 500 large US-based companies that trade

on NYSE or NASDAQNikkei 225 225 large companies listed on the TSEFTSE 100 100 most highly capitalized UK companies

listed on the LSE

Table 2.2: Well-known stock market indices

Although stock markets and markets to trade IaaS resources seem to have littlein common, there is an important lesson that can be learned from the discussion ofstock markets in the context of this thesis. Namely that even a market that is as largeand mature as the stock market still exists of multiple exchanges that all use differentrules, auction types and have different specializations. In other words, even afteryears of experience and change, markets still remain highly heterogeneous. As such,expecting that a single exchange market will emerge on which all IaaS resources willbe traded is not realistic. Instead, it is more likely that through a combination ofresearch and commercial incentives, multiple IaaS exchange markets will emerge thatuse different mechanisms, rules and practices. As the IaaS industry is particularlyheterogeneous (both in the resources that are offered as in the use of different priceand allocation models), there is certainly room for multiple IaaS exchange marketsthat have different characteristics and specialties. Given these remarks, it is clearthat this thesis does not claim to search for or present the solution, but rather topresent a solution that is based on well-founded ideas and principles.

2.2.2 Bond markets

The bond market facilitates the trade of bonds. A bond is a (typically long-term)debt security by which an investor (who buys the bond) lends money to the issuer ofthe bond. The issuer of a bond is then obligated to repay his debt in fixed slices alongwith an interest rate. This repayment is do be done at fixed intervals (usually oncea year). When mentioning the bond market, one usually refers to the government


bond market because of its size and liquidity. (Similarly to the stock market, thebond market can be divided in a primary and secondary market in which bonds areissued for the the first time or re-traded among market participants). Contrary tostock markets, most trading is done OTC in bond markets.

2.2.3 Foreign exchange markets

Also known as currency markets or forex markets, foreign exchange markets allowOTC trading of currencies1. Although different financial centers can be consideredas different forex markets, most of the time only a single global forex market ismentioned that depicts the worldwide decentralized market formed by these inde-pendent geographically dispersed forex markets. Note that this is also regularlydone in the IaaS cloud market context, where the previously mentioned differentdecentralized exchange markets are also commonly denoted as a single worldwidemarket. In the remainder of this thesis, context will indicate whether with the IaaSmarket only a single exchange market is meant, or the worldwide market formed bythe independent IaaS markets.

The primary goal of the global foreign exchange is to make international tradeeasier, by allowing businesses to convert one currency to another. The current forexmarket took shape during the early 1970s when countries gradually switched fromthe fixed exchange rate system as defined by the Bretton Woods System2 to floatingexchange rates.

2.2.4 Commodity Markets

A commodity market is characterized by the fact that it facilitates the exchangeof raw and easily described products. Typical examples of such products includeagricultural commodities such as cotton, wheat and cattle; (precious) metals suchas gold, silver, copper, aluminum and others; and energy commodities such as elec-tricity, natural gas and uranium. In order to do the actual trading, standardizedcontracts that describe good quality and characteristics, volume, delivery date andprice are bought and sold among the market participants.

Rooting from the middle of the nineteenth century in the United States to tradeagricultural products, commodity markets have grown to become the largest of allfinancial markets in terms of daily traded monetary volume. Trade can occur bothOTC or through market exchanges. Today about 30 major of such commodityexchange markets exist, of which the Chicago Mercantile Exchange (CME) is prob-ably the most well-known. Table 2.3 lists some other examples of large commoditymarkets along with the products that are traded on these markets.

As cloud computing, and IaaS in particular, deals with the commoditization of

1Note that the term exchange is used, while in reality most trade is done OTC.2The Bretton Woods system was a system of rules and procedures that was agreed upon by the

allied nations at the end of the second World War, in order to regulate the international monetarysystem.


Exchange Location Product Types

Chicago Mercantile Exchange Chicago, US Meats, Currencies, Eurodol-lars, Equity Index

New York Mercantile Exchange New York, US Energy, Precious Metals, In-dustrial Metals

European Climate Exchange Europe EmissionsTokyo Commodity Exchange Tokyo, Japan Energy, Precious Metals, In-

dustrial Metals, Agricultural

Table 2.3: Well-known commodity exchanges

IT products (as already mentioned in the introductory chapter), it is logical thatthe study of commodity markets will play an important role when looking intoIaaS market design. In order for an IaaS market to be organized, it is importantthat some of the key concepts of commodity markets such as standardized tradingcontracts, bidding languages and good delivery are further examined and studied inthe context of IaaS resources. To do so, section 2.5 discusses the core componentsand functionalities of commodity markets and explains how these apply to IaaSmarkets.

2.2.5 Derivatives markets

Derivative markets trade derivatives, which are agreements between two partiesbased on the expected value of an asset (such as a share, bond or currency) at aspecified time in the future. The term derivative is used to denote the fact thatthe agreement itself derives its value from the (future) valuation of the underlyingasset. Due to the speculative nature of the derivative markets, they tend to attractinvestors that are willing to take more risk in order to leverage profits.

When the trading of a derivative is done using an exchange (as opposed toOTC), the term futures market or futures exchange are typically used. Futuresmarkets deal with the exchange of futures contracts (or just a futures), which is anagreement between two parties to buy or sell a specified asset at a specific date fora specific price (that is determined on the day the futures contract is negotiated).All futures contracts have a binding character, as they legally obligate the owner ofthe futures contract to buy or sell the specified asset. The so-called option does nothave this obligation, it gives the owner the right to buy or sell the specified asset atthe specified date, at a price that is agreed upon when the option is bought or sold.

Although the term futures market is mostly used to refer to the stock, bond orforex market, it can also be used to refer to any other market that exchanges con-tracts that are to be fulfilled at some point in the future. For example, in commoditymarkets, a futures contract refers to an agreement to buy or sell a specified volumeof a commodity for a fixed price at some point in the future. Clearly, the idea of fu-


Figure 2.1: Evolution of the electricity spot price for 10th of February 2011 on theEuropean Energy Exchange. Time is represented on the horizontal axis, price onthe vertical axis. The bold black line represents the average price over 24 hours, thered bold line the average price during peak hours (between 08.00h and 20.00h).

tures (and options) can also be applied to the IaaS context, where a futures contractwould involve now buying or selling a set of computational resources that are to beused at some point in the future. By trading futures contracts, both IaaS providersand consumers can make medium and longterm provisioning decisions more easily.

2.2.6 Spot Markets

Whereas a derivative (futures) market facilitates the trading of contracts that are tobe fulfilled in the future, a spot market accommodates trading assets or commoditiesthat are to be delivered (almost) immediately. The word spot market refers to thefact the price negotiation and trading is done on the spot, much as it has beendone for centuries on street markets. Spot markets are unique in the fact that theso-called spot price for a good or asset is typically quite volatile as it reflects thedirect relation between supply and demand for the good (or asset) within a shorttimeframe. This price volatility is caused by the fact that it is generally impossibleto create additional supply of the good that is being traded within the requiredshort delivery timeframe. Because of this volatility, the less volatile future marketsare regularly used to acquire the bulk of a specific good or asset, while spot marketsare more frequently used to deal with the real-time differences between needed andalready owned goods or assets. Although this inherit volatility of the spot marketcan make it difficult to predict the spot price at a given moment in time, historicalanalysis of the spot price of many goods frequently shows hourly, daily and monthlypatterns. Successfully speculating on these patterns, can induce large profits within


very small timeframes.

Examples of large spot markets are the forex spot market, in which traded cur-rency needs to be delivered within 2 days, and various energy spot markets. Asalready mentioned in the introductory chapter, in electricity spot markets, differentproviders trade electricity on an hourly basis. Doing so allows them to cope withthe mismatch between their provisioned capacity and the capacity they actuallyneed. In many cases, it is cheaper to buy extra electricity from other providersthan to pay for the startup cost to power on an extra generator (also, depending onthe generator type, starting an extra generator might take a substantial amount oftime). Likewise, in many cases it is cheaper to keep producing electricity and to sellovercapacity, than to shutdown a generator and risking to have to restart it againwithin the next few hours (incurring the startup cost) [49].

Figure 2.1 shows the evolution of the electricity spot price of the 10th of February2011 of the European Energy Exchange (EEX) [50] - the leading energy exchangein Central Europe. On the graph, you can clearly see human behavioral patterns.That is, the spot price is lower during night hours (when demand is lower), and haspeaks in the morning when most people wake up and during the early evening whenpeople arrive home from work.

In the IaaS cloud context, spot markets can also play an important role to dealwith short timespan volatility in demand and supply of computational resources.Such a spot market can be global in nature, as by leveraging the fact that manyIaaS applications are essentially location and provider independent, it is possible to(live) migrate applications according to where computational resources are currentlythe cheapest. This can be done among consumers and providers, but also amongproviders themselves (inter-provider trading) as is regularly done in the power in-dustry. By doing so, providers can effectively cope with short-term shortages orexcesses of computational resources in their datacenters.

Note that the computational resources that are acquired on such a global spotmarket do not necessarily need to follow the spot allocation model as introducedby Amazon [17], in which resources are automatically deallocated if the spot priceexceeds a consumer specified maximum price. Also, as the heterogeneity betweendifferent kind of computational resources is large, it will be difficult to specify asingle spot price (if not impossible, see section 3.2.1 for more information about theproblems with a single price component model for IaaS resources) for all types of re-sources. Instead, it is more likely that in such a spot market, specific predefined setsof resources (instance types) are traded as that greatly simplifies the introductionof a single spot price.

The use case in which resources are traded in a spot like manner is however notnecessarily tight to the use of a single spot price. Indeed, when a price for a set of

2.3. AUCTION TYPES 40

computational resources is negotiated on the spot (that is, directly) and allocationtakes place directly afterwards, there are not that much fundamental differenceswith spot markets that use a single spot price. As such it is reasonable to call anyIaaS market that adheres to the principles of direct allocation and demand/supplybased price volatility a type of spot market.

2.3 Auction Types

As already noted, many exchanges use auction mechanisms to determine which buyeris matched with which seller. The use of auctions is however not limited to officialexchanges, they can be used as a general process to sell any good. In general, anauction refers to any mechanism or set of trading rules for the exchange of goods;it is the process of buying and selling goods or services by offering them up for bid,taking bids, and then selling the item to the highest bidder.

In reality, many variations on this basic auction form exists. Differences amongthese auctions include time limits, minimum or maximum limits on bid prices, andspecial rules for determining the winning bidder(s) and sale price(s). Also, partici-pants in an auction may or may not know the identities or actions of other partic-ipants. Depending on the auction, bidders may participate in person or remotelyusing telecommunication or computer networks (telephone, Internet, etc.). In orderfor the auctioneer (the entity that organizes the auction) to make profit, the sellerusually has to pay a commission that is either fixed or variable (e.g. based on apercentage of the final sale price).

Below, the most prevalent auction types are discussed in more detail.

2.3.1 English Auction

The English Auction is the most well-known of auction mechanisms. It is oftenused in cases where a single seller wishes to sell a single (or multiple) relatively raregood(s) to one of multiple buyers. During the auction, buyers iteratively bid for thegood that is being auctioned. More specifically, when placing a bid, a buyer shouldalways bid higher than the highest bid that has been placed until then (called thestanding bid). The buyer that eventually bids the highest price for the good wins theauction. Since all bidding is done publicly, all participants know what the highestbid is (and who placed it) at any given moment. In many cases, the seller is allowedto set a reserve price - a minimum price at which the auctioneer starts acceptingbids. English auctions end at a specific predetermined time, or when no new bidshave been made for a given length of time. The English auction typically favorsthe seller as bidders tend to overbid the good in the heat of the bidding process, aneffect commonly referred to as the winner’s curse.

A bidding system in which all bids are public is typically referred to as an open


outcry auction, as historically bidders cried out their bids (this is still done in somecases, but alternatives signaling such as raising a bidding paddle also exist).

2.3.2 Dutch Auction

The Dutch auction, also known as an open-outcry descending price auction, uses atotally different bidding mechanism. Instead of letting buyers compete with eachother directly by increasing the price of the standing bid, the dutch auction startswith a high asking price (set by the seller) that is iteratively lowered until a buyer isfound that is willing to pay the last announced price. As with the English Auction,the dutch auction is called an open-outcry auction since all other participants knowwho won the auction at which price. The dutch auction gets its name from theyearly tulip auctions in the Netherlands, in which it is used.

This type of auction is primarily adopted when goods are to be auctioned quickly,as completing the auction only requires a single bid. Additionally, the descending-price structure of dutch auctions increases competition among bidders as it makesacting fast a necessity.

2.3.3 Vickrey Auction

The Vickrey Auction, named after its inventor William Vickrey, is a second-pricesealed-bid auction which means that bids are not made public and that the bidderthat bids the highest price wins the auction, but only needs to pay the price ofthe second highest bid. Using this mechanism, it is relatively easy to prove [51]that the Vickrey Auction is incentive compatible in some situations. Section 2.4.3provides more details on incentive compatibility, while [52] gives more insight aboutthe situations in which the Vickrey Auction is incentive compatible.

However, despite the desirable incentive compatibility property in some situ-ations, there are some disadvantages to using the Vickrey auctioning mechanism.First of all, it allows for the seller to use so-called shill bids to increase his profit.That is, if the auctioneer and the seller have shared interests, the auctioneer mayinform the seller of the current highest bid, at which point the seller can make bidsthat are only slightly lower than the highest bid, forcing the winning participant to(almost) pay the price of the winning bid. A second downside to the Vickrey auctionis that is does not allow participants to get an idea of what a certain good is worthto others3 as the valuations (=bids) of the participants are not disclosed.

When multiple units are being auctioned, the Vickrey auction generalizes to auniform price auction (see section 2.3.6) which is not incentive compatible. However,a variation of the Vickrey auction - the Vickrey-Clarke-Groves (VCG) auction - existswhich remains incentive compatible for multi-unit auctions [53].

3The process of discovering what the current market value for a certain good is, is called pricediscovery and is discussed in more detail in section 2.5.4.


2.3.4 Reverse Auction

In the reverse auction (RA), the roles of buyers and sellers are reversed. That is,sellers compete to obtain the right to sell to a single buyer. The reverse auction isthus typically used in cases where a buyer wishes to buy a good or service (as iscommon with reverse auctions) and that several parties are interested in selling thegood or service to that buyer. Reverse auctions are typically organized in severalbidding rounds after which the lowest bid price is made public. New bids can onlybe lower than the standing bid. This mechanism has the effect that the price of thegood decreases as the auction progresses, in some cases significantly towards the endof the auction. This makes the reverse auction buyer-oriented, as the buying partyis the one that benefits from the competition among the sellers.

The reverse auction is primarily used in cases where considerable financial stakesare involved, or when a lot of different providers are available. For example, thereverse auction is commonly used to procure large service or construction contractsin the private or public sector (such as road construction or factory maintenance).

2.3.5 Double Auction

The double auction (DA) is used in situations when there are both numerous buyersand sellers (for example, in commodity markets). In the double auction, sellers offerthe quantity of the good they wish to sell, as well as the price at which they wishto sell the specified amount. Buyers use the same bidding language, that is, theiroffers contain both the amount they wish to buy, as well as the price at which theywish to buy the specified amount for. In order to avoid confusion, an offer placedby a seller is commonly known as an ask, while an offer placed by a buyer is referredto as a bid.

Typically, a double auction uses time frames during which it accepts asks andbids (e.g. an hour). At the end of such a time frame, a specific matching algorithm(also called the clearing algorithm) is used to match asks with bids. Since findingan optimal solution - that is, the matching that maximizes the allocative efficiency(for more details about allocative efficiency, see section 2.4.1) - might be a verytime-consuming or even intractable task (the number of combinations between asksand bids becomes rapidly very large), various heuristics are typically applied to finda reasonable matching within acceptable time.

A common variant of the double auction, the continuous double auction (CDA),uses such a heuristic to improve matching speed. In the CDA, a bid or ask is, whenpossible, immediately matched to an existing ask or bid. If no matching bid or askexists, it is added to the pool of asks or bids that have not been matched yet so thatit can be matched at a later time. By matching each offer at the time it arrives,the CDA avoids the combinatorial explosion that typically occurs at matching time.Of course the downside of this, is that the CDA can only be maximum allocativeefficient for the incoming bid and ask at that moment in time while the regular double


auction can (theoretically) find a maximum allocative efficient matching under allthe offers that arrive in a time frame4.

In order to reduce algorithm complexity even more, many CDA implementationsuse the double sorted queue algorithm. When using this algorithm, the auctioneerkeeps 2 queues: one for unmatched bids and one for unmatched asks. The queue ofunmatched bids are sorted by decreasing price, the queue of the unmatched asks aresorted by increasing price. It is the sorting of the queues that results in the reductionof algorithmic complexity. That is, when a new bid comes in, the algorithm iteratesover the unmatched ask queue and returns the first ask it finds that has a price thatis less or equal to the price of the incoming bid, and that has a matching quantity.The process for an incoming ask is analogue.

Whether quantities can be split or have to be matched exactly is something thateither has be agreed upon before the start of the auction or has to be specificallydefined for each bid or ask. Since the queue is ordered by increasing price, one issure that once a matching ask is found that there is no better ask, as matching anask that is further down the queue is always less allocatively efficient than matchingan ask that is higher up the queue. When no matching ask can be found, theincoming bid is added to the right place in the queue of unmatched bids. Note thatthis technique is not specific to continuous double auctions, it can also be used todecrease the complexity in the matching algorithm of regular double auction.

2.3.6 Uniform Price Auction

The uniform price auction is a multi-unit auction in which bidders bid both a numberof units and a unit-price. Bids are typically sealed. When the auction finishes, theauctioneer first serves the highest bidder, allotting him the requested number ofunits. Then the second highest bidder is served, than the third, and so on untilno more units are available. All bidders pay the same unit price - the lowest pricesubmitted by one of the buyers that are receiving units - independently of the unitprice they originally bid. Variations exists in which the winners pay the highestlosing bid instead of the lowest winning bid.

2.3.7 Combinatorial Auction

In a combinatorial auction, buyers place bids on combinations of items (called pack-ages) on sale, instead of bidding on individual items. After the bidding period, theauctioneer selects the set of bids that result in the most revenue for the seller withoutassigning any item to more than one bidder. Obviously, doing this is NP-complete(this problem can easily be reduced to the well-known NP-complete knapsack prob-lem [54]), so in many cases the auctioneer uses heuristics to approximate the optimal

4Note that choosing the size of this time frame is difficult, as it can always be that an offerarrives just outside of a time frame that would have resulted in a more allocatively efficient match.The exact size of the time frame is typically dependent of the specific case in which the doubleauction is used, as the industry domain itself typically dictates the time constraints.

2.4. DESIRABLE MARKET PROPERTIES 44

solution(s). Combinatorial auctions have been studied extensively, more informationcan be found in [55] [56].

2.3.8 Walrasian Auction

The Walrasian auction, named after its inventor Leon Walras, is a uniform priceauction auction in which all participants calculate their demand for a good at everypossible price and submit this to the auctioneer. The actual price is then determinedin such a way that the total demand among all participants is the same as the totalamount of the good on sale. In other words, the Walrasian auction will make surethat the demand always matches the supply by picking the right combination ofprice-quantity points among the participants. Walras suggests that this equilibriumcan be achieved through a process called tatonnement (French for “groping”), aform of hill climbing.

2.4 Desirable Market Properties

Before looking in more detail at how a commodity market can be designed, it isbeneficial to address market design from a more theoretical point of view. Morespecifically, expressing how an ideal commodity market might look like can providegood evaluation criteria for any real-world commodity market. Such an ideal marketshould conform to a set of principles (have a few key properties), of which the mostimportant are explained and discussed in this section.

2.4.1 Allocative efficiency

Allocative efficiency is a measure of profit or utility that results from a proposed oractual choice of a distribution of resources. In a market, the term allocative efficiencyindicates the received profit of the participants for a specific deal compared to themaximum profit they could receive for any deal. More concretely, the allocativeefficiency of a market gives an idea of how good a market is able to match offersof buyers and sellers with each other. Usually, the allocative efficiency is directlyinfluenced by the allocation and pricing rule (see section 2.5.3), specified by theused market mechanism, that is applied when a deal occurs. It is difficult to makemore specific statements about the effect of pricing rule on the allocative efficiency,as this is really market specific.

Allocative efficiency is usually less of an issue in situations where there is onlya single seller (or buyer), since the maximum allocative efficient deal is with thebuyer willing to pay the most (seller asking the least). In reality, many such marketmechanisms exist (English auction, Dutch auction, Reverse Auction). In situationswhere multiple buyers and sellers compete on the market, it is generally more dif-ficult to find the match that maximizes the allocative efficiency due to algorithmiccomplexity, although some mechanisms - such as the previously discussed DoubleAuction - exist. Note that for the maximum allocatively efficient match to be found,


it is imperative that both buyers and sellers submit prices that are truthful to theactual value they attach to the good they wish to sell or buy (a market is said to beincentive compatible if its matching mechanism encourages this behavior, see section2.4.3). Obviously, a market can only find the real maximal allocative efficient matchif it is given the real values participants attach to goods.

2.4.2 Individual Rationality

The individual rationality (IR) property is strongly related to rational choice theorywhich provides a framework for understanding and modeling social and economicbehavior. The core idea behind this framework is the assumption that the rationalbehavior of any individual in the current microeconomic model is to want morerather than less of a good. Although this theoretical assumption seems reasonable,there are many real-world situations in which it is rather difficult for an individualto actually obtain as many goods as possible. There can be many reasons for this.For example, it might be difficult for a buyer to find sellers willing to sell a specificgood (or the other way around). It is in such cases that participating in a marketthat trades the good makes sense to the buyer and seller. A market is said tohave the individual rationality property if joining the market is the most interestingchoice for all participants. That is, by taking part in the market each participantwill gain more than he would be able obtain on his own. As mentioned, this canbe because the market brings together buyers and sellers, but other reasons alsoexists. For example, if the market achieves a better (overall) allocative efficiencythan any other trading mechanism, joining the market is the obvious choice for allparticipants. An other example is a situation in which buyer or seller are unsureabout the valuation of the good they wish to buy or sell. In such a case, the marketmay provide price discovery mechanisms that allow the participants to discover howother participants valuate the goods on sale.

In the context of individual rationality, the concept of utility is often used toexpress whether joining a market is a logical choice for a buyer or seller. To makethe idea of utility more tangible, it is often expressed as the difference in value abuyer attaches to a good and the price he needs to pay to acquire the good: u = v−p.(Analogously for the seller: u = p−v). Joining the market is then individual rationalfor each participant i if ui > 0.

Note that this assumes that there is no other way to buy or sell the good, exceptby trading on the market. If there is another way to buy or sell the good that yieldsa higher utility ui for participant i, then trading the good via that other way thatis clearly the most rational choice for participant i. However, when stating thata market is individually rational for all participants, often the weaker requirementthat no participant incurs negative utility by taking part in the market is meant.

2.4.3 Incentive compatibility

A market is called incentive compatible (IC) if the combination of all its components,mechanisms and processes give the result that from a participant’s point of view, themost interesting strategy is to truthfully reveal private information. More concretely,


participants will only be able to maximize their own profit if they reveal their owninformation truthfully. In most cases, this information pertains to the participant’svaluation for the good. This means that participants cannot gain additional utilityby underbidding or overbidding. Note the subtlety in the fact that although inmany market mechanisms, under- or overbidding will change the outcome of a sale,it might not actually benefit the participant that did the false bidding. For example,if a buyer acquires a good by overbidding, he does not actually maximize his ownprofit as the good itself does not have the same value as the value the buyer attachesto the money he needed to buy the good (the buyer thus incurs negative utility).Similarly, if a buyer is not able to get hold of a good because of underbidding,opportunity loss occurs, as the buyer might have been able to increase his ownprofit by bidding truthfully and possibly acquiring the good that was on sale.

However, there are certainly examples of markets in which underbidding allowsparticipants to buy goods at prices that are lower than the value participants attachto the good, or overbidding allows participants to achieve the good at a price thatis within the participant’s valuation for the good. The first situation is the casein first-price open-outcry auctions, such as the English Auction, while the lattersituation might occur with auction mechanisms where the paid price is not the sameas the bid price, such as in a Uniform Price Auction. Such markets are obviouslynot incentive compatible.

Although incentive compatibility is a desirable property for any market, it isthus not easy to design a market that is incentive compatible, nor easy to verifythat a seemingly incentive compatible mechanism actually is.

2.4.4 Budget Balanced

A market mechanism or auction is said to be (weakly) budget balanced if the totalpayments from the buyers, less the revenues of the sellers and the needed transactioncosts is nonnegative. Put differently, a market mechanism is budget balanced if theauctioneer itself does not have to add extra money to the market in order for a marketclearing to occur. In other words, the payments of the buyers cover the revenues ofthe sellers (and any additional transaction costs). Having a budget balanced marketmechanism is clearly important as it is the main incentive for the auctioneer toorganize the auction or exchange of a good. In this context, the market mechanismis called weakly budget balanced if the auctioneer has a surplus after the matchingprocess (the auctioneer thus makes profit from the matching process), while themechanism is called strongly budget balanced (or just budget balanced) when thereis neither a surplus nor a deficit.

Although having a budget balanced mechanism is certainly very desirable, itturns out (as proven in [57]) that it is impossible to create a market mechanismthat is allocatively efficient, individually rational, incentive compatible and budgetbalanced at the same time. As such, it is clear that when designing any marketmechanism, certain concessions and choices will need to be made as it is impossible

2.5. DESIGNING A COMMODITY MARKET 47

to have a market mechanism that has all desirable properties. The fact that this isso emphasizes that there cannot be a single (or multiple) ‘ideal’ market mechanismor design that fits every situation. Instead, the choice of a certain mechanism isinherently dependent on the context in which it is used, as that context dictateswhich concessions are acceptable and which choices appropriate.

2.4.5 Computational Tractability

Today, many markets and exchanges make extensive use of electronic trading plat-forms for their speed, increased transparency and reliability. However, making useof such platforms has the obvious downside that they suffer from the shortcomingsof the current generation of computers. In particular, modern day computers areknown to be notoriously bad at non-deterministic problems (such as determiningwhich combination of offers is the most alloctively efficient). As such, it is impera-tive that during the design of any market, thought is given to whether the chosenmarket (auction) mechanism can efficiently be implemented using software.

A market mechanism is said to be computationally tractable if it can find matchesin polynomial time. Many auction mechanisms are inherently tractable as they onlyneed to compare new incoming bids against a single standing bid or reserve price.Examples include the English, Dutch, Vickrey and Reverse auctions. However, withDouble Auctions and more complex auctions such as the Uniform Price, Combina-torial and Walrasian Auctions this is not necessarily the case. As these auctionstypically need to compare multiple bids with each other and with a one or multipleasks, their time-complexity tends to grow exponentially. When the market mecha-nism is to be used in situations where a lot of simultaneous bids are expected andmatching needs to go fast, it might not be feasible to use those mechanisms with-out modification. The use of heuristics to find a pseudo-optimal match is in manycases a good alternative. Additionally, a parallelizable matching mechanism has theclear benefit that multiple servers can be used to work on the matching problemsimultaneously.

2.5 Designing a Commodity Market

After having discussed financial markets, auction mechanisms and desirable marketproperties, it is now time to have a closer look at how commodity exchange marketsare typically designed. This section will explore this topic in more detail by dis-cussing the various components and concepts that are typically part of a commodityexchange market.

2.5.1 Market actors

In any (commodity) market, there are several different market actors. With theterm market actor, any party that interacts with the market is meant. Sometimesthe word market player is also used to instead of market actor. The most prominent


of these actors are the buyers and sellers that trade goods on the exchange. Buyersare also referred to as consumers, while sellers are also called providers. Althoughboth terms can be used interchangeably, the terms ‘consumer’ and ‘provider’ laymore weight on the consumption aspects of the buyers and sellers. Note that it doesnot necessarily need to be the case that the seller is the same party that actuallyprovides the good that is being sold, nor does the buyer needs to be the party thatactually consumes the acquired good. Throughout the remainder of this thesis, theterms ‘buyer’ and ‘seller’ might occasionally be mentioned, but most of the time theterms ‘consumer’ and ‘provider’ will be used.

The market or exchange is most of the time also considered as a market actor.If the exchange is auction-based, it is sometimes also referred to as the auctioneer.Again, both terms can be used interchangeably. Throughout this thesis, the words‘market’, ‘market mechanism’, ‘auction’, ‘auction mechanism’ and ‘exchange’ arealso used to denote the same entity. For clarity; this entity is the component thataccepts offers from consumers and providers and finds matches between them. Ad-ditionally, this component may provide extra functionality such as price discoverymechanisms (section 2.5.4) and contract enforcement (section 2.5.6).

The role of brokersAlthough consumers and providers regularly take part in the market themselves, inmany situations they use also brokers to participate in their place. In this contextbrokers are also called bidding-agents, as they place bids on behalf of providersand consumers. The use of such bidding-agents can have many benefits for bothproviders and consumers. Such a key benefit is that brokers can hide some (possiblyall) of the complexity of the market by translating specific requests and constraintsof consumers and providers to the market’s - possibly complex - bidding language(section 2.5.2). Additionally, brokers can hide much of the different steps thattypically occur in a bidding process and only present the end-result to the consumeror provider. Such a service can be especially helpful to consumers and providers whenelectronic trading platforms are used that require complex hardware and softwaresetups to interact with. Examples include brokers that hide all complexity behindsimple user interfaces or implement certain bidding strategies in order to maximizeprofit.

Another more general role of bidding-agents can be that of the retailer thateffectively hides the market from the consumer or provider. That is, a retailer sellsgoods to consumers or buys goods from a provider using a custom price model (fixedprice per good, fixed part and variable price, etc.) that is apparently independentfrom what is going on the market itself. Doing so is advantageous to the customer(either consumer or provider) as using such a retailer effectively allows him to protecthimself against price fluctuations in the market.

Retailers make profit by leveraging the mid- to long-term differences betweenthe actual market price at which they buy or sells goods and the price they offer totheir customers. Note that the broker-retailer principle presented here is essentially


the same as in the retailer model used in the power industry as discussed in theintroduction (section 1.3.2).

Besides their roles as bidding-agents or retailers, brokers can also perform func-tions on behalf of the market itself. For example, in order to reduce the complexityof the market design and implementation, as well as to give market participantsthe freedom to use multiple alternatives, the market may outsource functions suchas price discovery, contract enforcement and accounting to external broker parties.Doing so has the advantage that the market becomes more self-contained and au-tonomous as certain responsibilities are no longer part of it. In the case of electronictrading platforms, this has the additional advantage that the platform becomes moremodular and the risk of introducing single points of failure is reduced.

Figure 2.2 shows an overview of how different market actors interact.

Figure 2.2: Market actors in an commodity market

IaaS cloud market actorsWhen considering the IaaS cloud market in particular, the following actors can beidentified.

Provider In the IaaS market, the IaaS providers are the organizations offering IaaSresources for sale on the market. These will typically be virtualized instance-like resources oriented towards computational use-cases. Other products suchas non-virtualized hardware and software, firewall or load balancing services


or persistent storage solutions can also be part of the products offered bythe providers. However, providers that charge similar to IaaS providers (e.g.per compute hour) but only offer more advanced platform services such asspecific database or programming framework deployments are not consideredIaaS providers, as these kind of services are clearly part of the PaaS spectrum.

Examples of IaaS providers are specialized companies such as Amazon, GoGridor Rackspace, but also non-specialized companies, research centers or univer-sities that want to earn some money by selling their excess computational orstorage capacity. In the future, groups of end-users might also want to takepart in the market as a provider by selling the idle resources of their desktopcomputers.

Consumer In the IaaS market, the consumer is the party that wants to lease com-putational resources for a certain time. Possible use-cases for these resourcesinclude hosting webapplications, performing (scientific) calculations (so-calledHigh Performance Computing (HPC) [58]), deploying development and testenvironments and many others. Note that the role of the consumer can alsobe taken by an IaaS provider that wishes to buy extra capacity from otherproviders to cope with temporary capacity shortages. Such inter-providertrading was already mentioned in the discussion of spot markets in section2.2.6.

Market (exchange) In the IaaS cloud context, the role of the exchange is to matchIaaS resources offered by IaaS providers with requests for certain resources asdemanded by the consumers.

Broker The role of brokers in the IaaS market is very similar to that of the brokersas depicted in Figure 2.2; they provide services that simplify market inter-action for the consumers and providers. These include providing easy-to-usefront-ends, implementation of bidding strategies, protection against rapid pricefluctuations and others. Additionally, brokers can be used to provide marketrelated services such price discovery or any of the auxiliary services describedin section 2.5.6.

2.5.2 Bidding language

Throughout the previous sections, it was already mentioned that the bidding lan-guage is a core component of each market. As the word suggests, the biddinglanguage of a market defines the language or protocol that is to be used to specifyand describe the contents of a bid. In order to avoid confusion, the term ask is com-monly used to denote a bid that is submitted by a provider as the provider is askingto sell a certain item. A bid by a consumer keeps the term bid as the consumeractually bids a price to buy something. Note that these terms were first mentionedin the context of the double auction (section 2.3.5), but they can easily also be usedhere in a more general context.


The main goal of specifying a clear bidding language is to avoid confusion ordiscussion during and after trading. That is, if trading is to occur, each party thatis involved must have a clear understanding of what other parties are offering, as wellas understand under which conditions the offers are valid. Moreover, in exchangemarkets, having a clear bidding language is an absolute requirement as asks andbids can only be compared and matched to each other if they are specified in thesame language. In this light, the importance of a formally defined bidding languagehas only increased with the introduction of electronic trading platforms.

Although it is difficult to make general statements about bidding languages with-out ruling out some specific use-cases, there are some similarities between the biddinglanguages of most commodity markets. That is, a typical offer (both bid and ask)contains at least a volume, a price and an expiration date. To represent such offersmore concisely, a tuple-based representation such as [volume, price, bid] can beused. In this notion, the tuple [5, 129, 12/07/11 13:14:28 GMT-2] thus repre-sents an offer with a volume of 5 and a price of 129, which expires on the 12thof July 2011 at the specified time. If this offer would be an ask (thus made by aprovider), the volume would indicate the number of units of a good the providerwishes to sell, while the price is the price at which the provider would wish to sellthe specified volume. The expiration date indicates until when the ask is valid (themarket should no longer match this ask after it has expired). Similarly, when theoffer would be a bid (thus made by a consumer), the volume would indicate thenumber of units the consumer wishes to buy, while the price indicates the price atwhich the consumer would be willing to buy the specified volume.

Note that up until now, it has been assumed that an offer cannot be split,meaning that the volume specified in the bid or ask needs to be traded at onceat the specified price or not at all. However, many commodity markets allow forso-called partial matches in which only a part of the original offer is matched, bysplitting offers into several sub offers. In such a case, the price that is part of theoffer specifies the unit price instead of the price of the whole volume. Of course,variations in which multiple prices are given (one for the whole volume, one for asingle unit, one for x units, etc) can also be thought of. In such cases, the biddinglanguage will need to be adjusted to accommodate this functionality.

Importance of the unit definition in a commodity marketStrongly related to the specification of the bidding language is the specification ofthe good under sale itself. An important aspect in this context is the specificationof the quantity of a good that is defined as a single unit. Doing so is an absoluterequirement as without it, trading of a certain volume of the good has no mean-ing at all as different actors might use different unit sizes. In existing commoditymarkets, these good units are therefore standardized. Examples include the oil bar-rel (158.9873 liter), the ounce (when trading gold) and the wheat bushel (althoughtoday replaced by standard weights).


Besides determining the quantity the unit represents, there should also be adescription of the properties and characteristics of the single unit of trade as in manycases there exists substantial qualitative differences between different units. Forexample, in the oil industry, a barrel of oil is categorized according to its crudeness5.Similar unit categorization exist for almost all commodities.

Although similar units of different categories are in many cases traded on differentmarkets (as is the case with oil), it is also possible to trade these units on the samemarket, as long as the bidding language is expressive enough to denote the differencesbetween the units of different categories.

Without going into to much detail for now, it is apparent that in the context ofcloud computing, the definition of the IaaS unit will be far from straightforward asthe goods that are being traded on such a market will be a lot less homogeneousthan the goods that are traded on more classic commodity markets. Indeed, whencomparing different offers of different IaaS providers (see section 1.2), it is clear thatthere exists considerable differences in both the resources that are being offered, aswell as in the way these resources are described. Additionally, extra information suchas the price for certain resources, the time that resources need to be allocated, extraquality of service constraints or the time a certain offer is valid, should also be easilydescribable. As such, a vital step in the design of any market for IaaS resources isthe specification of a clear bidding language that is able to express both the inheritresource heterogeneity as well as any additional constraints and properties of anoffer. In the context of this thesis, this topic is revisited in detail in section 3.1.

2.5.3 Market mechanism

In terms of choices that need to be made when designing an exchange market, thechoice of the market mechanism is certainly one of the most important ones as iteffectively determines how consumers and providers will trade with each other. Ineffect, the market mechanism is defined by the specification of two important rules:the so-called allocation and pricing rule.

Allocation ruleThe allocation rule (also called the matching or clearing rule or mechanism) de-termines which goods are allocated to which consumer or provider. Although notstrictly necessarily, in many cases, the allocation mechanism is chosen to be a vari-ation of one of the auction types that were discussed in section 2.3. The benefits ofchoosing from among these is that all of these auctions have been studied extensivelyand consequently their benefits and shortcomings are well-known. The exact choicebetween the different auctions is dependent on several factors. A very importantfactor is the relation between the different market actors. In the commodity marketcontext, where there are typically both many consumers and providers and wherethe auctioneer is a priori not biased to one of both sides, the (continuous) doubleauction is a logical choice as it is designed to accept multiple bids and asks and

5A measure indicating how much processing the oil has been through. The term crude oil denotesraw, unprocessed oil.


match them without favoring either consumers or providers.

Given that the auction mechanism determines when and how consumers andproviders can place offers, it is logical that it has a strong influence on the entirebidding process, and consequently the bidding strategy that is used by consumersand providers. Likewise, the auction mechanism has a large influence on the biddinglanguage as depending on this mechanism, the bidding language should to be ableto express the particularities of the bidding process. For example, when a DutchAuction is used (section 2.3.2), the bidding language only needs to be able to expressan ‘accept’, while in double auctions, the bidding language is far more complex as italso needs to be able to express what the consumer or provider is exactly offering.

Pricing ruleThe pricing rule is closely related to the allocation rule as it determines the pricethat is to be paid by the buyer once an allocation is found. Sometimes, this pricingrule is entirely determined by the allocation rule itself (as is the case in second-price auctions), but with other allocation rules, it is possible to use different pricingrules with the same allocation rule. For example, in double auctions many differentpricing rules can be used. The rather straightforward rule is to let the consumerpay the provider the price he actually bid, while another rule might be to let theconsumer only pay the price that was asked by the provider. The first choice favorsthe provider as he effectively receives all surplus that the consumer was willing topay above the provider’s requested price. The second choice than does the opposite;it favors the consumer by not letting him pay for the surplus he was willing to payfor the good. Note that this second pricing rule requires sealed bids, as makingbids public will heavily affect the consumers’ bidding strategy. That is, if consumersknow that they only need to pay for the price that is requested by the provider, theywill start overbidding to acquire a good knowing that they will not need to pay theprice they are actually bidding.

More complex pricing rules also exist. For example, some auctions use a pricingrule that lets consumer and providers share the surplus. In such a case, the clearingprice p that needs to be paid by the consumer is equal to (a + b)/2 where a is theprovider’s ask price and b the consumer’s bid price. A generalization of this rule,the so-called k-pricing rule parameterizes this rule with a parameter k by settingthe clearing price p to kb + (1 − k)a. Clearly, the basic case (all surplus is equallysplit between both parties) is given by setting k = 0.5, while any k > 0.5 favors theseller and any k < 0.5 favors the buyer.

The choice of the pricing rule has a large influence on both the bidding strategyof the participants and the allocative efficiency of the market. More informationabout the pricing rule in double auctions can be found in [59].

IaaS cloud market mechanismGiven the similarities between the power industry and IaaS clouds as indicated inthe introductory chapter, trying to design an IaaS cloud market using a double


auction seems to be a valid starting point, as it is the primary market mechanismfor electricity trading. However, given the fact that the traded goods (services)of the IaaS market are totally different than that of the power industry markets,it should not come as a surprise that it will turn out that using a classic doubleauction will not suffice in the IaaS cloud scenario. In fact, due to the potentialinfinite number of combinations between different hardware resources and serviceconstraints as well as pricing model heterogeneity among providers, it will turn outthat using a double auction is just not feasible at all in the current IaaS marketwithout leaving out a large number of use cases. The next chapter will elaborate onthis, and will put forward a new market mechanism that aims to solve some of theseproblems. Specifically, section 3.2 will explain the problems with the (continuous)double auction in the IaaS cloud setting, while section 3.3 introduces the new auctionmechanism that attempts to solve these issues.

2.5.4 Price discovery

The price discovery process describes the mechanisms that are used by buyers andsellers to determine or approximate the price of a good in the marketplace. Pricediscovery is often associated with asset valuation, which denotes the mechanismsused by the buyer or seller to determine the value they attach to the good (asset)for sale. Note the subtle difference between the two: the term price discovery denotesthe bi-directional process in which buyers and sellers arrive at a transaction price fora specific item at a given time, while asset valuation is a process that every buyeror seller does for themselves. Obviously, both processes are closely related to eachother as the price at which a trade occurs is heavily influenced by the value thatboth parties attach to the good for sale. Because of this tight coupling betweenprice discovery and asset valuation, the terms are frequently used interchangeablyin literature. In this thesis however, the term ‘price discovery’ will exclusively beused. The context in which the term is used should make its exact meaning clear.

The use of price discovery mechanisms has two distinct advantages. First of all,it allows new buyers or sellers to discover the price at which the market is currentlyclearing offers. For example, in a double auction, a good price discovery mechanismallows consumers and providers to discover a price at which they are likely to buyor sell goods. This can for example be the average price at which units are currentlybeing traded (in which case bidding a price that is higher than the average priceincreases the buyer’s chances of actually making a trade, while lowering the pricedecreases that chance; the opposite is true from the seller’s point of view).

The ability to do price discovery is also attractive because it allows consumersto make an upfront trade-off between the price they will need to pay on the marketand the resources and quality of service they will receive for that price. Providerson the other hand can use price discovery techniques to get an idea of the type ofresources and quality of service they will need to deliver for a certain price. Thisability to make trade-offs without having to make commitments effectively leads tothe second advantage of price discovery: it allows existing participants to determine


a bidding strategy that optimizes their own profit or utility by choosing the momentat which they buy or sell certain resources. Such a bidding strategy can be tailoredto short term periods, as well as medium to long term periods. For example, a shortterm bidding strategy might describe the bidding behavior needed to quickly obtaina large amount of a good for the lowest price possible, while a medium or long termbidding strategy might involve deciding a which point in time to buy certain goodsby carefully monitoring the prices of past trades, and looking for patterns. A goodexample of such a pattern can be found in electricity markets, in which the pricefor a Kilowatt/hour is usually lower at night when general power usage is lower (seeFigure 2.1 in section 2.2.6). Consequently, it is cheaper to run non-critical operationsthat consume substantial amounts of electricity at night than during the daytime6.

When considering price discovery in an IaaS cloud context, it is clear that thesame remarks hold. Both the ability to get an idea of the price at which computa-tional resources are traded for new providers and consumers as well as the possibilityfor existing providers and consumers to make better provisioning decisions are valu-able tools in the cloud computing setting. Note that this process of making goodprovisioning decisions is however more complex in the IaaS cloud context, as opti-mality is no longer only determined by the choice of provider or consumer and time,but also by the choice of the resources itself. That is, by choosing different typesof resources (that is, different hardware or software components), different levels ofperformance for different applications will be obtained. Additionally, by allocatingthe same resources via different allocation models (such as on-demand, spot, futureor reserved - detailed in section 2.5.5), the price, performance and behavior for thesame set of resources during the same time period can be totally different. As such,far more complex methods and heuristics will be needed to intelligently make theaforementioned trade-off between price and received or delivered quality of service.The discussion of such techniques is however beyond the scope of this thesis, andwill as such not be further considered.

The issue of price discovery itself is however an important topic in the contextof this thesis. However, due to the heterogeneity of the resources that are beingtraded, giving a good indication of what certain computational resources or serviceswill actually cost at a given time is far from trivial and closely related to the marketmechanism and bidding language that are being used. Because of this, it is hardto make more explicit statements about price discovery in an IaaS market withoutfirst having a closer look at its matching mechanism and bidding language. As such,exact considerations about the price discovery mechanism in the IaaS cloud marketare postponed until section 3.3.3, after the discussion of these other components.

6Given that electricity providers pass this price reduction to their consumers, as is the case withthe well-known night-tariffs.


2.5.5 Market Settlement

When designing a commodity market, it is not only important to clearly indicatewhich good is being traded by giving a good unit definition that both describes thequality as the quantity of a single unit; it is also important to make a decision onthe delivery time of the goods that are being traded on the market (the so-calledsettlement of the trade). In this light, the choice between a futures market (seesection 2.2.5), in which goods need to be delivered at some specified point in thefuture, and a spot market (see section 2.2.6), in which goods are to be delivereddirectly, needs to be made.

Although the choice of whether a market is a spot or futures market foremostdetermines which providers and consumers will take part in the market, it is possiblethat this choice also has a profound impact on other parts of the market such as thematching mechanism. For example, in immediate spot markets it is important thatmatches are found quickly as delivery needs to take place as fast as possible whilein a futures market, it is acceptable if the matching process takes a lot longer. Assuch, it is clear that in some spot markets, finding matches quickly has priority overfinding the best matches, while the opposite is generally true in futures markets.

As the choice of the matching mechanism has a profound impact on almost everyother part of the market (such as the bidding language, bidding process, actors’strategies and price discovery mechanism), it is clear that the choice of the deliverytime of the market is an important one.

In the IaaS cloud market, the situation is however a bit different. That is, as thegoods that are being traded are very heterogeneous in nature, the delivery time ofthese goods can easily be considered as yet another characteristic that differentiatesit from other goods. When studying the different offers of IaaS providers, it canbe observed that this is exactly what is done in practice: different IaaS providersoffer computational resources to their customers and allow them to allocate theseresources using a variety of different models such as on-demand, spot or reserved.When choosing a different allocation model, customers get the exact same resources,but get different guarantees in terms of price and resource availability.

Because delivery time is considered as any other property of an IaaS offer, itis possible to introduce a single market in which IaaS resources with different al-location models are all traded together. Note that this is generally not done inother commodity markets as delivery time is usually one of the few characteristicson which goods do differentiate. Because of this, it is usually decided to create sep-arate markets in order to keep the market, market mechanism and bidding languagesimple. In the IaaS market however, heterogeneity between goods is a core aspectthat cannot be ignored, making it easier to incorporate the aspect of delivery time


into the market itself7.

IaaS resource allocation modelsThe terms on-demand, futures/reserved and spot instance will further be used toindicate the resource (instance) behavior and price models similar to those specifiedby Amazon. In a more generic form, these different models are defined as follows.

On-demand When using the on-demand model, resources are available immedi-ately and can be used by a consumer for a non-specified amount of time.Consumers are billed at a fixed rate per time-unit (hour, minute, ...). Figure2.3 shows a timeline of how the on-demand model works.

Figure 2.3: On-demand resources, resources are allocated at a time t and stay allo-cated for an indefinite length of time (until deallocated by the consumer).

Futures When using the futures model, resources are available for a specified pointin the future and for a specified amount of time. Consumers pay an up-frontfixed price. Figure 2.4 shows a timeline of how the futures model works.

Figure 2.4: Futures market timeline, resources are allocated from a time t + x untilt + y.

Reserved The reserved pricing model offered by many IaaS providers is a specialfutures model that borrows principles from the on-demand model. Specifically,in the reserved price model an upfront fixed price is paid to make sure resourcesare available in a certain period, and additionally a (lowered) price is paid pertime-unit the resources are actually used. Figure 2.5 shows a timeline of howthe reserved model works.

Figure 2.5: Reserved market timeline, resources are reserved from time t until t + yand allocated at various intervals (chosen by the consumer) within the specifiedboundaries.

7Note that this does not mean that IaaS resources with different delivery times must be tradedon a single market, it is just easier to do so in the context of IaaS markets.


Spot When using the spot model, the model as introduced by Amazon is used.That is, consumers get access to resources immediately and specify a maximumspot price. Every time-unit (hour, minute), the spot price is changed by theprovider. The consumer always pays the spot price for that time unit. As longas the spot price is lower or equal than the maximum spot price specified by theconsumer, the resources will remain available. When the spot price is higherthan the spot price, the consumer should be either notified or the resourcesshould be released immediately (this can be a provider specific policy). Figure2.6 shows a timeline of how the spot model works.

Figure 2.6: Spot market timeline, resources are allocated from a time t and stayallocated until t+ 3 when the spot price of the provider exceeds the maximum priceof the bid.

2.5.6 Auxiliary services

After the market has found a match between a consumer and the provider anddetermined the price that needs to be paid by the consumer, its task is usuallyover. However, this does not mean that the actual trade between the consumerand provider has taken place. To facilitate the actual trade between both parties,several mutually trusted parties that can deliver the necessarily auxiliary servicesare typically used. Listed here are some of these auxiliary services.

Money transfer service Service that allows two parties that do not entirely trusteach other to exchange money. Note that the function is different as that ofa bank as the service knows the details of the deal made by the market, andcan thus verify that the consumer has paid the right amount of money at theright time. Additionally, the service can cope with different currencies.

Good delivery service Service that delivers the goods to the consumer at a pre-determined time and place. Service is either paid for by the consumer ofprovider, depending on market rules.

Contract enforcement service Service that verifies whether the quality of thedelivered good is adequate (e.g. purity of metal, crudeness of oil) and whetherthe good was delivered in time and on the right location. It is quite commonthat a mutually trusted third party (such as an government official or bailiff)is used to do the actual verification.

In the IaaS cloud context, the issue of contract enforcement is particularlychallenging, as it involves verifying that a provider actually supplies resourcesthat deliver the promised quality of service. Verifying this is not easy, as to do


this accurately, a trusted third party must be able to run a potentially largenumber of tests and benchmarks at regular randomized intervals in such a waythat the results cannot be influenced by the consumer or provider. Puttingthis into practice this is a rather difficult administrative and technical challengethat requires advanced monitoring and analysis tools. Today, some companiessuch CloudHarmony [16] and CloudSleuth [60] already provide rather detailedanalyses of the performance of well-known IaaS providers; integrating suchtools as broker services into the market has significant value and can be thesubject of future research.

Authentication and Authorization service Service that provides advanced andsecure user management. When making use of electronic trading platforms,security and trust is always an important topic. The correct and secure fa-cilitation of authentication and authorization services is however a complexissue, especially when many different parties and organizations with differentsecurity requirements and implementations interact with each other. As such,outsourcing such services to third parties can take away a lot of this burdenfrom the market.

CHAPTER 3

IaaS Cloud Computing Market: The Continuous Reverse Auction

After having discussed the importance of introducing a market for IaaS resourcesin the first chapter, as well as given an overview of how such a market should bebuilt in the second chapter, it is now time to elaborate on the most critical partsof the market, namely its bidding language and matching (auction) mechanism.This chapter will start with the introduction of a Continuous Double Auction basedmarket for grid resources that was developed by Nicolas Dube in [39]. Once thismarket model has been properly introduced, the author will point out some problemsthis model faces in the cloud computing setting. Working from the particularities ofthese problems, a new bidding language and market mechanism that is able to copewith the specific issues of the IaaS ecosystem will be iteratively constructed.

The ratio behind the choice to start from an existing CDA model is twofold.First of all, given the resemblances between the power industry and IaaS clouds asdiscussed in section 1.3.1, using a double auction based market is a reasonable firstchoice as this market mechanism is already being used in power markets. Indeed,without studying the problem in more detail, it seems that once the appropriatebidding language is introduced that can express the heterogeneity of IaaS resources,the CDA mechanism is a viable market mechanism as it is able to quickly matchbids and asks of multiple providers and consumers.

A second reason to start from an existing market model is the fact that a marketdesign is something that needs to grow steadily and is not something that justsprouts into the mind; this chapter starts from an existing solution before deviatingfrom it to reflect the iterative process the author went through while designing themarket model detailed throughout this chapter.

By the end of this chapter, a bidding language for cloud computing resources

3.1. CONTINUOUS DOUBLE AUCTION FOR IAAS RESOURCES 61

will be proposed that uses so-called tag sets to describe resources and constraint setsto specify consumer and provider constraints on these resources. Additionally, anew auction mechanism called the Continuous Reverse Auction is introduced whichattempts to solve the problems of the Continuous Double Auction in the cloudsetting.

3.1 Continuous Double Auction for IaaS Resources

Because of the many resemblances between the IaaS industry and power industry,looking at the market mechanisms used in the power industry is a logical first step.As discussed before, the Continuous Double Auction is used in multiple countriesas the primary market mechanism to do spot trading for electricity. Indeed, theCDA market model has already earned its name as a fast (both in terms of marketclearing speed, as in algorithmic tractability) and reliable mechanism. As such, it isinteresting to have a look at how the CDA mechanism could used as the matchingcomponent of an IaaS market.

3.1.1 Dube’s CDA-based market

To study a CDA-based matching mechanism for IaaS resources, the PhD thesis ofNicolas Dube [39] acts as a good starting point as it introduces such a market forgrid systems. In particular, Dube’s thesis contains some good ideas about how torepresent computer resources (using so-called resource sets), how to match bids andasks of consumers and providers, and makes an interesting contribution on how tointroduce a price discovery mechanism for such a market model1.

Dube’s market exists of a central market component that accepts bids fromconsumers and asks from providers. As with every CDA, the market maintains twoqueues, one for the bids and one for the asks. The queue containing the bids issorted by decreasing price, the queue containing the asks is sorted by increasingprice. When a new bid or ask comes in that cannot be matched against an existingoffer it is added to its according queue. A bid is matched against an ask if theresource set attached to the bid that describes the requested computing resourcesmatches that of the ask, and if the price of the bid is higher or equal to that of theask. Similarly, an ask is matched against an existing (not previously matched) bidif the resource set attached to the ask fulfills the bid’s resource requirements andthe price attached to the ask is lower than that of the bid.

The actual matching algorithm for an incoming bid works by iterating over thequeue containing the asks and trying to match each of the asks in the queue to thenew bid. Once a match is found, the mechanism removes the matching ask from

1Dube’s price discovery mechanism will not be discussed in detail here; more information aboutit can be found in [39].


its queue and notifies both parties (consumer and provider) that a match has beenfound. Matching an incoming ask with an existing bid works similarly.

Although it might seem that so far, there are no apparent differences betweenthe classic continuous double auction as used in the power industry and Dube’sversion that was just described, the introduction of the resource sets do introduce asignificant change to the model. That is, rather than only comparing bids an asksbased on price, they are now also compared using the resource sets with which theyare both annotated. This adds algorithmic complexity to the matching process.The key point is however, that in effect, Dube has extended the double auctionmechanism so that it can cope with very heterogeneous commodities. It is clearthat this change can have a significant impact on the characteristics of the doubleauction, as optimal matching is no longer only determined by price, but also by theoptimality of resource set matching. To better understand how this change altersthe matching mechanism, the following section describes the resource sets in moredetail, by explaining how they are actually used to describe the different componentsof a computational resource.

3.1.2 Resource sets in detail

A resource set is (as the words suggest) a description of computer resources that iseither requested by the consumer in case of a bid or offered by the provider in caseof an ask. When considering a set of resources that is requested by the consumer,Dube uses the term requirement set, while the term component set is used to denoteresources that are offered by a provider. Without going into to much detail, it isintuitively clear that the resource set S in 3.1 represents a computer which has anx86 system architecture, 2 core processor clocked at 2.4 Ghz, 4GB of RAM and16GB of hard disk space.

S = {x86proc arch, 2proc cores, 2.4proc clock, 4mem size, 16disk size} (3.1)

Obviously, the used unit as well as the type and domain of each of the itemsin the set should be clearly defined. Dube does this by specifying a discrete, finitedomain set for each of the types of resources, along with the used unit (if applicable).Some examples of such domain sets can be seen in 3.2.


Φproc arch = {x86, itanium, intel, amd,power,bleugene, sparc, nec}

Φproc clock = {1.6, 1.8, 2.0, 2.2, 2.4, 2.6, 2.8, 3.0, 3.2, 3.4, 3.6, 3.8, 4.0} (Ghz)

Φproc cores = {1, 2, 4, 6, 8, 16, 32}Φproc L2 = {1, 2, 4, 6, 8, 12, 16} (MB)

Φmem size = {1, 2, 4, 6, 8, 16, 32, 64, 128, 256} (GB)

Φdisk size = {1, 2, 4, 6, 8, 16, 32, 64, 128, 256, 512, 768, 1024} (GB)

Φnet tech = {100T,GigE, 10GigE, IB SDR, IB DDR,Myrinet,NumaLink4}Φnet bw = {10, 100, 1000, 2000} (MB/s)

(3.2)

Although resource sets contain hardware requirements or components most ofthe time, they can contain software (such as operating systems, software libraries) orbenchmarks results as well. Again, it is clear that the resource set S in 3.3 representsa computer with 4 GB of ram, a 32 GB harddisk, a processor capable of reaching12 GFlops as measured by the Linpack benchmark [61], and specific versions of theGlobus [62] toolkit software package and Infinipath driver installed2.

S = {4mem size, 32disk size, 12proc linpack,GLOBUS 4.0.6soft package,

InfiniPath 2.1soft driver} (3.3)

The domains of these software packages and benchmarks also need to be defined,as is done in 3.4.

Φsoft package = {BLAST 2.2.14,GROMACS 3.3.3,FLUENT 6.3,GLOBUS 4.0.6}

Φsoft driver = {OFED 1.2.5, InfiniPath 2.1}Φproc linpack = {4, 8, 12, 16, 20, 24, 32, 40, 48, 64, 96, 128} (GFlops)

Φproc specint = {20, 40, 60, 80, 100, 120, 140, 160, 180, 200} (2006, rate, base)

(3.4)

Matching resource setsTheoretically, Dube states that a match between a requirement set and a compo-nent set occurs if the intersection between the two results in the same set as therequirement set, as indicated by 3.5. In this equation, Rb represents a requirementset for a bid b and Co represents the component set for an offer (ask) o.

Rb matches Co ⇐⇒ Co matches Rb ⇐⇒ Rb ∩ Co = Rb (3.5)

2The specified example is Grid Computing oriented as it is based on the examples given inDube’s thesis. However, it is clear that any software package, library or benchmark can be used.


In practice, an algorithmic approach is needed to determine whether two resourcesets actually match. That is, to check whether a given requirement set matches agiven component set, the requirement set is iterated, and for each of the items inthe requirement set an item of the same type (e.g. proc arch) in the component setis searched. If such an item exists, the two items are compared more closely to seeif they actually match. If no such item is present in the component set, the resultof the matching procedure is negative (there is no need to check the other items).

Although not explicitly mentioned in his thesis, Dube actually considers twodifferent types of item matching: exact matching and what can be called comparativematching. An exact match occurs when two resources have the exact same value(e.g.: [proc arch = x86]), a comparative match occurs when the resources havethe required relation (e.g.: [RAM = 1GB] < [RAM = 2GB]).

Intuitively, it is thus clear that 3.5 expresses that a requirement set matches acomponent set if all items in the requirement set have a matching item in componentset. Consequently, a component set matches a requirement set if it contains amatching item for each of the items in the requirement set. Note that a componentset can thus match a requirement set with less items in it, while requirement setscan never match component sets that have fewer items.

Although matching bids and asks by matching their respective resource setsas just described is computationally intensive (due to the possibly high numberof comparisons between item sets), the number of bids and asks that are actuallycompared with each other should be relatively low due to the continuous nature ofthe matching algorithm and the fact that the queues are sorted by price. The factthat the auction is continuous guarantees that a single offer is matched immediatelywhen it arrives, compared to a regular double auction where all bids are comparedto all asks at the end of the offer submission period. The use of the sorted queueson the other hand, ensures that once a match is found there is no need to search foran other match since the found match will be maximally efficient (as discussed in2.3.5).

Furthermore, the matching itself can be made more efficient by implementingthe item lookup using hash sets. Also, by comparing the sizes of the requirementset and component set prior to comparing their items, the result of the match isalready known (that is, negative) when the component set contains less items thanthe requirement set.

3.1.3 Using resource sets to describe IaaS resources

Given the similarity between grid resources and IaaS resources, it seems reasonableto try and use the resource sets as introduced in the previous section to describeIaaS resources. Indeed, as with grid resources, IaaS resources are typically describedin terms of their hard- and software components and can thus be expressed byresource sets, as shown in example 3.6 which shows a resource set description of asmall instance [15] as offered by Amazon on its EC2 platform.critical) patches in


the provider’s management software (e.g. VM management) are applied.

small instance = {1ECU, 1.7mem size, 160disk size, 32platform} (3.6)

However, although resource sets provide a viable way to specify many of thehardware and software characteristics of IaaS cloud resources; they lack the expres-siveness and flexibility that is needed to express some other more complex constraintsthat are inherently part of IaaS solutions. In what follows, these shortcomings of re-source sets will be outlined, followed by the introduction of a more generic constraintspecification language that can deal with these issues.

When comparing IaaS resources against grid resources, there are besides hard-ware and software constraints almost always additional constraints - such as price,QoS and security constraints - that need to be taken into account. The fact thatthis is the case is based on the observation that in the cloud computing setting theseadditional constraints are first citizens, while they are far less prevalent in the gridcomputing setting. While most existing compute grids still only offer publicly fundedbest-effort services that are oriented towards scientific workloads, cloud computingservices aim to deliver general purpose, metered usage solutions with guaranteedquality of service. This broader and more commercial view means that providersand consumers consider a far larger and more complex set of criteria when specifyinga particular resource. The use of these extra criteria and requirements will becomeeven more pervasive with the advent of the so-called cloud 2.0, which promises tooffer value beyond the provision of raw hardware by focusing on delivering high per-formance, highly available and secure computing infrastructure for business-criticalproduction applications [63].

In order to get a more concrete idea of some of these criteria, consider Table 3.1which expresses them as constraints.

Although Table 3.1 is by no means meant to be exhaustive - a lot of additionalconstraints can easily be thought off - it does contains some interesting examplesthat are difficult or even impossible to express using Dube’s resource sets.

For example, a constraint such as the Refund policy constraint will typicallyinclude very provider specific conditions under which a refund can occur, as well aswhat this refund actually includes. As such a constraint is typically very complex, itcannot be expressed by a single value, and consequently does not fit in the currentresource set model. Instead, a more elaborate model that allows resources andconstraints to be characterized by multiple values is needed to express such complexconstraints.

Other constraints, such as the Region and the Software version range constraintsare also difficult to express using Dube’s resource sets as the resource sets do notprovide the more elaborate matching semantics that are needed by those constraints.In particular, to match a Region constraint, knowledge of geographical location is


Constraint Description

Quality of Service (SLAs)

Uptime Provider guaranteed hardware uptime.Refund policies Policy used by the provider when SLAs are violated.Delivered performance Actual delivered performance. May include many pa-

rameters: CPU performance (flops), memory perfor-mance, disk performance.

Network performance Average and maximum access times (network delay),available up and downstream bandwidth.

Price

Hourly price Price that is paid per hour for the resources.Bandwidth price Price that is paid for the used bandwidth.Future price Stability Provider guarantees about price future stability.Historic price information Information about the price of the resource in the past.

Security

Resource access Supported methods to access resources (SSH, FTP,Remote Desktop, etc.).

Certification Security certificates held the provider (e.g. SAS70).Patch policy How fast (critical) patches in the provider’s manage-

ment software (e.g. VM management) are applied.

Misc

Region Geographical region in which the resources are lo-cated.

Allocation model Whether on-demand, future, reserved or spot re-sources are used.

Software version range Support for different versions of the same softwarepackage.

API Software API used to access and manage resources.Additional services Other optional services (firewall, load balancer, . . . )

Table 3.1: Examples of additional constraints that are regularly used when describ-ing IaaS resources.

needed to know which regions are included or overlap with others. This kind ofmatching is not supported by the resource sets. Similarly, the possibility to specifya range of versions of a particular software suite (by either listing the supportedversions or by specifying a min and max version) cannot be done by Dube’s “exactmatching” or “comparative matching” semantics.

This lack of expressiveness in the matching mechanism of the resource sets be-comes even more apparent when trying to combine different simple constraints intonew, more complex constraints. Such situations can for example occur when con-sumers are willing to make compromises if the compensation is large enough. That


is, it is not unthinkable that a consumer is willing to accept lower uptime guaran-tees if more interesting refund policies come with it. Similarly, customers might bewilling to pay higher prices if the resources they can acquire will be able to get thejob done faster. Expressing these kinds of constraints is just not possible using thematching semantics of resource sets. Instead, in the case of a CDA, several differentbids that express these subtle trade-offs in requirements should be submitted to themarket. Note that doing so is typically problematic as the consumer only wishes tomatch a single of all those bids. In other words, giving consumers a language thatis able to express complex constraints such as trade-offs between simple constraintshas substantial value as it gives them the ability to model their bids much closer totheir actual requirements, increasing the overall usefulness of the market.

Besides the fact that when using resource sets certain constraints cannot beexpressed, there is a potentially other important issue that is of a more practicalnature. That is, when using resource sets, a market is needed that has specificknowledge about each of the types of resources that are used as it needs to knowhow to match these resources, i.e. it needs to know the different matching semanticsof each of the resources. Although using such a mechanism was still feasible whenconsidering a fixed set of resources with fixed domain sets and only two types ofmatching (that is, exact matching and comparative “greater than” matching); in asituation where more and more complex constraints are used with complex matchingsemantics, using such a system is no longer viable. Indeed, given the heterogeneityof IaaS solutions today, using a matching mechanism in which the market must beaware of all possible resources and constraints is not manageable nor future proof.Instead, a bidding language is needed in which offers can be compared and matchedto each other without the market needing to have specific knowledge about eachof the constraints and resources that are part of the offers. The following sectionintroduces such a market-agnostic resource specification language by letting theconsumers and providers specify constraint matching semantics.

3.1.4 Introducing tag and constraint sets

To solve the previously stated problems, the concept of the resource set will bereplaced by the notions of tag sets and constraint sets, which enable the specificationof more complex constraints and remove the need for the market to have knowledge ofeach type of item in the resource sets. In particular, the specification of a constrainton a resource that a consumer is requesting or a provider is offering will be separatedfrom the specification of the values that characterize that resource. By doing so, theproperties that characterize these resources are decoupled from the semantics thatare used to match them with each other during the matching process. In other words,by letting the consumer and provider specify the properties of a resource, as wellas the semantics that determine how these resource are to be matched, the marketitself no longer needs to have knowledge about the matching semantics of each kindof resource. Additionally, by letting providers and consumer specify the matchingsemantics, the market is no longer constrained to a different few types of matching(as is the case in Dube’s model), but can instead support any kind of matching.


Finally, by splitting the values from the matching semantics, the specification ofcomplex constraints that can express constraints that span multiple resources (suchas the QoS or consumer trade-off constraints that were mentioned before) can beintroduced relatively easily.

When a consumer or provider wants to describe a certain resource, requirement,service or feature they can do that using tag sets that contain tags that describe thedifferent characteristics of the resource or feature that is being considered. Formally,a tag T in a tag set T S is defined as the tuple in 3.7. It contains a name and a setof attributes that define the tag.

T = [NT ,AT ] (3.7)

The name NT of the tag can be any regular string3, but has to be unique foreach type of tag and within a tag set (there can not be two tags with the same namein the same tag set). The attributes AT associated with the tag (see 3.8) is a set ofkey-value pairs.

AT = {(x, f(x))}f : x ∈ string 7→ value

(3.8)

These attributes will typically contain the values that further characterize agiven tag. For example, in the case of a Disk tag (NT =“Disk”), the tag attributeswill contain both the unit in which the hard disk size is specified (typically GB)as well as the actual size of the disk, i.e. [Disk, {(unit, GB), (value, 160)}].Similarly, an OperatingSystem tag may contain both the name as well as the versionof an operating system, as in [OperatingSystem, {(name, Windows), (version,

7)}]. The values of attributes do not have to be singular values, they can also be ofa compound nature. That is, it is also possible that these values are lists or nestedtags. For example, if multiple operating systems are supported by a provider, hecan list them all as follows.

[SupportedOperatingSystems, {(list, {

[OperatingSystem, {(name, Windows), (version, 7)}],

[OperatingSystem, {(name, Ubuntu), (version, 11.04)}]}

)}]

The most apparent difference between the tag sets as defined here and Dube’sresource sets is that using tag sets multiple properties (or attributes as they arecalled here) can be under grouped under a single tag, allowing for a more hierarchicalapproach to resource specification. A less visible difference is that tag sets in itselfhave no significance during matching, they are just values that describe an offer. The

3When implementing the tag, this can just be the string type provided by the programminglanguage of choice.


real matching semantics of tags are given by so-called constraint sets that accompanytag sets.

These constraint sets contain constraints that define a certain matching restric-tion on an offer. Formally, a constraint (3.9) is a function that defines such amatching restriction on an offer O1, by expressing restrictions between the tags ofO1 and the tags of an other offer O2 against which O1 is matched.

C : (T SO1 , T SO2) 7→ {true, false} (3.9)

Note that constraints are defined on an offer and not only on a bid, implyingthat constraints can also be defined on the tag sets that accompany a provider’sask. By allowing this, providers effectively get the possibility to set constraints onthe bids they are willing to match with. Although this might not seem importantat first, allowing providers to set constraints does allow them to refuse certain bidsbased on specific conditions such as e.g. the consumer’s country of origin4 or e.g.the lack of certain other information in the consumer’s tag set.

Given the generic definition in 3.9, it is clear that in practice the outcome ofa constraint function can be influenced by a large number of factors. To betterunderstand how constraints work in practice, consider a consumer specified memoryconstraint as given by Algorithm 1.

Algorithm 1 MemoryConstraint(T S1, T S2)T1 ← T S1[“Memory”];T2 ← T S2[“Memory”];if T1 6= null and T2 6= null then

if (AT1 [“unit”] = AT2 [“unit”]) and (AT1 [“value”] ≤ AT2 [“value”]) thenreturn true;

end ifend ifreturn false;

As already noted, a constraint function always takes two arguments: the tag set(T S1) that belongs to the same offer to which the constraint belongs and the tagset (T S2) that belongs to the offer against which the constraint is verified. TheMemory constraint works as follows. First, the Memory tag is looked for in both tagsets. In practice, this can either be done by iterating over both sets and comparingthe names of each tag, or by using a hash table like structure to store tag sets thatmaps tag names to tags, in which case a lookup can happen in a single step. If oneof both sets does not contain the Memory tag than the result of the constraint isnegative. If both tag sets do contain the Memory tag, the algorithm first compares

4Certain providers apply restrictions on the countries of origin of their consumers. Most notably,all US-based providers may not do business with consumers from a certain number of countries onwhich the US government has set a trading restrictions.


the “unit” attributes to verify whether both tags are specified in the same unit, andthen checks whether “value” specified in T1 is less than or equal to that of T2 (notethat the efficiency remark concerning hash table lookup of attributes also applieshere). If this is the case, the result of the function is positive (the constraint is thusfulfilled), in all other cases the result is negative.

In this example, the Memory constraint was used to define matching semantics.Besides the Memory constraint, many other constraints obviously exist. Some of theseoperations should be standardized by the market to avoid confusion among the mar-ket participants, while others can remain consumer- or provider-specific. Examplesof constraints that should be standardized are simple hardware-related constraintssuch as the Memory, Disk and CPUClock constraints. More complex standardizedconstraints may include the aforementioned Region and SoftwareVersions con-straints.

A good example of a situation where e.g. the consumer would want to specifya custom constraint is the previously given case where a trade-off between resourceuptime and refund policy is required. A conceptual overview of how such a complexconstraint can be defined is given by Algorithm 2.

Algorithm 2 UptimeRefundConstraint(T S1, T S2)Tu1 ← T S1[“uptime”];Tr1 ← T S1[“refund”];Tu2 ← T S2[“uptime”];Tr2 ← T S2[“refund”];if Tu1 6= null and Tr1 6= null and Tu2 6= null and Tr2 6= null thendelta← 0if Tu1[“value”] > Tu2[“value”] thendelta← (Tu1[“value”]− Tu2[“value”])

end ifif Tr2[“value”] ≥ (Tr1[“value”] + delta * Tr1[“compensation”]) then

return true;end if

end ifreturn false;

Now, although up until now it may have seemed that verifying whether theseconstraints are fulfilled or not is straightforward as it only involves applying the givenalgorithms, in practice actually verifying the constraints is a bit more difficult. Thisis primarily caused by the fact that in reality, the consumer and provider must havea way to transfer the algorithm that embodies the constraint to the market. Themost obvious way to do this, is by allowing consumers and providers to run customcode that can verify these constraints within the market. Despite the fact that theaction of running custom programming code on a remote location itself is relativelystraightforward using some software libraries, doing so introduces a whole new set


of technical challenges as this remote code must be verified to be safe prior to beingexecuted on the market. That is, in order to guarantee the market’s integrity, itmust be verified whether the consumer’s or provider’s code that verifies a constraintdoes not contain malicious code. While introducing a system that actually does thisis certainly possible, there is also a way to avoid the technical hassle of having toexamine the code of each incoming constraint.

By giving the market a database of standard constraints for which it knows thematching semantics and leaving the matching of non-standardized constraints to theconsumer and provider themselves, the technical difficulties that remote code intro-duces are effectively avoided. In practice, such a system would work by letting theconsumers and providers specify the name of the constraint as part of the constraintset when a standard constraint is used and specify the endpoint of a webservicethat can verify custom constraints in the other cases. Note that by doing this, themarket still remains agnostic of the matching semantics of resources themselves, asthe relation between a resource (tag) and its matching semantics is still determinedby the consumer or provider.

When elaborating on this idea of using a set of standard constraints for whichthe market knows the matching semantics, it is interesting to introduce a set ofmore generic constraints in order to maximize the number of constraints that canbe matched by the market itself. The idea being that such constraints should beable to verify whether the properties in any given tag in both the provider andconsumer set have a certain relation. For example, having a constraint such as theLessThan constraint as defined in Algorithm 3, allows consumers and providers toexpress that the bid or ask against which their bid or ask is matched should have aspecific tag and that the value of a specific attribute in that tag should be less thanthe value of the attribute with the same name in the same tag that accompaniedthe bid or ask that specified the constraint. Other generic constraints such as theGreaterThan, Equal, LessThanOrEqual and GreatThanOrEqual would have similarsemantics. Constraints such as the EqualAll constraint defined in Algorithm 4 couldsimilarly be used to express that two tags sets should both contain a certain tag andthat all attributes in the tag of the first set should have an attribute with the samename and value in the tag of the second set5.

To further indicate the usefulness of such constraints, consider the Resource con-straint in Algorithm 5 which is a generalization of the Memory constraint of Algorithm1 that can also be used to implement many other hardware related constraints (suchas Disk or CpuClock).

In order to express which tag is checked in a generic constraint, the notationConstraint[Parameters...] (as in Resource[Memory] or Equal[AllocationType,value]) will be used in the remainder of this thesis.

5Note that this is different from requiring that both tags have the exact same attributes, as thesecond tag might have more attributes than the first tag. Obviously, the market could also chooseto define the semantics to be so that both tags must have the exact same attributes.


Algorithm 3 LessThanConstraint(T S1, T S2, tagName, attributeName)

T1 ← T S1[tagName];T2 ← T S2[tagName];if T1 6= null and T2 6= null then

if AT1 [attributeName] < AT2 [attributeName] thenreturn true;


Algorithm 4 EqualAll(T S1, T S2, tagName)

T1 ← T S1[tagName];T2 ← T S2[tagName];if T1 = null or T2 = null then

return false;end iffor all (key, value) ∈ AT1 do

if value 6= AT2 [key] thenreturn false;

end ifend forreturn true;

Algorithm 5 ResourceConstraint(T S1, T S2, tagName)

T1 ← T S1[tagName];T2 ← T S2[tagName];if T1 6= null and T2 6= null then

if (AT1 [“unit”] = AT2 [“unit”]) and (AT1 [“value”] ≤ AT2 [“value”]) thenreturn true;


The technicalities of how this system with standard and custom constraints (us-ing web services) would work exactly go beyond the scope of this chapter, but willbe explained in detail in the next chapter, as part of the software architecture of theIaaS market.

3.1.5 The CDA using tag and constraint sets

Using the tag and constraint sets, it is now possible to introduce a CDA basedmarket, similar to Dube’s CDA described in section 3.1, that can cope with theconstraints as needed in an IaaS setting. Figure 3.1 gives a schematic overview ofhow such a market would work.


Figure 3.1: Overview of the Continuous Double Auction for IaaS Resources

From this figure it can be seen that the market process can be split in threedistinct phases. In the first phase, consumers submit bids and providers submitasks to the market. The bidding language that is used to describe a consumer’s bidand a provider’s ask is technically the same and can be represented as the following5-tuple.

[Volume, Price, ExpirationDate, TagSet, ConstraintSet]

The different components of this tuple should be interpreted according to whetherthey represent a bid or an ask, similar to the explanation given in section 2.5.2. Thatis, in case the tuple represents a bid, the Volume indicates the number of specifiedgoods the consumer is willing to buy, the Price indicates the price at which the con-sumer is willing to buy a single unit and the ExpirationDate indicates until whenthe consumer’s bid is valid. The Tagset contains the tags that describe the resourcesthe consumer is requesting, possibly complemented with some additional (personal)information about the consumer and the request as to increase the likelihood thatany provider imposed constraints can be fulfilled. The ConstraintSet contains theactual constraints the consumer is imposing on the tag set of the provider’s offer.

If the offer is an ask, the Volume indicates the number of specified units theprovider is willing to sell, the Price indicates the price at which the provideris willing to sell a single unit and the ExpirationDate indicates until when theprovider’s ask is valid. The Tagset contains the tags that describe the resources theprovider is offering as well as some information about the provider itself, while theConstraintSet contains the constraints the provider is imposing on the tag set ofthe consumer.

As noted before (in section 2.5.2), in order to determine which bids can bematched with which asks, it is important that the market knows whether asks and


bids with a Volume larger than 1 can be split into several sub bids with smallervolumes or not. Although this decision could be made market-wide (that is, adecision is made whether splitting offers is allowed or not, and this decision appliesto all offers), using the tagsets it is also possible to let consumers and providers decidefor themselves whether they allow their offer to be split. Concretely, this is doneby adding the [Splittable, (value, x)] tag, which specifies that the number ofunits of an offer that has to be matched at once must be a multiple of x. The factthat this kind of consumer and provider specific behavior can be incorporated intothe market without needing to change the bidding language again demonstrates theflexibility of the tag and constraint based approach.

The second phase that is shown in Figure 3.1, is the matching phase within themarket itself. For this, the sorted queue based matching mechanism that has beendescribed before is used. An incoming bid matches an existing ask in the priceascending ask queue if the price of the incoming bid is higher than that of the askand if the constraints in the constraint sets of both the bid and ask are fulfilled (asshown formally in 3.10 for a bid B and ask A).

∀ C ∈ CSA : C(T SA, T SB) = true ∧ ∀ C ∈ CSB : C(T SB, T SA) = true (3.10)

If a match is found, the third phase of the market process is entered in whichthe consumer and provider are notified of the found match. If no match is found,the consumer’s bid is added to the bid queue that is sorted according to decreasingprice. The process for an incoming ask is completely analogous. If no match canbe found before the expiration date of an offer, it is automatically removed fromits queue. Whether the consumer or provider that submitted the offer is notified ofthis or not is of little overall importance and can be considered an implementationdetail.

3.1.6 Practical example

Now that tag and constraint sets, as well as a CDA-based market that uses these setshave been properly introduced, it is crucial to verify whether these concepts wouldreally function in the current IaaS environment by trying to map some existingIaaS solutions to the proposed model. To do so, this section will give a rudimentaryexample of how the process of matching an Amazon EC2 small instance would work.

Consider a market with 4 different consumers and a single provider (that is, Ama-zon), and let the bidding process proceed as follows.

1. Amazon submits an ask, specifying a volume of 5 small instances at the priceof $0.10 per hour.

Ask1 = [5, 0.10, 12/07/11 13:00:00 UTC-7, {

[Memory, {(unit, MB), (value, 1700)}], [EC2_ECU, {(value, 1)}],

[Disk, {(unit, GB), (value, 160)}], [Platform, {(value, x86)}],

[EC2_API, {(value, m1.small)}], [SLA_Uptime, {(value, 99.95)}],


[SupportedOperatingSystems, {(list, ...)}],

[SupportedEC2_AMI_IDS, {(list, ...)}],

[Splittable, {(value, 1)}],

[AllocationType, {(value, OnDemand)}],

[Time, {(value, 1), (unit, hour)}],

[Region, {(value, USA/Virginia)}]

[Provider, {(name, Amazon AWS), (url, http://aws.amazon.com)}] },

{ ConsumerLocationExclusion["Cuba"],

ConsumerLocationExclusion["Libya"], ... }

]

In this ask, the hardware aspects of the small instance are described usingvarious tags (Memory, Disk, etc). Additionally, the provider also added a listof supported operating systems, a list of supported AMIs (Amazon MachineImage), a Splittable tag indicating that the ask can be split into single units,a tag describing the region in which the resources are located, etc. Note amongthese tags the use of the AllocationType tag to indicate that the resources areoffered according to the on-demand allocation model. Other values for this tagcould be Spot, Future or Reserved as discussed in section 2.5.5. When thechosen allocation model is on-demand, future or reserved, the price specifiedin the ask should be considered as a price that needs to be paid periodicallyaccording to the time unit specified in the Time tag6. When Spot is chosen, thisTime tag keeps it’s meaning as the periodical length of the time after whichthe consumer needs to pay, but the price in the ask should be interpretedas the current spot price rather than the price that needs to be paid everytime unit. As this price changes after each period, the ask’s expiration dateshould consequently be set to expire after the passing of the current period. Ifthe provider wishes to offer spot resources for another period, he should thenresubmit the ask with an updated price.

Besides tags, Amazon also specifies multiple ConsumerLocationExclusion

constraints to make sure that the consumers it is matched with are from coun-tries with which it is willing (or allowed) to do business with.

2. A consumer submits a bid with a price of $0.08 and some hardware constraints.

Bid1 = [1, 0.08, 09/07/11 14:10:00 UTC+1, {

[Memory, {(unit, MB), (value, 1200)}],



[Consumer, {(name, "John Smith"),

(location, "Paris, France, EU")}] },

{ Resource["Memory"], Equal["AllocationType", "value"],

EqualAll["Time"] }

]

6If the reserved or future allocation model is used, an additional Duration tag that specifies howlong the resources will actually need to be allocated or reserved should also be specified as part ofa bid.


As the price of this bid is lower than that of Amazon’s ask in the ask queue,it is not matched and consequently added to the bid queue.

3. A consumer submits a bid with a price of $0.12 and some hardware constraints.

Bid2 = [2, 0.12, 05/06/11 02:00:00 UTC+1, {

[Disk, {(unit, GB), (value, 100)}],




[Consumer, {(name, "Jan Modaal"),

(location, "Brussels, Belgium, EU")}] },

{ Resource["Disk"], Resource["Memory"],

Equal["AllocationType", "value"], EqualAll["Time"] }

]

As the price of this bid is higher than that of the ask in the ask queue and bothprovider and consumer constraints are fulfilled, this bid is matched against theask.

4. A consumer submits a bid with a price of $0.20 and the following tags andconstraints.

Bid3 = [2, 0.20, 15/06/11 14:00:00 UTC-2, {

[CpuClock, {(unit, Ghz), (value, 2.2)}]

[CpuCores, {(value, 2)}],




[Consumer, {(name, "Jane Smith"),

(location, "Boston, Massachusetts, USA")}] },

{ Resource["CpuClock"], LargerThanEqual["CpuCores", "value"],

Resource["Memory"], Equal["AllocationType", "value"],

EqualAll["Time"] }

]

As this bids contains constraints on CpuClock and CpuCores tags that are notpresent in Amazon’s ask, it can not be matched and will thus be added to thebid queue.

As a side note, notice that the consumer is requesting specific processor hard-ware here, which is not supported by most IaaS providers. That is, most of thetime, a high level of abstraction is made when it comes to processor power. Byusing processor type agnostic units such as the ECU, cloud providers can hideheterogeneity among there own infrastructure while at the same time allowconsumers to specify a processor based on its delivered performance ratherthan on its specific hardware characteristics.

A few providers (such as Newservers [64] and CloudSigma [65]) do howeverallow consumers to express more specific processor requirements.


5. A consumer submits a bid with a price of $0.14 and the following tags andconstraints.

Bid4 = [4, 0.14, 05/06/11 02:00:00 UTC+2, {

[EC2_ECU, {(value, 1)}],




[Splittable, {(value, 1)}],

[Consumer, {(name, "Mohammed Naser"),

(location, "Tripoli, Libya, Africa")}] },


Equal["AllocationType", "value"], EqualAll["Time"] }

]

Although the offered price of this bid is higher than the price in Amazon’s askand the consumer constraints are all met, no match will occur. This is dueto the fact that this bid comes from someone from Libya, which means thatthe ConsumerLocationExclusion constraint set by the provider will not befulfilled and consequently the bid and ask will cannot be matched with eachother.

After these bids and asks, the bid and ask queue are as follows.

Bid Queue Ask QueuePrice ($) Volume Bid Price ($) Volume Ask

0.08 1 Bid1 0.10 3 Ask1

0.14 4 Bid4

0.20 2 Bid3

Although this example shows that it is indeed possible to use a CDA-basedmarket to trade IaaS resources, it neglects the fact that if different providers andconsumers use different time units to price their offers, any price-based sorting ofoffers in queues is effectively meaningless. That is, the prices that are attached tooffers can only be used to meaningfully compare offers with each other if the specifiedresource allocation times of the offers are the same.

While it is possible to solve this problem by only accepting timeframes of aspecific size (e.g. a single hour) or by normalizing offers and prices to the same timeunit before putting them in the appropriate queue, the presence of this problemactually suggests the existence of a far more fundamental issue with using the CDAfor IaaS resources. The next section will discuss this problem is in detail and presentthe first steps towards a solution.

3.2. ISSUES WITH THE CDA MODEL FOR IAAS RESOURCES 78

3.2 Issues with the CDA model for IaaS resources

Although tag and constraint sets allow for increased flexibility, when using them inconjunction with the CDA mechanism they still fail to address a more fundamentalissue related to the IaaS pricing model. That is, as cloud computing is based on thepay-what-you-use pricing model, the price that consumers pay for a set of resourcesis not only determined by the characteristics that define these resources, but also byhow these resources are used. More specifically, in the cloud setting, providers donot only charge consumers for the allocation of a specific set of resources, but also forhow these resources are actually used once they are allocated. To implement such amodel, the total cost for the use of a specific set of resources cannot be determinedby a single price but needs to be influenced by variety of different price componentsthat get different weights according to the actual use of the resources. Because ofthis, using the classic CDA that only takes into account a single price component(such as the mechanism described in section 3.1.5) is no longer feasible.

3.2.1 IaaS price models

In practice, most providers implement the multi-price-component model by offeringtheir customers a number of basic resource configurations for fixed price per time-frame and charging extra for the use of specific resources, features or services. Agood example of such a resource is the use of network bandwidth, which is typicallymeasured in transferred bytes and charged separately from the time-based cost ofthe allocation of the resources themselves. Table 3.2 gives an overview of how dif-ferent cloud providers charge bandwidth, while Table 3.3 lists some other resourcesand services that are often charged for separately by providers. Note that some ofthe entries in this table are optional services for some IaaS providers (most notably,those marked with an asterisk ∗), but are part of standard IaaS packages of otherproviders.

Provider Bandwidth Price Model

Amazon (EU-West) $0.10/GB inbound trafficprice outbound traffic varies (1 GB free, $0.15/GB for up to10 TB/month, $0.11/GB for the next 40 TB/month, . . . )

GoGrid inbound traffic is free$0.29/GB for outbound traffic (cheaper prices for larger up-front commitments)

NewServers 3GB of traffic (inbound or outbound) per hour included.$0.10 per extra GB

Flexiscale 5 units per GB (1000 units = $17.95)same price for inbound and outbound traffic

Table 3.2: Bandwidth prices of different IaaS providers

As already mentioned, the fact that IaaS providers use a multi-price-component


Cost Commonly Used Price Model

Bandwidth pay per amount of inbound and outbound trafficDisk I/O pay per number of used disk reads and writesStorage pay per timeframe and/or per stored GBOperating systems pay per timeframe for properietary OSesSoftware licenses pay per timeframe, per user or by an other measureIP address allocation* pay per allocated (block of) IP address(es)Firewall services* pay per timeframe and/or per amount of processed trafficLoadBalancer services* pay per timeframe and/or per amount of processed traffic

Table 3.3: Resources and services that are commonly charged separately by IaaSproviders

model to charge their customers poses a real problem to the use of the CDA mech-anism in the IaaS cloud setting, as the CDA model can only take into account asingle price. When looking at this problem for the first time, it might seem thatthere is a rather simple solution to this problem. That is, letting consumers andproviders specify a price as part of their offer for the allocation of a set of resourcesand letting them use constraints that define restrictions on the minimum or maxi-mum price of the extra price components. For example, a consumer can specify thathe is offering to pay 10ct per hour for a certain set of resources and additionallyspecify an OutboundBandwidthCost constraint that verifies that the price that hespecified in an OutboundBandwidthCost tag accompanying his bid is lower than thatof the provider. Analogous, a provider can specify that he is asking a price for aset of resources and additionally specify a constraint that checks that the price inthe consumers tag is higher or actual to the price the provider wants for a GB ofoutgoing bandwidth.

Although at first it thus may seem that using constraints that impose maximumor minimum values solves the problem of the extra price components, it actuallydoes not. There are several reasons for this. First of all, it might be very difficultfor the consumer to determine what a realistic price is for each of the different costcomponents (although this can probably be solved using some (advanced) price dis-covery mechanisms). Related to this, is the fact that many consumers are probablyonly interested in the total cost of using a set of resources, and not in the exactallotment of all different price components.

Secondly, when a minimum or maximum price constraint is set, it is not entirelyclear what the actual unit-price for use of the particular resource or service shouldbe once a match is made. That is, should the price specified by the consumer beused, the price specified by the provider, or is a more complex pricing rule needed?Although this is in essence not really a problem, but rather a decision that needsto be made, the fact that such decisions potentially need to be made on a per pricecomponent and per provider basis does add more complexity to this method.


The most important issue however, is the fact that when this kind of constraintsare used, the market can no longer guarantee optimal matches using the informationthat is currently provided. This comes directly from the fact that minimum/maxi-mum price constraints do not take the actual usage of the specific resource or featureinto account, only its unit price. To understand why this is an issue, consider Figure3.2, which depicts the total cost of 2 instances7 (at different providers) that use anincreasing amount of bandwidth within a single hour. It is assumed that a fixedhourly price of 10 ct is paid for instance A, while the fixed hourly price for instanceB is 13 ct. Additionally, the provider of instance A charges 10 ct per used GB ofnetwork bandwidth, while the provider of instance B charges 7 ct per used GB. Itis intuitively clear, and shown on the graph, that when no bandwidth is used, usinginstance A is the cheapest. However, when bandwidth is consumed, the differencein total cost between instance A and B becomes smaller. If more than 1 GB isconsumed, using instance B will minimize total costs. Figure 3.3 shows the same

Figure 3.2: Graph showing the importance of taking into account the usage ofadditional costs components. As long as less than 1 GB of data is transfered, usinginstance A is the cheapest, otherwise using instance B is the cheapest.

result, but varies both the used time (in hours) and the used bandwidth (in GB).

7The term instance in this context denotes a basic virtualized computing environment, in thesame way Amazon uses this term to denote the virtualized servers it offers on its EC2 platform.


Obviously, the provider corresponding to the lowest plane for any time-bandwidthcombination is the cheapest.

It is clear that this example can be generalized to incorporate n providers andm price components. Finding the cheapest provider for a specific set of usage valuesis then the same as searching through n hyperplanes in an m+ 1 dimensional space,and finding the hyperplane that has the smallest cost value at the point defined bythe m given usage values.

Figure 3.3: Graph showing the importance of taking into account the usage ofadditional costs components. Both time and bandwidth vary, the cheapest instancefor a given time and bandwidth combination is given by the lowest (smallest totalcost) plane.

In other words, figures 3.2 and 3.3 indicate that ordering bids and asks in queuesaccording to a single price-component as is done in the CDA-based market makesno sense for IaaS resources as this ordering does not reflect the actual price thatwill need to be paid or will be charged to make use of these resources. If a CDAmechanism based on a single price component would still be used, it is destined to begamed by providers that advertise very low prices for that single price component,but charge expensive prices for the actual use of resources. That is, in this mechanisma provider might be ranked at the top of the ask queue by advertising an hourly


price of $0.001 for a set of computational resources, while charging a whole dollar for1 MB of network bandwidth. Almost surely, this makes it a very expensive providerfor a whole range of consumers compared to other providers that advertise a higherhourly price. Consequently, this provider should be at the bottom of the ask queueand not on top. Obviously, such an IaaS market that only incorporates a single-pricecomponent system has limited applicability.

Usage and price tagsIt is thus clear that if any IaaS market is to make acceptable matches, it needs tohave consumer usage data for all of the different price components as well as theactual prices as determined by the providers. As such, it is imperative that con-sumers specify their usage of different resources as part of their bid and providersspecify the different price components as part of their ask. Luckily, this can bedone rather easily using the previously introduced tag sets. For example, ex-pressing that 3.6 GB of outgoing bandwidth will be needed is as easy as defining[OutgoingBandwidthUsage, {(unit, GB), (value, 3.6)}] as part of the tagsetof a bid. Similarly, specifying that the price for that bandwidth is $0.10 per GBis as easy as adding [OutgoingBandwidthPrice, {(unit, GB), (value, 0.10)}]to the tagset of the ask.

Using these usage values and prices, it is theoretically possible for the market todetermine the actual price for the use of a given set of resources. However, it will turnout that due to the heterogeneity of price models that are used by different providers,actually determining this price is still unfeasible without more information abouthow the different price components relate to each other. Before explaining this inmore detail, a remark concerning the importance of the accurateness of the consumerspecified usage values must be made. That is, as the choice of which consumeris matched with which provider is determined by the total cost of using a set ofresources, it is clear that the accurateness of the values in the usage tags is crucialas they directly influence the total cost for the use of these resources. However,in reality, determining these usage values is not always that easy. To accuratelydetermine the bandwidth usage or the disk I/O behavior of a certain application,advanced analysis is often necessary. As doing such analyses is time-consumingand requires a certain expertise, many customers might not be able or willing todetermine accurate (or at least sensible) usage values. To solve this, brokers shouldbe used that benchmark certain applications or use-cases for consumers in order todetermine good usage values. Alternatively, brokers might help consumers estimatethese values based on the consumer’s use-case (e.g. small website, scientific CPUintensive application, etc).

3.2.2 Using a multi price component model

Given the usage and price information as specified by the usage and price tags, it isnow seems possible for the market to determine the actual price that will chargedby provider for a specific consumer bid. In order to do so, the market could use amodel similar to the one given in 3.11.


P (a, b) =∑

T ∈ (T Sa ∪ T Sb)

v(T ) pA(T ) (3.11)

In this formula, the exact price P (a, b) for a certain bid b that is matched withan ask a of provider A is determined by the weighted sum (according to the usageof the consumer) of the different price components of the provider. The weights aregiven by v(T ), which is defined to be the value of the usage-tag that is used by theprice-component for the resource specified by tag T . The term pA(T ) is then theunit price specified by provider A for usage of the resource specified by tag T .

In practice, the formula in 3.11 would be translated to a combination of anOutgoingBandwidthUsage tag with and OutgoingBandwidthPrice tag for networkbandwidth or to a combination of a TimeUsage with a TimePrice for a memoryresource (specified by a Memory tag). The total cost for a given bid is then determinedby the sum of all these different combinations8.

As simple as the idea behind the formula in 3.11 might be, actually incorporat-ing it into the market mechanism and specifically the market’s bidding language isnon-trivial. This comes from the fact that the relation between the usage tags andthe price tags must be specified by the provider as the market itself does not havethis provider-specific knowledge. The way of expressing this must be flexible enoughto allow providers to indicate which prices belong to which resource and usage tagsas well as indicate that certain combinations of tags only incur a single cost. Forexample, when the consumer specifies a Memory tag and constraint, it should bepossible for the provider to indicate that the combination of the value in the con-sumer’s TimeUsage tag and the value in the provider’s MemoryPrice determine theactual cost of the memory. Similarly, a provider should have the means to indicatethat when when both CpuClock and Memory tags and constraints are present, theconsumer should only be charged once for both resources by using the values in theconsumer’s TimeUsage and the provider’s TimePrice tags.

Finally, note that it is very well possible that the usage is not specified in thetags attached to the bid but that the matching ask imposes a standard usage. Inpractice, this is particularly true when certain hardware resources are specified (e.g.:Memory) but no usage tags (such as TimeUsage) are given (the usage function shouldthan return e.g. a single hour for those hardware constraints).

A possible way to define the relations between the usage tags that define v(T )and the unit price tags that define pA(T ) is by using a table such as Table 3.4 thatspecifies which resource tags should by measured by which consumer usage tags aswell as the unit and unit price that should be used to determine the actual price forthe given resources.

8Note that the formula in 3.11 implicitly assumes that the usage unit in the usage and pricetags are the same (e.g. hour, MB, etc.). It is certainly also possible that only compatible unitsare required (e.g. minutes in the usage tag and hours in the price tag), and that v(T ) and pA(T )automatically convert between compatible units.


Resource Tag Usage Tag Usage unit Unit price

Memory TimeUsage hour 0.10CpuClockCpuCoresDiskSLA

OS (Windows based) TimeUsage hour 0.03

LoadBalancing TimeUsage hour 0.10

LoadBalancing OutgoingBandwidthUsage GB 0.02

LoadBalancing IncomingBandwidthUsage GB 0.02

OutgoingBandwidth OutgoingBandwidthUsage GB 0.13

Disk I/O DiskAccessUsage million accesses 0.05

Table 3.4: Provider specified table that defines the relations between usage and pricetags

Note that some resource types such as Memory and CpuClock are grouped to-gether while other resources, such as LoadBalancing occur multiple times in thetable. By grouping the resources, providers can indicate that the consumer shouldonly be charged once for those resources. Similarly, by adding a resource multipletimes with different usage tags, the provider can indicate that certain resources in-cur costs that are determined by multiple factors (in the case of the load balancingservice, both per hour and per in- or outgoing GB). Note that if a certain resourceis not specified in the table, no cost will taken into account for that resource. Alsonote that there might be tags in the first column that are not really resources butrather requirements that also incur a cost. For example, Table 3.4 indicates thatif an SLA tag is specified but no Memory, CpuClock, CpuCores or Disk tags, a costdependent on the TimeUsage tag will still be applied.

Although a solution using a model similar to 3.11 and the price table specified in3.4 would probably work, such a solution is still flawed in two important ways. Firstof all, it assumes that when certain tags are present in a bid, there are also constraintsspecified in the same bid that enforce the use of these tags. More concretely, themodel assumes that when a Memory tag is present in a bid, the consumer shouldautomatically be charged for that (typically according to the TimeUsage tag), eventhough it is perfectly possible that no Memory constraint is attached to the bid thatactually enforces any memory requirement. In order to solve this, a more complexbidding language, based on a more evolved form of Table 3.4, should be introducedthat also takes into account the relation between tags and constraints.

A second, more problematic issue with the proposed price model is that it islinear in nature as it makes a linear combination of the consumer’s usage values andthe provider’s price components. However, in reality many providers do not use such

3.3. THE CONTINUOUS REVERSE AUCTION FOR IAAS RESOURCES 85

a simple linear price model, but use more complex models to charge their customers.For instance, many providers use stepped pricing rules in which consumers pay lessper unit if they buy more units9. Besides such stepped prices, it is not unthinkablethat certain providers may wish to use more complex (non linear) functions or evena piece of software code to determine their exact quotation for a given bid. A goodexample of such a use case is given by NewServers [64], which provides 3 GB of freeoutgoing bandwidth per hour, but charges $0.10 per additional GB. Expressing suchcomplex pricing rules that take several units into account can only be done usingprogrammed code.

Towards a flexible solutionThe only real way of solving all price model related problems as well as the con-straint enforcement issue is by letting providers run custom code - that determinesa quotation for each matching bid - within the market itself. However, one can askwhether the term continuous double auction is still applicable in such case. In fact,such a market mechanism looks more like a reverse auction (since each providerprovides a new quotation for each matching bid based on custom code) than it lookslike a double auction. When further exploring this idea, one can ask whether takingthe difficult step of allowing remote code to run within the market (which intro-duces a lot of security issues that need to be taken care of) can not be replaced byletting that code run on the providers itself, similar to how the problem of customconstraint code was avoided by the use of web service calls as put forward in section3.1.4.

Indeed, by contacting each provider and asking for a quotation for each incomingbid, the market can determine exact prices for bids without needing to sacrifice pricemodel flexibility. The next section introduces a new market mechanism, called theContinuous Reverse Auction, that does exactly that.

3.3 The Continuous Reverse Auction for IaaS Resources

The Continuous Reverse Auction (CRA) is a new type of auction in which ideas ofthe Reverse Auction and the Continuous Double Auction are combined to trade (het-erogeneous) commodities among multiple buyers and sellers. A systematic overviewof how the CRA works, is provided in Figure 3.4.

3.3.1 Mechanism and Principles

In the Continuous Reverse Auction, bids made by the consumers are replaced byso-called requests, and asks are replaced by what will be called quotes. The reasonwhy the words request and quote are used instead of bid and ask is due to the waythe CRA functions. In the Continuous Reverse Auction, it is no longer the case

9This is the case for e.g. Amazon which charges a different price per outgoing GB depending onyour total outgoing monthly bandwidth usage, as already indicated by Table 3.2.


Figure 3.4: Schematic overview of the workings of the Continuous Reverse Auction

that consumers and providers specify prices for a predefined set of resources theyare requesting or offering. Instead, consumers submit requests to the market andproviders provide quotes for these requests through the market. Specifically, thebidding process is started with a consumer submitting a request to the market. Themarket will subsequently find the cheapest provider for that request by queryinga group of previously registered providers and ask them for a quote for the givenconsumer request. Providers answer to this request by either sending a quote to themarket specifying the price they will charge to fulfill the request or by indicatingthat they are unable to match the given request. After all providers have deliveredtheir quote, the market is able to select the cheapest of all the quotes and offer it asa match to the consumer who can then either accept or decline the match.

By using a system in which providers no longer submit prices to the marketbut are asked to give a price on a per request basis, the problems discussed in theprevious section are circumvented as providers are free to use the price model theywant without needing to resort to remote code. Additionally, as providers nowdetermine whether they can match a request with one of their offers or not, themarket no longer needs to know how to match constraints, nor does it need to haveconstraint specific knowledge. Indeed, when using the CRA mechanism, the marketis participant and constraint agnostic, and acts only as an intermediate.

Although the reasons why a quote-based approach for providers is used in theCRA have been explained extensively, the reason why consumers no longer submitprices in the CRA has not yet been covered. The rationale behind this choice stemsfrom the fact that in the CRA, it is no longer necessary for the consumer to do so.To understand this, consider the reasons why a consumer does submit a price in the


CDA.

First of all, in the CDA, the price specified in a bid is used to determine whichprovider asks need to be considered for matching and which do not. That is, theprice specified in a bid is in fact the maximum price the consumer is willing topay for the resources he is requesting. Any asks that are more expensive than theprice specified in the bid will not be considered when trying to match the bid, eventhough it is very well possible that only those asks fulfill the necessary constraints.By doing this, the market is able to match asks to bids more efficiently as it nolonger needs to perform computationally intensive constraint checks for asks thatare too expensive. In the CRA however, this is no longer an issue as the marketno longer needs to check provider constraints itself. As all quotes returned to themarket fulfill the consumer’s constraints per definition, there is no need for a priceto perform a selection among the provider offers.

A second, arguably more important, reason why a price is specified in a bid whenusing the CDA is to create an ordering among consumers. That is, if a bid is notmatched immediately, it is placed in the bid queue according to descending price(bids with lower prices are placed further down the queue) in order to be matchedat a later time. The reasoning behind this ordering is that consumers that arewilling to pay more for their resources should get preference above consumers thatare paying less as this increases the overall allocative efficiency. By first consideringthe bids at the top of the queue when matching an incoming ask, this preferenceis effectively created among the consumers. Concretely, this means that if twoconsumers are requesting the same resources, the consumer that is willing to paymost gets matched first as this is maximizes the overall profit. That is, by matchingwith the consumer that specified the higher price, the provider will be able to getmore money for the resources and the consumer that attaches to most value to theresources gets to actually allocate them.

However, in the IaaS ecosystem, this is no longer important as the principle of“infinite provider capacity” leads to the fact that providers currently do not differ-entiate among consumers, and will always serve both consumers at the same price.As such, there is no need for a direct ordering between consumers, and consequentlythe price specified as part of a bid is also no longer necessary. Section 3.3.5 makessome further considerations about this in the context of allocative efficiency.

Now, although it is theoretically still possible that a consumer specifies a maxi-mum price as part of his request to the CRA, not requiring this frees the consumerfrom the difficult task of having to determine an accurate price estimation for arequest without actually making it. By allowing the consumer to accept or decline aquote (match) himself given its price, the consumer can do accurate price discovery(this is discussed in more detail in section 3.3.3) without having to resort to moredifficult and less accurate techniques.


Also, observe that modifying the CRA model as to allow consumers to submitprices to the market in order to preserve the surplus between the price a consumeris willing to pay and the price the provider is asking, is not that difficult. Doing soonly involves letting the market check whether the consumer’s price is higher thanthat of the cheapest provider and apply a certain pricing rule as to determine whichprice will actually need to be paid by the consumer10. Although there is currently nodirect need to implement this kind of matching due to the infinite capacity model,it is theoretically not impossible that th view towards the infinite capacity modelwill change in the future and that providers will increasingly start differentiatingamong different consumers by using price and allocation models that follow demandand supply more closely. The EC2 spot market is already a good example of such amodel within a single company. The further study of these kinds of market modelis an interesting subject for future research.

Relation to the Reverse AuctionAs the name suggest, the Continuous Reverse Auction is based on principles of theReverse Auction. Although there consequently are a lot of resemblances betweenboth auction mechanisms, there are also some notable differences between them.The main differences between the CRA and the regular RA can be found in thefacts that the CRA is a continuous auction and deals with multiple buyers, whilethe RA is not continuous and also considers multiple sellers but only a single buyer.As with the CDA, the word continuous in Continuous Reverse Auction refers to thefact that the market tries to find a match immediately after a request is submitted,as compared to a regular double auction in which matches are made in a specificmatching phase or a regular reverse auction in which multiple price request roundsare used. In short, by using the word continuous in the name of the CRA, it is thusimplied that the CRA uses only a single price request round and that matches arereturned directly.

The reason why the CRA only uses a single price round is based on the observa-tion that for virtually all requests such multiple price rounds are just not beneficialas in the IaaS ecosystem providers are not interested in competing with each otherfor every single consumer request. That is, assuming an IaaS market that willhave to match a very large number of requests at any given time (in the order ofthousands per hour or even minute), the matching of a single additional request isfinancially insignificant compared to the total number of requests that are matchedby a provider; even if that specific request involves a considerable amount of money.As such, from a providers point of view, trying to ‘win’ a single additional request isjust not effective. Instead, in order to make significant financial gains, the providershould adjust its general matching strategy as to ‘win’ a whole range of similar re-quests. How a provider can do this is briefly explained in section 3.3.3, which dealswith price discovery in the CRA.

Note that the insight that the multiple price rounds are no longer needed in the

10Recall the pricing rule examples in section 2.5.3.


CRA is in fact derived from the reason why multiple rounds are needed in the regularreverse auction. That is, in the normal RA, multiple price rounds are used becauselarge amounts of money are involved and only a single (typically special) good orservice is being bought by the consumer. Because of this, winning the auctionis crucial to providers in order to do any trading at all. In the IaaS ecosystemhowever, the goods that are being bought are cheap and abundant (in other words,commodities). If a provider is unable to win a particular request, nothing is lost,as there are plenty of opportunities to do win requests in the future. As such, themultiple price rounds as used by the RA are no longer needed as providers do notneed to compete over a single request in order to trade resources.

Of course, only using a single price request round also has some benefits in termskeeping the system’s complexity limited. That is, there is no need for providersto maintain state as each price request is independent from any previous request.Additionally, as only a single message needs to be exchanged between the marketand providers for a match to occur, the load on the entire system is reduced, as wellas the time that is needed to find a match.

3.3.2 Bidding Language

Now that an overview of how the CRA functions as well as the core ideas behind ithave been discussed, it is time to elaborate on the bidding language of the mecha-nism. As Figure 3.4 provided a good overview of the different steps of the CRA’sbidding process, it will be reused as a basis to discuss the CRA’s bidding language.

The bidding process of the CRA begins with a consumer that wants to allocatea set of IaaS resources. In order to do so, the consumer submits a request to themarket. The bidding language that is used to specify this request, is based on thetag and constraint sets that have been introduced before.

Request = [Volume, TagSet, ConstraintSet]

As mentioned before, a consumer only has to accept or decline an offered price,meaning that it is no longer necessary for him to specify a price as part of hisrequest. Also, as a request in the CRA is either matched immediately or not at all(if no provider is able to fulfill the consumer specified constraints), there is no need tospecify a expiration date as part of the request. In other words, the only componentsof a request are a Volume indicating how many units a consumer wishes to allocate,and a TagSet and Constraint specifying the resource he wishes to allocate.

Once the request has arrived at the market, the market will send out a pricerequest to all registered providers, to which the providers respond with either anegative answer meaning that they cannot match the request or with a quote thatspecifies a price for the request.

Quote = [Price, Volume, Identifier]


Besides the Price, the quote also contains a Volume that indicates how manyunits the provider can or is willing to match (this is only important if the consumer’srequest is splittable, that is, tagged with the Splittable tag). The Identifier inthe quote allows a consumer to actually allocate resources after a match has beenfound on the market; it allows the consumer to proof the commitment a providermade when it handed out a certain quote. How this works exactly is detailed in thefollowing chapter.

After all registered providers have returned a quote (or have replied that theycannot match the request), the market returns the cheapest quote as a match tothe consumer. If bids can be split, the market can match different units of a con-sumer’s request with different providers and return multiple matches. Naturally, thismatching occurs according to the quote prices specified by the providers (cheaperproviders are matched first). Algorithm 6 shows how the market could implementsuch a matching procedure. In order to be concise, Algorithm 6 does not incorporatesplittable bids.

Algorithm 6 submitRequest(r: Request): Match

bestQuotes← {}for p in providers do

bestQuote← bestQuotes.first();currentQuote← p.getQuote(r);if bestQuote = null or currentQuote.getPrice() < bestQuote.getPrice() then

bestQuotes← {currentQuote}else if bestQuote.getPrice() = currentQuote.getPrice() then

bestQuotes.add(currentQuote)end if

end formatchingQuote← selectRandomElement(bestQuotes);return new Match(matchingQuote);

After the consumer has received the match (or multiple matches), he only needsto confirm or decline this match to the market in order for the matching process tocomplete.

Compound RequestsThis section detailed the bidding language and process of the CRA mechanism.Among other things, it defined which components are part of a consumer’s request.However, the given definition implicitly assumes that the application for which theconsumer is submitting a request has a clear and single behavioral profile. This doesnot necessarily need to be the case. That is, there exist several applications thatcan be split in several distinct phases that each have different behavioral profilesand resource requirements. A good example of such an application is a scientificbatch queue application in which a large amount of data is fetched in a first phase,


calculations occur in a second phase and results are reported back in a final phase.

In order to deal with these kinds of applications, it is interesting to introduce amore elaborate bidding language that is able to cope with these kind of requirements.Appendix B does so by introducing the concept of compound requests.

3.3.3 Price Discovery

Although the problems that were discussed in section 3.2 have been effectively ad-dressed with the introduction of the CRA, the issue of price-discovery has not beendiscussed yet. Recall from section 2.5.4 that price discovery is the process that de-scribes the mechanisms that are used by buyers and sellers to determine or approx-imate the price of a good in the marketplace. The use of price discovery techniquesallows consumers and providers to discover the price at which the market is currentlytrading goods. This can be useful for consumers or providers that are new to themarket to get an idea of the price at which resources are being traded. Consumersin particular can make use of price discovery techniques to make an upfront trade-offbetween the price they will need to pay on the market and the resources and qualityof service they will receive for that price.

Additionally, price discovery techniques allow consumers and providers to deter-mine bidding strategies by monitoring the price for a specific resource in the marketand choosing the time at which they buy or sell certain resources intelligently as tomaximize their personal profit.

In the CRA market, price discovery is in fact already built-in for consumersas they have the possibility to request a quote from the market at any given timewithout any further obligation. As such, both new as existing consumers can alwaysdiscover the price for any request and make decisions based on that.

The situation is a bit different for providers as they have no idea at which priceand rate certain resources are being traded except when they are the provider thatactually gets to trade the resources for a given request. The best providers can dois get an estimation about which resources are in popular demand according to theprice requests they are receiving from the market.

In order to solve this lack of price discovery tools for providers, the CRA model isextended with a billboard-like mechanism on which the market publishes anonymizedconsumer accepted matches. This is done in real-time; after a match is found, it isdirectly published on the billboard. Because of this real-time behavior, the billboardeffectively represents the current market situation. Whether the billboard containsonly the last match, or the last x matches is an implementation detail and does notchange its applicability. Figure 3.5 shows an updated design of the CRA that takesthe billboard into account.

Using the billboard, providers can analyze the anonymized matches and adjust


Figure 3.5: Price discovery in the CRA market using a billboard to publishanonymized bids

their pricing strategy accordingly. For example, if a provider is able to learn from thematches that the price of matches in a specific category of requests is consistentlylower than the price it is offering, it might be interesting for the provider to adjustits strategy and lower its quote price in order to get more matches. Analogously, if aprovider learns from the billboard that it is the only provider that has matches for aspecific category of requests, it might be interesting for the provider to increase thequote price for that category of requests, in order to increase its profit. This notionthat providers will adjust their strategy according to the prices of the matches theyobserve on the billboard introduces a form of indirect competition among providerswhich is desirable in terms of market dynamics. Section 3.3.5 briefly revisits thisidea in the context of allocative efficiency.

Decision Support SystemsDecision Support Systems (DSS) are computer-based information systems that sup-ports business or organizational decision-making activities. In the context of marketsand resource allocation, such systems often make heavy use of available price discov-ery techniques to determine optimal bidding strategies or trade-offs. As this involvescontacting the market (and through the market the providers) very frequently in caseof the CRA, the question whether the use of such systems put to much strain onthe market becomes very relevant. Although the CRA market mechanism itselfis computationally inexpensive, the amount of communication per request betweenconsumer, market and providers is considerable (see section 3.3.5). In order to relievesome of the strain from the market, it can consequently be interesting to allow con-


sumers to have access to the billboard, so that they can do analysis on the matchesthat are posted there. This would allow consumers to approximate the price of aset of resources, based on e.g. the average or normalized price for these resources.In other words, giving consumers a different way to do price discovery that puts noadditional burden on the market.

Note that doing this kind of analysis can become very complex as it involvesextracting price information from matches that rarely contain exactly the sameresources and usage values. Nonetheless, because of the potential benefits of doingsuch analysis, its further study is an interesting topic for future research.

3.3.4 Practical Example

To demonstrate the exact workings of the CRA, and to clarify how it differentiatesfrom the previously discussed CDA model, it is interesting to consider an examplesimilar to that of section 3.1.6 and see how the CRA finds matches for the consumers.

Assume that 3 providers (Amazon AWS, GoGrid, Rackspace [66]) are registeredwith the market prior to the arrival of any request and consider the arrival of fol-lowing requests.

1. A consumer submits a simple request to the market for that contains a Memoryconstraint.

Request1 = [1, {



[TimeUsage, {(value, 1), (unit, hour)}],

[Consumer, {(name, "John Smith"),

(location, "Paris, France, EU")}] },

{ Resource["Memory"], Equal["AllocationType", "value"],

EqualAll["TimeUsage"] }

]

After relaying this request to the providers, they will all respond with a quotethat contain the following prices11.

Provider Price ($)

Amazon 0.085GoGrid 0.38RackSpace 0.12

In order to return these prices, the different providers first have to make surethat they had an offer that actually matches all constraints that are specifiedin the request. To do this, providers should model their own offers using the

11In the following examples, prices are based on the actual prices of these providers on April 25th,2011.


tag and constraint sets and match the requests against these offers by verifyingwhether both the consumer’s and the provider’s constraints are satisfied. Notethat it is assumed here that providers understand the tags and constraints asspecified by the consumer. In reality this will only be the case if providersand consumers share a set of common tags and constraints. The next chapter(section 4.2 in particular) elaborates on how this sharing of tags and constraintscan be achieved.

Only when providers have determined whether they can match a request with acertain offer, they should determine the actual price for that offer based on theusage values specified in the consumer’s bid. Note that providers should specifytheir quote in the same time unit as the time unit specified in the request. Thisis enforced by the TimeUsage constraint in the consumer’s request. If no suchconstraint is present, then the quotes returned by the provider are in factmeaningless. As such, it is the consumer’s responsibility to make sure thatsuch a TimeUsage constraint is always present.

As was the case with the Time tag in the example of section 3.1.6, the mean-ing of the TimeUsage tag should be interpreted differently according to theAllocationType tag that is part of the same request as the TimeUsage tag.That is, if the specified allocation type is on-demand or spot, the TimeUsage

tag should be interpreted as the size of a single time-frame, while in case of afuture allocation (or if no AllocationType tag is present), the TimeUsage tagshould be interpreted as the total time the resources are requested. In case theallocation model is reserved, there will be a need for an additional tag (e.g.ReservationTimeFrame) that specifies what the reservation timeframe is.

In case of Request1, the market will return Amazon’s quote to the consumeras it has the lowest price. In reality, this quote corresponds to a small instancewith 1.7GB of RAM, 1 ECU and 160GB of hard disk space.

2. A consumer submits a more complex request to the market that contains somehardware, software and bandwidth descriptions.

Request2 = [1, {



[OperatingSystem, {(name, Windows), (version, Server 2008)}],

[OutgoingBandwidthUsage, {(unit, GB), (value, 3)}],





{ Resource["Disk"], Resource["Memory"], EqualAll["TimeUsage"]

Equal["AllocationType", "value"], EqualAll["OperatingSystem"] }

]


Note that the specified OutgoingBandwidthUsage tag should be interpretedas the usage within the timeframe defined by the TimeUsage tag. In otherwords, in this request the consumer has specified that he will be needing 3GBof outgoing bandwidth on an hourly basis. Note that if the consumer doesnot specify such a TimeUsage tag, the OutgoingBandwidthUsage is effectivelymeaningless.

After relaying this request to the providers, the market receives the followingprices back.

Provider Price ($)

Amazon 0.78GoGrid 1.25RackSpace 0.70

As Rackspace offers the cheapest solution, the market will return that quoteto the consumer. In reality, this quote corresponds to a Windows 2008 Serverinstance with 2GB of RAM, 4 virtual CPU cores and 80GB of hard disk space.

3. A consumer submits a request for a one-time allocation of 2 servers for a fixedtime (in other words, a Future request).

Request3 = [2, {



[OutgoingBandwidthUsage, {(unit, GB), (value, 50)}],

[IncomingBandwidthUsage, {(unit, GB), (value, 10)}],

[AllocationType, {(value, Future)}],


[StartTime, {(value, 15/06/11 14:00:00 UTC-2)}]




EqualAll["AllocationType"],

Equal["StartTime", "value"], Equal["TimeUsage", "unit"]

GreaterThanEqual["TimeUsage", "value"] }

]

The market once again relays the request to the providers, of which only tworespond that they can match the request (GoGrid does not have an offer withmore than 8GB of RAM).

Provider Price ($)

Amazon 776.70GoGrid -RackSpace 969.80

As Amazon is the cheapest of the two, it will be returned as match to theconsumer.


3.3.5 Desirable Market Properties

Now that the Continuous Reverse Auction has been properly introduced, it is inter-esting to review the desirable market properties that were discussed in section 2.4and see how they apply to a CRA-based market. Note that the intent is not theprovide an in-depth evaluation of these properties (as doing so requires advancedgame-theoretic [67] and complexity analysis [57]), but rather to address the differentproperties in a more intuitive, ad-hoc manner.

Allocative EfficiencyRemember that the term allocative efficiency indicates the measure in which a mar-ket is able to find the most optimal matches between consumers and providers. Asconsumers are always price takers in the CRA model (meaning that there is no suchthing as matching the consumer that is willing to pay most with the provider thatasks the most) is is very difficult to make definite statements about the allocativeefficiency in a CRA-based market.

From a consumer’s point of view, it is clear that a CRA-based market alwaysfinds the best match (that is, the cheapest match); as the CRA was specificallydesigned to do so. However, when looking at the market as a whole, things are morecomplicated. As direct competition - based on the prices specified in bids - betweenconsumers as it existed in the double auction is replaced by indirect competitionin the continuous reverse auction - based on different maximum prices differentconsumers are willing to accept for a given set of resources - the idea of one matchbeing better than another no longer exists for a provider. Consequently, makingstatements about which match is the most optimal is impossible.

The fact that this is impossible, is a result of the “infinite capacity” model cloudproviders currently convey to their customers. In this model, computing capacity isabundant and consequently direct competition for capacity among consumers doesnot exist as there is plenty of capacity for all. In reality however, forces of demandand supply do exist, and there are certainly times at which capacity is a lot scarcerthan at other times. In order to cope with this, while at the same time keepingthe illusion of infinity capacity, providers increasingly let their customers engage informs of indirect competition by increasing the price of their resources according todemand. The Amazon EC2 spot market is a good example of this: it let’s consumersspecify a maximum price they are willing to pay for a certain set of resources, therebygiving priority to consumers that are willing to pay higher prices.

By letting the consumer decide what the maximum price is at which he acceptsa certain match the CRA does in fact the same. As such, the CRA does have animplicit mechanism that let’s providers differentiate among different consumers.

The use of this mechanism in itself does not make the market allocative effi-cient on a short timeframe as no difference is made between consumers that acceptmatches at the same point in time. However, this indirect competition betweenconsumers does give the market allocative efficient-like characteristics when viewed


over a longer period of time. That is, by the fact that in the CRA-based marketthe price for a set of resources varies according to current demand, the market williteratively converge to a situation where scarcer resources are allotted to consumerswith higher valuations for those resources (as consumers with lower valuations willno longer accept the proposed price for those resources). As such, given enough com-petition among providers (which makes sure that the price for resources reflects theirdemand), a CRA-based market will exhibit optimal-like matching characteristics.

Individual RationalityIn its essence the term Individual Rationality conveys the question whether takingpart in the market is the most rational choice for each of the markets participants. Incase of the CRA, this is certainly so for consumers as the CRA’s matching mechanismguarantees that consumers get the lowest available price. Additionally, as consumerscan always decide to not accept a match, they are never forced to incur negativeutility.

The situation is a bit different from the provider’s point of view, as the CRAintroduces a form of increased competition among the providers by the use of thebillboard. In reality this will mean that large well-known providers (such as Amazon)will have the least added value by joining a CRA-based market as doing so increasescompetition among them and others. However, as these providers are large, theytypically enjoy better economies of scale than the smaller providers, meaning thatthe cost of their resources should be lower than that of other providers, allowingthem to have larger profit margins for the resources they sell on the market.

The small providers will however also benefit from entering the market, as thedirect competition allows them to directly target a very large audience. This isalso true for providers that are specialized in particular IaaS solutions, as theirspecialization usually allows them to offer their solutions at cheaper prices thansimilar solutions from other non-specialized providers, meaning that they will beable to ‘win’ most requests that fall under their specialization.

In any case, a provider does not gain negative utility by entering a CRA-basedmarket as he is never required to sell below the cost of production (or service). Notethat the fact that the provider might have to pay a fee to take part in the marketis ignored here. If a provider has to pay such a fee to take part in the market, hemight incur a negative utility (if this fee to take part in the market is larger thanthe profit he makes from trading resources on the market). The same is obviouslytrue for consumers.

Incentive compatibilityAs mentioned in section 2.4.3, a market is called incentive compatible if its match-ing mechanism forces the participants to bid truthfully in order to maximize theirpersonal utility. When considering whether the CRA model is incentive compatibleor not, it is clear that little can be said regarding the consumers, as in the CRAmechanism, consumers themselves do not submit prices. As the consumer is a price


taker in the CRA, he can never directly influence the price of his match. Also, asthe auction is continuous, the consumer cannot influence the price of his match bytaking the role of a provider and submitting shill bids to try and influence the priceof the other providers.

While there is not much to say about incentive compatibility for the consumers,some remarks do hold for the providers. That is, as all providers have perfect infor-mation about the price of the resources that are being traded due to the introductionof the billboard and given that there is enough competition among providers, therewill be an incentive among providers to send quotes to the market that containprices that are close to the actual value the provider attaches to offered resources atthat moment. This comes from the fact that the only way a provider can increaseits chance to match certain resources is to decrease its price. As such, it seemsreasonable to suggest that in general providers will have a tendency to lower theirprices to the point at which they have reached their current marginal value. In sucha scenario, the actual price for a given set of resources on the market will exhibitspot market behavior as it will reflect the current demand for those resources.

Budget BalancedRecall that a market is called budget balanced if the party that organizes the marketdoes not have to add extra money to the market in order for a market clearing tooccur. When considering a CRA-based market, it is clear that it is budget balancedas the consumer always pays to full costs that are asked by the provider when a matchis accepted. As such, the market never has to intervene to make sure that a trade isfinancially balanced. However, the market does have the additional cost of havingto run the electronic trading platform. To cover these costs, any real-world marketwill probably ask participants to pay a fee to take part in the market. Althoughrequesting a flat subscription based fee is surely possible, a more fair model is basedon the pay-what-you-use principles and charges participants based on the numberof interactions they have with the market. That is, consumers pay for each requestthey make, while providers pay based on the number of price requests they receive(and responses they send back). A more fine-grained model may charge participantsbased on the number of web service or even HTTP (GET, POST) requests. Whetherthe market is then weakly or strongly budget balanced is clearly dependent on theexact details of the payment model and of no further importance here.

Computational TractabilityWhen considering the subject of computational tractability in the light of the Con-tinuous Reverse Auction, it is intuitively clear that algorithmic complexity of thematching mechanism itself is rather low as the only real task the market has to dois determine which of the quotes for a specific request contains the lowest price. Byletting the providers determine whether they match a certain request as well as de-termine the price for that specific request, the market has effectively distributed thecomputational complexity that is inherent in the matching of the heterogeneous IaaSresources. Furthermore, as no bids or asks need to be stored in queues as is the casein the double auction, no state needs to be maintained between subsequent matches.


As such, parallelizing the market’s matching mechanism over different servers is pos-sible, adding to attractiveness of the CRA mechanism from a computational pointof view.

Figure 3.6: Hierarchical market setup to reduce CRA communication bottleneck.Different providers register with different leaf market components. Each leaf marketcomponent sends out n price requests to its registered providers and only sends thecheapest match to its parent market component. The top level market componentthen picks the cheapest of the matches it received from its children as the actualmatch for the consumer.

A downside to the CRA, is that the amount of communication between all in-volved actors will be larger than in other market mechanisms (particularly, the dou-ble auction) as the market sends out price requests to all providers (which send backreplies) for each consumer request. Concretely, if there are n registered providersthat can all fulfill a certain consumer request, 2n + 3 messages will be needed for amatch to occur. That is:

• 1 initial request from the consumer to the market• n price requests from the market to the providers• n responses from the providers to the market• 1 match message from the market to the consumer• 1 match acceptance message from the consumer to the market

When comparing this to the Continuous Double Auction in which no additionalmessages with providers need to be exchanged for an incoming consumer bid, it isclear that the flexibility the CRA offers comes at a considerable communication cost.In order to keep this cost manageable, providers might use brokers that have verygood connectivity to the market. However, in the end, it is of course the networkconnectivity of the market itself that becomes the bottleneck. In order to copewith this, a hierarchy of market components can be setup that each determine the


cheapest match and forward that to the their parent component. An overview ofhow this could work is depicted in Figure 3.6.

3.3.6 The Continuous Reverse Auction for other commodities

Although the Continuous Reverse Auction has been introduced to deal with spe-cific issues concerning IaaS solutions, the general principles behind the CRA arenot bound to the IaaS or cloud computing ecosystem. By taking a step back fromthe cloud computing context of this thesis, it becomes apparent that the proposedCRA model tries to solve the more general problem of trading heterogeneous com-modities in an environment with a seemingly infinite supply, many consumers andproviders, and many different price models. In fact, as tag and constraint sets arenot specifically tailored to the IaaS environment, but have the flexibility to describeany requirement or commodity, the CRA based market as described in the previoussections needs almost no adjustments to be used in a different context. An exampleof a market in which the CRA could be used is given by the following use case.

Use Case 1: Commodity market for Rental CarsIn the rental car market, many different cars are rented by many consumers at anygiven time. Because of the varied requirements of consumers, car rental companiesuse a pay-what-you-use model to charge users according to their specific needs. How-ever, the exact details of these price models vary wildly among different providers.Some providers charge their customers per day, others per hour. Some charge extraaccording to mileage, or if the car is returned at another location than it was pickedup. Extra charges may also apply for young drivers, (extra) insurance, returning thecar with a empty gas tank, the use of gps systems, etc. Some providers allow thecar to leave the country where the car was picked up, others charge extra or don’tallow it. Virtually all providers charge more for larger or more luxurious cars, etc.

Without going in further detail, it is clear that the car rental market exhibitsthe same heterogeneity (both in the used price models as in the description of thecommodity) as the IaaS market. As such, using the CRA to trade rental cars seemsto be a viable option.

CHAPTER 4

Market Implementation

The design of market mechanisms is mainly theoretical in nature. Nevertheless,investigating how markets can be implemented in practice is valuable as this imple-mentation is a complex technological undertaking. This stems from the fact thata market is inherently a distributed system in which many different parties withdifferent interests interact with each other. As such, any electronic trading platformthat brings those parties together will be distributed in nature, and will have to dealwith computer protocols, quality of service constraints and security.

In order to give an idea of the involved complexity, this chapter focuses on thepreviously introduced concepts of tag and constraint sets as well as the ContinuousReverse Auction, and discusses how they can be implemented concretely. Further-more, a prototype that was implemented as part of this thesis to show the real worldapplicability of the CRA based market will be briefly discussed at the end of thischapter.

4.1 Implementation of the Continuous Reverse Auction

As any market for cloud computing resources is inherently distributed, specifyingthe language with which the involved parties interact is a key architectural deci-sion. As most actors (consumers, providers, the market and different brokers) areloosely coupled, using service-oriented architecture design principles [68] will keepthe overall design simple, elegant and modular. Also, as the platform on which cloudcomputing is based (that is, the Internet) is an inherently heterogeneous ecosystemin terms of operating systems, used programming languages and application soft-ware, it is compulsory that the communication mechanism used by the differentmarket components is independent of these specific details. Furthermore, as one of

4.1. IMPLEMENTATION OF THE CONTINUOUS REVERSE AUCTION 102

the goals of this thesis is to design a market model for IaaS cloud resources that canbe used today, it is imperative that the chosen communication technology is matureand in wide-spread use. Based on these requirements, the choice for a WSDL/SOAPbased web service architecture seems valid as web service technology was specificallydesigned with these requirements in mind. As a discussion about the exact work-ings and principles of this technology is outside the scope of this thesis, readers arereferred to [69] for more information about web services.

Based on webservice technology, it is possible to deploy an electronic CRA-basedmarket, by implementing the different market components as depicted in Figure 4.1.

Figure 4.1: Implementation of a CRA-based market using webservices

The core components of the market itself are the webservices that allow theconsumers to make requests and providers to register in order to respond to pricerequests.

The ConsumerIaaSMarket service is a rather simple service that allows con-sumers to make requests and accept or decline previously matched requests. Ad-ditionally, the service allows the consumer to retrieve some information (such as aname, description, certificates, etc.) about the market. Note that although interface


of the ConsumerIaaSMarket is rather simple, its implementation is more complex asthe service needs to contact all registered providers for the submitted request andstore the request’s match for a certain amount of time (the service is thus stateful),awaiting the potential acceptance of the match. Once the match is accepted by theconsumer, the service can anonymize it, publish it to the billboard and remove itfrom its internal data structures. If the match is declined by the consumer, theservice can also safely discard it. If the consumer does not answer within a specifiedtimeframe, the market should also be allowed to discard the match as to avoid need-ing to permanently store matches that are never accepted or declined. Note thatthe size of this timeframe effectively determines the lifetime of a match (that is, howlong a match given by a provider is valid). Clearly, it is technically also possiblethat the lifetime of a match is specified by the provider on a per match basis; atthe cost of extra complexity in both the bidding language and the implementationof the market.

The ProviderIaaSMarket service is the webservice that allows providers or theirrepresentative (brokers) to register a PriceRequestService and an Execution-

Service. It should also allow providers to retrieve some extra information aboutthe market itself (as is the case with the ConsumerIaaSMarket service).

The PriceRequestService is a standardized webservice (standardized inter-face) implemented by all providers that allows the market to request prices forconsumer requests. By using such a standardized interface, the actual implemen-tation of the matching algorithm (as given in Algorithm 6 in section 3.3.2) for theConsumerIaaSMarket service is almost trivial. Listing 4.1 shows a (slightly simpli-fied) implementation of the ConsumerIaaSMarket’s submitRequest() operation inthe Java programming language. Note that through the use of the JAX-WS library[70], it is relatively straightforward to adjust the given code to use webservices1.

In practice however, it is important that some additional remarks are taken intoaccount. That is, the implementation of Listing 4.1 should be modified to contactthe different providers in parallel as this is obviously far more efficient. Additionally,it is important that the market enforces some quality of service rules when query-ing providers. Concretely, in order to guarantee that matches are found within aconsiderable amount of time (that is, within the order of a few seconds at most)the market requires the PriceRequestServices to answer requests within a certainamount of time (e.g. 500 ms). If a specific PriceRequestService does not answerwithin this timeframe, the market should be allowed to continue the matching pro-cess without waiting for a response, effectively ignoring this service for the request inquestion. When a PriceRequestService fails to meet the QoS constraints multipletimes, the market may apply more severe penalties. For example, it can decide toonly contact violating services for certain (randomly selected) consumer requests in

1Actually, the code in Listing 4.1 is very similar to the code that is actually used in the prototypethat accompanies this thesis; the only noteworthy additions are a few JAX-WS annotations on someinterfaces and classes.


1 public Match submitRequest(ConsumerRequest consumerRequest) {

2 List <Quote > bestQuotes = new ArrayList <Quote >();

3 Iterator <Provider > itor = providers.iterator ();

4 while (itor.hasNext ()) {

5 Provider provider = itor.next();

6 PriceRequestService service = Service.createService(

provider.getPriceRequestServiceDescription (),

PriceRequestService. c la s s );7 PriceRequest request= createPriceRequest(consumerRequest)

8 Quote quote = service.getQuote(request);

9

10 i f (quote != null) {

11 i f (bestQuotes.isEmpty ()) {

12 bestQuotes.add(quote);

13 } e l se {

14 Quote bestQuote = bestQuotes.get(0);

15 i f (quote.getPrice () < bestQuote.getPrice ()) {

16 bestQuotes.clear ();


18 } e l se i f (quote.getPrice () == bestQuote.getPrice ())

{


20 }

21 }

22 } / / e l s e n u l l : s e r v i c e c a n n o t h a n d l e r e q u e s t e d b i d

23 }

24

25 i f (! bestQuotes.isEmpty ()) {

26 RandomSelector <Quote > randomSelector = new RandomSelector

<Quote >( bestQuotes.toArray(new Quote []));

27 Match match = new Match(UUID.randomUUID ().toString (),

quote.getPrice (), quote.getProviderInfo (), quote.

getExecutionServiceDescription ());

28 pendingMatches.put(match.getIdentifier (), new MarketMatch

(consumerRequest , match , quote.getTicket (), new Date()

));

29 return match;

30 }

31 return null;

32 }

Listing 4.1: Java implementation of the CRA matching mechanism of theConsumerIaaSMarket (Simplified for readability).


order to lower the workload on that service. In extreme cases, the market may banthe service entirely.

The ExecutionService is a provider-specific service that allows consumers toaccess their acquired resources once they have accepted a match. When lookinginto this service in more detail, it becomes clear very quickly that the reoccurringproblem of standardization (or lack thereof), is again a serious issue when trying todefine this service in a uniform way. That is, as there is currently no standard wayin how different cloud providers let customers access their resources (although someinitiatives to solve this are on their way, most notably the OpenStack [12] initiative),it is impossible to define a single standardized interface for the ExecutionService.

In this respect, the only service the market really can provide is passing aprovider- and match-specific identifier from the PriceRequestService to the con-sumer once he accepts a match, so that the consumer can identify the acceptedmatch to the provider’s ExecutionService. Note that no further details on how aconsumer needs to do this are given here, as this is entirely provider specific. It ispossible that this process happens fully automatically through web service calls orthrough brokers, or the consumer could manually provide the given identifier throughthe provider’s website. The further issues in actually acquiring the resources (whichcan involve authenticating, passing e.g. certificates or tokens, installing tools, etc.)are not further discussed here.

On a side note, instead of passing an identifier, it is probably more interesting topass a more generic ticket-like datastructure that can contain more information thanonly a match specific identifier, such as an expiration date or extra provider specificinitialization data. Furthermore, by passing an encrypted hash that describes thematched resources instead of passing the identifier or ticket itself, providers cancompletely separate their PriceRequestService and ExecutionService. That is,by passing an encrypted hash between both services via the consumer, the need forany direct communication between both services is eliminated. This is particularlyinteresting when the PriceRequestService service is operated by a broker ratherthan the provider itself. By using asymmetric (public key) encryption, providerscan be sure that hashes cannot be altered with. Figure 4.2 shows how this wouldwork.

Besides the ProviderIaaSMarket and ConsumerIaaSMarket services, the marketshould also provide a secured MarketAdminstration webservice to allow easy in-spection and management of the market by administrators.

The Billboard webservice should allow the market to publish anonymized matches.Additionally, the operator of the Billboard webservice should also host a website

4.2. IMPLEMENTING TAGS AND CONSTRAINTS 106

Figure 4.2: Passing encrypted tickets between the PriceRequestService and theExecutionService via the consumer

on which the published matches can be found by the providers2.

An overview of all the operations of the different webservices can be seen in Figure4.3, which shows them in UML representation. Note that the PriceRequestServiceand ExecutionService parameters of the register() method of the Provider-

IaaSMarket represent descriptions (which contain among others, the WSDL loca-tion) of actual deployments of these standardized services.

4.2 Implementing Tags and Constraints

Now that is has been explained how the different components of a CRA-based marketcan be implemented using webservices, some light will be shed on the implementationof the tag and constraint sets that are part of a consumer’s request.

4.2.1 Implementing tags

The tags that were first introduced in section 3.1.4, can be implemented quite lit-erally as objects containing a name and a set of attributes. Figure 4.4 shows class

2This information can also be delivered in a more structured syndication format such as RSS orAtom to allow providers to process this information automatically.


Figure 4.3: UML diagram of the webservices of a CRA-based IaaS market

diagrams of the implementation of the Tag and TagSet in the Java programminglanguage.

Figure 4.4: UML diagrams of a Tag and TagSet

As already mentioned before, the name of the tag can just be of the regularstring type of the language (in this case, the native java.lang.String type). Thetag attributes are easily implemented using associative array like structures, such asJava’s java.util.Map. By using a HashMap, constant lookup times are guaranteed.The implementation of the TagSet also uses the HashMap to have fast Tag lookupwithin the TagSet.

4.2.2 Implementing constraints

The implementation of constraints is a bit more complicated as there exists a divisionbetween custom constraints and standardized constraints. Recall that standardizedconstraints are constraints of which the semantics are standardized and availableto all involved parties, while custom constraints allow customers and providers toexpress participant specific matching semantics through the use of webservices.

Implementing standardized constraintsThe implementation of standardized constraints is straightforward as it only involvesspecifying the name of the constraint and optionally some parameters that initialize


the constraint (this can again be implemented with the use of associative arrays). Fig4.5 shows an UML representation of how this can be done in the Java programminglanguage.

Figure 4.5: UML diagrams of a ConstraintDescription and Contraint-

Parameters that specify a standard constraint

Although the implementation of a constraint specification is thus quite simple,there are some remarks about the distribution of the semantics of the constraints.That is, in order to guarantee correct matching behavior, constraints should beprecisely defined so that there is a clear and unique understanding among all par-ticipants of what the constraint exactly specifies. As it is a core characteristic of theCRA that the market itself is unaware of constraint contents, an additional entitywill be needed that provides this exact definition. A separate (market independent)lookup service that clearly specifies how constraint names map to actual matchingsemantics can solve this problem fairly easy. In order to specify these constraintsemantics, the lookup service can make use of a set of (algebraic) rules, pseudocodeor a specific programming language.

It is clear that multiple such services can exist, possibly specializing in differentcategories of constraints. A good way to avoid constraint name clashes in the case ofmultiple lookup services is by using the Reverse DNS naming convention as regularlyapplied when using the Java programming language.

Figure 4.6 shows how the consumers and providers would interact with such alookup service. Note that the lookup table in the figure contains a column with anexpiration date for each given constraint name. This expiration date indicates howlong the given constraint definition is valid; that is, the date indicates until when thesemantics for a certain constraint name are guaranteed to be unchanged. Using thisexpiration date, both consumers and providers can effectively cache the semanticsof constraints, drastically reducing the amount of communication between themand the lookup service. It is clear that many constraints, such as the Resource-

Constraint or EqualityConstraint, are unlikely to change at all, and can thushave expiration dates that lay several months in the future.

Since lookup services define a unique binding between the name and the meaningof a constraint, it is important that lookup services are officially recognized andaccredited by a mutually trusted third party, as constraints are a core part of thecontract between provider and consumer when a match is accepted. This extra


Figure 4.6: Using a constraint lookup service to determine constraint semantics

third party (which could be the market, the lookup service or another broker), canthen be trusted to objectively verify whether the contract is met by both parties ornot. This includes making sure the consumer has paid for the resources, as well asverifying that provided resources actually meet the constraints that were specifiedin the consumer’s request3.

Note that the concept of the lookup service does not necessarily need to existas a webservice; as long as there is a way for providers and consumers to share thesemantics of particular constraints. In many cases it is thus sufficient that a providerstates the meaning of a certain constraint on his website. A good example of such aconstraint would be the ECUConstraint that can be used to model Elastic ComputeUnits.

In order to lower the entry-barrier for providers (this is especially important ina bootstrap phase), a standard set of the most common constraints can be providedas a software library. This way, providers will not need to download code from thelookup service or implement the constraints themselves based on the lookup service’sdefinition. If providers internally model their offers using tag and constraint sets,

3Note that doing so is actually non trivial as it involves thoroughly examining and monitoringthe allocated resources.


they can even use the software library to determine whether a given request matchesone of their offers or not.

Additionally, providing such a library will limit the proliferation of constraintsthat have similar or the same semantics. The market should also embrace a defaultlookup service that contains the definition of all the constraint operations in thesoftware library as a backup for customers and providers that do not have directaccess to the software library.

Using the reverse DNS naming convention that is used to identify constraints andreflection it is easy to implement such a standard constraint library. Figure 4.7 showsthe class diagrams of the command pattern that could be used to implement such alibrary in an elegant way. Specifically, the Constraint interface specifies a validate

method that is used at the provider side to actually validate the constraint. TheMyConstraint diagram represents a concrete constraint implementation (such asthe ResourceConstraint) that is provided as part of the library. When a consumerspecifies a constraint, he will specify the qualified name of the class that implementsthe desired constraint as part of a ConstraintDescription that he attaches to hisrequest. This will make validating the constraint easy for the provider, as he thenonly needs to create an instance of the specified constraint class using reflection, andcall the validate method on the resulting object, passing the consumers tags, thetags used to model the provider’s offer and the constraint parameters as specifiedby the consumer.

Listing 4.2 shows how a consumer would specify such a constraint as part of hisrequest specifically, while 4.3 shows the Java code that the provider can use to verifywhether the constraint is met.

Figure 4.7: Implementing a standard constraint library using reflection

1 ConsumerRequest request = new ConsumerRequest ();

2 Tag tag = new Tag("Memory");

3 tag.setProperty("unit", "GB");

4 tag.setProperty("value", 1.7);

5 ConstraintParameters params = new ConstraintParameters ();

6 params.addParameter("tag","Memory");

7 ConstraintDescription conDescr = new ConstraintDescription ();


8 constr.setConstraintName(ResourceConstraint. c la s s .getName ());9 conDescr.setParameters(params);

10 request.attachConstraint(conDescr);

11 / / a t t a c h o t h e r c o n s t r a i n t s

12 market.submitRequest(request);

Listing 4.2: Consumer-side code specifying a Memory Tag and ResourceConstraint

as part of a request

1 / / t h e v a r i a b l e c o n s t r a i n t D e s c r i p t i o n i s a c o n s t r a i n t f r o m

t h e c o n s u m e r ’ s r e q u e s t

2 Class <?> constraintClass = Class.forName(

constraintDescription.getName ());

3 Constraint constraint = (Constraint) constraintClass.

newInstance ();

4 / / t h e v a r i a b l e o f f e r i s t h e p r o v i d e r ’ s o f f e r

5 Set <ConstraintViolation > violations =

6 constraint.validate(request.getTags (), offer.getTags (),

constraintDescription.getConstraintParameters ());

7 i f (violations.isEmpty ()) {

8 / / c o n s t r a i n t m e t

9 } e l se {

10 / / c o n s t r a i n t n o t m e t

11 }

Listing 4.3: Provider-side code that verifies whether a constraint is fulfilled

Although using constraints with well specified semantics is the only real require-ment for market participants, it is quite unlikely that many matches will occur ifthe tags that consumers and providers use to model resources, requirements, servicesand features are not standardized. That is, most (if not all) constraints require thatthere are tags with specific names or attributes in the tag set that accompanies theconstraint. For example, when using the ResourceConstraint, a tag with a specificname that has value and unit attributes is required in both the consumer and theprovider tag set in order for a match to occur.

To address this, lookup services should also define tags (names and associatedattributes) besides constraint semantics. The Reverse DNS convention can again beused to avoid name clashes. The standard constraint library should also be extendedto include set of standard tags (and the default lookup service should consequentlyalso contain these tags). Figure 4.8 shows class diagrams of some of the tags thatare part of the standard library.

Implementing custom constraintsBesides the standardized constraints, consumers also have the possibility to specifycustom constraints that allow them to express specific matching behavior that isnot available as part of the standard library or in any of the lookup services. Todo this, the consumer should setup a webservice (or use a broker that does this forhim) that is able to check whether a given constraint is met given a tag set that was


Figure 4.8: Standardized tags in the standardized constraint library

Figure 4.9: Implementation of custom constraints using webservices

4.3. IMPLEMENTATION OF THE PROTOTYPE 113

specified by the consumer and a tag set that describes a provider’s offer. Figure 4.9shows how such a system would work.

When the consumer specifies a customConstraint as part of his request, heshould also specify an URL at which a webservice can be found that is able to verifywhether the particular constraint is met. When the provider encounters such a con-straint, he should then make a webservice call to the standardized CheckConstraint

interface at the specified URL, passing the name of the custom constraint, the con-sumer specified tag set, a tag set representing the offer against which the provideris validating the particular request and any additional ConstraintParameters thatwere specified by the consumer. Given this information, the webservice should beable to determine whether the constraint is met or not and return a boolean responseto the provider.

Note that from an efficiency standpoint, it is more interesting that the providerfirst verifies all standardized constraints as no time consuming webservice calls areneeded to check these constraints. When the provider finds out that one of thestandard constraints is violated (and that a specific offer can thus not be matchedwith the consumer’s request), there is no longer a need to check the other constraints.

If it turns out that certain custom constraints are often used by consumers, it isobviously beneficiary to add an implementation of the constraint directly to a lookupservice so that it can be downloaded by providers and consumers. Note however,that there may be situations in which the verification of the constraint involvesthe usage of proprietary code or contacting additional web services or databases.Submitting such code to a lookup service may not be desirable or even possible. Inthese cases, the use of the webservice-based custom constraint mechanism based isan appropriate alternative.

4.3 Implementation of the prototype

In order to prove the real-world applicability of the Continuous Revere Auction,a prototype was implemented as part of this thesis. The prototype implementsthe CRA-based market using the design from Figure 4.1, effectively implementingthe ConsumerIaaSMarket, ProviderIaaSMarket, MarketAdminstration and Bill-

board web services according to the interfaces as defined by Figure 4.3. The pricemodels of four different providers (Amazon, GoGrid, CloudSigma, Rackspace) wereimplemented in order to show that the CRA can indeed cope with the heterogeneityin resource specification and price models of different cloud providers. Web appli-cations were build to interact with the various web services. Screenshots of some ofthese web applications are shown in Figures 4.11 and 4.12.

To do the actual implementation, a variety of technologies - listed in Table 4.1 -


were used. A real-world deployment was made on Amazon’s EC2 platform, using 4small instances, as shown in figure 4.10.

Implementing a full production-level market is of course a far more vast andcomplex endeavor as it involves the implementation of many additional subsystemssuch as user management and security, accounting, contract enforcement and others.To implement such a system, a whole developer team, as well as far more elaboratesystem design is needed. As such, these components were not implemented in theprototype.

Additionally, some more advanced functionality such as compound requests,splittable requests, custom constraints and some allocation types have also not beenimplemented as part of the prototype as doing so is largely an implementation ef-fort4 that does not directly contribute to the notion that the CRA mechanism canindeed be implemented in practice.

The complete source code of the prototype can be found at http://jorisroovers.com/files/IaaSMarketPrototype.zip.

Figure 4.10: Prototype deployment

4Although implementing some of this functionality is actually non-trivial.

http://jorisroovers.com/files/IaaSMarketPrototype.zip

http://jorisroovers.com/files/IaaSMarketPrototype.zip


Technology Explanation

Java EE The Java programming language was chosen due theauthor’s personal experience and affinity with the lan-guage. Furthermore, as Java is in widespread use, de-ployment on different platforms is easy.

JAX-WS The JAX-WS library allows developers to expose reg-ular Java classes as web services by adding a few an-notations. JAX-WS effectively hides all complexityinvolved with interpreting WSDL, as well as marshal-ing SOAP messages.

Spring Framework The Spring framework provides developers with ex-tensions for building web applications on top of theJava EE platform more easily. Specifically, the Model-View-Controller model, dependency injection libraryand some of web service libraries were used in the pro-totype.

HTML, CSS, JSP, JSTL Various technologies that can be used for the imple-mentation of the representational aspect of websites.JSP is the server-side technology that can bring to-gether front-end markup and backend business logic.

Javascript Javascript (AJAX) allows asynchronous communica-tion between a website front-end and backend, allow-ing the server to interaction asynchronously with theuser when it is performing time-consuming tasks. Thisis used in the prototype to notify the consumer of theprogress of the matching process.

EAR The Enterprise ARchive allows the deployment of mul-tiple web services (packaged in Web ARchives) as asingle application. The prototype uses EARs to de-ploy the various grouped web services as single appli-cations.

Glassfish application server Java EE Application server used to host the differentEARs.

Table 4.1: Technologies used to implement the prototype


Figure 4.11: Screenshot of the resource specification user interface as offered by abroker to a consumer

Figure 4.12: Screenshot of the user interface offered to administrators to monitorand manage the market

CHAPTER 5

Conclusion

This final chapter summarizes the previous chapters and puts forward the core con-cepts and idea that were introduced in this thesis. To finish, it puts forward somefuture work and formulates a final conclusion.

5.1 Summary

This thesis started with an introduction to the topic of cloud computing whichis materializing the long-lived dream of a general-purpose utility-like computingplatform that can be accessed from anywhere at anytime. To do so, cloud computingintroduces core concepts from existing utilities such as quick on-demand provisioningand the pay-what-you-use model to allow users to focus on their core activities whileleaving the task to build and maintain IT solutions to expert companies. Duringthe study of cloud computing in the first chapter of this thesis, it was establishedthat the most hardware oriented layer of cloud computing, offered by the so-calledInfrastructure-as-a-Service solutions, is the one that most resembles existing utilities;power utilities in particular. From this observation it became apparent that in orderto have a genuine global computing utility, there is a need for an open market inwhich consumers and providers, as well as providers among each other, have thepossibility to trade computing resources on an immediate basis. Although attemptshave recently been made to introduce such a market (most notably [43]), currentsolutions are still in its infancy and require providers to fit strict service and pricemodels as well as implement specific APIs or use particular frameworks. Giventhe simplicity and imposed restrictions of these products, the question still remainswhether it is feasible to create a truly open and accessible IaaS market. As such, thecore question of this thesis has consequently been how such an open market, that isable to cope with the heterogeneity of the current IaaS landscape, can be built.

5.1. SUMMARY 118

In order to tackle this question appropriately, the second chapter looked furtherinto the principles behind market design. After looking into the different kindsof markets, existing auction mechanisms as well as theoretically desirable marketproperties were discussed. Towards the end of the second chapter, focus turnedto the design of commodity markets in particular; identifying core components,functionality and responsibilities. During this discussion, special attention was givento how these components and functionalities are applicable when designing an openIaaS market.

In the third chapter the two most important of these components, the market’sbidding language and matching mechanism, were discussed in further detail. To dothis, the author started from a Continuous Double Auction-based market for gridcomputing resources introduced by Nicolas Dube in [39]. In Dube’s model, so-calledresource sets are used by consumers and providers to describe the hardware andsoftware components they require or offer. Based on the observation that in theIaaS cloud computing setting, a more generic constraint mechanism is needed asin the commercial cloud context many other factors such as price and quality ofservice play a key role, the concepts of tag and constraint sets were introduced. Byusing tag and constraint sets, both consumers and providers have the possibilityto express complex constraints on the resources they are requesting or offering.This is achieved by separating the specification of a matching constraint on anoffer from the properties that characterize that offer. Specifically, so-called tagsthat contain attributes (key-value pairs) are used to describe a given offer, whileadditional constraint operations attached to the offer specify matching restrictionsbased on these tags.

Using this new tag and constraint mechanism, a market based on the ContinuousDouble Auction was presented. Although this CDA based market is simple andelegant, studying it in more detail revealed that because of the heterogeneity of theprice models used by IaaS providers today, the CDA mechanism cannot be usedwithout making considerable assumptions and forcing providers in a tight serviceand price model. As one of the goals of this thesis was to avoid forcing consumersor providers to adhere to a strict set of rules (based on the observation that ifan IaaS market is to be successful, it needs to accommodate current practices), adifferent solutions was needed. In the search for a solution, a new kind of auctionmechanism - the Continuous Reverse Auction (CRA) - was introduced, which appliesprinciples from the Reverse Auction to commodity markets by modifying its behaviorto cope with both multiple consumers and providers. By letting providers determinethemselves whether they wish to match a certain consumer bid, the CRA takes awaythis computational intensive task from the market and allows providers to use customprice models. Matching bids is greatly simplified by the use of the tag and constraintsets that were introduced before.

The issue of price discovery is solved by the CRA by the non-obligatory char-acter of the mechanism from a consumer point of view and by the introduction of

5.2. CONCLUSION 119

a billboard to allow providers to monitor prices for specific resources more closely.When studying the Continuous Reverse Auction in more detail, it became clear thatthe mechanism can also be used in other commodity markets that deal with com-plex heterogeneous goods and price models. Also, from a first examination, it seemsthat the CRA (partially) holds some desirable market properties such as individ-ual rationality, maximum allocative efficiency, budget balance and computationaltractability. However, more in-depth work is needed to verify this.

The one but last chapter of this thesis dealt with the more practical side ofbuilding an IaaS market, by considering how the previously introduced concepts canbe molded into a software architecture. It was explained that a lookup service willbe needed to easily share the semantics of tags and constraints among providers andconsumers. Additionally, it was argued that a software library containing a set ofstandard tags and constraints is desirable for efficiency and consistency reasons. Toimplement the market itself, it was pointed out that several different components areneeded. Interfaces for these components were defined and discussed in detail. Extraattention was given to the (simple) implementation of the CRA matching algorithmand to the issues concerning the lack of standardization in APIs to access acquiredresources.

To proof the real-world applicability of the tag and constraint sets as well asthe CRA based market, a web-service based prototype was implemented in the Javaprogramming language with the help of the Spring Library. In this prototype, theoffers and pricing models of different providers were implemented to show that theproposed solutions are indeed capable of dealing with the heterogeneity of currentIaaS solutions.

5.2 Conclusion

Based on the previous summary, the main conclusion of this work is that it is possibleto design a market for IaaS Cloud resources. However, to deal with the heterogeneityof the current IaaS landscape, a complex bidding language and market mechanismis needed. To this end, this thesis argues that the (Continuous) Double Auctionlacks the possibility to deal with this heterogeneity and proposed the ContinuousReverse Auction combined with a bidding language based on the concept of tag andconstraint sets as a solution.

Future work may include an in-depth study of the CRA mechanism to validateits ideas and possible significance. To this extent, the use of simulations can prob-ably provide valuable insights. Furthermore, the study of IaaS markets that areoriented towards specific use-cases such as high-performance or scientific computingcan be interesting as in these use-cases many assumptions can be made. Dube’sthesis already proved that given a simple price model in which only a single pricecomponent needs to be taken into account, it is possible to create a CDA-based

5.2. CONCLUSION 120

market for grid resources. Based on this, the author believes that using constraintsets, use-case specific CDA-based IaaS markets can be build in which the use of onlyuse a single price component is justifiable.

Additional future work may include the impact of allowing consumer to submitprices to the CRA mechanism as discussed in section 3.3.1 and the research of ad-vanced price discovery techniques as discussed in the section about Decision SupportSystems in the context of the CRA (section 3.3.3).

In the end, this thesis is an attempt to contribute to the development of a globalplatform in which providers and consumers come together to trade the abstractnotion of computing resources. Although the idea of a market in which IaaS resourcescan be traded is a natural evolution in the vision of cloud computing itself, actuallybuilding such a market turns out to be a complex task. The author hopes that thisthesis may contain some insights and ideas that can help to undertake this task, asto contribute to the realization of a global utility-like computing model.

Appendices

APPENDIX A

Cloud Computing in Practice: SaaS and PaaS

This appendix contains some examples of real-world Software-as-a-Service (SaaS)and Platform-as-a-Service (PaaS) products.

A.1 Software-as-a-Service

There are a plethora of SaaS applications on the Internet, ranging from personalphoto-sharing services such as SmugMug [71] to more enterprise oriented applica-tions such as the HreForce [72] HRM application. In order to be brief, only 2well-known SaaS suites are discussed here; a more complete listing of different SaaSapplications can be found online [73].

A.1.1 Google Apps

Google Apps [74] is a suite of several different online office applications. The mostprominent are listed here.

Gmail Complex online email application.

Google Calendar Online calendar application.

Google Docs Online word-processor, spreadsheet editor, powerpoint-editor andfile storage facilties.

Google Sites Online tool to quickly build and integrate simple webpages.

Google Reader Web-based aggregator, capable of reading Atom and RSS feeds.

Picasa Web Albums Online photo-sharing service.

Google Adwords Online advertisement tools.

A.2. PLATFORM-AS-A-SERVICE 123

Users can choose to either make use of the Google-branded versions of theseapplications that have advertisements, or choose to use custom branded versionsthat are hosted by Google but can be tightly integrated into a corporate website.A free tier with a limited amount of accounts and support exist for the corporate-version of Google Apps, but is mainly focused on very small businesses. Largercorporations pay $50 per user per year, and receive more storage, better securityand SLAs (99.9 percent uptime) for that price.

As of 2010, Google also has an Apps Marketplace [75] on which it allows inde-pendent developers and software vendors to offer free or subscription-based onlineapplications that integrate with the Google Apps application suite.

A.1.2 Salesforce

Salesforce [76] is well-known for its online Customer Relationship Management(CRM) products that include marketing, sales, analytics and service & supportapplications.

Sales Cloud The Sales Cloud application provides a wide set of tools that allowssales representatives to track customer profiles and account history, allowsthem to manage marketing campaigns and track other information unique tothe representative’s company sales process.

Service Cloud The Service Cloud application is a customer support applicationthat provides companies with a call center-like view that allows them to createand track cases and automatically route and escalate important information.

Chatter Chatter is a real-time collaboration platform, dubbed by Salesforce as“Facebook for the enterprise”, that allows employees to access a secure, privatesocial environment.

JigSaw Jigsaw is a community-driven contacts database that allows Salesforce cus-tomers to quickly find full and correct contact information about each other.

Salesforce uses a subscription-based pricing model for all of its service that re-quires customers to pay a certain fee per user per month. The exact prices dependon the which of the different editions of the services is used; more expensive editionsoffer more features and increased reliability and support. Details can be found online[77].

A.2 Platform-as-a-Service

Although the number of SaaS products is seemingly endless, PaaS products are alot more scarce; only a handful large IT vendors offer mature PaaS solutions today.Two of those, offered Google and Microsoft, are discussed here.


A.2.1 Google App Engine

Google App Engine (GAE) [78] is a platform for developing and hosting web appli-cations on top of Google-managed data centers. In practice, developers can writeapplications in the Java or Python programming languages which are subsequentlyuploaded to Google servers. Once uploaded, developers do not need to worry aboutthe deployment of their applications; provisioning, scaling and loadbalancing is takencare of by Google itself. Besides these services, Google also provides developers witha set of software libraries and tools that allows them to simplify common tasks. Themost important of these tools are listed here.

Google Web Toolkit Java-based development toolkit for building and optimizingcomplex browser-based applications.

App Engine Memcache Fast, transient distributed storage for caching the resultsof datastore queries and calculations.

App Engine URL Fetch Allows access to resources over the web, and to com-municate with other hosts using the HTTP and HTTPS protocols.

App Engine Images The Images service lets applications transform and manip-ulate image data in several formats.

App Engine Mail Allows App Engine applications to send email messages on be-half of the app’s administrators, and on behalf of users with Google accounts.

App Engine XMPP Allows App Engine applications to send and receive instantmessages to and from users of XMPP-compatible instant message services.

Google provides a rather substantial free tier for App Engine usage (exact quotasdepend on the used services, details can be found online [79]), but once more than thefree quota is used, Google charges the prices according to the amount of bandwidthused, data stored and CPU time used by their servers. Details can be found on theAppEngine billing webpage [80].

A.2.2 Microsoft Azure Services

Microsoft Azure Services Platform [81] is an online software platform that allowsdevelopers to run and host applications in Microsoft’s datacenters. Although someof the services that are delivered can arguably also be categorized as IaaS solutions,Microsoft markets the Azure product as a PaaS because of its tight integration withother Microsoft (cloud) products. More concretely, on the lowest level Microsoftprovides developers with a “cloud operating system” called Windows Azure thatserves as a runtime for applications that can be written in any of the programminglanguages supported by the NET framework [82] or a few other programming lan-guages such as Java, Ruby or PHP. More advanced functionality can be obtainedby making use of the AppFabric middleware that provides pre-built, higher levelservices by means of a pre-built class library, simplifying the development of cloud


applications. These services include advanced access control services, caching ser-vices, messaging services and integration with the Microsoft Bizztalk server [83] thatprovides Enterprise Application Integration (EAI) features. Additionally, applica-tions can integrate with the SQL Azure Database service which is a cloud computingoriented online database service based on Microsoft SQL Server technology [84]. Onan even higher level, Microsoft allows developers to integrate their applications withsome of the SaaS products Microsoft offers as part of their Windows Live brand,which includes online photo, email, contact and office applications.

For billing, Microsoft uses a model that is similar to that of IaaS providers bycharging developers by the hour depending on the instance type they choose to runthe Windows Azure operating system on (see section 1.2). Additional charges forbandwidth, storage and the usage of the AppFabric and SQL Azure services alsoapply. Details can be found online [85].

APPENDIX B

Compound Requests

Besides the normal requests, the market should also have a mechanism that allowsfor a different kind of request; the so-called compound request. The compoundrequest is a consumer request that is composed of several normal requests and canbe used in cases where the usage of the requested resources can be partitioned intoseveral distinct phases or stages. For example, it is quite common (especially withscientific workloads) that the time used by cloud resources can be partitioned intofetching input data, subsequently processing that input data, and finally writingback the output data. Similarly, there are applications that only periodically needto transfer a lot of data, while transmitting very few to no data at other moments.This occurs most typically in scenarios where periodical backups are taken.

In other words, there are multiple applications that exhibit this behavior ofhaving different distinct workload profiles at different points in time. Although suchprofiles are typically characterized by being either bandwidth- or compute-focused,there are certainly different scenarios possible in which e.g. an application mightperiodically need a lot of memory or CPU cycles to perform heavy tasks, while mostof the time only needing a fraction of those resources.

It is clear that by only using the normal requests that were defined before, ex-pressing such behavior is impossible. However, giving consumers a mechanism to doso, can have distinct advantages for both themselves as for the cloud providers sup-plying the resources. To better understand this, consider an example of a scientificworkload that exists of 3 distinct stages:

1. Transfer 30 GB of raw data to the cloud.

2. Process data in the cloud. No simultaneous data-transfer (except for occasionalheart-beat message), at least 10 computing hours necessary on requested re-sources.

127

3. Transfer 1GB of result data back.

When trying to specify such a workload using a normal request, it would beimpossible to express the fact that the bulk of the datatransfer needs to happen atthe start of the task. Instead, one would have to specify that the 30GB of inbounddata would be transferred across the whole (10+ hours) of processing. Additionally,all the requested compute resources would have to be present from in the beginning,while the consumer would only be using them after all the input data has beenreceived (and until the result data is sent back).

The main disadvantage of this discrepancy between allocating resources andactually using them is that the consumers will need to pay for the allocation of all theresources, even if they are not currently using them. Although this is partially solvedIaaS solutions by the usage of a multi-price component model as discussed in section3.2, when the difference between the allocated resources and the resources that arebeing used is too large, there will almost always be a significant financial penaltyassociated with that. Although at first it may also seem that another disadvantageis the fact that it could be that certain prices such as e.g inbound data transferchange during the time the workload is running1, consequently changing the totalcost for the consumer based on when certain resources are used, this is not anissue since a consumer and provider seal a contract about the price at the time ofmatching. In other words, a provider is never allowed to change the price of anyprice component for a given match. Allowing this would nullify the optimality of anymatching mechanism since in such cases, providers can potentially game the systemby initially offering low prices while raising them after a short period of time.

The allocation discrepancy can be solved by using compound requests, as in sucha case, providers are able to allocate resources that match the resources required bythe workload much more closely. In the example above, this could mean that in thefirst and third stage, the consumer would not have to pay for the computing resourcesthat are used in the second stage, while no bandwidth costs will be incurred in thesecond stage. Additionally, as the provider knows that the task in the second stagewill run for at least 10 hours, he might apply prices of reserved resources (instances)for 10 hours, while using an on-demand price model for the following hours. Finally,if the job that is running in second stage would transfer data, it could be priceddifferently than in the first and third stage. The ratio behind this is that from aprovisioning point of view, it is reasonable to expect that (in the future) providersmight apply lower bandwidth rates when they know that a specific amount of dataneeds to be transferred in a small timeframe, then when they know that that datais to be transfered somewhere within a much larger timeframe. Put differently, itis not to unlikely to expect that in time certain providers will use different price

1Such scenarios are not very likely at the time of writing (providers typically use fixed prices perGB), but it is not unthinkable that in the future, more spot like behavior will emerge in which theprices for data-transfer will vary according to current bandwidth demand.

128

models based on the flexibility that is requested by the consumer2.

Specifying Compound RequestsAs far as the bidding language is concerned, a compound request is not that differentfrom a regular request.

Compound Request = [Volume, TagSet*, ConstraintSet]

The main difference with regular requests is however that the tagset may containtags that have attributes that itself are requests. By doing so, sub requests caneffectively be nested inside other requests, and constraints can be specified betweenthese sub requests as part of the ConstraintSet that is part of the top-level request.Good examples of such constraints are for example a RequestDependency constraintthat specifies in which order the resources in the subrequests will have to be allocatedor a DataLocality constraint that specifies that the requests should fulfilled on asingle physical machine (or at least that the local data should be preserved betweenswitching between resources of sub requests).

To fully understand how this could be done, consider the the compound requestthat specifies the previously mentioned three-phase application.

Compound Request = [1,

{[SubRequests,

{(SubRequest1,

[1, { [Memory, {(value, 1500), (unit, 1)}],

[TimeUsage, {(unit, hour), (value, 1)}],

[IncomingBandwidthUsage, {(unit, GB), (value, 30)}]

},

{ [Resource["Memory"], EqualAll["TimeUsage"] }

]

),

(SubRequest2,



[MinDuration, {(unit, hour), (value, 10)}]

},

{ [Resource["Memory"], EqualAll["TimeUsage"],

Resource["MinDuration"] }

]

),

(SubRequest3,



[OutgoingBandwidthUsage, {(unit, GB), (value, 1)}]

2Actually, this already happens today. Many providers charge more expensive prices for theallocation of on-demand resources compared to the allocation of future or reserved resources.

129

},

{ [Resource["Memory"], EqualAll["TimeUsage"]}

]

)

}

]},

{ RequestDependency["SubRequest2", "SubRequest1"],

RequestDependency["SubRequest3", "SubRequest2"],

DataLocality["SubRequest1", "SubRequest2"],

DataLocality["SubRequest2", "SubRequest3"]

}

]

Note that no allocation type has been specified in the different subrequests, whichmeans that the provider can decide itself which kind of resources to allocate (this caneven be Spot !)3. The second request does contain a MinDuration request specifyingthe minimum duration the consumer will be using the resources. As already noted,the use of the RequestDependency and DataLocality ensure the desired behaviorof the resources once they are acquired.

Provider matching behavior should be slightly adjusted as to recognize sub re-quests within requests and match them first before trying to match the entire re-quest. However, besides this minor adjustment, no real changes are needed in themarket itself. As such, the ability to nest requests inside of other requests usingtags, as well as the possibility to define relations between these nested request usingconstraint sets, shows once again that the introduction of the tag and constraintsets add considerable flexibility to the market.

Additional remarksThis section contains some additional remarks concerning the use of compoundrequests.

The effects of using compound requests cannot be achieved by using multiplenormal requests. That is, it might seem that by chunking a request in multipledifferent request and matching each of these requests separately, the same resultcan be achieved. This is not the case, as it could be that the market matchesthe different normal requests with different providers as it could very well be thatdifferent providers are cheaper than each other for the different individual request.This is a real problem as only a single workload is considered; it is imperative thatthis workload is matched with a single provider as data locality is almost alwaysimportant. The concept of the compound request clearly expresses the fact thatthese different individual requests should be matched by the same provider and as awhole. When a compound request is matched with a certain provider, it is thus that

3Note that the consumer has not specified any additional bandwidth speed constraints, whichtogether with the lack of an allocation type specification may lead to the fact that the acquiredresources are deallocated after an hour without the data transfer being complete.

provider that offers the cheapest price for that whole compound request, althoughit might be that that provider is more expensive than all other providers for allindividual sub requests.

Despite the fact that implementing compound request as part of the market isnot very difficult, there are some technical difficulties to support the different parts(stages) of the compound request and the transition between them. Most impor-tantly, it is not trivial for cloud providers to provide the possibility to dynamicallychange the allocated resources (vertical scaling), or to only pay for transfered datawithout needing to pay for other cloud resources (instances) that are storing thetransfered data. However, some providers are already making (significant) effortsto allow for such increased flexibility. Indeed, certain cloud providers such as Ama-zon and GoGrid are already offering cloud storage facilities (Amazon S3 [20] andGoGrid Cloud Storage [86]) that allow customers to store data without needing torun instances. Similarly, Amazon allows users to dynamically change instance typesby using Elastic Block Devices [87].

Of course, as was the case before, the market only acts as an intermediate be-tween the consumer and provider. Actually giving the consumer access to the ac-quired cloud resources, helping the consumer setting up his requested environmentand coordination the transition between the different stages is an different matterentirely, and should be handled by separate broker services. Section 4.1 touches thissubject in a little more depth with the introduction of the ExecutionService.

Bibliography

[1] M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. H. Katz, A. Konwinski,G. Lee, D. A. Patterson, A. Rabkin, I. Stoica, and M. Zaharia, “Above theclouds: A berkeley view of cloud computing,” Tech. Rep. UCB/EECS-2009-28,EECS Department, University of California, Berkeley, February 2009.

[2] R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic, “Cloud comput-ing and emerging it platforms: Vision, hype, and reality for delivering comput-ing as the 5th utility,” Future Gener. Comput. Syst., vol. 25, no. 6, pp. 599–616,2009.

[3] “World internet users and population statistics.” http://www.

internetworldstats.com/stats.htm, 2011. Last accessed on 24/03/2011.

[4] R. S. Kaplan and W. Bruns, Accounting and Management: A Field Study Per-spective. Harvard Business School Press, 1987.

[5] D. Nurmi, R. Wolski, C. Grzegorczyk, G. Obertelli, S. Soman, L. Youseff, andD. Zagorodnov, “The eucalyptus open-source cloud-computing system,” in inProceedings of Cloud Computing and Its Applications, (Chicago, Illinois), Oc-tober 2008.

[6] “OpenNebula Virtual Infrastructure Manager.” http://www.opennebula.com.

[7] VMWare, “vCloud.” http://www.vmware.com/products/vcloud/overview.

html, 2009. Last accessed on 24/03/2011.

[8] Environmental Protection Agency, “Report to congress on server and data en-ergy efficiency,” 2007.

[9] Datacenter Knowledge, “Microsoft: PUE of 1.22 for data centercontainers.” http://www.datacenterknowledge.com/archives/2008/10/20/

microsoft-pue-of-122-for-data-center-containers/, 2011.

http://www.internetworldstats.com/stats.htm

http://www.internetworldstats.com/stats.htm

http://www.opennebula.com

http://www.vmware.com/products/vcloud/overview.html

http://www.vmware.com/products/vcloud/overview.html

http://www.datacenterknowledge.com/archives/2008/10/20/microsoft-pue-of-122-for-data-center-containers/

http://www.datacenterknowledge.com/archives/2008/10/20/microsoft-pue-of-122-for-data-center-containers/

[10] Google, “Data center efficiency measurements.” http://www.google.com/

corporate/datacenter/efficiency-measurements.html, 2011.

[11] “Cloud standards overview.” http://cloud-standards.org, 2011.

[12] “OpenStack.” http://www.openstack.org, 2010. Last accessed on18/03/2011.

[13] Amazon, “Online Bookshop.” http://amazon.com, 2011.

[14] Amazon, “Elastic Compute Cloud.” http://aws.amazon.com/ec2, 2008. Lastaccessed on 24/03/2011.

[15] Amazon, “EC2 Instance Types.” http://aws.amazon.com/ec2/

instance-types/, 2011. Last accessed on 24/03/2011.

[16] CloudHarmony, “Cloudharmony, benchmarking in the cloud.” http://www.

cloudharmony.com/. Last accessed on 22.05.2011.

[17] Amazon Web Services LLC, “Amazon EC2 Spot Instances,” 2009. Last accessedon 24/03/2011.

[18] “Cloudexchange.” http://cloudexchange.org/.

[19] Amazon Web Services LLC, “Amazon EC2 Pricing.” http://aws.amazon.com/

ec2/#pricing, 2010. Last accessed on 24/03/2011.

[20] Amazon, “Simple Storage Service.” http://aws.amazon.com/s3, 2008. Lastaccessed on 24/03/2011.

[21] Amazon, “Simple Queue Service.” http://aws.amazon.com/sqs/, 2008. Lastaccessed on 24/03/2011.

[22] Amazon, “SimpleDB.” http://aws.amazon.com/simpledb/, 2008. Last ac-cessed on 24/03/2011.

[23] Apache, “Hadoop.” http://hadoop.apache.org/.

[24] “GoGrid.” http://gogrid.com, 2008. Last accessed on 24/03/2011.

[25] GoGrid, “Cloudhosting.” http://www.gogrid.com/cloud-hosting, 2008.Last accessed on 24/03/2011.

[26] GoGrid, “GoGrid Exchange.” http://exchange.gogrid.com, 2010. Last ac-cessed on 24/03/2011.

[27] I. Foster and C. Kesselman, The Grid: Blueprint for a New Computing Infras-tructure. Morgan Kaufmann Publishers, Inc, first ed., 1998.

[28] I. Foster, Y. Zhao, I. Raicu, and S. Lu, “Cloud computing and grid computing360-degree compared,” in Proceedings of the Grid Computing EnvironmentsWorkshop (GCE 08), 2008.

http://www.google.com/corporate/datacenter/efficiency-measurements.html

http://www.google.com/corporate/datacenter/efficiency-measurements.html

http://cloud-standards.org

http://www.openstack.org

http://amazon.com

http://aws.amazon.com/ec2

http://aws.amazon.com/ec2/instance-types/

http://aws.amazon.com/ec2/instance-types/

http://www.cloudharmony.com/

http://www.cloudharmony.com/

http://cloudexchange.org/

http://aws.amazon.com/ec2/#pricing

http://aws.amazon.com/ec2/#pricing

http://aws.amazon.com/s3

http://aws.amazon.com/sqs/

http://aws.amazon.com/simpledb/

http://hadoop.apache.org/

http://gogrid.com

http://www.gogrid.com/cloud-hosting

http://exchange.gogrid.com

[29] N. Carr, The Big Switch: Rewiring the world, from Edison to Google. NewYork, NY, USA: W.W. Norton, 2008.

[30] M. Chetty and R. Rajkumar Buyya, “Weaving computational grids: How anal-ogous are they with electrical grids,” Computing in Science and Engineering,vol. July/agust, pp. 61–71, 2002.

[31] D. S. Kirschen and G. Strbac, Fundamentals of Power System Economics. WestSussex, England: John Wiley & Sons, Ltd, 2004.

[32] S. Hunt and G. Shuttleworth, Competition and Choice in Electricity. Chich-ester, England: John Wiley & Sons, Ltd, 1996.

[33] L. Leong, “Adopting cloud infrastructure as a service in the ’real world’,” tech.rep., Gartner, 2011.

[34] B. Schnizler, Resource Allocation in the Grid – A Market Engineering Approach.PhD thesis, University of Karlsruhe, 2007.

[35] G. Stuer, K. Vanmechelen, and J. Broeckhove, “A commodity market algorithmfor pricing substitutable grid resources,” Future Generation Computer Systems,vol. 23, no. 5, pp. 688–701, 2007.

[36] K. Vanmechelen, Economic Grid Resource Management using Spot and FuturesMarkets. PhD thesis, University of Antwerp, 2009.

[37] K. Vanmechelen, W. Depoorter, and J. Broeckhove, “Market-based grid re-source co-allocation and reservation for applications with hard deadlines,” Con-currency and Computation: Practice and Experience, vol. 21, p. 22702297, 2009.

[38] J. Broberg, S. Venugopal, and R. Buyya, “Market-oriented grids and utilitycomputing: The state-of-the-art and future directions,” Journal of Grid Com-puting, vol. 6, no. 2, pp. 255–276, 2008.

[39] N. Dube, SuperComputing Futures: the Next Sharing Paradigm for HPC Re-sources. PhD thesis, Universite Laval, 2008.

[40] “GridEcon Project.” http://www.gridecon.eu/, 2006. Last accessed on26/03/2011.

[41] J. Altmann, C. Courcoubetis, G. D. Stamoulis, M. Dramitinos, T. Rayna,M. Risch, and C. Bannink, “GridEcon: A Market Place for Computing Re-sources,” 2006.

[42] C. Courcoubetis, M. Dramitinos, T. Rayna, S. Soursos, and G. D. Stamoulis,“Market mechanisms for trading grid resources,” 2006.

[43] Enomaly, “SpotCloud.” http://www.spotcloud.com, 2011.

http://www.gridecon.eu/

http://www.spotcloud.com

[44] J. Kalagnanam and D. Parkes, “Auctions, bidding and exchange design,” inSupply Chain Analysis in the eBusiness Area (D. Simchi-Levi, S. Wu, andM. Shen, eds.), Int. Series in Operations Research and Management Science,ch. 5, Kluwer Academic Publishers, 2003.

[45] P. Hardwick, B. Khan, and J. Langmead, An Introduction to Modern Eco-nomics. Financial Times / Prentice Hal, 1999.

[46] A. Sullivan and S. M. Sheffrin, Economics: Principles in action. New Jersey:Pearson Prentice Hall, 2003.

[47] R. Bade and M. Parkin, Foundations of Microeconomics. Addison Wesley, 2003.

[48] A. Mas-Colell, M. D. Whinston, and J. R. Green, Microeconomic Theory. Ox-ford University Press, 1995.

[49] D. S. Kirschen and G. Strbac, Fundamentals of Power System Economics,ch. Markets for electrical energy (The Need for a Managed Spot Market),pp. 51–52. West Sussex, England: John Wiley & Sons, Ltd, 2004.

[50] European Energy Exchange. http://www.eex.com/. Last accessed on18/03/2011.

[51] Wikipedia, the free encyclopedia, “Proof of incentive compatibilty of vick-rey auction.” http://en.wikipedia.org/w/index.php?title=Vickrey_

auction&oldid=419008059#Proof_of_optimality_of_truthful_bidding.Permalink (verified correctness), Last accessed on 27/03/2011.

[52] T. Sandholm, “Limitations of the Vickrey auction in computational multiagentsystems,” in Proceedings of the First International Conference on Multi–AgentSystems (V. Lesser, ed.), (Cambridge, MA), pp. 299–306, MIT Press, 1996.

[53] P. Cramton, Y. Shoham, and R. Steinberg, Combinatorial Auctions, ch. TheLovely but Lonely Vickrey Auction. MIT Press, 2006.

[54] M. Silvano and P. Toth, Knapsack Problems: Algorithms and Computer Imple-mentations, ch. Knapsack Problem, pp. 13–15. John Wiley and Sons, 1990.

[55] P. Cramton, Y. Shoham, and R. Steinberg, Combinatorial Auctions. MIT Press,2006.

[56] R. V. Sven de Vries, “Combinatorial auctions: A survey.” December 2000.

[57] R. B. Myerson and M. A. Satterthwaite, “Efficient mechanisms for bilateraltrading,” Journal of Economic Theory, vol. 29, no. 2, pp. 265–281, 1983.

[58] C. Vecchiola, S. Pandey, and R. Buyya, “High-performance cloud computing:A view of scientific applications,” in Proceedings of the 10th International Sym-posium on Pervasive Systems, Algorithms and Networks (I-SPAN 2009, IEEECS Press, USA), 2009.

http://www.eex.com/

http://en.wikipedia.org/w/index.php?title=Vickrey_auction&oldid=419008059#Proof_of_optimality_of_truthful_bidding

http://en.wikipedia.org/w/index.php?title=Vickrey_auction&oldid=419008059#Proof_of_optimality_of_truthful_bidding

[59] S. Parsons, M. Marcinkiewicz, J. Niu, and S. Phelps, “Everything you wantedto know about double auctions but were afraid to (bid or) ask,” tech. rep., inpreparation.

[60] CloudSleuth, “Cloudsleuth, decoding the mysteries of the cloud.” https://

www.cloudsleuth.net. Last accessed on 22/05/2011.

[61] “High-Performance Linpack,” 2006. University of Tennessee.

[62] I. Foster, “Globus toolkit version 4: Software for service-oriented systems,”in Proceedings of the IFIP International Conference on Network and ParallelComputing (H. Jin, D. Reed, and W. Jiang, eds.), vol. 3779 of Lecture Notes inComputer Science, (Heidelberg), pp. 2–13, Springer-Verlag, 2006.

[63] D. Durkee, “Why cloud computing will never be free,” Communications ofACM, vol. 53, pp. 62–69, May 2010.

[64] “NewServers.” http://www.newservers.com/, 2009. Last Accessed on23/04/11.

[65] “CloudSigma.” http://cloudsigma.com, 2009. Last Accessed on 23/04/11.

[66] “Rackspace.” http://rackspace.com, 1998. Last Accessed on 23/04/11.

[67] D. Fudenberg and J. Tirole, Game Theory. Cambridge: MIT Press, 1991.

[68] T. Erl, Service-Oriented Architecture - Concepts, Technology, Architecture.Prentice Hall Service Oriented Computing Series, Upper Saddle River, NJ,USA: Prentice Hall PTR, 6 ed., 2006.

[69] K. Gottschalk, S. Graham, H. Kreger, and J. Snell, “Introduction to web ser-vices architecture,” IBM Systems Journal, vol. 41, no. 2, pp. 170 –177, 2002.

[70] “JAX-WS (JSR 224).” http://jax-ws.java.net. Last accessed on22/05/2011.

[71] “SmugMug,” 2011. Last accessed on 24/03/2011.

[72] “HreForce.” http://www.hreforce.com, 2011. Last accessed on 24/03/2011.

[73] “SaaS Directory,” 2011. Last accessed on 24/03/2011.

[74] Google, “Apps.” http://www.google.com/apps, 2006. Last accessed on24/03/2011.

[75] Google, “Apps MarketPlace.” https://www.google.com/enterprise/

marketplace, 2010. Last accessed on 24/03/2011.

[76] “Salesforce.” http://www.salesforce.com, 1999.

[77] “Salesforce pricing.” http://www.salesforce.com/eu/crm/

editions-pricing.jsp, 2011.

https://www.cloudsleuth.net

https://www.cloudsleuth.net

http://www.newservers.com/

http://cloudsigma.com

http://rackspace.com

http://jax-ws.java.net

http://www.hreforce.com

http://www.google.com/apps

https://www.google.com/enterprise/marketplace

https://www.google.com/enterprise/marketplace

http://www.salesforce.com

http://www.salesforce.com/eu/crm/editions-pricing.jsp

http://www.salesforce.com/eu/crm/editions-pricing.jsp

[78] Google, “Appengine.” http://code.google.com/appengine, 2008. Last ac-cessed = 24/03/2011.

[79] Google, “AppEngine Quotas.” http://code.google.com/appengine/docs/

quotas.html, 2011. Last accessed on 24/03/2011.

[80] Google, “AppEngine Billing.” http://code.google.com/appengine/docs/

billing.html, 2011. Last accessed on 24/03/2011.

[81] Microsoft, “Azure Services.” http://www.microsoft.com/windowsazure,2010. Last accessed on 24/03/2011.

[82] Microsoft, “.NET framework.” http://www.microsoft.com/net, 2002. Lastaccessed on 24/03/2011.

[83] Microsoft, “BizTalk Server.” http://www.microsoft.com/biztalk, 2000. Lastaccessed on 24/03/2011.

[84] Microsoft, “SQL Server.” http://www.microsoft.com/sqlserver, 1989. Lastaccessed on 24/03/2011.

[85] Microsoft, “Azure Services Pricing.” http://www.microsoft.com/

windowsazure/pricing, 2011. Last accessed on 24/03/2011.

[86] GoGrid, “CloudStorage.” http://www.gogrid.com/cloud-hosting/

cloud-storage.php, 2008. Last accessed on 24/03/2011.

[87] Amazon, “Elastic Block Storage.” http://aws.amazon.com/ebs/, 2008. Lastaccessed on 24/03/2011.

http://code.google.com/appengine

http://code.google.com/appengine/docs/quotas.html

http://code.google.com/appengine/docs/quotas.html

http://code.google.com/appengine/docs/billing.html

http://code.google.com/appengine/docs/billing.html

http://www.microsoft.com/windowsazure

http://www.microsoft.com/net

http://www.microsoft.com/biztalk

http://www.microsoft.com/sqlserver

http://www.microsoft.com/windowsazure/pricing

http://www.microsoft.com/windowsazure/pricing

http://www.gogrid.com/cloud-hosting/cloud-storage.php

http://www.gogrid.com/cloud-hosting/cloud-storage.php

http://aws.amazon.com/ebs/

a market design for iaas cloud resources

Documents