research article metricscloud: scaling-up metrics...

Research ArticleMetricsCloud Scaling-Up Metrics Disseminationin Large Organizations

Miroslaw Staron1 and Wilhelm Meding2

1Computer Science and Engineering University of Gothenburg Forskningsgangen 6 412 96 Gothenburg Sweden2Ericsson AB 164 83 Stockholm Sweden

Correspondence should be addressed to Miroslaw Staron miroslawstaronguse

Received 10 September 2014 Revised 17 November 2014 Accepted 18 November 2014 Published 10 December 2014

Academic Editor Phillip A Laplante

Copyright copy 2014 M Staron and W Meding This is an open access article distributed under the Creative Commons AttributionLicense which permits unrestricted use distribution and reproduction in any medium provided the original work is properlycited

The evolving software development practices in modern software development companies often bring in more empowerment tosoftware development teams The empowered teams change the way in which software products and projects are measured andhow the measures are communicated In this paper we address the problem of dissemination of measurement information bydesigning a measurement infrastructure in a cloud environment The described cloud system (MetricsCloud) utilizes file-sharingas the underlying mechanism to disseminate the measurement information at the company Based on the types of measurementsystems identified in this paper MetricsCloud realizes a set of synchronization strategies to fulfill the needs specific for these typesThe results from migrating traditional web-based folder-sharing distribution of measurement systems to the cloud show that thistype of measurement dissemination is flexible reliable and easy to use

1 Introduction

Managing software products and software development pro-jects in large organizations has evolved over the past decadefrom the central top-down planning of waterfall processesto distributed monitoring of empowered agile teams [1] Thepractices of software management have evolved as well fromfollowing the plans to adjusting the plans based on customerneeds [2] Inmodern software development processes it is thesoftware development team that is responsible for planningtheir work in order to deliver customer value (eg new fea-tures) in a (more-or-less) continuousmanner [3 4] From theperspective of software management in general and softwaremeasurement in particular this change requires a change inhow measuring software products and projects is conductedand disseminated in enterprises Using business intelligencetools [5 6] provides managers and productprogram leaderswith the insight into the organization but is usually burdenedwith high maintenance costs [7] The business intelligencetools are often complemented with the so-called informationradiators which are designed to spread the information

throughout enterprises [8] However both manners aremainly used to communicate vertically in the hierarchy of theenterprise (teams-management and management-teams) Inthe context of empowered teams this vertical communicationneeds to be complemented with a horizontal disseminationof measurement information between the teams withoutcausing information chaos or extensive information noise Inthis paperwe address this need by designing an infrastructurebased on the principles of IaaS (Infrastructure as a Service[9]) where empowered teams and individual stakeholderscan share their measurement information without creatinginformation chaos The infrastructure has evolved from anextensive measurement program at one of the developmentunits at Ericsson and addresses its needs for scalabilitydistribution reliability and low maintenance cost [10] Theinfrastructure is namedMetricsCloudMetricsCloud is basedon file synchronization in a controlledmanner specific to thepurposemdashvertical or horizontal dissemination and public orprivate access The infrastructure uses a central storage formeasurement systems and a set of local clients to distributevisualize and update the measurement systems At Ericsson

Hindawi Publishing CorporationAdvances in Soware EngineeringVolume 2014 Article ID 905431 12 pageshttpdxdoiorg1011552014905431

2 Advances in Software Engineering

this infrastructure uses network storage web pages and exe-cution engines based on our previous work [11]

In this paper we address the following research questionHow to increase dissemination of software metrics using cloudtechnologies

Wedescribe howMetricsCloud infrastructure is designedand we provide lessons learned for the companies willing touse cloud solutions internallymdashboth to increase themeasure-ment dissemination and as a means of internal education onhow IaaS cloud systems can be designed We also evaluateMetricsCloud by introducing it for a selected group ofmeasurement systems and interviewing thembefore and afterthe introduction of MetricsCloud

Our results show that cloud-based dissemination in-creases the reliability of the entire measurement infrastruc-ture (replication of information) and seamless availabilityof the measurement infrastructure (scaled-down distributedversion of measurement execution) but increases the need ofsecurity control andmonitoring ofmeasurement systems (forprivate and shared measurement systems)

The paper is structured as follows Section 2 presentsresearch related to our work Section 3 shows how measure-ment systems are designed and used at Ericsson Section 4describes the cloud environment used to increase the dissem-ination of metrics at the company Section 5 shows how thecloud design is realized at Ericsson Section 6 summarizeslessons learned from developing the cloud environmentfor a software development organization Finally Section 7outlines the conclusions and further work

2 Related Work

Pawluk et al [12] described the process of introducing a newcloud solution to a large enterprise The purpose of the cloudis similar to ours and we use their work when designing ourcloud system The current cloud system is an evolution ofthe previous work on ensuring information reliability donetogether with Ericsson [11] In this work we address the prob-lems of ensuring that information is available throughout theenterprise

Yoon et al [14] showed how to establish security in cloudcomputing The security of MetricsCloud is based on similarprinciples but is a simplification of the security policies Allsecurity is based on the enterprise log-in

Vecchiola et al [15] presented a solution based onNET forimplementing cloud systems Despite the similar technologytheir cloud solution does not address the issues specific tomeasurement programs in the enterprise In particular it doesnot address the need of combining IaaS with the executionenvironment (SaaS)This is addressed in the work of Liu et al[16]

Another approach was presented by Zhang and Zhou[17] and their CCOA framework Although a very elaborateframework could be used in our solution we preferred to usea simple approach and focus on the ease of use It is the easeof use and performance that are important for similar cloudsystems as described by Gong et al [18]

Security is an important aspect which has been recog-nized by Pearson et al [19] The MetricsCloud infrastructureimplements themost important security aspects based on theenterprise access control which is aligned with Person et alrsquosapproach

3 Measurement Systems at Ericsson

The organization within Ericsson which we worked closelywith develops large products for the telecommunicationnetworks The size of the organization is several hundredengineers and the size of the projects is up to a few hundredsProjects are increasingly often executed according to theprinciples of agile software development and lean productionsystem referred to as streamline development (SD) withinEricsson [20] In this environment various disciplines areresponsible for larger parts of the process compared totraditional processes design teams (cross-functional teamsresponsible for complete analysis design implementationand testing of particular features of the product) networkverification and integration testing and so forth

The organization uses a number of measurement systemsfor controlling the software development project (per project)described above a number of measurement systems tocontrol the quality of products in field (per product) anda measurement system for monitoring the status of theorganization at the top level

One example of a measurement system for the moni-toring of the status of the organization is the measurementsystem monitoring feature burndown It is updated dailyand is used by project management to control the statusof the development It is implemented using MS Excel andVBA (Visual Basic forApplications) scripts Another exampleis a measurement system for monitoring the velocity ofintegrating new features to the main product code base

All measurement systems are developed using the in-house methods described in [10 21] with the particularemphasis on models for design and deployment of measure-ment systems presented in [22 23]Theneeds of the organiza-tion evolved from metric calculations and presentations (ca8 years before the writing of this paper) to using predictionssimulations early warning systems and handling of vastquantities of data to manage organizations at different levelsand providing information from teams to managementThese needs are addressed by a number of action researchprojects which we conducted in the organization since 2006

The studied organization uses and maintains over 400different measurement systems (over 4000 MS Excel files)and a large number of persons are involved in the decisionprocesses based on the status of measures A graph withthe size of measurement systems at Ericsson is presented inFigure 1

As the figure shows there are a significant number offiles with over 1000 lines of VBA code which indicatesthe complexity of the updates and dissemination of themeasurement systems

Advances in Software Engineering 3

LOC per file9000

8000

7000

6000

5000

4000

3000

2000

1000

0

Figure 1 Size of measurement systemsmeasured in number of linesof VBA code

The measurement systems are at all levels of the organi-zation and include measures for for example

(i) measuring reliability of network products in oper-ation for the manager of the product managementorganization (Figure 3) example measures in thismeasurement system are

(a) product downtime per month in minutes(b) number of nodes in operation

(ii) measuring project status and progress for projectmanagers and team leaders who need to have dailyupdated information about such areas as require-ments coverage in the project test progress and costsexample measures in this measurement system are

(a) number of test cases executed during the currentweek

(b) cost of the project up till the current date(c) development efficiency(d) in-development product quality

(iii) measuring postrelease defect inflow for productman-agers who need to have weekly and monthly reportsabout the number of defects reported from productsin field examples of measures are

(a) number of defects reported from field operationof a product during the last month

(b) predicted number of defects per week

(iv) summarizing status from several projects for depart-ment managers who need to have an overview of thestatus of all projects conducted in the organizationfor example number of projects with all indicatorsgreen

The information is processed as a flow according to thepipes-and-filters architectural style as presented in Figure 2

To keep the measurement systems updated daily (or sev-eral times a day) the metrics team at Ericsson has developedits own infrastructure based on a set of simple scripts Thescripts export raw data from databases executemeasurementsystems and present the indicators for the stakeholders Thisis done (usually) during night and takes a few hours (for allmeasurement systems)

There are multiple advantages of this type of solution forexample

(i) using MS Excel files makes the measurement systemsavailable to all stakeholders (MS Office is a standardsoftware package installed on every computer)

(ii) centralized storage is backed up and kept up to dateby a dedicated metric team

(iii) using MS Sidebar gadgets and web pages is a simpleand effective way to spread the information across thecompany

There are however limitations with this way of workingOne of the main limitations is that the centralization requiresall measurement systems to be maintained by the metricteam Some stakeholders would like to maintain their ownmeasurement systems and only disseminate them usingthe measurement infrastructuremdashsimilar to Amazonrsquos cloudsolution [24]

4 Cloud Systems

Cloud computing [25 26] introduces new paradigms ininformation processing It implements a modern vision ofldquosharingrdquo resources and infrastructure between organiza-tions which is the focus of this paper (there exist other usesfor example information processing However these are notpart of this paper) Cloud systems are usually provided bycommercial vendors which have the right competence andresources to manage cloud systems whereas ldquouser organiza-tionsrdquo can focus on their primary business objectives

41 Cloud-Based Dissemination Patterns In general manycloud-based information dissemination systems use Infras-tructure as a Service (IaaS) where MetricsCloud (describedin Section 5) is the infrastructure to store update andshare measurement systems This approach is described byArmbrust et al [27]

One of the patterns is a simple file-sharing and one of thewidely spread general-purpose file-sharing systems based oncloud is Dropbox [28] which provides users with a rudimen-tary mechanism of synchronizing files across platforms andsharing the files with other users Systems like Dropbox poseno constraints on file types and leave it up to the user to designsharing patterns Dropboxrsquos synchronizationmechanisms aremarket leaders with batched synchronization and advancedclean-up mechanisms [29]

Another pattern is focusing on the information to beshared and is described by Rosenthal et al [30] who present acloud system for biomedical research which uses informationsharing as the main objective regardless of its form (eg filesor data) The pattern distinguishes two elements of a cloud-based dissemination of informationmdashinfrastructureserviceproviders and research consortium management

These patterns are similar toweb-based information shar-ing but have an advantage of replication of information andseamless synchronization Figure 4 presents a comparisonbetween the dissemination of information based on web-systems and cloud-based dissemination


People

Projects

Products

Source files

Source files

Source files

Raw data

Raw data

Raw data

Base measures

Derived measures Indicators

Green

RedYellow

Measurement system

Figure 2 Information flow in a measurement system

ISP

2010-05-02

Product A 803

Product B 1551Product C 000Product D 340

Figure 3 MS Sidebar gadget for visualizing product availability atEricsson

The major differences include (as mentioned before) theseparation of concernsmdashinformation provision and execu-tionstorage of informationThe stakeholders of themeasure-ment systems have access to their information on their clients(either computers or mobile devices) without the need toremember how to access the information over the internet(eg the URL of the server or a web page)

5 MetricsCloud

MetricsCloud addresses the needs of the organization forthe stakeholders (s-i) dissemination of self-managed mea-surement systems (s-ii) possibility to share measurementsystems and (s-iii) obtaining simple measurement executioninfrastructure MetricsCloud also provides benefits to themetric team (m-i) reducing the need to create ldquosimplerdquomeasurement systemsmdashnow done by stakeholders (m-ii)applying identity-based security and (m-iii) reducing theneed to constantly keep-alive theweb serverwith allmeasure-ment systems In the current solution described in Section 3the stakeholder relied on themetrics team to develop updateand disseminate the measurement systems to all partiesnecessary This could cause the metric team to become abottleneck of dissemination of measurement systems in alarge organization or introduce the necessary costs One ofthe intentions with MetricsCloud was to reduce the risk ofthe bottlenecks andor unnecessary costs

As shown in Figure 4 the dissemination separates theconcerns of information provision and executionstorage ofinformationThis separation of concerns is done by designingcloud systems based on layersmdashPallis [25] identifies such

layers in a cloud-based system in general for example plat-form infrastructure In this paper we instantiate three ofthese layers based on the division of responsibility (in theorganization) (i) information product provisioning (ii) exe-cution and information quality and (iii) storage and access aspresented in Figure 5

The top layer contains measurement systems managedindividually by stakeholders of measurement systems whoneed access to information (addressing the needs of s-i s-iiand m-i) The midlayer is the layer of execution and updateof measurement systems and is managed by the dedicatedmetric team The stakeholders of the measurement systemsare not concerned with execution of public measurementsystems but are notified if the measurement systems arenot updated (eg by information quality indicators [11])mdashfulfilling the needs of s-iii m-ii and m-iii Finally the lowestlayer is the standard IT infrastructure of the company withnetwork file servers web servers and client programs whichis managed by the IT department of the company

51 Architecture Thearchitecture of the cloud-based dissem-ination is presented in Figure 6The client is a programwhichprovides the access and synchronization of measurementsystems between the cloud (storage) and the client computersGadgets and web pages are used to provide access to themainserver with the measurement systems [31]

The cloud architecture uses the following servers

(1) storage servermdashprovides network access to all mea-surement systems

(2) execution servermdashupdates measurement systemsThe connection between the servers and the clients ispresented in Section 52

Each measurement system which is part of either thepublic or the private part is an MS Excel file which processesdata from various sources as shown in Figure 2

(1) Execution Environment The measurement systems can beupdatedmanually or automatically and both on the client andserver side The manual update of a measurement system isdone by the stakeholder and then the MetricsCloud is usedto disseminate the results of the stakeholderrsquos measurementprocess They are synchronized from the client to the server


User Clientcomputer

Server IP xxxxxxxx Documents

Internet access

UserClient

computer

Cloud

Documents

Web-based dissemination

Cloud-based dissemination

DocumentsSynchronization

Figure 4 Web-based and cloud-based dissemination patterns

Storage and access

Execution and information quality

Information product provisioning

DB Web serverStorage

Measurement executionsystem (MES)

Measurement system

ExecuteUpdatestatus

VBA IQ

ISP2010-05-02Product A 803Product B 1551Product C 000Product D 340

Figure 5 Layers in cloud infrastructure


Storage server Client

Public

Private

User 1

User 2

Public

User 1

ClientPublic

User 2

Execution server

Figure 6 Cloud infrastructuremdashexecution and storage servers andclients

However it is the automatically updated measurementsystems that are more interesting These need to be updatedand synchronized correctly This is done by a dedicatedprogrammdashmeasurement execution system MESmdashshown inFigure 7

Every public measurement system is executed by MESand synchronized to the clients However the MetricsCloudclient has a built-in simplified version of MES (ie excludedmechanisms for controlling the reliability of the measure-ment program infrastructure)The execution system executesthe private measurement systems if any of the conditions isfulfilled

(i) measurement system is an MS Excel file with amacronamed ldquostart hererdquo

(ii) measurement system is an executable file (exe)

(iii) measurement system is a Ruby script and a Rubyinterpreter is available on the client machine

MES is fail-safe and protects the userrsquos computer systemfrom potentially defective measurement systems It usesmultithreaded execution and monitors problems with theldquoexecution threadrdquomdashif needed it terminates the thread

(2) Size MetricsCloud is realized in C and consists of ca1000 lines of code of the client For the identification of meas-urement systems in the MetricsCloud we use such attributesas (i) file name (ii) stakeholder company identity or (iii)modification time

(3) Security Security in the MetricsCloud uses enterpriseidentities of stakeholders EachMetricsCloud client discoversthe corporate ID of the stakeholders and uses it when com-municating with the storage server The storage server steerswhich users have access to which folders The MetricsCloudclient synchronizes only those measurement systems whichare (i) public (ii) private for the stakeholder and (iii) sharedwith the stakeholder

(4) Energy Optimization One could observe that it is possibleto save the energy as the information is replicated to multiplemachines When the measurement systems are updated theexecution server can be put to sleep and only the access tothe updated files can be retained

52 Operation of MetricsCloud Based on our previous workon measurement systems [21] we have identified a number oftypes of measurement systems

(1) Public these measurement systems are spread acrossthe company Everyone can access thesemeasurementsystems designed by designated stakeholders (egproject managers product managers) An example ofsuch measurement system is status of the total defectbacklog in the projectmdashinformation which should bespread in the organization

(2) Private these measurement systems are designedby individual stakeholder and managed by themthat is updates are done locally The measurementsystems are visible only to the users who managethem An example of such measurement systemis a measurement system for defect backlog for adesigner the measurement system may include itsldquoownrdquo information about the status of the repair of thedefect

(3) Shared these measurement systems are private butthe owner has shared it with another user An exampleis defect backlog for a teammdashthis information shouldbe spread within the team

(4) Locally updated public these measurement systemsare public but they are updated by a single user Theuser can be a stakeholder responsible for inputtingdata to the measurement system For other users themeasurement system is visible as public (like the 1stitem) An example is the annotated feature backlogmdashteams update their information locally but it is spreadfor the entire project

(5) Centrally updated private these measurement sys-tems are private for one stakeholder but are main-tained by the central storage and execution serverThese can be dedicated measurement systems withcompany-sensitive information An example can bemeasurement systemswith budget informationwhichis dedicated to certain group of stakeholders withinthe organization

Each of the abovemeasurement systems requires differentmanagement by theMetricsCloudThesemanagement strate-gies are based on the following synchronization principles(see Figure 8)

(1) Public these measurement systems are alwaysupdated from the storage server to the client

(2) Private these measurement systems are alwaysupdated from the client to the storage server

(3) Shared these measurement systems are alwaysupdated from the owner to dedicated clients

(4) Locally updated public these measurement systemsare updated from the client to the storage server

(5) Centrally updated private these measurement sys-tems are updated from the storage server to thededicated client


Execution environment storage and configuration

Measurement executionsoftware system (MESS)

Web server

Measurement system

Information product

Raw data files

Databases

Products

People

ExecuteUpdate

status

Base measures

Derived measures

Indicators

Green

RedYellow ISP

2010-05-02

Product A 803


Figure 7 Measurement execution system

Storage serverClient

Public

Private

Client 1

Private

Public

Private

Shared Shared

Figure 8 Synchronization of measurement systems

In Figure 8 measurement systems of types 1ndash3 are syn-chronized along the solid lines using the pull mechanismsIt is the MetricsCloud client that is responsible for thesynchronization The measurement systems of types 4 and5 are synchronized along the dotted lines The dotted linesmean that these measurement systems have a file attributesetmdashofflineTheMetricsCloud client recognizes this attributeand based on the location of the file makes the appropriatesynchronization

53 Mapping Types to Existing Solutions The types of themeasurement systems identified in this section can be foundin other solutions (eg business intelligence tools) andsupport different types of communication in the company(as outlined in the Introduction) Table 1 maps the types tocommunication and other solutions

However these mechanisms are not integrated in theother solutions which can hinder the stakeholders (and thusthe organizations) from efficient dissemination Locally usedmetric tools require manual intervention and distributionlists to spread the information Shared measurement systems

Table 1 Mapping of types of measurement systems to communica-tion types and example of existing solutions

Type of MS Communication type Examples of existingsolutions

Public Manag ltmdashgt teams Business intelligence

Public Manag ltmdashgtmanag Business intelligencescorecards

Private Individual MS Excel metrictools [13]

Shared Team ltmdashgt team Business intelligenceManuallyupdated NA MS Excel

for teams require mechanisms for controlling the quality ofthe information and maintenance of the common storage(usually a website)

These limitations are addressed by the MetricsCloud

6 Evaluation

We evaluated the work by conducting interviews with themembers of the metric team at Ericsson

61 Design of Evaluation Before introducing MetricsCloudwe interviewed members of the metrics team and stakehold-ers of measurement systems at Ericsson We asked questionsregarding the following

(i) Howdo you use the current infrastructure for dissem-inating measurement systems

(ii) What are the main positivenegative aspects of thecurrent measurement infrastructure


(iii) How do these positivenegative aspects affect yourdaily work

(iv) Which improvements would you recommend to themeasurement programbased on your experiencewithit

(v) What are the main limitations to the dissemination ofmeasurement systems which stem from the currentinfrastructure

These aspects allowed us to understand what the lim-itations of the infrastructure were both before introducingMetricsCloud into the organization and during evaluationafter the introduction

62 Results

(1) Studying Growth of Measurement Systems As the firststep in the evaluation we collected the statistics aboutthe trends in indicators for one (out of many) softwaredevelopment program at Ericsson The program develops acomplex telecomproduct which is used formobile data trafficcommunications The trend in the number of measures andindicators for this program is presented in Figure 9

In the figure we also added the major drivers for boththe decreasing and the increasing trends Each generationof the measurement system is a milestone in the lifecycle ofthe product (ie a project) The 1st generation of the mea-surement systems was an automated status reporting whichcaused the number of measures and indicators to increaseThe 2nd generation of measurement systems included aprioritization of indicators and wide spread of basemeasuresAdding mechanisms of gadgets (as previously described inFigure 3) caused even wider spread of measures The 3rdgeneration of measurement systems was aligned with othersoftware development programs which reduced the numberof measures but also reduced the information provided bythe measurement systems The major driver for the latestincrease in the number of measures is the fact that softwaredevelopment teams started working with local personalizedmeasures that is measures which are designed and valid onlyfor specific teams or disciplines not for the company or forthe software development projects

The trend described above was common for all softwaredevelopment projects for the studied unit (although theactual numbers varied) Extrapolating this trend gives anoverview ofwhat the situation can look like for the entire unitGiven this trend the centralized measurement infrastructurecould not scale which was addressed by the MetricsCloud

(2) Interviews We conducted interviews with a number ofexperts workingwithmeasurement systems (i)measurementprogram leader with over 7 years of experience in designingimplementing andmanagingmeasurements at the company(ii) measurement system designer with 3 years of experiencein designingmeasurement systems (iii)measurement systemdesigner with 1 year of experience in designing measurementsystems and (iv) randomly selected stakeholder for themeasurement systemswith over 1 year of experience in settingrequirements and using measurement systems

24 8 7 5

5581

161184

13

282

0

50

100

150

200

250

300

1st g

ener

atio

n

2nd

gene

ratio

n

3rd

gene

ratio

n

4th

gene

ratio

n

5th

gene

ratio

n

Number of indicatorsNumber of base and derived measures

Fewtop levelindicators

Addingtrends and

gadgetsProgram status

(based on reporting)

Alignment

programs

Open upfor ldquolocalrsquorsquo

metrics

with other

Figure 9 Growing number of indicators in one of the softwaredevelopment programs at Ericsson

When discussing the positive and negative aspects ofweb-based dissemination of information the team has recog-nized the following issues

(i) positive aspects

(a) open access to measurement information forall stakeholders in the organization the publicmeasurement systems were accessible to allemployees such openness stimulates bench-marking comparison and sharing good prac-tices across the company

(b) centrally maintained execution environmentthanks to in-house developed execution soft-ware a small number of engineers of the metricteam could execute and monitor large numberof files of measurement systems

(c) common access to files all measurement sys-tems were stored centrally and accessed via webpages

(d) common naming conventions which simplifiedcataloguing of measurement systems

(ii) negative aspects

(a) maintenance and development of measurementsystems with limited resources in the metricteam posed a limitation for the organization thecomplexity of the infrastructuremade it difficultfor the stakeholders to easily place their ownmeasurement systems in the common storagea simpler solution would allow the stakeholdersto manage their own measurement systems andldquopushrdquo them to share with others as public

(b) rigid data flowmade the infrastructure viable tofailure propagation


(c) the main limitation of the current measurementinfrastructure was perceived to be the depen-dency onMSExcel almost allmeasurement sys-tems were MS Excel files (the minority includedRubyPython scripts or PHP pages)

The interviews after introducing MetricsCloud showedthe following


(a) seamless updates the measurement systemswere synchronized seamlessly with simple noti-fications which helped the stakeholders to focuson their business objectives rather than access-ing the measurement systems

(b) the open access was kept and the stakeholdersgot a simple and immediate access to the mea-surement systems

(c) complementing the centrally managed mea-surement system with the scaled-down execu-tion of private measurement systems on stake-holderrsquos computers allows the stakeholder totake full control of the update of measurementsystems (if needed)

(d) common access to files all measurement sys-tems were stored centrally and accessed via webpages

(e) common naming conventions simplified cata-loguing of measurement systems

(f) the stakeholders were able to add their ownmeasurement systems and share them withothers

(g) the metric team could potentially reduce theeffort of keeping the web-based infrastructureavailable all the time as the information was stillavailable on stakeholderrsquos computers (throughthe synchronization) short breaks in availabilitywere no longer as problematic to the organiza-tion as before


(a) new communication patterns since the cloud-based dissemination was a new approach thestakeholder interviewed was concerned aboutthe security aspects of the disseminated infor-mation for example the need for solutions as in[19] this was identified as the further work forMetricsCloud

(b) failure propagation the new dissemination pat-terns require new models of handling of infor-mation quality of measurement systems Dis-semination of information to clients needs tobe preceded with ensuring of quality in a morestrict manner Since the synchronization is con-trolled by distributed clients the central infras-tructure has to ensure that the information tobe distributed has the right quality for example[11]

Management

Teams Teams

Stand-ups

ScreensInfo radiators

StatusStatus

Excel

Web pages

Figure 10 Channels for disseminating metrics in the organization

Based on the evaluation we concluded that using cloud-based systems for dissemination of information separates theconcerns of stakeholders (information) and the metric team(execution and infrastructure) This increases the involve-ment of stakeholders in creating their own measurementsystems and spreading them in the organization using thecommon infrastructure This type of dissemination reducedalso the bottlenecks in creating and managing measurementsystems by the metric team The metric team could focus onmore advanced measurement systems

Based on the evaluation we have also defined types ofmeasurement systems based on the metrics disseminationpatterns presented in Figure 10

(3) Access Statistics One aspect of the evaluation of the impor-tance of the cloud-based solution is the measurement of howoften files are accessed For this purpose we use the statisticsof number of fetches of measurement systems which iscollected by the measurement infrastructure at EricssonThestatistics is checked for those measurement systems whichcontain indicators and are available publicly (ie not theprivate and not the sharedmeasurement systems)We select asample of 180 measurement systems which are representativefor the entire set of measurement systems and which presentdifferent type of access patterns The statistics were collectedover a period of 28 weeks

Figure 11 presents a histogram of access statistics forthe sample of 180 measurement systems The majority ofmeasurement systems are accessed between 0 and 10 timesin the period but there are measurement systems whichare accessed more often (including a measurement systemaccessed over 260 times during the period of 28 weeks)Due to confidentiality agreements with our industrial partnerwe changed the names of the measurement systems toldquomeasurement system XXrdquo

The total number of fetches for these measurementsystems shows only one side of the complexity the frequencyof access per week is the other side Figure 12 shows the accessstatistics per measurement systems as a heatmap [32] Inthe heatmap each row represents one measurement systemand each column represents one week The intensity of thecolor shows how many times the measurement system wasaccessed during the week The figure shows an excerpt of 180


123

2817

9 6 2 2 3 1 1 1 1 1 1 1 1 1 10

20

40

60

80

100

120

140

280

270

260

250

240

230

220

210

200

190

180

170

160

150

140

130

120

110

100

90

80

70

60

50

40

30

20

10

Figure 11 Access statistics for measurement systems

measurement systems and contains measurement systemswhich are accessed regularly on a weekly basis (horizontallines in the diagram eg measurement system 4) Thismeasurement system measures the status of the progress offeature implementation It is updated daily and provides aninsight into the development status of each feature of theproduct

Another pattern is illustrated by measurement system15 This measurement system measures the value flow inthe organization and is accessed periodically as this piece ofinformation is needed before specific periodicmeetings at thecompany Before these meetings the measurement system isused intensively and there are periods when it is not accessed(white spots in the line) but it is still updated

Measurement system 1 is another example of usagepatternmdashusing measurement system for a specific purposefor a limited time This measurement system measures thetest progress and defect status for one specific release Sincethe release is developed during a defined period of timethis measurement system is of interest to stakeholders mostlyduring that time

63 Threats to Validity To evaluate the validity of our eval-uation we follow the recommendations by Wohlin et al [33]

The main external validity threat stems from the factthat we used a single organization to evaluate the approachHowever since the organization ismature in itsmeasurementprograms and use of measurement infrastructure we believethat the results are generalizable to other mature organiza-tions The content of the feedback obtained showed such amaturity

The main internal validity threat is related to the fact thatwe conducted an action research project [34] In these kindsof projects the formulation of research questions can be proneto organizational context In order to minimize the threatwe have studied other types of systems based on the cloudtechnology and compared their requirementsfunctionalitywithMetricsCloud from the perspective of how general theseare

The main conclusion validity threat is the fact that wehave used interviews as a means of collecting feedbackon MetricsCloud as a dissemination technique In order

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

Measurement system 1Measurement system 2Measurement system 3Measurement system 4Measurement system 5Measurement system 6Measurement system 7Measurement system 8Measurement system 9Measurement system 10Measurement system 11Measurement system 12Measurement system 13Measurement system 14Measurement system 15Measurement system 16Measurement system 17Measurement system 18Measurement system 19Measurement system 20Measurement system 21Measurement system 22Measurement system 23Measurement system 24Measurement system 25Measurement system 26Measurement system 27Measurement system 28Measurement system 29Measurement system 30Measurement system 31Measurement system 32Measurement system 33Measurement system 34Measurement system 35Measurement system 36Measurement system 37Measurement system 38Measurement system 39Measurement system 40Measurement system 41Measurement system 42Measurement system 43Measurement system 44Measurement system 45Measurement system 46Measurement system 47Measurement system 48Measurement system 49Measurement system 50Measurement system 51Measurement system 52Measurement system 53Measurement system 54Measurement system 55Measurement system 56Measurement system 57Measurement system 58Measurement system 59Measurement system 60Measurement system 61Measurement system 62Measurement system 63Measurement system 64Measurement system 65Measurement system 66Measurement system 67Measurement system 68Measurement system 69Measurement system 70Measurement system 71Measurement system 72Measurement system 73Measurement system 74Measurement system 75Measurement system 76Measurement system 77Measurement system 78Measurement system 79Measurement system 80Measurement system 81Measurement system 82Measurement system 83Measurement system 84Measurement system 85Measurement system 86Measurement system 87Measurement system 88Measurement system 89Measurement system 90Measurement system 91Measurement system 92Measurement system 93Measurement system 94Measurement system 95Measurement system 96Measurement system 97Measurement system 98Measurement system 99Measurement system 100Measurement system 101Measurement system 102Measurement system 103Measurement system 104Measurement system 105Measurement system 106Measurement system 107Measurement system 108Measurement system 109Measurement system 110Measurement system 111Measurement system 112Measurement system 113Measurement system 114Measurement system 115Measurement system 116Measurement system 117Measurement system 118Measurement system 119Measurement system 120Measurement system 121Measurement system 122Measurement system 123Measurement system 124Measurement system 125Measurement system 126Measurement system 127Measurement system 128Measurement system 129Measurement system 130Measurement system 131Measurement system 132Measurement system 133Measurement system 134Measurement system 135Measurement system 136Measurement system 137Measurement system 138Measurement system 139Measurement system 140Measurement system 141Measurement system 142Measurement system 143Measurement system 144Measurement system 145Measurement system 146Measurement system 147Measurement system 148Measurement system 149Measurement system 150Measurement system 151Measurement system 152Measurement system 153Measurement system 154Measurement system 155Measurement system 156Measurement system 157Measurement system 158Measurement system 159Measurement system 160Measurement system 161Measurement system 162Measurement system 163Measurement system 164Measurement system 165Measurement system 166Measurement system 167Measurement system 168Measurement system 169Measurement system 170Measurement system 171Measurement system 172Measurement system 173Measurement system 174Measurement system 175Measurement system 176Measurement system 177Measurement system 178Measurement system 179Measurement system 180

Figure 12 Heatmap visualizing frequency of access ofmeasurementsystems per week


to minimize the threat of respondent bias we used severalinterviews

7 Recommendations for Other Companies

Based on the experiences from developing MetricsCloudand the experiences from previous work (eg introducingmeasurement systems using MS Sidebar gadgets [21 35])we can provide the following recommendations for othercompanies

(1) identify patterns of usingmetrics and the related typesof measurement systemsmdashdesign the measurementsystem based on the identified patterns and updatestrategies (eg local 997888rarr public)

(2) use both public and private measurement systemsand separate themmdashprovide the possibility for thestakeholders to share the measurement systems butalso to use some of the measurement systems forindividual purposes

(3) provide the possibility to execute the measurementsystems automaticallymdashthe users should be able totake advantage of automatization of measurementprocesses

(4) make the client as smart as possiblemdashcreate the possi-bility to use attributes of file system location of filesand other available pieces of information as contextand make the client context-aware

(5) when introducing the cloud use analogies to existingsystems for example Amazon Cloud or Dropbox

(6) use the approach of the minimum-viable-product[36]mdashthis provides the possibility to evolve the use ofmetrics together with designing the cloud dissemina-tion system

Following the above recommendations increases theprobability of a successful adoption of the disseminationsystem

8 Conclusions

Measurement programs in large software development orga-nizations are usually a blend of automatic and manualmeasurement systemsThemajority of measurement systemsare automated and are to be disseminated within the orga-nization However some of the measurement systems arededicated for specific stakeholders only or updated manuallyand disseminated to the entire organization Developmentmanagement and maintenance of a measurement programwith such a diversity can quickly become costly andor labor-intensive

In this paper we describe how to address the scalabilityproblem and how to overcome the main challenges of dis-seminating measurement information for example constantupdate of information sharing of measurement systemsand separation of competence areas We describe how cloudconcepts of IaaS can be applied to support the variety of

uses of measurement systems in modern companies Weshow which types of measurement systems exist the relatedsynchronization strategies and the design ofMetricsCloudmdasha system supporting these types and strategies

By developing a cloud-based dissemination systemmdashMetricsCloudmdashwe reevaluatedwhich dissemination patternsexist in the company and how to realize them By introducingMetricsCloud into the organization we evaluated how thecloud-based dissemination can address the needs of thestakeholders and the metrics team We organized the resultsinto a set of recommendations for other companies that arewilling to adopt similar solutions

In our further work we intend to study how the use ofMetricsCloud changed the patterns of using metrics at thestudied company We intend also to identify new patterns ofsuggesting metrics for stakeholders based on data mining ofthe storage of public metrics

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgment

This work has been partially supported by the SwedishStrategic Research Foundation under the Grant no SM13-0007 (Profiling Product and Organizational Performance)

References

[1] O Gassmann and M Von Zedtwitz ldquoTrends and determinantsof managing virtual RampD teamsrdquo R and DManagement vol 33no 3 pp 243ndash262 2003

[2] I J Chen ldquoPlanning for ERP systems analysis and future trendrdquoBusiness Process Management Journal vol 7 no 5 pp 374ndash3862001

[3] HHOlssonHAlahyari and J Bosch ldquoClimbing the ldquostairwayto heavenrdquondashamulitiple-case study exploring barriers in the tran-sition from agile development towards continuous deploymentof softwarerdquo inProceedings of the 38th EUROMICROConferenceon Software Engineering and Advanced Applications (SEAA rsquo12)pp 392ndash399 IEEE 2012

[4] MFowler andMFoemmelldquoContinuous integrationrdquoThought-Works httpwwwthoughtworkscomContinuousIntegrationpdf

[5] S Chaudhuri U Dayal and V Narasayya ldquoAn overview ofbusiness intelligence technologyrdquo Communications of the ACMvol 54 no 8 pp 88ndash98 2011

[6] H Chen R H Chiang and V C Storey ldquoBusiness intelligenceand analytics from big data to big impactrdquoMIS Quarterly vol36 no 4 pp 1165ndash1188 2012

[7] S Negash ldquoBusiness intelligencerdquo The Communications of theAssociation for Information Systems vol 13 no 1 p 54 2004

[8] E Whitworth and R Biddle ldquoThe social nature of agile teamsrdquoin Proceedings of the Agile Conference (AGILE rsquo07) pp 26ndash36Washington DC USA August 2007

[9] S Bhardwaj L Jain and S Jain ldquoCloud computing a studyof infrastructure as a service (IAAS)rdquo International Journal of


Engineering and Information Technology vol 2 no 1 pp 60ndash632010

[10] M Staron W Meding and C Nilsson ldquoA framework for devel-oping measurement systems and its industrial evaluationrdquoInformation and Software Technology vol 51 no 4 pp 721ndash7372009

[11] M Staron and W Meding ldquoEnsuring reliability of informationprovided by measurement systemsrdquo in Software Process andProduct Measurement pp 1ndash16 Springer New York NY USA2009

[12] P Pawluk B Simmons M Smit M Litoiu and S MankovskildquoIntroducing STRATOS a cloud broker servicerdquo in Proceedingsof the IEEE 5th International Conference on Cloud Computing(CLOUD rsquo12) pp 891ndash898 IEEE Honolulu Hawaii USA June2012

[13] A Shollo and K Pandazo ldquoImproving presentations of softwaremetrics indicators using visualization techniquesrdquo Rapport nrReportIT University of Goteborg 2008020 2008

[14] Y B Yoon J Oh and B G Lee ldquoThe establishment of securitystrategies for introducing cloud computingrdquo KSII Transactionson Internet and Information Systems vol 7 no 4 pp 860ndash8772013

[15] C Vecchiola X Chu and R Buyya ldquoAneka a software platformfor NET-based cloud computingrdquo inHigh Speed and Large ScaleScientific Computing pp 267ndash295 IOS Press 2009

[16] F Liu W Guo Z Q Zhao and W Chou ldquoSaaS integrationfor software cloudrdquo in Proceedings of the 3rd IEEE InternationalConference onCloudComputing (CLOUD rsquo10) pp 402ndash409 July2010

[17] L-J Zhang and Q Zhou ldquoCCOA cloud Computing OpenArchitecturerdquo in Proceedings of the IEEE International Confer-ence on Web Services (ICWS rsquo09) pp 607ndash616 Los AngelesCalif USA July 2009

[18] C Gong J Liu Q Zhang H Chen and Z Gong ldquoThe charac-teristics of cloud computingrdquo in Proceedings of the 39thIEEE International Conference on Parallel Processing Workshops(ICPPW rsquo10) pp 275ndash279 September 2010

[19] S Pearson Y Shen and M Mowbray ldquoA privacy manager forcloud computingrdquo in Cloud Computing pp 90ndash106 Springer2009

[20] P Tomaszewski P Berander and L-O Damm ldquoFrom tradi-tional to streamline developmentmdashopportunities and challen-gesrdquo Software Process Improvement and Practice vol 13 no 2pp 195ndash212 2008

[21] M Staron W Meding G Karlsson and C Nilsson ldquoDevelop-ing measurement systems an industrial case studyrdquo Journal ofSoftware Maintenance and Evolution vol 23 no 2 pp 89ndash1072011

[22] M Staron andWMeding ldquoMonitoring bottlenecks in agile andlean software development projectsndashamethod and its industrialuserdquo in Product-Focused Software Process Improvement pp 3ndash16 Springer 2011

[23] M Staron ldquoCritical role of measures in decision processesmanagerial and technical measures in the context of largesoftware development organizationsrdquo Information and SoftwareTechnology vol 54 no 8 pp 887ndash899 2012

[24] A E Compute Amazon Web Services vol 9 2011[25] G Pallis ldquoCloud computing the new frontier of internet com-

putingrdquo IEEE Internet Computing vol 14 no 5 pp 70ndash73 2010[26] B Hayes ldquoCloud computingrdquoCommunications of the ACM vol

51 no 7 2008

[27] M Armbrust A Fox R Griffith et al ldquoA view of cloud com-putingrdquo Communications of the ACM vol 53 no 4 pp 50ndash582010

[28] I Drago M Mellia M M Munafo A Sperotto R Sadre andA Pras ldquoInside dropbox understanding personal cloud storageservicesrdquo in Proceedings of the ACM Conference on InternetMeasurement Conference (IMC rsquo12) pp 481ndash494 November2012

[29] Z Li C Wilson Z Jiang et al ldquoEfficient batched synchroniza-tion in dropbox-like cloud storage servicesrdquo in Middleware2013 pp 307ndash327 Springer 2013

[30] A Rosenthal P Mork M H Li J Stanford D Koester andP Reynolds ldquoCloud computing a new business paradigmfor biomedical information sharingrdquo Journal of BiomedicalInformatics vol 43 no 2 pp 342ndash353 2010

[31] M Staron and W Meding ldquoUsing models to develop measure-ment systems a method and its industrial userdquo in SoftwareProcess and Product Measurement pp 212ndash226 Springer 2009

[32] M Staron J Hansson R Feldt et al ldquoMeasuring and visualizingcode stabilitymdasha case study at three companiesrdquo in Proceedingsof the Joint Conference of the 23rd International Workshop onSoftware Measurement and the 8th International Conference onSoftware Process and Product Measurement (IWSM-MENSURArsquo13) pp 191ndash200 Ankara Turkey October 2013

[33] CWohlin P Runeson M Host M C Ohlsson B Regnell andA Wesslen Experimentation in Software Engineering Springer2012

[34] P Checkland and S Holwell ldquoAction research its nature andvalidityrdquo Systemic Practice and Action Research vol 11 no 1 pp9ndash21 1998

[35] M Staron and W Meding ldquoTransparent measu res cost-effi-cient measurement processes in SErdquo in Software TechnologyTransfer Workshop Kista Sweden 2011

[36] E Ries The Lean Startup How Todayrsquos Entrepreneurs Use Con-tinuous Innovation to Create Radically Successful BusinessesRandom House 2011

Submit your manuscripts athttpwwwhindawicom

Computer Games Technology

International Journal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Distributed Sensor Networks


Advances in

FuzzySystems

Hindawi Publishing Corporationhttpwwwhindawicom

Volume 2014


ReconfigurableComputing

Hindawi Publishing Corporation httpwwwhindawicom Volume 2014


Applied Computational Intelligence and Soft Computing

thinspAdvancesthinspinthinsp

Artificial Intelligence

HindawithinspPublishingthinspCorporationhttpwwwhindawicom Volumethinsp2014

Advances inSoftware EngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Electrical and Computer Engineering

Journal of

Journal of

Computer Networks and Communications


Hindawi Publishing Corporation

httpwwwhindawicom Volume 2014

Advances in

Multimedia


Biomedical Imaging


ArtificialNeural Systems

Advances in


RoboticsJournal of



Computational Intelligence and Neuroscience

Industrial EngineeringJournal of


Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Human-ComputerInteraction

Advances in

Computer EngineeringAdvances in



this infrastructure uses network storage web pages and exe-cution engines based on our previous work [11]

In this paper we address the following research questionHow to increase dissemination of software metrics using cloudtechnologies

Wedescribe howMetricsCloud infrastructure is designedand we provide lessons learned for the companies willing touse cloud solutions internallymdashboth to increase themeasure-ment dissemination and as a means of internal education onhow IaaS cloud systems can be designed We also evaluateMetricsCloud by introducing it for a selected group ofmeasurement systems and interviewing thembefore and afterthe introduction of MetricsCloud

Our results show that cloud-based dissemination in-creases the reliability of the entire measurement infrastruc-ture (replication of information) and seamless availabilityof the measurement infrastructure (scaled-down distributedversion of measurement execution) but increases the need ofsecurity control andmonitoring ofmeasurement systems (forprivate and shared measurement systems)

The paper is structured as follows Section 2 presentsresearch related to our work Section 3 shows how measure-ment systems are designed and used at Ericsson Section 4describes the cloud environment used to increase the dissem-ination of metrics at the company Section 5 shows how thecloud design is realized at Ericsson Section 6 summarizeslessons learned from developing the cloud environmentfor a software development organization Finally Section 7outlines the conclusions and further work

2 Related Work

Pawluk et al [12] described the process of introducing a newcloud solution to a large enterprise The purpose of the cloudis similar to ours and we use their work when designing ourcloud system The current cloud system is an evolution ofthe previous work on ensuring information reliability donetogether with Ericsson [11] In this work we address the prob-lems of ensuring that information is available throughout theenterprise

Yoon et al [14] showed how to establish security in cloudcomputing The security of MetricsCloud is based on similarprinciples but is a simplification of the security policies Allsecurity is based on the enterprise log-in

Vecchiola et al [15] presented a solution based onNET forimplementing cloud systems Despite the similar technologytheir cloud solution does not address the issues specific tomeasurement programs in the enterprise In particular it doesnot address the need of combining IaaS with the executionenvironment (SaaS)This is addressed in the work of Liu et al[16]

Another approach was presented by Zhang and Zhou[17] and their CCOA framework Although a very elaborateframework could be used in our solution we preferred to usea simple approach and focus on the ease of use It is the easeof use and performance that are important for similar cloudsystems as described by Gong et al [18]

Security is an important aspect which has been recog-nized by Pearson et al [19] The MetricsCloud infrastructureimplements themost important security aspects based on theenterprise access control which is aligned with Person et alrsquosapproach

3 Measurement Systems at Ericsson

The organization within Ericsson which we worked closelywith develops large products for the telecommunicationnetworks The size of the organization is several hundredengineers and the size of the projects is up to a few hundredsProjects are increasingly often executed according to theprinciples of agile software development and lean productionsystem referred to as streamline development (SD) withinEricsson [20] In this environment various disciplines areresponsible for larger parts of the process compared totraditional processes design teams (cross-functional teamsresponsible for complete analysis design implementationand testing of particular features of the product) networkverification and integration testing and so forth

The organization uses a number of measurement systemsfor controlling the software development project (per project)described above a number of measurement systems tocontrol the quality of products in field (per product) anda measurement system for monitoring the status of theorganization at the top level

One example of a measurement system for the moni-toring of the status of the organization is the measurementsystem monitoring feature burndown It is updated dailyand is used by project management to control the statusof the development It is implemented using MS Excel andVBA (Visual Basic forApplications) scripts Another exampleis a measurement system for monitoring the velocity ofintegrating new features to the main product code base

All measurement systems are developed using the in-house methods described in [10 21] with the particularemphasis on models for design and deployment of measure-ment systems presented in [22 23]Theneeds of the organiza-tion evolved from metric calculations and presentations (ca8 years before the writing of this paper) to using predictionssimulations early warning systems and handling of vastquantities of data to manage organizations at different levelsand providing information from teams to managementThese needs are addressed by a number of action researchprojects which we conducted in the organization since 2006

The studied organization uses and maintains over 400different measurement systems (over 4000 MS Excel files)and a large number of persons are involved in the decisionprocesses based on the status of measures A graph withthe size of measurement systems at Ericsson is presented inFigure 1

As the figure shows there are a significant number offiles with over 1000 lines of VBA code which indicatesthe complexity of the updates and dissemination of themeasurement systems


LOC per file9000

8000

7000

6000

5000

4000

3000

2000

1000

0



















4 Cloud Systems







People

Projects

Products

Source files

Source files

Source files

Raw data

Raw data

Raw data

Base measures


Green

RedYellow

Measurement system


ISP

2010-05-02

Product A 803




5 MetricsCloud












User Clientcomputer


Internet access

UserClient

computer

Cloud

Documents





Storage and access





Measurement system

ExecuteUpdatestatus

VBA IQ





Public

Private

User 1

User 2

Public

User 1

ClientPublic

User 2

Execution server


























Web server

Measurement system

Information product

Raw data files

Databases

Products

People

ExecuteUpdate

status

Base measures

Derived measures

Indicators

Green

RedYellow ISP

2010-05-02

Product A 803




Public

Private

Client 1

Private

Public

Private

Shared Shared













6 Evaluation










62 Results





24 8 7 5

5581

161184

13

282

0

50

100

150

200

250

300

1st g

ener

atio

n

2nd

gene

ratio

n

3rd

gene

ratio

n

4th

gene

ratio

n

5th

gene

ratio

n



Addingtrends and



Alignment

programs


metrics

with other

























Management

Teams Teams

Stand-ups


StatusStatus

Excel

Web pages








123

2817

9 6 2 2 3 1 1 1 1 1 1 1 1 1 10

20

40

60

80

100

120

140

280

270

260

250

240

230

220

210

200

190

180

170

160

150

140

130

120

110

100

90

80

70

60

50

40

30

20

10









1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29














8 Conclusions








Acknowledgment


References




























51 no 7 2008


















Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in




LOC per file9000

8000

7000

6000

5000

4000

3000

2000

1000

0



















4 Cloud Systems







People

Projects

Products

Source files

Source files

Source files

Raw data

Raw data

Raw data

Base measures


Green

RedYellow

Measurement system


ISP

2010-05-02

Product A 803




5 MetricsCloud












User Clientcomputer


Internet access

UserClient

computer

Cloud

Documents





Storage and access





Measurement system

ExecuteUpdatestatus

VBA IQ





Public

Private

User 1

User 2

Public

User 1

ClientPublic

User 2

Execution server


























Web server

Measurement system

Information product

Raw data files

Databases

Products

People

ExecuteUpdate

status

Base measures

Derived measures

Indicators

Green

RedYellow ISP

2010-05-02

Product A 803




Public

Private

Client 1

Private

Public

Private

Shared Shared













6 Evaluation










62 Results





24 8 7 5

5581

161184

13

282

0

50

100

150

200

250

300

1st g

ener

atio

n

2nd

gene

ratio

n

3rd

gene

ratio

n

4th

gene

ratio

n

5th

gene

ratio

n



Addingtrends and



Alignment

programs


metrics

with other

























Management

Teams Teams

Stand-ups


StatusStatus

Excel

Web pages








123

2817

9 6 2 2 3 1 1 1 1 1 1 1 1 1 10

20

40

60

80

100

120

140

280

270

260

250

240

230

220

210

200

190

180

170

160

150

140

130

120

110

100

90

80

70

60

50

40

30

20

10









1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29














8 Conclusions








Acknowledgment


References




























51 no 7 2008


















Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in




People

Projects

Products

Source files

Source files

Source files

Raw data

Raw data

Raw data

Base measures


Green

RedYellow

Measurement system


ISP

2010-05-02

Product A 803




5 MetricsCloud












User Clientcomputer


Internet access

UserClient

computer

Cloud

Documents





Storage and access





Measurement system

ExecuteUpdatestatus

VBA IQ





Public

Private

User 1

User 2

Public

User 1

ClientPublic

User 2

Execution server


























Web server

Measurement system

Information product

Raw data files

Databases

Products

People

ExecuteUpdate

status

Base measures

Derived measures

Indicators

Green

RedYellow ISP

2010-05-02

Product A 803




Public

Private

Client 1

Private

Public

Private

Shared Shared













6 Evaluation










62 Results





24 8 7 5

5581

161184

13

282

0

50

100

150

200

250

300

1st g

ener

atio

n

2nd

gene

ratio

n

3rd

gene

ratio

n

4th

gene

ratio

n

5th

gene

ratio

n



Addingtrends and



Alignment

programs


metrics

with other

























Management

Teams Teams

Stand-ups


StatusStatus

Excel

Web pages








123

2817

9 6 2 2 3 1 1 1 1 1 1 1 1 1 10

20

40

60

80

100

120

140

280

270

260

250

240

230

220

210

200

190

180

170

160

150

140

130

120

110

100

90

80

70

60

50

40

30

20

10









1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29














8 Conclusions








Acknowledgment


References




























51 no 7 2008


















Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in




User Clientcomputer


Internet access

UserClient

computer

Cloud

Documents





Storage and access





Measurement system

ExecuteUpdatestatus

VBA IQ





Public

Private

User 1

User 2

Public

User 1

ClientPublic

User 2

Execution server


























Web server

Measurement system

Information product

Raw data files

Databases

Products

People

ExecuteUpdate

status

Base measures

Derived measures

Indicators

Green

RedYellow ISP

2010-05-02

Product A 803




Public

Private

Client 1

Private

Public

Private

Shared Shared













6 Evaluation










62 Results





24 8 7 5

5581

161184

13

282

0

50

100

150

200

250

300

1st g

ener

atio

n

2nd

gene

ratio

n

3rd

gene

ratio

n

4th

gene

ratio

n

5th

gene

ratio

n



Addingtrends and



Alignment

programs


metrics

with other

























Management

Teams Teams

Stand-ups


StatusStatus

Excel

Web pages








123

2817

9 6 2 2 3 1 1 1 1 1 1 1 1 1 10

20

40

60

80

100

120

140

280

270

260

250

240

230

220

210

200

190

180

170

160

150

140

130

120

110

100

90

80

70

60

50

40

30

20

10









1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29














8 Conclusions








Acknowledgment


References




























51 no 7 2008


















Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in





Public

Private

User 1

User 2

Public

User 1

ClientPublic

User 2

Execution server


























Web server

Measurement system

Information product

Raw data files

Databases

Products

People

ExecuteUpdate

status

Base measures

Derived measures

Indicators

Green

RedYellow ISP

2010-05-02

Product A 803




Public

Private

Client 1

Private

Public

Private

Shared Shared













6 Evaluation










62 Results





24 8 7 5

5581

161184

13

282

0

50

100

150

200

250

300

1st g

ener

atio

n

2nd

gene

ratio

n

3rd

gene

ratio

n

4th

gene

ratio

n

5th

gene

ratio

n



Addingtrends and



Alignment

programs


metrics

with other

























Management

Teams Teams

Stand-ups


StatusStatus

Excel

Web pages








123

2817

9 6 2 2 3 1 1 1 1 1 1 1 1 1 10

20

40

60

80

100

120

140

280

270

260

250

240

230

220

210

200

190

180

170

160

150

140

130

120

110

100

90

80

70

60

50

40

30

20

10









1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29














8 Conclusions








Acknowledgment


References




























51 no 7 2008


















Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in






Web server

Measurement system

Information product

Raw data files

Databases

Products

People

ExecuteUpdate

status

Base measures

Derived measures

Indicators

Green

RedYellow ISP

2010-05-02

Product A 803




Public

Private

Client 1

Private

Public

Private

Shared Shared













6 Evaluation










62 Results





24 8 7 5

5581

161184

13

282

0

50

100

150

200

250

300

1st g

ener

atio

n

2nd

gene

ratio

n

3rd

gene

ratio

n

4th

gene

ratio

n

5th

gene

ratio

n



Addingtrends and



Alignment

programs


metrics

with other

























Management

Teams Teams

Stand-ups


StatusStatus

Excel

Web pages








123

2817

9 6 2 2 3 1 1 1 1 1 1 1 1 1 10

20

40

60

80

100

120

140

280

270

260

250

240

230

220

210

200

190

180

170

160

150

140

130

120

110

100

90

80

70

60

50

40

30

20

10









1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29














8 Conclusions








Acknowledgment


References




























51 no 7 2008


















Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in








62 Results





24 8 7 5

5581

161184

13

282

0

50

100

150

200

250

300

1st g

ener

atio

n

2nd

gene

ratio

n

3rd

gene

ratio

n

4th

gene

ratio

n

5th

gene

ratio

n



Addingtrends and



Alignment

programs


metrics

with other

























Management

Teams Teams

Stand-ups


StatusStatus

Excel

Web pages








123

2817

9 6 2 2 3 1 1 1 1 1 1 1 1 1 10

20

40

60

80

100

120

140

280

270

260

250

240

230

220

210

200

190

180

170

160

150

140

130

120

110

100

90

80

70

60

50

40

30

20

10









1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29














8 Conclusions








Acknowledgment


References




























51 no 7 2008


















Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in

















Management

Teams Teams

Stand-ups


StatusStatus

Excel

Web pages








123

2817

9 6 2 2 3 1 1 1 1 1 1 1 1 1 10

20

40

60

80

100

120

140

280

270

260

250

240

230

220

210

200

190

180

170

160

150

140

130

120

110

100

90

80

70

60

50

40

30

20

10









1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29














8 Conclusions








Acknowledgment


References




























51 no 7 2008


















Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in




123

2817

9 6 2 2 3 1 1 1 1 1 1 1 1 1 10

20

40

60

80

100

120

140

280

270

260

250

240

230

220

210

200

190

180

170

160

150

140

130

120

110

100

90

80

70

60

50

40

30

20

10









1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29














8 Conclusions








Acknowledgment


References




























51 no 7 2008


















Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in














8 Conclusions








Acknowledgment


References




























51 no 7 2008


















Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in





















51 no 7 2008


















Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in










Advances in

FuzzySystems


Volume 2014












Journal of

Journal of





Advances in

Multimedia


Biomedical Imaging



Advances in


RoboticsJournal of










Advances in

