exploring the disruptive power of adopting devops for software …1334453/fulltext03.pdf · pers...

Master of Science in Industrial EconomicsJune 2019

Exploring the disruptive power ofadopting DevOps for software

development

Liang YuClemente Guerra

Department of Industrial Economics, Blekinge Institute of Technology, 371 79Karlskrona, Sweden

This thesis is submitted to the Faculty of Industrial Economics at Blekinge Institute of Tech-nology in partial fulfilment of the requirements for the degree of Master of Science in IndustrialEconomics.

The authors declare that they are the sole authors of this thesis and that they have not usedany sources other than those listed in the bibliography and identified as references. They furtherdeclare that they have not submitted this thesis at any other institution to obtain a degree.

Contact Information:Author(s):Liang YuE-mail: [email protected]

Clemente GuerraE-mail: [email protected]

University advisor:Shahiduzzaman QuoreshiDepartment of Industrial Economics

Department of Industrial Economics Internet : www.bth.seBlekinge Institute of Technology Phone : +46 455 38 50 00SE–371 79 Karlskrona, Sweden Fax : +46 455 38 50 57

Abstract

Background. DevOps is a software developer practice that is aiming at bridgingthe gap between development (Dev) and operations (Ops). This practice has beenwidely adopted successfully in many enterprises and helps them to overcome the lackof cohesiveness and collaboration that is a common place in the software develop-ment industry. Through DevOps, companies of any size, are empowered to deliverrapidly software features, fixes and updates for business objectives. However, despitea general consensus, there is still a large number of firms which do not employ theDevOps in their software production. During our research we were not able to findstudies that clearly identify either the challenges or the benefits of implementingDevOps methodology in their software production.Objectives. In this study, we aim to explore the challenges and benefits of adoptingDevOps in software companies, which could help teams and organisations to enhancetheir productivity and efficiency.Methods. We used the systematic literature review method, and identified 29 pa-pers that present the issues or challenges while adopting DevOps in software de-velopment. The evidence-based and thematic coding analysis is performed on theselected papers. In addition, we have also noticed that the evidence of the bene-fits while applying DevOps can be found in “grey literature” including conferenceabstracts, presentations, proceedings; regulatory data; government publications; re-ports (such as white papers, working papers, internal documentation); dissertation-s/theses; patents; policies and procedures.Results. Firstly, 42 challenges of adopting DevOps have been found. Secondly, allidentified challenges have been mapped to six high-level software development lifecycles, which are requirements, design, implementation, test, release, maintenance.Moreover, we highlight the considerable empirical things when applying DevOpspractices in an organisation.Conclusions. The challenges we identified could be a valuable guidelines to thepractitioners that intend to adopt DevOps in their organisations, and they are keyfactors that could affect software development life-cycle. In addition, we unearth alist of benefits of DevOps that practitioners have consistently agreed upon.

Keywords: DevOps, Software Development, Systematic Literature Review.

i

Contents

Abstract i

1 Introduction 21.1 Thesis Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Problem discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Problem formulation and purpose . . . . . . . . . . . . . . . . . . . . 4

1.3.1 The origin of DevOps . . . . . . . . . . . . . . . . . . . . . . . 51.3.2 Core differences between DevOps and Agile . . . . . . . . . . 51.3.3 The seven myths about DevOps . . . . . . . . . . . . . . . . . 51.3.4 DevOps Theory and Principles . . . . . . . . . . . . . . . . . 8

1.4 Delimitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Theory 11

3 Research Methodology 133.1 SLR processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.2 Research questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.3 Search string and databases . . . . . . . . . . . . . . . . . . . . . . . 14

3.3.1 Selected database . . . . . . . . . . . . . . . . . . . . . . . . . 153.3.2 Inclusion and exclusion criteria . . . . . . . . . . . . . . . . . 153.3.3 Search result . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.4 Data collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.5 Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3.5.1 Data synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.5.2 Synthesis procedure . . . . . . . . . . . . . . . . . . . . . . . . 19

3.6 Validity threats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4 Results 214.1 RQ1:what challenges have been reported on adopting DevOps in prac-

tices for software development? . . . . . . . . . . . . . . . . . . . . . 214.2 RQ1.1:What aspects of the software development have been covered

by the reported challenges in practices? . . . . . . . . . . . . . . . . . 214.3 RQ2:What are the necessary considerations to take into account when

implementing DevOps in an organisation? . . . . . . . . . . . . . . . 234.4 RQ3: What are the tangible benefits of applying DevOps in the soft-

ware development environment? . . . . . . . . . . . . . . . . . . . . 24

5 Discussion 26

ii

6 Conclusions and Future Work 29

iii

List of Figures

1.1 The mapping between DevOps and industry organizations . . . . . . 41.2 The three ways of thinking (Kim et al. 2016) . . . . . . . . . . . . . . 9

3.1 The overview of the SLR processes . . . . . . . . . . . . . . . . . . . 133.2 The collected codes mapping to primary studies . . . . . . . . . . . . 183.3 The thematic coding processes with an example . . . . . . . . . . . . 19

4.1 The challenges mapped in software development lifecycle . . . . . . . 23

5.1 The identified DevOps challenges in industry . . . . . . . . . . . . . . 27

iv

List of Tables

1.1 Differences between Agile and DevOps . . . . . . . . . . . . . . . . . 61.2 The 7 myths about DevOps . . . . . . . . . . . . . . . . . . . . . . . 7

3.1 The inclusion and exclusion criteria . . . . . . . . . . . . . . . . . . . 153.2 The search result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.3 Selected primary papers . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.1 A list of challenges identified in this study . . . . . . . . . . . . . . . 22

1

Chapter 1

Introduction

Technology is changing our lives, the way we interact with each other, the way wework and even our core values as human have been disrupted. This transformationis occurring at such an incredible pace and it is so profound that it is unmanageableto think that ten years ago a company like TomTom was able to sell a turn-by-turnmobile application for 100 dollars (Keizer 2009), Uber did not exist, neither was theiPad, Instagram or WhatsApp.

Technology advancements, e.g. artificial intelligent (AI), machine learning, block-chain, Internet of Things (IoT), Crowdfunding, DevOps, are fuelling further thistransformation. Enterprises are facing huge challenges to adjust their strategiesquickly and successfully (Jesse 2018). Meanwhile, customers are empowered withtools that can make or break a company, they become unsatisfied and frustrated(Abdelkebir et al. 2017).

“It is not necessary to change. Survival is not mandatory”.Dr.William Edward Deming

The ironic quote from Deming is perfectly encapsulating the idea of the evolution andchange that companies and people are required to undergo in order to survive “Theonly survivors will be companies with constancy of purpose for quality, productivity,and service” (Deming 2018).

Jeff Immelt, General Electric chairman and CEO explains that one of the majortarget of the company is to “become a top 10 software company” by 2020 (Kellner2017 (accessed April 14, 2019). The organizational agility, considered as a capabilityof responding to changes, becomes an important competence to be the winner in thecompetitive globalized environment (Jesse 2018).

Almost a decade ago on the Wall Street Journal, Marc Andreessen stated that“software is eating the world” (Andreessen 2011). He went on explaining how themost relevant organizations are software companies. The music industry back then,but even more today, was dominated by Apple, Spotify, Pandora, and YouTube (soft-ware companies), the music labels from dominant forces as they were, became justthe content providers. In the same article, (Andreessen 2011) was stated that thelargest video service provider was Netflix, another software company, which obliter-ated the traditional entertainment providers by adopting technological advancementsand implementation of disruptive methodology including DevOps.

The Web 2.0 principle, “forever beta”, has been a normal status of most Internet-based applications, which means that software are frequently updated online andreleased (Bai et al. 2018) In order to improve organization’s agility, methodology

2

3 Chapter 1. Introduction

such as agile, has been widely adopted in regulated development (Rajkumar et al.2016). DevOps does present some similarities to the agile practices, however the coreobjective is to reduce or join repetitive tasks by using automation solutions throughplanning, development, integration, testing, delivery and deployment (Laukkarinenet al. 2018). It connects both development and operation organizations to worktogether efficiently by removing communication barrier (Premchand et al. 2019). Itdrives business models to new technology strategies, and it helps organizations torun infrastructure resources with a low cost and remove the delegated liabilities tothe third-party suppliers (Al-Ruithe et al. 2018).

1.1 Thesis StructureWe have organised this study as follows: Section 1 after the introduction, dives on theproblems discussion framing the gap that exist in the literature. Section 2 exploresthe existing theories and background studies on the challenges occurring in the widesoftware development realm. Section 3 introduces the research methodology adoptedfor this study, the data collection and how the analysis was performed. Section 4presents the results obtained from the analysis conducted and answers to the researchquestions. Section 5 draws the conclusion of this study, while section 6 discussesfurther directions for the study.

1.2 Problem discussionResearchers have conducted a number of studies focused on the organisational agilitythat enables firms to be agile for changes (Jesse 2018, Ravichandran 2018, Sherehiy& Karwowski 2014). Furthermore, (Ravichandran 2018) they presented that digi-tized business processes, enable firms to be agile and the organisational agility hasa strong positive impact of firms’ performance. From the early 2000, agile was re-puted as the “must adopt” methodology for software development and currently isstill widely adopted. According to the Annual State of the Agile report (VersionOne2019),which surveyed over 1492 businesses worldwide, over 90% of the respondentsare still utilising agile methodologies. Furthermore, this methodology is not re-stricted to the software development realm, but it does have a great appeal on otherindustries too (Schwaber & Beedle 2002).

As other development principles in the past, agile was initially driven by practi-tioners to overcome their necessities. In the adoption process, parallel practices havetaken shape this may refer to: eXtreme Programming (XP), the Dynamic SystemsDevelopment Method (DSDM), Scrum, Crystal, Agile modelling, Feature Driven De-sign (FDD) and Lean Software Development (LSD), along with variants of each e.g.XP- Lite" (Conboy & Coyle 2009)

Empirical studies suggest that the capability of quick reactions with the technol-ogy changes is able to improve the performance of delivery (Ferrier 2001).

In the organizational agility literature, (Sherehiy & Karwowski 2014) investigatedhow organizations adopted agile strategies (focused on the employees’ management)to improve the workforce agility. Besides, (Jesse 2018) has found that the digitizationleadership improves the organizational agility to handle changes.


In the same report on the state of agile in 2019 (VersionOne 2019) is mentionedthat from all the surveyed organisations about 71% have already implemented or areplanning to implement DevOps initiatives in their organisations. This establishesthat the move from agile to DevOps could have even bigger implications, is not amerely trend but it could be a necessity, therefore the need of further investigationis central.

1.3 Problem formulation and purposeIn this paper we argue that DevOps has the power to disrupt most of the previousmethodologies and deliver tangible benefits to the businesses. However, the transitionfrom one methodology to another could be one of the major bottlenecks. Especiallyfor the software industries, there are challenges and problems that preventing theDevOps applied in their organizations as shown in Figure 1.1.

Requirements

Designarchitect teams

Dev teams

Test teams

Releaseteams

Supportteams

Software industrychallenges

DevOps

CONTINUOUS INTEGRATION

TEST

DEVELOPPLAN

RELEASE

MONITOR

DEV

OPS

Figure 1.1: The mapping between DevOps and industry organizations

As shown in Figure 1.1, the DevOps requires changes in the entire organization ina software industry when the decision is taken to apply the DevOps in an organizationor a company. Each team have connection to others in a collaboration environment,


and the teams are mapped with each component in DevOps processes also. Therefore,many barriers exist in different teams that need to be addressed.

The intent of this study is to aid professionals within the software industry whenthe consider to adopt DevOps methodologies. Moreover, we have not found a researchwhich is focused on the challenges or disruption of applying DevOps from a industryperspective.

1.3.1 The origin of DevOps

In 2009 at the O’Reilly Velocity Conference, two senior Flickr’s employees, PaulHammond and John Allspaw gave a presentation that has subsequently become thecatalyst of the DevOps movement. A belgium consultant named Patrick Deboiswhile watching the live streaming of the presentation named 10+ Deploys per Day:Dev and Ops Cooperation at Flickr (Paul Hammond 2009) had the eureka momentand announced his own conference to be hosted in Belgian city of Ghent named De-vOpsDays (DevOpsDays 2009). This is known as the official begin of the DevOpsmovement. During their presentation, Hammond and Allspaw were listing all therecurrent issues and stereotypes commonly occurring in most of the software com-panies and more precisely they draw the audience attention on the barriers betweenDevelopment and Operation departments. They went on proposing a more cohesivemindset between Dev and Ops, including a set of recommendations. These becamethe core foundation of DevOps. Another stepping stone for the movement come in2013 from a book, more precisely a fiction novel titled The Phoenix Project, writtenby Gene Kim, Kevin Behr and George Spafford(Kim et al. 2014), this book inspiredpractitioners and consolidate the movement growth and adoption.

1.3.2 Core differences between DevOps and Agile

Authentic software development as it is intended today, started in the 60s with theSystem Development Life Cycle (Royce 1987) giving birth to the Waterfall concept.Throughout the years, the evolution of development models has been systematic andrelentless. In 2001, the practitioners collected and organized most of the theoriesproposed under the Agile Manifest. The agile approaches partially faced up to thecore problem of the contemporary rapid software development. The prevalent ideais that teams can be more efficient in change management, if they are capable ofreducing the time and costs of exchanging information between participants in de-velopment, in such a way as to shorten the time period between the moment thedecision was made and feedback on the impact of this decision(Matković & Tumbas2010).

As shown in Table 1, there are several elements that explain the core differencesbetween the two practices.

1.3.3 The seven myths about DevOps

Below we list a series of what are considered, by the “founding father of DevOps” inthe famous vade mecum “The DevOps Handbook (Kim et al. 2016) to be myths:


Table 1.1: Differences between Agile and DevOpsParameter Agile DevOps

PurposeAgile helps to manage complexprojects adopting definedframeworks.

To manage end to end engineering processeswith focus on security and robustness.

Tasks Focuses on constant changes. Focus on continuous delivery.

Implementation

Agile can be developed withina range of tactical frameworkslike a sprint, scrum, kanbanand more.

Collaboration, culture shift and accountability,are the primary focus, however it does nothave any commonly defined framework.

Team skill setEmphasises in training all teammembers to have similar andequal skills.

DevOps tends to spread the skill setbetween the developers and operationteams.

Team sizeGenerally small teams. Thefewer people in it, the fasterthey can move (avoid frictions).

Relatively larger team size. The shiftoccurs from top to bottom and permeatesto the entire organisation.

Teamorganisation

Teams are organised aroundproducts and features, generallyare not allowed to mix withother teams.

The teams are organised by tasks. Seniorteam members can switch their roledepending on the project, everyone needsto have a deeper understanding of theentire software development life-cycle.

DurationAgile is managed in sprint units.The time is much less than amonth for each sprint.

DevOps strives for deadlines andbenchmarks with major releases.

Feedback Feedback is given by customersat the end of the cycle.

Feedback comes from the internal teamsconstantly, also known as Feedback Loop.

Target areas Software development primarily End to end business solution for continuousdelivery within software development.

OrganisationTeams are compartmentalisedand not responsible for theother departments.

Teams need to share, understand, embraceand resolve issues occurring in otherdepartments. Sharing knowledge is a key.

Main FocusAgile is primarily concernedwith development andquality insurance.

DevOps focuses on development, qualityassurance and operation aspect of theentire cycle.


Table 1.2: The 7 myths about DevOps

Myth1 DevOps is only for startup

Each company that able to transformtheir architecture, technical practiceand culture can successfullyimplement DevOps.

Myth2 DevOps replace Agile

DevOps and Agile are compatible.DevOps is not a continuation ofAgile. Agile is very often aneffective enabler of DevOps.

Myth3 DevOps is incompatiblewith ITIL

DevOps practice can be madecompatible with ITSM (IT servicesManagement). However, in order tosupport short lead time and increasedeployment frequency ITSMfeatures in the DevOps processesare generally automated.

Myth4DevOps is incompatiblewith information securityand compliance

There is an absence of "traditionalcontrols" such as change approvaland manual security review.However, with DevOps instead ofimplementing the compliance andsecurity activities at the end of theproject, controls are an integralpart of daily work of thedevelopment cycle.

Myth5 DevOps meanseliminating IT operations

Instead of performing manual tasksthat are brought forward by tickets,the IT staff through APIs andself-served platform become moreengaged in the product development

Myth6DevOps is justinfrastructure as code orautomation

DevOps is not solely aboutautomation, it requires cultural shiftthat allows the shared goals to beachieved throughout the IT valuestream.

Myth7 DevOps is only for opensources software

Achieving DevOps outcome isindependent from the technologybeing used.


1.3.4 DevOps Theory and Principles

There is a vast number of definitions that attempt to define DevOps at its core. Aqualitative study of DevOps conducted by (Erich et al. 2017) shows that “there is alack of consensus on the relevant characteristics of DevOps”. The agreement on thedefinition within the practitioners and promoters of DevOps is also not unanimous,however it presents a stronger and more resilient means of describing and defyingthis methodology and therefore we submit this version to the readers. The mostcomprehensive ways of describing what DevOps really is, in the authors’ opinion,belong to the thought leaders of the DevOps community and in particular John Willis(Willis 2012), that stated “DevOps is about humans, DevOps is a set of practice andpatterns that turn human capital into high-performance organization capital”.

DevOps culture changes automation, lean, sharing, learning, and innovation (Ra-jkumar et al. 2016). DevOps methodology addresses the objective of agility, elasticityand stability of a commercial products. It includes continuous integration, continu-ous delivery and continuous deployment (Itkonen et al. 2016), focusing the immediatedelivery and deployment of features instead of incremental or scheduled releases forproduct, and it allows organisations to make a quick response to customers’ needsand anticipate changes(Taryana et al. 2017). Back in 2010 after the first DevOpsDaysin U.S. Damon Edwards and John Wills coined the acronyms that encapsulated themovement’s idea namely CAMS (Culture, Automation, Measurement and Sharing)framework. Later, another founding father of the movement Jez Humble added anL for Lean and the updated acronyms became CALMS (Willis 2012).

The underpinning principles onto which DevOps is based are also know in jargonas the Three Ways, which is the concept that lays the foundation for all DevOpspatterns (Kim et al. 2016).

• “The First Way enables fast left-to-right flow of work from Development to Op-erations to the customer. In order to maximize flow, we need to make workvisible, reduce our batch sizes and intervals of work, build in quality by prevent-ing defects from being passed to downstream”.

This is also defined as “System Thinking” and it is really about the entiresystem been seen as one single unit rather than separate or team/group effort.

• “The Second Way enables the fast and constant flow of feedback from right to leftat all stages of our value stream. It requires that we amplify feedback to preventproblems from happening again, or enable faster detection and recovery.”

The primary concept here is about amplifying the feedback loop. This elementis used by the team to ponder and discuss on the output delivered before movingonto the next step.

• “The Third Way enables the creation of a generative, high-trust culture thatsupports a dynamic, disciplined, and scientific approach to experimentationand risk-taking, facilitating the creation of organisational learning, both fromour successes and failures.”

The Third way is about enabling, nurturing and encouraging a culture of ra-tional and measured experimentation, embrace failure and learn from it.


business customer

DEV OPS

DEV OPS

DEV OPS

First Way:System Thinking

Second Way:Amplify FeedbackLoops

Third Way:Culture of ContinualExperimentation AndLearning

Figure 1.2: The three ways of thinking (Kim et al. 2016)

Further principles used in the DevOps practice include adaptation of Lean Man-agement to software development, Improvement Kata and Kanban, which despitebeing outside the scope of this research are to be considered as a contributing elementin the DevOps practice.

1.4 Delimitations

Despite the authors’ best effort to conduct a research based on scientific evidence, anduse empirical evidences, some of the sources utilized are belonging to so-called greyliterature. These are "information produced on all levels of government, academia,business and industry in electronic and print formats not controlled by commercialpublishing i.e. where publishing is the not the primary activity of the producing


body"(White et al. 2013). Despite not having a scientific validity, however the au-thors realized the added value that DevOps movements and practitioners could have.

Choosing different methods of collecting the data does carry a risk of publicationbias, however in the best interest of the readers, the authors have tried to mitigatethis by applying a critical review and scrutiny of the material selected and recurringto establish conceptual references when no references were obtainable. Once thedata was analyzed by each single author, the conclusion where compared againsteach other.

Chapter 2Theory

The success of almost any business depends on the promotion of digital excellence(Jesse 2018). This online presence is necessary for the business to thrive, can assumedifferent forms, websites, apps, platforms running on cloud, IOT devices just to citea few. Any of these activities require a piece of code, a program written, maintainedand deployed. This is the reason why in this study we focus on software development.Today, software are developed in fast-changing environment, the unpredictability ofmarkets, together with complex and demanding customers’ requirements are addingpressure for delivering products to market (Claps et al. 2015). Incumbent firms arerequired to adapt and adjust at light speed, in order to rise entry barriers and keepout the entrants. On the other hand, the entrants are exploiting new technologies,methodology and strategies that will enable them to enter the market, gain tractionand attract investors. In such an intricate and disruptive environment, leaders areconstantly searching for the “holy grail” methodology that can empower, and optimizethe resources they have at their disposal.

The long term focus of the business unit is on globalization, efficiency and quality(Winter et al. 2014). Some industries are more prone to technology than others andtheir internal organization allows them to quickly adapt. However, big corporationshave a whole set of different complications “Reinventing organizations with legacy andcomplexity as we enter the digital revolution is a difficult and interesting challenge inwhich only the most adaptable will survive” (Smart 2018). Successful transformationis 70 to 90 percent leadership and only 10% to 30% management (Kotter et al. 1995),and this becomes the real setback. One of the key elements circulating in the softwaredevelopment industry is Continuous Delivery.

“Continuous Delivery (CD), is a software development discipline (at the core ofDevOps) in which the software is kept in such a state that in principle, it could bereleased to its user at any time”(Humble & Farley 2010).

CD has been implemented successfully by companies (e.g. Facebook, Atlassian,IBM, Adobe Tesla and Microsoft) that have succeeded in implementing these prac-tices to an extent that software is continuously deployed to their customers (Clapset al. 2015). Google’s achievements with continuous delivery are incredible:

• 40,000 code commits/day.

• 50,000 builds/day (on weekdays, this may exceed 90,000).

• 120,000 automated test suites.

• 75 million test cases run daily.

11

12 Chapter 2. Theory

• 100+ engineers working on the test engineering, continuous integration, andrelease engineering tooling to increase developer productivity (making up 0.5%of the R&D workforce) (Kim et al. 2016).

The Continuous Deployment (CD) is always utilised to automatically deploy therelease candidate artifacts in development, testing and production environments(Rajkumar et al. 2016). Furthermore, in the highly regulated software industriesthat require level compliance in order to meet regulatory requirements (medical,finance,food) , there is still not clear evidence that DevOps fits the regulatory stan-dards (Laukkarinen et al. 2018). One of the major obstacles that appear to becommon in many enterprises and companies is the different interests among differentdepartments (Winter et al. 2014). Each department tends to address what are themost pressing issues within the department itself, rather than looking at the overallpicture and potential benefits. DevOps is often coalesced with Agile principles andpractices” (Nagarajan & Overbeek 2018). Some researchers who conducted on bothmethodologies, highlighted the fact that DevOps can enhance the key characteristicsof Agile (Jabbari et al. 2016). However, DevOps methodology is incorporating in itsessence many elements of Lean Software Development (LSD) defined by Poppendieckand Poppendieck as “the application of the principles of the Toyota Product Devel-opment System to software development” (Poppendieck & Poppendieck 2007). Theadoption of DevOps presents two major observations: the first is that “infrastructureorganization and operations need to be made part of the simplification process andrationalize”, and the second is that ”operations and infrastructure teams may alsoneed to re-skill, cross skil land align to the new way of working” (Premchand et al.2019). In Jesse’s words “it is plausible to assume that without a unifying corporateculture “we spirit” and a share target system, a leap in flexibility will cause moretroubles than advantages” (Jesse 2018). The very familiar split between developersand operation is a common place in many organizations. This division is a frequentcause of disagreements and delays of software release , it is due to different goals,contrary mindsets, and incompatible processes (Wettinger et al. 2014).

This fundamental structural division between operational and development is themajor cause of deployment delays in the production environment (De França et al.2016). Despite the DevOps has sprung from a cloud-based product environment, itnot confined to exclusively to these type of enterprises, but is adaptable to differentorganization and can provide the same benefits (Zhu et al. 2016).

Chapter 3Research Methodology

This study aims to explore the disruption power and challenges of adopting theDevOps in software development.

To fulfil this goal, we need to get understand the knowledge of DevOps andthe practices of adopting it based on the previous research studies. The systematicliterature review (SLR) (Kitchenham & Charters 2007) is the chosen method toretrieve the state-of-the-art of DevOps practices in software development companies.The motivations of the SLR:

• Transparent and systematic reviews.

• Cover wide scope of studies.

• High quality of the chain of the evidence.

• Under controlled research bias.

In summary, we conducted SLR to get understanding the DevOps strategies andand practices dedicated to the software development.

3.1 SLR processes

We conducted the literature review by using the steps defined in Figure 3.1.

Step1: Establishresearch goal

Step2: Defineresearch questions

Step3: Create search string

Step4: Applysearch string in

databasesStep5: Store

relevant papers

OK

refine

NOKStep6: Screen ofrelevant papers Step7: Data extraction

Step8: Data analysis

Phase1: Plan Phase2: Search papers Phase3:Data analysis

inclusion exclusion

Figure 3.1: The overview of the SLR processes

As shown in Figure 3.1, firstly we finalised the research goal and define the re-search questions. Secondly, we searched in the bibliographic database “Scopus 1”

1www.scopus.com

13

14 Chapter 3. Research Methodology

and found a limited number of peer-reviewed studies. Thirdly, we collected data byreading all the identified articles, and we analysed the collected data. Finally, wepresented the result of the literature review.

3.2 Research questionsWe define the below research questions for our study.

• RQ1: what challenges have been reported on adopting DevOps in practices forsoftware development?

• RQ1.1: What aspects of the software development have been covered by thereported challenges in practices?

• RQ2: What are the necessary considerations to take in account when imple-menting DevOps in an organisation?

• RQ3: What are the tangible benefits of applying DevOps in the software de-velopment environment?

3.3 Search string and databasesWe piloted several experimental searches in April 2019 by following the steps definedin the Figure 3.1 “phase2”:

1. Identify a search string by using the keywords related with the research area,e.g. “DevOps”, “challenges” etc.

2. Apply the search string in the selected databases.

3. Store the search results.

4. Screen the relevant papers.

In order to cover the search scope better and describe the phenomenon appropri-ately, we executed the above steps many times during the pilot searches. Generally,there are three major steps which help us to find the most relevant articles:

Level 1 pilot search string: “DevOps” AND “software development”Level 2 pilot search string:

(limited to title,abstract,keyword): ("software development") AND("DevOps" OR "DevOps process" OR "DevOps design" OR "DevOpschallenge" OR "DevOps strategy" OR "DevOps adoption")

Level 3 pilot search string:

TITLE-ABS-KEY(("software development") AND ("DevOps process"OR "DevOps *" OR "DevOps challenge" OR "DevOps practice" OR"DevOps strategy" OR "DevOps adoption") AND ("challenge" OR "dis-ruption"))


3.3.1 Selected database

For the bibliographic databases, we used Scopus to search all primary articles, due toit is a reference database for peer-reviewed papers. Then we downloaded PDF filesfrom IEEE, Springer ACM, and Wiley based on the papers’ title found in Scopus.These chosen databases have good coverage of the software engineering literature,are commonly used for software engineering SLRs, and contain “keywords” metadatasearch and “command search” with a search string.

3.3.2 Inclusion and exclusion criteria

We limited the papers published between 2009 to 2019 in peer-reviewed and Englishlanguage including conference, journal and book chapter. The inclusion and exclusioncriteria used during our SLR is defined in Table 3.1.

The below steps are used to include and exclude papers:

1. Limit the publish year in search database and keep papers from 2009 to 2019

2. Check paper type and delete non-peer-reviewed papers.

3. Keep workshop, conference, journal and book chapter paper-types and excludethe others.

4. Exclude papers that published in the non-English language.

5. Store the papers containing "systematic literature review" in the title or ab-stract to a folder in Mendeley2 tool.

Table 3.1: The inclusion and exclusion criteriaInclusion criteria

- Published from 2009 to 2019- Peer-reviewed- English language- Including workshop, conference, journal, book chapter

Exclusion criteria- Books

-Non-peer-reviewed papers like keynotes, editorials,power point, lecture notes, patents,reviews, discussions

3.3.3 Search result

We applied the above three level search strings and executed the inclusion and ex-clusion criteria. As a result, we got the search result as shown in Table 3.2. Thelevel 1 search result is 581 papers, and the scope is very wide. Then we refined the

2www.mendeley.com


search string as level 2, and we got 185 papers which still contain some irrelevantstudies after screening the search result. Therefore, we conducted the level 3 search,and the result was 29 papers.

Table 3.2: The search resultSearch NO. of papers foundLevel 1 581Level 2 185Level 3 29

All selected papers are presented in Table 3.3:

3.4 Data collection

We read all the selected papers and collected the below two primary types of datato analyse:

1. The properties of all selected papers (e.g. author(s), article title, type of paper,published year, research type) as shown in Table 3.3.

2. Collect all challenges while adopting DevOps in software development includingDevOps architecture, management, tooling, automation, and environments etc.

We stored the collected data for further analysis, and the collected codes arevisualised in Figure 3.2.

As shown in Figure 3.2, we extracted four levels of codes from the selected papers.The first level is the “DevOps”, and the second level covers software development man-agement, architecture design, tooling design, processes automation, software specifi-cation, and environment. The third level includes the culture, teams, organisations,and data, log, tools, hardware etc. The fourth level is more detailed in softwaredevelopment issues or strategies, and they are mapped with the selected papers.

3.5 Data analysis

We followed below steps to analyse the extracted codes.

1. Familiarise with extracted data: we read and examined all extracted informa-tion, e.g. DevOps architecture challenges, disruptions, environment issues.

2. We reviewed and refined all extracted data. In some cases, we had to read thepapers again.

3. Connect initial data: we tried to combine different initial data of DevOpschallenges in details.

4. Summarised the findings.


Table 3.3: Selected primary papersIndex Authors Paper type Paper title Year

P1 Bartusevics et al. Conference An Approach for Development of Reusable FunctionLibrary for Automation of Continuous Processes 2016

P2 Beranek et al. Journal Architecting Enterprise Applications for the Cloud:The Unicorn Universe Cloud Framework 2018

P3 Shahin Conference Architecting for devops and continuous deployment 2015

P4 Baskarada et al. Journal Architecting Microservices:Practical Opportunities and Challenges 2018

P5 Bartusevics Conference Automation of Continuous Services:What Companies of Latvia Says about It? 2016

P6 Debroy et al. Conference Building lean continuous integration and delivery pipelinesby applying devops principles: A case study at varidesk 2018

P7 Nicolau de Franca et al. Conference Characterizing DevOps by hearing multiple voices 2016

P8 Hemon et al. Journal Conceptualizing the transition from agile to DevOps:A maturity model for a smarter is function 2019

P9 Bai et al. Conference Continuous delivery of personalized assessmentand feedback in agile software engineering projects 2018

P10 Chen Journal Continuous Delivery: Overcoming adoption challenges 2017

P11 Samarawickrama et al. Conference Continuous scrum: A framework toenhance scrum with DevOps 2018

P12 Fitzgerald et al. Conference Continuous software engineering and beyond:Trends and challenges 2014

P13 Rong et al. Conference DevOpsEnvy:An Education Support System for DevOps 2017

P14 Perera et al. Conference Evaluating the impact of DevOps practice inSri Lankan software development organizations 2017

P15 Rana et al. Conference First international Workshop on EmergingTrends in DevOps and Infrastructure 2016

P16 Hemon et al. Journal From Agile to DevOps:Smart Skills and Collaborations 2019

P17 Wiedemann et al. Conference How to implement clan control in DevOps teams 2018P18 Zhu et al. Conference If Docker is the Answer, What is the Question? 2018P19 Albuquerque et al. Journal Implementing DevOps in legacy systems 2019

P20 Nogueira et al. Conference Improving la redoute’s CI/CD pipeline and devopsprocesses by applying machine learning techniques 2018

P21 Cois et al. Conference Modern DevOps: Optimizing softwaredevelopment through effective system interactions 2015

P22 Van heesch et al. Conference Software specification and documentation in continuoussoftware development - A focus group report 2017

P23 Theunissen et al. Conference Specification in continuous software development 2017

P24 Bruza et al. Conference Teaming with silicon valley to enablemulti-domain command and control 2018

P25 Shahin et al. Conference The Intersection of Continuous Deployment andArchitecting Process: Practitioners’ Perspectives 2016

P26 Chen Conference Towards Architecting for Continuous Delivery 2015P27 Fazal-Baqaie et al. Conference Towards DevOps in multi-provider projects 2017

P28 Lwakatare et al. Conference Towards DevOps in the embedded systemsdomain: Why is it so hard? 2016

P29 Snyder et al. Journal Using Analytics to Guide Improvement duringan Agile-DevOps Transformation 2017


DevOps

Management

Architecture

Tooling

Automation

Specifications

Team

Culture Organisation, culture, changesDevOps communicationManagers, teams, projects

Complexity, diagnose,latency, design, logability

Cross teams, dependenciesChanges, processes

One tool does not fit all needsPackage managementRelease managementConfiguration & deploymentSystem monitoringLog

Technology, tools, data-flow

Level 1 Level 2 Level 3 Level 4 Paper mapping

[P5][P7][P9][P10][P27][P13][P27]

[P11][P12][P17]

[P18][P24][P27]

[P24][P6][P27]

[P18][P18][P18][P18][P18][P18]

[P20][P28]

the chain of evidence

Environments

Communication

Organization

PipelineDeploymentTechnology

Tools alternativeComponents

ProcessesData storageDeploy

DocumentsInformation Missing/inadequate

documentation[P2][P8]

[P22][P23]Hardware,Configurations,Changes

Systems,testing ENV,

visibility

[P3][P4][P24] [P27][P28][P29]

Config

Design [P1]

Figure 3.2: The collected codes mapping to primary studies

3.5.1 Data synthesis

In our study, the “thematic coding approach (Robson 2011)” is chosen to do the datasynthesis.Mechanism for coding the data is presented as follows:

1. Initial reading the paper content.

2. Identify specific segments of text.

3. Label the segments of text.

4. Reduce overlap and translate codes into themes.

5. Create a model of higher-order themes.

Motivation of selecting the “thematic coding approach”:

• Flexible to use with virtually all types of qualitative data.

• Relatively easy and quick method to learn and use compared to other methods.

• Accessible to researchers with little or no experience of qualitative research.

• The result of analysis has no difficulties to communicate with practitioners.

• Be able to handle large amount of qualitative data.

• Can be used in a wide variety of fields and disciplines (Robson 2011).


Comparisons to the other methods:Grounded theory approach:

• Used to develop a theory grounded in the data

• It a thematic coding, but the codes are from interaction with the data.

• Codes generated based on researchers’ interpretations of the meaning or pat-terns in the texts.

Quasi-statistical approach:

• Uses word or phrase frequencies and inter-correlations as key methods of de-termining the relative importance of terms and concepts (Robson 2011).

• Focus on content analysis.

3.5.2 Synthesis procedure

The synthesis procedure is presented in Figure 3.3.

1. Initial readingof selectedpapers' text

2. Identify specificsegments of text

3. Label thesegments of text

4. Remove overlap translate codes to

themes

5. Create high-level themes

thematic coding procedure

An e

xam

ple

of th

emat

ic a

naly

sis

Papers:- [P18]

Segments: e.g. Complexity indeployment due to thelarge number of services.

Label: e.g. 'complexity','deploy', 'services'

theme: e.g. 'challenge':'The number of servicesaffect deploy complexity'

codes-themes: e.g.'deployment complexity','services'

Figure 3.3: The thematic coding processes with an example

As shown in Figure 3.3, there are five steps to conduct the data synthesis basedon the collected data.

1. The first step is to read the source papers, for example, the paper P18 in Table3.3.


2. The second step is to identify the segment text from the target paper, e.g.“high complexity in deployment due to the large number of services”.

3. The third step is to label the segment with keywords, e.g. complexity, deployetc.

4. The fourth step is to translate the code to themes, e.g. “deployment complex-ity”, number of “services”.

5. The last step is to generate the high-level themes, e.g. The number of servicesin DevOps cluster affect the complexity of software deployment.

3.6 Validity threatsWe designed, refined, and validated the case study processes. Nonetheless, there arepotential threats to the validity of the case study.

Construct validity : The SLR steps are defined and refined, and formal data anal-ysis methods are applied.

Internal validity : The analysis of qualitative data requires interpretations, whichis subjective to potential bias. However, we take the actions below to triangulate theresults:

• Collect data from multiple sources.

• Pilot and validate pilot search during design phase by other researchers.

External validity : We defined filter criteria to select primary studies.Conclusion validity : The data collection process and data analysis steps are

demonstrated with sufficient information in order to improve the reliability of mea-sures and the evidence-of-chain of the conclusions. We kept track of all the collectedand analysed data to improve the reliability of the result.

Chapter 4

Results

In this section, we present the results of analysis to answer our research questions.The results are based on the synthesised data directly with our interpretations.

4.1 RQ1:what challenges have been reported on adopt-ing DevOps in practices for software develop-ment?

We identified a list of challenges through this SLR as shown in Table 4.1:As shown in Table 4.1, we identified 42 challenges (from C1 to C42) in nine differ-

ent software management areas extracted from the primary studies. These nine areasare team management, leadership culture, communication, strategy management, ar-chitecture, technology of tools, environment and automation, and documentation.

Based on the number of challenges located in different areas (architecture:10,tools:8, automation:6, documentation:6, environment:5, team management:4, lead-ership culture:1, communication:1, strategy management:1), this pattern indicatesthat software architecture is the most challenging part while adopting the DevOpsin software development. The second level of influences are tools, automation, doc-umentation, and environment, and the third layer is the team, culture, strategymanagement, and communication.

4.2 RQ1.1:What aspects of the software develop-ment have been covered by the reported chal-lenges in practices?

We mapped all the identified challenges to the software development lifecycle, andthe areas of software development, which were covered by our challenges, are shownin Figure 4.1.

As shown in Figure 4.1, our identified challenges of applying the DevOps con-nect through the full software development lifecycle stages which are “requirement,”“design,” “implementation,” “testing,” “release,” “maintenance.”

For the requirement phase, three areas are covered, e.g. documentation of therequirement, tools used to elicit requirements from customer and track requirementchanges, and culture of requirement management.

21

22 Chapter 4. Results

Table 4.1: A list of challenges identified in this studyID Challenge description Software

development

C1 Managers are challenged to bring team memberscloser together for achieving project success Team

managementC2 Lacking of management and guidance for DevOps teamC3 The resistance to change the existing way of working.C4 Integrating operations activities into a development team is a great challenge

C5 Deep-seated company culture Leadershipculture

C6 Insufficient communication communication

C7 Industry constraints and feasibility Strategymanagement

C8 Complexity in deployment due to the large number of services

Architecture

C9 The need to monitor execution to diagnose hardware and software failures quickly

C10 Network latency is a problem when services communicatewith each other a lot over the network

C11 Cognitive load of the extra complexity brought by the microservices architecture

C12 highly coupled architecture presents the biggest challenge whenorganizations are going to move to continuous deployment practice

C13 DevOps logability requires a well design to ensure performance

C14 DevOps monitorability and testability are necessaryto secure quality and readiness of release

C15 Migrating old architecture design to DevOps is a challenge

C16 Crossing team dependencies result in significant challenges for teammembers to make appropriate decision for deploying software continuously

C17 DevOps is unclear but also evolvingC18 One tool does not fit all needs

Technology- tools

C19 Software package management, e.g. Docker, Artifactory

C20 Service release management, e.g. change management,release approval, and release automation

C21 System configuration and deployment management,e.g. Jenkins, Puppet, Vagrant, Ansible etc.

C22 System monitoring, e.g. Nagios/Icinga, Monit, Collectd/Collectl.

C23 Log file analysis for diagnosis purposes,e.g. ELK (Elasticsearch, Logstash and Kibana).

C24 A tooling stack, e.g. a combination of an IDE (e.g. IntelliJ for Java),Git, JIRA, Confluence, Jenkins, and SonarQube.

C25 Service registration and discovery: e.g.Consul, Zookeeper, etc.

C26 Hard to ensure traceability of the events reported with thecomponents being created and deployed in the DevOps pipeline

Technology- automation

C27 Automation of processes:it is necessary to ensure that all datais easily collected and correlated via automated mechanisms.

C28 Data storage lifespan: the ability of keeping historical data.C29 Absence of feature usage data in system performance data

C30 Lack of technology to automatically and reliably deploynew features repeatedly in customer specific environments

C31 Reusability of libraries and functionsin the automation development

C32How to provide just-enough adequate specifications for reasoningabout architectural problems, supporting planning activities,and defining interfaces between team members?

DocumentationC33 One of the main challenges is dealing with software specificationsand documentation in a continuous development flow

C34 missing documentation is most problematic in large systemsC35 When to create documents and how much is enoughC36 Documentation version controlled, incompleteness, and dead documents.C37 Architecture principles and practices’ classifications are missing.C38 Hardware dependency and multiple version compatibility

Technology- environments

C39 Limited visibility of customers’ environment when configuring test environment

C40 the influence of ever-changing environments andtools on architecture design to enable CD practice

C41 Heterogeneous environmentsC42 Monolithic systems cannot be executed independently.


SoftwareDevelopment

lifecycle

Requirements Design Implementation Testing Release & Deployment Maintenance

Documents

Tools

Architecture

Culture

Communication

Automation

Tools

Teamsmanagement

Leadership

Automation

Tools

Teamsmanagement

Leadership

Automation

Tools

Strategy

Documents

Figure 4.1: The challenges mapped in software development lifecycle

For the software design step, the challenges are in the architecture design, com-munication among teams and organisations, the strategy of design.

In both, the implementation and the testing phase, the covered areas are devel-opment automation, tools, leadership and team management.

In the release and deployment phase, the challenges are located in automation ofdelivery and deployment, as well as tools.

In the maintenance phase, the focused area is the documentation of tracking thebugs or errors, which can be used to improve the software products in the nextiteration.

4.3 RQ2:What are the necessary considerations totake into account when implementing DevOpsin an organisation?

During this study we observed, that there is a set of principles and tools that con-tribute positively to organisations that are involved in the software development.Given the technical abilities of both development and operation teams, what appearsto be the core requirements to a successful implementation of DevOps practices are:

• Culture shift is the most impactful element of DevOps adoption. The siloedand monolithic configuration will not have enough flexibility that will enablean organisation to benefit from the DevOps practices.

• The ability to form multi-skilled teams, i.e. the programmer needs to havesome basic skills of analytic and measurement, in order to understand and beable to offer solutions. Understanding the underlined singularity of each of theelements that contributes to the success or failure becomes of vital importanceto each team member.


• The top management is the ultimate driver to the DevOps adoption within theorganisation. They ought to create a collaborative environment, where staffcan expand their set of knowledge/skills. Sharing and understanding the suc-cess, but most importantly the failures. To build such environment one of theelements at disposal of the managers is the adoption and the implementationof incentives and rewards based on collective achievements (Merchant & derStede 2012).

• Observation is an essential part of the implementation. Observing and un-derstanding where bottleneck may lay, could give a clear indication of directtargets and helping prioritise the initiatives. This can be achieved throughValue Stream Mapping.

• Telemetry allows to evaluate at any given moment the status of the entiresystem. Monitoring activities becomes an advantage, to detect and fix issuesrapidly.

• Automation is key. Repetitive tasks require a good amount of time to beperformed. Boring and frustrating tasks should be not performed manually.

• Start small and iterate. Selecting a small team that does not have a huge impacton the rest of the organisation. The learning lesson gathered from this can beapplied on a bigger team and gradually adopted by the entire organisation.

4.4 RQ3: What are the tangible benefits of apply-ing DevOps in the software development envi-ronment?

In this study we have conducted an SLR analysing a substantial number of publica-tions, however we did not find a clear evidence of the potential benefits that DevOpscould have on a firm within the software development manufacturing. Despite thenot so strong indications about the benefits, several considered studies are implicitlysuggesting that a more collaborative and experimental culture, where automation ispracticed to avoid repetitive and frustrating tasks, could have tangible benefits forthe whole organization.

In addition to the publications retrieved and used for the SRL, we have alsoexamined the so-called “grey literature” including researches, articles, books andcase studies that proved some benefits of DevOps initiatives.

Taking in account that the DevOps practice is still in its adolescence, it is fairto assume that more researches will be conducted in future. However, the authorssuggest that a study such as “The Accelerate State of DevOps Report” (referencehere), being the largest and longest running research conducted on DevOps, should betaken into consideration too. This report has been tracking and surveying technicalprofessionals worldwide, with over 30,000 survey responses in the past five years.

Furthermore, by simply conducting a Google search for “DevOps case study”there are currently (year 2019) about 7,340,000 results. In the result pages it’s easy


to recognize names such Netflix, Amazon, Esty, Capital One, ING bank, Infosys,Spotify, Facebook, just to mention few. Despite not being a scientific acceptedmethodology, what the Google search shows, is that the adoption of DevOps is notlimited to well define industries. This is also confirmed by The Accelerate State ofDevOps Report 2018, only 40% of the company surveyed are tech companies, therest includes financial services, retail, government, media, healthcare, education andothers.

According to the results of this report the tangible benefits are:

• Deployment frequency,how often a company deploy code can be increased fromonce a month to on-demand.

• Lead time for changes, the interval of time between a commit and running thecode in production. This can increase from every six month to less than anhour.

• Time to restore services, the time that generally takes to restore a service whenan incident occurs. With DevOps practices the recovery time can be reducedfrom one week to less than an hour.

• Change failure rate, this estimate the percentage of changes that can degradeservices that consequently lead to service outage, rollback, patch and so forth.Companies the employ DevOps can be up to 7 times faster in dealing with suchissues.

Chapter 5

Discussion

In this section, we present the interpretation and reflections according to the results.Generally, applying DevOps affects the entire organisation, and the challenges

exist in each of them. Through this study, we have found 42 challenges that preventDevOps been adopted in industrial organisations including software architecture,development, test, release, and support teams.

As shown in the Figure 5.1, the 42 identified challenges are mapped with the or-ganisations of a software industry. The 10 challenges are in the software architecturearea which is higher than other organisations, and it indicates that the architectureteams get the highest impact while adopting DevOps in a software company. Therelevant reasons are that, firstly, the DevOps adoption basically changes all existingroutines and habits in software development, test and release, and as a result, thearchitecture design needs a lot of effort to align to the DevOps workflow. Secondly,the automation pipeline for software implementation, testing, and delivery requiresthe architecture teams to rethink the components’ interactions and interfaces of asystem. In addition, these revolution changes have to be well communicated withcross functional teams and applied in a limited time plan.

By looking at the technology perspective, there are 19 challenges unveiled includ-ing software tools, process automation, and environments (e.g. develop and testingenvironment etc.). In the 19 challenges, the main issues are linked to the continuousintegration, continuous delivery, and continuous deployment tools, pipeline automa-tion, and environment. The reason is that DevOps connects the software develop-ment and operations, and automation is the key factor to remove the repetitive work,which means that there are more software tools introduced for automation, and thatthe configuration together with the maintenance require tremendous long workinghours in a company. Therefore, appropriate guidelines on how to adopt DevOps intocompanies and associated tools or technologies should also be well trained for thepractitioners in the company.

Furthermore, in our results, the documentation is also an important area to con-sider when companies adopt DevOps in practices. As shown in Figure 5.1, there are 6challenges that have been identified. The documentation includes requirement speci-fications, instruction of the continuous development flow, version control documents,continuous delivery guidelines, and design principles and practices etc. Our reflectionon these challenges is that documentation is very helpful for industrial practitionersto apply DevOps, and also it is useful to transfer knowledge to software architects,developers, testers, integration engineers, release engineers, and support engineers.Besides, the documentation is also a good way to remove the silos of communication

26

27 Chapter 5. Discussion

Strategy

10

8

65

1

Architecture

Technology:tools

Technology:automation

Industrialorganization

Technology:environment

6

4

1

1

42 identifiedchallenges

Documentation

Teams

Culture

Communication

Figure 5.1: The identified DevOps challenges in industry

among different teams or organisations in a company.Additionally, the leadership strategy and culture are also affected by adopting De-

vOps practices, and each of them has one challenge found in industrial cases. Thesetwo areas should be considered carefully; the reason is that we have mentioned De-vOps changes all organisations once it is applied, including the management teamsin a company. The success of applying DevOps needs to start from the top man-agement. They ought to create a collaborative environment, where staff can expandtheir knowledge/skills. Sharing and understanding the success (Merchant & derStede 2012).

The leadership strategy and culture impact each member in an organisation andtheir productivity performance as well. Organisations need to be well prepared tohandle technical and social adoption challenges with their existing expertise, pro-cesses and tools before adopting the CD process (Claps et al. 2015).

In summary, there is a list of challenges existing while adopting DevOps in asoftware companies. In our study, these identified challenges are important factorsthat can be taken into consideration for industrial practitioners prior DevOps prac-tices are implemented. Meanwhile, implementing DevOps methodology despite thechallenges indicates some tangible benefits, that can be also something to consider

28 Chapter 5. Discussion

while considering the adoption. As the adoption rate of DevOps increases within theindustries, more valuable the presented challenges could contribute.

This study for some aspects can be considered an hybrid due the mix of SRLand grey literature. However, in the authors’ opinion data and resources utilised aremitigating the existing gap between the scientific community and the practitionersconcerning DevOps. This also offer considerations for further studies and perhaps amore collaborative approach between them.

Chapter 6Conclusions and Future Work

In this section, we present our conclusions and future work.Through this study, we have found 42 challenges while adopting the DevOps in

software organisations, and these challenges have been mapped into different phasesof software development life cycle. Moreover, we have also presented the benefits ofapplying the DevOps in software industries for business cases.

We have analysed and categorised the collected challenges. They exist in differentphases of software development which illustrates that the DevOps has an impact onall teams while applying it in an industrial project.

In the software development life-cycle, the architecture is in the highest numberof challenges for DevOps adoption, and we found that the software architecture istightly connected to requirement management, software development, testing, andrelease. This is the main reason that it gets most of the impact by adopting DevOpspractices.

The second area that gets impacted by DevOps is the technology including De-vOps tools, automation pipeline, and testing environments etc. Thereby, the DevOpstechnology needs to get educated and trained to practitioners, and it also requires tobe documented in order to transfer and share knowledge or skills among industrialorganisations.

Moreover, the industrial leadership strategy and culture have a lower number ofchallenges found , but they are critical for software companies to get managementteams involved for a successful DevOps adoption.

All in all, the identified challenges in this paper could help industrial practitionersto apply DevOps in various business projects, and they also facilitate the speed ofDevOps adoption in software companies. However, the alternative solutions for eachchallenge are not investigated in this study, therefore, more case studies should beapplied for DevOps adoption in industrial cases as future work.

In addition, we have also presented the benefits of using DevOps methodology inpractices and in business cases. These benefits should be verified and validated inindustrial projects in the future studies.

29

Bibliography

Abdelkebir, S., Maleh, Y. & Belaissaoui, M. (2017), An agile framework for itsmanagement in organizations: A case study based on devops, in ‘Proceedingsof the 2nd International Conference on Computing and Wireless CommunicationSystems’, ACM, p. 67.

Al-Ruithe, M., Benkhelifa, E. & Hameed, K. (2018), ‘Key issues for embracing thecloud computing to adopt a digital transformation: A study of saudi public sector’,Procedia computer science 130, 1037–1043.

Andreessen, M. (2011), ‘Why software is eating the world’, Wall Street Journal20(2011), C2.

Bai, X., Li, M., Pei, D., Li, S. & Ye, D. (2018), Continuous delivery of personalized as-sessment and feedback in agile software engineering projects, in ‘2018 IEEE/ACM40th International Conference on Software Engineering: Software Engineering Ed-ucation and Training (ICSE-SEET)’, IEEE, pp. 58–67.

Claps, G. G., Svensson, R. B. & Aurum, A. (2015), ‘On the journey to continu-ous deployment: Technical and social challenges along the way’, Information andSoftware technology 57, 21–31.

Conboy, K. & Coyle, S. (2009), ‘People over process: the implications of agile for isskills and human resource management’.

De França, B. B. N., Jeronimo Junior, H. & Travassos, G. H. (2016), Characterizingdevops by hearing multiple voices, in ‘Proceedings of the 30th Brazilian Symposiumon Software Engineering’, ACM, pp. 53–62.

Deming, W. E. (2018), Out of the Crisis, MIT press.

DevOpsDays (2009), ‘About devopsdays’. Last accessed 16 April 2019.URL: https://www.devopsdays.org/about/

Erich, F., Amrit, C. & Daneva, M. (2017), ‘A qualitative study of devops usage inpractice’, Journal of Software: Evolution and Process 29(6), e1885.

Ferrier, W. J. (2001), ‘Navigating the competitive landscape: The drivers andconsequences of competitive aggressiveness’, Academy of management journal44(4), 858–877.

30

31 BIBLIOGRAPHY

Humble, J. & Farley, D. (2010), Continuous Delivery: Reliable Software Releasesthrough Build, Test, and Deployment Automation (Adobe Reader), Pearson Edu-cation.

Itkonen, J., Udd, R., Lassenius, C. & Lehtonen, T. (2016), Perceived benefits ofadopting continuous delivery practices, in ‘Proceedings of the 10th ACM/IEEEInternational Symposium on Empirical Software Engineering and Measurement’,ACM, p. 42.

Jabbari, R., bin Ali, N., Petersen, K. & Tanveer, B. (2016), What is devops?: Asystematic mapping study on definitions and practices, in ‘Proceedings of theScientific Workshop Proceedings of XP2016’, ACM, p. 12.

Jesse, N. (2018), ‘Organizational evolution-how digital disruption enforces organiza-tional agility’, IFAC-PapersOnLine 51(30), 486–491.

Keizer, G. (2009), ‘Tomtom launches $100 gps app for iphone’.URL: https://www.computerworld.com/article/2527054/tomtom-launches–100-gps-app-for-iphone.html

Kellner, C. (2017 (accessed April 14, 2019)), ‘A $100 billion idea: Ge’s jeff immelttalks to cnbc’s jim cramer about industry’s digital transformation’.URL: https://www.ge.com/reports/100-billion-idea-ges-jeff-immelt-talks-cnbcs-jim-cramer-industrys-digital-transformation/

Kim, G., Behr, K. & Spafford, K. (2014), The phoenix project: A novel about IT,DevOps, and helping your business win, IT Revolution.

Kim, G., Humble, J., Debois, P. & Willis, J. (2016), The DevOps Handbook:: How toCreate World-Class Agility, Reliability, and Security in Technology Organizations,IT Revolution.

Kitchenham, B. & Charters, S. (2007), ‘Guidelines for performing systematic litera-ture reviews in software engineering’.

Kotter, J. P. et al. (1995), ‘Leading change: Why transformation efforts fail’.

Laukkarinen, T., Kuusinen, K. & Mikkonen, T. (2018), ‘Regulated software meetsdevops’, Information and Software Technology 97, 176–178.

Matković, P. & Tumbas, P. (2010), ‘A comparative overview of the evolution ofsoftware development models’, International Journal of Industrial Engineering andManagement (IJIEM) 1, 163–172.

Merchant, K. A. & der Stede, W. A. V. (2012), Management control systems: per-formance measurement, evaluation and incentives, Pearson Education.

Nagarajan, A. D. & Overbeek, S. J. (2018), A devops implementation frameworkfor large agile-based financial organizations, in ‘OTM Confederated InternationalConferences" On the Move to Meaningful Internet Systems"’, Springer, pp. 172–188.

32 BIBLIOGRAPHY

Paul Hammond, J. A. (2009), ‘10+ deploys per day: Dev and ops cooperation atflickr’. Last accessed 16 April 2019.URL: https://cdn.oreillystatic.com

Poppendieck, M. & Poppendieck, T. (2007), Implementing lean software develop-ment: From concept to cash, Pearson Education.

Premchand, A., Sandhya, M. & Sankar, S. (2019), ‘Simplification of applicationoperations using cloud and DevOps’, Indonesian Journal of Electrical Engineeringand Computer Science 13, 85–93.

Rajkumar, M., Pole, A. K., Adige, V. S. & Mahanta, P. (2016), Devops cultureand its impact on cloud delivery and software development, in ‘2016 Interna-tional Conference on Advances in Computing, Communication, & Automation(ICACCA)(Spring)’, IEEE, pp. 1–6.

Ravichandran, T. (2018), ‘Exploring the relationships between it competence, inno-vation capacity and organizational agility’, The Journal of Strategic InformationSystems 27(1), 22–42.

Robson, C. (2011), Real world research, Vol. 3, Wiley Chichester.

Royce, W. W. (1987), Managing the development of large software systems: conceptsand techniques, in ‘Proceedings of the 9th international conference on SoftwareEngineering’, IEEE Computer Society Press, pp. 328–338.

Schwaber, K. & Beedle, M. (2002), Agile software development with Scrum, Vol. 1,Prentice Hall Upper Saddle River.

Sherehiy, B. & Karwowski, W. (2014), ‘The relationship between work organizationand workforce agility in small manufacturing enterprises’, International Journal ofIndustrial Ergonomics 44(3), 466–473.

Smart, J. (2018), ‘To transform to have agility, dont do a capital a, capital t agiletransformation’, IEEE Software 35(6), 56–60.

Taryana, A., Setiawan, I., Fadli, A. & Murdyantoro, E. (2017), Pioneering the au-tomation of lnternal quality assurance system of higher education (iqas-he) usingdevops approach, in ‘2017 International Conference on Sustainable InformationEngineering and Technology (SIET)’, IEEE, pp. 259–264.

VersionOne, C. (2019), ‘State of agile survey’. Last accessed 16 April 2019.URL: https://www.stateofagile.com/#ufh-i-521251909-13th-annual-state-of-agile-report/473508

Wettinger, J., Breitenbücher, U. & Leymann, F. (2014), Standards-based devopsautomation and integration using tosca, in ‘2014 IEEE/ACM 7th InternationalConference on Utility and Cloud Computing’, IEEE, pp. 59–68.

White, G., Thomas, J., Weldon, P., Lawrence, A., Galatis, H. & Tyndall, J. (2013),‘Grey literature in australian education’, Grey Journal 9(2), 103–108.

Willis, J. (2012), ‘Devops culture’. Last accessed 16 April 2019.URL: https://itrevolution.com/devops-culture-part-1/

Winter, J., Rönkkö, K. & Rissanen, M. (2014), ‘Identifying organizational barriers—acase study of usability work when developing software in the automation industry’,Journal of Systems and Software 88, 54–73.

Zhu, L., Bass, L. & Champlin-Scharff, G. (2016), ‘Devops and its practices’, IEEESoftware 33(3), 32–34.

Faculty of Industrial Economics, Blekinge Institute of Technology, 371 79 Karlskrona,Sweden