the state of agile development adoption - 2014

Contents Executive Summary 4 Purpose & Methodology 6 Respondents 9

Company Size 10 Job Role 11 Industry 11 Organizational Experience with Agile 12 Geography 12

Findings 13 The Agile Capability Gap 14 The Benefits of Agile 17 The Barriers to Agile 22 The Role of Experience 26 The Influence of Job Role on Agile 29 Lean Startup Driving Agile 30 Agile Segments 33

Conclusion 35 Appendix 37

Definitions of Statistical Terms 38 Factor Analysis Table Benefits 38 Factor Analysis Table Barriers 39 Agile Perceptions By Experience 39

About Intelliware 40

3 © 2014 Intelliware Development Inc.

Executive Summary

A gile software development has been around for over a

decade. While many now consider it to be a mainstream approach, there is a need for deeper understanding about how best to make it work for organizations. To gain a deeper understanding of the key drivers of success, challenges and relationships that various factors play in the successful adoption of Agile, Intelliware designed a survey on the state of Agile adoption in organizations today. We surveyed 117 individuals representing a broad range of industries, job roles, company sizes, and experience with Agile. Given our local presence in Toronto, Canada, the majority of our respondents – approximately 85% - were in Canada. There are several key findings from this survey: •  “The ability to adapt to changing priorities” is the most

common benefit of Agile. •  Close collaboration leads to more adaptive organizations.

There is a significant positive correlation between the Agile benefits of “greater collaboration” and “ability to adapt to changing priorities” with an organization’s success with Agile.

•  Organizations want to adopt Agile, but often aren’t having a great deal of success. There’s a significant gap between the importance organizations place on Agile and their capability with it.

•  Augmenting your existing team by hiring full-time staff with Agile is only part of the equation. Agile experience alone cannot be relied on, as we found that experience drives Agile capability but only to a limited extent.

•  Lip-service doesn’t work. “Approach – using Agile elements in a non-Agile framework” was the most frequently selected barrier in our survey; 24% of respondents stated this as the lone barrier that is inhibiting further adoption of Agile.

•  Lean Startup is driving Agile adoption, but it’s mostly with organizations that have been practicing Agile for some time.

Executive Summary


Purpose & Methodolgy

Purpose

A gile software development practices are becoming

increasingly prevalent these days. Articles and blogs abound discussing the pros and cons of this approach to software development. In fact, now the term “Agile” is being attached to all sorts of other business processes, ranging from business strategy to marketing*. To gain a deeper understanding of the key drivers of success, challenges, and relationships that various factors play in the successful adoption of Agile, Intelliware designed a survey on the state of Agile adoption in organizations today.

Methodology

This survey and corresponding analysis is intended to provide a general benchmark of the state of Agile adoption. We asked 6 broad questions:

•  How long has your organization been using Agile software development methodologies?

•  How important are Agile practices to your overall business strategy?

•  How would you rate your organization’s Agile capabilities?

•  What benefits of Agile practices are important to your organization?

•  What is inhibiting further adoption of Agile practices in your organization?

Purpose & Methodology


•  How much influence have “Lean Startup” concepts influenced your organization’s desire to adopt Agile development practices?

The survey was short enough to help generate a sufficient number of survey responses but long enough to capture enough data that would allow for a meaningful analysis. These questions provide the opportunity for us to get a better understanding of the business drivers, capabilities and challenges companies are facing in their adoption and use of Agile practices. We also included a question on Lean Startup, since this method is gaining traction as a viable alternative to traditional product development and employs several Agile practices. To add to the analysis, we wanted to learn more about the companies represented by the survey respondents. We researched the respondents’ companies to identify company size and industry. Company size was categorized by number of staff: Small-to-medium sized businesses (SMBs) = <100, Midmarket = 100-999 and Enterprise = 1000+. While we saw some variance by industry and company size, our sample size was not large enough to report conclusive findings based on these variables. Throughout the report, we refer to several statistical techniques. For definitions of these techniques, please see the Appendix.

Purpose & Methodology


*Books such as Lean Marketing for Startups: Agile Product Development, Business Model Design, Web Analytics, and Other Keys to Rapid Growth by Sean Ellis and Marketing Agencies that tout Lean or Agile Marketing practices are appearing more frequently than ever before.

Respondents

Respondents

O ver the course of 7 weeks, we surveyed 117 individuals

representing a broad range of industries, job roles, and company sizes. Given our local presence in Toronto, Ontario, Canada, the majority of our respondents – approximately 85% - were in Canada.

At a high level, respondents represent: •  A variety of job roles and companies of all sizes •  Multiple industries, such as banking, insurance, ICT

(information, communication and technology), retail •  Organizations with varied Agile experience. On average,

respondents represent companies with just over two years of experience with Agile

Company Size


34%SMB (100 or fewer staff)

39%Midmarket (101-1000 staff):

27%Enterprise (1000+ staff)

Respondents

Senior Management

24%

Developer

23%Project Management

18%Middle Management

15%

Change Agent

9%Other

11%

Job Role

Industry

10%Financial Services - Banking

10%Financial Services – Insurance

4%Consulting

11%Other

10%Unknown

55%Information Communications and Technology (ICT)


Respondents

Geography

85%Canada

15%International

Organizational Experience with Agile


Findings

B ased on the results, there is a gap between

organizational capability with Agile and the importance of Agile to those companies. In general, people do not believe that their Agile capabilities are on par with Agile’s importance to their organization. We believe that importance can be seen as a proxy for expected results from Agile. Therefore, this could also have been interpreted as the gap between the desired outcome from Agile practices and the ability to achieve this outcome. The following pages provide an overview of this gap as well as an exploration of the influences of organizational capability based on these 5 variables:

•  Benefits of Agile •  Barriers to Agile •  Practice Makes Perfect? The Role of Experience •  The Influence of Job Role on Agile •  Agile Segments - Two Groups

Findings


The Agile Capability Gap

R espondents were asked to rate how important Agile

practices are to their organization’s overall business strategy as well as how capable their organization is with Agile. The data shows a gap between the importance an organization places on Agile and its competence with it. The average score for Agile’s importance to business strategy was 6.82 out of 10 yet the average capability rating was only 5.66 out of 10. Figure 1 illustrates how respondents stated organizational capabilities are not in line with the importance of Agile for their organization. While the significance of this gap can be debated in subjective terms, there is a significant gap between them in statistical terms.* This gap could be caused by a number of things. Perhaps the barriers to Agile are getting in our way? Maybe many of the benefits of Agile are recognized as important but remain unrealized? Or, could it be that we are simply our own worst critic?

The Agile Capability Gap

5.66 6.82 0 1 2 3 4 5 6 7 8

Capabilty Importance

The Agile Gap

Figure 1: The gap between organizational capability with Agile versus the importance of Agile

*At the 95% confidence interval, importance scores would range from 6.35 to 7.29 and capability scores would range from 5.20 to 6.11.


The Benefits of Agile

Respondents were asked to choose which benefits of Agile are important to their organization. The results indicate diverse benefits of Agile. From most to least common, respondents reported the following benefits: ability to adapt to changing priorities; greater collaboration; faster time-to-market; reduced risk; better fit of features to product needs; and lower costs. One of the most interesting findings is that there is a relatively strong relationship between the two benefits of “greater collaboration” and the “ability to adapt to changing priorities” and the importance of Agile practices to overall business strategy.* In other words, organizations that believe Agile is important place a higher value on collaboration and adaptability. The same is true for capability with Agile.


Ability to adapt to changing priorities 78%Greater collaboration 62%

Faster time-to-market 60%Better fit of features to product needs 53%

Reduced risk 53%Lower costs 27%

Other 22%

The Benefits of Agile*

*Percentage of respondents who agreed that these Agile benefits are of importance to their firm


Further, there are relationships among some of the benefits of Agile (see Figure 2). While none of the benefits have a strong correlation (i.e. nothing above 0.5) with one another, there are several that are positively correlated with statistical significance (between .20 and .35), ), such as “better fit of features to product needs” and the ability to adapt to changing priorities”.** Other Agile benefits that have virtually no correlation to one another such as “lower costs” and “better fit of features to product needs” are not listed. Understanding the relationship between two benefits of Agile is helpful to get a sense of which benefits go well with one another. But alone, they leave an unclear picture of how these various benefits of Agile relate to broader business benefits.


Benefits by Correlation Correlation

Fit to product needs & the ability to adapt 0.320

Fit to product needs & time to market 0.311

Reduced risk & ability to adapt 0.279

Lower costs & time to market 0.268

Reduced risk & greater collaboration 0.259

Reduced risk & fit to product needs 0.245

Figure 2: Benefits with at least a moderate level of correlation, i.e. > 0.2. Note that correlation ranges from +1 between (perfect correlation) to –1 (perfect negative correlation). 0 indicates no relationship.


Consequently, we grouped these benefits together to create related “factors”, or broader categories of benefits. Using factor analysis, we found that the benefits listed in our survey could be loosely grouped into the following three factors:

•  Software quality •  Speed and agility •  Resources: saving time and money

“Software quality” includes four variables (or benefits): better fit of features of product needs, reduced risk, greater collaboration, and ability to adapt to changing priorities. “Speed and agility” includes time to market and ability to adapt. “Resources” include lower costs and faster time to market, which could be interpreted as saving staff time as well as satisfying the market with releasing products faster. These three factors show the true breadth of Agile benefits: effectiveness (software quality), efficiency (speed and agility) and money (resources).*** “Greater Collaboration” has a 0.30 correlation and 0.001 significance with Agile importance and has a 0.35 correlation and 0.000 significance with Agile capability. The “ability to adapt to changing priorities” has a 0.35 correlation and 0.000 significance with Agile importance and has a 0.31 correlation and 0.001 significance with Agile capability.

**Statistically significant at the 0.05 level, i.e. p<.05.

***To see the factor scores, see the Appendix.



The Barriers to Agile

Respondents were asked to choose which barriers are inhibiting further Agile adoption for their organization. The barriers to Agile adoption prove to be as diverse as the benefits. From most to least common, respondents reported the following barriers: approach – using Agile elements in a non-Agile framework; staff – not having the team setup or skill set for Agile; culture – not currently suitable to Agile; office layout – problems setting up project rooms; and management – not supportive of Agile. One of the most interesting findings is that the barrier of "management – not supportive of Agile" is negatively


The Barriers to Agile*

* Percentage of respondents who agreed that these barriers are inhibiting further Agile adoption in their organization


Approach – using Agile elements in a non-Agile 46%

Staff – not having the team setup or skill set for Agile 44%

Culture – not currently suitable to Agile 39%

Office layout – problems setting up project rooms 26%

Management – not supportive 23%Other 22%

correlated with both the perceived importance of, and capability with, Agile.* In other words, those who see management as a barrier to further Agile adoption in their organization are more likely to say that their organization places lower importance on Agile and are less capable with Agile relative to their peers. This is an unsurprising yet interesting finding because it shows how critical it is to have management support for Agile. A grassroots Agile effort may be the easiest way to implement Agile in an organization but it is by no means guarantees success if management isn’t involved. The barrier of “approach – using Agile elements in a non-Agile framework” is noteworthy. Interestingly, while it was the most frequently selected barrier in our survey (46% of respondents), it is the only barrier that does not have any significant correlation (p<0.05) with another barrier. Sixty three percent of those who selected “approach” as a barrier selected two or fewer barriers that are inhibiting further adoption of Agile practices at their organization, with 24% selecting approach alone. In other words, there are many people who find that their organization’s approach to Agile is the only or just one of a few barriers they face to improve Agile adoption. Using factor analysis again, we found that the barriers listed in this survey could be loosely grouped into the following two factors:

1.  Practice barriers 2.  Environmental barriers



“Practice barriers” include only two important variables (or barriers): approach and culture. “Environmental Barriers” include four important variables: staff, office layout, management, and culture. In essence, what’s getting in the way of Agile adoption appears to be either the environment in which one is trying to be Agile or the approach that one is taking with Agile.** “Management – not supportive of Agile” has a -0.39 correlation and 0.000 significance with Agile importance and has a -0.37 correlation and 0.000 significance with Agile capability. **To see the factor scores, see the Appendix.



Does Practice Make Perfect? The Role of Experience

G iven the gap between capability and importance with

Agile, we looked into the role of experience. Results indicate experience drives Agile capability but only to a limited extent. In the first year of implementing Agile, the average respondent rated organizational Agile capability at 5 out of 10. After more than a year of experience with Agile, the average increased to over 6 out of 10. Then, it appears organizations remain stuck at this level for a few years before they moderately improve to 7 out of 10.

The Role of Experience

2.44

5 6.19 6.22

7.06

3.62

6.93 7.29 7.11 8.12

0 1 2 3 4 5 6 7 8 9

10

Rat

ing

Agile Experience

Capability

Importance

Practice Makes Perfect? The Role of Experience

Figure 3: Capability and importance ratings by Agile experience


While greater experience with Agile generally leads to better Agile capabilities, the rate of improvement is underwhelming. One might expect capability to increase more steadily with experience. It seems as though organizations cannot rely on Agile experience alone to improve their capabilities. One theory is this is due to the common barriers to Agile, such as culture, approach, and staff (as described in “Barriers to Agile” section). Regarding the perceived importance of Agile, the good news is that experience and average perceived importance are positively correlated. That is, the greater one’s experience with Agile, the more important it becomes. The bad news is that the rate of growth is rather slow – only about 7 percent per year after the first year with Agile. This may be due to the results or challenges from gaining experience with Agile. It’s worth noting that this finding is based on the respondents’ perceptions. Consequently, it is possible that organizations are improving their capability at a steady and greater rate but are overestimating their organization’s Agile capabilities when new to Agile and are underestimating it once they are experienced. It is also possible that there is more than meets the eye with Agile and that those who are new to it underestimate what it entails.

The Role of Experience


The Influence of Job Role on Agile

P eople at all job levels share common views about Agile

(see Figure 4). While it’s encouraging that the results show that Agile is seen as at least moderately important across job levels, this may be the result of response bias. After all, those who respond to an Agile survey likely think that Agile is somewhat important. The findings are similar for organizational capabilities with Agile; moderately important across job levels but likely influenced by response bias (see Appendix for details). While we saw some variance in these ratings by job role, our sample size was not large enough to report any conclusive findings, as the margin of error is too large.

The Influence of Job Role on Agile

0 5 10

Other

Change Agent

Senior Management

Middle Management

Project Management

Developer

Rating

Agile Perceptions - By Job Role

Importance

Capability

Figure 4: Capability and Importance of Agile to Organizations Strategy – Perceptions By Job Role


Lean Startup: Driving Agile Adoptions Further, Not Encouraging New Adoptions

Lean Startup

F or those who are unfamiliar with Lean Startup, it is a

method for developing businesses and products and is primarily adopted for software development. Its central tenet is “validated learning”, which is a process to get end-user customer feedback and then iterate accordingly. The Lean Startup movement and Agile are viewed as complementary. The Lean Startup concept of validated learning goes well with the Agile principles of regularly reflecting on how to become more effective and of welcoming change. Both Agile and Lean Startup use iterative processes, from development to product releases. Given these similarities, we were curious to understand how much influence Lean Startup concepts are having on the organizational desire to adopt Agile development practices. We found that Lean Startup is influencing Agile adoptions, but to a limited and specific extent. When we asked survey respondents how much influence, on a scale of 1 to 10 with 10 being the highest, Lean Startup concepts have had on their organization’s desire to adopt Agile development practices, the median response was 4 and the most common response was a 1. However, despite this minimal impact, when we considered experience with Agile, the effect of Lean Startup on Agile became more evident. Experience with Agile is positively correlated with the perceived influence Lean Startup is having on Agile adoption. This implies that those who have been Agile for longer are more likely to belong to an organization whose Agile adoption rate is influenced by the Lean Startup concept. Figure 4 illustrates that for every experience category, the longer one has practiced Agile, the higher the level of influence that Lean Startup tends to have on their organization’s Agile adoption.


Lean Startup

Because the sample size was not large enough to report any conclusive findings, it seems as though industry sector may be an influence of Lean Startup on Agile adoptions. On average, ICT respondents provided a 4.22 rating whereas Financial Services respondents provided a 2.70.* *For the ICT rating, there is a 95% confidence level of a rating between 3.5 to 5.0. For the Financial Services industry, it is only 1.8 to 3.6 at the same confidence level.

2.37 2.8 3.83

4.78 4.94

0

1

2

3

4

5

6

None <1 year 1-2 years 3-5 years >5 years

Rat

ing

(out

of 1

0)

Agile Experience

Influence of the Lean Startup on Agile

Figure 5: Lean Startup's Influence on Agile


Agile Segments – Two Groups

W e explored whether there are any natural groups of people with similarities related to Agile. Using cluster analysis, we determined that while there were no tightly-knit clusters in our sample, there were two loosely-connected groups that can be described as follows:

•  The Veterans: Those who consider Agile to be among the top priorities for their organization, are very good with Agile, and are influenced by Lean Startup

•  The Rookies: Those who consider Agile as only having moderate importance to their organization, are subpar at it, and are not influenced by Lean Startup

The Veterans:

•  Experience: about 3 years •  Importance of Agile: 9 of 10 •  Capability with Agile: 7 of 10 •  Influence of Lean Startup on Agile: 6 of 10 •  Believe Agile helps their organization have greater

collaboration •  Do not believe staff or company culture are inhibiting

Agile adoption The Rookies:

•  Experience: about 1.5 years •  Importance of Agile: 5 of 10 •  Capability with Agile: 4 of 10 •  Influence of Lean Startup on Agile: 2 of 10 •  Do not believe Agile helps their organization have

greater collaboration •  Believe both staff and company culture are inhibiting

Agile adoption

Agile Segments


Conclusion

T he response data from this survey reveals several

insights into the adoption of Agile in today’s organization. There are two main areas that should be considered by anyone in an organization that is Agile or is considering this approach: 1.  Agile is enabling a more adaptive organization, and, 2.  Despite a will to be Agile, there is a gap between good

intentions and actual capability. Organizations are recognizing the imperative to be nimble and action-oriented in order to manage changing priorities. The results of this survey indicate that Agile is a key enabler for this: from “the ability to adapt to changing priorities” to “greater collaboration”, organizations are coming more adaptive because of Agile. Although Agile is helping companies become more adaptive, the Agile capability gap needs to be addressed. Organizations are headed in the right direction in terms of placing strategic value on Agile. Now they need to address their actual ability to implement and maintain a high-performing Agile culture. Hiring experienced Agile practitioners is only part of the solution. Other aspects of the organization must be addressed, most importantly, meaningful support from the top – not only in embracing the idea, but also by removing the barriers that inhibit Agile and creating an environment that supports Agile.

Conclusion


Appendix

Definitions of Statistical Terms Correlation: A statistical measure that indicates the extent to which two or more variables fluctuate together. A positive correlation means that the variables tend to be either large or small together. For example, education and income are positively correlated. A negative correlation means that as the value of one variable increases, the other(s) decrease(s). For example, TV viewing and grades (for students) are negatively correlated. Correlation ranges from +1 between (perfect correlation) to –1 (perfect negative correlation). Statistically significant: The probability that an effect is not likely due to just chance alone. P-values are often coupled to a significance or alpha (α) level, which is also set ahead of time, usually at 0.05 (5%), as they are in this report. Factor Analysis: a statistical method used to describe variability among observed, correlated variables in terms of a potentially lower number of unobserved variables called factors. It helps identify groups of related items (called factors). For example, in a decathlon, each athlete competes in 10 events, but several of them can be thought of as "speed" events, while others can be thought of as "strength" events, etc. Thus, a competitor's 10 event scores might be thought of as largely dependent on a smaller set of 3 or 4 types of athletic ability. Cluster Analysis: the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters).

Appendix


Appendix

Benefits Factor 1 (Software quality)

Factor 2 (Speed and agility)

Factor 3 (Resources: time & money)

Time To Market 0.182 0.541 0.533

Fit to Product Needs

0.606 0.329 -0.064

Reduced Risk 0.768 -0.144 0.1

Lower Costs 0.057 0.005 0.884

Greater Collaboration

0.563 0.035 0.235

Ability to Adapt 0.563 0.482 -0.241

Other 0.028 -0.839 -0.073

Factor Analysis Table – Benefits

The 3 Categories of Agile Benefits. Note that the figures listed above are “factor loadings”. The closer a factor loading is to 1, the more representative the variable is to the factor. E.g. Lower Costs is very representative of the factor “resources: time & money” with a factor loading of 0.884.

Factor Analysis Table – Barriers

Barriers Factor 1 – Environmental Barriers Factor 2 – Practice Barriers

Staff .749 -.017

Layout .631 -.048

Mgmt .516 .332

Approach -.197 .715

Culture .504 .511

OtherBarrier -.196 -.756


About Intelliware Development Inc. Intelliware is a custom software, mobile solutions and product development company headquartered in Toronto, Canada. Intelliware is a leader in Agile software development practices which ensure the delivery of timely, high quality solutions for clients. Intelliware is engaged as a technical partner by a wide range of national and global organizations in sectors that span Financial Services, Healthcare, ICT, Retail, Manufacturing, and Government.

About Intelliware

https://plus.google.com/+Intelliware

/intelliware_inc

/intelliware.inc

/company/intelliware-development-inc-

www.intelliware.com


About Intelliware


About the Authors Lawrence Ludlow is Director, Process at Intelliware Development. He has over 24 years experience managing technical projects, the last 16 in the software development field. Lawrence holds a Bachelors degree in Civil Engineering from the University of Waterloo and a Masters degree in Environmental Engineering from the University of Toronto. Lawrence is a Director and co-founder of the Toronto Agile Community group who organize the annual Agile Tour Toronto conference. For the past 10 years Lawrence has also been the administrator for the XP/Agile Toronto users group. Joe Kevens is a B2B marketing Specialist at Intelliware Development. He has 6 years experience in a variety of marketing and business development roles at PricewaterhouseCoopers, the Greater Toronto Marketing Alliance, and Eloqua. Joe holds a Bachelors degree in Commerce with a Marketing Management major from Ryerson University and a Masters degree in International Business from the Queen’s School of Business.