forrester_2015 state of agile development report

7/23/2019 Forrester_2015 State of Agile Development Report

http://slidepdf.com/reader/full/forrester2015-state-of-agile-development-report 1/42

The 2015 State Of Agile

Development

Forrester Research

Authors: Diego Lo Giudice

August 3, 2015



© 2015 Forrester Research, Inc. Reproduction Prohibited

Why read this report

• Agile development is high on every BT agenda, but many still struggle to be

truly Agile and achieve the real benefits beyond a single team’s experience.

• This 2015 online survey, contrary to many existing surveys in the market that

survey Agile and non-Agile practitioners, focuses on understanding the

strategies being adopted by 215 Agile practitioners from around the world.

• The survey primarily illustrates:

• What the benefits are, how success is measured, and which Agile practices

are adopted.

• Which Agile methods they prefer and use both upstream and downstream of

the application delivery-cycle process.

• How they deal with large enterprise scale issues and what the most common

barriers are.

• How teams organize and with which core and extended roles.

• What Agile testing strategies they adopt.




Agenda

• Demographics

• Agile practices and organization

• Agile testing and outsourcing

• Measuring Agile value

• Methodology and related research




27%

33%

28%

12%

I am responding on behalf of my team only

I am responding on behalf of more teams(departmental/divisional level)

I am responding for the entire technologymanagement organization (global level)

Other

Base: 215 professionals with knowledge of their firm’s Agile practices

Source: Forrester's Q2 2015 Global Agile Software Application Development Online Survey

215 respondents, 68% responding on behalf of their team, division,and enterprise

4

Sixty-one percent use Agile in

multiple teams at the departmental

and enterprise levels.

“Which of the following best describes the scope of your management authority?(Select one)”



© 2015 Forrester Research, Inc. Reproduction Prohibited 5

Primary group of respondents are executives: 69% of respondentsare VPs or managers; 8% are transformation leaders

Base: 190 professionals with knowledge of their firm’s Agile practices.


“What is your primary role? (Select one)”

Directors/managers: 48%

VPs/other execs: 21%

Deployment/ops: 1%

QA/testers: 2%

13%

1%

1%

2%

2%

3%

5%

6%

7%

8%

8%

21%

24%

Other (please specify)

Deployment or Operations (e.g., sysadmin, DBA,deployment manager)

Portfolio manager

DevOps manager

Software quality assurance manager (includingtesters)

Business analyst

Product manager

Development manager

Chief architect

Transformation or change leader

Program manager

Vice-president or C-level executive

Mid-level manager (e.g., Manager, Director, etc.)




“Which of the following best describes the industry to which your company

belongs? (Select one)”

In line with previous years, IT/BT industry participationleads. This year, 25% come from financial services orinsurance; 6% from retail and wholesale



27%

18%

16%

9%

8%

6%

5%

3%

3%

2%

3%

IT/BT professional services (System integration,outsourcing, consultancy, web agency)

Software products (ISV, SW Vendor)

Financial Services

Insurance

Public sector and healthcare

Retail and wholesale

Utilities and telecommunications

Manufacturing

Media, entertainment, and leisure

Business services and construction

Other

Agile survey 2015 2013 2011

Tech

industry

45% 34% 37%

IT/BT end

users

47% 58% 49%

Government 8% 8% 8%


http://slidepdf.com/reader/full/forrester2015-state-of-agile-development-report 7/42© 2015 Forrester Research, Inc. Reproduction Prohibited

7%

7%

7%

3%

22%

23%

30%

Less than 100

Between 100 - 249

Between 250 - 499

Between 500 - 999

Between 1,000 - 4,999

Between 5,000 - 19,999

20,000 or more

7

“Using your best estimate, how many employees work for your company worldwide?

(select one)”

More than 50% of respondents are from very large or largeenterprises. However, smaller shops have higher adoption of Agilethan the larger enterprises.

Base: 149 (*37 from small enterprises) professionals with knowledge of their firm’s Agile practices.


Respondents from

small enterprises*

(those with fewer than

1,000 employees) lead

Agile adoption. Sixty-

five percent of people

from this group report

having 100% of

development teams

using Agile.


http://slidepdf.com/reader/full/forrester2015-state-of-agile-development-report 8/42© 2015 Forrester Research, Inc. Reproduction Prohibited

“What is your estimate of the number of developers that are part of your internal and

external staff? (select one for internal, and one for external)”

7%

29%

36%

29%

14%

10%

21%

17%

6%

4%

9%

7%

7%

4%

Internal staff

External staff

1-10 11-49 50-99 100-499 500-999 1,000 - 4,999 5,000 or more

8

The bulk (36%) have between 11 and 49 developers asinternal staff; 14% have between 50 and 99; and 21%have between 100 and 499

Base: 184 professionals with knowledge of their firm’s Agile practices





“What kind of software development is your organization primarily responsible for?

(Check all that apply — select one if you are representing only one team)”



Agile is pervasive across all types of businesses anddevelopment organizations. Respondents are from bothcentralized IT/BT as well as from divisional teams.

Agile is used by software

vendor product groups, customapplication development groups

that develop mobile and web

apps, and by packaged

application development teams

(e.g., SAP).

2%

8%

9%

24%

25%

29%

36%

36%

42%

43%

45%

46%

Console games

PLM or embedded software/systems engineering(consumer)

PLM or embedded software/systems engineering(business/government)

Departmental IT for internal applications

Departmental IT or app dev team for mobileapplications development

Central IT app dev team for mobile applicationsdevelopment

Delivering software products for desktop, contentmanagement, collaboration tools

Departmental IT or App dev team for customapplication development

Central IT for internal applications

Central IT App dev team for custom applicationdevelopment

Delivering software products for development,platform/middleware or deployment tools

Custom software (services or system integrationcompany)




“In which region or country do you work?”(select one)

We received responses from all over the globe.Geography: Americas — 55%, Europe — 26%, APAC —15%, Other — 4%.



52%

25%

15%

3%

1%

4%

North America

Western Europe

Asia Pacific

South America

Eastern Europe

Other




What It Means

• All types of businesses have responded to the survey: large system

integrators and consulting organizations, software product vendors, and IT/BTorganizations of end users coming from various industries.

• Although the bulk of respondents are from the US, almost half come from the

rest of the world, making it a global online survey.

• With 25% coming from financial services or insurance, further data cuts and

considerations can be made on the Agile adoption trends for this industry. If

you are interested and a Forrester client, check with your account manager.

• Respondents are mostly IT/BT executives.

Demographics

11




Agenda

• Demographics








10%

15%

21%

25%

18%

10%

All team members are in the same location

90% to 99% of the team is in the same location




Less than 25% of the team is in the same location

13

“To what extent is your primary team geographically distributed?”

(Select one)

Distributed Agile teams are the norm. Only 10% havefully collocated teams



Seventy-one percent,

the majority, keep more

than half of the team in

the same place.




5%

11%

22%

15%

8%

40%

12%

15%

28%

17%

10%

18%

15%

22%

20%

18%

10%

14%

Less than six months

More than six months, but less than an a year

More than a year, but less than two years

More than two years, but less than three years

More than three years, but less than four years

Four or more years

2015

2013*

2011**

14

“How long have Agile practices been in use in your organization? (select one)”

The proportion of professionals with four or more yearsof Agile adoption at their firms has increasedsignificantly

Base: 184, *149 and **205 professions with knowledge of their firm’s Agile techniques.


• Forty percent have been

doing Agile for more than

four years and are likely

to be beyond large scale

adoption.

• Thirty-eight percent have

been doing Agile for less

than two years and most

likely still need to scale.




23%

20%

15%

18%

24%

45%

21%

12%

11%

11%

40%

17%

13%

12%

19%

Under 25%

Between 25% and 49%

Between 50% and 74%

Between 75% and 99%

100%

2015

2013*

2011**

15

“Roughly what percent of development teams in your organization are using Agile

practices? (select one)”

Base: 182 , *149 and **205 professionals with knowledge of their firm’s Agile techniques.


Over half of respondents have scaled the use of Agile tomore than 50% of teams, and 24% have 100% adoption.




24%

22%

22%

20%

13%

Under 25%

Between 26% and 49%

Between 50% and 74%

Between 75% and 99%

100%

16

“And among those using Agile practices, what percent are using those practices to a

satisfactory level? (select one)”

Fifty-five percent of respondents have between 50% and100% of teams using Agile to a satisfactory level



M t l i A il ti 20%




Many teams are leveraging core Agile practices. 20% areleveraging Agile from ideation to deployment (includingDevOps). Interestingly, 51% use Agile in upstream.

86%

84%

84%

80%

78%

76%

68%

64%

58%

58%

56%

50%

47%

47%

39%

16%

10%

Short iterations (between 2-4 weeks)

Daily Scrum meetings

Story-based requirements

Product Owner

Early and frequent feedback

Scrum Master

Use Burndown or burn-up boards (prioritized list)

Up front p lanning initial estimate (dev&testing, sprint 0)

Continuous Integration

Continuous Delivery

Up front requirements (sprint 0)

Refactoring

Up front a rchitecture (sprint 0 )

User acceptance test-driven development

Test-driven development

Behavior-driven development

Other

Base: 176 professionals with knowledge of their f irm’s Agile practices.


“Which core Agile practices do you apply? (Check all that apply)”

Agile upstream (51% of

respondents do all of the

following):

• Short iterations

• Daily scrum meetings

• Product owner

• Early and frequent

feedback• Burn-down/burn-up

Agile downstream (24% of

respondents do all of the

following):

• Continuous integration

• Continuous delivery

• Test-driven

development

Agile upstream anddownstream (15% of

respondents do all

upstream and downstream

activities)

O l 13% dh t l A il th d l M t




13%

54%

27%

6%

25%

44%

24%

7%

17%

31%

39%

13%

We adhere to one particular Agile methodology asclosely as we can.

We use a mix of Agile methodologies

We deliberately mix Agile and non-Agile techniques

Other

2015

2013*

2011**

18

“How would you describe your adherence to a particular Agile methodology?)”

(select one)

Only 13% adhere to only one Agile methodology. Mostuse a mix of Agile methods, which is a rising trend.

Base: 175, *149, and **205 professionals with knowledge of their firm’s Agile techniques.


Twenty-seven percent are mixing

Agile with non-Agile methodologies.This is not necessarily a bad thing

when done right! We call this Water-

Agile-Fall.




86%

57%

57%

50%

49%

35%

35%

35%

33%

29%

23%

20%

20%

17%

12%

8%

6%6%

5%5%

Scrum

Iterative

Kanban

DevOps

Lean

Hybrid Waterfall and Agile (WaterScrumFall)

Test-Driven Development (TDD, test cases are implemented first…

Waterfall

Scaled Agile Framework (SAFe)

Extreme Programming (XP)

Feature-Driven Development (FDD)

Capability Maturity Model Integration (CMMI)

Lean Startup

Behavior-Driven Development (BDD)

Rational Unified Process (RUP)

Disciplined Agile Delivery (DAD)

Microsoft Solutions Framework (MSF) For Agile

Spiral

Dynamic Systems Development Method (DSDM)

Other derivative of the Unified Process (AUP, OUP, etc.)

19

Scrum leads, but Kanban and DevOps adoption is alsoquite widespread with 57% and 50% use, respectively



“Which of the following methods do you partially or wholly apply? (Check all that

apply)”

XP is still under considered,

although continuous

integration, XP’s most popular

practice, ranked 58% among all

Agile practices adopted.

Enterprise Agile practices are

also making in roads: Water-Agile-Fall (done right) and SAFe

(which seems to enjoy fast

growth).




59%

49%

40%

39%

38%

38%

37%

36%

36%

36%

People behavioral change

Lack of skilled Product Owners from the business

Lack of dedicated cross functional teams

Lack of Agile skills in project management practices, backloggrooming, planning, requirements, architecture, etc.

Lack of Product Owners commitment from the business

Scaling Agile at the enterprise by replicating success throughoutthe organization to more teams

Lack of Agile skills in downstream or DevOps

Lack of Agile testing skills

Lack of automation tools and/or infrastructure

Lack of Agile executive leadership

20

Behavioral change is the biggest barrier to adoption;properly skilled business product owners are also a bigimpediment for Agile adoption success.



• Organizations are

finally realizing

that the lack of

cross-functional

teams is one of the

top impediments

for successful

Agile adoption.

• A lack of properexecutive

leadership is also

an important

impediment.

“Which of the following are barriers to your organization’s Agile adoption? (check all

that apply)”




69%

77%

80%

85%

88%

93%

93%

94%

Employee applications for back end systems

Architecture implementation (e.g., ESB, SOA)

Application integration

API development

Employee applications for serving customers

Website development

Product development

Mobile development

21

“For which of the following areas is Agile development important? (check one)”

Agile development is recognized as most important formobile, software products, and website development

Base: Between 137 and 161 professionals with knowledge of their f irm’s Agile practices (each row has a unique sample

size within this range.)


While mobile is

considered most

important, it’s not the

top area where Agile

is practiced. This may

be because:

1. Mobile

development might

have even shorter

cycles and smaller

teams, making

people identify

themselves as not

sticking to any

method.2. Or responding on

backend (SoR)

which might not be

done in Agile.




7%

44%

46%

48%

49%

59%

67%

76%

82%

84%

87%

Other

Designer

UX design professional

Project manager

Product owner from IT/BT

Architect

Product owner from Business

Back-end developer

Tester

Scrum master

Front-end developers

22

“Who’s on a typical team using Agile practices? (Select all that apply)”

Two-thirds have a business product owner. Testers arenow peers to developers and first-class citizens of Agileteams (a good sign).



However, some double

down on the business

product owner with an

IT/BT product owner

(these might be eitherplaying the role of the BA

or are SIs POs in Agile

outsourcing).




What It Means

• Just over half of respondents show an expert or mature adoption pattern. This group

has between 50% and 100% of the teams practicing Agile development to satisfactory

levels. These teams:

• Know when to use certain Agile practices and how to mix various methods. Scrum

is still the top Agile method for project management, followed by Kanban. DevOps

has made inroads too, and since 2013, a newcomer, the SAFe framework for

enterprise Agile adoption, has also made important strides.

• Integrate Agile development with waterfall practices in an overall enterprise

governance framework. When done well, Forrester calls this Water-Agile-Fall.

“Done well” means upfront planning and an understanding of very high-level

product requirements (water), then sprints kick off with further user stories of

refinement, design, development, testing, and integration (Agile), and, lastly,

release packaging and deployment (fall).

• Organize in cross-functional product (value stream) teams with core roles such as

product owners, scrum masters/Agile coaches, testers, and developers. Thesework exclusively on one product or value stream at a time. Additional members

might extend the core team on call, such as architects, UX designers, business

analysts, and operations managers.

Agile practices and organization

23




What It Means (cont.)

• Main impediments to scaling Agile successfully:

• Lack of strong leadership.• No business product owner or a lack of skilled business product owners.

• No cross-functional or integrated end-to-end teams; people not dedicated to

one product/project/stream of work at a time.

Agile practices and organization

24




Agenda

• Demographics








“Which of the following statements are true about your testing organization

structure? (Check all that apply)”

Most testers are allocated fulltime to project teams. It isnot clear if they are most likely to work on one project ata time, which would be highly desirable.



4%

6%

18%

28%

29%

35%

36%

41%

57%

61%

Don't know

Other

Testing is managed and executed in the TCoE

Automation architects are shared across projects

Testers belong to a Testing Center of Excellence(TCoE) where testing is managed and executed

Performance and Load testers are in project teams

We have a Testing Practice Center of Excellence withspecialized skills

Performance and Load testers are centralized

Testers are members of an integrated delivery team

Testers are allocated full time to project teams

• Testing center of excellence (TCOE)

breakdown, as Forrester predictedin 2013, is a growing reality.

• Performance and load testing

activities are also moving in

projects. This confirms performance

testing is also shifting to the left.

Manual testing prevails and unit testing follows UAT




3%

5%

8%

6%

13%

11%

14%

21%

28%

29%

53%

49%

11%

14%

16%

19%

19%

31%

29%

33%

38%

38%

25%

33%

BDD

Service Virtualization Testing

TDD

Test Data Management – Synthetic Generation

Test Data Management – Masking, Obfuscation

Test Data Management – Production Data Sub-setting, etc.

Exploratory Testing

Functional Test Automation

Functional Regression Tests Automation

User Acceptance Testing In Sprints

Unit Testing

Manual Testing

Always Usually

27

“How frequently do you use the following testing practices? (select one)”

Manual testing prevails and unit testing follows. UATdone early in sprints enjoys significant adoption



Automation

enjoys growing

adoption for

both functional

regression and

nonregression

tests

Service

virtualization,

TDD, and BDDare still far from

where they need

to be.




Agile teams do more functional than non-functional testautomation. GUI and non-GUI automation still coexist.Levels of test automation are starting to increase.

% Automation

(mean across responses)

Unit tests 53%

Functional tests (GUI) 45%

Functional tests (Beyond GUI, e.g., APIs) 42%

We implement continuous testing — tests are kicked off at each

code check in and integration (CI)42%

We automate nonfunctional (performance) 38%

We automate nonfunctional (integration) 35%

We automate test environment provisioning 30%



“Using your best estimate, how much do you automate the following?”




“Which of the following are barriers forusing system integrators on your Agile

projects? (Check all that apply)”

Few claim they use SIs on Agile projects successfully;24% don’t use SIs because they develop only withinternal resources



FALSE77%

TRUE23%

“I have no barriers for using systemintegrators on my Agile projects

and I'm using them successfully.”

9%

8%

8%

12%

17%

24%

Other

I don’t care what developmentmethod my SI uses; I just buy the

output as a managed service

I don’t trust industry contractingpractices are mature enough

I don’t feel SIs have sufficient skills

My current contractual aspects

and SLAs make it hard to use Agile

I develop only with my internalresources




What It Means

• Although Agile testing has matured overall, it’s still the biggest impediment for

teams to truly develop applications with speed and quality because:• The testing center of excellence, as we know it, is a concept of the past. It’s

becoming a federated practice testing center of excellence, rather than a

highly centralized testing execution shared service.

• The majority are including testers in development teams.

• TDD, BDD, and other modern testing practices, such as servicevirtualization for integration testing, are still lagging.

• Levels of test automation seem to have improved from as low as 10% in the

past years to between 30% to 53%. While a great improvement, in any case

it is still too low to not compromise agility.

• When doing Agile, many still rely on internal resources. The biggest

impediment to Agile outsourcing is still in old vendor governance practices and

waterfall contracts, and there is still some mistrust in true SI Agile expertise.

Agile testing and outsourcing

30




Agenda

• Demographics





The top e pected benefits are s all achie ed b t ith




25%

36%

46%

54%

58%

58%

60%

60%

69%

71%

72%

Lower maintenance costs

Continuous Delivery

Improved technical quality

Greater predictability of resultsaligned with requirements

Greater frequency of releases

Greater predictability of releases

Improved functional quality

Increased team

motivation/morale

Faster delivery of solutions

More opportunities for mid-course corrections

Better business/IT alignment

32

“What are the benefits expected by

your business for using Agile?

(select all that apply)”

The top expected benefits are usually achieved but withsmall gaps



“What are the benefits achieved/realized

by your business for using Agile?

(select all that apply)”

40%

57%

58%

63%

63%

67%

68%

70%

75%

79%

83%

Lower maintenance costs

Continuous delivery

Improved technical quality

Greater predictability of releases

Increased teammotivation/morale

Greater frequency of releases

Greater predictability of resultsaligned with requirements

Improved functional quality

More opportunities for mid -course corrections

Faster delivery of solutions

Better business/IT a lignment




3%

9%

13%

15%

22%

23%

33%

37%

37%

45%

49%

56%

56%

57%

Other metrics

Decreased technical debt

Data analytics generated during the development & delivery process

Decreased mean time to repair

Completed tests at each sprint/iterations

Function points completed

Increased predictability of planned releases

Cycle time

Increased perceived value of software delivered

Features completed

Business value

Quality

Velocity

User stories completed

33

“How do you measure the value of Agile? (Check all that apply)”

Productivity and quality are the top metrics that Agileteams use; business value closely follows.






“How do you measure the value of Agile? (check all that apply)”

Quality, velocity, and business value can be measured ina variety of ways



Quality Velocity Business value

32%

50%

Increase in thestability of deployed

sprints/iterations

Reduction of number of defects in

production

18%

23%

43%

Growth of plannedfeatures in backlog

over time

Increase in thenumber of features

of backlog

Increase in thenumber of user

story points

32%

40%

As measured bysales increase,client retention,

% of revenueincrease over

period of time, or end user…

As measured byincreased

user/customer

satisfaction

St i t b f th i it f i




4%

4%

12%

23%

35%

45%

49%

65%

Other

Lines of code

Function points

Features

Hours of work

User stories

Sizes (e.g., XL, L, M, S, XS)

Story points

35

Q24: “How do you size epics, themes, and your product backlog?

(Check all that apply)”

Story points are by far the primary unit of measuringsize; t-shirt size follows in popularity



Sizing in terms of hours of work

is no longer the primary sizing

technique




What It Means

• The top three expected and achieved benefits of becoming Agile are the same

– with small gaps between the two:• Better business and IT alignment

• Faster delivery of solutions

• More opportunities for midcourse corrections

• The top three metrics used by Agile teams are:

• User stories completed

• Velocity

• Quality

• However, among Agile expert teams, another top metric is predictability. They

still need to prove when a release will happen and what is being delivered.

• Business value is getting closer to the top three metrics.

• The most common sizing technique is story points with poker planning and

Fibonacci.

Benefits and measuring agile value

36




Agenda

• Demographics

• Agile practices and organization• Agile testing and outsourcing






Survey and data methodology

• Name of survey: Forrester's Q2 2015 Global Agile Software Application

Development Online Survey• Survey sample size: 215 professionals with knowledge of their firm’s Agile

practices. Exact sample sizes are displayed for each question.

• Methodology: Forrester fielded the survey during April and May 2015.

Respondent incentives included a summary of the survey results.

38




Thank you to the following vendors who helped ussocialize our survey

• CI&T

• Cigniti• Cognizant

• CollabNet

• IGate Global Solutions

• Imola Informatica

• IBM

• Interactive Intelligence

• L&T Infotech

• Magenic

• Parasoft• Rally Software Development

Corporation

• Softek Solutions

• Software Development Experts

• Tata Consultancy Services

• Tech Mahindra

• Wipro

39




Want to know more?

• See more research from the author, Diego Lo Giudice.

http://www.forrester.com/home#/Diego-Lo-Giudice• Access The Modern Application Delivery Playbook For 2015.

https://www.forrester.com/The+Modern+Application+Delivery+Playbook+For+2

015/-/E-PLA750?objectid=PLA750

• Please give us feedback on this report.

http://www.forrester.com/forr/reg/contact.jsp?id=38

40

http://www.forrester.com/home#/Diego-Lo-Giudice

https://www.forrester.com/The+Modern+Application+Delivery+Playbook+For+2015/-/E-PLA750?objectid=PLA750





http://www.forrester.com/home#/Diego-Lo-Giudice




Selected Forrester Research

• “The 2015 State Of Agile Development: Learn From Agile Expert Firms”

Forrester report• “Five Must-Do's For Testing Quality At Speed” Forrester report

• “Overcoming Barriers To Modern Application Delivery” Forrester report

41



Thank you

forrester_2015 state of agile development report

Documents