2014 state of code review survey results

Quality Matters:

The 2014 State of Code Review

Survey Results

Contents

About the survey

Why does code review matter

Current state of code review – Ad-hoc, meeting-based, tool-supported

– By industry, company size, dev team size & team distribution

Obstacles to reviews – Code review in general, tool-supported

Conclusion & recommendations

About the survey

Conducted by SmartBear from August-

October 2014

2nd annual survey – first launched in 2013

Over 600 respondents, 560 completed

responses

Conducted via email, social media, website

Contents

About the survey

Why does code review matter

Current state of code review – Ad-hoc, meeting-based, tool-supported

– By industry, company size, dev team size & team distribution

Obstacles to reviews – Code review in general, tool-supported

Conclusion & recommendations

What Do You Feel is the Number One Thing a

Company Can Do to Improve Code Quality?

0%

10%

20%

30%

40%

CodeReview

UnitTesting

IntegrationTesting

Other FunctionTesting

35%

24%

17% 13% 12%

% Responded

Satisfaction with Software Quality is Linked with Ability

to Ship Releases

87%

13%

Able to Ship Regularly

54%

46%

Unable to Ship Regularly/Neutral

Satisfied Dissatisfied/Neutral

What Do You Think Are The Most Important Benefits of

Code Review?

0%10%20%30%40%50%60%70%80%90%

84%

62% 61% 56% 48%

27% 26% 23% 21% 16%

% Responded

Contents

About the survey

Why does code review matter

Current state of code review – Ad-hoc, meeting-based, tool-supported

– By industry, company size, dev team size & team distribution

Obstacles to reviews – code review in general, tool-supported

Conclusion & recommendations

Ad-Hoc Code Review Adoption by Industry

0%

20%

40%

60%

80%

100%100%

83% 81% 80% 79% 75% 75% 73% 72% 69% 58%

% Responded

* Indicates small subsample

Meeting-Based Code Review Adoption by Industry

0%

20%

40%

60%

80%67% 65% 63% 60% 57%

52% 52% 52% 48% 40% 40%

% Responded

* Indicates small subsample

Tool-Based Code Review Adoption by Industry

0%

20%

40%

60%

80%

100% 86%

72% 70% 67% 66% 65% 63% 60% 56% 55%

35%

% Responded

* Indicates small subsample

Ad-Hoc Code Review Adoption by Company Size

39%

49%

63%

40%

77% 76% 83%

74%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Less than100

employees

100 to 500employees

500 to2000

employees

2000+employees

Uses Ad-Hoc CodeReview

Uses Ad-Hoc CodeReview Daily/Weekly

Meeting-Based Code Review Adoption by Company Size

17% 16% 19% 18%

53%

43%

54% 57%

0%

10%

20%

30%

40%

50%

60%

Less than100

employees

100 to 500employees

500 to2000

employees

2000+employees

Uses Meeting-BasedCode Review

Uses Meeting-BasedCode ReviewDaily/Weekly

Tool-Based Code Review Adoption by Company Size

52%

63%

75% 71%

33%

41%

56%

45%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Less than100

employees

100 to 500employees

500 to2000

employees

2000+employees

Uses Tool-BasedCode Review

Uses Tool-BasedCode ReviewDaily/Weekly

Ad-Hoc Code Review Adoption by Dev Team Size

67%

81% 76%

81%

27%

51% 50%

42%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Less than5 people

5 to 20people

20 to 50people

More than50 people

Uses Ad-Hoc CodeReview

Uses Ad-Hoc CodeReview Daily/Weekly

Meeting-Based Code Review Adoption by Dev Team Size

44%

51%

61%

68%

8%

16%

28% 31%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Less than5 people

5 to 20people

20 to 50people

More than50 people

Uses Meeting-BasedCode Review

Uses Meeting-BasedCode ReviewDaily/Weekly

Tool-Based Code Review Adoption by Dev Team Size

48%

66%

80% 81%

27%

44%

59% 53%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Less than5 people

5 to 20people

20 to 50people

More than50 people

Uses Tool-Based CodeReview

Uses Tool-Based CodeReview Daily/Weekly

Ad-Hoc Code Review Adoption by Team Distribution

73% 78% 78%

46% 41%

47%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

One location Two or morelocations or

countries

Multiple sites, butteam is colocated

Uses Ad-Hoc CodeReview

Uses Ad-Hoc CodeReview Daily/Weekly

Meeting-Based Code Review Adoption by Team Distribution

52% 53% 54%

16%

20%

14%

0%

10%

20%

30%

40%

50%

60%

One location Two or morelocations/countries

Multiple sites, butteam is colocated

Uses Meeting-BasedCode Review

Use Meeting-Based CodeReview Daily/Weekly

Tool-Based Code Review Adoption by Team Distribution

51%

70% 65%

28%

49% 44%

0%

10%

20%

30%

40%

50%

60%

70%

80%

One location Two or morelocations/countries

Multiple sites, but teamis colocated

Uses Tool-BasedCode Review

Use Tool-BasedCode ReviewDaily/Weekly

% of Ad-Hoc Code Review Adoption by Level of

Satisfaction of Software Quality

76%

24%

Satisfied

74%

26%

Dissatisfied/Neutral

Uses Ad-Hoc Code ReviewDoes Not Use Ad-Hoc Code Review

% of Meeting-Based Code Review Adoption by Level of

Satisfaction of Software Quality

56%

44%

Satisfied

44% 56%

Dissatisfied/Neutral

Uses Meeting-Based Code ReviewDoes Not Use Meeting-Based Code…

% of Tool-Based Code Review Adoption by Level of

Satisfaction of Software Quality

67%

33%

Satisfied

56%

44%

Dissatisfied/Neutral

Uses Tool-Based Code ReviewDoes Not Use Tool-Based Code Review

Do You Have A Preferred Source Control System?

0%

10%

20%

30% 27% 26%

15%

10% 9%

4% 4% 5%

% Responded

Contents

About the survey

Why does code review matter

Current state of code review – Ad-hoc, meeting-based, tool-supported

– By industry, company size, dev team size & team distribution

Obstacles to reviews – Code review in general, tool-supported

Conclusion & recommendations

What Obstacles Prevent You From Doing Any Type of

Code Review

0% 20% 40% 60% 80%

Workload

Deadline/Time Constraints

Lack of Manpower

Reviews are too Time-Consuming

Location of Team Members

Reviews areTedious/Repetitive

63%

46%

34%

25%

18%

15%

% Responded

What Obstacles Prevent You From Doing Tool-Based

Type of Code Review

38%

36%

30%

26%

17%

14%

14%

9%

8%

0% 10% 20% 30% 40% 50%

Workload

Lack of Budget

Deadline/Time Constraints

Lack of Managerial Buy-In

Lack of Manpower

SCM Integration

Reviews are too Time-Consuming

Reviews are Tedious/Repetitive

Location of Team Members

% Responded

I Often Find It Challenging to Collaborate with Team

Members on Large Projects

6%

29% 27%

34%

4%

0%

5%

10%

15%

20%

25%

30%

35%

40%

1 -StronglyDisagree

2 -Disagree

3 - NeitherDisagree

Nor Agree

4 - Agree 5 -StronglyAgree

% Responded

Satisfaction with Quality is Linked with How

Challenging it is to Collaborate with Team Members

65%

35%

Satisfied

50% 50%

Dissatisfied/Neutral

Not Challenging/Neutral Challenging

My Company Is Able to Get Releases Out On Time

Regularly

1%

15% 18%

51%

15%

0%

10%

20%

30%

40%

50%

60%

1 -StronglyDisagree

2 -Disagree

3 - NeitherDisagree

Nor Agree

4 - Agree 5 -StronglyAgree

% Responded

I am Satisfied with the Overall Quality of the Software I

Help Produce

1%

11% 13%

61%

15%

0%

10%

20%

30%

40%

50%

60%

70%

1 - StronglyDisagree

2 -Disagree

3 - NeitherDisagree

Nor Agree

4 - Agree 5 - StronglyAgree

% Responded

Contents

About the survey

Why does code review matter

Current state of code review – Ad-hoc, meeting-based, tool-supported

– By industry, company size, dev team size & team distribution

Obstacles to reviews – Code review in general, tool-supported

Conclusion & recommendations

Developers and Testers, Managers and End-Users

Agree on the Importance of Tool-Based Code Review

66%

53%

65% 63%

0%

10%

20%

30%

40%

50%

60%

70%

Development Testing Manager End-User

Implementing a Code Review Tool Is An Important Priority

Conclusion and recommendations

Code quality matters – Frequent releases, time to market, satisfied customers

Companies in many industries do code review – Ad-hoc reviews are more popular than meeting-based

• 75% ad hoc reviews, 50% meeting-based reviews, 60% tool-based reviews

– Perceived to improve quality

Obstacles to effectiveness – Workload and deadlines = planning (as ever)

Tool-based reviews – Use more regularly

– Use tools to support reviews – best of both worlds