managing software debt - quality debt focus - qasig kirkland

Managing So)ware Debt in Prac2ce

Quality Debt Focus

2

First Things First...

@csterwa #swdebt

Get Rally! www.rallydev.com

3

Chris Sterling

• Director of Engineering at Rally So)ware in Kirkland, WA office

• Author of Book “Managing So)ware Debt: Building for Inevitable Change”

• Consults on so)ware technology, Agile technical prac2ces, Scrum, and effec2ve management techniques

• Innova2on Games® Trained Facilitator • Cer2fied Scrum Trainer • Open Source Developer

Email: [email protected] Web: hWp://www.rallydev.com Follow me on TwiWer: @csterwa Blog: hWp://www.geYngagile.com Hashtag for presenta2on: #swdebt

4

Agenda

• Managing So)ware Debt Overview

• So)ware Debt Types •  Technical •  Quality •  Configura2on Management •  Design •  Pla]orm Experience

• Asser2ng Quality •  Defini2on of Done •  Test Automa2on

•  The “3 Amigos” PaWern •  Test-‐Driven Design

• Monitoring Quality •  The Power of 2 Scripts •  Con2nuous Integra2on •  Automated Promo2on •  Advanced Quality Metrics Trending and Analysis

• Wrap Up •  So)ware Debt Management Strategy •  The “No Defect” Mindset

Managing So)ware Debt Overview Let me tell you a poten2ally familiar story…

Lack of emphasis on so)ware quality aWributes contributes to decay

Principle: No maWer what, the cost of addressing so)ware debt increases with 2me.

Types of So)ware Debt Technical, Quality, Configura2on Management, Design, and Pla]orm Experience

10

• Technical debt tended to focus more on programming aspects of so)ware delivery and le) out full so)ware development life cycle

• Each type of so)ware debt can be managed and monitored using different tools and approaches

• Focusing on managing each type of so)ware debt simplifies crea2on of an overall strategy that promotes holis2c perspec2ve

Why not just call it all “Technical Debt”

Types of So)ware Debt •  Technical Debt: These are the ac2vi2es that a team or team members take shortcuts on now that will impede future development if le) as is.

•  Quality Debt: There is a diminishing ability to verify the func2onal and technical quality of so)ware: the “Break/Fix” mentality.

•  Configura2on Management Debt: Integra2on and release management become more risky, complex, and error-‐prone.

•  Design Debt: The cost of adding features is increasing toward the point where it is more than the cost of wri2ng from scratch.

•  Pla]orm Experience Debt: The availability and alignment of people to business objec2ves that involve so)ware changes is becoming more limited or cost-‐prohibi2ve.

8

Asser2ng Quality Teams must focus on asser2ng sustainable quality to support future customer needs in a 2mely manner

Defini2on of Done So)ware developments assert what finished, working so)ware entails to support predictable delivery into the future

14

Defini2on of Done -‐ Assert Quality " Acceptance defined criteria for each

user story �

" Unit tests written and passed �

" Code compiles with no errors and no warnings �

" New code doesn’t break existing code �" Test case review (Dev to review test

case written) �

" Architectural impact assessed and artifacts updated if necessary�

" Comments in code �

" Error codes added�

" Code reviewed by peer �

" Code checked in with reference to US#/Task# �

" Tested on FE�

" Integration test written & passes �

" Test code reviewed�

" Environment requirements documented �

" Interface document updated/added and checked in to SVN �

" Acceptance criteria verified complete �

" All P1-P3 bugs for the story are closed �

" Test approves user story �

" Story demonstrated to product owner and accepted on Target Platform�

Release Defini2on of Done

• Every release should have clear quality criteria • With a “Release Defini2on of Done” you can understand targets beWer

• Measure the gap between the teams’ Defini2on of Done and a Release Defini2on of Done. • This gap is a source of quality issues and represents significant risk to schedule

Case Study: Test Automa2on Reduces Cost of Change

17

Manual Regression Tes2ng

• Tes2ng was taking 75 person hours during 2 full test runs consis2ng of: • Comprehensive manual regression tes2ng • Data conversion and valida2on

• Cost for tes2ng was $17,000 each itera2on

18

Introducing Fit into Tes2ng Process

• A)er 8 itera2ons team had introduced healthy amount of Fit fixtures and automated tests

• Reduced 70+ hour test run2me down to 6 hours which now included: • Fit automated regression tes2ng • Data conversion and valida2on automated with Fit fixtures

• Reduced cost of tes2ng each itera2on from $17,000 to $7,000

19

The Agile Regression Tes2ng Triangle*

* The Agile Triangle has been modified from Mike Cohn’s original version

Automated Unit Tests Make up largest por2on of regression tests and are developed by programmers

Integra2on Tests Automated & Exploratory

Smoke++ Tests Risk-‐based UI & API Automated Tests

The “3 Amigos” PaWern* Quickly get testers, coders, and business on the same page before building based on a requirement

* The Three Amigos paWern originally coined by George Dinwiddie hWp://www.s2ckyminds.com/s.asp?F=S17232_COL_2

21

The Three Amigos PaWern

As a Shopper I want to receive updates on incredible deals that are located near my home so that I can save money on my purchases

Acceptance Criteria: • Save Shopper’s location • Ask Shopper if they want to receive localized deals daily • Send notification of incredible deals to Shoppers located within 10 miles each morning • Allow Shopper to stop receiving localized deals

22


As a Shopper I want to receive updates on incredible deals that are located near my home so that I can save money on my purchases

• What areas of the applica2on will this affect? • What is the overall design? (UI, API, UX, etc…) • What are the details test cases for this user story and it’s acceptance criteria? • What about nega2ve test condi2ons? • What about boundary condi2ons? • How might we put exis2ng func2onality at risk?

23


• At minimum include tester, coder & business rep in discussion

• Should only take 30 minutes to 1 hour for user stories

• Focus on clarifica2on and design through testable inputs/outputs

24

Acceptance Test-‐Driven Development

Release Management “If releases are like giving birth, then you must be doing something wrong.” -‐ Robert Benefield

26

Case Study: Enterprise Agile Adop2on •  180+ person “Web 2.0” product organiza2on • Waterfall SDLC that development uses to deliver in 6 month release cycles

• Want to use Agile methods to be more responsive to users and keep up with other “Web 2.0” companies

•  Transi2oned to Agile methods on 15 teams in 3 months •  Changed release management strategy, added XP technical prac2ces, and implemented Scrum product development framework for scaled coordina2on

•  Able to release every week to users within 4 months •  Used streamlined deployment environment process to validate product changes daily using Con2nuous Integra2on and automated promo2ons

27

The Power of 2 Scripts: Deploy & Rollback

28

Tradi2onal Source Control Management

Main Branch Debt

Death March { Debt accrues quickly within stabiliza2on periods

Version 1 Branch

Integrate for Version 2

Code Complete

29

Flexible Source Control Management

Main Branch

Version 1 Version 2 {Not Easy! Must have proper infrastructure to do this.

30

Scaling Con2nuous Integra2on

Component Valida2on

Integrated Component Valida2on

End-‐to-‐End & Load/Stress

31

Automated Promo2on to Environments

Advanced Quality Asser2ons Using Automated Tools and Dashboards

33

Con2nuous Integra2on

34

Quality Dashboard -‐ Sonar

35


36


37


38

Early Warning Signs

Early Warnings: • Broken Builds • Broken Automated Tests • Broken Custom Thresholds

39

Early Warnings: • Design Debt in Duplica2on (DRY) • Technical Debt in Code Complexity • Quality Debt in Bug DB (Break/Fix) • Other Custom Thresholds

Early Warning on Quality Dashboard

The “No Defect” Mindset “What he needs is some way to pay back. Not some way to borrow more.” -‐-‐ Will Rogers

39

Ken Schwaber “For every [dollar] of compe22ve advantage gained by cuYng quality, it costs $4 to restore it; and so)ware is an organiza2onal asset and decisions to cut quality must be made by execu2ve management and reflected in the financial statements.” hWp://www.infoq.com/presenta2ons/agile-‐quality-‐canary-‐coalmine

Case Study: Field Support Applica2on

• 2000+ users access applica2on each day • Applica2on supports mul2ple perspec2ves and workflows from Field Support Opera2ons to Customer Service

• Team of 5 people delivering features on exis2ng Cold Fusion pla]orm implementa2on

• Migra2ng Architecture to Spring/Hibernate in slices while s2ll delivering valuable features

• 36 2-‐week Sprints, 33 produc2on releases, and only 1 defect found in produc2on

• So, what was the defect you say? Let me tell you…

40

Can We Afford a “No Defect” Policy? •  This team worked on legacy codebase inherited from another vendor

• Other vendor had been slowing down month a)er month and cost of development was increasing

•  In first itera2on this team was able to deliver more than other vendor was able to in previous 2 months

•  A)er 24 itera2ons this team was 10 2mes faster delivery than1st itera2on

•  Acceptance Test-‐Driven Development and Con2nuous Integra2on were greatest technical factors to support team in these results

•  Can you afford not to have a “No Defect” policy?

41

Thank you! Ques2ons and Answers [Time permiYng]

45

Chris Sterling Director of Engineering at Rally So)ware in Kirkland, WA office Author of Book “Managing So)ware Debt: Building for Inevitable Change” Consults on so)ware technology, Agile technical prac2ces, Scrum, and effec2ve management techniques Innova2on Games® Trained Facilitator Cer2fied Scrum Trainer Open Source Developer

Email: [email protected] Web: hWp://www.rallydev.com Follow me on TwiWer: @csterwa Blog: hWp://www.geYngagile.com Hashtag for presenta2on: #swdebt

managing software debt - quality debt focus - qasig kirkland

Documents