getting the most value from feedback systems: daily, every sprint, and every release

AW10 ConcurrentSession11/11/152:45pm

“Getting the Most Value from Feedback Systems: Daily, Every Sprint, and Every

Release”

Presented by:

Satish Thatte

VersionOne

Broughttoyouby:

340CorporateWay,Suite300,OrangePark,FL32073888-268-8770·904-278-0524·[email protected]·www.techwell.com

Satish Thatte VersionOne

Satish Thatte is an agile/lean coach and product consultant at VersionOne. He has fifteen years of software development and management experience at Texas Instruments, Bellcore, LG Electronics, and start-up companies in a variety of agile roles. Satish is a SAFe Program Consultant, Certified ScrumMaster, Certified Scrum Product Owner, and Certified Scrum Practitioner. He specializes in tailoring agile-lean methods to individual organizations (one size doesn’t fit all) and scaling up agile methods your way. Learn more about Satish at LinkedIn.

9/14/2015

1

Getting the Most Value from Agile Feedback Systems: Every Day, Every Sprint and Every Release

Satish Thatte, PhDSPC, CSM, CSPO, CSPAgile/Lean Coach and Product Consultant

Agile Development Conference EastOrlando, FL11 November 2015

[email protected]

© 2015 VersionOne 2

Accurate and Timely Feedback is Essential for Agile to Work

• Agile methods are empirical • Daily feedback system: benefit team members and teams• Sprint feedback system: benefit teams and projects• Release feedback system: benefit programs and organization

• Double feedback loops: Primary and Derivative feedbacks• Many agile teams have dysfunctional feedback systems

– No or delayed primary feedback– No derivate feedback: little learning or continuous improvements

• The key to improving feedback systems– Use templates or tools to implement double feedback loops

• VersionOne platform subsumes much of template functionality

9/14/2015

2


Feedback Framework: Evidence, Relevance, Consequence, Action

Feedback Loop in Action: An Example that Works Real Well

Adapted from: “The feedback loop” by Thomas Goetz, Wired magazine, July 2011

1. Evidence: The radar sign

flashes a car’s speed

2. Relevance: The sign also

displays legal speed limit

3. Consequence: People are

reminded of the downside of

speeding: traffic tickets, accidents

4. Action: Drivers slow down

10% - usually for several miles

Driver

Feedback


Framework for Effective Agile Feedback Systems

1. Daily Scrums2. Sprint reviews and retrospectives3. Release reviews and retrospectives

1

3

2

• Is your system “Feedback” or an “open loop” system?• Quality of feedback

• Accurate or Ambiguous, Fast or Delayed, Strong or Weak

• Improve the effectiveness of feedback in terms of:• Evidence, Relevance, Consequence, Actions

• Cost and Ease of Use of Feedback System

9/14/2015

3


Daily Feedback System is experienced through Daily Scrums: Burndown, Burnup and Storyboard

Broken Feedback System: Beware of Garbage-in, Garbage-out


Ineffective Daily Scrum

•What did you do since the last Daily Scrum?– Lots of development

•What’s your plan until the next Daily Scrum?– Will fix a bunch of defects

•What are the likely impediments? – Too many meetings, such

as this Daily Scrum!

• Quality of feedback: Ambiguous, Fast, Weak

• Very Weak in: Evidence, Relevance, Consequence, Actions

Effective Daily Scrum

•What did you do since the last Daily Scrum against your plan?– Completed implementation of US 37, but

could not complete all unit testing.•What’s your plan until the next Daily

Scrum? – Complete unit testing of US37 and Fix

Defects 367, 431•What do you need from your team

members?– Need Joe (analyst) to explain US46

•What are the likely impediments? – Awaiting clarification on US59 from Dave

(Product Owner)• Quality of feedback: Accurate, Fast,

Strong• Strong in: Evidence, Relevance,

Consequence, Actions – If Written

Daily Feedback System is experienced through Daily Scrums

9/14/2015

4


Daily Scrums: Effective as well as Efficient

Until the next Daily Scrum• My proposed work plan:

• Be specific, and result-oriented

• Work against my planned task in the agile tool: Task 3 and 4

• My needs from others in the next few days

• My impediments on the horizon

Since the last Daily Scrum

• Variance in my work against my plan, and the reasons for the variance

• Task 1: Done• Task 2: Not Done;

Reason


Daily Scrums: Not Very Effective; You Can Do Better

Daily Scrum in front of a “Default” Taskboard

There is a weak feedback between the plan presented in today’s Daily Scrum (or in the last Daily Scrum) and what will be actually done until the next Daily Scrum (or was actually done since the last Daily Scrum).

Weak in Evidence, Relevance, Consequence, Action

9/14/2015

5


Daily Feedback System: Experienced through Daily Scrums

• There may be no or weak connection between the planned work until the next Daily Scrum and what will be completed– Similarly, there may be no or weak connection between the plan

presented in last Daily Scrum, and what was actually completed.

• Primary feedback loop must indicate whether the daily planned work was completed in that day as planned.

• Derivative feedback loop must identify any systemic issues to enable learning and continuous improvements.

• My 5-part blog series (Making Daily Scrums Really Effective and Efficient: http://bit.ly/ODMybJ) describes a Daily Feedback System with these feedback loops. – The template with this blog has been downloaded over 1,000 times– Making Daily Scrums really effective and efficient is a challenge– Functionality of this template is subsumed in the appropriately

designed VersionOne Taskboard to facilitate Daily Scrums


Highly Effective and Efficient Daily Scrums

There is a strong feedback between the plan presented in today’s Daily Scrum (or in the last Daily Scrum) and what will be actually done until the next Daily Scrum (or was actually done since the last Daily Scrum).

Daily Scrum in front of the Redesigned Daily Scrum Taskboard

Strong in Evidence, Relevance, Consequence, Action

http://bit.ly/ODMybJ

9/14/2015

6


Derivative Feedback on Daily Feedback: Discuss during Sprint Retrospective

Daily Scrum Taskboard at the Sprint End

Goal: Maximize the number of SMART tasks Completed OnTime within 1 day of work


Sprint Feedback System: Experienced through Sprint Reviews and Retrospectives

• Effective sprint reviews and retrospectives are essential for continuous team-level process improvements.

• Primary feedback loop must indicate whether and how well the sprint-level goals are achieved

• Derivative feedback loop must identify any systemic issues to enable learning and process improvements by the team.

• My 4-part blog series (Making Sprint Retrospectives really Effective: http://bit.ly/R0Kyfa) describes a Sprint Feedback System with these two feedback loops. – The template with this blog has been downloaded over 300 times– Making the Sprint Feedback system really effective is a challenge

faced by many agile teams.• Sprint Feedback System (Reviews and Retrospectives) is less

common than the Daily Feedback System (Daily Scrums)• Many teams often fail to implement the action plan coming out

of their sprint reviews and retros.

http://bit.ly/R0Kyfa

9/14/2015

7


Sprint Review

• Review the Sprint accomplishments against the Sprint Goals

• A sprint demo(s) by the team–Demonstrate the completed, fully tested, potentially

shippable sprint release just completed–No or minimal PowerPoint slides!

• Different types of sprint reviews–Company management, Marketing, Sales, Customer

support, any other Stakeholders–Selected customers

• Needs planning and engineering during sprint–Otherwise, sprint review will not be effective


Sprint Retrospective

Product Name:

Release ID Sprint ID

A) What worked well,

& will be continued:

15 min

• Team consensus on

up to top 3 factors

C) Present and Discuss Key Statistics and Impediments: 30 min

• Sprint goals planned vs. achieved in a sprint

• Planned vs. accepted story points (i.e., velocity) for the sprint

• Estimated effort vs. Actual effort spent in the Sprint

• Scope change, and any additional key statistics.

• Top 3 impediments or impediment patterns, and their causes

B) What was

problematic & will be

changed: 15 min

• Team consensus on

up to top 3 factors

D) Develop SMART Action plan to improve agile process: 45 min

Did we meet our Sprint Goal? Any gaps?

• Specific

• Measurable

• Achievable

• Realistic

• Time-bound

E) Capture the results of Sprint Retrospective in the agile tool,

including SMART stories in the next sprint backlog: 15 min

9/14/2015

8


Sprint Retrospectives: Examples of Factors Working Well and Posing Most Problems

Factors working well

• Small stories improved estimation and planning• Use of VersionOne as the agile tool provided us an integrated project database with good transparency

• Coordination among team members facilitated

Factors Posing Most Problems

• Daily Scrum meetings were not very effective as people usually weren’t prepared for the meeting

• Some stories were too big; we should have split them into smaller stories

• Too many stories accepted in the last few days of the sprint


Sprint Retrospectives: Examples of Key Statistics and Major Impediments

Key Statistics• Planned vs. accepted story points (i.e., velocity)

– Velocity was 20% below the planned story points• Estimated effort vs. Actual effort

–Estimated effort was 5% higher than the team capacity, while Actual effort was 21% higher than the team capacity

• Scope change –4 new stories were added in the middle of the sprint!

Major Impediments• Work on stories could not start -- waiting for upstream dependent stories to complete

• Unplanned work kept people unavailable for planned work

• Slow daily build process

9/14/2015

9


Sprint SMART Actions: Make SMART Stories for the Next Sprint Backlog

SMART Action Plan mapped on Epics and Stories

We had too many

stories accepted in

the last week of

the sprint

Story: Apply rigorous WIP controls to Kanban storyboard.

Story: Implement the Kanban board with WIP control in VersionOne production environment

Build process was

slow, which often

caused delays

waiting for build

to complete

Epic: Invest in Continuous Integration infrastructure

for future sprints.

Story: Evaluate open-source options (such as Hudson)

Story: Trial and experiment with Continuous Integration platform

Story: Complete training of the team members for the selected Continuous Integration platform

Story: Implement and deploy the Continuous Integration server


Derivative Feedback on Sprint Feedback: Discuss during Release Retrospective

Analyze Sprint Retrospective Summary over a sequence of many sprints:• Are we actually improving as planned?• Are we taking two steps forward and one step backward, or sideways• Are there recurring issues? If so, what are their root causes?• What actions are needed to fix the root causes?

9/14/2015

10


Release Feedback System: Experienced through Release Reviews and Release Retrospectives

• Effective release reviews and retrospectives are essential for strategic improvements at the program or organizational level:– Time-to-market reduction, productivity improvements, quality

improvements, release cost reductions, etc. • Primary feedback loop must indicate whether and how well the

release-level goals are achieved• Derivative feedback loop must identify any systemic issues to

enable learning and process improvements by a program or the organization

• My 5-part blog series (Making Release Retrospective Strategic and Effective: http://bit.ly/GYGePS) describes a Release Feedback System with these two feedback loops – The template with this blog is downloaded over 100 times – Making the Release Feedback system really effective is a challenge

faced by many programs and organizations• Release Reviews and Retros are less common than Sprint

Reviews and Retros, which is less common than Daily Scrums• Programs and organization often fail to implement the action

plan coming out of their release reviews and retros


Release Retrospectives Require Strategic Metric

Strategic Metric Measurement Approaches

Field data on product

feature usage

Customer surveys to understand feature usage

Feature usage distribution and frequency

Features that are missing

Features that need to be improved or simplified or

streamlined

Concept to Customer

value realization cycle

time

End-to-end cycle time

End-to-end process bottlenecks

(Value producing time / Total cycle time)

Release cost Fully loaded development and delivery costs: people, material,

equipment, licenses, shared IT service charges, etc.

Release productivity =

(Release velocity /

Release cost)

Normalize the velocity numbers across sprints and teams to

account because story points across sprints and teams may

represent different amounts of work.

Number of customer-

reported new issues

Typically reported on a quarterly basis or for the entire release

cycle.

http://bit.ly/GYGePS

9/14/2015

11


Strategic Action Plan following Release Retrospectives

Strategic Metric Action Plan to Address the Issues

Field data on product

feature usage

Discontinue rarely used features Add important missing features or epics Improve, simplify or streamline features with poor user

experience

Concept to Customer

value realization cycle

time

Reduce end-to-end cycle time by simplifying and streamlining the process, reduce delays, and remove bottlenecks

Release cost Reduce development and delivery costso eliminate rarely used features (see Measure A )o improve release productivity (see Measure D)o increase automation

Release productivity =

(Release velocity /

Release cost)

Improve team work, cross-functional team training

Reduce release cost (see Measure C)

Number of customer-

reported new issues

Improve quality with reviews for feature specification,

design, code

Offer training and resources for test-driven development,

refactoring, test automation, and technical debt reduction


Summary

Daily Feedback System

Experienced with

Daily Scrums, Story Board, Burn-Down,Burn-up

Focused on Individual team members & team

PrimaryFeedback

Feedback loop between daily commitment and its fulfillment: Enabled by Daily Scrum Taskboard

DerivativeFeedback

Feedbacknecessary to allow continuous improvements to deliver on daily commitments

Tools Daily Scrum with Revised Taskboard

Sprint Feedback System

Sprint Reviews and Retrospectives

Agile teams and projects

Feedback loop between Sprint goals and accomplishments; SMART action plan drive SMART stories in the next sprint

Are SMART actions giving the expected results? Is the team regressing from or building upon prior improvements?

SMART storiesSprint retro reports

Release Feedback System

Release Reviews and Retrospectives

Programs, Portfolios, and Organization

Feedback loop between Release goals and accomplishments, captured with Strategic Metrics: Time-to-market reduction, productivity improvements, quality improvements, release cost reductions, etc.

Is the program or organization regressing from or building upon prior improvements?

Strategic metrics system, and implement action plan to improve strategically

getting the most value from feedback systems: daily, every sprint, and every release

Software