the zero defect project

The Zero Defect ProjectCase StudyPMI Reston Luncheon 01/18/2017, Jamey Lees

www.pmiwdc.org/calendar

Jamey Lees• An accomplished Information

Technologist (IT) with over 18 years of overall IT experience and 14 years in Project Management with software delivery. Jamey is an Iowa native, graduated from Iowa State University, and is ITILv3 Certified, a Certified Scrum Master (CSM), and Project Management Professional (PMP). He is an active PMI member of Washington DC chapter.

• www.linkedin.com/in/jameylees

Introduction

• How many times have you ever heard a Project Manager (PM) say they received zero defects on the first User Acceptance Testing (UAT) cycle?

• Why do project manager’s plan for failure in their first UAT and schedule several UAT cycles to fix issues with their product?

• Project management as an industry struggles with delivering successful products to their client’s satisfaction• Project success is usually measured by Scope, Schedule and Budget, not

necessarily the client’s or Business area’s satisfaction • This case study observed an instance where the project received zero

defects in the first UAT cycle and met the client’s satisfaction. • The project introduced three new techniques called:

• Project Total Complexity Number (PTCN) • The "right of voice" concept • Requirements Acceptance Criteria (RAC)

• Adopting these techniques alone did not guarantee zero defects, but it is how these techniques were applied that resulted in a collaborative environment that facilitated the product receiving zero defects as part of UAT

Primary Cause for Project to Fail?

PMI’s Pulse of Professionals, February 2015

Project Synopsis

• The Zero Defect Project was valued at half a million dollars and was the second of two contracts that were both fixed price

• The first contract created the application and the Zero Defect Project performed enhancements to the product

• The first contract’s estimated value was $10 million, and was also the project team’s first attempt at running an agile methodology

• This resulted in the client and project team members getting into shouting matches on a daily bases

• The working environment was very hostile

Project Synopsis (continued)

• Establishing the Zero Defect Project team was problematic since most of the original team members from the first contract refused to work on it again

• A new team was formed from people on the contractor's bench and was significantly smaller in size

• 1 Project Manager, 3 developers, 1 analyst, and part-time database administrator

• 3 Business area members, and 1 Contract Officer

• New team members that differed from the first contract

• Project Manager, analyst and 1 developer

Project Synopsis (continue)

• The original contract established that after the initial product was delivered, a subsequent contract would be procured for any product enhancements

• Sole-Sourced

The Zero Defect Project Objective

• From the Contracting firm

• To make this a referenceable contract from the client

• The first contract was not referenceable

• Make a profit

• Deliver on time and under budget

• Change client’s perception of the contracting firm

• Repair relationship and receive more contracts

• From the client

• Deliver the product enhancements that they should have received under the original contract

Lesson Learned from the First Contract• The first contract relied on an agile project methodology without

establishing some project basics

• Define project success criteria• Project team and Client having the same expectations

• Receiving or establishing project boundaries• Requirement approvers or sign-off

• When key individuals returned from vacation they would change the requirements

• Define project roles• Client’s formed a consensus decision-making group with several business units

that did not always agree

• Project team tried to treat the consensus group as the Product Owner role defined by an agile project

• Training on the new project methodology for both the client and project team

• Change Management• Project team accepted changes on top of changes as an agile iteration without

a project impact analysis being performed

Lesson Learned from the First Contract• With no end in sight for finishing the project, the team

stopped accepting new or updated requirements once it was realized they were going to lose money

• Arguments started between the clients and the project team

Sole Sourced Contract Proposal

• To avoid disputes between the project team and the client on defining the scope of requirements during the project

• The project team establish a Project Total Complexity Number (PTCN) for each high level requirement in the Request for Proposal (RFP)

• 60 – very complex (was not clear or understood)

• 30 – complex

• 15 - minor complex (between 1 to 1.5 weeks of effort)

• 05 – not very complex

• Had 2 lead developers that worked on the previous project establish the PTCN for each RFP requirements

Sole Sourced Contract Proposal (continue)

Source 15 July 2007 : http://www.ibm.com/developerworks/rational/library/jul07/temnenco/

Understood technology Not all the

resources were subject matter

experts New estimating technique No Roles and

Responsibility defined

What happens whencomplexity is not

understood?

Estimation Challenges and Trends from IBM


• Work for 1 programmer = # of weeks in the project – (requirement sessions + UAT) * 15

• Project Total Complexity Number (PTCN) = Work for 1 programmer * 3 developers

Planning Execution UAT

Total Project Complexity

Number

23% 43% 34%

1.38 Months

2.58 Months

2.04 Months


• In the proposal • We gave them the PTCN threshold for the whole project

• Complexity number for each requirement in the RFP

• Explained they could pick any of the requirements so long as the added complexity number did not exceed the PTCNthreshold


• Proposal Strategy• Project Total Complexity Number

• Forced the 3 different Business units to determine their priorities together to represent the product as a whole• The requirements needed to be reconciled across each business line

• Define the delivery scope of the project so it would not be a point of contention during the requirement sessions

Project Kickoff meeting• The client was an environmental

agency and the PM brought printouts of the Project Kick-off presentation• No one would touch the paper

• PM was consistently interrupted with remarks of:• “we have heared that before”• “I’ll believe it when I see it”• “we are just trying to get the

application changes you owe us from the previous contract”

• The PM finally asked if each person in the room making the remarks if the project had their commitment

• The Client did not want the project team to waste their Business areas time in the requirements sessions and wanted the duration shorter

Project Resource Changes

• On the Client’s side the Contracting Officer changed

• On the project team

• The key lead developer moved to another project

• Replaced with a junior developer

• The analyst was replaced with an analyst that worked on the previous contract

• Once the lead developer moved to the other project it was revealed that they performed the PTCN with the expectation they would be doing the coding changes

Planning Goals• Optimize the Business area’s time during requirements gathering

process

• Receive all the requirements during the requirement sessions without the need to receive requirement clarity during testing

Planning Strategy• Leverage requirement questionnaires

• Review screen-mockups made from requirements received in previous requirement sessions

• Introduced the concept of the “right of voice” (defining authority over application requirements, avoided accountability)

Sun Mon Tue Wed Thurs Fri Sat

Requirement Sessions• In the kick-off meeting a request was made for the Business area

to complete the requirement questionnaire before the sessions started

• No one filled them out and said they would not, ever…

• The requirement questionnaires was used as a guide to review and extract the business requirements from the high level RFP requirements

Requirement Sessions (continue)• The requirement questionnaires were quickly removed

from the meetings and replaced with a Requirements Acceptance Criteria (RAC)• We needed to quickly train the Business area on what an acceptable

requirement was without offending them

Requirements Acceptance Criteria (RAC)Requirements Acceptance Criteria Criteria Guidelines

1. Is the requirement not an opinion?An opinion cannot be implemented and is unclear as a specific business need and requires more refining to be a factual statement.

2. Is the requirement accurate?Supports business process and objective of the application and does not conflict with other requirements.

3. Is the requirement feasible?Technically feasible and can be done with the existing technology and within cost restraints for the project.

4. Is the requirement not ambiguous? Does not have more than one interpretation.

5. Is the requirement testable? It can be measured that the change has occurred or not occurred.

Screen-Mockups• Before collecting new requirements in each session a

screen-mockups would be presented and validated for correctness

• The screenshots and callout logic defining users actions of Graphical User Interface (GUI) and backend data logic

Something Started to Change

• The RAC became a game to see who could get the requirements right…

• Each Business area started holding each other accountable for passing the 5 RACquestions

• With the requirements fresh in everyone's minds the screen-mockup reviews dropped from 1 hour to less than 15 minutes

Execution and UAT

• Execution

• The developers had a clear understanding of the expected results of the changes

• The team performed their own system validation of their coding changes and compared the results with the expected screen-mockup and logic material

• UAT

• The Business area validated all changes within a day and found zero defects in the first and only UAT cycle

The Results…• The consulting firm received a referenceable contract from the

agency

• The requirement sessions created a focal point for all participants

• RAC process removed us verses them mentality

• Screen-mockups and logic defined what to expect as a final product result for both the project team and the Business area

• The project team finished two month early

• The project’s Business area used the RAC processes to train other Business areas within the Client’s site

So What Happened?• Predefined expectations as

the project end state• Project Total Complexity Number

(PTCN)

• Screenshot mockup with logic

• The "right of voice" concept• Empowered the business area

• Requirements Acceptance Criteria (RAC)• Created a project belonging

Maslow's hierarchy of needs

Questions?