quality delivery management

Click to edit Master title style

Quality Delivery Management

The ideal model for Effective Quality Implementation

ByRamkumar RamachandranGM – Quality & TestingRenault Nissan Technology & Business Centre India Pvt. Ltd (IS&IT Division)[email protected]

mailto:[email protected]


15/04/2023 Copyrighted material 2

Contents

• Objective

• The Quality Team Structure

• Teams & Responsibilities

• Quality Team – Ideal Composition

• Selling Quality – Best Practices

• Risks & Mitigations

• Summary



Objective

• How do you setup a Quality Team?

• How do you effectively create Quality Culture?

• How do you justify investments in quality?

• What is war-time and peace-time activities?

• Mix of Practitioners in Quality Team

• Determine Facilitation Productivity

• Monitoring mechanisms

• Awarding Quality in deliveries



Disclaimer

Best Attempts are made to Provide inputs on a workable Quality Organization ModelResults vary based on your

Organization ecosystem

Honestly, nothing of this is rocket science, just pure common sense


Quality Organization

The typical ‘Quality Structure’ for well-oiled operations



Quality Delivery Management

Senior Management

Quality Head

Process Definition

Process Specialists

Internal Audit

Auditors

Facilitation

Facilitators / SQAs

Process Compliance

Review

Process Reviewers /

SQAs

Innovation

MBB / BB



Inter-related Quality Wheels

Senior Management

Commitment

Policies

Processes

Implementation

Audits

CMMI/ISO

ITIL/Six Sigma

Customer Satisfaction

Product

Quality

Delivery



Desired Eco System

• Committed Senior Management

• Strong Engineering focus

• Long term growth plans for the organization

• Strong estimation mechanism

• Focused Risk Management

• Highly matured middle management

• Inclined to encourage innovation

Signs of a

Growing O

rganiza

tion


Justifying Investment

Creating a quality team with the right crowd and right composition



Quality Team Composition

Nascent Mid-Way Matured0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1020

30

4035

35

2025

2530

2010

DefinitionAuditorsImplementersInnovators

Ratio of the quality crowd



Quality Rollout Team Strength

Nascent Mid-Way Matured0

2

4

6

8

10

12

1 1 1

7

4

2

10

7

3

Non-FocussedFocussedHighly Focussed

Note: This is very subjective and needs to be applied along with several other parameters

% Quality Rollout Team Vs. Orgn. HC

Includes Full time, Part

time & Practitioners



Practitioners Composition

• Practitioners can contribute for: -– Process Championing– Process Definition – through EPG– Process Implementation– Process Audits

• In the nascent levels it can be as much as 30%

• In matured state it can be at 15%

• Higher composition of Practitioners leads to better quality culture and sustainability of initiatives



Typical Quality Team Member Profile

• Comfortable in software engineering (Eg. Tech Savvy)

• Delivery experience

• Faith in systemic working

• Strong conflict resolution skills

• Excellent communication skills

• Knowledge of Models / Standards / FrameworksBasic Q

ualities o

f a

Good SQA


Process Definition

Create rules for delivery that meet the business objectives

Click to edit Master title styleFit the Model into Business

Sales

KT

Engineering

• Identify Prospects• Submitting RFPs• Transferring

knowledge• Validating KT• Design Quality

• Code Quality

CMM

IISO

ITILMaps

To

Business

15/04/2023 15Copyrighted material



Rules for Defining Process

• Choose Model / Standard / Framework that suits the business

• Map the Model into the business – never the other way

• Create a process architecture to understand various process interactions

• Make the process architecture scalable

• Call the QMS as BMS – Business Management System – melting pot of all Best Practices

• BMS should be able to accommodate QMS, SMS, ISMS etc.



CUSTOMER 1CUSTOMER 1 CUSTOMER 2CUSTOMER 2

Maintenance Support

Java .NET OTHERSM/F SAP

ABC Corporation

Business Service Catalogue

Technical Service Catalogue

Service Level ManagementBusiness Relationship Management

Service Continuity ManagementStrategic Service Management

Information Security Management Supplier ManagementService Management Objectives, Policy , Scope and Plan

IT Financial Management

Requirements Management

Service System Development Change Management

Release and Deployment Management

Incident ManagementProblem Management

Service Request Management

Configuration Management

Availability Management

Capacity Management Risk Management Work Planning

Work Monitoring & Control

Measurement & Analysis

Quality Assurance

Organizational Training

Integrated Work Management

Organizational Process Definition

Conti

nual

Ser

vice

Impr

ovem

ent Continual Service Im

provement

Continual Service Improvement

SAMPLE



BMS Release Mechanism

• After initial baseline, plan quarterly releases of BMS

• Encourage Practitioners to come out with process change requests

• Process changes needs to be approved by EPG and queued for the forthcoming release

• Have provision for ‘Hotfix’ releases

• Provide a version number for each BMS release

• Communicate to all stakeholders on each BMS release



• Design a very effective training mechanism to all stakeholders

• Split the training into two parts: -– Generic BMS training– Role based BMS training

• Everyone should undergo a training of these two components

• Role based training should be with case studies / scenarios that is applicable to the given roles

• Have a very strong mechanism for evaluating training effectiveness

Effective Training on BMS



• Evaluation mechanism can be through online quiz / interviews / tests

• Arrive at a threshold score that everyone should get in these evaluations

• Plan for Refresher Training for all roles

• Bring in the training completion as a pre-requisite for deployment into Projects

Effective Training on BMS


Engineering Process Group

How to make Practitioners own the process?



Having an Active Engineering Process Group

• Engaging Practitioners to define processes is key to effective buy-in

• EPG should be headed by Practitioner

• Identify ‘Process Owners’

• Co-ordination of EPG Meetings by QA rep

• High visibility of Process ‘Tailorings’ & ‘Deviations’ to EPG

• Cross unit representation in EPG

• Incentives for Practitioners to be part of EPG



The EPG Structure

EPG Members

Senior Mgmt. Committee

Chairperson

EPG

Objectives

SEPG Plan

QA Representative

Quarterly Reviews

Monthly reviews/action plans

PMO

Middle Mgmt. Committee

CMMI / ISO / ITIL

Operations & Functions

Monthly updates

QA QA Plan

Model Updates

PIF & Tailorings

Data

SBU Head

V.Chairperson

Convener

SAMPLE



EPG Responsibilities

• Process should be owned by EPG

• EPG should be predominantly comprised of Practitioners

• QA should vet the processes for compliance with CMMI / ISO / ITIL as applicable

• EPG should analyze various parameters to sense patterns and act on the same

• Sample parameters for analysis: -– Tailoring / Deviation Requests– Non-Conformances– Risks– PPB

• An annual plan for analysis should be drawn and monitored



Jan Feb Mar Apr May Jun Jul Aug

Defects Analysis

PCR Analysis

Risks Analysis

Best Practices Analysis

Tailoring / Deviations Analysis

Annual EPG Analysis Calendar

SAMPLE


Process Rollout

Effective facilitation of process adherence in delivery



Effective Facilitation Planning

• Arrive at Facilitation Productivity– A very scientific one OR– A ball park figure (one SQA for ‘n’ projects, of given category)

• Consider other activities and allocate ‘effective available mandays’ only (Eg. 18d One person month)

• Consider multiple skill sets for defining productivity Fresher / Standard / Senior

• Have a Facilitation Tracker to allocate SQA to projects

• Know the load of each SQA at any point in time



Work Load DistributionTotal Current Work Units Projects Count

Total Available EffortNot Considered Effort

arriving the Work Units

SQA

Available Hrs per Month(18days*8 hrs)

Allocated %

Actual Hours

Conducting PCR (2days*8hrs)

Meetings, Trainings and Leaves (2days*8hrs)

John 144 100% 144 16 16Umesh 144 15% 22 16 16Yashitha 144 95% 137 16 16Venkat 144 10% 14.4 16 16Malathi 144 100% 144 16 16

Prashanth 144 100% 144 16 16

Jothi 144 85% 122.4 16 16Total Available Effort 727

SQA Run Change Build Grand Total

John 6 13 19Umesh 1 1 1 3Yashitha 3 14 1 18Venkat 1 16 1 18Malathi 5 5Prashanth 8 9 17Jothi 7 18 25Total 31 71 3 105

Low(1) Medium(3) High(5) Total Work UnitsJohn 6 39 0 45Umesh 1 3 5 9

Yashitha 3 42 5 50Venkat 1 48 5 54Malathi 5 0 0 5Prashanth 8 27 0 35Jothi 7 54 0 61

Total Current Work Units 259

Total Current Work Units 259Organization "A" Work Units 14Organization "B" Work Units 245

Work Unit Per Resource(Organization "B" Work Units(245) / No. of Resources(5)) 49Total Available Effort(Organization "B" Resources) 691Effort per work unit(Total Effort / Total current work units) 2.82



Doing Effective Facilitation

• Effectively train SQAs on BMS

• Have a strong validation mechanism for judging SQAs knowledge on BMS (Eg. Quiz)

• Train SQAs on Doing Effective Facilitation

• Make SQA ALWAYS see the DELIVERY perspective of a compliance requirement

• Ensure that simple Facilitation Log is maintained with facilitation findings and shared

• Define escalation mechanism for non-closure of facilitation findings

• Measure the leakage of non-conformances to PCR



Facilitation Log

<<To be filled by SQA>> << To be filled by PM / PL>> << To be filled by SQA>>

S.No

Review Date Description of Issues Details of Action Status Closure

Date Closure Verified On

SAMPLE



Know the Practitioners Pulse

• Conduct CSAT on Facilitation team’s performance with Delivery Team

• Frame questions to know how much the SQA is delivery focused / BMS knowledge etc

• Take this CSAT as an indicator of Delivery Team’s perception on Facilitation

• Answers should NEVER be mapped to SQAs

• Answers should ONLY be mapped to projects

• NEVER use this CSAT to appraise SQA

• Do systemic changes based on this CSAT



Plan localized Facilitation Services

• Meet delivery units at define frequency with special facilitation services. For eg: -– Analyze certain delivery patterns and provide a solution

– Study certain challenges (Requirements / Testing etc) of delivery and offer solutions

– Identify opportunity for improvements and advice the same

– Discuss beyond ‘audit’, ‘compliance’, ’PCR’ etc.



• Average Projects / SQA

• % of findings leaked to IQA from Facilitation– IQA NCs not reported in Facilitation Log / Total IQA NCs

• SLA Adherence %ge to Facilitation Findings closure

• Kaizen contribution per SQA

• Implemented process improvements per SQA

Interesting Facilitation Metrics


Process Compliance Reviews

The in-between of Facilitation and Internal Audits



Findings Leakage

Findings in Facilitation – Facilitation Log

Findings in PCR – PCR Report

Findings in IQA – Audit Report

Findings in External Audits



Effective PCR Framework

• Cross allocate SQA for PCR

• Ensure independence of PCR activity

• Maintain an elaborate PCR template with– Process-wise questions– Graded scores for each question (0 to 5)

• All scores <= 2 should be treated as NCs and tracked for closure

• Define ‘hygiene factors’ as pre-requisite to start a PCR

• PCR template should have provision to capture details of compliance



Crosslink PCRs

SQA-1

SQA-2

SQA-3

Project A

Project B

Project C

Project DFacilitates

Facilitates

Facilitates

Conducts PCR

Conducts PCR

Facilitates

Conducts PCR

Project E

Facilitates

Conducts PCR



PCR Planning

• Have PCR as a monthly activity

• Decide on sample criteria, for eg.:-– Projects > USD 1M – Monthly sample– Projects in Support – Bi-Monthly sample– Hi-visibility Projects – Monthly sample

• Publish PCR Schedule to all stakeholders

• Create high visibility to PCR results through emails and posters

• Do PCR trend analysis to identify patterns



Finer aspects of PCR

• Ensure min 80% of PCR activity is offline and only max 20% engages the delivery team

• Define SLAs to close PCR findings along with escalation mechanisms

• Capture findings leakage from Facilitation

• Make Leads to randomly validate the PCRs

• Minimum 80% of PCR score should be made as KPI at all levels

• Provide high visibility for PCRs that are cancelled



• Arrive at the Corroborative Compliance Score– Compliance score only when activities are done in the

chronological order– Nil compliance for artifacts created in ‘one-go’

• Identify ‘Sustainability Index’– This score tells how much is the PCR compliance sustainable– Purely refers to artifacts in relevant location– CM Plan becomes the base for Sustainability Index

Going Beyond PCR


Internal Audit

How to design the best internal business control



Internal Audit Organization

QA HeadOversees Audits

AuditorsPractitionersConducts Audit

AuditorsQA

Conducts Audit

AuditorsSupport Function

Conducts Audit

Audit Manager

Manages Audits



Internal Audit Framework

• Audit team should be trained on auditing skills

• The Audit Manager should oversee the audits

• Audit frequency should be defined – ideally quarterly

• Support functions also should be audited

• Strive for common approach across auditors

• Normalization of findings should be done



Internal Audit Framework……(Cntd)

• Audit analysis leading to effective CAPA should be done

• Pareto analysis should be done to know ‘Key Contributors’ of Non-conformances



Internal Audit – Activities

• Maintain an annual audit calendar, with named audit cycles

• Have a well defined project sampling criteria

• Have 70% to 80% of Auditors from delivery

• Rest 20% to 30% of Auditors from support

• Clearly define SLA at every level

• Ensure formal Opening & Closing meetings

• Provide high visibility to findings closure status



Internal Audit SLAs

Audit Announcement

Audit Report Submission

Audit Report Queries Raising

Audit Queries Resolution

Findings Closure

Audit Team

Auditor Auditee Auditor Auditee

SLA for Findings closure depends upon finding category. Eg. 2 weeks for Major and 1 week for Minor. SLA for Findings Closure is followed by Escalation Mechanism.

Two weeks lead time to announce

audit schedule

Audit report to be

provided within 2

Business Days

Queries to be raised within

2 business days

Queries should be responded

within 4 business days

Findings should be

closed as per NC

classification



NC Closure Specific SLA Escalation

After 1 weekProgram Manager

After 2 weeksVertical Head

After 3 weeksQA Head

After 4 weeksCentre Head

Becomes a Organization

RISK beyond 5 weeks of non-

closure



NC Closure Specific SLA Escalation

NC Details

Open within SLA

Open beyond SLA

Escalated Level

PM Unit Head QA Head VP

NC 1

NC 2

NC 2

NC Details

Closed Closed beyond SLA Closed with

Proposed Action

within SLA

Escalated Level

PMUnit

HeadQA Head VP

NC 1 NC 2 NC 2

Open beyond SLA

Closed beyond SLA



Standard Audit Analysis

• NC score across delivery units

• Audits Planned Vs. Conducted

• Pareto analysis of Process Areas of NCs

• Process Area-wise / Clause-wise NC analysis

• NC Open / Close Status



Interesting Audit Metrics

• NC Density / Delivery Unit

• Auditors Effectiveness Score

• SLA Adherence – Delivery Unit-wise

• NCs leakage percentage – from PCR

• Average TAT to close NC – Delivery Unit-wise



Internal Audit – Key Success Factors

• Clear sampling criteria

• Comprehensive coverage of all operations & support functions

• Auditors Training

• Auditees Awareness Session

• Findings normalization

• Strict adherence to SLA & related escalations

• Providing high visibility to audit outcome to all Stakeholders



• Mechanism to track NCs that are closed with ‘Proposed Corrective Action’ in future releases

• All NCs should be tracked as following: -– Open – Within SLA / Beyond SLA– Closed – With Correction / With Proposed Corrective Action– Closed – Within SLA / Beyond SLA

• Each audit cycle should track last cycle’s NC closed with ‘Proposed Corrective Action’

• Closure verification can be done by SQAs and later concurred with concerned Auditors

Internal Audit – Finer Aspects


Innovation

Building a strong Innovation Culture is key for sustained growth



Innovation Framework

Six Sigma

Innovation Kaizen

Lean



Innovation Framework

• Build a culture of innovation in the organization slowly & methodically

• Innovation is disruptive thinking – needs a planned growth

• Expect enough resistance in the initial period

• Create a three to five year plan

• Start with high visibility – posters, mailers, roadshows etc.

• Form a strong Innovation Team with Master Blackbelt and Blackbelts



Rolling out Innovation Culture

• Start with Kaizens

• Conduct multiple Kaizen waves every year

• Define Kaizen as ‘small, implemented improvement’

• Never harp on hard savings

• Stress on benefits, soft savings achieved through Kaizen

• Create a common body to evaluate Kaizens

• Have defined criteria to ‘qualify’ a Kaizen



Awarding Kaizens

• Kaizens can scored based on ‘Originality’, ‘Re-usability’, ‘Cost Savings’, ‘Effort Spent’

• Provide Wave-wise awards for Kaizens Eg. Top ‘n’ Kaizens

• Have an Annual Best-of-Best Award across Kaizen Waves

• Make Kaizen Contribution as part of KPI for Managers

• Provide awards in public forums to convey its importance



Graduate into Six Sigma

• Kickoff Six Sigma projects on strong Kaizen base

• A very conducive Innovation Culture is key for successful Six Sigma rollout

• Kickoff Annual Six Sigma Cycle with identification of key business pain points

• Business pain points should be decided by the Senior Management

• Evaluate and rank business pain points

• Choose the Top ‘n’ pain points as candidates of Six Sigma Projects



Evaluating Pain Points

• Pain points can be evaluated on: -– Cost savings that will be achieved on fixing it– Time that would be taken for fixing it– Availability of data on the said pain points

• This will justify the time & effort spent on addressing the pain points

• Engage the Finance Department to closely track and confirm the savings on completion



Six Sigma Rollout

• Get nominations for Green Belt training

• Nominated persons will be trained and allocated to GB Six Sigma projects

• It is a good practice to localize the GB training content, with local examples

• GB Training should be exhaustive and on workshop mode

• Ensure that the Trainer is an MBB with relevant industry experience



Stages of Six Sigma Project

Define Define Problem Activities

Measure Measure problem Activities

Analyze Analyze Causes Activities

Improve Implement solution Activities

Control Sustain Success Activities



Monitoring Six Sigma Projects

• Draw a schedule for each Six Sigma Projects

• Schedule should never go beyond 3 to 4 months – under normal circumstances

• A long running Six Sigma project may lose Organization’s focus

• Plan review with MBB, Sponsor, QA Head and Senior Management at end of each phase

• Take a conscious call on progressing to subsequent phase

• Collectively, have the courage to call off a project if a strong un-resolvable problem is evidenced



Key Success Factors for Six Sigma Projects

• Focus on proper and pragmatic cause attribution

• Ensure good data quality for analysis

• Active participation of Finance person in all phase end reviews is mandatory

• Proposed solutions should be on causes identified and should not be superfluous

• Sponsor should be fully support effective implementation of proposed solution

• Focus should be on sustaining the benefits achieved

• Award Green Belt Certificate ONLY on successfully completing the project



Honoring Green Belts

• Conduct a high visibility Certification Ceremony for successful Green Belts

• Share the success details viz. Hard Savings in Dollars, Person Days etc.

• Use GB Certification as a Differentiator during appraisals for promotions

• Publish GB Success Stories in organization newsletters to spread Best Practices

• Set KPIs for Seniors on number of GB projects to be sponsored in a year


The Other Practices

Few standard and non-standard practices



Quality Award

• Institute a Quality Award based on defined parameters

• These should NOT be on compliance score alone

• Few parameters: -– SLA honoring in IQA– SEPG Attendance– Count of Process changes proposed to SEPG

• Make the award a quarterly affair

• Announce the Winner with high visibility



Train on Art of Facilitation

• Train the SQA on mastering the art of facilitation

• Facilitation is the ‘art of making a point without making an enemy’

• SQA needs to be soft but stubborn

• All Facilitation inputs should be formal in Facilitation Log

• Change projects allocated at defined frequency to avoid bias and monotony



Plan the Activities

• Make Leads provide a broad Activity Report on weekly basis

• Activity Report shall contain: -– Activities completed last week– Activities planned for current week– Activities that spans weeks– All the above with dates and status as applicable

• This needs to be a simple report conveying the snapshot of activities of the week



Sample Activity Report

Activities Planned for current week Activities Completed last week

Activity Description

Planned Completion Date Remarks Activity Description Completed On Remarks

Facilitation for Project A Facilitation for Project BPCR for Project XYZ

Notes: - Activities In-Progress

Activity Description

Planned Completion Date Remarks

Process Definition - DARProcess Review is pending with Process Owner, could be delayed

SAMPLE



Report Comprehensively

• Quality Team many a times fails in reporting developments

• Design a comprehensive monthly report to Senior Management

• Broad contents would be: -– Key accomplishments last month– Key activities planned for current month– Key Indicators – PCR Score, SLA adherence, Team Load,

Project Risks etc



Create High Team Morale

• Keep the team morale high

• Provide independence to report factual findings without fear

• Make it clear – Compliance is Delivery Responsibility, enabling is SQA responsibility

• Measure facilitation effectiveness through NC leakage, Facilitation PCR IQA

• Organize informal group activities for better team bonding

• Remember that your subordinate’s career growth is your responsibility as well



• High Compliance Score Vs. High Product Defects

• Delivery Accountability – QA Vs. Project Team

• Snapshot Compliance Vs. Sustainable Compliance

• Is Facilitation a policing activity?

• To CSAT or not to CSAT for SQA services

• War Time and Peace Time activities of QA Function

Case Studies



• What, When and How to escalate Project Risks to Senior Management

• Facilitation Productivity – Myth or Reality

• Compliance Vs. Defects Vs. CSAT – Can they be correlated?

• Best job rotation models for SQAs

Case Studies



Quick Recap

• Quality Organization – Sub-divisions

• Defining process

• Implementing process

• Effective NC defect leakage

• Strong Audit Controls

• Planning & Monitoring QA activities

• Team Morale



Thank You…!

Questions Please…!



Join today!



• No part of this presentation may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted.

• Requests to the publisher for permission should be addressed to [email protected]

Copyright

Thank You…!

mailto:[email protected]

quality delivery management

Software

master title stylerules

master title stylefit

master title styleobjective

master title styledisclaimer

copyrighted material

quality crowd

typical quality structure

better quality culture