AgileKPIsvs.TraditionalKPIs–Amindshift

1. ExecutivesummaryManagersencountercontinuedpressuretodelivermoresoftwareinlesstimeandtheytendtointroducemanydifferentKPIstomeasuresuccess.ButwhydotheyintroduceKPIsinthefirstplace?WhicharethegoodKPIs? Whichonesarenotuseful?Andwhichonesaretheharmfulones?Thiswhitepaperpresents someof themost commonKPIs and the expectedoutcomesthatyoucouldfindusingthem.

2. Historicallywhydidwemeasuresuccess?Successisdifficulttomeasurebecauseithasmanydifferentvariablesandeachstakeholderwillhaveadifferentviewofthedefinitionofsuccess:thebusinessownerusuallyconsiderssuccessashort“timetomarket”andwithinbudget,whileatechnicalarchitectmightconsiderstabilityorscalabilityasthesuccesscriteria.HistoricallyKPImeasurehasbeenusedforthefollowingreasons:

1. Analysethecurrentprocesstoidentifyandunderstandgaps;2. Promotebehavioursthatwillleadtobetterresults;3. Understandhowsuccessfulwehavebeenandproducealessonslearntreport;4. Punishorrewardteams,individualsordepartments

ThewaytheKPIswillbeusedintheteamscanleadtogreatsuccessortremendousfailure.AKPI is like any other tool that you could find in a toolbox. A hammer can be use to hang apaintingoritcanbeusedtobreakawindow.ItisuptotherelevantstakeholderstounderstandhowshouldtheyapplytheinformationprovidedbytheKPI.The strength of Agile is to fail fast and cheaply, learning from themistakes occurred.WhilstsettinguptheAgileenvironment,oneshouldalwaysaskthefollowingtwoquestions:

A) IstheKPIallowingmetoimprovetheprocess?B) IstheprocessgoingagainsttheAgileprinciples?

UsingthepunishmentapproachinanAgileenvironmentwilldiscouragethepeopletobefreeto speak up their mind. That can lead to the situation when the people would hide theirmistakesandthereforenousefulandrelevantlessonslearntreportcanbeapplied.Ultimately,theteamswillincreasethetechnicalandnon-technicaldebt.Eventually,therewillbeapricetopayforthatdebtandthelongerisunidentifiedandconsequentlyfixed,themoreexponentiallycostlyitwillbetorepairthedamagesproduced.Furthermore, one of the strengths of Agile comes from an environment where the teammemberscanfeelfreetosearchforthebestvaluefortheproductownersandforthefinalusers.SetupKPIsthatwillnotaffectthestability,theprocessesortheteam.Thepositiverewardapproach isnotnecessarilyagoodapproacheither.Forinstance,donotbelievethatworkinghardformanyhoursisagoodKPI.TheUniversityofNebraskaprovedthatworkingextra10hoursaweekcanreducetheoverallproductivitybyupto40%(Raynar1997).Also,itisimportanttonotethatpositivereinforcementforoneteammembercanbeseenasapunishment for another one. As a consequence, avoiding punishment or reward is the bettersolution.

2.1. Traditionalvs.AgileKPIsTheKPItounderstandsuccess isaninheritanceoftheWaterfallmentalityashistoricallytheorganisations did not have any earlier warning tools. Managers tend to use this mechanismbecausethe“lessonslearnt”stageisattheendoftheproject.However,theKPIstoimprovetheprocessarethefoundationofAgile:Tips:

1. SettinguptherightKPIsthatwillimprovetheprocessesandwillleadtomorequalitysoftwareinlesstime.

2. SettingupmanagerialKPIstoanalysethequantityorqualityofyoursoftwarewillharmyourAgileprocesses.

3. Worksmart,nothard.2.2. TherightKPIsinAgileAgile is amethodology that promotes change and adaptability.Managers have to understandthat theKPIs should reflect the reality, should be careful not to restrict freedom, and shouldavoid rigidity and static behaviours. The classic example of this is to standardise a value perpointintermsoftimeorcost.Theultimateoutcomeofsuchactionisthattheteamswon’thaveasenseofachievementandsuccessduringthesprints.

Finding the best KPIs for your environment requires a high level of understanding of thesituation, empathy and patience. The golden rule should be: find your teams’weakness andsetupKPIsthatwillpromoteAgileprinciples.Alsoanalyse:

a) DoesthisKPIpromoteindividualsandinteractionsorprocessesandtools?b) DoesthisKPIpromotemoreworkingsoftwareormorecomprehensivedocumentation?c) DoesthisKPIpromotecustomercollaborationormorecontractnegotiation?d) DoesthisKPIpromoteflexibilitytochangeorrigiditytofollowaplan?

the SothatsoweWewanttodosoftware

Wemakemoresoftwareinthenextsprint

ProcessesSetupaKPItoimprove

sowe sothatWewanttodosoftware

Howmuchsoftwarewewillhavedeveloped

Wecanunderstand

projectsuccess

SetupaKPItounderstand

ExamplesofKPI(let’splayabitofGameTheory):

• “Wearegoingtocompareteam’sefficiencytopromotethenextmanagers”.IfweapplysomeGameTheoryprinciplestothisKPIwewillcometothefollowingconclusion:teammembers are competing among themselves for the best results.What initially seemslikeagoodidea,inreality,itcontradictsprinciple“A”and,forthelongterm,theoutputof this isKPIwillbe that teammemberswillno longer collaborate.Thus, thegeneralcombinedefficiencyoftheteamswillbereduced.

• “Wearegoingtomeasurethevelocityoftheteams”.Velocityisagreatinternaltoolthat

each team uses to understand their strengths and weaknesses, and bring betterestimatesinfuture,thereforereducingtherisk.Ifamanagertriestocomparevelocities,then the logical reaction is that therewill bean inflationof thepointvalue.Thatwilldilutetheinformationandtheestimationswillbecomeunstableandincreasetherisk.Thishappenedbecauseitwasagainsttheprinciple“A”.

• “We are going to measure client satisfaction”. Satisfaction of the clients can only be

achievedbyincreasingcustomercollaboration(principle“C”). ThatKPIwillpositivelyinfluenceyourteamstoimproveyourprocess.

• “WearegoingtomeasurethenumberofAgilebestpracticesusedduringthesprints”.

Thiswillpromotealmostallprinciples,improvingprocessesthatwillthusleadtomoreworkingsoftware.

TonotethatthereareafewKPIsthatcanbeusedinanyenvironment:

• ContinuousImprovementIndex;createacultureofimprovingandofsharingideas;

• Agilebestpractices;• Engagementlevelswiththeusers;• Teamsvelocity.

3. AgileMaturityAssessmentManypeoplehavetheoreticalknowledgeofAgilebutnoteveryoneisawareoftheculturalandorganizational challenges that the transformation towards Agile requires. Generally, creatingtherightenvironmentsrequiresadualapproach:

a) Top-downapproach: theorganizationalneeds tounderstandhowmature theyareanditsnextsteps.Thiswillrequirebuyinandsupportfromseniorstakeholders;

b) Bottom-up approach: how are the technical processes, architecture and toolsrunningonthedaytoday.

Once the assessment has been completed, the right KPIs can be suggested to promote Agilebehavioursinyourenvironment.Importanttip:DonottrytosetupKPIsbeforeconductinganAgileMaturityAssessmentbyanAgile Coachwith extensive practical experience. He should understand deeply the strengthsand weakness of the individuals, the teams, the organization, the dynamics of the businessprocessesandtheappetiteforchange.

4. KPIanalysisThefollowingtableshowssomeofthepossibleKPIsandwhatdotheypromote.ItisimportanttonotethatsomeKPIsareuseful to improvetheprocesses for theAgile teams(i.e.,chickens)butsomeotherstakeholders(i.e.,pigs)couldmisusethem.

ManagerialKPI

AgileKPI

Punishorreward

Understandsuccess

Promotebehaviours

Understandgaps

KPI TheKPIpromotesCustomersatisfaction X X X

DonotuseanyKPIaspunishm

entorreward

X X ClientorientedfocusBestAgilepracticesusedeg.XP,Lean,SAFE,KanbanScrum. X X X Increaseproductivity

Velocity(pointspersprint) X X X IncreaseefficiencyTeamstability X X Willgeneratebetterteamwork Attrition X X TeamcollaborationTimetomarket X X X X X FlexibilityoftheteamstoadaptAgileadoption X X X Promotesbestpractices Businessinvolvement X X X X ClosethegapbetweentheITandthe

businessWorkingsoftware

X Itcanleadtoanendlessloopunderstandingproductivity(quantityvs.quality)

Numberofreleases X X X Itcanpromotereleasingmoreorlessthanwhatisactuallyneeded

Ontimedelivery X X X Thiscouldleadtounder-commitmentfromtheteams

Valuedelivered X X GoodKPIbutproblematicassometimesvalueisoutofthecontroloftheteams

Compareteamsvelocity X InflationofpointsIndividualvelocity PoorKPIasitwilllowermoraleand

increasemicromanagementNumberofcodelines PoorKPIasdeveloperswriteinefficientlyPercentangeofsprintcommitment X X X AttentionrequiredasthisKPIcanreduce

quantitycommittedandreducemoralDefectsdensity X X X GoodKPIbutattentionnottolower

amountofsoftwareneededValueofpoints PoorKPI-InflatepointsResponsivenesstochange X X X X ImprovescollaborationTimespentinimprovements X X X X Improvesprocesses Unfinishedstories X X X X X UnderstandwasteandvalueUseandAgileMaturityassessment(Fuller 2014) X X X X UnderstandAgilejourneyandcreatethe

rightKPI Convertormeasurepointsintodays X X PoorKPI:motionlessmentality.Theteam

doesnotfeelthattheyaresucceedingCompareteams/people X TerribleKPI:nocollaborationPeopleperformances X TerribleKPI:Burnouteffect/lowmoral.

LowcollaborationTheseKPIscanbeusedinanyenvironmentastheypromotegoodAgilepractices.

SomeotherKPIsthatyoucouldconsidercouldbe:WhilechoosingyourKPIyoucanalso takea looktosomeof the followingmetrics: leadtime,defect count (at various phases; i.e., what’s a bug?), work in progress, code coverage,responsivenesstochange,teamstability,velocity(storypointsorstorycountpersprint),returnoninvestment,innovationspersprint,continuousimprovementtime,failureload(fire-fightingtime), iteration burn-down, unfinished stories, innovation index, customer satisfaction, un-deployed stories, number of blocks, budget/schedule compliance flow efficiency (leadtime/touchtime),releaseburn-up,projectedcapacityofnextsprint(inpersondays),projectedcapability (in story points range) of next sprint, planned capability of the sprint (in storypoints), accepted velocity - story points done and accepted by product owner in the sprint,discoveredwork added to the backlog (in story points and stories), scope increase/decreaseduring the sprint/release, technical debt incurred this sprint (in story points), cost of sprint,sprint predictability, actual stories completed vs. committed stories, retrospective processimprovement, communication, understanding of sprint scope and goal, user stories deliveredversususerstoriesaccepted,defect-removalefficiency(DRE),valuedelivered,ontimedelivery(Leigh2015).

5. Summaryandtakeawaytips

a) SettingupKPIsthatwillpromotetheimprovementoftheprocessesandwillleadyourteamstomorequalitysoftwareinlesstime.

b) SettingupmanagerialKPIstoanalysethequalityandquantityofthesoftwarewillharmyourprocesses.

c) “Increments of improvements” is the best KPI to make more software with betterquality.DonotuseKPIsasmeanstopunishorrewardyourpeopleinAgile.

d) Every fewsprints conduct anAgileMaturityAssessment tounderstandyourprogressandthenextsteps.

AbouttheAuthor

JavierEspinosadelosMonterosForetisanindependentAgileCoachthatafterobtainingtwofirstclassdegrees,ManagementandIT,workedformorethan10years in international projects across Europe leading Agile and Waterfallsoftwareprojects.HeobtainedhisMBAfromIEBusinessSchoolandheisalsoaScrum Master Certified, Change Manager certified APMG, Project Manager(Prince2Practitioner)andProgrammeManager(MSPPractitioner)certified.

yo@javierespinosa.comhttps://uk.linkedin.com/in/javierespinosadelosm

AcknowledgementsSpecialthankstoSimsonLeigh,mygoodfriendandexcellentAgilecoach,forhisconstantadviceandsupport.

BibliographyFuller,Dan.MeasuringAgileAdoptionMaturity.21May2014.http://www.slideshare.net/cognizant/shuhari-measuring-agile-adoption-maturity(accessedSeptembre10,2014).Raynar,RandolphThomasandKarlA.“SCHEDULEDOVERTIMEANDLABORPRODUCTIVITY:QUANTITATIVEANALYSIS.”JournalofConstructionEngineeringandManagement123(1997):2.