what works best with tspi for small team productivity and quality · 2014-07-18 · tspi for small...

1W. L. Honig, TSPi Symposium 2006 San Diego, CA

What Works Best withTSPi for Small Team

Productivity and Quality

William L. Honig, Ph.D.Associate Professor, Department of Computer Science


TSPi Effectiveness withSmall Teams

• TSPi impact on software teams– 23 teams of 7 to 12 graduate students on real world

developments– Software process awareness and impact

• Productivity coupled with quality• Result of planning and analysis• Extensive data collection

• Bringing real world software experience to theclassroom– R&D leadership in communications companies– Land line, wireless, satellite, private and public

networks• Voice, data, land line, mobile, satellite, network

management


What Results?• Data Summary – Productivity

– Source Lines of Code (LOC) per Person Hour• High 47.4• Average 13.5• Low 1.8(complete Cycle 2 development, including reuse – all phases)

• Data Summary – Quality– Defects Injected per Total KLOC

• Low 2.8• Average 24.1• High 86.3


How do the teams work?

• Team composition• Students assigned to Team

» Based on From INFO

• Roles matched to background• Demographic mixture• Well trained individual programmers

• Learning environment• 14 to 17 weeks of class• Strict enforcement of team discipline• Face to face team meetings required• Students ? Employees, but can be “fired”

Team Phoenix

Fall 2001

Dr. William L. Honig2005

COMP 474 Software Engineering

Team Roles at a Glance(Five Specialized Roles)

• Support Manager

• Quality/ProcessManager

• Planning Manager

• DevelopmentManager

• Team Leader


Team Productivity – Cycle 2Source Lines Of Code (LOC) per Hour

0.000

5.000

10.000

15.000

20.000

25.000

30.000

35.000

40.000

45.000

50.000

To

tal L

OC

/ T

ota

l Tim

e in

Ph

ase

Ram

bler

Ice

Coo

l

Luci

d

Pho

enix

Blu

e B

ee

Dim

Sum

Doc

Max

Soc

rote

s

Kite

s

Tita

ns

G10

eUph

oria

Bet

a

Sea

ls

Bee

s

Sha

rp

Sili

con

Rai

ders

Evo

lutio

n

Ava

lanc

he T3

Cod

e W

arrio

rs

Pho

enix

II

Vol

ki

Teams

Productivity

Productivity of Each Team

Good


How is TSPi used in theclassroom?

• Student teams complete two cycles ofof development

• Same team assignment for both cycles• Some switch roles for cycle two

• “Customer” provides starting point• Product Needs Statement (not full requirements)• 2 to 4 meetings with customer to clarify needs and

review requirements and plans

• Teams present key milestones and demonstrateproduct to faculty, research assistants, customer


COMP 474 Software Engineering

Strategy

The Process at a Glance(TSPi)Plan

Requirements

Design

Implementation

TestPostmortem

Repeat

A controlled, data driven,step-by-step process for

software life cycle

Launch


How do students learnPSP first?

• Personal Software Process (PSP)– Required for individuals– Prerequisite for TSPi

• PSP trial introduction– Undergraduate programming course– Plan (estimate time), track defects, record time spend

• Only some TSPi student teams have thisexperience before TSPi begin– Quick two day introduction– One programming project


Development Projects

• “Real World” Development– University staff groups as customer

• working system or, • prototype or,• requirements clarification,…

• Wide range of applications– Prospect tracking for Graduate School– Summer visit registration for College of Arts and Sciences– Student Portal for Information Technology– Grant Approval and Tracking for VP Research

• Many technologies– C++, Java, XML, ColdFusion, …

Titans

Fall 2002


How are data collected?

• Textbook: Watts S. Humphrey, Introduction tothe Team Software Processsm

• Key data entered weekly into 21 forms:– Product Summary (SUMP)– Quality Summary (SUMQ)– Work Tasks/Effort (TASK)– Schedule and Earned Value (SCHEDULE)– Defect Identification and Correction (LOGD)– Inspection Reports (INS)– Time Recording Log (LOGT) Phoenix

Fall 2001


Initial Findings, FEB 2002

TSPi Plan Summary: Form SUM P

Name Date Team Instructor Part/Level Cycle

Product Size Plan Actual Requirements pages (SRS) Other text pages High-level design pages (SDS) Detailed design lines Base LOC (B) (measured) Deleted LOC (D)

(Estimated) (Counted)

Modified LOC (M) (Estimated) (Counted)

Added LOC (A) (N-M) (T-B+D -R)

Reused LOC (R) (Estimated) (Counted)

Total New and Changed LOC (N) (Estimated) (A+M)

Total LOC (T) (N+B-M -D +R) (Measured)

Total New Reuse LOC Estimated Object LOC (E) Upper Prediction Interval (70%) Lower Prediction Interval (70%)

Time in Phase (hours) Plan Actual Actual % Management and miscellaneous Launch Strategy and planning Requirements System test plan Requirements inspection High-level design Integration test plan High-level design inspection Implementation planning Detailed design Detailed design review Test development Detailed design inspection

If it’s notdocumented, it’s notthere…

If you can’t measureit, it’s not there…


What Results?Defects Injected per LOC

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

Tota

l Def

ects

Inje

cted

/ To

tal L

OC

Ram

bler

Ice

Coo

l

Luci

d

Pho

enix

Blu

e B

ee

Dim

Sum

Doc

Max

Soc

rote

s

Kite

s

Tita

ns

G10

eUph

oria

Bet

a

Sea

ls

Bee

s

Sha

rp

Sili

con

Rai

ders

Evo

lutio

n

Ava

lanc

he T3

Cod

e W

arrio

rs

Pho

enix

II

Vol

ki

Teams

Quality--a

Quality(a) of EachTeam

Good


Quality Results fromCycle Testing ONLY

• In Cycle Testing determines the qualitynumbers– No “production” use recorded

• “Testing can only show the presence ofbugs, not their absence”– Fault Seeding– Bug Density / Arrival Rate Analysis


Where are the Hours Used?Total Time by Phase

Total Cycle 2 Hours by Phase

20%

3%

9%

8%

16%

24%

15%

5% Mgmt&Misc

Launch

Strat&Plan

Requirements

Design

Implementation

Test

PostMortem6396 Total Hours to Date


Student Outcomes• Student Perceptions – Popular Course

• Team work experiences very positive learning• Understand process – appreciation varies• Data collection a struggle

– Volume of data needed– Needed for timely team cooperation

• My Viewpoint• Students well equipped to join industrial teams;

larger team sizes work well• TSPi textbook is great on metrics and quality, limited

on coverage of design, testing,…• Volume of “paper work” can lead to cybercrud

Volki

Spring 2005

Pot Luck



Students “Value” Forms

StudentSurvey:Choosetheformsusefulto yourteam.

Question 15

0%

25%

50%

75%

100%

CCR INS

TASKLO

GDSCHEDULE

SUMS

LOGT

SUMP

SUMPQCSR IT

LLO

GTESTSTR

ATW

EEKPEER

INFO

Greatest perceived value in forms that manage change anddefects (red) and project plan creation and tracking (blue)


How do these findingsapply to industry?

• Student teams approximate small industry task teams /development groups– Importance of (self) policing team behavior– Specialized roles help (in addition to developer role)

• Training / Coach / Observer role is critical to rapidintroduction of process such as TSPi– Get through one cycle quickly to speed learning– Need Process Coach / Facilitator

• Face to face regular meetings– Weekly cycle of data, analysis, action– Emphasis on analysis and quality is key

• Lead teams to analysis (not just data generation)

• Historical data a real help for getting started– If none, BEGIN NOW!


What about TSPi andSmall Teams?

• Team data for 23 student teams show industrylevel productivity early in learning TSPi- Quality *always* needs focus

• TSPi can be learned efficiently andapplied rapidly– Team composition and coaching

• The “academic” learning approach likelyapplicable to other types of organizations– Value of discipline, data collection, metrics

G10

Fall 2002


LOC Vary GreatlyTotal LOC and Its Max. Min. and Avg.

0

2000

4000

6000

8000

10000

12000

14000

Ramble

rIce

Coo

lLu

cidPho

enix

Blue B

eeDim

Sum

Doc M

axSoc

rotes

Kites

Titans

G10eU

phor

iaBet

aSea

lsBee

sSha

rp

Silicon

Raid

ers

Evolut

ionAva

lanch

e T3

Code W

arrio

rsPho

enix

IIVolk

i

Teams

To

tal L

OC Total LOC of Each Team

MaxMinAverage



Ramblers Team Metric Chart

Planned Value vs. Earned Value

11%

32%

93%100%

0%

20%

40%

60%

80%

100%

120%

Week1 Week2 Week3 Week4

Time

Val

ue PV

EV

Week1 Week2Week3

Week4

PHAH

90 85

141

82

89 8989 89

-

50

100

150

Val

ue

Time

Planned Hours vs. Actual Hours

PH

AH

Defects Injected vs. Removed

5

5

4

5

5

4

Week1

Week2

Week3

Week4

Removed

Injected

CCR Tracking Chart

-7

16

--7

16

-- - - --

5

10

15

20

Week1 Week2 Week3 Week4

Time

No.

Submitted Approved Rejected

No. INS Finished

5

4

6

0 2 4 6 8

Week1

Week2

Week3

Week4

No.

Time

No. INS

Ramblers Team Metric Chart

5-UpChart

Pert Chart

Lau Strategy Plan Request Design Implement Test Postmortem

Size and time Estimation

(All members)15 hrs

Set goals (All members)

5 hrs

Updaterisk & issues (Support M.)

5 hrs

Updateconfiguration management

procedure(Support M.)

5 hrs

UpdateProduct listand size

estimation(Plan M.)14 hrs

Allocate tasks among members

(Plan M.)3 hrs

Estimating the defects

(Quality M.)10 hrs

Produce SRS(Develop M.)

11 hrs

Produce STP(Develop M.)

11 hrs

Inspect SRS(Quality M.)

6 hrs

Produce SDS(Develop M.)

9 hrsInspect SDS(Quality M.)

8 hrs

Detaileddesign

(All members)14.5 hrs

Test plan and development (Develop M.)

8 hrs

Build & Integration

(Support M.)9 hrs

Documentation (Support M.)

11.5 hrs

System test(Develop M.)

9.5 hrs

Week 1 Week 2 Week 4Week 3

Proj

ect

Task

s

Task

Dependency

Time

Assign roles(All members)

3 hrs

ITL

SUMSTASK

SCHEDULESUMPSUMQ

Pert Chart

30 hrs

Inspect STP(Quality M.)

6 hrs

SRSSTP

SRS-INSSTP-INS

Produce ITP(Develop M.)

7 hrsInspect ITP(Quality M.)

5 hrs

SDS-INSITP-INS

SDS-INSITP-INS

Implementationplanning

(Develop M.)11.5 hrs

Unit Test Plan(All members)

9.5 hrs

Detailed Design inspection

(Quality M.)11 hrs

Code(All members)

22 hrs

Code inspection

(Quality M.)9 hrs

Quality review(All members)

7.5 hrs

Code, CCRSUMS, SUMPSUMQ, LOGTLOGD, INS

LOGDLOGTEST

SUMPSUMQ

Finish documentations (Support M.)

24 hrs

Update douments(All memebers)

22 hrs 17.5 hrs 58 hrs

PIIC

CYCLE 2



Larger Team Size Works• Flexibility in Roles:

– Some ability to switch roles– Easier to recover from “drop outs”

• Student Feedback:– Students identified the problems their team encountered– 20% felt a smaller team size of 5 would lessen the problems



What are some next steps?

Expand Focus on Analysis Metrics for In cycle QualityImprovement

Ease Data Gathering Travail Mobile Tool

Incorporate Teaching Materials on Technique Best Practices

Effectiveness of TSPi to Accelerate Transition to CMMI

Questions, follow-ups, ideas…. contact

William L. [email protected], 1-312-915-7988

what works best with tspi for small team productivity and quality · 2014-07-18 · tspi for small...

Documents