cse 7315 - sw project management / module 27 - project tracking and oversight copyright ©...

CSE 7315 - SW Project Management / Module 27 - Project Tracking and OversightCopyright © 1995-2006, Dennis J. Frailey, All Rights Reserved Slide 1

CSE7315M27

January 7, 2006

SMU CSE 7315 / NTU SE 584-NPlanning and Managing a

Software Project

Module 27Project Tracking and

Oversight

Slide # 2January 7, 2006

CSE 7315 - SW Project Management / Module 27 - Project Tracking and

OversightCopyright © 1995-2006, Dennis J. Frailey,

All Rights ReservedCSE7315M27

Objectives of This Module• To present an overview of tracking

and oversight• To discuss some basic terminology

regarding measurement





Tracking and Oversight Occur ContinuouslyManage Risks

Definethe Approach

GenerateDetailed Plans

Understandthe Need

Execute and Monitortext, chapters 6, 15





SW Project MonitoringTypical Symptoms of a Problem

We’re six months behind schedule, and

nobody knew it!

Why did it take us so long to find

out?

January





SW Project MonitoringSymptoms of Another Problem

They’ve beenworking on that module for eight weeks and everyone

else is waiting for it.

Did thedevelopers in charge

know how many people depend on

themodule?





SW Project MonitoringOther Examples

• The project manager promised a new feature to the customer -- but never told any of the programmers!– “I thought you knew about this!”

• The software takes up too much disk space.– Nobody ever thought it would get that big.





SW Project Tracking and Oversight

PurposeTo provide adequate visibility into actual progress so that

management can take effective actions when the

software project’s performance deviates

significantly from the software plans





SW Tracking and Oversight Goals from the SEI CMM/CMMI

1) Actual results and performance are tracked against software plans– Plans are revised to reflect actual

performance and changes in requirements or commitments

2) Corrective actions are taken and managed to closure when actual performance deviates significantly from software plans





3PM Today

SW Tracking and Oversight Goals from the SEI CMM/CMMI

3) Changes to software commitments are communicated to all affected groups and individuals





SW Project Tracking & Oversight

Practices Recommended by SEI• Use a software development plan for

tracking and communicating status• Track schedule, size, effort, computer

resources, technical activities and risks• Hold periodic reviews and take corrective

actions• Revise plans and schedules to reflect

changes -- using a defined procedure• Review customer commitments on a

regular basis





Tracking Progress





Things You Can Estimate and Track

• Costs• Sizes• Quality• Reliability• Schedules• Staffing• etc.





Establish a Data Base• Know about your organization– Performance on past projects– Lessons Learned

• Know about your industry and competitors– What is best in class?– Improvement rates

HistoricalData Base

- Data- Lessons- etc.

Facts to help you manage





Example of Experience vs. History

• History: for C++ doing your kind of software, you should be generating– 25 lines of code per day during the

coding phase, with– 3 errors per 1000 lines of code during

module test• Actual experience on your project:– 40 lines of code per day, with– 0.5 errors per 1000 lines of code

during module test





Optimist’s Conclusion

Do you have a solid reason to explain this difference?

Ask questions. Why are you better?– Is the process different?– Are the people a lot better?– Are the tools better?

We are doing much better than in the past!





Pessimist’s Conclusion

Ask questions. Find out what is really happening.– Are the tests being performed?– Is the coverage adequate?– Are there higher rates of customer complaints

after shipment?

Our testing is no good (perhaps because it is being

rushed due to deadlines)





Knowing the Competition Can Give You Insights

• But what if the norm in your industry is an improvement of 25%?

• And what if your competitors have all switched to Java and are 50% more productive as a result?

Weimproved our “C” language

productivity by 15%





Measurement





Every Measure Should Have a Purpose -- You Want to Get

Information

Data Analysis Information





But for Every Analysis there are Two Possible Results

Information - tells you something right–We are (or are not) on schedule–Our risks are (or are not) under control

Misinformation - tells you something wrong –We are (or are not) on schedule–Our risks are (or are not) under control

And there will always be changes in the organization when you measure it





Key Issues• Define how to interpret measurements– To form a basis of consistent analysis

• Choose consistent display techniques– So people know how to interpret the data

• Define how to use each measurement– You must also demonstrate that you are using

it that way, so people will believe you– Given any measure, people will change to

make it look to their advantage• you want to make their behavior change in a

positive way





Organizational Framework• There must be an Organizational

Framework for understanding the importance of measurements– i.e., people do not sabotage the data

collection effort– And people do not abuse measures– And people do not draw wrong

conclusions





Achieving an Effective Organizational Framework

• Educate everyone in the proper use of measurements

• Develop the right measures– Involve those who are being measured– Measure only what you can benefit from

• Use the measurements– To make decisions about the product and

the process– But NOT to make decisions about people





You Want to Achieve Optimal Performance

• Don’t over-measure or under-measure• Don’t over-test or under-test• Don’t over-inspect or under-inspect• etc.

• Track the things that represent your greatest risks and concerns

• Remember that it costs time and money to track - make it worthwhile





Basic Definition and Terminology Issues

• Levels of Measurement• What to Measure• Who Cares about What

Measures





Levels of Measurement

• What do we mean by a measure?• Does everyone mean the same

thing?• How do we resolve discrepancies?• Consider the example on the next

slide:





Information may be Measured in Many Ways!

Cost is best

measured in staff

days!

Why not

dollars?

I measure

it by drop in

stock price.

However they measure it, they had better figure out how to reduce cost!





Information, Measures and Data

Process in Execution

Data Data Data DataData

Information Need

Measure Measure Measure





Example

Process in Execution

UnitsProduced

Head-count

Lines of Code $ SpentMonths

Productivity

Units Per Month LOC perStaff Month $ per Line of Code





Definitions: Data

Data (or Primitive Measures)The fundamental, factual quantities that

characterize a process or product

– Specific facts that are countable or otherwise obtainable

– Minimum amount of analysis





Key Issues: Data

• Consistent definitions• Effective collectionExamples:– Hours worked– Number of employees– Lines of code





Consistent Definitions• What do you mean by a “day” of work?

Does everyone mean the same thing?• Such questions should be asked about

any data you propose to collect, such as lines of code, people, dollars, defects, variance, tests completed, complaints

You must define what you want … -- How many hours per day? -- What is a “line” of code? -- Which lines to include in LOC?





How Long Is a Staff Day?

Salaried Staff

Staff Day

OvertimeRegular Time

Hourly StaffSalaried Staff Hourly Staff

Unpaid

PaidUnpaid

Paid





Effective Collection• Can you collect accurate and

consistent data?• Can you collect it efficiently?• How will the organization and the

process change when you collect the data?





Definitions: MeasuresMeasure (sometimes called Compound Measure)The result of counting or otherwise quantifying

an attribute of a process or product

– How we quantify – Something we can analyze and interpret– Generally associated with specific graphs

and rules of interpretation





Key Issues: Measures• Proper interpretation• How measures are displayed• Effective use of measuresExamples:– lines of code per hour– ratio of current to historical test

performance– turnover rate





Display and Interpretation:Consider the Following Graph

Modules Tested

05

1015202530

0 1 2 3 4 5 6 7 8 9 10 11 12

HistoricalPlanActual





Effective Use• You must define how you will use each measure– A measure can typically be used many ways– Some of these ways can be ineffective or even harmful– People tend to fear measurement because of past

experience with misuse• You must also demonstrate that you are using it

as planned, so people will believe you• Given any measure, people will change to make

it look to their advantage– So you want to make their behavior change in a

positive way





Example: A Measure & Its Impact

Information Need: ProductivityMeasure 1: Lines of code per dayUse: reward those who produce the most lines of code per dayResult: people may produce bloated, inflated code in order to look goodMeasure 2: requirements met and tested, weighted by complexity of requirementUse: track against history and use to identify

process bottlenecksResult: people may use the data to make the process more efficient, resulting in lower cost





A Less Common Measure That May be Very Effective

Information Need: ProductivityMeasure 3: Number of customer complaintsUse: reward those who produce the code that is responsible for the fewest customer complaintsResult: developers may pay particular care to the

customer’s need, resulting in future sales and

business success





Definitions: InformationInformation

A measured quantity that provides insight into a software issue or concept or goal.

– What and why we measure– An indicator – A standard of measurement that ties to a

goal or purpose





Key Issues: Information• Selecting a measure that provides

the desired insight or information regarding the goal or purpose

Examples:– Comparisons (planned vs. actual)– Indices (performance ratio,

productivity, return on investment)





Focusing on the Goals and Needs

• Customer satisfaction can be measured in terms of response to a survey that biases customer response and hides their real views

• Or it can be measured in terms of something that accurately reflects customer views

• The same applies to measures of productivity, quality, cycle time, defect density, etc.

Each measure should provide information that addresses some higher level objective, such as

“reducing cost” or “on time delivery”





Select the Right Measure• There are many ways to measure a

desired quantity and different organizations measure differently

• The key is to spend time to select the right measure, so you satisfy the information needs in support of the higher level objective(s)





Good and Not So Good Measures

Goal: Produce software more efficientlyInformation Need: Efficiency

Measure 1: tests completed per weekResult: easy tests done first; corners cut in testing; hard problems ignored or deferred

Measure 2: reworkResult: process and methods are improved to reduce rework, resulting in more efficient software development





Rework Is an Excellent Measure of Efficiency or

Productivity• It tells you where you are wasting

resources

However …• Rework is a lagging indicator.– It does not spot problems in advance





What Should We Measure?Product ProjectProcess determine

ssuccess of

determines

quality of

root causesroot causes

Process Measures–Effectiveness of the process–How well are we following the process?–Risk monitoring

Product Measures–Performance and quality–How well is the product meeting its requirements?

Project Measures–The state of the project–How are we doing relative to cost, schedule, staffing, etc.?





Product ProjectProcessAttributesWhat

Resources

Quality

Time Are We On Schedule?

Expenses vs. Budget?

How Fast can we Manufacture?

What Is our Cycle Time?

Post-release Defects?

What will it Cost?

What is our Productivity?

Customer Satisfaction?

In-process Defects?

Performance

Meets Perfor- mance Goals?

Meets Mgt. Goals?

Does it Work?

What Attributes Can We Measure?

• We want attributes that relate to our goals – time, resources, performance, quality etc.

• The following type of matrix can help:





Who Cares About What?• Managers usually care about project

measures - that’s how they are judged -- But if the project is in trouble they need

to know more• Developers usually care about product

measures -- that’s what they are evaluated by• Both should care about process measures -- this is usually where you learn the

reasons why a project or a product is in trouble





Summary of Module• Measure to know what is really

happening• Avoid misinterpreting the data• Define how to measure, interpret,

and display the data• There are many ways to measure a

given quantity – choose one that works best for you





References• Baumert, John H., and Mark S. McWhinney,

Software Measures and the Capability Maturity Model, CMU/SEI-92-TR-25, ESC-TR-92-025, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pa., 1992.

• DeMarco, Tom, Controlling Software Projects: Management, Measurement, and Estimation, New York, Yourdon Press, 1982.

• Grady, Robert B. Practical Software Metrics for Project Management and Process Improvement. Englewood Cliffs, N.J., Prentice-Hall, Inc., 1992. ISBN 0-13- 720384-5.





Possible Exam Questions Explain why tracking and

oversight are important for project management

Explain the difference between an information need and a measure.

Explain several risks with defining and collecting data properly





END OFMODULE 27

cse 7315 - sw project management / module 27 - project tracking and oversight copyright ©...

Documents