process mapping and discrete event simulation for...

Process Mapping and Discrete Event Simulation For Defect Costing And Scheduling Estimation

20th International Forum on COCOMO and Software Cost ModelingBy Jack H. Arabian, Comparative Management Associates, Milton, MA 02186

Key Words...........................................................................................................................1Acknowledgements..............................................................................................................1Introduction..........................................................................................................................1Requirements.......................................................................................................................2Procedure.............................................................................................................................2Outcome...............................................................................................................................7Analysis:..............................................................................................................................8Live Demo...........................................................................................................................9Future Work.........................................................................................................................9References..........................................................................................................................10End.....................................................................................................................................11

Figure 1 Generalized Defect Detection Cost and Schedule Estimation Process.......................................2Figure 2 An Overview of the Engineering Process Expanded to Three Phases.......................................4Figure 3 Example of the Sub-Model entitled Collate and Collect Defects................................................5Figure 4 Dialog for Activity labeled Review Board Sort Contained Requirements Development Defects..............................................................................................................................................................6Figure 5 Capsule of the Final Report for the Normal Run Scenario with Relevant Variables Highlighted......................................................................................................................................................7Figure 6 Capsule of the Final Report for the Scenario_2 Run with Relevant Variables Highlighted.............8

Key WordsProcess, Mapping, Modeling, Simulation, System of Systems, Six Sigma, Estimation.

AcknowledgementsThe author gratefully acknowledges the contributions of Robert J. Floyd, who suggested the paper and advised on the model.

IntroductionProcess engineering tools have grown from the need of industrial engineers to describe and map out the sequence of events in engineering, manufacturing and business processes. [1] Laid out on paper, a long and complex process was cumbersome at best until the computer relieved the burden of multiple changes, notification, and information storage.[6] [7] [8] [9] The process mapping application combined with discrete event simulation [2] capability now fills the need for quick turnaround, reliability and accuracy in modeling.Recently, the marriage of process modeling with performance models has expedited the ability of engineers and programmers to branch out and integrate related sub-processes in the enterprise. The technique presents new opportunities to create true System of Systems (SoS) models [3] [5] and raise levels of abstraction, which are otherwise not as easy to achieve.[4] Prediction of defects, defect density, defect containment by phase, cost of

Jack H. Arabian, Comparative Management Associates, Milton, MA 02186, Tel (617) 733-3363 1

quality and other Six Sigma considerations widen the scope of simple modeling. This paper introduces a tool to ease the burden of data collection and estimation. The beauty of modeling and simulation is the ability to predict future outcomes by creating scenarios with “what-if” parameters. Once modified, the original “as is” model becomes a tool to estimate and forecast effects on the changed model. Such a technique is usable in many software /system estimation environments.This paper describes a novel application, which is used for an engineering process to detect defects and predict/estimate costs and scheduling.

RequirementsA need existed to document the defect detection and solution procedures of an engineering product development process, and combine the process with a performance model. The following steps were taken:

1. Examine and document the engineering defect detection process.2. Create a visual diagram/display of the step-by-step analysis with defects to be

identified, tagged, and counted.3. Collect/collate the parameters of each step, which will be integrated into a global

array for meaningful reporting. Statistical distributions and estimations were used to cover the spectrum of non-constant, time-varying parameters.

4. Process the parameters with meaningful calculations, which will yield a range of estimates of cost and schedule to compare with past performance.

5. Time-simulate the process with known “as-is” parameters, then create required scenarios with “what-if” parameters for a range of outcomes where the object is future estimation of costs and schedules.

6. Identify related and sub-processes which influence the model’s performance.


Procedure1. We documented and diagrammed the process procedure using a commercial

process mapping application,1 as shown in Figure 1 below.

Figure 1 Generalized Defect Detection Cost and Schedule Estimation Process

1 ProcessModel™, Provo, UT 84601

Jack H. Arabian, Comparative Management Associates, Milton, MA 02186, Tel (617) 733-3363

ImportGenerated

KSLOCDetect Defects

Count & Document In-Phase Defects

Out of PhaseDefects Identifiedand Processed in Later Phases

Collect/Collate All

Defects

SolveDefect

ProblemImplement and Review Release

Estimate FutureCosts and Schedule

In-PhaseDefect

Out-of-PhaseDefect

To PhasesFollowing

3

2. Many processes consist of several phases, where each succeeding phase duplicates the previous one except for the parameters. It is convenient to map out each phase, i.e., replicate Figure 1 above, for as many replicates as are needed. For purposes of this paper, we used three replicates as shown in Figure 2. The mapping application conveniently accomplishes our goal. For purposes of this paper, Figure 2 was reduced in scale, but can be expanded electronically.

Figure 2 An Overview of the Engineering Process Expanded to Three Phases


3. Figure 1 is a higher level of abstraction of each of the process blocks; e.g., Collate and Collect Defects, is a collection of steps or activities which can be represented by a sub-process (or sub-model), as depicted in Figure 3. For each of the blocks in Figure 1, we created similar sub-models.

Figure 3 Example of the Sub-Model entitled Collate and Collect Defects


4. Within each sub-model, each of the activities in the collection can now be assigned parameters, such as cost per activity and time per activity in its own dialogue box. Figure 4 shows the dialogue for the activity labeled Review Board Sort Contained Requirements Development Defects

Figure 4 Dialog for Activity labeled Review Board Sort Contained Requirements Development Defects

5. Additions to the three phase model of Figure 2 were made in the form of interconnecting activities to demonstrate the effect on performance, viz., additional non-contained phase defect simulation to enhance the model.

6. The process map was converted by the application to its simulator mode to show the existing process and performance with actual historical, statistical data.

7. The simulator was re-run in Scenario_2 with “what-if” parameters to yield future estimates of cost and schedule.

Jack H. Arabian, Comparative Management Associates, Milton, MA 02186, Tel (617) 733-3363

This Activity has this dialogue box

Multiple tabs for categories of information

Review Board Sort

Contained Requirements Development

Defects

Multiple tabs for categories of information

Variable time, “Q” for different scenarios

6

Outcome1. The simulator has yielded estimates of cost and schedule with scenario

perturbation of the installed parameters. Figures 5 & 6 show four selected parameters for the two scenarios of Normal Run, and Scenario_2:

a. v_RDReviewBoardCostb. v_P1HX (Total Time to accomplish the processing of all Requirements

Development defects through this first phase, which we have labeled P1)c. And similarly for v_P2HX, and v_P3HX.

Figure 5 Capsule of the Final Report for the Normal Run Scenario with Relevant Variables Highlighted

General ReportOutput from G:\ProcessMod\12COCOMO.modDate: Sep/03/2005 Time: 11:55:20 AMScenario : Normal RunReplication : 1 of 1Simulation Time : 30.75ACTIVITIES

Activity Name

Scheduled Hours Capacity

Total Entries

Average Minutes Per

Average Contents

Maximum Contents

Current Contents % Util

Start Requirements Development 30.75 999999 498 0.0 0 498 0 0Trigger Requirements Development inQ 30.75 999999 1 0 0 1 0 0Trigger Requirements Development 30.75 1 1 2 0 1 0 0.11Sort Requirements Development Criticals inQ 30.75 999999 498 248.5 67.07 497 0 0.01Sort Requirements Development Criticals 30.75 1 498 1 0.26 1 0 26.99VARIABLES ( * indicates observation based variables)

Variable NameTotal

Changes

Average Minutes Per Change

Minimum Value

Maximum Value

Current Value

Average Value

v P1HX* 231 6.36 0 1471 1471 657.63v P2HX* 189 8.64 0 1093 1093 692.64v P3HX* 132 13.9 0 756 756 464.78v RDCostCounter1* 499 2.85 0 498 498 249v RDReview BoardCost* 499 2.85 0 11454 11454 5727


Figure 6 Capsule of the Final Report for the Scenario_2 Run with Relevant Variables Highlighted

Analysis: The Activity labeled Review Board Sort Contained Requirements Development Defects has a time variable ,Q, in Phase 1, as shown in Figure 4. The equivalent variables in Phases 2 & 3 (not shown) are R & S. In the first run (Normal Run),(Figure 5) the values for Q, R, & S were 1 time unit each. In Scenario_2, the values for Q, R, & S were 5 time units each. The cost of each defect was chosen at $23 per defect. Comparison of the two runs shows the cost jumped from $11,454 to $57,270.


The scheduled time (v_P1HX) for the Requirements Development Phase jumped from 1471 time units to 2808 time units. Similar results for the Design Phase with its time variable, v_P2HX (1093 to 2922) and the Code Phase with its time variable, v_P3HX (756 to 1352) were automatically calculated. Note that in all three times, the first phase took longer to complete than the second phase, and similarly for the third phase. The explanation is that the Out of Phase Defects had to be recycled back from each succeeding phase, thus requiring more time for each previous phase[4]. More time means more cost, as was shown earlier. Below is a table for ease of comparison:

2. Automatic pre-programmed scenarios with selected parameter variation enabled multiple, faster runs to yield outcomes.

3. Additional parameters (56 variables) showing number of defects, percent density of defects, defect containment, total KSLOC, total hours consumed, etc. were listed in a longer and detailed comprehensive report (17 pages) automatically generated at the end of a simulation run, for each scenario.

4. Additional bar graphs, pie charts, cost summaries, and plots were automatically generated using this technique.

5. Model parameters were exported to a spreadsheet, and global changes were made as needed in the spreadsheet. Changes were successfully imported directly into the model to create a new scenario to generate new estimates.

6. This technique becomes immediately extendable to appended systems to create a system of systems (SoS) model, by escalating upwards to a higher architectural level.

Live DemoFor this project, a 10-minute live demonstration of process mapping and simulation shows dynamic, graphic animation of the flow of defects from one phase to another. A dashboard during each run displays relevant variables in real time to show performance in defect prediction and estimation capabilities for software development cost and schedule.

Future WorkThis model is generic to many processes in which defects or anomalies can occur, as, e.g., hardware defects in a manufacturing production line. In conjunction with Quality Assurance and Six Sigma practices other processes can be similarly treated such as in business (order process, Help desk), finance (transactions), healthcare (claims processing), aerospace (radar tracking, checklist, countdown, communications, [5] command and control) and shipbuilding (welding, supply chain).


ReferencesRef [1]

Arabian, J.H Computer Integrated Electronics Manufacturing and Testing, Comparative Management Associates, Milton, MA 02186, Marcel Dekker, Inc., 270 Madison Ave, New York, N.Y. 10016 Tel (212) 696 9000.

Ref [2]

Ulrich, E. Concurrent Simulation Associates, Burlington, MA., Agrawal, V, AT&T Bell Laboratories, Murray Hill, New Jersey, and Arabian, J.H., ibid Concurrent And Comparative Discrete Event Simulation, Kluwer Academic Publishers, 101 Philip Drive, Assinippi Park, Norwell, MA 02061, Tel: 617 871 6300

Ref [3]

Lane, J., Valerdi, R., Synthesizing System-of-Systems Concepts, for Use in Cost Estimation, Center for Software Engineering, University of Southern California, 941 W. 37th Place, SAL Room 330, Los Angeles, CA 90089-0781Ref [4]

Boehm, B., Valerdi, R., Lane, J., and Brown, W., COCOMO Suite Methodology and Evolution, CSE, Working Draft, January 2005.Ref [5]

Stouder, R.L., Allgood, G.O, BDA Operational Analysis and FCS Impacts and Stressors, FCS Integrated Support Team (FIST),Oak Ridge National Laboratory, Oak Ridge, Tennessee, April 2003, http://www.ornl.gov/adm/directorates/nsd/pdf/bda_finalreport.pdfRef [6]

Arabian, J.H., User's Requirements for a Dynamic High Speed Tester/Diagnoser (DHSFT/D), Proceedings International Conference and Exhibition 1982, Palais de Congres, C.I.P. Paris, France

Ref [7]

Arabian, J.H., User's Requirements for Automated Handling in Computer Manufacturing and Board Test, Proceedings International Test Conference 1983, Philadelphia, USA, October 1983 pages 226-237, IEEE Computer Society

Ref [8]

Hebert, D. and Arabian, J.H. ,Implications of the Technique for Dynamic High Speed Functional Testing, Proceedings International Test Conference 1982, Philadelphia, PA, USA, November 1982,pages 548-557, ITC, IEEE Computer Society

Ref [9]

Arabian, J.H., Dynamic Thermal Model Simulation, Charles Stark Draper Laboratory, MIT/MS Thesis, May 1970


http://www.ornl.gov/adm/directorates/nsd/pdf/bda_finalreport.pdf

End


process mapping and discrete event simulation for...

Documents