(Draft) Data Collection and Normalization Policy

August 29, 2008

(Updated Software Metrics Section – September 11, 2009)

1

1.1.1 Software (Tabs 8a, 8b, 8c)

The Software Section of the Normalization Template contains both technical software inputs from the contractor (in two standard software data sheets) and software metrics. It has been designed to provide the analyst with useful data in a convenient format while enforcing data normalization consistency and usage. For this reason, the Normalization Template should not be altered. If an idea for improving the template has been identified, submit the proposed change to the appropriate Government Point of Contact for consideration. Do not simply modify the Normalization Template.

Modification is defined as any change made outside the process boundaries defined in the DCNP. However, there are some significant changes that are allowable under the established process. If software is not in the scope of the data collection effort, all three standard software tabs can be deleted from the normalization workbook. Conversely, if software is within the effort scope and there is relevant software information not addressed in the existing tabs, an additional tab in Section 8 (i.e., Tab 8d, 8e, etc.) can be included. Use judgment to determine if it is necessary to include this information in the normalization workbook, or if the information can be linked into the Related Documents section of the data point in SCATTR once the data point is parsed.

Other changes, though seemingly less significant, are considered modifications (outside the process) and are not allowed. Columns in the SW Metrics tab and software data sheets cannot be deleted, reordered, or modified in any way. If a particular column does not apply to the dataset, allow the link (or calculation) to demonstrate that no data has been provided. For direct data input, leave it blank. Do not delete the column. Likewise, do not add columns to accommodate data outside the scope of the software tabs. (Adding a specialized tab is permitted, but modifying standard tabs is not.) Also, do not insert additional rows above the column headings. Such modifications will prevent the tab from parsing into SCATTR and potentially create formula errors elsewhere within the normalization workbook/software tabs.

Inadvertent modifications and errors can be avoided by adopting the following “Best Practices” from the outset of every normalization effort:

First and most importantly, begin by downloading the Normalization Template from SCATTR. Resist the temptation to start with an existing template and “just update”. Starting with a clean template from SCATTR ensures that the most current (and accurate) template is being used. The template has been structured and formatted to ensure proper functioning and successful parsing into SCATTR. Recycling an old template could cause unexpected problems that are not necessarily easy to correct.

Download the Simple Normalization Example from SCATTR for use during normalization. Since the Normalization Template does not contain formulas, the example can serve as the source for many equations. Remember to verify that copied formulas contain the proper cell references, and update them if they do not. As with the template, do not rely on an old (possibly outdated) copy of the Simple Normalization Example. It may reflect old structures (or less obvious formatting differences) that could cause problems.

Get into the habit of using “Paste Special – Formulas” when copying links/equations. This will protect the formatting required for the worksheets to function and display properly.

2

Get into the habit of using “Paste Special – Values” when copying raw data inputs from the contractor’s Standard Data Sheets. This will protect the formatting required for the worksheets to function and display properly.

The following sections drill down into the specifics of each standard software tab. Note that each tab is discussed in the order that it appears in the normalization workbook. However, it generally proves easier to populate the data sheet tabs prior to establishing links with the SW Metrics tab. (If that is the approach you prefer to take, proceed to section 3.6.1.1.2 of this document.)

1.1.1.1 SW Metrics (Tab 8a)

This section of the DCNP will briefly describe the process and metrics formulas used to complete the SW Metrics tab. This is the first tab of the normalization workbook Software Section and serves to consolidate software costs and technical inputs in order to create useful metrics. Refer to the Metrics Definition Appendix C for additional background information on each metric.

1.1.1.1.1 Linking SW Metrics Tab and Final Normalized Costs Tab

All software-related SWBS elements (including SW SEIT/PM) from the Final Normalized Costs tab should be linked into the SW Metrics tab. Be thorough in establishing these links. If your program contains simulation/test code, it will not be carried in the software section of the Final Normalized Costs tab. The NRO CAIG has established the ground rule that cost and hours associated with simulation/test code is mapped to the appropriate System Test and Evaluation (STE) WBS element.

Begin by individually linking each cell in Column A (Standard WBS #) of the SW Metrics tab to the Standard WBS # column in the Final Normalized Costs tab for software-related elements. Be sure to include software related roll-ups (summing items), simulation/test code carried in STE, and software SEIT/PM, if separately identified. At a minimum, include the roll-up of unique WBS items into a standard WBS element. If SEIT/PM is separately reported, be sure to include a roll-up that combines software/CSCI data and SW SEIT/PM for a total. It is possible that SEIT/PM will be reported at multiple levels. If lower-level SEIT/PM costs are available, show (and link) each level reported. Do not combine all SEIT/PM into a single total line.

Next, select and copy the formulas (links) from Column A of the SW Metrics tab and paste into Column B (WBS Description) of the SW Metrics tab. Since these columns are consecutive in the Final Normalized Costs tab, there is no need to update column references. Simple links are all that is required for these columns:

='6. Final Normalized Costs'!A24

Before linking any other cells, use the Standard WBS # and WBS Description information to properly format content within the SW Metrics tab. Indent SWBS numbers and descriptions to properly reflect their relative position within the SWBS structure (parent and child items). Determine software end items (vice roll-ups) and color code them using blue font. Apply these formatting conventions to the remainder of each row. If software SEIT/PM is separately reported, the cells in the associated rows should be shaded grey. Also, enter “N/A” in the cells of each SEIT/PM row for the following columns: C (Language) and I through X (code counts

3

and metrics). These items do not apply to SEIT/PM. If SEIT/PM is not separately reported, Column F (SEIT/PM Factor / Allocation Ratios) will not be used. Go ahead and enter “N/A” into all the cells of this column. Finally, group rows to create an “end item only” viewing option and columns containing metrics to create a “technical data only” viewing option.

Now continue establishing links to the Final Normalized Costs tab. Link each cell in Column D (Unique SWBS Title) of the SW Metrics tab to the appropriate cell in the Unique SWBS Title column of the Final Normalized Costs tab using the following formula structure:

IF('6. Final Normalized Costs'!C24=0," ",'6. Final Normalized Costs'!C24)

Using this IF Function will result in a blank cell, vice a 0 (zero), if no information is available. It will also establish a link that will automatically update if blank cells in the Final Normalized Costs tab are populated later in the process. Once the links are properly established for Column D of the SW Metrics tab, copy the formulas from Column D into the Unique SWBS Description (Primary) column (Column E). Again, this approach can be used since the target columns within the Final Normalized Costs tab are consecutive. Note that these two columns will often be blank.

The last two columns to be linked to the Final Normalized Costs tab are Total Hours in Column G and Total Cost (FY00$K) in Column H. The cells in Column G (Total Hours) of the SW Metrics tab should be linked to the cells in the Final Normalized Costs tab column titled Direct Labor Hours / NR+R. This can be accomplished by individually linking each cell or by copying the formulas from Column A (Standard WBS #) and updating the references to the proper target column in the Final Normalized Costs tab. The same options apply for linking Total Cost (FY00$K) in Column H of the SW Metrics tab to the Cost Thru G&A / NR+R column in the Final Normalized Costs tab. Simple links are all that is required for these columns:

='6. Final Normalized Costs'!M24

Again, a simple “cut and paste” from Column A is not appropriate and will result in linking to the wrong columns and errors since target columns are not consecutive. If links are copied, references to columns in the Final Normalized Costs tab must be updated in these formulas. Each cell can be manually updated with the proper reference. Another option would be to highlight (select) each column, one at a time, and conduct a “Find and Replace” to update the column references.

Perform a final quality review of the links to the Final Normalized Costs tab to ensure no errors have occurred. Remember, all linked cells within a single row of the SW Metrics tab should link to a single row within the Final Normalized Costs tab. Verify the sum of lower-level items by selecting the cell range and noting the total shown at the bottom right of the Excel window. This value should match the total at the roll-up level.

1.1.1.1.2 Linking SW Metrics Tab and SW Data Sheets 1-3 Tab

Contractor inputs for code counts and software language identification should be linked into the SW Metrics tab. The contractor supplies this information via standard software data sheets, which are included in the SW Data Sheets 1-3 tab of the Normalization Workbook. (Some may find it easier to populate the data sheet tab prior to establishing links. See section 3.6.1.2 for

4

more detail.) Keep in mind that there are occasions where code count details are reported at a lower WBS level than present in the Final Normalized Costs tab. This lower-level detail should be linked into the SW Metrics tab even though associated costs are not available. This will require you to enter unique Standard WBS numbers in Column A (Standard WBS #) of the SW Metrics tab using the established naming convention. WBS descriptions for these items can be entered into Column B of the SW Metrics tab by linking to the contractor provided Item ID provided in Column C of the SW Data Sheets 1-3 tab.

Begin by individually linking the lowest level end item cells in Column C (Language) of the SW Metrics tab to the associated cells in Column D (Source Language) in the SW Data Sheets 1-3 tab using a simple link:

='8b. SW Data Sheets 1-3'!D17

Do not merely link one cell and drag the formula down. The language cells in the SW Data Sheets 1-3 tab are not consecutive and errors will occur.

Once the language data is linked, examine the roll-up (summing or parent levels). If all lower-level items (children) are written in the same language, enter that language into the associated parent level in Column C. If multiple languages are present in the children, enter the word “various” in the summing levels.

The next step is to link the contractor’s code count data and input summing formulas. This activity will occur in Columns I through M of the SW Metrics tab. Begin by linking the lower-level end item cells in Column I (New) to the appropriate cells in Column E (Unique SLOC) of the SW Data Sheets 1-3 tab. Since the target cells are not consecutive, do not simply link one cell and drag the formula down. Link each cell individually. Another option would be to copy the formulas from the language column and then update the column references within the formulas. Remember to use “Paste Special – Formulas” if this approach is taken. Once Column I (New) of the SW Metrics tab is properly linked, the formulas can be copied and pasted into the next three columns: J (Autogen), K (Reuse), and L (Deleted). No formula reference updates are required in this case because the target columns in the SW Data Sheets 1-3 tab are consecutive. Again, simple links are all that is needed for these columns:

='8b. SW Data Sheets 1-3'!E17

Once these lower-level end item links are successfully established, it is time to examine roll-up (summing) level rows. Begin by inserting summing equations in Column I (New). Sum lower-level items first and work up from there. Lower-level items are consecutive by row, so the following simple summing formula (using a range) can be applied:

=SUM(I7:I13)

In the Simple Normalization Example, this equation sums the CSCI level code into a total for Spacecraft Bus Flight Custom Software. The next step is to sum lower-level subtotals to the next higher level. In the Simple Normalization Example, there is only one subtotal, so the equation is very simple:

=SUM(I6)

5

However, there may be multiple subtotals involved. In such a case, use the formula structure demonstrated by the notional equation below, keeping in mind that your target cells will be different:

=SUM(I6,I15,I20,I23)

Notice that the formula structure is not a range but a series of specifically selected cells. This is because the higher-level summing activities are not contained within consecutive cells. This construct is easily achieved by using the SUM Function, holding down the CTRL key, and then selecting the desired cells.

Once completed, these summing formulas can be copied into Columns J (Autogen), K (Reuse), and L (Deleted). Remember the summing equations are unique to their row and should only be pasted into cells of the same row. Also, there are no code counts associated with SEIT/PM. Cells associated with the SEIT/PM row, if there is one, should already contain “N/A”, and should not be included in summing equations.

Next move to Column M (Total) and insert the equation for calculating the total delivered code into the first row of the column. Use the following formula to subtract Deleted code from the sum of New, Autogen, and Reuse code:

=SUM(I4:K4)-L4

After that, copy this formula into the other cells of the column that contain data…except SEIT/PM, which should already contain “N/A”.

Perform a final quality review of the links to the SW Data Sheets 1-3 tab to ensure that no errors have occurred. Remember, all linked cells within a single row of the SW Metrics tab should link to a single row within the SW Data Sheets 1-3 tab. Verifying that the sum of lower-level items in the Total column (by selecting the range and noting the total shown at the bottom right of the Excel window) matches the total is one quick way to check that links and equations have been properly implemented.

1.1.1.1.3 Direct Entries – Summary Description and USC Code Counts

The SW Metrics tab reserves cell A1 for a summary description. Enter a brief description with relevant information specific to the program. Be sure to highlight any unusual circumstances or data anomalies that future users of the data should know. (This cell should already be linked with the corresponding cell in the Table of Contents tab; be sure to verify.)

Two code counters are normally run on each software program. The contractor (commonly abbreviated as KTR or CONTR) submits code counts that were run using their own applications, and these are reflected in the Standard Software Data Sheets. The USC Code Counter is a tool used by the NRO CAIG to standardize code counts across contractors. Results of USC Code Counter runs are uploaded and stored within the NRO CAIG Software Database. USC Code Counter results must also be entered into the SW Metrics tab. Contact the Software Database POC for assistance in this area.

If the USC Code Counter has been run against the program being collected and normalized, the results should be obtained from the Software Database and entered into Column R (USC Code

6

Count Total) of the SW Metrics tab. Understand that the USC Code Counter only counts and provides the Total software lines of code. Enter the code counting results in the lower-level end product rows. If there is no code count for a given cell, simply enter 0 (zero). This will present as a centered dashed line: “-”. Once this is accomplished, add summing equations for roll-ups. The summing equations are the same as those used for summing the contractor’s code counts and can be copied from there. Compare the resulting sums with the USC Code Counter results as a quality check against your data inputs. Again, cells associated with a SEIT/PM row should already contain “N/A”. Calculating USC code counts for New, Autogen, Reuse, and Deleted is covered in the next section.

1.1.1.1.4 Inserting Simple Metric/Factor Calculations and Calculations Containing Links

At this point, the SW Metrics tab should contain most, if not all, of the information required to calculate the various factors and metrics encompassed in this tab, including the calculation of USC Code Count Distributions.

USC/KTR % Delta – The USC/KTR % Delta (Column S) is used to define the relationship between the contractor’s code counting tool (and results) and the USC Code Counter (and results). This ratio is used to calculate both the USC Code Count Distributions and ESLOC values, so it is an important calculation.

The ratio divides the difference between the lines of code counted by each tool by the lines of code counted by the contractor’s tool at each SWBS level. There are two formulas used to accomplish this; one for parents (summing levels) and one for the lowest-level children. Begin by entering the following formula structure into the cells associated with lowest level children:

=IF(OR(R7=0,M7=0),"N/A",(R7-M7)/M7)

The formula checks to see if both USC and contractor code counts are available for the item. If there are, the ratio is calculated. If a 0 (zero) value appears in either the contractor or USC Total, “N/A” is automatically entered into the cell. For summing levels (parents) a modification of the formula is used. Note that the range in the “COUNTIF” section of the formula should be set to include all children:

=IF(OR(R6=0,M6=0),"N/A",IF(COUNTIF(S7:S13,"N/A")>=1,"N/A",(R6-M6)/M6))

This formula structure includes an additional value test to avoid errors. If any of the lower-level items (children) contain “N/A”, the parent will automatically be assessed as “N/A”. This is because the ratio generated at the parent level is only valid if comparative code counts are available for all contributing children.

Understanding the USC/KTR ratio will be useful when applying NRO CAIG metrics to future estimates. For example, if Contractor A has a ratio of -20% and submits software data sheets for a new proposal, then the NRO CAIG can decrement their SLOC counts by 20% so that it is normalized against NRO CAIG data.

Calculated USC Code Count Distribution – Once the USC/KTR % Deltas are calculated, the USC Code Count Distributions can be determined. Normalized values for New, Autogen, Reuse, and Deleted code (Columns N – Q) are generated at the end item level by adding 1.0 to the USC/KTR % Delta (in Column S) for a given row and then multiplying this sum by the

7

corresponding contractor code count. This creates estimated USC code count values that are based on the contractor’s distribution of code among the various software categories. It also serves to normalize for differences between the code counting tools. Begin by calculating USC values for New code in Column N of the SW Metrics tab. Perform the calculations for lower-level end items using the following formula structure:

=IF($R7=0,0,I7*(1+$S7))

This formula is structured to verify that USC code counts are available in (Column R) for a given end item. If they are, the normalized USC value is calculated. If not, the formula automatically enters a 0 (zero) into the cell, thereby avoiding errors in the worksheet. It also locks the column references to USC Code Count Total (Column R) and USC/KTR % Delta (Column S). The basic formula is the same regardless of SWBS level, so once the equation is inserted it is an easy matter of copying it into the remainder of the non-summing cells in the column. Of course, the equation would not be entered into any cells associated with a SEIT/PM row.

After generating normalized line of code values for New code, insert the summing equations. New code values should sum in the same manner as USC Code Count Totals, so a simple copy and paste from the summing equations in Column R should be all that is required.

Once the accuracy of Column N (New) values has been confirmed, all the equations can be copied into the following three columns to calculate values for Autogen (Column O), Reuse (Column P), and Deleted (Column Q). Since these columns are consecutive, and the column references to the USC Code Count Total and USC/KTR % Delta within the code count calculation are locked, the formula can be copied across columns and down rows without creating reference errors. Copying the summing formulas is also a simple matter given that all the reference cells are consecutive. Even so, always double check cell references and cross check totals. At this stage, perform a quality check by calculating total values using the same formula structure used to calculate totals for the contractor in Column M (New + Autogen + Reuse - Deleted). This should yield results that match the summed code count values you entered into the USC Code Count Total (Column R) of the tab.

DSLOC and ESLOC Code Counts – If available, USC code counts are used as the DSLOC values. If not, the contractor’s DSLOC values are used. To accomplish this, insert the following formula into the DSLOC column (Column T) of the tab for the lower-level end items:

=IF(R7>0,R7,M7)

This formula tests to see if USC code counts are available and uses those values if they are. Otherwise, the contractor’s code count value is inserted in the cell. The basic formula is the same for all non-summing WBS items, so once the equation is inserted into a cell it is an easy matter of copying it into the remainder of the non-summing cells in the column. If SEIT/PM is separately identified, the cells associated with a SEIT/PM row should already contain “N/A”.

After populating the end item DSLOC values, insert the summing equations. DSLOC values should sum in the same manner as USC Code Count Totals, so a simple copy and paste from the summing equations in Column R should be all that is required.

The ESLOC calculation is a bit more complicated in that it draws from data contained in the SW Data Sheets 1-3 tab. Further, the calculation uses the USC/KTR % Delta to normalize the

8

contractor ESLOC values for code counting tool differences. Use the following formula structure to enter the equations into the ESLOC column (Column U) for lower-level end items, or copy the formula structure from the Simple Example:

=IF(S7="N/A",'8b. SW Data Sheets 1-3'!L17,'8b. SW Data Sheets 1-3'!L17*(1+S7))

Exercise extreme care in establishing reference cells. Do not merely link one cell and drag the formula down. The target ESLOC cells in the SW Data Sheets 1-3 tab are not consecutive and errors will occur. A simple copy and paste of formulas within the ESLOC column is not appropriate.

Once the end item ESLOC values are populated, insert the summing equations for higher-level elements. ESLOC values should sum in the same manner as DSLOC Totals. In this case, a simple copy and paste from the summing equations in Column T is acceptable and can be utilized. Again, cells associated with a SEIT/PM row should already contain “N/A”.

Take a moment to examine your results for reasonableness and accuracy. Remember, ESLOC should be equal to DSLOC if it is all new code. Otherwise, it should be less than DSLOC. For further information on DSLOC and ESLOC, refer to the detailed descriptions included in the Metrics Definition Appendix C.

SEIT/PM Factor / Allocation Ratios – If SEIT/PM is separately reported, Column F is used to calculate both SEIT/PM Factor(s) and Allocation Ratios. If it is not, then this column should already contain “N/A”. Move on to the Productivity Metrics section below if SEIT/PM is not separately reported on the program.

The SEIT/PM Factor is a simple ratio of the SEIT/PM cost to the unloaded software cost shown in Column H. SEIT/PM factors are calculated for each SEIT/PM row (shaded grey) and entered in the associated cell within Column F. Calculate the factor using this straightforward formula structure:

=H5/H6

In the Simple Example, SEIT/PM cost is reported on row 5 (SWBS 1.2a.2.8.1) of Column H. This value is divided by the unloaded custom software cost, reported on row 6 (SWBS 1.2a.2.8.2) of the same column.

Allocation Ratios are calculated for all non-SEIT/PM rows and demonstrate the proportional distribution of total SEIT/PM (hours or dollars) to lower-level items. The standard basis for allocating SEIT/PM is hours, so the data in Column G (Total Hours) will be used to develop the ratios. Apply the following formula structure to enter the equation into Column F, or copy the formula structure from the Simple Example:

=IF((G4/$G$6)>1,0,G4/$G$6)

This equation divides the hours associated with a given non-SEIT/PM item by the total unloaded hours for software. It then assesses if the result is greater than one. If it is, zero is entered in the cell. This portion of the formula deals with summing level items that already contain SEIT/PM. If the result is equal to or less than one, the ratio is calculated and entered into the cell. Make

9

sure to anchor the cell associated with the total unloaded software hours so that you can easily copy the formula into the remaining, non-SEIT/PM cells …which is the next step.

As the Allocation Ratio formula is copied, make sure to operate only within a single WBS section. In the Simple Example, cell G6 (Spacecraft Bus Flight Custom Software) is the anchored cell and relates to the Spacecraft Bus Flight Software section of the SWBS. Notice all listed items fall within SWBS 1.2a.2.8. However, if software from another SWBS section were included, for example, SWBS 1.3a.6.3 (Ground Mission Data Processing Software), the formula anchor cell would need to be updated to lock on the cell containing the total unloaded hours for the ground software. The message here is treat separate software/SWBS areas separately.

Do a quick quality check to ensure the equation has been properly applied and that anchor cells are correct if multiple SWBS sections are represented. Allocation Ratios within a single SWBS section should total to 100%. If they don’t, there is an error to correct.

Understanding the information provided in the SEIT/PM Factor / Allocation Ratios column will be useful when developing estimates or tailoring metrics. The SEIT/PM Factor appears on the SEIT/PM row (SWBS 1.2a.2.8 in the Simple Example). The Factor should only be used against the unloaded software subtotal cost in dollars:

Factor: (for use against dollars only)

SEIT/PM Cost (dollars) = (factor * unloaded software subtotal $)Loaded Total Software Cost (dollars) = (1+ factor) * unloaded software subtotal $

Allocation Ratios can be applied against hours or dollars. Not as commonly needed as the Factor, Allocation Ratios allow the analyst to isolate lower-level items for use in detailed estimating or CER tailoring circumstances. The top-level total (SWBS 1.2a.2.8 in the Simple Example) already contains SEIT/PM; hence, the Allocation Ratio of 0% means no SEIT/PM needs to be allocated to the cost or hours numbers shown. The unloaded software totals (SWBS 1.2a.2.8.2 in the Simple Example) has an Allocation Ratio of 100%. This means that the SEIT/PM total should be added to create a loaded total…or the analyst can merely refer to the summing line (SWBS 1.2a.2.8). For lower-level items, the Allocation Ratio is applied against the SEIT/PM total (cost or hours) to determine the amount of SEIT/PM that needs to be added to the unloaded figure to determine the loaded total:

Allocation Ratios: (for use against hours or dollars)

Loaded Software Item = (Allocation Ratio * Total SEIT/PM) + unloaded hours or dollars

Remember: Factors appear on SEIT/PM rows; Allocation Ratios appear on non-SEIT/PM rows.

Productivity Metrics – Several useful productivity metrics are calculated within the SW Metrics tab. They include $/ESLOC (Column V), ESLOC/hr (Column W), and Hrs/ESLOC (Column X). The metrics are used often for software estimates when performing analogy based estimates or for cross checks. Each of the metrics is a simple ratio of the corresponding columns in the SW Metrics tab itself. To avoid “#DIV/0!” and “#VALUE!” errors, an “IF Function” structure is used.

Before generating productivity metrics, the analyst must identify the scope of the costs and hours used in these calculations. Determine if costs within a given activity include subcontractor costs. If this is the case, there may be a scope discrepancy between total costs and total hours. Typically, the effort (total hours) reflects only prime contractor labor whereas the cost (total

10

cost) reflects prime contractor, materials, and subcontractor costs. If possible, the analyst should query the prime to determine if subcontractor labor hours are reported (included) or not. If only prime hours are reported, insert a comment in the associated cells in Columns W (ESLOC/hr) and X (Hrs/ESLOC) stating that these metrics are calculated against prime hours only and do not reflect subcontractor hours worked against the item. Also, indicate that some activities contain subcontractor costs but not the associated hours in the Tab Summary Cell (A1).

The metric for $/ESLOC is the simple ratio of cost in dollars to effective lines of code. Since costs are carried in thousands, the cost column must be multiplied by 1000 prior to dividing by ESLOC. The following formula should be used to calculate the $/ESLOC metric:

=IF(OR(H4=0,U4=0),"N/A",(1000*H4)/U4)

Notice that this IF Function checks for 0 (zero) values in both the cost and ESLOC cells. If detected, an “N/A” is entered into the cell. Using this structure will not only avoid the errors mentioned above, it will establish a calculation that will automatically update if data becomes available later in the process. This formula is the same for all items (except SEIT/PM), so once entered, it can be easily copied into the others cells. This is true for all the metric calculations.

The metric for ESLOC/hr is the simple ratio of effective lines of code to cost in hours. No unit adjustment is required for this ratio. The following formula should be used to calculate ESLOC/hr:

=IF(OR(G4=0,U4=0),"N/A",U4/G4)

The formula used to calculate the Hrs/ESLOC metric is almost the same as the formula presented above. However, this productivity metric is the ratio of cost in hours to effective lines of code:

=IF(OR(G4=0,U4=0),"N/A",G4/U4)

Remember, for all productivity metrics (within a single column), the formula is the same regardless of SWBS level, so once the equation is inserted it is an easy matter of copying it into the remainder of the cells in the column, except for SEIT/PM rows where it does not apply.

For further information on these software metrics, refer to the detailed descriptions included in the Metrics Definition Appendix C.

1.1.1.2 SW Data Sheets 1-3 (Tab 8b)

The SW Data Sheets 1-3 tab is used to capture the contractor’s technical software inputs. It is designed to mirror the Standard Data Sheets used to collect data and contains information such as contractor code counts and complexity attribute ratings. Refer to the Standard Data Sheets link under the Data Collect section of SCATTR for a detailed description of how the Software Standard Data Sheets are populated.

Begin by entering a summary description in cell A1. Recall from the SW Metrics tab instructions, this cell is reserved for a summary description and should already be linked to the corresponding cell in the Table of Contents tab. (Take the time to verify this.) Enter a brief description of information relevant to the data contained in this tab including any unusual circumstances or data anomalies that future users of the data should know.

11

Enter the contractor data from the Standard Data Sheets. These inputs can be hand entered, or they can be copied if electronic copies are available. If the data is copied, make sure to use the “Paste Special – Values” approach to preserve the necessary formatting in the SW Data Sheets 1-3 tab (and to avoid inadvertently inserting unwanted links into the Normalization Workbook). Expand the number of lines in the SW Data Sheets 1-3 tab, as necessary, to show all of the software end items. Note that all software end items should be listed on one tab, even if the contractor has submitted separate sheets for different software functions. More on this later.

As you enter the contractor’s data, you will notice some differences between the SW Data Sheets 1-3 tab and the Standard Data Sheets. First, the SW Data Sheet 1 in the tab has a column that does not appear in Standard Data Sheet 1. This is Column L (ESLOC), which is used to calculate ESLOC using contractor code count inputs. (Recall the cells in this column are linked into the SW Metrics tab through the formula used to calculate the USC normalized ESLOC count in Column U.) Apply the following formula structure to enter the equation into Column L, or copy the formula structure from the Simple Example:

=E17+(F17*$S$43)+(G17-H17)*(((I17/100)*$S$40)+((J17/100)*$S$41)+((K17/100)*$S$42))

This formula performs multiple operations. It sums code counts, subtracts deleted code, multiplies net pre-existing code by the amount of rework required, and applies the appropriate software category weightings to calculate contractor ESLOC values. As indicated in the formula, be sure to divide the contractor inputs for Redesign, Reimplement, and Retest by 100 to convert the values into percent for the calculation. The formula is the same for all items so once the equation is inserted, it can be copied into the other necessary rows. However, note that there are several anchored cells in this equation. Before copying the equation into other rows, make sure the cells are properly anchored to the ESLOC calculation weighting. This table is a second difference between this tab and Standard Data Sheet 1.

A small table that contains ESLOC calculation weightings is included in the SW Data Sheets 1-3 tab. In the Simple Example, this table starts in cell R39. (If you inserted additional rows to accommodate your data, then this table will be lower down the page.) The table contains the standard weightings that the NRO CAIG uses in the calculation of ESLOC. The values are carried in this simple table for both easy reference and in the event that the methodology changes overtime. If this happens, the update only needs to be applied here and it will automatically ripple through the calculations.

If the contractor has submitted software inputs on a single data sheet, the next two items will not apply to you. However, as suggested earlier, contractors will not always provide code count data in a single data sheet. In fact, many times separate sheets are prepared for different software subsystems and prepared by different individuals. The SW Data Sheets 1-3 tab has features to deal with this information, which would normally appear in the Header portion of the sheet. (Reference the Software Normalization Template for this discussion.) Begin by expanding Column A. This will reveal cells for entering Preparer IDs and subsystem association information. The title cell in the column is already filled in and links to a legend at the bottom of the sheet. Assign ID numbers by preparer. Next enter the segment association (Flt, Grd, STE…) and short name for the software subsystem. Next proceed to the legend and enter this information along with version, date, subsystem long name, and preparer’s phone number. All

12

this information is taken from the Header section of the contractor’s inputs. The ID cells in Column A would look something like this:

Prep ID and subsystem

1 - STEABC Sim

The associated legend would look something like this:

Prep ID LegendPrep ID Version Date Segment

1 1 15-Apr-05 STE A Big Code Simulator 888-4321Les Paul 812-555-1234Subsystem open phone secure phonePreparer

In the Header section of the tab input the overall software system being described in the merged cell named “SYSTEM”. In the “PREPARER NAME” merged cell, enter the phrase “multiple – see legend below”. Next enter “see legend” for all other Header items. This only needs to be done in the Header for data sheet 1 since the cells are linked to the Header cells in data sheets 2 and 3.

Before leaving this tab, perform a quick quality check. Verify that ESLOC values make sense given the data. Just as in the SW Metrics tab, ESLOC should be equal to DSLOC if it is all new code. Otherwise, it should be less than DSLOC. For further information on ESLOC calculation, refer to the detailed descriptions included in the Metrics Definition Appendix C.

1.1.1.3 SW Data Sheet 4 (Tab 8c)

The SW Data Sheet 4 tab is used to capture additional technical and descriptive data from the contractor. It is designed to mirror the Standard Data Sheets used to collect data and contains information such as contractor assessment of total delivered and ESLOC code, contractor labor rates, anticipated code growth, and other developmental factors. Refer to the Standard Data Sheets link under the Data Collect section of SCATTR for a detailed description of how the Software Standard Data Sheets are populated.

Just like the other tabs in the software section, the SW Data Sheet 4 tab reserves cell A1 for a summary description and should already be linked to the corresponding cell in the Table of Contents tab. (Take the time to verify this.) Enter a brief description of information relevant to the data contained in this tab including any unusual circumstances or data anomalies that future users of the data should know.

Enter all data provided by the contractor data from Standard Data Sheet 4. These inputs can be hand entered, or they can be copied if electronic copies are available. If the data is copied, make sure to use the “Paste Special – Values” approach to preserve the necessary formatting in the SW Data Sheet 4 tab (and to avoid inadvertently inserting unwanted links into the Normalization Workbook). The tab is intended to include the contractor’s inputs as they were provided. If the analyst needs to add clarifying statements, use brackets [ ] around the added information and change color the font brown. This will make it clear what is from the contractor and what has been added by the analyst.

13

If multiple sheets are provided, use an ID approach similar to that described in the SW Data Sheets 1-3 tab section. In this case, a special column is not provided. Instead enter ID information in brackets in front of the contractor text and use a legend at the bottom of the sheet. In the Header section enter the phrase “see legend” as was done in the SW Data Sheets 1-3 tab. One difference is establishing a link to the legend. Since there is no dedicated column for this in the SW Data Sheet 4 tab, use the text in the “Preparer Name:” to create the hyperlink. The Normalize Template has a legend at the bottom of the SW Data Sheet 4 tab. It can be deleted or ignored if it is not needed.

Once all the contractor data is entered, perform a quick quality review. Are the ESLOC and DSLOC total values consistent with the inputs reported in SW Data Sheets 1-3 tab? If not, does the contractor indicate calculation methods different from the NRO CAIG standard? This may account for the difference. Insert comments into the affected cells to document your findings, if necessary.

1.1.1.4 Software Debugging Tips/Checklist

Begin by downloading a Normalization Template from SCATTR to avoid working in an outdated format and perpetuating errors that may exist in old Normalization Workbooks.

Columns in all three software tabs cannot be deleted, reordered, or modified in any way.

Rows may be inserted to accommodate additional data, but never insert rows over the header information. This will cause parsing errors in SCATTR.

Formulas may be copied from the Simple Normalization Example, but remember to use “Paste Special – Formulas” to protect formatting and avoid inadvertent links.

Raw data from the contractor may be copied from the Standard Data Sheets if electronic copies are available, but remember to use “Paste Special – Values” to protect formatting and avoid inadvertent links.

Perform quality checks as you go. It is easy to recognize, isolate, and correct errors when the work you performed is fresh in your mind.

No document can address every contingency. Unusual situations and issues unique to a program can, and do, occur. If you are not sure how to handle a specific issue…ask

1.2 Software Metrics and Definitions This section describes software metrics and definitions. They are appropriate for all types of software whether spacecraft flight, ground segment operational, simulation, or test support software.

1.2.1 Software Size

Software size can be defined for a Computer Software Configuration Item (CSCI) or at lower levels, such as a Computer Software Component (CSC). Both CSCIs and CSCs are frequently referred to as software end items. The NRO CAIG standard for counting code is the USC Code Counter. Code count inputs provided by contractors reflect a variety of code counting tools. Some may use the USC Code Counter, but other counters are often employed including those

14

developed by the contractor themselves. The code counting normalization process is discussed in section 3.6.1.1.4, “Inserting Simple Metric/Factor Calculations and Calculations Containing Links”, of this document.

The two key size metrics for software CSCs and CSCIs are delivered source lines of code (DSLOC) and effective source lines of code (ESLOC). Note that all software size metrics and data sheet code count values are measured in single units. DSLOC is calculated using the following formula:

DSLOC = Unique SLOC + Autogen SLOC + (Pre-existing Total SLOC – Pre-existing Deleted SLOC)Inputs: Unique SLOC – completely new delivered source lines of code Autogen SLOC – tool generated new lines of code Pre-Existing Total SLOC – pre-existing source lines of code identified for reuse on

development Pre-Existing Deleted SLOC – pre-existing source lines of code deleted during development

ESLOC measures size in terms of the relative effort (defined as effective lines) required to design, develop, code, and test developed software. New code is the standard for effort required and is weighted at 1 (one). Other types of code (tool generated code and pre-existing code) are weighted less than 1 to indicate the reduced effort required as compared to developing new code. ESLOC is calculated using the following formula:

ESLOC = Unique SLOC + (Autogen SLOC * 0.25) + (Pre-existing Total SLOC –Pre-existing Deleted SLOC) * (%RD*0.40 + %RI*0.25 + %RT * 0.35)

Inputs: Unique SLOC – completely new delivered source lines of code Autogen SLOC – tool generated new lines of code Pre-Existing Total SLOC – pre-existing source lines of code identified for reuse on

development Pre-Existing Deleted SLOC – pre-existing source lines of code deleted during development %RD (percent redesign) – This is the portion of the pre-existing software that requires

redesign, reverse engineering, redocumentation, and revalidation to work the new design into the preexisting software. It also reflects effort required to delete unnecessary code.

%RI (percent reimplementation) – portion of the pre-existing code that requires reimplementation (coding and unit test) to make it functional within the software item.

%RT (percent re-test) – effort required to test the preexisting software, expressed as a portion of the effort, that would have been required had the software been entirely new. %RT reflects re-use of existing test procedures.

Higher-level software size totals encompassing multiple CSCIs or multiple CSCs are obtained by simply summing lower-level values of DSLOC and ESLOC.

1.2.2 Software Productivity Metrics

Software productivity can be measured in several ways. These measures, whether assessed at summary or CSCI/CSC levels, are simple ratios against labor hours and/or cost dollars. This section describes several software productivity metrics typically used at the NRO CAIG.

15

The most common measure of software productivity is based on cost. This metric is expressed in units of dollars per effective line of source code. It is a simple ratio, but be sure to verify the units represented by cost figures. Since costs are often carried in thousands, cost values may need to be multiplied by 1000 prior dividing by ESLOC.

Cost Based Software Productivity = (Total Cost*1000)/ESLOC

Inputs: Total Cost – total cost of all software development activities ESLOC – size reflecting relative effort required to design, develop, code, and test software

Labor based software productivities are a simple ratios using labor, instead of dollars, in the metric. One common approach ratios the number of effective source lines of code to development labor hours. It is calculated using the following formula:

Labor Based Software Productivity 1 = ESLOC/(Total Hrs)

Inputs: ESLOC – size reflecting relative effort required to design, develop, code, and test software Total Hrs – total number of staff hours devoted to all facets of the software development

Total Hours includes activities such as requirements definition, algorithm development, software systems engineering design, implementation (coding and unit test), and CSC / CSCI integration and testing (I&T), support for higher-level I&T, software quality assurance, and software configuration management and documentation.

Care must be taken when comparing productivity values from different sources to ensure that each value includes the full scope as described above. When the scope excludes one or more of these tasks, the excluded tasks should be clearly described when the metric value is stated.

Effort related to hardware development, production and acquisition are excluded from software productivity. This includes the costs of hardware upon which the software is developed. Similarly, the effort devoted to creating and maintaining software development facilities are excluded from software productivity.

It is also common to describe software productivity in terms of the average amount of effort required to generate each effective line of code. The calculation is similar to the one demonstrated above and merely inverts the numerator and denominator. It is calculated using the following formula:

Labor Based Software Productivity 2 = (Total Hrs)/ESLOC

Inputs: Total Hrs – total number of staff hours devoted to all facets of the software development ESLOC – size reflecting relative effort required to design, develop, code, and test software

An alternate form for expressing productivity is on a monthly basis:

Labor Based Software Productivity 3 = ESLOC/PM

Inputs:

16

ESLOC – measure of size in terms of effort required to design, develop, code, and test software

PM – total number of equivalent person-months devoted to all facets of the software development

An equivalent person month is usually defined as the average number of direct hours per month for the developing organization. It is normally calculated from the average number of direct hours per year divided by 12. Keep in mind that the number of hours per equivalent per month can vary between contractors. A default value of 152 direct hours per month may be used, when insight into the number of hours per month is not available.

17