tabular cost analysis requirements description (card

CARD Sufficiency Review Handbook

Tabular Cost Analysis Requirements Description (CARD)

Sufficiency Review Handbook: Completion Requirements by Acquisition Milestone

Jeff McDowell and Marge Evriviades 3 February 2017

Prepared for Air Force Cost Analysis Agency


Use or disclosure of data contained on this page is subject to Page 1 the restriction on the title page of this document.

TABLE OF CONTENTS

1: Introduction ............................................................................................................................. 2 2: CARD Tables .......................................................................................................................... 3 3: CARD Temporal Expectations Overview .............................................................................. 4 4: Detailed Expectations and Guidance ...................................................................................... 5

4.1 Tabular CARD Completion Expectations ........................................................................ 5 4.1.1 Milestone A ............................................................................................................... 6

4.1.2 Milestone B ............................................................................................................... 7

4.1.3 Milestone C ............................................................................................................... 7

4.1.4 Post Milestone C And Annual Updates .................................................................... 7

4.2 Technical Parameter Expectations by Key Milestone...................................................... 7 5: Tabular CARD Features to Enable Full System Description ............................................... 10

5.1 Alternate Design Solutions............................................................................................. 10 5.2 Pedigree and Variability of ValueS ................................................................................ 10

5.2.1 Estimate or Actual................................................................................................... 11

5.2.2 Ranges ..................................................................................................................... 11

5.2.3 Source ..................................................................................................................... 12

5.3 Unknowns, TBDs and N/As ........................................................................................... 12 5.4 CARD Tailoring ............................................................................................................. 12

6: Security Considerations ........................................................................................................ 13 7: Resources .............................................................................................................................. 14

7.1 CARD Tailoring ............................................................................................................. 14 7.2 CARD Checklist ............................................................................................................. 14

8: CARD Insufficiency ............................................................................................................. 16 9: Summary and Conclusion ..................................................................................................... 16

TABLES Table 1: Key Data Sources ........................................................................................................... 15

FIGURES Figure 1: List of CARD Tables by Category .................................................................................. 3 Figure 2: Overview of Progressive Tabular CARD Completion .................................................... 4 Figure 3: Tabular Card Completion Expectatons by Key Milestone .............................................. 5 Figure 4: CEM Templates support CARD Completion assessment (Example ONly) ................... 9 Figure 5: Example With Multiple Reference Designs .................................................................. 10 Figure 6: CARD Tables Contain Pedigree And Variability Columns .......................................... 11 Figure 7: Estimate or Actual CARD Table Columns ................................................................... 11



1: INTRODUCTION

The DoD cost community—including Cost Assessment and Program Evaluation (CAPE) and the Service Cost Centers, to include the Air Force Cost Analysis Agency (AFCAA) have committed to improving the efficiency and effectiveness of our cost estimating capability. The CAPE has launched several initiatives to systematically and continuously create, update, and modernize cost estimating data, methods and tools. The initiatives are not only aimed at improving the overall cost community but also immediately, positively affecting the programs they support. Specifically, the CAPE has released new guidelines for the preparation and maintenance of a tabular Cost Analysis Requirements Description (CARD). The CARD Tables by specific commodity address several objectives.

• The Tables contain key programmatic and technical data required to estimate costs at a sufficient level of detail to support program acquisition reviews (e.g. Milestone Reviews) or PPBE process reviews (e.g. POM submission reviews).

• Over time, the completed Tables serve as a record of program evolution. • The Tables are designed to support future automation via a database that will be used

for cost estimating, analysis and research. The role of the CARD sufficiency reviewer is to ensure these stated objectives are met to the fullest extent at each CARD submission. The CARD reviewer needs not only to focus on the near term cost analysis needs, but also understand and propagate long term analytical opportunities that populated CARDs will present. Given that it is the responsibility of the CARD reviewer to assess the sufficiency of a tabular CARD in terms of the above objectives, this Handbook identifies technical population expectations by key acquisition Milestone. Generally, the reviewer should expect the pedigree of the data provided by the Program Management Office (PMO) to be commensurate to the acquisition stage or upcoming program Milestone. Additionally, the reviewer needs to ensure that CARD features, which provide flexibility and facilitate the ability to fully describe a system even at its earliest program stages, are exercised to the fullest. This Handbook provides the CARD reviewer with the tools to assess and advocate for the quality and completeness of the data by delineating key data sources and recommended best practices.



2: CARD TABLES The CARD Tables describe the key technical, programmatic, and operational characteristics of an acquisition program using concise tabular entries. The tabular CARD includes the following Tables by category:

FIGURE 1: LIST OF CARD TABLES BY CATEGORY

Tables to Present Cost Drivers by Work Breakdown Structure (WBS) Element

Prime Mission Product (PMP) Hardware Technical Table Software Development Table Nonhardware Technical Table

Operations and Support (O&S) Table Software Maintenance Table

Tables to Describe the Program’s Quantities

Quantities and O&S Time Phased Table Configuration Table

Manpower Time Phased Table

Tables to Describe Material Components Line Reparable Units (LRU) Table

Parts Table Government Furnished Equipment (GFE) Table

Tables to Describe Program Context

Program Table Milestones Table Acquisition Table

Roles Table Budget Plan Table

Metadata Table WBS/Cost Reporting Structure (CRS) Definitions Table

Of the 18 CARD Tables, 15 are common to all commodities (e.g. Program, Milestones, Roles, Budget Plan, Quantities, Manpower, etc.). The Commodity specific Tables include PMP Hardware Technical, Non-Hardware Technical, and Configuration Tables. The completed Tables provide a thorough description of the acquisition to include key cost drivers and the detailed execution plan that enables not only immediate cost analysis requirements but also future analytics. It is the responsibility of the CARD sufficiency reviewer to ensure the full spectrum of system and program attributes are accurately and completely recorded across all Tables.



3: CARD TEMPORAL EXPECTATIONS OVERVIEW In the role of CARD reviewer, it is important to first acknowledge that the CARD is a living document that will evolve over time. Understandably, program maturity will drive the granularity of data available (and expected) for a given tabular CARD. Figure 1 illustrates an overview of the general progressive population expectations for the tabular CARD by Milestone.

FIGURE 2: OVERVIEW OF PROGRESSIVE TABULAR CARD COMPLETION

At each Milestone, expect that Government requirements, as sourced from the Capability Development Document (CDD) Key Performance Parameters (KPP), Technical Requirements Documents (TRD), or Statement of Objectives (SOO) will be available for CARD population. As program definition matures from Milestone A to Milestone C and from a notional Government technical baseline to a specific Contractor design solution, so too should the pedigree of CARD technical input parameters. Expect CARD values to evolve from Government engineering estimates to Contractor estimated values to the final measured and tested actual values. As the following sections will describe, the Tabular CARD construct also provides ample opportunities and approaches to facilitate complete population throughout a program’s life. As a CARD sufficiency reviewer, the cost analyst needs to be aware of the features, approaches and techniques that will enable thorough population of the Tables over the life of the program.



4: DETAILED EXPECTATIONS AND GUIDANCE

4.1 TABULAR CARD COMPLETION EXPECTATIONS The CARD reviewer should first understand that program maturity will drive the granularity of data available (and expected) for a given tabular CARD. During the early stages of a program, CARD population focus should be on the highest-priority, core cost-driving parameters - even if notional or estimated, while subsequent CARD updates will describe in further detail the system being procured with actual values.

To provide consistent guidance across all commodities and amongst all CARD reviewers, Figure 3 identifies completion expectations for each CARD Table by key program event. From Milestone A through Technology Maturation and Risk Reduction, only the highest level technical and programmatic parameters are expected. However, both Preliminary and Critical Design Reviews should provide more detailed information on subsystem and LRU levels to populate the CARD Tables.

FIGURE 3: TABULAR CARD COMPLETION EXPECTATONS BY KEY MILESTONE



While CARD updates are not generally required for or aligned to each of those events, CARD sufficiency reviewers should be alert to ensuring that each CARD submission reflects the design solution shown at the most recent engineering event. The following sections provide some additional guidance for CARD sufficiency reviewers on the expected degree of tabular CARD completion by Milestone.

4.1.1 MILESTONE A

Acquisition programs at Milestone A generally have a lack of detailed program definition and data, precluding an in-depth completion of the Tables. However, at MS A, program requirements (e.g. Weight Constraints, Environmental Factors, Bandwidth and Range) and the Government Technical Baseline parameter values should be available. It is acknowledged that the parameters available at MS A will be at a higher system level, most likely without detailed information at the subsystem or Line Reparable Unit (LRU) levels. However, even at MS A, data may be available from either an analogous system(s) or from engineer-provided inputs. It might be possible to identify COTS vs development solutions at the LRU level or software sizing at the Configuration Item (CI) level.

Material Solutions Analysis

O&S

ASR MS A SRR SFR PDR MS B CDR SVR FCA PRR MS C MS C PCA IOC FOC AnnualCARD Table Name

Alternative Systems Review

System Requirements

Review

System Functional

Review

Preliminary Design Review

Critical Design Review

System Verification

Review

Functional Configuration

Audit

Production Readiness

Review LRIP Decision FRP Decision

Physical Configuration

Audit

Tables to Present Cost Drivers by WBS Element

PMP Hardware Technical Table

Software Development Table

Nonhardware Technical Table

O&S Table

Software Maintenance Table

Tables to Describe the Program’s QuantitiesAcquisition Quantities, Deliveries, and O&S Time Phased Table

Configuration Table

Manpower Time Phased

Tables to Describe Details of the Program

LRU Table

Parts Table

GFE Table


Program Table

Milestone Table

Acquisition Table

Roles Table

Budget Plan Table

WBS/CRS Definitions Table

Note: This table is notional. Expectations of data granularity varies considerably by commodity and program type (NDI versus Developmental).

Technology Maturation and Risk Reduction

Engineering and ManufacturingDevelopment Production and Deployment

A few of this table's parameters are expected at this event. But acknowledge a large number of n/a's are expected.

A fair portion of this table's parameters are expected at this event. Substantially all of this table's parameters are expected at this event.

The majority of this table's parameters are expected at this event. Little-to-no expectations of data for this table at this event.



Given gaps and uncertainty within a new program, the CARD Tables can be populated with well researched, estimated values. The CARD can accommodate multiple design solutions and convey multiple points of view. The CARD sufficiency reviewer should encourage the program team to populate the CARD with the best data available from alternate sources and using ranges of values to mitigate any gaps for cost estimating purposes.

4.1.2 MILESTONE B At MS B and post CDR, many of the notional inputs and assumptions from MS A can be replaced or expanded with more complete design specific inputs based on the selected contractor’s technical solution. Expect the CARD to be updated with the contractor’s unique design values including re-use plan for hardware and software, Make versus Buy determinations, and subcontracting strategies. Other programmatic artifacts such as updated schedules, a risk register, organization charts, and the Life Cycle Management Plan should be available by MS B and provide key additional inputs to the CARD Tables. Note, that in all program phases the CARD should reflect the Program Management team’s best judgment of program description and parameter values. It is the responsibility of the CARD sufficiency reviewer to clearly understand the sources of CARD values and to ensure they are properly characterized as contractor proposal, engineering estimates or measurable actuals.

4.1.3 MILESTONE C With full test results available and development complete, the CARD sufficiency reviewer should expect the PMP Hardware Technical Table to be fully populated with actual values at MS C. Although the system is not final until it is fielded, at MS C the program possesses near perfect information to fully describe the program in all the CARD Tables. Each cell of each Table should be regarded as a must and as an opportunity to describe the program to the fullest.

4.1.4 POST MILESTONE C AND ANNUAL UPDATES

Post Milestone C, CARD Tables must support Full Rate Production (FRP), Initial Operational Capability (IOC), Operations and Support and annual funding decisions. Expect annual CARD updates post System Delivery to include more detailed and accurate information for Tables supporting software/hardware maintenance, Mean Time Between Failure (MTBF), recurring annual activities, scheduled Depot events, and other O&S information. Note that each subsequent CARD should contain an increased amount of detail as well as additional populated technical parameter fields. 4.2 TECHNICAL PARAMETER EXPECTATIONS BY KEY MILESTONE

To further support a comprehensive evaluation of the completeness of CARD Tables, the AFCAA Cost Estimating Modeling (CEM) Data Collection Templates contain columns not included in the CARD Tables as displayed in Figure 4.



First, the CEM Templates convey CARD completion expectations. Each unique technical parameter has been assigned a temporal rating for completion by key program Milestone. These temporal ratings can be found in the far right-hand columns (AX – BB) of each commodity Prime Mission Product (PMP) Hardware Technical and Nonhardware Technical Table. The CARD reviewer can compare program office CARD inputs to these population expectations to ensure the CARD is completed to the fullest extent at each submission. Secondly, the CEM Templates identify the authoritative source for the final actuals of each unique commodity specific technical parameter. The authoritative source has been categorized as either a Government or Contractor input and is specified in Column BD. The CARD reviewer should expect in the early stages of the program Government estimated values. However, as the program matures, expect notional values to be replaced with contractor measured and tested values. Finally, each parameter in the CEM Template has been given a priority rating for the Development and Production phases (Columns BF-BG). While the priority columns are intended to facilitate future contractor technical data reporting, these priority ratings can also inform CARD Table population requirements. While all population of all relevant parameters is the ultimate objective, suggest that the CARD reviewer focus on the Priority 1 parameters in the earliest program phases, and encourage population of Priority 2 parameters, as available. These additional source and priority ratings can be found in the far right-hand columns of each commodity Prime Mission Product (PMP) Hardware Technical and Nonhardware Technical Template. Figure 4 provides an example of these CEM Template rating extensions. Check with your AFCAA Technical Advisor to locate and use the CEM Templates as an aid in assessing the CARD’s completeness.



FIGURE 4: CEM TEMPLATES SUPPORT CARD COMPLETION ASSESSMENT (EXAMPLE ONLY)

Note that the format for CARD completion expectations in the CEM Templates is being revised. The proposed reformatting is presented and discussed in Attachment B.



5: TABULAR CARD FEATURES TO ENABLE FULL SYSTEM DESCRIPTION

To facilitate a complete system description as well as to annotate the maturity of inputs, the CARD Tables are designed to accommodate a full spectrum of approaches and sources. This section of the Handbook describes specific features built into the CARD Tables that provide the opportunity to characterize the source and pedigree of parameter values at each Milestone. The CARD reviewer should ensure that these features are used to the maximum effectiveness. 5.1 ALTERNATE DESIGN SOLUTIONS

Early in a program, multiple approaches might exist to deliver the required capability. At Milestone A, when competition is anticipated, there is potential for alternate vendor design solutions. Additionally, to provide a baseline independent from contractor proposals, the program office might develop a Government Reference Architecture. To acknowledge and describe the various design approaches, technical input parameter columns are duplicated on the PMP Hardware Technical Table to accommodate multiple Reference Design Solutions (e.g. Government, Contractor A, B etc.). Given the ability to describe multiple approaches within a single CARD, the CARD sufficiency reviewer should fully expect population early in a program, and be undaunted by claims that it is too early to know what the winning contractor’s approach will be. Figure 5 below presents the PMP Hardware Table with multiple designs.

FIGURE 5: EXAMPLE WITH MULTIPLE REFERENCE DESIGNS

5.2 PEDIGREE AND VARIABILITY OF VALUES

The CARD Tables provide several columns to describe the pedigree of technical parameter values. As shown in Figure 6, there are opportunities to identify estimate vs actual values, range values, objective and threshold values and to cite input sources. The CARD reviewer should understand that each of these columns support a complete understanding of each value provided and, therefore, allow for the complete population of the Tables using the best data available at the time of submission.

olumnHead Government Reference Design Contractor A Contractor B

WBS/CRS Number

WBS/CRS Element Parameter Name Value Unit of

MeasureEstimate or Actual Source Value Estimate

or Actual Source Value Estimate or Actual Source

1.0 C4I Radar System1.1.2.2.2 Signal Processor Software Release 11.1.2.2.2.1 Signal Processor Software Release CSCI 1

New Code 50,000 SLOC Estimated Govt 5,000 Estimated Model 10,000 Estimated Contractor BModified Code 50,000 SLOC Estimated Govt 15,000 Estimated Model 50,000 Estimated Contractor BReused Code SLOC Estimated Govt 80,000 Estimated Model 15,000 Estimated Contractor B



FIGURE 6: CARD TABLES CONTAIN PEDIGREE AND VARIABILITY COLUMNS

5.2.1 ESTIMATE OR ACTUAL

The Tabular CARD construct acknowledges that a program technical baseline and associated parameters values will evolve throughout the acquisition life cycle. The technical and schedule baseline, described initially at Milestone A with estimated values, will evolve into a specific design solution with actual values. Many of the CARD Tables include columns to specify if the parameter value is an Estimate or an Actual. The CARD Tables that provide the opportunity to indicate Estimate or Actual values are shown in Figure 7:

FIGURE 7: ESTIMATE OR ACTUAL CARD TABLE COLUMNS

CARD Table Estimate or Actual Column

Milestone F Quantities and O&S Time Phased D-N

PMP Hardware Technical K Nonhardware Technical K Software Development Per CSCI Software Maintenance Per Release

O&S J and others The CARD reviewer should emphasize to CARD developers that to establish a technical baseline sufficient for cost estimating purposes, it is necessary that all cost driving parameters have an estimated value at key program Milestones and critical PPBE submission reviews.

5.2.2 RANGES The High and Low range columns also facilitate the ability to express the variability around a technical parameter value before a specific design solution or actuals are available. These columns are not intended to express or support risk or probabilistic uncertainty. The range columns can either be used to express a range of likely values with a High and a Low, or a band of estimated values around the most likely input. Much like the estimated or actual columns, the range columns increase the ability to describe a program in its early stages to ensure CARD sufficiency.

WBS/CRS Number

WBS/CRS Element Parameter Name Value Low High Margin Estimate

or Actual Objective Threshold Source Notes

1.0 C4I Radar SystemTarget Environment Platform

1.1 Prime Mission Product (Radar)WeightRadar Cross Section (RCS)Noise Figure - System

1.1.1 AntennaWeightBandwidthTunable BandwidthAntenna Type - Radar

If applicable use the Margin column to express design

margin. (If a unique parameter exists for Weight Growth

Allowance enter it as a value on that row, do not use the

Margin column for Weight.)

If parameter value is uncertain, and cannot be expressed as a single value use the Low and

High columns to enter a range. (Not meant for full probabilistic

expression of parameter uncertainty. But if needed enter

the most-likely value in the Value column, the bounds in the Low

and High column, and the confidence level in the Notes

Enter Requirements Specification Objective and

Threshold.



5.2.3 SOURCE

The Source column provides the basis or origin of a parameter value. Early in a program the source might be expressed as the Government Reference Architecture. The reviewer should ensure that references to other program documents are noted with full document title and date recorded in this column. If a parameter is provided by an engineer or acquisition specialist, the CARD reviewer should expect the name of a subject matter expert and office symbol to be provided. The source column, used effectively, not only communicates the origin of the technical parameter value, but also conveys its pedigree.

5.3 UNKNOWNS, TBDS AND N/AS

Even though a lack of detail and unknowns are expected in early acquisition phases, CARDs should not include any blank cells. The CARD reviewer should interpret blank cells as parameters not-yet-examined and waiting to be filled for this CARD version.

If the parameter is significant and needed to develop a realistic cost estimate, then the analyst should work closely with program office personnel to get the best available estimate for it and include that value in the CARD. However, for less critical parameters, unknown at this time, suggest working with the program office team to understand when values will be available and from what source and then annotate these cells as To Be Determined (TBD).

The use of TBDs, however, should be minimized and appropriate. For example, for a software intensive system, identifying the software sizing as TBD would be inappropriate.

The CARD reviewer should also advise the program office not to delete CARD rows, since it is easier in subsequent updates to unhide a row rather than insert a new row.

The use of Non-Applicable (NA) should be the result of a line by line review of the Commodity WBS and technical parameters. N/As should only be used when both the CARD reviewer and program office mutually agree that the parameter is not relevant to the system being procured. (See Section 5.3 CARD Tailoring). Note that when a WBS element or technical parameter is determined as N/A, the row should be hidden (not deleted) for presentation purposes.

5.4 CARD TAILORING

Even though CARD temporal expectations have been expressed, each program is unique. Aligning a CARD completeness assessment to appropriate expectations is very program-specific and places judgmental burden on the CARD reviewer. The CARD analyst should use common sense and their knowledge of the program to determine the level of parameter input requirements. Additionally, while commodity-specific CARD Tables provide an ample starting point for CARD creation, each program is different and tailoring of the CARD Tables is fully expected. Tailoring is not only available (for most Tables) it is indeed encouraged. While those parameters identified as core priority 1 from the CEM template are generally the most fundamental to completing a cost estimate, the priority 2 parameters should not be ignored and may be key to a particular program. The CARD



sufficiency reviewer should work proactively with the program office and the AFCAA Technical Advisor when tailoring the tabular CARD to ensure all echelon estimating needs are accommodated, and that the CARD is populated to the fullest extent possible. 6: SECURITY CONSIDERATIONS

As the reviewer of the CARD, the analyst should read and understand the program Security Classification Guide (SCG). The SCG is the authoritative document for identifying the sensitivity and classification of technical and/or programmatic parameters. Note that all tabular CARDs must adhere to SCG guidance and either be UNCLASSIFIED/FOUO, or if CLASSIFIED follow appropriate procedures. The CARD sufficiency reviewer should not populate the CARD directly. The program office has the responsibility to populate the CARD and to ensure the SCG is not breached. Additionally, the CARD reviewer needs to be sufficiently familiar with the SCG to ensure that they do not combine the CARD with any other cost estimate documentation that would then become CLASSIFIED through inadvertent compiliaton. Security best practices to share with the program office are as follows:

• Do not enter the value of a CLASSIFIED parameter value into an UNCLASSIFIED CARD.

• For UNCLASSIFIED CARDS, if the program baseline includes some key CLASSIFIED parameters, best practice is to annotate on the CLASSIFIED parameter row “See SCG”.

• Do not specify the classification level of a CLASSIFIED parameter as “S” or “TS”. In some cases, security issues or compilations can occur solely by indicating the classification level of a particular parameter.

Additionally, check that CARD inputs reflecting contractor design approaches or pricing data are marked Contractor Proprietary Data and protected in accordance with Non-Disclosure Agreements.



7: RESOURCES The commodity specific CARD Tables are available on the CADE Portal http://cade.osd.mil/policy/card in conjunction with the updated guidance. Tables feature Work Breakdown Structures (WBS) that are MIL STD 881 product extensions and align with standard Cost and Software Data Reporting (CSDR) requirements and parameters, and with the future Technical Data Report (1921-T). Additional CARD references include:

• Cost Analysis Guidance and Procedures Instruction (5000.73), 09 Jun 15 • Guideline for the Preparation and Maintenance of the Cost Analysis Requirements Description

(CARD) • Guidelines for the Preparation and Maintenance of CARD Tables

7.1 CARD TAILORING

As the CARD sufficiency reviewer be familiar with both Government and contractor standard data sources to enable successful tabular CARD completion. Expect that by Milestone B both Government and Contractor sources will be required for successful CARD population. Understand how various documents from the Contract Data Requirements List (CDRL) can support CARD completion, and encourage program office to utilize these sources to the fullest effectiveness. Given that Contractor technical design parameters will be required to complete the CARD at later Milestones, recommend working with the program office to ensure data reported at PDR and CDR, and later Program Management Reviews (PMR) contains required attributes and information. Judicious use of the Technical Data Report (DD Form 1921-T) may also facilitate tabular CARD completion. 7.2 CARD CHECKLIST

The CARD reviewer needs to ensure that the populated CARD content is consistent with prioritized expectations and program maturity. Attachment A, the CARD Table Checklist provides a useful guideline for assessing the tabular CARD completeness. Checklist questions are identified for each CARD Table.

http://cade.osd.mil/policy/card



TABLE 1: KEY DATA SOURCES

Government Contractor Security Classification Guide (SCG) Integrated Master Schedule (IMS) Test & Evaluation Master Plan (TEMP) Bill of Materials (BOM)/ Parts List Work Breakdown Structure (WBS) Preliminary and Critical Design Reviews

Reports System Engineering Plan (SEP) Vendor Lists Acquisition Program Baseline (APB) Power Allocation Summary Production Plan Mass Properties (detailed) Technical Requirements Description (TRD) Configuration Drawings Program Milestone Schedule Software Resources Data Report (SRDR),

Initial Developer Report Acquisition Strategy Contractor Work Breakdown Structure (CWBS) Contract History/Data (detailed) 1921-T Technical Data Reports Capability Development Document (CDD) Configuration Audit Capability Production Document (CPD) Size, Weight and Power (SWAP) Reports President’s Budget / Budget Estimate Submission (BES)

Store Technical and Mass Property Sheet (STAMP)

Selective Acquisition Report (SAR) Beddown Plan Statement of Objectives/Work (SOO/SOW) SW Development/Sustainment Plan Performance Work Statement (PWS) Life Cycle Management Plan (LCMP) Life Cycle Sustainment Plan (LCSP) Detailed Test Execution Plans Manpower Estimates/Actuals (detailed) Source Of Repair Assignment Process (SORAP)

Acquisition Decision Memorandum (ADM) Software Resources Data Report (SRDR), Initial Government Report

Analysis of Alternatives (AoA) Program Environmental, Safety and Occupational Health Evaluation (PESHE)

Program Protection Plan (PPP) Item Unique Identification Implementation Plan System Threat Assessment Report (STAR) Spares Provisioning Report



8: CARD INSUFFICIENCY A CARD reviewer should deem a CARD insufficient if any of the following circumstances occur:

• CARD detail is insufficient to support Cost Component Analysis (CCA) or Independent Cost Estimate (ICE) development

• CARD detail does align with population expectations at program Milestones • CARD has not been updated with additional parameter values over subsequent submissions

and life cycle • Estimated CARD values not updated to reflect actuals when actuals should be available • CARD features for multiple solutions, estimated vs actuals and sources not used or consistent

with available program information Note that the list above is not absolute or exhaustive. However, when the CARD reviewer observes any of the circumstances above, recommend elevating to the AFCAA Technical Advisor for adjudication and support. Working proactively with the program office, it is expected that the CARD ultimately will be sufficiently populated and accepted. 9: SUMMARY AND CONCLUSION Reliable and comprehensive cost data is essential to produce credible cost estimates as required by statute and regulation. CAPE, along with the Service cost centers, have come together to strategically plan for and implement better data collection. Air Force goals with these improvements are to make data collection easier; improve productivity and quality of cost estimates; close data gaps; and provide cost and acquisition community with the data required for better analysis and decision making. The CARD sufficiency reviewers play a key role in meeting these strategic objectives. It is their responsibility to ensure that the CARD Tables are populated to the fullest extent using estimated and/or actuals values and the flexibility embedded in the Tables themselves that facilitates the ability to fully describe a system even at its earliest program stages.



ATTACHMENT A

CARD Table Checklist

The CARD Table Checklist provides a guideline for assessing the tabular CARD completeness.

Overall

• Is the populated content consistent with program maturity? See technical parameter expectations by milestone in the AFCAA CEM Data Collection Templates. In particular see columns AX–BB of the PMP Hardware Technical and Nonhardware Technical templates, and columns AQ-AU of the O&S template. These can be obtained from your AFCAA Technical Advisor.

• Is the populated content consistent with prioritized expectations? Refer to the AFCAA CEM Data Collection Templates to assess if the highest priority parameters have been addressed. In particular see columns AX–BB of the PMP Hardware Technical and Nonhardware Technical templates, and column Q of the O&S template. These can be obtained from your AFCAA Technical Advisor.

• Is this CARD consistent with tailoring previously discussed between the PM, Service Cost Agency analyst, and CAPE analyst?

• Have N/As been fully vetted with Technical Advisors and CAPE analysts? • Have N/A rows been hidden? • Have TBDs been minimized, and has the program office identified when the value will be

available and the anticipated source? • Is the Security Classification Guide (SCG) being used to ensure CARD meets classification

requirements?

Tables to Present Cost Drivers by WBS Element

PMP Hardware Technical Table

• Are Physical, Performance, and Configuration parameters shown for each WBS element? • For those that are “TBD,” have the PM, SCA, and CAPE discussed the implications of each

“TBD” to their respective estimating approach? • Is Material Mix stated for structural items? • Are Manufacturing Processes stated? • Are any parameters new? If so, are they defined? • Are software elements defined either here or on the Software Development Table? • If the WBS has been tailored for multiple variants, are the unique and common parameters of

each WBS element clear? • Is the design heritage clear for each WBS item? • Has the Alternate Reference Design feature been used to identify alternate design approaches

and values?



Nonhardware Technical Table

• Is this Table replicated for each phase? • Are the test resources adequately described? • Are dates and number of test assets by location clear?

Software Development Table

• Is there a Table for each software WBS element? • Are key software cost model input parameters provided? • Is each CSCI or CI described?

Software Maintenance Table

• Is there a Table for each software WBS element? • Are key software cost model input parameters provided?

O&S Table

• Are system activity rates provided? Are the rates clear by location and/or organization?

Tables to Describe the Program’s Quantities

Quantities and O&S Time Phased Table

• Is every year shown? Prior years? • Are the end item names clear on this Table and consistently used on other Tables throughout the

CARD? • Are prior and concurrent quantities clear? • In addition to procurement quantities, is the fielded/supported inventory throughout O&S

described? • Are known future upgrades/resets shown?

Configuration Table

• Are the column names the same as those on the Time-phased Quantity Table? • Are the differences between test configurations and fieldable/operational configurations clear? • For each WBS/subassembly item noted as unique, are its technical parameters on the PMP

Hardware Technical Table? • Have GFE items been included?

Manpower Time Phased Table



• Has the program office and other Government acquisition staff been described? • Is this information organized consistently with the phases in the Acquisition Table? • Is the operator and maintainer headcounts provided for every unit receiving the system? • Is this information organized consistently with the inventory and organizations shown in the

Time-phased Quantity Table?

Tables to Describe Details of the Program

LRU Level Table

• Are all details necessary for maintenance estimating included? • Are Mean Time Between Failure (MTBF) parameters shown?

Part Level Table

• If a cost element is largely COTS/catalog, is the Bill of Materials (BOM) included? • If not a COTS program, have the high-cost items been listed?

GFE Table

• Have GFE items provided? Is it clear to which WBS element they belong?


Program Table

• If this CARD is for an existing MDAP, is the information correct and consistent with DAMIR and DCARC metadata?

Milestone Table

• Are the dates expressed as full day, month and year? • Are the milestones and dates sufficient to time-phase the cost estimate? • Are the milestones and dates sufficient to determine durations for time-sensitive costs?

Acquisition Table

• Is every planned life cycle phase described?



• Is there a row for every major contractor and participating Government Agency? • Is contractor fee specified? • If applicable is subcontractor fee specified? • Is withhold rate included? • Are the begin/end dates sufficient to estimate time-sensitive costs? • Is competition clearly described?

Roles Table

• Are relative participation of Government, Prime Contractor and subcontractors clear by each common WBS element?

• Are major second and third tier vendors shown by each PMP WBS element?

Budget Plan Table

• Has the source of the budget (i.e., POM, BES, or PB) been provided? • Is the quantity on which this budget is based provided?

WBS/CRS Definitions Table

• Are the WBS elements consistent throughout all the Tables? • If the WBS differs by phase, has each been provided?

Metadata Table

• Is the preparer of this Table identified?



ATTACHMENT B

4.2 (UPDATE) TECHNICAL PARAMETER EXPECTATIONS BY KEY MILESTONE Section 4.2 references the completeness columns on the CEM templates. Revisions to the CARD tables are underway and this Attachment is provided to align Section 4.2 with the anticipated future versions of the CARD Tables. To further support a comprehensive evaluation of the completeness of CARD Tables examine the CARD Expectations columns to the far right of each technical parameter table.

FIGURE4: CARD COMPLETION EXPECTATIONS

First, to support the data collection objective of the CEM Templates, the criticality of each CARD technical parameter has been reviewed and assessed. To focus CEM Template population on the most key cost driving parameters, “Core” parameters have been annotated with an “X” in column AY. The CARD reviewer can compare program office CARD inputs to the CEM Template Core set to ensure the CARD, at the minimum, addresses the most important parameters required for cost analysis purposes. Secondly, in the CEM Templates each unique technical parameter has been assigned an authoritative source at each key program Milestone. These source ratings can be found in the far right-hand columns (BA – BE) of each commodity Prime Mission Product (PMP) Hardware Technical and Nonhardware Technical Table. The CARD reviewer can compare program office CARD inputs to these population expectations to ensure the CARD is completed to the fullest extent at each submission with values from the most authoritative source. The CARD reviewer should expect a preponderance of estimated values in early program stages. Inputs will most likely be either a Government Forecast (GF) or, once the contract has been awarded, a Contractor Forecast (CF). (Note that if both sources exist for a single parameter, with a yet to be determined actual, the PMP Hardware Technical Table Alternate Reference Design feature can be used to present both values.) However, as the program matures, expect notional values to be replaced with Government actuals (GA) or contractor measured and tested actuals (CA).

olumnHead CARD ExpectationsWBS/CRS Number

WBS/CRS Element Parameter Name Core

MS A CARD

Post-PDR CARD

MS B CARD

Post-CDR CARD

MS C CARD

Parameter 1 X - - GF GF CAParameter 2 X GF CF CF CA CAParameter 3 - GF CF CA CAParameter 4 X GF GF GF GF GA

- N/AGF-Government ForecastGA-Government ActualCF-Contractor ForecastCA-Contractor Actual