Brian LihaniDeputy Chief, Aerospace Warning BranchHQ NORAD/J35W11 May 2011

Analysis of Alternatives (AoA)

This Briefing is ClassifiedUNCLASSIFIED

Purpose of Analysis of Alternatives (AoA)

• Evaluate performance, operational effectiveness, operational suitability, and estimated costs of alternative systems to meet a mission capability

• Assess advantages and disadvantages of alternatives being considered to satisfy capabilities. Includes sensitivity of each alternative to possible changes in key assumptions or variables

• A key input to defining system capabilities in the Capability Development Document

2

Bottom line Conduct analysis to determine the best solution based on cost, risk, and capability

What determines an AoA is required?

• DoDI 5000.02 implies that AoAs are mandatory for all programs

• In reality, the true driver is statutory requirements where AoAs are mandatory for:• All potential and designated Acquisition Category (ACAT) I

efforts• ACAT-I: Major Defense Acquisition Program (MDAP)

• 10 USC § 2366a & b: Milestone Certifications to Congress

• ACAT-IA: Major Automated Information System (MAIS)• 40 USC § 1401 et seq.: Clinger-Cohen Act (CCA) of 1996• PL 107-248 § 8808: CCA Compliance & MAIS Funding

• Can be designated ACAT I / IA if “JROC Interest” – for programs where the capabilities have a significant impact on joint warfighting or have a potential impact across Services

3

UNCLASSIFIED

UNCLASSIFIED

AoA Defined

• An AoA is the evaluation of the performance, operational effectiveness, operational suitability, and estimated costs of alternative systems to meet a mission capability. (CJCSI 3170.01)

4

Which one, in what configuration… or is it a combination (i.e., system of systems [SoS] or Family of Systems [FoS])?

UNCLASSIFIED

UNCLASSIFIED

What is an AoA?

• Key AoA components• Operational environment

• i.e., threats and scenarios

• Alternative development• Employment concept development• Risk analysis • Effectiveness analysis• Cost analysis• Cost-effectiveness analysis

• Value to the decision process• Identifies the viable trade space (not just the 100% solution)• Creates audit trail of the analysis process• Provides an analytical basis for any acquisitions that may follow

5

Joint CONOPS

Joint Capabilities Integration & DevelopmentSystem (JCIDS)

VCJCS or Service Chief Oversight

Defense AcquisitionSystem

MDAOversight

Calendar Driven

Need Driven

Event Driven

Planning, Programming, Budgeting & Execution (PPBE) Process

DEPSECDEF Oversight

ANALYSIS

UNCLASSIFIED

UNCLASSIFIED

When are AoAs performed?

• Suggested by an Initial Capabilities Document (ICD)

• Authorized at the Material Development Decision (MDD)

• Conducted during the Materiel Solution Analysis (MSA) phase

• Updated prior to Milestones B and C as directed by MDA

Operations and Support

Engineering and Manufacturing Development

Production and Deployment

Technology Development

MaterielSolutionAnalysis

Defense AcquisitionManagement System

(DODI 5000-02)

Development Planning &MDD Prep

Capability Development Document

Technology Development Strategy

AoAAoA

MDD MS A MS B MS C

ConceptDevelopment(AFI 10-601)

Capability Production Document

Initial Capability Document

AoA Study Guidance and Study Plan

RequiredDocumentation

How long do AoAs take and cost?

• Short answer: it depends!• Key Drivers:

• Decision maker expectations

• Available resources: time, personnel/expertise, funding

7

Study Results

“Trade Space”

Good

FastCheap

“Good, fast, or cheap… pick two!”

UNCLASSIFIED

UNCLASSIFIED

• Data for 56 AoAs **• Average length 20 months• 61% took between 13-24 months

• Variations generally correlate to• Complexity of the problem• Stakeholder politics• Funding levels• How much analysis already done

• Times are approximate, slightly conservative

• Those longer than 36 months have extenuating circumstances

• Data for 27 AoAs (2000-2008) **• Average cost $3.9M• Gov’t salaries not included• Earlier AoA costs very suspect

• Primary factors affecting cost:• Blue Suit vs Contractor mix• Non-AoA work counted in the total

(engineering design, test, etc.)• Length of the AoA (consistent

$300-400K/mo)

• Costs are approximate; often the study lead reluctant to give us costs

** Data provided by OAS

Cost of AoA

• 2000-present: cost range from $500K to $11.5M (Ground Moving-Target Indicator (GMTI)• GMTI estimated cost was $25M – provided by Cost lead

• 3600 funding only – money requested by Space and Missile Center (SMC) to directly fund work

• 3400 funding and/or “in-house” contractor support

• Prompt Global Strike (PGS) AoA• Pre-AoA prior to 2003 – all work “out of hide” funded

• ADM assigned

• Reported to AFROCC 2008; Final Report signed in 2009

• Requested analysis continues

8

** Single country, single component, single service, single MAJCOM, single mission, single technology

UNCLASSIFIED

UNCLASSIFIED

North American Air Domain Awareness – Surveillance Analysis of Alternatives

9

UNCLASSIFIED/FOUO

2. Develop Analysis Plan• Review existing knowledge• Develop appropriate

measures• Develop analysis

methodologies• Socialize the plan and get

approval

5. Interpret the Results• Determine meaning of the data

generated• Get independent sanity check

7. Package Results• Focus on issues• Make it clear and understandable• Orient to decision maker• Socialize the results and get approval

6. Prove the Answer• Recheck assumptions• Analyze sensitivities• Identify additional analysis needed

3. Gather & Review Data• Identify input data

requirements • Accredit/certify• Identify voids

Data Types• Scenarios• Tools• Performance• Cost

1. Define the Problem (J58)• Determine purpose• Determine/discuss issues• Define alternatives/approaches• Determine audience for results• Identify potential detractors

4. Execute Analysis Plan

Conduct the analysis• Effectiveness• Cost• Risk• Affordability

• Documents required for AoA start:• Joint Integrated Air & Missile Defense Organization

(JIAMDO) IAMD Initial Capabilities Document (ICD)• Approved by JROCM dated 29 Oct 2010

• OSD/CA&PE for AoA Study Guidance• N-NC/J58 working comments

• N-NC/J5 Study Plan• Currently on hold – awaiting Study Guidance

finalization• Material Development Decision (MDD) – formal entry point

into acquisition process

• AoA required for ACAT 1 acquisitions (DoDI 5000.02)

• Purpose:• Develop an International/Interagency solution to

address North American air/cruise missile surveillance gaps and deterioration while maintaining current capabilities

• Scope:• AoA is unique due to scope – current planning has N-

NC & ACC as co-leads

• Timeline/Funding:• Currently advertised as 18-24 months after approval;

$5M

• N-NC/J84 supporting as follows:• Effectiveness Analysis Team Lead, Risk Team Lead,

Integrating Analyst

AoA Activities

Etc…

UNCLASSIFIED/FOUO

10