1

5th Generation Fighter Operational Risk

Management

Systems-of-Systems Engineering Workshop

Maj Gen Matt “Zap” MolloyCommander, Air Force Operational Test and Evaluation Center

Albuquerque, NM

Distribution A: Approved for public release; distribution is unlimited

2

• AFOTEC Mission…Background

• 5th Gen Fighter Testing …a lens to the Future:

– What is a “5th Generation” Fighter?

– F-22 & F-35 Overview

• 5th Gen “Systems-of-Systems” & Managing Risk…it’s Complicated!

• Closing Thoughts & Discussion

Overview

3

Test and Evaluate

New Capabilities in

Operationally

Realistic

Environments to

Inform Warfighters

and Influence

National Resource

Decisions

Mission

4

Will it Work?(Developmental Test)

5

Will it Work…In Combat?(Operational Test)

6

Why Does AFOTEC Exist?

7

– “OT&E. . .is done most effectively when

OT&E organizations report directly to the

Chief of Service, representing both the

developer and the user, but

organizationally independent of each.”

– “OT&E can and should contribute

significantly to decision-making at all

levels of the DoD.”

Why Does AFOTEC Exist?

8

“Fly before buy”

–Initial Operational Test

and Evaluation

–Independent

Operational Test

Agency

–Production

Representative Data

Why Does AFOTEC Exist?

9

Army

Navy

Marine

Operational

Test Agencies

Air Force Major Commands

Direct Reporting

Coordination

External Interactions

10

Today’s AFOTEC

AFOTEC Headquarters

Operating Location

Kelly Field, TX

Detachment 1

Joint Strike Fighter

Program Office

Arlington, VA

Detachment 2

Detachment 4Operating Location

Vandenberg AFB, CA

Space and Missile

Systems Center

Los Angeles AFB, CA

Detachment 5

Operating Location

Everett, WA

Operating Location

Hurlburt Field, FL

Detachment 6

Operating Location

Nellis AFB, NV

Operating Location

Buckley AFB, CO

Operating Location

Marietta, GA

11

TESTING 5TH GENERATION FIGHTERS

12

What is a 5th Gen Aircraft?

13

5th Gen Stealth….

14

This is our Situation…“A2/AD”

15

• Tactical Surprise Lost• Tracked, Locked and Shots Taken Upon• Mission Effectiveness Degraded and/or Lost

SOF

SOF

SOF

Legacy Aircraft in Anti-Access Environment …High Mission Risk

16

• Stealth, Speed, Precision, Sensor Fusion, maneuverability

5th Gen operational schema:• Battlefield system-of-systems approach:• Integration/interdependencies:

• Space/ISR/Air Superiority/C2/Grnd/IAMD

SOF

SOF

SOF

Asymmetric Advantage of 5th Gen

31

17

F-35 & F-22 as a System-of-Systems

Fusion and SOS

Integration

EWMore Capability at1/2 the Cost -Digital Modules,Production Processes, Apertures

DAS

Demonstrated

Significant Contributions

to Situation Awareness

Weapons

Finalized Carriage Concepts

Support All Weapons

CNI

Critical H/W Elements and

S/W Reuse Demonstrated

Mission Support

Finalized Carriage Concepts

Support All Weapons

EOTS

Low Profile Window

Optimized for Visibility &

Configuration Integration

Radar

Half the Weight,

Half the Cost Goal

Being Realized

ICP

COTS Technology Refresh Demonstrated

Common

Components

Economy of Scale Savings

CockpitProjection

Display Technology for

Enlarged Screen Area

and Availability

Open Architecture / Software

Standard Interfaces, Loose

Coupling, Object Oriented

Design, Commercial Tools

for Simpler Integration

Figure 2.1.x-x Display Portal concept w/ Virtual HUD Image

Figure 2.1.x-x Display Portal concept w/ Virtual HUD Image

18

4th Gen Federated Displays

Radar Warning Receiver Threat Emitter Location / Type

Radar Display Radar targets

ID info

Navigation points

Weapons Attack Steering

Situation/JTIDS Display Wingman / flight Position info

Pre-planned threat locations

Link-16 Data Link targets & ID info

Countermeasures info

19

Situation Display Position info

Simple line map

Navigation points

Pre-planned threat locations

Link-16 Data Link targets & ID info

Wingman/flight info

Radar search volume

Countermeasures info

Threat envelopes

Defensive Display Threat Emitter Location / Type

Countermeasures

Missile launch warning

Missile tracks

Threat envelopes

Attack Display ID Info

Weapons Attack Steering

“Fused tracks”

Side view of altitudes

Threat launch ranges

5th Gen Sensor FusionIt’s a mini “Systems-of-Systems”

(F-22 Cockpit)

20

212/23/2016 Docs 55445

“System-of-Systems-of-Systems”

CV

LH/LP

Air Base

PHM

Datalink

SATCOM

Beyond Line of Sight

JFAC

Joint Tactical Radio System (JTRS)

Functionality

Variable Message Format

VMF

Ground Forces

94 Information Exchange Requirements to Ensure Interoperability

Across US and Coalition Forces

JFAC

E-2

F/A-18

AWACs

JSTARS

Link -16

Aviation & Air Defense Assets

F-16

V-22

Blue SAMs

AEGIS

• Link 4/ACLS, ICLS

• Link 16

• LAN/WAN/Internet

• UHF/VHF Voice/Data

• SATCOM 2 Way Voice/Data

Autonomic Logistics

Information System

“ALIS” Global Supply Chain Mngmt

Adversary

systems

22

S-o-S Testing Challenges

• Where do you draw the test boundaries on a SoS

weapons system?– Fifth-generation platforms are redefining classical boundaries as

systems evolve from cooperative…to integrated…to interdependent

• How do you appropriately measure “suitability”,

“effectiveness” and “Mission Capability” in a System-

of-Systems weapons platform?– If the autonomic Logistics Information System is compromised can

the F-35 be Suitable and Mission Capable in the operational

environment?

• Open-air testing constraints– GPS jamming restrictions

– Full system employment constraints due to security/exploitation

concerns

23

5th-Gen Testing Challenges

• Raptors & Lightnings have voracious appetites; they

chew through the adversary with volume & velocity…– Very difficult to generate the threat density and complexity to meet

5th-gen operational test requirements Advanced SAMs, advanced adversary aircraft, and advanced jamming

• Creating a complex, threat-dense environment is key

to an effective 5th-gen IOT&E and for accurately

quantifying operational capability and risk– Limits to what can be accomplished in the air

• How do we crack this nut???

24

5th Gen Modeling & Sim

• Creating an advanced modeling & simulation environment– There are tough challenges posed by 5th gen M&S:

Creating the virtual battlespace and how the stealthy air vehicle dynamically

interacts within it

Accurately modeling advanced threats

Inserting blue forces into the virtual battlespace

• Creating this environment is key to an effective 5th gen

IOT&E and for accurately quantifying operational risk

25

F-35 Developmental Timeline

• Concurrent development, production, test, & operations

• Risk-Return tradeoffs in schedule, cost, and performance– Allows for early availability and maintaining schedule

– Risk in performance…retrofit cost & substandard capability

• AFOTEC has to adapt from its classical IOT&E approach – We are inserting our support and our assessments rendered to the

Services & Acq community in unique ways

F-35

Development Milestone A Milestone B Milestone C IOC

USMC

IOC

USAF

Block Buy

Robust LRP

400+ jets fielded

IOT&E

IOC

USN/

GBR

FRP?

FOC?

Cyber Vulnerability Assessments

Red Team Penetration Analysis

Networks

ALIS

Air Vehicle

USS WASP

Operational Assessment

(“OA”) iso USMC

Mt Home Deployment

OA iso USAFWeapons Delivery Events

A/A & A/G

26

CLOSING THOUGHTS

Managing Risk in 2020…..

27

Reducing Risk in 2020

• Advancing and Maturing M&S– Testing’s center-of-gravity has tipped toward M&S (w/ open-air validation)

– Live, virtual, and Constructive (“LVC”) holds great promise and opportunity

for the testing community …for industry

– Trend toward Gov’t owned systems Data rights, integration with other synthetic weapons systems

Training & Mission rehearsal

Affordability

Need for both closed-loop & Distributed Mission Operations simulation

– We need to better educate or testers, developers, operators, & acq officers

for the virtual world ahead…

• Packard’s imperative is indeed sound:– “Fly b-4 you buy…”

– F-35 is giving us a new perspective & CONOP as to how we execute

this principle in a non-standard environment

28

Reducing Risk in 2020

• We understand and quantify System Reliability Growth Curves when

fielding new weapons systems:

• This same methodology can be used for Operational Performance:– Manages technology risks assoc w/ 5th gen

Time

Reliability

System Reliability

Time

Capability

Operational

Performance

29

Reducing Risk in 2020

• 2020’s testing terrain is different– S-o-S interdependencies; M&S; Cyber…

• Must develop the “next generation” of testers…– Ready to meet the challenges of 2020, 2030, and beyond with test-

ready experienced teams Recruit, train, mentor & retain key personnel

Develop cyber OT skills; develop live virtual constructive & VVA skills

On the horizon: Long Range Strike Bomber, Long Range Stand Off

weapon, Ground-Based Strategic Deterrence system

Over the horizon: Directed energy weapons? Hypersonic systems?

• AFOTEC’s enduring Trump Card:– Our Airmen… Integrity, Service, Excellence

30

Questions & Comments

31

Excellence

32

33

Detachment 1 – F-35

34

Back up material

35

Cyber testing 5th Gen

• Complex task to define “cyber boundaries”

– Must take a system-of-systems approach

• Cyber warriors are “HD/LD” assets

– Can’t grow them fast enough to meet demand signal

• AFOTEC is reaching out for assistance:

– To the Total Force for assistance

Mutual benefit to share cybersecurity

knowledge

NM Guard brings wisdom & leading-edge practices from

industry

Assist AFOTEC with forecasted capacity shortfall

– To AFRL--bringing a “STEM” analytical approach to security

36

International Participation in IOT&E…Managing Risk

• 4 partner nations participating in F-35 IOT&E…unprecedented– US, UK, Dutch, Australia

• The Advantages:– Partner buy in ($$, strengthens the relationship)

– Program transparency

– Informs partner nation resource decision-making

• The Challenges:– Information disclosure

Unprecedented access to sensitive information

– Expectation management

Security/

DisclosurePartnering

Risk Management

372/23/2016 Docs 55445

JSF Autonomic Logistics System

Support System

Sustaining Engineering

24/7 Help Desk

Electronic Joint-Service Tech Data

Intelligent Maintenance Management

Global Supply Chain Insight

Support Equipment Management

Training System

Integrated Training

Embedded Pilot Training

On Demand Maintenance Training

Air Vehicle Software Reuse

Highly Supportable Aircraft

Smart / Reliable Design

Prognostics and Health Management

Remove and Replace (R/R) Maintenance

On Condition Maintenance

Autonomic Logistics Information System

Distributed Information System

Enterprise Resource Solution

Secure

Scalable

Deployable

Cyber-enabled Jet….

38

F-22

Development Milestone A Milestone B

IOT&E

FRPMilestone C IOC FOC

• F-22 followed a more classical acquisition timeline

40

• Mentoring New Mexico High School and

College Students

• Meals-on-Wheels and Habitat for Humanity

• Inmate Recovery and Prison Ministries

• Church Leaders, Music Coordinators,

Ushers, Chaplain Services

• Girl Scouts, Boy Scouts and 4-H

Instructors

• Children’s sports coach volunteers

• Leadership for Albuquerque Tuskegee

Airmen Chapter

• New Mexico Animal Humane Society

• Big Brothers / Big Sisters Programs

• New Mexico Boys and Girls Clubs

• New Mexico Roadrunner Food Bank

• Various High School Parent Teacher

Organization volunteers

• Kirtland Against Drunk Driving Program

AFOTEC in the Community

41

Questions?

CSAF Artwork; A C-17 from Charleston AFB flying medical and food relief to

the earthquake victims of Haiti