1

34th Annual ITEA Symposium: T&E in a Time of Risk and Change

Supporting “Rapid Acquisition” with Speed and Agility

Mr. Derrick HintonActing Director

Test Resource Management Center5 October 2017

2

Topics of Discussion

• TRMC Overview• “Rapid Acquisition” (RA) Defined• Alternative Pathways for Acquisition• Sample RA Use Case• Characteristics of the RA Process• Required Characteristics of T&E

Infrastructure to Support RA• Identifying and Overcoming Challenges• Ultimate Vision• Points of Contact

3

TRMC Overview

4

TRMC OrganizationAs of 1 Oct 2017

Chief Financial Officer

Chief Operating Officer

Under Secretary of Defense for Acquisition, Technology & LogisticsThe Honorable Ellen M. Lord

Deputy EA for Cyber Test Ranges

AgencyRange

Oversight

ArmyRange

Oversight

Air ForceRange

Oversight

NavyRange

Oversight

Deputy Director, T&E Range Oversight

Deputy PM, CTEIP PM, REP

PM, CTEIP

Deputy PM, T&E/S&T

Director, TENA SDA

Deputy PM, JMETC

Director, NCR Complex

Deputy Director,

NCR Complex

Deputy Director, Major Initiatives and Technical AnalysesPM, T&E/S&T // PM, JMETC

Principal Deputy, TRMC

Ms. Sheila Wright (Acting)

PM, NCR Complex Expansion

Deputy PM, NCR Complex Expansion

Director, TRMC

Mr. G. Derrick Hinton (Acting)

5

TRMC Missions (1 of 2)DoD (Charter) Directive 5105.71

Statutory Regulatory

MRTFB OVERSIGHT / T&E INFRASTRUCTURE• Plan for and assess the adequacy of the MRTFB to provide adequate

testing in support of the development, acquisition, fielding, and sustainment of defense systems

• Support the Department’s objective of ensuring compliance with DoDD 7000.14-R

• Review proposed significant changes to T&E facilities and resources of the MRTFB before they are implemented by the DoD Components

• Issue guidance to the DoD Components, through the USD(AT&L), with respect to MRTFB planning

• Maintain an awareness of other T&E facilities and resources, within and outside the DoD, and their impacts on DoD requirements

• Serve as Executive Agent for Cyber Test RangesSTRATEGIC PLAN

• Complete a strategic plan for T&E not less often than once every 2 fiscal years

6

TRMC Missions (2 of 2)DoD (Charter) Directive 5105.71

Statutory Regulatory

BUDGET CERTIFICATION• Submit report to the SECDEF containing the comments of the Director

concerning all such proposed budgets, together with the Director’s certification as to whether such proposed budgets are adequate

PROGRAMS• Administer the CTEIP (Central Test and Evaluation Investment

Program) and T&E/S&T ProgramCAPABILITIES

• Manage and operate the JMETC (Joint Mission Environment Test Capability) Multiple Independent Levels of Security (MILS) Network and the Regional Service Delivery Points (RSDP) cloud computing environments

• Manage and operate the NCR (National Cyber Range) (IAW RMD 407A1, Issue #1, Title: Cyber, Jan 12, 2015) to provide test capability and capacity for the T&E Community

7

“Rapid Acquisition”(RA) Defined

8

“Rapid Acquisition” (RA) Defined

CJCSI 3170.01I, 23 January 2015, Glossary page GL-7:

“Rapid Acquisition—A streamlined and tightly integrated

iterative approach, acting upon validated urgent or

emergent capability requirements, to: conduct analysis

and evaluate alternatives … contract using all available

statutory and regulatory authorities and waivers and

deviations of such, appropriate to the situation; identify

and minimize technical development, ..; and rapidly

produce and deliver required capabilities.”

9

Alternative Pathwaysfor Acquisition

10

A New Acquisition Pathway

• Today, the DoD predominately uses two acquisition pathways that are well understood and used regularly:

– Major Defense Acquisition Programs (DoDI 5000.02)

– Urgent/Emerging Operational Needs (DoDD 5000.71)

• DoD’s response to Congress on NDAA FY 2017 section 901, requiring the re-org of AT&L, states:

“…in today's near-peer competitive environment, DoD needs alternative pathways to acquire capabilities”

• FY 2016 NDAA provided such an additional pathway:

‒ Middle Tier Acquisition

11

Current DoD Policy on Rapid Acquisition

DoDD 5000.71, August 24, 2012, “Rapid Fulfillment of Combatant Commander Urgent Operational Needs”

“DoD’s highest priority is to provide warfighters involved in conflict or preparing for imminent contingency operations with the capabilities urgently needed to overcome unforeseen threats, achieve mission success, and reduce risk of casualties.”

DoDI 5000.02February 02, 2017“Operation of the

Defense Acquisition

System”

“Rapid Fielding of Capabilities”

Enclosure 13,“URGENT CAPABILITY

ACQUISITION”

CJCSI 3170.01IJanuary 23, 2015

“Joint Capabilities Integration and Development

System”

Complies with PUBLIC LAW 107–314—DEC. 2, 2002BOB STUMP NATIONAL DEFENSE

AUTHORIZATION ACT FOR FISCAL YEAR 2003SEC. 806. RAPID ACQUISITION AND DEPLOYMENT PROCEDURES.

12

Sample RA Use Case

13

14

Characteristics of theRA Process

15

Characteristics of the RA Process

• Accepts greater risk than deliberate process• Includes rapid development, rapid prototyping, and/or

rapid procurement and rapid fielding• Synchronizes abbreviated requirements, resourcing

and tailored processes and eliminates approval layers• Deviates from and/or waives authorities• RA delivery includes a “CONOP refinement” effort after

equipment delivery• RA organization assists with operationalizing solutions• Includes an operational assessment• Iterative process; leads to spiral development• Solution includes a training element• Initial sustainment generally includes Field Service

Representatives

Examples:• Global Hawk

• Joint Improvised Explosive Device Defeat Office

• Mine Resistant Ambush Protected (MRAP) Vehicles

• Predator Hellfire Missiles

Adapted from Chris O’Donnell (D,JRAC) “Rapid Acquisition” brief to Product Support Manager Workshop on 8 June 2017

16

Required Characteristics of T&E Infrastructure

to Support RA

17

Required Characteristics of T&E Infrastructure to Support RA

Examples:• Global Hawk

• Joint Improvised Explosive Device Defeat Office

• Mine Resistant Ambush Protected (MRAP) Vehicles

• Predator Hellfire Missiles

T&E must rapidly present environments, targets, threats using:• Modeling and Simulationo Red, Blue, and Gray digital intel packages to rapidly

integrate and update profiles into simulationso Rapid virtualization of system under testo Common protocols across all M&S and T&E activities

• Hardware in the Loop & Installed System Test Facilitieso Maximize digital/modular interfaces for rapid reconfiguration

• Open Air Rangeso Core of peer competitors’ order of battle laydown establishedo Digital/reconfigurable emulatorso Contract authority

• Real time Data Collection and Presentationo Big data miningo High-speed analysis tools for rapid transition of data to reports

18

Rapid Acquisition Test Infrastructure Enablers

When speed of delivery is a driving priority,

program leadership needs to assess

the ability of a candidate range

not only to provide technical services,

but also to deliver those services

and critical administrative enabling functions

at a speed acceptable to the fielding

requirement.

19

To Support RA Programs, an MRTFB Must…

• Have agile internal processes and procedures

• Have modeling and simulation support services

• Be configurable to support RA (ISTFs)

• Be able to develop and produce tailored T&E products

• Be able to develop tactics, techniques and procedures concurrent with T&E

• Be able to begin immediate construction of infrastructure

• Be able to clear non-traditional air vehicles in time to support RA

20

…and Must Also Have the Following Already in Place…

• A variety of funding constructs for RA

• Appropriate contract vehicles to support immediate activity

• Environmental Impact Clearances for intended T&E

• Special Use airspace

• Spectrum allocations (in place or immediately resolvable)

• Rapidly configurable Govt and contractor workforce

• TS/TS-SCI billets and workspaces for Govt, contractors, and foreign partners

21

T&E Infrastructure SupportMissile engagement effectiveness requires successful

execution of all segments of the offensive kill chain

Next GenerationJammer (NGJ) EA-18G F-35

DynamicTargets (5)

Rapidly configurable environment

Programmable threats

HWIL / ISTFM&S

MILCON Waiver- DIADS- ITASE- JSE- Etc…

• Emulate necessary environment

• Accept higher risk• Demonstrate

performance• Iterate

Programmable & Mobile Threats

Teaming Arrangements

OAR TransportableInstrumentation

22

Identifying and Overcoming Challenges

23

• Increasing RF spectrum demand

• Increasing demand for data accuracy and precision

• Increasing data quantity driving network and storage capacity

• SUTs driving need for miniaturization and modularity

• Common hardware and standards for telemetry and TSPI

Common Range InstrumentationChallenges & Solutions

Challenges Solutions• CTEIP Projects

o Common Range Integrated Instrumentation System (CRIIS)

o Integrated Network Enhanced Telemetry (iNET)

o Advanced Range Tracking and Imaging System (ARTIS)

• Supporting T&E/S&T Effortso Spectrum Efficient Technology Efforts

ULTIMATE GOAL:Wireless Inter-Range Network Environment

That Seamlessly Supports Any & All Range Operations

24/7/365 In the Most Spectrum-Efficient Manner Possible

24

• Range control and safety

• Establishing user trust

• Non-intrusive instrumentation

• Testing see-and-avoid technologies

• Repeatable testing of cognitive autonomous system interactions

• DMS/ISTF/OAR correlation

• Replicating swarming autonomy scenarios

AutonomyChallenges & Solutions

Challenges Solutions• CTEIP Projects

o Swarm Autonomy and Scoring (SWARM) • Supporting T&E/S&T Efforts

o Autonomy T&E Infrastructure Gap Assessment and Investment Plan (IGAP)

o Stress Testing of Autonomy Architectures (STAA)

o Robustness Inside-Out Testing (RIOT) o Safe Testing of Autonomy in Complex,

Interactive Environments (TACE)

25

• Keeping pace with rapidly changing cyber threats

• Ability to present representative operational environments

• Visualization of cyber battlefield and effects

• Controlling propagation of threat (isolated from operational systems)

• Sanitization of the test environment

• Maximize utility of distributed cyber test environments

Cyber SecurityChallenges & Solutions

Challenges Solutions• National Cyber Range (NCR)

o Multiple Independent Levels of Security (MILS) architecture; rapid emulation; automation; sanitization capabilities

o Cyber Table Topso Regional Service Delivery Points (RSDPs)o NCR Complex Buildout

• CTEIP Projects o CTEIP Cyber Test Analysis and Simulation

Environment (CyberTASE) • Supporting T&E/S&T Efforts

o Advanced Cyber Threat Representation (ACTR)

o Framework for Automated and Verified Sanitization (FAVS)

26

Ultimate Vision

VISION: Live-Virtual-Constructive (LVC) Test Environment

ConfigurationManagement

InformationAssuranceSecurity

V V & A

C R O S S D O M A I N O P E R A T I O N S

VirtualizationMassively Parallel Processing Big Data Storage

Big Data Analytics

ProtocolTranslators

High Level Architecture

Distributed Interactive SimulationSystem

Adaptors

TENAMiddleware

Gateways Range Protocols

Standard Processes and Repositories

Distributed Data Collection

Common Tools

Cloud Services

Test Support Network

System Integration

Mission BasedScenarios

Virtual & ConstructiveRed & Blue System

Simulations

Tactical Network Emulation

Programs of Record(e.g., F-35)

MOPFramework

EVENTMonitoring, Management, Control

POST-EVENTData Reduction, Analysis,

Visualization

PRE-EVENTPlanning

TE

NA

So

luti

on

s Cross DomainSolutionsJTEN JIORJMETC

Networks

VISION: LVC Test EnvironmentA Closer Look

29

TRMC Points of Contact

30

TRMC Points of ContactMr. Derrick Hinton

Acting Director, TRMC Acting Executive Agent (EA) for Cyber Test Ranges

george.d.hinton2.civ@mail.mil571-372-2761

Mr. Chip FergusonDeputy EA, Cyber Test Rangesbernard.b.ferguson.civ@mail.mil571-372-2697

Mr. AJ PathmanathanDirector, National Cyber Range (NCR) Complexarjuna.pathmanathan.civ@mail.mil571-372-2702

Dr. Robert TamburelloDeputy Director, NCR ComplexPM, NCR Complex Expansionrobert.n.tamburello.civ@mail.mil571-372-2753

Ms. Denise De La CruzDeputy Director, T&E Range Oversightdenise.b.delacruz.civ@mail.mil571-372-2733

Mr. George RumfordDeputy Director, Major Initiatives & Technical AnalysesPM, Test & Evaluation / Science & Technology (T&E/S&T)PM, Joint Mission Environment Test Capability (JMETC)george.j.rumford.civ@mail.mil571-372-2711

Mr. Christopher PaustPM, Central Test & Evaluation Investment Program (CTEIP)christopher.w.paust.civ@mail.mil571-372-2732

Mr. G. Derrick Hinton (Acting)Director, TRMC

EA, Cyber Test Rangesgeorge.d.hinton2.civ@mail.mil

571-372-2761

Ms. Sheila Wright (Acting)Principal Deputy, TRMC

sheila.r.wright.civ@mail.mil571-372-2735

As of 1 Oct 2017

http://www.acq.osd.mil/dte-trmc

31

Backup

32

Concept for Alternative Paths

• Report to Congress...Response to Section 901…August 2017– Alternative Pathway is not uncommon to smaller acquisitions– While DoDI 5000.02 and DoDD 5000.71 processes remain in place– USD(R&E) will set policy for implementing the alternative pathways to speed the delivery of

capability to the warfighter

• Additional DoD Initiatives:– Project MAVIN -- Agile -- Autonomy

Initiated by operators defining a need or researchers identifying an opportunity

Resources applied to procure limited numbers for:– Testing, prototyping, user evaluations, operational assessments

Deliberate decision point:– Further invest to scale up,– Divest,– Continue modifying, or– Harvest technology and move on

“…each echelon of the Department must be structured to rapidly adapt and field capabilities that leverage the advances that are occurring at an ever increasing pace in the commercial and defense

technology sectors” (Report to Congress,.. Response to Section 901… August 2017)

Figure 6: Alternate Pathways

Report to Congress,.. Response to Section 901… Alternate Pathways

33

RA programs may exploit test domains to a greater or lesser extent balancing risk to meet schedule

Test Domain Contribution

Open Air Live Launch

• High cost, low volume of test data, low statistical significance

• Complexity and cost of replicating complex scenario and environments

• Supports M&S validation• Limited by flight safety,

target operating limits

Modeling & Simulation

• Low cost, high volume, high statistical significance

• Validation of engagement models

• Digital representation of threat offensive capabilities

• Replication of operational context

Open Air Captive Carry

• Moderate cost, moderate volume, moderate statistical significance

• Focused on evaluating missile seeker commands, prior to live fire test

• Complexity and cost of replicating complex scenario and environments

• Supports M&S validation• Limited by flight safety,

captive carry operating limits

HW in the Loop &Installed Sys Test Facility• Intermediate step in systems

engineering development• Complexity of replicating

phenomenology; RF and IR environment

• Representative HW and SW• Replication of operational

context

Ground Test Resources Flight Test Resources

T e s t D o m a i n s

34

Major Range and Test Facility Base (MRTFB): The “Critical Core”

23 Sites: Army-8; Navy-6; Air Force-7; Defense Agency-2Legend:

Army, Navy, AF, Defense Agency

30th Space WingVandenberg AFB, CA

NAWC-WD Point Mugu, CA

Nevada Test and Training RangeNellis AFB, NV

Utah Test and Training RangeHill AFB, UT

West Desert Test CenterDugway Proving Ground, UT

White Sands Test Center, White Sands Missile Range, MN

Electronic Proving GroundFort Huachuca, AZ

96th Test Wing (Includes 96th Test Group), Eglin AFB, FL

Atlantic Undersea Test and Evaluation Center, Andros

Island, Bahamas

45th Space Wing, Patrick AFB, FL

Arnold Engineering Development Complex,

Arnold AFB, TN

Aberdeen Test Center, Aberdeen

Proving Ground, MDDISA

Ft. Meade, MD

Cold Regions Test Center

Fort Greely, AK

NAWC-WDChina Lake, CA NAWC-AD

Patuxent River NAS, MD

DISA,JITCFort Huachuca, AZU.S. Army Kwajalein Atoll

Reagan Test SiteTropic Regions

Test Center, various locations

PMRFKauai, HI

Yuma Test CenterYuma Proving Ground, AZ

412th Test WingEdwards AFB, CA

NUWC Keyport (Nanoose & Dabob ranges)Keyport, WA

Updated 22 Jul 2015 • Created by Congress• 25,000+ Personnel• ~$4B in Funding Annually

• Army – 5.5M Acres Land• Navy – 1M Acres Land, 165K

sq mi Sea, 103K sq mi Air• Air Force – Western Ranges

ALONE – 5M Acres Land

35

Annual Broad Agency Announcements (BAAs) Academia Industry Government Laboratories

Tri-Service working groups Validate requirements Evaluate proposals Facilitate technology transition

Central Oversight – Distributed Execution

Mission: Develop Technologies Required to Test Future Warfighting Capabilities

Established in FY02 Joint DDR&E / DOT&E Initiative Transitioned to TRMC in FY05

RDT&E Budget Activity 3 funds Purpose

High Risk / High Payoff R&D for Testing Foster technology transition to major DoD test ranges Risk reduction for test capabilities developments

58 ActiveProjects

High SpeedSystems

15 Active Projects

Eight Test Technology Areas

Directed Energy4 Active Projects

Cyberspace3 Active Projects

C4I & SoftwareIntensive Systems5 Active Projects

Electronic Warfare10 Active Projects

Unmanned & Autonomous Systems

4 Active Projects

AdvancedInstrumentation9 Active Projects

Shaping Technology into Tomorrow’s T&E Capabilities

Test & Evaluation / Science & Technology (T&E/S&T) Program

Spectrum Efficiencies

8 Active Projects

36

Requirements Drivers

Central Test & Evaluation Investment Program (CTEIP)

Established in FY91 by Congress with 6.4 RDT&E funds

Mission: Develop or Improve Major Test Capabilities that have multi-Service Utility

31 JIM, 6 EW, 17 Hypersonics, 19 REP, 13 TSP = 86 Projects

Near-Term Investments Long-Term multi-Service Investments

JIM-Core JIM-EW• Special DoD area of

emphasis• EMD of electronic warfare

(EW) test capabilities• Assess aircraft performance

against complex new threats.

• Service budget for O&M• Total cost ~$465 over ~7

years

Threat Systems Project(TSP)

• 1-2 year horizon• Address shortfalls

in threat systems representation

• Coordinated with DOT&E

• $3-5M/year

• 1-2 year horizon• EMD of

instrumentation to address near term OT shortfalls

• Coordinated with DOT&E

• $18-20M/year

Resource Enhancement

Project(REP)

Joint Improvement & Modernization (JIM)

Annual review of OT shortfalls

Biennial multi-Service T&E Reliance Nomination

ProcessMultiple DoD EW studies Annual review

of threat needs

JIM-Hypersonics• Special DoD area of

emphasis• EMD of hypersonic ground

test capabilities• Focus on hypersonic cruise

& boost glide missiles• Service budget for O&M• Total cost ~$350 over ~5

years

DoD Approved Roadmap

• 3-5 year requirement horizon

• EMD of major multi-Service test capabilities

• Development, not procurement

• Services & Agencies budget for O&M

• $110-140M/year, $600-$700M over 5 years

37

JMETC SECRET Network (JSN)

Ft Hood (2): CTSF, TTEC

WPAFB: SIMAF

As of 21 Aug 2017

Pax River:(11) E2C, E2D, MCL, ATRACETEF, SAIL, MFS, P-8,E-2C SIL, UASIL, EP-3 Triton Lab JMETC SYSCONEast Interconnect Rtr.

Aberdeen: ATC-STE, ARL

Langley TDLITCJS J6 DDC4 C4AD

Wallops Island SCSC

Dahlgren:(3) IWSL, DOC, ICSTFRapid-SIL/C2

Dam Neck CDSA

Newport News NGC VASCIC

McLean MITRE NSEL

Redstone (12): SMDCRTC: DTCC, DISTL, AvSTILAMRDEC SSDDAMRDEC SED: Patriot, THAAD, FAAD, GSIL, JLENS, MUSE, C-RAM

Crane (2):NSWC RDT&E, NTN Lab

Moorestown (2): CSEDS, CPTS

GTRI

Ft Huachuca (4):JITC, JTDL, CMIS, JTRS

Metro Park JMETC / TENA SDA Lab

Dugway Proving Ground

Eglin AFB (4):AOC, DTF, GWEF, KHILS Melbourne

NGC JSTARS

Keyport NUWC

Rome NY AFRLNewport NUWC

Tinker AFB AWACS

Greenville Rivet JointWSMR IRCC

Raytheon Tucson

LMCO Global Vision

Network

Pittsfield GD LCS

Arlington IDT

Manassas LM ASW

Ft. Sill, SMART Lab

Camp Pendleton:MCTSSA

China Lake (4):F/A-18, IBAR,

TSPIL, AV-8BLand Range, WSCI

Edwards (2):

Corona: NSWC

Point Loma : (2)SSC-PAC 59140 SSC-PAC CTB LMMTFTEC (Keyport Det.)

Point Mugu (4):ITEC, EW CyCon,AEA, Sea Range

MAGTF EW

W. Interconnect Rtr.Rancho Bernardo NGC Triton

412th EWG IFASTRidley

Port Hueneme: NSWC

Hanscom AFB CEIF

Leverages the SECRET Defense Research and Engineering Network (SDREN) for connectivity

Operates at SECRET classification, available 24/7 Continuous monitoring, troubleshooting, and optimization of the

end-to-end network infrastructure Capable of supporting numerous simultaneous test events

Functional JSN Locations: 46 (access to 77 labs/facilities)Planned JSN Locations: 4Connection Points to Other Networks: 5

DISTRIBUTION A. Approved for public release: distribution unlimited.

ArmyAir ForceNavyMarines

JointIndustry

MHPCCHawaii

Pearl Harbor FIST2FAC (Keyport Det.)

38

• Provides the ability to rapidly design, deploy and sanitize large scale, high fidelity test and training environments

– Extremely malicious threats can be unleashed on operationally, representative systems and networks to assess the impact on the network, networked weapon systems, the associated mission

• Significant technological advancements incorporated into integrated tool suite

– Highlighted by the significant level of automation – Ability to support up to four concurrent events, executed in completely

isolated testbeds, at different levels of classification• Automated sanitization capability

– Expunges all malicious code from every component used within the event and returns it to a pristine state ready for reuse

– Sanitizes not only storage but also servers (including non-volatile RAM) and network equipment

• Time saving– In many cases, what previously took weeks or months has been reduced to

hours or days – Allowing users the ability to rapidly plan, execute, and analyze their test or

training initiatives• NCR also provides SME

– Available, at the discretion of the user, to support the planning, execution and analysis of test and training events

• NCR is able to collaborate with other cyber ranges using secure networks

– When test events require special capabilities or additional scale

National Cyber Range (NCR) Complex

• Encapsulation Architecture & Operational Procedures• Computing Assets/Facility

• Integrated SW Testing Toolsuite