Experimentation of Complex, Adaptive Aerospace Mission Capabilities Experimentation of Complex, Adaptive Aerospace Mission Capabilities - An International Code of Best Practice for T&E - An International Code of Best Practice for T&E & Experimentation – & Experimentation – Spousal Family of Systems ParableSpousal Family of Systems Parable

Si ego Certiorem Faciam, Mihi Tu Delendus Eris – the Greek Goddess of Chaos! HomerSi ego Certiorem Faciam, Mihi Tu Delendus Eris – the Greek Goddess of Chaos! Homer

Wing Commander Malcolm G. Tutty MEng, CPEng, FRAeS, FIE(Aust)UniSA DASI PhD Candidate / HQSRG SO1TC – Airmalcolm.tutty@defence.gov.au / www.maltutty.com – Use visitor / Maven1

The FICS & ‘FOC Effectors’The FICS & ‘FOC Effectors’

FICS – Fundamental Inputs to FICS – Fundamental Inputs to CapabilityCapability

DrivenDriven by OCDs & Conops by OCDs & Conops

•PersonnelPersonnel•OrganisationOrganisation•Collective TrainingCollective Training•Major SystemsMajor Systems•SuppliesSupplies•Facilities & Training AreasFacilities & Training Areas•Command & ManagementCommand & Management

•User perspectives at all levels User perspectives at all levels (Sqn/Wing/Group/HQs) need to (Sqn/Wing/Group/HQs) need to be identified and argued for ...be identified and argued for ...

FOC Effectors - Fundamental FOC Effectors - Fundamental OutputsOutputs of Capability of Capability

ResultsResults expected of the Conops ... expected of the Conops ...SIPOCSIPOCCritical Operational/Technical Critical Operational/Technical Issues…Issues…Measures Of EffectivenessMeasures Of EffectivenessMeasures Of PerformanceMeasures Of PerformanceIV&VIV&V

Rate of EffortRate of EffortPreparedness – both readiness & Preparedness – both readiness & sustainmentsustainmentInteroperability with other SoS?

Question/Observation: How many Question/Observation: How many Conops do you have/need, really?Conops do you have/need, really?

2UnclassifiedSETE 2011 Tutty

Technology / Systems Technology / Systems Readiness LevelsReadiness Levels

3UnclassifiedSETE 2011 Tutty

CAPABILITYCAPABILITY IDENTIFIEDIDENTIFIED

CAPABILITYCAPABILITYDEFINITION-ACQUISITIONDEFINITION-ACQUISITION

INTRODUCTION INTRODUCTION INTO SERVICEINTO SERVICE

CAPABILITY MAINTAINEDCAPABILITY MAINTAINED

CONOPS / OCDCONOPS / OCDREQT DEFNREQT DEFN

00

Initial OT&E Initial OT&E

66

Acceptance T&EAcceptance T&E

55

INTRO INTRO INTOINTO

SERVICESERVICE

VEHICLE & SYSTEMS SPO(s)VEHICLE & SYSTEMS SPO(s)ACQN & SYST ENGACQN & SYST ENG

(Acft-Msn Cap – EDP / Sim Survey)(Acft-Msn Cap – EDP / Sim Survey)

22

CAPMGRCAPMGRORD MET?ORD MET?

Y / NY / N

Complex Capability Management ModelComplex Capability Management Model- Lust to Dust Methodology- Lust to Dust Methodology

LUSTLUST WOMB WOMB FISCAL REALITYFISCAL REALITY! ! CLEARANCE CLEARANCE CERTIFICATION CERTIFICATION ADOLESCENCE MATURITY TOMB DUSTADOLESCENCE MATURITY TOMB DUST

EXPERIMENT’NEXPERIMENT’N

11

Qualification T&EQualification T&E

44

ONGOING IN-SERVICE ONGOING IN-SERVICE Tactics, Techniques & ProceduresTactics, Techniques & Procedures

EXPERIMENTATIONEXPERIMENTATION

88

Developmental T&EDevelopmental T&E

33

CONOPS REQTCONOPS REQTACHIEVED &ACHIEVED &MAINTAINEDMAINTAINED

99

PPWWDD1010

4UnclassifiedSETE 2011 Tutty

Follow-on OT&E(s)Follow-on OT&E(s)

77

• F-111 – AGM-142 Demo and DMO F-111 – AGM-142 Demo and DMO win/fail/winwin/fail/win

• P-3 MU90 Light Weight Torpedo failP-3 MU90 Light Weight Torpedo fail• P-3 EO/IR SSHD & TCDL win/winP-3 EO/IR SSHD & TCDL win/win• Classic Hornet - JDAMER fail/winClassic Hornet - JDAMER fail/win• Classic JDAM/JDAMER/JSOW winsClassic JDAM/JDAMER/JSOW wins• Tiger ARH AGM-114 Hellfire winsTiger ARH AGM-114 Hellfire wins• Joint Fires (JTAC) and Danger Close Joint Fires (JTAC) and Danger Close

impact wins and future criticalityimpact wins and future criticality• Develop Spousal FoS CapabilityDevelop Spousal FoS Capability

SETE 2011 Tutty 5Unclassified

Proposed Case StudiesProposed Case Studies

• The power to achieve a desired operational effect in a nominated environment, within a specified time, and to sustain that effect for a designated period.

• Conops: To see, experience & change the world for the better (world peace / hunger / control (Pinky) solved) with a friend and lover until Death (Planned Withdrawal Date) with at least 2.6 children & with an online availability of daily missions …

FICS – Fundamental Inputs to Capability

- Driven by Conops, of course!a. Personnel: sensuous & sultry wife & fun kids,

wider families.b. Organisation: wife & family, decentralised

parents. c. Collective training: marriage, schools, Uni,

entertainment, Swingers Club ??. d. Major Systems: Home Ent, Cars, Computer

Avatar LVC Sim, Guns, power, ... e. Supplies: Alcohol, Food Markets, Financial,

contraception ... f. Facilities and Training Areas: House, Bar,

holiday home/caravan, shooting range. g. Support: social / financial / ETSA, Gas, water. h. Command and Management: Comms

essential (so I’m told), others?

FOCS - Fundamental Outputs of CapabilityThe Conops in User perspective ...Scenarios: ok be realistic nowSIPOC: ...RoE: key – average loving/sex / daily / weekly / monthly / yearly for some SEs?

Preparedness: readiness & sustainability?COI/CTIs: could be interesting to get agreement on these…MOE: Ops suitability & Effectiveness!, World Peace or World Hunger solved, kids in Uni/TAFE, Govt stable Pop/social/economic betterment

MOP: Min 2 kids Max 4... Other Perf measuresIV&V – should be interesting!Interoperability with other SoS: hmm…

6SETE 2010 Tutty Unclassified

Spousal Capability – a FoS parable!Spousal Capability – a FoS parable!

Girls

Boys

OUTCOMEOUTCOMEFemaleFemale – Male configurations that meet – Male configurations that meet operational suitability, effectiveness and operational suitability, effectiveness and

preparedness criteria!preparedness criteria!

The Right ‘Capabilities’Road

Roadmapmap

CAPABILITYCAPABILITY IDENTIFIEDIDENTIFIED

CAPABILITY CAPABILITY DEFINITION - ACQUISITIONDEFINITION - ACQUISITION

INTRODUCTION INTRODUCTION INTO SERVICEINTO SERVICE

CAPABILITY MAINTAINEDCAPABILITY MAINTAINED

Traditionally seen as a System/SoS -> FoS, obviously

To see, experience & change the world for the better (world peace/hunger/control (Pinky) solved) with

a friend and lover until Death (the Planned Withdrawal Date) with at least 2.6 children &

with an online availability of daily missions …

LUSTLUST WOMB WOMB FISCAL REALITIESFISCAL REALITIES! ! CLEARANCE CLEARANCE CERTIFICATION CERTIFICATION ADOLESCENCE TOMB DUSTADOLESCENCE TOMB DUST

Affairs!

Spousal Capability – a FoS parable!Spousal Capability – a FoS parable!

2000 2015 2030 2045 2060 2075 2100

Girlfriend 2

2045 GOALS 2060 GOALS

YOD

DELIVERY

2030 GOALS

Wife 2

Girlfriend 1

2075 GOALS

Boyfriend 1

FOT&E / IOS - .7

SCOPE/R&D/ORD .1 / .2

PWDIOC

Increment / Spiral / P3I

You

Husband 2

2015 GOALS

LINKS

Wife 3

? FoS - to achieve capability others are involved in the ‘mission’ – messy, complex, real? Value of / How to experiment early with what you have – simulation & LVC, -> 2nd Life / avatars? Note that the Conops has more dimensions…

A / IOT&E - .5 / .6

IN SERVICE - .8 / .9

ACQUISITION - .3 / .4

Wife 1

Spousal Capabilities - Spousal Capabilities - coming sometime soon…coming sometime soon…

• Dear Tech Support,• I have upgraded from Boyfriend 5.0 to Husband 1.9 and I

noticed a distinct slowdown in the overall system performance, particularly in the flower and jewellery applications, which operated flawlessly under Boyfriend 5.0.

• Husband 1.9 uninstalled many other valuable programs, such as Romance 9.5 and Personal Attention 6.5, and then installed undesirable programs such as NEWS 5.0, NOMONEY 3.0 and FOOTBAL 4.1, Conversation 8.0 no longer runs, and Housecleaning 2. 6 simply crashes the system.

• Please note that I have tried running Nagging 5.3 to fix these problems, but to no avail.

• What can I do?

9UnclassifiedSETE 2011 Tutty

IT Support’s ResponseIT Support’s Response• DEAR Madam,• First, keep in mind, Boyfriend 5.0 is an Entertainment Package,

while Husband 1.X is an operating system. Please enter command: ithoughtyoulovedme.Html and try to download Tears 6.2 and do not forget to install the Guilt 3.0 update. If that application works as designed, Husband1.X should then automatically run the applications Jewellery 2.0 and Flowers 3.5.

• However, overuse of the above application can cause Husband 1.0 to default to Silence 2.5 or Beer 6.1. Note that Beer 6.1 is a very bad program that will download the Snoring Loudly Beta.

• Whatever you do, DO NOT under any circumstances install Mother-In-Law 1.0 (it runs a virus in the background that will eventually seize control of all your system resources) and DO NOT attempt to reinstall the Boyfriend 5.0 program. These are unsupported applications and will crash Husband 1.9.

• In summary, Husband 1.9 is a great, mature program, but it does have limited memory and cannot learn new applications quickly. You might also consider buying additional software to improve memory and performance. We recommend: Cooking 3.0 and Good Looks 7.7.

10UnclassifiedSETE 2011 Tutty

WarningA Flight Test Armament Engineer is a person who

passes as an expert on the basis of being able to turn out, with prolific fortitude, indefinite strings of

incomprehensible mathematical figures, calculated with micromatic precision from vague assumptions which are based on debatable figures taken from inconclusive data,

carried out through forms of problematic accuracy by persons of doubtful

reliability and questionable mentality about things designed to kill or negate targets and/or non-allies, for the avowed

purpose of annoying and confounding a hopelessly befuddled group of key personnel and who

may never read the damned reports anyway!

11SETE 2011 Tutty Unclassified

CAS & Experimentation CAS & Experimentation 3 Levels of Abstraction3 Levels of Abstraction

12Unclassified

V&V & ImplicationsQA

Structured Expirmentation/T&E of Systems for Joint End-Effects

Exp/T&E of SoS boundaries - Sellers Risk

Exp/T&E with FoS continuously - Buyers Risk

V&V & ImplicationsQA

Structured Expirmentation/T&E of Systems for Joint End-Effects

Exp/T&E of SoS boundaries - Sellers Risk

Exp/T&E with FoS continuously - Buyers Risk

Complex & adaptive - Family of SystemsLots of unknowns and unknowablesMission Level PICs & ‘ilities’ proven (sic) “FOS” ‘ilities’ are rare but WITH “Mission Expectations” -> adaptability?SOA / OSA predominate – but not commonContinuous measurement of MS MOE/MOP

Complex & adaptive - Family of SystemsLots of unknowns and unknowablesMission Level PICs & ‘ilities’ proven (sic) “FOS” ‘ilities’ are rare but WITH “Mission Expectations” -> adaptability?SOA / OSA predominate – but not commonContinuous measurement of MS MOE/MOP

Complex Adaptive - Cross Capability Engineered Integrated Mission Environment-Seeking understanding and adaptability of FoS Emergent properties at Buyers Risk!

Complex Acquisitions of FOSSynchronisation

Proven Config/Role/Environment (CRE) for EACH SoS in the FoS

“Rub Points” & hence all ‘Key Interfaces’ are identified

PICS Critical

Complex Acquisitions of FOSSynchronisation

Proven Config/Role/Environment (CRE) for EACH SoS in the FoS

“Rub Points” & hence all ‘Key Interfaces’ are identified

PICS Critical

Simple - ‘Traditional’ Engineered Systems- Mature COTS & MOTS ‘Heaven / Nirvana’ at Buyers RiskSimple - ‘Traditional’ Engineered Systems- Mature COTS & MOTS ‘Heaven / Nirvana’ at Buyers Risk

V&V & ImplicationsQA checks ’things’ from Contractor

Buyer must be aware of any/all cross dependencies with COTS/MOTS support and

updates

Systems – essentially Buyers Risk

V&V & ImplicationsQA checks ’things’ from Contractor

Buyer must be aware of any/all cross dependencies with COTS/MOTS support and

updates

Systems – essentially Buyers Risk

Straightforward ‘Simple’ Standalone SystemsNo Unknowns and No Unknowables!We’re buying something ‘other’ s working systemA Collection of Managed Things (ie. 5 apples & 3 bananas)No ilities neededNo dedicated training needed (ie known by Users)‘Metaphysics rules’ & all characteristics ‘thought to be known’

Straightforward ‘Simple’ Standalone SystemsNo Unknowns and No Unknowables!We’re buying something ‘other’ s working systemA Collection of Managed Things (ie. 5 apples & 3 bananas)No ilities neededNo dedicated training needed (ie known by Users)‘Metaphysics rules’ & all characteristics ‘thought to be known’

Simple AcquisitionsRare coordination

Buy each ‘thing’ OTS

CIOG & DSG space critical in Aust

What are PICS?

Simple AcquisitionsRare coordination

Buy each ‘thing’ OTS

CIOG & DSG space critical in Aust

What are PICS?

Complicated - Bounded ‘Systems Engineered’ Capabilities- Pick and maintain a model with agreed standards to measure and control system, or any SoS emergent properties, at Sellers RiskComplicated - Bounded ‘Systems Engineered’ Capabilities- Pick and maintain a model with agreed standards to measure and control system, or any SoS emergent properties, at Sellers Risk

V&V & ImplicationsQA

Structured Exp/T&E to verify agreed ‘ilities’, Training & TTPs

SoS – essentially Sellers Risk

V&V & ImplicationsQA

Structured Exp/T&E to verify agreed ‘ilities’, Training & TTPs

SoS – essentially Sellers Risk

Complicated Systems of SystemsFew unknowns unknownsBounded ‘Things’ with controlled connectivityEngagement ‘ilities’ & PICs specified & known but no “prescriptive” Mission Level MOE / MOPUsers need Training / TTPs developed on the new and/or unique Tools

Complicated Systems of SystemsFew unknowns unknownsBounded ‘Things’ with controlled connectivityEngagement ‘ilities’ & PICs specified & known but no “prescriptive” Mission Level MOE / MOPUsers need Training / TTPs developed on the new and/or unique Tools

Typical Complicated AcquisitionsMinimal coordination

Config/Role/Environment (CRE)

‘Isolated’ with few connections

Sub-systems can be simply acquired

PICS Important

Typical Complicated AcquisitionsMinimal coordination

Config/Role/Environment (CRE)

‘Isolated’ with few connections

Sub-systems can be simply acquired

PICS Important

JAIME CODEx ‘Tests’JAIME CODEx ‘Tests’TABLE 1. JAIME CODEx FRAMEWORK / TAXONOMY Draft v 0.7

Sphere of Influence of Actor / Problems with compatibility are usually all about the interface(s) / / CoBP has to be evidence based and have a ‘whole of life’, learning view...

Scientific / Engineering / Technical Discipline

Sub-Discipline

4D: Veh, Sensor, C2, Shooter, EW, Ranges ... Time ->

Lab / Virtual & Constructive

M&S Analysis – 0.-.-

LiveSystem

Gnd Test - 1.-.-

LiveSystemFlt Test

- 2.-.-

LiveIn-service Exercise,

Trng, R&M 3.-.-

Physical –Form and Fit

-.1.-

Fit of ‘Form’-.-.1

011110 (PreProd)

111 (101)

211 flexible struct on acft

and store?311

Physical –‘Victim’ – Safety

Structural & Environment-.2.-

Loads-.-.1

021120(141)121(131)

220(252) *221 (200)

321

Vibrations-.-.2

022122

(152)222

(221)322

Aero-acoustic-.-.3

023123

(152)223

(222)323

EMC / EMI-.-.4

024124

(154)224 324

HERO-.-.5

025125

(153)225

224 EW325

Temperature-.-.6

026126

(155)226

(223)326

Noise (Aeroacoustics)-.-.7

027 127 227 327

Physical - ‘Victim’ – SafetyAerodynamics,

Aeroelasticity, Captive Carriage, Flying /Handling

Qualities & Aircraft Performance

-.3.-

Aeroelasticity-.-.1

031130 (143)131(142)

232(210)

331

Take-Off, Handling Qualities,(+ W&B) & Landing (+Carrier)

-.-.2

032132

(120)230

(251) *231 (230 & 260)

332032 W&B (132) W&B

Perf’ and Drag-.-.3

033133

(142 Update) 233

(240)333

Endurance-.-.4

? ? 234 vs 200 (253) *

Physical -‘Victim’ – Safety

Employment & Jettison-.4.-

Separation-.-.1

041140 (144) 141 (110)

F’stream & Sep Coeff?241 (271)

(291 or 293?)341

Dispense-.-.2

042142

(162)242

(272)342

Gun Firing-.-.3

043143

(161)243

(273)343

Launch-.-.4

044144

(144)244

(274)344

Jettison-.-.5

045145

(144)245

(280)345 13UnclassifiedSETE 2011 Tutty

JAIME CODEx ‘Tests’JAIME CODEx ‘Tests’

Information- Function, States, Modes of Op/Delivery, Vehicle Level /

Role/Ballistics & OFP Validation & Verification

Note needs to cover sensor, C2 and shooter...

‘Source’ Mission Critical Rx & Tx ranges – 2D / 3D at various security levels

-.5.-This is where most

MOE/MOPs should come into play

Function – Comms – C2 IntraNetworks

-.-.1051 (Lab ?) 151 (Acft) 251 351

Function – Sensor System

-.-.2052

152(102)

DT EW / EMC ? 352

Function- Shooter Role – In-range envelope V&V

-.-.3

Ball 053 153

DT 255(292)

353

OT 255 +(294)

OT

0PGMs 1PGM 2PGM 3PGM

Function – DE ??? 054 154

Other Mission ???

Range ?? Y Y OT Y

FTS ??

RCS / Noise Signatures

Cognitive –System Level

Procedures V&V and Human Factors

-.6.-

-1 061 161 261 361-32 0632 1632 2632 3632-33 0633 1633 N/A -

-34 / Tacman 0634 1634 2634 3634Msn Planning

Tools06MPS ? 16MPS ? DT & OT 256 ? 36

EBO Tools 06 EBO? 16EBO?

Operator – 1472 & NEO HF

? ? ? ?

Social – ie pure SoS / focused on

FoS & especially C2-.7.- 

MissionSOS -605th SOS – 605th SOS – 605th ?

TBMCS ... FOS? FOS? FOS ? ?...........

14UnclassifiedSETE 2011 Tutty