Life Cycle Cost Minimization & the Impact of Early Implementation of Human Systems Integration Requirements

Liana Algarín, Ph.D. Candidate, The George Washington University Thomas A. Mazzuchi, D.Sc., The George Washington University Shahram Sarkani, Ph.D., P.E., The George Washington University

Presented at Systems Engineering in DC Chantilly, VAApril 3-5, 2014

Overview1. Problem Statement 2. Motivation 3. Life Cycle of an Acquisition Program4. Human Systems Integration Requirements5. Case Study: WIN-T6. Training Burden7. Methodology8. Pilot Study9. Conclusion & Future Research

Problem Statement Incorporating Human Systems Integration (HSI)

requirements earlier in the acquisition life cycle may minimize cost. One way to predict the effect that HSI requirements will have on life cycle cost would be to identify HSI language within acquisition documents. Regression analysis using specific variables identified from the text of HSI language is a potential method for making a prediction about life cycle cost. The predictor variables will be generally categorized by time (when HSI language appeared in documentation), training recommendation and component, and specific HSI issue.

HSI has nine domains:1. Human Factors Engineering, 2. Environment, 3. Safety, 4. Occupational Health, 5. Habitability, 6. Survivability, 7. Manpower, 8. Personnel, & 9. Training

Project Manager (PM) Warfighter Information Network-Tactical (WIN-T) is a tactical communications network

“WIN-T is like the ‘Verizon’ of the Army.” Dr. Bev Knapp, Army G-1, MANPRINT

WIN-T is an ubiquitous tactical communications network that the Army will continue to implement and improve during the coming years

The Army is committed to making WIN-T work efficiently

4

Problem Statement

Motivation Primary Stakeholders

• US Army’s Manpower Personnel Integration (MANPRINT) program • Human Research and Engineering Directorate (HRED), which is one of six directorates in

the US Army Research Laboratory (ARL) MANPRINT program mission is to cut costs and prevent soldier loss during

the system development life cycle MANPRINT’s goal is for Army’s Soldiers to experience less of a burden in

both training and long-term Operations and Maintenance (O&M) My goal is to deliver new analytics that would assess the benefits (or

costs) of implementing HSI early in the life cycle Stakeholders have been wanting to examine this idea, and they would like

to have insight on how WIN-T can/could have been improved

Acquisition Program Life Cycle

Major decision points occur during Milestones A, B, and C; some requirements are not included at Milestone A but are later introduced during Milestone C

Schedule slippage and cost overruns are affected by long-term manpower planning In addition to poor planning, poor project management strategy can cause a program to fail

to meet its objectives When Systems Engineering is applied early in the life cycle, the Technology Development

Phase and Knowledge Points are benefited Function allocation early in the life cycle helps a complex work system, such as WIN-T, to be

more effective It is also beneficial to create a system value from stakeholders’ opinions and apply this system

value to the life cycle

HSI Requirements Human Systems Integration (HSI) involves incorporating people with the systems

that they use to complete various tasks When an acquisition program begins, there may not be a clear idea of what the

end user will need in the long-term However, this is not a justification for entirely leaving out HSI requirements at

Milestone A Usability defines how well a human can use a system; usability issues can be

anticipated early in the life cycle Including HSI up-front at the life cycle planning stage helps to reduce long-term

costs by means of reducing the burden of training the workers Early incorporation of HSI requirements and HSI methods can benefit a system or

program later in its life cycle HSI requirements can be successfully incorporated when the stakeholders are

open to including HSI requirements and following HSI methods and when HSI happens early in the life cycle

Case Study: WIN-T A case study with the Army’s tactical communications network, known as

Warfighter Information Network-Tactical (WIN-T), will illustrate how there have been cost overruns as a result of integrating HSI requirements late in the life cycle

WIN-T will be used at locations where Soldiers must communicate with other units

There is no common interface across all of WIN-T’s components• WIN-T combines user interfaces from several individual systems• Combining these interfaces has created some confusion for the Soldier end user• Confusion has created a heavy training burden on Soldiers

There are 4 WIN-T increments being completed/scheduled since 2004 onward• Each increment of WIN-T is expected to show improvements for usability • WIN-T Increment 2’s team is currently conducting usability analysis of WIN-T’s

components

Training Burden As more time is spent training soldiers to interact with and use WIN-T, cost

increases Some Military Occupational Specialties (MOS) have the appropriate

experience for working with the types of interfaces that are incorporated with WIN-T

However, Soldiers with different MOS may sometimes use WIN-T and find it difficult to use, particularly due to the several user interfaces

Methodology Objective is to create new analytics to measure the extent to which early

implementation of HSI requirements will affect an acquisition program Data collection will be taken from source documents provided by

stakeholders at the MANPRINT and HRED offices Potential WIN-T Source Documents

• Assessments (including MANPRINT Assessments)• Statements of Work• Analysis of Alternatives • Engineering Change Proposals • Emails • Test Reports • Meeting Minutes • Initial Capability Document• Capabilities Production Document• Capability Development Document • etc.

Pilot Study Using existing data from WIN-T Increments 1 & 2, we can make

predictions for WIN-T Increments 3 & 4 Regression analysis is a statistical method that can be used for making a

general prediction based on previously made observations The planned analysis will be a regression analysis with one Dependent

Variable and several Predictor Variables

Pilot Study Dependent Variable

• Will be defined using MANPRINT assessment scores Predictor Variables

• Time: Date in month/year (number of months into WIN-T’s acquisition life cycle)

• Time: If a document is released sometime after the events to which the document refers, such as a report about a test from a few months ago, then the date in month/year will indicate closest approximation to the appropriate time in WIN-T’s life cycle

• Training: Recommendations for training improvement (by each WIN-T component)

• HSI-Related Language: Previously selected HSI words/phrases – Selected by Expert Elicitation – Asking stakeholders what words or phrases they

think are relevant – Selected by Cluster Analysis (CA) – Do CA on a primary word that is synonymous

with HSI, such as a unique word related to any of the nine HSI domains

Conclusion & Future Research New analytics will be created to demonstrate that applying HSI

requirements earlier in the acquisition life cycle would be beneficial The method of looking for HSI language within documents and noting

when certain language was used can be repeated with any program Milestones and assessments are commonly used within systems

acquisition programs, particularly for military and weapons systems These new analytics could potentially be used to:

• increase accuracy of schedule predictions, • minimize Life Cycle Costs for hours spent on training, and • minimize component usability issues related to HSI

Questions

???

ReferencesAhram, Tareq Z., and Waldemar Karwowski. "A Framework for Human Total Ownership Cost Based on

Universal Human Performance Cost Components." Paper presented at the Proceedings of the Human Factors and Ergonomics Society 56th Annual Meeting, HFES 2012, October 22, 2012 - October 26, 2012, Boston, MA, United states, 2012.

Boehm, Barry, Ricardo Valerdi, and Eric Honour. "The Roi of Systems Engineering: Some Quantitative Results for Software-Intensive Systems." Systems Engineering 11, no. 3 (2008): 221-34.

Brinkman, W. P., R. Haakma, and D. G. Bouwhuis. "Component-Specific Usability Testing." IEEE Transactions on Systems Man and Cybernetics Part a-Systems and Humans 38, no. 5 (Sep 2008): 1143-55.

Browning, Tyson R., and Eric C. Honour. "Measuring the Life-Cycle Value of Enduring Systems." Systems Engineering 11, no. 3 (2008): 187-202.

Challenger, R., C. W. Clegg, and C. Shepherd. "Function Allocation in Complex Systems: Reframing an Old Problem." Ergonomics 56, no. 7 (Jul 2013): 1051-69.

Chatterjee, Samprit, and Ali S. Hadi. Regression Analysis by Example. Wiley Series in Probability and Statistics. Fifth ed. Hoboken, NJ: John Wiley & Sons, Inc., 2012.

Chung, Lawrence, and Kendra Cooper. "Defining Goals in a Cots-Aware Requirements Engineering Approach." Systems Engineering 7, no. 1 (2004): 61-83.

Colombi, John M., Michael E. Miller, Michael Schneider, Major Jason McGrogan, Colonel David S. Long, and John Plaga. "Predictive Mental Workload Modeling for Semiautonomous System Design: Implications for Systems of Systems." Systems Engineering 15, no. 4 (2012): 448-60.

Dacus, Chad L. "Improving Acquisition Outcomes through Simple System Technology Readiness Metrics." Defense ARJ 19, no. 4 (October 2012): 444–61.

Deitz, Daniel, Timothy J. Eveleigh, Thomas H. Holzer, and Shahryar Sarkani. "Improving Program Success through Systems Engineering Tools in Pre-Milestone B Acquisition Phase." Defense ARJ 20, no. 3 (October 2013): 283–308.

Eigbe, Arekhandia Patrick, Brian J. Sauser, and John Boardman. "Soft Systems Analysis of the Unification of Test and Evaluation and Program Management: A Study of a Federal Aviation Administration's Strategy." Systems Engineering 13, no. 3 (2010): 298-310.

Hardman, Nicholas, and John Colombi. "An Empirical Methodology for Human Integration in the Se Technical Processes." Systems Engineering 15, no. 2 (2012): 172-90.

Hart, C. G., Z. He, R. Sbragio, and N. Vlahopoulos. "An Advanced Cost Estimation Methodology for Engineering Systems." Systems Engineering 15, no. 1 (2012): 28-40.

ReferencesHorowitz, B. M., and J. H. Lambert. "Assembling Off-the-Shelf Components: 'Learn as You Go' Systems

Engineering." Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on 36, no. 2 (2006): 286-97.

Karwowski, W. "A Review of Human Factors Challenges of Complex Adaptive Systems: Discovering and Understanding Chaos in Human Performance." Human Factors 54, no. 6 (Dec 2012): 983-95.

Knapp, Dr. Bev. "Army Manprint Perspective." Paper presented at the NDIA HS Workshop, Washington, DC, 2012.

———. "Moving Manprint (Manpower Personnel Integration) Left (Mml)." 2013. ———. "Moving Manprint to the Left (Mml)." 41. Washington, DC, 2013. Lattin, James M., J. Douglas Carroll, and Paul E. Green. Analyzing Multivariate Data. Belmont, CA:

Brooks/Cole, 2003. Liu, Kevin K., Ricardo Valerdi, Donna H. Rhodes, Larry Kimm, and Alvis Headen. "The F119 Engine: A

Success Story of Human Systems Integration in Acquisition." Defense ARJ (April 2010). Madni, Azad M. "Integrating Humans with Software and Systems: Technical Challenges and a Research

Agenda." Systems Engineering 13, no. 3 (2010): 232-45. Marchant, Alan B. "Obstacles to the Flow of Requirements Verification." Systems Engineering 13, no. 1

(2010): 1-13. Patanakul, Peerasit, and Aaron Shenhar. "Exploring the Concept of Value Creation in Program Planning

and Systems Engineering Processes." Systems Engineering 13, no. 4 (2010): 340-52. Pflanz, Mark, Chris Yunker, Friedrich N. Wehrli, and Douglas Edwards. "Applying Early Systems

Engineering: Injecting Knowledge into the Capability Development Process." Defense ARJ 19, no. 4 (October 2012): 422–43.

Pharmer, James A. . "The Challenges and Opportunities of Implementing Human Systems Integration into the Navy Acquisition Process." Defense ARJ (February 2007).

Sohoni, Milind G., Ellis L. Johnson, and T. Glenn Bailey. "Long-Range Reserve Crew Manpower Planning." Manage. Sci. 50, no. 6 (2004): 724-39.

Tan, W. P., B. J. Sauser, J. E. Ramirez-Marquez, and R. B. Magnaye. "Multiobjective Optimization in Multifunction Multicapability System Development Planning." IEEE Transactions on Systems Man Cybernetics-Systems 43, no. 4 (Jul 2013): 785-800.

U. S. A. R. L. "Improved Performance Research Integration Tool." U. S. Army Research Laboratory, http://www.arl.army.mil/www/default.cfm?page=445.

U. S. Army. "Manprint Program Mission." U. S. Army, http://www.manprint.army.mil/manprint/about.html.

———. "Our Mission." U. S. Army, http://www.manprint.army.mil/directorate/about.html.

Biographies Liana Algarín is working toward a Ph.D. in Engineering Management and Systems Engineering

at The George Washington University. She is also a Human Factors Engineer at Alion Science and Technology, where she has designed user interfaces, training manuals, and user guides to support the Army and NASA. She has also conducted a critical task and usability analyses of Coast Guard cutters.

Thomas A. Mazzuchi, D.Sc., is Professor of Engineering Management and Systems Engineering at The George Washington University. His research interests include reliability, life testing design and inference, maintenance inspection policy analysis, and expert judgment in risk analysis. He served as Research Mathematician at Royal Dutch Shell, and has conducted research for the U.S. Air Force, Army, and Postal Service; for NASA, and for the Port Authority of New Orleans, among others.

Shahram Sarkani, Ph.D., P.E., is Professor of Engineering Management and Systems Engineering, and Director of EMSE Off-Campus Programs, at The George Washington University. He designs and administers graduate programs that enroll over 1,000 students across the U.S. and abroad. In over 150 technical publications and in sponsored research with NASA, NIST, NSF, AID, and Departments of Interior, Navy, and Transportation, his research has application to risk analysis, system safety, and reliability.