about the ccrp · about the ccrp the c4isr cooperative research program (ccrp) has the mission of...

About the CCRP

The C4ISR Cooperative Research Program (CCRP) hasthe mission of improving DoD’s understanding of thenational security implications of the Information Age.Focusing upon improving both the state of the art andthe state of the practice of command and control, theCCRP helps DoD take full advantage of the opportunitiesafforded by emerging technologies. The CCRP pursuesa broad program of research and analysis in informationsuperiority, information operations, command andcontrol theory, and associated operational concepts thatenable us to leverage shared awareness to improve theeffectiveness and efficiency of assigned missions. Animportant aspect of the CCRP program is its ability toserve as a bridge between the operational, technical,analytical, and educational communities. The CCRPprovides leadership for the command and controlresearch community by:

n articulating critical research issues;n working to strengthen command and control

research infrastructure;n sponsoring a series of workshops and symposia;n serving as a clearing house for command and

control related research funding; andn disseminating outreach initiatives that include the

CCRP Publication Series.

A DoD CCRP/NDU Collaboration

This collaborative effort is a continuation of the seriesof publications produced by the Center for AdvancedConcepts and Technology (ACT), which was createdas a “skunk works” with funding provided by the AssistantSecretary of Defense (C3I). The early success of ACTled to the creation of ACTIS when the president of theNational Defense University (NDU) merged theexperimental School of Information Warfare and Strategywith ACT and ASD (C3I) made the Director of ACTISthe executive agent for the DoD Command and ControlResearch Program (CCRP). ACTIS has demonstratedthe importance of having a research program focusedon the national security implications of the InformationAge and in providing the theoretical foundations forproviding DoD with information superiority, as well asthe importance of an educational program designed toacquaint senior military personnel and civilians withthese emerging issues. As a result, ACTIS’s educationalprograms are being merged with the Colleges of NDUand ACTIS’s research programs are being transitionedto OSD under the direction of ASD (C3I).

DoD C4ISR Cooperative Research Program

Senior Civilian Official and Chief Information Officer, OASD (C3I)Mr. Arthur L. Money

Director, Performance AssessmentDr. Margaret E. Myers

Director, Research and Executive Agent for CCRPDr. David S. Alberts

Opinions, conclusions, and recommendations expressed or impliedwithin are solely those of the authors. They do not necessarilyrepresent the views of the National Defense University, theDepartment of Defense, or any other U.S. Government agency.Cleared for public release; distribution unlimited.

Portions of this publication may be quoted or reprinted without furtherpermission, with credit to the Institute for National Strategic Studies,Washington, D.C. Courtesy copies of reviews would be appreciated.

Library of Congress Cataloging-in-Publication Data

Krygiel, Annette J. Behind the Wizard’s curtain : an integration environment for a system of systems / Annette J. Krygiel. p. cm. -- (CCRP publication series) Includes bibliographical references. ISBN 1-57906-018-8 1. Military telecommunication--United States. 2. United States- -Armed Forces--Communication systems. 3. Internetworking (Telecommunication) 4. Expert systems (Computer science) I. Title. UG593.K79 1999 355.6’884--dc21 98-32110 July 1999 CIP

The Institute for National Strategic Studies (INSS) is a majorcomponent of the National Defense University (NDU) that operatesunder the supervision of the President of NDU. It conducts strategicstudies for the Secretary of Defense, Chairman of the Joint Chiefsof Staff, and unified commanders in chief; supports nationalstrategic components of NDU academic programs; and providesoutreach to other governmental agencies and the broader nationalsecurity community.

The Publication Directorate of INSS publishes books, monographs,reports, and occasional papers on national security strategy, defensepolicy, and national military strategy through NDU Press that reflectthe output of NDU research and academic programs. In addition,it produces the INSS Strategic Assessment and other work approvedby the President of NDU as well as Joint Force Quarterly, aprofessional military journal published for the Chairman, JointChiefs of Staff.

The National Defense University

Behind the Wizard�s Curtain:

An Integration Environment for aSystem of Systems

Annette J. Krygiel

A Collaborative Effort Between theNational Defense University

and theDoD CCRP

Table of Contents

List of Figures .......................................................... iii

Acknowledgments .................................................... v

Preface ..................................................................... ix

Introduction............................................................... 1

Chapter 1—The System of Systems ......................... 9

Chapter 2—Systems of Systems and Federationsof Systems............................................................... 31

Chapter 3—The Two Case Studies ......................... 49

Chapter 4—The Integration Experiences ................ 99

Chapter 5—Lessons Learned on Integratinga SOS .................................................................... 143

A Lesson on Preliminaries ................................. 148

A Lesson on Continuous Integration.................. 150

A Lesson on Testing Strategy ............................ 154

A Lesson on Planning Resources ....................... 158

A Lesson on the Importance of an IntegrationFacility ............................................................... 160

A Lesson on “Who’s in Charge” ........................ 163

ii

A Lesson on Common Processes andInfrastructure ...................................................... 167

A Lesson on Efficiencies and Effectiveness ...... 170

A Lesson on Evolutionary Acquisition .............. 172

Chapter 6—Lessons Learned on Training witha SOS .................................................................... 175

A Lesson About Training Operators .................. 191

A Lesson on Infrastructure Support forTraining .............................................................. 194

A Lesson About Training the Wizard’s Team .... 199

Chapter 7—An Integration Environment for theFuture .................................................................... 203

Appendix A—The Lessons Learned ..................... A-1

Appendix B—The Integration Environment......... B-1

Appendix C—Acronym List ................................. C-1

Appendix D—Glossary of Terms ......................... D-1

Appendix E—Bibliography .................................. E-1

iii

List of Figures

Figure 2-1. Hierarchy of a System of Systems ....... 35

Figure 2-2. System of Systems (SOS) andFederation of Systems (FOS) .................................. 43

Figure 3-1. Digital Production System ................... 61

Figure 3-2. DPS Schedule ....................................... 66

Figure 3-3. DPS Management Structure ................. 70

Figure 3-4. Task Force XXI ArchitectureSchedule.................................................................. 77

Figure 3-5. Task Force XXI Tactical OperationsCenters .................................................................... 82

Figure 3-6. Task Force XXI Core Capabilitieson a Weapons Platform ........................................... 84

Figure 3-7. Task Force XXI .................................... 87

Figure 3-8. Task Force XXI ArchitectureManagement ........................................................... 91

Figure 3-9. DPS and Task Force XXIComparison (Notional) ........................................... 96

Figure 4-1. Integration of the Task Force XXISystem Architecture .............................................. 127

Figure 6-1. DPS Training Schedule ...................... 178

iv

Figure 6-2. Task Force XXI Training Schedule ..... 185

Figure 7-1. A Way Ahead ..................................... 210

v

Acknowledgments

The Honorable Emmett Paige, Jr., the formerAssistant Secretary of Defense for Command,Control, Communications, and Intelligence, has

had a powerful interest in systems integration. When Ifirst broached the prospect of my leaving a seniormanagerial role to investigate large-scale systemsintegration, he provided strong encouragement andoffered many suggestions, among them an assignmentto the National Defense University. What was thengerminating in my mind as a seed quickly took strongerroot. When I sought the advice of the Honorable JohnDeutch, for whom I had worked since he was DeputySecretary of Defense, he endorsed the National DefenseUniversity as an institution of excellence. He made myconversion to an academic happen when Lt.Gen. ErvinJ. Rokke (USAF), then President of the University,extended his hospitality. To these three I owe theopportunity that resulted in this book.

My home at the National Defense University began andended in the Institute for National Strategic Studies,directed by Dr. Hans Binnendjik. It was he who piquedmy interest in the TF XXI experiment when hesuggested that I accompany him on a trip to view theU.S. Army’s endeavors at Fort Hood, Texas. Dr. DavidS. Alberts has given unstinting and outstanding supportto this book. He not only provided the necessarysponsorship in his role as manager of the C4ISR

vi

Cooperative Research Program, but was available whenneeded, reviewed many drafts tirelessly, and providedpositive suggestions constantly.

My investigation began by reviewing how the privatesector was approaching integration on a large scale. Anumber of companies active in the defense sectoroffered both senior managers and engineers asparticipants in informal discussions with me in a(somewhat) organized agenda. Topics includedmethodology, best practices, trends in architecture andtechnology, resource implications, partnerships,customer roles, and acquisition practices. I am indebtedto the following companies for the generosity of theirtime and their insights: Booz-Allen & Hamilton Inc.,Computer Sciences Corporation, Raytheon SystemsCompany (IGS), Harris Corporation, Hughes AircraftInformation Systems, IBM, Intergraph, LockheedMartin Corporation, The Mitre Corporation, MRJ,Rational Software Corporation, Sun MicrosystemsFederal, Inc., and TASC.

I am grateful also to many participants of the DigitalProduction System program and the TF XXI programwho engaged in many dialogues with me, providedinformation, contributed documentation, answeredmany questions, and freely offered their ownperspectives of the programs. Many of theseindividuals were asked on repeated occasions to clarifyconflicting data, amplify existing material with details,and provide information when none was available fromexisting sources.

vii

My reviewers deserve a special acknowledgement. Asbusy people, they often accomplished the reading ofmy many versions on their own time. Using theirexpertise and many suggestions, I was able to makethis a better and more readable book.

Special recognition is given to Lois Burke of the NDUGraphics Department and to John McDaniel of EvidenceBased Research, Inc., for their graphics support. KarinaBraszo of the NDU Graphics Department was the creatorof the magical cover. Thanks go to Dr. Richard E. Hayesand his staff at Evidence Based Research, Inc., for theirinstrumental support in transforming the book into itsfinal edited form.

The last and ultimate words of acknowledgment arefor the National Imagery and Mapping Agency (NIMA).Although established as recently as October 1996, ithas been my parent organization for most of my career.Today it retains the vestiges of the Air Force’sAeronautical Chart and Information Center, and thoseof the Defense Mapping Agency and the CentralImagery Office, both former Department of Defenseagencies. Collectively these organizations comprised37 years of my professional life. Through NIMA’ssupport of my assignment to the National DefenseUniversity, I was able to write this book. Today and inthe future, NIMA faces the challenge of integrating theUnited States Imagery and Geospatial InformationSystem. It is my hope that this work provides practicalassistance for its team behind the Wizard�s curtain.

viii

While many sources were used to construct anunderstanding of the events that occurred during thetwo program ventures that comprise the case studies,the conclusions and recommendations are mine alone.I am responsible for errors of fact or interpretation, andthe opinions expressed in this book are solely my own.

ix

Preface

Twenty-first century concepts of operations thatpromise to transform us into an information-agemilitary are beginning to emerge from each of

the services and the joint military community. Theseconcepts share a common vision, articulated in JointVision 2010, of being able to generate greatly increasedcombat power by creating and leveraging informationsuperiority to achieve shared awareness, increased speedof command, a higher tempo of operations, greaterlethality, increased survivability, and improvedsynchronization. The concepts also share a commonassumption: the infostructure necessary to support theseoperational concepts will be there when needed.

To transform these emerging concepts into realoperational capabilities will require much work tomigrate the current collection of legacy systems infusedand augmented by new and emerging capabilities intoa coherent infostructure. Lack of interoperability andsecurity are arguably the two most critical areas ofshortfall. They are among our most vexing andpersistent problems, and constitute the two largestobstacles we need to overcome. In some cases we willbe able to exert the kind of leadership and impose thekind of standardization required to forge a system ofsystems and achieve the level of holistic behavior thatis implicit in this term. In other cases we will find thatwe need to take a federated approach to developing the

x

necessary degree of coherence. The extent to which asystem of systems approach will be practical and thedegree of success we will achieve with a federatedapproach remain to be seen. Our success will dependin large part on the degree of delivered interoperabilityand security.

This will not be an easy journey; it will be easier if we,as a community, develop a better understanding of thechallenges involved, the problems encounteredpreviously, and the actions needed for success. This isnot just a technical problem that systems developerscan solve. The problem encompasses moving beyonda collection of largely independent and stove-pipedsystems to a coherent infostructure capable ofsupporting emerging operational concepts. The taskbegins with how we think about enterprise processesand the information necessary to support them, and notjust with individual systems. It includes how we developfuture architectures and design and build participatingsystems to generate and use information in support ofvarious missions and tasks. It extends to how we makeinfostructure investment decisions and how we thinkabout achieving desired behaviors.

As Dr. Krygiel points out, there will be many mission-oriented systems of systems or federations of systemsthat share common information, computing, andcommunications resources. For each one of these it willbe the end-to-end flow of information from varioussources to its many uses that will need to drive the trainrather than just the component processes along the way.The distinctions between the business side of the

xi

Department of Defense and its warfighting side alreadyare beginning to disappear as more information is madeavailable to warfighters in a timely manner. This is trulya fundamental shift in the way we think about and dothings today and it will require cultural change. Suchsignificant changes will occur only if preceded bywidespread understanding and a vigorous program ofexperimentation. Here, too, is the rub. It is hard to dothe kind of discovery experiments necessary to reallyexplore the opportunities to do business differentlyunless you can imagine interactions that cannot takeplace today. It is hard to imagine what is possiblewithout some hands-on experience, but to get thisnecessary experience requires at least a critical mass ofthe systems of systems be available. We must ensurethat our efforts at experimentation are well supportedby an experimental infostructure.

Dr. Krygiel has performed a valuable service to thecommunity by analyzing two very significant programs,each of which involved a system of systems. Her focusis on their integration efforts to extract the nuggets weneed to develop a better understanding of the tasksahead. We need to work together to ensure that theselessons recorded become lessons learned. As will beclear to her readers, there is still much to do and moreto learn and understand about developing and fieldingan effective and durable infostructure as a foundationfor the twenty-first century. Without successfullyfielding systems of systems, we will not be able toimplement emerging concepts in adaptive and agilecommand and control, nor will we reap the potential

xii

benefits of network centric warfare. The C4ISRCooperative Research Program (CCRP) will continueto pursue this line of research and to seek outcollaborators to improve our understanding of thisimportant area.

David S. AlbertsDirector, Research OASD (C3I)

1

Introduction

Our U.S. defense strategy seeks new levels ofeffectiveness by harnessing the power ofadvanced technologies, particularly those of

information technologies. A central premise to futuremilitary strategy is the formation of a system of systems(SOS) to attain dominant battlespace knowledge. Bycoalescing data from collection and processingsystems, the resulting information can be integratedwith systems of weaponry and warriors for a seamlesssensor-to-shooter flow. Linking these with thecapabilities of maneuver, strike, logistics, andprotection will allow decision makers at every levelto respond significantly faster than any adversary andin any operational situation.

Because a SOS is necessary to realize such powerfuleffects, I believe the integration environment to formsuch an entity should merit considerable focus. Havingbeen a program manager for the integration of arelatively small number of individual systems, I findthe achievement of this much larger venture a dauntingchallenge, and not one to which I would automaticallyascribe success.

I had the opportunity to observe the appropriate levelof attention on such processes as the U.S. Armylabored to integrate hundreds of information systemsto support a major warfighting experiment—Task

2 Behind the Wizard’s Curtain

Force XXI (TF XXI). The Central Technical SupportFacility at Fort Hood, where the integration wasoccurring, resembled a crisis operations center. Thewalls were papered with configuration item drawings,the boards covered with problem assessments.Engineers and developers rushed in a frenzy ofactivity to correct and change hardware and software.Meantime, hundreds of soldiers trained on theintegrated product around the clock, occasionallyexpressing frustration with systems that did not workas intended or required. In short, it appeared so likethe integration and training environment that had beenused during my own program experience that I wasimmediately intrigued with the similarities.

Also analogous was the conviction demonstrated bythe participants that the integration of multipleinformation systems is indeed challenging. Today inthe era of the Internet and advertised interoperability,we are undermined by the expectation that properlyengineered systems will plug-and-play and snap-in,snap-out. Instead integration is more typically astrenuous and complex undertaking to obtain the resultsintended. This is an era of chaos and non-linearitytheory, and the relationships between networkedinformation systems increasingly are described in termsof biological phenomena—unpredictable phenomena,at that.

This book then emerged from two case studies, thecontinuing analysis of the Army’s TF XXI, andrevisiting the Digital Production System (DPS) program

3Introduction

with which I had been associated for many years whileat the (then) Defense Mapping Agency.

This work is directed at recovering successful strategiesand determining an environment that supports not onlythe integration of a SOS but its use for operationaltraining. I characterize the activities that transpire asoccurring behind the Wizard�s curtain, a referenceto L. Frank Baum’s (1900, 1903) wonderful book, TheWizard of Oz . The reader may recall that the powerfulwizard is finally revealed to the heroine Dorothy andher companions as a mere mortal, laboring behind thescreen to produce the magic effects. This is anappropriate simile because it requires skilled peoplearmed with modern arts to achieve the technologicalmagic required to integrate and sustain a SOS. It is thepeople and the processes and infrastructure they usethat this work examines.

Why my focus on the integration phase? The answer ismany-faceted. It is true that the entire life cycle of aSOS merits attention, and this need is paramount todeliver the capabilities needed for the future defensestrategy. More on this theme is covered in the sectionof this book on future work. The integration processand its associated environment do not provide anarchitecture or offer an alternative to a good design. Sowhy emphasize integration? One reason is that it canbe used effectively as an adjunct to the requirementsand design processes if accomplished early enough inthe life cycle. The U.S. Army demonstrated this withTF XXI.


I view the integration phase as the last certainopportunity to deliver an integrated product before itsdeployment for operations. Frequently it will benecessary to assemble such a capability despiteinadequate previous processes—and using systemsdeveloped and operated for other and different purposes.Circumstances are not always optimum. Still, theintegration process and environment, if sufficientlyrobust, can be used to overcome some disadvantages.They are certainly necessary to garner sufficient qualityin the product no matter how robust the requirements,architecture, and design processes; otherwise, theoperational community will suffer the impacts whileattempting to accomplish its mission.

The Road Taken Through This BookThe road taken in exploring the topic of integrationtraverses many subjects: Joint Vision 2010, the defenseenterprise, Greek philosophy, characteristics of a SOS,digital mapping, battlefield digitization, operationaltraining, and recent experiences in Bosnia.

This book begins with a brief look at a SOS from theviewpoint of the U.S defense strategy and theframework of Joint Vision 2010. The intent is to providean impression of SOS scope and the level ofexpectations for its operational support. There is notone SOS that is a one-time new start, as a singleinformation system project. Rather a SOS moretypically will be assembled from shared reusablecomponents and with many existing systemsindependently developed for other and various missions.

5Introduction

Past problems with the interoperability of informationsystems have resulted in several strategies that arenecessary and relevant to achieve the integration of aSOS. These are reviewed briefly, but the key questionis whether they are sufficient of themselves. The answeris no. Current operations demonstrate this, as do thetwo case studies. The chapter also examines the natureof change and the consequences on a SOS. A principleis developed from the Greek philosopher Heraclitus:“You can never experience the same SOS twice.” Thereis not one SOS but many SOSs. This arises from theneed to use particular systems for different missionsand the rate of change of circumstances and technology.As a result, the process of integrating a SOS occursmany times.

Chapter 2 poses the question: “What is a SOS?”Unfortunately there are no commonly accepteddefinitions. An answer is provided by building upon afew basic definitions. A SOS has constituent systems,which themselves are large-scale systems. There aredifferent kinds of SOS, and the discussion of one typeintroduces the concept of a federation of systems (FOS).Coalition activities more characteristically require aFOS capability. The implications of a FOS in the JointVision 2010 era are not immediately assessed. Ratherthey are deferred until after the discussions of the SOSintegrations accomplished during the two programventures examined in this book.

The two case studies are the core of the book. Bothprograms, the DPS and TF XXI, produced an integratedproduct characterized as a SOS. Both were ventures


that applied revolutionary changes in operationalconcepts through advanced information technology.Although different methodologies were used to developtheir architectures, they followed similar strategies forintegration. Their overviews are provided in chapter 3,and the integration experiences are described in detailin chapter 4. The two approaches are deliberatelyjuxtaposed to illustrate the high degree of correlation,and there are more similarities than differences. Despiteextensive preparation for integration, both the DMAand Army managers initially found themselves in realdifficulties. Both recovered and moved on. Theirexperiences are worth telling and worth reading.

In chapter 5 the conclusions from this examination takethe form of nine lessons learned about integration. Theseprovide practical strategies for the team behind theWizard�s curtain to achieve a successful SOSintegration. The lessons learned are fundamental andsupplement good practices for program managers. Useof a structured integration environment in a singlefacility enables success—and with more effectivenessand efficiency. It also results in a more robust integratedproduct. Not the least of the lessons deals with thenecessity for skilled people properly empowered,staffed, trained, and chartered within the acquisitionprocess to accomplish the integration of a SOS.

Three additional lessons learned are described in chapter6. These differ from those related to integration andapply to operational training on a SOS. Both DMA andthe Army prepared their operators by training them fortheir missions on the SOS being integrated behind the

7Introduction

Wizard�s curtain. What was demonstrated was the needfor more and iterative training with a SOS. There isalso a requirement to adapt training materials withconsiderably more exposition of the SOS capabilities,in addition to providing the operator with informationabout his or her individual system.

Considerable efforts were required to integrate the DPSand the TF XXI SOS, yet they are simple in comparisonto the demands of the era of Joint Vision 2010. Becausea FOS will be required for many missions, refinementsor differences in the strategy for integration will beneeded. The final chapter explores these topics andconcludes that future experimentation should continueassessments of SOS and FOS activities behind theWizard�s curtain. Considering that future operations willbe joint and coalition, integration will require processescharacterized by increased collaboration. The requiredcollection of systems will reflect a greater diversity.Processes will need to bridge many different cultures.

The lessons learned provide a foundation upon whichto build. Several recommendations are provided in thefinal chapter, including the use of an integrationenvironment. As defined, an integration environmentdescribes who and what should appear behind theWizard�s curtain. Still, the more complex futuredemands continued investigation of other strategies andpractices as well. Specific work is proposed includingthe analyses of more case studies. The extensiveexperimentation to implement Joint Vision 2010planned for the decade ahead provides the opportunityfor these additional assessments and investigations.

9

Chapter 1

The System of Systems

The U.S. Defense Strategy and theSystem of Systems

The National Security Strategy relies on the U.S.military to play an essential role in ways thatprotect and promote U.S. interests (The White

House, 1998). Accordingly the future military strategyis proceeding in consonance with the concepts statedin Joint Vision 2010 (Joint Force Quarterly, 1996;Concept for Future Joint Operations, Expanding JointVision 2010, 1997). Among the important precepts thatcomprise this powerful framework is that ofinformation superiority:

—the capability to collect, process, anddisseminate an uninterrupted flow of informationwhile exploiting or denying an adversary’sability to do the same.1

From a position of information dominance, the strategyseeks a powerful and seamless sensor-to-shooter flow.We will be in a vastly improved position to “see” ourenemies, “decide” on a course of action, and

1 As defined in Joint Vision 2010.


subsequently “destroy” or “influence,” whichever isconsistent with our national objectives. We will succeedby using information technology as a strong enablerfor our decision-makers at every level and in everyoperational situation.

Central to this strategy is the formation of a “system ofsystems” (SOS) to achieve dominant battle spaceknowledge. This is a concept renewed and expandedby Admiral William Owens while serving as ViceChairman of the Joint Chiefs of Staff (JCS) (Owens,1996). He noted the superior technologies emerging inthree areas:

• those of sensors for intelligence, surveillance, andreconnaissance (ISR)

• those computer processing capabilities supportingcommand, control, communications, andintelligence (C4I), and

• those surrounding and supporting precisionweapons.

By coalescing the data from systems being developedin the collection and processing domains, we canrealize a significant awareness and knowledge of thebattlespace unable to be achieved previously. We canextend this strategy to integrate further with thesystems of weaponry and warriors to achieve thatseamless sensor-to-shooter flow so desired. And then,coupled with emerging technology, we can transformand link with the systems of maneuver, strike,logistics, and protection.

11Chapter 1

Among the essential components to implement thisstrategy is that of a capable network sufficient tosupport the aggregation of all these systems.2 Butultimately, it is the integration of these manyindividual systems that will provide the means tocollect, evaluate, and deliver the information neededto support the decision process and enable asignificantly faster response.

Secretary of Defense William Cohen (1997), in buildingon the President’s National Security Strategy, adoptedthe Joint Vision 2010 plan as the template for militaryoperations of the future. Contained within Joint Vision2010 are four operational concepts that, in the aggregate,are intended to provide the capability to dominate anyadversary and control any situation. These are dominantmaneuver, precision engagement, focused logistics, andfull-dimensional protection. Certain key expectationsabout a SOS emerge from this framework and itspowerful concepts, as follows:

• Joint Vision 2010—The future defense strategyrequires that U.S. forces support a full spectrumof operations ranging from humanitarianassistance and peacekeeping to high intensityconflict. These vary in scale from smallcontingencies to major theater events of warfare.Coalition operations are essential to protect,

2 A system is broadly characterized here as any sensor, platform,or weapon (in addition to the more traditional computer ornetwork) through which bits flow and which can be connectedto a network. As such, it can be embedded in a sensor and it canbe strapped to a soldier. It can accommodate many purposes.


promote, and conduct our national interestsinternationally and to achieve global reach.Therefore collaborating with coalition allies—many of them in partnerships that are neithertested nor experienced before—becomes animportant element of the strategy.

• Dominant Maneuver—Forces will be widelydispersed as joint air, land, sea, and space assets.There will be fewer overseas points from which tolaunch forces. Even today, forces have beenmigrating from a forward deployment toward onebased in the continental United States. Moreextensive maneuverability will be required toaccommodate force projections over strategicdistances than ever before. Dispersion necessitatesbroader and more rapid collaborations from allassets and elements of the forces—and to massgreater effects. And rapid re-dispersion of forcesis required to cover concurrent operationaldemands. This has implications on all levels, ontactical as well as on joint operations.Maneuverability also requires an evolution oforganizations to become more agile and versatile.

• Focused Logistics—With the need for deployingand sustaining more expeditionary-likeoperations as well as to mass effects, the need toaccurately locate, track, assess, and transportassets across geographic regions is important foragility. Information technology promises toachieve this without skewing the “tooth-to-tail”ratio unduly toward support. Tailoring logistics

13Chapter 1

to the needs of a specific mission and positioningsupplies as needed are both desirable objectives.Nevertheless, “just-in-time” provisioning mustbe balanced carefully against “just-in-case”provisioning, with a view to the consequencesand risks.

• Precision Engagement—Precision engagementwill be broadly applied. The future will providemore capable weaponry, real-time information oftargets, situational awareness of friendly andunfriendly forces, more lethal munitions, andincreasingly precise delivery against the objective.Future performance will be assessed by anexternal environment with expectations set at zerocollateral damage and no fratricide. The capabilityto engage precisely and react accordingly willevolve to the level of an individual combatant;this requires a ubiquitous integration across alllevels of an operation.

• Full Dimensional Protection—The future bringsnew threats, and therefore increasedvulnerabilities. Consequently, there will be a needfor increased protection. Collection assets ofsurveillance, reconnaissance, and intelligence willbe used to detect and track attacks fromconventional and unconventional means. Newsensors for the characterization and detection ofchemical and biological warfare agents arerequired to deal with these threats, increasinglyused by rogue nations and terrorist groups.Protection of the forces is paramount. Protection


of the information infrastructure,3 upon whichthese future operational forces will rely soextensively, is also necessary.

Operational Expectations of a SOS

What do these strategies and concepts demand of aSOS? A few declarative statements synopsize theoperational expectations:

• A SOS must support an operational temposubstantially increased from that of today.

• The individual systems of a SOS must beconfigured and linked to sustain both strategic andtactical level activities, accessed and applied by atheater force as well as a highly mobileexpeditionary unit.

• A SOS must reach widely dispersed sites aroundthe globe.

• A SOS must encompass the systems providinginformation superiority with those of commandand control, with weaponry and engagement, andwith those supporting maneuver, strike,protection, and logistics.

• A SOS must be appropriately secure and protected.

3 The term “infrastructure” is broadly inclusive. It comprisesthe underlying computers, communications, organizational base,people, and processes to support the function specified by thecontext.

15Chapter 1

• A SOS must be joint, linking service andagency assets.

• A SOS must support coalition operations,connecting to the systems of international partnersto the appropriate extent.

These expectations indicate a substantial scope for aSOS, and an integration of systems beyond anythingpreviously attempted or achieved.

The Journey to the System of SystemsThe SOS is not just a futuristic concept initiated byJoint Vision 2010 and the potential of informationtechnology. In one sense, the Department of Defense(DoD) has been on a journey to evolve something likea SOS for some time, although not of such breadth,scale, coherence, or level of connectivity andinteroperability.4 It has long been recognized thatexchanging information and sharing services betweenand among information systems is important. However,experiences during Operations Desert Shield /DesertStorm revealed serious deficiencies in the ability to dothis. Shortfalls even resulted in the fratricide of friendlytroops (Stanley, 1998).

In reaction to the severity of interoperability problemsin the Desert War, a program, “C4I for the Warrior,”was initiated by the JCS. This was structured into several

4 Interoperability is the ability of two or more systems orcomponents to exchange and use information (IEEE Standard610.12, 1990).


phases and moved in the direction of applying acommon set of standards to improve interoperability.The program also promoted a transition from militarystandards to commercial standards (Starr, 1996).

A defense enterprise architecture is being evolved.Standards and guidelines for the enterprise facilitateinteroperability. An agreed-to subset of the standardscomprises the current version of the Joint TechnicalArchitecture (JTA)5 (DoD Joint Technical Architecture,1998). Compliance with the JTA is consideredmandatory. With the multitude of systems beingacquired, enhanced, and maintained, efforts have beenincreased to ensure compliance—including oversightof acquisitions, the establishment of a Chief InformationOfficer6 in each defense organization, and the expansionof certification testing.

The growth of common services is also a fundamentalstrategy to foster interoperability within the defenseenterprise. The Defense Information Infrastructure (DII)comprises networks, computers, software, data bases,applications, interfaces, and services, and it provides acommon operating environment (COE). It incorporateskey joint systems that provide significant functionalitycommon for all users. Two prominent examples of theseare the Global Command and Control System (GCCS)and the Defense Information System Network, both in

5 The first version focused on C4I systems. The second versionexpanded into domains such as combat support, weapon systems,and modeling and simulation.6 This resulted from the Clinger–Cohen Act of 1996 (formerlythe Information Technology Management Reform Act), PL 104–106.

17Chapter 1

operation. The GCCS is a C4I system that providescommon services and a shared data environment,accessible by information systems that fit within theoverall architecture (Butler, Diskin, Howes, and Jordan,1996; DII Master Plan, 1998). When service-uniqueextensions of GCCS must be acquired, they comprisean adjunct to the joint, common capability. Thiscontrasts with past practices of proceeding with separateand sometimes disparate acquisitions by the servicesand agencies.

The defense enterprise will evolve as its variouscomponents evolve. It provides an actual foundationfor a future SOS. A SOS is rarely a one-time new start,but rather it will build on common capabilities such asthe GCCS. A SOS will include legacy systems. It alsowill incorporate new capabilities, many specificallydeveloped to implement the Joint Vision 2010.7

Legacy Systems and the SOS

A unique factor that compounds the challenge ofassembling a SOS from existing systems is the sheernumber of legacy systems in the DoD. These systemswere developed without benefit of the enterprisearchitecture definition and before the JTA was defined.

7 A task force identified at least 32 critical functional capabilitiesrequired for Joint Vision 2010 and conceived a future globalSOS spanning all levels, strategic to tactical. The results weredocumented in the Advanced Battlespace Information System(ABIS) Task Force Report (1996). The roadmap for science andtechnology initially provided has since been refined to mesh thenew with current capabilities to plan assimilation incrementally(Joint Warfighting Science and Technology Plan, 1998).


Many do not even use modern digital technology andwere developed for standalone environments.

Using legacy systems to support missions will continueindefinitely. Defense downsizing, the enormous fundingrequired for recapitalization, as well as the time requiredfor migration all compound their duration. Somesystems take as many as 10 to 15 years to acquire, andtypically remain in the inventory between 20 to 40years.8 Legacy systems are challenging for the privatesector as well, but the length of the acquisition cycle9

and the number and age of systems in the inventory areprobably unique to DoD.

With the rate of change in technology, today’sdevelopmental systems become tomorrow’s legacysystems. They add difficulty to constructing a SOSbecause generally they are not compliant with thethen-current version of the defense architecture.Interoperability with them is problematic. Differencesmust be reconciled with those of systems that fitwithin the overall enterprise framework. Legacysystems must be migrated to a state of compliancethrough enhancements.

8 Former Secretary of Defense William Perry, at the 1997Association for Computing Machinery (ACM) conference, “TheNext 50 Years of Computing.”9 DoD continues to streamline the acquisition process bymoderating requirements for oversight, processes, anddocumentation based on the risk of a specific acquisition.

19Chapter 1

The Journey to a SOS—Emulating theCommercial Sector

The DoD strategy to achieve interoperability is adoptingmany processes and methods that have provensuccessful in the commercial information technologysector (Schaeffer, 1998). An architecture should becomprised of systems that are “open”10 as well asmodular, with various components that can be reusedin future evolutions.

Emulation of successful business practices of the privatesector has resulted in some streamlining of defenseacquisition processes and wider application ofcommercial engineering, design, and developmentprocesses. It also has brought the greater use andleveraging of commercial technology and commercialservices in lieu of specialized components and uniqueelements. While commercial products have all the meritof reduced development expenditures, these bringadditional advantages for achieving openness, and,therefore, interoperability. There is evidence that thecommercial sector of the information technologydomain has been marching smartly toward open

10 An open system is defined as “a system that implementssufficient open specifications for interfaces, services, andsupporting formats to enable properly engineered applicationssoftware: (1) to be ported with minimal changes across a widerange of systems, (2) to interoperate with other applications onlocal and remote systems, and (3) to interact with users in astyle that facilitates user portability” (DISA DII Master Plan,1998).


architectures for some time11 12 because open systemsprovide economic benefits.

Consequently commercial products and services willconstitute a larger percentage of the future defensearchitecture (and a SOS). Industry standards comprisedmore than 60 percent of those in the initial version ofthe JTA and nearly 70 percent of the second version.Subsequent versions increasingly will rely oncommercial enterprises and international standards.

The Reality CheckWith the current migration toward an enterprise, theemphasis on standardization, common components, andthe expanded use of commercial products and services,the questions are: Are they sufficient to achieve afunctioning SOS? And given the conceptual frameworkof Joint Vision 2010, will they remain sufficient?

What is learned from current operations such as inBosnia and from joint and service exercises anddemonstrations is that interoperability between systemsand the integration of multiple information systemscontinues to be difficult. A good synopsis of problems

11 In an interview with Kevin Kelly, John Hagel discussed hisbook Net Gain and noted that between 1985 and 1990 there wasa huge redistribution of shareholder value between companiesthat were owners of proprietary architectures and those that werechampioning open architectures (Hagel, 1997).12 Kevin Kelly (1997) noted the phenomenon of open systemsand common standards emerging to maximize the potential ofnetwork infrastructure as Rule 8 of the “New Rules of the NewEconomy.”

21Chapter 1

and causes connected with the lack of interoperabilityis provided in the Commission on Physical Sciences,Mathematics, and Applications, “Realizing the Potentialof C4I: Fundamental Challenges” (1999). The strategiesfollowed, when viewed as a complete solution, arenecessary but not sufficient. The case studies discussedwithin this work will illustrate this also.

Larry Wentz’s summary of lessons derived fromOperation Joint Endeavor in Bosnia provides insightinto the challenges and difficulties of achieving a SOS(Wentz, 1998). To succeed with the integration ofcommunications and information systems (CIS) tosupport the implementation force (IFOR), he recountsthe following:

The challenge facing NATO and the nations wasto build a long haul and regional CIS networkout of a mixture of military and commercialequipment that would vary widely in age,standards, and technology and would be builtvery quickly once given the order to deploy.Putting the pieces of the puzzle together wouldmost likely not result in a true ‘system of systems’for IFOR. Furthermore, there would be a needto interface systems that had not been plannedor designed for interfacing. The independentnational systems would be tied together, notengineered as a single system. Given theuncertainty of the situation it would most likelybe a case of integrating what you get, notnecessarily what you need, and then making thebest of it. (Wentz, 280–282)


Other ventures outside of defense also indicate thechallenge. Unlike Fermat’s last theorem,13 the proof ofthese difficulties is not left entirely as an exercise forthe reader. There is a rich body of literature availableon the challenge of succeeding (and failing) withcomplex information systems. The Standish Group(1995) undertook a survey called CHAOS oninformation technology projects in the commercialsector. Results were grim. Generally, more than 30percent of projects were canceled before completion;more than 50 percent cost nearly twice their originalestimates; and only 16 percent were completed on timeand within budget.

Capers Jones has characterized the difficulties insucceeding with software intensive information projectsas a function of their size, as characterized by functionpoints (Jones 1996a, 1996b, 1996c). These provide unitsof measure for software size using logical functions asan alternative to lines of code.14 Only 14 percent ofprojects that exceed 100,000 function points aredelivered on time. Risk rises dramatically from the10,000 to 100,000 function point range and higher.

Of the two case studies discussed in this book, one isequivalent to 100,000 function points, and the other to

13 Pierre Fermat was a French mathematician who, in proposinga new assertion, said “I have a truly marvelous demonstrationof this proposition, which this margin is too narrow to contain.”Three and one half centuries lapsed before anyone could discoverthe demonstration.14 To do rigorous function point analysis of a software applicationrequires inputs, outputs, inquiries, logical files, and interfaces.

23Chapter 1

10 times that.15 The two programs are more complexthan the software project of a single system, but thissimple comparison provides a lower bound on difficulty.This theme is developed later in this work. And whilethese endeavors are challenging, even morecomprehensive capabilities will be required to supportthe Joint Vision 2010 era.

Learning about the System ofSystems from HeraclitusThe Greek philosopher Heraclitus observed in the fifthcentury B.C. that change was continuous (de Laguna,1921). His philosophy is epitomized by the line:

You can never step into the same river twice.

The Joint Vision 2010 era must deal with continuouschange. Many capabilities in a SOS are founded oninformation technology, which itself is an acceleratingphenomenon.16

Adapting to the fast pace of technological change hassignificant implications. Accommodating change itselfbecomes a primary requirement for a SOS.

Technological innovation is not limited to friendlyforces, but rather it is proliferated globally. High-tech15 Based on lines of code estimates of approximately 10,000,000for DPS and 30,000,000–40,000,000 for TF XXI (and using 100lines of code per function point).16 For example, W. Brian Arthur compared the rate of technologyevolution to biology evolution. He concluded that: “…technologyis evolving at roughly 10 million times the speed of naturalevolution”(Arthur, 1997).


weaponry is available in the marketplaces of the world,as are commercial communications, navigation, andtransportation assets. The net result is a dynamicsituation with respect to the capabilities of adversariesand, therefore, the overall threat.

Asymmetric threats to the United States can arisethrough the manipulation of information technology byunscrupulous nations or groups with otherwise inferiorassets. While technological superiority is onecornerstone of the national military strategy, fastertechnology insertion is needed for the United States toretain its competitive edge. Accommodating rapidtechnological turnover has become a strategic necessity.

The pace of innovation also will require fasteradaptation of the enterprise framework and an increasedpace for migration of legacy systems. Strategies to dealwith rapid change such as the reliance on commercialtechnology are anticipated to ease the accommodation.However, the revolution in the information technologyteaches that increased capabilities lead to increasedrequirements and increased complexity. The ChiefTechnology Officer of Microsoft observed that softwareis limited only by human expectations:

The software crisis is perpetual because thebenefits of panacea solutions are absorbed byrising expectations. The real driver isexpectations.17

17 “The Next 50 Years of Software,” by Dr. Nathan Myrhvold,1997 Association for Computing Machinary conference, “TheNext 50 Years of Computing.”

25Chapter 1

The global environments also are changing rapidly.With an increasingly wider set of players on the world’sstage, more diverse geopolitical environments areanticipated. Alterations in economic, technical, societal,religious, cultural, and physical conditions will occur.The U.S. defense strategy is based on the reality of livingin a dangerous, uncertain, and unpredictable worldwhere rogue nations can use asymmetrical strategiessuch as terrorism and weapons of mass destruction withdevastating consequences. The net result of thesecircumstances is flux, a spiral of altering operationalconcepts, revised strategies and doctrine, andorganizational adaptations, all continuouslyaccommodating diverse and exceptional circumstancesin a changing world. These changes, in turn coupledwith technological innovation, drive the need for moreadaptation of capabilities. These circumstances areinterrelated, linked, and coevolved. The net result is astate characterized by continuous change.

Considering the circumstances surrounding the SOS, a“Heraclitan principle” is derived by paraphrasing:

You can never experience the same SOS twice.

Different Systems of Systems for Different Missions

This Heraclitan principle is confirmed when consideringthat, as in the past, the future will require that U.S. forcesengage in distinct operations with differentcombinations of systems. These differences arise basedon the specific type of mission, the nature of theoperational environment, the duration of the mission,


as well as many other factors. Many components doremain the same, but others vary, some unique for thegeopolitical environment. Humanitarian assistance inSomalia requires certain capabilities that are differentthan those for a desert war in Iraq.

Peacekeeping is a mission different than that ofconflict and involves a significantly differentapproach to command and control and decisionmaking. Operation Joint Endeavor is characterizedas an operation other than war (OOTW). Missionrequirements for such operations, when contrastedwith those of major conflicts like Operations DesertShield/ Desert Storm, demand different combinationsof systems.

The accounts of peacekeeping in Bosnia are illustrative.For Operation Joint Endeavor, a unique situation arosebecause NATO was out of its normal area, and becausea large number of countries that had never workedtogether collaborated. These circumstances lead to anextraordinary command and control structure, andconsequently to extraordinary combinations ofcommand and control systems (Layton, 1998).

Weather and time of day are also among factors thatcontribute to differences in capabilities for differentmissions. They establish conditions whereby onereconnaissance information system is moreadvantageous than another. For example, electro-opticalimaging sensors cannot penetrate through clouds or thenight so that other types of sensors are used. Terrain isa factor that distinguishes the selection of surveillancecapabilities. In Bosnia, extensive terrain masking and

27Chapter 1

inadequate resolution initially precluded extensive useof the Joint Surveillance Target Attack Radar System(JSTARS), which was better adapted to detectingopposing wartime force movements rather thandistinguishing intertwined friendly and unfriendlyforces. Later when there was increased freedom ofmovement, it was used for tracking military vehicles inconjunction with other assets (Wentz, p.102).

Participants in actual operations noted these variationsin suitability for OOTW missions:

Systems that work in deserts may be useless injungles, forests, or urban centers. Tools that aresafe in open areas may have unacceptableconsequences in crowded areas. Where theimmediate threat is low, technologies that workslowly or require detailed preparation areuseful, but they cannot help in urgent situations.

Complicating the process further is the fact thattechnical requirements vary with the location,type of operation, and the time available forapplication…. In Desert Storm, for example,soldiers reported that they could spot buriedmines using night vision devices. While thisworked in that desert environment, it does notwork in forest or jungle areas. Likewise,technologies that work in fields may not work inhills and probably won’t work in urbanenvironments. (Alberts, 1995)

“Stepping into a SOS” could be an experience of veryshort duration. A particular SOS may be configured and


used for a period of days or weeks to support a mission-transient operation. Other combinations of systems maybe integrated and sustained for longer periods of time.During that time individual components or systems mayundergo adaptation, proceeding through many versions.For prolonged operations, such as in Bosnia, even thecommon framework capabilities may evolve. Also, newadvanced capabilities may be introduced at any time toprovide an operational advantage.

The U.S. forces are expected to conduct multipleoperations concurrently. This implies sustaining variouscombinations of systems concurrently.

Behind the Wizard�s CurtainConsiderable attention will remain focused on theimplementation of Joint Vision 2010 in the decadeahead. Changes to personnel, organizations, anddoctrine will be evaluated and addressed. Advances intechnology will be used to achieve the capabilitiesrequired. The SOS so central to the defense strategymust support the operational expectations. However,the SOS is an integrated product for operationalmissions. An integration process will be needed, notjust one time and not just for a single product. Therewill be many integrations, and many combinations ofsystems integrated, and a continuum of integrations,depending on the different objectives intended and theduration of missions.

To support each operational activity that must play oncenter stage, another production also must play. That

29Chapter 1

production is the orchestration of people, processes, andthe information systems behind the Wizard�s curtainto provide the technological magic so necessary for themission’s success. While these activities are not at thecenter and front of the stage, they are essential.

It can be argued that with sufficient prescience,anticipation, warning, and resources, the Wizard’s teamwill deliver the required SOS. A modicum ofintegration of systems was achieved for OperationsDesert Shield/Desert Storm, although manyparticipants would consider this a stretch. In Bosnia,with sufficient lead-time, many systems wereintegrated, including many with advancedtechnologies. On the other hand, by imposingconventional limits on resources and time, or withoutsufficient warning, the rapid integration of largenumbers of systems is well beyond our current abilitiesto achieve today. Compounding the difficulty is thefact that the nature of today’s threats results inresponses that often combine forces and systems inunanticipated ways. Among the most formidablechallenges to realizing the promise inherent in thefuture defense strategy is the integration of informationsystems, a process based on interoperability. It meritsconsiderable attention.

31

Chapter 2

Systems of Systems andFederations of Systems

What is a system of systems (SOS)? This shortchapter develops an answer by building upona few basic definitions. There are distinctions

between a single system and a SOS. A commonunderstanding is necessary to appreciate the descriptionsand analyses of the two case studies provided insubsequent chapters.

The concept of a federation of systems also is discussed.It is representative of the capabilities required for theJoint Vision 2010 era and its discussion provides acontext for integration methods needed for the future.

The terms and definitions are listed in Appendix D,Glossary of Terms.

Systems and Systems of SystemsThe concept of a “system of systems” has been used invarious sciences for many years. A 1964 paper on NewYork City refers to “cities as systems within systems ofcities” (Berry, 1964). The social, biological, as well asthe physical sciences make use of the concept. However,


as applied to information systems, there is no widelyaccepted definition or agreement on how a SOS differsfrom more conventional systems.

The term “system” has more universal acceptance.Eberhardt Rechtin and Mark Maier (1997), who havedeveloped the art of architecting systems, have provideda definition which would be recognizable in manysciences:

A system is defined as a set of different elementsso connected or related as to perform a uniquefunction not performable by the elements alone.

This description conveys that a system has someessential ingredients—components, and relationships,and implicitly a boundary, that separates it from therest of the environment. For an information system, theelements and components refer to hardware, software,and even people. The relationships are its interfaces,interrelationships between and among softwarecomponents and hardware components, or between auser and any or all of these.1 There can be interfaces tothe external environment as well.

A system produces results unachievable by thecomponents alone. Rechtin offers a general exampleto illustrate this aspect of a system, worth repeatinghere:

…imagine that your automobile was completelydisassembled and laid out on your driveway. All

1 A more formal definition of interface is “shared boundariesacross which information is passed” (IEEE Standard 610.12,1990).

33Chapter 2

the elements individually would be just as before,all in working order. But you would have notransportation. Transportation, the uniquesystem function, only exists when all the elementsare connected together and function as a whole.(Rechtin, 1991)

But what is a SOS? There is no commonly accepteddefinition, and there are differing classifications oflarge complex systems as SOSs (Shenhar, 1994; Eisner,1993; Maier, 1996, 1998). To illustrate theinconsistency, at least one large defense system hasbeen both characterized and disavowed as a SOS.2 Bothcase studies analyzed in this book—the DefenseMapping Agency’s3 Digital Production System and theU.S. Army’s TF XXI—are characterized here as a SOS.

In this book, the more conventional definition of a singlesystem is expanded to that for a SOS:

A system of systems is a set of different systemsso connected or related as to produce resultsunachievable by the individual systems alone.

Characteristics of a System of Systems

Maier (1996, 1998) has provided characteristics todistinguish a SOS from more conventional systems.Certain of his descriptions are useful in the context ofthe two case studies and for considering the needs offuture operations in the Joint Vision 2010 era—manysystems managed by many organizations integrated to2 Global Command and Control System3 Now the National Imagery and Mapping Agency


achieve results to meet the objectives of a specificmission. He notes that in a SOS the various componentsare large-scale systems in their own right.

• Each is capable of independent action and fulfillsa purpose of its own.

• The individual systems of the set are managedindependently—to fulfill their stated purposes.

In contrast to Rechtin’s example of an automobile givenearlier, in a SOS the individual entities would not remainlaid out on the driveway until assembled. They arecapable of independent action. These constituents fulfillpurposes of their own and can operate whendisassembled from the whole. They are managed fortheir own purposes.

As for any system, there are links in a SOS as well.These are between the individual systems of the set andare the interface relationships necessary to accomplishthose objectives not able to be obtained by the individualconstituents alone.

Figure 2-1 illustrates the use of these terms for a singlesystem and a SOS.

In addition to the two characteristics discussed, Maier(1996) provides three others that are useful todistinguish a SOS from the more conventional system:

• A SOS manifests emergent behavior because itachieves purposes not resident in the individualconstituents.

35

Ch

ap

ter 2

Figure 2-1. Hierarchy of a System of Systems


• It is evolutionary—functions are added, removed,and modified over time.

• It usually is distributed on a large geographicscale.

Prominent examples of a SOS cited4 are the Internet,integrated air defense networks, and intelligenttransportation systems.

The Internet provides a simple example of a SOS. Itsconstituents are many computer networks and majorcomputer nodes distributed globally. Some of theindividual networks may be further decomposed, suchas into other subnetworks and computer systems.Internet nodes collaboratively exchange informationusing protocols. The Internet evolves with phenomenalgrowth. It also exhibits emergent behavior, typifiedby the complex distributed applications on itscommunications layer, including that of the WorldWide Web.

Maier’s examples also illustrate the point that theconstituent systems of a SOS can themselves be a SOS,such as the World Wide Web, or a corporate enterprise.

Interoperability and Integration

A SOS carries out a purpose separate from those of itsindividual systems—through the relationships of itsconstituent systems. The interface relationshipsbetween and among the systems of the SOS achieve

4 Maier (1996, 1998) provides discussions of these examples inhis papers.

37Chapter 2

synergy—through the passage of information.Therefore, interoperability is an essential requirementfor a SOS:

Interoperability connotes the ability of two ormore systems or components to exchange anduse information. (IEEE Standard 610.12, 1990)

Interoperability enables the relationships, andintegration ensures that the synergy of the individualsystems realizes the purpose of a SOS. The integrationevent unifies the individual systems of a SOS to achievethe desired holistic behavior—to deliver the requiredresults. Integration is as essential as interoperability fora SOS.

Usually the process of integration is both an incrementaland a cyclic one—not just one of plug-and-play,although having it be so would be the ultimate objective.It is one of build and test, iteratively refining a system’scomponents and interfaces until the required purposeis produced. For a SOS, it is building and testing theindividual systems and their interfaces to determine ifthe result, the integrated product, meets the operationalrequirements. As shall be seen from the narration ofthe integration events of each case study, a great dealof effort was necessary to ensure each SOS deliveredthe results intended.

The Architecture of the SOS

The integration event of a SOS could be the milestoneevent comically characterized by “a miracle occurshere”—if it occurs without the proper system


architecting and engineering preceding it. Whileintegration is the focus of this book, the architecturalframework and engineering efforts are relevant. In theoverview of the two case studies provided in the nextchapter, background on their architectures is providedas pertinent information.

The architecture of a system is its organizationalstructure.5 It can be described using multiple andvarious perspectives, called viewpoints, each of whichprovides different information. The descriptions of thetwo case studies apply the same three6 viewpointstypically used in describing the defense enterprise—operational, technical, and system architectures. As ageneral characterization, the systems architecture isdeveloped using the standards described in thetechnical architecture to meet the requirements of theoperational architecture.

Operational Architecture

Generally the operational architecture captures what theuser expects to do and what information will be neededand exchanged by the organizational units. For theDMA case, the user was the Agency’s workforce,primarily cartographers. For the Army’s TF XXIprogram, it was a brigade of soldiers equipped with newdigital capabilities. Descriptions of the operationalarchitecture communicate the user functions, the5 As defined in IEEE Standard 610.12, 1990.6 Rechtin and Maier (pp. 120–122) present six viewpoints intheir book, including a separate information (data) viewpoint.In contrast, the information model is embedded within the threeviewpoints used in this book.

39Chapter 2

information required, operational relationships, and, ifknown, performance bounds:

A description (often graphical) of theoperational elements, assigned tasks, andinformation flows required to support thewarfighter. It defines the type of information,the frequency of exchange, and what tasks aresupported by these information exchanges.(DISA DII Master Plan, 1998)

Technical Architecture

The technical architecture describes the standards andguidelines with which the system must comply, suchas information, processing, and transport protocols:

…the services, interfaces, standards, and theirrelationships. It provides the technical guidelinesfor implementation of systems upon whichengineering specifications are based, commonbuilding blocks are built, and product lines aredeveloped. (DISA DII Master Plan 1998)

As later described in the two case studies in this book,DMA used very broad guidelines for its architecture,while the Army developed a detailed technicalarchitecture.

Systems Architecture

The systems architecture provides a physical view anddescribes the real system components, as built andimplemented, i.e., a description of:


…the physical connection, location, andidentification of key nodes, circuits, networks,warfighting platforms, etc. and specifies systemand component performance parameters.… Thesystems architecture shows how multiple systemswithin a subject area link and interoperate, andmay describe the internal constructions oroperations of particular systems within thearchitecture. (DISA DII Master Plan 1998)

Most of the constituent systems of DMA’s SOS werenew developments initiated at the same point in time.In contrast, the Army’s SOS incorporated many existingsystems in addition to new initiatives. The systemsdescriptions of these legacies were an important startingpoint for developing the architecture for the TF XXI.

Federations of SystemsEach of the two case studies produced a SOS. As willbe apparent in subsequent chapters, these werestrenuous undertakings. Yet each was simple incomparison to integrating a SOS that matches theoperational expectations for the Joint Vision 2010 era.One reason is that both programs had centralizedmanagement and authority that controlled thedevelopment and subsequent operations. Generally,there was control exercised over the three architecturalviewpoints—operational, technical, and systems.

The Joint Vision 2010 era includes missions that willrequire a SOS when there is not the same centralizedcontrol and authority. Coalition operations will

41Chapter 2

involve dispersion of power and authority andintroduce many viewpoints from coalition partners.At the same time partnerships will bring greaterdiversity of assets than was experienced in either casestudy. This type of SOS is distinguished here by theterm, “federation of systems.”

A federation of systems (FOS) is a SOS—but onemanaged without central authority and direction. Theconstituent systems of a FOS are independentlymanaged, and have a purpose of their own. But thedegree of management independence is much greater.Power and authority are decentralized in management,development, and operations. Because there is nocentral power or authority for direction, theparticipation of the constituents occurs throughcollaboration and cooperation to meet the objectivesof the federation. Consequently a FOS is generallycharacterized by a greater degree of autonomy,heterogeneity, and distribution.

This concept of a FOS borrows from other work,7 suchas the taxonomy introduced for federated databases byAmit Sheth and James Larson (1990) and fromcollaborative structures by Maier (1998). It alsodovetails with the principles of federations providedby Charles Handy (1992), albeit these were developedin the context of organizations. Handy defined five

7 The intent is not to present a complete taxonomy, but to discusscharacteristics of federations to develop an understanding of thecomplexity of integration events in the future. The interestedreader should consult the references.


principles8 of a federation, the most important of whichis subsidiarity, the assignment of power at the lowestpossible point in the organization. In such a structure,each element has great autonomy in participating tomeet the overall objectives of the federation.

This relationship of a SOS to a FOS is captured infigure 2-2. This figure is notional and indicates therelative positions of a SOS from a FOS. There is noalgorithm provided to define precisely thedemarcation. Nor are there values on the axes todelineate any boundaries.

The region labeled SOS is characterized by (more)centralized management and control over developmentand operations, and therefore less autonomy. While thesystems of the SOS are managed independently andhave a purpose of their own, as a constituent in the SOSthey are subject to some form of direction. The result is(more) uniformity in architectural framework,guidelines, standards, and development principles, anda (more) uniform operational view than would be thecase for a FOS.

A SOS for a coalition operation, such as in Bosnia, canbe viewed as positioned farther along the autonomy axisinto the FOS region. A SOS supporting service oragency missions is closer to its origin. A SOS for a

8 The other four principles are: interdependence to spread powerand avoid risk; a uniform way of doing business; separation ofpowers to keep management, monitoring, and governance insegregated units; and twin citizenship to ensure a federal presencein an independent region (Handy, 1992).

43

Ch

ap

ter 2

Figure 2-2. System of Systems (SOS) and Federation of Systems (FOS)


joint mission (not shown) would be depicted betweenthe two regions.

Figure 2-2 also depicts different shapes for each SOSand FOS to convey different degrees of heterogeneityand dispersion. Each could vary by degree dependingon the nature and complement of the systems supportingthe needs of a particular mission. The degree (and shape)could change with evolution of the SOS or FOS—assystems are added or altered. Also, a specific FOS couldbe less dispersed geographically than a specific SOS.

The attributes of autonomy, heterogeneity, anddistribution are interrelated. As power shifts fromcentral direction to more collaborative management ofdevelopment and operations, the characteristics canchange by degree. As control is decentralized andlocalized, then local requirements and localinterpretations can become more diversified. Localpower with autonomy contributes to variations inrequirements and therefore system capabilitiessupporting these. Similarly, greater geographicdispersion can contribute to increased autonomy, whichresults in more localization of processes, concepts, andeven cultures. These, in turn, can lead to singularrequirements, which in turn leads to more heterogeneity.

To return to the example given earlier, the Internet beganas a SOS under the management and direction of theDepartment of Defense. Today it is a FOS. Little centralmanagement or power of enforcement exists.Collaboration by participants is voluntary. Today a

45Chapter 2

footprint of its constituent systems and characteristicswould be very different than one from its early years.

The Implications of a Federation of Systems

Generally a FOS is more heterogeneous than a SOS.Any SOS will exhibit some diversity, even one adaptedto a single architectural framework. The individualsystems are managed and operated independently byvarious organizations (e.g., services and defenseagencies). When multiple communities of humansmanage, engineer, and operate, there will be differentinterpretations of requirements, standards, and priorities.The legacy systems in the defense enterprise alreadyhave been flagged as amplifying heterogeneity.

The technical architecture of a FOS could comprisewidely varying standards and guidelines—theequivalent of multiple technical architectures used byits constituents. Accordingly, the individual systems asimplemented would be (more) heterogeneous than aSOS in hardware components, platforms, operatingsystems, programming languages, softwareapplications, and data structures. There would be moreissues of interoperability.

The coalition operational environments will bringmultiple cultures and multiple languages, as well asdifferent rules, guidelines, processes, values, andconstraints. The rich mix of international players willintroduce interoperability issues of semantics—disparate meanings, and different interpretations, inaddition to different languages. There will be differing


management, development, and operationalenvironments. A FOS will be comprised of diverseindividual systems managed by collaboratingorganizations but developed and operated using theirown methods, processes, and technologies.

The operational viewpoints, when as diverse as incoalition operations, result in individual systems in aFOS that mirror the varieties of operational approachesand relationships in their individual architectures.Decentralized control implies variations in howoperational communities perform their missions—andat what level. This affects how their information systemsare developed and operated. For example, commandand control might be exercised differently by variousorganizations in a coalition operation. Such differenceswould be reflected in the individual systems of the FOSthat are used by the various partners.

From every architectural perspective—operational,system, and technical—for a FOS the interoperabilityissues will be greater and the integration morechallenging than that of a SOS. Nonetheless, when it ispossible to simplify architecture and relationshipsamong the constituents, such as in the case of theInternet, phenomenal synergy is possible.

Hard, Harder, and Hardest

To an extent, the term “system of systems” is anunsatisfactory one. It masks the complexity anddifficulty of the integration challenge. The unitarynature of the term screens the multiplicity, diversity,

47Chapter 2

and autonomy of the individual systems and theirmanagement communities. However, it is powerful inconveying the synergy of integration.

While integrating a single large scale system is hard,the integration of a set of systems independentlydeveloped and operated for distinct missions is harder,and a FOS even more so. As such, these differenceshave resource, schedule, performance as well asmanagement implications.

The approaches for integrating a SOS will be examinedthrough the experiences of the two programs thatcomprise the case studies. Each is characterized as aSOS, but one (the TF XXI case) is positioned relativelycloser to the FOS domain than the other. They providea good starting point for determining how to succeedwith a SOS integration.

It may be inferred that the approach and proceduresused to integrate the components of a single system aresufficient for success. They provide a good basis, butin dealing with a SOS, refinements to methods andprocesses are required for activities behind theWizard�s curtain.

It can be anticipated that a FOS integration requires yetstill another evolution of strategy from that of a SOSbecause a FOS is developed with diverse architecturesand managed through collaboration rather thandirection. The intent is to examine the implications ofthe FOS in the final chapter of this work by buildingupon the lessons learned for SOS integration developedin earlier chapters.

49

Chapter 3

The Two Case Studies

This chapter provides an overview of the twoprograms with which this book is concerned—the Defense Mapping Agency’s1 Digital

Production System and the U.S. Army’s Task Force XXI.They share many characteristics in common, not the leastof which was their goal to deliver a SOS withrevolutionary capabilities.

Because the emphasis of this book is on the integrationenvironment, the architecture, engineering, anddevelopment aspects of the two ventures are discussedonly to the degree necessary. But these are relevant.Many bibliographic references also are provided whichcan be used by the interested reader for moreinformation.

Management structures for the programs are discussedbecause they are particularly germane to the integration.

1 Now the National Imagery and Mapping Agency.


Defense Mapping Agency’s DigitalProduction SystemThe Digital Production System (DPS) was a 10-yeardevelopment by the Defense Mapping Agency (DMA)to deliver an end-to-end digital processing pipeline forproduction of mapping, charting, and geodesy(MC&G)2 products. It was conceived with a sense ofurgency in the early 1980s and resulted in one of thethen-largest development programs3 undertaken in theDepartment of Defense (DoD).

Its genesis was a series of studies, many congressionallysponsored, to look at collection platforms for the 1990sin the context of emerging requirements, particularlyfor weapons systems. Its birth was the direct result ofone of those studies—the Hermann Panel Report.4 Thisreport recommended that DMA expedite developmentof a modernized production line that accommodateddigital softcopy source materials and used computer-assisted techniques. Dr. Richard DeLauer, then UnderSecretary of Defense for Research and Engineering(USDR&E), directed DMA in February 1982 simply

2 MC&G comprises “the collection, transformation, generation,dissemination, and storing of geodetic, geomagnetic,gravimetric, aeronautical, topographic, hydrographic, cultural,and toponymic data” (USIGS Glossary, 1998).3 An advisory board made this program assessment just beforethe DPS critical design review. Board members observed thatwhile the scope of the software and complexity of the integrationeffort made it comparable to major special programs within theDepartment of Defense, there was no comparable productionprogram.4 The panel was chaired by Dr. Robert Hermann.

51Chapter 3

to implement the Hermann report. DMA delivered thefull operational capability (FOC) in November 1992.

The program was considered critical to the success ofthe defense mission. At the time there were significantbacklogs of requirements for MC&G products whilethere were growing dependencies on them, particularlyin weapons systems. In addition there were requirementsfor products providing greater fidelity of the earth’sterrain and culture to drive simulators to supportrehearsals for military missions. While the panelprofessed skepticism about the specificity of particularrequirements, it concluded that the requirements wereparamount and growing.

Weapon systems were increasingly dependent ondescriptions of terrain and cultural features to aid theirnavigational capabilities. The digital data productsprepared by DMA were used for smart weapons, as wellas for the air-based, sea-based, and land-based missiles.As an added impetus, there was also great reliance onDMA processes to provide targeting information withincreasing precision.

The Hermann panel anticipated future scenarios thatwere even more time-sensitive than then-currentdemands, requiring a crisis-like turnaround responsefrom DMA. The members saw the steady trend ofincreasing precision with respect to the relative andabsolute accuracy of the positions of points and featureson the earth, although this would vary depending onthe nature of the product. With some prescience (thiswas 7 years before the end of the Cold War), the panel


anticipated the growing unpredictability of the locationsof crisis events. This situation, when viewed in theaggregate with the increasing reliability on MC&G data,gave urgency to the need to produce products moreaccurately and in a more timely manner. Extraordinaryefforts were expended by DMA to meet crisis demands,5

which at the time, depending on the product need, couldrequire many months of activity.

By the early 1980s DMA long had been regarded asthe world’s premier map-making organization. Whilethe conversion to digital products was incorporated intothe Agency’s longer term strategy, the production plantswith the installed and aging technology base reliedlargely on film-based source materials and a hybrid ofanalog and digital processes. At the time, DMA’s abilityto enhance the speed of its production processes andthe accuracy of its products relied on a modestlyendowed research and development program. Most ofthe production systems within the organization werestratified, fragmented by processes tied directly to thenature and formats of the source materials used, as wellas by the type of individual products required. Changesin the formats of source materials or in customers’product requirements inevitably resulted in adaptationsthat introduced further delays in meeting requirements.

5 Examples were DMA’s support to the hostages’ rescue in Iranand operations in Africa and the Middle East.

53Chapter 3

Need for Revolutionary Technology andRevolutionary Concepts

In assessing the long-range requirements in concert withemerging collection and processing technologies, theHermann panel perceived a chance to achieve asignificant breakthrough. The opportunity was markedto position for increased precision, adapt for crisisscenarios, and also accommodate the growing need forMC&G data. But the modernization of DMA into digitalprocesses needed significant investment, a faster-pacedresearch and development program, and a largeacquisition venture. The panel anticipated significantadvances from automation to substantially reduce thetimelines, particularly those involving the correlationof stereo imagery, critical to the derivation of particularDMA products.

Exciting possibilities from technology were envisioned.At the time, some of DMA’s most labor-intensive (andtherefore time-intensive) processes included usingstereo imagery to extract elevation and feature data.Pipeline times of 2 to 3 years were not unusual toproduce new products from source materials. It wasbelieved that delivery times could be substantiallyreduced with the introduction of more automatedprocessing techniques.

The Hermann panel was not concerned with impactson production resources as a result of such significantchange, nor even expected that reduced resources forproduction processes would result. In fact, the paneldid not consider this aspect to any great extent. It noted


that the impact of accommodating new materials wasprobably underestimated, and probably not understoodvery well. But as the modernization programproceeded, DMA put tremendous effort into reducingproduction inefficiencies. The design approachadopted to do this had a significant impact on theoverall success of the program.

The Hermann panel concluded its report in 1982 withrecommendations to USDR&E to proceed withsubstantial investments in creating a new productioncapability at DMA using digital, time-responsiveprocessing and based on digital source materials.Anticipating the extensive scope of the venture, thepanel further recommended the establishment of aseparate organization responsible for the modernizationand acquisition of substantial equipment and systems,and that DMA develop a plan to accomplish this.

Responsible Organization Established

DMA responded to USDR&E by establishing a SpecialProgram Office for Exploitation Modernization(SPOEM) in February 1982. Despite no previousexperience with a development of this magnitude, theAgency proceeded aggressively with the program calledthe Digital Production System (DPS) under the strongleadership of a program director specifically appointedto the task. By design, a single organization and a singlesenior manager were designated as accountable for theacquisition and development.

55Chapter 3

The Defense Mapping Agency Program—in TwoPhases

The program was partitioned into two phases—aninterim phase called Mark 85 and the final phase calledMark 90. Mark 85 was so-named because the deliverieswere to begin arriving in 3 years—in 1985. Analogously,Mark 90 derived is name because the initial operatingcapability (IOC) was to occur in 1990.6

From today’s perspective, a 10-year development mightbe judged as non-time critical. However, there was agreat sense of urgency to get on with the conversion todigital materials and develop a new digital productioninfrastructure. The venture was assessed as an enormousundertaking with a great deal of technical risk anduncertainty. At one time DMA had conceived that sucha capability would require at least 15 years to achieve.But DMA, in reaction to the DeLauer letter, determinedthat the early 1990s provided a reasonable deliveryschedule. DMA moderated expectations by earlyintroduction of the understanding that only several yearsafter the program’s final delivery of hardware andsoftware would a full production capacity be achieved.7

6 The IOC schedule was slipped a year to 1991 because ofconsequences of the Balanced Budget and Emergency DeficitControl Act of 1985, 2 USC 901 (Gramm–Rudman–HollingsLaw).7 The design relied on population of a data base of MC&G data,which when reaching sufficient detail over geographic regions,could be used to generate or revise products more efficiently.Because this initial compilation of data required several years,full production throughput was also delayed.


Mark 85, an Incremental Step

As initially conceived, Mark 85 had as its objectives adelivered capability to ingest and exploit new sourcematerials using film-based processes. The strategy wasfixed on retaining many of the then-existing productionsystems retrofitted with new software and hardware.Mark 85 capitalized on some work that had occurred inprevious research and development efforts by theAgency and incorporated several softcopy techniquesand digital components that were enhancements incapability. Some of these provided important insightsfor Mark 90 developers.

Primarily the approach emphasized enhancements ofthen-current processes. It was an evolutionary step—not a revolutionary one. As a result, the operationalconcept retained some of the disadvantages of thefragmented, product-specific processes and standalonecomputer processing systems, all of which wereconnected primarily through manual processes. Onenew addition was that of an automated productionmanagement capability, which became heritage to theMark 90.

Importantly, Mark 85 was also a risk-mitigationimplementation. By delivering this interim capability,DMA ensured that there would be a production capacityin place long before the more ambitious second phasewas ready. This interim approach turned out to becritical and enormously successful, especially whenunanticipated operational requirements exploded in

57Chapter 3

Operations Desert Shield/Desert Storm before thedelivery of the Mark 90.

Mark 90, a Revolutionary Step

This book focuses on the Mark 90 phase of the DPS; itwas truly revolutionary in its conception. It used hithertountried digital source materials. It went beyond the stateof the art in extraction of geospatial8 data using entirelysoftcopy techniques. It developed digital automatedprocesses for cartography akin to those of imageprocessing. At its inception, it required advances in datamanagement to support volumes of imagery andgeospatial data and enormous bandwidth in datacommunications to move that information locally andto geographically dispersed sites.

In the early 1980s, the notion of storing MC&G data ina digital database and using that as a base from whichto generate a variety of cartographic products wasrevolutionary. As an overall strategy, it promised greatflexibility for adaptability and tailoring of productinformation to accommodate the plethora of emergingweapons systems, as well as increased currency ofinformation through more rapid revision. Suchcapability had never before been implemented and onlyconceptually articulated.

Production processes would access imagery that was100 to 1,000 times greater than the average size of a

8 This includes “information that identifies the geographiclocation and characteristics of natural or constructed featuresand borders of the earth” (USIGS Glossary, 1998).


digital data set in use at the time, and produce a unit ofMC&G data that was about 10 times greater. As a result,the concept of accessing, exploiting, and disseminatingmultiple images and extracting and storing MC&G datawent well beyond the available technology andmethodology of the time. At the outset9 of the programin 1982, only 10 percent of the technology required forthe DPS was commercially available.

In addition to its innovations in use of digital imageryand digital processes, the DPS was based on the strategyof multiproduct operations. The cartographer extractedinformation for many products at the workstation inone job assignment, rather than extracting informationfor one product over multiple job assignments atdifferent points in time. Optimization of this time andlabor intensive process promised a breakthrough in theoverall annual production rate of Agency products.

The Demise of a Prototype

At one point in the program history there was a Mark87, but it had an early demise.10 Its elimination wasbarely noticed at the time, but it later resulted in seriouscomplications for Mark 90, evident in the integrationphase with which this book is concerned. Its purposewas the delivery of a fully integrated digital prototype—an engineering model—less ambitious in performance

9 By IOC in 1991, approximately 90 percent of the technologywas commercially available. Some became available fromvendors who developed commercial versions of the Mark 90technology.10 Its termination as a development was announced in September1983 at a concept design review status briefing.

59Chapter 3

than the total Mark 90, but offering full functionalityand including all individual systems.

Because the prototype could not be completed withsufficient lead-time to influence the manufacturing, itsbenefits were considered marginal. At the time of itstermination, it was believed that other means could beused to verify the DPS engineering.

The Digital Production System Architecture

As the DPS undertaking was evolved, it was partitionedinto aggregates of functionality termed “segments” inthe program terminology, but here called “systems”consistent with the definitions provided earlier. TheMark 85 consisted of six systems that were principallyhardware and software enhancements augmenting then-existing systems or ongoing development initiatives.Later two of these, the Hardcopy Extraction Systemand the Source Acquisition System, were furthermodified and became legacy systems included in theset of systems of the Mark 90. A substantial heritage ofsoftware modules and some hardware components fromthe Data Integration System of Mark 85 were subsumedinto the Mark 90 system.

Mark 90 was the DPS system of systems (SOS). Generaloverarching requirements for functionality,performance,11 and specific MC&G products werepartitioned and aggregated into separate parts. These,

11 The DMA established objectives of 75 percent reduction inproduction time and 50 percent reduction in production costsfor products generated in the DPS.


in turn, were used for the acquisitions of five individualsystems and the three heritage Mark 85 systems alreadymentioned. When integrated and delivered, the Mark90 SOS had seven individual systems interfaced throughcomplex relationships between and among theindividual systems.

The DPS was designed to produce 2412 MC&Gproducts, primarily from digital imagery but also usingvarieties of other information including foreign-produced maps and charts, film, and text. While the 24products constituted a small number of the hundreds ofproducts produced for customers, at the time theycollectively required a majority of the Agency’sresources to produce.

Figure 3-1 illustrates the DPS architecture of sevenconstituent systems as a pipeline. Based onrequirements for MC&G products from users, aproduction program was developed. Productionmanagers then used many factors to determine thesubsequent flow of work and information. Theseincluded the availability of imagery (if unavailable, itwas acquired), the preparation and georeferencing ofsource materials for the assignment, subsequentextraction by cartographers of terrain and featureinformation, and eventually generation of graphicproducts, both hardcopy and digital. Source imageryand source materials, the extracted MC&G data, andvarieties of intermediate data were stored, accessed,

12 The number of products varied during the program as a resultof evolving requirements, priorities, and the efficiencies of thedigital processes. At FOC there were 24 products.

61

Ch

ap

ter 3

Figure 3-1. Digital Production System


and disseminated at many points through the pipelineby using a set of data and communications services.

The DPS was installed at three geographically dispersedProduction Centers. At the time of delivery, the usercommunity of DPS was the major subset of theAgency’s production workforce and numbered about3,000 professionals. They were in organizations whichwere structured along the lines of the pipelinedproduction process.

The operational concept for the DPS graduallyevolved from an early one of relatively independentactions by the various user organizations to one thatrelied on a great deal of coordination andcommunication to deliver the intended results—a setof mapping products derived from imagery, fromother sources of information, and (ultimately) from adata base of existing geospatial information. Thisbecame true for users and organizations within aProduction Center, for those coordinatingassignments and information between Centers, aswell as for those in the Headquarters. The principalfactor contributing to this change was thecentralization of authority in production managementand its increasing role in scheduling and allocationof jobs, people, and equipment. Over time thisresulted in a DPS structure characterized as tightly

63Chapter 3

coupled,13 with interdependencies among theconstituent systems that complicated the integrationchallenge and ultimately adaptability. At the time itwas believed that this would lead to significantimprovements in production throughput andreductions in the numbers of people required.

The systems architecture “as built” for FOC wasextensive, with large numbers of hardware and softwarecomponents developed and delivered for the DPS. Morethan 3,000 pieces of equipment were acquired, including2,000 workstations, about a thousand of which werebased on developments specialized to the mappingprocesses. About 10 million lines of code weregenerated, integrated, and tested, including nearly 2million knowledge-based rules.14 As installed, the DPSrequired more than 380,000 square feet of facility spaceat the Agency’s three Production Centers.

Program Methodology and Schedule

The DPS was developed using a classic waterfallmethodology (Winston Royce, 1970). From generalrequirements for the SOS, aggregates of functionalitywere partitioned into individual acquisitions. Theseresulting systems of the DPS became aggressivedevelopments by different companies begun at about

13 Two systems are coupled if they are interdependent (i.e., if atleast one system requires information from the other, or requirescomponents, services, or people). Tighter coupling indicatesgreater (i.e., multiple) interdependencies between systems thandoes loose coupling.14 Heuristics implemented in software to increase the degree ofautomated assistance to the cartographer.


the same time (with the exception of the Mark 85 legacysystems). After the allocations of functionality wereinitially made for the individual systems, there werefew changes in the segmentation.

The DPS architecture evolved principally bottom-up,through the increasingly detailed functionality of theindividual systems and the growing specificity of therelationships15 between them. There was no equivalentof the defense enterprise common operatingenvironment or the joint technical architecture availableat the time. There were a few key standards that wereapplicable, particularly those relevant to customers’product requirements for MC&G data.

As implemented, the various systems of the DPS wereheterogeneous, the diversity among them amplified bythe nature of the special developments in each of them.The unique hardware and software components rangedin technology from image processing to cartographicgeneralization software to network switches. The verynature of the graphic MC&G products requireddevelopments specialized for the mapping applications,such as printers, scanners, and plotters with large sizeformats. Differences in the missions of the individualProduction Centers required variations on components,equipment, and system configurations.

Designs were constrained by a general framework ofprogramming practices and conventions and the

15 As an example of complexity, one interface document requiredapproximately 10,000 pages to express the details of theinformation exchanges that one system had with the others.

65Chapter 3

standards applicable at the time. By the time of FOC,15 different programming languages had been used.Standardization was realized more within thecomponents of an individual system than across theSOS. However, there were partial successes in limitingthe numbers of platforms and operating systems throughcommon mainframes and minicomputer environments.Multiple commercial products were incorporated,primarily for the information processing and data baseenvironments, and some for communications.

Figure 3-2 provides a simplified DPS programschedule—requirements and design reviews, integrationevents, IOC, and FOC. At all the major DPS milestonesthere were assessments to examine the SOS architecturewhile considering the state of the individual systemsand the interfaces between and among them. Modeling,analyses, and simulations were used to examine theviability of achieving functionality and performance,and issues were identified and addressed.

The integration phase occurred after completion ofthe development and testing of the individual systemsand their interfaces. The SOS integration approachused a series of formal demonstrations to verify thecorrectness of the interfaces between systems of theset. These events included production-like jobs togenerate MC&G products. Informal integrationactivities began even earlier. Discussions on thedetails of the integration events will be provided inthe next chapter.

66

Be

hin

d th

e W

izard

’s Cu

rtain

Figure 3-2. DPS Schedule

67Chapter 3

The DPS was delivered to a single site for the IOC.This delivery had reduced functionality, a decisionforced by late closure of all the interface relationships,primarily those for production management. A similarorchestration of activities occurred before the FOCdelivery, which included full functionality of the SOS.The individual systems and interfaces were completedand tested, then reintegrated as a SOS using anotherand different series of demonstrations at a single site.Then the integrated product was incrementally deliveredto all three Centers. After achievement of FOC, thebusiness of using the DPS for production and populationof the MC&G data base began.16 As DMA had planned,residual discrepancies were addressed while productionusage gradually increased.

After Full Operational Capability

The FOC occurred in November 1992, 10 years afterthe initiation of the program. By the time of delivery,the customers’ needs had dramatically altered andcontinued to do so over subsequent years, drivenprimarily by changes in the geopolitical environment.In response, the Agency migrated from a posture ofproducing MC&G products toward one of providinggeospatial information and services. While the DPS is

16 A caveat was that the actual use of DPS for production beganbefore FOC. The production processes using DPS were serializedso that after intermediate stages of the pipeline were judged readyfor production, the turnover to production occurredincrementally.


used for production today, it represents only a subsetof the Agency’s current systems architecture.

The DPS was not used as conceived because the needsand production scenarios had changed by the time ofits delivery. Its design precluded easy and rapidadaptation. Alternative production capabilities werespun from certain DPS technologies, some in responseto Operations Desert Shield/Desert Storm, whichoccurred before FOC; others in response to differentcustomer requirements. A good general referencedescribing the DPS and its subsequent evolution isfound in (Littlefield 1995). This paper also providesinsight as to the effect of the DPS development on thecommercial availability of systems and workstations—for cartographic processes and feature extraction. Thestate-of-the-art of commercial cartographic technologyadvanced by building upon the DPS developmentinvestments.

The overall DPS performance, which required a stableproduction program, a populated data base, and astrategy of multiproduct extraction (conditions neverfully realized), was never achieved. Yet even today,for certain production processes, it surpasses evencurrent commercially available capabilities.

With the DPS the DMA achieved the key objective itwas given: to establish digital softcopy processes toproduce MC&G products from digital source materials.

69Chapter 3

The Management Structure

The DPS was a development managed by a singleAgency and intended for its own internal operationaluse. Figure 3-3 depicts the organizational structure usedto manage the development and its transition toproduction. The responsibility for the acquisition anddevelopment of the DPS resided with its SpecialProgram Office for Exploitation Modernization(SPOEM), later re-organized as the DMA SystemsCenter. One Agency Program Executive Officer (PEO)was accountable and reported directly to the Director,DMA. This individual was in a position to resolve anyprogrammatic, engineering, and funding issuesattendant to the acquisitions, including any connectedwith the integration.

At peak activity, as many as 400 DMA people workedon aspects of the integration. These included theSPOEM’s program managers with their developmentteams, and the operational community leading thetransition processes and participating in the acquisitions.One organizational element, called the “SOS cadre” inthis book, was responsible for the DPS SOS. Amongtheir responsibilities was leading the integration eventwith system integration and system engineeringcontractor resources supporting.

DMA’s operational community participated throughoutthe program. To prepare for the acceptance of the DPSand its turnover to production, an Activation ControlTeam (ACT) was established in June 1989. Themembers played a significant role during the integration.

70

Be

hin

d th

e W

izard

’s Cu

rtain

Figure 3-3. DPS Management Structure

71Chapter 3

Their leader reported to the PEO but was dual-hattedas the Director of the Production Center where the SOSintegration occurred. He provided a focused voice forthe operational community. The Director of DMA alsoconvened an advisory team of senior leaders from theAgency’s organizations, both production andacquisition, named the Agency Transition ManagementTeam (ATMT). This group was chartered to ensure thatan integrated planning and implementation approachwas taken for integration, verification, and productionramp-up activities.

The U.S. Army’s Task Force XXIThe Task Force XXI (TF XXI) resulted from asignificant movement toward change in the U.S.Army. In 1992 the (then) Army Chief of Staff GeneralGordon Sullivan recognized that the convergence ofseveral factors called for a significantly alteredstrategy for the Army. The geopolitical environmenthad changed as a result of the end of the Cold War.The U.S. Army was being downsized and basedprimarily in the United States. But the threats werebecoming unpredictable, not only in their nature, butalso in their location. Also information technologywas becoming increasingly available.

General Sullivan concluded that the Army needed toshift toward a strategy of force projection with ademonstrated ability to rapidly alert, deploy, andconduct operations anywhere in the world (Sullivan &Dubik, 1994; Sullivan & Coroalles,1995). He


recognized that information technology could providea key enabler. He expected fundamental changes inevery aspect of the Army, including structure, doctrine,capabilities, training, and tactics. However, the natureof these would require time and effort to understand.

A sequence of exercises and experiments called theLouisiana Maneuvers was conceived as a kind oflaboratory to learn about the Army of the 21stcentury (Sullivan, 1992). Over the next 2 years muchinternal examination occurred. Many activities suchas major exercises and simulations were used toassess the Army’s ability to meet different forceprojection scenarios. What emerged among theconclusions was the importance of informationtechnology and the opportunity to organize aroundinformation to mass effects.

The results of the Louisiana Maneuvers were far-reaching. Three complementary processes wereinitiated—the re-design of the operational Army, there-design of the institutional Army, and the promotionof information age technology. This third componentthe Army called “digitization,” and it was to befacilitated by a newly created Army Digitization Office(ADO). The TF XXI program and processes thenemerged with the objective of transforming the currentArmy to one organized around information andinformation technologies for the 21st century.

73Chapter 3

The Task Force XXI Plan and Beyond

The Army subsequently defined a comprehensiveprogram for battlefield digitization. The goal was:

to apply information technologies to acquire,exchange, and employ timely digital informationtailored to the needs of each decider, shooter,and supporter (Providing the Means, 1994)

By providing the communications and processingcapability to influence speed, space, and time withinthe battlespace, two key advantages could be gained:shared situational awareness17 and enhanced battlecommand. Implicitly such a strategy relies on the abilityto accomplish integration between information systems;consequently, the ADO’s mission statement includedthe coordination of integration.

The execution of the TF XXI plan was to provide theunderstanding of how to evolve. The Army neededdecisions on doctrine, structure, and capabilities by theyear 2000 to field the “Army XXI”18 after that date.The TF XXI events would be used to evaluate neededchanges. Subsequent evolution would result in “theArmy after Next,” a longer term objective, anticipating

17 Defined as: “the ability of a unit to know where its friends arelocated, where the enemy is, and to share that information withother friends, both horizontally and vertically, in near real-time”(Providing the Means, 1994).18 The Army XXI program includes fielding a digitized divisionby 2000 and a digitized corps by 2004 (Reimer, 1998).


a significantly different force with greater strategic andoperational mobility (Reimer, 1996, 1997).

A key element of the TF XXI plan was to use thestrategy of an Advanced Warfighting Experiment(AWE). The magnitude of changes for the Army XXInecessitated an entire series of experiments, eachsuccessively building upon lessons learned from theprevious. The AWEs were unique in providing theoperational conditions to focus the technologicaldevelopments, innovative operational concepts, andnew force structure with experimental doctrine forevaluation. They, in turn, also built upon previoustechnology demonstrations.

The Army embarked on battlefield digitization using abottom-up approach for experimentation, echelon byechelon, involving multiple systems and digitizationinitiatives. Early efforts began at the company level. InApril 1994 the Army conducted the first of a series oflarge-scale exercises applying digital informationtechnologies—one at the battalion level.

The Desert Hammer AWE took place at the NationalTraining Center (NTC) at Fort Irwin, CA. Some keylessons derived from the experiment affectedpreparations for the TF XXI AWE, specifically the needto prepare extensively for fielding of and training withexperimental systems. Over the next 3 years, exercisesand AWEs were used to evolve concepts, doctrine, andcapabilities. These included Prairie Warrior/MobileStrike Force, Roving Sands Theater Missile Defense,Focused Dispatch, and Warrior Focus.

75Chapter 3

Task Force XXI Advanced Warfighting Experiment

The TF XXI AWE consisted of a series of live andconstructive simulations that began in March 1996. Itculminated at the NTC during 2 weeks of March 1997with a major force-on-force encounter between theopposing force (OPFOR) and an experimental brigadetrained and fully equipped with all the capabilities theArmy’s digitization program could provide at the time.This event required a SOS, the second case study ofthis work.

The essence of the AWE was to examine the effects ofdigitization on lethality, survivability, sustainability, andoperational tempo. New doctrine was developed for theexperiment and organizational changes were made inorder to examine the effects of these changes when theexperiment was conducted.

The brigade selected as the experimental force(EXFOR) was the 1st Brigade from the Fourth InfantryDivision (4ID) based in Fort Hood, TX. The EXFORwas a brigade task force of 5,000 soldiers. It wascomprised of three battalions of mechanized infantry,light infantry, and armor, with supporting field artillery,aviation, and engineering elements, and areconnaissance troop (Hanna, 1997). The preparationof the EXFOR began in January 1996 and continued24 hours a day until equipment deployment to the NTCin December 1996. Planning the experiment anddefining and engineering its architecture began muchearlier, at least as early as January 1995.


A simplified schedule for the TF XXI, highlighting theevolution of the SOS architecture, is shown in figure3-4. The Army identified particular key core digitizationcapabilities needed for the field experiment, but wasalso willing to consider additional initiatives andprototypes that might provide a significant advantageon the battlefield. Some of these were in laboratoriesor in various stages of development. Therefore a “callfor good ideas” went out while operators and developerscollaborated to evolve operational concepts, doctrine,and required capabilities for the TF XXI.

Though activities on the TF XXI SOS architecture wereunderway, the cut-off date for initiatives to beincorporated did not occur until June 1995. Thecompletion of developments or enhancements andtesting of individual systems occurred between Januaryand June 1996. By June 1996 these systems werescheduled to be in place for the final SOS integration atFort Hood to support a subsequent series of exercisespreparatory for the NTC event. The discussion of theseevents surrounding the integration will be given in thenext chapter.

Task Force XXI Architecture

The process to define the operational, technical, andsystems architecture for the TF XXI experiment wasunderway in January 1995. In one sense it is misleadingto infer that all the necessary preparation occurred afterthis date. Rather the accomplishment built upon thelessons from the Louisiana Maneuvers, which beganin Spring 1992, the previous events of the TF XXI plan,

77

Ch

ap

ter 3

Figure 3-4. Task Force XXI Architecture Schedule


and the earlier efforts underway to establish a technicalarchitecture for the Army.

The essence of the experiment had to be determined,and then translated into key digitization capabilities thatwere essential in the systems architecture to enable theexperiment. Key to defining the operational architecturewas the postulation of how a brigade would conductoperations with all the assistance of informationtechnology. Operators had to determine whatinformation was needed and how it would be used.Mission threads had to be examined end-to-end.Doctrine and tactics needed to change, too. Theoperational architecture for TF XXI focused on thenature and form of the information required, howoperators would actually exercise their functions, andoperational and organizational relationships.Performance bounds were assessed as well.

The implications of the evolving operationalarchitecture and the core capabilities had to be analyzedand engineered with a view to determining neededchanges to existing systems, new developments, andinitiatives, while ensuring interoperability, capacity, andperformance. A TF XXI system engineering master planemerged. The substantial participation and coordinationneeded was achieved through various forums andthrough integrated product teams.19

The technical architecture used was the (then) definedArmy technical architecture,20 versions of which

19 Examples of these included fires, chemical, aviation, mountedbattle/armor, and communications/signal.

79Chapter 3

evolved over the duration of the experiment (ArmyTechnical Architecture, 1996). At the time the TF XXIarchitectural planning began and while efforts wereunderway, the Joint Technical Architecture (JTA) forthe defense enterprise had not yet been issued. However,the Army’s technical architecture was used as a primarysource21 for the initial version of the JTA, mandated inAugust 1996. The TF XXI AWE was among the earliestof programs to field a large-scale systems architecturebased on a defined technical architecture analogous tothe JTA.

The Army’s technical architecture22 provided aminimum set of standards to facilitate integration ofthe systems of the TF XXI. Where possible, thearchitecture used commercial standards and provideda framework for information modeling and dataexchange, for information processing, informationtransport, human-computer interface standards, andinformation security. As examples, the informationprocessing standards addressed a distributed computerenvironment for UNIX systems. The informationtransport standards specified the Internet protocol. Dataand message standardization was provided, includingthe definition of a variable message format and tactical

20 Version 4.0 of the Army’s technical architecture was publishedin January 1996, and Version 4.5 in November 1996, which wasin use at the time of the NTC event. This subsequently hasevolved to the Joint Technical Architecture–Army.21 About two-thirds of the joint technical architecture was derivedfrom the Army’s technical architecture.22 The scope of the Army’s technical architecture was principally,but not exclusively, focused on C4I systems.


digital information link messages. A command andcontrol core data model was developed. Key messageswere defined, such as a “call for fire.”

The technical architecture incorporated the DII/COEconcept. The then-available version of the DII/COE wasnot used in the TF XXI; however a similar layeredframework was implemented using many of the samesoftware products. Others were surrogates, many ofwhich were commercial products. The net result wasthat a common operating environment and commonservices were provided as part of the architecturalframework for the SOS.

The architectural process was a spiraling one,continuing through the integration phase, andadjustments in operational and systems architectureswere made until the conclusion of the SOS integration(Boutelle & Grasso, 1998). For example, the human-computer interfaces were enhanced with featurestailored to the user and with common symbology. Aswitch-based network was migrated to a commercialrouter-based network for the operations centers.Solutions for firewalls and intrusion detection systemswere completed after experimentation with multiplecommercial products. System administration, directoryservices, and start-up were improved. And changes weremade to tactics, techniques, and procedures.

The key revolutionary operational capability introducedfor the AWE was situational awareness, achievedthrough the integration of a number of key systems andcomponents into the overall architecture. From these

81Chapter 3

all soldiers could derive a near-real time commonpicture of the battlefield—to know where friendly forcesand enemy forces were positioned. The locations offriendly forces were automatically transmitted. Theenemy locations were identified using intelligence,reconnaissance, and surveillance assets. Positionlocation devices were linked to the Global PositioningSystem (GPS).

An integrated brigade command and controlcapability—part of the Army Battle Command System(ABCS)23—was provided for the brigade task force ina dozen Tactical Operations Centers (TOCs). Thesewere connected to maneuver, air defense, artillery,combat support, and intelligence systems, and in turn,to the division command and control systems. This isshown in figure 3-5. The information was disseminatedwith a mobile router-based network, also illustrated.

The networks linked the TOCs to the level of theindividual soldier and vehicle, and allowed commandand control messages and information to flow. Messagesflowed in various formats, and message content rangedfrom position reports to graphical overlays. In turn,automatic position reports flowed upwards through thebattalion and brigade TOCs to provide the commonpicture of the battlefield at various levels.

In addition to ABCS, the core capabilities included:

23 A critical aspect of digitization is to structure the Army BattleCommand System to allow seamless connectivity acrossechelons and connect to the joint defense enterprise GlobalCommand and Control System.

82

Be

hin

d th

e W

izard

’s Cu

rtain

Figure 3-5. Task Force XXI Tactical Operations Centers

83Chapter 3

• Digital appliqués, which were small processors infour different versions, mounted on all vehicles orcarried by dismounted soldiers, interfaced to GPS,and connected to a communications system ofdigital radios

• A Precision Lightweight GPS Rreceiver (PLGRS)

• An Enhanced Position Location Reporting System(EPLRS), which was jam-resistant and secure,used for data hauling and for providing near real-time position reporting of the tactical forces

• A Battlefield Combat Identification System(BCIS) to differentiate equipped friendly forcesfrom foes

• A Single Channel Ground and Airborne RadioSystem (SINCGARS) Improvement Program witha commercially-based Internet controller fordigital communications, and

• A Tactical Internet based on commercial routersand switches linking all these computers, radios,satellite terminals, and reconnaissance/surveillance platforms.

Situational awareness was achieved for dismountedtroops, for vehicles, as well as for various platformsthrough the integration of these core digitizationcapabilities, as illustrated generically on figure 3-6, whichshows them incorporated into a weapons platform.

84

Be

hin

d th

e W

izard

’s Cu

rtain

Figure 3-6. Task Force XXI Core Capabilities on a Weapons Platform

85Chapter 3

The TF XXI architecture fielded at the NTC was anaggregate of enhanced legacy systems and 72 separatedigitization initiatives, many of which were individualsystems that included developments of several years’duration. Some of these were concepts or systems thatwere tried in previous experiments but were transformedby the newer digitization technology. For example, inthe late 1980s, the transmission of target locations hadbeen accomplished by using a radio system linked toground vehicles with computers and position locationsystems (Holcombe, 1998). This concept wassignificantly advanced through the core digitizationcapabilities of the TF XXI.

At one point there were as many as 171 initiatives thatwere part of the TF XXI architectural baseline, inresponse to the call for good ideas. These were graduallywinnowed to 72 to ensure that the systems deployed tothe NTC were sufficiently mature to support conceptexperimentation in the field. Screening experimentalcapabilities sufficiently to support their fielding was alesson learned from the Desert Hammer and WarriorFocus experiments.

The initiatives were not only diverse in technology, butrequired multiple configurations, given the variousplatforms and vehicles. Two references describing manyof these are Goedkoop (1997) and Hester (1996).Initiatives ranged from a mortar fire control system to atactical end-to-end encryption device to a ground controlstation for the unmanned aerial vehicles. Of the 72, about60 were innovative digital information systems. A senseof the breadth of the experiment is graphically


communicated by figure 3-7, which is provided withoutfurther elaboration (TF XXI outbrief).

There were organizations external to the Army that alsoparticipated and/or brought experimental capabilitiesto the TF XXI complement of systems, although thesewere limited in number. These included elements ofthe U.S. Marine Corps, the Special Operations Force,the Defense Advanced Research Projects Agency, theAir National Guard, and the U.S. Air Force.

The scope of the systems architecture was vast,comprising hundreds of systems, including thoseproviding fire support, air defense, maneuver, logistics,command and control, communications, andintelligence. There were more than 5,000 pieces ofequipment with more than 900 vehicles and 180different configurations required to support theactivities at the NTC (Hanna 1997). Thousands ofpieces of new equipment were installed on existingplatforms24 “including 873 appliqué packages, 336EPLRS, 1,550 SINCGARS, 62 BCIS, and 1,386 othertypes of TF XXI equipment” (Goedkoop, 1997). Thedetails of the architecture are described at the TF XXIweb site.

24 Platforms included wheeled and tracked vehicles, aircraft, andeven personnel. Many existing platforms were used but had tobe retrofitted with the key digitization capabilities. Someinitiatives required new platforms.

87

Ch

ap

ter 3

Figure 3-7. Task Force XXI


After the Experiment

The TF XXI AWE occurred at the NTC in March 1997as planned. There were about a thousand officialobservers and controllers, augmented by subject matterexperts. Substantial data collection enabled evaluationof the results afterward. The key question that had tobe answered was: If information age battle commandcapabilities and connectivity exist across all battlefieldoperating system functions, will increases in lethality,survivability, and tempo be achieved?

Numerous additional questions were asked about theimpacts of specific technologies and weapons as wellas about the effects of force structure, doctrine,organization, and tactics, techniques, and procedures.To obtain answers, data were compiled not only fromevents at the NTC, but from predictive and post-NTCconstructive simulations, as well as from assessmentsof EXFOR training events before and after. Therewere special reports generated, including one ontraining effectiveness. The analyses and final reportsaddressed the potential of digitization, relying not juston the actual force-on-force encounter at the NTC,but on the opportunities to replay some events laterusing simulations.

There were differing views on the degree of success ofthe experiment, not unusual for an undertaking of suchmagnitude. Some observers did not attribute anoperational advantage to digitization, at least to the then-fielded version. The interested reader will find a rich

89Chapter 3

stock of publications25 available for perusal. The Army’sexecutive summary of the results is provided in Hartzog(1997), which highlights the achievements withoutdismissing the problems and challenges. The immaturityof certain capabilities along with the connectivityproblems did impact the activities at the NTC; however,the Army viewed the TF XXI as an experiment, not asan operational test. Its assessment, which usedqualitative and quantitative data, acknowledged thetremendous potential of digitization, and on balancecredited the overall success as much greater than anyspecific failure. The executive summary stated:

The TF XXI AWE was a highly successfulexperiment that exceeded the expectation ofplanners and participants alike. Not only did itreveal a clear vision of the dynamic potential inthe digital land force, but it incidentallyvalidated the Army’s whole approach toexperimentation. (Hartzog, 1997)

Management Structure for the Task Force XXI

The TF XXI AWE was primarily an Army event, whichsimplified the lines of accountability. Many Armyorganizations participated, reflecting the priorityaccorded the battlefield digitization program. Strategicguidance and direction came directly from the Army’ssenior leadership, which comprised a forum akin to a

25 Stanley (1998) provides some first-hand comments. A selectivebibliography on TF XXI compiled from many sources isavailable on the Internet (Gibish, 1997).


“Board of Directors” chaired by the Army’s Chief ofStaff. The Army’s Digitization Office (ADO)coordinated the integration of digitization activitiesacross the Army.

The Commanding General, Training and DoctrineCommand (TRADOC), had the overall responsibilityfor the TF XXI experiment. He was supported by theForces Command (FORSCOM) and the Army MaterielCommand (AMC). The Program Executive Officer forCommand, Control, and Communications Systems(PEOC3S) was accountable for the systems architecturesupporting the TF XXI AWE although other programexecutives provided essential capabilities such as forair defense and aviation. Because one PEO wasdesignated accountable for the SOS architecture, thedecision processes and integration of capabilities weresimplified behind the Wizard�s curtain.

The Director of Information Systems, Command,Control, Communications, and Computers (DISC4)evolved the technical architecture of standards andguidelines.

The convergence to the operational architecture andsystems architectures for the experiment relied onteamwork. An experimental working group, withgeneral officer participation, provided definition anddirection of the TF XXI for TRADOC. A TF XXIprocess action team and a coordination cell planned theexperiments, finalized the set of initiatives to be used,refined the mission threads with tactics, techniques, andprocedures, and coordinated with the users. The systems

91

Ch

ap

ter 3

Figure 3-8. Task Force XXI Architecture Management


architecture steering group, managed by the PEOC3S,with participation from the ADO, the DISC4, and AMC,factored the effects of the initiatives, coordinating thedevelopment activities required for the TF XXI SOS.

The “trail boss” for integration reported directly to thePEOC3S, directing efforts at the CTSF. The programmanagers, their teams and systems, came from manyorganizations to accomplish the integration on-goingthere, as described earlier. The Digital IntegrationLaboratory was one of several organizations supportingtesting. Figure 3-8 highlights many of these roles.

Comparison of the Two Case StudiesThe overviews of the two programs illustrate manycharacteristics in common. Both were ventures ofconsiderable scope and boldness of conception. Theirrespective organizations developed and appliedoperational concepts that were dramatic departures fromthose of then-current operations. To implementinnovative and leading-edge capabilities, they bothattempted to bridge technological gaps. This resultedin integrating some systems and components of varyinglevels of (im)maturity.

The as-built architectures were considerable in size—thousands of pieces of equipment were fielded forthousands of users. The DPS when installed requiredmore than 380,000 square feet of facility space. TheTF XXI required as many as 600 railroad cars totransport the necessary equipment and supplies fromFort Hood to the NTC at Fort Irwin.

93Chapter 3

The DPS comprised the integration of about 10 millionlines of code; the TF XXI required about 40 millionlines of code. For both programs there was acombination of developed software as well ascommercial off-the-shelf (COTS) software.26

Both programs were risky undertakings from manyviewpoints, including technology and size. While theArmy’s venture was greater in scope, both the DPS andthe TF XXI, if assessed as a single information system,would qualify as highest risk on the scale of difficulty.27

Complexity was amplified by the relationships betweenand among their individual systems. For example, theDPS developers characterized it as tightly coupledbecause the various constituent systems were highlydependent on one another to execute their own activities.For TF XXI battlefield digitization, the command andcontrol strategy applied has been characterized as tightlycoupled because of its need for near real-time

26 DPS used COTS for information processing and somecommunications capabilities when commercial products couldmeet performance requirements. The digital photogrammetricand cartographic applications required development of softwareand hardware. Later many such components were transformedinto commercial products.The Army introduced many commercial products into TF XXIto facilitate the SOS integration. Examples included thedistributed computing environment, the TOC backbone, andclient/server applications to complement systems like the ABCS.27 Per a model using function points, described in chap. 1. DPSand TF XXI fall into the project category of 100,000 functionpoints and 1 million function points respectively, which resultsin characterizing them as high risk based on the performance ofother projects of comparable size.


synchronization of information (Czerwinski, 1996). Theimplication of a complex, tightly coupled structure isgreater risk, unpredictable behavior, and vulnerabilityto failure according to Charles Perrow’s classicquadrants’ model (1984; Czerwinski, 1998).

Both the DPS and the TF XXI are characterized as aSOS. They were managed, developed, and operated byone organization, i.e., one agency and one service,respectively. Their development and operation weresubject to direction, rather than collaborative in nature.

The DMA management structure was more centralized.As a smaller institution, DMA had only one internalorganizational unit responsible for directing anddeveloping the DPS. The Army used multipleorganizations but focused leadership for the TF XXI byclearly designating accountability. Both organizationstransferred considerable decision-making power to a fewleaders to manage the respective ventures.

The DPS operational architecture was a postulation byDMA as to how the Agency’s workforce would operatethe SOS. The systems architecture included constituentsystems managed, developed, and operated by theDMA. Yet each was a large-scale system in its ownright, developed independently by different acquisitionteams and contractors, under broad program guidanceand direction. The independence of each system wasamplified by freedom of design and developmentmethods. The technical architecture, such as it was,imposed few standards and guidelines, a circumstanceinfluenced by the technological challenges of the

95Chapter 3

program. A common environment was applied to asmall degree.

The operational architecture for the TF XXI was anArmy view of a digitized brigade, although participants(and systems) external to the Army did engage in theexperiment. The systems architecture encompassedmany Army legacy and developmental systems, andmany platforms and configurations. The individualsystems served many different purposes, developed byvarious acquisition teams and contractors at differentpoints in time, although under overall Army direction.The technical architecture was evolved by the Armyand provided a set of standards primarily focused onC4I systems, but far more comprehensive than DMA’s.

Figure 3-9 provides a notional comparison of the DPSand the TF XXI. The more centralized, almost unitarycontrol and direction, moves the DPS closer to the originon the autonomy axis. The TF XXI appears as moreheterogeneous, with the breadth and numbers ofdifferent systems, and the use of legacy and someexternal systems. The DPS, developed using fewer than10 sites and operated from three principal geographiclocations, is depicted as less dispersed than the TF XXI.Although operated at the Fort Irwin NTC, the TF XXISOS reflects operational scenarios with informationassets geographically dispersed, even on a global scale.

There are differences in the two case studies relative totheir overall objectives and methodology. The DPS wasdeveloped as a production capability. TF XXI was a“product mature enough for an experiment to provide

96

Be

hin

d th

e W

izard

’s Cu

rtain

Figure 3-9. DPS and Task Force XXI Comparison (Notional)

97Chapter 3

data to the Army leadership for making investmentdecisions…” (Boutelle, 1996).

DPS, developed in the 1980s, generally resulted froma classic methodology characterized as a waterfall,and required nearly 10 years to deliver. Thearchitecture developed over time from a bottom-upapproach building upon the individual systems andtheir interfaces. The TF XXI architecture began top-down and resulted from a spiral evolutionarydevelopment process, which itself became a by-product of the experiment.

99

Chapter 4

The IntegrationExperiences

This chapter describes the events that transpiredduring the integrations of the DPS and the TF XXI.The narration builds on the overviews of the two

programs, their architectures, and their organizations asdescribed in the previous chapter.

With an understanding of the challenges both theseprograms faced, it is now possible to examine howeach approached the integration of the individualsystems in order to achieve a powerful new entity—asystem of systems (SOS). This chapter explores theconsiderable efforts undertaken to plan and preparefor each integration and what actually occurred.Among the characteristics that both ventures sharedis that neither had prototyped the integrated productpreviously and this circumstance made the integrationtask even more difficult.

A single integration facility with its supportingenvironment of people, processes, and infrastructurewas essential to manage the integration successfully.In addition, in the TF XXI case, the environment


fostered an evolutionary acquisition and developmentprocess. This chapter discusses these experiences.

Defense Mapping Agency’s DigitalProduction System

Before Integration of the Digital ProductionSystem

The DPS behavior was anticipated to be the union ofthe behavior of the individual systems that comprisedthe SOS. The DPS was perceived by developers andusers alike as a digital processing pipeline. The wholewas expected to be the sum of the parts. Each individualsystem would contribute its required functionalitywithin its allocated timelines and in a particularsequence. In simple systems where relationships arelinear, this is more generally true (Czerwinski, 1998).

Between the time of the DPS critical design review andthe start of its first integration event for initial operatingcapability (IOC), a series of formal requirementsverifications and functional tests were accomplished foreach individual system and its interfaces to othersystems. This series was repeated before the fulloperational capability (FOC) integration as well.1

In addition, a complementary and independent seriesof activities occurred to verify the correctness of theinterfaces. Because of the data dependencies among the1 At IOC, some interfaces, primarily those associated with theproduction management system, were not defined. Their testingand verification was completed before the FOC milestone.

101Chapter 4

individual systems, the exchange of simulated and realdata provided an additional verification for thecorrectness2 of the interfaces. One such activity requiredthat the systems generating data provide that data tothe systems consuming it, a check that consumers andproviders interpreted data format and content identicallyon both sides of the interface. A second verificationactivity used an independent agent to interpret theinterfaces and generate data in accordance with thatinterpretation, again requiring the consumers to verifyand the producers to corroborate. A tool called a“scenario generator” was used for this testing, whichwas conducted at the various factories of individualsystem developers. Some of these verification activitiestranspired as much as a year before the first SOSintegration phase. However, the early exchanges werehampered by late closure of some interfaces.Nonetheless, these efforts were critical in resolvinghundreds of interface-related issues.

Also before the integration, a few subsets of the SOSwere integrated and tested, as were commoncomponents shared between individual systems. Certainservices common for all systems in the SOS wereinstalled and tested at the various factory sites. Thenetwork services subsystem was one prominentexample.

A single site was planned for SOS integration, a newProduction Center, not yet operational. This reduced

2 Verification of interfaces focused primarily on syntax; however,the development of a MC&G data model enabled verifying datamessages of that type for content as well.


the impact of integration activities on the two existingProduction Centers, where significant production wasunderway. The first delivery of the integrated SOS wasintended for the new Center, where the almost exclusivefocus was on transitioning the DPS to production use.The new Center was similarly used for the secondintegration event for FOC. The facility comprised about200,000 square feet of space, half for the installed DPS,the other half to support integration activities. The sizeof the DPS facility requirement was indicative of themagnitude of the integration event.

At least 2 years before the SOS integration, detailedpreparation for its verification was underway. A seriesof demonstration events was defined. As each eventwas successful, it signaled the SOS readiness to supportthe mission. Some demonstrations focused on formalverification of all the information exchanges, formats,and content; others focused on inter-Production Centerrelationships. Based on engineering analysis,operational jobs and tasks were defined for inclusionin key demonstrations used for both the IOC and FOC.These were constructed to test the functionality andrequirements for the DPS as exercised by operators inend-to-end threads of operational activities. Theschedules for each integration were estimated basedon projections for each task in these demonstrations,factoring product difficulty, skill levels, and marginfor reserve.

The SOS integration was anticipated to be difficult. Thiswas well recognized at least 4 years3 before the FOC.DMA management had wrestled with difficulties in the

103Chapter 4

Mark 85 program, a far less ambitious undertaking. Theplethora of misinterpretations of MC&G feature datacaused difficulties in the earlier program and providedstrong incentives for early interface testing. The dataexchange programs for the Mark 90 DPS, as well asearly integration of some subsets, were used to reducerisk and resolve many problems before the integration,which they did.

DMA’s program management expected that thecommunications services of DPS would be problematic,and that interoperability issues would arise from thediversity of platforms and operating systems. Becausemany individual systems had technological challenges,the constituent systems were at varying levels ofmaturity. Their residual defects were expected tocontribute problems despite their assessed readiness.4

All participants recognized that the SOS integrationschedule was compressed and risky.

The Start of the Digital Production SystemIntegration Event

The installations and tests of individual systems at theDPS integration site were time-consuming and resource-intensive. Their engineering teams moved to the site to

3 An advisory board chartered by the Director of DMA assessedthe DPS integration as very risky and recommended earlyintegration efforts.4 A review ascertained that systems met requirements and weresufficiently robust to ship for installation and DPS integration.Discrepancies were assessed as acceptable or unacceptablebefore shipment Unacceptable discrepancies were correctedbefore shipment; nonetheless, many defects remained.


complete the integration testing with the other systems,and to support the SOS demonstrations. Informal testingamong the individual systems began as soon as possiblebecause of the difficulties anticipated.

When the systems were linked to provide initial operatoraccess to the SOS, assign jobs, and initiate the flow ofimagery, a staggering number of problems notpreviously manifested occurred, despite the earliertesting on individual systems. The integration eventrevealed the naivete of the assumption that the SOSwould behave as the sum of the constituent systems,that the proper functionality would occur, and occur inthe sequence anticipated. While difficulties wereexpected, the DPS behavior appeared unpredictable, andthe nature and number of problems were confounding.Ascertaining the source of problems overwhelmed thosereporting them.

The systems providing services were tightly coupledwith those systems using them. The data in one systemaffected and altered the behavior of another. In manycases responses were unanticipated if the data were outof range or interpreted in different and unexpected ways.Varieties of reactions from anomalous data occurredthat resulted in significant delays of hours, even days,in the flow of imagery and management data amongthe various systems, and even in permitting legitimateaccess to the systems. Because the DPS was intendedto be a map-producing engine fueled largely by imagery,this initial situation was catastrophic.

105Chapter 4

Problems mounted while personnel threaded throughlogons and job assignments. Systems with workstationsawaited scheduling, or when scheduled, awaited data,or when data arrived, prompted unexpected results,5 notall of which were adverse. Dependencies imposed bythe centralized production management and data andcommunications services were difficult to anticipateuntil the sequence of end-to-end threads of operationswere exercised in the demonstrations used forintegration.6 Then it quickly became clear that theeffects were definitely underestimated in severity.

The Integration Event Produced a System ofSystems Prototype

The phenomenon observed at the integration site wasakin to the birth of a new personality—the SOS—whichsubsumed or altered the behavior of the individualsystems. It became clear that, as an entity in its ownright, the SOS had received insufficient attention whencompared to the attention directed to the individualsystems. The importance of having a prototype of the

5 Not all unexpected results were viewed as adverse. Whencartographers saw the rich imagery detail at the newworkstations, they extracted more information than planned;however, this lead to unintended consequences on productiontimelines and mass storage requirements.6 The production management system managed and scheduledthousands of production events that required resourcedeconfliction. Each event had dependencies with predecessorand successor events, all frequently changing. As the DPSbehavior emerged, the numbers and complexities of theserelationships increased, resulting in a combinatorial problem.As a result, a complete set of SOS end-to-end threads wasvirtually impossible to identify, maintain, and test.


entire SOS became obvious with all the prescience ofhindsight.7 In one sense, the IOC integration eventproduced the first prototype.

Did this happen because the individual systems of theDPS were not well understood? No. Their individualbehavior was far better anticipated. Many systemsbenefited from previous development initiatives that hadbeen successfully implemented in the earlier Mark 85phase. Many had been prototyped with operatorsproviding feedback to the developers at the factories.What was not prototyped nor well understood was theDPS SOS as a functioning entity.

An equivalent revelation occurred in the second SOSintegration for FOC. After the IOC concluded, thedevelopers added functionality to the individualsystems. The production management system, inrelative contrast to the others, had substantial changes8

between IOC and FOC. In addition, the FOC milestonesignaled the readiness of the new three ProductionCenter operational capabilities (and new operationalinterdependencies). With such changes, the secondintegration event was as difficult as the first and theSOS behavior equally confounding; it was, after all, adifferent entity. Nonetheless, the participants were better

7 The DPS Mark 87 prototype was cancelled when it could notbe developed in time to support the individual systemsmanufacturing cycles. Integration revealed the importance ofthe prototype in providing insight into the behavior of the SOS.8 Because of schedule difficulties, only minimal systemproduction capability was delivered for IOC.

107Chapter 4

prepared and the environment supporting them wasconsiderably enhanced.

Operational and Engineering Leaders at theIntegration Site

Of paramount importance to the SOS integration wasauthoritative leadership and a unified team at theintegration site. The team had to be seamless betweenoperational and development communities, and betweenthe teams managing the individual systems andmanaging the SOS.

Before the onset of the SOS integration an ActivationControl Team (ACT) was established to support thetransition from development to production capability.The ACT was operating at the integration site nearly 2years before IOC. The Chairman of the ACT functionedas a commanding officer to provide a focused voice forthe operational community. A senior engineer, whoreported directly to the Program Executive Officer,represented the development and acquisitioncommunity. Both leaders were positioned at theintegration site to facilitate overall progress.

The problems resulting during the integration of theSOS demanded an almost continuous decision-makingprocess at the integration site. It was essential todetermine the causes of anomalies in functionality,develop solutions, and allocate responsibilities foractions to the teams of the individual systems.Requirements had to be reviewed to preserve the needsof the operational community as engineering decisions


were made to accomplish the integration. The views ofthe operational community had to be clear because theindividual system capabilities were changing, and theSOS was in a dynamic state.

On-site processes for decisions were coordinated withacquisition processes to maintain program controlwhile supporting the resolution of engineering issues.Daily engineering boards, acquisition boards, andtransition activity boards were all augmented withcrisis-like schedules to accommodate the continuouscoordination of testing and integration activities.Removing conflicts, setting priorities, and schedulingevents required an iron hand.

A real threat was the prospect of a marching army ofengineers and developers halted in their tracks byproblems, with the effect of costly schedule delays.Authoritative decision processes, well-organized issuedispositions, and alternative plans for daily activitieswere among the means used to avoid unnecessaryimpacts and optimize time and resources.

At first the DPS integration was a confrontationalprocess as the engineers of the individual systems,surrounded by users of many different persuasions,required resolution of issues, the source of which andthe disposition of which were not immediately obvious.The importance of the on-site engineering boardoperating at a single geographic location cannot beoverstated. The Senior Engineer was responsible forthe SOS and had the authority to adjudicate betweenthe competing demands of the individual system

109Chapter 4

managers, who were given “face time” to present theiranalyses and recommendations.

The Need for An On-Site Engineering Board

The DPS Engineering Review Board, lead by the SeniorEngineer and staffed by the SOS cadre and contractorassets, determined the responsibility for solutions totechnical issues and problems in the SOS. This was anon-trivial undertaking because it required anunderstanding of the SOS entity, the means to determinethe cause of the problems, and the means to understandhow to resolve them. This staff was small9 comparedto those management and support personnel for theindividual systems. However, it was a staff withknowledge of the DPS as a SOS.

In contrast, program managers and engineers of theindividual systems had spent years managing theirventures, and identified strongly with them. They andtheir teams were not in the best position to assess theneeds of the DPS, although all were committed to makeit a success. Their diverse views of the SOS, like thetale of six men viewing the elephant, werecomprehensive but specific to individual systems. Theywere less able, as a result of their specializedperspectives, to grasp the holistic behavior.

The individual system development teams also feltsignificant ownership of their own issue solutions; againthis was a situation best resolved by the SOS

9 One-twentieth of the personnel affiliated with the individualsystems.


Engineering Review Board. It was important to maintainobjectivity and understanding to arrive at the optimumsolution for the SOS.

Any disputes (and the resulting funding consequences)of the SOS could be appealed to the higher authority ofthe DPS Program Executive Officer. This was invokedrarely because the on-site leadership was recognizedas empowered.

Adding People to the System of Systems Cadre

For the DPS program management, full realization ofthe level of resources required and the extent of expertiseneeded to make a SOS out of a set of individual systemsdid not come until the beginning of the IOC integrationactivities. Each day brought more problems requiringdisposition and resolution. More people were requiredto handle the increased activity. Confounding theparticipants was the circumstance that causes and effectsof problems among the various systems could not bedetermined easily because of the complexity of the DPS.

Not only was there a need for more people, but forpeople with knowledge of the DPS as a whole (incontrast to that of the individual systems). Theyneeded to understand the end-to-end behavior of theSOS that resulted from interactions among individualsystems. Unexpectedly, a partitioning of knowledgeabout the SOS had occurred even in the SOS cadre.Members had worked so long with the teams of theindividual systems that their own knowledge hadbecome specialized.

111Chapter 4

The amount of information an individual needed toabsorb and understand overwhelmed many. By the timeof the integration, many individuals had participated inthe program for nearly 10 years. They had engaged inDPS reviews, technical exchanges, interfaceimplementations, and pre-demonstration activities. Yetdespite this, they were at a loss to explain behavior thatwent well beyond the constituent systems.

The on-site operational leader characterized the situationas follows:

We had some idea of how many (people) wereneeded and did in fact program for them, but whatwas unknown was the amount of informationpeople had to absorb and understand.

Attempts were made to supplement the SOS cadre, butat the time of greatest demand, knowledgeable talentwas in short supply. Understanding the SOS in all itsmanifestations required vast amounts of time, and thisdetailed knowledge was exactly what was needed forthe integration. This shortfall was unanticipated andtherefore not addressed, even in an elementary trainingprogram or in one that would have transferredexpanding knowledge. With schedule pressures and thenumber of concurrent activities at integration, this wasproblematic. The situation resulted in performanceabove and beyond the call of duty for those who werein a position to contribute, and dedicated people wereconsumed by the effort.


Problems Understood and Resolved Faster UsingOne Site

The physical site for integration with its supportingenvironment afforded the opportunity to learn theemerging SOS from first-hand observation. It offeredimmediate access to details of behavior, allowing morerapid resolution of problems to prepare the DPS forproduction readiness. The demonstration leader,reflecting on the experience, said:

Problem resolution required continuous face-to-face communication.

The need for communication, coordination, andcollaboration among participants was intense.Information exchange and analyses of issues werebest accomplished with on-site personnel who werein a position to observe, understand, and resolve. Thiswas a shift in strategy because previously keyengineering resources remained at factories where therobust infrastructure and large resources werepositioned to respond.

Participants were prepared to deal with the complexitiesattendant to their individual systems. This wasunderstandable because these developments weresubstantial undertakings in their own right. Eachrequired expertise to develop; each was technologicallychallenging; each had an individual user community tosatisfy; each had specific functionality to demonstrate;and each had performance and reputations tied to thesuccess of the individual systems.

113Chapter 4

However, the SOS integration event demanded thatengineers pool their diverse and detailed knowledge ofindividual systems to begin to achieve understandingof the integrated product—akin to dynamicallydeveloping a knowledge base about the DPS. Theenvironment at the one physical site enabled theengineers of individual systems and the SOS cadre tocompile, exchange, develop, and aggregate the diverseinsights to describe the whole. Then, in turn, the on-site teams for the individual systems translated theseunderstandings for their counterparts at the factories ofthe individual systems.

One Site Helped Make a System of Systems Team

The environment was essential to reforming a newteam—the SOS team. The location of the integration atone site coalesced team work. It helped shift participantsfrom their more parochial individual system views toone focused on the SOS. It helped transform many teamsdedicated to succeeding with their parts to one teamintent on succeeding with the whole.

The synergy of allegiance to the national mission, totalprofessionalism, and pride of being part of one of themost challenging ventures of its kind emerged for allparticipants. Commitment, integrity, and self-sacrificeabounded. Today, even with the lapse of years, manyparticipants still view it as one of the greatest and mostrewarding experiences of their professional lives.


Management Tools at the Integration Facility

At the outset of the DPS program, a number of processesand practices were required by the program leadershipto manage the individual system acquisitions. However,how these functions were implemented was at thediscretion of individual system managers who hadconsiderable latitude in design and implementation. Forexample, different tools and processes were used toaccomplish project management, resource scheduling,action item identification and tracking, issue resolution,and configuration management. Many of these toolswere brought in and installed at the integration facilityby individual team managers.

The means used often reflected individual corporatepractices of the developers. Because personnel affiliatedwith a particular system were familiar with and trainedon specific tool capabilities in their own companies,they naturally selected these tools to supportmanagement of their acquisitions. Government teamsmanaging these acquisitions then also became familiarwith these individual implementations.

Limited Interoperability in CollaborativeMechanisms

At the outset of integration, project managers of theindividual systems anticipated the need for their ownsupporting infrastructure at the DPS integration site.Their first priority was the relationship between theirsite team and their counterparts at their factory. Thesite team required key engineering resources and

115Chapter 4

development assets to respond to the problemsidentified at the integration site. They planned andpositioned themselves accordingly.

During integration, the priority quickly shifted to theirrelationships with the teams of other individual systemsand the SOS cadre—driven by the needs of integration.Before turning to the factories to correct the problems,they needed more understanding of the nature of theproblem, what to fix, and an agreed-to-plan. All ofthis required intense collaboration, communication,and coordination with their counterparts at theintegration site through processes managed by theEngineering Review Board.

While their support processes and tool sets wereappropriate for managing their own teams, they werenot necessarily compatible for exchanging informationwith managers of other teams and the SOS cadre. Thissituation interfered with and slowed coordination andcollaboration among the various individual systemteams. The diversity in approaches subsequently wasmitigated by decreed standards for selected informationexchange as well as for common tool sets and systemsspecifically to support the needs of SOS integration andissue resolution.

While this facilitated the necessary collaboration, theshift to commonality resulted in the need to assimilatenew tools and processes at a time (integration) whenthe resources and schedules of the individual teams


already were stressed.10 In addition to elevating schedulerisk, cost impacts (acquisition of tool sets and personneltraining) resulted. Difficulties also arose fromreconciliation of the level of detail necessary to scheduleactivities and resources at individual system levels withthose necessary at the SOS level. These impacts couldhave been reduced if the collaboration needs had beenaccommodated early on at the program outset and wellbefore the start of the integration.

Examples

A good example is that of discrepancy tracking. TheDPS program guidelines defined hardware and softwarediscrepancies as defects in meeting programrequirements. The methods, priorities, and timelines fortheir resolution were based on the level of severity ofthe defect. However the specific implementations(processes, tools, and systems used) to accomplishdiscrepancy handling at the factories where softwareand hardware were developed was left to the discretionof the management of the individual system teams.

For the SOS integration, it was agreed to migrate alldiscrepancy information to one computer system tocontrol the entire SOS status, including that of theindividual systems. The implementation of thismigration lagged. As problems were identified duringthe integration event, what quickly became apparentwas that the various individual system implementations

10 While this need for commonality was foreseen beforeintegration, the migration to common tools sets and commonpractices fell behind schedule.

117Chapter 4

had spawned differing interpretations of severity levels,categorizations, and the content of informationrequired—despite exchanges among the individualsystem teams before the integration event.

When discrepancies arose in one system that affectedothers, collaboration and coordination were critical forrapid resolution and disposition. However, informationand time were lost in translating the local dialects amongthe teams, then re-translating for their factorycounterparts. These inconsistencies eventually werereconciled, but it required a year before all theinformation flowed smoothly among the various teamsat the site and the factories. This situation burdened analready-stressed Engineering Review Board.

Another example is illustrated by the varieties of toolsand processes supporting configuration management ofthe hardware and software baselines. The experiencewas analogous to that of discrepancy identification andtracking in that the variations in the individualimplementations caused difficulties in managing theSOS baselines as a whole. There also was a need todifferentiate the level of configuration item for whichthe management at the integration site would identifyand control the baselines, versus the level required bythe factory sites.


Adding Tools and Infrastructure to the IntegrationFacility

Several processes and automated tools proved beneficialin supporting integration management for the DPS. Inaddition to discrepancy tracking and configurationmanagement, these included tools for requirementstraceability, action item tracking, and scheduling dailyevents and resources (people, equipment, and facilityspace). The resource scheduling required the capabilityto project multiple contingencies for every eventbecause activities did not progress as predicted.Eventually the SOS team established one configurationmanagement system for the SOS baselines at theintegration site, one SOS discrepancy tracking system,and one action item and issue tracking system.

Integration activities demanded a common means forcommunicating and documenting information. Acommon work environment unified the teams ofmany individual (and disparate) managementinfrastructures. Because the integration site was aProduction Center, its office automation environmentprovided a common means for interpersonalcommunications for participants.

A telecommunications infrastructure united theProduction Center that functioned as the integration sitewith the other two Production Centers. This wasbeneficial to link the Agency’s managers, who wereresident at all three locations. It was also essential tostage DPS software deliveries to all three sites. Laterthe infrastructure supported the Agency’s operations

119Chapter 4

when DPS was ready for production. An equallynecessary communications infrastructure linked theintegration site with the development factories ofindividual systems. This essential infrastructuresupported interpersonal communications, videoteleconferencing, and transmission of software changes,data, training materials, and documentation.

A Digital Production System “Core”

The implementation of a SOS “core” to augment thetesting strategy was started before the integration event.This was a mini-DPS called the “DPS System TestMode.” Analogous implementations have been used onother information technology projects where there is aneed to assess changes without perturbing theoperational mission. The offsite facility in support ofthe Cheyenne Mountain Complex is similar in concept.

The DPS System Test Mode was a minimum set11 ofall the unique hardware components and all the softwarebaselines of the individual DPS constituent systems.As such, it was a specifically configured standalone SOScomprised of all individual systems, detailed to theirhardware and software components. It was equivalentin functionality to the operational DPS except for thoseconfigurations when unique hardware components were

11 There were some caveats to “all the unique components.” Themajor exception was the imagery fetch and disseminationcapabilities that were not included in the system test mode as adedicated asset, but could be redirected from production assetswhen needed.


deliberately omitted because they were unavailable. Ithad logically independent, parallel networks andphysically separate data bases and could operateconcurrent with, but isolated from, production use orother integration activities.

The System Test Mode supported end-to-end multiplethread testing for the DPS and included tools thatallowed software flows to be altered, bypassed, ormodified to facilitate testing of a particular configurationitem. It was used for testing and build verification ofthe SOS and came under the management responsibilityof the Engineering Review Board. While it was notavailable in time for the IOC integration, it was usedfor the second integration event before the FOCmilestone, and subsequently for DPS maintenance andevolution. It was installed at the integration site, andlater at the other two Production Centers, and was usedfor geographically distributed (inter-Center) testing. Itwas valued as a necessary tool of the SOS integrationenvironment.

The purpose of the System Test Mode was grounded inverification, not for concept definition or designevolution, as a SOS prototype might be applied. It couldbe tailored into specific configurations as needed fortesting except for certain kinds of stress loading andresource contention (because the equipment suite usedwas intended to be minimal). Consequently, it provideda capability to assess the effects of changes to theindividual systems and to evaluate them on the SOSholistically using a complement of known results. Whensufficiency of testing and correctness were established,

121Chapter 4

the changes then were migrated to the installed SOSbaseline.

As test results were gradually compiled, a sizeable setof benchmarks accrued. These eventually included allend-to-end flows and associated test data as well astraining data. Engineering analysis was used to selectthe subsets of the benchmarks to be used, based onwhere the risk was highest, or the change greatest, oreven which test was on the critical path. Consequently,the quality of the change processes improveddramatically and adverse side effects from changescould be determined earlier and with increasingpredictability.

A key adjunct to the System Test Mode was itscompanion set of tools, including the scenario generator,which could initiate message traffic and file transfers,act as a passive or active receiver of service requests,and then respond accordingly.12 This tool was evolvedbased on information gained from the earlierindependent data exchange program for testing theinterfaces between individual systems.

The System Test Mode proved invaluable to supportevolution and maintenance of the DPS after FOC. Afterthe production equipment was operational at theProduction Centers, the hardware componentsautomatically could be extracted from production use,

12 It acted in accordance with the interface specifications. If amessage received was intended to produce a certain response,that response was generated to the appropriate systems as partof the verification.


configured, and dedicated as testing and verificationassets to assess the consequences of changes across theSOS. This avoided inadvertently introducing adverseside effects on the production floor.

Conclusion of the Integration Phase

The successful completion of a series ofdemonstrations comprised the conclusion of theintegration phases for IOC and FOC. In addition tothe message exchanges, the series included theproduction of four MC&G products for IOC and eightproducts for FOC. The products selected exercisedmajor aggregates of functionality, were high priority,and were considered difficult to generate.13 Forexample, a coastal chart was selected for FOC becauseit included aspects of both land and water featureattribution. In the pre-DPS era at DMA, some of theseproducts required 2 years to produce.

The two demonstrations for IOC required 7 months;the four demonstrations for FOC required 10 months.Their completion, coupled with the correction of themore severe discrepancies, signaled readiness for thenext program phase, which after FOC was production.

13 For IOC, the four products produced were the 1:50,000topographic linemap, digital terrain elevation data at level 1resolution, and two types of point target graphics. For FOC, theeight products included a l:50,000 topographic line map, a jointoperations graphic-ground, a coastal chart, a digital featureanalysis data product at level 2 resolution, a digital terrainelevation product at level 1, and three terrain contour matchingproducts.

123Chapter 4

The U.S. Army’s Task Force XXI

Before Integration of the Task Force XXI System ofSystems

The difficulties of SOS integration were demonstratedagain when the many individual systems intended forthe TF XXI experiment first were integrated in June1996. Systems arrived at the Central Technical SupportFacility (CTSF) at Fort Hood for the final integrationphase. Despite considerable preliminary testing, whenthe systems were interfaced, they failed in the firstexercise, LZ Phantom, to support operators. Onemanager described the initial situation as “disastrous!”This circumstance stems from the same causesexperienced in the DPS integration. This can beattributed to the complexity of the interactions of theinformation systems independently and individuallydeveloped and tested, and the emergence of behaviordifficult to predict and comprehend when the systemsare first integrated.

The Army had prepared for this event through anextensive and complementary series of tests andverifications of the participating systems, coupled withmultiple training events. As described earlier, previousactivities included a series of large-scale experimentsbeginning with Desert Hammer in 1994. Using acombination of Advanced Warfighting Experiments(AWEs), simulations (virtual, live, and constructive),and actual exercises, individual systems and subsets ofsystems gradually were tested. By advancing the


digitization concept through exercises and experiments,the Army developed and integrated some systemsrequired for the TF XXI SOS, testing them inoperational settings. Holcombe (1998) provides a goodreference for certain aspects of this progression. Duringthe Desert Hammer AWE at the NTC in 1994,experimentation with a partially digitized armor forceused: an appliqué-like capability; Tactical OperationsCenters (TOCs) equipped with all-source intelligenceworkstations; and a Brigade and Below Command andControl System. Similar capabilities were later used inTF XXI.

In August 1995 Exercise Focus Dispatch centered onheavy forces and used a combination of virtual and livesimulations with a field exercise. The digital passageof information was explored, such as for moving largeamounts of imagery and graphics. A key objective wasthat of evolving techniques, tactics, and procedures, butthis was complicated by connectivity and contentiondifficulties. One of the important by-products wasprogress on testing the communications system withthat of the radio. This was one of the core systemsincluded in the SOS for the TF XXI. The exerciseexposed the problems of factoring “real world” effects,such as those from hilly terrain, into the communicationstraffic between tanks.

The Warrior Focus AWE at Fort Polk in November 1995examined digitization in a light infantry environment.The results were used to improve command and controlsystems. During the experiment, a computer systemlinked to the GPS, the Brigade and Below Command

125Chapter 4

and Control System, a commercial communicationstechnology test bed, and voice and data radio systemsall were exercised. Also included were a lightweightTactical Operations Center, versions of the ManeuverControl System, and a secure packet radio system.

The tactical internet, also one of the core capabilitiesof the TF XXI, was exercised at multiple events,including some using a variable message format. Therewas one virtual exercise of the tactical internet nodesfor architectural compliance. All of these tests were usedto identify and correct a variety of problems.14

In the early planning phases, a TF XXI SystemIntegration Plan was developed that provided guidancefor testing and experimentation to integrate the criticalelements and systems (TF XXI Plan, 1996). It focusedon risks and issues that had to be resolved for the SOSintegration to proceed. Agreed-to methods to testresolution, such as analysis or modeling and simulation,were identified and used by the teams managing theindividual systems. As the target battlefield architectureevolved for the digitized experiment at the NTC, eachindividual system in the SOS complement for the TFXXI also evolved by subsequent enhancements and byincorporating applicable results from previous tests andexercises.

Each initiative or modified system in the TF XXI SOShad its own integration, testing, and verification tasks

14 The tactical internet did not achieve the level of reliabilitydesired by the TF XXI event during the force-on-force encounterat the NTC. (Hartzog, 1997; Caldwell, 1997)


to complete successfully, and then each underwentanother level of testing in preparation for an integratedSOS. Examples of testing for the SOS included that ofthe all-important message formats and data protocols,where considerable effort was expended. Theintegration plan was followed to resolve issues by themethods agreed.

The plan partitioned the SOS integration into logicalcomponents and subsets, such as the Army BattleCommand System, the various platforms, the TacticalOperations Centers, and the tactical internet. Thoseinitiatives not integrated through this strategy wereindividually integrated into the tactical internet. Figure4–115 illustrates this approach.

Because the TF XXI focused on experimentation, therewere many initiatives in the SOS complement that werein early stages of development. A set of success criteriawas developed to ensure that these initiatives weresufficiently mature and deployable in time for the NTCfield activities. Fifteen criteria were used in assessingfactors such as safety, manpower availability, andreadiness. As a result, the number of initiatives waspared to the final 72 in time for fielding at the NTC.

A Certification Process Also Established

When testing of the individual systems was completed,another stage of testing began. This was administeredindependently by the Army’s Digital Integration

15 Extracted directly from the TF XXI System Integration Plan(1996).

12

7C

ha

pte

r 4

Figure 4-1. Integration of the Task Force XXI System Architecture


Laboratory (DIL) and generally occurred just before asystem’s shipment to Fort Hood.16 This series of testsaddressed compliance with the technical architecture,with military specifications and standards (such as themessage format), and testing for interoperability. Thisprovided an independent verification for each systemand its interfaces. The DIL complement of testing wasintended not only to ensure compliance with standardsand architecture, but also to establish a certificationprocess. Individual systems in the SOS for TF XXI hadto pass DIL testing before proceeding into the finalintegration phase at Fort Hood.

Use of Fort Hood in Task Force XXI

Originally the planned use of Fort Hood was primarilyfor training for TF XXI, although the site had a historyof integration support. The Army’s Tactical Commandand Control System Integration Office had a facility atthat site. Previously it had supported large contingentsof soldiers for training on new capabilities. However,the role of Fort Hood’s CTSF initially was viewed as anominal one for the TF XXI integration. It was to beused for correction of discrepancies in the softwareidentified by soldiers in training.

Preparations for the TF XXI experiment previously hadinvolved use of other facilities. Fort Hood was to bethe place where the SOS was brought together for thefirst time in June 1996. All the existing systems would

16 Some testing occurred before integration at the Fort Hoodcentral technical support facility, which functioned as anextension of the DIL.

129Chapter 4

be integrated with the new capabilities to provide a SOSfor training purposes as well as to evolve the finalconfiguration for deployment to the NTC. The plan thenconcluded with a series of training events on theintegrated SOS from August through December 1996.

The Reality of System of Systems Integration

In June 1996, the date on the overall schedule foreverything to be in place, most systems17 arrived at FortHood but few were ready for a SOS integration. Thiswas demonstrated during the first exercise with theoperational community. The LZ Phantom exerciseconducted there in June 1996 provoked the comment,“disastrous!”

When the individual systems were hooked together, theSOS did not function. Despite all the previous testing,the TF XXI SOS could not support the Army’s EXFORin executing the operational mission. The complexityof the integration was fully revealed. The TacticalOperational Centers did not work well together. Therewas questionable performance between vehicles. Therewere difficulties in scaling to larger numbers ofoperational components, as well in supporting variableconfigurations, which numbered as many as 180.

The integration plan reflected the expectations that,given the circumstances of the earlier complementarysuite of testing of the individual systems, the integrationof the SOS would occur readily in a relatively short

17 A major exception was allowed for the tactical internet, whicharrived some weeks later.


window of a few months. The early days of theintegration proved otherwise. All of these capabilitieshad been developed independently by programmanagers who believed they were building forintegration. Despite their best efforts at developing orenhancing systems to fit within an overall architecturalframework, testing systems and interfaces forcompliance, and building upon previous exercises, theindividual systems did not interface together, nor didthey interoperate, fundamental requirements for a SOS.

Vitalizing Fort Hood’s Central Technical SupportFacility

When the reality of the challenges emerged, the solutionwas also at hand. The Program Executive Officer(PEOC3S) for the TF XXI experiment immediatelyresponded to the problems by increasing the leadershipfocus on the integration, augmenting the engineeringsupport, and evolving the role of the Central TechnicalSupport Facility (CTSF) at Fort Hood. It now becamea facility:

for rapid integration of dissimilar software andhardware systems, through real time interactionwith soldiers, contractors, testers, programmanagers, and the requirements community.(Boutelle &Grasso, 1998)

This decision increased the discipline and rigor of theintegration process. In retrospect, this was a stroke ofgenius because it created the environment behind theWizard�s curtain to support the integration of the SOS,

131Chapter 4

and thus its creation and delivery to the NTC in time tosupport the TF XXI exercise.

The PEOC3S also designated a leader in July 1996 tomake the integration happen at the then-growing CTSF.This individual became the decision-maker “trailboss,” adjudicating solutions to engineering issues, andmoderating acquisition decisions to align withsuccessful integration. This appointmentacknowledged the need to arbitrate among thecompeting interests of the individual programmanagers and to prioritize the engineering of the SOSover that of the individual systems.

This empowerment was endorsed by the Army’s Chiefof Staff and the Commanding General of TRADOC,and it brought quick acknowledgement throughout theArmy organizations, including all the participantsbehind the Wizard�s curtain. These included theprogram managers of the individual systems, whocontrolled the bulk of the necessary resources with theircontractor teams.

The response to the integration difficulties also includedenhanced engineering resources. In fact it was asubstantial response, sequestering many engineers atthe CTSF and increasing the core staff to nearly 100engineers. While approximately l,200 personnel and 48separate companies supported TF XXI overall, onlyseveral hundred personnel were resident at theintegration site at any one time. Many remained at thedevelopment factories for the individual systems, or atother sites, tethered by communications infrastructure


to the CTSF. However, there was a substantialenhancement at the site, and the CTSF grew markedly.

The TF XXI participants initially were apprehensive atthe convergence of large numbers of program managers,engineers, testers, and developers at Fort Hood.However, one leader, on observing the team spirit thatevolved there, commented on the magic of it—that sometransformation occurred—and that everyone becamededicated to “making it happen.” When one participantwas asked how many teams were at the CTSF, he replied“one.”

At Fort Hood the entire focus of the resident teambecame the successful achievement of TF XXI. Theuse of the CTSF with its supporting infrastructuregenerally is recognized as a major contributor to taskforce success.

Experimentation and Training at the CentralTechnical Support Facility

A tremendous synergy arose at the CTSF with thephysical collocation of the operational community andthe development community. The EXFOR in residencewas partitioned into operational enclaves (such asbattalion Tactical Operational Centers) for training.Soldiers engaged in a series of exercises that threadedoperational activities end-to-end. The participantsdubbed them “battle drills” because they providedsubstantive training experiences. In addition, theyserved to test the integration of the SOS.

133Chapter 4

As operators trained with new capabilities in mission-like activities on a daily basis, they provided instantfeedback to the developers. Fort Hood changed into afactory-like setting, where vehicles and platforms werenewly retrofitted, and the next day soldiers were trainedon them. Developers stood next to operators. Whencapabilities did not work, they were modified by thedevelopers, sometimes at the site, then reintegrated, andretested in an iterative spiraling process. Wherenecessary, additional functions were added. Operationaland developmental compromises about the SOSsometimes were made in near real-time.

Many systems relied on commercially availablecomponents. Many pieces of equipment tested werecommercial off-the-shelf, some were ruggedversions and some were of military specification.But there was great diversity in vehicles andplatforms, and accommodations had to be made tomount and install capabilities. Alternatives forinstallations were evaluated by soldiers to minimizeinterference when operated. There were more than40 different vehicle types with new equipment andnearly 1,000 vehicles with about 180 differentconfigurations (Goedkoop, 1997).

Operators were positioned to learn throughexperimentation and to use their knowledge to drivedevelopment. The developers gained considerableunderstanding of operational needs and responded. Bothcommunities derived synergy through their collocationat the CTSF and the activities occurring there.


While this process was on-going, problems attendantto the integration continued, but resolution was focusedby the leadership and this phase progressed. Activitiesin support of integration and training operated 7 days aweek, 24 hours a day. The battle drills revealed defectsin the integrated product, but gradually progress wasmade, which resulted in a SOS entity that was able tosupport more comprehensive activities. The events atFort Hood concluded successfully with the brigade-on-brigade force-on-force exercise in December 1996.

On-Site Engineering Board Essential

The on-site engineering board processes, managed bythe trail boss, were critical to the quality of architecturalchanges, resolution of technical issues, and acquisitiondecisions. Adjudication for resolution of issues anddiscrepancies occurred quickly and continuously.Having access to the right people at the site to resolveproblems was essential. The requirements forengineering expertise were demanding. To fitcapabilities within the overall framework wherestandards did not exist were anomalous, or wereinsufficiently detailed, decisions were made favoringemerging standards and future SOS integrations. Theadjudication processes were open for participation.Coordination was facilitated by the presence of programmanagers and participants at the site.

The number of ongoing daily events necessitated a firmdiscipline for defects correction. As changes werepropagated continuously, there was a need for rigorousconfiguration control of the SOS hardware and software

135Chapter 4

baselines being integrated and tested. Over time, controlincreased in extent but was never completely managedby the CTSF because the change processes for someindividual systems continued to be managed by theirrespective teams at their own sites or factories.

Exercising Authority Behind the Curtain

As discussed earlier, there was one primary ProgramExecutive Officer for the TF XXI architecture.However, responsibility, management, and funding forthe individual systems remained with the programmanagers of those individual systems in the SOScomplement. Many of these managers, in turn, reportedto other program executives such as those responsiblefor ground combat and aviation. The trail boss was thedesignated decision-maker for the integration at theCTSF, but the program managers retained the majorityof the assets used. This resulted in a form of creativetension that required consensus for some decisions.

Because there were many systems in the TF XXI SOS,there were many managers and many differentinterpretations of needs and priorities. Some whomanaged legacy systems felt constrained because theyhad only limited funding programmed for a venture ofsuch scope. This situation heightened concerns.

During integration, funding pressures increased asdecisions resulted in unanticipated and unprogrammedchanges to individual systems and interfaces.Requirements dynamically emerged or evolved.Sometimes the missions and funding for individual


systems competed with the needs of the TF XXI SOSintegration. Three factors eased this situation. First, theempowerment of the trail boss established a decisionauthority at the site. Second, the collocation of theoperators and the developers improved theunderstanding of operational needs and reduced frictionin favor of collaboration. Third, the participation ofprogram managers and teams at the CTSF aided theresolution of disparate priorities and lent reconciliationto the decision-making process. The views and insightsof the individual program managers were important tothe quality of the decisions. Their availability supportedthe needs of a process that ran continuously.

Lines of authority and decision processes were wellestablished and important to overall integration success.As discussed in the program overview, strategicdirection came from the Army’s most senior leadership,using a forum analogous to a Board of Directors, chairedby the Army’s Chief of Staff. The “Board” providedauthority and remained engaged throughout the TF XXI.An Experimental Working Group of general officersprovided oversight for TRADOC.

An experimental coordination cell was comprised of a“Council of Colonels” to focus attention on key issuesand to coordinate to achieve unanimity in the users’viewpoint. They wrestled with overarching TF XXIissues such as the effects of force restructuring andadjustments in overall schedules. They had recourse tothe oversight boards as necessary. The Chairman of theCouncil of Colonels, who was resident at Fort Hood,was cognizant of the on-site operational issues and able

137Chapter 4

to coordinate the operational viewpoint. The TF XXIIntegration Office and the trail boss reported directlyto the TF XXI Program Executive Officer.

Many Army organizations were involved with the TFXXI, and they were represented in the working groups,steering groups, and coordination cells that wereestablished. The lines of authority were clear to theparticipants. The day-to-day decision making was madepossible by empowerment and by teamwork at the site,and there were clear avenues to follow when issuesappeared too contentious or significant. Generally thisstructure was very effective.

Integration Environment

The role of the CTSF changed dramatically in responseto the challenges of the TF XXI integration. Becausethe schedule was compressed, management was notprepared to acquire an environment specifically adaptedfor integration, although the infrastructure was enhancedduring the integration activities. This situation improvedafter TF XXI when the Army was able to assess theprocesses conducted at the CTSF as critical to the overallsuccess of the experiment.

Certain infrastructure that contributed to managementof the integration was in place early or evolved. Onecritical capability was that which supportedconfiguration management of the changing SOSbaseline. Another was the communicationsinfrastructure. The latter tethered the factories of theindividual system teams, which were geographically


dispersed, to the integration site. It also linked certainengineering simulators necessary for integration, andconnected to simulation centers that were used fortraining scenarios.

Initially the bandwidth of this communicationsinfrastructure was severely limited but eventuallybecame sufficiently robust to support videoteleconferencing for the management teams. The CTSFwas deliberately linked to the DIL, which in turnprovided connectivity to many other sites.

Certain office automation support also was available.The CTSF supported participants with whiteboards,personal computers, and a distributed computingenvironment.

Infrastructure for Scheduling

The integration schedule was a risk. This resulted fromseveral factors, not the least of which wasunderestimating complexity. Some individual systemslagged in meeting their own development and testingschedules, compressing the integration schedule further,and increasing the difficulty. The initial planinsufficiently accounted for the time needed to resolvethe problems encountered in integration, and a high rateof changes resulted from the collaboration of operatorsand developers.

Initially there was no detailed integration schedule,rather individual and separate understandings of it. Thisled to difficulties in coordinating activities among the

139Chapter 4

various teams, compounded by daily activities for SOSintegration that were dynamic and changeable.

Project management tools were used to develop dailyschedules, although their use was resource-intensive.Daily activities were posted, and a web-based approachwas successfully used to disseminate schedules. It wasessential to have contingencies available, and testdirectors at the integration site were prepared forschedules with many alternative plans. When one setof activities was prematurely aborted, others couldproceed. Consequently, the trail boss was able tosucceed in pushing the daily integration events forward.

Configuration Management Challenges

Configurations of individual systems were not alwayssynchronized with those in the SOS baseline. This wasexacerbated by the high rate of changes of the systemsin the SOS, and because some managers other than thoseof the CTSF leadership controlled the baselines of someindividual systems.

Teams at the CTSF sometimes found themselves inthe situation of preparing for an event that had beenrescheduled, or teams found themselves in thesituation of testing their system in conjunction withanother that had a different-than-expectedconfiguration of hardware and software, a situationinvalidating the test. Corrections to discrepancieswould not always succeed because of hardware andsoftware changes made elsewhere. Parts of the


configurations were controlled, while others were not.Changes and reiterations of tests resulted.

There was insufficient opportunity and means todevelop a suite of SOS test results and compilebenchmark data sets for the operational threads throughthe SOS, such as from the battle drills and exercises atFort Hood. This made some regression tests moreinformal than formal, sometimes with unexpected andundesirable consequences.

Conclusion of the System of Systems IntegrationPhase

While the LZ Phantom exercise in June 1996 signaledthe start of SOS integration, analogous training exercisesmarked interim progress and completion. Platoon Lanesoccurred from August through mid-September,Company and Team Lanes in October, and finally aTask Force/Brigade Team Field Training Exercise inmid-December 1996.

These built on the end-to-end scenarios postulated asto how a brigade task force would use the newdigitization capabilities. The SOS providing these,though still perturbed by immaturity levels in some keysystems needed for the experiment, was deemedsufficiently mature for the field experiment. The piecesof equipment were shipped from Fort Hood to the NTCin December 1996 to support the March 1997engagement.

With leadership and commitment, the teams behindthe Wizard�s curtain coped. Their goal was to

141Chapter 4

deliver a “product mature enough for an experimentto provide data to the Army leadership for makinginvestment decisions…” (Boutelle, 1996), and in thisthey succeeded.

143

Chapter 5

Lessons Learned onIntegrating a SOS

It is perhaps simplistic, but not overly so, to observethat the integration phase of both programs wasto some extent an act of enlightenment. This is

not meant to detract from the expertise andunderstanding of their program management andparticipants. Rather it is to observe that the complexityof integrating individually developed heterogeneoussystems, managed autonomously, each complex inits own right, is staggering. The difficulty is wellbeyond that experienced in managing a singleinformation system. It is the SOS entity that deliversthe results required. It is not the individual systemsthat provide the necessary effects, although eachcontributes to them. The whole is greater than thesum of the parts and the value-added is achieved inthe integration process. It is a strenuous undertaking.

The early stages of integration were confounding andoverwhelming with the nature and number of problems,but both programs moved on. To the credit and successof both ventures, some adjustments were possible inmethods and strategies, and the programs succeeded.If they had not achieved the integration, there would


have been no DPS or TF XXI to speak of. There wouldhave been only individual systems.

The lessons extrapolated from their experiences arepresented in this chapter. They are also summarized inAppendix A. For future integrations, they provide adescription of those participants who should comprisethe team behind the Wizard�s curtain and of theenvironment that should support them.

Summary of the Approaches toIntegrationThere were many similarities between the twoprograms in their approach to SOS integration. Neitherorganization previously had integrated the set ofsystems in the SOS. DMA accomplished two SOSintegrations—for the DPS IOC and FOC milestones.Both prepared for the integrations through an elaborateand complementary cascade of testing, which beganwith the individual systems and interfaces, usedindependent review processes, and followed withanother series of tests of interfaces by independentmeans. Both based their strategy on an analysis of risks,accomplished years1 in advance of the operational needdate for the SOS.

Characteristically they relied on testing eventsprimarily, but not exclusively, targeted at individualsystems and their interfaces. Both engaged external1 DMA assessed the integration risk at least 4 years before FOC,and the Army completed a risk assessment at least 2 years beforeTF XXI.

145Chapter 5

organizations for independent testing. Both usedprototyping of individual systems for many individualsystem developments.

Both recognized the need to reduce risk with earlyintegration efforts. The Army made extensive use ofreal and virtual simulations, deriving early knowledgeabout the behavior of core (to the experiment) systemsand subsets of the SOS. The Army also integratedsubsets of the SOS for use and experimentation indiscrete exercises well before the SOS integration.DMA did this to a lesser extent but did integrate specificSOS subsets at the factories of individual systems toidentify problems.

Both tested the integrated SOS in conjunction withoperators. Both used a spectrum of end-to-end threadsof operational activities to drive out problems in the SOSand signal completion of the integration phase. Theirobjective was to deliver an integrated product sufficientlycorrect for their needs,2 which were different.

They were typical in their approach of “build-a-little,test-a-little” for the process of integration. As the end-to-end threads were exercised by operators, problemswere identified and corrected. In the case of the DPS,the operational scenarios were demonstrations,including the use of digital source materials to producespecific MC&G products. In the case of the TF XXI, it

2 The Army’s objective was to achieve a product “mature enoughfor an experiment to provide data to the Army leadership formaking investment decisions…”(Boutelle, briefing slide, 1996).For DMA, it was to acquire a production capability.


was the conduct of the LZ Phantom exercise and a seriesof battle drills and exercises at Fort Hood.

Both relied on one site as the place where the SOS wouldall come together for the first time. The DMA hadplanned for this with a new Production Center; the Armycapitalized on the CTSF at Fort Hood.

Both were schedule-driven at the start of integration.The DPS was at the ending phase of a 10-yeardevelopment, and DMA required the integratedcapability to meet mission requirements. The Armyneeded the results of the TF XXI experiment forsubsequently fielding capabilities as part of the ArmyXXI program. While both schedules were compressed,they were considered a reasonable risk.

There were some differences in their approaches. Oneis a similarity as well—the early integration of subsetsof the SOS. The Army did this to a greater extent andplanned the SOS integration incrementally, such as bycomponents and systems, platforms, communications,and command and control. The DMA approachreflected the perception of the DPS as a digital pipeline,and progressed through the SOS integration of thesystems serially through the various stages.

One primary difference in their integration strategiesstems from the differences in their developmentmethodologies. The Army defined an architecturalframework and imposed it top-down on the individualsystems. Careful attention was given to architecturalcompliance (and architectural evolution) as part ofthe testing strategy. As the SOS integration

147Chapter 5

proceeded, this attention to compliance was sustainedthroughout the process. DMA accomplished the DPSarchitecture development bottom-up, within a farmore generalized framework.

A second primary difference is that the Army used aspiral process of acquisition and development whilethe SOS integration continued. The DMA used aclassic waterfall approach. For the TF XXI, theintegration event was a discovery process ofrequirements and refinement of designs—iterating tomodify the capabilities to accommodate therequirements as realized through the operators’experiences in conjunction with the developers whostood ready to respond. With respect to integration,this resulted in more impetus for changes in theindividual systems, thereby affecting the dynamics ofthe integration process.

At the outset of the integration phase, the managementof both ventures found themselves in similarcircumstances. Both had underestimated the challengeof the SOS but moved forward to deal with it.

The Lessons LearnedNine lessons learned are discussed here based on thetwo case studies and the influence of the integrationenvironment on them. As articulated, the lessons learnedappear fundamental. However, they serve as guidepostsfor dealing with the complexity of a SOS, providingpractical strategies to supplement other goodengineering practices. For future integrations, they


provide a description of those personnel who shouldcomprise the team behind the Wizard�s curtain, theprocesses they should apply, and the commoninfrastructure they should use.

A Lesson on Preliminaries

Lesson 1

Certain activities should precede a SOSintegration. These include:

defining the SOS architecture;

developing and testing the individual systemconstituents of the SOS;

developing and testing the interfaces betweenand among the individual systems of the SOS;

independently certifying compliance with theSOS architecture.

The promise of plug-and-play is a seductive one. Neitherprogram presumed this advantageous circumstance, yetthe difficulties of the integration and the complexity ofthe emerging entity far exceeded their expectations. Thislesson serves to remind that verifying the individualconstituent systems and interfaces and certifying themfor architectural compliance are not activities whichconclude the integration process but rather are necessaryto begin it.

The lesson is applicable to a SOS which is a “new start”or one which incorporates many legacy systems. Whatboth programs illustrated is that testing the individual

149Chapter 5

systems and their interfaces is not equivalent tointegrating and testing a SOS. Never mind that theconstituents in each SOS were designed (e.g., DPS) orengineered (e.g., TF XXI) to function as parts of thecohesive whole from the outset. The whole is greaterthan the sum of the parts. In effect, a SOS integrationprocess starts by using well-tested, certified individualsystems and interfaces—and an architecturalframework.3

For the two case studies, all the earlier developmentand testing—the prototyping, testing of individualsystems, the independent verification of interfaces,and the integration of subsets—were necessary todeliver functioning individual capabilities to theintegration site. But the functioning of individualsystems is not equivalent to the functioning of a SOS.Previous activities like independent certificationhelped to resolve a myriad of problems ofinteroperability and non-compliance with thearchitecture. However, in the aggregate theseactivities were not sufficient of themselves to achieveintegration of a SOS. The extraordinary efforts4 ofan integration process were required to accomplishthat task. The integration process was necessary tocomplete not only the “plugs” but establish the “play”for each SOS. Even when the interoperability betweenand among the constituent systems is readilyachieved, the integration process is needed to ensurethat the SOS provides the results required.3 The architecture may convey common applications used byconstituent systems, such as from the DII/COE.4 Chap. 4 describes these in detail.


Both programs did encounter an increase in difficultiesbecause some individual systems were not maturebefore, during, or even after the SOS integration. Manyindividual systems were themselves cutting-edgedevelopments and therefore at different levels ofmaturity. Both programs progressed to produce anintegrated product, but its quality was lower in earlyuse than later use. Typical impacts included discoveriesof new discrepancies, interruptions or delays in services,and the down-time of components.

A Lesson on Continuous Integration

Lesson 2

Use early, incremental, and iterative integrationto achieve a SOS.

In the acquisition cycle, the more traditional timingfor the integration phase follows the development andtesting phases of all the individual systems andinterfaces of a SOS. There are benefits to beginningthe integration of a SOS earlier than this classicmethodology. There is risk in dealing withcomplexities and unanticipated behavior late in thecycle. The difficulty can be so great as to result infailure to deliver—or to deliver the integrated productextremely late.

An additive suite of events, including early use ofsimulations and exercises, and early integration ofsystems and subsets of the SOS, enhances thepreparation for SOS. This is less risky than approachingthe integration as a one-time “big-bang” event when

151Chapter 5

all the constituent systems are available. SOSintegration is a resource-intensive process of build-and-test and should be planned accordingly, preferablyin iterative increments. It is not a simple process ofsnap-in, snap-out. It is a difficult undertaking in thebest of circumstances.

The experiences of both programs, as well as goodengineering practices, argue to tailor the strategy notonly to early, but also to an incremental, iterativeapproach.5 This means beginning the integration of aSOS with a subset of systems, then reintegrating thesewith additional subsets of systems in a series ofiterations, until a last phase of integration of all thesystems is completed. Early and iterative integrationsenable a more systematic accommodation of changes,and allow sufficient time in the program venture to adaptwhen the unanticipated occurs.

A first-time integration occurs when all the systemsof the SOS have never been combined in their then-current form. Both programs produced the firstexample of a SOS because they had not integrated allthe systems previously.

The Army exercised a strategy that relied on integratingsubsets of the TF XXI SOS. Yet the program stillexperienced considerable difficulties when all thesystems were integrated the first time. The immaturityof some components did not always allow systematicincrements. Difficulties also resulted with their5 For example, see Rechtin and Maier (1997) on stableintermediate forms and Walker Royce (1998) on iterative life-cycle processes.


incremental approach because changes continued tocomponents, systems, and subsets after their earlierintegration. The architecture incorporated constituentsystems that had been integrated previously to supportearlier experiments. However, these systems evolvedin the interim. For the TF XXI, they were then combinedwith new and different digitization initiatives, resultingin a new integration challenge.

A second integration event of the DPS occurred forFOC—started less than a year after the first. Thisreintegration manifested all the difficulties andcomplexities equivalent to that of the first-timeintegration because so many changes had occurred. TheSOS was comprised of the same set of individualsystems but their functionality and interfaces had allchanged in non-trivial ways.

Both external and internal forces will propagate changesduring the acquisition of a single SOS. The cautionarynote warranted for a SOS integration is connected withthe effects of changes to systems or components alreadyintegrated—as illustrated in the two case studies. Whenchange occurs, consequences occur for reintegration.A SOS, which characteristically is comprised ofconstituent systems that themselves are large-scale, canbe so complex as to behave as a non-linear system. Thismeans that small changes can produce disproportionateresults; the whole is more than the sum of the parts;behavior does not always repeat; and causes and effectsmay not be demonstrable (Czerwinski, 1998).

153Chapter 5

With the strategy of iterative and incrementalintegration, the question arises as to how to begin.Generally good engineering practices argue that theapproach should be a risk-based one (i.e., taking thesystems or components that are most difficult and riskyto integrate and focusing early-on to resolve issues).This is certainly consistent with DoD acquisitionpractices (Schaeffer, 1998). If the systems fail to matureor the issues are not resolved within the allotted time,there is more time to make adjustments.

Early integration also should focus on core capabilitiesneeded for the mission or experiment. While it iscritical to “do the hard parts first,”6 it is equallyimportant to concentrate on the parts that matter.Progress allows secondary objectives to be introducedin later iterations or subsequent phases of the programventure. Because success in integrating a SOS is astrenuous undertaking, narrowing the focus to coreobjectives is a practical strategy. Both case studiesprovided examples of requirements (e.g., centralizedprogram management in DPS) or initiatives (e.g., inTF XXI) that migrated beyond those that wereessential, complicating the integration. Many venturesso encumbered will not succeed.

The argument for early integration also aligns withrecommendations from experiences incorporatingsubstantial numbers of COTS products into informationsystems and enterprises of heterogeneous systems (Fox,et al., 1998; Brownsword, et al., 1998):

6 Rechtin and Maier (p.42) use this as a heuristic.


…in today’s reality, software COTS productsseldom plug into anything easily. Most productsrequire some amount of adaptation to workharmoniously with other commercial or customcomponents in the system…. Adaptation musttake into account the interactions among customcomponents, COTS products, any non-developmental item components, any legacycode, and the architecture, includinginfrastructure and middleware elements(Brownsword, et al.).

To deal with the rapid turnover of COTS products,recommendations include an immediate up-frontintegration and test activity with other systems andcomponents, as well as construction of an environmentto do so (Fox, et al.).

An early start and the use of the incremental iterativeapproach, while not eliminating the considerablecomplexity of the undertaking, serves to reduce the riskand apportion the difficulty into more manageablesegments. It also provides insight into the holisticbehavior of the SOS earlier and incrementally.

A Lesson on Testing Strategy

Both programs recognized the need to use operations-like activities typical of the mission to build and testthe integrated product. They also recognized theimportance of using all the actual systems in the SOSat the integration site (in contrast to heavy reliance onsynthetic stimuli), for the final integration of the SOS.

155Chapter 5

Lesson 3

The testing strategy for the integration of a SOSrequires:

an agreed-to plan and process for testing, basedon a risk assessment;

a suite of activities representative of theoperational requirements of the mission theSOS supports;

the exercising of a full spectrum of the SOSactivities (end-to-end) by operators, using theactual constituent systems of the SOS�or atleast a core SOS.

This lesson synopsizes what is required. Anything lessis a compromise and elevates risk. One of the challengesattendant to integrating a SOS is understanding itsbehavior sufficiently to make informed decisions abouta compromise. Earlier integration (and testing) ofsubsets of the SOS serves to reduce the risk, as well asto establish patterns of operational usage and SOSbehavior. These provide information needed to deviseand refine the testing strategy.

Each SOS integrated in the case studies was highlyoperator-interactive7—in contrast to one deployed on aspace-based platform. When this is the case, test plansneed scenarios built by operators and exercised byoperators. Both programs developed test plans usingend-to-end scenarios. With limitations on time (thereare always limitations), the challenge lies in assessingthe sufficiency of the end-to-end threads and of the risk


incurred by a compromise in testing with less. Bothorganizations were influenced by the demands ofschedule but judged the risk reasonable.

The DPS demonstrations included end-to-endmission threads, but not for every mission nor allproducts. The TF XXI used a set of battle drills andexercises. The sufficiency of these for completenessof integration testing is vulnerable to the ingenuityof the operators and how accurately the set reflectsactual operational scenarios. Also the nature ofoperational use of a SOS evolves. In both case studies,problems were found after integration when theunexercised threads were exercised.

In addition to early integration of subsets, a usefulstrategy to assess completeness of the testing is toanalyze defect trends. There are methods to predict thenumber of defects in an information system.8 Assessingthe number and nature of discrepancies detected andcorrected provides information on errors remaining. Adecline in problems uncovered by operators over timeshould signal growing stability of the SOS.

During the SOS integrations of both programs, therewere also compromises when certain systems orcomponents were unavailable; simulated capabilitieswere used.

With SOS complexity, a better guarantee of sufficiencyof testing is to use the actual systems. Models and

7 Operational missions will rely on a human-interactive SOSbecause warfare is based on human behavior.

157Chapter 5

simulators are, after all, only an approximation of thereal thing. However, they are important complementsto testing methodology.

A core SOS brings significant benefits for integrationand operations. DMA successfully constructed such aminiature DPS—called the System Test Mode(STM)—dedicated specifically to verification.9 Thiswas used for the systematic assessment of changes inthe integrated SOS baselines while evaluating theresults with an increasing number of benchmarks. Thisenabled resolving adverse side effects beforeoperational use and verifying that changes did indeeddeliver the results intended, both objectives importantfor maintenance and evolution.

A stand-alone core SOS can be exercised concurrentlyto the SOS being used for operations. As such it shouldbe physically isolated from that being used in actualoperations, including independent data bases. Changesto components and systems of the SOS then can beexercised with full benefit of regression testing and theuse of end-to-end test suites. For DPS, when it wasdifficult to include certain unique components in theSTM, simulators such as for the imagery network wereused. However, such substitutions frequently introducedanomalies of their own into the verification process.

A core consisting of a minimum set of capabilities maybe inadequate to verify performance of the integratedproduct under peak stress conditions—when the8 At least three methods were used for the DPS, including onecalibrated for large-scale DOD models.9 See chap. 4, section entitled, “A DPS Core.”


maximum is needed. In this circumstance, other means,such as modeling and simulation, may be the only orbest supplement.

A Lesson on Planning Resources

Lesson 4

To integrate all the systems of a SOS, plan forsubstantial difficulties and significant time andresources.

This lesson rejects outright the assumption of plug-and-play, even while presupposing that the appropriatetesting of individual systems and their interfaces hasoccurred. It provides a counter-assumption forplanning purposes based on current experience—thatthe integration of a SOS is a strenuous undertaking.As such it does not imply faulty testing of the individualsystems. Rather it underscores the efforts that must beexpended to achieve the emergent and requiredbehavior of the SOS.

The TF XXI integration phase at the CTSF at Fort Hoodrequired 6 months. Generally activities occurred aroundthe clock 7 days of the week. Approximately 500personnel supported just the integration activities.10 Forthe DPS, the first formal integration event required 7months and the second required 10 months. Activitiesalso occurred around the clock every day. About athousand personnel supported the DPS integrationevents.11 For both case studies, factors contributing tothe resource requirements were that all systems were

159Chapter 5

not previously integrated and that many constituentsystems were cutting-edge.12

A SOS is a complex venture, with individual systemstypically in large scale. For military missions, advancedand cutting-edge capabilities often are inserted toprovide an operational advantage. For manyoperations, forces and systems are combined inunanticipated ways. Planning for substantial resourcesfor a SOS integration behind the Wizard�s curtainis warranted. The resources, if not properlyprogrammed a priori, can be problematic to acquirebecause large expenditures are needed.

Planning requires good estimation methods, as well asa good experience basis for estimation. While this lessonpoints to the need to be prepared for the ravages ofintegration of all the SOS systems, developing credibleestimates of how much time and resources are necessaryis challenging. Both programs used methods todetermine the time and resources needed that provedinadequate in light of the actual effort required.

10 This estimate included personnel at the CTSF who supportedthe processes and infrastructure for integration, and factorypersonnel who modified and corrected constituent systems andinterfaces for the integrated product. This number does notinclude operators.11 The number was less for IOC; it was more for FOC becausethe latter milestone affected three Production Centers. The DPSestimate includes the DMA personnel supporting integration andtransition to production and contractors supporting integrationand correcting constituent systems and interfaces at factories.The number does not include operators.12 After the DPS FOC and the TF XXI event at the NTC,reintegration of the SOS required fewer resources as constituentsystems stabilized and changes slowed.


Methods for estimating time and effort for complexinformation system projects continue to evolve toprovide the means to factor in complexity, modernmethodologies, and larger-scale systems.13 Today’sprogram manager has many such models available forestimation. Nevertheless, while systematicmeasurement should be applied, current estimationmethods generally are based on software projects orinformation systems. Because a SOS involvesconstituents of large systems independently managed,developed, and operated and its behavior is emergent,and because the venture is resource intensive, moreaccurate means are warranted. The need for accurateestimation is compelling for an unanticipated real-worldmilitary operation, where it is essential for successfulmission planning.

To improve this situation, estimation models needtailoring and calibrating using actual expenditures fromSOS programs. This will be discussed in chapter 7 asfuture work.

A Lesson on the Importance of an IntegrationFacility

The use of a single site and its supporting environmentfor SOS integration was essential in achieving success.In an era of digital communications, the transport offaces and data can be accomplished in seconds. But avirtual team cannot sustain the empathy nor the

13 For example, see discussion on Cocomo II (Boehm, et al.,1996).

161Chapter 5

continuous interactions necessary for the birth of a SOS.In addition, the SOS did not exist until it was integrated.

The collocation at the single facility of programmanagers, engineers, and users enhanced theunderstanding between developer and operator to thebenefit of all. The single site environment facilitatedteam work focused on the success of the SOS (ascompared to individual systems). It enabled morejudicious resolution of problems in the shortest possibletime. The integration site was used by both programsas a means to maintain schedule after underestimatingthe complexity of the SOS integration. It enabled theteams to form the parts into a whole. Finally, the SOScapabilities “born there” were necessary to support thetraining of the operational community in the use of theSOS, an aspect discussed in chapter 6.

So critical for the success of the TF XXI was the CTSFenvironment that the Army subsequently expanded itsuse for other integrations, including the TF XXIDivision Exercise.

Lesson 5

The use of a single facility�with anenvironment of people, processes, andinfrastructure�substantially facilitates theintegration of a SOS from individual systems.

The value of a single facility with its supportingenvironment was clearly demonstrated in these two casestudies. Their experiences showed the following effects:


• More effective leadership of the SOS integrationresulted from the collocation of management andengineering resources from the constituents.

• A SOS team emerged where previously there hadbeen individual teams.

• The holistic behavior of the SOS wasconstructed dynamically at the site through thecollaboration of those individuals responsiblefor the SOS, those with the diversified,specialized understanding of the individualsystems, and the users.

• Standardized processes and infrastructure in theenvironment enabled effective, efficient, andunambiguous exchange of information.

• Complex information to identify and resolveissues was communicated and comprehendeddynamically and continuously with necessarypersonnel participating.

• The integration was achieved within the allottedschedule.

How, or if, distributed integrations of a SOS might beaccomplished successfully is discussed in the contextof future work in chapter 7.

163Chapter 5

A Lesson on “Who’s in Charge”

Steps were taken at the outset of both programventures to ensure that the overarching authoritiesand responsibilities for delivery of a SOS were clear.Ambiguity or lack of clear decision-making authoritywas not a problem in either case study. Rather, theclarity of lines of responsibility contributed to thesuccess. The acquisition responsibilities weresimplified in that primary accountability rested witha single program executive officer (PEO) within asingle organization. DMA’s PEO controlled theresources. In the Army’s case, multiple PEOscontrolled multiple pools of resources, but theaccountability was allocated to one, the PEOC3S, forthe TF XXI experiment. The operationalresponsibilities were defined clearly for bothprograms. Steering groups and working groups forthe participating organizations provided direction,coordination, and recourse for problems and issuesas necessary.

In both cases there were also clear lines of authoritydelineated specifically for the integration and at theintegration facility. A cadre was responsible for the SOSand was distinct from those teams managing theindividual systems of the SOS. The experiences lead tothe following lesson:

Lesson 6

The process for SOS integration should overtlyaddress the leadership of the integration asfollows:


an on-site acquisition leader empowered for theintegration of the SOS and an on-site leaderempowered for the operational community;

supported by a SOS cadre�with sufficientresources and authority;

supported by participants who manage,develop, and operate the constituent systemsof the SOS.

The question, “who’s in charge of the SOS integration,”was addressed openly by the management of bothprograms. The DMA appointed a senior leader-engineerat the integration site who reported to the DPS PEO,and the Army designated a trail boss acting on behalfof the PEOC3S at the CTSF.

Also demonstrated was the importance of an on-siteleader empowered for the operational community. TheSOS and the individual systems are in a dynamic stateduring integration, requiring decisions that must beclosely coordinated between the development andoperational communities. An authority to speak foroperations at the integration site preserves a uniformoperational view and agreement on operationalpriorities. This is particularly important becauserequirements will be changed, refined, introduced, anddeleted during SOS integration. Presence at the siteprovides first-hand understanding of the operationalexperience with the integrated product.

A SOS cadre was established by both programs, andamong their duties was direct support to the on-siteleaders of the SOS integration. The case studies illustrate

165Chapter 5

that they required more resources, painfully evident atthe time of SOS integration. For both DPS and TF XXImanagement, realization of the level of resourcesrequired and the extent of expertise needed emergedfully at the time of integration. The problemsencountered in the early stages of SOS integrationresulted in immediate reactions by the leadership of bothprograms to increase these assets.

The SOS cadre has duties that are distinct from thoseof the program managers of the individual systems.They are required for more than the SOS integrationevent, in fact for all phases of a SOS undertaking—such as in evolving the specific SOS architecture. Assuch, there are advantages to retaining these sameindividuals throughout all the various stages of the lifecycle of a SOS.

The experiences may indicate that the peak level fortheir resources occurs during the SOS integration phase,and both sets of events reveal the difficulties ofintroducing new resources late in the process. But moredata are necessary.

The cadre must have power and influence in the decisionprocesses, both acquisition and engineering. Currentacquisition processes recognize the roles andresponsibilities for program management of individualsystems. In both the DPS and TF XXI ventures, theprogram managers of individual systems were perceivedas carrying greater responsibilities than the SOS cadre.They managed large acquisitions, many people, andcontrolled significant dollars.


Substantial adjudication from both the programmaticand engineering perspectives was required on behalfof the SOS. Decisions had to be made favoring thewhole SOS entity at the expense (literally) of theindividual systems. This of itself illustrates the needfor a cadre with leadership responsibility and authorityfor the SOS.

When the SOS integration required additional funding,adjudication across these various interests wasrequired. Consensus or negotiation is not always themost expedient process. Greater autonomy andheterogeneity in management teams are morerepresentative of future SOS ventures. This impliesthe need for adequate staffing and more influence inthe SOS cadre, plus control of funding by the SOSintegration leadership sufficient to deal withunanticipated problems in the integration.

Program managers of the individual systems, and theirdevelopment teams, as well as users who operate thesesystems for integration testing, are also essentialmembers of the “one team” engaged in activitiesbehind the Wizard�s curtain. Collectively they areresponsible for the “parts” that comprise the SOS butalso support missions of their own. They provide theneeded engineering and operational insights into theseconstituents, and accomplish what is necessary todevelop or adapt, integrate, and sustain those systemsin a SOS.

167Chapter 5

A Lesson on Common Processes and Infrastructure

While there were some differences in commoninfrastructure at their respective integration sites, therewas a correlation on many essential processes for thetwo case studies—engineering boards, configurationmanagement, and external communications.

This lesson communicates the need for certain commonprocesses and common infrastructure in the integrationenvironment that are the same for all teams. It is likelythat the organizations that develop and operate theindividual systems of the SOS use different processes,tools, and infrastructure to manage their owndevelopmental and operational capabilities. It isexpected that they will continue to do so. But for theSOS integration environment there is commonality thatis minimal but sufficient—and available at theintegration facility.

Lesson 7

Certain common processes and commoninfrastructure in the integration environmentare essential to manage a SOS integrationsuccessfully. These include the following:

an Engineering Board with responsibility andauthority for identification and resolution ofSOS issues and discrepancies, including theassignment of responsibility for correction;


establishment of processes (and the automatedmeans) for identification of SOS issues anddiscrepancies, their disposition, tracking, andresolution, under the management of theEngineering Board;

automated support for the tracking and tracingof SOS operational requirements;

configuration management and control of thehardware and software baselines of thesystems of the SOS by the integrationleadership, supported with: automated meansfor identifying and controlling the baselinesand subsequent changes; a formal build,verification, and re-integration process forchanges;

a robust communications infrastructure linkingthe teams internal to the integrationenvironment and their external counterparts;

an office automation environment to supportthe integration�s administrative processes aswell as to support interpersonal processing andcommunications for the participants.

There is a requirement for the integration leadership todesignate the actual tools14 and infrastructure thatconstitute the common integration environment for theSOS being integrated. Early specification is important,

14 For example, designating a specific tool for hardware andsoftware baseline configuration management enables teams thatmanage the individual systems (and use other configurationmanagement systems) to plan accordingly.

169Chapter 5

as the DPS case study demonstrated in the example ofthe common discrepancy (defects) tracking system.

A subtle but important nuance on configurationmanagement processes is articulated in this lesson. Theconfiguration management of the individual systemsin the SOS should be controlled by the SOS integrationleadership, not by the management of the individualsystems. This arises from both sets of experiences.DMA was able to substantiate the benefits of thisapproach on its second DPS integration.

Both programs exercised configuration managementof the baselines of the individual systems during SOSintegration. However, for some specified systems theyallowed the primary change control to be managedby the factories of those systems. These exceptionsproduced problems. As a check on process after SOSintegration, DMA re-exercised the end-to-end threadswith the integrated product and identified anomaliesin the individual systems’ baselines. Consequently,during the second SOS integration, the primarycontrol of all the individual systems was allocated tothe SOS integration leadership and no exceptionswere allowed. This resulted in a more robust processfor change control.

Because an individual constituent system is likely tobe independently operated for multiple purposes (otherthan that of the SOS being integrated), it may requiremanagement of multiple baselines. The change controlof the baseline in the integrated product should be


managed by the integration leadership. A plan andprocess for subsequent synchronization is then required.

Consistent with the dynamics of change,configuration management tools used should besufficiently robust to allow rolling backward andforward through many versions.

Robust communications infrastructure is essential tolink the developers at the integration facility with theirfactory counterparts. Linking the integration site tolocations where the SOS is operated also supports itssubsequent deployment, sustainment, and evolution.

A Lesson on Efficiencies and Effectiveness

There is a need to optimize use of the large resourcesattendant to a SOS integration, and to preserve schedule.Both programs relied on daily status, on alternative andcontingent plans, and the dissemination of programinformation. Because the constituents of a SOS areindependently operated, developed, and managed withtheir own individual processes, it is beneficial toestablish these methods and automated infrastructureas part of the common integration environment as earlyas feasible.

Lesson 8

Certain common processes and infrastructurein the SOS integration environment promoteeffectiveness and efficiencies. These include:

171Chapter 5

daily planning and scheduling of resources(people, equipment, facilities) for integrationevents�with contingency plans and schedulesreadily available;

timely dissemination of information pertinentto each integration event, such as test status,equipment availability, and results;

daily status meetings, with results immediatelyavailable.

Daily planning and scheduling tools aid the integrationmanagement in optimizing participants’ time andenergies while moving the overall program schedulesforward. The DPS integration leadership used suchcapability extensively to manage the 700 daily eventsthat were ongoing for SOS integration, affecting morethan 1,000 people.

Such tools facilitate progress through the integrationschedule. While automated means can project the dailyevents, the dynamics of SOS integration often precludethem from taking place as planned. Automated toolsshould provide the means to develop alternative orcontingent plans and link related activities. For bothprograms, being prepared with such alternatives for themultiple daily events ensured that progress continued.

Both programs experienced considerable contention forresources—people, equipment, and facilities. Usingautomated planning for the daily events supporteddeconfliction and optimization of resources.


The immediate availability of status was importantto gain a common understanding by all participantsat the same time, as opposed to individual, disparate,and unsynchronized understandings—which carrythe consequences of unnecessary, uncoordinated, andfutile activities.

A Lesson on Evolutionary Acquisition

Lesson 9

Prototyping a SOS can provide early insight intooperational requirements and into the SOSsystems architecture.

Generally this lesson encapsulates the merits ofprototyping, which can be applied to any new softwareproject or any new system. However, articulating thelesson here emphasizes the distinction that it is the SOSthat delivers the necessary results—not the individualsystems. The performance of the individual systemsmust be understood in the context of their contributionto the SOS behavior. How the whole entity behaves isthe crux of the matter. In turn, all the systems in thewhole must be integrated to realize the full extent ofthe holistic behavior.

It is the integrated SOS that provides the insight as towhether the delivered capability renders the resultsexpected—and required. However, the results also maypropagate a different understanding of what isneeded—and imply changes in the operationalarchitecture. When using experimental capabilities tosupport experimental concepts, the key is to use the

173Chapter 5

results of integration as a contributor to therequirements and architectural processes for theintegrated product intended for field production.

For each case study, the SOS provided revolutionarycapabilities and supported revolutionary operationalconcepts. The entity emerging from the integration ofthe individual systems was difficult to predict at theoutset, a theme frequently played within this work.

The Army integrated a specific SOS for the first timeto put digitization capabilities in the hands of theoperators for the 2 weeks of the TF XXI at the NTC.As a result, the Army gained considerableunderstanding of the consequences of that capability,and from many aspects—operational, doctrinal, andstructural. The information derived was used for manypurposes—including follow-on acquisitions for theArmy XXI program.

The Army viewed the acquisition process that occurredat its CTSF as one of the fundamental successes of theTF XXI experiment. The process that ensued wasbroader than just prototyping—it was one ofevolutionary acquisition (Boutelle & Grasso, 1998).The collocation of developers with operators trainingon revolutionary capabilities produced a spiral offeedback on requirements and refinement ofcapabilities, trimming years from the usual acquisitioncycles. This advantage cannot be overlooked as animportant benefit of the SOS integrationenvironment—and a future opportunity.


DMA elected to forego prototyping the DPS15 as astrategy in an era where the more classic waterfallmethodology prevailed, and one can only speculate asto the impact of a decision that appeared sound at thetime. After FOC, the integrated product provided newunderstanding, which resulted in changes to the initialoperational concepts, such as those for productionmanagement. Without insights into the behavior of theSOS earlier in the process, the surprises can only begreater, considering the complexity of the undertaking.

While the integration environment can supportmanagement’s ability to move forward afterunderestimating SOS complexity, it cannot overcomethe shortfalls of bad requirements or a poor design, orbe a substitute for good architecture. However,integrating a SOS as a prototype for experimentationcan provide significant insights into the consequencesof operational requirements and design decisionsearlier in the acquisition process, and it is an alternativeto a methodology that is more serial, rendering thatunderstanding near the end of the acquisition cycle. Itis also far better than relying primarily on theknowledge of the capabilities and performance of theindividual systems.

The caution that must be sounded is that, in accordancewith the Heraclitan principle of change, and consistentwith the experiences of both case studies, the integrationof the SOS capability ultimately fielded will still be astrenuous undertaking.

15 The concept of a Mark 87 engineering prototype of Mark 90was considered briefly, but subsequently terminated.

175

Chapter 6

Lessons Learned onTraining with a SOS

This chapter discusses the training experiencesfor both programs. Training and the SOSintegration occurred concurrently. Because of

the desirability to train operators on capabilitiesidentical to that used for operations, the integrationenvironment was essential for training. The integratedproduct was needed to accomplish the mission, andthe SOS for each program was incrementally evolvingat the integration facility through the “build-a-little,test-a-little” process of integration.

Three lessons learned about training with a SOS arediscussed in this chapter. What emerged from both setsof experiences was the importance of allowing moreand iterative training on the integrated SOS capability—and the need to teach the whole in addition to the parts.

Training lessons are summarized in Appendix A.


The Digital Production SystemTraining ProgramFor the DPS, the training program focused on theAgency production workforce. In addition to managers,this workforce included primarily cartographers andthose in related fields. The total training population wasabout 2,700 people.

The DMA management provided training on theindividual systems of the DPS, coupling these withprerequisite and related courses (i.e., computer skillsand knowledge engineering). Generally the trainingprogram assumed that students knew their professionand focused on the new digital capabilities with newoperational scenarios. Multiple training events preparedthe workforce for the daily business of using the DPSto produce MC&G products. More than 8,000 trainingevents were conducted.

The most challenging training events for operators werethe Exercises and Rehearsals (E&R). These consistedof rehearsals coupled with exercises to produce a set ofDMA products end-to-end using the integrated SOS.These events were conducted first at the ProductionCenter that served as the integration facility, thenexpanded to the other two Production Centers. The setof products included those produced in the integrationdemonstrations and others as designated by productionpriorities, which varied over time. The scheduleserialized events to engage operators in increasinglycomprehensive production activities with the newcapability, while training operators in time for their

177Chapter 6

production assignments using DPS. The events for E&Rrequired many months—consistent with the longproduction time required for Agency products, evenwith the new capabilities. The timelines were alsodeliberately lengthened to allow for learning. Thetraining window extended well past FOC toaccommodate the large training population. Figure 6-1illustrates the training schedule embedded within thatof the DPS program.

Early in the series of training events, operators exercisedthe functionality of an individual system in thestandalone mode, while training for their part in thenew DPS operational concepts. An operator on thesystem for data extraction learned to apply the functionsof the newly developed workstations—how to performfeature extraction for multiple products, and how topopulate the MC&G data base. Both multiproductextraction and attributing the data base were innovativeconcepts introduced by the DPS.

The operators’ initial access to the integrated suite ofDPS SOS hardware and software occurred during thedemonstrations before IOC, but this was only by a smallcadre. During the E&R and FOC demonstrations, thenumber of users increased substantially as 20 percentof the workforce was trained. Their experiences wereused to generate production procedures for subsequentuse. After FOC, some operators helped train theremaining 80 percent of the workforce for DPSproduction while the majority moved to productionassignments.

17

8B

eh

ind

the

Wiza

rd’s C

urta

in

Figure 6-1. DPS Training Schedule

179Chapter 6

The strategy emphasized training an operator for theindividual system he or she would operate in production;an operator was not trained on multiple systems of theDPS. Only general information was provided about theSOS capabilities and other (than the operator’s)constituent systems. At the time, this was not perceivedas a limitation. One training program intended forproduction managers attempted to provide a detailedexposition of the SOS and all individual systems andinterfaces; however, it quickly became obsolete. Onlya small population of the workforce attended.

In E&R, operators exercised production tasks analogousto their future work assignments When using their ownworkstations, such as to accomplish data extraction, theyalso engaged the support of other systems, such as thoseproviding imagery and ancillary source information, andthose managing the MC&G data bases. Yet the DPSoperational concepts and the operators’ trainingmaterials included limited information about theserelationships. Materials did not provide the rapidlyevolving and detailed “as built” interfaces;consequently, operators obtained only a top-levelunderstanding as to how the processes and methods ofother systems in the SOS impacted them in doing theirown job. In reflecting on the training experience later,one senior DMA leader noted the rarity of people whounderstood the end-to-end process of making a mapwith the DPS, and the value of those who did.

During E&R and early production start-up with DPS,it was difficult for individual operators to understandwhat, why, and how functionality was executed beyond


the boundaries of the operator’s own individual system.The results of an individual’s actions, which createdripple effects in the mapping pipeline, were notaddressed in the training program. These effects weredynamic, changeable, and often non-repeatable. Attimes these experiences were inexplicable to anoperator. The understanding of how an individual’sactions related to the whole SOS was elusive.

The DPS underwent numerous corrections during E&R,primarily resulting from problems with interfaces duringintegration. Despite this, training continued as long asworkarounds1 could be developed. Trainingconcurrently while the SOS was stabilizing wasburdensome to operators, who were frustrated by delays,errors, changes, and rework.

Despite the objective to make E&R just likeproduction, the full complement of equipment neededwas not always located at the integration facility noravailable for training. Some pieces were unique, suchas scanners or plotters, and were not allocated to theintegrated training environment. Training carried alower priority than integration needs and thanproduction needs after FOC.

These omissions resulted in training cartographers onabnormal configurations, which then introduced otheranomalies. Because there were different production

1 Workarounds were temporary procedures that deviated fromnormal production processes but allowed recovery from afunctional defect. Workarounds were not intended to bepermanent, but they gave developers time to correctdiscrepancies.

181Chapter 6

assignments and unique subsystems at each of the threeProduction Centers, configurations supporting thetraining baseline varied. The result was that operatorswere not always able to train on an environmentidentical to the one they used for production, and theyexperienced atypical problems.

More equipment was moved into production andunavailable for training events, and greater use wasmade of simulators to provide interface stimuli. This,too, created abnormalities. As more elements migratedto production, the E&R concluded earlier than planned.The Agency decided to accelerate production with theDPS to maximize its benefits. Nonetheless, theincreasingly synthetic nature of the trainingenvironment decreased its effectiveness in preparingoperators for production.

Digital Production System Training Program WellReceived

Overall the training program was well received by theoperator community; it exceeded a 90 percentacceptability level by operators and managers. Whensoftcopy training materials were available at individualprocessors or operator workstations, they were foundto be more effective. DPS operators preferred embeddedtraining with self-paced instruction. In the opinion oftrainers and trainees, this approach produced superiorresults. This level of embedded training was developedin only one individual system of the SOS, and it becamea model for future training programs.


The Task Force XXI TrainingProgramIn fact, TF XXI was an experiment coupled with atraining event intended to provide an experience as closeas possible to an actual conflict. The operator cadre wasthe Experimental Force (EXFOR) comprised ofapproximately 5,000 soldiers of a Brigade Task Force.The training addressed not only the digitizationcapabilities but also new tactics, techniques, procedures,and organizational initiatives.

Training for earlier digitization efforts, includingAWEs, exercises, and simulations, had provided severallessons (i.e., that units should be proficient on combatfundamentals, be proficient with the digital systems,and train using these systems) (TRADOC IntegratedReport Annotated Briefing Slides, 1997). In the yearpreceding the TF XXI, there were platoon- andcompany-level field training events; however,simulation networks were used as the principal meansfor battalion-level training.

The EXFOR’s training on new equipment ramped inMarch 1996 following that of a small cadre of soldier-trainers who began 2 months earlier. A rigorousschedule was enforced because the trainingrequirements were so vast. The equivalent of a “digitaluniversity” was established to enable soldiers to learnto operate one or several pieces of equipment. Morethan 90 classrooms and 28 motor pool areas were usedfor hands-on training (Goedkoop, 1997).

183Chapter 6

When most of the individual systems arrived at FortHood in June 1996, integration difficulties occurred,as discussed in chapter 4. However, training continuedaround the clock. Developers worked at correcting oradapting individual system capabilities based onoperators’ feedback in training. A continuous processof train, refine or correct, and train, occurred withoperators influencing the integrated TF XXI SOScapabilities to reflect the realities of operational needs.

As the stability and completeness of the SOSintegration advanced, a series of training exercisescalled battle drills were conducted with the EXFOR.These training activities used configurationssupporting operational enclaves, like the TacticalOperational Command Centers. They were used inconjunction with other exercises at Fort Hood to enablethe soldiers to operate with discrete subsets of the SOSas well as the integrated product.

According to Boutelle and Grasso (1998):

The development and execution of battle drillswas key to the success achieved in training thewarfighter. These battle drills simulated specificthreads of operation and allowed the warfighterto better understand the tools and capabilitiesavailable in the context of his or her mission.

Exercises were conducted at Fort Hood to providephased training in a realistic field setting. Soldiersadvanced from using basic connectivity to usingincreasingly comprehensive capabilities and applyingnew tactics and organizational initiatives. The training


events are highlighted in figure 6-2, relative to theschedule for the TF XXI architecture and integration.The platoon-level exercise in August 1996 provided unitexperience in a tactical environment. Company Lanesduring October trained soldiers using the digitizedcombat support and combat service support capabilities.A brigade-level exercise, concluded in December 1996,gave all soldiers of the Brigade Task Force their firstopportunity to maneuver together in a realisticenvironment using the SOS with digitization initiativesand legacy systems fully integrated.

The field exercises were supplemented with simulation-based training events. A classroom facility, equippedwith surrogate appliqué workstations to simulatesituational awareness, was used to enable EXFORcommanders and staffs to engage in war games withspecific objectives. Simulated GPS data and certainintelligence assets, along with tactical vehicles, alsowere networked. This allowed exercising tacticaldecision-making processes and supplemented fieldexercises. When the equipment was shipped to NTC inlate December, the EXFOR trained with battalion- andbrigade-level simulations in January 1997.

Training experiences occurred on capabilities in flux,the result of constant corrections to individual systems,refinements induced by operator feedback, andprogressing stability of the integrated SOS. Newtrainees used the capabilities differently than theirpredecessors as more changes and corrections weremade. Engaging the operators in the exercises fueledthese dynamics further by identifying more problems

18

5C

ha

pte

r 6

Figure 6-2. Task Force XXI Training Schedule


for correction, more user modifications, and morechanges to the integrated product.

As a result, operators trained on an evolving, constantlychanging SOS, though one better aligned withoperational requirements. Interruptions wereburdensome to the training experience; and those whotrained early found themselves using altered capabilitiesduring the actual NTC event.

A high rate of turnover in soldier participants occurredearly in the EXFOR training cycle. Despite early effortsto sequester specific assignees to the EXFOR and retainthem until the TF XXI experiment concluded, theimplementation of this strategy lagged. Many soldierswho received early classroom training were reassigned.Between January 1996 and the AWE at NTC, personnelturnover, including duty changes, exceeded 40 percent.When the TF XXI occurred at Fort Irwin, many EXFORparticipants did not have full training on current versionsof SOS capabilities because of turnover and changes.For example, despite at least three major upgrades tothe appliqué during the training period, only about halfthe soldiers previously trained received updates(TRADOC Integrated Report Annotated BriefingSlides, 1997).

187Chapter 6

Conclusions about Training with aSystem of SystemsThe training strategy for both programs appropriatelyemphasized operators’ access to new capabilities usingoperations-like scenarios. Both programs used acomplement of training events. The TF XXI and DPSprograms recognized the need to train for the operationalmission in a fully integrated SOS environment andprovided this access. This training was in addition tothat provided on individual systems and components.

The results illustrate that the operational communitywas able to perform the mission required. Nonetheless,more and different training opportunities were necessaryto master the SOS capabilities. The Army’s specialtraining assessment found the TF XXI training programeffectiveness was affected by various factors, includingimmature equipment, a high rate of changes, andturnover (TRADOC Integrated Report AnnotatedBriefing Slides, 1997).

In training on the integrated SOS, both cartographersand soldiers experienced interactions that weredifferent from training on an individual system in astandalone mode. This situation arose from threeprincipal causes—interactions arising from the holisticbehavior of the SOS, changes deliberately introducedbecause of user feedback (in the TF XXI case), andproblems remaining in the integrated SOS. Allprompted responses from operators for actions anddecisions that had not arisen from training in thestandalone mode or in a less than fully integrated mode.


From the perspective of “train-as-you-would-fight,”this is problematic. However, it should be expectedwith a SOS.

The concurrency of training and integration withinterruptions caused by problems, corrections, andworkarounds affected the quality of the trainingexperience. Both programs wrestled with aggressiveschedules and the dilemma of when best to train theoperators, considering the immaturity in the integratedproduct. A premature start elevates user frustration andprovides an experience unlike actual operations,whereas a delayed start jeopardizes readiness for theoperational mission.

In both cases the problems and changes duringintegration contributed to more training time than wasallotted. Nevertheless, even with a stable, more matureintegrated product, SOS mastery requires stepwise,incremental training.

In retrospect, the DPS training program provided toosegmented a view of the SOS. The program succeededadmirably at training the workforce on individualsystems, consistent with production assignments. Butit allowed inadequate time in the integrated SOSenvironment, and the insufficiency was furtherexacerbated by changes and latent defects.

Although operator access to the DPS in E&R was anacknowledged objective, the availability was reducedbecause of integration schedule pressures and theAgency’s decision to accelerate the production use ofthe DPS. Furthermore, all operators were not provided

189Chapter 6

entrée to the SOS. Some of the training events at a singleProduction Center offered that access; most eventscombined subsets of the SOS integrated with interfacesimulators or partial configurations of hardware, orsoftware modified for training.

All these limitations created a steep learning curve foroperators to capitalize on the SOS and to adapt processeswith it. The DPS production community was expectedto respond to changing scenarios and requirements,essential for crisis response, which often necessitatedactions different from normal production activities.Without the longer training investment on the fullyintegrated DPS, the production community struggledto develop alternatives to the new processes on whichthey had been trained. They frequently turned to non-DPS production systems and established processes.2

This situation was aggravated by the complexinterrelationships of the various systems of the DPSand the limited information about them provided in thetraining. Over time this situation was mitigated by on-the-job experience and management efforts. A kind of“SOS team” approach later was introduced as a resultof user dissatisfaction with the segmented view. Butthe experience underscores the need for training thatprovides a coherent understanding of the SOS.

An analogous result occurred with the TF XXI case. Inassessing the force-on-force encounters at the NTC,participants and observers noted that when the EXFOR

2 Numerous defects and changes in customer requirementsexacerbated the frustration with the DPS.


engaged the OPFOR, it did not always exploit the newcapabilities. Naylor (1997) observed:

…although the EXFOR troops have beenlearning about and training with the newsystems, as well as developing tactics to exploitthem, for more than a year, they are still not asadept with their new gear as conventionallyequipped brigades are with current systems anddoctrine. Thus, in many cases they failed tomaximize their newfound capabilities.

In fact, in many observed examples, the soldiersreverted to more conventional strategies and processesthey knew better.

When later reconstructing the events at the NTC usingthe data collected during the experiment, the potentialfor capitalizing on the new digitization capabilities wasevident, although sometimes in contrast to how eventsactually had transpired. This was why some participantscharacterized the training as the biggest shortfall.

The conclusion from this is that the EXFOR requiredmore time and more iterations of experiences with thefull complement of capabilities to evolve dramaticallynew strategies.

The post-NTC assessments underscored this conclusion.Effectiveness increased when several missions of theTF XXI AWE were replayed by the EXFOR and otherparticipants using simulations. Repeated opportunitiesreinforced this result:

191Chapter 6

The consensus from those involved in analyzingthe AWE was that numerous opportunities toexploit digitization were provided to the EXFOR,but they either failed to take advantage or failedto recognize their presence. Modeling three ofthese opportunities with different tools, atdifferent locations, with different players led toa consistent finding that, if in fact the EXFORhad reacted to the information available in atimely fashion, then significant value-added toforce effectiveness would have been observed.As with any new concept or system, knowing howto best employ it takes trial, error, and retrial.(TRADOC Integrated Report AnnotatedBriefing Slides, p.57)

A Lesson About Training Operators

These two sets of experiences on training operators touse a SOS lead to a first lesson.

Training Lesson 1

Train operators on a SOS using a full spectrumof operational activities, and train allowingiterations.

Training should include iterative exercises with afull spectrum of operational activities using the SOS.This is in addition to the training provided onindividual systems that are standalone or notintegrated into the SOS.


Training on an individual system will not supply thenecessary understanding of how the whole SOSfunctions or behaves. Such limitations in the trainingexperience can inhibit operator effectiveness from twoperspectives—capitalizing on the full benefits of theintegrated whole and dealing with the relationships ofother systems in the SOS.

Operating with SOS capabilities requires more trainingthan operating with an individual system. The casestudies show that increased time and types of trainingin the SOS environment were necessary to master thenew capabilities offered by the integrated whole. Thisaspect of learning is not one-time, but rather iterative.

Both sets of experiences indicate the importance oftraining to assimilate the total picture of how the SOSsupports the mission and how the individual systemscontribute to the whole. The user must master his orher role but in the full context of the whole. For using aSOS capability, more than usual attention should befocused in training content on the interfaces among theconstituent systems and how their relationships affectthe operator. In actual operations, an operator must beprepared to respond when actions are initiated by otheroperators on other systems. While it may be difficult toanticipate the full spectrum of activities that a particularoperator will experience with a SOS, the case studiesreveal that the more opportunities the operator has, thebetter prepared he or she will be.

SOS training should use a wide spectrum of operationalmissions and a full complement of systems. The DMA

193Chapter 6

experience illustrated the disadvantage of excludingsome MC&G products from the suite of trainingexercises. Later, the operational community struggledto complete the detailed processes for producing theunexercised products and operators uncovered defectsnot previously manifested. Both training programs alsoreinforced the desirability of using a full complementof all constituent systems and equipment with little-to-no reliance on synthetic stimuli. Training with the SOScapabilities as near as possible to actual operationalconfigurations enhanced the operator’s preparation forproduction assignments.

This lesson is summarized neatly in a synopsis of theEXFOR’s experience at the CTSF (Boutelle & Grasso,1998):

The final ingredient in fielding a highlyintegrated set of capabilities is to ensure thatthe end user understands the power behind thesystem. The end user must make the system partof his or her business. Consequently trainingmust not be focused on how to punch the keys,but on how to better conduct business. Trainingmust also include the collective set of capabilitiesavailable to the end user.

One TF XXI leader-soldier eloquently characterized theobjective as follows:

I call this the fourth dimension of leadership—the full understanding of what the informationtechnology is and what it can actually do foryou. The new capabilities require a different


thinking, an assimilation of the potential and theability to leverage it faster. This requires newcourses of action by iterations on exercisescenarios. Using these approaches, and bycreating new and harder constraints on theoperation, the leader will find ways toaccomplish the end game. We must find ways totrain for this holistic approach.

A Lesson on Infrastructure Support for Training

The case studies point to the need to supplement thetraining infrastructure to support training with a SOS,as noted in this next lesson. This augmentation isnecessary even when the integration event is not asdynamic as was the case with both programs. Theoperator requires a training experience which providesthe perspective of the SOS as a whole, and this carriesimplications for the training infrastructure—the people,processes, organization, components, materials,documentation, and automated implementation.

Training should not provide the perspective of a singlediscrete system absent the context of the whole. Iftraining gives the operator a perspective primarily orexclusively that of the individual system, the trainingexperience will naturally be less comprehensive, eveninadequate, in exposing the relationships between andamong the other capabilities in the SOS—or inproviding the holistic view. This is a shortfall morereadily caused when the constituents of the SOS arethemselves existing systems; however the DPS though

195Chapter 6

a “new start” program also provided too-segmented aperspective to the operators.

There will be many SOSs; systems will becomeconstituents in many SOSs. Therefore there is a needfor generation of new training content to teach thecapabilities of each SOS, and to expose the new anddifferent relationships. This will be a recurring need.

Training Lesson 2

The training infrastructure should beaugmented to provide the perspective of a SOS.

The infrastructure supporting training should addressthe SOS supporting the mission, as well as thecapabilities of the constituents and theirrelationships. Because operating with a SOS requiresmore training than operating with an individualsystem, an augmentation in the traininginfrastructure naturally follows.

The training infrastructure should be sufficiently robustto accommodate an iterative learning experience andtraining that is concurrent with integration. This requiresa ready revision to training materials. Training contentmust advance as operators progressively master thecapabilities of the whole. This need for flexibility carriesspecial challenges when training content is embeddedin individual systems.

Both programs felt the disadvantage of dealing withproblems while SOS integration and operator trainingwere concurrent. Despite this, cartographers andsoldiers demonstrated great ingenuity and commitment


in coping with the problems—and in getting their jobsaccomplished. The Army leadership commented on theimpact of focusing on integration to the detriment oftraining at the completion of the EXFOR’s encounterwith the OPFOR at the NTC (Naylor 1997). Alsosubsequent evaluations of results marked the adverseeffects of immature systems on the EXFOR’s training(TRADOC Integrated Report Annotated BriefingSlides, 1997; Slabodkin, 1997).

As a practical matter, training while integrating maybe more representative of pressing operational missions.What then follows is the need to lessen thedisadvantages when they are concurrent—or when thereis a great rate of change in the SOS.3 And there clearlyare benefits to the integration processes when operatorstrain while the integration continues. First, the natureand number of operator interactions increases thebreadth of testing, improving the quality of the deployedSOS. Second, the intense experience by users can leadto an alignment of the integrated product withoperational needs—when developers stand ready torespond. The TF XXI demonstrated this.

Training with immature systems and components and/or a stabilizing integrated product will always beburdensome. When training is concurrent withintegration more engineering support is required torespond to changes and defects to minimize3 In fact, the difficulty of training soldiers on new equipmentwith a high rate of software changes has been noted in priorexercises such as Focused Dispatch. The Army continues toevolve training methods for the digitized forces (Meigs, 1998;Ruocco & Smith, 1998).

197Chapter 6

interruptions and workarounds for the operator.Different operators will uncover different defects, evenin a well exercised and stable SOS environment. Thisis characteristic of highly interactive informationsystems. And the Heraclitan principle4 argues that theexternal environment, not just the one internal to theSOS, will cause changes to occur. The training supportmust be strengthened accordingly.

Impacts can also be lessened by refreshing trainingsoftware and training data with the current SOSbaselines and by updating training materials frequently.The timely dissemination of revisions to trainees isequally important. Without this added investment forsupport, the difference in the training experience fromthat of actual operations will be amplified. This disparitycan detract from operators’ performance. The anomaliesand interruptions certainly lower user confidence andincrease user frustration with the new capabilities, asboth programs experienced.

More refresher training should be provided. There is aneed to continue to educate users on changedcapabilities. There is also a requirement to update auser’s knowledge of the whole, as it is mastered,building upon earlier information for subsequent use.

Accommodating change while maintaining the currencyof the training experience carries implications forplanning and supporting the infrastructure. For the DPSprogram, the training infrastructure included thehardware and software baselines of the SOS, step-by-

4 “You can never experience the same SOS twice” (see chap. 1).


step operational procedures, and data synchronized tothose steps. All of these were resource-intensive tomaintain when the software baselines of the individualsystems were changing more rapidly than those usedfor the training materials. To accommodate changes anddocument workarounds for problems, considerableresources were used to annotate changes on trainingdocumentation and to distribute information to operatorsin training. Because the SOS software used for trainingwas not updated as frequently as the SOS software usedfor integration, anomalies of experience and informationresulted. Reconciliation was burdensome for theoperators when they moved from training to aproduction assignment. Maintaining currency wascomplicated further by the different trainingconfigurations required for the three Production Centers.Analogously, the TF XXI faced equivalent demands tomaintain the currency of the training experience foroperators in light of substantial software revisionsduring integration. The program also experienced thechallenges of training on nearly 200 configurations inthe systems architecture.

If the training site is the integration site, as it was forthe TF XXI at Fort Hood and for the DPS at a DMAProduction Center, this infrastructure support fortraining becomes an important element of theenvironment there.

199Chapter 6

A Lesson About Training the Wizard’s Team

The training programs for the DPS and the TF XXIwere focused appropriately on preparing the operationalcommunity to accomplish the mission. There was littleattention on the training needs of the players behindthe Wizard�s curtain. These included engineers,integrators, hardware and software developers, testers,infrastructure and administrative support personnel, andgovernment and contractor managers. An implicitassumption was made in both programs that theseparticipants understood the SOS.

Unlike the TF XXI experiment, the DPS was aproduction capability with a life-cycle that includedmaintenance and evolution; consequently, the DPStraining program did address DMA personnel who hadhardware and software maintenance responsibilities orwho had the administration of the networks and the databases. As it had for the DMA operator community, thetraining focused on individual systems, but was limitedin addressing the interfaces between systems. Thisresulted in gaps in understanding that had to beovercome through on-the-job experience.

In light of the integration experiences,5 these omissionsresult in a third lesson on training.

Training Lesson 3

Training should be provided to those behindthe Wizard�s curtain.

5 See chap. 4.


Training people behind the Wizard�s curtain aboutthe SOS architecture being integrated contributes totheir effectiveness and that of the process. Here thecontent of training provides information about theoperational, technical, and systems architectures, andthe relationships among the constituent systems of thespecific SOS being integrated.

Both programs experienced shortfalls in personnel withthe necessary knowledge of the SOS at the time of theintegration. While their numbers were addressed rapidlythrough augmentation of assets, the transmission ofknowledge was not. No training program was availableto describe the SOS architecture and relationshipsamong the individual systems with explicit “as-built”details in the context of the end-to-end mission scenariosoperators were using.

The SOS team needed extensive knowledge to masterthe holistic view, and it was in short supply when itwas needed most. The complex and detailed informationwas difficult to absorb as an on-the-job experience andrequired time to assimilate. For the DPS, with therelatively small size of the SOS cadre, only a smallpercentage of personnel actually achieved such masteryof the whole entity and the relationships among theconstituent systems—about 1 percent of the participants.New personnel, particularly those rushed to theintegration site, had to be educated, but there were onlybriefing materials compiled as a tutorial to use and theywere more general, rather than detailed, in nature.

201Chapter 6

In hindsight, it was a training need that was overlookedand underestimated. With the large number6 of peoplewho supported both the integration phase and thesubsequent sustainment, a training program will realizeefficiencies and effectiveness.

In the context of rapidly configuring a SOS for anymission, jump-starting the knowledge andunderstanding of the Wizard’s team must be consideredas essential. A modicum of training also will provideuniversal understanding of the common processes,practices, and tools used in the SOS integrationenvironment.

While it is of primary importance that the SOS cadremanaging integration receive such training, theexperiences show that the teaming of all participantsbehind the Wizard�s curtain was critical to success.The benefits of collaboration and the synergy from thediverse and specific views of the individual systems inthe SOS argues for providing training to a broaderpopulation, which includes personnel supporting theconstituent systems.

Training the participants for the integration of a SOSand for evolution and maintenance requires resources.Sustaining current training materials takes increasedeffort, just as it does to train the operational communityto support the mission. The complex and dynamic natureof the SOS capability presents special challenges foreducation and presentation.

6 See discussion in chap. 5, lesson 4.

203

Chapter 7

An IntegrationEnvironment for the Future

Considerable efforts were required to integrate theDPS and the TF XXI. Yet each was a relativelysimple system of systems (SOS) in the context

of the Joint Vision 2010 era. Operations typically willbe joint and coalition. It is important to look towardintegrations that go beyond the complexity of the twocase studies, and consider federations of systems (FOS)as well. This chapter briefly explores these topics andconcludes that future experimentation should includeassessments of SOS and FOS activities behind theWizard�s curtain.

Nonetheless, the lessons from both programs provide afoundation upon which to build. Severalrecommendations are given in this chapter, includingthe use of an integration environment. This approachincorporates and supplements that of the U.S. Army’sCentral Technical Support Facility (CTSF), so essentialto the TF XXI experiment. For integrating a FOS,further refinements will be needed.

Some changes in the acquisition culture arerecommended as well. A template for future work is


put forward—many open questions remain to beresolved. Attention should be directed to the entire lifecycle of a SOS and FOS, whereas this book has focusedon the integration phase. Finally, the link between anintegration environment and future training is discussed.

Recent Coalition OperationsCoalition operations are intrinsic to the era of JointVision 2010. Yet problems with even rudimentaryinteroperability occur today in coalition operations—as demonstrated in Operation Restore Hope (Starr,1996). The compilation of lessons from Operation JointEndeavor in Bosnia by Larry Wentz (1998) illustratesthe difficulty. So detailed is his examination that itcomprises the equivalent of a case study, one focusedon a recent coalition operation.

The environments where today’s forces must functionare increasingly complex and heterogeneous and willgrow more so. While U.S. defense systems are managedby separate organizations in consonance with a defenseenterprise architecture, coalition partners bring theirown information systems. These are and will bedeveloped with different methodology, technology, andstandards. There will be greater diversity along withmultiple architectural frameworks, as was the situationin Bosnia.

According to Wentz (p 273):

Coalition operations such as Joint Endeavorpresent a complex set of challenges for the

205Chapter 7

military C4ISR1 system planners, implementers,and operators. The most difficult challenge isthe provision of integrated C4ISR services andcapabilities to support the needs of ad hocmultinational military force structures andpolitically driven command arrangements.Although integrated C4ISR services are thedesired objective, the realities tend to drive thesolution to stove-piped implementations. In spiteof technology advances, this will likely be thecase for some time to come. There will continueto be uneven C4ISR capabilities among coalitionmembers who will continue to rely on systemswith which they are most comfortable—theirown.

Wentz observes that this situation in Bosnia resultedpartially from the widely disparate informationtechnology capabilities in and among the coalitionmemberships. Often these were used in the absence ofin-country infrastructure. Examples abounded ofinfrastructures comprised of diverse parts. There wereindependent and separately managed NATO systemsfor voice, messages, and data and video networks. Thenational tactical assets of the framework nationsprovided telecommunication capabilities because theBosnia infrastructure was destroyed, some of it byNATO air strikes. These were supplemented withUnited Nations satellite terminals and commercialproducts and services.

1 Command, Control, Communications, Computers, Intelligence,Surveillance, and Reconnaissance.


There was a strategy to use commercial products andservices, but interoperability between military andcommercial systems continued to plague forces.Numerous examples occurred where strategic, theater,and tactical systems had difficulties in exchanginginformation. Networking applications also were notinteroperable. As anticipated, language was aninteroperability issue with 34 separate nationsparticipating (Wentz, pp. 351-353).

Various systems were acquired by many differentcoalition organizations, such as NATO. Somerequired integration with the information systems ofnon-government organizations and with those ofprivate volunteer organizations. In Bosnia thissituation was complicated by the number of civilianand non-government organizations with whominformation had to be exchanged, and with whomactivities required coordination. For example, civilactivities collectively constituted an importantintelligence asset. Some non-governmentalorganizations, private volunteer organizations, andinternational organizations were included inintelligence collection plans. Often such organizationsrelied on information infrastructures separate fromthose of the government or military organizations.

While Wentz asserts problems will continue, heacknowledges progress toward interoperability andcredits the good efforts of people in variousorganizations who did achieve a successful integrationof the disparate systems. However, the achievementconsumed time and resources. To integrate

207Chapter 7

communications and information systems, it required9 months of planning and lead roles by three nations toestablish the capability to allow the ImplementationForce (IFOR) to take command and control of theoperation (Wentz, pp. 280-282).

The Joint Vision 2010 Era andFederations of SystemsThe recent experiences in Bosnia presage the future era.While students of Joint Vision 2010 acknowledge itsreliance on a SOS, the operational needs for manymissions will require the capability of a FOS2 as well,where management of development and operations iscollaborative. Joint Vision 2010 embodies certainoperational concepts that rely on collaboration ratherthan central direction and management. However, howthese are actually implemented is still to be determined.

Joint Vision 2010 relies on partnerships, as typified bythe situation in Bosnia. As in Bosnia, systems in a FOSare acquired and managed by many organizations,including foreign, civil, and commercial organizations,military and non-military, governmental and non-governmental. Usually all partnerships are not the same;relationships and information sharing differ. In someoperations, there is even ad hoc participation byorganizations (and their systems), as also occurred inSomalia during Operation Restore Hope.

2 S/FOS characteristics are discussed in chap. 2.


With this reliance comes the heterogeneity of the FOS.This derives from the systems architectures that are thelegacies of many nations and many organizations.Diversity will propagate in the architectures throughthe application of different technical standards andthrough the rich mix of multiple cultures, languages,and semantics used to interpret them. The FOS alsomirrors the diversity in the differing relationships ofthe partners and their various operational architectures.The coalition organizations use systems that are alignedwith their individual requirements, organizationalstructures, and operating scenarios.

In the Joint Vision 2010 era, the local commander willbe given more assets, more information, and morecontrol over them. There will be more managementcontrol exercised at the tactical level, and in some casesto the level of the individual combatant. This is aconcept more consistent with federation principleswhere power is delegated, usually to the lowestpossible unit in an organization.

However these U.S. operational scenarios may besubstantially different from those of other coalitionpartners. Not only are organizational structures ofinternational partners frequently different, so are theirapproaches to command and the rate at which decisionsare made. In a coalition mission, control may beaccomplished at quite different levels by the variousparticipants. These differences in their operationalscenarios migrate into implicit differences in theirinformation systems, giving rise to a whole body of

209Chapter 7

effects that become obvious only when they areexercised together.

There are many relationships and interdependencies thatwill increase in complexity in the Joint Vision 2010era. Maneuverability by forces will require not onlygreater dispersion but also more agility across globaldistances. The use of coalition assets is critical to attainglobal reach. The global dispersion of joint capabilitiesand assets—at strategic, theater, and tactical levels—must collaborate in execution for massed effects.

The consequence of operating with greater dependencyon collaboration, increased heterogeneity, and greatlydispersed assets is increased vulnerability to failures ofinteroperability and integration. The implications of aFOS are portentous. Failure to achieve an integratedproduct could bring serious consequences to anoperational mission or result in capabilities that fail toprovide the best operational advantage.

A Way AheadThe more complex future undermines the viability ofintegrating the disparate parts of a FOS or a SOS, orintegrating them with limitations on time andresources. Certain strategies adapted in both the DPSand TF XXI programs proved beneficial. For now,these provide a foundation for a future SOS or a FOS,but more work is necessary. Several recommendations,including a template for future work, are summarizedin figure 7–1.


A Way Ahead

Direct attention behind the Wizard�s curtainin future experiments

✧ ✧ ✧ ✧ ✧ Include coalition partners inexperiments

Use an integration environment

Evaluate more case studies

Use a single facility for integration

✧ ✧ ✧ ✧ ✧ Investigate distributed integrations

Develop acquisition specialists for SOS andFOS

Address best practices for the life cycle

Tailor methods for estimating time and effort

✧ ✧ ✧ ✧ ✧ Compile SOS project databases

Address methods to train for a SOS and FOS

Figure 7-1. A Way Ahead

211Chapter 7

Direct Attention Behind the Wizard�s Curtain

There will be opportunities in the future to scrutinize,assess, and improve activities behind the Wizard�scurtain—if the attention is directed. Theimplementation of Joint Vision 2010 will proceed fromconcepts to joint warfighting capabilities through a long-term continuous process. (Shelton,1997–1998, 1998;Reimer, 1997–1998; Coats, 1997–1998; Hallion, 1997–1998; Barnett, 1997–1998; Hoffman, 1997–1998). Jointand service experimentation, exercises, anddemonstrations will continue to comprise elements ofthe evolution. The commitment to experimentation asa means to shape the implementation of Joint Vision2010 has led to the designation of the U.S. AtlanticCommand as the leader for joint experimentation. Asactivities proceed, various SOS or FOS (S/FOS) willbe configured; consequently, attention can be focusedon the efforts behind the Wizard�s curtain in additionto that directed on the operational performances playing“front and center stage.”

And to address the realities of the future,experimentation should include coalition partnersengaged in a spectrum of operations using their ownsystems integrated into a S/FOS.

Use an Integration Environment

To integrate a S/FOS, an integration environmentshould be used. Here this environment is labeled SFIE.3

It is a concept, not an organization. It is the environment

3 S/FOS integration environment.


of people, processes, and infrastructure used by a teamconsisting of acquisition and operational personnel tomanage the integration before the product is deployedfor an operation or an experiment and to sustain itafterward. As such, it describes who and what appearbehind the Wizard�s curtain. The environment canbe convened by any organization and constituted in anyappropriate facility. The SFIE is a mechanism to achievethe results required, to accelerate the integration process,and to garner sufficient quality in the product.Otherwise, the operational forces must expend the effortin the field—absent the critical mass of engineeringexpertise. Not only do the two case studies demonstratethis need when revolutionary capabilities are involved,but a similar lesson was derived in Operation JointEndeavor:

Exercises and training demonstrated the valueof setting up the expected C4I configurationsin advance of the deployment to sort outintegration and interoperability problems. Theexercises also served to train and do some teambuilding for those personnel who would deploy.(Wentz, p. 376)

Use of an integration environment does not replace theneed to acquire and test individual systems and certifytheir compliance to the overall architecture of the SOS.4

Nor does it replace the earlier phases of the life cycle.As methodology, the efforts in the integrationenvironment are additive to the essential activities

4 See the discussion on Lesson 1 in chap. 5.

213Chapter 7

accomplished before the integration begins, not “in lieuof.” Using a SFIE does not negate any efforts by theservices and agencies to develop and acquire robustsystem capabilities and build common architectures.Rather the application of the SFIE is tractable with thecontinuation of such efforts.

Elements Included in the Integration Environment

The case studies indicate the need for a set of processesand infrastructure that are common for all teamsparticipating in the integration—and established at theintegration facility. This set is fundamental and intendedas minimal—because the constituent systems of a S/FOS are managed, developed, and operated by manydifferent organizations for their own purposes with theirown processes and infrastructure.

The elements of the SFIE, compiled directly fromthe lessons of the case studies, are listed inAppendix B. The SFIE includes leadershipempowered for the integration, a S/FOS cadre, andparticipants who manage, develop, and operate theindividual S/FOS systems.

The inclusion of a “core” in the commoninfrastructure is necessary for integration5 andsustainment of the S/FOS during mission operation—to accommodate changes, such as from technologyinsertion, COTS turnover, and corrections. While this

5 A core is the minimum set of all the hardware components, allthe software, and the architectural framework in the S/FOS. Seechap. 5, lesson 3.


constitutes something of an investment, its value mustbe assessed against the risk of unanticipated andadverse impacts from changes while the integratedproduct is deployed in the field. For the DPS core,the investment was less than 1 percent of the cost ofthe total program.

This expenditure can provide added advantages forpurposes of training—but only with augmentation ifthe training population is large.6 Access to the S/FOScan facilitate the iterative learning process needed tomaster the capabilities of the integrated product, whileproviding training on the integrated capability actuallydeployed. These were important benefits experiencedin the two case studies and in Operation Joint Endeavor.

The Need for More Case Studies

Two case studies were used. Therefore it cannot beargued that these common processes and infrastructurein the SFIE constitute the complete set needed. Whileit was tempting to include more, caution is warrantedwhen the learning curve by participants could be steep,when adaptation by individual teams consumesschedule and resources, and when technologicalimplementation of the common infrastructure requirescontinuous change.

More assessments are required to determine otherelements that should be standardized and those that can

6 For verification, a core may require only a single workstationof a unique type. Hundreds of workstations would be requiredto train hundreds of operators concurrently.

215Chapter 7

remain particularized to the individual teams. Inreviewing both case studies, individual teams retainedmany processes and tools that remained uniquely theirown. For example, none of the individual teamsparticipating in the DPS or TF XXI had identical qualityassurance processes or tools.

Additional case studies would provide the means tocompare whether methods for one SOS unequivocallyapply to another and to a FOS as well. For example,determining the sensitivities of approaches to numbersof systems in the set or to measures of coupling7 betweensystems, or to the degree of autonomy, heterogeneity,and dispersion are comparative analyses of interest.

Test environments requiring integration of multiplesystems have been applied by the services. One exampleis that of the Fort Franklin/CUBE,8 used by the U.S.Air Force to manage the problems of technologyinsertion during Operation Joint Endeavor (Starr, 1996;Wentz). This also illustrated the advantages of retaininga core SOS to sustain an integrated product as changeswere required for a continuing operation.

7 Two systems are coupled if they are interdependent (i.e., if atleast one system requires information from the other, or requirescomponents, services, or people).8 Technology insertion was a major problem in Operation JointEndeavor. Wentz reports on replication of C3I systems for theCAOC in Vicenza at a laboratory called the C2 UnifiedBattlespace Environment (CUBE), at Air Force ElectronicSystems Center at Hanscom AFB. This was used for systemintegration testing of new capabilities before operationaldeployment to theater (p. 366-368).


Another rich source for examination is the effort toaddress the Year 2000 defect, conducted on a globalscale. Individual systems are being tested, certified forcompliance, and reintegrated into enterprises ofsystems. The successes (and failures) of the methodsused will render important lessons, many of which willapply to a S/FOS.

Periodic evaluations of the dynamics of integration ina S/FOS are warranted, given the march to a moreenterprise-centric strategy in the Department ofDefense, discussed in chapter 1. Neither case study usedin this work was able to take advantage of the (nowavailable) joint technical architecture,9 and more robustdefense information infrastructure and commonoperating environment—because of timing and phasing.The Army’s TF XXI established a common operatingenvironment using surrogates for the defense enterpriseapplications, many of them commercial products. Muchof the first version of the joint technical architecturewas based on the Army’s technical architecture. Howthese new(er) versions of the defense enterprise mightchange the lessons from future SOS and FOSintegrations should be evaluated.

More studies should assess integration environmentswith coalition partners. The differing methods usedby the various teams engaged in a FOS integrationcoupled with their multiple cultures and languagespresent challenges to defining common processes andinfrastructure. Achieving collaboration among

9 The Army’s technical architecture was a primary source forthe defense enterprise Joint Technical Architecture (JTA).

217Chapter 7

diverse communities of participants may requiredifferent strategies and incentives. There are examplesof SOSs and FOSs emerging from the private sector(Maier, 1998) that also provide additional materialfor evaluation.

A Word on Leadership

The two case studies provide limited insight formanaging behind the Wizard�s curtain when theenvironment is that of a FOS, which is collaborative innature. Rather, both programs exercised some form ofdirection over the various teams participating. Bothexamples illustrate the advantage of someone in charge.The overt empowerment of leadership of theintegrations was a strategy successfully used.

The two programs were simple in organizationalstructure. They were managed by a single agency and asingle service, respectively. The responsibility,accountability, and resources resided within a singleorganization for each.

The compelling need in future S/FOS ventures when itis not clear “who is in charge” is to resolve thatambiguity at the outset—or to determine a process forresolving it.

The leadership and management structures best suitedfor integration environments which are collaborativeor voluntary and ad hoc, rather than directed in nature,should evolve. Recent coalition operations and futureJoint Vision 2010 experiments with coalition partnersshould be assessed. Protecting the U.S. critical


infrastructure, the subject of national attention,requires collaboration among federal, state, local, andnon-governmental organizations; the resultingmanagement structures may provide examples whichare applicable (Presidential Commission Report,1997).

Use a Single Facility

The lessons from the two case studies concluded thatthe use of a single facility with a supporting environmentof common processes and infrastructure was essentialto the success of their integration events. Themanagement of each program made early use ofgeographically dispersed and distributed integrations—of individual systems, and of subsets of the SOS.However, to achieve the integrated product beforedeployment, each installed the set of all the individualsystems that comprised the SOS at a single facility. Andonly limited use was made of substitutes for individualsystems and components.

In an era of information technology and virtualcollaboration, it is perhaps surprising to express the needfor one real physical site for integration rather than fora virtual facility, but the evidence of both cases arguesstrongly10 for this. With the importance of managingnot only the achievement of a complex integratedcapability from diverse parts, but also in constructingone team from many, the collocation of teams cancontribute mightily. And for a military mission, the

10 See chap. 5, lesson 5.

219Chapter 7

operational community may be placed at increased riskif the integration is insufficient.

The management of a FOS integration is morechallenging than that of a SOS because collaborativerelationships must be developed and sustained across abroader spectrum of heterogeneous cultures,organizations, developments, and operations. The singlefacility and common environment can bringcohesiveness to build the twin citizenship necessary forfederations to succeed (Handy, 1992).

A facility with an integration environment could bedesignated when a mission is initiated. A S/FOS cadrecould be assembled temporarily with members drawnfrom various organizations and countries. However,creating at least some level of permanent staff andfacility is also an option (e.g., the Army has done thisfor integration with its CTSF at Fort Hood). Thisapproach has the merit of building resident expertise,expediting establishment of the integrationenvironment, and accelerating the integration processes.

Investigate Distributed Integrations

For coalition or joint missions, one integration sitemay not be feasible. The question that must beaddressed is: if collocation is not viable, whatenvironments (if any), virtual and distributed, can lendthe same success and effectiveness—to support theintegration of all systems in the set? Virtualenvironments must expand well beyond tools suchas video teleconferencing (extensively used by both


programs) to attain the communications, insights, andsynergies required for success.

Future experimentation must establish ifcollaborating but dispersed service integrationenvironments are sufficient for an integration of aS/FOS that supports joint missions—and sufficientfor an integration of a S/FOS that supports jointmissions and sufficient for integration of a FOSwhich supports coalition missions. The method fordistributing the integration may hold the key toeffectiveness. Partitioning subsets of the S/FOS bymission area (command and control) is oneapproach. There may be effective distributionsdeterminable when entire end-to-end threads ofactivities to support a mission are exercised on theS/FOS by specific (but different) operationalcommunities.

If integration proceeds incrementally with ever-growingnumbers of systems and subsets using distributedenvironments, can the sufficiency of the integrationprocess for all systems before deployment be preserved?This is a question that should not be too quicklyanswered or dismissed, but the evidence of the casestudies suggests that “no” is the current answer. Andthe reasons extend beyond just the integration ofconstituent systems to the cohesion of the operationalviewpoints and of the Wizard’s team.

221Chapter 7

Develop Acquisition Specialists

The Heraclitan principle says, “You can neverexperience the same SOS twice.” The future will requiremany integrations of a SOS or a FOS. It is important tohave an appropriately skilled and sufficiently staffedcadre of people responsible.

A S/FOS cadre should be developed within theacquisition community. Because it is the whole, ratherthan the parts, that delivers the required results the futureU.S. defense strategy is based on, the acquisition cultureshould endorse the merit and evolve the skills forproducing the integrated product. The expertise requiredis in short supply, even in the private sector, and mustbe developed, as well as retained, in sufficient numbers.

Such roles are distinct from those of the programmanagers of individual systems. Program managers willcontinue to develop and maintain individual andindependent systems to serve other and variousmissions. Within the acquisition environment, they willcontinue to have considerable authority, controlsignificant dollars, and consequently be providedsignificant recognition.

A S/FOS cadre merits equivalent recognition andposition. In addition to providing needed expertise, suchspecialists could evolve and reinforce architectures thatare enterprise-centric for joint and coalition missions,while instantiating processes for integratingindependently managed constituents. As missions crossinstitutional lines, the program and engineeringadjudication process intrinsic to the cadre transcends


the interests of parent organizations as well as ofindividual systems. Institutionalizing the cadre couldpromote universal recognition as well as provide anecessary balance in authority to that of the programmanagers who must manage within their own cost,schedule, and performance constraints.

The DPS case study illustrated how an Agency’sculture affected the SOS cadre. The acquisition teamsof the individual systems delivered functionalcapabilities identified with core business of theAgency. They were more easily recognized ascontributing to Agency business than those in the SOScadre. Less value and responsibility were perceivedin engineering a SOS than in delivering a productionsystem. Furthermore the SOS cadre was considerednecessary only until FOC, when it was reduced innumber and rank. Unfortunately this occurred justwhen the need to evolve the DPS became greater.

Life Cycle of System of Systems and Federation ofSystems

While this book has focused on the integration phase,attention to all the phases of the life cycle of a S/FOS iswarranted. Both program ventures dealt with challengesother than those of integration. For example, thearchitecting processes for the TF XXI allocatedrequirements across a broad spectrum of existing anddevelopmental systems and their interfaces. Thisconsumed time, requiring 7 months for an initialarchitectural baseline, which subsequently was refined.

223Chapter 7

For other ventures, the best strategies to expedite andoptimize this process would be advantageous.

A guidebook for managers should be developed for aS/FOS; architectural and design principles tailored fora S/FOS would provide substantial benefits. Thesecould be derived from those practices and lessonscompiled from appropriate case studies and programventures. An example of an analogous reference thatdelineates best practices and supporting tools is the DoDSoftware Acquisition Best Practices Initiative (1997),and there are others available.11 However, as are many,it is oriented to the management of software projects.

Considerable work is proceeding to frame the practicesand processes for software-intensive projects and thosefor large and complex information systems into anintegrated whole (Schaeffer, 1998). There are currentand emerging standards for engineering a system andaddressing the life-cycle processes; these are discussedin Wright (1998). An assessment of practices andprocesses specifically tailored for a S/FOS and evolvedfrom this integrated framework merits attention.

Address Architecting in Best Practices

A key question is: How does a manager develop thearchitecture for a S/FOS? While the U.S. defenseenterprise provides a substantial framework of commonsystems and applications as a foundation, there will be11 The Defense Systems Management College disseminatesinformation on best practices and lessons learned for theacquisition work force, available through its web site (DefenseSystems Management College).


other and various systems in the set of systems used fora specific mission. And in dealing with a FOS, there isgreater heterogeneity. This brings increased problemsin interoperability, and therefore in integration.

The diversity in architectural frameworks in the FOSand the plethora of legacy systems contributed bythe participating organizations compounds thechallenge. Different standards and guidelines not onlycan impose different rules, but de facto the sets ofrules may not be interoperable. Perhaps moreproblematic, because it is less obvious, are the moresubtle architectural disconnects that arise fromsystems designed for different purposes or bydifferent development communities operating withimplicit understandings of their standards andguidelines that are not readily assimilated, traceable,or recoverable. And this can and does occur evenwith commercial products and services.12

One approach lies in reducing the multiplicity offrameworks, possibly by narrowing the choices ofstandards, similar to the mandate of the joint technicalarchitecture within the U.S. defense enterprise. At leastone recommendation has been made to accomplish thisthrough the application of commercial standards(Commission on Physical Sciences, Mathematics, andApplications, “Realizing the Potential of C4I:Fundamental Challenges,” 1999). Perhaps a coalition

12 One of the effects of increasing reliance on COTS packagesmay be the introduction of architectural mismatches becausecommercial products are developed for various architecturalframeworks (Gacek & Boehm, 1998).

225Chapter 7

technical architecture is viable, but it will not occurquickly, but rather evolve deliberately, rigorously, and(probably) incrementally with specific coalitionpartners. Another strategy is to allow the multiplicitybut use automated means to identify and reconcilearchitectural disconnects or to translate diversity intocommonality. Even if feasible, this also is not anoptimum or short-term strategy.

Adapting architecting principles for a S/FOS in thecontext of Joint Vision 2010 would address perhapsthe most important phase of the acquisition life cycle.However there is more basic work required, includinggeneral agreement on taxonomy and thosecharacteristics which should be used to distinguishclasses of “systems of systems.” (In this book the degreeof autonomy, heterogeneity, and dispersion have beenused.) Bounding a SOS in an enterprise of many systemsor bounding a FOS in the enterprises of coalitionpartners also present fundamental challenges, thestrategies for which are not currently obvious.

In addition to a proposed taxonomy, Maier (1998)already has provided some basic principles forarchitecting systems of systems through severalheuristics that are refinements of more generalguidelines. The Army’s success in developing anarchitecture for the TF XXI SOS, using the viewpointsof operational, systems, and technical architecture,provides a good example of architecting processes. Thefact that the technical architecture used for theexperiment correlated highly to the initial version ofthe joint technical architecture makes the TF XXI


example a significant one for practices related to thedefense enterprise.

A compilation of principles and design guidelines wouldbenefit from those used in prominent examples like theInternet, as well as from more robust applications thatare emerging in various business sectors such asintelligent transport systems (Maier, 1997).

Refine Methods for Estimating Time and Effort

For conducting activities behind the Wizard�scurtain, it is important to plan adequately. To do thisrequires the ability to estimate cost and scheduleresources for producing a S/FOS from a collection ofindependently developed and operated systems. Thelesson13 derived from the two case studies points to theneed to plan for significant time and effort. However,the discussion flagged difficulties with good estimationmethods and models tailored to a S/FOS.

Typical models available are based on a history ofsoftware projects. Many models have evolved usingexperience-based estimation, based on project databases compiled over the lives of numerous projects.Other models use parameters, but are tailored to specificdomains, such as the military developmentenvironment. A good summary of estimationmethodology is provided in “Software EstimatingTechnology” (Stutzke).

13 See chap. 5, lesson 4.

227Chapter 7

Resources can be estimated using various factors thatcharacterize the project, such as developmentapproaches, product complexity, team experience, andwork environments (Boehm, et al., 1996).They can beprojected for different phases of the life cycle as well.However, even for conventional projects, Capers Jones(1998) notes that most tools available today areinadequate in treating data bases, multiple domains,14

and enterprise estimation. And they are calibrated toprojects different than a S/FOS.

A S/FOS comprises existing systems independentlydeveloped for other and specific purposes. Tailoredestimation methods are needed for projects that includea single enterprise as well as multiple enterprises, andwith components that include substantial numbers oflegacy systems as well as developmental systems.

Good estimation processes also require a data base ofprojects that are analogous. Royce (1998, p 29) says:

Extrapolating from a good estimate, an idealestimate would be derived from a mature costmodel with an experience base that reflectsmultiple similar projects done by the same teamwith the same mature processes and tools.

Two developments are needed for more accurateestimation of resources for a S/FOS—tailored methodsand an empirical project data base. The opportunity tocompile data on S/FOS activities behind the Wizard�s

14 Products constructed from hardware components, softwarecomponents, data base components, and microcodedcomponents.


curtain comes in conjunction with the Joint Vision2010 experimentation and from other S/FOS projects.Compilation of data coupled with the best methodsfor estimation should evolve to enable estimation apriori, for each phase of the life cycle, and for eachspecific S/FOS.

Address Training with a System of Systems andFederation of Systems

During the integration phase of both case studies, theoperational community trained using the integratedproduct during integration. The lessons learned 15 canbe translated into three training principles applicableto missions that require a S/FOS:

• Train operators on the S/FOS in iterations

• Train operators for the S/FOS, not just forindividual systems

• Train the team behind the Wizard�s curtain.

Determining the best methods to implement thesesimple principles needs to be addressed.

During integration, changes (including corrections andadaptations) are continuous. They occur in theconstituent systems and all the relationships. The simpleapproach of waiting to train until stability is reached isnot an option—because the mission schedule is usuallyurgent, because the training population is large, andbecause Heraclitus is right. A window of time without

15 See chap. 6, training lessons 1–3.

229Chapter 7

changes is a small one. Developing the best methods tooffset the adverse effects of change during integrationwhile training to teach the synergy of the whole usingthe capability deployed deserves future attention.

The dynamics of integration complicate the task ofproviding and maintaining a current perspective ofthe whole—which emerges more accurately with theintegration and evolves through the operational use.The orientation of the operator to an individual systemshould be supplemented with the perspective of theS/FOS—then refreshed. And the orientation needsto be reformed for each mission, when a differentSOS and FOS are required. The relationships withinand the synergy of the whole are different.

The dynamics of integration complicate the training ofthe teams behind the Wizard�s curtain. “Training” inthis context is about the S/FOS architecture—theoperational, technical, and systems viewpoints. Manyparticipants bring perspectives appropriately but entirelyfocused on the management and development of theirown systems. Providing a current and commonunderstanding of the architecture and the relationshipsamong the constituent systems while communicating thedynamic and emerging behavior of the whole entity is atraining need that should not be overlooked. The methodsdealing with change and maintaining current informationon the integrated product for training the operationalcommunity are applicable for this population as well,but the means remain to be determined.


Realizing the PromiseThe concluding recommendation is the same as thefirst—attention must be directed at those activitiesbehind the Wizard�s curtain that produce, as wellas integrate, a SOS or FOS. The joint experimentationfor Joint Vision 2010 provides the opportunity. Thecase studies and current coalition operations presagethe complexity of future ventures. The foundationestablished through the case studies—the lessonslearned, and the integration environment—are neithera final solution nor a silver bullet for the Joint Vision2010 era. Future experimentation and futureassessments are essential.

The U.S. defense strategy and Joint Vision 2010, asenvisioned, rely on an integration of systems to form aSOS, and, implicitly, a FOS. Many differentcombinations of systems will be required based on thetype of mission and the nature of the geopoliticalenvironment. “You can never experience the same SOStwice.” There is not one SOS or one FOS, but many.This reinforces the need to continue to examine theprocesses to determine how systems, independentlymanaged, developed, and operated, may be integratedmore quickly and more effectively.

There will be future opportunities to scrutinize allaspects of producing, integrating, deploying, andsustaining a SOS and a FOS. But the data must becompiled and the analyses conducted for the Wizard’steam. Most importantly, as lessons are learned throughthese examinations, they must be used as a foundation

231Chapter 7

for better principles and practices which address everyaspect of a successful SOS and FOS. This evolution isnecessary to produce the technological magic forrealizing the promise of Joint Vision 2010. To continuethe assessments of activities behind the Wizard�scurtain is the most important message of this work.

A-1

Appendix A

The Lessons Learned

Nine Lessons Learned on IntegrationLesson 1

Certain activities should precede a SOSintegration. These include:

defining the SOS architecture;

developing and testing the individual systemconstituents of the SOS;

developing and testing the interfaces betweenand among the individual systems of the SOS;

independently certifying compliance with theSOS architecture.

Lesson 2

Use early, incremental, and iterative integrationto achieve a SOS.

Lesson 3

The testing strategy for the integration of a SOSrequires:

A-2 Behind the Wizard’s Curtain

an agreed-to plan and process for testing, basedon a risk assessment;

a suite of activities representative of theoperational requirements of the mission theSOS supports;

the exercising of a full spectrum of the SOSactivities (end-to-end) by operators, using theactual constituent systems of the SOS�or atleast a core SOS.

Lesson 4

To integrate all the systems of a SOS, plan forsubstantial difficulties and significant time andresources.

Lesson 5

The use of a single facility�with anenvironment of people, processes, andinfrastructure�substantially facilitates theintegration of a SOS from individual systems.

Lesson 6

The process for SOS integration should overtlyaddress the leadership of the integration asfollows:

an on-site acquisition leader empowered for theintegration of the SOS and an on-site leaderempowered for the operational community;

supported by a SOS cadre��with sufficientresources and authority;

A-3Appendix A

supported by participants who manage,develop, and operate the constituent systemsof the SOS.

Lesson 7

Certain common processes and commoninfrastructure in the integration environmentare essential to manage a SOS integrationsuccessfully. These include the following:

an Engineering Board with responsibility andauthority for identification and resolution ofSOS issues and discrepancies, including theassignment of responsibility for correction;

establishment of processes (and the automatedmeans) for identification of SOS issues anddiscrepancies, their disposition, tracking, andresolution, under the management of theEngineering Board;

automated support for the tracking and tracingof SOS operational requirements;

configuration management and control of thehardware and software baselines of thesystems of the SOS by the integrationleadership, supported with: automated meansfor identifying and controlling the baselinesand subsequent changes; a formal build,verification, and re-integration process forchanges;


A-4 Behind the Wizard’s Curtain

an office automation environment to supportthe integration�s administrative processes aswell as to support interpersonal processing andcommunications for the participants.

Lesson 8

Certain common processes and infrastructurein the SOS integration environment promoteeffectiveness and efficiencies. These include:

daily planning and scheduling of resources(people, equipment, facilities) for integrationevents�with contingency plans and schedulesreadily available;

timely dissemination of information pertinentto each integration event, such as test status,equipment availability, and results;

daily status meetings, with results immediatelyavailable.

Lesson 9

Prototyping a SOS can provide early insight intooperational requirements and into the SOSsystems architecture.

Three Lessons Learned on TrainingTraining Lesson 1

Train operators on a SOS using a full spectrumof operational activities, and train allowingiterations.

A-5Appendix A

Training Lesson 2

The training infrastructure should beaugmented to provide the perspective of a SOS.

Training Lesson 3

Training should be provided to those behindthe Wizard�s curtain.

B-1

Appendix B

The IntegrationEnvironment

The elements of the integration environment for aSOS and a FOS (S/FOS) include:

An integration facility with its internaladministrative infrastructure;

A team consisting of:

an on-site acquisition leader empowered for theintegration of the S/FOS and an on-siteoperational leader empowered for theoperational community;

a S/FOS cadre;

participants from organizations who manage,develop, and operate the constituent systemsof the S/FOS.

Common processes consisting of:

a testing strategy for integration whichincludes a suite of activities representative ofthe full spectrum of missions the S/FOSsupports; exercising of those activities end-to-end by operators, on the actual or core S/FOS;

B-2 Behind the Wizard’s Curtain

an engineering board with the authority foridentification and resolution of S/FOS issuesand discrepancies, including the assignment ofresponsibility for correction;

the tracking and tracing of S/FOS operationalrequirements;

configuration management and control of thehardware and software baselines of thesystems of the S/FOS, including a formal build,verification, and re-integration processes forchanges;

daily planning and scheduling of activities,including contingencies;

the dissemination of information pertinent toeach integration event;

the dissemination of daily status.

Common infrastructure consisting of:

the actual S/FOS or a core S/FOS;


an office automation infrastructure to supportadministrative processes of the integration.

C-1

Appendix C

Acronym List

4ID—Fourth Infantry Division

A2C2S—Army Airborne Command and ControlSystem

ABCS—Army Battle Command System

ABIS—Advanced Battlespace Information System

ACM —Association for Computing Machinery

ACT—Activation Control Team

ADA—Air Defense Artillery

ADO—Army Digitization Office

AFATDS—Advanced Field Artillery Tactical DataSystem

AMC —U.S. Army Materiel Command

AMPS—Aviation Mission Planning System

AN/TPQ–36v8—Firefinder Radar System

ANBACIS —Automated Nuclear Biological andChemical Information System

ASAS—All Source Analysis System

C-2 Behind the Wizard’s Curtain

ATA —Army Technical Architecture

ATCCS—Army Tactical Command and ControlSystem

ATM —Asynchronous Transfer Mode

ATMT —Agency Transition Management Team

AVN—Aviation

AVTOC —Aviation Tactical Operations Center

AWE—Advanced Warfighting Experiment

B2—Brigade and Below

BCIS—Battlefield Combat Identification System

BCV—Battle Command Vehicle

BDE—Brigade

BN—Battalion

BSFV-E—Bradley Stinger Fighting Vehicle–Enhanced

BTRY—Battery

C2—Command and Control

C2V—Command and Control Vehicle

C4I—Command, Control, Communications,Computers, and Intelligence

C4ISR—Command, Control, Communications,Computers, Intelligence, Surveillance, andReconnaissance

C-3Appendix C

CCRP—C4ISR Cooperative Research Program

CGSP—COTS Ground Station–Prototype

CIS—Communications and Information Systems

COE—Common Operating Environment

CSSCS—Combat Service Support Control System

CTSF—Central Technical Support Facility

CUBE—Command and Control Unified BattlespaceEnvironment

DBS—Direct Broadcast Satellite

DII —Defense Information Infrastructure

DIL —Digital Integration Laboratory

DISC4—Director of Information Systems, Command,Control, Communications, and Computers

DMA —Defense Mapping Agency

DPS—Digital Production System

E&R —Exercises and Rehearsals

ENG—Engineer

EPLRS—Enhanced Position Location ReportingSystem

EXFOR—Experimental Force

FAAD—Forward Area Air Defense

FAASV—Field Artillery Ammunition Supply Vehicle


FOC—Full Operational Capability

FORSCOM—U.S. Army Forces Command

FOS—Federation of Systems

FSB—Forward Support Battalion

GBS—Ground Based Sensor

GCCS—Global Command and Control System

GPS—Global Positioning System

HH—Handheld

IEWCS—Integrated Electronic Warfare CommonSensor

IFOR—Implementation Force

INC—Internet Controller

IOC—Initial Operating Capability

IREMBASS—Improved Remotely MonitoredBattlefield Sensor System

ISR—Intelligence, Surveillance, and Reconnaissance

JSTARS—Joint Surveillance Target Attack RadarSystem

JTA—Joint Technical Architecture

LAN —Local Area Network

LGSM—Light Ground Station Module

LINC —Lightweight Internet Controller

C-5Appendix C

LLDR —Lightweight Laser Designator andRangefinder

LRAS3—Long Range Advanced Scout SurveillanceSystem

M1A1—M1 Main Battle Tank, Improved Version

MC&G —Mapping, Charting, and Geodesy

MCS/P—Maneuver Control System/Phoenix

MCS—Maneuver Control System

MECH —Mechanized

MET MEAS —Meteorological Measuring Set

MFCS—Mortar Fire Control System

MI —Military Intelligence

MP—Military Police

MSE—Mobile Subscriber Equipment

NIMA —National Imagery and Mapping Agency

NMT —Network Management Terminal

NTC—National Training Center

OH-58D—Kiowa Warrior Helicopter

OOTW—Operations Other Than War

OPFOR—Opposing Force

OTN—Own the Night


PEOC3S—Program Executive Officer for Command,Control, and Communications Systems

PEO—Program Executive Officer

PLGRS—Precision Lightweight GPS Receiver System

PLS—Palletized Loading System

PLT—Platoon

POS/NAV—Position/Navigation Initiative

PSSCS—Personnel Service Support Control System

RECON—Reconnaissance

RF—Radio Frequency

RWS—Remote Work Station

S/FOS—System of Systems and/or Federation ofSystems

SDR—Surrogate Data Radio

SFIE—SOS and FOS Integration Environment

SINCGARS—Single Channel Ground and AirborneRadio System

SIP—System Improvement Program

SIV—System Integration Van

SOS—System of Systems

SPOEM—Special Program Office for ExploitationModernization

C-7Appendix C

STAMIS—Standard Army Management InformationSystem

STM—System Test Mode

TELE-MED —Telemedicine

TF XXI —Task Force XXI

TMG —Tactical Multinet Gateway

TNS—Tactical Name Server

TOC—Tactical Operations Center

TPN—Tactical Packet Network

TRADOC —U.S. Army Training and DoctrineCommand

TSIP—Task Force XXI System Integration Plan

UAV-SR—Unmanned Aerial Vehicle–Short Range

USDR&E—Under Secretary of Defense for Researchand Engineering

USIGS—United States Imagery and GeospatialInformation System

WAM —Wide Area Mine

X-FIST (BRAD)—Experimental Fire Support Team(Bradley)

D-1

Appendix D

Glossary of Terms

Several sources have been used for thesefrequently used terms. It is noted whenadaptations have been made.

Architecture. The organizational structure of a systemor component (IEEE Standard 610.12, 1990).

Battlefield digitization. The U.S. Army’s program toapply information technologies to acquire, exchange,and use timely digital information tailored to the needsof each decider, shooter, and supporter (Providing theMeans, 1994).

Core SOS. A minimum set of all unique hardwarecomponents and all software baselines of individualsystems of the SOS and the architectural framework.

Coupling. Two systems are coupled if they areinterdependent (i.e., if at least one system requiresinformation from the other, or requires components,services, or people). Tighter coupling indicates greaterinterdependencies between systems than does loosecoupling.

A federation of systems. A system of systems managedwithout centralized authority and direction.

D-2 Behind the Wizard’s Curtain

Geospatial data. Information that identifies thegeographic location and characteristics of natural orconstructed features and borders of the earth (USIGSGlossary, 1998).

Heraclitan principle. A paraphrase of the sayings ofthe Greek philosopher, Heraclitus: “You can neverexperience the same SOS twice.”

Information superiority. The capability to collect,process, and disseminate an uninterrupted flow ofinformation while exploiting or denying an adversary’sability to do the same (Joint Vision 2010, 1996).

Infrastructure. The underlying processing,communications, and organization base, people, andprocesses, to support the function specified by thecontext in which the term is used.

Integration. The process of combining the componentsof a system into an overall system, or the process ofcombining the systems of a set of systems into a SOS(adapted from IEEE Standard 610.12-1990).

Integration environment. The people, commonprocesses, and common infrastructure established at anintegration site to support SOS (or FOS) integration.See Appendix B.

Integration testing. An orderly progression of testingin which the components of a system or systems of aset of systems are combined and tested until the systemor set of systems has been evaluated (adapted from IEEEStandard 610.12-1990).

D-3Appendix D

Interface. A shared boundary across which informationis passed (IEEE Standard 610.12-1990).

The interfaces of a system. A connecting link orinterrelationship between two systems, two devices, twoapplications, or the user and an application, device, orsystem (DISA DII Master Plan, 1998).

Interoperability. The ability of two or more systemsor components to exchange information and to use theinformation that has been exchanged (IEEE Standard610.12-1990).

Mapping, Charting, and Geodesy (MC&G). Thecollection, transformation, generation, dissemination,and storing of geodetic, geomagnetic, gravimetric,aeronautical, topographic, hydrographic, cultural, andtoponymic data (USIGS Glossary, 1998).

Open system. A system that implements sufficient openspecifications for interfaces, services, and supportingformats to enable properly engineered applicationssoftware: (1) to be ported with minimal changes acrossa wide range of systems; (2) to interoperate with otherapplications on local and remote systems; and (3) tointeract with users in a style that facilitates userportability (DISA DII Master Plan).

Operational architecture. A description, oftengraphical, of the operational elements, assigned tasks,and information flows required to support thewarfighter. It defines the type of information, thefrequency of exchange, and what tasks are supported

D-4 Behind the Wizard’s Curtain

by these information exchanges (DISA DII MasterPlan).

Shared situational awareness. The ability of a unit toknow where its friends are located, where the enemyis, and to share that information with other friends, bothhorizontally and vertically, in near real-time (Providingthe Means, 1994).

SOS cadre. The people responsible for architecting,engineering, testing, and integrating a SOS usingsystems managed, developed, and operated by (other)organizations and people.

System. A collection of components organized toaccomplish a specific function or set of functions (IEEEStandard 610.12-1990).

System. A collection of different things that togetherproduce results unachievable by themselves alone(Rechtin & Maier, 1997).

Systems architecture. A description that defines thephysical connection, location, and identification of keynodes, circuits, networks, warfighting platforms, andso forth, and specifies system and componentperformance parameters. The systems architecture isconstructed to satisfy operational architecturerequirements per standards defined in the technicalarchitecture. The systems architecture shows howmultiple systems within a subject area link andinteroperate and may describe the internal constructionsor operations of particular systems within thearchitecture (DISA DII Master Plan).

D-5Appendix D

A system of systems. A set of different systems soconnected or related as to produce results unachievableby the individual systems alone.

Technical architecture. A description that identifiesthe services, interfaces, standards, and theirrelationships. It provides the technical guidelines forimplementation of systems upon which engineeringspecifications are based, common building blocks arebuilt, and product lines are developed (DISA DII MasterPlan).

E-1

Appendix E

Bibliography

Advanced Battlespace Information System (ABIS) TaskForce Report. Sponsored by Director for Command,Control, Communications, and Computers (JointStaff) and Director of Defense Research andEngineering (OSD). May 1996.

Alberts, Davis S. Operations Other Than War: TheTechnological Dimension. Washington, DC:National Defense University Press. 1995. 3–4.

Army Technical Architecture, Version 4.5. At web sitewww.hqda.army.mil/webs/techarch/. 1996.

Arthur, W. Brian. “How Fast is Technology Evolving?”Scientific American. February 1997. 105, 107.

Barnett, Roger W. “Grasping 2010 with NavalForces.” Joint Forces Quarterly, 17. Autumn/Winter 1997–1998. 25-31.

Baum, L. Frank. The Wizard of Oz. Illustrated JuniorLibrary. New York: Grosset & Dunlap. FirstCopyright, 1900, 1903 by L. Frank Baum andW. W. Denslow.

E-2 Behind the Wizard’s Curtain

Berry, B.J.L. “Cities as Systems within Systems ofCities.” Papers of Regional Sciences Association,13. 1964. 147–163.

Boehm, Barry, Bradford Clark, Ellis Horowitz, ChrisWestland, Ray Madacy, and Richard Selby. “TheCocomo-2 Software Cost Estimation Model.”American Programmer, IX:7. Cutter InformationCorp. July 1996.

Boutelle, BG Steven, USA, and Alfred Grasso. “A CaseStudy: The Central Technical Support Facility.”Army RD&A., March–April 1998. 30–33.

Boutelle, COL Steven. TF XXI briefing slide. 1996.

Brownsword, Lisa, David Carney, and TriciaOberndorf. “The Opportunities and Complexities ofApplying Commercial-Off-the-Shelf Components.”Crosstalk: The Journal of Defense SoftwareEngineering. April 1998.

Butler, S., D. Diskin, N. Howes, and K. Jordan.“Architectural Design of a Common OperatingEnvironment.” IEEE Software, New York: Institutefor Electrical and Electronics Engineers, 13:6.November 1996. 57–46.

Caldwell, BG John, USA. Interview. Jane’s DefenceWeekly. April 9, 1997. 32.

Coats, Dan. “Joint Experimentation—Unlocking thePromise of the Future.” Joint Forces Quarterly, 17.Autumn/Winter 1997–1998. 13–19.

E-3Appendix E

Cohen, William S. “The Secretary’s Message.” Reportof the Quadrennial Defense Review. May 1997. Atwebsite http://www.defenselink.mil/pubs/qdr/msg.html.

Commission on Physical Sciences, Mathematics, andApplications. Realizing the Potential of C4I:Fundamental Challenges. National ResearchCouncil. Washington, DC: National Academy Press.1999.

Concept for Future Joint Operations: Expanding JointVision 2010. Office of Primary Responsibility:Commander, Joint Warfighting Center, FortMonroe, VA. May 1997.

Czerwinski, Thomas J. Coping with the Bounds:Speculations on Nonlinearity in Military Affairs.Washington, DC: National Defense UniversityPress. 1998. Chaps. 1-2.

Czerwinski, Thomas J. “Command and Control at theCrossroads.” Parameters, 26:3, Autumn 1996. 121–132. At website http://carlisle-www.army.mil/usawc/parameters/issues.htm.

de Laguna, Theodore. “The Importance ofHeraclitus.” The Philosophical Review, 30:3. May1921. 238–254.

Defense Information Systems Agency. DefenseInformation Infrastructure (DII) Master Plan.Version 7.0. Glossary. March 1998.


Defense Systems Management College. “Best Practicesand Lessons Learned.” At web sitewww.dsmc.dsm.mil/.

Department of Defense Joint Technical Architecture(JTA), Version 2.0. At DISA web site, www-jta.itsi.disa.mil. May 1998.

Department of Defense Software Acquisition BestPractices Initiative. The Program Manager’s Guideto Software Acquisition Best Practices. Version 2.0.April 1997.

Eisner, H. “RCASSE: Rapid Computer-Aided Systemsof Systems (S2) Engineering.” Proceedings of the3rd International Symposium of the National Councilon System Engineering (NCOSE), 1. 1993. 267–273.

Fox, Greg, Steven Marcom, and Karen W. Lantner. “ASoftware Development Process for COTS-BasedInformation Infrastructure.” Crosstalk: The Journalfor Defense Engineering. April 1998.

Gacek, Cristina and Barry Boehm. “ComposingComponents: How Does One Detect PotentialArchitectural Mismatches?” Workshop onComputational Architectures. Monterey, CA.January 6–8, 1998.

Gibish, Jane E. “Force XXI: A Selected Bibliography.”U.S. Army War College Library. At web site http://carlisle-www.army.mil/library/bibs/forcexxi.htm.April 1997.

E-5Appendix E

Goedkoop, COL Thomas R., USA, and CPT Barry E.Venable, USA. “Task Force XXI: An Overview.”Military Review. March–April 1997.

Hagel, John. In an interview with Kevin Kelly. “It Takesa Village to Make a Mall.” Wired, 5.08. August1997. 4–6.

Hallion, Richard P. “Airpower and the Changing Natureof Warfare.” Joint Forces Quarterly, 17. Autumn/Winter 1997–1998. 39-46.

Handy, Charles. “Balancing Corporate Power: A NewFederalist Paper.” Harvard Business Review, 70:6.November–December 1992. 59–72.

Hanna, LTC Mark, USA. “Task Force XXI: TheArmy’s Digital Experiment.” National DefenseUniversity Strategic Forum, 119. Washington, DC:National Defense University Press. July 1997.

Hartzog, GEN William W., USA. “Task Force XXIFinal Report Executive Summary.” HQ TRADOC,DCSCD (ATCD-J) Force XXI/Joint Venture. FortMonroe, VA. October 1997.

Hester, CPT Henry M., Jr., USA. “Digitization in TaskForce XXI.” Field Artillery Journal, September–October 1996. 42–44.

Hoffman, F.G. “Joint Vision 2010—A MarinePerspective.” Joint Forces Quarterly, 17. Autumn/Winter 1997–1998. 32-38.


Holcombe, Robert. “Some Lessons Learned WhileDigitizing the Battlefield.” Proceedings of theBattlefield Systems International 98 Conference.June 9–11, 1998. 73–89.

Institute for Electrical and Electronics Engineers. IEEEStandard 610.12-1990, IEEE Standard Glossary ofSoftware Engineering Terminology. New York.1990.

“Joint Vision 2010, American’s Military—Preparingfor Tomorrow.” Joint Force Quarterly, 12. Summer1996. 34–49.

Joint Vision 2010. Office of Primary Responsibility:Chairman of the Joint Chiefs of Staff, The Pentagon,Washington, DC. 1996.

Joint Warfighting Science and Technology Plan.Department of Defense, Director of DefenseResearch and Engineering, Office of the Secretaryof Defense, Washington, DC. February 1998.

Jones, Capers. “Software Project Management in the21st Century.” American Programmer, XI: 2. CutterInformation Corp. February 1998.

Jones, Capers. Applied Software Measurement. NewYork: McGraw-Hill. 1996a.

Jones, Capers. “Large Software System Failures andSuccesses.” American Programmer. CutterInformation Corp. 1996b.

E-7Appendix E

Jones, Capers. Patterns of Software Systems Failureand Success. International Thomson Press. 1996c.

Kelly, Kevin. “New Rules for the New Economy.”Wired. September 1997.

Layton, Richard L. “Command and Control Structure.”In Wentz, Larry, Ed. Lessons From Bosnia: TheIFOR Experience. Washington, DC: NationalDefense University Press. 1998. 35–52.

Littlefield, Keith E. “The Defense Mapping Agency’sDigital Production System (DPS).” Cartographyand Geographic Information Systems, Journal ofAmerican Congress on Surveying and Mapping,22:2. 1995. 119–127.

Maier, Mark W. “Architecting Principles for Systems-of-Systems.” Preprint submitted to SystemsEngineering. 1998.

Maier, Mark W. “On Architecting and IntelligentTransport Systems.” IEEE Transactions onAerospace and Electronic Systems, 3: 2. 1997.610–662.

Maier, Mark W. “Architecting Principles for Systems-of-Systems.” 6th Annual International Council onSystem Engineering (INCOSE) SymposiumProceedings. 1996. 567–574.

Meigs, LTG Montgomery C., USA, and COL EdwardJ. Fitzgerald, III, USA. “University After Next.”Military Review. March–April 1998. 37–45.


Naylor, Sean D. “Army Fires First Round in Battle forthe Future.” Army Times. April 28, 1997.

Owens, Adm William A., USN. “The Emerging U.S.System-of-Systems.” National Defense UniversityStrategic Forum, 63. Washington, DC: NationalDefense University Press. 1996.

Perrow, Charles. Normal Accidents: Living with HighRisk Technologies. New York: Basic Books, Inc.1984.

Providing the Means. U.S. Army Digitization Office.Washington, DC: Office of the Chief of Staff Army.1994.

Rechtin, Eberhardt. Systems Architecting: Creating andBuilding Complex Systems. Englewood Cliffs, NJ:Prentice Hall. 1991.

Rechtin, Eberhardt and Mark W. Maier. The Art ofSystems Architecting. CRC Press. 1997.

Reimer, GEN Dennis J., USA. “The Year in Review—The Year Ahead.” Remarks at the Institute of LandWarfare. 1998.

Reimer, GEN Dennis J., USA. “Leaping Ahead to the21st Century.” Joint Forces Quarterly, 17. Autumn/Winter 1997–1998. 20-24.

Reimer, GEN Dennis J., USA. “Preparing Now to Meet21st Century Challenges.” Army Green Book, 47:10.October 1997.

E-9Appendix E

Reimer, GEN Dennis J., USA. Dwight D. EisenhowerSpeech. October 15, 1996.

Presidential Commission on Critical Infrastructure.Critical Foundations: Protecting America’sInfrastructures (report).Washington, DC. 1997.

Royce, Walker. Software Project Management: AUnified Framework. Reading, Mass.:Addison-Wesley. 1998.

Royce, Winston. “Managing the Development of LargeSoftware Systems.” Proceedings WESTCON,California. 1970.

Ruocco, LTC Anthony, USA, and MAJ Todd L. Smith,USA. “Designing a Digital Library to SupportGlobal Army Training.” Crosstalk: The Journal ofDefense Software Engineering. October 1998. 12.

Schaeffer, Mark. “Software Engineering and SystemsEngineering in the Department of Defense.”Crosstalk: The Journal of Defense SoftwareEngineering, 11:10. October 1998. 3–6.

Shelton, GEN Henry H., USA. “A Word from the NewChairman.” Joint Forces Quarterly, 17. Autumn/Winter 1997–1998. 6-8.

Shelton, GEN Henry H., USA. “Remarks at theGeneral Graves B. Erskine Distinguished LectureSeries.” Quantico, VA: Marine Corps University.February 23, 1998.


Shenhar, A. “A New Systems Engineering Taxonomy.”Proceedings of the 4th International Symposium ofthe National Council on System Engineering, 2.1994. 261–276.

Sheth, Amit P. and James A. Larson. “FederatedDatabase Systems for Managing Distributed,Heterogeneous, and Autonomous Databases.” ACMComputing Surveys, 22: 3. 1990. 183–236.

Slabodkin, Gregory. “First-Blush Review of Force XXIFinds Army Training Inadequate.” GovernmentComputer News. 1997.

Stanley, Elizabeth A. Evolutionary Technology in theCurrent Revolution in Military Affairs: ArmyTactical Command and Control System. StrategicStudies Institute, U.S. Army War College. 1998.

Starr, Stuart H. “Perspectives on C4I Interoperability.”Proceedings of the Naval War College, Commandand Control Research and Technology Symposium.Monterey, CA: Naval Post Graduate School. 1996.331–345.

Stutzke, Richard D. “Software Estimating Technology.”At web site www.saic.com/publications/.

Sullivan, GEN Gordon R., USA. Memorandum, March9, 1992. Re: Louisiana Maneuvers 1994. In Sullivanand Coroalles. Seeing the Elephant: LeadingAmerica’s Army into the Twenty-first Century.Cambridge, MA: Institute for Foreign PolicyAnalysis. 1995. 55–56.

E-11Appendix E

Sullivan, GEN Gordon R., USA, and COL AnthonyM. Coroalles, USA. The Army in the InformationAge. Strategic Studies Institute, U.S. Army WarCollege. 1995.

Sullivan, GEN Gordon R., USA, and COL James M.Dubik, USA. War in the Information Age. StrategicStudies Institute, U.S. Army War College. 1994.

Task Force XXI Architecture. At web sitewww.monmouth.army.mil/cecom/lrc/forcexxi/equipa.html.

Task Force XXI AWE Integrated Report AnnotatedBriefing Slides. U.S. Army Training and DoctrineCommand (TRADOC). 1997.

Task Force XXI outbrief to the U.S. Army Trainingand Doctrine Command (TRADOC) staff. At website www.monroe.army.mil/pao/tradoc/sld001-sld021.html.

Task Force XXI System Integration Plan (TSIP). 1996.

The Standish Group, CHAOS survey. At web sitewww.standishgroup.com. 1995.

The White House, A National Security Strategy for aNew Century. Washington, DC. 1998.

United States Imagery and Geospatial InformationSystem (USIGS) Glossary, Revision A. NationalImagery and Mapping Agency (NIMA). 1998.


Wentz, Larry, Ed. Lessons From Bosnia: The IFORExperience. Washington, DC: National DefenseUniversity Press. 1998.

Wright, Randall R. “Process Standards and CapabilityModels for Engineering Software-IntensiveSystems.” Crosstalk, The Journal of DefenseSoftware Engineering, 11:10. October 1998. 7–11.

about the ccrp · about the ccrp the c4isr cooperative research program (ccrp) has the mission of...

Documents