1cc4-ok' i b s: se-cy-o · attached is a pdf of our product gao-06-831, "enterprise...

EDO Principal Correspondence Control

FROM: DUE: 10/23/06

Randolph C. HiteU.S. Governmental Accountability Office (GAO)

EDO CONTROL: G20060773DOC DT: 09/12/06

FINAL REPLY:

P1ATS I- S eULq-x0(X -oM54

Agency Heads

FOR SIGNATURE OF: ** PRI ** CRC NO: 06-0450

Chairman Klein

DESC: ROUTING:

GAO Report "Enterprise Architecture: LeadershipRemains Key to Establishing and LeveragingArchitectures for organizational Transformation"(GAO-0.6-831) (Due to GAO: 11/10/06)

DATE: 09/12/06

ReyesVirgilioKaneSilberJohns onCyr/BurnsMalloy, QEDOGAO File

ASSIGNED TO:

QU is

CONTACT:

Baker

SPECIAL .INSTRUCTIONS OR REMARKS:

1Cc4-ok' 2- e I b S: SE-Cy-o /

EDATS Number: SECY-2006-0154 Initiating Office: SECY

Geea I~fraio

Assigned To: OIS QEDO Due Date: 10/23/2006 5:00 PM

Other Assignees: SECY Due Date: 10/27/2006 5:00 PM

Subject: GAO Report, "Enterprise Architecture: Leadership Remains Key to Establishing and Leveraging Architecturesfor Organizational Transformation"Description:

ADAMS Accession Numbers

Incoming:

Response:

Package:

Doumn Infrato

Originating Organization: GAO Originator Name: Randolph C. HiteIncoming Task Received: E-mail Date of Incoming Document: 9/12/2006

Document Received by OEDO Date: 9/12/2006

Date Response Requested by Originator: 11/10/2006

,Addressee: Agency Heads

PoesI fraio

Action Type: Letter Priority: Medium

Sensitivity: None

Signature Level: Chairman Diaz Urgency: NOQEDO Concurrence: YES

0CM Concurrence: YES

Special Instructions:

Ote IfrmtoCross Reference Number: GAO-06-83 I ,LTR-06-0450,G20060773

Related Task:

File Routing: EDATS Agency Lesson Learned: NO

Page 1 of I

OFFICE OF TIlIE SECRETARY

CORRESPONDENCE CONTROL TICKET

Date Printed: Sep 12, 2006 11:03

PAPER NUMBER:

ACTION OFFICE:

LTR-06-0450

EDO

LOGGING DATE: 09/12/2006

AUTHOR:

AFFILIATION:

ADDRESSEE:

Randolph Hite

GAO

Federal Agencies

SUBJECT: Release of GAO-06-0831 "Enterprise Architecture: Leadership ERemains Key to Establishingand Leveraging Architectures for Organizational etc;

ACTION:

DISTRIBUTION:

LETTER DATE:

ACKNOWLEDGED

SPECIAL HANDLING:

Signature of Chairman

RF..Encls to: EDO

09/A-%006

No

Comr. Correspondence

NOTES:

FILE LOCATION: ADAMS

10/27/2006DATE DUE: DATE SIGNED:

EDO - -G200607 73

lelinda Malloy- Release of GAO-06-831, ""Enterprise Architecture: Leadership Remains Key to Establishing... PgPage 11

From: "GAO Reports" <GAO Reports@ gao.gov>To: <gao-oigliaison @dhs.gov>, <steven.pecinovsky@ dhs.gov>, <brian.lee@ do.treas.gov>,<murphy.iris @dol.gov>, <martin.gertel @dot.gov>, <tiber.steve @epa.gov>, <ralph.boldt @gsa.gov>,<debra.perrry@ hhs.gov>, <dianne.l.williams @hq.doe.gov>, <kathryn...A._Nicholson @hud.gov>,-<[email protected]>, <deborah..gaojliaison [email protected]>, <[email protected]>,<evelyn.r.klemstine @nasa.gov>, <john.d.werner@ nasa.gov>, <lmiller@ nasa.gov>,<paul.roberts ©nasa.gov>, <william.s.clement @nasa.gov>, <mxm @nrc.gov>, <dcureton @nsf.gov>,<IGRes @oig.hhs.gov>, <rlowe @opm.gov>, <[email protected]>, <candace.skurnik@ ssa.gov>,<shinnickj @state.gov>, <thompsonznb ©state.gov>, <bemcdonald ©usaid.gov>,<andrea.g.nicholson @usdoj.gov>, <suzanne.r.johnson @usdoj.gov>, <[email protected]>Date: 0911212006 9:17:30 AMSubject: Release of GAO-06-831, " "Enterprise Architecture: Leadership Remains Key to Establishing...

Attached is a PDF of our product GAO-06-831, "Enterprise Architecture: Leadership Remains Key toEstablishing and Leveraging Architectures for Organizational Transformation." This product is beingisslijed in electronic rather than printed form, as part of GAO's pilot project on electronic dissemination.

This report contains recommendations to you. As you know, 31 U.S.C. 720 requires the head of a federalagency to submit a written statement of the actions taken on our recommendations to the SenateCommittee on Homeland Security and Governmental Affairs and to the House Committee on GovernmentReform not later than 60 calehdar days from the date of the report and to the House and Senate'Committees on Appropriations with the agency's first request for appropriations made more than 60calendar days after that date. Since the congressional requester has asked that the distribution of thereport be restricted, as provided by GAO's Congressional Protocols, the 60-day period will begin on thedate the report is released and e-mailed to you. Because agency personnel serve as the primary source ofinformation on the status of recommendations, GAO requests that you also provide GAO with a copy ofyour agency's statement of action to serve as preliminary information on the status of openrecommendations. Please send your statement of action to me or Mark Bird at [email protected] appreciate the assistance and cooperation of your staff during our review.

Sincerely,

Randolph C. HiteDirector, Information Technology Architectureand Systems Issues

CC: CC:"Mark T Bird" <BirdM @GAO.GOV>

I c:\temp\GWIOOOO3.TMP Page 1

c:\temP\GWJ00003.TMP Page 1"I

Mail Envelope Properties (4506B355.28D:4:33421)

Subject: Release of GAO-06-83 1, " "Enterprise Architecture: Leadership.Remains Key to Esta blishing...Creation Date 09/12/2006 9:16:02 AMFrom: "GAO Reports" <[email protected]>

Created BY: [email protected]

Recipientsnrc.gov

TWGWPOOI1.HQG WDOOI1MXM (Melinda Malloy)

GAO.GOVBirdM CC (Mark T Bird)

va.govj ames.vanzandt

usdoj .g ovsuzanne.r.johnsonan dre a g.n ich ols on

us aid go vbemcdonald

state.govthompsonznbshinnickj

ssa.govcandace.skurnik

sba.govtiffani.cooper

*opm.govriowe

oig.hhs.govIGRes

nsf.gov

c:\temp\GWJObOO3.TM P Page 2

dcureton

nasa.govwilliam.s.clementpaul.roberts]millerjohn.d.wernerevelyn.r.klemstinedbloxon

ios.doi.govdeborah~gao-liaison-williams

hud.govla-y-e.-mcgheekathryn-A..Nicholson

hq.doe.govdianne.I.williams

hhs.govdebra.perrry

gsa.govralph.boldt

epa.govtiber.steve

dot.govmartin .gertel

dol.govmurphy.iris

do.treas.govbrian.lee

dhs.govsteven.pecinovskygao-oigliaison

Post Office RouteTWGWPOOI1.HQGWDOO 1 nrc.gov

GAO.GOV

I c:\temp\GW)00003.TMP Page 3 1I c:\temp\GW}00003.TMP

Page 3 I

va.govusdoj .govusaid.govstate.govssa.govsba.govopm. go voig.hhs.govnsf.govnasa.govios.doi.govhud.govhq.doe.govhhs.govgsa.govepa.govdot.govdol .govdo.treas.govdhs.gov

Files Size Date & TimeMESSAGE 1569 09/12/2006 9:16:02 AMd06831l.pdf 4381839Mime.822 6000601

OptionsExpiration Date: NonePriority: StandardRep lyRequested: NoReturn Notification: None

Concealed Subject: NoSecurity: Standard

Junk Mail Handling Evaluation ResultsMessage is eligible for Junk Mail handlingThis message was not classified as Junk Mail

Junk Mail settings when this message was deliveredJunk Mail handling disabled by UserJunk Mail handling disabled by AdministratorJunk List is not enabled.Junk Mail using personal address books is not enabledBlock List is not enabled.

GAOUnited States Government Accountability Office

Report to the Chairman, Committee onGovernment Reform, House ofRepresentatives

August 2006 ENTERPRISEARCHITE CTURE

Leadership RemainsKey to Establishingand LeveragingArchitectures forOrganizationalTransformation

G GAO0*1Accountability * Integrity * Reliability

GAO-06-83 1

GAOSAccountabiltvy Integrity- Reiab~i ty

HighlightsHighlights of GAO-06-831, a report to theChairman, Committee on GovernmentReform, House of Representatives

Ag st 00

ENTERPRISE ARCHITECTURE

Leadership Remains Key to Establishingand Leveraging Architectures forOrganizational Transformation

Why GAO Did This StudyA well-defined enterprisearchitecture is an essential tool forleveraging information technology(MT to transformn business andmission operations. GAO'sexperience has shown thatattempting to modernize andevolve IT environments without anarchitecture to guide and constraininvestments results in operationsand systems that are duplicative,not well integrated, costly tomaintain, and ineffective insupporting mission goals. In light ofthe importance of enterprisearchitectures, GAO developed a,five stage architecture managementmaturity framework that defineswhat needs to be done toeffectively manage an architectureprogram. Under GAO's framework,a fully mature architecture program*is one that satisfies all elements ofall stages of the framework. Asagreed, GAO's objective was todetermine the status of majorfederal department and agencyenterprise archiftecture efforts.

Wha GAO Recomend

To increase the maturity of federalenterprise architecture efforts,GAO is making a recommendationto each department and agencyaimed at improving their respectivearchitecture programs. Incommenting on a draft of thisreport, all but one department fullyagreed with its recommendationand all but six departments andagencies fully agreed with ourfindings about its program. Twodepartments did not comment.

www.gao.gov/cgi-bin/getrpt?GAO-06-831.

To view the full product, including the scopeand methodology, click on the link above.For more information, contact Randolph C.Hite at (202) 512-3439 or [email protected].

What GAO FoundThe state of the enterprise architecture programs at the 27 major federaldepartments and agencies is mixed, with several having very immatureprograms, several having more mature programs, and most beingsomewhere in between. Collectively, the majority of these architectureefforts can be viewed as a work-in-progress with much remaining to beaccomplished before the federal government as a whole fully realizes theirtransformational value. More specifically, seven architecture programs haveadvanced beyond the initial stage of the GAO framework, meaning that theyhave fully satisfied all core elements associated with the framework'ssecond stage (establishing the management foundation for developing,using, and maintaining the architecture). Of these seven, three have alsofully satisfied all the core elements associated 'with the third stage(developing the architecture). None have fully satisfied all of the coreelements associated with the fourth (completing the architecture) and fifth(leveraging the architecture for organizational change) stages. Nevertheless,most have fully satisfied a number of the core elements across the stageshigher than the stage in which they have met all core elements, with all 27collectively satisfying about 80, 78, 61, and 52 percent of the stage twothrough five core elements, respectively (see figure). Further, most havepartially satisfied additional elements across all the stages, and seven needto fully satisfy five or fewer elements to achieve the fifth stage.

The key to these departments and agencies building upon their currentstatus, and ultimately realizing the benefits that they cited archidtecturesproviding, is sustained executive leadership, as virtually all the challengesthat they reported can be addressed by such leadership. Examples of thechallenges are organizational parochialism and cultural resistance, adequateresources (human capital and funding), and top management understanding;examples of benefits cited are better information sharing, consolidation,improved productivity, and reduced costs.

Percentage of Framework Elements Collectively Satisfied by All Departments and AgenciesIn Each StageStage 5 _____________ __

Stage 4

Stage 3

Stage 2

I

0 10 20 30 40 50 60 70 80Percentage

Source: GAO analysis of departmentlagency data.

Note: There are no framework elements in stage 1.

.United States Government Accountability Off ice

Contents

Letter IResults in Brief 2Background 4Overall State of Enterprise Architecture. Management Is a

Work-in-Progress, Although a Few Agencies Have LargelySatisfied Our Framework 18

Conclusions 37Recommendations for Executive Action 37Agency Comments and Our Evaluation 38

Ap1pendixesAppendix 1:

Appendix II:

Appendix III:

Appendix IV-

Appendix V-.

Appendix VI:

Appendix VII:

Appendix VIII:

Appendix IX:

Appendix X:

Appendix XI:

Appendix XII:

Appendix XIII:

Appendix XIV:

Reported Enterprise Architecture Costs Vary, withContractors and Personnel Accounting for Most Costs

Departments and Agencies Reported Experiences with TheirArchitecture Tools and Frameworks

Objective, Scope, and Methodology

Detailed Assessments of Individual Departments andAgencies against Our EA Management MaturityFramework

Comments from the Department of Commerce

Comments from the Department of DefenseGAO Comments

Comments from the Department of Education

Comments from the Department of Energy

Comments from the Department of Homeland SecurityGAO Comments

Comments from the Department of Housing and UrbanDevelopment

Comments from the Department of the Interior

Comments from the Department of JusticeGAO Comments

Comments from the Department of StateGAO Comments

Comments from the Department of the Treasury

45

51

56

64

135

136138139

141

142144

146

150

151153

154160161

Page 1 Page GAO-06-831 Enterprise Architecture

Contents

Appendix XV:

Appendix XVI:

Appendix XVII:

Appendix XVIII:

Appendix XIX:

Appendix XX:

Appendix XXI:

Comments from the Department Veterans Affairs

Comments from the Environmental Protection AgencyGAO Comments

Comments from the General Services Administration

Comments from the National Aeronautics and SpaceAdministration

Comments from the Social Security AdministrationGAO Comments

Comments from the U.S. Agency for InternationalDevelopmentI

GAO Contact and Staff Acknowledgements

163

164166

167

168

169172

173

174

Tables Table 1: Federal Enterprise Architecture Reference ModelsTable 2: 0MB Enterprise Architecture Assessment Framework

Capability AreasTable 3: Summary of EAMMF Version 1.1: Core Elements

Categorized by GroupTable 4: Maturity Stage of Major Department and Agency

Enterprise Architecture ProgramsTable 5: Percent of Framework Elements Satisfied by Department

arnd Agency Architecture Programs within Each MaturityStage

Table 6: Departments and Agencies That Need to Satisfy 5 or FewerCore Elements to Achieve Stage 5

Table 7: Degree to Which Departments and Agencies AreExperiencing Enterprise Architecture Challenges

Table 8: Enterprise Architecture Benefits Reported As Being or ToBe Achieved to a Significant Extent

Table 9: Department and Agency Reported Satisfaction withTools

Table 10: Department and Agency FrYamework Satisfaction LevelsTable 11: List of Architecture Programs Included in this ReportTable 12: Stage 2 Evaluation CriteriaTable 13:,Stage 3 Evaluation CriteriaTable 14: Stage 4 Evaluation CriteriaTable 15: Stage 5 Evaluation CriteriaTable 16: Department of Agriculture Satisfaction of EAMMFTable 17: Department of the Air Force Satisfaction of EAMMFTable 18: Department of the Army Satisfaction of EAMMF

7

17

19

21

33

34

36

53555658596061646770

Page ii Page iiGAO-06-831 Enterprise Architecture

Contents

Table 19: Department of Commerce Satisfaction of EAMMF 73Table 20: DOD Business Enterprise Architecture Satisfaction of

EAMMF 75Table 21: DOD Global Information Grid Satisfaction of EAMMF 78Table 22: Department of Education Satisfaction of EAMMF 81Table 23: Department of Energy Satisfaction of EAMMF 83Table 24: Department of Health and Human Services Satisfaction of

EAMMF 85Table 25: Department of Homeland Security Satisfaction of

EAMMF 87Table 26: Department of Housing and Urban Development

Satisfaction of EAMMF 89Table 27: Department of the Interior Satisfaction of EAMMF 91Table 28: Department of Justice Satisfaction of EAMMF 93Table 29: Department of Labor Satisfaction of EAMMF 95Table 30: Department of the Navy Satisfaction of EAMMF 97Table 31: Joint Enterprise Architecture Satisfaction of EAMMF 100Table 32: Department of Transportation Satisfaction of EAMMF 103Table 33: Department of the Treasury Satisfaction of EAMMF 105'Table 34: Department of Veterans Affairs Satisfaction of

EAMMF 107Table 35: Environmental Protection Agency Satisfaction of

EAMMF 110Table 36: General Services Administration Satisfaction of

EAMMF 113Table 37: National Aeronautics and Space Administration

Satisfaction of EAMMF 116Table 38: National Science Foundation Satisfaction of EAM~MF 119Table 39: Nuclear Regulatory Commission Satisfaction of'

EAMMF 122Table 40: Office of Personnel Management Satisfaction of

EAMMF 125Table 4 1: Small Business Administration Satisfaction of EAMMF 127Table 42: Social Security Administration Satisfaction of EAMMF 130Table 43: U. S. Agency for International Development Satisfaction of

EAMMF 132

Figures Figure 1: Summary of EAMMF Version 1. : Maturity Stages, Critical

Figure 2: Overall Satisfaction of Core Elements Associated withArchitecture Governance 25

Page iii Page iiiGAO-06-831 Enterprise Architecture

Contents

Figure 3: Overall Satisfaction of Gore Elements Associated withArchitecture Content 27

Fidure 4: Overall Satisfaction of Gore Elements Associated withArchitecture Use 28

Figure 5: Overall Satisfaction of Core Elements Associated withArchitecture Measurement 29

Figure 6: Department/Agency Maturity Stage Based on FullyVersus Partially Satisfied Criterion 31

Figure 7: Reported Development Costs to Date for Departmentsand Agencies 46

Figure 8: Reported Estimated Completion Costs for Departmentsand Agencies 47

Figure 9: Reported Estimated Annual Maintenance Costs forDepartments and Agencies 48

Figure 10: Breakdown of Enterprise Architecture DevelopmentCosts for all Departments and Agencies 49

Figure 11: Reported Enterprise Architecture Contractor Costs byCategory 50

F igure 12: Enterprise Architecture Tools Used by Departments andAgencies 52

Figure 13: Frameworks Used by Departments and Agencies 54

Page iv Page ivGAO-06-831 Enterprise Architecture

Contents

Abbreviations

BEA Business Enterprise Architecture0I0 Chief Information OfficerDHS Department of Homeland SecurityDOD Department of DefenseDODAF Department of Defense Architecture FrameworkDOJ Department of JusticeEA Enterprise ArchitectureEAMMF Enterprise Architecture Management Maturity FramewbrkEAMS Enterprise Architecture Management SystemEPA Environmental Protection AgencyFAA Federal Aviation AdministrationFBI Federal Bureau of InvestigationFEAF Federal Enterprise Architecture FrameworkFEAPMO Federal Enterprise Architecture Program Management OfficeGIG Global Information GridGSA General Services AdministrationHHS Department of Health and Human ServicesHUD Department of Housing and Urban DevelopmentIT Information TechnologyIV&-V Independent Verification and ValidationNASA National Aeronautics and Space AdministrationNIST National Institute of Standards and TechnologyNRC Nuclear Regulatory CommissionNSF National Science Foundation0MB Office of Management and BudgetOPM Office of Personnel ManagementSBA Small Business AdministrationSSA Social Security AdministrationTOGAF The Open Group Architecture FrameworkUSAID United States Agency for International DevelopmentUSDA United States Department of AgricultureVA Department of Veterans Affairs

This is a work of the U.S. government and is not subject to copyright protection in theUnited States. it may be reproduced and distributed in its entirety without furtherpermission from GAO. However, because this work may contain copyrighted images orother material, permission from the copyright holder may be necessary if you wish toreproduce this material separately.

Page v Page vGAO-06-831 Enterprise Architecture

I&GAO

An. ccountablity - Integrity - Reliability

United States Government Accountability OfficeWashington, D.C. 20548

August 14, 2006

The Honorable Tom DavisChairmanCommittee on Government ReformHouse of Representatives

Dear Mr. Chairman:

A wvell-defined enterprise architecture' 'is an essential tool for leveraginginformation technology (IT) in the transformation of business and missionoperations. Our experience with federal departments and agencies hasshown that attempting to modernize and evolve IT environments withoutan enterprise architecture to guide and constrain investments often resultsin operations and systems that are duplicative, not wvell integrated,unnecessarily costly to maintain and interface, and ineffective insupporting mission goals. Moreover, the development, implementation, andmaintenance of architectures are widely recognized as hallmarks ofsuccessful public and private organizations, and their use is required by theClinger-Cohen Act and the Office of Management and Budget (0MB). Inlight of the importance of these architectures, you requested that wedetermine, the current status of major federal department and agencyenterprise architecture efforts.

To accomplish our objective, we surveyed 27 major federal departmentsand agencies using a questionnaire that was based on our maturityframework for assessing and improving enterprise architecturemanagement,' and we collected and reviewed documentation to verifyagency responses. We then analyzed the results to determine the extent towhich each of the 27 satisfied our maturity framework,' and the challengesand benefits that each department and agency sees. We also collectedinformation about, for example, department and agency architecture costsand architecture framework and tool use and satisfaction, which is

'An enterprise architecture is a blueprint for organizational change defined in models thatdescribe (in both business and technology terms) how the entity operates today and how itintends to operate in the future; it also includes a plan for transitioning to this future state.

'GAO, Information Technology: A Framewvork-for Assessing and Improving EnterpriseArchitecture Managemnent (Version 1.1), GAO-03-584G (Washington, D.C.: April 2003).

'Our analysis reflects the state of department and agency architecture efforts as of Mlarch2006.

Page I Page 1GAO-06-831 Enterprise Architecture

summarized in appendixes I and Il. Because our framework defines whatneeds to be done to effectively manage an enterprise architecture program,and not the details surrounding how it needs to be done, the scope of ourreview did not include assessing the quality of enterprise architectureproducts and activities and associated management structures andprocesses that make up our framework. As such, scoring high on ourmaturity scale should be viewed as an indicator of, and not a guaranteethat, a department or agency necessarily has a well-defined architectureand that it is being effectively implemented. We conducted our work inaccordance with generally accepted government auditing standards.Details of our objective, scope, and methodology are in appendix III.

Results in Brief The state of the enterprise architecture programs at the 27 major federaldepartments and agencies is varied, with several having very immatureprograms, several having more mature programs, and most beingsomewhere in between. Collectively, this mqans that the bulk of the federalgovernment's enterprise architecture efforts can be viewed as a work inprocess with much to be accomplished before their transformation value isfully realized. To effectively establish and leverage enterprise architecturesas instruments of organizational transformation, research by us and othersshow that architecture programs should be founded upon both aninstitutional commitment to the architecture and a measured and verifiedorganizational capability to properly develop and use it to affectoperational and technological change. Our five stage architectureframework for managing and evaluating the status of architecture effortsconsists of 31 core elements related to architecture governance, content,use, and measurement that reflect these basic attributes.4 Of the 27departments and agencies, 7 have advanced beyond the initial stage of ourframework, meaning that they have fully satisfied all the core elementsassociated with the framework's second stage (establishing themanagement foundation for developing, using, and maintaining thearchitecture). Of these seven, four have also fully satisfied all the coreelements associated with the third stage (developing the architecture).None have fully satisfied all of the core elements associated with the fourth(completing the architecture) and fifth (leveraging the architecture fororganizational change) stages. Nevertheless, most of the departments andagencies have fully satisfied a number of the core elements across stages

4 GAO-03-584G.

Page 2 Page 2GAO-06-831 Enterprise Architecture

higher tha~n that at which they have met all core elements. When this isconsidered, the profile shows that about 77 percent of the programsreviewed have fully satisfied the architecture governance core elements, 68percent have fully satisfied the architecture content core elements, 52percent have fully satisfied the architecture use core elements, and 47 havefully satisfied the architecture measurement core elements. Moreover,most of the departments and agencies have also partially satisfiedadditional core elements across all the stages. Seventeen of thedepartments and agencies have at least partially satisfied the core elementsassociated with achieving the framework's third stage, with four havingpartially satisfied the elements associated with achieving higher stages.

As we have previously reported, the key to these departments and agenciesbuilding upon their current status, and ultimately realizing the manybenefits that they cited architectures providing, will be sustained executiveleadership, as virtually all the barriers that the agencies reported can beaddressed through such leadership. Examples of these barriers orchallenges are overcoming organizational parochialism and culturalresistance, having adequate resources (human capital and funding), andfostering top management understanding. Examples of the benefits includebetter information sharing, consolidation, improved productivity, andreduced costs. To assist the departments and agencies in addressing theirarchitectural barriers, managing their architecture programs, and realizingtheir architecture benefits, we are making recommendations to heads ofmajor departments and agencies for developing and implementing plansaimed at satisfying all of the conditions in our architecture managementmaturity framework.

We received written or oral comments on a draft of this report from 25 ofthe departments and agencies in our review.' Of the 25, 24 fully agreed withour recommendation and one department partially agreed. Nineteendepartments and agencies agreed with our findings and six partially agreed.Of the six that disagreed with certain aspects of our findings, thedisagreements largely centered around (1) the adequacy of thedocumentation that they provided to demonstrate satisfaction of a specificcore element and (2) recognition of steps that they reported taking after weconcluded our review. For the most part, these isolated areas of

'The Department of Defense submitted a single letter that included comments from theDepartments of the Air Force, Army, and Navy. Representatives of the Departments ofHealth and Human Services and Transportation stated that they did not have comments.

Page 3 Page 3GAO-O6-831 Enterprise Architecture

disagreement did not result in any changes to our findings for two primaryreasons. First, our findings across the departments and agencies werebased on consistently applied evaluation criteria governing the adequacy ofdocumentation, and were not adjusted to accommodate any one particulardepartment or agency. Second, our findings represent the state of eacharchitecture program as of March 2006, and thus to be consistent do notreflect activities that may have occurred after this time. Beyond thesecomments, several departments and agencies offered suggestions forimproving our framework, which we will consider in issuing the nextversion of the framework, and several provided technical comments, whichwe have incorporated, as appropriate, in this report.

Background An enterprise architecture is a blueprint that describes the current anddesired state of an organization or functional area in both logical andtechnical terms, as well as a plan for transitioning between the two states.Enterprise architectures are a recognized tenet of organizationaltransformation and IT management in public and private organizations.Without an enterprise architecture, it is unlikely that an organization will beable to transform business processes and modernize supporting systems tominimize overlap and maximize interoperability. The concept of enterprisearchitectures originated in the mid-1980s; various frameworks for definingthe content of these architectures have been published by governmentagencies and OMB. Moreover, legislation and federal guidance requiresagencies to develop and use architectures. For more than a decade, wehave conducted work to improve agency architecture efforts. To this end,we developed an enterprise architecture management maturity frameworkthat provides federal agencies with a common benchmarking tool forassessing the management of their enterprise architecture efforts anddeveloping improvement plans.

Enterprise ArchitectureDescription and Importance

An enterprise can be viewed as either a single organization or a functionalarea that transcends more than one organization (e.g., financialmanagement, homeland security). An architecture can be viewed as thestructure (or structural description) of any activity. Thus, enterprisearchitectures are basically systematically derived and captureddescriptions-in useful models, diagrams, and narrative.

More specifically, an architecture describes the enterprise in logical terms(such as inter-related business processes and business rules, information


needs and flows, and work locations and users) as wvell as in technicalterms (such as hardware, software, data, communications, and securityattributes and performance standards). It provides these perspectives bothfor the enterprise's current or "as-is" environment and for its target or "to-be" environment, as well as a transition plan for moving from the "as-is" tothe "to-be" environment.

The importance of enterprise architectures is a basic tenet of bothorganizational transformation and IT management, and their effective useis a recognized hallmark of successful public and private organizations. Forover a decade, we have promoted the use of architectures, recognizingthem as a crucial means to a challenging end: optimized agency operationsand performance. The alternative, as our work has shown, is theperpetuation of the kinds of operational environments that burden mostagencies today, where a lack of integration among business operations andthe IT resources supporting them leads to systems that are duplicative,poorly integrated, and unnecessarily costly to maintain and interface.'Employed in concert with other important IT management controls (suchas portfolio-based capital planning and investment control practices),architectures can greatly increase the chances that the organizations'operational and IT environments will be configured so as to optimizemission performance.

Brief History of During the mid-1980s, John Zachman, widely recognized as a leader in the

Architecture Frameworks field of enterprise architecture, identified the need to use a logicaland Management Guidance construction blueprint (i.e., an architecture) for defining and controlling

the integration of systems and their components.' Accordingly, Zachman

'See, for example, GAO, Homeland Security: Efforts Under Way to Develop EnterpriseArchitecture, but Much Work Remains, GAO-04-777 (Washington, D.C.: Aug. 6, 2004); DODBusiness Systems Modernization: Limited Pr-ogress in Development of BusinessEnterprise Architecture and Oversight of Information Technology Investments, GAO-04-731R (WVashington, D.C.: MIay 17, 2004); Information Technology: Architecture Needed toGuide NASA's Financial Management Modernization, GAO-0-143 (Washington, D.C.: Nov.21, 2003); DOD Business Systems Modernization: Important Progress Made to DevelopBusiness Enterprise Architecture, but Much Work Remains, GAO-03-1018 (Washington,D.C.: Sept. 19, 2003); and Informnation Technology:- DLA Should Strengthen BusinessSystems Modernization Architecture and Investment Activities, GAO-01-631 (Washington,D.C.: June 29, 2001).

7J. A. Zachman, "A Framework for Information Systems Architecture," IBM SystemsJournal vol. 26, no. 3 (1987).


developed a structure or framework for defining and capturing anarchitecture, which provides for six perspectives or "windows" from whichto view the enterprise.' Zachinan also proposed six abstractions or modelsassociated with each of these perspectives.' Zachman's frameworkprovides a way to identify and describe an entity's existing and plannedcomponent parts and the parts' relationships before the entity begins thecostly and time-consuming efforts associated with developing ortransforming itself.

Since Zachman introduced his framework, a number of frameworks haveemerged within the federal government, beginning with the publication ofthe National Institute of Standards and Technology (NIST) framework in1989. Since that time, other fed~ral entities have issued frameworks,including the Department of Defense (DOD) and the Department of the'freasury. In September 1999, the federal Chief Information Officers (CIO)Council published the Federal Enterprise Architecture Framework(FEAF), which was intended to provide federal agencies with a commonconstruct for their architectures, thereby facilitating the coordination ofcommon business processes, technology insertion, information flows, andsystem investments among federal agencies. The FEAF described anapproach, including models and definitions, for developing anddocumenting architecture descriptions for multi-organizational functionalsegments of the federal government.1 0

More recently, 0MB established the Federal Enterprise ArchitectureProgram Management Office (FEAPMO) to develop a federal enterprisearchitecture according to a collection of five reference models (see table1). These models are intended to facilitate governmentwide improvementthrough cross-agency analysis and the identification of duplicativeinvestments, gaps, and opportunities for collaboration, interoperability,and integration within and across goverunment agencies.

sThe windows include (1) the strategic planner, (2) the system user, (3) the system designer,(4) the system developer, (5) the subcontractor, and (6) the system itself.

9The models cover (1) how the entity operates, (2) what the entity uses to operate,(3) where the entity operates, (4) who operates the entity, (5) when entity operations occur,and (6) why the entity operates.

'0Similar to the Zachman framework, FEAF's proposed models describe an entity's business,data necessary to conduct the business, applications to manage the data, and technology tosupport the applications.


Table 1: Federal Enterprise Architecture Reference Models

Reference model DescriptionPerformance Provides a common set of general performance outputs andReference Model measures for agencies to use to achieve business goals and

objectives.Business Reference Describes the business operations of the federal governmentModel independent of the agencies that perform them, including defining

the services provided to state and local governments.Service Component Identifies and classifies IT service (i.e., application) componentsReference Model that support federal agencies and promotes the reuse of.

components across agencies.Data and Information Describes, at an aggregate level, the types of data and informationReference Model, that support program and business line operations, and the

relationships among these types.Technical Reference Describes how technology is supporting the delivery of serviceModel components, including relevant standards for implementing the

technology.Source: GAO.

0MB has identified multiple purposes for the Federal EnterpriseArchitecture, such as the following:

" informing agency enterprise architectures and facilitating theirdevelopment by providing a common classification structure andvocabulary;

" providing a governmentwide framework that can increase agencyawareness of IT capabilities that other agencies have or plan to acquire,so that they can explore opportunities for reuse;

" helping 0MB decision makers identify opportunities for collaborationamong agencies through the implementation of common, reusable, andinteroperable solutions; and

" providing the Congress with information that it can use as it considersthe authorization and appropriation of funding for federal programs.

Although these post-Zachman frameworks differ in their nomenclaturesand modeling approaches, each consistently provides for defining anenterprise's operations in both logical and technical terms, provides fordefining these perspectives for the enterprise's current and targetenvironments, and calls for a transition plan between the two.


Several laws and regulations address enterprise architecture. For example,the Clinger-Cohen Act of 1996 directs the CIOs of major departments andagencies to develop, maintain, and facilitate the implementation ofinfor-mation technology architectures as a means of integrating agencygoals and business processes with information technology." Also, 0MBCircular A-130, which implements the Clinger-Cohen Act, requires thatagencies document and submit their initial enterprise architectures to 0MBand that agencies submit updates when significant changes to theirenterprise architectures occur. The circular also directs 0MB to use variousreviews to evaluate the adequacy and efficiency of each agency'scompliance with the circular.

A Decade of GAO Work HasFocused on ImprovingAgency EnterpriseArchitecture Efforts

We began reviewing federal agencies' use of enterprise architectures in1994, initially focusing on those agencies that were pursuing major systemsmodernization programs that were high risk. These included the NationalWeather Service systems modernization,'12 the Federal AviationAdministration (FAA) air traffic control modernization,"3 and the InternalRevenue Service tax systems modernization.'" Generally, we reported thatthese agencies' enterprise architectures were incomplete, and we maderecommendations that they develop and implement complete enterprisearchitectures to guide their modernization efforts.

S ince then, we have reviewed enterprise architecture management at otherfederal agencies, including the Department of Education (Education),'" the

"40 U.S.C. sections 11101-11703.

"2GAO, Weather Forecasting: Systems Architecture Needed for National Weather ServiceModernization, GAO/AIMD-94-28 (Washington, D.C.: Mar. 11, 1994).

"GAO, Air Tr-affic Control: Coinpletc and Enforced Architecture Needed for FAA SystemnsAlodernizat ion, GAO/AIMD-97-30 (Washington, D.C.: Feb. 3, 1997).

" GAO, Tax Systemns Modernization: Blueprint Is a Good Start but Not Yet SufficientlyComnplete to Build or Acquire Systems, GAO/AJMD/GGD-98-54 (Washington, D.C.: Feb. 24,1998).

"5GAO, Student Financial Aid Infonnalion: Systems Architecture Needed to ImproveProgramns' Efficiency, GAO/AlMD-97-122 (Washington, D.C.: July 29, 1997).


Customs Service,"6 the Immigration and Naturalization Service,"7 theCenters for Medicare and Medicaid Services,'8 FAA,'` and the FederalBureau of Investigation (FBI)."0 We have also reviewed the use ofenterprise architectures for critical agency functional areas, such as theintegration and sharing of terrorist watch lists across key federaldepartments2 ' and DOD financial management,22 logistics management,23

combat identification,'24 and business systems modernization .25 Thesereviews continued to identify the absence of complete and enforcedenterprise architectures, which in turn has led to agency businessoperations, systems, and data that are duplicative, incompatible, and not

"6GAO, Customs Service Modernization: Architecture Must Be Complete and Enforced toEffectively Build and Maintain Systems, GAO/AIMD-98-70 (Washington, D.C.: Mlay 5,1998).

"7GAO, Information Technology: INS Needs to Better Manage the Development of ItsEnterprise Architecture, GAO/AIAID-00-212 (Washington, D.C.: Aug. 2000).

"8GAO, Medicare: Information Systems Modernization Needs St ronger Management andSupport, GAO-01-824 (Washington, D.C.: Sept. 20, 2001).

"GAO, Federal Aviation Administration: StrongerArchitecture Program Needed to GuideSystems Modernization Efforts, GAO-05-266 (Washington, D.C.: April 2005).

"GAO, Information Technology: FBI is Taking Steps to Develop an EnterpriseArchitecture, but Much Remains to Be Accomplished, GAO-05-363 (Washington, D.C.:Sept. 9, 2005).

2"GAO, Information Technology: Terrorist Watch Lists Should Be Consolidated to PromoteBetter Integration and Sharing, GAO-03-322 (Washington, D.C.: April 15, 2003).

'~GAO, Information Technology: Architecture Needed to Guide Modernization of DOD'sFinancial Operations, GAO-01-525 (Washington, D.C.: Mlay 17,2001).

2"GAO-01-631.

24 GAO, Combat Identification Systems: Strengthened Management Efforts Needed toEnsure Required Capabilities, GAQ-01-632 (Washington, D.C.: June 25, 2001).

25GAO, DOD Business Systems Modernization: Improvements to Enterprise ArchitectureDevelopment and Implementation Efforts Needed, GAO-03-458 (Washington, D.C.: Feb. 28,2003); Information Technology: Observations on Department of Defense's Draft EnterpriseArchitecture, GAO-03-571R? (Washington, D.C.: Mfar. 28, 2003); DOD Business SystemsModernization: Longstanding Management and Oversight Weaknesses Continue to PutInvestments at Risk, GAO-03-553T (Washington, D.C.: Mfar. 31, 2003); Business SystemsModernization: Summary of GA 0's Assessment of the Department of Defense's InitialBusiness Enterprise Architecture, GAO-03-877R (Washington, D.C.: July 7, 2003); DODBusiness Systems Modernization: Long-Standing Weaknesses in Enterprise ArchitectureDevelopment Need to Be Addressed, GAO-05-702 (Washington, D.C.: July 22, 2005).


integrated; these conditions have either prevented agencies from sharingdata or forced them to depend on expensive, custom-developed systeminterfaces to do so. Accordingly, we made recommendations to improve therespective architecture efforts. In some cases progress has been made,such as at DOD and FBI. As a practical matter, however, considerable timeis needed to completely address the kind of substantive issues that we haveraised and to make progress in establishing more mature architectureprograms.

In 2002 and 2003, we also published reports on the status of enterprisearchitectures governmentwide. The first report (February 2002)2" showedthat about 52 percent of federal agencies self-reported having at least themanagement foundation that is needed to successfully develop, implement,and maintain an enterprise architecture, and that about 48 percent ofagencies had not yet advanced to that basic stage of maturity. We attributedthis state of architecture management to four management challenges: (1)overcoming limited executive understanding, (2) inadequate funding, (3)insufficient number of sk~illed staff, and (4) organizational parochialism.Additionally, we recognized OMB's efforts to promote and overseeagencies' enterprise architecture efforts. Nevertheless, we determined thatOMB's leadership and oversight could be improved by, for example, using amore structured means of measuring agencies' progress and by addressingthe above management challenges.

The second report (November 2003)27 showed the percentage of agenciesthat had established at least a foundation for enterprise architecturemanagement was virtually unchanged. We attributed this to long-standingenterprise architecture challenges that had yet to be addressed. Inparticular, more agencies reported lack of agency executive under-standingof enterprise architecture and the scarcity of skilled architecture staff assignificant challenges. OMB generally agreed with our findings and theneed for additional agency assessments. Further, it stated that fullyimplementing our recommendations would require sustained managementattention, and that it had begun by working with the CIO Council toestablish the Chief Architect Forum and to increase the information 0MBreports on enterprise architecture to Congress.

"6GAO, Informat ion Technology: Enterprise Architecture Use across the FederalGovernment Can Be Improved, GAO-02-6 (Washington, D.C.: Feb. 19, 2002).

27 GAO, Information Technology: Leadership Remains Key to Agencies Making Progress onEnterprise Architecture Efforts, GAO-04-40 (Washington, D.C.: Nov. 17, 2003).


Since then, 0MB has developed and implemented an enterprisearchitecture assessment tool. According to 0MB, the tool helps betterunderstand the current state of an agency's architecture and assistsagencies in integrating architectures into their decision-making processes.The latest version of the assessment tool (2.0) was released in December2005 and includes three capability areas: (1) completion, (2) use, and (3)results. Table 2 describes each of these areas.

Table 2: 0MB Enterprise Architecture Assessment Framework Capability Areas

Capability area DescriptionCompletion Addresses ensuring that architecture products describe the agency

in terms of processes, services, data, technology, and performanceand that the agency has developed a transition strategy.

Use Addresses the establishment of important management practices,processes, and policies, such as configuration management,communications, and integration of the architecture with capitalplanning processes..

Results Addresses the effectiveness and value of the architecture byencouraging performance measurements and using it to ensureagency policies align to 0MB IT policy.

Source: 0MB.

The tool also includes criteria for scoring an agency's architecture programon a scale of 0 to 5."8 In early 2006, the major departments and agencieswere required by 0MB to self assess their architecture programs using thetool. 0MB then used the self assessment to develop its own assessment.These assessment results are to be used in determining the agency's e-Government score within the President's Management Agenda.

GAO's EnterpriseArchitecture ManagementMaturity Framework(EAMMF)

In 2002, we developed version 1.0 of our Enterprise ArchitectureManagement Maturity Framework (EAMMF) to provide federal agencieswith a common benchmarking tool for planning and measuring their effortsto improve enterprise architecture management, as well as to provide 0MBwith a means for doing the same governmentwide. We issued an update of

2Ascore of 0 means undefined, I means initial, 2 means managed, 3 means utilized, 4 meansresults-oriented, and 5 means optimized.


the framework (version 1. 1) in 2003.29 This framework is an extension of APractical Gjuide to Federal Enteiprise Architecture, Version 1. 0,published by the CI0 Council.' Version 1. 1 of the framework arranges 31core elements (practices or conditions that are needed for effectiveenterprise architecture management) into a matrix of five hierarchicalmaturity stages and four critical success attributes that apply to each stage.Within a given stage, each critical success attribute includes between oneand four core elements. Based on the implicit dependencies among thecore elements, the EAMMF associates each element with one of fivematurity stages (see fig. 1). The core elements can be further categorizedby four. groups: architecture governance, content, use, and measurement.

EAMMF Stages Stage 1: Creating EA awareness. At stage 1, either an organization doesnot have plans to develop and use an architecture, or it has plans that donot demonstrate an awareness of the value of having and using anarchitecture. While stage 1 agencies may have initiated some enterprisearchitecture activity, these agencies' efforts are ad hoc and unstructured,lack institutional leadership and direction, and do not provide themanagement foundation necessary for successful enterprise architecturedevelopment as defined in stage 2.

Stage 2: Building the EA management foundation. An organization atstage 2 recognizes that the enterprise architecture is a corporate asset byvesting accountability for it in an executive body that represents the entireenterprise. At this stage, an organization assigns enterprise architecturemanagement roles and responsibilities and establishes plans for developingenterprise architecture products and for measuring program progress andproduct quality; it also commits the resources necessary for developing anarchitecture-people, processes, and tools. Specifically, a stage 2organization has designated a chief architect and established and staffed aprogram office responsible for enterprise architecture development andmaintenance. Further, it has established a committee or group that hasresponsibility for enterprise architecture governance (i.e., directing,overseeing, and approving architecture development and maintenance).This committee or group membership has enterprisewide representation.

*At stage 2, the organization either has plans for developing or has started

19GAO-03-584G.

30CIO Council, A Practical Guide to Federal Enterprise Architecture, Version 1.0 (February2001).


developing at least some enterprise architecture products, and it hasdeveloped an enterprisewide awareness of the value of enterprisearchitecture and its intended use in managing its IT investments. Theorganization has also selected a framework and a methodology that wvill bethe basis for developing the enterprise architecture products and hasselected a tool for automating these activities.

Stage 3: Developing the EA. An organization at stage 3 focuses ondeveloping architecture products according to the selected framework,methodology, tool, and established management plans. Roles andresponsibilities assigned in the previous stage are in place, and resourcesare being applied to develop actual enterprise architecture products. Atthis stage, the scope of the architecture has been defined to encompass theentire enterprise, whether organization-based or function-based. Althoughthe products may not be complete, they are intended to describe theorganization in terms of business, performance, information/data,service/application, and technology (including security explicitly in each)as provided for in the framework, methodology, tool, and managementplans. 3' Further, the products are to describe the current (as-is) and future(to-be) states and the plan for transitioning from the current to the futurestate (the sequencing plan). As the products are developed and evolve, theyare subject to configuration management. Further, through the established

enterprise architecture management foundation, the organization istracking and measuring its progress against plans, identifying andaddressing variances, as appropriate, and then reporting on its progress.

Stage 4: Completing the EA. An organization at stage 4 has completedits enterprise architecture products, meaning that the products have beenapproved by the enterprise architecture steering committee (established instage 2) or an investment review board, and by the CIO. The completedproducts collectively describe the enterprise in terms of business,performance, information/data, service/application, and technology forboth its current and future operating states, and the products include a planfor transitioning from the current to the future state. Further, anindependent agent has assessed the quality (i.e., completeness andaccuracy) of the enterprise architecture products. Additionally, evolutionof the approved products is governed by a written enterprise architecturemaintenance policy approved by the head of the organization.

"1This set of products is consistent with O1M13s federal enterprise architecture referencemodels.


Stage 5: Leveraging the EA to manage change. An organization atstage 5 has secured senior leadership approval of the enterprisearchitecture products and a written institutional policy stating that ITinvestments must comply with the architecture, unless granted an explicitcompliance waiver. Further, decision makers are using the architecture toidentify and address ongoing and proposed IT investments that areconflicting, overlapping, not strategically linked, or redundant. As a result,stage 5 entities avoid unwarranted overlap across investments and ensuremaximum systems interoperability, which in turn ensures the selection andfunding of IT investments with manageable risks and returns. Also, at stage5, the organization tracks and measures enterprise architecture benefits orreturn on investment, and adjustments are continuously made to both theenterprise architecture management process and the enterprisearchitecture products.

EAMMF Attributes Attribute 1: Demonstrates commitment. Because the enterprisearchitecture is a corporat e asset for systematically managing institutionalchange, the support and sponsorship of the head of the enterprise areessential to the success of the architecture effort. An approved enterprisepolicy statement provides such support and sponsor-ship, promotinginstitutional buy-in and encouraging resource commitment fromparticipating components. Equally important in demonstratingcommitment is vesting ownership of the architecture with an executivebody that collectively owns the enterprise.

Attribute 2: Provides capability to meet commitment. The success ofthe enterprise architecture effort depends largely on the organization'scapacity to develop, maintain, and implement the enterprise architecture.Consistent with any large IT project, these capabilities include providingadequate resources (i.e., people, processes, and technology), defining clearroles and responsibilities, and defining and implementing organizationalstructures and process management controls that promote accountabilityand effective project execution.

Attribute 3: Demonstrates satisfaction of commitment. Satisfactionof the organization's commitment to develop, maintain, and implement anenterprise architecture is demonstrated by the production of artifacts (e.g.,the plans and products). Such artifacts demonstrate follow through-thatis, actual enterprise architecture production. Satisfaction of commitment isfurther demonstrated by senior leadership approval of enterprisearchitecture documents and artifacts; such approval communicates


institutional endorsement and ownership of the architecture and thechange that it is intended to drive.

Attribute 4: Verifies satisfaction of commitment. This attributefocuses on measuring and disclosing the extent to which efforts to develop,maintain, and implement the enterprise architecture have fulfilled statedgoals or commitments of the enterprise architecture. Measuring suchperformance allows for tracking progress that has been made towardstated goals, allows appropriate actions to be taken when performancedeviates significantly from goals, and creates incentives to influence bothinstitutional and individual behaviors.


Figure 1: Summary of EAMMF Version 1.1: Maturity Stages, Critical Success Attributes, and Core Elements

.. .......................................................... .......................................... Stagee 5:............................................................... Stg 3:Sae4 Leveraging the EA to.. ................ competin 3:mngecagStage 1::5~Ji[~ing theDeveloping EA pout

Creating Emage nt productsE foundation

awareness:

Attribute 1: Adequate resources exist. Written and approved Written and approved Written and approvedDemonstrates Committee or group representing the organizatinplc xss ognzto oiyeit o organization policy

comtetfor EA dvlpet. EA maintenance, exists for IT investmentcommitment ~enterprise is responsible for directing, Adee:pe

overseeing, and approving EA. compliance with EA.

Attribute 2: Program office responsible for EA EA products are under EA products and Process exists toProvides development and maintenance exists.: configuration management processes formally manage EAcapability to Cifacietxss.management. undergo independent change.meet verification and validation. E sitga opnn

comtetEA is being developed using of EAiT investa ment en:framework, methodology, and o Tivsmn

automted oolmanagement process.

Attribute 3: EA plans call for describing both the EA products describe or EA products describe or will EA products areDemonstrates "as-is" and the "to-be" environments will describe both the describe both the "as-is" and the peniodically updated.satisfaction of of the enterprise, as well as a "as-is" and the "to-be" "to-be" environments of the irnvsm tscplcommitment sequencing plan for transitioning environments of the enterprise, as well as awihE

from the "as-is" to the "to-be." enterprise, as well as a sequencing plan for transitioningEA plans call for describing both the sequencing plan fdr from the "as-is" to the "to-be"' Ognztinha a"as-is" and the "to-be" environments transitioningfoth Both the *as-is" and the "to-be" apove curntvrsoin terms of business, performance, "as-is" to the "to-be. evrnet redsrbdoinformation/data, application/ Both the "as-is" and the will be described in terms ofservice, and technology. "to-be" environments are business, performance,EA plans call for business, described or will be information/data,performance, information/data, described in terms of application/service, andapplication/service, and technology buies:efrac ehooydescriptions to address security. ifraio/aa Business, performance,

application/service, and information/data,technlogyapplication/service, and

Business, performance, technology descriptions addressinformation/data, security.application/service, and Organization CIO has approvedtechnology descriptions current version of EA.addessuri wiltddes Committee or group representing

secuitythe enterprise or the investmentreview board has approvedcurrent version of EA.

Attribute 4: EA plans call for developing Progress against EA plans Quality of EA products is Return on EA investmentVerifies metrics for measuring EA is measured and reported. measured and reported. is measured andsatisfaction of progress, quality, compliance, and reported.commitment return on investment.CopinewtEAs

Measured and reporied.

MaturationSource: GAO.

Note: Each stage includes all elements of previous stages.


EAMMF Groups The framework's 31 core elements can also be placed in one of four groupsof architecture related activities, processes, products, events, andstructures. The groups are architecture governance, content, use, andmeasurement. These groups are generally consistent with the capabilityarea descriptions in the previously discussed OMB enterprise architectureassessment tool. For example, OMB's completion capability area addressesensuring that architecture products describe the agency in terms ofprocesses, services, data, technology, and performance and that the agencyhas developed a transition strategy. Similarly, our content group includesdeveloping and completing these same enterprise architecture products. Inaddition, OMB's results capability area addresses performancemeasurement as does our measurement group, and OMB's use capabilityarea addresses many of the same elements in our governance and usegroups.

Table 3 lists the core elements according to EAMMF group.

Table 3: Summary of EAMMF Version 1.1: Core Elements Categorized by Group

Group Core element

Governance Adequate resources exist (stage 2).Committee or group representing the enterprise is responsible for directing, overseeing, and approving EA(stage 2).Program office responsible for EA development and maintenance exists (stage 2).Chief architect exists (stage 2).EA being developed using a framework, methodology, and automated tool (stage 2).EA plans call for describing "as-is" environment, "to-be" environment, and sequencing plan (stage 2).EA plans call for describing enterprise in terms of business, performance, inform ation/data,application/service, and technology (stage 2).EA plans call for business, performance, information/data, application/service, and technology to addresssecurity (stage 2).Written and approved policy exists for EA development (stage 3).Written and approved policy exists for EA maintenance (stage 4).Organization CI0 has approved EA (stage 4).Committee or group representing the enterprise or the investment review board has approved current versionof EA (stage 4).Written and approved organization policy exists for IT investment compliance with EA (stage 5).Organization head has approved current version of EA (stage 5).

Content EA products are under configuration management (stage 3).


(Continued From Previous Page)Group Core element

EA products describe or will describe "as-is" environment, "to-be" environment and sequencing plan (stage3).Both "as-is" and "to-be" environments are described or will be described in terms given in stage 2 (stage 3).These descriptions address or will address security (stage 3).EA products and management processes undergo independent verification and validation (stage 4).EA products describe "as-is" environment, "to-be" environment, and sequencing plan (stage 4).Both "as-is" and "to-be" environments are described in terms given in stage 2 (stage 4).These descriptions address security (stage 4).Process exists to formally manage EA change (stage 5).EA products are periodically updated (stage 5).

Use EA is integral component of IT investment management process (stage 5).IT investments comply with EA (stage 5).

Measurement EA plans call for developing metrics to measure EA progress, quality, compliance, and return on investment(Stage 2).Progress against EA plans is measured and reported (stage 3).Quality of EA products is measured and reported (stage 4).Return on EA investment is measured and reported (stage 5).Compliance with EA is measured and reported (stage 5).

Source: GAO.

Overall State ofEnterpriseArchitectureManagement Is a Work-in-Progress, Although aFew Ag encies HaveLargely Satisfied OurFramework

Mostof the 27 major departments and agencies have not fully satisfied allthe core elements associated with stage 2 of our maturity framework. Atthe same time, however, most have satisfied a number of core elements atstages 3, 4, and 5. Specifically, although only seven have fully satisfied allthe stage 2 elements, the 27 have on average fully satisfied 80, 78, 61, and 52percent of the stage 2, 3, 4, and 5 elements, respectively. Of the coreelements that have been fully satisfied, 77 percent of those related toarchitecture governance have been fully satisfied, while 68, 52, and 47percent of those related to architecture content, use, and measurement,respectively, have been fully satisfied. Most of the 27 have also at leastpartially satisfied a number of additional core elements across all thestages. For example, all but 7 have at least partially satisfied all theelements required to achieve stage 3 or higher. Collectively, this meansefforts are underway to mature the management of most agency enterprisearchitecture programs, but overall these efforts are uneven and still a work-in-progress and they face numerous challenges that departments andagencies identified. It also means that some architecture programs provideexamples from which less mature programs could learn and improve.


Without mature enterprise architecture programs, some departments andagencies wvill not realize the many benefits that they attributed toarchitectures, and they are at risk of investing in IT assets that areduplicative, not wvell-integrated, and do not optimally support missionoperations.

The Degree to which MajorDepartments and AgenciesHave Fully Satisfied OurFramework's Core ElementsIs Uneven and TheirCollective Efforts Can BeViewed as a Work-in-Progress

To qualify for a given stage of maturity under our architecture managementframework, a department or agency had to fully satisfy all of the coreelements at that stage. Using this criterion, three departments and agenciesare at stage 2, meaning that they demonstrated to us through verifiabledocumentation that they have established the foundational commitmentsand capabilities needed to manage the development of an architecture. Inaddition, four are at stage 3, meaning that they similarly demonstrated thattheir architecture development efforts reflect employment of the basiccontrol measures in our framework. Table 4 summarizes the maturity stageof each architecture program that we assessed. Appendix IV provides thedetailed results of our assessment of each department and agencyarchitecture program against our maturity framework.

Table 4: Maturity Stage of Major Department and Agency Enterprise ArchitecturePrograms

Stage when programrequired to fully satisfy allelements In one stage to

Department/Agency advance to the next

Department of Housing and Urban Development (HUD) 3Department of the Interior (Interior) 3Department of Justice (DOJ) 3Department of Labor (Labor) 3Department of Agriculture (USDA) 2Department of Homeland Security (DHS) 2Office of Personnel Management (OPM) 2Department of the Air Force (Air Force) 1Department of the Army (Army) 1Department of Commerce (Commerce) 1Department of Defense -Business Enterprise 1Architecture (BEA)Department of Defense - Global Information Grid (GIG) 1


(Continued From Previous Page)Stage when program

required to fully satisfy allelements in one stage to

Department/Agency advance to the next

Department of Education (Education) 1Department of Energy (Energy)1Department of Health and Human Services (HHS)1Department of the Navy (Navy)1Department of State (State)1Department of the Treasury (Treasury)1Department of Transportation (Transportation)1Department of Veterans Affairs (VA)1Environmental Protection Agency (EPA)1General Services Administration (GSA) 1National Aeronautics and Space Administration (NASA)1National Science Foundation (NSF)1Nuclear Regulatory Commission (NRC)1Small Business Administration (SBA) 1Social Security Administration (SSA) 1U.S. Agency for International Development (USAID) 1Source: GAO analysis of department and agency data.

While using this criterion provides an important perspective on the state ofdepartment and agency architecture programs, it can mask the fact that theprograms have met a number of core elements across higher stages ofmaturity. When the percentage of core elements that have been fullysatisfied at each stage is considered, the state of the architecture effortsgenerally shows both a larger number of more robust architectureprograms as well as more variability across the departments and agencies.Specifically, 16 departments and agencies have fully satisfied more than 70percent of the core elements. Examples include Commerce, which hassatisfied 87 percent of the core elements, including 75 percent of the stage5 elements, even though it is at stage 1 because its enterprise architectureapproval board does not have'enterprisewide representation (a stage 2 coreelement). Similarly, SSA, which is also a stage 1 because the agency'senterprise architecture methodology does not describe the steps fordeveloping, maintaining, and validating the agency's enterprise architecture'(a stage 2 core element), has at the same time satisfied 87 percent of all theelements, including 63 percent of the stage 5 elements. In contrast, theArmy, which is also in stage 1, has satisfied but 3 percent of all frameworkelements. Overall, 10 agency architecture programs fully satisfied more


than 75 percent of the core elements, 14 between 50 and 75 percent, and 4fewer than 50 percent. These four included the three military departments.Table 5 summarizes for each department and agency the percentage of coreelements fully satisfied in total and by maturity stage.

Table 5: Percent of Framework Elements Satisfied by Department and Agency Architecture Programs within Each Maturity Stage

Percent of Percent of Percent of Percent of Percent offramework stage 2 stage 3 stage 4 stage 5

elements elements elements elements elementsDepartments/Agencies and Maturity Stages satisfied satisfied satisfied satisfied satisfied

Stage 3Department of the Interior 97 100 100 88 100Department of Housing and Urban Development 94 100 100 75 100Department of Labor 87 100 100 88 63Department of Justice 77 100 100 63 50Stage 2Office of Personnel Management 94 100 83 88 100Department of Homeland Security 77 100 83 75 50Department of Agiriculture 61 100 67 50 25Stage 1Department of CommerceSocial Security AdministrationDepartment of EducationDepartment of EnergyNational Aeronautics and Space AdministrationSmall Business AdministrationDepartment of the TreasuryDepartment of Health and Human ServicesEnvironmental Protection AgencyDepartment of Defense - Global Information GridDepartment of Defense - Business Enterprise ArchitectureDepartment of Veterans AffairsDepartment of TransportationDepartment of StateGeneral Services AdministrationNuclear Regulatory CommissionNational Science FoundationDepartment of the Air Force

87878477

71717171747168

65

8989

8989

67

78

7889

89

89

7878

10010010083

10067

831008367

67

83

88

1007588637563.3888

756350

7563

7550

63636363

3850

6350

65

58

7867

83

67

5063

50

3850

2555 67 50 50

55 67 83 50

52 78 67 25 3845 56 67 38 25


(Continued From Previous Page)Percent of Percent of Percent of Percent of Percent offramework stage 2 stage 3 stage 4 stage 5

elements elements elements elements elementsDepartments/Agencies and Maturity Stages satisfied satisfied satisfied satisfied satisfiedAgency for International Development 39 67 50 13 25Department of the Navy 32 44 50 25 13Department of the Army 3 11 0 0 0

Source: GAO analysis of department and agency data.

Notwithstanding the additional perspective that the percentage of coreelements fully satisfied across all stages provides, it is important to notethat the staged core elements in our framework represent a hierarchical orsystematic progression to establishing a well-managed architectureprogram, meaning that core elements associated with lower frameworkstages generally support the effective execution of higher maturity stagecore elements. For instance, if a program has developed its full suite of "as-is" and "to-be" architecture products, including a sequencing plan (stage 4core elements), but the products are not under configuration management(stage 3 core element), then the integrity and consistency of the productswill be not be assured. Our analysis showed that this was the case for anumber of architecture programs. For example, State has developedcertain "as-is" and "to-be" products for the Joint Enterprise Architecture,which is being developed in collaboration with USAID, but an enterprisearchitecture configuration management plan has not yet been finalized.

Further, not satisfying even a single core element can have a significantimpact on the effectiveness of an architecture program. For example, nothaving adequate human capital with the requisite knowledge and skills(stage 2 core element), not using a defined framework or methodology(stage 2 core element), or not using an independent verification andvalidation agent (stage 4 core element), could significantly limit the qualityand utility of an architecture. The DOD's experience between 2001 and2005 in developing its BEA is a case in point. During this time, we identifiedthe need for the department to have an enterprise architecture for itsbusiness operations, and we made a series of recommendations groundedin, among other things, our architecture management framework to ensure


that it was successful in doing So.Y2 In 2005,33 we reported that thedepartment had not implemented most of our recommendations. Wefurther reported that despite developing multiple versions of a wide rangeof architecture products, and having invested hundreds of millions ofdollars and 4 years in doing so, the department did not have a well-definedarchitecture and that what it had developed had limited utility. Amongother things, we attributed the poor state of its architecture products toineffective program governance, communications, program planning,human capital, and configuration management, most of which are stage 2and 3 foundational core elements. To the department's credit, we recentlyreported that it has since taken a number of actions to address thesefundamental weaknesses and our related recommendations and that it isnow producing architecture products that provide a basis upon which tobuild.

The significance of not satisfying a single core element is also readilyapparent for elements associated with the framework's content group. Inparticular, the framework emphasizes the importance of planning for,developing, and completing an architecture that includes the "as-is" and the"to-be" environments as well as a plan for transitioning between the two. Italso recognizes that the "as-is" and "to-be" should address the business,performance, information/dataI application/service, technology, andsecurity aspects of the enterprise. To the extent these aspects are notaddressed in this way, the quality of the architecture and thus its utility willsuffer. In this regard, we found examples of departments and agencies thatwere addressing some but not all of these aspects. For example, HUD hasyet to adequately incorporate security into its architecture. This issignificant because security is relevant to all the other aspects of itsarchitecture, such as informnation/data and applications/services. Asanother example, NASA's architecture does not include a plan fortransitioning from the "as-is" to the "to-be" environments. According to theadministration's Chief Enterprise Architect, a transition plan has not yetbeen developed because of insufficient time and staff.

'See, for example, GAO-01-525, GAO-03-458, GAO-04-731R, GAO-05-702, and GAO, DODBusiness Systems Modernization. Important Progress Made in EstablishingFoundational Architecture Products and Investment Management Practices, but MuchWork Remains, GAO-06-219 (Washington, D.C.: Nov. 23, 2005).

'3GAO-05-702.


Looking across al the departments and agencies at core elements that arefully satisfied, not by stage of maturity, but by related groupings of core'elements, provides an additional perspective on the state of the federalgovernment's architecture efforts. As noted earlier, these groupings of coreelements are architecture governance, content, use, and measurement.Overall, departments and agencies on average have fully satisfied 77percent of the governance-related elements. In particular, 93 and 96 percentof the agencies have established an architecture program office andappointed a chief architect, respectively. In addition, 93 percent have plansthat call for their respective architectures to describe the "as-is" and the"to-be" environments, and for having a plan for transitioning between thetwo (see fig. 2). In contrast, however, the core element associated withhaving a committee or group with representation from across theenterprise directing, overseeing, and approving the architecture was fullysatisfied by only 57 percent of the agencies. This core element is importantbecause the architecture is a corporate asset that needs to beenterprisewide in scope and accepted by senior leadership if it is to beleveraged for organizational change.


Figure 2: Overall Satisfaction of Core Elements Associated with Architecture Governance

Framework elements

Adequate resources exist (stage 2)

Committee or group representing the enterprise Is responsible for directing, _____________________

overseeing, and approving EA (stage 2) _

Program off ice responsible for EA development and maintenance exists(stage 2).

Chief architect exists (stage 2)

EA Is being developed using a framework, methodology, and automated tool_________________________(stage 2)

EA plans call for describing both the "as-is" and the "to-be" environments ofthe enterprise, as well as a sequencing plan for transitioning from the "'as-is"

to the "to-be"_(stage_2)____________ _____

EA plans call for describing both the "as-is" and the "to-be" environments interms of business, performance, Information/data, application/service, and

_____________technology (stage 2)

EA plans call for business, performance, info rmatio n/data, application/service,__________ ..- and technlg descriptions to address security (stage 2) _____

Written and approved organization policy exists for EA development (stage 3)

Written and approved organization policy exists for EA maintenance (stage 4)

Written and approved organization policy exists for IT Investment compliance II- -with EA (stage 5)

Organization CIO has approved current version of EA (stage 4)

Committee or group representing the enterprise or the Investment review __________________________board has approved current version of EA (stage 4)

Organization head has approved current version of EA (stage 5)

0 20Percentage

L I Not satisfied

SPartially satisfied-Satisfied

40 60 80 100

Source: GAO analysis of department/agency data.

Note: Numbers might not add to 100 percent due to rounding.


In contrast to governance, the extent of full satisfaction of those coreelements that are associated with what an architecture should containvaries widely (see fig. 3). For example, the three content elements thataddress prospectively what the architecture will contain, either in relationto plans or some provision for including needed content, were fullysatisfied about 90 percent of the time. However, the core elementsaddressing whether the products now contain such content were fullysatisfied much less frequently (between 54 and 68 percent of the time,depending on the core element), and the core elements associated withensuring the quality of included content, such as employing configurationmanagement and undergoing independent verification and validation, werealso fully satisfied much less frequently (54 and 21 percent of the time,respectively). The state of thbse core elements raises important questionsabout the quality and utility of the department and agency architectures.


Figure 3: Overall Satisfaction of Core Elements Associated with Architecture Content

Framework elements

EA products are under configuration management (stage 3)p

EA products describe or will describe both the "as-is" and the "to-be"environments of the enterprise, as well as a sequencing plan for

transitioning from the "as-is" to the "to-be" (stage 3)

Both the "as-is" and the "to-be" environments are described or will bedescribed In terms of business, performance, Inform ation/data,

application/service, and technology (stage 3)

Business, performance, Information/data, application/service, and technology__________________________descriptions address or will address security (stage 3)no

EA products and management processes undergo Independent verificationand validation (stage 4)

EA products describe both the "as-is" and the "to-be" environments of theenterprise, as well as a sequencing plan for transitioning from -

the "as-is" to the "to-be" (stage 4)

Both the "as-is" and the "to-be" environments are described In terms ofbusiness, performance, Information/data, application/service,

and technology (stage 4)

Business, performance, Information/data, application/service, and technologydescriptions address security (stage 4)

Process exists to formally manage EA change (stage 5) i.EA products are periodically updated (stage 5) I I

0 20 40 60 80 100Percentage

W Not satisfiled

rllmM Partiallysatisfied

MSatisfied



The degree of full satisfaction of those core elements associated with the'remaining two groups-use and measurement-is even lower (see figs. 4and 5, respectively). For example, the architecture use-related coreelements were fully satisfied between 39 and 64 percent of the time, whilethe measurement-related elements were satisfied between 14 and 71percent. Of particular note is that only 39 percent of the departments andagencies could demonstrate that IT investments comply with theirenterprise architectures, only 43 percent of the departments and agencies


could demonstrate that compliance with the enterprise architecture ismeasured and reported, and only 14 percent were measuring and reportingon their respective architecture program's return on investment. As ourwork and related best practices show, the value in having an architecture isusing it to affect change and produce results. Such results, as reported bythe departments and agencies include improved information sharing,increased consolidation, enhanced productivity, and lower costs, all ofwhich contribute to improved agency performance. To realize thesebenefits, however, IT investments need to comply with the architecture andmeasurement of architecture activities, including accrual of expectedbenefits, needs to occur.

Figure 4: Overall Satisfaction of Core Elements Associated with Architecture Use

Framework elements

EA Is Integral component of IT Investment management process (stage 5)

IT Investments comply with EA (stage 5) __. I0 20 40 60 80 100Percentage

WNot satisfied

SPartially satisfied-SatisfiedSource: GAO analysis of department/agencry data.



Figure 5: Overall Satisfaction of Core Elements Associated with Architecture Measurement

Framework elements

EA plans call for developing metrics for measuring EA progress,quality, compliance, and return on Investment (stage 2)

Progress against EA plans Is measured and reported (stage 3)

Quality of EA products Is measured and reported (stage 4)

Return on EA Investment Is measured and reported (stage 5)

Compliance with EA Is measured and reported (stage 5)

0 20 40 60 80 100Percentage

WNot satisfied

SPartially satisfied

Satisfied



Most Agencies Have at LeastPartially Satisfied MostFramework Elements

In those instances where departments and agencies have not fully satisfiedcertain core elements in our framework, most have at least partiallysatisfied"' these elements. To illustrate, 4 agencies would improve to atleast stage 4 if the criterion for being a given stage was relaxed to onlypartially satisfying a core element. Moreover, 11 of the remaining agencieswould advance by two stages under such a less demanding criterion, andonly 6 would not improve their stage of maturity under thesecircumstances. A case in point is Commerce, which could move from stage1 to stage 5 under these circumstances because it has fully satisfied all butfour core elements and these remaining four (one each at stages 2 and 4and two at stage 5) are partially satisfied. Another case in point is the SSA,which has fully satisfied all but four core elements (one at stage 2 and threeat stage 5) and has partially satisfied three of these remaining four. If thecriterion used allowed advancement to the next stage by only partiallysatisfying core elements, the administration would be stage 4. (See fig. 6 for

31Partially satisfied means that a department or agency has addressed some, but not all,aspects of the core element.


a comparison of department and agency program maturity stages under thetwo criteria.)


Figure 6: Department/Agency Maturity Stage Based on Fully Versus Partially Satisfied Criterion

Program

Department of Agriculture

Department of the Air Farce

Department of the Army

Department of Commerce

Department of Defense - Business Enterprise Architecture

Department of Defense - Global Information Grid

Department of Education

Department of Energy

Department of Health and Human Services

Department of Homeland Security

Department of Housing and Urban Development

Department of the Interior

Department of Justice

Department of Labor

Department of the Navy

Department of State

Department of Transportation

Department of the Treasury

Department of Veterans Affairs

Environmental Protection Agency

General Services Administration

National Aeronautics and Space Administration

National Science Foundation

Nuclear Regulatory Commission

Off ice of Personnel Management

Small Business Administration

Social Security Administration

U.S. Agency for International Development

Maturity stage based onhighest stage In which allelements are fully satisfied1 2 3 4 5H 0EL ElL0"1ElLILILIU El LI El E"EDl El E"EDl0 D lIU El EL ELiE]* -El IDLI* ElD Li ELI" EL DI lI L0 U -El IDM NE E LID0 NMDE]EM NElILI1E 00ElILI* LED EEL EliNE] L ElDI0EDL ILILIE"El ELI ELEl" EL [E] ElI LU ELI LII ElI00000ILIN ELELEl* LI ELI LI00ElI0ILI0N 0 [IEj0E* LI LII 0LEEL LID0 [I F0 1- lED EIL

Maturity stage based on higheststage In which all elementsare fully or partially satisfied12 34 5

EMKLE]IDONE0 Eli 0IIU LI1: LID LI0000.N U0Ej LI LN EMN lI

EM NEELEU 0I LDlEU N 0I LI

NE 0 LI El EMU LI N lMU LI 0 l :EM U NI Li-:ME 0 LI LIEMU LI 00ELIMU LI [IEIMMM Q lINU U LI ElMU 0- 00LI0 UNE, LI LI00EUlLILIMMMD LIMMMD0 lIMMMDL0I0NMMMDLIMM0 W 1 0MEUNHNEL0El 0 l E



As mentioned earlier, departments and agencies can require considerabletime to completely address issues related to their respective enterprisearchitecture programs. It is thus important to note that even though certaincore elements are partially satisfied, fully satisfying some of them may notbe accomplished quickly and easily. It is also important to note theimportance of fully, rather than partially, satisfying certain elements, suchas those that faIJ within the architecture content group. In this regard, 18,18, and 21 percent of the departments and agencies partially satisfied thefollowing stage 4 content-related core elements, respectively: "EA productsdescribe 'as-is' environment, 'to-be' environment and sequencing plan";"Both 'as-is' and 'to-be' environments are described in terms of business,performance, information/data, application/service, and technology"; and"These descriptions fully address security." Not fully satisfying theseelements can have important implications for the quality of an architecture,and thus its usability and results.

Seven Departments orAgencies Need to SatisfyFive or Fewer CoreElements to Be at Stage 5

Seven departments or agencies would meet our criterion for stage 5 if eachwas to fully satisfy one to five additional core elements (see table 6). Forexample, Interior could achieve stage 5 by satisfying one additionalelement: "EA products and management processes undergo independentverification and validation." In this regard, Interior officials have drafted astatement of work intended to ensure that independent verification andvalidation of enterprise architecture products and management processesis performed. The other six departments and agencies are HUD and OPM,which could achieve stage 5 by satisfying two additional elements;Commerce, Labor, and SSA, which could achieve the same by satisfyingfour additional elements; and Education which could be at stage 5 bysatisfying five additional elements. Of these seven, five have not fullysatisfied the independent verification and validation core element.

Notwithstanding the fact that five or fewer core elements need to besatisfied by these agencies to be at stage 5, it is important to note that insome cases the core elements not being satisfied are not only veryimportant, but also neither quickly nor easily satisfied. For example, one ofthe two elements that HUD needs to satisfy is having its architectureproducts address security. This is extremely important as security is anintegral aspect of the architecture's performance, business,information/data, application/service, and technical models, and needs tobe reflected thoroughly and consistently across each of them.


Table 6: Departments and Agencies That Need to Satisfy 5 or Fewer Core Elements to Achieve Stage 5

Number ofunsatisfied

Department/agency elements Unsatisfied element(s)

Department of the 1 * EA products and management processes undergo independent verification and validation.InteriorDepartment of '2 * EA products and management processes undergo independent verification and validation.Housing and Urban - Business, performance, information/data, application/service, and technology descriptionsDevelopment address security.Office of Personnel 2 * Progress against EA plans is measured and reported.Management * EA products and management processes undergo independent verification and validation.Department of 4 9 Committee or group representing the enterprise is responsible for directing, overseeing, andCommerce approving EA.

- Committee or group representing the enterprise or the investment review board has approvedcurrent version of EA.

* Written and approved organization policy exists for IT investment compliance with EA.* IT investments comply with EA.

Department of Labor 4 - EA products and management processes undergo independent verification and validation.- IT investments comply with EA.* Return on EA investment is measured and reported.* Compliance with EA is measured and reported.

Social Security 4 * EA is being developed using a framework, methodology, and automated tool.Administration * Organization head has approved current version of EA.

- Return on EA investment is measured and reported.* Compliance with EA is measured and reported.

Department of - Committee or group representing the enterprise is responsible for directing, overseeing, andEducation approving EA.

- EA products and management processes undergo independent verification and validation.- Committee or group representing the enterprise or the investment review board has approved

current version of EA.* Organization head has approved current version of EA.* Return on EA investment is measured and reported.

Source: GAO.

Departments and AgenciesReport NumerousChallenges Facing Them inDeveloping and UsingEnterprise Architectures

The challenges facing departments and agencies in developing and usingenterprise architectures are formnidable. The challenge that mostdepartments and agencies cited as being experienced to the greatest extentis the one that having and using an architecture is intended to overcome-organizational parochialism and cultural resistance to adopting aenterprisewide mode of operation in -which organizational parts are sub-optimized in order to optimize the performance and results of theenterprise as a whole. Specifically, 93 percent of the departments andagencies reported that they encountered this challenge to a significant(very great or great) or moderate extent. Other challenges reported to this


same extent were ensuring that the architecture programr had adequatefunding (89 percent), obtaining staff skilled in the architecture discipline(86 percent), and having the department or agency senior leadersunderstand the importance and role of the enterprise architecture (82percent).

As we have previously reported, sustained top management leader-ship isthe key to overcoming each of these challenges. In this regard, ourenterprise architecture management maturity framework provides for suchleadership and addressing these and other challenges through a number ofcore elements. These elements contain mechanisms aimed at, for example,establishing responsibility and accountability for the architecture withsenior leaders and ensuring that the necessary institutional commitmentsare made to the architecture program, such as through issuance ofarchitecture policy and provision of adequate resources (both funding andpeople). See table 7 for a listing of the reported challenges and the extent towhich they are being experienced.

Table 7: Degree to Which Departments and Agencies Are Experiencing EnterpriseArchitecture Challenges

Percentage of departmentsand agencies experiencing the

Challenge challenge to a great or very great extentOvercomingparochialism/cultural resistance 76Ensuring adequate funding 52Fostering top managementunderstanding 48Obtaining skilled staff 48Source: GAO analysis based on departmentlagency data.

Many Departments andAgencies Reported ThatThey Have Already RealizedSignificant ArchitectureBenefits, While Most Expectto Do So in the Future

A large percentage of the departments and agencies reported that they havealready accrued numerous benefits from their respective architectureprograms (see table 8). For example, 70 percent said that have alreadyimproved the alignment between their business operations and the IT thatsupports these operations to a significant extent. Such alignment isextremely important. According to our IT investment management


maturity framework,' alignment between business needs and ITinvestments is a critical process in building the foundation for an effectiveapproach to IT investment management. In addition, 64 percent respondedthat they have also improved information/knowledge sharing to asignificant or moderate extent. Such sharing is also very important. In 2005,for example, we added homeland security information sharing to our list ofhigh-risk areas because despite the importance of information to fightingterrorism and maintaining the security of our nation, many aspects ofhomeland security information sharing remain ineffective andfragmented."6 Other examples of mission-effectiveness related benefitsreported as already being achieved to a significant or moderate extent byroughly one-half of the departments and agencies included improvedagency management and change management and improved system andapplication interoperability.

Beyond these benefits, departments and agencies also reported alreadyaccruing, to a significant or moderate extent, a number of efficiency andproductivity benefits. For example, 56 percent reported that they haveincreased the use of enterprise software licenses, which can permit costsavings through economies of scale purchases; 56 percent report that theyhave been able to consolidate their IT infrastructure environments, whichcan reduce the costs of operating and maintaining duplicative capabilities;41 percent reported that they have been able to reduce the number ofapplications, which is a key to reducing expensive maintenance costs; and37 percent report productivity improvements, which can free resources tofocus on other high priority matters.

Notwithstanding the number and extent of benefits that department andagency responses show have already been realized, these same responsesalso show even more benefits that they have yet to realize (see table 8). Forexample, 30 percent reported that they have thus far achieved, to little orno extent, better business and IT alignment. They similarly reported thatthey have largely untapped many other effectiveness and efficiencybenefits, with between 36 and 70 percent saying these benefits have beenachieved to little or no extent, depending on benefit. Moreover, for all thecited benefits, a far greater percentage of the departments and agencies (74

3ýGAO, JInformnat ion Technology Investment Mlanagement: A Fr-amewvork forAssessing andImproving Process Mlaturity. GAO-04-394G. (Washington, D.C.: Mar. 2004).

'~GAO, High Risk Series: An Update. GAO-0.5-207. (Washington, D.C.: Jan. 2005).


to 93 percent) reported that they expect to realize each of the benefits to asignificant or moderate extent sometime in the future. What this suggests isthat the real value in the federal government from developing and usingenterprise architecture remains largely unrealized potential.

Our architecture maturity framework recognizes that a key to realizing thispotential is effectively managing department and agency enterprisearchitecture programs. However, knowing whether benefits and results arein fact being achieved requires having associated measures and metrics. Inthis regard, very few (21 percent) of the departments and agencies fullysatisfied our ýtage 5 core element, "Return on EA investment is measuredand reported." Without satisfying this element, it is unlikely that the degreeto which expected benefits are accrued will be known.

Table 8: Enterprise Architecture Benefits Reported As Being or To Be Achieved to aSignificant Extent

PercentPercent reporting reporting thatthat the benefit is the benefit will

Benefit being achieved 'be achieved

Improved business and information technologyalignment 70 93Improved information/knowledge sharing 64 93Improved agency and change management 54 89Increased infrastructure consolidation 56 81

Increased use of enterprise licenses 56 89Improved systems interoperability 48 93Improved application interoperability 46 81Fewer applications 41 89Optimized business processes 41 89Improved data integration 39 89Enhanced productivity 37 81Improved data reuse 33 93Lower system-related costs 30 85Reduced system complexity 30 74Source: GAO based on department and agency data.

Note: Significant extent means a very great, great, or moderate extent.


Conclusions If managed effectively, enterprise architectures can be a useful changemanagement and organizational transformation tool. The conditions foreffectively managing enterprise architecture programs al-c contained in ourarchitecture management maturity framework. While a few of the federalgovernment's 27 major departments and agencies have fully satisfied all theconditions needed to be at stage 2 -or above in our framework, many havefully satisfied a large percentage of the core elements across most of thestages, particularly those elements related to architecture governance.Nevertheless, most departments and agencies are not yet where they needto be relative to architecture content, use, and measurement and thus thefederal government is not as well positioned as it should be to realize thesignificant benefits that a well-managed architecture program can provide.Moving beyond this status will require most departments and agencies toovercome some significant obstacles and challenges. The key to doing socontinues to be sustained organizational leadership. Without suchorganizational leadership, the benefits of enterprise architecture will not befully realized.

Recommendations forExecutive Action

To assist the 27 major departments and agencies in addressing enterprisearchitecture challenges, managing their architecture programs, andrealizing architecture benefits, we recommend that the Administrators ofthe Environmental Protection Agency, General Services Administration,National Aeronautics and Space Administration, Small BusinessAdministration, and U.S. Agency for International Development; theAttorney General; the Commissioners of the Nuclear RegulatoryCommission and Social Security Administration; the Directors of theNational Science Foundation and the Office of Personnel Management; andthe Secretaries of the Departments of Agriculture, Commerce, Defense,Education, Energy, Health and Human Services, Homeland Security,Housing and Urban Development, Interior, Labor, State, Transportation,Treasury, and Veterans Affairs ensure that their respective enterprisearchitecture programs develop and implement plans for fully satisfyingeach of the conditions in our enterprise architecture management maturityframework.


Agency Comments andOur Evaluation

We received written or oral comments on a draft of this report from 25 ofthe departments and agencies in our review.3"7 O the 25 departments andagencies, all but one department fully agreed with our recommendation.Nineteen departments and agencies agreed and six partially agreed withour findings. Areas of disagreement for these six centered on (1) theadequacy of the documentation that they provided to demonstratesatisfaction of certain core elements and (2) recognition of steps that theyreported taking to satisfy certain core elements after we concluded ourreview. For the most part, these isolated areas of disagreement did notresult in any changes to our findings for two primary reasons. First, ourfindings across the departments and agencies were based on consistentlyapplied evaluation criteria governing the adequacy of documentation, andwere not adjusted to accommodate any one particular department oragency. Second, our findings represent the state of each architectureprogram as of March 2006, and thus to be consistent do not reflectactivities that may have occurred after this time. Beyond these comments,several agencies offered suggestions for improving our framework, whichwe will consider prior to issuing the next version of the framework. Thedepartments' and agencies' respective comments and our responses, aswarranted, are as follows:

" Agriculture's Associate CIO provided e-mail comments stating that thedepartment will incorporate our recommendation into its enterprisearchitecture program plan.

" Commerce's CI0 stated in written comments that the departmentconcurred with our findings and will consider actions to address ourrecommendation. Commerce's written comments are reproduced inappendix V.

" DOD's Director, Architecture and Interoperability, stated in writtencomments that the department generally concurred with ourrecommendation to the five DOD architecture programs included in ourreview. However, the department stated that it did not concur with theone aspect of the recommendation directed at the GIG architectureconcerning independent verification and validation (IV&V) because itbelieves that its current internal verification and validation activities are

"7Representatives from the Departments of Health and Human Services and Transportationstated that they did not have comments.


sufficient. We do not agree for two reasons. First, these internalprocesses are not independently performed. As we have previouslyreported, IV&-V is a recognized hallmark of well managed programs,including architecture programs, and to be effective, it must beperformed by an entity that is independent of the processes andproducts that are being reviewed. Second, the scope of the internalverification and validation activities only extends to a subset of thearchitecture products and management processes.

The department also stated that it did not concur with one aspect of ourfinding directed at BEA addressing security. According to DOD, becauseGIG addresses security and the GIG states that it extends to all defensemission areas, including the business mission area, the BEA in effectaddresses security. We do not fully agree. While we acknowledge thatGIG addresses security and states that it is to extend to all DOD missionareas, including the business mission area, it does not describe how thiswill be accomplished for BEA. Moreover, nowhere in the BEA is securityaddressed, either through statement or reference, relative to thearchitecture's performance, business, information/data,application/service, and technology products. DOD's written comments,along with our responses, are reproduced in appendix VI.

9 Education's Assistant Secretary for Management and Acting 0I0 statedin written comments that the department plans to address our findings.Education's written comments are reproduced in appendix VII.

* Energy's Acting Associate CIO for Information Technology Reformstated in written commients that the department concurs with ourreport. Energy's written comments are reproduced in appendix VIII.

* DHS's Director, Departmental GAO/OIG Liaison Office, stated in writtencomments that the department has taken, and plans to take, steps toaddress our recommendation. DHS's written comments, along with ourresponses to its suggestions for improving our framework, arereproduced in appendix IX. DUS also provided technical comments viae-mail, which we have incorporated, as appropriate, in the report.

*HUD's 010 stated in written comments that the department generallyconcurs with our findings and is developing a plan to address ourrecommendation. The CI0 also provided updated information aboutactivities that the department is taking to address security in itsarchitecture. HUD's written comments are reproduced in appendix X.


*Interior's Assistant Secretary, Policy, Management and Budget, stated inwritten comments that the department agrees with our findings and

*recommendation and that it has recently taken action to address them.Interior's written comments are reproduced in appendix XI.

* DOJ's 010 stated in written comments that our findings accuratelyreflect the state of the department's enterprise architecture program andthe areas that it needs to address. The CI0 added that our report willhelp guide the department's architecture program and providedsuggestions for improving our framework and its application. DOJ'swritten comments, along with our responses to its suggestions, arereproduced in appendix XII.

" Labor's Deputy 010 provided e-mail comments stating that thedepartment concurs with our findings. The Deputy CI0 also providedtechnical comments that we have incorporated, as appropriate, in thereport.

" State's Assistant Secretary for Resource Management and ChiefFinancial Officer provided written comments that summarize actionsthat the department will take to fully satisfy certain core elements andthat suggest some degree of disagreement with our findings relative tothree other core elements. First, the department stated that itsarchitecture configuration management plan has been approved by boththe State and USAID GI~s. However, it provided no evidence todemonstrate that this was the case as of March 2006 when we concludedour review, and thus we did not change our finding relative toarchitecture products being under configuration management. Second,the department stated that its enterprise architecture has been approvedby State and USAID executive offices. However, it did not provide anydocumentation showing such approval. Moreover, it did not identifywhich executive offices it was referring to so as to allow adetermnination of whether they were collectively representative of theenterprise. As a result, we did not change our finding relative to whethera committee or group representing the enterprise or an investmentreview board has approved the current version of the architecture.Third, the department stated that it provided us with IT investmentscore sheets during our review that demonstrate that investmentcompliance with the architecture is measured and reported. However,no such score sheets were provided to us. Therefore, we did not changeour finding. The department's written comments, along with moredetailed responses, are reproduced in appendix XIII.


" Treasury's Associate CI0 for E-Government stated in written commentsthat the department concurs with our findings and discussed steps beingtaken to mature its enterprise architecture program. The Associate CI0also stated that our findings confirm the department's need to provideexecutive leadership in developing its architecture program and tocodify the program into department policy. Tryeasury's written commentsare reproduced in appendix XIV.

" VA'S Deputy Secretary stated in written comments that the departmentconcurred with our recommendation and that it will provide a detailedplan to implement our recommendation. VA's written comments arereproduced in appendix XV.

" EPA'S Acting Assistant Administrator and CI0 stated in writtencomments that the agency generally agreed with our findings and thatour assessment is a valuable benchmarking exercise that will helpimprove agency performance. The agency also provided comments onour findings relative to five core elements. For one of these coreelements, the comments directed us to information previously providedabout the agency's architecture committee that corrected ourunderstanding and resulted in us changing our finding about this coreelement. With respect to the other four core elements concerning use ofan architecture methodology, measurement of progress against programplans, integration of the architecture into investment decision making,and management of architecture change, the comments also directed usto information previously provided but this did not result in any changesto our findings because evidence demonstrating full satisfaction of eachcore element was not apparent. EPA~s written comments, along withmore detailed responses to each, are reproduced in appendix XVI.

" GSA'S Administrator stated in written comments that the agencyconcurs with our recommendation. The Administrator added that ourfindings will be critical as the agency works towards furtherimplementing our framework's core elements. GSA'S written commentsare reproduced in appendix XVII.

" NASA'S Deputy Administrator stated in written comments that theagency concurs with our recommendation. NASA's written commentsare reproduced in appendix XVIIH. NASA'S GAO Liaison also provided

* technical comments via e-mail, which we have incorporated, asappropriate, in the report.


- NSF'S CI0 provided e-mail comments stating that the agency will use theinformation in our report, where applicable, for future planning andinvestment in its architecture program. The 0I0 also provided technicalcomments that we have incorporated, as appropriate, in the report.

* NRC'S GAO liaison provided e-mail comments stating that the agencysubstantially agrees with our findings and describing activities it hasrecently taken to address them.

e OPM's 010 provided e-mail comments stating that the agency agreeswith our findings and describing actions it is taking to address them.

- SBA's GAO liaison provided e-mail comments in which the agencydisagreed with our findings on two core elements. First, andnotwithstanding agency officials' statements that its architectureprogram did not have adequate resources, the liaison did not agree withour "partially satisfied" assessment for this core element because,according to the liaison, the agency has limited discretionary funds andcompeting, but unfunded, federal mandates to comply with that limitdiscretionary funding for an agency of its size. While we acknowledgeSBA'S challenges, we would note that they are not unlike the resourceconstraints and competing priority decisions that face most agencies,and that while the reasons why an architecture program may not beadequately resourced may be justified, the fact remains that anyassessment of the architecture program's maturity, and thus itslikelihood of success, needs to recognize whether adequate resourcesexist. Therefore, we did not change our finding on this core element.Second, the liaison did not agree with our finding that the agency did nothave plans for developing metrics; for measuring architecture progress,quality, compliance, and return on investment. However, our review ofdocumentation provided by SBA and cited by the liaison showed thatwhile such plans address metric development for architecture progress,quality, and compliance, they do not address architecture return oninvestment. Therefore, we did not change our finding that this coreelement was partially satisfied.

*SSA's Commissioner stated in written comments that the report is bothinformative and useful, and that the agency agrees with ourrecommendation and generally agrees with our findings. Nevertheless,the agency disagreed with our findings on two core elements. First, theagency stated that documentation provided to us showed that it has amethodology for developing, maintaining, and validating its


*architecture. We do not agree. In particular, our review of SSA provideddocumentation showed that it did not adequately describe the steps tobe followed relative to development, maintenance, or validation.Second, the agency stated that having the head of the agency approvethe current version of the architecture is. satisfied in SSA's case becausethe Clinger-Cohen Act of 1996 vests its CI0 with enterprise architectureapproval authority and the CI0 has approved the architecture. W~e donot agree. The core element in our framework concerning enterprisearchitecture approval by the agency head is derived from federalguidance and best practices upon which our framework is based. Thisguidance and related practices, and thus our framework, recognize thatan enterprise architecture is a corporate asset that is to be owned andimplemented by senior management across the enterprise, and that akey characteristic of a mature architecture program is having thearchitecture approved by the department or agency head. Because theClinger-Cohen Act does not address approval of an enterprisearchitecture, our framework's core element for agency head approval ofan enterprise architecture is not inconsistent with, and is notsuperseded by, that act. SSA~s written comments, along with moredetailed responses, are reproduced in appendix XIX.

*USAID's Acting Chief Financial Officer stated in written commentsstated that the agency will work with State to implement ourrecommendation. USAID's written comments are reproduced inappendix XX.

As agreed with your offices, unless you publicly announce the contents ofthis report earlier, we plan no further distribution until 30 days from thereport date. At that time, we will send copies of this report to theAdministrators of the Environmental Protection Agency, General ServicesAdministration, National Aeronautics and Space Administration, SmallBusiness Administration, and U.S. Agency for International Development;the Attorney General; the Commissioners of the Nuclear RegulatoryCommission and Social Security Administration; the Directors of theNational Science Foundation and the Office of Personnel Management; andthe Secretaries of the Departments of Agriculture, Commerce, Defense,.Education, Energy, Health and Human Services, Homeland Security,Housing and Urban Development, Interior, Labor, State, T7ransportation,T7reasury, and Veterans Affairs. We will also make copies available to othersupon request. In addition, the report will be available at no charge on theGAO Web site at http://wwwv.gao.goN.


If you have any questions concerning this information, please contact me at(202) 512-3439 or by e-mail at [email protected]. Contact points for our Officesof Congressional Relations and Public Affairs may be found on the lastpage of this report. Key contributors to this report are listed in appendixXXI.

Sincerely yours,

Randolph C. HiteDirector, Information Technology Architecture

and Systems Issues

Page 44 Page 44GAO-06-83 1 Enterprise Architecture

Appendix I

Reported Enterprise Architecture Costs Vary,with Contractors and Personnel Accountingfor Most Costs

Department- and agency-reported data show wide variability in their coststo develop and maintain their enterprise architectures. Generally, the costscould be allocated to several categories with the majority of costsattributable to contractor support and agency personnel.

Architecture Developmentand Maintenance Costs Vary

As we have previously reported, the depth and detail of the architecture tobe developed and maintained is dictated by the scope and nature of theenterprise and the extent of enterprise transformation and modernizationenvisioned. Therefore, the architecture should be tailored to the individualenterprise and that enterprise's intended use of the architecture.Accordingly, the level of resources that a given department or agencyinvests in its architecture is likely to vary.

Departments and agencies reported that they have collectively invested atotal of $836 million to date on enterprise architecture development.Across the 27 departments and agencies, these development costs rangedfrom a low of $2 million by the Department of the Navy to a high of $433million by the Department of Defense (DOD) on its Business EnterpriseArchitecture (BEA). Department and agency estimates of the costs tocomplete their planned architecture development efforts collectively totalabout $328 million. The department and agencies combined estimates ofannual architecture maintenance costs is about $146 million. Thesedevelopment and maintenance estimates, however, do not include theDepartments of the Army and Justice because neither provided these costestimates. Figures 7 through 9 depict the variability of cost data reportedby the departments and agencies.


Appendix IReported Enterprise Architecture Costs Vary,with Contractors and Personnel Accountingfor Most Costs

Figure 7: Reported Development Costs to Date for Departments and Agencies

Dollars In millions

440

430

420

410

400

70

60

50

40

30

20

10

0 0

A C2 C2 cA 0 C' A' 0

*Departments and agencies

Source: Cited depaflmenis and agencies.

Note: The Departments of the Army and Justice did not provide development costs.



Figure 8: Reported Estimated Completion Costs for Departments and AgenciesDollars In millions

70

60

50

40

30

20

10

7-

IHnn FlF-0- 11 1..IL.j Li I I I I I II

C,~ 4. 0 ~~ ) r 0 2 C

Departments and agenciesSource: Cited departments and agencies.

Note: The Departments of the Air Force, Army, Energy, Justice, and Navy, the DOD's BEA and GlobalInformation Grid (GIG), and OPM did not provide completion costs. NSF reported completion costs($15,000), which are not identified on this figure.


Appendix IReported Enterprise Architecture Costs Vary,with Contractors and Personnel Accountingfor Mlost Costs

Figure 9: Reported Estimated Annual Maintenance Costs for Departments and Agencies

Dollars in millions

25

20 K-

10

5

0 FL Hnnri H m-- n F-I H Fý F-I r--i , F-1 - H F]~h~hi

Departments and agencies

Source: Cited departments and agencies.

Note: The Departments of the Army, Justice, and Navy did not provide maintenance costs.

Contractor SupportAccounts for the Majority ofArchitecture DevelopmentCosts

All of the departments and agencies reported developing their architecturein-house using contractor support. All but two of the departments andagencies allocated their respective architecture development costs to thefollowing cost categories:t contractor support, agency personnel, tools,methodologies, training, and other.' These 26 agencies accounted for about$741 million of the $836 million total development costs cited above. Thevast majority (84 percent) of the $741 million were allocated to contractorservices ($621 million), followed next by agency personnel (13 percent or$94 million). The remaining $26 million were allocated as follows: $12million (2 percent) to architecture tools; $9 million (1 percent) to "other"

'Tine Departments of the Army and Justice (lid not. provide cost dlata.

'Mhe "other" cost category includes costs that cannot be allocated to the other categories.


*Appendix IReported Enterprise Architecture Costs Vary,with Contractors and Personnel Accountingfor Mlost Costs

costs; $4 million (1 percent) to architecture methodologies; and $2 million(less than 1 percent) to training. (See fig. 10.)

Figure 10: Breakdown of Enterprise Architecture Development Costs for allDepartments and Agencies

<1%Training

1 %Methodology

2%Tools

1 %Other.

Agency personnel

Contractor costs


Note: Numbers do not add to 100 percent due to rounding.

Architecture DevelopmentActivities Were Reported asLargest Component ofContractor-Related Costs

The departments and agencies allocated the reported $621 million incontractor-related costs to the following five contractor cost categories:architecture development, independent verification and validation,



methodology, support services, and other.' Of these categories,architecture development activities accounted for the majority of costs-about $594 million (87 percent). The remaining $85 million was allocated asfollows: $51 million (7 percent) to support services, $13 million (2 percent)to "other" costs, $11 million (2 percen~t) to independent verification andvalidation, and $10 million (1 percent) to methodologies. (See fig. 11.)

Figure 11: Reported Enterprise Architecture Contractor Costs by Category

____ ___1%

Methodology

2%Independent verification & validation

2%Other

Support services

87% Architecture development


Note: Numbers do not add to 100 percent due to rounding.

3The "other" cost category is intended to include costs that cannot be allocated to thecategories wve specified.


Appendix II

Departments and Agencies ReportedExperiences with Their Architecture Toolsand Frameworks

Departments and agencies reported additional informnation related to theimplementation of their enterprise architectures. This information includesarchitecture tools and frameworks.

Departments and AgenciesReported Using a Variety ofEnterprise ArchitectureTools with Varying Degreesof Satisfaction

As stated in our enterprise architecture management maturity framework,an automated architecture tool serves as the repository of architectureartifacts, which are the work products that are produced and used tocapture and convey architectural information. An agency's choice of toolshould be based on a number of considerations, including agency needsand the size and complexity of the architecture.'

The departments and agencies reported that they use various automatedtools to develop and maintain their enterprise architectures, with 12reporting that they use more than one tool. In descending order offrequency, the architecture tools identified were System Architect (18instances), Microsoft Visio (17), Metis (12), Rational Rose (8), aridEnterprise Architecture Management System (EAMS) (4). In addition, 21departments and agencies reported using one or more other architecturetools.' Figure 12 shows the number of departments and agencies using eacharchitecture tool, including the other tools. 3

'GAO-03-5S4G.

'TMe "other" tool category is intended to include various tools that were not listed on oursurvey.

'Other tools reported by the departments and agencies include: Adaptive InformationTechnology Portfolio Manager, Adaptive-USDA EA Repository, Caliber-RNI, Catalyze bySteelTrace, Defense Architecture Repository System (DARS), DARS MIS Office,Embarcadero-ER Studio, Erwin, FRA Portal, MIS W~ord, 0MIG Component CollaborativeArchitecture (Component X Tool), ProSight, Serena Tracker and Version Manager.


Appendix 11Departments and Agencies ReportedExperiences with Their Architecture Toolsand Frameworks

Figure 12: Enterprise Architecture Tools Used by Departments and Agencies

Number of agencies

25

20

15

10

5

0Other System Vislo Metis Rational EAMS

Architect RoseTools


The departments and agencies also reported various levels of satisfactionwith the different enterprise architecture tools. Specifically, about 75percent of those using Microsoft Visio were either very or somewhatsatisfied with the tool, as compared to about 67 percent of those usingMetis, about 63 percent of those using Rational Rose, about 59 percent ofthose using System Architect, and 25 percent of those using EAMS. Thismeans that the percentage of departments and agencies that weredissatisfied, either somewhat or very, with their respective tools rangedfrom a high of 75 percent of those using EAMS, to a low of about 6 percentof those using System Architect. No departments or agencies that usedMetis, Rational Rose, or Microsoft Visio reported any dissatisfaction. Seetable 9 for a summary of department and agency reported satisfaction withtheir respective tools.


Appendix HIDepartments and Agencies ReportedExperiences with Their Architecture Toolsand Frameworks

Table 9: Department and Agency Reported Satisfaction with Tools

Number ofdepartments and

agenciesSomewhat

Very satisfied or Neither satisfied nor or very Undecided (tooTool name (Vendor) Using tool somewhat satisfied dissatisfied dissatisfied early to say)

EAMS 4 1 0 3 0Metis (TrouxTechnologies) 12 8 1 0 3Rational Rose (IBMCorporation) 8 5 2 0 1System Architect(PopkinSoftware/re lelogic AB) 18 10 4 1 2Visio (MicrosoftCorporation) 17 12 3 01Other 21 17 0 0 1

Source: GAO based on department and agency data.

Note: One agency did not indicate its satilsfaction or dissatisfaction with System Architect and Visio.

Departments and AgenciesReported Using a Variety ofEnterprise ArchitectureFrameworks with VaryingLevels of Satisfaction

As we have previously stated, an enterprise architecture frameworkprovides a formal structure for representing the architecture's content andserves as the basis for the specific architecture products and artifacts thatthe department or agency develops and maintains. As such, a frameworkhelps ensure the consistent representation of information from across theorganization and supports orderly capture and maintenance of architecturecontent.

The departments and agencies reported using various frameworks todevelop and maintain their enterprise architectures. The most frequentlycited frameworks were the Federal Enterprise Architecture ProgramManagement Office (FEAPMO) Reference Models (25 departments andagencies), the Federal Enterprise Architecture Framework (FEAF)4 (19depa~rtments and agencies), and the Zachman Framework (17 departmentsand agencies), with 24 reporting using more than one framework. Other,less frequently reported frameworks were the Department of Defense

'1This framework was issued in September 1999 by the federal CIO Council.


Appendix 11Departments and Agencies ReportedExperiences iMth Their Architecture Toolsand ]Frameworks

Architecture Framework (DODAFT), the National Institute of Standards andTechnology (NIST) framework, and The Open Group ArchitectureFramework (TOGAF). See figure 13 for a summary of the number ofdepartments and agencies that reported using each framework.

Figure 13: Frameworks Used by Departments and Agencies

Number of agencies

25

20

15

10

5

0FEAPMO FEAF Zachman DODAF Other NIST TOGAF

Framework


Departments and agencies also rep orted varying levels of satisfaction withtheir respective architecture. Specifically, about 72 percent of those usingthe FEAF indicated that they were either very or somewhat satisfied, andabout 67 and 61 percent of those using the Zachman framework and theFEAPMO reference models, respectively, reported that they were similarlysatisfied.5 As table 10 shows, few of the agencies that responded to oursurvey reported being dissatisfied with any of the frameworks.6

" Some agencies and departments (lid not indicate their level of satisfaction or dlissatisfactionwvith the framework(s) they reported using.

'Mhe number of responses regarding frameworks is larger than the number of agenciessurveyed because some agencies reported using more than one framework.


Appendix 11Departments and Agencies ReportedExperiences with Their Architecture Toolsand Frameworks

Table 10: Department and Agency Framework Satisfaction Levels

Number ofDepartmentsand Agencies

Very satisfied Neither Somewhat orUsing or somewhat satisfied nor very Undecided (too

Framework (source) framework satisfied dissatisfied dissatisfied early to say)

DODAF (Department of Defense) 7 4 2 0 0FEAF (CIO Council) 19 13 3 1 1

FEAPMO Reference Models (0MB) 25 14 4 3 2

NJST Framework (NIST) 2 1 0 1 0

TOGAF (The Open Group) 1 1 0 0 0

Zachman Framework (The ZachmanInstitute for Framework Advancement) 17 10 4 1 0Other 3 .3 0 0 0

Source: GAO based on department and agency data.


Appendix III

Objective, Scope, and Methodology

Our objective was to determine the current status of federal departmentand agency enterprise architecture efforts. To accomplish this objective,we focused on 28 enterprise architecture programs relating to 27 majordepartments and agencies. These 27 included the 24 departments andagencies included in the Chief Financial Officers Act.' In addition, weincluded the three military services (the Departments of the Army, AirForce, and Navy) at the request of Department of Defense (DOD) officials.For the DOD, we also included both of its departmentwide enterprisearchitecture programs-the Global Information Grid and the BusinessEnterprise Architecture. The U.S. Agency for International Development(USAID), which is developing a USAID enterprise architecture and workingwith the Department of State (State) to develop a Joint EnterpriseArchitecture, asked that we evaluate its efforts to develop the USAIDenterprise architecture. State officials asked that we evaluate their agency'senterprise architecture effort based the Joint Enterprise Architecture beingdeveloped with USAID. We honored both of these requests.

Table 11 lists the 28 department and agency enterprise architectureprograms that formed the scope of our review.

Table 11: List of Architecture Programs Included in this Report

AgencyDepartment of AgricultureDepartment of the Air ForceDepartment of the ArmyDepartment of CommerceDepartment of Defense - Business Enterprise ArchitectureDepartment of Defense - Global Information GridDepartment of EducationDepartment of EnergyDepartment of Health and Human ServicesDepartment of Homeland SecurityDepartment of Housing and Urban DevelopmentDepartment of the Interior

'This Act requires 24 departments and agencies to establish chief financial officers. See 31U.S.C. section 901.


Appendix IIIObjective, Scope, and Methodology

(Continued From Previous Page)AgencyDepartment of JusticeDepartment of LaborDepartment of the NavyDepartment of StateDepartment of TransportationDepartment of the TreasuryDepartment of Veterans AffairsEnvironmental Protection AgencyGeneral Services AdministrationNational Aeronautics and Space AdministrationNational Science FoundationNuclear Regulatory CommissionOfftice of Personnel ManagementSmall Business AdministrationSocial Security AdministrationU.S. Agency for International DevelopmentSource: GAO.

To determine the status of each of these architecture programs, wedeveloped a data collection instrument based on our EnterpriseArchitecture Management Maturity Framework (EAMMF),2 and relatedguidance, such as 0MB Circular A-1303 and guidance published by thefederal Chief Information Officers (0I0) Council,' and our past reports andguidance on the management and content of enterprise architectures.' Wepretested this instrument at one department and one agency. Based on theresults of the pretest, we modified our instrument as appropriate to ensurethat our areas of inquiry were complete and clear.

2GA003584G.

'Office of Management and Budget, Mlanagement of Federal Information Resources,Circular A-130 (Nov. 28, 2000).

'Chief Information Officers Council, Federal Enterprise Architecture F~ramewvork-, Version1.1 (September 1999) and Chief Information Officers Council, A Practical Guide to FederalEnterprise Architecture, Version 1.0 (February 2001).

5GAO-02-6; GAO-04A40; and, for example, GAO-OS-l0iS, GAO-04-777, GAO-05-702,GAO-06-219.



Next, we identified the Chief Architect or comparable official at each of the27 departments and agencies, and met with them to discuss our scope andmethodology, share our data collection instrument, and discuss the typeand nature of supporting documentation needed to verify responses to ourinstrument questions.6

On the basis of department and agency provided documentation to supporttheir respective responses to our data collection instrument, we analyzedthe extent to which each satisfied the 31 core elements in our architecturem .aturity framework. To guide our analysis, we defined detailed evaluationcriteria for determining whether a given core element was fully satisfied,partially satisfied, or not satisfied. The criteria for the stage 2, 3, 4, and 5core elements are contained in tables 12, 13, 14, and 15 respectively. Tofully satisfy a core element, sufficient documentation had to be provided topermit us to verify that all aspects of the core element were met. Topartially satisfy a core element, sufficient documentation had to beprovided to permit us to verify that at least some aspects of the coreelement were met. Gore elements that were neither fully nor partiallysatisfied were judged to be not satisfied.

Table 12: Stage 2 Evaluation Criteria

Core element Evaluation criteriaAdequate resources exist. Agency responded that "very adequate," "somewhat adequate," or "neither adequate

nor inadequate" resources exist for funding, personnel, and tools.Committee or group representing the enterprise is Agency (1) responded that a committee or group representing the enterprise isresponsible for directing, overseeing, and responsible for direction, oversight, and approval of the enterprise architecture; (2)approving EA. provided a charter or other documentation supporting the group's responsibilities;

and (3) provided sample meeting minutes or other documentation confirming thatmeetings have been held.

Program office responsible for EA development Agency (1) responded that a program office is responsible for EA development andand maintenance exists. maintenance and (2) provided documentation supporting their assertion.Chief architect exists. Agency (1) responded that chief architect exists and (2) provided documentation or

assertion that the chief architect is responsible and accountable for EA and serves asthe EA program manager.

67nie Social Sectuity Administration wvas the only agency to decline such an initial meeting.

Page 5S Page 68GAO-06-831 Enterprise Architecture


*(Continued From Previous Page)Core element Evaluation criteria

EA being developed using a framework, Agency (1) responded that the enterprise architecture is being developed using amethodology, and automated tool, framework, methodology, and automated tool; (2) provided documentation supporting

the use of a framework and automated tool; and (3) provided a documentedmethodology that includes steps for developing, maintaining, and validating theenterprise architecture.

EA plans call for describing "as-is" environment, Agency (1) responded that EA plans call for describing the "as-is" and "to-be""to-be" environment, and sequencing plan. environments and a sequencing plan and (2) provided plans that document this

assertion; or agency (1) responded that the EA describes the "as-is" and "to-be"environments and a sequencing plan and (2) provided documentation to support thisassertion.

EA plans call for describing enterprise in terms of Agency (1) responded that EA plans call for describing the enterprise in terms ofbusiness, performance, information/data, business, performance, information/data, application/service, and technology and (2)application/service, and technology, provided plans that document this assertion; or agency (1) responded that the EA

describes the enterprise in terms of business, performance, information/data,application/service, and technology and (2) provided documentation to support thisassertion.

EA plans call for business, performance, Agency (1) responded that EA plans call for business, performance, information/data,information/data, application/service, and application/service, and technology descriptions to address security and (2) providedtechnology to address security, plans that document this assertion; or agency (1) responded that the business,

performance, information/data, application/service, and technology descriptionsaddress security and (2) provided documentation to support this assertion.

EA plans call for developing metrics to measure Agency (1) responded that EA plans call for developing metrics to measure EAEA progress, quality, compliance, and return on progress, quality, compliance, and return on investment and (2) provided plans toinvestment, support this assertion; or responded (1) that EA progress, quality, compliance, and/or

return on investment is measured and reported and (2) provided support for thisassertion.

Source: GAO.


Core element Evaluation criteria

Written and approved policy exists for EA Agency (1) responded that a written and approved organization policy exists for EAdevelopment, development and (2) provided a policy that supported this assertion.EA products are under configuration Agency (1) responded that EA products are under configuration management and (2)management. provided their formally documented configuration management approach.EA products describe or will describe "as-is" Agency (1) responded that EA plans call for describing the "as-is" and "to-be"environment, "to-be" environment, and environments and a sequencing plan, (2) provided plans that document thissequencing plan. assertion, and (3) responded that it is "in the process of developing the EA" or that it

"has developed an EA"; or agency (1) responded that the EA describes the "as-is"and "to-be" environments and a sequencing plan and (2) provided documentation tosupport this assertion.



(Continued From Previous Page)Core element Evaluation criteria

Both "as-is" and "to-be" environments are Agency (1) responded that EA plans call for describing the enterprise in terms ofdescribed or will be described in terms of business, performance, information/data, application/service, and technology; (2)business, performance, information/data, provided plans that document this assertion; and (3) responded that it is "in theapplication/service, and technology, process of developing the EA" or that it "has developed an EA"; or agency (1)

responded that the EA describes the enterprise in terms of business, performance,information/data, application/service, and technology and (2) provideddocumentation to support this assertion.

These descriptions address or will address Agency (1) responded that EA plans call for business, performance, information/data,security, plans that document this assertion; and (3) responded that it is "in the process of

developing the EA" or that it "has developed an EA"; or agency (1) responded that thebusiness, performance, information/data, application/service, and technologydescriptions address security and (2) provided documentation to support thisassertion.

Progress against EA plans is measured and Agency (1) responded that it measures and reports progress against plans; (2)reported. provided a description of how progress against plans is measured and reported; and

(3) provided sample reports that include sample measures.Source: GAO.


Core element Evaluation criteriaWritten and approved policy exists for EA Agency (1) responded that a written and approved organization policy exists for EAmaintenance. maintenance and (2) provided a policy that supported this assertion.EA products and management processes Agency (1) responded that EA products and management processes undergoundergo independent verification and validation, independent verification and validation; (2) provided proof that independent

verification and validation activities were conducted by an independent third party andreported outside the span of control of the chief architect; and (3) provided sampleindependent verification and validation reports to the audit team. Independence was acritical element for satisfaction of this item.

EA products describe "as-is" environment, "to-be" Agency (1) responded that the EA describes the "as-is" and "to-be" environments andenvironment, and sequencing plan. a sequencing plan; (2) provided documentation to support this assertion; and (3)

responded that it has developed an EA. In addition, an agency could not receive fullcredit for satisfying this element unless it fully satisfied the element, "Both 'as-is' and'to-be' environments are described in terms of business, performance,information/data, application/service, and technology."

Both "as-is" and "to-be" environments are Agency (1) responded that the EA describes the enterprise in terms of business,described in terms of business, performance, performance, information/data, application/service, and technology; (2) providedinformation/data, application/service, and documentation to support this assertion; and (3) responded that it has developed antechnology. EA. Agencies that completed four or five required descriptions in both the "as-is" and

"to-be" environments received a yes for this item. Agencies that addressed less thattwo of the five required descriptions in both the "as-is" and "to-be" environmentsreceived a no for this element.



(Continued From Previous Page)Core element Evaluation criteria

These. descriptions address security. Agency (1) responded that the business, performance, information/data,application/service, and technology descriptions address security; (2) provideddocumentation to support this assertion; and (3) responded that it has developed anEA.

Organization 0I0 has approved EA. Agency (1) responded that that CIO has approved the current version of the EA and(2) provided a signature page or other proof that the 0I0 has approved currentversion of EA.

Committee or group representing the enterprise Agency (1) responded that a committee or group representing the enterprise or theor the investment review board has approved investment review board has approved current version of EA and (2) providedcurrent version of EA. meeting minutes or other proof that a committee or group representing the enterprise

or the investment review board has approved current version of EA.Quality of EA products is measured and Agency (1) responded that it measures and reports product quality, (2) provided areported. description of how quality is measured and reported, and (3) provided sample reports

that include sample measures.Source: GAO.


Core element Evaluation criteria

Written and approved organization policy exists Agency (1) responded that a written and approved organization policy exists for ITfor IT investment compliance with EA. investment compliance with EA and (2) provided a written policy to support this

assertion.Process exists to formally manage EA change. Agency (1) responded that a process exists to formally manage EA change and (2)

provided evidence to support this assertion.EA is integral component of IT investment Agency (1) responded that EA is an integral component of IT investmentmanagement process. management process; (2) provided documentation describing how the EA is used

when making IT investment decisions; (3) provided evidence that a sequencing planexists to guide IT investments; and (4) partially or fully satisfied at least one of thefollowing stage 3 elements: (a) EA products describe or will describe "as-is"environment, "to-be" environment, and sequencing plan, (b) both "as-is" and "to-be"environments are described or will be described in terms of business, performance,information/data, application/service, and technology, or (c) these descriptionsaddress or will address security.

EA products are periodically updated. Agency (1) responded that EA products are periodically updated and (2) provided adescription of the process used for updating EA products.

IT investments comply with EA. Agency (1) responded that IT investments comply with EA; (2) provided evidence thatIT is not selected and approved under the organization's capital planning andinvestment control process unless it is compliant with the EA; and (3) partially or fullysatisfied at least one of the following stage 3 elements: (a) EA products describe orwill describe "as-is" environment, "to-be" environment, and sequencing plan, (b) both'as-is" and "to-be" environments are described or will be described in terms ofbusiness, performance, information/data, aipplication/service, and technology, or (c)these descriptions address or will address security.



(Continued From Previous Page)Core element

Organization head has approved current versiof EA.

Evaluation criteria

on Agency (1) responded that the organization head has approved the current version ofthe EA; (2) provided a signature page or other proof that organization head or adeputy organization head has approved current version of EA or provided proof offormal delegation of this activity and subsequent approval; and (3) partially or fullysatisfied at least one of the following stage 3 elements: (a) EA products describe orwill describe "as-is" environment, "to-be" environment, and sequencing plan, (b) both"as-is" and "to-be" environments are described or will be described in terms given inStage 2, or (c) these descriptions address or will address security.Agency (1) responded that it measures and reports return on investment; (2)provided a description of how return on investment is measured and reported; and (3)provided sample reports that included sample measures.Agency (1) responded that it measures and reports compliance, (2) provided adescription of how compliance is measured and reported, and (3) provided samplereports that included sample measures.

Source: GAO.

Return on EA investment is measured andreported.

Compliance with EA is measured and reportec

Our evaluation included first analyzing the extent to which eachdepartment and agency satisfied the core elements in our framework, andthen meeting with department and agency representatives to discuss coreelements that were not fully satisfied and why. As part of this interaction,we sought, and in some cases were provided, additional supportingdocumentation. We then considered this documentation in arriving at ourfinal determinations about the degree to which each department andagency satisfied each core element in our framework. In applying ourevaluation criteria, we analyzed the results of our analysis across differentcore elements to determine patterns and issues. Our analysis made use ofcomputer programs that were developed by an experienced staff; theseprograms were independently verified.

Through our data collection instrument, we also solicited from eachdepartment and agency information on enterprise architecture challengesand benefits, including the extent to which they had been or were expectedto be experienced. In addition, we solicited information on architecturecosts, including costs to date and estimated costs to complete and maintaineach architecture. We also solicited other information, such as use of andsatisfaction with architecture tools and frameworks. We analyzed theseadditional data to determine relevant patterns. We did not independently



verify these data. The results presented in this report reflect the state ofdepartment and agency architecture programs as of March 8, 2006.'

We conducted our work in the Washington, D.C., metropolitan area, fromMay 2005 to June 2006, in accordance with generally accepted governmentauditing standards.

'The Department of Defense submitted updated information to Congress about its BusinessEnterprise Architecture on March 15, 2006. This information was also considered as part ofour evaluation.


Appendix IV

Detailed Assessments of IndividualDepartments and Agencies against Our EAManagement Maturity Framework

Department of Agriculture Table 16 shows USDA's satisfaction of framework elements in version 1. 1 ofGAO's EAMME

Table 16: Department of Agriculture Satisfaction of EAMMF

GAO basis for partially satisfied or not satisfiedStages and core elements Satisfied? determination

Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA management foundationAdequate resources exist. YesCommittee or group representing the enterprise is Yesresponsible for directing, overseeing, and approvingEA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Yesand automated tool.EA plans call for describing "as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, information/data,application/service, and technology.EA plans call for business, performance, information/data, Yesapplication/service, and technology to address security.EA plans call for developing metrics to measure and Yesreport EA progress, quality, compliance, and return oninvestment.Stage 3: Developing EA productsWritten and approved organization policy exists for EA Yesdevelopment.EA products are under configuration management. No USDA plans to develop specific steps for configuration

management. However, specific configuration managementsteps have not been developed.

EA products describe or will describe "as-is" environment, Yes"to-be" environment, and sequencing plan.Both "as-is" and "to-be" environments are described or Yeswill be described in terms of business, performance,information/data, application/service, and technology.These descriptions address or will address security. Yes


Appendix IVDetailed Assessments of IndividualDepartments and Agencies against Our EAManagement Maturity Framework

(Continued From Previous Page)GAO basis for partially satisfied or not satisfied

Stages and core elements Satisfied? determination

Progress against EA plans is measured and reported. Partial USDA has limited reporting of EA progress. However,progress is not measured and reported relative to an EAprogram plan.

Stage 4: Completing EA productsWritten and approved organization policy exists for EA Yesmaintenance.EA products and management processes undergo No According to USDA's EA division, USDA componentindependent verification and validation, agencies conduct reviews of the EA products of other

component agencies. However, the documentation provideddid not address verification and validation of EAmanagement processes and did not provide sufficientassurance that the EA product reviews were independent.

EA products describe "as-is" environment, "to-be" No According to USDA's EA division, EA products describe theenvironment, and sequencing plan. "as-is" environment. However, a description of the "to-be"

environment and a sequencing plan have not beendeveloped.

Both "as-is" and "to-be" environments are described in No According to USDA's EA division, the "as-is" environment isterms of business, performance, information/data, described in terms of business and service/application.application/service, and technology. However, the "as-is" environment is not described in terms of

performance, information/data, and technology and the "to-be" descriptions have not been developed.

These descriptions address security. Partial According to USDA's EA division, it has begun developing asecurity architecture. However, the security architecture isstill under development and does not address all therequisite descriptions.

Organization CI0 has approved current version of EA. YesCommittee or group representing the enterprise or the Yesinvestment review board has approved current version ofEA.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managing changeWritten and approved organization policy exists for IT Partial USDA has a policy that encourages IT investmentinvestment compliance with EA. compliance with the EA. However, the policy does not

explicitly require IT investment compliance with the EA.

Process exists to formally manage EA change. Partial USDA has assigned responsibility for formally managing EAchange to a board and has begun to develop a process toformally manage EA change. However, evidence of thisprocess was not provided.

EA is integral component of IT investment management Partial USDA provided documentation indicating that EA is anprocess. integral component of the IT investment management

process. However, the EA does not include a sequencingplan to guide such investments.

EA products are periodically updated. Yes

Page 65 Page 65GAO-06-83 1 Enterprise Architecture




IT investments comply with EA. Partial According to USDA EA officials, IT investments comply withthe EA. However, documentation provided to GAO did notclearly indicate that all IT investments comply with the EA.

Organization head has approved current version of EA. YesReturn on EA investment is measured and reported. No According to USDA EA officials, they are beginning to

measure and report return on EA investment. However,documentation indicated that these reports address theintegration of applications as a result of specific initiativesthat are independent of the EA.

Compliance with EA is measured and reported. Partial Compliance with USDA component agency architectures,which are segments of the departmentwide EA, is measuredand reported. However, USDA did not providedocumentation that compliance with the department EA as awhole is measured and reported.

Source: GAO analysis of agency provided data.



Department of the Air Force Table .17 shows the Air Force's satisfaction of framework elements inversion 1.1 of GAO's EAMMF.

Table 17: Department of the Air Force Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA management foundationAdequate resources exist. YesCommittee or group representing the enterprise is Partial The Air Force has three committees representing theresponsible for directing, overseeing, and approving enterprise, each of which is responsible for directing,EA. overseeing, and approving one of three segments of the EA.

However, the charter for one of these committees is notapproved and no evidence was provided to support that thiscommittee has conducted any meetings.

Program office responsible for EA development and Partial A program office responsible for EA development andmaintenance exists. maintenance exists. However, the directive establishing this

office has not been approved.Chief architect exists. YesEA being developed using a framework, methodology, and Partial The Air Force uses a framework and automated tool toautomated tool. develop their EA. However, our analysis of the Department of

Defense Architecture Framework (DODAF), which was citedas the Air Force's EA methodology, determined that it is notan EA methodology.

EA plans call for describing 'as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of business, Yesperformance, information/data, application/service, andtechnology.EA plans call for business, performance, information/data, Yesapplication/service, and technology to address security.EA plans call for developing metrics to measure EA Partial The Air Force plans to develop metrics to measure EAprogress, quality, compliance, and return on investment, progress, compliance, and return on investment. However,

plans to develop metrics to measure EA quality were notprovided.

Stage 3: Developing EA productsWritten and approved organization policy exists for EA Yesdevelopment.EA products are under configuration management. No. According to Air Force Architecture Policy and Guidance

Office officials, EA products are not under configurationmanagement.





EA products describe or will describe "as-is" environment, Yes"to-be" environment, and sequencing plan.Both "as-is" and "to-be" environments are described or Yeswill be described in terms of business, performance,information/data, application/service, and technology.These descriptions address or will address security. YesProgress against EA plans is measured and reported. Partial The Air Force has limited reporting of EA progress. However,

progress is not measured and reported relative to an EAprogram plan.

Stage 4: Completing EA productsWritten and approved organization policy exists for EA Yesmaintenance.EA products and management processes undergo No According to the Air Force Architecture Policy and Guidanceindependent verification and validation. Office,*EA products and management processes have not

undergone independent verification and validation.EA products describe "as-is" environment, "to-be" No According to the Air Force Architecture Policy and Guidanceenvironment, and sequencing plan. Office, EA products describe the "as-is" environment and "to-

be" environment but do not describe a sequencing plan.Both "as-is" and "to-be" environments are described in Yesterms of business, performance, information/data,application/service, and technology.These descriptions address security. No According to the Air Force Architecture Policy and Guidance

Office, the "as-is" and "to-be" descriptions do not addresssecurity.

Organization CIO has approved current version of EA. YesCommittee or group representing the enterprise or the No According to the Air Force Architecture Policy and Guidanceinvestment review board has approved current version of Office, a committee or group representing the enterprise hasEA. not approved the current version of the EA.Quality of EA products is measured and reported. No According to the Air Force Architecture Policy and Guidance

Office, quality of EA products is not measured and reported.Stage 5: Leveraging the EA for managing changeWritten and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. Partial According to the Air Force Architecture Policy and Guidance

Office, a process exists to formally manage EA change.However, the process by which changes are made to the EAis not documented.

EA is integral component of IT investment management No According to the Air Force Architecture Policy and Guidanceprocess. Office, EA is not currently an integral component of the IT

investment management process.EA products are periodically updated. Yes

Page 6S Page 68GAO-06-831 Enterprise Architecture




IT investments comply with EA. Partial According to the Air Force Architecture Policy and GuidanceOffice, IT investments comply with the EA. However, nodocumentation of IT investment compliance with the AirForce EA was provided.

Organization head has approved current version of EA. No According to the Air Force Architecture Policy and GuidanceOffice, the organization head has not approved the currentversion of the EA.

Return on EA investment is measured and reported. No According to the Air Force Architecture Policy and GuidanceOffice, return on EA investment is not measured andreported.

Compliance with EA is measured and reported. Partial According to the Air Force Architecture Policy and GuidanceOffice, compliance with the EA is measured and reported.However, compliance measurement and reporting is withrespect to only one segment of the EA.



*Appendix IVDetailed Assessments of IndividualDepartments and Agencies against Our EAManagement Maturity Framework

Department of the Army Table 18 shows Army's satisfacti on of framework elements in version 1. 1 ofGAO's EAMMF.

Table 18: Department of the Army Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. Partial According to Army officials, adequate tools exist. However,

according to these same officials, personnel resources aresomewhat inadequate and funding resources are veryinadequate.

Committee or group representing the enterprise is No According to Army officials, committees or groups addressresponsible for directing, overseeing, and approving EA. architecture issues. However, no committee or group has

specific responsibility for EA. Further, documentation did notinclude a charter or other description of a committee or grouprepresenting the enterprise that is responsible for directing,overseeing, and approving the EA.

Program office responsible for EA development and Partial According to Army officials, a program office responsible for EAmaintenance exists. development and maintenance exists. However, the program

office charter is not approved.Chief architect exists. YesEA being developed using a framework, methodology, Partial EA is being developed using a framework and automated tool.and automated tool. However, according to Army officials, the EA is not developed

using a methodology.EA plans call for describing "as-is" environment, "to-be" Partial According to Army officials, EA plans call for describing the "as-environment, and sequencing plan. is" environment, "to-be" environment, and sequencing plan.

However, these EA plans are not approved.EA plans call for describing enterprise in terms of Partial According to Army officials, EA plans call for describing thebusiness, performance, inform ation/data, enterprise in terms of business, performance, information/data,application/service, and technology. application/service, and technology. However, these EA plans

are not approved.EA plans call for business, performance, information/ No According to Army officials, EA plans call for business,data, application/service, and technology to address performance, info rmation/data, application/service, andsecurity, technology to address security. However, documentation of

these plans were not provided.EA plans call for developing metrics to measure EA Partial According to Army officials, EA plans call for developing metricsprogress, quality, compliance, and return on investment, to measure EA progress, quality, compliance, and return on

investment. However, documentation of plans for developingmetrics to measure compliance and return on investment werenot provided.




Stages and core elements Satisfied? determinationStage 3: Developing EA productsWritten and approved organization policy exists for EA Partial The Army policy for EA development is not approved.development.EA products are under configuration management. Partial Some but not all EA products are under configuration

management.EA products describe or will describe "as-is" Partial According to Army officials, EA products will describe the "as-environment, "to-be" environment, and sequencing is" environment, "to-be" environment, and sequencing plan.plan. However, plans to develop these EA descriptions are not

approved.Both "as-is" and "to-be" environments are described or Partial According to Army officials, EA plans call for describing thewill be described in terms of business, performance, enterprise in terms of business, performance, information/data,inform ation/data, application/service, and technology, application/service, and technology. However, these EA plans

are not approved.These descriptions address or will address security. No According to Army officials, EA plans call for business,

performance, inform atio n/data, application/service, andtechnology to address security. However, documentation ofthese plans was not provided.

Progress against EA plans is measured and reported. Partial Progress against plans is measured and reported for oneportion of Army EA. However, progress against EA plans is notmeasured and reported for other EA segments that are underdevelopment.

Stage 4: Completing EA productsWritten and approved organization policy exists for Partial The Army policy for EA maintenance is not approved.EA maintenance.EA products and management processes undergo No According to Army officials, EA products undergo review byindependent verification and validation, engineering boards and others. However, they did not provide

evidence that these reviews are independent, constituteverification and validation, and include EA managementprocesses.

EA products describe "as-is" environment, "to-be" No According to Army officials, the EA currently consists of aenvironment, and sequencing plan. partial description of the "as-is" environment. These officials

stated that a complete description of the "as-is" environment aswell as descriptions of the "to-be" environment and sequencingplan will be developed in the future.

Both "as-is" and "to-be" environments are described No According to Army officials, the "as-is" environment isin terms of business, performance, information/data, described in terms of technology. These officials stated thatapplication/service, and technology, descriptions of the "as-is" environment in terms of business,

performance, and inform ation/data as well as descriptions ofthe "to-be" architecture will be developed in the future.

These descriptions address security. No According to Army officials, business, performance,inform ation/d ata, application/service, and technologydescriptions address security. However, documentation ofthese descriptions was not provided.

Organization 0I0 has approved current version No According to Army officials, the organization chief informationof EA. officer has not approved the EA.



(Continued From Previous Page)

Stages and core elements

Committee or group representing the enterpriseor the investment review board has approved currentversion of EA.Quality of EA products is measured andreported.

GAO basis for partially satisfied or not satisfiedSatisfied? determination

No According to Army officials, a committee or group representingthe enterprise or the investment review board has not approvedthe current version of the EA.

Partial According to Army officials, the quality of EA products ismeasured and reported. Further, documentation indicated thata quality evaluation was performed. However, documentation ofthe evaluation results was not provided.

Stage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists No Army officials did not provide a written and approvedfor IT investment compliance with EA. organization policy that explicitly requires IT investment

compliance with the EA.Process exists to formally manage EA change. No Army officials did not provide evidence of a process to formally

manage EA change.EA is integral component of IT investment No Army officials did not provide documentation indicating that EAmanagement process. is an integral component of the IT investment management

process.EA products are periodically updated. No According to Army officials, EA products are not periodically

updated.IT investments comply with EA. No Army officials did not provide documentation indicating that IT

investments comply with the EA.Organization head has approved current version of No According to Army officials, the organization head has notEA. approved the current version of the EA.Return on EA investment is measured and reported. No Army officials did not provide documentation indicating that

return on EA investment is measured and reported.Compliance with EA is measured and reported. No Army officials did not provide documentation indicating that

compliance with EA is measured and reported.Source: GAO analysis of agency provided data.


Appendix IVDetailed Assessments of IndivridualDepartments and Agencies against Our EAManagement Maturity Framework

Department of Commerce Table 19 shows Commerce's satisfaction of framework elements in version1.1 of GAO's EAMMF.

Table 19: Department of Commerce Satisfaction of EAMMF

GAO basis for partially satisfied or not satisfiedStages and core elements Satisfied? determinationStage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the enterprise is Partial Commerce provided evidence that the Enterprise Architectureresponsible for directing, overseeing, and approving Advisory Group and the Enterprise Architecture Review BoardEA. are responsible for directing, overseeing, and approving EA.

However, the documentation did not indicate that either grouprepresents the enterprise.

Program office responsible for EA development and Yesmaintenance exists..Chief architect exists. YesEA being developed using a framework, Yesmethodology, and automated tool.EA plans call for describing "as-is" environment, "to- Yesbe" environment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, information/data,application/service, and technology.

EA plans call for business, performance, Yesinform ation/data, application/service, and technologyto address security.EA plans call for developing metrics to measure EA Yesprogress, quality, compliance, and return oninvestment.Stage 3: Developing EA productsWritten and approved organization policy exists for YesEA development.EA products are under configuration management. YesEA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencingplan.





Both "as-is" and "to-be" environments are described Yesor will be described in terms of business,performance, information/data, application/service,and technology.These descriptions address or will address security. YesProgress against EA plans is measured and Yesreported.Stage 4: Completing EA productsWritten and approved organization policy exists for YesEA maintenance.EA products and management processes undergo Yesindependent verification andvalidation.EA products describe "as-is" environment, "to-be" Yesenvironment, and sequencing plan.Both "as-is" and "to-be" environments are described Yesin terms of business, performance, information/data,application/service, and technology.These descriptions address security. YesOrganization CI0 has approved current version of YesEA.Committee or group representing the enterprise or Partial Commerce provided evidence that the EA Review Board hasthe investment review board has approved current approved the current version of the EA. However, theversion of EA. documentation did not indicate that the board represents the

enterprise.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for IT Partial Commerce provided evidence that includes general references toinvestment compliance with EA. IT investment compliance EA. However, the documentation,

including a draft policy, did not require IT investment compliancewith the EA.

Process exists to formally manage EA change. YesEA is integral component of IT investment Yesmanagement process.EA products are periodically updated. YesIT investments comply with EA. Partial Commerce provided evidence that some but not all IT

investments comply with the EA.Organization head has approved current version of YesEA.

Compliance with EA is measured and reported. YesSource: GAO analysis of agency provided data.



Department of Defense - Table 20 shows the BEA~s satisfaction of framework elements in version 1. 1Business Enterprise of GAO's EAMMF.Architecture

Table 20: DOD Business Enterprise Architecture Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA management.foundationAdequate resources exist. YesCommittee or group representing the enterprise is Yesresponsible for directing, overseeing, and approvingEA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, Yesmethodology, and automated tool.EA plans call for describing "as-is" environment, Yes"to-be" environment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, information/data,application/service, and technology.EA plans call for business, performance, Partial According to SEA officials, the BEA will address security asinform ation/data, application/service, and evidenced by the fact that the GIG extends to all DOD missionstechnology to address security, areas, including the business mission area. However, documented

plans for how and when this will be accomplished for the businessmission were not provided.

EA plans call for developing metrics to measure EA Partial EA plans call for developing metrics to measure EA progress,progress, quality, compliance, and return on quality, and compliance. However, according to the chief architect,investment. EA plans do not call for developing metrics to measure return on

investment.Stage 3: Developing architecture productsWritten and approved organization policy exists for YesEA development.EA products are under configuration management. Partial According to BEA officials, EA products are under configuration

management, consistent with the configuration management planthey provided. However, no documentation was provided to showthat BEA is complying with this plan.




Stages and core elements Satisfied? determinationEA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencingplan.Both "as-is" and "to-be" environments are described Yesor will be described in terms of business,performance, information/data, application/service,and technology.These descriptions address or will address security. Partial According to BEA officials, the BEA will address security as

evidenced by the fact that the GIG extends to all DOD missionsareas, including the business mission area. However, documentedplans for how and when this will be accomplished for the businessmission were not provided.

Progress against EA plans is measured and Yesreported.Stage 4: Completing architecture productsWritten and approved organization policy exists for YesEA maintenance.EA products and management processes undergo Yesindependent verification and validation.EA products describe "as-is" environment, "to-be" No EA products describe the "to-be" environment and sequencingenvironment, and sequencing plan. plan. However, EA products do not yet describe the "as-is"

environment.Both "as-is" and "to-be" environments are described No The "to-be" environment is described in the requisite terms.in terms of business, performance, information/data, However, the "as-is" environment is not yet described.application/service, and technology.These descriptions address security. No According to the chief architect, these descriptions do not address

security.Organization C10 has approved current version of YesEA.Committee or group representing the enterprise or Yesthe investment review board has approved currentversion of EA.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for YesIT investment compliance with EA.Process exists to formally manage EA change. Partial According to BEA officials, a process exists to formally manage

EA change. However, BEA officials provided a configurationmanagement plan, which begins to address EA changemanagement, but did not provide evidence that EA changemanagement processes are fully documented.

EA is integral component of IT investment Yesmanagement process.




Stages and core elements Satisfied? determinationEA products are periodically updated. YesIT investments comply with EA. Partial According to the chief architect, IT investments comply with the

EA to some or little extent. However, BEA officials providedevidence that some investments have been assessed for theircompliance with the BEA.

Organization head has approved current version of YesEA.Return on EA investment is measured and reported. No According to the chief architect, return on EA investment is not

measured and reported.Compliance with EA is measured and reported. Yes



Appendix IVDe tailed Assessments of IndividualDepartments and Agencies against Our EAManagement Maturity Framework

Department of Defense - Table 21 shows the GIG's satisfaction of framework elements in version 1.1Global Information Grid of GAO's EAMMF.

Table 21: DOD Global Information Grid Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the Yesenterprise is responsible for directing, overseeing,and approving EA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Yesand automated tool.EA plans call for describing "as-is" environment, "to- Yesbe" environment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, inform ation/data,application/service, and technology.EA plans call for business, performance, Yesinformation/data, application/service, and technologyto address security.EA plans call for developing metrics to measure and Partial A GIG official provided plans that call for developing metrics toreport EA progress, quality, compliance, and return measure and report EA progress, quality, and compliance.on investment. However, plans that call for developing metrics to measure and

report return on EA investment were not provided.Stage 3: Developing EA productsWritten and approved organization policy exists for YesEA development.EA products are under configuration management. Partial According to a GIG official, EA products are under configuration

management and they provided documentation of configurationmanagement procedures. However, the documentation did notdescribe detailed steps to ensure the integrity and consistency ofEA products.

EA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencingplan.





Both "as-is" and "to-be" environments are described Yesor will be described in terms of business,performance, information/data, application/service,and technology.These descriptions address or will address security. YesProgress against EA plans is measured and Partial According to a GIG official, progress against EA plans isreported. measured and reported. However, the evidence provided

consisted of (1) EA plans but no reports of progress relative tothose plans and (2) contractor progress reports that did not relateto the EA plans.

Stage 4: Completing EA productsWritten and approved organization policy exists for YesEA maintenance.EA products and management processes undergo Partial According to a GIG official, EA products and managementindependent verification and validation, processes undergo independent verification and validation.

However, the evidence provided consisted of reports on a subsetof EA products and processes.

EA products describe "as-is" environment, "to-be" Partial According to a GIG official, EA products describe the "as-is"environment, and sequencing plan. environment and "to-be" environment. However, only one of six

planned sequencing plans has been completed.Both "as-is" and "to-be" environments are described Yesin terms of business, performance, information/data,application/service, and technology.These descriptions address security. YesOrganization CI0 has approved current version of YesEA.Committee or group representing the enterprise or Yesthe investment review board has approved currentversion of EA.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for YesIT investment compliance with EA.Process exists to formally manage EA change. YesEA is integral component of IT investment Partial According to a GIG official, EA is an integral component of the ITmanagement process. investment management process. However, five of six

sequencing plans to guide IT investments have not beencompleted.

EA products are periodically updated. YesIT investments comply with EA. Partial According to a GIG official, IT investments comply with the EA.

However, documentation indicated that IT investments complywith the Information Assurance portion of the GIG.




Stages and core elements Satisfied?

Organization head has approved current version of YesEA.Return on EA investment is measured and reported. No

Compliance with EA is measured and reported. Partial

GAO basis for partially satisfied or not satisfieddetermination

Source: GAO analysis of ager

According to a GIG official, return on EA investment is notmeasured and reported.According to a GIG official, compliance with the EA is measuredand reported. However, documentation indicated that complianceis measured and reported relative to a portion of the EA.

ncy provided data.


Appendix IVDetailed Assessments of IndividualDepartments and Agencies against Our EAManagement Maturity Framewvork

Department of Education Table 22 shows Education's satisfaction of framnework.elemnents in version1.1 of GAO's EAMME

Table 22: Department of Education Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the enterprise is Partial According to the chief architect, the EA executive steeringresponsible for directing, overseeing, and approving committee is responsible for directing and overseeing the EAEA. and the investment review board is responsible for approving

the EA. However, the investment review board charter does notdiscuss approving the EA and no documentation of investmentreview board approval was provided.

Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Yesand automated tool.EA plans call for describing "as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, information/data,application/service, and technology.EA plans call for business, performance, Yesinfo rmat ion/data, application/service, and technologyto address security.EA plans call for developing metrics to measure EA Yesprogress, quality, compliance, and return on investment.Stage 3: Developing EA productsWritten and approved organization policy exists for EA Yesdevelopment.EA products are under configuration management. YesEA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencing plan.Both "as-is" and "to-be" environments are described or Yeswill be described in terms of business, performance,information/data, application/service, and technology.These descriptions address or will address security. Yes




Stages and core elements

Progress against EA plans is measured and reported.

.Stage 4: Completing EA productsWritten and approved organization policy exists for EAmaintenance.

__GAO basis for partially satisfied or not satisfiedSatisfied? determination

Yes

Yes

EA products and management processes undergo Partial Some EA products have undergone independent verificationindependent verification and validation, and validation. However, according to the chief architect, EA

management processes have not undergone independentverification and validation.

EA products describe "as-is" environment, "to-be" Yesenvironment, and sequencing plan.Both "as-is" and "to-be" environments are described in Yesterms of business, performance, information/data,application/service, and technology.These descriptions address security. YesOrganization 0I0 has approved current version of EA. YesCommittee or group representing the enterprise or the Partial According to the chief architect, the investment review boardinvestment review board has approved current version of has approved the current version of the EA. However, noEA. documentation of investment review board approval was

provided.Quality of EA products is measured and reported. YesSta ge 5: Leveraging the EA for managing changeWritten and approved organization policy Yesexists for IT investment compliance with EA.Process exists to formally manage EA change. YesEA is integral component of IT investment management Yesprocess.EA products are periodically updated. YesIT investments comply with EA. YesOrganization head has approved current No According to the chief architect, the organization head has notversion of EA. explicitly approved the current version of the EA.Return on EA investment is measured and reported. Partial Return on EA investment is measured and reported. However,

this activity is limited to one EA segment.Compliance with EA is measured and reported. Yes



Appendix IIIDetailed Assessments of IndividualDepartments and Agencies against Our EAManagement Maturity Framework

Department of Energy Table 23 shows Energy's satisfaction of framework elements in version 1. 1of GAO's EAMMF.

Table 23: Department of Energy Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the enterprise is Yesresponsible for directing, overseeing, and approvingEA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Yesand automated tool.EA plans call for describing "as-is" environment, "to- Yesbe" environment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, information/data,application/service, and technology.EA plans call for business, performance, Yesinformation/data, application/service, and technologyto address security.EA plans call for developing metrics to measure and Partial EA plans call for developing metrics to measure and report EAreport EA progress, quality, compliance, and return on progress, quality, and compliance. However, they do not call forinvestment. developing metrics to measure and report EA return on

investment.Stage 3: Developing EA productsWritten and approved organization policy exists for EA No According to the chief architect, a written and approveddevelopment, organization policy for EA development does not exist.EA products are under configuration management. YesEA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencingplan.Both "as-is" and "to-be" environments are described or Yeswill be described in terms of business, performance,information/data, application/service, and technology.These descriptions address or will address security. Yes





Progress against EA plans is measured and reported. YesStage 4: Completing EA productsWritten and approved organization policy exists for No According to the chief architect, a written and approvedEA maintenance. organization policy for EA maintenance does not exist.EA products and management processes undergo Yesindependent verification and validation.EA products describe "as-is" environment, "to-be" Yesenvironment, and sequencing plan.Both "as-is" and "to-be" environments are described Yesin terms of business, performance, information/data,application/service, and technology.These descriptions address security. YesOrganization CIO has approved current version of YesEA.Committee or group representing the enterprise or Yesthe investment review board has approved currentversion of EA.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. Partial According to the chief architect, a process exists to formally

manage EA change. However, the process does not includespecific steps to be followed.

EA is integral component of IT investment Yesmanagement process.EA products are periodically updated. YesIT investments comply with EA. Partial According to the chief architect, IT investments comply with the

EA to a "great extent." However, evidence provided to GAO showsthat some IT investments do not meet Energy's EA compliancecriteria.

Organization head has approved current version of No According to the chief architect, the organization head has notEA. approved the current version of the EA.Return on EA investment is measured and reported. No According to the chief architect, return on EA investment is not





Department of Health and Table 24 shows HHS's satisfaction of framework elements in version 1. 1 ofHuman Services GAO's EAMMF.

Table 24: Department of Health and Human Services Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the enterprise is Partial The HHS CI0 council is responsible for directing, overseeing, andresponsible for directing, overseeing, and approving approving EA. However, a charter for the council is underEA. development.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Yesand automated tool.EA plans call for describing "as-is" environment, Yes"to-be" environment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, inform ation/data,application/service, and technology.EA plans call for business, performance,. Yesinform ation/data, application/service, and technologyto address security.EA plans call for developing metrics to measure and Yesreport EA progress, quality, compliance, and return oninvestment.Stage 3: Developing EA productsWritten and approved organization policy exists for EA Yesdevelopment.EA products are under configuration management. YesEA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencingplan.Both "as-is" and "to-be" environments are described Yesor will be described in terms of business, performance,information/data, application/service, and technology.These descriptions address or will address security. Yes





Progress against EA plans is measured and reported. YesStage 4: Completing EA productsWritten and approved organization policy exists for EA Yesmaintenance.EA products and management processes undergo No EA products and management processes have not undergoneindependent verification and validation, independent verification and validation.EA products describe "as-is" environment, "to-be" Partial EA products describe the "as-is" environment and "to-be"environment, and sequencing plan. environment. However, a sequencing plan has been developed for

some EA segments but has not been completed.Both "as-is" and "to-be" environments are described in Partial Both "as-is" and "to-be" environments are described. However, theterms of business, performance, inform ation/data, chief architect indicated that HHS plans to develop theapplication/service, and technology, -descriptions in more detail.These descriptions address security. YesOrganization CI0 has approved current version of EA. No According to the chief architect, the EA has not been approved by

the chief information officer.Committee or group representing the enterprise or No According to the chief architect, the current EA has not beenthe investment review board has approved current approved by a committee or group representing the enterprise.version of EA.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. YesEA is integral component of IT investment Partial HHS provided evidence that EA is an integral component of the ITmanagement process. investment management process. However, a sequencing plan to

guide IT investments has not been completed.EA products are periodically updated. YesIT investments comply with EA. YesOrganization head has approved current version of No According to the chief architect, the organization head has notEA. approved the current version of the EA.Return on EA investment is measured and reported. No According to the chief architect, HHS plans to measure and report

return on EA investment. However, return on EA investment is notcurrently measured and reported.



Appendix IVDetailed Assessments of IndividualDepartments and Agencies against Our EAManagement Maturity Framewvork

Department of Homeland Table 25 shows DHS's satisfaction of framework elements in version 1. 1 ofSecurity GAO'sE.AMMF.

Table 25: Department of Homeland Security Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the enterprise is Yesresponsible for directing, overseeing, and approvingEA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Yesand automated tool.EA plans call for describing "as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, information/data,application/service, and technology.EA plans call for business, performance, Yesinformation/data, application/service, and technologyto address security.EA plans call for developing metrics to measure EA Yesprogress, quality, compliance, and return oninvestment.Stage 3: Developing EA productsWritten and approved organization policy exists for EA Yesdevelopment.EA products are under configuration management. Partial According to the chief architect, EA products are under

configuration management. However, the configurationmanagement guidance documents are in draft and not approved.

EA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencingplan.Both "as-is" and "to-be" environments are described or Yeswill be described in terms of business, performance,inform ation/data, application/service, and technology.





These descriptions address or will address security. YesProgress against EA plans is measured and reported. Yes

Stage 4: Completing EA productsWritten and approved organization policy exists for EA Yesmaintenance.EA products and management processes undergo No According to the chief architect, EA products and managementindependent verification and validation, processes undergo independent verification and validation.

However, the contractor that reviewed EA products andprocesses was not independent and the reviews did not includeverification and validation.

EA products describe "as-is" environment, "to-be" Partial According to the chief architect, EA products describe the "as-is"environment, and sequencing plan. environment, "to-be" environment, and sequencing plan.

However, the sequencing plan is in draft and not approved.Both "as-is" and "to-be" environments are described in Yesterms of business, performance, inform ation/d ata,application/service, and technology.These descriptions address security. YesOrganization 0I0 has approved current version of EA. YesCommittee or group representing the enterprise or the Yesinvestment review board has approved current versionof EA.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. Partial According to the chief architect, DHS has a process to formally

manage EA change. However, the policy that describes EAchange management is being revised and is not approved.

EA is integral component of IT investment Yesmanagement process.EA products are periodically updated. YesIT investments comply with EA. Partial DHS has an IT investment management process that requires

investment compliance with the EA. However, the IT investmentmanagement process does not include a methodology withdetailed compliance criteria.

Organization head has approved current No According to the chief architect, the agency head has notversion of EA. approved the current version of the EA.Return on EA investment is measured and reported. No According to the chief architect, return on EA investment is not




Appendix [VDetailed Assessments of IndividualDepartments and Agencies against Our EAManagement Maturity Framework

Department of Housing and Table 26 shows HUD's satisfaction of framework elements in version 1. 1 ofUrban Development GAO's EAMMF.

Table 26: Department of Housing and Urban Development Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the enterprise is Yesresponsible for directing, overseeing, and approvingEA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, Yesmethodology, and automated tool.EA plans call for describing "as-is" environment, "to- Yesbe" environment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, information/data,application/service, and technology.EA plans call for business, performance, Yesinformation/data, application/service, andtechnology to address security.EA plans call for developing metrics to measure EA Yesprogress, quality, compliance, and return oninvestment.Stage 3: Developing EA productsWritten and approved organization policy exists for YesEA development.EA products are under configuration management. YesEA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencingplan.Both "as-is" and "to-be" environments are described Yesor will be described in terms of business,performance, information/data, application/service,and technology.

Page 89 Page 89GAO-OG-831 Enterprise Architecture


(Continued From Previous Page)GAO basis for partially satisfied or not satisfieddeterminationStages and core elements

These descriptions address or will address security._Progress against EA plans is measured andreported.Stage 4: Completing EA productsWritten and approved organization policy exists forEA maintenance.

Satisfied?

YesYes

Yes

EA products and management processes undergo No According to the chief architect, EA products and managementindependent verification and validation, processes undergo verification and validation. However, the

verification and validation were not independent.EA products describe "as-is" environment, "to-be" Yesenvironment, and sequencing plan.Both "as-is" and "to-be" environments are described Yesin terms of business, performance, information/data,application/service, and technology.These descriptions address security. Partial HUD provided evidence that it is addressing security in the

requisite descriptions. However, according to HUD officials theyrecognize the need to further develop these security descriptions.

Organization CI0 has approved current version of YesEA.Committee or group representing the enterprise or Yesthe investment review board has approved currentversion of EA.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for YesIT investment compliance with EA:Process exists to formally manage EA change. YesEA is integral component of IT investment Yes.management process.EA products are periodically updated. YesIT investments comply with EA. YesOrganization head has approved current version of YesEA.Return on EA investment is measured and reported. YesCompliance with EA is measured and reported. Yes


Page 90GAO-06-831 Enterprise ArchitecturePage 90


The Department of the Table 27 shows DOI's satisfaction of framework elements in version 1.1 ofInterior GAO's EAMMF.

Table 27: Department of the Interior Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the enterprise is Yesresponsible for directing, overseeing, and approvingEA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Yesand automated tool.EA plans call for describing "as-is" environment, Yes"to-be" environment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, information/data,application/service, and technology.EA plans call for business, performance, Yesinformation/data, application/service, andtechnology to address security.EA plans call for developing metrics to measure Yesand report EA progress, quality, compliance, andreturn on investment.Stage 3: Developing EA productsWritten and approved organization policy exists for YesEA development.EA products are under configuration management. YesEA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencingplan.Both "as-is" and "to-be" environments are described Yesor will be described in terms of business, performance,info rmat ion/data, application/service, and technology.These descriptions address or will address security. Yes





Progress against EA plans is measured and Yesreported.Stage 4: Completing EA productsWritten and approved organization policy exists for YesEA maintenance.EA products and management processes undergo No According to the chief architect, EA products and managementindependent verification and validation, processes undergo independent verification and validation.

However, the evidence provided did not show that EA productand process quality were assessed.

EA products describe "as-is" environment, "to-be" Yesenvironment, and sequencing plan.Both "as-is" and "to-be" environments are described Yesin terms of business, performance, information/data,application/service, and technology.These descriptions address security. YesOrganization CI0 has approved current version of YesEA.Committee or group representing the enterprise or Yesthe investment review board has approved currentversion of EA.Quality of EA products is measured and Yesreported.Stage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for YesIT investment compliance with EA.Process exists to formally manage EA change. YesEA is integral component of IT investment Yesmanagement process.EA products are periodically updated. YesIT investments comply with EA. YesOrganization head has approved current version Yesof EA.Return on EA investment is measured and reported. YesCompliance with EA is measured and reported. Yes




Department of Justice Table 28 shows DOJ's satisfaction of framework elements in version 1.1 ofGAO's EAMME

Table 28: Department of Justice Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in Stage 1Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the enterprise is Yesresponsible for directing, overseeing, and approvingEA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Yesand automated tool.EA plans call for describing "as-is" environment, "to- Yesbe" environment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, information/data,application/service, and technology.EA plans call for business, performance, Yesinfo rmation/data, application/service, and technologyto address security.EA plans call for developing metrics to measure EA Yesprogress, quality, compliance, and return oninvestment.Stage 3: Developing EA productsWritten and approved organization policy exists for YesEA development.EA products are under configuration management. YesEA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencingplan.Both "as-is" and "to-be" environments are described Yesor will be described in terms of business,performance, information/data, application/service,and technology.These descriptions address or will address security. Yes





Progress against EA plans is measured and reported. YesStage 4: Completing EA productsWritten and approved organization policy exists for YesEA maintenance.EA products and management processes undergo No According to the chief architect, EA products and managementindependent verification and validation, processes have not undergone independent verification and

validation.EA products describe "as-is" environment, "to-be" Yesenvironment, and sequencing plan.Both "as-is" and "to-be" environments are described Yesin terms of business, performance, information/data,application/service, and technology.These descriptions address security. YesOrganization CI0 has approved current version of YesEA.Committee or group representing the enterprise or Partial According to the chief architect, a committee or groupthe investment review board has approved current representing the enterprise has approved the current version ofversion of EA. the EA. However, no documentation of committee approval was

provided.Quality of EA products is measured and reported. Partial According to the chief architect, DOJ uses the 0MB EA

assessment tool to measure and report product quality. However,no report documenting a quality assessment or results wasprovided.

Stage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. YesEA is integral component of IT investment Yesmanagement process.EA products are periodically updated. YesIT investments comply with EA. Partial D0J has an IT investment management process that requires

investment compliance with the EA. However, the IT investmentmanagement process does not include a methodology todetermine compliance.

Organization head has approved current version of No According to the chief architect, the organization head has notEA. approved the current version of the EA.Return on EA investment is measured and reported. No DOJ did not provide documentation describing how return on EA

investment is measured and reported and no sample reports wereprovided.

Compliance with EA is measured and reported. Partial DOJ has begun to measure and report IT investment compliancewith the EA. However, no compliance reports were provided.




Department of Labor Table 29 shows Labor's satisfaction of framework elements in version 1. 1 ofGAO's EAMMF.

Table 29: Department of Labor Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA management foundationAdequate resources exist. YesCommittee or group representing the enterprise is Yesresponsible for directing, overseeing, and approvingEA.Program office responsible for EA development and YesmAintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Yesand automated tool.EA plans call for describing "as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, inform ation/data,application/service, and technology.EA plans call for business, performance, information/ Yesdata, application/service, and technology to addresssecurity.EA plans call for developing metrics to measure and Yesreport EA progress, quality, compliance, and return oninvestment.Stage 3: Developing EA productsWritten and approved organization policy exists for EA Yesdevelopment.EA products are under configuration management. YesEA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencingplan.Both "as-is" and "to-be" environments are described or Yeswill be described in terms of business, performance,information/data, application/service, and technology.These descriptions address or will address security. YesProgress against EA plans is measured and reported. Yes




Stages and core elements Satisfied? determinationStage 4: Completing EA productsWritten and approved organization policy exists for EA Yesmaintenance.EA products and management processes undergo No According to the EA program manager, EA products andindependent verification and validation. management processes undergo independent verification and

validation. However, the verification and validation wereperformed by a contractor that had also performed other EAprogram activities and therefore were not independent.

EA products describe "as-is" environment, "to-be" Yesenvironment, and sequencing plan.Both "as-is" and "to-be" environments are described in Yesterms of business, performance, information/data,application/service, and technology.These descriptions address security. YesOrganization CIO has approved current version of EA. YesCommittee or group representing the enterprise or the Yes'investment review board has approved current version ofEA.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. YesEA is integral component of IT investment management Yesprocess.EA products are periodically updated. YesIT investments comply with EA. Partial According to the EA program manager, IT investments comply

with the EA. However, the evidence provided did not verifythat IT investments are not approved unless they comply withthe EA.

Organization head has approved current version of EA. YesReturn on EA investment is measured and reported. No According to the EA program manager, return on EA

investment is not measured and reported.Compliance with EA is measured and reported. Partial According to the EA program manager, compliance with the

EA is measured and reported. However, evidence of EAcompliance reporting was not provided.




Department of the Navy Table 30 shows Navy's satisfaction of framework elements in version 1. 1 ofGAO's EAMMF.

Table 30: Department of the Navy Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. 'n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the enterprise is Partial According to the Navy CIO office, Navy has a committee that isresponsible for directing, overseeing, and approving responsible for directing, overseeing, and approving EA.EA. However, the Navy did not provide meeting minutes or other

documentation to verify that committee meetings have occurred.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, Partial The Navy uses a framework and automated tool to develop theirmethodology, and automated tool. EA. However, our analysis of the Department of Defense

Architecture Framework (DODAF), which was cited as theNavy's EA methodology, determined that the DODAF is not anEA methodology.

EA plans call for describing "as-is" environment, "to- Partial EA plans call for describing the "as-is" and "to-be" environments.be" environment, and sequencing plan. However, EA plans do not include describing a sequencing plan.EA plans call for describing enterprise in terms of Partial EA plans do not include a "to-be" environment description inbusiness, performance, information/data, terms of technology.application/service, and technology.EA plans call for business, performance, Yesinformation/data, application/service, and technologyto address security.EA plans call for developing metrics to measure EA Partial EA plans call for developing metrics to measure EA progress,progress, quality, compliance, and return on quality, and compliance. However, no evidence was providedinvestment. indicating plans to measure EA return on investment.Stage 3: Developing EA productsWritten and approved organization policy exists for YesEA development.EA products are under configuration management. Partial Navy provided evidence of their EA repository tools which Navy

officials said they use to track EA product configurationmanagement. However, the evidence did not describe detailedsteps that would ensure the integrity and consistency of EAproducts.





EA products describe or will describe "as-is" Partial EA plans call for describing the "as-is" and "to-be" environments.environment, "to-be" environment, and sequencing However, EA plans do not include describing a sequencing plan.plan.Both "as-is" and "to-be" environments are described Partial EA plans do not include a "to-be" environment description inor will be described in terms of business, performance, terms of technology.inform ation/data, application/service, and technology.These descriptions address or will address security. YesProgress against EA plans is measured and reported. YesStage 4: Completing EA productsWritten and approved organization policy exists for YesEA maintenance.EA products and management processes undergo No Navy indicated EA products and management processes areindependent verification and validation, subject to quality, integration, and verification reviews. However,

these reviews are not performed by independent entities.EA products describe "as-is" environment, "to-be" No EA products describe the "as-is" environment. However,environment, and sequencing plan. according to the Department of the Navy 0I0 office, their EA

products will include descriptions of the "to-be" environment andsequencing plan in the future.

Both "as-is" and "to-be" environments are described No The "as-is" environment is described in business,in terms of business, performance, information/data, information/data, application/service, and technology, but notapplication/service, and technology, performance terms. However, according to the Navy 010 office,

their "to-be" environment will be described in the future.These descriptions address security. No According to the Navy CI0 office, EA descriptions will explicitly

address security in the future.Organization 0I0 has approved current version of YesEA.Committee or group representing the enterprise or No According to the Navy 0I0 office, EA approvals are rendered bythe investment review board has approved current either the Assistant Secretary for Research and Development orversion of EA. a duly appointed representative, in addition to the Navy 0I0.

However, a committee or group representing the enterprise doesnot approve the EA.

Quality of EA products is measured and reported. Partial According to the Navy CI0 office, quality of EA products ismeasured and reported. However, Navy officials providedevidence that quality of EA products is measured and reportedfor the Marine Corps portion of the architecture, but did notprovide documentation that quality of Navy architecture productsare measured and reported.

Stage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for No Navy policy does not explicitly require IT investment complianceIT investment compliance with EA. with the EA.Process exists to formally manage EA change. No Navy officials did not provide evidence of a process to formally

manage EA change.




Stages and core elements Satisfied? determinationEA is integral component of IT investment Partial According to the Navy CI0 office, the department has begun tomanagement process. integrate EA into its IT investment management process.EA products are periodically updated. YesIT investments comply with EA. Partial According to the Navy CI0 office, IT investments comply with

the EA. However, documentation of IT investment compliancewith the EA was not provided.

Organization head has approved current version of No According to the Navy 010 office, the organization head has notEA. approved the EA.Return on EA investment is measured and reported. No According to the Navy 010 off ice, the department does not

currently measure and report return on EA investment.Compliance with EA is measured and reported. Partial Navy provided documentation indicating that compliance with

the EA is measured. However, sample measurement reportswere not provided.


Note: This analysis primarily focuses on the FORCEnet architecture because FORCEnet wasidentified by the Department of the Navy as the overall Department of Navy enterprise architecture.Additional information from the Marine Corps Integrated Architecture Picture (MCIAP) wasincorporated where appropriate.



Department of State Table 31 shows State's satisfaction of framework elements in version 1. 1 ofGM's EAMMF.

Table 31: Joint Enterprise Architecture Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the enterprise is Partial According to the chief architect, State and USAID areresponsible for directing, overseeing, and approving EA. developing a governance plan that will describe management

processes for directing, overseeing, and approving their EA.However, this plan is not approved.

Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Partial According to the chief architect, the EA is being developedand automated tool. using a framework, methodology, and automated tool.

However, the methodology does not describe specific stepsfor maintenance.

EA plans call for describing "as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms Yesof business, performance, information/data,application/service, and technology.EA plans call for business, performance, info 'rmation/data, Yesapplication/service, and technology to address security.EA plans call for developing metrics to measure EA Partial State and USAID have plans to measure and report EAprogress, quality, compliance, and return on Investment, progress, quality, and compliance. However, no plans for

developing metrics to measure EA return on investment wereprovided.

Stage 3: Developing EA productsWritten and approved organization policy exists for EA Yesdevelopment.EA products are under configuration management. Partial A draft configuration management plan for EA products has

been developed. However, this plan has not been approved.EA products describe or will describe "as-is" environment, Yes"to-be" environment, and sequencing plan.




Stages and core elements Satisfied? determinationBoth "as-is" and "to-be" environments are described or Yeswill be described in terms of business, performance,information/data, application/service, and technology.These descriptions address or will address security. YesProgress against EA plans is measured and reported. Partial State has limited reporting of EA progress. However,

progress is not measured and reported relative to an EAprogram plan.

Stage 4: Completing EA productsWritten and approved organization policy exists for EA Yesmaintenance.EA products and management processes undergo No According to the chief architect, EA products andindependent verification and validation. management processes have not undergone independent

verification and validation.EA products describe "as-is" environment, No EA products describe the "as-is" and "to-be" environments as"to-be" environment, and sequencing plan. well as high-level transition activities. However, a sequencing

plan for transitioning between the "as-is" and "to-be"environments is currently under development.

Both "as-is" and "to-be" environments are described in Yesterms of business, performance, information/data,application/service, and technology.These descriptions address security. YesOrganization CIO has approved current version Yesof EA.Committee or group representing the enterprise or the No According to the chief architect, a committee or groupinvestment review board has approved current version representing the enterprise has not approved the currentof EA. version of the EA.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. No According to the chief architect, a process to formally

manage EA change does not currently exist.EA is integral component of IT investment management Partial According to the chief architect, EA is an integral componentprocess. of the IT investment management process. However, the

sequencing plan to guide IT investments is currently underdevelopment.

EA products are periodically updated. YesIT investments comply with EA. YesOrganization head has approved current version of EA. No According to the chief architect, the organization head has

not approved the current version of the EA.Return on EA investment is measured and reported. No According to the chief architect, return on EA investment is

not measured and reported.





Compliance with EA is measured and reported. Partial According to the chief architect, compliance with the EA ismeasured and reported and officials provided a descriptionof how compliance is to be measured. However, nodocumentation demonstrating compliance reporting wasprovided.


Note: Department of State officials asked that we evaluate their agency's enterprise architecture baseden efforts to develop the Joint Enterprise Architecture, which is being developed by State and the U.S.Agency for International Development.


Appendix IVDetailed Assessments or IndividualDepartments and Agencies against Our EAManagement Maturity Framework

Department of Table 32 shows Transportation's satisfaction of framework elements inTransportation vers5ion 1.1 of GAO's EAMME

Table 32: Department of Transportation Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA management foundationAdequate resources exist. YesCommittee or group representing the enterprise is Yesresponsible for directing, overseeing, and approving EA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, and Partial Program officials stated that the EA is being developedautomated tool. using a framework, methodology, and automated tool.

However, the methodology and other documentation did notinclude steps for EA maintenance.

EA plans call for describing "as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of business, Yesperformance, information/data, application/service, andtechnology.EA plans call for business, performance, information/data, Yesapplication/service, and technology to address security.

EA plans call for developing metrics to measure EA Partial EA plans call for developing metrics to measure EA progressprogress, quality, compliance, and return on investment. and quality. However, EA plans do not call for developing

metrics to measure compliance and return on investment.

Stage 3: Developing EA productsWritten and approved organization policy exists for EA Yesdevelopment.EA products are under configuration management. Partial A configuration management plan for EA products is being

defined. However, this plan has not been approved.EA products describe or will describe "as-is" environment, Yes"to-be" environment, and sequencing plan.Both "as-is" and "to-be" environments are described or will Yesbe described in terms of business, performance,information/data, application/service, and technology.These descriptions address or will address security. Yes


*Appendix IVDetailed Assessments of IndividualDepartments and Agencies against Our EAManagement Maturity Framework



Progress against EA plans is measured and reported. YesStage 4: Completing EA productsWritten and approved organization policy exists for EA Yesmaintenance.EA products and management processes undergo No According to the chief architect, EA products andindependent verification and validation, management processes do not undergo independent

verification and validation.EA products describe "as-is" environment, "to-be" Yesenvironment, and sequencing plan.Both "as-is" and "to-be" environments are described in Yesterms of business, performance, inform ation/data,application/service, and technology.These descriptions address security. No The business, performance, information/data,

application/service, and technology descriptions do notaddress security.

Organization CI0 has approved current version of EA. No According to the chief architect, the organization CI0 hasnot approved the EA.

Committee or group representing the enterprise or the No According to the chief architect, a committee or groupinvestment review board has approved current version of representing the enterprise or the investment review boardEA. has not approved the current version of the EA.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. Partial According to the chief architect, a process exists to formally

manage EA change. However, the department providedevidence that a structure is in place to manage EA change,but a description of this process for formally managing EAchange was not provided.

EA is integral component of IT investment management Yesprocess.EA products are periodically updated. YesIT investments comply with EA. YesOrganization head has approved current version of EA. No According to the chief architect, the organization head has

not approved the current version of the EA.Return on EA investment is measured and reported. No According to the chief architect, return on EA investment is

not measured and reported.Compliance with EA is measured and reported. No According to the chief architect, compliance with the EA is

measured and reported. However, department officials didnot provide evidence that compliance with the EA ismeasured and reported.




Dep artment of the Treasury. Table 33 shows the Treasury's satisfaction of framework elements inversion 1.1 of GAO's EAMMF.

Table 33: Department of the Treasury Satisfaction of EAMMF

GAO basis for partially satisfied or not satisfiedStages and core elements Satisfied? determinationStage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA management foundationAdequate resources exist. YesCommittee or group representing the enterprise is* Partial Two committees are responsible for directing andresponsible for directing, overseeing, and approving EA. overseeing the EA. However, the committee charters do

not indicate that either committee is responsible forapproving the EA or represent the enterprise (i.e., includeexecutive-level representation from each line of business).

Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, and Yesautomated tool.EA plans call for describing "as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of business, Yesperformance, information/data, application/service, andtechnology.EA plans call for business, performance, information/data, Yesapplication/service, and technology to address security.EA plans call for developing metrics to measure and report Partial According to the EA program manager, EA plans call forEA progress, quality, compliance, developing metrics to measure and report EA progress,and return on investment. quality, and compliance. However, evidence provided did

not include plans for EA compliance or return oninvestment metrics.

Stage 3: Developing EA productsWritten and approved organization policy exists for YesEA development.EA products are under configuration management. YesEA products describe or will describe "as-is" environment, Yes"to-be" e~nvironment, and sequencing plan.Both "as-is" and "to-be" environments are described or will Yesbe described in terms of business, performance,inform ation/data, application/service, and technology.These descriptions address or will address security. Yes




Stages and core elements Satisfied? determinationProgress against EA plans is measured and reported. Partial According to the EA program manager, progress is

measured and reported against plans, including an EAprogram management plan. However, the EA programmanagement plan is in draft.

Stage 4: Completing EA productsWritten and approved organization policy exists for EA Yesmaintenance.EA products and management processes undergo No According to the EA program manager, EA products andindependent verification and validation. management processes have not undergone independent

verification and validation.EA products describe "as-is" environment, "to-be" Yesenvironment, and sequencing plan.Both "as-is" and "to-be" environments are described in terms Yesof business, performance, information/data,application/service, and technology.These descriptions address security. YesOrganization 0I0 has approved current version of EA. YesCommittee or group representing the enterprise or the No A committee or group representing the enterprise has not.investment review board has approved current version of EA. approved the current version of the EA.Quality of EA products is measured and reported. Partial According to the EA program manager, the quality of EA

products is measured and reported. Further, evidenceshowed that a quality evaluation was performed. However,evidence of the evaluation results was not provided.

Stage 5: Leveraging the EA for managingchange

*Written and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. YesEA is integral component of IT investment management Yesprocess.EA products are periodically updated. YesIT investments comply with EA. Partial According to the EA program manager, IT investments

comply with the EA. However, the documents provided didnot demonstrate that the IT investment managementprocess requires IT investments to comply with the EA.

Organization head has approved current version of EA. No . According to the EA program manager, the organizationhead has delegated approval of the EA to the C10, whohas approved the current version. However, the evidenceprovided did not explicitly show such a delegation.

Return on EA investment is measured and reported. No According to the EA program manager, return on EAinvestment is not measured and reported.




Department of Veterans Table 34 shows VA~s satisfaction of framework elements in version 1.1 ofAffairs GAO's EAMMF.

Table 34: Department of Veterans Affairs Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundation*Adequate resources exist. Yes

Committee or group representing the enterprise is Partial A committee representing the enterprise is responsible forresponsible for directing, overseeing, and approving directing and overseeing the EA. However, the committeeEA. charter does not indicate that the committee is responsible for

approving the EA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Yesand automated tool.EA plans call for describing "as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, information/data,application/service, and technology.EA plans call for business, performance, Yesinformation/data, application/service, and technologyto address security.EA plans call for developing metrics to measure and Partial Metrics to measure and report EA progress and quality havereport EA progress, quality, compliance, and return been developed. However, plans do not include developingon investment. metrics to measure and report EA compliance and return on

investment.

Written and approved organization policy exists for YesEA development.EA products are under configuration management. Partial According to the chief architect, EA products are under

configuration management, which is accomplished throughthe version control features of the EA repository. However,configuration management procedures did not define specificsteps that would ensure the integrity and consistency of EAproducts.




Stages and core elements Satisfied? determinationEA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencingplan.Both "as-is" and "to-be" environments are described or Yeswill be described in terms business, performance,information/data, application/service, and technology.These descriptions address or will address security. YesProgress against EA plans is measured and reported. YesStage 4: Completing EA productsWritten and approved policy exists for EA maintenance. YesEA products and management processes undergo No According to the chief architect, EA version 4.0 hasindependent verification and validation, undergone independent verification and validation. However,

no documentation to support this statement was provided.Further, plans for independent verification and validation ofEA management processes have not been implemented.

EA productsdescribe "as-is" environment, "to-be" Yesenvironment, and sequencing plan.Both "as-is" and "to-be" environments are described in Yesterms of business, performance, information/data,application/service, and technology.These descriptions address security. Partial According to VA officials, their security architecture addresses

all the requisite descriptions. However, evidence provided tosupport this statement shows that the security architecturedoes not address all of the requisite descriptions.

Organization CI0 has approved current version of EA. No According to the chief architect, the 010 delegated EAapproval authority to the chief architect. However, no evidenceto support this delegation was provided.

Committee or group representing the enterprise or No No evidence that a committee or group representing thethe investment review board has approved current enterprise has approved the current version of the EA wasversion of EA. provided.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. Partial According to the chief architect, the EA repository is used to

manage EA change. VA provided documentation describingthe repository and its contents. However, evidence provideddid not demonstrate that the repository is used to manage EAchange or include detailed steps for managing EA change.

EA is integral component of IT investment Yesmanagement process.EA products are periodically updated. YesIT investments comply with EA. Yes





Organization head has approved current version of No According to the chief architect, the organization headEA. delegated EA approval to the CI0, who delegated this

authority to the chief architect. However, no evidence tosupport these delegations was provided.

Return on EA investment is measured and reported. No According to the chief architect, return on EA investment ismeasured and reported. However, no evidence that the returnon EA investment is measured and reported was provided.

Compliance with EA is measured and reported. No According to the chief architect, compliance with the EA ismeasured and reported. However, no evidence that

*compliance with the EA is measured and reported wasprovided.


Page 109 Page109GAO-06-831 Enterprise Archiltecture

* Appendix IVDetailed Assessments of IndividualDepartments and Agencies against Our EAManagement Maturity Framework

Environmental Protection Table 35 shows EPA'S satisfaction of framework elements in version 1.1 ofAgency GAO's EAMMF.

Table 35: Environmental Protection Agency Satisfaction of EAMMF

GAO basis for partially satisfied or not satisfiedStages and core elements Satisfied?. determination

Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the enterprise is Yesresponsible for directing, overseeing, and approvingEA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Partial EA is being developed using a framework, methodology, andand automated tool. automated tool. However, the methodology does not include

steps for maintaining the architecture and has not been*approved.

EA plans call for describing "as-is" environment, Yes"to-be" environment, and sequencing plan. .EA plans call for describing enterprise in terms of Yesbusiness, performance, inform ation/data,application/service, and technology.EA plans call for business, performance, Yesinform ation/data, application/service, and technology toaddress securityýEA plans call for developing metrics to measure EA Yesprogress, quality, compliance, and return oninvestment.Stage 3: Developing EA productsWritten and approved organization policy exists for EA Yesdevelopment.EA products are under configuration management. YesEA products describe or will describe "as-is" .Yesenvironment, "to-be" environment, and sequencingplan.Both "as-is" and "to-be" environments are described or Yeswill be described in terms of business, performance,information/data, application/service, and technology.




Stages and core elementsThese descriptions address or will address security.

Progress against EA plans is measured and reported.

Stage 4: Completing EA productsWritten and approved organization policy exists for EAmaintenance.EA products and management processes undergoindependent verification and validation.

EA products describe "as-is" environment, "to-be"environment, and sequencing plan.Both "as-is" and "to-be" environments are described interms of business, performance, information/data,application/service, and technology.These descriptions address security.Organization CI0 has approved current version of EA.Committee or group representing the enterprise or theinvestment review board has approved current versionof EA.Quality of EA products is measured and reported.Stage 5: Leveraging the EA for managingchange

GAO basis for partially satisfied or not satisfieddeterminationSatisfied?

Yes

Partial EPA has limited reporting of EA progress. However, progressis not measured and reported relative to an EA program plan.

Yes

No According to the chief architect, EA products andmanagement processes have not undergone independentverification and validation.

Yes

Yes

YesYesYes

Yes

Written and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. Partial According to th~e chief architect, a process exists to formally

manage EA change. However, evidence that the process hasbeen implemented was not provided and the process is notapproved.

EA is integral component of IT investment Yesmanagement process.EA products are periodically updated. YesIT investments comply with EA. Pdrtial An IT investment management process exists and the process

considers investment compliance with the EA. However,evidence that the process has been implemented was notprovided.

Organization head has approved current version of No According to the chief architect, the organization head has notEA. approved the current version of the EA.Return on EA investment is measured and reported. No According to the chief architect, return on EA investment is not

measured and reported.

Page III Page 111GAO-06-831 Enterprise Architecture



Stages and core elements Satisfied? determinationCompliance with EA is measured and reported. Partial According to the chief architect, compliance with EA is

measured and reported and documentation describes howcompliance is to be measured. However, reports documentingmeasurement were not provided.




General Services Table 36 shows GSA's satisfaction of framework elements in version 1.1 ofAdministration GAO's EAMMF.

Table 36: General Services Administration Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the enterprise is Yesresponsible for directing, overseeing, and approvingEA.Program office responsible for EA developrrient and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Partial According to the chief technology officer (CTO), the EA is beingand automated tool. developed using a framework, methodology, and automated

tool. However, the methodology does not include specific stepsto maintain the architecture.

EA plans call for describing "a s-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, information/data,application/service, and technology.EA plans call for business, performance, Partial Draft EA plans call for business, performance,inform ation/data, application/service, and technology information/data, application/service, and technologyto address security, descriptions to address security. However, these plans are not

approved.EA plans call for developing metrics to measure EA Partial According to the CTO, EA plans call for developing metrics.progress, quality, compliance, and return on However, these plans do not specifically address EA progress,investment. quality, compliance, and return on investment.Stage 3: Developing EA productsWritten and approved organization policy exists for YesEA development.EA products are under configuration management.' Partial According to the CTO, EA products are under configuration

management. However, the configuration management plan isbeing defined and has not been approved.

EA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencingplan.





Both "as-is" and "to-be" environments are described or Yeswill be described in terms of business, performance,information/data, application/service, and technology.These descriptions address or will address security. Partial Draft EA plans call for business, performance,

information/data, application/service, and technologydescriptions to address security. However, these plans are notapproved.

Progress against EA plans is measured and reported. No According to the CTO, progress against EA plans is notmeasured and reported.

Stage 4: Completing EA productsWritten and approved organization policy exists for EA Yesmaintenance.EA products and management processes undergo No According to the CTO, EA products and managementindependent verification and validation, processes have not undergone independent verification and

validation.EA products describe "as-is" environment, "to-be" Partial EA products describe both the "as-is" environment and the "to-environment, and sequencing plan. be" environment. However, the sequencing plan has not been

approved.Both "as-is" and "to-be" environments are described Partial Both the "as-is" and "to-be" environments are described inin terms of business, performance, info rmation/data, terms of business, application/service, and technology.application/service, and technology. However, according to the OTO, descriptions of performance

and information/data have not been completed.These descriptions address security. No According to the CTO, business, performance,

information/data, application/service, and technologydescriptions do not address security.

Organization CIO has approved current version of YesEA.Committee or group representing the enterprise or Yesthe investment review board has approved currentversion of EA.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managing changeWritten and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. No According to the OTO, no process to formally manage EA

change exists.EA is integral component of IT investment Partial According to the CTO, EA is an integral component of the ITmanagement process. investment management process. However, a sequencing plan

to guide IT investments has not been approved.EA products are periodically updated. YesIT investments comply with EA. Yes

Organization head has approved current version of YesEA.




Stages and core elements Satisfied? determinationReturn on EA investment is measured and reported. No According to the OTO, return on EA investment is not

measured and reported.Compliance with EA is measured and reported. No According to the OTO, compliance with the EA is not measured

and reported.Source: GAO analysis of agency provided data.



National Aeronautics and Table 37 shows NASA'S satisfaction of framework elements in version 1. 1 ofSpace Administration GAO's EAMMF.

Table 37: National Aeronautics and Space Administration Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. Partial According to the chief technology officer, NASA has adequate

funding and tools. However, the chief technology officer alsostated that the EA program has somewhat inadequate EApersonnel resources.

Committee or group representing the enterprise is Partial According to the chief technology officer, the operationsresponsible for directing, overseeing, and approving management council, CIO council, and strategic managementEA. council are responsible for directing, overseeing, and approving

the EA. However, the charters for these groups are awaitingexecutive approval and the draft charters provided did notspecifically mention EA.

Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Partial The EA is being developed using a framework and automatedand automated tool. tool. However, documentation did not indicate the EA is being

developed using a methodology that includes specific stepsrequired to develop, maintain, and validate the EA.

EA plans call for describing "as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, information/data,application/service, and technology.EA plans call for business, performance, Yesinformation/data, application/service, and technologyto address security.EA plans call for developing metrics to measure EA Yesprogress, quality, compliance, and return oninvestment.Stage 3: Developing EA productsWritten and approved organization policy exists for EA Yesdevelopment.EA products are under configuration management. Yes





EA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencingplan.Both "as-is" and "to-be" environments are described or Yeswill be described in terms of business, performance,

*information/data, application/service, and technology.*These descriptions address or will address security. Yes

Progress against EA plans is measured and reported. YesStage 4: Completing EA productsWritten and approved organization policy exists for EA Yesmaintenance.EA products and management processes undergo Yesindependent verification and validation.EA products describe "as-is" environment, "to-be" No EA products describe the "as-is" environment. However, EAenvironment, and sequencing plan. products do not fully describe the "to-be" environment and do

not include a sequencing plan.Both "as-is" and "to-be" environments are described Partial According to NASA's chief technology officer, the "as-is"in terms of business, performance, information/data, environment is described in terms of business,application/service, and technology. application/service, and technology. However, the "as-is"

environment is not described in terms of performance andinformation/data and EA products do not describe the "to-be"environment.

These descriptions address security. Partial The "as-is" description addresses security. However, the "to-be"description does not address security.

Organization CIO has approved current version of EA. YesCommittee or group representing the enterprise or. Yesthe investment review board has approved currentversion of EA.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. YesEA is integral component of IT investment Partial NASA's IT investment management process guidancemanagement process. recognizes EA as an integral component. However, a

sequencing plan to guide IT investments does not exist.EA products are periodically updated. YesIT investments comply with EA. Partial NASA evidence showed that some investments were certified

as compliant with the EA. However, evidence of the certificationcriteria used to assess IT investment compliance was notprovided.

Page 117 Pag 11 GAO-06-831 Enterprise Architecture




Organization head has approved current version of YesEA.Return on EA investment is measured and reported. No According to the chief technology officer, return on EA

investment is not measured and reported.Compliance with EA is measured and reported. Yes




National Science Table 38 shows NSF'S satisfaction of framework elements in version 1.1 ofFoundation GAO's EAMMF

Table 38: National Science Foundation Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the enterprise is Partial A committee or group representing the enterprise isresponsible for directing, overseeing, and approving responsible for directing and overseeing the EA. However, noEA. committee or group has responsibility for approving the EA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Partial According to the chief architect, the EA is being developedand automated tool. using a framework, methodology, and automated tool.

However, the methodology does not document steps for EAdevelopment.

EA plans call for describing "as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, information/data,application/service, and technology.

EA plans call for business, performance, information/ Yesdata, application/service, and technology to addresssecurity.EA plans call for developing metrics to measure EA Yesprogress, quality, compliance, and return on investment.Stage 3: Developing architecture productsWritten and approved policy exists for EA development. Partial According to the chief architect, a policy exists for EA

development. However, the policy is not approved.EA products are under configuration management. YesEA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencingplan.Both "as-is" and "to-be" environments are described or Yeswill be described in terms of business, performance,information/data, application/service, and technology.





These descriptions address or will address security. YesProgress against EA plans is measured and reported. Partial According to the chief architect, progress against EA plans is

measured and reported. However, the plan against whichprogress is measured is not approved and no documentationof progress reporting was provided.

Stage 4: Completing architecture productsWritten and approved organization policy exists for EA Partial According to the chief architect, a policy exists for EAmaintenance. maintenance. However, the policy is not approved.

EA products and management processes undergo No According to the chief architect, EA products andindependent verification and validation, management processes have not undergone independent

verification and validation.EA products describe "as-is" environment, "to-be" Partial According to the chief architect, EA products describe the "as-environment, and sequencing plan. is" environment, "to-be" environment, and sequencing plan.

However, the "as-is" environment is not fully described interms of performance and the "to-be" environment is not fullydescribed in terms of performance and information/data.

Both "as-is" and "to-be" environments are described in Partial According to the chief architect, the "as-is" is not fullyterms of business, performance, information/data, described in terms of performance and the "to-be"application/service, and technology, environment is not fully described in terms of performance

and information/data.These descriptions address security Partial The business, information/data, application/service, and

technology descriptions address security. However, theperformance description does not address security.

Organization 010 has approved current version of EA. YesCommittee or group representing the enterprise or the No According to the chief architect, no committee or groupinvestment review board has approved current version representing the enterprise has approved the current versionof EA. of the EA.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managing changeWritten and approved organization policy exists for IT Partial The chief architect provided an organization policy for ITinvestment compliance with EA. investment compliance with the EA. However, the policy is not

approved and does not explicitly require IT investmentcompliance with the EA.

Process exists to formally manage EA change. YesEA is integral component of IT investment management Yesprocess.EA products are periodically updated. YesIT investments comply with EA. Partial According to the chief architect, IT investments comply with

the EA and NSF provided evidence of procedures intended todetermine compliance. However, evidence supporting that ITinvestments are required to be compliant with the EA beforethey are approved was not provided..





Organization head has approved current version of EA. No NSF did not provide evidence that the organization head hasapproved the current version of the EA.

Return on EA investment is measured and reported. No According to the chief architect, return on EA investment ismeasured and reported. However, evidence to support thisstatement was not provided.

Compliance with EA is measured and reported. Partial According to the chief architect, compliance with the EA ismeasured and reported, and NSF provided evidencedescribing how compliance is measured and reported.However, evidence of compliance was not provided.



Appendix IV-Detailed Assessments of IndividualDepartments and Agencies against Our EAManagement Maturity Framework

Nuclear Regulatory Table 39 shows NRC'S satisfaction of framework elements in version 1.1 ofCommission GAO's EAMMF.

Table 39: Nuclear Regulatory Commission Satisfaction of EAMMF

GAO basis for partially satisfied or not satisfiedStages and core elements Satisfied? determinationStage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA management foundationAdequate resources exist. -YesCommittee or group representing the enterprise is Partial According to the chief architect, a committee or groupresponsible for directing, overseeing, and approving EA. -representing the enterprise is responsible for directing,

overseeing, and approving the EA. However, the charter forthis committee does not clearly state that it is responsible fordirecting, overseeing, and approving the EA and the charter isnot approved.

Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. Partial A chief architect has been designated. However, the chief

architect's roles and responsibilities do not include functioningas the EA program manager.

EA being developed using a framework, methodology, Yesand automated tool.EA plans call for describing "as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, inform ation/data,application/service, and technology.EA plans call for business, performance, information/data, Yesapplication/service, and technology to address security.EA plans call for developing metrics to measure EA Partial EA plans call for developing metrics to measure EA progress,progress, quality, compliance, and return on investment, quality, and compliance. However, EA plans do not call for

developing metrics to measure return on investment.Stage 3: Developing EA productsWritten and approved organization policy exists for EA Partial A draft policy exists for EA development. However, this policydevelopment, has not been approved.EA products are under configuration management. YesEA products describe or will describe "as-is" environment, Yes"to-be" environment, and sequencing plan.Both "as-is" and "to-be" environments are described or Yeswill be described in terms of business, performance,inform ation/data, application/service, and technology.

Page 122 Page122GAO-06-831 Enterprise Architecture.


(Continued From Previous Page)GAO basis for partially satisfied or not satisfieddeterminationStages and core elements

These descriptions address or will address security.Progress against EA plans is measured and reported.Stage 4: Completing EA productsWritten and approved organization policy exists for EAmaintenance.EA products and management processes undergoindependent verification and validation.EA products describe "as-is" environment, "to-be"environment, and sequencing plan.Both "as-is" and "to-be" environments are described interms of business, performance, inform at ion/data,application/service, and technology.These descriptions address security.Organization CIO has approved current version of EA.Committee or group representing the enterprise or theinvestment review board has approved current version ofEA.

Satisfied?

YesYes

Partial A draft policy exists for EA maintenance. However, this policyhas not been approved.EA products and management processes have notundergone independent verification and validation.

No

Yes

Yes

YesYesNo According to the chief architect, a committee or group

representing the enterprise or the investment review boardhas not approved the current version of the EA.

Quality of EA products is measured and reported. Partial NRC provided information on how quality of EA products is tobe measured and reported. However, documentationdemonstrating that quality is actually being measured andreported was not provided.

Stage 5: Leveraging the EA for managing changeWritten and approved organization policy exists for IT Partial A draft policy exists for IT investment compliance with EA.investment compliance with EA. However, this policy has not been approved.Process exists to formally manage EA change. Partial According to the chief architect, a process exists to formally

manage EA change. However, evidence that the process hasbeen implemented was provided for some EA products butnot others.

EA is integral component of IT investment management Yesprocess.EA products are periodically updated. YesIT investments comply with EA. Partial According to the chief architect, IT investments comply with

the EA and evidence demonstrates that EA is consideredduring the IT investment management process. However,documentation provided did not demonstrate IT investmentcompliance.

Organization head has approved current version of EA. No According to the chief architect, the organization head has notapproved the current version of the EA.

Return on EA investment is measured and reported. No According to NRC officials, return on EA investment is notmeasured and reported.




Stages and core elementsCompliance with EA is measured and reported.

GAO basis for partially satisfied or not satisfiedSatisfied? determinationPartial According to the chief architect, compliance with EA is

measured and reported. However, documentationdemonstrating that compliance is actually being measuredand reported was not provided.

Source: GAO analysis based on agency provided data.



Office of Personnel Table 40 shows OPM's satisfaction of framework elements in version 1.1 ofManagement GAO's EAMMF.

Table 40: Office of Personnel Management Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the enterprise is Yesresponsible for directing, overseeing, and approving EA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Yesand automated tool.EA plans call for describing "as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, information/data,application/service, and technology.EA plans call for business, performance, information/ Yesdata, application/service, and technology to addresssecurity.EA plans call for developing metrics to measure EA Yesprogress, quality, compliance, and return on investment.Stage 3: Developing EA productsWritten and approved organization policy exists for EA Yesdevelopment.EA products are under configuration management.. YesEA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencingplan.Both "as-is" and "to-be" environments are described or Yeswill be described in terms of business, performance,information/data, application/service, and technology.These descriptions address or will address security. YesProgress against EA plans is measured and reported. Partial OPM has limited reporting of EA progress. However, progress

is not measured and reported relative to an EA program plan.





Stage 4: Completing EA productsWritten and approved organization policy exists for EA Yesmaintenance.EA products and management processes undergo No According to the chief architect, EA products andindependent verification and management processes have not undergone independentvalidation, verification and validation. The chief architect stated that the

cost of independent verification and validation is not justified.EA products describe "as-is" environment, "to-be" Yesenvironment, and sequencing plan.Both "as-is" and "to-be" environments are described in Yesterms of business, performance, information/data,application/service, and technology.These descriptions address security. YesOrganization CIO has approved current version of EA. YesCommittee or group representing the enterprise or the Yesinvestment review board has approved current version ofEA.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managing changeWritten and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. YesEA is integral component of IT investment management Yesprocess.EA products are periodically updated. YesIT investments comply with EA. YesOrganization head has approved current version of EA. YesReturn on EA investment is measured and reported. YesCompliance with EA is measured and reported. Yes




Small Business Table 41 shows SBA~s satisfaction of framework elements in version 1. 1 ofAdministration GAO's EAMMF.

Table 41: Small Business Administration Satisfaction of EAMMF

GAO basis for partial or not satisfiedStages and core elements Satisfied? determination

Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA management foundationAdequate resources exist. Partial According to SBA officials, the agency has EA

program activities that do not have adequateresources.

Committee or group representing the enterprise is responsibl Yesfor directing, overseeing, and approving EA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, and Yesautomated tool.EA plans call for describing "as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of business, Yesperformance, information/data, application/service, andtechnology.EA plans call for business, performance, information/data, Yesapplication/service, and technology to address security.EA plans call for developing metrics to measure and report EA Partial EA plans call for developing metrics to measure andprogress, quality, compliance, and return on investment. report EA progress, quality, and compliance.

However, documentation did not include plans fordeveloping metrics to measure and report EA returnon investment.

Stage 3: Developing EA productsWritten and approved organization policy exists for EA Yesdevelopment.EA products are under configuration management. No Configuration management procedures provided to

GAO do not describe steps to ensure the integrityand consistency of EA products.

EA products describe or will describe "as-is" environment, Yes"to-be" environment, and sequencing plan.Both "as-is" and "to-be" environments are described or will be Yesdescribed in terms of business, performance, information/data,application/service, and technology.



(Continued From Previous Page)GAO basis for partial or not satisfied

Stages and core elements Satisfied? determinationThese descriptions address or will address security. YesProgress against EA plans is measured and reported. No SBA officials did not provide evidence that the

agency is measuring and reporting progress againstEA plans.

Stage 4: Completing EA productsWritten and approved organization policy exists for EA Yesmaintenance.EA products and management processes undergo independent Yesverification and validation.EA products describe "as-is" environment, "to-be" environment, Yesand sequencing plan.Both "as-is" and "to-be" environments are described in terms Yesof business, performance, information/data, application/service,and technology.These descriptions address security. YesOrganization CI0 has approved current version of EA. YesCommittee or group representing the enterprise or the No According to SBA officials, a committee or groupinvestment review board has approved current version of EA. representing the enterprise approved the current

version of the EA. However, documentationindicated approval of the 2003 EA program policiesand procedures, not the current version of the EA.

Quality of EA products is measured and reported. No According to SBA officials, the quality of EAproducts is measured and reported. However, SBA

* did not provide documentation that supports this* statement.

Stage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. YesEA is integral component of IT investment management Yesprocess.EA products are periodically updated. YesIT investments comply with EA. YesOrganization head has approved current version of EA. No According to SBA officials, the organization head

has approved the current version of the EA.However, documentation indicated approval of the2003 EA program policies and procedures, not thecurrent version of the EA.

Return on EA investment is measured and reported. No According to SBA officials, return on EA investmentis not measured and reported.



(Continued From Previous Page)GAO basis for partial or not satisfied


Compliance with EA is measured and reported. No According to SBA officials, compliance with EA ismeasured and reported. However, these officials didnot provide documentation that supports thisstatement.




Social Security Table 42 shows SSA's satisfaction of framework elements. in version 1. 1 ofAdministration GAO's EAMMF.

Table 42: Social Security Administration Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundationAdequate resources exist. YesCommittee or group representing the enterprise is Yesresponsible for directing, overseeing, and approving EA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Partial According to the chief architect, the EA is being developedand automated tool. using a framework, methodology, and automated tool.

However, the methodology does not include steps fordeveloping, maintaining, and validating the agency's EA.

EA plans call for describing "as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, inform ation/data,appffcation/service, and technology.EA plans call for business, performance, information/data, Yesapplication/service, and technology to address security.EA plans call for developing metrics to measure EA Yesprogress, quality, compliance, and return on investment.Stage 3: Developing EA productsWritten and approved organization policy exists for EA Yesdevelopment.EA products are under configuration management. YesEA products describe or will describe "as-is" Yesenvironment, "to-be" environment, and sequencing plan.Both "as-is" and "to-be" environments are described or Yeswill be described in terms of business, performance,information/data, application/service, and technology.These descriptions address or will address security. YesProgress against EA plans is measured and reported. Yes




Stages and core elements Satisfied? determinationStage 4: Completing EA productsWritten and approved organization policy exists for EA Yesmaintenance.EA products and management processes undergo Yesindependent verification and validation.EA products describe "as-is" environment, Yes"to-be" environment, and sequencing plan.Both "as-is" and "to-be" environments are described in Yesterms of business, performance, inform ation/data,application/service, and technology.These descriptions address security. Yes.Organization 0I0 has approved current version of EA. YesCommittee or group representing the enterprise or the Yesinvestment review board has approved current versionof EA.Quality of EA products is measured and reported. YesStage 5: Leveraging the EA for managingchangeWritten and approved organization policy exists for IT Yesinvestment compliance with EA.Process exists to formally manage EA change. YesEA is integral component of IT investment management Yesprocess.EA products are periodically updated. YesIT investments comply with EA. YesOrganization head has approved current No SSA officials stated that CIO approval of the current version ofversion of EA. the EA constitutes approval by the organization head.

However, they did not provide documentation delegating EAapproval authority from the organization head to the 010.

Return on EA investment is measured and reported. Partial According to the chief architect, return on EA investment ismeasured and reported and a description of how return oninvestment is measured and reported was provided. However,sample measures and reports were not provided.

Compliance with EA is measured and reported. Partial According to the chief architect, compliance with the EA ismeasured and reported and a description of how complianceis measured and reported was provided. However, samplemeasures and reports were not provided.




U.S. Agency for Table 43 shows USAID's satisfaction of framework elements in version 1. 1International Development of GAO's EAMMF.

Table 43: U. S. Agency for International Development Satisfaction of EAMMF


Stage 1: Creating EA awarenessAgency is aware of EA. .n/a No core element exists in stage 1.Stage 2: Building the EA managementfoundation'Adequate resources exist. No According to the chief architect, USAID has "somewhat

inadequate" EA resources.Committee or group representing the enterprise is Yesresponsible for directing, overseeing, and approving EA.Program office responsible for EA development and Yesmaintenance exists.Chief architect exists. YesEA being developed using a framework, methodology, Partial According to the chief architect, USAID is developing theirand automated tool. architecture using a framework, methodology, and

automated tool. However, the methodology does notdescribe steps required to maintain and validate thearchitecture.

EA plans call for describing "as-is" environment, "to-be" Yesenvironment, and sequencing plan.EA plans call for describing enterprise in terms of Yesbusiness, performance, inform ation/data,application/service, and technology.EA plans call for business, performance, information/data, Yesapplication/service, and technology to address security.EA plans call for developing metrics to measure EA Partial According to the chief architect, USAID has plans forprogress, quality, compliance, and return on investment, developing metrics to measure EA progress, quality,

compliance, and return on investment. USAID provideddocumentation of its plans to measure and report EAcompliance. However, documentation of plans to develop theother metrics was not provided.

Stage 3: Developing EA productsWritten and approved organization policy exists for EA No According to the chief architect, USAID does not have adevelopment, written and approved policy for EA development.EA products are under configuration management. No USAID provided evidence that discusses the need for EA

products to be under configuration management. However,the evidence did not describe detailed steps that wouldensure the integrity and consistency of EA products.




Stages and core elements Satisfied? determinationEA products describe or will describe "as-is" environment, Yes"to-be" environment, and sequencing plan.Both "as-is" and "to-be" environments are described or Yeswill be described in terms of business, performance,information/data, application/service, and technology.These descriptions address or will address security. YesProgress against EA plans is measured and reported. No According to the chief architect, progress against EA plans is

- not measured and reported.Stage 4: Completing EA productsWritten and approved organization policy exists for EA No According to the chief architect, USAID does not have amaintenance. written and approved policy for EA maintenance.EA products and management processes undergo Partial EA products and management processes have undergoneindependent verification and validation. independent verification and validation. However, the current

EA version has not undergone independent verification andvalidation.

EA products describe "as-is" environment, "to-be" No According to the chief architect, EA products do not fullyenvironment, and sequencing plan. describe the "as-is" environment, "to-be" environment, and

sequencing plan.Both "as-is" and "to-be" environments are described in Partial Both "as-is" and "to-be" environment descriptions are beingterms of business, performance, information/data, developed. However, the descriptions currently address onlyapplication/service, and technology. one segment of the architecture.These descriptions address security. Partial The "as-is" and "to-be" environment descriptions partially

address security. However, the descriptions currentlyaddress only one segment of the architecture and do notaddress security in each of the required terms.

Organization CI0 has approved current version of EA. YesCommittee or group representing the enterprise or the Partial According to the chief architect, committees representing theinvestment review board has approved current version of enterprise have approved the current version of the EA.EA. However, documentation of these approvals was not

provided..Quality of EA products is measured and reported. No According to the chief architect, USAID has not implemented

plans to measure and report quality of EA products.Stage 5: Leveraging the EA for managing changeWritten and approved organization policy exists for IT No USAID has a program plan that encourages IT investmentinvestment compliance with EA. compliance with the EA. However, the plan does not

explicitly require IT investment compliance with the EA.Process exists to formally manage EA change. Partial According to the chief architect, USAID has chartered a

configuration control board to manage EA change. However,USAID did not provide any documentation that this board isfunctioning as chartered.

EA is integral component of IT investment management Partial USAID provided documentation indicating that EA is anprocess. integral component of its IT investment management

process. However, USAID does not have an enterprisewidesequencing plan to guide IT investments.





EA products are periodically updated. YesIT investments comply with EA. Partial USAID provided documentation indicating IT investments

comply with the EA. However, investment compliance islimited to the one segment of the EA that has beendeveloped.

Organization head has approved current version of EA. No According to the chief architect, the organization head hasnot approved the current version of the EA.

Return on EA investment is measured and reported. No According to the chief architect, USAID is developing metricsfor measuring and reporting return on EA investment, butthese metrics have not been completed.


Note: The U.S. Agency for International Development is working with the Department of State todevelop the Joint Enterprise Architecture. However, USAID asked that we evaluate its agency'senterprise architecture based on efforts to develop the USAID enterprise architecture.


Appendix V

Comments from the Department ofCommerce

/!Z\ UNITED STATES DEPARTMENT OF COMMERCEChief Information Officer

, Washington. G.C 20230

jull ZV

Mr. Randolph C. HiteDirector, Information Technology Architecture

and Systems IssuesUnited States Government Accountability OfficeWashington, DC 20548

Mr. Hite:

Thank you for the Government Accountability Office's draft report entitled "EnterpriseArchitecture: Leadership Remains Key to Establishing and Leveraging Architectures forOrganizational Transformation (GAO-06-83 1)." We appreciate the thoroughness ofyourreview, concur with your findings regarding the Department of Commerce, and willconsider actions to address your concerns.

Sincerely,


Appendix VI

Comments from the Department of Defense

Note: GAO commentssupplementing those inthe report text appearat the end of thisappendix. ASSISTANT SECRETARY OF DEFENSE

6000 DEFENSE PENTAGONWASHINGTON, DC 20301-6000

June 30, 2006

NETWORKS AND INFORMAT'ONINTEGRATION

Mr. Randolph C. HiteDirector, Information Technology and Systems Issues,U.S. Government Accountability Office,441 G Street, NWWashington, D.C. 20548.

Dear Mr. Hlite,

"This is the Department of Defense (DoD) response to the GAO draft report,'ENTERPRISE ARCHITECTURE: Leadership Remains Key to Establishing and LeveragingArchitectures for Organizational Transformation,' dated May 31, 2006, (GAO Code3 10604/GAO-06-83 I)."

DoD has reviewed the Draft Report and plans to use the GAO Enterprise Architecturematurity framework as one of the benchmark best practices as DoD components continuouslywork to improve enterprise architecture management maturity. In general, DoD concurs withthe GAO recommendation but with one exception, and that is the criteria for independentvalidation and verification as it pertains to the Global Information Grid architecture. Also wetake exception to one of the techiiical findings published in Appendix IV of the GAO report asit pertains to the Business Transformation Agency. Specific comments regarding thesestatements arc enclosed. My action officer for this effort is Mr. Roy Mabry, (703) 380-0964.roy.mabryftosd.mil

See comment 1.

George WauerDirector, Architecture and Interoperability

(Office ofthe DeputyCIO)

Enclosures:As stated

cc:Head, Business Transformation AgencyDepartment of the Army, Chief Information OfficerDepartment of the Navy, Chief Information OfficerDepartment of the Air Force, Chief Information Officer


Appendix VIComments from the Department of Defense

GAO DRAFT REPORT - DATED MAY 31, 2006GAO CODE 310604/GAO-06-831

"ENTERPRISE ARCHITECTURE: LEADERSHIIP REMAINS KEY TO ESTABLISHINGAND LEVERAGING ARCHITECTURES FOR ORGANIZATIONAL

TRANSFORMATION"

DEPARTMENT OF DEFENSE COMMENTSON THE RECOMMENDATION

RECOMMENDATION: The GAO recommended that the Secretary of Defense ensure thatrespective enterprise architecture programs develop and implement plans for fully satisfyingeach of the conditions in GAO's Enterprise Architecture Management Maturity Framework.GAO CODE 310604/GAO-06-83 (p. 41/GAO Draft Report)

DOD RESPONSE: The GAO framework will be used as one of the benchmark hest practicesas DoD components continuously work to improve enterprise architecture managementmaturity. In general DoD concurs with the GAO recommendation with one exception, andthat is the criteria for independent validation and verification as it pertains to the GlobalInformation Grid architecture. DOD believes that the internal DoD process for reviewing andvalidating is sufficient and meets the intent of this GAO criteria and therefore will remain theDoD practice until otherwise changed. However, DoD will revisit this position asimplementation plans for fully satisfying each of the conditions in the GAO EnterpriseArchitecture Management Maturity framework are developed.

Regarding the GAO findings, which are shown in Appendix IV of the GAO report, it is clear thatDoD components do not fully meet the criteria of the GAO Enterprise Architecture ManagementMaturity Framework at this time, although some DoD components are further along than others.Even after further discussions between GAO and DoD), there are findings in the report withwhich DoD continues to disagree. The Business Transformation Agency (BTA) maintains thatthe Stage Two and Three security/i nformation assurance findings, for reasons previously statedin written technical comments and discussions with the GAO audit team, do not accuratelyreflect the situation and therefore DoD) must non-concur with those findings as presented inAppendix IV of the GAO Report. The DoD view is that the GIG architecture defines how allmembers of DoD EA will address security/information assurance and that its application must beaccepted by GAO as sufficient representation in the DoD EA component parts such as BTAotherwise having each component adopt security/information assurance other than that of theGIG negates one of the fundamental tenants of the GIG architecture and therefore fundamentallyworks counter to the DoD) efforts.

See comment 1.

See comment 2.


Appendix VIComments from the Department of Defense

GAO. Comments 1. We do not agree for two reasons. First, DOD's internal processes forreviewing and validating the Global Information Grid (GIG), whileimportant and valuable to ensuring architecture quality, are notindependently performed. As we have previously reported,independent verification and validation is a recognized hallmark ofwell-managed programs, including architecture programs.' To beeffective, it should be performed by an entity that is independent of theprocesses and products that are being reviewed to help ensure that it isdone in an unbiased manner and that is based on objective evidence.Second, the scope of these internal review and validation efforts onlyextends to a subset of GIG products and management processes.According to our framework, indepenident verification and validationshould address both the architecture products and the processes usedto develop them.

2. While we acknowledge that GIG program plans provide for addressingsecurity, and our findings relative to the GIG reflect this, this is not the.case for DOD's Business Enterprise Architecture (BEA). Morespecifically, how security will be addressed in the BEA performance,business, information/data, application/service, and technologyproducts is not addressed in the BEA either by explicit statement orreference. This finding relative to the BEA is consistent with our recentreport on DOD's Business System Modernization.'

'GAO-O3-584G

'GAO. Business Systems Modernization: DOD Continues to Improve InstitutionalApproach, but fu4rther Steps Needed, GAO-06-658. (Mlay 15, 2006).


Appendix VII

Comments from the Department of Education

JUNZ~J~

Mr. Randolph C. HiteDirector, Information Technology Architccture and SystemsU. S.* Government Accountability Office441 G Street. NWRoom 4085Washington. DC 20548

Dear Mr. Illite:

-Thank you for the opportunity to review the draft report Fitcerprise Architcture:Leuders/u~p Remains Key To Establishing and Leveraging Architectur's forOrganizational Transvformnation (GAO-06-83 1). The Department of Education usesEnterprise Architecture (EA) as a strategic planning tool to drive the Department'soperational performance and deliver business results. The Department has madesignificant progress in improving its EA maturity, and plans to continue its progresstoward greater EA maturity.

Figure 6 of thc report reflects that the Department needs to satisfy five or fewver coreelements in order to achieve an "Optimized" score of 5 in which all core elements arefully or partially satisfied. With respect to these remaining core elements, theDepartment plans to take the following step:

- With respect to the "Committed or group representing the enterprise is responsiblefor ... approving EA- core element, the Department's investment review board(IRB) charter will be amended to reflect this responsibility more explicitly, andfuture IRB minutes will also reflect this approval more explicitly.

. With respect to the "EA products and management processes undergoindependent verification and validation" core element, the Department isconsidering the business need and available altemnatives for indepcndentverification and validation of the Department's Enterprise Architecture productsand management processes.

0 With respect to the" ... the investment review board has approved current versionof EA" core element, future Department IRB meeting minutes will moreexplicitly reflect the IRB's approval.

a With respect to the "Organization head has approved current version of the EA"core element, the Department's IRB charter will be amended to more explicitly


Appendix VIIComments from the Department ofEducation

reflect delegation or EA approval authority to the Chief Operating Officer as theChair of the Investment Review Board.

*With respect to the "Return on EA investment is measured and reportcd- corcclement, the Department plans to implement a value metrics framework toimprove Rctumn on Investment (ROD) and Value of Investment (VOI)mcasurenlcnts in support of capital planning.

Again, thank you for the opportunity to review and comment on the draft report. ThcDcpartmcnt is continuing to work to improve its Enterprise Architecture.

Sincerely,


Appendix VIII

Comments from the Department of Energy

Y* Department of EnergyWashington, DC 20585

J une 22, 2006

Michael P. HollandSenior Information Technology AnalystU.S. Govcrmecnt Accountability Office441 G St. NW, Room 4Q26Washington. DC 20548

Dear Mr. H-olland:

We have reviewcd the draft Government Accountability Office report entitled,"Enterprise Architectture: Leadership Remnains Key to Establishing and LeveragingArchitectures for Organizational Transformation (GAO-06-83 I)" and concur with thereport with no further comment.

Thank yoti for the opportunity to participate in the development of this report. If youhave any questions, please contact Ms. Valerie Young at (202)586-8853 or via email atvalerie.voting r&W~doe.L'ov.

Sincerely,

TheAnne GordonActing Associate Chief. Information Officer

for Information Technology Reform

Page 141 Page141GAO-06-831 Enterprise Architectuire

Appendix IX

Comments from the Department of HomelandSecurity

Note: GAO commentssupplementing those inthe report text appearat the end of this U.S. Department or Hlomeland Security

appendix. WsigoD 02

HomelandUSecurity

June 28, 2006

Mr. Randolph C. HiteDirectorInformation Technology Architecture and

Systems IssuesU.S. Government Accountability Office441 G Street, NWWashington, DC 20548

Dear Mr. Ilite:

RE: Draft Report GAO-06-83 I, Enterprise Architecture: Leadership RemainsKey to Establishing and Lever-aging Architectures for OrganizationalTransformation (GAO Job Code 310604)

The Department of Homeland Security (DHS) appreciates the opportunity to review andcomment on the draft report. Thc Government Accountability Office (GAO)recommends that DHS and 26 other major departments and agencies ensure that our"respective enterprise architecture programs develop and implement plans for fullysatisfying each of the conditions in [GAO's] enterprise architecture management maturityframework." We have taken and continue to take steps to do so.

DHS is encouraged by GAO's recognition of the significant progress made since GAOcommented on the HLS Enterprise Architecture program in 20041 and again last yearwhen homeland security information sharing was first added to the GAO High RiskSeries2 .

We were pleased to see your assessment in Table 5 that found DH-S already satisfies 75%of the Stage 4 elements and half of Stage 5 elements. In our mission to protect thehomeland, issues of system survivability and vulnerability affect information technologyinvestment choices in a way not necessarily applicable to many agencies. Priority hasbeen, of necessity, rightly given to our mission of securing the homeland, during a periodof natural disasters and a very real war on terror.

1'GA0, Hlomeland Security: Efforts Under Way to Develop Enterprise.Architecture but Mluch WorkRemains. GAO-04.777.

'GAO, High Risk Series. An Update. GAO-O5-207.

www.dhs.gov


Appendix IXComments from the Department of HomelandSecurity

See comment 1.

2

DHS, as mentioned in prior GAO reports on the subject, has developed a businessprocess that reviews and evaluates planned investments for compliane to a formalEnterprise Architecture vision a% an integral step in the annual budget developmentprocess. Since the development of the 2005 High Risk report, DHIS has made significantprogress in definition and management of appropriate information sharing technologics,policies and procedures. Newly implemented initiatives, [like Homeland SecurityOperations Center (IISOC), United States Public and Private Partnership (USP-3),Integrated Public Alert and Warning System (IPAWS), Emergency Data ExchangeLanguage (EDXL) and National Information Exchange Model Compliance (NIEM)standards, alongside assessment efforts in DJta Fusion, Screening, and Alerts andWarnings], capitalize on technology investments with management of resources andoperating practices to improve processes based on previous GAO findings.

The Department has moved to implement the philosophies of Enterprise Architecture as ameans to' assuring mission performance, while developing the Department and itsfunctional infrastructure. To continue to make progress in meeting all of its oversightrequirements, DHS would like to recommend that GAO work with the Office ofManagement and Budget to coordinate and align their respective definitions of EnterpriseArchitecture maturity. As the Department matures, so will the architectural documentsand formal procedures, but the success of the mission must always be given priority tosafeguard our nation.

We agree with GAO's assertion that the key to building upon our current progress andstatus is executive leadership. We believe that based on demonstrated progress to datesuch leadership exists at the Department of Homeland Security.

We are providing technical comments to your office under separate cover.

Sincerely,

Steven J. PecinovskyDirectorDepartmental GAO/OIG Liaison Office

MMCP



GAO0 Comments 1. We acknowledge this recommendation and offer three comments inresponse. First, we have taken a number of steps over the last 5 years tocoordinate our framework with 0MB. For example, in 2002, we basedversion 1.0 of our framework on the 0MB-sponsored 010 CouncilPractical Guide to Federal Enterpgrise Architecture,1 and we obtainedconcurrence on the framework from the practical guide's principalauthors. Further, we provided a draft of this version to 0MB forcomment, and in our 2002 report in which we assessed federaldepartments and. agencies against this version, we recommended that0MB use the framework to guide and assess agency architectureefforts.' In addition, in developing the second version of our frameworkin 2003,' we solicited comments from 0MB as well as federaldepartments and agencies. We also reiterated our recommendation to0MB to use the framework in our 2003 report in which we assessedfederal departments and agencies against the second version of theframework.'

Second, we have discussed alignment of our framework and OMB'sarchitecture assessment tool with 0MB officials. For example, after0MB developed the first version of its architecture assessment tool in2004, we met with 0MB officials to discuss our respective tools andperiodic agency assessments. We also discussed OMB's plans forissuing the next version of its assessment tool and how this nextversion would align with our framework. At that time, we advocatedthe development of comprehensive federal standards governing allaspects of architecture development, maintenance, and use. In ourview, neither our framework nor OMB's assessment tool provide suchcomprehensive standards, and in the case of our framework, it is notintended to provide such standards. Nevertheless, we plan to continueto evolve, refine, and improve our framework, and will be issuing anupdated version that incorporates lessons learned from the results ofthis review. In doing so, we will continue to solicit comments fromfederal departments and agencies, including 0MB.

'CIO Council, A Practical Guide to Federal Enterprise Architecture, Version 1.0 (February2001).

2 GAO-02-6

I'GAO-03'-584G

4 AO-04-40



Third, we believe that while our framework and OMB's assessment toolare not identical, they nevertheless consist of a common cadre of bestpractices and characteristics, as well as other relevant criteria that,taken together, are complementary and provide greater direction to,and visibility into, agency architecture programs than either does alone.


Appendix X

Comments from the Department. of Housingand Urban Development

U.S. DEPARTMENT OF HOUSING AND URBAN DEVELOPMENTWASHINGTON, D.C. 20410-3000

THE CHIEF INFORMATION OFFICER

June 21, 2006

Mr. Randolph C. HiteDirector, Information Technology ArchitectureAnd System IssueUnited States Government Accountability Office

Dear Mr. Hite:

The Department of Housing and Urban Development (HUD) would like to thank you forthe opportunity to review and provide comments on the Draft Enterprise Architecture (EA)Report GAO 310604. We are providing the following comments on the report findings andrecommended actions:

" We agree that our EA practice has been assessed accurately as a Stage 3practice, and, in accordance with the report recommendations, we are in the process ofdeveloping and implementing as EA program plan to satisfy each of the core elementsin the enterprise architecture management maturity framework (EAMMF) in 2007.

" We request that information describing the status of HUD's securityarchitecture on page 27 and pages 35-36 of the draft report be written to reflect workthat is currently underway to develop an enterprise security architecture. While weagree with a "Partial" assessment for this Stage 4 core element, we believe that the.attached information (Attachment(s) 1) demonstrates that a security architecture will becompleted by January 2007, and will be reflected thoroughly and consistently acrosseach element of the Department's enterprise architecture.

HUD is committed to ensuring the continued development of the enterprise architectureprogram by implementing plans for fully satisfying each of the conditions in GAO's enterprisearchitecture management maturity framework.

N


Appendix XComments from the Department of Housingand Urban Development

2

Thank you again for the opportunity to review and comnment on the draft report. In theinterim, if you have any questions regarding the information provided please contact BeverlyHacker, Chief Architect, Department of Housing and Urban Development, Office of the ChiefInformation Officer on 202- 708-0614 x 8338.

Attachment(s)

Sincerely yours,

Lisa SchlosserChief Information Officer


Appendix XComments from the Department of Housingand Urban Development

Attachment I: Security Architecture Development

The Department of H-ousing and Urban Development (IIUD) recognizes the importance of incorporatingsecurity into the Enterprise Architecture (EA). Consequently, the Office of IT Security is in the proceas ofdocumenting the integration of security within eacht applicable element of HURD's EA. As early as April?7.2005. HUD initiated a project to update the Technical Reference Model (TRM) to include securityproducts. The overall goal of this effort was not only intended to identify applicable security productswithin the TRNI. but also to begin the process of fully integrating security within all applicable elements ofHIUD's EA. Ultimately, the results of this project contributed to an improvement in HUJD's ability toprotect systems and the information that directly support the business needs of H-UD.

The process included:

" Identification and integration of relevant security standards and products into the I-UtD Technical

Reference Model (TRM).

" Expansion of HfUD's EA version 1.0's TRM to include recommended security product upgrades.and

" Identification and mapping of applicable TRM security products wzth components, applications,and/or technologies within HUD's LA version 1.0's Business Reference Model (BRLM) andService Reference Model (SRM).

A follow-on project wan initiated on Jan. 30,2006 and is scheduled to be completed by January 2007. Thiseffort is intended to:

. Developal security architecture that supports the achievement of HUD's business objectives, and

. Incorporate IT security throughout the HUtD EA.

This project further emphasizes the need to integrate security within HUD's EA using principles thatdirectly correlate to HIUD's business functions. recommended standards, applicable policies, best practices.and technology trends. The principles and recommendations and work being performed are driven byHUID's IT security policies and procedures. The outcome of this project will produce the following:

" HUD's Baseline for Network Infrastructure Security Architecture.

" Security requirements within the components of the BRM.

" Identification of the security requireme nts for the SRM component.

" Security requirements for the TRIM component.

" A process to continuously integrate security into HUD's LA. and

" Formal I-UD LA Security Architecture documentation.

To date, this project has yielded the following results:

" Identified Federal mandates, policies and guidelines and industry best practices that are applicable

for inclusion into HUlD's Network Infrastructure Security Architecture,

" Developed a preliminary list of legislation, regulations, guidelines, and policies related to HUID


* Appendix XComments from the Department of Housingand Urban Development

I3AfSecurity integration,

" Created an approach for EAlSecurity integration, and

" Developed a list of potential EAlSiccurity requirements from the BRM.

Lastly. BIUD is dedicated to ensuring that the integration of security into EA is a continuous process. Thiswill enable refinement of the EA as necessary to keep it aligned with federally mandated initiatives.business strategy requirements, emerging standards, and changing technology pertaining to security.

The results of this ongoing project, as well as any future integration efforts, shall be fully in HUD'senterprise architecture commencing in 2007.


Appendix X1

Comments from the Department of theInterior

United States Department of the InteriorOFFICE OF THE ASSISTANT SECRETARYPOLICY. MANAGEMENT AND BUDGET TAKE P1 EWashington, DC 20240 1 NAM E RICA

JUN 2 0 2006

Mr. Randolph C. HiteDirector, Information Technology Architecture and SystemsU.S. Government Accountability OfficeWashington, D.C. 20548

Dear Mr. Hite:

Thank you for the opportunity to review and provide comments on the U.S. GoverunmentAccountability Office's (GAO) draft report GAO-06-831I titled Enterprise Architecture -Leadership Remains Key to Establishing and Leveraging Architecture for OrganizationalTransformation. The Department of the Interior (DOI) has reviewed the report andconsiders it reflective of the current state of maturity of its enterprise architecture (EA)program with respect to the GAO's EA Management Maturity Framework (EAMMF)Version 1.1. The draft report identifies that DOI has met 97% of all criteria within thematurity framework and needs only to complete one criterion: "EA products andmanagement processes undergo Independent Verification and Validation"' to achieve astage 5 maturity and satisfy' 100% of all elements. To that end, DOI has recentlyawarded a contract to conduct an independent verification and validation (IV&V) of itsEA program which will satisfy this remaining element.

DOI takes the results of independent assessments and reviews of its EA programseriously and incorporates recommendations for improvement to continually mature itsEA program. DOI is committed to utilizing its EA to guide the overall management andmaturation of its IT portfolio and infrastructure to improve its overall missionperformance.

If you have any questions, please contact W. Hord Tipton, DOI CIO, at 202-208-6194.Staff may contact Ms. Colleen Coggins. DO! Chief Architect, at 202-208-5911.

Sincerely.

R. Thomas WeimerAssistant Secretary


Appendix MII

Comments -from the Department of Justice

Note: GAO commentssupplementing those inthe report text appearat the end of thisappendix.

U.S. Department or Justice

Wash ington, D.C. 20530

JUN 2 1 206

Randolph C. HiteDirector.Information Technology Architecture

and Systems IssuesGovernment Accountability Office441 G Street, NWWashington, DC 20548

Dear Mr. Hite:

Thank you for the opportunity to review and comment on the Government AccountabilityOffice's (GAO) draft report entitled "Enterprise Architecture: Leadership Remains Key toEstablishing and Leveraging Architectures for Organizational Transformation (GAO-06-831/310604). The Department believes that your draft report accurately reflects the state of ourenterprise architecture (EA) program and our needed focus areas for the future.

Overall, the feedback and analysis in this report is beneficial to our program, and will help guideour efforts going forward. Of the seven Core Elements in Stages 4 and 5 identified as partially ornot satisfied in the assessment, all are being analyzed and incorporated into our plans for theprogram. In particular, the elements relating to EA compliance and utilizing the EA in theinvestment process are a key near-term focus of our program and we are making significantstrides in these areas. Also, we are establishing processes to capture and report the impact of theEA program to allow us to demonstrate the return on investment for the program.

We would also like to take this opportunity to address some potential improvements for GAO'sEA evaluation and oversight process from our perspective. There arc several areas that wouldassist us in streamlining our responses and improve our EA program, including the following:

Align the GAO assessmenh criteria with the 0MB framework - Although there aremany common themes between the two frameworks, the focus is different many areasrequiring us to develop separate submission packages for evaluation. We also feel thatthere are some key areas in the latest OMB framework, particularly relating to the use ofthe EA, which could enhance the GAO framework.

See comment 1.


Appendix XIIComments from the Department of Justice

Randolph C. Hite, DirectorPage 2

See comment 2.

See comment 3.

I ncorporate the principles offederatcd and segmented architecture developmentthis is a key principle of our program, as it is for most large department level programs.This approach addresses the need to collaborate with component or bureau-level EAprograms to capture mission specific requirements and institutionalize the program.

* Provide a mechanism to share best practices - The DOJ EA could benefit fromthe insights that GAO gains through its evaluations of programs across the federalgovernment. Although programs need to be tailored to the particular agencies, there aremany foundational elements that are common, and could be leveraged across multipleagencies for greater efficiency.

Again, we appreciate the assessment and feedback of our EA program, and we look forward toadditional collaboration to assist us in further improving our implementation of EA at DOJ. TheDOJ EA program has made significant strides over the past year, and we plan to continue withthis momentum and leverage the work to make a significant impact across the Department.

If you have any ques tions regarding our comments, please contact Richard P. Theis, AssistantDirector, Audit Liaison Group on (202) 514-0469.

Sincerely,

Vance E. HitchChief information Officer


Appendix XIIComments from the Department of Justice

GXAO Comments 1. See DHS comment 1 in appendix IX. Also, while we do not have a basisfor commenting on the content of the department's 0MB evaluationsubmission package because we did not receive it, we would note thatthe information that we solicit to evaluate a department or agencyagainst our framework includes only information that should be readilyavailable as part of any wvell-managed architecture program.

2. We understand the principles of federated and segmentedarchitectures, but would emphasize that our framework is intentionallyneutral with respect to these and other architecture approaches (e.g.,service-oriented). That is, the scope of the framework, by design, doesnot extend to defining how various architecture approaches shouldspecifically be pursued, although we recognize that supplementalguidance on this approach would be useful. Our framework wascreated to organize fundamental (core) architecture managementpractices and characteristics (elements) into a logical progression. Assuch, it was intended to fill an architecture management void thatexisted in 2001 and thereby provide the context for more detailedstandards and guidance in a variety of areas. It was not intended to bethe single source of all relevant architecture guidance.

3. We agree, and believe that this report, by clearly identifying thosedepartments and agencies that have fully satisfied each core element,serves as the only readily available reference tool of which we areaware for gaining such best practice insights.


Appendix XIII

Comments from the Department of State

Note: GAO commentssupplementing those inthe report text appearat the end of thisappendix. * United States Department of State

Assistant Secretaiy for Resource Managementand Chief Financial Officer

Washington. D.C. 20520

JUtl 9 ZCUMs. Jacquelyn Williams-BridgersManaging DirectorInternational Affairs and TradeGoverniment Accountability Office441 G Street, N.W.Washington, D.C. 20548-0001

Dcar Ms. Williams-Bridgers:

We appreciate the opportunity to review your draft report,"ENTERPRISE ARCHITECTURE: Leadership Remains Key toEstablishing and Leveraging Architectures for OrganizationalTransformation," GAO Job Code 310604.

The enclosed Department of State comments are provided forincorporation with this letter as an appendix to the final report.

If you have any questions concerning this response, please contactChan Kim, IT Specialist, Bureau of Information Resource Management, at(202) 663-1066.

Sincerely,

cc: GAO -Mark BirdIRM - James VanderhoffState/OIG - Mark Duda


Appendix XIIIComments from the Department of State

Department of State Comments on GAO Draft Report

ENTERPRISE ARCHITECTURE: Leadership Remains Key toEstablishine and Leveraeing Architectures for Oreanizational

Transformation,(GAO-06-831. GAO Code 310604)

Thank you for the opportunity to comment on the GAO's report onENTERPRISE.ARCHITECTURE:- Leadership Remains Key to Establishing andLeveraging Architect ures for Organizational Transformation. The Departmentof State agrees with the GAO findings and recommendations.

The Department is engaged in a unique Joint Enterprise Architecture(JEA) effort to identify' common areas of collaboration with the U.S. Agencyfor International Development (USAID). This fact makes senior managementinvolvement essential in making Enterprise Architecture decisions affecting thetwo agencies. This joint relationship adds overall complexity to the effort aswell. The Department's Undersecretary for Management and USAID's DeputyAdministrator chair 'the Joint Management Council (JMC) executive committee.They have endorsed the use of Enterprise Architecture methodology and havemade it a critical component of their evolving governance process.

We are providing comments to address the partially satisfied or negativecriteria in Table 1: Joint Enterprise Architecture Satisfaction of EAMMIF underAppendix IV: Detailed Assessments of Individual Departments and Agencies.

Stage 2: Building the Enterprise Architecture (EA) managementfoundation:

a. Committee or group representing the enterprise is responsible fordirecting, overseeing, and approving EA

GAO Comment: According to the chief architect, State, andUSAID are developing a governance plan that will describemanagement processes for directing overseeing, and approvingtheir EA. However, this plan is not approved.



2

Dept EA Comment: The Joint Management Council (JMC) willapprove the Joint Enterprise Architecture (JEA) Governance planas part of the JMC Governance Plan.

b. EA being developed using a framework, methodology, andautomated tool

GAO Comment: iAccording to the chief architect, the EA is beingdeveloped using a framework, methodology, and automated tool.However, the methodology does not describe specific steps for themaintenance.

Dept EA Comment: The JEA Governance plan will support thisfunction.

C. 'EA Plans call for developing metrics to measure EA progress,quality, compliance, and return on investment

GAO Comment: State and USAID have plans to measure andreport EA progress, quality and compliance. However, no plans fordeveloping metrics to measure EA return on investment wereprovided.

Dept EA Comment: The new JMC Governance process willprovide an opportunity to incorporate appropriate metrics to showEA return on investment.

Stage 3: Developing EA products

a. EA products are under configuration management

GAO Comment: A draft configuration, management plan for EAproducts has been developed. However this plan has not beenapproved.

See comment 1. Dept EA Comment: The draft JEA configuration plan wassubmitted to 0MB in February 2006. It was approved by bothState's and USAID's CIOs.



3

b. Progress against EA plans is measured and reported

GAO Comment: State has limited reporting of EA progress.However, progress is not measured and reported relative to an EAprogram plan.

Dept EA Comment: The Joint State/USAID EA Team reports itsprogress to 0MIB on a quarterly basis. The JEA Team is alsodeveloping future EA metrics to comply with GAO and 0MBrecommendations as part of its EA Program Plan revision.

Stage 4: Completing EA products

a. EA products and management processes undergo independentverification and validation

GAO Comment: According to the chief architect, EA products andmanagement processes have not undergone independentverification and validation.

Dept EA Com ment: Independent verification and validation will beincluded as part of the JEA's new go'vemnance process. To ourknowledge, no federal agency EA products currently enjoy suchscrutiny.

b. EA Products describe "as-is" environment, "to-be" environment, andsequencing plan

GAO Comment: EA products describe the "as-is" and "to-be"environments as well as high level transition activities. However, asequencing plan for transitioning between the "as-is" and "to-be"environments is currently under development.

Dept EA Comment: The revised JEA Transition Strategy will meetthis requirement.


Appendix X111Comments from the Department of State

4

c. Committee or group representing the enterprise or the investmentreview board has approved the current version of EA

GAO Comment: According to the chief architect, a committee orgroup representing the enterprise has not approved the currentversion of the EA.

See comment 2. Dept EA Comment: All previous and current versions of the JEAhave been approved by the executive offices of both State andUSAID.

Stage 5: Leveraging the EA for management change

a. Process exists to formally manage EA change.

GAO Comment: According to the chief architect, a process toformally manage EA change does not exist.

Dept EA Comment: The draft JMC JEA Governance process willaddress this requirement.

b. EA is an integral component of IT investment management process

GAO Comment: According to the chief architect, EA is an integralcomponent of the IT investment management process. However,the sequencing plan to guide investments is currently underdevelopment.

Dept EA Comment:The revised JEA Transition Strategy will meetthis requirement.

c. Organization head has approved current version of EA

GAO Comment: According to the chief architect, the organization

head has not approved the current version of the EA.

Dept EA Comment: Both State's and USAID's CIOs haveapproved the current version of the JEA. The new JMC



5

Governance process will address the issue of gaining OrganizationHead approval for future versions of the JEA.

d. Return on EA investment is measured and reported

GAO Comment: According to the chief architect, return on EAinvestment is not measured and reported.

Dept EA Comment: The new JMC Governance process willprovide an opportunity to incorporate appropriate metrics to showEA return on investment.

e. Compliance with EA is measured and reported

GAO Comment: According to the chief architect, compliance withEA is measured and reported and officials provided a descriptionof how compliance is to be measured. However, no documentationdemonstrating compliance reporting was provided.

Dept EA Comment: EA is an integral part of the CPIC process atState, including providing project management training, evaluatingand scoring project proposals against EA alignment. EA ITInvestment score-sheets were provided to GAO as examples ofEAICP1C compliance.

See comment 3.


Appendix X11IComments from the Department of State

G-1AOC m et 1. We acknowledge the comment that both CIOs approved theGAO Co mentsconfiguration management plan. However, the department did notprovide us with any documentation to support this statement.

2. We acknowledge the comment that the architecture has been approvedby State and USAID executive offices. However, the department did notprovide any documentation describing to which executive offices thedepartment is referring to allow a determination of whether they werecollectively representative of the enterprise. Moreover, as we state inthe report, the chief architect told us that a body representative of theenterprise has not approved the current version of the architecture, andaccording to documentation provided, the Joint Management Council isto be responsible for approving the architecture.

3. We acknowledge that steps have been taken and are planned to treatthe enterprise architecture as an integral part of the investmentmanagement process, as our report findings reflect. However, our pointwith respect to this core element is whether the department'sinvestment portfolio compliance with the architecture is beingmeasured and reported to senior leadership. In this regard, State didnot provide the score sheets referred to in its comments, nor did itprovide any other evidence that such reporting is occurring.


Appendix XIV

Comments from the Department of theThe asury

DEPARTMENT OF THE TREASURYWASHINGTON, D.C. 20220

JUN 2 3 2006Mr. Michael P. HollandSenior Information Technology AnalystU.S. Government Accountability OfficeGeneral Accounting Office441 G Street, N.W.Washington. D.C. 20548

Dear Mr. Holland:

Thank you for the opportunity to review and comment on the General AccountingOffice (GAO) draft report, "Leadership Remains Key to Establishing and LeveragingArchitectures for Organizational Transformation (GAO Report # 06-831). Overall,our review of the "Draft" report found the report to be consistent with our ownassessment of the Treasury Department's Enterprise Architecture at the time of theGAO review.

The Department of Treasury recognized the need to take proactive steps to matureits Enterprise Architecture program and took the following affirmative steps inFY2005:

- Hired Executive (Associate Chief Information Officer for ElectronicGovernment) to develop and lead the Department's Enterprise ArchitectureProgram.

- Adopted the enterprise-wide repository toolset (Metis) to normalize datacollection and streamline the EA Department's Data Capture requirements.

The above actions, in conjunction with effective leadership and collaboration acrossthe Department, resulted in Treasury receiving a score of three out of a possible fiveon its Federal Enterprise Architecture (FEA) Assessment. The score of three raisedthe Department's rating from "Red" to "Yellow" in Enterprise Architecture in theOctober 2005, President's Management Agenda, E-Government Scorecard. TheGAO report confirms and highlights the Department's need to continue to provideexecutive leadership to the developing Enterprise Architecture program, and tocodify the program into Departmental policy. The Department is committed tocontinuing the Enterprise Architecture maturation process and, subsequently,identified a vendor to perform an Independent Verification and Validation of theDepartments Enterprise Architecture \program. A key deliverable from the IV&V is*a comprehensive "Gap Analysis". The Chief Enterprise Architect utilizing the "GapAnalysis" will identify Program deficiencies. and formulate appropriate mitigationstrategies to continue maturing the Treasury "One Enterprise Architecture".


Appendix XIVComments from the Department of theTreasury

Thank you again for the opportunity to review the "Draft" GAO report. If you haveany questions regarding these comments, please contact me at (202) 622-1200 orvia e-mail at Lawrence. Gross(~do.Treas.gov.

Lawrence GrossAssociate CIO E-GovemnmentOffice of the Chief Information OfficerU.S. Department of the Treasury


Appendix XV\

Comments from the Department VeteransAffairs

THE DEPUTY SECRETARY OF VETERANS AFFAIRSWASHINGTON

June 21,.2006

75

Mr. Randolph C. HiteDirector, Information Technology Architecture*and Systems IssuesInformation Technology TeamU. S. Government Accountability Office441 G Street, NWWashington, DC 20548

Dear Mr. Hite:

The Department of Veterans Affairs (VA) has reviewed your draft report,ENTERPRISE ARCHITECTURE:- Leadership Remains Key to Establishingand Leveraging Architectures for Organizational Transformation (GAO-06-831). VA concurs with your recommendation that VA's enterprise architectureprograms develop and implement plans for fully satisfying each of the conditionsin the Government Accountability Office's (GAO) enterprise architecturemanagement maturity framework. The Department will provide a detailed plan to*implement GAO's recommendation when commenting on your final report.

VA appreciates the opportunity to comment on your draft report.

ISincerely yours,

Gordon H. Mansfield


Appendix XVI

Comments from the EnvironmentalProtection Agency


1fea 4r~UNITED STATES ENVIRONMENTAL PROTECTION AGENCY

WASHINGTON. D.C. 20460

JUN 20 2D06

OfFFCEOFEWMRONMENTAL. eIORMATION

Mr. Randolph C. Hite, DirectorInformation Technology Architecture and Systems IssuesU.S. General Accountability Office441 G Strect,NW1Washington, D.C. 20548

Dear Mr. Hite:

Thank you for the opportunity to respond to the U.S. General Accounting Office(GAO) draft Report to Congressional Requesters "Enterprise Architecture: LeadershipRemainus Key to Establishing and Leveraging Arc hiteciures for OrganizationalTransformation" GAO-06-83 1). The U.S. Environmental Protection Agency (EPA) ispleased to note that the reviewers acknowledge the Agency's significant efforts to maturethe management of its Enterprise Architecture (EA) Program. The Agency agrees withthe reviewers' find ings on most of the EA Management Maturity Framework (EAMMF-)core elements. However, the current assessment of some core elements does notaccurately reflect our true status. EPA formally requests that GAO reconsider itsassessment before issuance of the final Report. We recognize the complexity of GAO'stask, and provide the following evidence and rationale for changing EPA's assessmentfrom the current Stage I to Stage 3.

EA Committee (2.2) - EPA's Quality Information Council (QIC) is the Agencyoversight committee for the Enterprise Architecture and is responsible for directing,overseeing, and approving the EA. The QIC operates a formal integrated process for allquality and information management responsibilities as a corporate board of Agencysenior executives, including the ChiefFinancial Officer and the CIO. All EAmanagement decisions are brought to QIC and I, as CIO and QIC Chair, have theauthority to approve the QIC's direction and oversight of the Enterprise Architecture.The 'CIO Delegation ofAuthority' and the 'QIC Charter' formally document thisstatement.

EA Methodology (2.5~ - EPA's EA methodology does include steps formaintaining the architecture, which were documented and approved within the official2002 release of the EPA EA as the EPA EA Practice and Proposed Policy, and the EPAEA Approach. EPA has since established fortmal 'EA Policy' and 'EA GovernanceProcedure' which further authorizes and requires the use ofa standardized EAmethodology, framework, and automated tcolset for the update and maintenance of theEA.

Internit Address (.JRLI - hitplh-eps qoý

See comment 1.

See comment 2.

Page 164 Page 164GAO-06-831 Enterprise Architectture

Appendix XVIComments from the EnvironmentalProtection Agency

2

See comment 3.

See comment 4.

EA Progress Measurement (3.6) - EPA's EA Transition Strategy and SequencingPlan illustrates how EPA's progress in building out the target architecture is documentedand measured. The principle metric used was the rate of incorporation of EPA's sharedservice components in the individual business eases within the IT Portfolio. This metricwas reported to the Chief Architect, the CIO, and to OMB. Additionally, progressagainst EPA's EA Project Plan is tracked and reported to Chief Architect and CIOrepresentatives on a mnonthly basis.

Also; EPA's EA management is more mature on two Stage 5 core elements thanthis report reflects: EPA's IT investments do comply with EPA's EA, and the EA changemanagement process is more mature than stated. I lowever, we recognize that there is oneStage 4 core element that has not yet been achieved, and thus our maturity assessment isappropriately a '3' on this scale.

EPA requests that GAO revise the assessment to reflect EPA's EA maturity atStage 3 before issuing the final report. All of the above referenced supportive materialshave been provided to GAO in previous responses to this assessment inquiry. We wouldbe happy to guide GAO to some specific language in the documentation we haveprovided that substantiates the above rationale.

Again, thank you for the opportunity to respond to this important report. EPA isproud of our successful Enterprise Architecture as recognized by OMB's "green" EAmatunity assessment in 2006. We consider this GAO assessment to be a valuablebenchnrarking exercise that will help us improve our performanice. Ifyou have anyquestions relating to this information, please contact Steve Tiber, EPA-GAO Liaison, at202-564-5184.

Sincerely,

Linda A. TraversActing Assistant Administrator

and ChiiefInformnation Officer

cc: Marysann Froehiich6 Acting CFO


Appendix XVI*Comments from the EnvironmentalProtection Agency

G-1AOC m et 1. We agree and have modified our report to recognize evidence containedin the documents.

2. We do not agree. The 2002 documents do not contain steps forarchitecture maintenance. Further, evidence was not provideddemonstrating that the recently prepared methodology documentswere approved prior to the completion of our evaluation.

3. We do not agree. While we do not question whether EPA'S EATRansition Strategy and Sequencing Plan illustrates how annualprogress in achieving the target architectural environment is measuredand reported, this is not the focus of this core element. Rather, this coreelement addresses whether progress against the architecture programmanagement plan is tracked and reported. While we acknowledgeEPA's comment that it tracks and reports such progress against planson a monthly basis, neither a program plan nor reports of progressagainst this plan were provided as documentary evidence to supportthis statement.

4. We do not agree. First, while EPA'S IT investment management processprovides for consideration of the enterprise architecture in investmentselection and control activities, no evidence was provideddemonstrating that the process has been implemented. Second, whileEPA provided a description of its architecture change managementprocess, no evidence was provided that this process has been approvedand implemented.


Appendix XVII

Comments from the General ServicesAdministration

H GSA Administrator

Julyý 14, 2006

The Honorable David M. WalkerComptroller General of the United StatesGovernment Accountability OfficeWashington, DC 20548

Dear Mr. Walker:

Thank you for the opportunity to respond to the Government Accountability Office(GAO) draft report entitled: "ENTERPRISE ARCHITECTURE: Leadership RemainsKey to Establishing and Leveraging Architectures for Organizational Transformation"(GAO-06-831). We are proud that GSA has achieved significant progress inimplementing an agency wide enterprise architecture and anticipate realizing significantbenefits from this effort. We have reviewed the draft report and we concur with therecommendation contained in the report that each agency "ensure that their respectiveenterprise architecture programs develop and implement plans for fully satisfying eachof the conditions in the GAO enterprise architecture management maturity framework."

In particular, the table 1, in Appendix IV, indicating GSA's satisfaction of frameworkelements, will be critical as we work in fiscal year 2007 to further implement coreelements of the Enterprise Architecture Framework.

Should you have any questions regarding this matter, please feel free to contact me.Staff inquiries may be directed to Mrs. Viola Hubbard, Audits, Office of the ChiefInformation Officer, on (202) 208-6155.

Sincerely

Lurita DoanAdministrator

U.S. General Services Administration1800 F Street. NWWashington, DC 20400-0002Telephone; (202) 501-08000Fax: (202) 219-1243www.gsa.gov


Appendix XVIII

Comments from the National Aeronautics andSpace Administration

National Aeronautics andSpace AdministrationOffice of the Administrator

Washington, DC 20546-0001 iMr. Randolph C. HiteDirectorInformation Technology Architecture and Systems IssuesUnited States Government Accountability OfficeWashington, DC 20548

- Dear Mr. Hite:

NASA has reviewed the draft Government Accountability Office (GAO) report"Enterprise Architecture: Leadership Remains Key to Establishing and LeveragingArchitectures for Organizational Transformation" (GAO-06-83 1). Thank you for theopportunity to provide comments on the recommendations in the draft report.

The draft report contains one recommendation addressed to several Governmententities, including NASA, that recommends these agencies, "ensure that their respectiveenterprise architecture programs devclop and implement plans for fully satisfying each ofthe conditions in [GAO's] enterprise architecture management maturity framework."

NASA concurs with this recommendation. The NASA Administrator continues to fullysupport the efforts of the NASA Chief Enterprise Architect to develop and implement theNASA Enterprise Architecture. NASA remains committed to achieving Stage 5 of GAO'sEnterprise Architecture management maturity framework.

If you have any questions, or require additional information, please contact Dr. JohnMcManus, NASA Chief Enterprise Architect, at (202) 358-1802 or Mr. Ken Griffey,NASA Deputy Chief Enterprise Architect, located at the Marshall Space Flight Center,at (228) 813.6209.

Sincerely, <

Shana DaleDeputy Administrator


Appendix MXX

Comments from the Social SecurityAdministration


SOCIAL SECURITYThe Commissioner

June 20, 2006

Mr. Randolph C. HiteDirector, Information Technology Architecture

and Systems IssuesU.S. Government Accountability Office441 G Street, NWWashington, D.C. 20548

Dear Mr. Hite:

Thank you for the opportunity to review the draft report,lEnterprise Architecture: LeadershipRemains Key to Establishing and Leveraging Architectures for Organizational Transformationi'(GAO-06-83 1). Our comments are enclosed.

If you have any questions, please have your staffcontact Candace Skumik, Director, AuditManagement and Liaison Staff, at (410) 965-4636.

Sincerely,

Enclosure

SOCIAL SECURITY ADMINISTRATION BALTIMORE MUD 21235-0001


Appendix XIXComments from the Social SecurityAdministration

COMMENTS OF THE SOCIAL SECURITY ADMINISTRATION (SSA) ON THEGOVERNMENT ACOUNTABILITY OFFICE (GAO) DRAFT REPORT,"EN'TERPRISE ARCHITECTURE: LEADERSHIP REMAINS KEY TOESTABLISHING AND LEVERAGING ARCHIITECTURES FORORGANIZATIONAL TRANSFORMATION (GAO-06-831)

Thank you for the opportunity to review and provide comments on this GAO draft report.We appreciate GAO's effort to evaluate the state of enterprise architecture (EA) withinthe major federal government departments and agencies. This information is bothinformative and useful in the strategic planning of our EA program. We agrceewithGAO's conclusions and recommendation for executive action contained in this GAOreport.

GAO Recommendation

To assist departments and agencies in addressing enterprise architecture challenges,managing their architecture programs, and realizing architecture benefits, GAOrecommends that SSA ensure that its enterprise architecture program develop andimplement plans for fully satisfying each of the conditions of GAO's enterprisearchitecture management maturity framework.

SSA Comment

We agree. We have form-ed an Enterprise Architecture Governance Committee and havededicated EA staff to ensure continued development and maturity of our EA program inaccordance with GAO's Enterprise Architecture Management Maturity Framework(EAMMF) and the Federal Enterprise Architecture Program Management Office EAAssessment Framework 2.0 requirements.

Since the time this GAO effort was completed:, we have improved and documented ourEA framework, models, documentation and implementation. As a result, the Office ofManagement and Budget (0MB) scored SSA's Framework 2.0 assessment as Green inall categories (Completion, Use and Results) during OMB's March 2006 review.

Other Comments

We have the following comments on the Appendix TV table pertaining to SSA with

regard to SSA satisfaction of core elements of the noted EAMMF stages.

Satisfaction of Stacze 2 Core Element - "EA is being developed using a framework.methodology. and automated tool"

The GAO report states in the Appendix IV table (and also on page 24) that the SSA EAmethodology does not describe the steps for developing, maintaining and validating theAgency's EA. We disagree. The Enterprise Architecture Governance Committee charter



See comment 1.

See comment 2.

and the formnal Configuration Management Plans previously submitted to GAO followfull Systems Development Life Cycle practices and thus adequately describc thc steps fordeveloping, maintaining and validating the Agency's EA.

Satisfaction of Stave 5 Core Element - " Orpanization head has roved --- v-,'=14 onof EA"

The GAO report states SSA did not provide documentation delegating EA-approvalauthority from the organization head to the Chief Information Officer (CIO). We believethe Clinger-Cohen Act of 1996, and specifically 40 U.S.C. 113 15, vests the Agency CIOwith that authority and therefore, no formal delegation in writing from the Agency headto the CIO is required.

in consideration of the above comments concerning Stages 2 and 5, it appears fromGAO's evaluation and comments that if the Stage 2 core element discussed above issatisfied by the referenced documents, then SSA's EA Maturity level would rise from a"I" to a "4" for the stage where all elements are fully satisfied.

It also follows that if the Stage 5 core element discussed above is satisfactory, then SSA'sEA Maturity level would rise from "4" to "Y" in the stage where all elements are at leastpartially satisfied.



GAO0 Comments 1. We do not agree. Neither the governance committee charter nor theconfiguration management plan explicitly describe a methodology thatincludes detailed steps to be followed for developing, maintaining, andvalidating the architecture. Rather, these do'cuments describe, forexample, the responsibilities of the architecture governance committeeand architecture configuration management procedures.

2. We do not agree. The core element in our framework concerningenterprise architecture approval by the agency head is derived fromfederal guidance and best practices upon which our framework isbased. This guidance and related practices, and thus our framework,recognize that an enterprise architecture is a corporate asset that is tobe owned and implemented by senior management across theenterprise, and that a key characteristic of a mature architectureprogram is having the architecture approved by the department oragency head. Because the Clinger-Cohen Act does not address approvalof an enterprise architecture, our framework's core element for agencyhead approval of an enterprise architecture is not inconsistent with,and is not superseded by, that act.


Appendix XX

Comments from the U.S. Agency forInternational Development

SUSAIDFROM THE AMERICAN PEOPLE

June 22, 2006

Randolph C. Hite, DirectorInformation Technology Architcctu~re

and Systems IssuesU.S. Government Accountability Office441 G Street, N.W.Washington, D.C. 20548

Dear Mr. Hite:

I am pleased to provide the U.S. Agency for International Development's(USAID) formal response on the draft GAO report entitled "EnterpriseArchitecture: Leadership Remains Key to Establishing and LeveragingArchitectures for Organizational Transformation (GAO-06-83 1)" (August 2006).

By 0MB direction, the Agency is focusing most of its resources indeveloping a Joint Department of State and USAID Enterprise Architecture. Asboth agencies now receive a combined Annual EA Asscssmcnt, we will worktogether to implement the recommendations from this report.

Thank you for the opportunity to respond to the GAO draft report and for thecourtesies extended by your staff in the conduct of this review.

David OstermeyerActing Chief Financial OfficerU.S. Agency for International Development

US. Agency for Intemational Development1 300 Pennsylania Avenue. NWWdshi~ngton. DC 20523w.usaidgav


Appendix XXI

GAO-1 Contact and Staff Acknowledgements

GAO Contact Randolph C. Hite, (202) 512-3439

StaffAcknowledgments

In addition to the person named above, Edward Ballard, Naba Barkakati,Mark Bird, Jeremy Canfield, Jamey Collins, Ed Derocher, Neil Doherty,Mary J. Dorsey, Marianna J. Dunn, Joshua Eisenberg, Michael Holland,Valerie Hopkins, James Houtz, Ashfaq Huda, Cathy Hurley, CynthiaJackson, Donna Wagner Jones, Ruby Jones, Stu Kaufman, Sandra Kerr,George Kovachick, Neela Lakhmani, Anh Le, Stephanie Lee, JayneLitzinger, Teresa M. Neven, Freda Paintsil, Altony Rice, Keith Rhodes,Teresa Smith, Mark Stefan, Dr. Rona Stillman, Amos Tevelow, and JenniferVitalbo made key contributions to this report.

(31060-1) Page 174 (31004)Page174GAO-06-831 Enterprise Architecture

GAO's Mission The Government Accountability Office, the audit, evaluation andinvestigative arm of Congress, exists to support Congress in meeting itsconstitutional responsibilities and to help improve the performance andaccountability of the federal government for the American people. GAOexamines the use of public funds; evaluates federal programs and policies;and provides analyses, recommendations, and other assistance to helpCongress make informed oversight, policy, and funding decisions. GAO'scommitment to good government is reflected in its core values ofaccountability, integrity, and reliability.

Obtaining CopiesoGAO Reports andTestimony

The fastest and easiest way to obtain copies of GAO documents at no costis through GAO's Web site (wwxNv.gao.gov). Each weekday, GAO postsnewly released reports, testimony, and correspondence on its Web site. Tohave GAO e-mail you a list of newly posted products every afternoon, go toww-mgao.gov and select "Subscribe to Updates."

Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each.A check or money order should be made out to the Superintendent ofDocuments. GAO also accepts VISA and Mastercard. Orders for 100 ormore copies mailed to a single address are discounted 25 percent. Ordersshould be sent to:U.S. Government Accountability Office441 G Street NW, Room LMWashington, D.C. 20548To order by Phone: Voice: (202) 512-6000

TDD: (202) 512-2537Fax: (202) 512-6061

To Report Fraud, Contact:

Waste, and Abuse in Web site: wvww.gao.gov/fraudnet/fraudnet.htmE-mail: [email protected]

Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470

Congressional Gloria Jarmon, Managing Director, [email protected] (202) 5124400U.S. Government Accountability Office, 441 G Street NW, Room 7125Relations Washington, D.C. 20548

Public Affairs Paul Anderson, Managing Director, [email protected] (202) 5124800U.S. Government Accountability Office, 441 G Street NW, Room 7149Washington, D.C. 20548

PRINTED ON GDRECYCLED PAPER