business driven enterprise architecture assessment methodology josh arceneaux august 16, 2011

Business Driven Enterprise Architecture Assessment Methodology

Josh Arceneaux

August 16, 2011

Agenda

• Introduction and Approach

• Assessment Methodology

• Maturity Model

• Conclusions

Introduction

• Enterprise Architecture is often thought of incorrectly as an “IT” activity

• Typically the Enterprise Architecture function resides within the CIO organization which does nothing to dispel this thought (Clinger Cohen Act)

• To derive real and tangible value, Enterprise Architecture needs to not only reside within the IT community but also be understood and practiced within the business community

Introduction

• Objectives: – Create an Enterprise Architecture Assessment methodology to determine how IT

services align with customer business needs– Create a methodology that will enable assessment of how to evolve to meet

customer needs– Create an assessment methodology that is reusable

• This presentation will discuss how we created two things to meet our objectives

1. An assessment methodology that ties Enterprise Architecture Assessments directly to key business needs

2. A maturity model that enables an objective assessment of how mature, and by therefore how valuable, an Enterprise Architecture capability is to the organization

Approach for Establishing a Comprehensive Methodology for Enterprise Architecture Assessments and Maturity Assessments

EA Assessment Methodology

NPR 2830.1

IRD Needs

NASA EA Assessment

Checklist

Industry Standards and Best Practices

(ex. IDEF0)

CMMI Principles

Enterprise Strategy and

Design Framework

(DoDAF 2.0, Zachman, FEA)

Assessment Methodology

Specific Goals

• The Enterprise Architecture assessment methodology needed to answer the following:

– Why is the assessment being performed?

– By what means is the assessment being performed?

– What resources and skills are required to perform an EA assessment?

– What are the expected products the EA assessment will deliver and how will these products be utilized in the decision making process?

Our approach begins with an Enterprise Strategy and Design framework built up from integrating leading Enterprise Architecture Frameworks from across government and industry

FEA Reference Models

Zachman Principles

DoD AF 2.0 Views

What Does the Business/Customer Want to Know?

This is typically the best starting point for most stakeholders

Level 1:EnterpriseStrategy

Level 2:Enterprise

Design

Level 3:Segment

Architecture

Level 1:EnterpriseStrategy

Level 2:Enterprise

Design

Level 3:Segment

Architecture

Str

ateg

yS

olu

tio

n

Level 4:Solution

Architecture

Str

ateg

yS

olu

tio

n

Level 4:Solution

Architecture

• How is my organization going to be impacted and evolve to accommodate change across NASA and maximize our ability to thrive?

Tying Questions and Answers to Views (DoDAF 2.0 Views)

This section is often best left to the EA performing the assessment to complete

Stakeholders typically say yes to everything creating unrealistic goals and expectations

Question View Notes

UndeterminedWould you like to identify the project vision, goals , objectives, plans, activities, events, conditions, measures, effects (outcomes) and produced objects? AV-1

Undetermined Would you like to know what terms are used throughout your enterprise architecture? AV-2

Undetermined Would you like to have a graphic to capture your operational concepts? OV-1

UndeterminedWould you like to understand the resources that flow between each activity that your organization performs? OV-2

Undetermined Would you like to further understand the resource flows and capture the attributes of those flows? OV-3

UndeterminedWould you like to understand how your organization is structured and the relationships within your organization? OV-4

Undetermined Would you like to have a hierarchy of operations that your organization performs? OV-5a

UndeterminedWould you like to know what busienss activities are being performed, their sequecning, their inputs, outputs, and controls placed on them, and how they are performed? OV-5b

Undetermined Would you like to know the business rules that constrain your operations? OV-6a

Undetermined Would you like to understand how your business processes respond to various events? OV-6b

Yes Would you like to understand how your organization operates in a given scenario? OV-6c

YesWould you like to understand the high level data concepts and relationships throughout your organization? DIV-1

EA Overview Related Questions

Operational and Business Process Related Questions

Refining the Assessment Details for building the Event Trace Description (OV6c) and the Conceptual Data Model (DIV1)

View View Title Additional context and description Knowledge and Skills Predecessor Successor

OV-6cEvent-Trace Description

Helps define performer interactions and operational threads, and captures the end-to-end business processes in a readable, graphical representation; additionally it portrays a relative order to Operational Activity execution

Supports the analysis of operational events, behavioral analysis, the identification of non-functional user requirements, and operational test scenarios by providing a time-ordered examination of the resource flows as a result of a particular scenario

Knowledge: Understanding of how the organization business processes are executed under defined scenariosSkills: Building a scenario and demonstrating how the business process would operate in that scenario.

OV-2, OV-5a, OV-6a

Refining the Assessment Details for building the Event Trace Description (OV6c) and the Conceptual Data Model (DIV1)

The next step in the assessment planning is to identify the predecessor views needed

Identifies additional views needed

Identifies additional knowledge and skills needed

View View Title Additional context and description Knowledge and Skills Predecessor Successor

DIV-1Conceptual Data Model

Information in the DIV-1 represents the same data in the DIV-2 though at a high-level. Relationships between data elements are only declared and then decomposed and made explicit in the DIV-2

Describes an architecture’s information requirements and the structure and hierarchy for that information, as well as the structural business process rules to capture data that is important to the business (usually defined in doctrine, SOPs, etc.)

Knowledge: An understanding of the top level data items that constitute the information the organization receives, processes, and delivers.Skills: Data Modeling OV-3 DIV-2

Working through the methodology worksheet you arrive at a model of what is needed to exercise use cases based on external change and evaluate the impacts to the organization business proceses

Event Trace Description

(OV6c)

Event Trace Description

(OV6c)Operational Activity Tree

(OV5a)

Operational Activity Tree

(OV5a)

Operational Resource Flow

Description (OV2)

Operational Resource Flow

Description (OV2)

Conceptual Data Model

(DIV1)

Conceptual Data Model

(DIV1)

Operational Resource Flow Matrix (OV3)

Operational Resource Flow Matrix (OV3)

High Level Operational

Concept (OV1)

High Level Operational

Concept (OV1)

Operational Activity Model

(OV5b)

Operational Activity Model

(OV5b)

Operational Rules Model

(OV6a)

Operational Rules Model

(OV6a)

What is HappeningWhen it HappensWhy it Happens

What information is being consumedWhat information is being produced

What is HappeningWhen it HappensWhy it Happens

What information is being consumedWhat information is being produced

Overview and Summary

Information (AV1)

Overview and Summary

Information (AV1)

Knowledge and Skills Needed

This will provide the Business/stakeholder with a high level model to assess various use cases against and determine how business operations will be impacted by change

Is it complete; No, but it will enable decision makers to make educated decision about how to accommodate change and further expand the architecture to answer more detailed and technical questions

View View Title Knowledge and Skills

OV-6c Event-Trace DescriptionKnowledge: Understanding of how the organization business processes are executed under defined scenariosSkills: Building a scenario and demonstrating how the business process would operate in that scenario.

DIV-1 Conceptual Data Model

Knowledge: An understanding of the top level data items that constitute the information the organization receives, processes, and delivers.Skills: Data Modeling

OV-2Operational Resource Flow Description

Knowledge: Understanding of how resources (information, people, funding, etc.) flow throughout the organizationsSkills: Modeling using various techniques and tools to capture how resources flow through an organization

OV-5aOperational Activity Decomposition Tree

Knowledge: An understanding of what operations/activities/business processes the organization executes and how they are hierarchically related.Skills: Work Breakdown Structure creation

OV-6a Operational Rules ModelKnowledge: An understanding of what rules the organizations business processes must follow.Skills: Applying rules logic to business process models

OV-5b Operational Activity ModelKnowledge: An understanding of the organizations business processes and how they are executedSkills: Business Process Modeling with BPMN and/or IDEF0

OV-1High-Level Operational Concept Graphic

Knowledge: high level understanding of what the organizations missions is and how they perform it.Skills: Ability to create a one-page graphic that conveys the organizations mission and how they execute it.

AV-1Overview and Summary Information

Knowledge: Sponsor and Stakeholder needs, EA assessment planning, EA Frameworks and dependenciesSkills: Stakeholder/sponsor relationship management

OV-3Operational Resource Flow Matrix

Knowledge: information within OV-2Skills: Ability to translate the model from OV-2 into a matrix report

Build a Project Plan and Assign Schedule and Resources

ID Task Mode

Task Name

1 Create Views2 Overview and Summary (AV1)3 High Level Operational Concept (OV1)4 Operational Activity Tree (OV5a)5 Operational Activity Model (OV5b)6 Operational Resource Flow Description (OV2)7 Operational Rules Model (OV6a)8 Event trace Description (OV6c)9 Operational Resource Flow Matrix (OV3)

10 Conceptual Data Model (DIV1)11 Build Use Cases12 Use Case 113 Use Case 214 Use Case 315 Evaluate Outcomes16 Create Alternate (to-be) Views17 Implement Transition

FJun 5, '11

Balancing scope against resources is critical to meeting expectations while delivering an actionable results to the business/stakeholders

Putting Everything Together

• An agreed upon project plan:– What business driven questions are being answered– What products are being produced from the assessment to answer the questions– What information is needed to produce the products– What resources (people, funding, and tools) are needed to produce the products– What schedule will the products be produced on

Maturity Model

Specific Goal

• Objectively measure maturity of the Enterprise Architecture Program and identify concrete activities needed to increase the maturity level

Maturity Model• There are five maturity levels that

apply to five categories– Maturity levels

0. No capability

1.Initial capability

2.Under development

3.Defined

4.Managed

5.Optimized

– Categories1.People

2.Processes

3.Technology

4.Controls

5.Strategy

• Each category has specific actions that must be in practice throughout an organization to meet a specific maturity level

• Example: To achieve Level 1 for “processes” the EA must perform the following actions

– Processes are ad-hoc, unstructured, informal and/or inconsistent across technologies and/or lines of business

– Documentation of business drivers, technology standards, and other artifacts are informal and inconsistent

– Center has decided on a methodology to support the EA Program.

– CIO communications are not standardized.

Maturity Model with Key Activities for Each Level Provide an Objective means of assessing Organizational EA Maturity

Level 0 Level 1 Level 2 Level 3 Level 4 Level 5No Capability Initial Capability Under Development Defined Managed Optimized

People

> Senior Management unaware of EA and its benefits. > Senior Management aware of EA.> Center Enterprise Architect hired. > Plans developed for EA training and certification.> IT and business organizations attempt to work together via business analyst and project manager involvement.

> Senior Management understands benefits of EA Program to the organization.> Enterprise Architecture Team established.> One or more members of the EA team are EA certified.> Portfolio Management concepts emerge and portfolios are identified.> Solution Architect role identified.

> Senior management accepts methods, costs, and schedule required to achieve EA Program benefits for the organization.> Commitment to fully developing and using EA methods and artifacts is recognized as a key to success.> Portfolio Managers assigned.> Portfolios management responsibilities defined.> Training is provided for senior management and members of EA committees.> Solution Architects viewed as trusted suppliers to the business.

> Senior management is committed to achieving principles and objectives of the EA Program and actively promotes, markets and explains the benefits of the program to leaders and managers of peer organizations across the Center.> A network of Portfolio Managers across the Center exists and informal sharing of best practices occurs.> Solution Architects are viewed as trusted advisors.

> Senior management across the Center can explain the benefits of the EA Program and actively participate and support EA functions.> EA staff focus on customer needs and measurements.> Portfolio Management is recognized as a key activity across the Center.> Solution Architects viewed as trusted partners with the business.

Processes

> Need for Enterprise Architecture processes, standard documents and templates NOT recognized.

> Processes are ad-hoc, unstructured, informal and/or inconsistent across technologies and/or lines of business.> Documentation of business drivers, technology standards, and other artifacts are informal and inconsistent.> Center has decided on a methodology to support the EA Program.> CIO communications are not standardized.

> Basic EA Program is documented, processes are planned and tracked, and the organization has begun to reuse methods for capturing critical EA information.> The organization has begun to develop plans for EA educational sessions and materials to increase awareness and understanding of EA concepts and processes.> The organization has begun to develop a compliance process to ensure projects / enhancements are consistent with EA standards. > Portfolio Management processes emerge.> Various touch-points between management processes and the EA Program framework have been mapped.

> Life cycle architecture processes have been defined and documented.> Process models have been prepared and templates are used to ensure information captured is consistent.> The architecture is well defined and communicated.> Solution Architects have a deep understanding of customer requirements that drive the EA "To-Be" state.> Portfolio Management Processes established.> Projects approved and managed within portfolios.

> Regular meetings are held to review modifications to the EA framework.> Compliance to EA standards has become common practice throughout the Center.> Implementation of customer needs assessments, cost controls and evaluation of customer satisfaction are part of the strategic planning process.> The Enterprise Architecture is segmented by Portfolio.> Portfolios are actively managed.

> Captured business and technology information is reviewed in conjunction with Innovation Management's monitoring of new technology and business trends to improve business decisions and operations. > Agencies and departments work together as contributors to the architecture and its processes.> Business influences technology and technology influences business.> There is sustainable commitment to continuous business process improvement and ROI.> Future projects pipeline is prioritized and balanced across portfolios.

Technology

> No tools or technology are used to support development of the EA.

> Analysis and rating of EA tools for organization use has begun.

> Implementation of EA tools is under way and preliminary products are available for review.> Portfolio Management tools are evaluated for implementation.

> Implementation of EA tools is complete and users are proficient.> Portfolio Management tools are implemented.

> EA tools are used to actively model current and future states and to make investment decisions.> Portfolio Management tool is implemented with fully functional workflow and users are proficient.

> Tools are continually upgraded and enhanced to provide meaningful information for management decision-making.> Integrated EA and Portfolio Management tools support reporting, collaboration, and analysis.

Controls

> Need for Enterprise Architecture in IT Governance is NOT recognized.> No EA compliance exists across the organization.

> EA compliance is informal and unstructured.> Compliance cannot be measured effectively, because processes and procedures are not consistent across areas and/or projects.

> Governance committees are forming.> EA Project reviews are standardized.> Classification of existing technology standards, business drivers, and strategic information is consistent.

> Project costs are captured and forecast within portfolios and business benefits are related to strategy within the portfolio. > The EA Compliance process is followed consistently throughout the enterprise.> Architecture Governance committees are defined, and work together smoothly.

> Metrics are captured to measure progress against the established EA Transition and Sequence Plan.> The organization captures metrics to measure savings in resources including time and money> Other metrics identify awareness, participation, acceptance, and satisfaction levels for the EA Program.> Portfolios are modeled and optimized.> EA benefits are identified and tracked.

> Governance committees review progress and effectiveness metrics. > Action plans based on captured metrics are used to increase effectiveness of the EA Program. > Architecture metrics are used to drive continuous process improvements in investment initiative summaries and business cases.> Portfolio managers have their own financial resources and full life cycle cost visibility by asset and portfolio.

Strategy

> Need for developing an Enterprise Architecture strategy NOT recognized.

> Center has decided to develop an EA Program.> EA committees and their role in IT governance have been identified.> Actions to increase awareness of EA have been identified. > CIO strategic plans developed are internally focused.> Little or no customer involvement in strategic objectives.

> Business Drivers, and strategic information have been identified.> EA-based IT governance included in CIO strategic plan.> The EA team works with other centers / agencies to share ideas for improvements to the EA Program and processes.> CIO strategic plans now identify the importance of understanding customer needs and strategic direction.

> EA Program plans are well-defined including a timeline for developing the EA Program, financial, and staffing resource requirements. > Senior management, business, and technical staff participate in various EA Program committees.> EA Program is integrated with strategic planning and budgeting processes.> Strategic planning and project planning is emphasized and focused on customer needs.

> EA Program provides consistent guidance to development and acquisition through the IT Governance process.> Strategic planning is more refined addressing customer needs, expectations, and results -- including management of ROI.

> Optimized EA Program is directed toward real business priorities and value delivered to center can be measured.> EA Program measurements help identify steps to correct deviations and support program changes.> Strategic plans are flexible and adapt to new ways of meeting customer needs, help reduce process and development costs, and achieve superior ROI.

Levels of Maturity

Enterprise Architecture Maturity Model Level 0 Level 1No Capability Initial Capability

People

> Senior Management unaware of EA and its benefits. > Senior Management aware of EA.> Center Enterprise Architect hired. > Plans developed for EA training and certification.> IT and business organizations attempt to work together via business analyst and project manager involvement.

Processes

> Need for Enterprise Architecture processes, standard documents and templates NOT recognized.

> Processes are ad-hoc, unstructured, informal and/or inconsistent across technologies and/or lines of business.> Documentation of business drivers, technology standards, and other artifacts are informal and inconsistent.> Center has decided on a methodology to support the EA Program.> CIO communications are not standardized.

Levels of Maturity

Evaluating Current Maturity Level

Overall Maturity is based on how many activities are true across each category with weighting provided to account for a mixture of level 1, 2, 3, 4, and 5 activities ongoing concurrently


People

The set of activities that the people within the organization need to be performing to achieve level 0. Predefined

ProcessesTechnologyControlsControlsStrategy

Actions Required to Attain Level:

6 16 19 21 16 17

Actions Completednumber of level 0 actions currently being performed

number of level 1 actions currently being performed




number of level I5actions currently being performed

Current % Complete calculated value calculated value calculated value calculated value calculated value calculated value

Current Maturity Level

Near Term Planned Actions

Number of not currently being performed level 0 actions that the organization will plan to be performing by the next assessment






Near Term Target % Complete

calculated value calculated value calculated value calculated value calculated value calculated value

Near Term Target Date

target date for planned actions to be operational



target date for planned actions to be operational target date for planned actions to be operationaltarget date for planned actions to be operational

Near Term Target Maturity Level

Future Planned Actions

Number of not currently being performed level 0 actions that the organization will plan to be performing at the desired long term future state






Future Target % Complete


Future Target Datetarget date for planned actions to be operational




Future Target Maturity Level

Completed ActionsNear Term Planned Actions


Levels of Maturity

Legend: (to be used in the matrix above)

In the matrix above, use these color coding to identify what activities fall within these three categories.

Level 0 Level 1No Capability Initial Capability

PeopleProcessesTechnologyControlsControlsStrategy


6 16



Current % Complete calculated value calculated value


Levels of Maturity

Planning Enterprise Architecture Maturity Improvement

By altering what activities you decide to improve upon you can predict and create what-if scenarios for your near term and long term maturity level. This provides input into investments with rationale

tied to organizational goals and objectives for EA maturity.


People

The set of activities that the people within the organization need to be performing to achieve level 0. Predefined

ProcessesTechnologyControlsControlsStrategy


6 16 19 21 16 17






number of level I5actions currently being performed

Current % Complete calculated value calculated value calculated value calculated value calculated value calculated value































Completed ActionsNear Term Planned Actions


Levels of Maturity

Legend: (to be used in the matrix above)

In the matrix above, use these color coding to identify what activities fall within these three categories.





calculated value calculated value









calculated value calculated value




Conclusions

Conclusions• This activity produced two artifacts1. A workbook for:

• Planning and guiding Enterprise Architecture assessments that fundamentally tie back to key questions that the business needs answers to

• Evaluating Enterprise Architecture maturity throughout the organization and planning improvements

2. A primer on enterprise architecture that provides an introductory level description of what Enterprise Architecture is and gives the basic knowledge needed to begin Enterprise Architecture assessments

• Benefits to Enterprise Architects across NASA– The methodology can be utilized across NASA to establish and guide architecture assessments

for any organization

– The Fundamental starting point for performing Enterprise Architecture assessments begins with a business need to understand answers to key questions

– There is a maturity model that provide objectives evaluation of how functional Enterprise Architecture is within an organization and provides clear guidance on improving

business driven enterprise architecture assessment methodology josh arceneaux august 16, 2011

Documents