enterprise modeling

Enterprise Modeling

Saad El BeleidyPeyman Jamshidi

Jared KovacsGabriel Lewis

Improving the Analysis of Alternatives (AoA) Process for IT Solutions Government

Contract Proposals

Presented at:

Contents

Context and Stakeholder AnalysisProblem and NeedDesign AlternativesSimulation DesignResults and Conclusions

2

Contents

3


IT Solutions Government Contractors

4

Project Sponsor: Civilian and National Security (CNS) Division, Vangent, Inc.

Proposal Development is critical to gaining revenueAoA: develop complex IT solution, crucial to proposal15-25 Proposals per year, of varied sizesFederal Contract Spending decreased $40 billion from 2009 to 2011

1 2 3 4 5 6 7 8 90

0.4

0.8

AoA Duration (Weeks)

Like

lihoo

d

2009 2010 2011$500

$530

$560

Fiscal Year

Fede

ral C

on-

trac

t Sp

endi

ng

(Bill

ions

)

Adapted from the Federal Times, 2011

5

Acquisitions CommitteeSolicitation Proposals

Selected Solution

Government Contractor

Bid Decisio

n

Government Entity

Requirements

Technical Solution Development

Proposal Writing

Budget & Management

Technical Solution Development / AoA

Process Level 1Proposal Development

Process

Highly competitive: 5-10 competitors for each proposal

Process Level 2 Top Level AoA Process

6

Define the Problem Domain

Define Evaluation

Criteria

Explore Alternate Solutions

Evaluate Solutions

Solicitation

Ranked List of

Alternatives

High-level “Decision and Analysis Resolution” (DAR) model provided by Vangent sponsors

0%20%40%60% 20% 10% 30% 40%

Mea

n Du

ra-tio

n (%

AoA

)

Task Category Classification1. Labor Intensive:

Little expertise required: parsing, documenting.Time Variability = Low (±22%)

2. Decision Making:Expert required: difficult decisions, judgment calls.Time Variability = High (±54%)

3. Experience Recall:Expert required: subjectively making judgments based on previous experience. Time Variability = Medium (± 44%)

4. Networking:Personal dialogue and collaboration with co-workers and others.Time Variability = High (±100%) 7

Process Level 2Top Level AoA Process

8


Define Evaluation

Criteria


Evaluate Solutions

Solicitation

Ranked List of Alternatives

High-level “Decision and Analysis Resolution” (DAR) modelprovided by Vangent sponsors

Process Level 3Phase 1: Define Problem Domain

9

Data acquired via knowledge elicitation from stakeholder SMEs

Task Category

% of Phase

Labor Intensive 25%

Decision Making 23%

Experience Recall 21%

Networking 31%


10


Define Evaluation

Criteria


Evaluate Solutions

Solicitation



Process Level 3Phase 2: Define Evaluation Criteria

11

Data acquired via knowledge elicitation from stakeholder

SMEs

Task Category

% of Phase

Labor Intensive 42%

Decision Making 20%


Networking 20%


12


Define Evaluation

Criteria


Evaluate Solutions

Solicitation



Process Level 3Phase 3: Explore Alternate

Solutions

13


SMEs

Task Category

% of Phase

Labor Intensive 36%

Decision Making 15%


Networking 36%


14


Define Evaluation

Criteria


Evaluate Solutions

Solicitation



Process Level 3Phase 4: Evaluate Solutions

15

Task Category

% of Phase

Labor Intensive 38%

Decision Making 21%


Networking 18%


SMEs

Current Issues in AoA1. Limited and variable availability

of past research due to proprietary restrictions

2. Limited and variable applicability of past research once gained

3. Variable difficulty of AoA—need for past research

16

0% 50% 100%0

0.25

0.5

Relative Expected Availabil-ity

Prob

abili

ty

0% 50% 100%0

0.25

0.5

Relative Expected Appli-cability

Prob

abili

ty

Current Issues in AoA4. Quality of AoA suffers from

the information’s lack of availability and applicability

5. One employee handles entire AoA, so potentially parallel processes are conducted in series

6. Time spent for AoA results is an entirely overhead cost

17

0% 50% 100%0%

100%

Relative Expected Availabil-ity

AoA

Out

-pu

t Re

la-

tive

Qua

l-ity

0% 50% 100%0%

50%100%

Relative Expected Appli-cability

AoA

Out-

put

Rela-

tive

Qua

l-it

y

1 2 3 4 5 6 7 8 90

0.4

0.8

$32,000

$16,000

$6000

AoA Duration (Weeks)Pr

obab

il-it

y

Key Stakeholder GoalsSolutions ArchitectsPerform Analysis of Alternatives– Maximize productivity– Avoid overtime– Maximize labor rewards

Capture ManagersManage the transition from opportunity discovery to contract award.Oversees bid strategies, pricing, and teaming– Maximize probability of winning contracts

• Increase AoA discriminability and quality

Proposal ManagersDevelop and manage the proposal plan and schedule– Maximize proposal/AoA throughput 18

Key Stakeholder Interactions

19

InteractionsProposal DemandSolution Demand

Tension:Managers and

Solutions ArchitectsLimited time and personnel resources to conduct AoAs.

Contents

20


Problem StatementDuring a time of national economic downturn, federal contract spending cuts have led to a decrease in available contract revenue and an increase in competition between government contractors. These factors have increased the time sensitivity of proposal development, specifically in the AoA process.

21

Need Statement

There is a need for Analysis of Alternatives process improvements to reduce the mean time duration by at least 33%, and the variability by 25%, while maintaining or increasing AoA proposed solution quality and keeping maximum costs below $100,000 per AoA.

22

Need validated with key stakeholders

Contents

23


Design Alternative Approach

Optimize AoA Staffing Levels

– Target Parallel Tasks– Reduce mean time duration

Information Management System

– Target efficiency increases– Reduce duration and variability

24

AoA Phase Parallel Tasks

Define Problem Domain

6Define Evaluation Criteria

4Explore Alternate Solutions

5Evaluate Solutions 4

Task Category % AoA Variabilit

yLabor Intensive 35.2% Low

±22%Decision Making 19.2% High ±

54%Experience Recall 26.3% Med ±

44%Networking 19.2% High

±100%

Optimize AoA Staffing Levels

1 Additional Solutions Architect collaborates to conduct AoA

Reduce mean time duration of AoA– Additional resource to conduct parallel tasks– Increases size of social network– Potential for conflict in making decisions

Cost: Approx. $200,000 salary including benefits per yearFive year cost: $1,000,000

25

Task Category

Labor Intensive

Decision Making

Experience Recall

Networking

Efficiency Gain --- -10% --- +10%

Information Management Alternatives

26

Implementing a File Management System– Database searching capability improves availability of information in the AoA– Increases efficiencies of task categories, improves re-use of past research

Implementing a Content Management System– Enhanced capabilities greatly improves availability of information in the AoA– Increases efficiencies of task categories, improves re-use of past research

Maintaining a Sanitized Repository– Adds new content to the information pool– Changes high-variability task categories to low-variability categories

File Management System

Benefits– Integrates with current system– Organized file structure– Promotes collaboration– Easily scalable

Drawbacks– Requires permissions for file

access– Limited search functionality

Cost: IntravationInitial Cost: 100 User LicenseGDIT(Parent Company) has 25 user license; $1000 one time fee per active user

First Year Cost: $77,000Annual Maintenance: $1600Five Year Cost: $83,000

27

Task Category

Labor Intensive

Decision Making

Experience Recall

Networking

Efficiency Gain +10% +10% +15% +5%

Content Management System

Benefits– Robust searching and indexing– Preconfigured user roles based upon

content–access needs– Authentication, check in/out,

tracking– Workflow management

Drawbacks– High complexity

• High learning curve• Expensive Technical Support

Cost: DocumentumInitial Cost:100 User License Cost: $110,665

System Cost:First Year Cost: $129,000Annual Maintenance: $19,000Five Year Cost: $203,000

28

Task Category

Labor Intensive

Decision Making

Experience Recall

Networking

Efficiency Gain +15% +15% +20% +10%

Sanitized Document Repository

Benefits– Quality of information

improvement• Promotes availability• Increases applicable content

– Virtually eliminates security risks

– Provides quicker access to data– Minimal technical support

Drawbacks– Low initial benefit

CostEstimate 12 labor hours per AoA

Five Year Cost: $150,000

29

Task Category

Labor Intensive

Decision Making

Experience Recall

Networking

Efficiency Gain +15% +10% +10% +5%Variability Low High Med High

% of AoAOriginal New

35% 40% 20% 17% 19% 17% 26%

Contents

30


Simulation DesignMonte Carlo Discrete Event Simulation of the AoA1000 ReplicationsEach Replication covers 25 Proposals (1 year)

Key Model Assumptions:1. Solutions Architects work on one task at a time2. Only one proposal is being worked on at any given time3. The four task categories adequately capture AoA time

consumption4. All tasks are of equal importance to the quality of AoA

output31

Simulation Design

32

Simulation Model

AoA Process Definition

Quality Metric

AoA Duration

Variability

AoA Mean Duration

Number of Technologie

s

AoA Difficulty

Information Availability

Information Applicability

Task Category

Efficiencies

Simulation Video

: Task Categories (Labor Intensive, Decision Making, Experience Recall, Networking)

: Inherent Task Delay : Task Category Efficiency Index (% Expected Time): Task Category Weight (% of task with specific task category): Task Category Variability Factor (RV): Technology Variability (RV): Number of Technologies/AoAs in the Proposal

Task Time Delay Equation

34

𝐷𝑒𝑙𝑎𝑦=∑𝑇𝐶

[𝐷∗𝐸∗𝑊∗𝑉 ∗𝑇 𝑉

𝑁 ]

Task Quality Metric Equation

35

: Availability Factor (RV): Applicability Factor

(RV): Stakeholder weights

Design of Experiment Matrix

36

A1: Staffing LevelsA2: Sanitized RepositoryA3: File ManagementA4: Content Management

Run Configuration AlternativesA1 A2 A3 A4

1 Baseline - - - -2 A1 x - - -3 A1, A2 x x - -4 A1, A2, A3 x x x -5 A1, A2, A4 x x - x6 A1, A3 x - x -7 A1, A4 x - - x8 A2 - x - -9 A2, A3 - x x -10 A2, A4 - x - x11 A3 - - x -12 A4 - - - x

Contents

37


A1 A1, A2 A1, A2, A3

A1, A2, A4

A1, A3 A1, A4 A2 A2, A3 A2, A4 A3 A40.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

36.53%43.91%

49.47% 51.66%

43.08% 45.42%

11.15%

20.10%23.38%

10.11%13.97%

Alternative Configurations

Perc

ent D

ecre

ase

in A

oA

Mea

n Ti

me

Dur

atio

n

38

33.00%

A1: Optimized Staffing Levels; A2: Sanitized Repository;A3: File Management; A4: Content Management

Simulation ResultsPercent Decrease in Mean Duration

A1 A1, A2 A1, A2, A3

A1, A2, A4

A1, A3 A1, A4 A2 A2, A3 A2, A4 A3 A40.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

34.38%37.50%

50.00% 50.00%

37.50% 37.50%

6.25%

12.50%

18.75%

6.25% 6.25%


Perc

ent D

ecre

ase

in A

oA

Tim

e Du

ratio

n Va

riab

ility

Simulation Results

Percent Decrease in Duration Variability

39

25.00%


Simulation ResultsQuality Metric

40

Baseline

A1 A1, A2

A1, A2, A3

A1, A2, A4

A1, A3

A1, A4

A2 A2, A3

A2, A4

A3 A40.7

0.72

0.74

0.76

0.78

0.8

0.82

0.84


Qua

lity

Met

ric

A1: Optimized Staffing Levels; A2: Sanitized Repository;

A3: File Management; A4: Content Management

Using a Sanitized Repository is the only proposed alternative that directly affects quality.

~10% Increase in Quality

0.74Baselin

e

Utility

Mean Duration

Duration Variability

AoA Output Quality

Solution Quality

41

Stakeholders’ Utility Function

Value Hierarchy obtained via Stakeholder values elicitation weighted with the Swing Weight Method

.238 .048 .143 .571

Alternative Utility Ranking

42

RankAlternative Configuratio

nUtility

1 A1, A2, A4 4.252 A1, A2 3.953 A1, A2, A3 3.904 A2, A4 3.905 A2 3.546 A1, A4 3.497 A2, A3 3.288 A4 3.189 A1, A3 3.1410 A1 3.0911 Baseline 2.5012 A3 2.37

A1: Optimized Staffing Levels A2: Sanitized Repository A3: File ManagementA4: Content Management

Meet The Need

Don’t Meet The Need

43

Sensitivity AnalysisPercent Change in Criteria Weight Necessary to Change Utility Rank

Alternative Configuration

s

Utility Function Criteria

Mean Time Usability Integrability

A1, A2 +33% +41% +69%A1, A2, A3 +34% +48% +119%

A2, A4 +60% +34% +60%

-$5,000.00 $15,000.00 $35,000.00 $55,000.00 $75,000.00 $95,000.001

1.5

2

2.5

3

3.5

4

4.5

5

Baseline

A1

A1, A2 A1, A2, A3A1, A2, A4

A1, A3A1, A4

A2A2, A3

A2, A4

A3

A4

Cost per AoA

Util

ity

44

Cost-Benefit Analysis


Meet the Duration and Variability Reduction NeedDo Not Meet the Duration and Variability Reduction Need

-$5,000.00 $15,000.00 $35,000.00 $55,000.00 $75,000.00 $95,000.001

1.5

2

2.5

3

3.5

4

4.5

5

Baseline

A1

A1, A2 A1, A2, A3A1, A2, A4

A1, A3A1, A4

A2A2, A3

A2, A4

A3

A4

Cost per AoA

Util

ity

45

Cost-Benefit Analysis


Meet the Duration and Variability Reduction NeedDo Not Meet the Duration and Variability Reduction Need

More Desirable

Closest to Desirable Region

46

RecommendationsOptimize Staffing Levels and Maintain a Sanitized Repository (A1, A2)

– Percent Reduction in Time Duration: 44%– Percent Reduction in Duration Variability: 38%– Percent Increase in Quality: 10%– Total Utility: 3.95– Max Expected Cost per AoA: $50,000– Total Implementation Cost: $230,000/year

47

RecommendationsOptimize Staffing Levels, Maintain a Sanitized Repository, and Implement a Content Management System (A1, A2, A4)– Percent Reduction in Time Duration: 52%– Percent Reduction in Duration Variability: 50.00%– Percent Increase in Quality: 10%– Total Utility: 4.25– Maximum Expected Cost per AoA: $78,000– Total Implementation Cost: $111,000 plus

$249,000/yr

(Potential Value, if Cost Reduced)

Questions?

48

Backup Slides

49

Project Plan

50

WBS – Top LevelEnterprise Modeling

Project

1.0 Project Definition

2.0 Requirements Development

3.0 Solution Development

4.0 Modeling and Testing

5.0 Results Analysis

6.0 Communications and Management

Syst 495-Spring 2012

Context

Syst 490-Fall 2011 51

Output of AoA: Ranked Set of Alternatives

AoA output for “Case Management Platforms” (Vangent, Inc)

Vendors Considered: Custom Developed Siebel IBM Microsoft

Total Scores

Syst 495-Spring 2012 52

Stakeholders’ Value Hierarchy

Value Hierarchy obtained via Stakeholder values elicitation weighted with the Swing Weight Method

AoA Process Improvement

Objective

Reduce Mean Time

Maximize Usability

Reduce Time Variability

Maximize Integrability

Maximize Tailorability

Maximize AoA Quality

Maximize Tech Support

Maximize Scalability

.238 .048 .143 .024

.071.119.167.190

Information Management Alternatives



Define Evaluation

Criteria


Evaluate Solutions

Retrieve Past Proposal Requirements

Maintain List of SME’sRetrieve Past Alternative Material

Retrieve Past Criteria from similar proposals

Provide reference of past analyses

Optimizing Staffing Levels Alternative


Time ReductionParallel Processes


Define Evaluation

Criteria


Evaluate Solutions

Percent Decrease

55Syst 495-Spring 2012

Baseline

A1 A1, A2

A1, A2, A3

A1, A2, A4

A1, A3

A1, A4

A2 A2, A3

A2, A4

A3 A40.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

Mean Time (Filled) Variability (Hollow)A1: Optimized Staffing Levels; A2: Sanitized Repository;

A3: File Management; A4: Content Management

33%

25%

Alternatives Efficiency Summary


Alternative

Labor Intensiv

eDecision Making

Experience Recall

Networking

Baseline --- --- --- ---Optimizing Staffing Levels

--- -10% --- +10%Sanitized Repository

+15% +10% +10% +5%File Management

+10% +10% +15% +5%Content Management

+15% +15% +20% +10%

Alternatives’ Efficiency Indexes


Base-line

A1 A1, A2 A1, A2, A3

A1, A2, A4

A1, A3 A1, A4 A2 A2, A3 A2, A4 A3 A40.6

0.7

0.8

0.9

1

1.1

1.2Labor Intensive Decision Making Experience Recall Networking


Effici

ency

Inde

x



Acquisitions Committee

Solicitation Proposal

Selected Solution

Government Contractor

Bid Decisio

n

Government Entity

Requirements


Proposal Writing

Budget & Management


Process Level 1Proposal Development

Process

Proposal Development is critical to gaining revenueHighly competitive: 5-10 competitors for each proposal


Process Level 2Technical Solution

Development

59

Alternatives

AnalysisIntegration of Assets

SolicitationRanked List

of Alternatives

Assets for Proposal

Solution to be Proposed

Analysis of Alternatives

(AoA)

Model developed based on team research and knowledgeelicitation from Vangent sponsors

Process Level 3 Top Level AoA Process



Define Evaluation

Criteria


Evaluate Solutions

Solicitation



enterprise modeling

Documents

level decision

networking20 process

level aoa process6define

level aoa process8define

level aoa process10define

level aoa process12define

time sensitive5process

vangent sponsors