enterprise modeling - george mason university...project plan and budget syst 495-spring 2012 2 . ......

Enterprise Modeling

Saad El Beleidy Peyman Jamshidi

Jared Kovacs Gabriel Lewis

Improving the Analysis of Alternatives (AoA) Process for IT Solutions Government Contract Proposals

Contents

!   Context and Stakeholder Analysis !   Problem and Need

!   Design Alternatives !   Simulation Design !   Design of Experiment !   Results !   Project Plan and Budget

Syst 495-Spring 2012 2

Contents

!  Context and Stakeholder Analysis !   Problem and Need !   Design Alternatives !   Simulation Design !   Design of Experiment !   Results !   Project Plan and Budget


IT Solutions Government Contractors


Project Sponsor: Civilian and National Security (CNS) Division, Vangent, Inc.

!   Proposal Development is critical to gaining revenue

!   15-25 Proposals per year, of varied sizes

!   Complex proposals, large scope of technology

!   Federal Contract Spending decreased $40 billion from 2009 to 2011

0

0.4

0.8

0 2 4 6 8 10

Like

lihoo

d

AoA Duration (Weeks)

$500

$530

$560

2009 2010 2011 Fede

ral C

ontr

act

Spen

ding

(Bi

llion

s)

Fiscal Year Adapted from the Federal Times, 2011


Acquisitions Committee Solicitation Proposals

Selected Solution

Government Contractor

Bid Decision

Government Entity

Requirements

Technical Solution Development

Proposal Writing

Budget & Management

Technical Solution Development / AoA

Process Level 1 Proposal Development Process

Highly competitive: 5-10 competitors for each proposal

Process Level 2 Top Level AoA Process


Define the Problem Domain

Define Evaluation

Criteria

Explore Alternate Solutions

Evaluate Solutions

Solicitation

Ranked List of Alternatives

High-level “Decision and Analysis Resolution” (DAR) model provided by Vangent sponsors

20% 10% 30% 40%

0%

20%

40%

60%


Define Evaluation Criteria


Evaluate Solutions Mea

n D

urat

ion

(% A

oA)

Task Category Classification

1. Labor Intensive: Little expertise required: parsing, documenting. Time Variability = Low (±22%)

2. Decision Making: Expert required: difficult decisions, judgment calls. Time Variability = High (±54%)

3. Experience Recall: Expert required: subjectively making judgments based on previous experience. Time Variability = Medium (± 44%)

4. Networking: Personal dialogue and collaboration with co-workers and others. Time Variability = High (±100%)





Define Evaluation

Criteria


Evaluate Solutions

Solicitation



Process Level 3 (AoA) Phase 1: Define Problem Domain


Data acquired via knowledge elicitation from stakeholder SMEs

Task Category

% of Phase

Labor Intensive 25%

Decision Making 23%

Experience Recall 21%

Networking 31%




Define Evaluation

Criteria


Evaluate Solutions

Solicitation



Process Level 3 (AoA) Phase 2: Define Evaluation Criteria

11


Syst 495-Spring 2012

Task Category

% of Phase

Labor Intensive 42%

Decision Making 20%


Networking 20%




Define Evaluation

Criteria


Evaluate Solutions

Solicitation



Process Level 3 (AoA) Phase 3: Explore Alternate Solutions

13 Syst 495-Spring 2012


Task Category

% of Phase

Labor Intensive 36%

Decision Making 15%


Networking 36%




Define Evaluation

Criteria


Evaluate Solutions

Solicitation



Process Level 3 (AoA) Phase 4: Evaluate Solutions


Task Category

% of Phase

Labor Intensive 38%

Decision Making 21%


Networking 18%


Current Issues in AoA 1.  Limited and variable availability of

past research due to proprietary restrictions

2.  Limited and variable applicability of past research once gained

3.  Variable difficulty of AoA—need for past research


0

0.25

0.5

0% 50% 100%

Prob

abili

ty

Relative Expected Availability

0

0.25

0.5

0% 50% 100%

Prob

abili

ty

Relative Expected Applicability

Current Issues in AoA 4.  Quality of AoA suffers from the

information’s lack of availability and applicability

5.  One employee handles entire AoA, so potentially parallel processes are conducted in series

6.  Time spent for AoA results is an entirely overhead cost


0%

50%

100%

0% 50% 100%

AoA

Out

put

Rela

tive

Q

ualit

y

Relative Expected Availability

0%

50%

100%

0% 50% 100%

AoA

Out

put

Rela

tive

Q

ualit

y

Relative Expected Applicability

$32,000

$16,000

$6000 0

0.4

0.8

0 2 4 6 8 10

Prob

abili

ty

AoA Duration (Weeks)

Key Stakeholder Goals !   Solutions Architects

Perform Analysis of Alternatives

–  Maximize productivity –  Avoid overtime –  Maximize labor rewards

!   Capture Managers Manage the transition from opportunity discovery to contract award Oversees bid strategies, pricing, and teaming

–  Maximize probability of winning contracts •  Increase AoA discriminability and quality

!   Proposal Managers Develop and manage the proposal plan and schedule –  Maximize proposal/AoA throughput


Key Stakeholder Interactions


Interactions Proposal Demand Solution Demand

Tension: Managers and

Architects Limited time and personnel resources to conduct solutions development.

Contents

!   Context and Stakeholder Analysis

!  Problem and Need !   Design Alternatives !   Simulation Design !   Design of Experiment !   Results !   Project Plan and Budget


Problem Statement

During a time of national economic downturn, federal contract spending cuts have led to a decrease in available contract revenue and an increase in competition between government contractors. These factors have increased the time sensitivity of proposal development, specifically in the AoA process.


Need Statement

There is a need for Analysis of Alternatives process improvements to reduce the mean time duration by at least 33%, and the variability by 25%, while maintaining or increasing AoA proposed solution quality and keeping maximum costs below $100,000 per AoA.


Need validated with key stakeholders

Contents


!  Design Alternatives !   Simulation Design !   Design of Experiment !   Results !   Project Plan and Budget


Design Alternative Approach

!   Optimize AoA Staffing Levels

–  Target Parallel Tasks –  Reduce mean time duration

!   Information Management System

–  Target efficiency increases –  Reduce duration and variability

24

AoA Phase Parallel Tasks

Define Problem Domain 6

Define Evaluation Criteria 4

Explore Alternate Solutions 5

Evaluate Solutions 4

Task Category % AoA Variability

Labor Intensive 35.2% Low ±22%

Decision Making 19.2% High ± 54%

Experience Recall 26.3% Med ± 44%

Networking 19.2% High ±100%


Optimize AoA Staffing Levels

1 Additional Solutions Architect collaborates to conduct AoA

!   Reduce mean time duration of AoA –  Additional resource to conduct parallel tasks –  Increases size of social network –  Potential for conflict in making decisions

!   Cost: Approx. $200,000 salary including benefits per year Five year cost: $1,000,000


Task Category

Labor Intensive

Decision Making

Experience Recall

Networking

Efficiency Gain --- -10% --- +10%

Information Management Alternatives

Division-wide database system to facilitate storage, retrieval and sharing of technical solution material in AoA

!   Stakeholders currently use Microsoft Sharepoint for file sharing –  Insufficient level of access to past data, lacking product features –  Low quality system that is underutilized

!   Alternatives aim to: –  Reduce mean duration of AoA

•  Enable re-use of past technical solution material –  Reduce time variability

•  Change high variability task types to low variability task type –  Maintain or increase quality

•  Increase availability of information


File Management System

!   Benefits –  Integrates with current system –  Organized file structure –  Promotes collaboration –  Easily scalable

!   Drawbacks –  Requires permissions for file access –  Limited search functionality

!   Cost: Intravation Initial Cost: 100 User License GDIT(Parent Company) has 25 user license; $1000 one time fee per active user

First Year Cost: $76,615 Annual Maintenance: $1615 Five Year Cost: $83,075

27

Task Category

Labor Intensive

Decision Making

Experience Recall

Networking

Efficiency Gain +10% +10% +15% +5%


Content Management System

!   Benefits –  Robust searching and indexing –  Preconfigured user roles based upon

content–access needs –  Authentication, check in/out, tracking –  Workflow management

!   Drawbacks –  High complexity

•  High learning curve •  Expensive Technical Support

!   Cost: Documentum Initial Cost: 100 User License Cost: $110,665

System Cost: First Year Cost: $129,080 Annual Maintenance: $18,415 Five Year Cost: $202,740

28

Task Category

Labor Intensive

Decision Making

Experience Recall

Networking



Sanitized Document Repository

!   Benefits –  Quality of information improvement

•  Promotes availability •  Increases applicable content

–  Virtually eliminates security risks –  Provides quicker access to data –  Minimal technical support

!   Drawbacks –  Low initial benefit

!   Cost Estimate 12 labor hours per AoA

Five Year Cost: $150,000

29

Task Category

Labor Intensive

Decision Making

Experience Recall

Networking


Variability Low High Med High

% of AoA Original New 35% 40% 20% 17% 19% 17% 26%


Contents


!   Design Alternatives

!  Simulation Design !   Design of Experiment !   Results !   Project Plan and Budget


Simulation Design

!   Monte Carlo Discrete Event Simulation of AoA !   1000 Replications !   Each Replication covers 25 Proposals (1 year)

!   Key Model Assumptions: 1.  Solutions Architects work on one task at a time 2.  Only one proposal is being worked on at any given time 3.  The four task categories adequately capture time

consumption in AoA 4.  All tasks are of equal importance to the quality of AoA output


Simulation Design


Simulation Model

AoA Process Definition

Quality Metric

AoA Time Variability

AoA Mean Time

Number of Technologies

AoA Difficulty

Information Availability

Information Applicability

Task Category Efficiencies

Task Time Delay Equation


: Task Categories (Labor Intensive, Decision Making, Experience Recall, Networking)

: Inherent Task Delay

: Task Category Efficiency Index (% Expected Time)

: Task Category Weight (% of task with specific task category)

: Task Category Variability Factor (RV)

: Technology Variability (RV)

: Number of Technologies/AoAs in the Proposal

Task Quality Metric Equation


: Availability Factor (RV)

: Applicability Factor (RV)

: Stakeholder weights

Contents


!   Design Alternatives !   Simulation Design

!  Design of Experiment !   Results !   Project Plan and Budget


Design of Experiment Matrix


!   A1: Staffing Levels !   A2: Sanitized

Repository !   A3: File

Management !   A4: Content

Management

Run Configuration Alternatives A1 A2 A3 A4

1 Baseline - - - - 2 A1 x - - - 3 A1, A2 x x - - 4 A1, A2, A3 x x x - 5 A1, A2, A4 x x - x 6 A1, A3 x - x - 7 A1, A4 x - - x 8 A2 - x - - 9 A2, A3 - x x - 10 A2, A4 - x - x 11 A3 - - x - 12 A4 - - - x

Alternatives Efficiency Summary


Alternative Labor Intensive

Decision Making

Experience Recall Networking

Baseline --- --- --- --- Optimizing Staffing Levels

--- -10% --- +10%

Sanitized Repository +15% +10% +10% +5%

File Management +10% +10% +15% +5%

Content Management +15% +15% +20% +10%

Alternatives’ Efficiency Indexes


0.6

0.7

0.8

0.9

1

1.1

1.2

Baseline A1 A1, A2 A1, A2, A3

A1, A2, A4

A1, A3 A1, A4 A2 A2, A3 A2, A4 A3 A4

Effi

cien

cy In

dex

Alternative Configurations

Labor Intensive Decision Making Experience Recall Networking

A1: Optimized Staffing Levels; A2: Sanitized Repository; A3: File Management; A4: Content Management

Contents


!   Design Alternatives !   Simulation Design !   Design of Experiment

!  Results !   Project Plan and Budget


36.53% 43.91%

49.47% 51.66%

43.08% 45.42%

11.15%

20.10% 23.38%

10.11% 13.97%

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

A1 A1, A2 A1, A2, A3

A1, A2, A4

A1, A3 A1, A4 A2 A2, A3 A2, A4 A3 A4 Perc

ent

Dec

reas

e in

AoA

Mea

n Ti

me

Dur

atio

n



33.00%


Simulation Results Percent Decrease in Mean Duration

34.38%

37.50%

50.00% 50.00%

37.50% 37.50%

6.25%

12.50%

18.75%

6.25% 6.25%

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

A1 A1, A2 A1, A2, A3

A1, A2, A4

A1, A3 A1, A4 A2 A2, A3 A2, A4 A3 A4 Perc

ent

Dec

reas

e in

AoA

Tim

e D

urat

ion

Vari

abili

ty


Simulation Results Percent Decrease in Duration Variability


25.00%


Simulation Results Quality Metric


0.7

0.72

0.74

0.76

0.78

0.8

0.82

0.84

Qua

lity

Met

ric



Using a Sanitized Repository is the only proposed alternative that directly affects quality.

10.2% Increase in Quality

0.74 Baseline

Utility

Mean Duration

Duration Variability

AoA Output Quality

Solution Quality


Stakeholders’ Utility Function

Value Hierarchy obtained via Stakeholder values elicitation weighted with the Swing Weight Method

.238 .048 .143 .571

Alternative Utility Ranking


Rank Alternative Configuration Utility

1 A1, A2, A4 4.25 2 A1, A2 3.95 3 A1, A2, A3 3.90 4 A2, A4 3.90 5 A2 3.54 6 A1, A4 3.49 7 A2, A3 3.28 8 A4 3.18 9 A1, A3 3.14 10 A1 3.09 11 Baseline 2.50 12 A3 2.37

A1: Optimized Staffing Levels

A2: Sanitized Repository

A3: File Management

A4: Content Management

Meet The Need

Don’t Meet The Need


Sensitivity Analysis Percent Change in Criteria Weight Necessary to Change Utility Rank


Utility Function Criteria

Mean Time Usability Integrability A1, A2 +33% +41% +69%

A1, A2, A3 +34% +48% +119% A2, A4 +60% +34% +60%

Baseline

A1

A1, A2 A1, A2, A3

A1, A2, A4

A1, A3 A1, A4

A2 A2, A3

A2, A4

A3

A4

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

-$5,000.00 $15,000.00 $35,000.00 $55,000.00 $75,000.00 $95,000.00

Uti

lity

Cost per AoA


Cost-Benefit Analysis


Meet the Duration and Variability Reduction Need

Do Not Meet the Duration and Variability Reduction Need

Baseline

A1

A1, A2 A1, A2, A3

A1, A2, A4

A1, A3 A1, A4

A2 A2, A3

A2, A4

A3

A4

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

-$5,000.00 $15,000.00 $35,000.00 $55,000.00 $75,000.00 $95,000.00

Uti

lity

Cost per AoA


Cost-Benefit Analysis


Meet the Duration and Variability Reduction Need

Do Not Meet the Duration and Variability Reduction Need

Closest to Desirable Region


Recommendations

!   Optimize Staffing Levels and Maintain a Sanitized Repository (A1, A2)

–  Percent Reduction in Time Duration: 43.91% –  Percent Reduction in Duration Variability: 37.50% –  Percent Increase in Quality: 10.21% –  Total Utility: 3.95 –  Max Expected Cost per AoA: $50,000 –  Total Implementation Cost: $230,000/year


Recommendations

!   Optimize Staffing Levels, Maintain a Sanitized Repository, and Implement a Content Management System (A1, A2, A4) –  Percent Reduction in Time Duration: 51.66% –  Percent Reduction in Duration Variability: 50.00% –  Percent Increase in Quality: 10.17% –  Total Utility: 4.25 –  Maximum Expected Cost per AoA: $78,000 –  Total Implementation Cost: $111,000 plus $249,000/yr

(Potential Value, if Cost Reduced)

Contents


!   Design Alternatives !   Simulation Design !   Design of Experiment !   Results

!  Project Plan and Budget


Project Plan

51

!  WBS – Top Level Enterprise Modeling

Project

1.0 Project Definition

2.0 Requirements Development

3.0 Solution Development

4.0 Modeling and Testing

5.0 Results Analysis

6.0 Communications and Management


Project Budget


$ Total Project Expected Cost $130,000.00

Total Burden per Person $62.40

Management Reserve $8,000.00

Budget Cost Work Performed (BCWP) $132,500.00

Actual Cost Work Performed (ACWP) $137,500.00

Budget Cost Work Scheduled (BCWS) $124,800.00 Cost Performance Index (CPI) 0.96 Schedule Performance Index (SPI) 0.94

Project Budget


$0.00

$20,000.00

$40,000.00

$60,000.00

$80,000.00

$100,000.00

$120,000.00

$140,000.00

$160,000.00

1 5 9 13 17 21 25 29 33 37

BCWP ACWP BCWS

Time (Week)

Cost

Management Reserve

Questions?


Backup Slides


Appendix A:

Syst 490-Fall 2011 56

!   Output of AoA: Ranked Set of Alternatives

AoA output for “Case Management Platforms” (Vangent, Inc)

Vendors Considered: !   Custom Developed !   Siebel !   IBM !   Microsoft

Total Scores

Stakeholders’ Value Hierarchy

Reduce Mean Time

Maximize Usability

Reduce Time Variability

Maximize Integrability

Maximize Tailorability

Maximize AoA Quality

Maximize Tech Support

Maximize Scalability


Appendix B: Stakeholders’ Value Hierarchy

Value Hierarchy obtained via Stakeholder values elicitation weighted with the Swing Weight Method

.238 .048 .143 .024

.071 .119 .167 .190


Appendix C: Simulation Design - 1


Simulation Initiation



AoA Phase 1: Define the Problem Domain



AoA Phase 2: Define

Evaluation Critera



AoA Phase 3: Explore

Alternate Solutions



AoA Phase 4: Evaluate Solutions



Simulation Variable

Assignments



Simulation Calculations


enterprise modeling - george mason university...project plan and budget syst 495-spring 2012 2 . ......

Documents