abstract 2009 aashto value engineeringsp.design.transportation.org/documents/19-hays.pdf ·...

Abstract

2009 AASHTO Value Engineering Conference

Improving the Value of VE Studies on Transportation Projects

By

R. Terry Hays, CVS‐Life

Nine years ago at a Caltrans VA Program meeting, Jack Boda, then head of Caltrans Project and Program Management, asked the question, “While the [Value Analysis] studies are resulting in significant cost savings, what is it doing for the Performance of the projects?” That question led to the development of the Value Metrics approach, where the benefit to the project is now measured in terms of Value (Performance/Cost). As this process has been continually refined over the past 9 years, one of the most critical aspects is how properly to define Performance Attributes for transportation projects. This paper will discuss what constitutes a good performance attribute and why, and what kinds of attributes should be avoided. With well‐defined performance attributes, valid performance improvements can be identified in conjunction with cost savings, and the value of the improved project documented. The paper will focus on this topic and how to assess changes in project performance appropriately. The results from years of application of Value Metrics will be discussed and presented in support of this approach.

Time Duration: 30‐45 minutes

Presentation Format: Presentation of paper and software demonstration

Final Paper Contents: PowerPoint, Word, and Excel files

Audio/Visual Needs: LCD Projector

About the Author:

R. Terry Hays, CVS‐Life, Fellow ‐ SAVE is President of Value Management Strategies, Inc. Terry has over 25 years experience in leading value studies, providing VE training, and implementing VE in organizations. He has been working as a consultant to the California Department of Transportation (Caltrans) for 13 years. During that time, Terry has been the consultant contract manager on over 700 studies for Caltrans projects and led over 150 VE studies on transportation projects. Terry is also the past president of SAVE International.

Contact Address:


President

Value Management Strategies, Inc.

613 West Valley Parkway, Suite 240

Escondido, California 92025

(760) 741‐5518

terry@vms‐inc.com

www.vms‐inc.com

Improving the Value of VE Studies on Transportation Projects

By


INTRODUCTION

Nine years ago at a Caltrans VA Program meeting, Jack Boda, then head of Project and Program Management at Caltrans, asked the question, “While the [Value Analysis] studies are resulting in significant cost savings, what is it doing for the Performance of the projects?” That question led to the development of project performance measures, and eventually to the Value Metrics approach, where the benefit to the project is now measured in Value (Performance/Cost). More importantly, making Value Metrics (VM) inherent to the Value Analysis (VA) / Value Engineering (VE) process has resulted in creating a more robust VE process, whereby the project’s performace requirements and attributes are integral to each step of the VE Job Plan.

As this process has been continually refined over the past 9 years, one of the most important issues to emerge is how to define Performance Attributes properly for transportation projects. This paper will discuss what constitutes a good performance attributes and why, and what types of attributes should be avoided. With well‐defined attributes, performance improvements can be measured in conjunction with cost savings, and improvements to project value documented.

WHY IS VALUE METRICS IMPORTANT?

Value Engineering has traditionally been perceived as a “cost cutting” tool – one that has been mistakenly perceived as sacrificing project performance for the sake of cost. This paradigm only addresses one part of the value equation, often at the expense of the role that VE can play with regard to improving project performance. Project costs are fairly easy to quantify and compare, performance is not. As a result, many organizations have been reluctant to use VE to address their real problems and only use the methodology when required.

With Value Metrics integrated into VE studies, it is possible to identify those project requirements and attributes that are most important to project stakeholders and to integrate this information into studies so that more holistic, value‐based solutions can be developed. This approach has consistently resulted in the development of alternatives that are better balanced with respect to the project’s purpose and need. Value Metrics also permits project stakeholders to base their decisions on total value to the project, not merely cost savings.

WHAT IS VALUE METRICS?

Value Metrics is a system of techniques designed to enhance the traditional Value Methodology. It is predicated upon the theory that value is an expression of the relationship between the performance of a function and the cost of acquiring it. Value Metrics integrates critical performance attributes into the VE process at each stage of the Job Plan to provide better results.

This process, Value Metrics, emphasizes the interrelationship between cost and performance and can be quantified and compared in terms of how they contribute to overall value. The basic equation used for calculating value is:

In other words, value is equivalent to the relationship of the resources needed to provide a certain level of performance for a given function. Recently, and primarily through the integration of Risk into VE studies, we have seen the benefit of expanding this relationship further to better incorporate Schedule and Risk into the process.

If V = value, F = function (i.e., project objective), P = performance, C = cost, S = schedule and R = risk, then we can potentially expand on this algorithm in the following ways:

or The main difference with these two equations is how time (i.e., schedule) is addressed. In the first equation, schedule is considered a resource (i.e., input). In the second, schedule is considered an aspect of performance (i.e., output). While schedule is typically one of the performance attributes assessed, for many projects, schedule is driving the project as much or more than cost.

This expanded concept permits us to take one step further and incorporate the concept of uncertainty (Risk) through the application of quantitative risk analysis. This allows us to assess the impact of uncertainty on cost and schedule. If R = risk, then the following formulas could be applied:

or Basically, cost and schedule are not discrete numbers, rather ranges of numbers based on their probability of occurring. The Value Metrics process has been refined to accommodate these variables. In the end, it is not the number that is important, rather the information provided to the decision makers to help them in their assessment of the alternatives and their decisions.

Value Metrics provides a standardized means of identifying, defining, evaluating, and measuring performance. Once this has been achieved and costs for all value alternatives have been developed, measuring value is a relatively simple matter.

This paper will first consider project schedule as an output, that is, a performance attribute. At the end of this paper, methods to consider schedule as an input (like cost) will be presented.

Value PerformanceCost

VF PC S or VF P S

C

VF PC S x R VF P S x R

C x Ror

ISN’T VALUE METRICS JUST A WEIGHTED MATRIX?

No. While Value Metrics had its origins in the Performance Measures process, the measurement of Performance and Value is just a part of the overall Value Metrics process. It is the shift in focus from Cost Reduction to Total Value Improvement throughout the application of the VE Job Plan that is responsible for improving the VE study. Being able to identify, define, understand and compare performance attributes is important. Maintaining project performance in a manner consistent with project costs leads to the development of better alternatives for the project and permits decisions to be made based on total value.

It is important to note that the performance attributes of a project are seldom of equal importance. Therefore, a systematic process must be utilized in order to determine their relative importance in meeting the project’s purpose and need. Many systems and matrices are used by practitioners, but most are inadequate due to a variety of practical and/or mathematical problems.

Value Metrics utilizes a methodology known as the Analytic Hierarchy Process (AHP), developed by Dr. Thomas Saaty. This process has been thoroughly vetted within the academic community for over 30 years and is widely regarded as the preeminent decision structure in existence today. AHP makes use of a mathematical algorithm known as an eigenvector. Data is elicited from project stakeholders and is organized using this algorithm. The system utilizes scaled pairwise comparisons to develop numerical preferences for performance attributes. Performance scales are employed to rate how well alternative concepts are performing. This data is then synthesized to generate value indices which allow stakeholders to evaluate alternatives based on total performance. This process will be discussed in more detail later in this paper.

DOES VALUE METRICS REALLY IMPROVE VE STUDIES?

Value Metrics has been in place in the Caltrans for over nine years. In addition, numerous other agencies and companies in the U.S. and overseas have adopted this as the standard way to conduct VE studies. While the process has continued to be refined over this period, the results show:

♦ The total number of alternatives developed on studies has reduced while implementation rates have increased. This indicates that a more efficient system has evolved that produces alternatives that benefit projects and do not waste stakeholder time and effort.

♦ The image of the Caltrans VA Program has markedly improved. Project Managers are more prone to view the process as a resource rather than a requirement because the alternatives are value‐based rather than merely cost‐based.

♦ Value Metrics encourages participation from external stakeholders and ensures that their preferences for performance have been considered. Many view this as an important opportunity to have their voices heard and provide valuable project input.

♦ Performance‐reducing alternatives – often of the cost‐cutting, scope‐reducing type – have been minimized or eliminated.

♦ Value Metrics has provided a structure for better evaluating alternatives that improve project performance at an additional cost.

♦ Total cost savings per study have increased by over 50%.

♦ Performance on projects has been improved, on average, by over 15%.

ARE THERE OTHER BENEFITS THAT VALUE METRICS PROVIDES?

The application of Value Metrics has led to numerous other benefits for both participants and projects. For participants, Value Metrics has been observed to:

♦ Facilitate an improved understanding of the relationship of cost and performance to function;

♦ Differentiate between performance requirements and attributes;

♦ Enhance F.A.S.T. diagramming by allowing performance dimensioning;

♦ Augment group creativity through the consideration of performance attributes;

♦ Improve the evaluation of ideas through the consideration of performance attributes;

♦ Develop alternative solutions to problems that equally consider cost and performance;

♦ Prepare performance assessments of value alternatives;

♦ Evaluate alternative solutions relative to total value improvement;

♦ Evaluate competing solutions through the comparison of value indices; and

♦ Allow for the consideration of risk, either qualitatively or quantitatively.

For projects, Value Metrics:

♦ Builds consensus among project stakeholders (especially those holding conflicting views);

♦ Develops a better understanding of a project’s goals and objectives as they relate to Purpose and Need;

♦ Develops a baseline understanding of how the project is meeting performance goals and objectives;

♦ Identifies areas where project performance can be improved through the VM process;

♦ Develops a better understanding of an alternative concept’s effect on project performance;

♦ Develops a deeper understanding of the relationship between performance and cost in determining value; and

♦ Uses value as the basis for selecting the best project or design concept.

IMPROVING THE VE JOB PLAN

Value Metrics is a fully integrated process that compliments traditional Value Methodology at every step of the traditional Job Plan. One important benefit is that the value team is sensitized to and focuses on total value improvement and not just cost reduction. How the Job Plan is enhanced is summarized below. Discussion of each technique will be included later in the paper.

Preparation Phase

Value Metrics begins prior to the value study. During the pre‐study, or Preparation Phase, performance requirements and attributes are first identified and defined by the project team and stakeholders. Once this is completed, the development of performance rating scales is initiated in preparation for the value study. Major activities include:

♦ Identifying and defining performance requirements;

♦ Identifying and defining performance attributes; and

♦ Developing performance rating scales for the attributes.

Information Phase

At the VE study Kick‐off Meeting, Value Metrics is used to establish the project stakeholders’ preference for performance and rating the performance of the baseline concept. Key activities in this phase include:

♦ Validating that the baseline concept satisfies the performance requirements;

♦ Finalizing the performance rating scales for the various attributes;

♦ Determining the relative importance (priority) of the performance attributes through the use of an AHP paired comparison;

♦ Rating the performance of the baseline concept relative to the attributes; and

♦ Rating any other design options that are being considered by the project team.

Function Phase

In the Function Phase, the functions with the greatest impact on the project’s performance attributes are identified. Key activities in this phase that are performed in conjunction with function analysis include:

♦ Identifying the relationships between functions and performance;

♦ Identifying the relationships between functions and cost;

♦ Developing an understanding of how function cost is related to function performance; and

♦ Identifying which functions may be deficient in supporting project performance.

Speculation Phase

In the Speculation Phase, the functions with the greatest influence on project performance and cost are targeted for idea generation (i.e., brainstorming). In addition, brainstorming can be supplemented by using the performance attributes as question statements to generate additional ideas. For example, the team could be asked, “How can we reduce Environmental Impacts?” or “How can we improve Maintainability?” Brainstorming need not be limited to functions; the objective of the Speculation Phase is to generate as many ideas as possible.

Evaluation Phase

In the Evaluation Phase, each idea generated in the Speculation Phase is evaluated with respect to the performance attributes to determine the impact of the idea on the project’s performance as well as cost. This information is expanded upon in the listing of Advantages and Disadvantages. Key activities in this phase include:

♦ Testing each idea against the Performance Attributes (potentially improves, does not change, or reduces performance attribute) when compared to the baseline design concept;

♦ Noting under “Advantages or Disadvantages” if there is a potential change in the performance attribute or requirement;

♦ Assessing the potential cost impact of an alternative (potentially improves, does not change, or reduces project cost);

♦ Prioritizing ideas for development based on value improvement potential, using a 1‐to‐7 scale;

Ratings 4‐7 – The idea is worth examining in more detail in the Development Phase. The following rating scale is used. This rating convention provides a level of priority to the ideas to ensure those with the greatest potential are addressed first in case a lack of time limits the development of VA alternatives.

7 = Major Value Improvement Potential 6 = Moderate Value Improvement 5 = Minor Value Improvement 4 = Possible Value Improvement

While this rating is subjective, it is based on the team’s discussion and evaluation of the idea and does provide a useful prioritization of the ideas to guide the development of the VE alternatives.

Note: For years, in the development of this process, VMS used a 1‐to‐5 scale with ratings 4 and 5 being developed. We found that did not provide a reasonable level of differentiation to prioritize the ideas for development. With greater fidelity of the priorities, the development process is better managed and more time may be allocated to the ideas with the greatest potential benefits.

Rating 3 – The idea has no apparent significant cost or performance benefit but is an alternative approach that the design team may want to consider. These may be developed as a Design Suggestion as time permits.

Rating 2 – The idea is a cost‐reduction idea that reduces project scope or performance to an unacceptable level. These ideas become very apparent during the evaluation phase and are easy to separate from the truly value‐improving ideas. It is important that these ideas be separated, as typically, developing these as VE alternatives is not desirable. In cases where the project is so far over budget that the owner is looking for any option, these can easily be summarized for consideration. It is important that under “Disadvantages” on an Idea Evaluation form the reason this idea should not be developed is clearly stated.

Rating 1 – These ideas are rejected as they clearly do not satisfy the project’s purpose and need; some are nonsense ideas that arose during the creative session, while others are ideas that simply do not apply to this project.

A key benefit of the Value Metrics Evaluation Phase is that it curtails cost‐cutting ideas that degrade performance or reduce scope from development. While a more thorough, yet subjective evaluation of the ideas requires more time, overall, time is saved as there are typically fewer ideas for the VE team to develop.

Development Phase

The result of the more stringent Evaluation Phase is that there is more time to develop better documentation and rationale for the alternatives that are carried forward. This typically leads to better implementation rates and greater savings. For each alternative, the rationale for change to the performance attributes is documented. It is important that the impacts of each of the performance attributes be assessed on how the alternative affects the total project and not just an element of the project. This rational leads to a performance rating for that attribute as compared to the baseline, so that the total value improvement of that alternative can be determined.

Presentation Phase

Once the documentation of the alternatives is completed, the team considers how they can be logically combined into strategies, or groups of complimentary alternatives. Typically, the strategies fall into one of three categories:

♦ VE Team Recommended,

♦ Lowest Cost, or

♦ Best Performance.

By providing project decision makers with a way to consider the synergistic effect of the value alternatives, they can better assess how the project can be improved and why. Each of these strategies is rated with respect to the baseline concept and the rationale for the rating for each attribute discussed.

Ultimately, the report documentation and presentation of results is also enhanced as decision makers are given a much more comprehensive overview of the alternatives and strategies to improve the project. The report documentation discusses how each of the alternatives impacts the critical project performance attributes in addition to cost.

Also, the cost‐cutting, scope‐, performance‐, and quality‐reducing alternatives that are responsible for creating a negative image in the minds of many regarding VE are not considered in the report. These types of ideas will have been assessed during the process and have been dropped for valid reasons. The rationale for not pursuing the idea is documented on an Idea Evaluation form in the report.

Implementation Phase

The purpose of Value Metrics is to develop better VE alternatives – alternatives that clearly improve total project value and not simply cost. The discussions held during the Implementation Meeting focus on total project improvement offered by the VE alternatives. The project stakeholders and design team are asked to validate the performance improvements and cost savings as part of the discussions. Ultimately, the added information that results from the Value Metrics process has been found to reduce greatly the concerns for potential risks associated with the implementation of the VE alternatives. This is because the impacts to key performance attributes have been analyzed in the study and discussed in the report. The transparency of this information increases the confidence in the alternatives presented for their consideration. The result: greater implementation rate, higher savings, and documented performance improvements.

FINDING TIME FOR VALUE METRICS

Many questions have been asked as to how much added time is needed to include Value Metrics into a VE study. In the beginning (9+ years ago), it added roughly 2 hours pre‐study and 4 hours to the VE study, and a couple of hours to producing the report of the study. Today, however, we find it takes no added time and can be easily integrated into a 40‐hour VE study. The ability to complete the added steps in the same timeframe is largely due to the experience and comfort level that a team leader has with these techniques and the development of tools to aid the facilitation of the Value Metrics process. Also, over the years there has been a standardization and refinement to the Project Requirements and Attributes used by an agency. While there is a variation of the attributes used from project to project, they generally are extracted from a standard set. Even when a new user is involved, the added time to develop the list for the project is just an hour or two in most cases.

While there is time necessary for each of the added Value Metrics activities, the analysis of the project becomes more focused and efficient. Moreover, time is not wasted developing VE alternatives that will not improve value.

There is pressure today from many clients to do VE studies in less time. We are constantly faced with what can be cut out of the process or out of the study schedule and still conduct a valid VE study. When time is constrained, it is still possible to use all or part of the Value Metrics process to improve VE study results.

VALUE METRICS – PROCESS

Project performance must be properly defined and agreed by the stakeholders at the beginning of the VE study. The performance requirements and attributes developed are then used throughout the study to identify, evaluate, and document alternatives. This process, Value Metrics, emphasizes the interrelationship between cost and performance and can be quantified and compared in terms of how they contribute to overall value.

The Value Metrics process includes these steps:

1. Define Performance Requirements

2. Define Performance Attributes

3. Develop Performance Attribute Scales

4. Determine Weights of Performance Attributes

5. Determine Performance of Baseline Design Concept

6. Determine Performance of VE Alternatives

7. Define Value Strategies

8. Determine Value of VE Strategies

Performance Requirements

Performance requirements represent essential, non‐discretionary aspects of project performance. Any concept – whether it was developed during the project’s design process or during the course of the VE study – that does not meet a performance requirement fails to meet the project’s basic objectives and, therefore, cannot be considered as a valid solution.

For example, a concept that did not meet a performance requirement for a key project milestone could not be considered further as an acceptable design solution. It should be noted that in some cases, performance requirements may also represent the minimum acceptable level of a performance attribute.

The following are typical performance requirements for highway projects.

Performance Requirement Definition

Design Standards Any deviation from the Highway Design Manual must be approvable by the Agency’s Design Reviewer.

Performance Requirement Definition

Structures Design Any structure on the project must comply with current seismic design standards and meet the Load Resistance Design Factor (LRDF).

Critical Milestones Several critical schedule milestones must be met in order to meet legislative and/or funding requirements.

Delivery Method The project must utilize [Delivery Method].

Construction Traffic All existing mainline lanes must be kept operational during peak hours. Evening closures for up to 50% of the current capacity will be permitted between 10 PM and 5 AM.

Environmental Any concept or design modification considered must comply with state and federal environmental laws and be compatible with the environmental review process.

Funding Project must be able to be funded via the following funding sources: [Delivery Method].

Performance Attributes Performance attributes represent those aspects of a project’s scope and schedule that may possess a range of potential values. For example, Project Schedule may have a range of acceptable values for a project that are between 24 months and 36 months. Obviously, a concept that had a total project delivery schedule of 24 months would perform at a higher level than one that required 36 months to complete, but both would meet the project’s Purpose and Need and their value (i.e., the relationship between performance and cost) could be rationally compared.

Typical highway project performance attributes are listed below. It is important that each attribute be discrete and clearly defined.

Performance Attribute Definition

Mainline Operations

An assessment of traffic operations and safety on the mainline facility(s), including off‐ramps and collector‐distributor roads. Operational considerations include level of service relative to the 20‐year traffic projections, as well as geometric considerations such as design speed, sight distance, lane widths, and shoulder widths.

Local Operations

An assessment of traffic operations and safety on the local roadway infrastructure, including on‐ramps and frontage roads. Operational considerations include level of service relative to the 20‐year traffic projections; geometric considerations such as design speed, sight distance, lane widths; bicycle and pedestrian operations and access.

Environmental Impact

An assessment of the permanent impacts to the environment including ecological (i.e., flora, fauna, air quality, water quality, visual, noise); socioeconomic impacts (i.e., environmental justice); impacts to cultural, recreational, and historic resources.

Project Schedule An assessment of the total project delivery as measured from the time of the VE study to completion of construction.

Construction Impacts

An assessment of the temporary impacts to the public during construction related to traffic disruptions, detours, and delays; impacts to businesses and residents relative to access, visual, noise, vibration, dust, and construction traffic; environmental impacts related to water quality, air quality, soil erosion, and local flora and fauna.

Maintainability

An assessment of the long‐term maintainability of the transportation facility(s). Maintenance considerations include the overall durability, longevity, and maintainability of pavements, structures, and systems; ease of maintenance; accessibility and safety considerations for maintenance personnel.

In addition to these typical six performance attributes, up to two additional attributes should be made available to address site specific issues. The use of these attributes should be based upon the discretion of the project team and/or stakeholders. A list of commonly used attributes that may be relevant is provided below. It should be noted that this list is not all inclusive and that the VE Process must be flexible enough to consider any potential aspect of performance.

Optional Performance Attributes for Transportation Projects


Phaseability

An assessment of how easily a transportation facility can be improved or expanded upon at some future date. This attribute considers the degree of “throw‐away work” involved as well as future traffic and public impacts when the planned future improvements are made.

Land‐Use Compatibility

An assessment of the overall compatibility of transportation facilities with existing and planned land uses. This attribute considers how a transportation facility will directly affect the quality and viability of the land‐uses around it. [NOTE: This attribute is often used for projects that involve significant right‐of‐way acquisition and that will have significant impacts to municipalities and/or private entities.]

Cultural Impacts

An assessment of the permanent impacts to cultural, recreational and historic resources. [NOTE: Sometimes it is desirable to split the standard attribute “Environmental Impacts” into multiple, free‐standing attributes. This is in recognition that sometimes socioeconomic, cultural and natural resources are in conflict with one another.]

Ecological Impacts

An assessment of the permanent impacts to the ecological resources including flora, fauna, air quality and water quality. [NOTE: Sometimes it is desirable to split the standard attribute “Environmental Impacts” into multiple, free‐standing attributes. This is in recognition that sometimes socioeconomic, cultural and natural resources are in conflict with one another.]

Hydrological Impacts

An assessment of the project’s impact to lakes, rivers and streams in its vicinity. Also considered under this attribute are drainage and hydraulic issues. This attribute also considers the performance of the transportation facility during flood events.

Note when one of the environmental impacts listed above is significant enough in a project that it needs to be used as a specific performance attribute, the performance attribute definition for Environmental Impacts is revised to exclude the added performance attribute(s) to ensure the environmental attributes used are discrete and do not overlap.

The use of the following performance attributes (or any variation of these) should be strongly discouraged.

Perceived Performance Attributes that should be Avoided


Public Acceptance

This attribute commonly appears but should be avoided due to the difficulty of trying to assess the broad notion of community or public acceptance by such a small group of individuals possessing a relatively narrow perspective (i.e., the Project Design Team). In reality, “public” or “community” acceptance is a byproduct of the “standard” performance attributes described previously. In other words, the public is more likely to accept a design solution that performs well in these areas (and/or costs less) and less likely to accept one that does not (and/or costs more). Therefore, the use of such an attribute is redundant.

Constructibility

This is commonly stated as a potential attribute on VE studies; however, this is really a byproduct of “Project Schedule,” “Construction Impacts,” and cost. A design solution that is more constructible than another will involve trade‐offs between these three areas. Therefore, inclusion of an attribute such as “Constructibility” is redundant and unnecessary.

Right‐of‐Way Impacts

This attribute is better described by attributes such as “Environmental Impacts,” “Land‐Use Compatibility” or possibly “Cultural Impacts” as well as cost. Experience has shown that when this attribute is used, in effect, performance is really related to cost, which results in “double counting” by considering this as both an output (i.e., performance) and an input (i.e., cost).

What about Safety?

Safety is certainly a critical aspect of the performance of our highway system. It is also a very controversial and sensitive subject. First of all, there are legal issues related to any discussion of safety. If we are to evaluate safety quantitatively, we must come up with a reasonable, non‐emotional rationale to do so. This is difficult to do – basically, we only have past accident data to go on. It is problematic to predict what an improvement will have in the future on past rates for a given facility and a given improvement. We can declare a facility “safe” based on it meeting certain safety standards (i.e., design criteria); however, we cannot know how safe “safe” really is until we have data after the fact confirming or denying our predictions. Furthermore, we must consider the fact that DOTs commonly allow “design exceptions” (usually due to financial limitations and environmental concerns) which allow for design features that do not meet current standards. Logically, this translates to the acceptance of highway facilities that are less “safe” than those that fully meet standards. However, the terms “unsafe” or “less safe” are generally not acceptable and are never used due to fears of litigation. We all are aware of this.

The typical approach is to say all highway facilities are “safe” because they were approved by a DOT. The DOTs do not build “unsafe” projects according to language in official public documents. The concern with using “Safety” as a performance attribute relates to these legal issues. While we could qualitatively evaluate highway facilities for relative safety (i.e., a 10‐foot‐wide shoulder is “safer” than a 4‐foot‐wide shoulder), it could open the door for serious legal issues in the future by having a public document (i.e., a VE/VA study report) available to the legal community for use in litigation against state DOTs.

Furthermore, “Safety” is an emotional issue that people have a difficulty evaluating objectively. This is certainly true from a public perspective. In the past when “safety” is used as an attribute, it completely dominates all other attributes purely due to its emotionally charged nature. It is more logical to view “Safety” as an aspect of traffic operations. This is a logical and sensible way to discuss safety as safety and operations go hand in hand. Conditions that lead to accidents also tend to have poorer operations, and poor operational conditions lead to higher accident rates. Conditions that lead to better operations result in fewer accidents. “Safety” should therefore be regarded as a requirement as DOTs do not build “unsafe” highways.

Performance Attribute Scales

Specific rating scales must be developed for each performance attribute in order to evaluate the performance of the baseline and alternative concepts for each of the Performance Attributes. A standard scale is utilized for all attributes; however, it is important to note that the values and definitions of the scales vary significantly for each. Provided on the following pages is a preliminary description of these scales for each of the previously identified attributes.

VERBAL RATING

PERFORMANCE ATTRIBUTES, RATING SCALES, AND DEFINITIONS NUMBER RATING Mainline Operations Local Operations Environmental Impacts

Excellent

Mainline operations equivalent to Level of Service A (LOS A) during peak hour. Highest level of traffic operations. Meets or exceeds all design standards.

Local operations equivalent to LOS A during peak hour. Highest level of traffic operations. Significantly maintains or improves upon existing local access. Meets or exceeds all design standards.

The project improves upon the existing environmental conditions while introducing no new impacts.

1.0

Very Good

Mainline operations equivalent to LOS B during peak hour. High level of traffic operations. Meets all mandatory design standards. Meets all or most advisory design standards.

Local operations equivalent to LOS B during peak hour. High level of traffic operations. Maintains or improves existing local access. Meets all mandatory design standards. Meets all or most advisory design standards.

The project introduces no new environmental impacts.

0.8

Good

Mainline operations equivalent to LOS C during peak hour. Good level of traffic operations. Meets all or most design standards.

Local operations equivalent to LOS C during peak hour. Good level of traffic operations. Maintains existing local access. Meets all or most design standards.

The project introduces some new environmental impacts that can be addressed through standard and accepted mitigation approaches.

0.6

Fair

Mainline operations equivalent to LOS D during peak hour. Fair level of traffic operations. May require some design exceptions.

Local operations equivalent to LOS D during peak hour. Fair level of traffic operations. Somewhat impacts existing local access. May require some design exceptions.

The project introduces many new environmental impacts that will require extensive mitigation.

0.4

Poor

Mainline operations equivalent to LOS E during peak hour. Poor level of traffic operations. May require multiple design exceptions.

Local operations equivalent to LOS E during peak hour. Poor level of traffic operations. Significantly impacts existing local access. May require multiple design exceptions.

The project introduces environmental impacts that are both significant in number and impact that require extensive mitigation.

0.2

0

Unacceptable

Mainline operations equivalent to LOS F during peak hour. Very poor level of traffic operations. May require multiple design exceptions.

Local operations equivalent to LOS F during peak hour. Very poor level of traffic operations. Severely impacts existing local access. May require multiple design exceptions.

The environmental impacts are severe and the project does not comply with state and/or federal environmental laws.

VERBAL RATING

PERFORMANCE ATTRIBUTES, RATING SCALES, AND DEFINITIONS NUMBER RATING Project Schedule Construction Impacts Maintainability

Excellent The project will be completed significantly earlier by 6 months.

There will be no temporary traffic or environmental impacts during construction.

The project provides the highest possible level of maintainability and far exceeds expectations when compared to comparable facilities statewide. Examples are the use of long‐life pavement, low maintenance water quality facilities, low maintenance structures, etc.

1.0

Very Good The project will be completed earlier by 4 months.

There will be some minor temporary traffic and/or environmental impacts expected during construction. Impacts will be less than typical.

The project provides a high level of maintainability. The facility utilizes many low maintenance features and is better than average in terms of expected maintenance.

0.8

Good The project will be completed earlier by 2 months.

Some nighttime lane closures and/or temporary ramp closures anticipated. There will be some minor to moderate temporary environmental impacts. Impacts will be fairly "typical" for this type of project and can be handled through normal processes and procedures.

The project provides a satisfactory level of maintainability and is typical of a highway facility of this kind statewide.

0.6

Fair The project will be completed later by up to 1 month.

Temporary traffic impacts will be significant and be much greater than what would normally be anticipated for similar projects. Temporary environmental impacts will be more significant in nature and require greater mitigation measures and/or inconveniences to the public.

The highway facility is expected to require greater than normal maintenance due to existing site conditions or materials selection.

0.4

Poor The project will be completed significantly later by 4 months.

Temporary traffic impacts will be extensive, lengthy and very disruptive. Temporary environmental impacts will require extraordinary mitigation measures and create major inconveniences to the public.

The project is expected to require maintenance that far exceeds the norm for a facility of its kind.

0.2

0

Unacceptable

The project cannot be delivered in a manner that will meet current funding and/or legislative mandates.

Temporary traffic and/or environmental impacts will be severe and create impacts that are unacceptable to the public.

The anticipated level of maintenance for the project will be extreme and unacceptably high.

Determine Weights of Performance Attributes

The performance attributes of a project are seldom of equal importance. Therefore, a systematic process must be utilized in order to determine their relative weights in meeting the projects purpose and need. The Performance Attribute Matrix is used to determine the relative importance of the performance attributes for the project.

The performance attributes are compared in pairs, asking the question: “An improvement to which attribute will provide the greatest benefit relative to the project’s purpose and need?” The methodology employed to perform these pairwise comparisons draws upon the AHP. In this method, a pair of attributes is compared using the Fundamental Scale as defined below:

Intensity of Importance

Definition Explanation

The two attributes contribute equally to the project’s purpose and need.

1 Equal Importance

Experience and judgment slightly favor one attribute over another.

3 Moderate Importance

Experience and judgment strongly favor one attribute over another.

5 Strong Importance

Experience and judgment very strongly favor one attribute over another.

7 Very Strong Importance

The evidence favoring one activity over another is of the highest possible importance.

9 Extreme Importance

Sometimes there is a need to compromise between the preceding values in which case these intermediate values can be used.

2, 4, 6, 8 For compromises between the preceding values

A whole number indicates that the attribute in the row is more important. A fraction indicates that the attribute in the column is more important.

n, 1/n Whole numbers and fractions

The project team and other stakeholders evaluate the relative importance of the performance attributes that are used to evaluate the baseline concept and VE alternatives. The pairwise comparisons can be made using a standard matrix using Microsoft Excel using a mathematical algorithm known as an eigenvector. For the purposes of this paper, we will call this matrix the “Performance Attribute Matrix.”

The diagram below provides guidance on how to properly interpret pairwise comparisons appearing on the Performance Attribute Matrix:

The 1/5 indicates that Local Operations is strongly more important than Mainline Operations

Performance Attributes Mainline

Ope

ratio

ns

Local O

peratio

ns

Environm

ental

Impacts

Total

Mainline Operations 1 1/5 1/3 0.104

Local Operations 5 1 4 0.665

Environmental Impacts 3 1/4 1 0.231

Total 9.00 1.4 6.33 1.000

The inverse (5) of the previous judgment is placed in the corresponding cell

This number is derived by adding the quotients of each number in this row divided by the corresponding total in each column and dividing by the number of attributes (3). The number represents the weight of the attribute relative to the other attributes.

Based on the example above, we can determine the relative importance of the performance attributes as being approximately: Mainline 10.4% (0.104), Local Operations 66.5% (0.665), and Environmental Impacts 23.1% (0.231).

(For additional information concerning the mathematics involved in calculating the relative weights of the Performance Attributes, it is recommended that the reader visit Wikipedia.org and query “Analytic Hierarchy Process.”)

The sum of the relative weights for the Performance Attributes equals 1. Therefore, an attribute with a weight of 0.4 would indicate that it is twice as important as an attribute with a weight of 0.2. A Performance Attribute Matrix for a typical VE study is shown below.

SAMPLE PERFORMANCE ATTRIBUTE MATRIX

Which attribute is more important to meeting the project's Purpose & Need?

Performance Attributes

Mainline

Ope

ration

s

Local

Ope

ration

s

Environm

ental

Impa

cts

Project

Sche

dule

Construction

Im

pacts

Maintaina

bility

Priority

Mainline Operations 1 1/5 1/3 1/5 1/5 1/5 0.038

Local Operations 5 1 5 5 5 5 0.397

Environmental Impacts 3 1/5 1 1/5 1 3 0.124

Project Schedule 5 1/5 5 1 1/5 1/3 0.142

Construction Impacts 5 1/5 1 5 1 1/3 0.145

Maintainability 5 1/5 1/3 3 3 1 0.155

SUBTOTAL 24 2 12.667 14.4 10.4 9.867 1.000

Determine Performance of Baseline Design Concept

Assuming the Performance Attributes and their associated scales have been defined and their weights derived, the next step is to establish the performance of the Baseline Design Concept. The Project Stakeholders should take the lead in this process. Using the performance scales, each attribute should be rated accordingly. It is essential that a detailed description for the rating rationale be developed and recorded.

SAMPLE RATING AND RATING RATIONALE: BASELINE CONCEPT

Performance Attribute Rating Rationale for Rating

Mainline Operations 0.8

On‐ramps will be metered for the benefit of mainline traffic. The merge and diverge points of the interchange will be improved over the existing conditions. Storage for the off‐ramps is not expected to queue onto the mainline, except for the auxiliary lane upstream of the southbound off‐ramp. Without the ramp meters Mainline Operations would degrade to a LOS D from their projected LOS B.

Local Operations 0.8

California Oaks Road will be widened from four to six through‐lanes (eliminate the two left‐turn lanes). The lanes will be widened to 12 feet. The ramp intersections that are being replicated in the proposed design will experience significant queue reductions and related LOS improvement. The project removed one signalized intersection from California Oaks Road due to the consolidation of the southbound off‐ramp and Madison Avenue. Pedestrian access along California Oaks Road remains essentially the same. Design increased the vertical clearance to 15'‐0" (from 14'‐10").

Environmental Impacts 0.6

Aerially deposited lead will be buried on site above groundwater. There is a retention basin in the southeast quadrant that captures 2.5 times the runoff caused by the new impervious area. No real noise issues (commercial area). Lighting in the median closer to the intersection is not an issue (commercial area). There is no issue with temporary lease (discretionary City license) for baseball field in the southeast quadrant.

Project Schedule 0.5 The project is expected to begin construction January 2010 with construction duration of 18 months. No timeframe is tied to funding.

Performance Attribute Rating Rationale for Rating

Construction Impacts 0.5

The staging will require maintaining all traffic during construction and is carried out by: 1) Bridge substructure; 2) New ramps and bridge superstructure; 3) Pavement replacement; and 4) Curb and gutter and sidewalk. Dry and wet utilities may require relocations or protection in place. There are concerns that the construction activities may damage existing utilities, especially due to the remedial removal being proposed due to poor soils. The road is being replaced with engineered fill with an R‐value of 20, with the removal remediation at other locations being reconditioned. Overall, earthwork export job.

Maintainability 0.5

More structure (undercrossing widening, tie‐back walls and MSE walls) to maintain and inspect. Retaining wall will require attention by maintenance forces. Infiltration basins will require maintenance effort.

Determine Performance of VE Alternatives

As VE alternatives are developed during the course of the VE study, the VE team must assess their performance using the previously established rating scales.

SAMPLE RATING AND RATING RATIONALE: VE ALTERNATIVE 1

Performance Attribute

Priority Baseline Rating

Alternative Rating

Baseline Score

AlternativeScore Rationale for Rating

Mainline Operations

0.038 0.8 0.8 0.0304 0.0304 No significant change.

Local Operations 0.397 0.8 0.9 0.3176 0.3573

Change will improve on and off northbound turning movements and improve or eliminate an advisory design exception.

Environmental Impacts

0.124 0.6 0.6 0.0744 0.0744

The retention basin in the southeast quadrant can be increased 20% and be better able to contain peak runoff flows.

Project Schedule 0.142 0.5 0.8 0.071 0.1136

The project schedule can be reduced by 4 months as two sequential phases can be combined and work will be performed in parallel at this quadrant.

Performance Attribute

Priority Baseline Rating

Alternative Rating

Baseline Score

AlternativeScore Rationale for Rating

Construction Impacts

0.145 0.5 0.7 0.0725 0.1015

Relocation of critical utilities in this area is eliminated, greatly simplifying coordination with access to local businesses and night work required to minimize business disruptions.

Maintainability 0.155 0.5 0.8 0.0775 0.124

Better maintenance access is provided, which is off the local and state highway. This provides less exposure to traffic for maintenance workers.

Performance Total

0.6434 0.8012

Performance Improvement

25%

This shows that the VE alternative has a 25% improvement in performance. Importantly, it also documents why the performance is improved. While this alternative also saved over $2.2 million on a $60 million project, we typically do not calculate the value improvement by VE alternative. It is possible to do so, but instead, we focus on developing strategies for improvement as all VE alternatives cannot necessarily be implemented; some may be competing ideas or developed only to address an issue raised by a project stakeholder and not recommended by the team.

Define Value Strategies

Once the performance ratings for all of the VE alternatives have been determined, the next step is to identify specific value strategies which represent sets of complimentary VE alternatives. Value strategies are typically organized using the following logic:

♦ VE Team Recommended

♦ Lowest Cost

♦ Best Performance

For example, if there were 20 VE alternatives (and assume that none of them were mutually exclusive), and, say, 15 of them had cost savings, these 15 alternatives would then constitute a Value Strategy called “Lowest Cost.” We could then organize a second strategy by including only those alternatives that resulted in an improvement to performance. Assuming that there were 10 such VE alternatives, these could be organized into a Value Strategy called “Highest Performance.” Lastly, we could have the

team select those alternatives that they feel should be implemented. Perhaps there were 13 such alternatives which could be called “VE Team Recommended.”

The intent of assembling value strategies is to show the potential of certain combinations of alternatives that relate to a specific theme. In reality, the Project Team will likely not implement all the alternatives in a value strategy. Those that are eventually implemented can then be assessed as a group to determine the final change in value from the Baseline Design Concept.

Example – Determining Total Performance of Strategies

Strategies VE Alternatives Performance Score (x 100)

Baseline Design Concept N/A 56.70

Value Strategy A – Recommended 1.0 ‐ 8.0, 10.0, 12.0 ‐ 15.0 62.30

Value Strategy B – Lowest Cost 2.0, 6.0 ‐ 9.0, 11.0 ‐ 20.0 52.05

Value Strategy C – Best Performance 3.0, 7.0 ‐ 13.0, 17.0, 19.0 68.32

The next step is to determine the total costs of the value strategies. Any cost savings and/or increases must be determined and used to determine the total cost to construct the project. If costs are being considered as the only input (in other words, schedule is being considered as an output like performance), then the raw dollar cost can be used directly as money is already expressed as a ratio scale. For this calculation, construction costs are typically expressed in millions of dollars.

Example – Using Just Project Costs to Determine Value

Strategies VE Alternatives PerformanceScore (x100)

Construction Cost ($M)

Value Index

Change in Value

Baseline Design Concept N/A 56.70 $60.0 0.95

Value Strategy A – Recommended

1.0 ‐ 8.0, 10.0, 12.0 ‐ 15.0

62.30 $55.0 1.13 20%

Value Strategy B – Lowest Cost

2.0, 6.0 ‐ 9.0, 11.0 ‐ 20.0

52.05 $40.0 1.30 38%

Value Strategy C – Best Performance

3.0, 7.0 ‐ 13.0, 17.0, 19.0

68.32 $57.0 1.20 27%

However, if schedule is being considered alongside cost as an input, then both time and money to be converted into the same ratio scale (0 to 1). This is easily done by adding the costs for the various options together and then dividing the cost of each option by the total to develop a relative score.

Example – Converting Costs of Strategies to a Relative Score

Strategies Total Cost Relative Cost Score

Baseline Design Concept $60,000,000 0.2830

Value Strategy A – Recommended $55,000,000 0.2594

Value Strategy B – Lowest Cost $40,000,000 0.1887

Value Strategy C – Best Performance $57,000,000 0.2689

Total $212,000,000 1.0000

Depending on which value equation is used (which depends on how project schedule will be considered), the following steps are needed to determine the value index for the Baseline Design Concept and each Value Strategy.

If schedule is assessed as an output (i.e., like performance), than VFP SC

is used. In this method,

schedule has already been calculated and included as part of the total performance score.

If schedule is assessed as an input (i.e., like cost), than VFP

C S is used. In this case, schedule must

be converted into a ratio scale as well (similar to the above example for cost) and the relative importance of cost and schedule must be determined. This is easily achieved by directly assigning

Example – Using Costs of Strategies to a Relative Score

Strategies VE Alternatives PerformanceScore (x100)

Construction Cost ($M)

Value Index

Change in Value

Baseline Design Concept N/A 0.5670 0.2830 2.00

Value Strategy A – Recommended

1.0 ‐ 8.0, 10.0, 12 ‐ 15.0

0.6230 0.2594 2.40 20%


2.0, 6.0 ‐ 9.0, 11.0 ‐ 20.0

0.5205 0.1887 2.76 38%


3.0, 7.0 ‐ 13.0, 17.0, 19.0

0.6832 0.2689 2.54 27%

weights to them. For example, let us assume that for our example project, the Project Team decides that cost is more important than schedule. Based on internal discussion, they decide that cost should represent 0.7 and schedule 0.3 of the total inputs.

Example – Converting Schedules of Strategies to a Ratio Scale

Strategies Total Time Relative Score

Baseline Design Concept 36 months 0.3051

Value Strategy A – Recommended 28 months 0.2373

Value Strategy B – Lowest Cost 30 months 0.2542

Value Strategy C – Best Performance 24 months 0.2034

Total 118 months 1.0000

Example – Computing the Value Indices of Strategies (Value Matrix)

Strategies Performance

Score Cost Score

(0.7) Schedule Score

(0.3) Value Index

Change in Value

Baseline Design Concept .5670 0.1981 0.0915 1.957873

Value Strategy A ‐ Recommended

.6230 0.1816 0.0712 2.464399 25.87%


.5205 0.1321 0.0763 2.497601 27.57%


.6832 0.1882 0.0610 2.741573 40.03%

If a quantitative risk analysis was performed, the resulting affects can be included in the totals for cost and schedule when computing the value indices.

Regardless of which equation is selected, the Value Matrix is essential for understanding the relationship of cost, performance, schedule, risk and value of the Baseline Concept and Value Strategies. The comparison of these elements in this manner exposes the trade‐offs between inputs and outputs and provides useful information to decision makers in acting upon the information developed during the VE study.

CONCLUSION

Integrating Value Metrics into the VE process properly creates a more robust process which has demonstrated better results in VE studies (greater cost savings with documented performance improvements). This process has also been shown to better engage project stakeholders in the VE process as they see that the goal of the VE study is to improve the project and ensure that the right project is being developed for the right cost and not simply looking for cost cutting of their project.

This paper shows the current evolution of the Value Metrics approach. The refinements of the mathematical model (adaptation of AHP) used in the Value Metrics process are to enhance the sensitivity and flexibility of the process.

"The sciences do not try to explain, they hardly even try to interpret, they mainly make models. By a model is meant a mathematical construct which, with the addition of certain verbal interpretations, describes observed phenomena. The justification of such a mathematical construct is solely and precisely that it is expected to work."

Truth is much too complicated to allow anything but approximations."

– John von Neumann

abstract 2009 aashto value engineeringsp.design.transportation.org/documents/19-hays.pdf ·...

Documents