NCHRP Study 20-24 (30)

Performance Measures for Context Sensitive Solutions – A

Guidebook for State DOTs

As presented by Sally Oldham at the 2005 Midwest CSS Conference

Performance Measurement

CSS and CIA

Why Measure?

• Make CSS state of the practice not state of the art

• To Strengthen Agency Leadership Support for CSS Principles.

• To Maintain Focus on Strategic CSS Goals.

• To Strengthen Trust with Stakeholders and Customers.

Project Level – Key Characteristics

• CSS principles rooted at project level

• Work for one or many projects

• Foster team commitment

• Rely on collaborative self-assessment

• Apply at milestones in project delivery

• Strengthen stakeholder trust

Guiding Concepts for Project Measures

• Integrate measures into project development process as an aid

• Develop tailored consensus-based concepts to measure against – Problems and needs for all projects – Project vision or goals

Project Measures --Key Process Focus Areas

• Use of multi-disciplinary teams

• Public engagement

• Project problems and needs

• Project vision or goals

• Alternatives analysis

• Construction and maintenance

Project Measures – Key Outcome Focus Areas

• Achievement of project vision or goals

• Stakeholder satisfaction

• Quality assurance review

Organizational Level - Characteristics

• Used to assess performance of entire organization

• Fewer in number than project-level measures

• Rely on central reporting of data

• Vital resource for senior management

• Monitored on regular schedule

Organization Measures Key Process Focus Areas

• Training

• Manuals

• Policies

• Motivation

Organization Measures – Key Outcome Focus Areas

• Timeframe and budget

• Stakeholder satisfaction

Matrix of CSS Measures

Project Level

(Process)

Use of Multi-Disciplinary Teams.

• Suggestions for Measuring– Were the right people on the team?– Did the team function effectively?– Focus on Context Sensitive Solutions’

principles?

Public Engagement

• Suggestions for Measuring– Presence of a public involvement plan?– Were external champions for the project

created?– Was public input sought and used at key

decisions points?– Adequacy of DOT expertise and

resources?– Quality of public involvement strategy?

Project Problems, Opportunities and Needs.

• Suggestions for Measuring– Support for statement of Problems,

Opportunities and Needs?– Linkage of Problems, Opportunities and

Needs to evaluation of alternatives?

Achievement of Project Vision or Goals.

• Suggestions for Measuring– Match between original Problems,

Opportunities and Needs statement and final project?

– Consistency with Local Plans?– Tracking and adherence to project

commitments?– Were project Vision or Goals met?– Does the project support community values?– Are environmental resources preserved or

enhanced?– Did the project leverage other resources?

Alternatives Analysis

• Suggestions for Measuring– Adequacy of range of alternatives

developed?– Existence of criteria for evaluation of

alternatives? – Design considerations:

• Design speed• Level of service• Safety

– Need for redesign– Multi-modal considerations

Construction & Maintenance

• Were construction staff involved with the project team at all key milestones?

• Was a list of commitments to stakeholders maintained throughout the planning and design phases and incorporated into construction documents prior to beginning construction?

• Was the project monitored to ensure that commitments were acted on?

• Were there many requests for change orders during construction?

Construction & Maintenance

• Were maintenance staff involved with the project team at all key milestones?

• Were maintenance needs/requirements taken into consideration when alternatives were evaluated?

• Is a maintenance plan in place to ensure that the project investment will be maintained?

• As a reflection of community buy-in and support, has the local government or has a local organization agreed to maintain some portion of the project improvements?

Project Level

(Outcomes)

Achievement of Project Vision and Goals

• Match between original Problems, Opportunities and Needs statement and final project?

• Tracking and adherence to project commitments?

• Were project Vision or Goals met?• Does the project support community values?• Are environmental resources preserved or

enhanced?• Did the project leverage other resources?

Stakeholder Satisfaction

• Suggestions for Measuring– Tailored surveys of key stakeholders– Achievement of consensus during project? – Impacts of construction

Quality Assurance Review

• Suggestions for Measuring– Maryland SHA Charrettes– Maryland SHA Peer Reviews Impacts of

construction– Connecticut DOT Project Reviews

Organizational Level

(Process)

Training

• Suggestions for Measuring– Quantity of training?– Focus of training?– Quality of training?

Manuals

• Suggestions for Measuring– Changes in manuals?– Effectiveness of manual changes?

Policies

• Suggestions for Measuring– Changes in policies?– Effectiveness of policy changes?

Staff Motivation Strategies

• Suggestions for Measuring– NYDOT CSS awards program – Utah’s management through performance

plan requirements

Organizational Level

(Outcomes)

Timeframe and Budget

• Suggestions for Measuring– What proportion of projects is completed on,

or ahead of schedule?– Were few or no project redesigns required?– Were low-build options selected? – Were there added costs attributed to changes

in scope mid-way through the design process?

– Were there cost overruns during construction attributable to changes in design during the construction phase?

Stakeholder Satisfaction

• Suggestions for Measuring– Aggregate results of project-level surveys,

etc.– Conduct opinion polls; focus groups

Creating Measures:Two Key Ingredients

• Leadership– Executive management must show

considerable support– Need a day-to-day champion– Create a work group (internal & external

members

• Strategic Planning– CSS should be consistent with any strategic

planning efforts

Implementing Measures:A Tailored, Collaborative, Self-

Assessment Approach.

• May likely rely on self-administered surveys – team members and – stakeholders.

• “Charrette” style sessions may be a practical strategy

Timing

• Organization-wide– Quarterly or once a year

• Project-level– Process: At project milestones and at project

completion– Outcomes: after project completion

Tips For Getting Started

• Create a Champion for CSS Measurement

• Start Small• Incorporate Feedback from External

Sources• Focus on Planning and Preliminary

Design• Measures for Small Projects are as

Important as Those for Large Projects.

Other Ways to Measure

• Utilize the language of the “Thinking Beyond the Pavement” workshop.

• Now a part of SAFETEA-LU

• Process related

• Design related

SAFETEA-LUTITLE VI--Transportation Planning And Project Delivery

• Amends Title 23 § 109

Characteristics of the Process

Contributing to Excellence

• Open, honest, early, continuous communication

• Multidisciplinary team

• Involve full range of stakeholders in project scoping phase.

Characteristics of the Process

Contributing to Excellence

• Tailor processes to meet the circumstances/project.

• Multiple Alternatives; reach consensus.

• Be committed to the process.

• Valued resources are understood before engineering design is started.

• Use a full range of tools.

Qualities of Excellence in Transportation Design

• The project:– Satisfies the purpose and needs as agreed to

by a full range of stakeholders.– Is a safe facility both for the user and the

community.– Is in harmony with the community, natural

resources and values of the area.

Qualities of Excellence in Transportation Design

• The project:– Exceeds the expectations of both designers

and stakeholders – Involves efficient and effective use of the

resources (time, budget, community) of all involved parties.

– Is designed and built with minimal disruption to the community.

– Is seen as having added lasting value to the community.

FHWA CSS Criteria

• Written CSS commitment and/or policy.

• Technical staff is trained in CSS approach, both field and central offices, and across disciplines with a substantial portion of staff trained in CSS for project development.

• At a minimum, all new projects are being developed in accordance with CSS concepts, consistent with scope, size, and type of project.

FHWA CSS Criteria

• Early, continuing, and interactive public involvement throughout the project development process.

• Interdisciplinary teams are involved in the process from the beginning until the end.

CIA Measures

• CUTR, U. of South Florida

• October 2005

Basic Structure

• Application of Core Measures

• Checklist of Questions

• Suggested Public Involvement Methods

Sample Checklist Question

You can find the full report at:

http://www.dot.state.fl.us/research-center/Completed_Proj/Summary_PTO/FDOT_BC353_28_rpt.pdf