Mn/DOT CS Consultant Evaluation

Best Practices Review

May 01, 2003

Slide 2May 01, 2003

Agenda

• Best Practices research plan (reprise)• How we determined Best Practices • Results of Best Practices Review

– The States with best practices

• Best Practice Findings– Doing Consultant Evaluations– Using Consultant Evaluations

• Mn/DOT Consultant Evaluations– Level 2 Documentation– Level 3 Documentation

Slide 3May 01, 2003

The July 2002 Plan

• Research 13 states preliminarily identified for best practices in pre-qualification;

• Research 10 states identified for best practices in consultant evaluation.

• Complete the question-set for all.

Slide 4May 01, 2003

What is “Best Practice”

• Theory: Process Maturity– Five Levels of Process Maturity

• Refer to: Process Maturity Article on L&C Web Page

• Application of theory focused on:• Criterion 1: Documented addresses the extent to which an organization’s

processes are documented. The least mature state of documentation maturity is one in which processes typically are ad hoc, perhaps even chaotic. . . Alternatively, in the most mature state, written documentation is not only consistent throughout the organization, it is an intrinsic element of the process itself.

• Criterion 2: Practiced criterion addresses the consistency of process performance. In the least mature state, processes are practiced in an ad hoc or, at best, intermittent manner. By contrast, the processes of a more mature organization are practiced consistently and throughout the organization

Slide 5May 01, 2003

Best Practice States

DOT Pre-Qualified System Post SelectionEvaluation

1. A lberta √ √

2. Ar izona √

3. Colorado √

4. Florida √ √

5. Georgi a √ √

6. Iowa √ √

7. Kansas √ √

8. Massachusetts √

9. Michigan √ √

10. Nevada √

11. New Mexico √

12. Ohio √ √√√

13. Ontario √ √

14. Texas √ √

count 13 10

Slide 6May 01, 2003

Doing Consultant Evaluations

1. An evaluation process is documented via formal policies & procedures.(Ohio, Iowa, Kansas)

2. There is a clear separation between technical performance and project management performance (I.e., schedule, budget, communication). (Ohio, Iowa, Kansas, Florida)

3. Performance scales are pre-defined and explained in detail sufficient to ensure consistent application. (Ohio, Iowa, Ontario)

4. Evaluation criteria are pre-defined and explained in detail sufficient to ensure consistent application. (Ohio, Iowa, Florida)

Slide 7May 01, 2003

Doing Consultant Evaluations

6. Evaluation criteria require project-specific customization by Project Manager. (Ohio)

7. Performance ratings must be supported with written comments. (Ohio, Kansas)

8. Performance evaluation is shared with the Consultant in a performance evaluation conference. (Ohio, Iowa, Kansas)

9. Consultant is given the opportunity to respond to the evaluation. (Ohio, Iowa, Kansas)

Slide 8May 01, 2003

Using Consultant Evaluations

1. Consultant evaluations are explicitly used in project selection decisions. (Ohio, Kansas, Iowa, Florida, Ontario)

2. Poor Consultant evaluations are explicitly grounds for suspension of pre-qualification status. (Ohio, Florida, Ontario)