asbo vendor performance eval chicago 092509

RICHARD L. GAY, CPPO, RSBOPURCHASING MANAGEROFFICE OF PURCHASINGOctober 2009

VENDOR PERFORMANCE EVALUATIONS

Compliance with Board Policy & Rule 3231-Vendor Performance Evaluation, approved September 2007.

To effectively manage vendors doing business with the Baltimore County Public Schools.

Vendors have the right to receive performance feedback, whether it is positive or negative.

If feedback is negative, the vendor is entitled to be informed of why their performance is unsatisfactory and what corrective action is required.

This procedure applies to all contracts with a value greater than $500,000, and is voluntary for other contracts.

Why Monitoring is Fair…. Should be 360°

Each participating party needs to be able to rate the other allowing for open and constructive feedback.

Constructive Comments Should pinpoint exactly what needs to be improved and how to do it.

Consensus Isolated low scores look insignificant when the other team members are rating

everyone as well.

No Surprises Because the process is ongoing through the life of the project, there are no

surprises and no frustrations over “after the fact” evaluations, when it is too late to change.

Many Opportunities Multiple intervals offer many opportunities to improve based on suggestions

received.

Why Monitoring is Fair….

Hard and Easy Raters Administrators can determine whether certain raters are consistently demanding

and disgruntled, or alternatively indiscriminant in passing out 10’s to everyone.

20 Criteria, 4 Categories Breaking the interval criteria into categories provides many opportunities to score

well.

Positive Emphasis Project managers can use the ratings process to emphasize teamwork, positive

and constructive feedback, rewards, and recognition.

Averaged Final Scores Final scores are averaged on multiple intervals and ratings by multiple raters.

Recognition Many opportunities to recognize good or excellent performance.

Why Monitoring Can Be Done

Performance-based contracting typically requires procurement, technical, legal, and program staff to do more up-front work than more traditional acquisition processes.

Further, it requires considerable collaboration among staff. However, this investment of time yields a high payoff for all, allowing public sector agencies to get the best value for the dollars spent.

When performance evaluations are used there were demonstrated savings of at least 15%, better contractor performance, and higher satisfaction with the results of the contracts.

Develop a performance work statement (PWS) or statement of objectives (SOO).

This is a description of how the work to be done under the contract is framed as a desired outcome or set of results, rather than a set of tasks.

To generate the statement, an interdisciplinary team typically conducts one or more brainstorming sessions to consider what the ultimate "customer" really needs.

Decide how to measure and manage performance.

Measurable performance standards Incentives/remedies Performance assessment or quality

assurance plan

"Projects that would be considered as most appropriate for the use of the PerformanceEvaluation include:

Large Projects: For example, a project requiring a large investment of resources and staff, with an equal opportunity to gain large benefits.

New Technology Projects: For example, a project requiring new technology where the agency does not have an in-house staff with the needed skill or expertise.

High-Risk Projects: A project that contains enough cost, new technology, or impact on the business that the agency should look outward for solutions and expertise.

Among the kinds of projects that are not good candidates for performance evaluation are those;

with easy to define deliverables, such as hardware replacement, projects where it is not possible to assemble an interdisciplinary team,

and projects that affect governmental functions with no performance measurements.

MONITORING

BCPS’ procurement personnel or project staff shall verify that the certified MBE’s listed in the MBE participation schedule are actually performing the work.

BCPS’ procurement personnel shall ensure that MBE subcontractors are receiving compensation as set forth in the MBE participation schedule by ensuring that the contractor submits monthly reports, listing any unpaid invoices over 30 days old received from any certified MBE subcontractor, the amount of each invoice, and the reason payment has not been made.

The MBE liaison and/or the Public School Construction Program will conduct reviews as deemed necessary to confirm compliance with the minority business enterprise participation requirements.

The MBE liaison will maintain appropriate records, and shall assist the Public School Construction Program in on-site or post-audit reviews upon request.

NON-CONSTRUCTION CONTRACTS • The new Vendor Performance

Report – Non-Construction Vendors (VPR) is used to rate the performance of non-construction vendors such as publishers and office supply vendors.

• The VPR is found on the Purchasing Intranet webpage, under Forms & Publications.

https://intranet.bcps.org/offices/purchasing/xls/Form_VendorPerform_NonConst.xls

NON-CONSTRUCTION CONTRACTS Vendor Performance Report – Non-Construction Vendors (VPR)

• The Office of Purchasing will notify responsible managers when a VPR is required to be completed under the Rule.

• A VPR may be completed for contracts with a value less than $500,000, and for any open ended long-term or blanket order contracts.

• A VPR may be completed at the discretion of BCPS personnel, whether performance is positive or negative.

• Purchasing cannot take action on the negative performance of any vendor without adequate documentation!

• The VPR establishes the means to provide necessary vendor performance documentation.

NON-CONSTRUCTION CONTRACTS Complete the VPR with as

much information as possible. Instructions are included in the “Instructions” tab of the on-line document.

Send the completed VPR to Carol Yingling, Office of Purchasing.

Carol Yingling should also be contacted if you have any questions or need additional information, x4334 or cyingling@bcps.org

Vendor Performance Report – Non-Construction Vendors (VPR)

NON-CONSTRUCTION CONTRACTS Report Disposition

Based on the documented information provided, the Office of Purchasing will contact the vendor and investigate the matter.

The Office of Purchasing will report to the vendor the findings, their right to respond, and their right to appeal the disposition.

The Office of Purchasing will inform the report initiator of the results.

The Office of Purchasing maintains a file on all vendor performance evaluation documentation.

CONSTRUCTION CONTRACTS

CONSTRUCTION CONTRACTS The Office of Purchasing uses Rating

Source, an internet evaluation application, to evaluate performance while projects are under way.

Rating Source is a 360° evaluation application wherein each party rates the other.

The Office of Purchasing works with Facilities to set up project intervals and determines project raters (team members).

Project intervals and team members are chosen based on the scope of the project. Team members will be provided instructions entitled Vendor Performance Evaluation Procedures.

Team members are asked to submit evaluations at predetermined intervals or milestones.

Project Intervals

Design Development Construction Documents 50% Complete 100% Construction Document Submission Bid Phase Initial Construction End of First Phase End of Second Phase Final Completion

CONSTRUCTION CONTRACTS Project Rating Request: A notification email will be

generated to every project team member when it is time to rate a project.

The email provides each team member with a link to access the project to be rated, along with login and password information.

CONSTRUCTION CONTRACTS After clicking on the project link: Sign on with the user information provided to

you by Rating Source.

CONSTRUCTION CONTRACTS Click on Complete An Open Survey.

CONSTRUCTION CONTRACTS

Click on the survey that you have been asked to complete.

*Detailed instructions entitled Vendor Performance Evaluation Procedures,will be provided to you by the Office of Purchasing.*

CONSTRUCTION CONTRACTS Ratings Results Results

Upon completion of interval ratings by all team members, Rating Source compiles the results. The results summary is shared with the Project Manager for feedback.

Team members can request access to Rating Source to view their results by contacting the Office of Purchasing. An email to grant access with login and password information be will be sent to the requestor.

Click on the link to login to Rating Source.

CONSTRUCTION CONTRACTS Ratings Results Results-Beta Version

Click on Reports.

CONSTRUCTION CONTRACTS Ratings Results Results

Click on the survey that you would like to review for an overall score.

CONSTRUCTION CONTRACTS Ratings Results Results For more detailed results, click on the monitor name.

CONSTRUCTION CONTRACTS Ratings Results Results-Beta Version

Review each survey by clicking on the View button.

CONSTRUCTION CONTRACTS Ratings Results

CONSTRUCTION CONTRACTS Ratings Results

CONSTRUCTION CONTRACTS Ratings-Legacy System

BCPS first began rating construction projects using the older Legacy System version of Rating Source. Enter your User ID and Password; click on the Legacy System button.

CONSTRUCTION CONTRACTS Ratings-Legacy System Click on the Team Monitor link.

CONSTRUCTION CONTRACTS Ratings-Legacy System

Select your Monitor Name in the drop-down list.

CONSTRUCTION CONTRACTS Ratings-Legacy System

Click View Reports button along bottom of page.

CONSTRUCTION CONTRACTS Ratings-Legacy System

Below is a view of Overall Reports that can also be printed by clicking on the Print button.

CONSTRUCTION CONTRACTS Ratings-Legacy System

More detail is obtained by clicking on the Rated Company link.

BCPS GENERAL CONTRACTORS Vendor Performance Evaluations

A Rating Source-based evaluation process was performed on BCPS BCDPW pre-qualified general contractors in April, 2007.

Each general contractor provided client references on jobs over past three years.

BCPS GENERAL CONTRACTORS Vendor Performance Evaluations

These references were contacted and asked to provide vendor performance information via a survey.

Vendor Performance Evaluation Sets

BCPS GENERAL CONTRACTORS Vendor Performance Evaluations

• Upon completion of surveys, Rating Source compiles the results and rates each vendor.

• Requests for completed evaluation surveys should be made to the Office of Purchasing. Surveys will be received via email.