bullet-proofing your rfps · the building blocks of the evaluation process 1. establishing...
TRANSCRIPT
Bullet-proofing Your RFPs
1
RFPs are not new!
There is a large body of common knowledge
Bullet-proofing Your RFPs
2
https://www.rfpmentor.com/pages/chapter11
The Regular Price is $170 (US). Reduce by $85
for this Conference. So, just go to our store at
www.rfpmentor.com, order a book and use the
Discount Code “retired” at checkout. You’ll save
$85.
5th Edition of RFP and public procurement best
practices expert Michael Asner’s best-selling
text
“Michael Asner's book removes a lot of the RFP process mystery. It is one of very few
RFP books that address the negotiation process. I refer to Mr. Asner's book often and
it is recommended reading for public procurement officers.” ~ John Adler, CPPO,
former NASPO President/State Procurement Officer (Arizona)
3
SPECIAL Book Offer: $85 OFF The Request for Proposal
Handbook
For more details, go to:
https://www.rfpmentor.com/
Inside the Evaluation
Process
The Building Blocks of the
Evaluation Process
4
Inside the Evaluation
Process
The Building Blocks of theEvaluation Process
3
5
Constructing the Evaluation
The Building Blocks of The Evaluation Process
1. Establishing Compliance With Mandatory Requirements
2. Scoring the Proposals
3. Imposing Upset Levels
4. Evaluating the Cost
5. Developing a Short List
6. Interviewing Suppliers
7. Negotiating the Contract
8. Requesting Best and Final Offers
9. Using References/Past Performance
1. Establishing Compliance With Mandatory Requirements
Be precise, legalistic.
Establish the rules for rejecting proposals.
Definitions are important.
6
1. Establishing Compliance With Mandatory Requirements
Best Practices
1. Put Definitions in the RFP.
2. Provide a Table of Mandatory Requirements in the RFP.
3. Publish the Rules for Rejecting a Proposal in the RFP.
4. Restrict the number of Mandatory Requirements.
7
8
Constructing the Evaluation
The Building Blocks of The Evaluation Process
1. Establishing Compliance With Mandatory Requirements
2. Scoring the Proposals
3. Imposing Upset Levels
4. Evaluating the Cost
5. Developing a Short List
6. Interviewing Suppliers
7. Negotiating the Contract
8. Requesting Best and Final Offers
9. Using References/Past Performance
2. Scoring the Proposals
Evaluation Factors
Weights
Scoring Systems
9
Evaluation Factors (Criteria)
Families of Criteria Technical
Management
Cost
Effective CriteriaObjective
Related to the requirements definition
Discriminating
Non-discriminatory
Measurable
Economical to use
10
Weights
Determining the Weights
Publishing the Weights
11
Scoring Systems
The Bar Can’t Get Any Lower!
An Excellent Approach #1
An Excellent Approach #2
12
Scoring Systems
The Bar Can’t Get Any Lower
10 - Excellent - meets all requirements/very desirable
7 - Good - most requirements met, it is good enough
4 - Satisfactory - some requirements met, not sufficient
1 - Unsatisfactory - requirements essentially not met
13
Scoring Systems
An Excellent Approach #1
Air Force Regulation 70-15
If the majority of the (sub)factors evaluated meets standards, the requirement is not overly difficult to meet, and the (sub)factors which are deficient are of a very minor nature or are susceptible to easy correction, the item should be scored "5".
14
Scoring Systems
An Excellent Approach #2
Ontario Government Factor 2: Project Team (Weight = 40)
1. Project Manager is experienced in all 3 key areas
2. Project Manager has managed large, similar projects
3. Key assistants (2 or 3) are experienced in 3 key areas
4. Extra (contingency) resources are available
5. Two or more specialists to assist team
6. Experience with a similar system
7. Commitment/dedication of resources
15
16
Constructing the Evaluation
The Building Blocks of The Evaluation Process
1. Establishing Compliance With Mandatory Requirements
2. Scoring the Proposals
3. Imposing Upset Levels
4. Evaluating the Cost
5. Developing a Short List
6. Interviewing Suppliers
7. Negotiating the Contract
8. Requesting Best and Final Offers
9. Using References/Past Performance
3. Imposing Upset Levels
What are they?
When do you use them?
Best Practices
17
18
Constructing the Evaluation
The Building Blocks of The Evaluation Process
1. Establishing Compliance With Mandatory Requirements
2. Scoring the Proposals
3. Imposing Upset Levels
4. Evaluating the Cost
5. Developing a Short List
6. Interviewing Suppliers
7. Negotiating the Contract
8. Requesting Best and Final Offers
9. Using References/Past Performance
Evaluating Proposal Costs
What Works and What Doesn’t
19
Cost is Always Under the Microscope
For Political Reasons
o Some stakeholders want the band-aid
solution so they can fund another project
o Some stakeholders want the best (most
expensive) product or solution
For Practical Reasons
o You need to mitigate project risks
o You always need to justify the cost of the
award20
Agenda
1. Cost Evaluation Best Practice Fundamentals You Always Need to Consider
2. Arithmetic is Supreme: Numeric Scoring Strategies and How They Work
3. A Better Way: Non-Numeric Evaluation Strategies
4. The Nightmare Scenario: What to do When All the Proposals You Receive are Over Budget
21
PART 1
Cost Evaluation Best Practice Fundamentals You
Always Need To Consider
1. Life Cycle Costs
2. The Two-Envelope System
3. Finance, Negotiation and Contract Skills
4. Management/Stakeholder Buy-In
22
PART 2 Arithmetic is Supreme: Numeric Scoring Strategies & How They Work
23
Best Solution (Within Budget)
1. Score the proposals on technical
and management evaluation
factors combined.
2. Select the proposal with the highest
score, regardless of price or within
budget.
24
Standard Arithmetic Strategies
Strategy #1 - Cost is Just Another Evaluation Factor
Strategy #2 - Bang for the Buck
Strategy #3 - Two Steps: First Merit, Then Cost
25
Standard Arithmetic
Strategies
Strategy #1Cost Is Just Another Evaluation Factor
o Arguments against
this approach
o Assigning weights
o Determining the
score
26
Assigning Weights:
Results of an Informal Survey
All over the Map: No one ‘Best
Practice’ for Cost Emerged
Standard points are sometimes used
Survey Results (Cost is weighted from
20% to 80%)
Can’t be more than 30%
Must be more than 60%
27
How To Do It: Using Ratio of Costs
28
Proposal Cost Calculation of Points Points
A $300,000 (250,000/300,000) x 100 83
B $250,000 (250,000/250,000) x 100 100
C $275,000 (250,000/275,000) x 100 91
Strategy #1
Cost Is Just Another Evaluation Factor
Strategy #2
‘Bang’ for the Buck
Concept of value
Determine the score for
management and technical factors
Divide by dollars: points per dollar
29
Strategy #3
Two-Step Approach: First Merit, then Cost
STEP ONE – MERIT
Determine management and technical
scores. Eliminate all those that fail to get
some minimum established by
evaluators.
Proponent
Technical &
Management Score
Company A 86
Company B 75
Company C 70
Company D 82
Company E 65
Proponent
Technical &
Management Score Cost
Company A 86 $250,000
Company B 75 $225,000
Company C 70 xxx
Company D 82 $200,000
Company E 65 xxx
STEP TWO – COST
For those remaining, look at the cost
and select the minimum.
3131
PART 3
Adopting a Non-Numerical Evaluation of Cost
Are numbers better? NO!
o There is no ‘best practice’ for cost
o Standard Points
o Survey Results (20% to 80%)
o Can’t be more than 30%
o Must be 60%
Adopting a Non-
Numerical
Evaluation of Cost
o Are numbers worse?
o Possibly!
o Probably!
32
3333
A Better Way
o Determine the score for each proposal for the Technical and Management Factors.
o Calculate the cost of each proposal.
o Analyze the difference in points and the difference in price to see which proposal is ‘best value’.
o Write a narrative discussing the strengths and weaknesses of each proposal, ‘best value’ and why the recommended proposal represents ‘best value’.
A Few Best Practices Approaches from
Real Agencies
34
o Federal Transit Administration
o State of Alaska
o John Adler
U.S. Department of TransportationFederal Transit Administration
Best Practices Procurement Manual
http://www.fta.dot.gov/grants/13054_6037.html
Best Practices Procurement Manual
‘Highlights’
o Scores, without substantive explanations of the relative strengths and
weaknesses of the competitive proposals, including the perceived
benefits to the agency, are an insufficient basis for paying a higher
price.
o No one is smart enough to predict in advance how much more should
be paid for certain incremental improvements in technical scores or
rankings.
o Price is evaluated and considered alongside technical merit in a
tradeoff fashion using good business judgment to choose the proposal
that represents the best value to the agency. 36
State of Alaska
RFP Evaluators’ Guide
www.alaska.gov/admin/dgs/docs/rfpguide.doc
37
Alaska RFP Evaluators’ Guide ‘Highlights’
NUMERICAL SCORING SYSTEMS
. . . you are not required to provide a written explanation of your individual scores, however, this does not open the door to arbitrary or capricious scoring. . . . You should always have a reasonable, rational, and consistent basis for your scores, as you might be required to explain them to a hearing officer or judge in the event of a protest.
NON-NUMERICAL SCORING SYSTEMS
Sometimes the evaluation criteria can be too difficult to categorize into specific questions or is too subjective to use a numerical rating system; making it necessary to use a non-numerical scoring system. This type of evaluation requires you to provide written documentation about how you came to your conclusions and how you ranked each proposal. Your explanation must be rational and consistently applied from proposal to proposal.
38
What To Do When ALL
the Proposals Come
In OVER Budget?
A challenging situation
39
What can you do before creating your RFP to
avoid this problem?
Do your homework to establish a realistic budget.
During marketing research talk to vendors; invite them in
for presentations.
Hire an expert to establish a budgetary range.
40
What Can You Put in your RFP to Avoid this
Problem?
Publish the budget.
Use appropriate language.
Let the weights and scoring calculations reflect the
budget.
41
Summary
42
o Cost is always an issue.
o Stakeholders’ input is critical for success.
o Points are NOT better than the non-numerical approach.
o There is no reason for proposals to exceed your budget.
43
Constructing the Evaluation
The Building Blocks of The Evaluation Process
1. Establishing Compliance With Mandatory Requirements
2. Scoring the Proposals
3. Imposing Upset Levels
4. Evaluating the Cost
5. Developing a Short List
6. Interviewing Suppliers
7. Negotiating the Contract
8. Requesting Best and Final Offers
9. Using References/Past Performance
5. Developing a Short List
Consider these scores:
82
80
78
72
65
63
50
4844
45
Constructing the Evaluation
The Building Blocks of The Evaluation Process
1. Establishing Compliance With Mandatory Requirements
2. Scoring the Proposals
3. Imposing Upset Levels
4. Evaluating the Cost
5. Developing a Short List
6. Interviewing Suppliers
7. Negotiating the Contract
8. Requesting Best and Final Offers
9. Using References/Past Performance
6. Interviewing the Suppliers
Clarification meetings
Oral Presentations
Demonstrations
46
47
Constructing the Evaluation
The Building Blocks of The Evaluation Process
1. Establishing Compliance With Mandatory Requirements
2. Scoring the Proposals
3. Imposing Upset Levels
4. Evaluating the Cost
5. Developing a Short List
6. Interviewing Suppliers
7. Negotiating the Contract
8. Requesting Best and Final Offers
9. Using References/Past Performance
7. Negotiating the Contract
Solid reasons for negotiating:
Increase the number of high-scoring proposals (providing greater competition).
Reduce risk to both parties.
Eliminate unnecessary costs.
Reduce costs.
Improve benefits (better quality, performance, delivery etc.).
Identify alternative solutions not initially apparent.
Clarify requirements and proposals.
Create better understanding and relationships between the parties.
Improve the contract.
Improve the proposal.
Identify/explore opportunities for partnership.48
Negotiations Can Be Scary!
Many procurement people receive little if any training.
The supplier is better prepared.
Roles and responsibilities are poorly defined.
Many people find negotiations awkward.
49
The Negotiation Process
1. Preparation
2. Fact Finding
3. Bargaining
50
51
Constructing the Evaluation
The Building Blocks of The Evaluation Process
1. Establishing Compliance With Mandatory Requirements
2. Scoring the Proposals
3. Imposing Upset Levels
4. Evaluating the Cost
5. Developing a Short List
6. Interviewing Suppliers
7. Negotiating the Contract
8. Requesting Best and Final Offers
9. Using References/Past Performance
Best and Final Offers
What is BAFO?
Why do it?
How does BAFO work?
What are the benefits?
Examples
Why isn’t BAFO used more often?
52
What is BAFO?
ABA Model Procurement Code
ND Ch 54-44.4.10-3 State Purchasing Practices
ND RFP Evaluators Guide
53
American Bar
Association
54
American Bar Association
Model Procurement Code
For Competitive Sealed Proposals (3-203)
Discussions with responsible offerors
Revisions to proposals
Fair and equal treatment
No disclosure of information from competitors
(Commentary)
Negotiations based on ethical business standards
Auction techniques prohibited
Safeguards against abuse in negotiations55
ND Ch 54-44.4.10-3
State Purchasing
Practices
Discussions may be conducted with responsible offerors who submit proposals
determined to be reasonably susceptible of being selected for award for the purpose
of clarification to assure full understanding of, and responsiveness to, the solicitation
requirements. Offerors must be accorded fair and equal treatment with respect to any
opportunity for discussion and revision of proposals, and revisions may be permitted
after submissions and before award for the purpose of obtaining best and final offers. In
conducting discussions, there may be no disclosure of any information derived from
proposals submitted by competing offerors.56
ND RFP Evaluators
Guide
o Discussions are optional
o Initiated by State
o Some or all vendors
o Letter with terms & issues
o Only one (maybe)
o Revise scores using same team, criteria and process
57
Why use BAFO?
Revise proposals to better fit the requirements
Reduce costs
Increase the spread between proposals
58
How does it work?
RFP without BAFO
RFP with BAFO
59
The RFP Process (Without BAFO)
60
Step 1 - Creating the Statement of Work
Step 2 – Involving Procurement
Step 3 – From Issuing the RFP to Closing
Step 4 - Evaluating Proposals
Step 5 - Finalizing the Contract
Problems without BAFO?
Accept a mediocre proposal
Unofficial negotiations occur
Change orders are commonplace
Vendor’s contract prevails
Negotiate contract with ‘winner’ – no leverage
61
Next, the Road Less Travelled: BAFO
62
The RFP Process (With BAFO)
63
Step 1 - Creating the Statement of Work
Step 2 - Involving Procurement
Step 3 - From Issuing the RFP to Closing
Step 4 - Evaluating Proposals
Step 5 - Negotiating Best & Final Offers - NEW!
Step 6 - Finalizing the Contract
How does BAFO work?
64
Negotiate before BAFO
Then Request BAFO
65
What are the Benefits in Negotiating and
then using BAFO?
o Increase the number of high-scoring proposals (providing greater competition).
o Reduce risk to both parties.
o Eliminate unnecessary costs.
o Reduce costs.
o Improve benefits (better quality, performance, delivery etc.).
o Identify alternative solutions not initially apparent.
o Clarify requirements and proposals.
o Create better understanding and relationships between the parties.
o Improve the contract.
o Improve the proposal.66
Go Directly to BAFO
67
New Mexico’s RFP Guide
provides some insight into
the use of BAFO
The best and final offer is the only step in the process where the proposal can be amended. If the offeror’s proposal contains unacceptable contract terms and conditions, this is the step in the process where that problem is resolved. If an offeror stamped every page of the proposal “proprietary” or “confidential”, this is the step in the process where that problem is corrected. If costs were not proposed on exactly the same basis as the other offerors, this is the step in the process to correct that problem.
Montana’s RulesBest and Final Offers
The committee may decide to seek
best and final offers from one or more
offerors if additional information is
necessary or responses will be altered
in order to make a final decision.
The committee may request only one
best and final offer.
Offerors may not request an
opportunity to submit a best and final
offer. The procurement officer must be
notified of the offerors who are
provided the opportunity to submit
best and final offers and the areas to
be addressed.
69
Montana’s RulesBest and Final Offers
(Cont’d.)
The procurement officer will send out the request for best and final offers in a letter stating the areas to be covered and the date and time in which the best and final offer must be returned.
Proposal scores are adjusted in light of the new information received in the best and final offer.
A best and final offer cannot be requested on price/cost alone unless so stated in the RFP.
70
Some specific
examples
From Oregon
o An overview of BAFO
From Wisconsin
o BAFO Process and
Procedures
71
From Oregon
An overview of BAFO
o A request for BAFO
may be used during
the evaluation
process for any of the
following reasons:
o To clarify or amend
the RFP.
o To request revised
cost proposals from
the proposers.
o To allow proposers to
submit clarifications.
72
http://www.oregon.gov/das/OPM/Pages/proposal.aspx
From Wisconsin
BAFO Process and
Procedures
PRO-C-39
I. Definition
II. When used
III. Procedures
IV. Content and structure
of solicitations
V. Scoring
73
vendornet.state.wi.us/vendornet/procman/proc39.pdf
From Federal Transit
Administration
fta-best-practices-procurement-and-lessons-learned-manual-2016.pdf
Discussion with Offerors, Request for Revised Proposals, Request for BAFO
4.3 Competitive Proposals Evaluation Process
4.3.6 Discussions with Offerors
4.3.7 Request for Revised Proposals
4.3.8 Request Best and Final Offer
74
From NC
BAFO Letter
https://files.nc.gov/ncdit/documents/files/BAFO-
Request-Form.docx
75
From ND
BAFO Letter
https://www.nd.gov/omb/sites/omb/files/documen
ts/agency/purchasing/best-and-final-offer.docx
76
Why isn’t BAFO used more often?
a) Some public agencies are not be aware they can do it
b) There is a perceived lack of legal authority for BAFO (the misconception that negotiations
are always prohibited)
c) Sometimes management doesn’t want the agency to do it (for their own reasons)
d) Some procurement people prefer to avoid negotiations
77
78
Constructing the Evaluation
The Building Blocks of The Evaluation Process
1. Establishing Compliance With Mandatory Requirements
2. Scoring the Proposals
3. Imposing Upset Levels
4. Evaluating the Cost
5. Developing a Short List
6. Interviewing Suppliers
7. Negotiating the Contract
8. Requesting Best and Final Offers
9. Using References/Past Performance
9. Using References/Past Performance
As a prequalification condition
To confirm your selection
As an evaluation factor
79
Inside the Evaluation Process
The Building Blocks of The Evaluation Process
Establishing Compliance
Scoring the Proposals
Imposing Upset Levels
Evaluating the Cost
Developing a Short List
Communicating with Suppliers
Negotiating the Contract
Requesting Best and Final Offers
Checking References/Past Performance
80
Two Other Important Considerations
o Gaming the Evaluation Process
o Constructing the Recommendation Memo
81
Critical Success Factors
A well constructed RFP
A tested evaluation process
An empowered Evaluation Committee
Documentation and probity
A knowledgeable, experienced, ethical procurement officer
82
Conclusions
What have we learned???
83