source selection processes aeronautical systems center dominant air power: design for...
TRANSCRIPT
Source Selection ProcessesSource Selection Processes
Aeronautical Systems CenterAeronautical Systems CenterDominant Air Power: Design For Tomorrow…Deliver Today
ASC ACE
Dan Fulmer
Chris Telepak
Bob Wonderly
2
Dominant Air Power: Design For Tomorrow…Deliver Today
This ISN’T how we do it!and
It’s not always low price!
What Is Source Selection?
3
Dominant Air Power: Design For Tomorrow…Deliver Today
Training Objective
Provide introduction to source selection methods for lower dollar acquisitions of varying complexity
Show the relationship between acquisition planning and choosing the right source selection process
Emphasis is on FAR Part 15 source selection processes
Training covers the Air Force source selection processes under AFFARS 5315.5 and Mandatory Procedure MP5315.5
Also applicable to higher dollar, non-complex acquisitions
4
Dominant Air Power: Design For Tomorrow…Deliver Today
If you compete a buy, what tools are available to you?
5
Dominant Air Power: Design For Tomorrow…Deliver Today
FAR Part 8 and 12FAR Part 8 and 12
FAR Part 15 and AFFARS
Full Trade-Off
PPTPerformance/
Price Trade-Off
LPTALow-Price/Technically Acceptable
Simplified & Sealed Bid
FAR Part 13 & 14
Non-CostNon-Cost
Low Low PricePrice
CostCostPricePrice Perf
Trade-offTrade-off
Tech Acceptable
The Best Value Continuum
Greater Importance of Price Lesser
Lesser Technical Complexity Greater
6
Dominant Air Power: Design For Tomorrow…Deliver Today
FAR Part 12 Commercial Items
Use competition methods in FAR Parts 13, 14, or 15
Choose method appropriate for your buy
Emphasizes market research before solicitation
Maximize sources available for competition
Allows less complex solicitation
Less detail in solicitation format and content
Simpler Instructions to Offerors and Evaluation Criteria
Encourages clear, performance-based item descriptions
12.603 permits combination of synopsis and solicitation
FFP or FFP with Economic Price Adjustment contracts
7
Dominant Air Power: Design For Tomorrow…Deliver Today
FAR Part 13 Solicitations
Buys under Simplified Acquisition Threshold
Under $100K
Supplies or services supporting contingency operations or to facilitate defense against or recovery from nuclear, biological, chemical, or radiological attack
$250,000 for contract awarded and performed, or purchase to be made, inside the United States;
$1 million for contract awarded and performed, or purchase to be made, outside the United States.
Exempt from FAR Part 6 Competition requirements
Solicitations can be oral or written
Solicitation must state basis for evaluation
Price or price and other factors (e.g. past performance)
Formal evaluation plans, discussions, competitive range determinations not required
8
Dominant Air Power: Design For Tomorrow…Deliver Today
FAR Part 14 Sealed Bidding
Adequate re-procurement data or buy part number
FFP or FFP with Economic Price Adjustment contracts
Solicitation sections A-K similar to FAR Part 15 RFP
Section L
Instructions specific to submission of sealed bids
Section M
Specify price related factors if used
Criteria for technical evaluation in two-step sealed bid
Evaluate price or price-related factors
No Discussions
9
Dominant Air Power: Design For Tomorrow…Deliver Today
FAR Part 15 Source Selection
Competitive negotiated acquisitions
Source Selection Procedures are addressed in
FAR Part 15 - Contracting by Negotiation
AFFARS Mandatory Procedures MP5315.3
FAR Part 15.002(b) - Competitive procedures are intended to:
Minimize complexity of the solicitation, evaluation and the source selection decision
Foster an impartial and comprehensive proposal evaluation
Lead to selection of the proposal representing the best value to the Government
10
Dominant Air Power: Design For Tomorrow…Deliver Today
Source Selection Processes And Techniques
Lowest Price Technically Acceptable (LPTA) Process
Tradeoffs are not permitted
Proposals are evaluated for acceptability but not ranked using non-cost/price factors
Price/Performance Trade-off (PPT) Process
Permits tradeoff between price and past performance risk after ranking of technically acceptable offers based on price
Full Tradeoff Process
Permits tradeoffs among cost or price and non-cost factors and allows the Government to accept other than the lowest priced offer
Non-cost factors: mission capability, proposal risk, past performance
11
Dominant Air Power: Design For Tomorrow…Deliver Today
How do you choose the right process?Planning is the key!
12
Dominant Air Power: Design For Tomorrow…Deliver Today
Source Selection Activities
Acquisition StrategyContract Type/LengthMilestonesSource Selection ProcessBudget Considerations
Acquisition StrategyContract Type/LengthMilestonesSource Selection ProcessBudget ConsiderationsRequirements
DocumentRequirements
Document
External InputsUser requirementsFunding
External InputsUser requirementsFunding
RFPRFP
ITOITO
RFP Development
SSPSSP
RequirementsDocument
RequirementsDocument
EvalFactors
EvalFactors
ProposalProposal
Source SelectionSource
Selection
ContractAward &
Execution
ProposalProposal
ProposalProposal
MarketResearch
MarketResearch
RiskAssessment
RiskAssessment
13
Dominant Air Power: Design For Tomorrow…Deliver Today
External Inputs
Consist of two principal components
User Requirements – documented pursuant to local procedures
Funding commensurate with requirement
These inputs must be firm in order to proceed with the next steps in the process
Baseline established herein used for market research, risk assessment, requirements document development
14
Dominant Air Power: Design For Tomorrow…Deliver Today
Requirements Document
Defined: A performance based document that defines the government need without dictating the specific design solution.
Exceptions include commodities, construction, . . .
Benefits
A well written requirements document can reduce government costs by allowing commercial practices and contractor innovation to drive the solution
Process
It is almost always better to start with a clean sheet, as opposed to working backwards from an existing document
Focus on what is absolutely essential to the user/customer
Less is generally more
15
Dominant Air Power: Design For Tomorrow…Deliver Today
Market Research
Defined: An analysis of the market to determine
The extent to which non-developmental or commercial products or services may be used to fulfill government requirements
The potential offerors
Benefits
Increases your knowledge so you can design the most appropriate acquisition strategy
Process
Many options e.g. industry days, web searches, industry associations, Subject Matter Experts….
16
Dominant Air Power: Design For Tomorrow…Deliver Today
Risk Assessment
Defined: An analysis of the pre-mitigated risks associated with the successful completion of required tasks. These should include cost, schedule, and performance requirements, and their interrelationships.
Benefits
Those risks considered “high” should be considered as viable discriminators when developing evaluation factors, and in determining the appropriate acquisition strategy.
Helps focus the RFP and the resultant contract on what’s important
Reduces resources necessary to conduct the source selection and in contract administration
17
Dominant Air Power: Design For Tomorrow…Deliver Today
Risk “Thought Process”
1. Understand Requirements2. Identify Risks to Meeting Requirements3. Develop Mitigations for High Risk Areas
Understanding Risks, Knowing Market, and User Requirements help you to establish appropriate acquisition strategy
Source Selection MethodEvaluation criteriaContract TypeIncentive arrangementsDelivery/performance schedules
18
Dominant Air Power: Design For Tomorrow…Deliver Today
Risk Assessment – Discriminators
Those areas, topics, or requirements that will enable the source selection evaluation team to distinguish among offerors
These may include something as broad as past performance or price – or
Something as narrowly focused as a particular desired capability expressed in the requirements document
The identification of discriminators is arguably the most important determinant in establishing contract objectives, incentives, and evaluation factors.
At this stage, draft the evaluation factors
Use those factors to determine best source selection process
19
Dominant Air Power: Design For Tomorrow…Deliver Today
Industry/User Involvement
Industry is the customer of the RFP
The user is the customer of the eventual product or service produced by industry in order to meet government requirements
Leaving either out of the early phases of the competitive process is short sighted, and will likely lead to surprises during source selection or contract execution
Involve all stakeholders in
Market research
Risk Assessment
Development of requirements document
20
Dominant Air Power: Design For Tomorrow…Deliver Today
Source Selection Activities
Acquisition StrategyContract Type/LengthMilestonesSource Selection ProcessBudget Considerations
Acquisition StrategyContract Type/LengthMilestonesSource Selection ProcessBudget ConsiderationsRequirements
DocumentRequirements
Document
External InputsUser requirementsFunding
External InputsUser requirementsFunding
RFPRFP
ITOITO
RFP Development
SSPSSP
RequirementsDocument
RequirementsDocument
EvalFactors
EvalFactors
ProposalProposal
Source SelectionSource
Selection
ContractAward &
Execution
ProposalProposal
ProposalProposal
MarketResearch
MarketResearch
RiskAssessment
RiskAssessment
21
Dominant Air Power: Design For Tomorrow…Deliver Today
Acquisition Strategy
For the purposes of this training, the acquisition strategy consists of five parts:
Determination of contract type/length
Performance incentives
Milestones of the pre-award process
Selection of the source selection process
Budget considerations
The results of market research, the risk assessment, and user requirements are important considerations in making all of the required decisions
22
Dominant Air Power: Design For Tomorrow…Deliver Today
Acquisition Strategy -Contract Type/Length
Contract type should be determined based on the schedule, cost and performance risks identified during the risk assessment
Should high performance risk be identified, precluding accurate cost estimation within a reasonable degree, a Cost-Reimbursement contract vehicle may be appropriateShould there be little performance risk, a Fixed Price contract may be the better choice
Contract length for product acquisitions should be influenced by the contractors’ feedback regarding how long it will actually take to complete contract requirements
Contract length for all acquisitions is influenced by stability of requirements, statutes, funding, and ability to price
23
Dominant Air Power: Design For Tomorrow…Deliver Today
Performance Incentives
Target specific aspects of performance
Reward technical performance
Reward exceptional service performance
Reward schedule performance
Emphasize cost control
Product Quality
Choose incentives based on
What is important to your customer
What motivates your contractor
Make sure incentives are realistic and attainable
Use both positive and negative incentives
24
Dominant Air Power: Design For Tomorrow…Deliver Today
Acquisition Strategy –Milestones
The milestones to contract award are a required component of most acquisition planning documents and Source Selection Plans
The results of the acquisition team’s market research and risk assessment should contribute to the scheduling of these events
For example, should a significant number of proposals be anticipated, the time allotted for the source selection should be longer
Set realistic expectations on the acquisition schedule
Communicate with your customer
25
Dominant Air Power: Design For Tomorrow…Deliver Today
Acquisition Strategy –Source Selection Process
As mentioned earlier, there are three source selection processes applicable to negotiated acquisitions
Lowest Price Technically Acceptable
Performance Price Tradeoff
Full Tradeoff
Based upon the discriminators identified through the risk assessment process, one of the processes should be chosen, and documented in the acquisition strategy and source selection plan
26
Dominant Air Power: Design For Tomorrow…Deliver Today
Acquisition Strategy –Budget Considerations
There are three principal budget considerations that are tied to the acquisition strategy
Type(s) of money (i.e. 3400 for O&M)
Amount (do you have enough to cover the requirement?)
If not, what requirement can you live without?
Are there alternative ways to meet the requirement?
Annual distribution
Is the funding available at the right time and in the right increments?
The risk assessment should be able to help the team make all required acquisition strategy decisions pertaining to funding
27
Dominant Air Power: Design For Tomorrow…Deliver Today
Your Request for Proposal (RFP) is the next step!
28
Dominant Air Power: Design For Tomorrow…Deliver Today
Source Selection Activities
Acquisition StrategyContract Type/LengthMilestonesSource Selection ProcessBudget Considerations
Acquisition StrategyContract Type/LengthMilestonesSource Selection ProcessBudget ConsiderationsRequirements
DocumentRequirements
Document
External InputsUser requirementsFunding
External InputsUser requirementsFunding
RFPRFP
ITOITO
RFP Development
SSPSSP
RequirementsDocument
RequirementsDocument
EvalFactors
EvalFactors
ProposalProposal
Source SelectionSource
Selection
ContractAward &
Execution
ProposalProposal
ProposalProposal
MarketResearch
MarketResearch
RiskAssessment
RiskAssessment
29
Dominant Air Power: Design For Tomorrow…Deliver Today
Request for Proposals
Requirements
Evaluation Factors (Section M)
Instructions to Offerors (Section L)
Other contract information (e.g. special clauses, representations and certifications)
30
Dominant Air Power: Design For Tomorrow…Deliver Today
Section A Solicitation / Contract FormSection B Supplies or Services / Prices / CostsSection C Description / Specs / Work StatementSection D Packaging and MarkingSection E Inspection and AcceptanceSection F Deliveries or PerformanceSection G Contract Administration DataSection H Special Contract RequirementsSection I Contract ClausesSection J List of AttachmentsSection K Contractor Representations & CertificationsSection L Instructions, Conditions, and Notices to Offerors or QuotersSection M Evaluation Factors for Award
Parts of an RFP
31
Dominant Air Power: Design For Tomorrow…Deliver Today
Sections L and M of the RFP
Section L – “Instructions, Conditions, and Notices to Offerors or Quoters”
Establishes baseline proposal contentSpecifies proposal format and page limitationsTeams should familiarize themselves with this section to ensure that scope of evaluation is not broadened during exchanges
Section M – Evaluation Factors for AwardGeneral Basis for Contract Award
Defines “best value” to the GovernmentIntegrated Assessment of offersSubjective judgment is implicit in the evaluation processEverything is being evaluated– Most items are pass/fail– Sub-Factors represent focus areas
For PPT and FTO, state relative order of importance of factors Two components
FactorSub factor
32
Dominant Air Power: Design For Tomorrow…Deliver Today
Evaluation Factors
Specific characteristics that are tied to significant requirements having an impact on the source selection decision
Expected to be discriminators between the proposals
Uniform baseline against which an offeror’s solution is evaluated to determine its value to the Government
Used to measure how well each offeror meets RFP requirements
Established specifically for each source selection
Written by evaluation team
Derived from requirements, market research and risk assessment
Approved by Source Selection Authority
Used to determine which source selection process is appropriate for the acquisition
33
Dominant Air Power: Design For Tomorrow…Deliver Today
Evaluation Factors
Possible evaluation factors
Mission Capability/Technical
Use sub-factors if necessary
Proposal Risk
Past Performance
Cost/Price
Selecting Evaluation Factors
Factors are selected based on results of Market Research, Risk Assessment and Requirements Document
The choice of the evaluation factors determines which source selection process is appropriate
LPTA, PPT, or FTO
34
Dominant Air Power: Design For Tomorrow…Deliver Today
Evaluation Factors – Mission Capability/Technical
The evaluation provides an assessment of the offeror’s capability to technically satisfy the Government’s requirements
For LPTA and PPT, technical acceptability is evaluated on pass/fail basis
For FTO, mission capability/technical capability is evaluated
Evaluation is expressed by color ratings: blue/exceptional, green/acceptable, yellow/marginal, or red/unacceptable
Narrative assessment is expressed in terms of strengths, uncertainties, and deficiencies
All technical requirements are not evaluated; only those characteristics, or discriminators, that truly differentiate one proposal from another
Subjective judgment implicit in the evaluation
35
Dominant Air Power: Design For Tomorrow…Deliver Today
Evaluation Factors – Proposal Risk
Optional for FTO acquisitions at or below $10 million; not normally used for LPTA or PPT
Assesses the likelihood or risk that the proposed approach will cause significant disruption of schedule, increased cost or degraded performance
Evaluation is expressed by High, Medium, Low risk ratings
Narrative assessment is expressed in terms of weaknesses
36
Dominant Air Power: Design For Tomorrow…Deliver Today
Evaluation Factors – Performance Confidence Assessment
Assesses the degree of confidence the AF has in an offeror’s ability to provide products/services based on his demonstrated record of contract compliance
Evaluation is expressed by Confidence Ratings: High, Significant, Satisfactory, Unknown, Little, or No Confidence
Applies to FTO and PPT and could apply to LPTA
If used in LPTA, past performance would be a part of the evaluation to decide if an offer is technically acceptable
37
Dominant Air Power: Design For Tomorrow…Deliver Today
Evaluation Factors – Cost/Price
Evaluated for reasonableness and realism in accordance with FAR 15.4, as supplemented
Mandatory for all source selection processes
38
Dominant Air Power: Design For Tomorrow…Deliver Today
Instructions to Offerors
What are ITOs?
Proposal preparation instructions for offerors
Format and content
Section L in RFPs
Steps to develop ITOs
Look at your evaluation factors
Only ask for what is necessary to evaluate those factors
You have to evaluate everything you ask for
You have to determine that offeror meets all RFP requirements
39
Dominant Air Power: Design For Tomorrow…Deliver Today
You wrote the RFPYou have proposals
How do you evaluate the offers?
40
Dominant Air Power: Design For Tomorrow…Deliver Today
Source Selection Activities
Acquisition StrategyContract Type/LengthMilestonesSource Selection ProcessBudget Considerations
Acquisition StrategyContract Type/LengthMilestonesSource Selection ProcessBudget ConsiderationsRequirements
DocumentRequirements
Document
External InputsUser requirementsFunding
External InputsUser requirementsFunding
RFPRFP
ITOITO
RFP Development
SSPSSP
RequirementsDocument
RequirementsDocument
EvalFactors
EvalFactors
ProposalProposal
Source SelectionSource
Selection
ContractAward &
Execution
ProposalProposal
ProposalProposal
MarketResearch
MarketResearch
RiskAssessment
RiskAssessment
41
Dominant Air Power: Design For Tomorrow…Deliver Today
Lowest Price Technically Acceptable (LPTA) Process
When to use LPTA?
Appropriate when best value is expected from selection of technically acceptable proposal with the lowest evaluated price
Past Performance does not have to be an evaluation factor
When not to use LPTA?
If you want to trade off past performance, price, or technical parameters
Tradeoffs are not permitted in LPTA source selections
Proposals are evaluated for technical acceptability but not ranked using non-cost/price factors
Exchanges may occur
Team members are the Contracting Officer and technical lead
42
Dominant Air Power: Design For Tomorrow…Deliver Today
RFP SECTION M LPTA Factors & Ratings
Mission Capability/Technical Capability
Acceptable
Reasonably susceptible of being made acceptable
Unacceptable
Cost/Price
Reasonableness
Realism
RFP requirements (Ts & Cs, Certs & Reps, etc.)
43
Dominant Air Power: Design For Tomorrow…Deliver Today
Reasonably susceptible of being made acceptable. The proposal does not meet all the minimum mandatory requirements in the solicitation identified as subfactors within the Mission Capability Evaluation Standards, however, there is reason to believe that through minor revisions an acceptable proposal could result. For award without discussions these proposals are considered “unacceptable.”
Mission/Technical Capability Ratings
Acceptable: The proposal meets all the minimum mandatory requirements in the solicitation identified as subfactors within the Mission Capability Evaluation Standards. Only those proposals determined acceptable, either initially or as a result of exchanges will be considered for award. Once deemed acceptable, all mission capability proposals are considered to be equal.
Unacceptable: Fails to meet all the minimum mandatory requirements in the solicitation identified as subfactors within the Mission Capability Evaluation Standards. Proposals with an unacceptable rating will not be considered for award.
44
Dominant Air Power: Design For Tomorrow…Deliver Today
LPTA Evaluation Steps
Establish evaluation factors and sub factors before solicitation release, i.e., technical acceptability
Tech eval shall document evaluations in sufficient detail to explain each pass/fail decision
Discussions are possible for offers initially rated as “reasonably susceptible of being made acceptable”
For award without discussions, these offers are “unacceptable”
Award made to lowest evaluated cost (price) offer that meets all minimum mandatory criteria
PCO makes award decision and ensures award decision is documented
45
Dominant Air Power: Design For Tomorrow…Deliver Today
PPT – When To Use It
Allows the government to award to other than the offeror with the lowest evaluated price and award to an offeror with a better performance confidence rating
Appropriate to broad spectrum of requirements, e.g.,
Replenishment spares
Operational contracting acquisitions
Non-developmental, noncomplex service or supplies
Service contracts with only pass/fail technical requirements
Low technical complexity “build to print” contracts
46
Dominant Air Power: Design For Tomorrow…Deliver Today
Performance Price Trade-Off (PPT) Process
Permits tradeoff between price and past performance confidence
Technically acceptable proposals are determined then tradeoffs are made between price and past performance evaluation to determine successful offeror
Technical factors may be evaluated on pass/fail basis only but cannot be traded off for price
Only factor being traded off with price is past performance
Need to determine relative importance of past performance and price factors (equal, more significant, less significant)
47
Dominant Air Power: Design For Tomorrow…Deliver Today
PPT Factors & Ratings
Mission Capability/Technical CapabilityAcceptableReasonably susceptible of being made acceptableUnacceptable
Past Performance EvaluationHigh ConfidenceSignificance ConfidenceSatisfactory ConfidenceUnknown ConfidenceLittle Confidence No Confidence
Cost/PriceReasonablenessRealism
RFP requirements (Ts & Cs, Reps & Certs, etc.)
Tradeoff
zone
48
Dominant Air Power: Design For Tomorrow…Deliver Today
Source Selection OrganizationPerformance Price Tradeoff
Source Selection Authority(SSA)/PCO*
Source Selection Authority(SSA)/PCO*
Past PerformanceFactor
Cost/Price Factor
Technical Acceptability Factor
*SSA level depends on dollar value, local policy and regulation
49
Dominant Air Power: Design For Tomorrow…Deliver Today
PPT Evaluation Steps
When technical proposals are required, determine technical acceptability of each offeror
Based on pass/fail evaluation criteria in the solicitation
Offeror must pass all criteria to be considered acceptable
If contractor is determined technically unacceptable, do not go further with price, performance evaluations
Price for each technically acceptable proposal will be evaluated for price reasonableness, then ranked by total evaluated price to determine the low offeror
Assess performance confidence for each offeror
50
Dominant Air Power: Design For Tomorrow…Deliver Today
PPT Evaluation Steps (cont’d)
Past Performance Evaluation
Past Performance Information on relevant contracts submitted with proposal
Government obtains performance feedback through questionnaires, telephone surveys and automated systems
e.g., Contractor Performance Assessment Reporting System; Past Performance Information Retrieval System
Performance Confidence assigned using the performance ratings established in MP5315.3, Table 3
High Confidence---Significant Confidence---Satisfactory ConfidenceUnknown Confidence---Little Confidence---No Confidence
51
Dominant Air Power: Design For Tomorrow…Deliver Today
PPT Evaluation Steps (cont’d)
May award without discussionsIssues dealing with Past Performance don’t qualify as discussions (relevancy of data, adverse performance reports)
Discussions may be necessaryOffers initially rated “reasonably susceptible of being made acceptable”Technical IssuesCost/Price
Request Final Proposal Revisions
Evaluate Final Proposal Revisions
Award may be made to the technically acceptable, low price offeror with “acceptable” performance confidence
52
Dominant Air Power: Design For Tomorrow…Deliver Today
PPT Tradeoff Considerations
Consider potential performance-price benefit of all offerors with more highly rated past performance than lowest priced offeror in accordance with RFP
Determine benefit to Air Force of awarding to offeror with better past performance at higher cost
The government has the right to make a trade-off decision and award to other than the low offeror based on better performance confidence
Basis for decision must be thoroughly documented
53
Dominant Air Power: Design For Tomorrow…Deliver Today
Full Tradeoff Process
Permits tradeoffs among cost or price and non-cost factors and allows the Government to accept other than the lowest priced offer if you can justify paying more money for better performance.
Use this method when you want, or need, the maximum flexibility to trade off all aspects of your requirement (technical, management, past performance, cost/price) to select the best overall offer.
This method offers the maximum flexibility in making a “best value” decision.
If you don’t need this flexibility, LPTA, PPT, or perhaps sealed bidding are better alternatives.
Permits tradeoffs among cost or price and non-cost factors and allows the Government to accept other than the lowest priced offer if you can justify paying more money for better performance.
Use this method when you want, or need, the maximum flexibility to trade off all aspects of your requirement (technical, management, past performance, cost/price) to select the best overall offer.
This method offers the maximum flexibility in making a “best value” decision.
If you don’t need this flexibility, LPTA, PPT, or perhaps sealed bidding are better alternatives.
54
Dominant Air Power: Design For Tomorrow…Deliver Today
Evaluation Factors
Evaluation Factors
EvaluatIon
EvaluatIon
Strengths
Deficiencies
Weaknesses
Ratings
Initial Evaluation
Debrief
FTO Evaluation Process
OfferorProposals
Award w/oDiscussion
Award w/oDiscussion
Initial Evaluation
Briefing
Discussions
CompetitiveRange
Determination
CompetitiveRange
Determination
DiscussionsDiscussions
FinalProposal
FinalProposal
ENs
Final Evaluation
Revise
Strengths
Deficiencies
Weaknesses
Ratings
AwardAward
Final Evaluation
Briefing
Best Value Decision
ENs
55
Dominant Air Power: Design For Tomorrow…Deliver Today
FTO Factors & Ratings
Mission Capability
Blue - Exceptional
Green - Acceptable
Yellow - Marginal
Red - Unacceptable
Past Performance Evaluation
High Confidence
Significant Confidence
Satisfactory Confidence
Unknown Confidence
Little Confidence
No Confidence
Proposal Risk
High
Moderate
Low
Cost/Price
Reasonableness
RealismCost/Price Risk
– Same ratings as Proposal Risk
RFP Requirements (Ts & Cs, Certs & Reps, etc.)
56
Dominant Air Power: Design For Tomorrow…Deliver Today
Source Selection OrganizationFull Tradeoff
Source Selection Authority(SSA)/PCO*
Source Selection Authority(SSA)/PCO*
Mission CapabilityFactor
Mission CapabilityFactor
Proposal RiskFactor
Proposal RiskFactor
SubFactorSubFactor
SubFactorSubFactor
Cost/PriceFactor
Cost/PriceFactor
Past PerformanceFactor
Past PerformanceFactor
Other RFP Requirements
Other RFP Requirements
*SSA level depends dollar value, local policy and regulation
57
Dominant Air Power: Design For Tomorrow…Deliver Today
FTO Evaluation - Mission Capability
Mission Capability/Technical Capability Factor
The Mission Capability assessment focuses on how the proposal exceeds, meets, or fails to meet the Mission Capability subfactor requirements.
Each Mission Capability subfactor will receive one color rating.
58
Dominant Air Power: Design For Tomorrow…Deliver Today
Mission Capability Evaluation
Can be done at the factor level, if you don’t use sub-factors
If you use sub-factors, evaluation is at the sub-factor level (shall not roll up the ratings)
Focus on strengths, uncertainties and deficiencies
Written assessment shall include:
Strength: a significant, outstanding, or exceptional aspect of an offeror’s proposal that has merit and exceeds specified performance or capability requirements in a way that is advantageous to the government, and either will be included in the contract or is inherent in the offeror’s process.
Deficiency: A material failure of a proposal to meet a government requirement
Uncertainties: You can’t tell if they meet all requirements
59
Dominant Air Power: Design For Tomorrow…Deliver Today
Mission Capability Subfactors Color Ratings
BLUE-EXCEPTIONAL: Exceeds specified minimum performance or capability requirements in a way beneficial to the Government; proposal must have one or more strengths and no deficiencies to receive a blue.
BLUE-EXCEPTIONAL: Exceeds specified minimum performance or capability requirements in a way beneficial to the Government; proposal must have one or more strengths and no deficiencies to receive a blue.
GREEN-ACCEPTABLE: Meets specified minimum performance or capability requirements delineated in the Request for Proposal; proposal rated green must have no deficiencies but may have one or more strengths.
GREEN-ACCEPTABLE: Meets specified minimum performance or capability requirements delineated in the Request for Proposal; proposal rated green must have no deficiencies but may have one or more strengths.
YELLOW-MARGINAL: Does not clearly meet some specified minimum performance or capability requirements delineated in the Request for Proposal, but any such uncertainty is correctable.
YELLOW-MARGINAL: Does not clearly meet some specified minimum performance or capability requirements delineated in the Request for Proposal, but any such uncertainty is correctable.
RED-UNACCEPTABLE: Fails to meet specified minimum performance or capability requirements; proposal has one or more deficiencies. Proposals with an unacceptable rating are not awardable.
RED-UNACCEPTABLE: Fails to meet specified minimum performance or capability requirements; proposal has one or more deficiencies. Proposals with an unacceptable rating are not awardable.
60
Dominant Air Power: Design For Tomorrow…Deliver Today
FTO Evaluation - Proposal Risk
Proposal Risk Factor
• Focus on the Mission Capability sub factors
• The Proposal Risk assessment focuses on:
- The weaknesses associated with an offeror's proposed approach and
- Includes an assessment of the potential for disruption of schedule, increased cost, degradation of performance, and
- The need for increased Government oversight,
- Likelihood of unsuccessful contract performance
- For each offeror identified risk, the assessment also addresses the offeror's proposal for mitigating the risk and why that approach is or is not manageable
• Each Mission Capability subfactor will receive one Proposal Risk rating.
61
Dominant Air Power: Design For Tomorrow…Deliver Today
Proposal Risk Ratings
High
Likely to cause significant disruption of schedule, increased cost, or degradation of performance. Risk may be unacceptable even with special contractor emphasis and close government monitoring.
Likely to cause significant disruption of schedule, increased cost, or degradation of performance. Risk may be unacceptable even with special contractor emphasis and close government monitoring.
ModerateCan potentially cause disruption of schedule, increased cost, or degradation of performance. Special contractor emphasis and close government monitoring will likely be able to overcome difficulties.
Can potentially cause disruption of schedule, increased cost, or degradation of performance. Special contractor emphasis and close government monitoring will likely be able to overcome difficulties.
Has little potential to cause disruption of schedule, increased cost, or degradation of performance. Normal contractor effort and normal government monitoring will likely be able to overcome any difficulties.
Has little potential to cause disruption of schedule, increased cost, or degradation of performance. Normal contractor effort and normal government monitoring will likely be able to overcome any difficulties.
Low
62
Dominant Air Power: Design For Tomorrow…Deliver Today
FTO Evaluation – Past Performance
• Perform Past Performance Evaluation -•Focuses on, not limited to, mission capability sub-factors •Reviews recent and relevant past performance as predictor for success
• Rating is established through a review and analysis of the offeror’s recent, current, and relevant contract performance
• Emphasis is on demonstrated performance• Measures the level of confidence the government has in the
offeror’s ability to perform• Assigns a confidence assessment rating based on
assessment of offeror’s past performance
High Confidence---Significant Confidence---Satisfactory ConfidenceUnknown Confidence---Little Confidence---No Confidence
63
Dominant Air Power: Design For Tomorrow…Deliver Today
After Receipt of ProposalsRating Methodology
AssignRelevancy Rating
to each subfactor for each contract
Assign
Performance Rating to each relevant
subfactor for each contract
Assign Confidence rating to each
subfactor for each offeror
Assign FactorConfidence Rating
to each offeror
64
Dominant Air Power: Design For Tomorrow…Deliver Today
Offeror A Evaluation Summary
Factor Rating – High Confidence
PCAG EVALUATION SUMMARY
Contract F
Relev.Relev. Rating Relev. Relev.Rating Rating Rating
E
VG
VGM
S
E
E
PE
HRHR
HRRR
E
EN/A
S
E
VG
VGVG
Sub factorRatings:
HRHRR
HRNRNR
N/A
N/A
RR
SRR
R
HRHRHRRRR
N/A E
Program
SIGNIFICANTCONFIDENCE
HIGHCONFIDENCE
HIGHCONFIDENCE
HIGHCONFIDENCE
VG
VG
VG
E
Sub factor 2Sub factor 1 Sub factor 3 Cost/Price
NR
NR
Contract E
Contract D
Contract C
Contract B
Contract A
65
Dominant Air Power: Design For Tomorrow…Deliver Today
FTO Evaluation – Cost/Price
Team evaluates cost proposals to determine:
Is the cost or price reasonable?
Price a prudent person would pay when consideration is given to prices in the market
Is the cost or price realistic
What is the risk the offeror can perform at the cost or price proposed?
Degree of analysis varies with
Contract complexity
Degree of competition
Type of product/service being acquired
Contract type
Entire source selection team should have access to cost/price data
Part of integrated assessment of offers
66
Dominant Air Power: Design For Tomorrow…Deliver Today
FTO Evaluation – Cost/Price
Cost Reimbursement ContractsCost Realism Analysis
Costs realistic for the work to be performed;Costs reflect understanding of the requirements; andCosts are consistent with the performance and materials described in the offeror’s proposal
Possible methods you can useTechnical evaluationsCost modelsComparison to similar efforts or market history
Firm Fixed Price ContractsPrice Analysis required
Price competitionComparison to similar itemsPublished prices
No cost/price data beyond CLIN prices
67
Dominant Air Power: Design For Tomorrow…Deliver Today
Sample Source Selection Factor Evaluation Matrix – Full Tradeoff
Integrated assessment of above factors and sub factors will be made in order to determine “best value” to the Government.
ProposalRisk 1
ProposalRisk 2
ProposalRisk 3
ProposalRisk 4
Mission CapabilityS
ub
fac
tor
1 W
arfi
gh
ter
Su
pp
ort
Su
b f
acto
r 2
Res
po
nsi
ven
ess
Su
b f
acto
r 3
Tea
m S
tru
ctu
re
Su
b f
acto
r 4
SB
/SD
BB
usi
nes
s S
trat
egy
Price/Cost
Past Performance**assessed at subfactor, rated at factor level
68
Dominant Air Power: Design For Tomorrow…Deliver Today
Source Selection Types
Mission Capability / Past ProposalPrice Tech Acceptability Performance Risk
Low price LPTA +/-PPT +/- FTO
69
Dominant Air Power: Design For Tomorrow…Deliver Today
Source Selection Activities
No matter what process you use, some activities are common to all …
No matter what process you use, some activities are common to all …
70
Dominant Air Power: Design For Tomorrow…Deliver Today
Preparing to Evaluate Proposals
Read key RFP documents
Government Executive Summary
RFP Sections A-K, especially:
Requirements document (SOO, SOW, PWS, Spec)
Delivery schedule (Section F)
Special Contract Requirements (Section H)
CDRL (DIDs)
Section L (Instructions to Offerors)
Section M (Evaluation Criteria)
71
Dominant Air Power: Design For Tomorrow…Deliver Today
Source Selection - Key Participants and Responsibilities
Source Selection Authority (SSA)Makes the SS decisionEnsures Source Selection Inbriefing certificates signedSigns Source Selection Decision Document (SSDD)Establishes the competitive range
Contracting Officer (CO)May be the SSA for smaller dollar actionsControls access to and release of source selection information (SSI) after RFP release, including evaluation notices (ENs)Ensures protection of SSIEvaluates cost (price) proposals and negotiates contractsConducts discussions with offerors
Mission Capability (MC)/Technical leadEvaluates and rates MC/technical factors and sub factorsPrepares evaluation noticesProvides input on recommendations for the competitive range
72
Dominant Air Power: Design For Tomorrow…Deliver Today
Initial Evaluation
Evaluators analyze proposals against evaluation factors from solicitation
Focus on strengths, deficiencies, uncertainties, weaknesses, performance confidence and cost/price
Proposals generally reviewed in their entirety prior to moving to second offer
Facilitates evaluation against RFP, not other offer
Evaluation Notices (ENs) should be written concurrent with review
Results documented to support SSA decision
73
Dominant Air Power: Design For Tomorrow…Deliver Today
Atch 3
74
Dominant Air Power: Design For Tomorrow…Deliver Today
Initial Evaluation DocumentationCompetitive Range Determination
Documentation of initial evaluation results
Strengths, uncertainties, deficiencies, weaknesses
Past performance
Cost/Price
Summarizes evaluation findings, ratings, narrative, ENs, significant issues
SSA makes Competitive Range Determination when one or more offerors recommended for elimination
Offerors who can effectively compete-most highly rated
Approval to enter discussions
Release ENs
Award without discussions if possible
75
Dominant Air Power: Design For Tomorrow…Deliver Today
Evaluation Notices (ENs)
After proposal receipt, communication with offerors via Evaluation Notices (ENs)
Used to better understand offeror proposals or notify offerors about problems, questions, or issues with proposals
Adverse past performance, uncertainties, deficiencies
Write clearly and concisely with specific references to the RFP and proposal documents
Contracting Officer and technical lead screen ENs, legal reviews them, and the Contracting Officer issues them
Always identify the type of EN
Clarification, Communication, Discussion
EN must clearly identify deficiencies
76
Dominant Air Power: Design For Tomorrow…Deliver Today
Clarifications*as needed with AWOD
Communications*to determine competitive range
before discussions
Negotiations / Discussions
*Neither clarifications nor communications allow for an opportunity for proposal revision.
Adverse past performance
PPI relevance
Minor or clerical errors
Shall
Adverse PPI is determining factor in exclusion
May -
Inclusion or exclusion uncertain
Enhance gov’t understanding
Reasonable interpretation
Facilitate gov’t evaluation process
Ambiguities, perceived weaknesses, omissions, mistakes, etc . . .
PPI relevance
Conducted with offerors in the competitive range
Goal is to get best value
Discuss uncertainties, weaknesses, deficiencies, and other aspects of the proposal to enhance award
Discuss efforts above mandatory minimums
Types Of ENs
77
Dominant Air Power: Design For Tomorrow…Deliver Today
Atch 2
78
Dominant Air Power: Design For Tomorrow…Deliver Today
Discussions
Negotiations with offerors in the competitive range
Resolve Evaluation Notices
Must discuss
Significant weaknesses, uncertainties, deficiencies
Adverse past performance information not previously known to offeror
Other aspects to enhance award potential
Government may provide actual interim ratings
As reviewed by the SSA
At beginning and end of discussions, if possible
Telephonic and/or face to face discussions if necessary
Proposal revisions form baseline for final evaluation
79
Dominant Air Power: Design For Tomorrow…Deliver Today
Release of Interim Ratings
You may release an offeror’s interim ratingsConcurrent with ENs, andImmediately prior to request for Final Proposal Revision (FPR)
Provides description of offeror’sStrengthsWeaknessesDeficiencies
Actual color codes and risk rating charts
Ensure complete understanding of evaluation
80
Dominant Air Power: Design For Tomorrow…Deliver Today
Don’t favor one offeror over another
Don’t reveal technical solution or other information that compromises an offeror’s intellectual property
Don’t reveal an offeror’s price without that offeror’s permission
Don’t reveal the names of individuals who provided reference information about past performance
Don’t furnish source selection information in violation of FAR 3.104
Limits on ENs and Discussions
81
Dominant Air Power: Design For Tomorrow…Deliver Today
Final Proposal Revision
After discussions offerors are requested to submit final proposal revision
Request for FPR includes the model contractTechnical baselineTerms and conditionsBusiness arrangement
Offeror will price final negotiated agreement
Contracting Officer establishes a common cutoff date for discussions
No further discussions after cutoff date
Recommend leaving time for questions about FPR request
Evaluators review the final proposal revisions
Focus on changes to original proposal
All changes should be traceable to original proposalAll changes should be traceable to original proposal
82
Dominant Air Power: Design For Tomorrow…Deliver Today
Final Evaluation & SSA Decision
Final Evaluation Documentation
Contains final evaluation results
Evaluation team makes an award recommendation to SSA
Documentation must include the integrated assessment of
Cost/Price
Performance Confidence
Color ratings for mission capability/technical sub factors
Proposal risk ratings for each subfactor
SSA makes decision based on analysis
SSA writes and signs Source Selection Decision Document
Evaluation team may assist in drafting the SSDD
83
Dominant Air Power: Design For Tomorrow…Deliver Today
Documentation is a key to avoid protests
How do you document your evaluation and decision?
84
Dominant Air Power: Design For Tomorrow…Deliver Today
Source Selection Documentation
Evaluation
ENs Price
WeaknessesUncertaintiesDeficiencies
PCAG
Evaluation
ENs Price
WeaknessesUncertaintiesDeficiencies
PCAG
Evaluation Summary
Decision Briefing Charts
Proposal Analysis Report (PAR) or
Simplified Source Selection Report
SourceSelectionDecision
Document(SSDD)
Source
SelectionPlan(SSP)
Source
SelectionPlan(SSP)
85
Dominant Air Power: Design For Tomorrow…Deliver Today
• Information the disclosure of which would jeopardize the integrity or successful completion of the acquisition
• Sensitive government information pertaining to the source selection
- Examples: Source Selection Plan
Analysis Worksheets
Proposal Analysis Report
What is in Contractor Proposals
Proposals from competing contractors
Number and identity of offerors
Who is serving as evaluators
Anything about the evaluation
Past Performance Information (PPI)
Source Selection Information (SSI)
86
Dominant Air Power: Design For Tomorrow…Deliver Today
(continued)
• Requirement to protect SSI extends into the post-award period
• Includes talking about what happened during the source selection
• FAR 3.104 covers protection of SSI
• Use Cover sheet
- Mark as:
“SOURCE SELECTION INFORMATION
SEE FAR 2.101 and 3.104”
&
“FOR OFFICIAL USE ONLY”
Source Selection Information
87
Dominant Air Power: Design For Tomorrow…Deliver Today
(continued)
• Do not discuss information on source selection proceedings with non-members (during and after)
• Each member is personally responsible for security of data
• Don’t remove data from evaluation area
• Use Cover sheet if you must remove SSI
• Source selection information briefing certificate required
• PCO serves as the sole point of contact between the offerors and the government after RFP release
Source Selection Information
88
Dominant Air Power: Design For Tomorrow…Deliver Today
Source Selection Plan (SSP)
Contents:
Brief description of the requirement
Evaluation factors and relative importance
Evaluation process
Techniques to be used in evaluating proposals
Signed by the SSA prior to RFP release
The SSP is Part I of the Simplified Source Selection Report
Does not have to be a formal plan
Can be documented in briefing charts
89
Dominant Air Power: Design For Tomorrow…Deliver Today
PPT Source Selection Documentation
Source Selection Plan
Not required, but recommended if SSA is other than the CO
Draft Request for Proposal (if used)/Request for Proposal
Proposals
Evaluation Worksheets and Summaries
Competitive Range Determination, if applicable
Evaluation Notices, if applicable
Decision Briefing, if SSA is other than the CO
Source Selection Decision Document
Include comparative assessment
90
Dominant Air Power: Design For Tomorrow…Deliver Today
Simplified Source Selection Report
Used for smaller dollar and less complex source selections
Section I – Source Selection Plan and Acquisition Description
Streamlined Source Selection Plan
Section II – Evaluation
Rating Team Worksheets
Price Competition Memorandum
Section III – Comparative Analysis
Concise comparative analysis of offerors
Supports the source selection decision
Section IV – Source Selection Decision Document
Signed by SSA
Debriefing documentation can be attached to this section
91
Dominant Air Power: Design For Tomorrow…Deliver Today
Requirement for the PAR
Used to document evaluations for more complex, higher dollar value source selections
Objectives of the PAR
Document the results of the evaluation
Provide comparative analysis of offerors in competitive range
Integrated assessment of all factors and sub factors
FAR 15.308, AFFARS 5315.308, Mandatory Procedure MP5315.308
SSA decision based on comparative assessment
Against all evaluation criteria
Documentation
Rationale for business judgments
Tradeoffs, including benefits associated with additional costs
92
Dominant Air Power: Design For Tomorrow…Deliver Today
Content of the PAR
Introduction
Description of Proposals
Evaluation Results
Comparative Analysis of Offers
Contract Considerations
93
Dominant Air Power: Design For Tomorrow…Deliver Today
Content of the PAR Introduction
State the authority for the acquisition
PMD or other reference
Source Selection Procedure Used
Part 15 (FTO, PPT, LPTA), Part 12 etc.
Process overview and describe significant events
Evaluation Criteria
Offerors
All Offerors who responded to RFP
Offerors in competitive range
94
Dominant Air Power: Design For Tomorrow…Deliver Today
Content of the PAR Description of Proposals
Key Technical features for each offeror
Unique technical features
Other unique attributes of the proposal
Key contract features for each offeror
Type of contract
Unique terms or conditions
No judgments, comparisons or ranking in this section of the PAR
95
Dominant Air Power: Design For Tomorrow…Deliver Today
Content of the PAREvaluation Results
Present results by factor and subfactor
Mission capability
Strengths, deficiencies, weaknesses, risks
Performance Confidence Assessment
Results of PCAG evaluation
Data gathered
Results for evaluation criteria
Confidence Assessment
Summary
Cost/Price evaluation
Summarize evaluation for each offeror
96
Dominant Air Power: Design For Tomorrow…Deliver Today
Content of the PAR Comparative Analysis
Compare each offeror in competitive range
Compare by factor and subfactor
Mission capability
Color and risk rating for each offeror
State differences between each offeror
Overall comparison
Comparison of cost/price for each offeror
Scope and method of evaluation
Proposals
OGCs
97
Dominant Air Power: Design For Tomorrow…Deliver Today
Content of the PAR Contract Considerations
Summarize questions and answers for each offeror
Results of ENs
Clarifications, communications and discussions
Compare differences in contracts
Key or unique contract features
Unique terms and conditions
98
Dominant Air Power: Design For Tomorrow…Deliver Today
Source Selection Decision Document(SSDD)
SSA memorandum documenting, justifying and explaining the award decision
Includes SSA’s rationale for the decision, including tradeoffs
SSDD represents the SSA’s independent judgment and comparative analysis of offeror's proposals
SSDD is required for all competitive negotiated acquisitions, including LPTA, PPT, and FTO
SSA writes the SSDD
Legal review recommended
99
Dominant Air Power: Design For Tomorrow…Deliver Today
SSDD Content
LPTA
No comparative analysis of technical proposals
Decision based on low price
SSDD can address responsibility of offerors
PPT
No comparative analysis of technical proposals
Address comparative analysis and tradeoffs of past performance and price
Address responsibility of offerors if necessary
FTO
Address comparative analysis and tradeoffs of all evaluation factors
Address responsibility of offerors if necessary
100
Dominant Air Power: Design For Tomorrow…Deliver Today
SSDD Content
Referencing phrase (“Pursuant to AFFARS 5315, the SSP, and etc.”)
Award decision rationale statement
Mission, scope, contract approach, offerors
Evaluation method (list the factors consistent with RFP Section M and include their importance compared to cost; explain rating method(s) used - colors, adjectival, weights, and/or ordinal rankings)
The Mission Capability factor and its sub factors (sub factors listed in descending order of importance)
The Mission Capability factor’s subfactor comparative analysis (the analysis is not comparative for LPTA and PPT acquisitions because the technical evaluation is not part of the tradeoff)
101
Dominant Air Power: Design For Tomorrow…Deliver Today
SSDD Content
Proposal Risk factor’s subfactor comparative analysis (the analysis is not comparative for LPTA and PPT acquisitions because the technical evaluation is not part of the tradeoff)
Comparative analysis of the performance confidence assessment at the factor level (the analysis is not comparative, if used, for LPTA acquisitions because the past performance evaluation is not part of the tradeoff)
Cost/Price factor (except that this is usually just termed “Price” on LPTA and PPT acquisitions)
Recommendation(s) from the SSET or SSAC. Include rationale for any disagreement with the recommendation(s).
Summary determination.
SSA signature block
102
Dominant Air Power: Design For Tomorrow…Deliver Today
Summary Documentation Matrix
Activities/Documents
Source Selection Plan
Non-complex requirement (P/N, NSN, Drawings, Spec)
Mission Capability Evaluated
Proposal Risk Evaluated
Past Performance Evaluated
Price Evaluated
LPTA PPT FTO
Formal plan not required
Formal plan not required
Yes, as Part 1 of the SSSR
Yes Yes No
Pass/Fail Pass/Fail Yes
No No Optional
Optional* Yes Yes
Yes Yes Yes
*Past Performance need not be evaluated if the CO documents why past performance is not appropriate (i.e. not a discriminator)
*Past Performance need not be evaluated if the CO documents why past performance is not appropriate (i.e. not a discriminator)
103
Dominant Air Power: Design For Tomorrow…Deliver Today
Summary Documentation Matrix
Tradeoffs
Notify offerors that they have been excluded from competitive range
Price Competition Memo required
Source Selection Authority
Source Selection Decision Document
LPTA PPT FTO
NoYes,
between Past Performance and
Price
Yes,
between all factors
N/A Yes Yes
Yes Yes Yes
PCO* PCO* PCO*
Yes Yes Yes
*SSA level will depend on dollar value of acquisition and local review and approval threshold requirements
104
Dominant Air Power: Design For Tomorrow…Deliver Today
Summary
Each Source Selection is unique
Source Selection is a subjective process
The purpose of Source Selection is to provide best value products and services to your customer