a top ten you can use: ten key fixes that avoid big problems in procurements and proposals breakout...

A Top Ten You Can Use:

Ten Key Fixes that Avoid Big Problems in Procurements

and Proposals

Breakout Session #: G02

Presented by: Jacqueline Owens Lancaster

Date: 24 July 2013

Time: 9:45 a.m.

About Your Speaker

3

Jacqueline Owens Lancaster (Jackie O.)

CBDO/VP, ASI Government, since 2011

BD/Capture/Proposal executive roles in major Government contracting firms since 1996

Worked on first proposal in 1985

Started career supporting “Big A” acquisition for U.S. Navy (NAVAIR)

MS, Management, University of Maryland

BS, Mass Communications, Virginia Commonwealth University

Who’s in the audience?

World Congress 2013 4

I have written or contributed to developing RFPs/ RFQs for the Government I have written

proposals in response to Government RFPs/RFQs

I’ve done both!

Right hand up

Left hand up

Both hands up

More and more of us have been on both sides, so let’s face it…

…we need to stop hurting each other!

Little things can lead to big trouble

5World Congress 2013

Government Industry

Failure to allot adequate market research time

Generic proposal pricing requirements

Delays caused by need to amend RFP

Discussions required to complete evaluation

Winner selected, but paper trail very messy

Limited debriefs to prevent protest

Failure to respond with good pricing data

Multiple questions on pricing requirements

Confusing proposals based on RFP changes

Further confusion in Final Proposal Revisions

Protest based on conflicting requirements

Everyone but winner angry and dissatisfied

5

6World Congress 2013 6

How do we recognize, avoid, or -- as a last resort –

fix these “little things” before they become big problems?

So here’s the

multi-million-dollar

question:


Start by learning from the past …

with a little help from late night TV

Top 10 List:

Ten Key Fixes that Avoid Big Problems in

Procurements and Proposals


(First person to match one on my list gets a

prize!)

But before we get

started…Let me ask you…

What do you do to avoid problems in acquisitions before they occur?


Number 10:

ExampleIn 2011-2012, an in-depth review of the

Joint Tactical Radio System (JTRS) program revealed that inadequate

market research had been conducted as changes in commercial items and

user needs occurred. Upon conduct of market research that revealed user

needs had dramatically changed and non-developmental items were

available to address them, the existing contract was modified to end all remaining development efforts.

- GAO Report, “Defense Acquisitions: Assessments of Selected Weapons Programs,” March 2013

If you don’t do it, what happens?

No understanding of “realm of the possible”No picture of support requirementsUnawareness of commercial productsFailure to bundle for efficiencyLimited understanding of potential risksNo insight on industry standards/benchmarksInadequate number of qualified vendorsUnfamiliarity with small business providersNo visibility into potential incentivesInclusion of unnecessary constraintsFew criteria to evaluate good/not so goodUnreasonably high prices

Conduct market research to enable selection of effective, differentiating

evaluation criteria


Number 10:Conduct market research to enable

selection of effective, differentiating evaluation criteria

Do:

Conduct market research across all acquisition phases

Maximize contact with industry – don’t fall victim to myths!

Keep your mind open – don’t assume you know the answer

Focus research on identifying differentiating characteristics

Target what you need to know – avoid “free form” researchContact the right people from industry – multiple disciplinesIterate – it’s generally not a “one-and-done” approach

Research existing research – don’t start from scratch


Number 9:

ExampleSince 2009, DoD has cancelled three MDA programs due to concerns about technical issues, schedule delays, and

cost-effectiveness and operational roles. A GAO investigation documented

in 2013 revealed that the issues that ultimately caused these cancellations

would most likely have been discovered before the programs began if more

robust Analyses of Alternatives (AoAs) had been conducted as part of

acquisition planning.- GAO Report, “Missile Defense: Opportunity to Refocus on

Strengthening Acquisition Management,” April 2013


Lack of input from some stakeholdersLack of market research to support strategyInability to integrate (logically) inputs receivedInconsistency among requirementsFailure to address security requirementsFailure to consider support needsInadequate or unfounded IGCEDiscovery that funding is unavailableNo use of performance-based techniquesDiscouragement of competition Requirements that don’t yield desired result

Take the time required to develop a comprehensive, integrated

Acquisition Plan (AP)


Number 9:Take the time required to develop

a comprehensive, integrated Acquisition Plan (AP)

Do:

Use iterative market research to make AP outcomes-focused

Consider full realm of needs (e.g., support, security, IT)

Reach out to varied experts to get the IGCE right

Consider criteria early – what separates good from better?

Use cross-disciplinary reviews to check for integration

Maximize up-front, cross-disciplinary stakeholder input to AP

Use the “So what?” test – is the AP going to improve results?

Make senior level sign-off/supervisory approval mandatory


Establish and enforce accountability for the acquisition

ExampleIn 2005, GAO sustained protests from two

companies challenging the Air Force's award of a sole-source contract based on an unusual

and compelling urgency J&A. GAO agreed with the protestors’ claim that the agency's

circumstances did not fulfill the criteria for an unusual and compelling urgency; that the

urgency of the requirements was the result of the agency's inadequate procurement planning;

that the agency unreasonably concluded that the awardee was the only firm capable of

meeting the requirements; and that the agency failed to obtain competition to the maximum

extent practicable.- GAO Protest Decision, Worldwide Language Resources;

SOS International Limited, November 2005


Rushed, “thrown over” requirementsNo ownership of requirements integrationFailure to consider connection to strategyLoss of perspective on foundational needNo consideration of past approaches to needFailure to consider impact on competitionLoss of connection between needs and costsInadequate knowledge base for acquisitionLoss of focus on supportive servicesNo consequences for unsatisfactory inputUnbalanced acquisition that is hard to defend

Number 8:


Do:

Use systems that provide visibility of status (e.g., SharePoint)

Evaluate and provide feedback on quality of input

Employ “color review” approach to build in quality

Define key milestones at which authority must sign off

Number 8:Establish and enforce

accountability for the acquisition

Identify ultimate acquisition authority early

Make sure COR is identified early and involved throughout

Use methods that build stakeholder engagement while cutting cycle time (e.g., Acquisition Acceleration Centers)


Intensify up-front stakeholder engagement to build consensus on

requirementsExample

In 2010, GAO sustained a protest challenging the elimination of company’s proposal from competition for a Federal

Transit Administration PMO contract. GAO agreed with the protest claim that

FTA’s RFP criteria were internally in conflict. Specifically, the RFP did not

require the submission of cost or pricing data, but the protesting firm was

evaluated against, and excluded for, not meeting the requirements of FAR Section

15.408, Table 15-2, which only apply when cost or pricing data are required. - GAO Protest Decision, PMO Partnership Joint Venture,

October 2010


Inconsistent internal view of requirement No ID of “must haves” versus “nice-to-haves”Many inconsistencies/contradictions in termsLimited understanding of overall timeline Delays caused by over-the-transom approachUnbalanced view based on involvement levelFailure to consider novel approachesLoss of focus on supportive servicesMany difficult questions after RFP releaseInconsistent proposalsFailure to meet objectives with acquisition

Number 7:


Do:

Identify approaches that are collaborative versus sequential

Use performance-based methods to focus on outcomes

Use tools that improve visibility of status (e.g., SharePoint)

Consider every discipline that will be involved in the program

Identify key stakeholders early and bring them together

Identify Authoritative, Responsible, Consulted, and Informed (ARCI) stakeholders by task

Number 7:Intensify up-front stakeholder engagement

to build consensus on requirements

Establish an acquisition schedule that promotes collaboration

Integrate efforts through the ultimate acquisition authority


Number 6:

ExampleIn 2013, DOE issued a Policy Flash that identified the importance of DRFPs in

helping match the proposal and selection process to the requirement. Specifically, DOE noted that using DRFPs to guide

selection of the proposal type can lead to significant cost savings and productivity

enhancements; reduced proposal preparation and evaluation time; reduced

need for solicitation amendments that disrupt timely completion; and better

proposals, end products, and services. -DOE Policy Flash 2013-45,

April 2013


Too few bidders based on overly limiting reqsUnqualified bidders based on overly loose reqsToo time-consuming considering needUnnecessarily paper-intensiveToo lengthy considering scope of requirementMisalignment between prop & execution needsNo ability to meet key staff during bid processInappropriate re-use of existing materialsInterpretation that “it’s locked” for someoneFailure to consider oral presentation approachInability to select the best possible solution

Select a proposal type and process that aligns with overarching

objectives


Do:

Promote electronic submittal to greatest extent possible

For small bids, keep it simple (complexity drives out bidders)

Carefully review re-used artifacts – are they appropriate?

Use market research to gain specific RFP recommendations

Relate the proposal requirements to the work requirements

Put DRFPs out early; include as much content as possible

Number 6:Select a proposal type and process

that meets overarching objectives

Consider schedule in selecting response methods (e.g., orals) Directly ask for specific RFP recommendations from industry


Number 5:

ExampleIn 2009, GAO sustained a protest claiming

that an RFP from the Army Medical Research Acquisition Activity unreasonably required certification of proposed equipment by the Joint Interoperability Test Command at the time of proposal submittal. “The Army has simply not explained why the apparent

purpose behind the certification requirement -- to ensure that a system be certified prior to

the time it must be fielded -- requires that evidence of certification be provided as early

in the process as the time at which quotations are submitted.”

-GAO Protest Decision, SMARTnet ,Inc., January 2009


Many questions based on evaluation criteriaIndustry self-elimination for the wrong reasonsInappropriate focus on less significant aspectsUneven, difficult-to-evaluate proposalsUnintended conflicts with other RFP elementsMisalignment with Acquisition PlanLoss of confidence in fair and open processLoss of focus on overarching objectivesPotential protest before proposal submittalPoor props that inhibit selection of best option

Use critical thinking skills when selecting evaluation criteria


Do:

Consider use of sample tasks to promote realism

Provide adequate time for offerors to meet expectations

Think about how you would choose if it was your money

Limit the number of criteria to focus on key discriminators

Make criteria specific to needs/objectives (limit re-use!)

Define to greatest extent possible what makes an offer great

Define key expectations (schedules, processes, templates) Make sure proposal instructions align with selected criteria

Number 5:Use critical thinking skills when selecting evaluation criteria


Number 4:

ExampleIn 2012, GAO partially sustained a protest

claiming that an RFP from the Defense Intelligence Agency did not clearly state that use of uncleared personnel by the awardee

demonstrated a lack of requirements understanding and should have been

considered in a price realism evaluation. GAO also upheld the protestor’s claim that a cost realism evaluation must be conducted if it is defined in the evaluation criteria, even if the evaluation criteria do not specifically state by name that a cost realism evaluation will be

conducted. -GAO Protest Decision, Science Applications International

Corporation, November 2012


Annoyed, contentious offerorsPerception that ambiguity is deliberateHigh potential for unintended conflictsMultiple questions focused on gaining clarityDifficulty in tracking back to AP for answersMany different but reasonable interpretationsUneven, difficult-to-evaluate proposalsIncreased need for lengthy discussionsDelays in completing acquisition processHigh potential for successful protests

Challenge ambiguity in all aspects of process


Do:

Assign at least one person to search for ambiguity issues

Use a color team type approach to build in clarity in phases

Document your thinking so it is defensible downstream

If you find ambiguity, enlist fresh perspective to help fix it

Remember: if it seems ambiguous to you, it is ambiguous

Use industry to identify ambiguities and propose solutions

Use an iterative approach that considers holistic alignment Write short, direct statements – less is often more (clear)

Number 4:Challenge ambiguity in all aspects of

process


Number 3:

ExampleIn 2011, the U.S. Court of Federal Claims determined that GSA’s inclusion of a 15-

year fixed-pricing schedule violated customary commercial practice and

therefore also violated FAR 12.301(a)(2). The court found that GSA’s market

research failed to show that the solicitation requirement for providing fixed prices at contract outset was

consistent with commercial practice, and it therefore found the pricing requirement

to be in violation of FAR 12.301(a)(2). -COFC Decision, CW Government Travel v. the United

States, August 2011


Use of outdated guidance/regulatory artifactsConflicts among accurate/inaccurate clausesImproper application of clausesMultiple questions from annoyed biddersLoss of bidder confidence in processAutomatic necessity for RFP revisionNeed (often) for re-review of entire solicitation Delays in completing acquisition process“Technicalities” with potential to derail processHigh potential for successful protests

Review all clauses for currency,

accuracy, and applicability


Do:

Schedule a final clause review immediately before release

Apply tools and references that focus on clause updates

Use automation for generation, but review manually

Provide resources/training on clause changes/updates

Make an individual responsible for clause accuracy

Document identified issues/inconsistencies and resolutions

Number 3:Review all clauses for currency,

accuracy, and applicability

Conduct research and seek guidance if you identify an issue


Number 2:

ExampleIn 2009, GAO sustained a protest claiming

that TRICARE failed to apply its evaluation criteria properly in the areas of

technical approach, past performance, and cost realism in selecting the initial

awardee for a contract. In its decision, GAO noted that the instructions and evaluation

criteria were complex and could be interpreted differently. The initial awardee followed all RFP instructions, but GAO held that TRICARE failed

to apply its evaluation criteria appropriately. GAO upheld the protest.

-GAO Decision, Health Net Federal Services, LLC, November 2009


Confusion over what to cover/to what extentVaried proposals that are hard to evaluateHigh potential for conflictsMultiple questions from annoyed bidders“Everything is equally important” mentalityLoss of focus on key acquisition objectivesDelays in the proposal development timelineChallenging evaluation processConfusing, overly complicated proposals

Make Section L (Instructions) and Section M (Evaluation Criteria) the same


Do:

Consider pass/fail for some elements of the proposal

Match what you will evaluate with what you need in proposals

Write Sections L and M concurrently with SOO/SOW

Limit the information required to what will be evaluated

Use past RFPs as references – What worked? What didn’t?

Complete evaluation plan before RFP release; map L/M to it

Consider offerors’ reactions to differing Sections L and M

Number 2:Make Section L (Instructions) and Section M

(Evaluation Criteria) the same

If Sections L and M differ, document your rationale


Number 1:

ExampleIn 2007, GAO sustained a protest claiming that the Army improperly

evaluated proposals with regard to each of the solicitation’s evaluation factors (fill

rate, experience, and transition). Specifically, the evaluations of offerors’

responses to the evaluation factors could not be aligned to specific criteria in the

evaluation factors. It was therefore determined that there was inadequate support for the Army’s source selection

determination. -GAO Decision, L-3 Communications Titan Corporation,

March 2007


No “red-threading” of changes to solicitationNo “fresh eyes” to find errors/inconsistenciesMissed errors/inconsistenciesMisalignment with AP and Evaluation Plan Disappointment from requirement initiator“Amend it later” mentalityDelays from questions/amendmentsConfusion caused by multiple changesIncreased potential for successful protests

Allow sufficient time to conduct iterative reviews and to respond to flaws

they disclose


Do:

Consider process changes and training based on reviews

Consider and correct ambiguities – even if they’re not “wrong”

Establish a timeline that supports an iterative review cycle

Remember: it is always better to fix it when you find it

Use tools to promote rapid, effective, timely collaboration

Create teams that combine historic and fresh perspectives

Document, document, document – maintain your file

Number 1:Allow sufficient time to conduct iterative

reviews and respond to flaws they disclose

Use review results as learning opportunities

Little things can lead to big successes

29World Congress 2013

Government Industry

Market research identifies key criteria

Schedule/Acquisition Plan/DRFP reflect input

Unambiguous, direct RFP includes best ideas

Fewer amendments keep timeline intact

Winner selected based on clear eval plan

Debriefs clearly explain logical rationale

Feedback sharpens focus on discriminators

DRFP yields good suggestions from industry

Well-prepared offerors have fewer questions

Offerors submit clear, consistent proposals

Program awarded! Protests avoided!

Offerors receive notification/request debriefs

29


Success begins with the

basicsSignificant problems are generally avoidable or

resolvable by focusing on simple-to-implement

acquisition best practices


Questions?

a top ten you can use: ten key fixes that avoid big problems in procurements and proposals breakout...

Documents