California Department of Technology

Market Research Guidelines

Version # 2

Mar

ket R

esea

rch

Guid

elin

es

TABLE OF CONTENTS

Table of Contents

TABLE OF CONTENTS .......................................................................................................... 2 DOCUMENT HISTORY .......................................................................................................... 3 INTRODUCTION AND PURPOSE............................................................................................ 4 WHAT IS MARKET RESEARCH? .......................................................................................... 6 WHY DO MARKET RESEARCH? .......................................................................................... 7 WHO SHOULD BE INVOLVED IN MARKET RESEARCH? ......................................................

............................................................................. ...............................................................................

................................................... ......................................................

..........................................................

8 WHEN IS MARKET RESEARCH DONE? 10 HOW IS MARKET RESEARCH DONE? 11 WHERE IS MARKET RESEARCH DONE - RESOURCES? 23 FURTHER CONSIDERATIONS OF MARKET RESEARCH 27 WRAPPING UP WITH CHECK LISTS & NEXT STEPS 29

2

DOCUMENT HISTORY

Revision Number

Revision Date Summary of Changes Author

1 06/30/15 Published Guideline Department of Technology

2 10/10/18 Revised to update web links Department of Technology

Mar

ket R

esea

rch

Guid

elin

es

3

Mar

ket R

esea

rch

Guid

elin

es

INTRODUCTION AND PURPOSE

Market research is an organized effort to gather information about target markets or potential vendors. It is a very important component of business strategy. Market research provides essential information to identify and analyze the market need, market size, new, and existing competition. Through Market research information one can know the prices of different commodities and services in the market, as well as the supply and demand situation.

The results of Market research must be substantial, credible, current, and supportable and aligned with the project’s business, technical and functional objectives. Many Agencies/state entities make the mistake early on of thinking that they know their solution before conducting the appropriate level of research which often results in some very expensive mistakes later on.

Market research is a vital means of arming an IT Project Team with the expertise needed to conduct an effective procurement. This type of information helps determine the suitability of the marketplace for satisfying a need or requirement. Market research is the continuous process of collecting information to maximize reliance on the commercial marketplace and to benefit from its capabilities, technologies, and competitive forces in meeting an Agency/state entity need.

Market research is an essential process enabling the State to buy best-value products, services and solutions that solve mission-critical objectives. When little or no knowledge exists for the desired IT product, service, solution or available supplier resources, the Market research can help identify:

New, existing or innovative products, services or solutions available in the marketplace that address the business problem or meet similar requirements as well as how others have procured similar solutions

Accurate cost estimates (rough order of magnitude (ROM)) and/or realistic delivery schedules

Industry practices such as service level agreements (SLA’s), warranty standards, and installation procedures

Product distribution and maintenance support capabilities of potential suppliers Availability and status of potential supplier sources (i.e. Small Business Vendors) Competitive prices that have been obtained by other customers when acquiring

products, services or solutions Refinement of business technical and/or functional requirements based on industry

standards and best practices Provide an awareness of the market vocabulary, industry trends, standards, service

and product concepts, and competitive business trends

4

INTRODUCTION AND PURPOSE

The Department of Technology developed Market Research Guidelines as a tool to support Agencies/state entities in the basics of conducting and vetting of industry research and input.

It aides in the “how” to gather research surrounding the IT project needs, including but not limited to; business technical and functional requirements, industry trends, risk factors, resourcing, schedules, and budgetary information. These Guidelines will assist in identifying research tactics and serve as a basis to create more robust market research results and ultimately collect more reliable data.

Adherence to the Department of Technology’s Market Research Guidelines is not required to complete your Stage 2 Alternative Analysis; however, the Department of Technology does require Market research to be conducted by Agencies/state entities to determine the suitability of the marketplace for satisfying an IT project’s business, technical and functional needs and/or requirements. In addition, recent Market research will aide in ensuring the alignment of the recommended solution with current and feasible industry offerings.

The results of an Agency/state entity’s Market research is required to be submitted for Department of Technology’s assessment and review before an Agency/state entity may move into to Stage 3 Solution Development.

These Guidelines are not intended to take precedence over any applicable State or Federal Statues, Rules, or Regulations. For IT Reportable projects contact IT Project Oversight Division (ITPOD) and Statewide Technology Procurement Division (STPD) when initializing Market research. Special care should be taken when the Market research is complex, has high public visibility, proprietary, confidential in nature, or if the Agency/state entity has any questions on research activities that could potentially put the IT project at risk.

Mar

ket R

esea

rch

Guid

elin

es

5

Mar

ket R

esea

rch

Guid

elin

es

WHAT IS MARKET RESEARCH? Market research is the process of collecting valuable information to determine if there is a market for your proposed product, solution, or service. The information gathered from Market research will assist the Agency/state entity to make wise and profitable business recommendations and ultimately decisions.

Market research is process for gathering data on a product or solution characteristics, the suppliers’ capabilities, any professional service offerings, and any business practices that surround the market.

Market research requires one to collect and analyze information about the market that subsequently can be used to determine whether the Agency/state entities need can be met by products or services available in the current marketplace; whether commercial practices regarding customizing, modifying products or tailoring services are available to meet customer needs; what are the industry practices, including standard warranty, maintenance and operations and pricing models; and whether the distribution and logistics support capabilities of potential suppliers are sufficient to meet the needs of the chosen solution.

Market research information is used to shape the procurement strategy, recommended solution, to identify and determine the type and content of the product description or statement of work, to develop the maintenance support strategy, to refine the requirements, and to assist in the development of the evaluation factors used for the procurement.

Market research may make the difference between the right and wrong decisions that affect project IT solutions. It may reveal unfilled needs, suggest different strategies or identify a product or services strengths and weaknesses. Market research could involve asking questions, researching, surveying the market with Requests for Information (RFI) questionnaires, recording information and taking time to learn from the information. Market research develops an information base to estimate budget, develop solution strategy and decide how to approach a recommended solution.

Market research is the process of analyzing data to help you understand which products and services are available, and if applicable how they are competitive. Market research will provide valuable insight to help project’s reduce business risks, spot potential current and upcoming problems in the industry and identify all opportunities.

Additionally, Market research is an important strategy tool to assist in identifying the Vendors available such as Small Business and Disabled Veteran Business Enterprise (DVBE) vendors. Small businesses and DVBE participation is important to promoting competition in the procurement process and growing the overall economy for the State of California.

6

WHY DO MARKET RESEARCH?

Market research is essential to optimize the use of commercial items, commercial services and to meet the Agency/state entity needs. The benefits of a competitive, supplier base can not only reduce costs, but can also reduce the acquisition cycle times. Access to the most current trends in the commercial marketplace will also provide access to the latest advances in technology.

Knowing your market also means understanding external factors that could affect your success. Standards emerge, regulations change, economic conditions shift, and technological breakthroughs occur. Today's assumptions may not apply tomorrow. Conducting market research early - and often - will help you stay current with the industry. Information obtained through market research will help you to focus your product or service development efforts, reduce risk and define project data needs to improve your procurement and project success.

Market research provides the basis for:

Collecting relevant data to support industry capabilities, product availability, competitive market forces, and alternative sources

Availability and status of potential supplier resources Supporting effective procurement planning Determining if the IT project requirements can/should be modified, or enhanced Defining and refining of new requirements Components such as Scope, Schedule and Budget Recommendations for the components to include in the Statement of Work (SOW) Provide information on specific risks or pitfalls specific to the solution Identifies unanticipated work, services or products to be included Provides historical date/Lessons Learned

Failure to conduct market research may have significant consequences on your procurement, solution development, and resulting contracts. For example, if you find your solution is only available by one or a limited pool of suppliers and you will have to procure via a non-competitive or conduct a procurement that limits competition. If this is the scenario, then the Market research conducted must be compelling and plays a very important supporting factor, it must be valid and well documented; if Market research is not valid or incorrect the procurement may be subject to protests. This could result in delays, as well as re-procurement and additional project costs. Additionally, new commercial market technology, capabilities, processes, or resources could be overlooked if Market research is not conducted. This is why the Market research is such a critical factor and must not be overlooked or done without a diligent effort. It creates signpost and guides to the roadmap of success for an IT project!

Mar

ket R

esea

rch

Guid

elin

es

7

Mar

ket R

esea

rch

Guid

elin

es

8

WHO SHOULD BE INVOLVED IN MARKET RESEARCH?

The State of California does not have a specific group of people that conduct Market research; instead, an Agency/state entity may rely upon their IT Project Team to perform Market research related activities, the Agency/state entity procurement official or contracting office, and/ or will request assistance from ITPOD and STPD. The level of knowledge performing Market research within in the Agency/state entity may vary, depending on the Agency/state entity’s staffing levels and experience an IT Project Team research effort that includes procurement official expertise is the best approach. Many times the business, functional and technical requirements need to be researched and the expertise of Subject Matter Experts (SMEs) to interpret and apply the results of the research information gathered is vital to the resulting analysis.

For successful research that leads to successful procurements, consider the following questions when identifying who should have input to the market research:

Who are the stakeholders (Counties, Local Agencies, other Agencies/state entities)? Who would know what information is needed to make the decision to buy from the

commercial market? To prepare the SOW you need to know the performance characteristics upon which product, solutions, and services are valued and distinguished from one another.

Who would know of the product/services need, follow-on support, or maintenance and operations?

Who Is testing or tracking success metrics for the product, services, or solution? Who would know the business practices and standards? What practices are standard for the insertion of new technology?

The Market research results at minimum, must involve an IT Project Team effort in the resulting analysis because of the many business and technical aspects and dependencies involved. The Agency/state entity’s research IT Project Team may be composed of the following:

Procurement Analyst/Official or Contracting Office Executive management, CIO/AIO/OIS Technical Staff/Technical Leads Business/Functional/Project Management Staff Stake Holders/End Users/Users/Critical partners (may include outside agencies) Subject Matter Experts (SME’s) Cost Analysts/Rough Order of Magnitude (ROM)/Budgetary staff Legal Representatives

California Department of Technology roles that may participate in some capacity. Statewide Technology Procurement Division (STPD) Information Technology Project Oversight Division (ITPOD) Office of Information Security (OIS) Consulting and Planning (C&P) OTech

WHO SHOULD BE INVOLVED IN MARKET RESEARCH? Examples of the Knowledge, Skills, and Abilities (KSA’s) necessary to perform Market Research include:

Knowledge of: Market research activities and sources of collecting data on business trends, IT

technology cycles, and forecasting Fundamentals of supply and demand IT industry information systems and similar current applications in production for IT

the area being researched Using on-line information services and skills at performing internet searches such Identifying financial information on IT markets and individual firms.ie. Dunn and

Bradstreet, Moodys, Standard and Poors, etc. Identifying similar or like IT business goals and strategies, the market environments,

market measurement, market segments and product differentiation, product life cycles and market evolution, market prices, market channels/middlemen, physical distribution, industrial markets, and source selection factors and procedures in industrial and public markets

Skills and Abilities to: Identify market data relevant to State IT business decisions — “collecting and

analyzing information about capabilities within the market to meet Agency/state business entity needs or opportunity”

Communication effectively Perform Internet/Intranet research Perform Library research Perform Literature research Perform Classification and catalog of data Assign importance and relevancy of data Collate disparate pieces of information Discern fact from fiction Identify high public visibility concerns and policy implications Maintain a focus of purpose and objectivity

Market research is typically performed based on a need to gain intelligence on a key business problem (or opportunity) that needs to be acted upon, but there is a lack of information (can be both technical or non-technical) to make that decision securely; the job of a Market researcher is most commonly to inform decision makers by providing solid data, and allowing f or a recommended s olution to come forward that has current industry Market research to support it. The objective of the Market researcher is to maintain a level playing f ield when conducting research and to avoid the possibility of partiality or unfairness.

Mar

ket R

esea

rch

Guid

elin

es

9

Mar

ket R

esea

rch

Guid

elin

es

10

WHEN IS MARKET RESEARCH DONE? Market research should be viewed as a continuous learning process in any Agency/state entity. Knowing what the environment has and keeping abreast of technology is always the recommended approach. However, Market research is needed for IT Reportable projects early in the IT Project Approval Lifecycle (Pal) and typically leads up to the Stage 3 Solution Development phase. The level of specificity and scope varies at different points, but Market research, done early in the PAL process provides a plethora of information that can be used to shape the procurement strategy, support implementation, create metric for test plans, development of SOW requirements and deliverables, solicitation evaluation factors, contract conditions, and performance metrics.

The first step in conducting Market research is communicating the core business need or opportunity that is being researched and the application or solution requirements, if known, to the Vendor and IT industry. The information must reach the IT industry early in the process for many reasons:

It allows vendors to identify potential a “commercial off-the-shelf” (COTS) available products or services that can meet the requirements and business needs. It may also identify, if suppliers can meet the requirement by modifying their products ,as a “modified off-the-shelf” (MOTS).

It provides the upfront analysis of flexibility in the industry solutions (i.e. to what degree cans a product me modified? Do vendors have to modify source code or customize it?). This may lead a project to modify their technical or business requirements to allow for the flexibility needed.

Early communication to the IT industry of the solution requirements saves time in correcting or revising requirements later in the PAL process. This is especially noticed in the procurement stages as fewer addendums will be required and far less vendor questions. Earlier research allows the IT industry to better prepare responses during subsequent procurement phases, if applicable. It is important to communicate the requirements early so that the industry can easily understand the need and provide to the Agency/state entities the best and most current information.

Provides the IT industry information to provide suggestions to aide in shaping the procurement strategy, support and test plans development, product description, SOW components, and assist in the development of the solicitation evaluation or even the post contract management factors.

Market research should be an ongoing process and can be performed continuously to assist Agency/state entities to keep abreast of technology trends, products and services. Specifically a time when Market research data analysis is required is in the Department of Technology Project Approval Lifecycle (PAL). In PAL Stage 2 Alternative Analysis (S2AA) asks for a more comprehensive research and is conducted in response to a specific IT business solution needs, opportunity, product and/or services. Mainly, early exchanges of information prior to procurement are the discipline that should be followed.

HOW IS MARKET RESEARCH DONE?

The Agency/state entity has many possibilities on how Market Research is developed and conducted. For example; if using written questionnaires/surveys it might be best to develop questions about how the agency would approach an identified need. The Agency/state entity researcher will want to contact the business and technical partners or SME’s involved for assistance in developing the questionnaires. Market Research should not have to settle on one methodology either. The Agency/state entity may utilize any of (or a combination of) the following example methods to conduct Market research:

Acquire technical and or business analysis publications information about products, trends, product availability, business practices, product/service reliability and prices.

Contact via phone, email, or physically knowledgeable individuals in State or other government and industry regarding market capabilities to meet requirements.

Conduct unbiased industry briefings/ pre-procurement discussions via phone, email, or physically with potential suppliers to discuss solution needs. Ask about the purchase history of same or like solutions to collect data to help determine the level of competition, prices and performance results. Be sure to seek advice with the Department of Technology when conducting industry briefings for an IT project that is proprietary, complex, or publicly sensitive.

Contact via phone, email, or physically the internal Procurement or Contracting office to discuss publishing and processing a RFI to survey the market. Remember the “Who should be involved” section above!

How can the general Market research steps be conducted?

Step 1. Define the Objective & the “Business Problem”

Perhaps the most important step in the Market research process is defining the business problem, opportunity, and objectives of the project. The core of defining the objective(s), opportunity or problem(s) is to ask questions that will achieve quality Market research outcomes. The questions developed in the questionnaires/surveys are imperative to the success. Examples of “business problem(s) questions developed might be “How should we budget for this new widget?” or “Which features should we prioritize?” “What are the risk factors involved”.

By allowing the Industry to understand the objective(s)clearly, it will keep the research focused and effective. The best technique is well before any research has been conducted; imagine what a “perfect” final research report would look like to help answer the business objectives(s). Consider preparing a mock-up report, with hypothetical data. Is there enough information to make an informed choice?” If yes, proceed with getting the real data. If no, continue working with the technical and business SME’s until the objective is clear.

Mar

ket R

esea

rch

Guid

elin

es

11

Mar

ket R

esea

rch

Guid

elin

es

12

HOW IS MARKET RESEARCH DONE?

Step 2. Determine the “Research Method”

Now that the research objective(s) is known, plan out the type of research deliverable that will best obtain the necessary data. Think of the “research method” as the detailed plan of attack. In this step first determine the Market research method (survey, focus group, etc.). Think through specifics about how to identify and choose a sample (who to go after? Where to find them? etc.). Keep the end goal in mind (final outcome). Identify the types of data needed to conduct the analysis which will dictate the structure of questions to ask. This is also the time to plan where to conduct research (telephone, in-person, mail, internet, etc.). The research methodologies will be based on the nature of the data collected.

Research methodologies can further define itself by its classification type, This is a means to segment the data collected. For example an Agency/state entity can differentiate the types of different research by classification. Below are two types of classifications:

Exploratory Research – This form of research is used when the topic is not well defined or understood, the hypothesis is not well defined, and/or the knowledge of a topic is vague. Exploratory research will help gain broad insights, help narrow the focus, and learn the basics necessary to go deeper. Common exploratory market research techniques include focus groups, interviews and demonstrations. An RFI at this level should contain high level details questions and/or surveys.

Descriptive or Qualitative Research – For research requiring detailed data on a specific topic, conducts quantitative descriptive research. The goal of this form of market research is to measure specific topics of interest, usually in a quantitative way. Surveys are the most common research instrument for descriptive research. There are also mini focus groups or focused vendor industry briefings/ pre-procurement discussions. An RFI at this level should contain detailed level questions and/or possible sampling of requirements.

Step 3. Design & Prepare the “Research Deliverable”

If using an RFI as the deliverable it may contains a survey or questionnaire, begin by writing and forming the questions with the IT Project Team input. Include the business and technical requirement needs. If a focus group is the preferred choice to start the research development, start preparing questions and materials for the moderator. Typical drab or piles of written market research report deliverables for your project may fail to connect with stakeholders or inspire executives to take action. Sharing research RFI’s or questionnaires via hands-on workshops or meetings can be a solution to better engage stakeholders. If complex research is needed, be sure to test the survey instrument or RFI with a small group prior to broad deployment. Take the sample data or mock a response to questions and add to a spreadsheet, verify there are no issues with the data being provide or how its structured.

HOW IS MARKET RESEARCH DONE? Prepare the deliverable with the end in mind. Let the questions drive the response and if not applicable, do not be prescriptive in the questions. All RFI’s must provide a fair and equitable level of surveying.

Do not prepare a RFI questionnaire of survey that acts in the same format of a solicitation document. Always seek to express to the industry that an RFI deliverable is not a procurement/solicitation vehicle.

Step 4. Administer and Collect Your Data

Begin administering the RFI, survey, etc. running the focus groups, conducting the interviews, holding pre-bid conferences, demonstrations etc. Collect and thoroughly record the answers, choices, and observations in spreadsheet format or create a way to organize your data. Collection of data can take many forms (spreadsheet, charts, etc.), most importantly the way it is collected should be meaningful and should be customized to provide, stakeholders, IT project teams, business and technical staff, end users, SME’s and executives clear and unbiased data that leads to sound decision making.

Step 5. Analyze Your Data

Analyze the outcome and read through all the data. Compile the data in a clean organized format. Run summaries and scenarios with the information by building spreadsheets, tables and graphs, and segment the results by groups or needs. If ratings and rankings are used for building recommendations, consider computing a mean, or average. Ensure that all IT project participants involved have an understanding of the feedback and have sufficient input to analysis, summary or the outcome of results. Provide participants the chance to carefully review and discuss the findings.

Step 6. Document the Data and Communicate Results

Compile the data into a report or presentation. Start with the research objectives and business problem that were identified in Step 1. Restate the business questions, and present recommendations based on the data, to address those issues. When presenting the results, present insights, answers and recommendations, and not just charts and tables.

Mar

ket R

esea

rch

Guid

elin

es

13

Mar

ket R

esea

rch

Guid

elin

es

14

HOW IS MARKET RESEARCH DONE?

When completing the steps to conduct Market Research, be aware this requires time. Many Agency/state entities are often strapped for time and budget and may take shortcuts that can later backfire. See common pitfalls below;

Using only secondary research. Relying on the published work of others doesn't give the full picture. It can be a great place to start, of course, but the information you get from secondary research can be outdated. It can miss out on other factors relevant to your business.

Using only web resources. When you use common search engines to gather information, you get only data that are available to everyone and it may not be accurate. Surveying only the products you know. Agency/state entities sometimes interview only those commonly used or historically used products or services when conducting research. To get the most useful and accurate information, you need to talk to real customers about their needs, wants and expectations.

Market research questions, information to include, or activities to consider:

Availability of commercial off-the-Shelf, Modified-off-the-Shelf, or custom Software solutions

- Salient characteristics and past performance - Market shares and niches/corporate strategies - Price/feature tradeoffs - Support services - Product reliability and history - Typical customizing, modifying or tailoring for customers - Potential cost of modifying the item to meet particular needs - Industry practices and trends - Industry specifications and industry standards - Industry terms and conditions (e.g., discounts, warranties, financing, tier

pricing) - Laws and regulations unique to the item/service - Production and delivery lead-times - Technological/product changes and forecasts - Trends in market prices - Trends in supply/demand - Factors that affect market prices (e.g., cost and demand of product, raw

materials prices) - Status of availability (e.g., small business, DVBE, other) - Production capability - Distribution and support capabilities - Technical strengths and weaknesses - Business and organizational strengths and weaknesses - Patent and data rights

HOW IS MARKET RESEARCH DONE?

Market research questions, information to include, or activities to consider (Continued):

Practices and trends in State procurements of the same or similar requirements - Other State buyers - Trends in State demand - Trends in prices paid by the Government Agencies-Problems and issues in

the award and administration of prior similar contracts Review solicitation histories on current or prior project/contract files for the same or

similar requirements - Prior project Files/Contract files - Contacts with other buyers (State and out of state) who have experience in

buying the supply or service Determine the scope and extent of research

- Review information already in hand (including your personal knowledge of the market from prior contract actions, information supplied by the requiring activity, and the findings of recent research on like requirements)

Plan the collection of market information (i.e., when and how) Consult the appropriate State Oversight/Procurement/Contracts teams about roles

and responsibilities for conducting market research Ensure the Market research questions developed are reviewed and accepted by the

IT project teams, users, executive management, legal (if applicable) oversight authorities and Procurement/Contracts teams

Ensure the planning of the Market research and its evaluation activities are in place prior to gathering the data. Actively document solutions to problems, weighing the results and consequences of each solution. Evaluations typically will be simple, informal, efficient, practical, low- cost and easily carried out and understood by all

- Evaluations won't just be delegated to one person – highly encourage everyone in the team become involved in evaluating and reviewing the results

- Evaluations should be honest, and place emphasis on feasibility and credibility

Be sure to select a method or multiple methods for researching information. Research methods may include:

- Industry panels - Request for Informal Market Survey/Research for Planning and Budgetary

Purposes (written) - Questionnaires/surveys - Pre-solicitation for Industry Comments - Pre-bid conferences - One-on-One Meetings - Presentations/Demonstrations - Forums - Forming of panels with members from State and Vending community

Mar

ket R

esea

rch

Guid

elin

es

15

Mar

ket R

esea

rch

Guid

elin

es

16

HOW IS MARKET RESEARCH DONE

Initiate the selected Research methods with fairness

For example, Market research could include one-on-one meetings with multiple vendors who provide relevant solutions. Having meetings with the industry, will minimize the perception that procurements are being steered toward a particular vendor. Moreover, by conducting one-on-one meetings prior to solicitation, the State can benefit from greater insight into vendor solutions — insight that vendors are hesitant to provide in an RFI, which is a public document. Market research should also serve as a key input into the State entity’s assessment of alternative solutions and approaches. Be sure to seek advice with the Department of Technology when conducting industry briefings for an IT project that is proprietary, complex, or publicly sensitive. Example conference activities include;

- Open conference and brief participants on the proposed requirement - Answer questions to aide in the industry understanding of State needs - Obtain verbal feedback on SOW or technical requirements - Identify interest in submitting response to future bids or phased procurements - Provide information (location, time, map, contact info) to bidders for potential

site visits - Identify conditions under the existing contract that may impact new contract

(where applicable)

Building an RFI document for Planning and Budgetary Purposes and/or to Survey the Industry

Developing and conducting an RFI is a typical format to collect written information about the capabilities of various suppliers to provide identified IT solutions, services or products.

The RFI acts as an inquiry to the marketplace to determine which suppliers for IT products, services, or solutions are available in the market to solve an Agencies/state entities’ IT business problem. The information obtained through an RFI may be used in developing a subsequent Request for Offer (RFO) Invitation for Bid (IFB) or Request for Proposal (RFP) after determining that IT suppliers or solutions are available to satisfy the IT technical or business problem.

An RFI is not a procurement method and the results of an RFI cannot be made into a contract. Responses to an RFI will assist the Agency/state entity in determining an appropriate course of action that may or may not involve a new procurement to solve their IT needs. An RFI can be best described as an effort to seek ideas, perspectives and information on the proposed IT procurement from potential suppliers so that a formal project scope and set of requirements may be developed.

HOW IS MARKET RESEARCH DONE

An lengthy and detailed RFI is typically used when robust or specific knowledge is needed regarding the availability of unique or custom set of IT suppliers, IT solutions, IT approaches, services or products are being sought and little is known about them in the general market or IT industry. Below are a few of the examples of specific instances where the use of an RFI can be helpful:

When an Agency/state entity lacks in-house expertise or resources qualified to determine what type of solution may achieve its IT project’s business goals and/or what types of solutions are available in the marketplace.

When supplier responses to an RFI will assist an Agency/state entities in developing future solution requirements. Suppliers may propose multiple solutions in response to an RFI inquiry and need not be limited to proposing one solution. Each solution proposed may be an option for consideration and may help an Agency/state entity shape their requirements.

When vendors can educate an Agency/state entity on available options that will help mitigate risks and/or reduce overall technical needs.

Examples of RFI Topics

Remember each project’s business objectives are unique and the resulting RFI should be aligned with those unique needs, the following are general examples to follow in preparing a successful RFI. These examples are not intended to be a complete collection of topics to be covered in a RFI or survey nor are they expected to be used in every survey.

These examples are not intended to be a complete collection of topics to be covered in a RFI or survey nor are they expected to be used in every survey. Each survey should be tailored to the particular needs of the IT Project the information already available to the State.

Generally these examples below apply to both suppliers of products, suppliers, and services. However, in some cases they may apply only to products or only to services. Information available through other sources(tradeshows, forums) and not the RFI should generally be retrieved through those sources and then combined in the analysis in order to keep the size and organization of the RFI consistent.

As a general rule the more succinct the RFI or survey, the faster the response rate. Some of this information you may already know as a result of your on-going Market research. While each IT project’s business objectives are unique and the resulting RFI should be aligned with those unique needs, there are general recommended examples to follow in preparing a successful RFI:

Be concise and include a clear statement of the problem for which a solution may be solicited in the future

Request that suppliers respond to questions concerning the particular IT topic, business need or solution being considered

Inform vendor that they bear all costs in providing an RFI response and that there is no guarantee that a resulting contract will be issued

Mar

ket R

esea

rch

Guid

elin

es

17

HOW IS MARKET RESEARCH DONE?

Mar

ket R

esea

rch

Guid

elin

es

18

Inform vendors that the information provided is public and the State has the right to use it in any future solicitations, without stating which vendor it came from and by using it, the state has no liability in any way to the vendor who provided it. Ask suppliers to provide information on their qualifications, experience and ability to solve the IT problem posed by the RFI

Ask vendors to provide lessons learned white papers or other credible data regarding the solution being considered to obtain sound information, facts and knowledge-share

Do not use the RFI to select an IT vendor or a solution Any subsequent IFB or RFP should not be written to a particular vendor, product,

service or solution discovered during the RFI process The State may ask vendors for a ROM or cost driver estimates for budgetary and

planning purposes

Information to obtain from Industry on performance, specifications and product Information

Product data sheets Availability of product samples/length of time the product has been produced or

service provided Purchase item descriptions used by other State activities or used in commercial

transactions, including commercial specifications standards and statements of work

Product quality, reliability, and maintainability experience of similar users List of products and company services satisfying identical or similar service

requirements. Applicable regulatory and de facto standards Average time between model changes and practice of providing continued parts

inventories, upgrades, or production for phased-out models Supplier plans for handling upgrades and obsolescence

Information to obtain for Supplier capability Number of suppliers Size and location of suppliers and their current market Product distribution channels Business practices in sales and distribution from manufacturers to wholesaler,

distributor or Value Added Distributor or Master Value Added Reseller, Value Added Reseller, reseller or retailer, to user

Production capacity to meet requirements as part of commercial sales Packaging, handling, storage, and transportation practices

Information to obtain for Market acceptance criteria Annual sales Anticipated future orders

HOW IS MARKET RESEARCH DONE?

Description of supplier/contractor’s quality controls including extent of statistical process controls and any ISO or other quality certifications

Warranty terms and practices, annual returns under warranty Need for any pre-production or production qualification testing and special quality

assurance requirements Product evaluation criteria (including life-cycle criteria, if applicable)

Information to obtain for System Integration (SI) professional services

How are technology vendors adapting their services offerings to capitalize on the trend toward the solution be provided?

What is the scope of the SI market, and what are the primary drivers that affect its growth and development?

How is SI project delivery changing, and what impact will this have on skill sets going forward?

Which are the leading SI vendors, and what is their market position? Which are the leading integrators for the specific industry and around specific

solution areas? What are the industry benchmarks for key performance indicators such as hourly

rates, utilization rates, breadth of services offerings, global delivery capabilities, etc.?

Information to obtain about supportability Repair parts availability and lead times, documentation, pricing, and distribution

systems Customer service, installation, checkout, and user operation and maintenance

instructions Requirements and provisions for manpower and personnel Competitive or sole source repair Training and training support requirements Requirements for and availability of tools, test equipment, computer support

resources, calibration procedures, operations, and maintenance manuals Commercial repair capabilities Supplier calibration, repair, and overhaul practices and capabilities documentation Supplier commitment to maintenance and operation support Degree of technical data package availability Stability of current configuration of technology Security requirements

Information to obtain about Test data Hardware, software, and manpower interface issues such as human factors and

product safety as experienced by similar users

Mar

ket R

esea

rch

Guid

elin

es

19

Mar

ket R

esea

rch

Guid

elin

es

20

HOW IS MARKET RESEARCH DONE?

Manufacturer test results Certification or test results from independent test organizations List of those currently using the product

Information to obtain about Business Data

Distribution practices Minimum order quantities or TIER or package pricing and practices Typical contract terms and conditions Warranty procedures Financing practices Typical Service Level Agreements

Information to obtain about supportably and maintenance

What is the anticipated staff requirements that the solution may require for the with the operation, administration and maintenance of the system

What should be included in Knowledge Transfer activities What should the State know about Maintenance and Operations risks

Information to provide to vendors may include:

General Information

Operating characteristics for hardware and software Environmental conditions for use

Logistics Support Information

Planned maintenance echelons Software maintenance plans Maintenance environment (weather, mud) Supply support, support equipment needs, limitations Training needs Technical data needs Transportability

Services Information

Description of services desired Period of performance Expected deliverables Expected labor categories Usage (e.g., fixed, airborne, tactically deployable)

HOW IS MARKET RESEARCH DONE?

System Interface or Integration Requirements

Computer language, speed, throughput, ports, memory and expansion potential. Radio transmission frequency requirements and allocation status Rules for government use of frequency spectrum Human factors considerations

Maintainability Information

Self-test requirements Limitations, if any, on organizational-level support equipment

Communications-Computer System Interface Information

Software portability to other communications-computer systems Interoperability Reliability, Maintainability, and Survivability data Input power quality (drops, surges, spikes, noise) Essential safety characteristics Operating duty cycle (e.g., 24 hours, intermittent)

Market research disclaimers

Inform Suppliers/Respondents to always read the RFI before responding Always discuss the RFI being is issued for information and planning purposes only

and does not constitute a solicitation for services or products. I.e. “A response to this RFI is not an offer and cannot be accepted by the Agency/state entity to form a binding contract.”

Suppliers/Respondents s are solely responsible for all expenses associated with responding to this RFI

Suppliers/Respondents are advised that the responses to this RFI may be subject to the Public Records Act (California Government Code sections 6250-6276.48)

Suppliers/Respondents s are requested to respond to the requirements based on actual or planned product or service offerings which are available for delivery within the next (Insert Planned months)

Market Research Logistical Information

Provide a Purpose Statement Provide a Scope Statement Provide Key Action Dates with activities tied to them Provide Background Information Provide Contact and Correspondence information Provide any required content format when response

Mar

ket R

esea

rch

Guid

elin

es

21

HOW IS MARKET RESEARCH DONE?

Provide any additional information that might be of benefit to gather meaningful data

NOTE OF CAUTION:

Mar

ket R

esea

rch

Guid

elin

es

Always follow the applicable State and Federal Statues, Rules, regulations. See-Competitive Solicitations section of SCM, Vol.3, Revision 1, Chapter 4

A1.7 - A1.11 Request for Information For IT Reportable projects seek Department of Technology for guidance for

Market Research assistance Do not allow for any unfair or biased discussions or activities. Always be open,

competitive, and allow for data and statistics on competition!

22

WHERE IS MARKET RESEARCH DONE? - RESOURCES?

RESEARCH RESOURCES

Reading Trade Journals

Contacting knowledgeable people (state, government and industry) in specific

markets

Contacting Procurement Officials, Program Managers, Contract Officers in other

Federal/state agencies and private industry (other users). Take advantage of the

lessons that these individuals have learned in previous solicitations

Contacting known sources or those communities of those providing services for the

applicable IT solution being researched

Reviewing market surveys prepared by independent companies

Attending trade shows, conferences, and symposia

Querying State and other Government databases (eProcurement, FI$Cal, Bidsync,

Govwin, FPDS-NG, PPIRS, GSA, FedBizOpps, SBA databases, that provide relevant

information on acquisitions

Reviewing results of recent market research on similar or identical requirements.

Reviewing catalogs and literature published by service providers

Reviewing State contracts, Federal Supply Schedule contracts, and other

Government contracts

Don’t reinvent the wheel if it isn’t necessary – Some of the most important questions may

have already been asked by another entity… “Who else is using the solution/service I need”

How did they solve this objective/problem? Who/What did they use? How current is the data?

Is it viable? How is it different then how we do it today? What Knowledge transfer will I need ?

What is the maintenance?

Mar

ket R

esea

rch

Guid

elin

es

23

Mar

ket R

esea

rch

Guid

elin

es

WHERE IS MARKET RESEARCH DONE? - RESOURCES?

INTERNET RESEARCH RESOURCES

General Search Engines

www.google.com https://www.yahoo.com/ https://ixquick.com/ https://www.bing.com/ http://www.thesearchenginelist.com/ - A comprehensive list of search engines

Labor

http://www.bls.gov/cpi/ - Published monthly by the Bureau of Labor Statistics. The Consumer Price Index (CPI) provides a guide to changes to "retail" prices. It is used as a price adjustment factor in some areas of procurement (leasing in particular) and as a general gauge of inflation. However, the Producer Price Index (PPI) is usually a better source of data on price trends for the Government, since the Government is usually a "wholesale" buyer

https://www.bls.gov/ppi/- Producer Price Index (PPI) from the Bureau of Labor Statistics - Published by the Bureau of Labor Statistics (BLS). It tracks changes in the prices which producers receive from the initial commercial purchasers of their products. It reflects the selling prices of selected manufacturers or the prices quoted on organized exchanges such as the Chicago Board of Trade. The prices are generally f.o.b. origin for immediate delivery

Information Technology

http://nist.gov/ - The appendices to the Council on Competitiveness Report on U.S. Research & Development Competitiveness, entitled Endless Frontier, Limited Resources U.S. R&D Policy For Competitiveness, contain studies of research & development practices in the Aircraft, Automotive, Chemical, Electronics, Information Technologies and the Pharmaceuticals industries

http://www.informationweek.com/ - Information Technology news and research

24

WHERE IS MARKET RESEARCH DONE? - RESOURCES?

http://washingtontechnology.com/Home.aspx = Latest news and information on the business of delivering technology and services to government including government contractors, the integrator community

http://www.forbes.com/technology/ = Forbes is a leading source for reliable news and updated analysis on Technology

http://itri.uark.edu/ - Information Technology Research Institute, University of Arkansas

http://ist.mit.edu/ MIT Information Systems and Technology (IST)

http://www.infoworld.com/news - InfoWorld's IT news and technology stories cover all aspects of information ... Latest Reviews

http://www.pricewatch.com/ - A fast way to find the "street price" on a computer product. Price Watch Corporation is an independent research firm that studies the street prices of computer components. Their web database is updated about 3 times a day at undetermined times

Commodities www.thomasregister.com – Product and Supplier sourcing

http://www.industrynet.com/ - Industrial search engine. Discover who makes it, who supplies it, and who does it among 430,000 manufacturers & suppliers. Silicon Valley, Inc.'s on-line database of seller information sorted by purchasing categories. A valuable tool for preliminary market research

http://www.betterbuys.com/ - Office Equipment- Offers a guide to various kinds of office equipment. It refuses to accept advertising, so as to offer an unbiased analysis. There are Better Buys reports on all the major types of office equipment -- copiers, fax, multifunctional equipment, printers, phones systems, mailing equipment, and more. You can either subscribe to all the guides or buy individual ones

Mar

ket R

esea

rch

Guid

elin

es

25

WHERE IS MARKET RESEARCH DONE? - RESOURCES?

State Resources

Mar

ket R

esea

rch

Guid

elin

es

26

http://www.dgs.ca.gov/pd/Programs/eprocure.aspx - eProcurement and/or FI$Cal is the portal for the California State Contracts Register (CSCR), the State Contracting and Procurement Registration System (SCPRS), the online Small Business and Disabled Veteran Business Enterprise (DVBE) queries and the statewide Leveraged Procurement Agreements (LPAs)

http://www.cio.ca.gov/ - The IT Project Oversight and Consulting Division (ITPOD) and Statewide Technology Division (STPD) has approval and oversight authority, as applicable, for California State Information Technology projects

Federal Resources

http://fpdc.gov/fpdc/fpdc_home.htm - Federal Procurement Data Center www.gsaadvantage.gov – GSA Advantage- Federal Supply catalogs https://www.fbo.gov/ - Federal Business Opportunities. This is where the Federal

Government posts there Pre Solicitation, Sources Sought notices, Solicitations, and Award Notices etc.

Financial Resources

https://www.sec.gov/ - U.S. Securities & Exchange Commission Corporate Information Database. This data base contains financial filings required of companies as required by the SEC

http://www.dnb.com/ - Dun and Bradstreet - A catalog of tools to identify and assist in evaluating potential suppliers based on purchasing needs; demographics such as size, geography, woman or minority owned business; ISO 9000 registration; past performance information; non-delivery or insolvency information. D&B offers reports, publications, supplier base analysis and software

General Business Search Engines

http://www.hoovers.com/ - A D&B Company. This database contains listings for over 10,000 companies. Hoover's Company Profile Database, which is only available to subscribers, contains in-depth profiles of over 2,480 companies

http://us.kompass.com/ - 2.3M Companies in 70 countries referenced by 57.000 product & service keywords 860.000 trade names and 4.6M executive names. A guide for worldwide sourcing

http://www.thesearchenginelist.com/ - contains more than 400,000 listings within about 65,000 categories. Search results are preceded by four types of paid links

FURTHER CONSIDERATIONS OF MARKET RESEARCH

In discussing Market research, here are five principles that need to be kept in mind:

1. Start early and often

Begin Market research early, while the solution requirements are still flexible! In a Department of Technology PAL process, begin Market research as early as possible, while the project needs and objectives are being defined. Early research can support the development of the requirements document. Further defined research in Stage 2 AA will be performed in the PAL to identify not only scope, budget, schedule, risks, and resources needed but also the alignment of the S2AA recommendation of IT technical, business, functional approach and needs, the subsystems or components for the design and development of mature project requirements (specific details related to technical characteristics) and related market practices. In addition, the PAL process also includes research activities in the Stage 3 Solution Development by providing the industry a draft of the solicitation.

Stage 3 Solution Development industry a draft of the solicitation process will be within the oversight of Department of Technology, ITPOD and STPD.

2. Always Involve Users (Critical Partners/Stakeholders)

Involve users in the Market research process, not merely in just defining the requirements. Users can be active participants in the Market research for some items and services or testers for others. Try to involve users formally in working groups, and also maintain constant informal lines of communication between them and the market research analysts. Users in the field most likely will have access to new technology and new product information that needs to be conveyed to those who define the business and technical project requirements. Users also play a major role in identifying problems with the current equipment or service.

3. Communicate

Good communication across all IT project members, functional, business and technical areas, SME’s, the Vendors/ industry, users and executives is the most important factor to the success of a Market research. For this reason an extensive and collaborative Market research approach will always work best as a team effort. The effect of linking each member’s prior experience and area of expertise with the information gathered will guarantee more balanced, “best-value” decisions.

Reminder, in communicating with industry, consider the use of multiple different types of formats (written, online, physical) to ensure you are communicating with all vendors, solution providers near and far – in addition to those incumbent contractors that are already providing services or that have traditionally dealt in the State market.

Mar

ket R

esea

rch

Guid

elin

es

27

Mar

ket R

esea

rch

Guid

elin

es

FURTHER CONSIDERATIONS OF MARKET RESEARCH

4. Tailor the research effort

The amount of time and money spent on Market research should be related to factors such as the value of the Project/Procurement, the complexity of the Business need or opportunity. If market surveillance/research information tells you there is little potential for commercial use, further Market research should be minimal. If commercial potential is high, an extensive more detailed Market research may be called for.

Do not invest more resources (e.g., lead-time, available staff hours, and money) in Market research than are warranted by the potential benefits received from the business need!

5. Refine as you proceed

The research should always proceed from the general to the specific. Acquire a little bit of information on many producers and products and a lot of information on the few products/services that are likely to fit and meet the requirements. The Market research should be structured to acquire only enough information to decide whether to proceed to a more detailed examination of possibilities. In addition, keeping the end in mind, by including in the analysis, ways to determine (rankings, best value factors) the best fitting recommended solution(s).

Market research can in some rare cases even be ongoing even during t he procurement process, this is highly cautioned and you must seek Department of Technology advice on this process, if applicable. Market research in most cases is used prior to any solicitation and is used to determine the availability of commercial capabilities, practices, items, and services available. It may also be done in different stages within the PAL to confirm and identify commercial components available or for incorporation into systems already developed for the State and to get more specific, detailed information or to make various decisions during PAL development. Research components may, if applicable, assist in providing negotiation drivers specific to that solution or identify project risks after implementation.

28

WRAPPING UP WITH CHECK LISTS & NEXT STEPS

Market Surveillance/Research Checklist - General

Start Early and Prepare

Define your problem/opportunity (Statement of Need)

Get approval of Market Research Deliverable (i.e. Sponsors/Stakeholders/Users)

Communicate Openly – Involve users, teams, executives, Agency/state entity

procurement

For IT Reportable projects seek the advice Department of Technology ITPOD and

STPD when initializing Market research. Special care should be taken when the

project is complex, high public visibility, proprietary, confidential in nature, or if the

Agency/state entity has any questions on research activities that could potentially

put the IT project at risk.

Document results

Market Knowledge (Review the “How” section)

What are the industry's current products and service capability?

What new development programs are they working on?

What trends are noticed within the industry?

Who are the industry leaders in terms of performance, quality, price, delivery

capability, capacity, depth of products or services, etc.?

What are the industries levels of quality and ability to maintain quality products and

services?

Listen to industry feedback

Product/Service Knowledge (Review the “How” section)

Seasonal trends that impact the industry

Economic trends that impact the industry

Other industries whose trends impact the industry

Distribution channels

Mar

ket R

esea

rch

Guid

elin

es

29

Mar

ket R

esea

rch

Guid

elin

es

30

WRAPPING UP WITH CHECK LISTS & NEXT STEPS

Look for products / services / works that best meet the needs of the Agency/State

entity. Are there possible alternatives for consideration the vendor can provide?

Market Surveillance/Research Checklist Continued (Review the “How” section)

Decide which product and alternatives need further specification? What are the

going rates (level of prices/ROM)?

Company Knowledge (Review the “How” section)

What is the company’s financial condition? Are they financially stable?

What markets do they serve? State of California, other states, Federal, Local,

commercial?

What is the company’s degree of innovation in the market place? Are they a trend

setter or follower?

What is the State’s past performance experience with the companies?

What is the company’s distribution make up? Some instances you can obtain a

better service/product dealing direct with a service provider or manufacture and

sometimes it is better to go through a distribution partner, Master Value Added

Reseller, Value Reseller, Retailer, Authorized Service Provider etc. Find out how the

industry operates and allow open competition to vet out who offers a better solution.

Next Steps – Research completed Below are the basic next steps (and these should all be documented):

Have your documented Market research gone through all the IT Project Team

members for review and concurrence with the findings/recommendation

The Market Research will need to align with the S2AA recommended

solution/approach chosen

o See Department of Technology SIMM instruction for further S2AA

instructions on what criteria is needed for Market research

WRAPPING UP WITH CHECK LISTS & NEXT STEPS

Next Steps – Research completed, Continued

The Market Research will need to be documented/attached and include a summary

with the results and provide supporting documentation.

Did the Market research lead to current and applicable information?

Did the Market research Requirements yield expected results?

Did the agency develop core criteria (requirements/goals) and use these as a first

step in the market research process?

Did the agency obtain sufficient information through the preferred method(s) so that

the information can be evaluated against the requirements; this would include costs.

Did the Agency/state entity communicate and document the results from each

responder/entity for further analysis and decision making? (A spreadsheet works

well for this purpose – seek Department of Technology ITPOD or STPD as a

resource for examples of previously used material.

Be prepared to analyze and document results that are “apples and oranges” since

different types of market research will yield different results.

Did the Market research data fill in the gaps, risks, and support the business

decisions ?

Did the Market research assist the IT Project Team, Stakeholders and Executives be

on the same page with regard to priorities.

Evaluation, analysis, clear unbiased data, and a well-informed feasible and current

decision or recommendation(s) should be the conclusion of the Market research: Did the

Agency/state entity evaluate all the facts and data open and fairly when documenting their

decision? ( This should include documenting that the market offers with no alternatives or if

the Agency/state entity was not able to obtain viable information and why).

Mar

ket R

esea

rch

Guid

elin

es

31