quantitative and qualitative evaluation methods
TRANSCRIPT
-
7/30/2019 Quantitative and Qualitative Evaluation Methods
1/2
Quantitative and Qualitative Evaluation Methods
Evaluationmethods and the data they produce are grouped into two basic categories quantitative andqualitative. In general, quantitative methods produce hard numbers while qualitative methods capture moredescriptive data. The method(s) you choose are determined by the purpose(s) of your evaluation and the resourcesyou have to design and conduct it. In practice, most researchers and evaluators agree that combining quantitativeand qualitative techniques (sometimes called mixed method evaluations) produces a richer and morecomprehensive understanding of a projects accomplishments and learnings.
How are Quantitative and Qualitative Data Different?
At the most basic level, data are considered quantitative if they are numbers and qualitative if they arewords. Qualitative data may also include photos, videos, audio recordings and other non-text data. Those who favorquantitative data claim that their data are hard, rigorous, credible and scientific. Those in the qualitative camp counterthat their data are sensitive, detailed, nuanced and contextual. Quantitative data best explain the whyand howofyour program, while qualitative data best explain thewhat, who and when.
Different techniques are used to collect and analyze quantitative and qualitative data:
Quantitative Techniques Qualitative TechniquesSurveys/Questionnaires ObservationsPre/post Tests InterviewsExisting Databases Focus GroupsStatistical Analysis Non-statistical (methods vary)
In general, evaluators agree that qualitative and quantitative data and methods have different strengths, weaknesses,and requirements that affect decisions about which methodologies are appropriate for which purposes.
Quantitative Data and Evaluation Methods
What are Quantitative Data?
Pieces of information that can be counted mathematically
Usually gathered by surveys from large numbers of respondents selected randomly
Secondary data such as census data, government statistics, etc. often included in quantitative evaluationsAnalyzed using statistical methods
Best used to answer what, when and who questions
Not well suited to how and why questions
Strengths Limitations
Findings can be generalized, if selectionprocess well-designed and sample isrepresentative of study population
Related secondary data sometimes notavailable, or accessing available data isdifficult/impossible
Relatively easy to analyze Difficult to understand context of programactivities
Data can be very consistent, precise,reliable
Data may not be robust enough to explaincomplex issues
Data collection is usually cost efficient
How do you collect Quantitative Data?Surveys
o Most common methodo Self-administered or by someone elseo Face-to-face, telephone, mail, web-based
Secondary Data
http://www.civicpartnerships.org/docs/tools_resources/Evaluation%209.07.htmhttp://www.civicpartnerships.org/docs/tools_resources/Evaluation%209.07.htmhttp://www.civicpartnerships.org/docs/tools_resources/Evaluation%209.07.htm -
7/30/2019 Quantitative and Qualitative Evaluation Methods
2/2
o Often used in conjunction with survey datao Includes census data, knowledge/attitude/behavior (KABB) studies, criminal justice statistics, performance
data, non-confidential client information, agency progress reports, etc.
Qualitative Data and Evaluation Methods
What are Qualitative Data?
Usually gathered by observations, interviews or focus groups
May also be gathered from written documents and through case studies
Less emphasis on countingnumbers of people who think or behave in certain waysand moreemphasis on explainingwhypeople think and behave in certain ways
Involves smaller numbers of respondents
Utilizes open-ended questionnaires or protocols
Best used to answer how and why questions
Not well suited to what, when and who questionsStrengths Limitations
Complement and refine quantitative data Findings usually can not be generalized tothe study population or community
Provide more detailed information toexplain complex issues
More difficult to analyze; dont fit neatly instandard categories
Multiple methods for gathering data onsensitive subjects
Data collection is usually time consumingand costly
How do you collect Qualitative Data?Observations
o Looking at what is happening rather than directly questioning participantso Used to better understand behaviors, their social context and meanings attached to themo Useful for certain populations - children, infantso Can identify unanticipated outcomes
Interviews (in-depth, individual)
o Usually provide rich data, details, insights from program participants and stakeholders about theirexperiences, behaviors and opinions
o Particularly useful for complex or sensitive subjectso Use open-ended questions
Focus Groups
o 8-12 people selected by non-random method, share some characteristics or experience relevant to theevaluation, ideally do not know each other, respond to questions from group facilitator
o Use group dynamics to generate data and insights
o Useful for generating ideas and strategies, defining problems in project implementation, assist withinterpreting quantitative findingso Open-ended questions or topics designed to stimulate discussion; topics usually broader than interview
questions
http://www.civicpartnerships.org/docs/tools_resources/Quan_Qual%20Methods%209.07.htm