w · web viewjohnson, b. (2017). dialectical pluralism. journal of mixed methods research, 11 (2),...

Writing Effective Mixed Method andMulti-Method Grant Proposals

Workshop Booklet

Table of Contents

Section I: Overview of Mixed Methods Designs Page 2-3

Section II: Mixed Methods Research Designs Table Page 4

Section III: Mixed Methods Study Design Examples Page 5-8

Section IV: Resource List (Books) Page 9-10

Section V: Resource List (Articles) 0-13

Section VI: Resource List (Political Science/ Sociology) Page 13-14---------------------------------------------------------------------------------

Acknowledgements

The Social Science Committee would like to acknowledge the generous support from the Office of Research to offer this workshop. In particular, we would like to thank Aleister Saunders, Christine Chacko and MaryAnn Skedzielewski for their assistance in the Office of Research. We would also like to thank Dom Gullo and Sherri Manson for their help in the School of Education. Invited guest speakers Jason Seawright and Nataliya Ivankova and faculty mentors Joke Bradt, Steve Lankenau, Kim Blankenship and Toni Sondergeld, provided valuable expertise and recommendations. This Booklet, prepared by Jen Katz-Buonincontro and Elaine Perignat, was made possible with input from Philip Ayoub, Phillip Massey, Collette Sosnowy, Dali Ma and Janell Mensinger of the Social Science Committee, who crafted the pedagogical design of this workshop.

Sponsored by the Social Science Committee in partnership with the Office of Research

Workshop: Writing Effective Mixed Method and Multi-Method Grant Proposals

Overview of Mixed Methods Designs1

Typically scholars can mix qualitative and quantitative data at four different stages of their research process: during design, data collection, analysis, and/or interpretation (see Creswell and Clark 2011, 66–8). The choice to triangulate can be “fixed,” decided on before the research is undertaken, or “emergent,” in response to questions or problems that arise during the research or data analysis process (Creswell and Clark 2011, 54). Upon deciding on the need to triangulate, scholars must also select the type of mixed-method design that best suits their particular research questions or interests. The three most commonly used mixed-method research designs are convergent parallel, explanatory sequential, and exploratory sequential (see Creswell and Clark 2011, 68–104).

Convergent Designs: According to Creswell (2014), a convergent design is when qualitative and

quantitative data are collected and analyzed separately, then compared to assess whether the findings contradict or confirm each other (219). An example from sociology using this type of design is Ferree et al.’s (2002) seminal comparative examination of abortion discourse in Germany and the United States. In Shaping Abortion Discourse, public opinion surveys and quantitative content analysis of newspaper articles are triangulated with in-depth interviews of activists and journalists in both countries. Through their novel use of these quantitative and qualitative data, the authors made important empirical (setting a new standard for research on framing) and theoretical (further developing the concept of “discourse opportunity structures”) contributions to research, while demonstrating how the strategies and actions of numerous state, media, and civil society actors in different institutional and cultural contexts can interact to produce varying results.

Exploratory Designs: Sequential mixed-method research designs can either be exploratory or

explanatory. In exploratory designs, “the researcher starts by qualitatively exploring a topic before building to a second, quantitative phase” (Creswell and Clark 2011, 86). The main goal of these types of designs “is to generalize qualitative findings based on . . . the first phase to a larger sample gathered during the second [quantitative] phase” (86). In a recent example, based on a newly created comprehensive database of Jewish

1 This section was developed using prose from Ayoub, Phillip, Sophia Wallace, and Chris Zepeda- Millán. 2014. “Triangulation in Social Movement Research.” In Donatella della Porta (ed.) Methodological Practices In Social Movement Research, Oxford: Oxford University Press.

2


victimization in the Netherlands during the Holocaust, Braun (2013) identified four pairs of Dutch villages that were socially similar but differed significantly in terms of Jewish evasion. For each of these pairs he consulted administrative records of German security forces and post-war testimonies which led to the hypothesis that Christian minority churches played a crucial role in the emergence of collective rescue networks because their leaders could exploit social insulation to setup clandestine movements. His inductively created hypothesis was then tested by pairing the quantitative database with geo-coded information of Christian church communities throughout the Netherlands. Through this exploratory sequential research design, Braun found that it was the structural position of Protestant or Catholic communities and not something inherent to either religion that produced collective networks of assistance to threatened Jewish neighbors.

Explanatory Designs:Alternatively, in an explanatory sequential designs, “the researcher collects

quantitative data in the first phase, analyzes the results, and then uses the results to plan (or build on to) the second, qualitative phase” of the research (Creswell 2014, 224). Several classic (Mansbridge 1986; McAdam 1988) and contemporary (Taylor et al. 2009; Hadden 2011, Ayoub 2016) studies have employed this type of triangulation. One of its main strengths is that it can help explain the mechanisms—how quantified variables interact—through the use of qualitative data (Creswell 2014, 224). Hadden’s (2011) work on the international climate change movement, for example, combined quantitative network analysis with over 90 qualitative interviews. The design aimed to explain how the tactical choices of social movement organizations are affected by their positions in an inter-organizational network, and the qualitative interviews were employed to provide evidence of the causal mechanisms underlying the correlations observed in the quantitative network analysis. Since senior scholars have recently called for more mechanism driven approaches to studying contentious politics (McAdam et al. 2001; Tilly and Tarrow 2006), the applied examples we describe in detail later also utilize this type of design. However, before we do so, below we discuss more of the benefits, as well as some of the concerns and complexities, of mixed-method research.

3


Mixed Methods Research Designs (based on Creswell & Plano Clark, 2003)

Design Central Purpose Data Mixing Example

Convergent More complete understanding of data

Concurrent quantitative and qualitative data

Data transformation or validation

Explanatory Explain quantitative results

Phase 1: quantPhase 2: qual

Explanation of survey results

Exploratory Measure qualitative findings

Phase 1: qualPhase 2: quant building on phase 1

Instrument developmentTheory development

Embedded Exploration before, during or after an experimental trial

Concurrent or sequential data collection that supports an experimental trial

Embedded experimentMixed methods case study

Transformative Address social injustices

Concurrent or sequential data collection guided by social justice framework

Disability focusFeminist lens

Multiphase Program development and evaluation

Concurrent or sequential data collection over multiple phases of a program

Large-scale program development or evaluation projects

4


Mixed Methods Study Design Examples

5


6


7


8


MIXED METHODS RESOURCE LIST

BOOKSCambridge University PressAyoub, Phillip M. (2016). When states come out: Europe's sexual minorities and the

politics of visibility. New York: Cambridge University Press.

Della Porta, Donatella and Michael Keating. (2008). Approaches and Methodologies in the Social

Sciences: A Pluralist Perspective. Cambridge University Press.

Ferree, Myra, William Gamson, Jurgen Gerhards, and Dieter Rucht. (2002). Shaping Abortion Discourse:

Democracy and the Public Sphere in Germany and the United States. New York: Cambridge

University Press. 1Seawright, J. (2016). Multi-method social science: Combining qualitative and quantitative tools.

Cambridge University Press.

Sage PublicationsCreamer, E. (2017, February). An introduction to fully integrated mixed methods research. Thousand Oaks,

CA: Sage.

Creswell, J. C. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd Ed).

Thousand Oaks, CA: Sage.

Creswell, J. C. (2014). A concise introduction to mixed methods research. Thousand Oaks, CA: Sage.

Creswell, J. C. & Plano Clark, V. (2017). Designing and conducting mixed methods research (3rd Ed).

Thousand Oaks, CA: Sage.

Curry, L. & Nunez-Smith, M. (2014). Mixed methods in health sciences research. Thousand Oaks, CA:

Sage.

DeCui-Gunby, J. T. & Schutz, P. A. (2016). Developing a mixed methods proposal: A practical guide for

beginning researchers. Thousand Oaks, CA: Sage.1Ivankova, N. V. (2014). Mixed methods applications in action research. Thousand Oaks, CA: Sage.

Maxwell, J. A. (2012). A realist approach for qualitative research. Sage.

Maxwell, J. A. (2013). Qualitative research design: An interactive approach (3rd ed.). Los Angeles, CA:

Sage.

Mertens, D. (2017). Mixed methods design in evaluation. Thousand Oaks, CA: Sage.

Padgett, D. K. (2011). Qualitative and mixed methods in public health. Sage publications.

Plano Clark, V. L. & Creswell, J. W. (2007). The mixed methods reader. Thousand Oaks, CA: Sage.

Plano Clark, V. L. & 1Ivankova, N. V. (2016). Mixed methods research: A guide to the field. Thousand

Oaks, CA: Sage.

Tashakkori, A. & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative

approaches. Thousand Oaks, CA: Sage.

9


Tashakkori, A., & Teddlie, C. (Eds.). (2010). Sage handbook of mixed methods in social & behavioral

research. Thousand Oaks, CA: Sage.

Workshop Guest Speaker

OxfordAyoub, Phillip, Sophia Wallace, and Chris Zepeda- Millán. 2014. Triangulation in Social Movement

Research. In Donatella della Porta (ed.) Methodological Practices In Social Movement Research,

Oxford: Oxford University Press

Brady, Henry E., and David Collier. (2004). Rethinking Social Inquiry: Diverse Tools, Shared Standards.

Oxford: Rowman & Littlefield Publishers.

Hesse-Biber, S. N., & Johnson, R. B. (Eds.). (2015). The Oxford handbook of multimethod and mixed

methods research inquiry.

WileyGreen, J. C. (2007). Mixed methods in social inquiry.

Borbasi, S., Jackson, D. (2015). Navigating the maze of research (4th Ed). Elsevier.

Griffin, K. A. & Museus, S. D. (Editor). (2011). Using mixed methods to study intersectionality in higher

education: New directions in institutional research. Number 151

Maruyama, G., & Ryan, C. S. (2014). Research methods in social relations, 8th Edition.

Mertens, D. M., Hesse-Biber, S. (2013). Mixed methods and credibility of evidence in evaluation: New

directions for evaluation. Number 138

ARTICLESCaracelli, V. J. & Greene, J. C. (1993). Data analysis strategies for mixed-method evaluation

designs. Educational Evaluation and Policy Analysis, 15(2), 195-2017.

Creswell, J. W., Klassen, A. C., Plano Clark, V. L., Smith, K, for the Office of Behavioral and Social

Sciences Research. (2011). Best practices for mixed methods research in the health sciences.

National Institutes of Health. Retrieved from http://obssr.od.nih.gov/mixed_methods_research.

Crouch, M., & McKenzie, H. (2006). The logic of small samples in interview-based qualitative research.

Social Science Information, 45(4), 483–499.

Curtis, S., Geslerb, W., Smith, G., & Washburn, S. (2000). Approaches to sampling and case selection in

qualitative research: Examples in the geography of health. Social Science & Medicine, 50, 1001-

1014.

Draucker, C. B., Martsolf, D. S., Ross, R., & Rusk, T. B. (2007). Theoretical sampling and category

development in grounded theory qualitative health research. Qualitative Health Research, 17,

1137-1148.

Fetters, M., Curry, L., & Creswell, J. W. (2013). Achieving integration in mixed methods designs—

principles and practices. Health Services Research, 48(6), Part II, 2134 – 2156.

Fetters, M., & Freshwater, D. Publishing a methodological mixed methods research article (Editorial).

Journal of Mixed Methods Research, 9(3) 203–213.

10


Francisa, J. J., Johnston, M., Robertson, C., Glidewella, L., Entwistle, V., Eccles, M. P., & Grimshawe, J.

M. (2010). What is an adequate sample size? Operationalizing data saturation for theory-based

interview studies. Psychology and Health, 25, 1229-1245.

Groger, L., Mayberry, P. S., & Straker, J. K. (1999). What we didn’t learn because of who would not talk to

us. Qualitative Health Research, 9, 829-835.

Guest, G. (2013). Describing mixed methods research: An alternative to typologies. Journal of Mixed

Methods Research, 7, 141-151.

Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data

saturation and variability. Field Methods, 18, 59-82.

Guetterman, T. C., Fetters, M. D., & Creswell, J. W. (2015). Integrating quantitative and qualitative results

in health science mixed methods research through joint displays. Annals of Family Medicine, 13,

554-561.

Hagaman, A., & Wutich, A. (2017). How many interviews are enough to identify metathemes in multisited

and cross-cultural research? Another perspective on Guest, Bunce, and Johnson’s (2006) landmark

study, Field Methods, 29, 23-41. 1Ivankova, N. V. (2014). Implementing quality criteria in designing and conducting a sequential QUAN→

QUAL mixed methods study of student engagement with learning applied research methods

online. Journal of Mixed Methods Research, 8(1), 25-51.1Ivankova, N. V. (2017). Applying mixed methods in community-based participatory action research: A

framework for engaging stakeholders with research as a means for promoting patient-centredness.

Journal of Research in Nursing, 22(4), 282-294.1Ivankova, N. V., Creswell, J. W., & Stick, S. L. (2006). Using mixed-methods sequential explanatory

design: From theory to practice. Field Methods, 18(1), 3-20.1Ivankova, N. V., & Stick, S. L. (2007). Students’ persistence in a distributed doctoral program in

educational leadership in higher education: A mixed methods study. Research in Higher

Education, 48(1), 93-135. DOI: 10.1007/s11162-006-9025-4

Johnson, B. (2017). Dialectical pluralism. Journal of Mixed Methods Research, 11(2), 156-173.

Katz-Buonincontro, J., Hass, R. W., & Friedman, G. (2017). “Engineering” student creativity in a

probability and statistics course: Investigating perceived versus actual creativity. Psychology of

Aesthetics, Creativity, and the Arts, 11(3), 295-308.

Knaggs, C. M., Sondergeld, T. A., & Schardt, B. (2015). Overcoming barriers to college enrollment,

persistence, and perceptions for urban high school students in a college preparatory program.

Journal of Mixed Methods Research, 9(1), 7-30.

Lankenau, S. E., Teti, M., Silva, K., Bloom, J. J., Harocopos, A., & Treese, M. (2012). Initiation into

prescription opioid misuse amongst young injection drug users. International Journal of Drug

Policy, 23(1), 37-44.

11


Myers, K. K., & Oetzel, J. G. (2003). Exploring the dimensions of organizational assimilation: Creating

and validating a measure. Communication Quarterly, 51(4), 438-457.


Onwuegbuzie, T., & Leech, N. (2006). Linking research questions to mixed methods data analysis

procedures. The Qualitative Report, 11(3), 474-498.

O’Reilly, M., & Parker, N. (2012). “Unsatisfactory Saturation”: A critical exploration of the notion of

saturated sample sizes in qualitative research. Qualitative Research, 13, 190-197.

Plano Clark, V. L., Schumacher, K.,West, C., Edrington, J., Dunn, L. B., Harzstark, A., Melisko, M.,

Rabow, M. W., Swift, P. S., & Miaskowski, C. (2013). Practices for embedding an interpretive

qualitative approach within a randomized clinical trial. Journal of Mixed Methods Research, 7,

219-242.

Polit, D. F., & Beck, C. T. (2010). Generalization in quantitative and qualitative research: Myths and

strategies. International Journal of Nursing Studies, 47, 1451-1458.

Safman, R. M., & Sobal, J. (2004). Qualitative sample extensiveness in health education research. Health

Education & Behavior, 31, 9-21.

Sandberg, J., & Alvesson, M. (2011). Ways of constructing research questions: Gap-spotting or

problematization? Organization, 18, 23-44.

Sandelowski, M. (1995). Sample size in qualitative research. Research in Nursing & Health, 18(2), 179–

183.

Sandelowski, M. (2000). Combining qualitative and quantitative sampling, data collection and analysis

techniques in mixed-method studies. Research in Nursing & Health, 23, 246-255.

Sandelowski, M. (2008). Theoretical saturation. In L. M. Given (Ed.), The Sage encyclopedia of qualitative

research methods (Vol. 2, pp. 875-876). Thousand Oaks, CA: Sage.

Sandelowski, M., & Barroso, J. (2003). Writing the proposal for a qualitative research methodology

project. Qualitative Health Research, 13, 781-820. 1Seawright, J. (2016). The case for selecting cases that are deviant or extreme on the independent variable.

Sociological Methods & Research, 45(3), 493-525.1Seawright, J. (2016). Better multimethod design: The promise of integrative multimethod research.

Security Studies, 25(1), 42-49.

Sharp, L., Mobley, C., Hammond, C., Withington, C., Drew, S., Stringfield, S., & Stipanovic, N. (2012). A

mixed methods sampling methodology for a multisite case study. Journal of Mixed Methods

Research, 6(1), 34–54.

Smaling, A. (2003). Inductive, analogical, and communicative generalization. International Journal of

Qualitative Methods, 2(1), Art. 5. Retrieved from

http://www.ualberta.ca/~iiqm/backissues/2_1/pdf/smaling.pdf

12


Sondergeld, T. A., & Koskey, K. L. (2011). Evaluating the impact of an urban comprehensive school

reform: An illustration of the need for mixed methods. Studies in educational Evaluation, 37(2),

94-107.


Tariq, S., & Woodman, J. (2013). Using mixed methods in health research. JRSM Short Reports, 4(6),

2042533313479197.

Wittink, M. N., Barg, F. K., & Gallo, J. J. (2006). Unwritten rules of talking to doctors about depression:

integrating qualitative and quantitative methods. The Annals of Family Medicine, 4(4), 302-309.

Wood, M., & Christy, R. (1999). Sampling for possibilities. Quality & Quantity, 33(2), 185–202.

Reference List from Political Science and Sociology:

Adcock, Robert and Collier, and David. (2001). Measurement Validity: A Shared

Standard for Qualitative and Quantitative Research. American Political Science Review. 95(3)

529-546.

Ahmed, Amel and Rudra Sil. (2009). “Is Multi-Method Research Really ‘Better’?” Qualitative and Multi-

Method Research. 7(2): 2-5.

Ahmed, Amel and Rudra Sil. (2012). “When Multi-Method Research Subverts Methodological Pluralism—

or, Why We Still Need Single-Method Research.” Perspectives on Politics. 10(4): 935-953.

Braun, Robert. (2013) “Minority Churches and High Risk Mobilization: the Collective rescue of Jews in

the Netherlands During the Holocaust.” Paper presented at the 39th Annual Meeting of the Social

Science History Association, Chicago, IL.

Collier, David, and James Mahoney. (1996). “Research Note: Insights and Pitfalls: Selection Bias in

Qualitative Research.” World Politics 49 (1): 56–91.

Goertz, G. (2006). Social Science Concepts: A User’s Guide. Princeton: Princeton University Press.

Greene, Jennifer, Valerie Caracelli, and Wendy Graham. (1989). "Toward a Conceptual Framework for

Mixed-Method Evaluation Designs." Educational Evaluation and Policy Analysis. 11(3): 255-274.

Hadden, J. (2011). “Contesting Climate Change: Civil Society Networks and Collective Action in the

European Union”. Dissertation, Ithaca: Cornell.

Jick, Todd. D (1979). “Mixing Qualitative and Quantitative Methods: Triangualtion in

Action.” Administrative Science Quarterly. 24(4):602-611.

King, Gary, Robert Keohane, and Sidney Verba. (1994). Designing Social Inquiry: Scientific Inference in

Qualitative Research. Princeton University Press.

Klandermans, Bert, Suzanne Staggenborg, and Sidney Tarrow. (2002). "Conclusion: Blending Methods and

Building Theories in Social Movement Research." In Bert Klandermans and Suzanne Staggenborg

(eds), Methods of Social Movement Research. Minneapolis: U of Minnesota Press.

Lieberman, Evan S. (2005). “Nested Analysis as a Mixed-Method Strategy for Comparative Research.”

American Political Science Review 99 (03): 435–452.

McAdam, Doug. (1988). Freedom Summer: The Idealists Revisited. New York: Oxford University Press.

13


Przeworski, Adam, and Henry Teune. (1970). The Logic of Comparative Social Inquiry. Wiley-

Interscience.

Rohlfing, Ingo. (2008). "What You See is What You Get: Pitfalls and Principles of Nested Analysis in

Comparative Research." Comparative Political Studies, 41 (11): 1492-1514.

Sil, Rudra, and Peter J. Katzenstein. (2010). “Analytic Eclecticism in the Study of

World Politics: Reconfiguring Problems and Mechanisms across Research Traditions.”

Perspectives on Politics 8(02): 411–431.

Small, Mario Luis. (2011). "How to Conduct a Mixed Methods Study: Recent Trends in a Rapidly Growing

Literature." Annual Review of Sociology. Vol. 37: 57-86.

Snijders, Tom A. B., and Roel J. Bosker. (1999). Multilevel Analysis. London: Sage

Tarrow, Sidney. (1995). “Bridging the Quantitative-Qualitative Divide in Political Science.” The American

Political Science Review. 89 (2): 471.

———. (2004). “Bridging the Quantitative and Qualitative Traditions.” In Henry E. Brady and David

Collier (eds), Rethinking Social Inquiry: Diverse Tools, Shared Standards, 171–179. Rowman &

Littlefield Publishers.

Tarrow, Sidney, Bert Klandermans, and Suzanne Staggenborg. (2002). “Conclusion: Blending Methods

and Building Theories in Social Movement Research.” In Bert Klandermans and Suzanne

Staggenborg (eds), Methods of Social Movement Research, 314–350. Minneapolis: U of

Minnesota Press.

Taylor, Verta, Katrina Kimport, Nella Van Dyke, and Ellen Ann Andersen. (2009). "Culture and

Mobilization: Tactical Repertoires, Same-Sex Weddings, and the Impact on Gay Activism."

American Sociological Review. 74 (6): 865-890.

Yeasmin, Sabina, and Khan Ferdousour Rahman. (2012). Triangulation Research Method as the Tool of Social Science Research. BUP Journal, 1(1), 154-163.

--------------------------------------------

14

w · web viewjohnson, b. (2017). dialectical pluralism. journal of mixed methods research, 11 (2),...

Documents