research design, philosophy and methods

Research design, philosophy and methods

Mark Reed

Plan

• Choosing your research topic: where are you?• Research philosophy: what is knowledge? • Research design

1 What is knowledge?

Data• Raw numbers &

facts

Information• Useful data (that

has been analysed/ interpreted)

Knowledge• Information that is

known by an individual/group

Wisdom• “Constructive” use

of knowledge (Matthews, 1997)

• “Use of knowledge ...to achieve a common good” (Sternberg, 2001)

Universal truth generated by reducing the world to its constituent parts to

test hypothesesKnowledge as a social construction leading to multiple realities

Different ways of viewing and constructing knowledge...

Different types of knowledge...

Raymond CM, Fazey I, Reed MS, Stringer LC, Robinson GM, Evely AC (2010) Integrating local and scientific knowledge for environmental management: From products to processes. Journal of Environmental Management 91: 1766-1777

Knowledge Type

Implicit(not yet articulated)

Local

Informal

Novice

Tacit(cannot be articulated)

Traditional

Generalised/Universal

Formal

Expert

Explicit(articulated)

Scientific

Extent to which knowledge is locally generated/relevant versus universal

Extent to which knowledge generated via formal, codified processes

Extent to which those generating knowledge are regarded as experts

Extent to which knowledge is articulated and accessible to others

Extent to which knowledge is embedded in and reflects traditional cultural values/norms, or in the scientific method

Different types of knowledge...

Raymond CM, Fazey I, Reed MS, Stringer LC, Robinson GM, Evely AC (2010) Integrating local and scientific knowledge for environmental management: From products to processes. Journal of Environmental Management 91: 1766-1777

Knowledge Type

Implicit(not yet articulated)

Local

Informal

Novice

Tacit(cannot be articulated)

Traditional

Generalised/Universal

Formal

Expert

Explicit(articulated)

Scientific

Extent to which knowledge is locally generated/relevant versus universal

Extent to which knowledge generated via formal, codified processes

Extent to which those generating knowledge are regarded as experts

Extent to which knowledge is articulated and accessible to others

Extent to which knowledge is embedded in and reflects traditional cultural values/norms, or in the scientific method

Post-modern PositivistEpistemology

Different ways of managing knowledge...

Knowledge Transfer

Producers UsersProducers Users

One-way flow of existing knowledge

Knowledge Exchange

Producers Users

Two-way flow of existing knowledge

Knowledge generation

Producers

Producers generate or co-generate knowledge

together

Knowledge application

Users

Users apply knowledge gained through transfer

or exchange and provide feedback to or become producers of knowledge

Know-ledge

Storage

Reed MS, Fazey I, Stringer LC, Raymond CM, Akhtar-Schuster M, Begni G, Bigas H, Brehm S, Briggs J, Bryce R, Buckmaster S, Chanda R, Davies J, Diez E, Essahli W, Evely A, Geeson N, Hartmann I, Holden J, Hubacek K, Ioris I, Kruger B, Laureano P, Phillipson J, Prell C, Quinn CH, Reeves AD, Seely M, Thomas R, van der Werff Ten Bosch MJ, Vergunst P, Wagner L (2011) Knowledge management for land degradation monitoring and assessment: an analysis of contemporary thinking. Land Degradation & Development

2 Research design

How to choose research design

Choice influenced by:• Research questions you want to answer• Epistemology• Preferences towards qualitative/quantitative

Designing to questions

The questions you can answer will depend on:• Existing data availability• Can you measure/collect relevant new data?

– Skills, equipment, time etc.• The more focused your question, the easier it

will be to design your research

Epistemology

• How do you perceive knowledge, how it is generated and what constitutes valid knowledge?

• Positivists: define hypotheses and quantify, proving beyond doubt

• Post-modernists: more open-ended research questions and qualitative, providing a range of perspectives to build credible arguments

Qualitative versus quantitative

• Examples of reasons to choose qualitative versus quantitative in different contexts?

• Benefits/challenges of mixing both?

Qualitative or quantitative?

• Depending on research question and epistemology, qual/quant may be obvious

• Alternatively, start with a qual/quant preference and select research questions accordingly

• More on choosing qual/quant later

Writing up research design

• Methodology chapter: difference between research design and methods

• Create a sub-section for both• Explain your design and methods in enough

detail for someone else to replicate• Justify your choice – theoretically and/or

empirically

3 Methods

Should I use or collect Primary or secondary Data?

Primary data

• Primary data is collected by you, first-hand

Secondary data

• Secondary data has been collected by someone else, and you are using it “second-hand”

What should I use?

• For your dissertation it is safest to focus on primary data collection– Easier to demonstrate originality– Harder to fall into trap of writing extended lit

review• Supplement your primary data with secondary

data to check/deepen your analysis– Handy if you don’t think you’ve got enough

primary data

Qualitative or quantitative?

What is qualitative?

• Understanding the quality or nature of things, rather than their quantity– Good for asking “why” questions and gaining an in-

depth understanding of many different perspectives on an issue (i.e. often subjective)

– Not so suited to statistical analysis and clear-cut, “objective” answers

– Typically use quite small sample sizes (e.g. 20 interviews and a focus group)

– Can be flexible – adapt your methods as you go

Examples of qualitative

–Examples of qualitative data collection methods:• Open-ended questions in questionnaires• Semi-structured interviews• Focus groups• Participant observation• In-depth case studies

–Examples of qualitative data:• Transcripts, audio, interview notes, documents

–Examples of qualitative analysis:• Content analysis e.g. Grounded Theory Analysis

What is quantitative?

• Understanding the quantity of things – being able to quantify relationships and describe them mathematically or in terms of their statistical significance– Good when you need to be able to answer a research question

with precision, determine if there is a relationship between two things (x varies with y) or you need to determine something is statistically significant

– Harder to determine causality (x causes y to vary) and answer “why” questions

– Typically large data sets (min 50 data points, ideally >100)– Inflexible – have to stick to and replicate your method

Examples or quantitative

• Examples of quantitative data collection methods:– Ecological and soil-based survey techniques e.g.

counting plants in quadrats or along transects– Experiments– Closed questions in questionnaires e.g. Likert scale and

categorical or numerical questions• Examples of quantitative analysis

– Calculating percentages, means & standard deviations– Statistical analyses

Qualitative or quantitative?

– I need to ask mainly what, where and when questions

– I need to understand exactly how something has changed or might change in future

– I need to understand if something influences something else

– I need to know of something is significantly greater or lesser than something else

– The people reading my research want a precise or “objective” answer to my research questions

– PROBABLY QUANTITATIVE

Qualitative or quantitative?

– I need to ask why questions– I want an in-depth understanding of the issue– I want to understand what happens in one

particular area in-depth– I want to interview people– I want to consider differing perspectives– I don’t like numbers– PROBABLY QUALITATIVE

Qualitative or quantitative?

– All of the above!– MIXES METHODS APPROACH

Quantitative Methods

Quantitative research design

• Representing reality– Systematic e.g. transects– Random and random stratified (i.e. random within

different groups such as socio-economic classes or habitats)

Quantitative data collection

• Counting things…• Closed ended question surveys with large

samples e.g. via internet• Ecological and soil-based techniques e.g.

chemical analysis or counting plants in quadrats

Quantitative data analysis

– Descriptive statistics e.g. mean, median, standard deviation, percentages

– Parametric statistics (sample size >50, not too much variation)

• Significant differences e.g. T-Test• Correlations e.g. regression• Multi-variate e.g. multiple-regression, ordination

– Non-parametric (sample size <50, lots of variation)• Significant differences e.g. Mann Whitney U• Correlations e.g. Pearson Product Moment Correlation

Qualitative Methods

Qualitative research design

• Purposive sampling– Selecting respondents on the basis of pre-defined categories that

cover key aspects of your research question• Snowball sampling

– Keep interviewing within a category till no new ideas– Get respondents to recommend others for you to interview

• Case studies– Common in qualitative research– In-depth understanding of a particular case from which you may be

able to generalise more widely– Multiple cases representing different perspectives, locations or

components of your issue

Qualitative data collection

• Understanding the quality/nature of things…• Open ended question surveys with large

samples e.g. via internet• Semi-structured interviews with small samples

(e.g. 12-20 people)• Participant observation – transcripts and

behaviour• Make sure you get informed consent from

respondents

Qualitative data analysis

– Different types of content analysis and ways of summarising large bodies of text

• Key word counts (aggregating synonyms)• Coding for themes – preset or emergent (Grounded Theory

Analysis)• Discourse analysis to capture context and power relations• Recursive abstraction – summarising and summarising

summaries and so on, to reach core themes

Triangulation

• Simply “checking” your data and interpretation of results

• Commonly used to increase the reliability of qualitative studies

• Is there another way of collecting data to answer the same question a different way?– Follow your interviews with a focus group– Follow up historical documents to check an oral

history

Summary

• Primary or secondary?• Qualitative or quantitative?

– Research design– Data collection methods– Analysis methods– Triangulation