Research Design: Using Quantitative Methods

Objectives

By the end of this session you will be able to:

• Describe the experimental and quasi-experimental research approaches.

• Formulate appropriate questions and hypotheses.

• Identify populations and samples.• Describe the principles of research tool

design (validity and reliability).

Stages in the experimental design process

Identifying issues

‘Good’ research topics might emerge:

• From the literature.

• Within workplace settings.

• From previous projects/assignments.

• From a sponsor.

Review the literature

The literature review of the

research topic

What are the key sources?

What are the major issues and debates about the topic?

What are the origins and definitions of the

topic?

What are the key theories, concepts

and ideas?

What is the epistemological and ontological basis for

the subject?

What are the main questions/problems

that have been addressed to date?

Source: adapted from Hart (1998)

Develop questions/hypotheses

Kerlinger and Lee (2000) argues that a good research question:

• Expresses a relationship between variables (e.g., company image and sales levels).

• Is stated in unambiguous terms in a question format, and …

• Must be capable of being operationally defined (Black, 1993).

Types of applied research questions – with examples

Type of research question Example

Descriptive How common is drug use amongst university students?

Normative How serious is drug abuse amongst university students

Correlation What is the relationship between gender, academic performance and drug use amongst university students?

Impact Has the drug awareness campaign had any impact on the level of university student drug use?

A hypothesis

• Is a speculative statement of the relation between two or more variables.

• Describes a research question in a testable format which predicts the nature of the answer.

• Can be written as a directional statement, such as, ‘When this happens, then that happens’.

Identifying independent and dependent variables

• Dependent variables - a variable that forms the focus of research, and depends on another (the independent or explanatory) variable.

• Independent variable - used to explain or predict a result or outcome on the dependent variable.

• Intervening variable – a hypothetical internal state, used to explain relationship between two observed variables.

.

Conducting the study

• Planning the design.• Gathering data.• Storing data.• Observing ethical

guidelines.

Using descriptive and inferential statistics

0

10

20

30

40

50

60

70

80

90

1stQtr

2ndQtr

3rdQtr

4thQtr

East

West

North

• T-test data• Mann-Whitney U data• Chi-square data• Spearman’s rho data• Pearson Product

Moment data• ANOVA

Descriptive statistics Inferential statistics e.g.

Accept or reject the hypothesis

• A hypothesis cannot be ‘proved’ to be right – all theories are provisional/tentative (until disproved).

• Acceptance or rejection of the hypothesis based upon the weight of statistical evidence and probability entails: – The risk of accepting the hypothesis as true

(when it is in fact false).– The risk of rejecting the hypothesis as false

(when it is in fact true).

Preparing the formal report

Why the study was conducted

What research questions and hypotheses were evaluated

How questions were turned into a research design

What differences were observed between the hypotheses

and the resultsWhat conclusions can be drawn – do these support or contradict the hypothesis and existing theories?

Experimental design

• The researcher has control over the experiment in terms of:– Who is being researched (subjects randomly

assigned).– What is being researched.– When the research is to be conducted.– Where the research is to be conducted.– How the research is to be conducted.

Typically, researchers often have no control over the ‘who’, having to use pre-existing groups – hence, a quasi-experimental

design.

Quasi-experimental designs

Quasi-experimental designs are best used when:• Randomization is too expensive, unfeasible to

attempt or impossible to monitor closely.• There are difficulties, including ethical

considerations, in withholding the treatment.• The study is retrospective and the programme

being studied is already underway.

Differences in quantitative research design

Differences between experimental, quasi-experimental and non-experimental research

Faulty quantitative designs

One group, pre-test/post-test problems:• Maturation effects• Measurement procedures• Instrumentation• Experimental mortality• Extraneous variables

Sound quantitative designs (1)

Experimental group with control

Sound quantitative designs (2)

Quasi-experimental design with non-equivalent control

Generalizing from samples to populations

To generalize, samples must be representative of the population, through:

• Random probability sampling (but note problem of sampling error).

Types of probability sample

• Simple random sample (where the sampling frame is equal to the population).

• Stratified random sample (sampling from strata according to some characteristic e.g., geographical area, age, gender).

• Cluster sample (e.g., a county, households in a street, schools in a town, etc.)

• Stage sample (cluster sample followed by random selection from cluster).

Non-random sampling

• Purposive: Subjects selected against one or more trait.

• Quota: Non-random selection of subjects from identified strata until the planned number of subjects is reached.

• Convenience or volunteer.• Snowball: Researcher identifies a small

number of subjects, who, in turn, identify others in the population.

Instrument design: validity

• Internal validity: The extent to which changes in the dependent variable can be attributed to the independent variable.

• External validity: This is the extent to which it is possible to generalize from the data to a larger population or setting.

• Criterion validity: How people have answered a new measure of a concept, with existing, widely accepted measures of a concept .

• Construct validity: The measurement of abstract concepts and traits, such as ability, anxiety, attitude, knowledge, etc.

Instrument design: reliability

Reliability is the consistency between two measures of the same thing such as:

• Two separate instruments.• Two like halves of an instrument (for

example, two halves of a questionnaire).• The same instrument applied on two

occasions.• The same instrument administered by two

different people.

Summary• Experimental research generally comprises two stages: the planning stage

and the operational stage.• Experimental research begins from a priori questions or hypotheses that the

research is designed to test. • Research questions should express a relationship between variables. A

hypothesis is predictive and capable of being tested.• Dependent variables are what experimental research designs are meant to

affect through the manipulation of one or more independent variables.• In a true experimental design the researcher has control over the

experiment: who, what, when, where and how the experiment is to be conducted. This includes control over the who of the experiment – that is, subjects are assigned to conditions randomly.

• Where any of these elements of control is either weak or lacking, the study is said to be a quasi-experiment.

• In true experiments, it is possible to assign subjects to conditions, whereas in quasi-experiments subjects are selected from previously existing groups.

• Research instruments need to be both valid and reliable. Validity means that an instrument measures what it is intended to measure. Reliability means that an instrument is consistent in this measurement