Presentation on Research designs:

Observational, Quasi-experiemental and

RandomizationBy

Lévis KAHANDUKYA NYAVANDA,( DipDES, BDES,MCHD, Candidate PhD).

Content

1. Randomised Control Trial (RCT)2. Quasi-Eperimental3. Observational4. Open discussion ( with the audience).

Randomized Control Trial (RCT)

RCT is a benchmark technique for a number of raisons:

- It has evidence at Level I and Level II- It implies random allocation ( area of the

study) and random sellection ( population for the study).

- It concern is allocation of volunteers to be part to one of two groups ie. Control and treatment group.

RCT

- It also implies verification whereby samples are compared in order to overcome bias.

- It also concludes by describing characteristics and difference among groups may only occurs due to chance.

Types of randomization

1. Simple randomisation: no restrition on allocation ( groups may be unequally sized);

2. Block randomisation: allocation is performed in blocks, so that groups are equally sized within each block;

3. Stratified randomisation: factors such as age, sex, social ladders are randomised separately, so that they are equally distributed among the groups.

4. Proportional systematic randomisation: allocation is performed proportionally with equal distribution of groups.

Randomized controlled trials (RCT)• A study design that randomly assigns

participants into an experimental group or a control group. As the study is conducted, the only expected difference between the control and experimental groups in a randomized controlled trial (RCT) is the outcome variable being studied.

• The randomized controlled trial is considered as the most rigorous method of determining whether a cause-effect relationship exists between an intervention and outcome .

• Basic outline of the design of a randomized controlled trial

Advantages and disadvantages.• Advantages • Good randomization will prevent any

population bias • Easier to blind/mask than observational

studies • Results can be analyzed with well known

statistical tools • Populations of participating individuals are

clearly identified

Disadvantages • Expensive in terms of time and money • Volunteer biases: the population that

participates may not be representative of the whole

• Does not reveal causation • Loss to follow-up attributed to treatment

QUASI EXPERIMENTAL DESIGN.• Quasi-experimental designs are commonly

used in the evaluation of programmes when random assignment is not possible or practical. Like experimental designs, quasi-experimental designs involve comparing the changes between one group that receive the programme and a no intervention control group. The decision of who receives the programme and who doesn’t is not random and is usually determined by either systematic allocation or convenience

Types of Experimental Design

• There are two basic types of research design:• True experiments• Quasi-experiments• The purpose of both is to examine the cause of

certain phenomena.• True experiments, in which all the important

factors that might affect the phenomena of interest are completely controlled, are the preferred design.

Types of Experimental Design Continuous….

• Often however, it is not possible or practical to control all the key factors, so it becomes necessary to implement a quasi-experimental research design.

• In a true experiment, participants are randomly assigned to either the treatment or the control group, whereas they are not assigned randomly in a quasi-experiment

Strengths of Quasi-Experimental Design

• Useful in generating results for general trends in social sciences.

• Easily integrated with individual case studies–Generated results can reinforce the findings in a

case study.–Allow statistical analysis to take place.

Strengths of Quasi-Experimental Design Continuous….

• Enable to reduce the time and resources required for experimentation.–Not required extensive pre-screening and

randomization. The context and way in which the intervention or programme is delivered is often more natural or more similar to ‘usual practice’ than in an RCT, meaning the results may be more generalizable to other schemes/settings.

Weakens of Quasi Experimental Design.

• The lack of random assignment is the major weakness of the quasi-experimental study design. Associations identified in quasi-experiments meet one important requirement of causality since the intervention precedes the measurement of the outcome. difficulty in measuring or controlling for important confounding variables, particularly unmeasured confounding variables, which can be viewed as a subset of the selection threat in .

Threats to Internal Validity

1. Ambiguous temporal precedence: Lack of clarity about whether intervention occurred before outcome

2. Selection: Systematic differences over conditions in respondent characteristics that could also cause the observed effect

3. History: Events occurring concurrently with intervention could cause the observed effect

4. Maturation: Naturally occurring changes over time could be confused with a treatment effect

Threats to Internal Validity Continuous…6. Regression: When units are selected for their extreme scores, they will often have less extreme subsequent scores, an occurrence that can be confused with an intervention effect7. Attrition: Loss of respondents can produce artifactual effects if that loss is correlated with intervention8. Testing: Exposure to a test can affect scores on subsequent exposures to that test9. Instrumentation: The nature of a measurement may change over time or conditions10. Interactive effects: The impact of an intervention may depend on the level of another intervention

Observation research method• Observation deals with actions and behavior.

If you want to find out what people do, you should observe them. If you want to find out what they think (e.g., attitudes, beliefs, expectations, or knowledge), you should ask them directly. Although there are exceptions, observation is generally the best method for studying natural behavior, while interviews and questionnaires are more appropriate for exploring opinions and beliefs.

Observation research method Continuous…• Reliability is always a problem in observation.

For systematic observation, the use of two independent observers is recommended during the early stages of the study. No matter how simple and straightforward the behavior being studied, it still is wise to check on reliability

Types research observation methods

• Controlled Observation• Controlled observations (usually a structured

observation) are likely to be carried out in a psychology laboratory. The researcher decides where the observation will take place, at what time, ch with which participants, in what circumstances and uses a standardised procedure. Participants are randomly allocated to each independent variable group.

Types research observation methods continuous…• Naturalistic Observation• Naturalistic observation (i.e. unstructured observation)

involves studying the spontaneous behaviour of participants in natural surroundings. The researcher simply records what they see in whatever way they can. Participant Observation

• Participant observation is a variant of the above (natural observations) but here the researcher joins in and becomes part of the group they are studying to get a deeper insight into their lives. If it were research on animals we would now not only be studying them in their natural habitat but be living alongside them as well!

Strengths and limitation of controlled observation method

Strengths1. Controlled observations can be easily replicated by other researchers by using the same observation schedule. This means it is easy to test for reliability.

2. The data obtained from structured observations is easier and quicker to analyze as it is quantitative (i.e. numerical) - making this a less time consuming method compared to naturalistic observations.

3. Controlled observations are fairly quick to conduct which means that many observations can take place within a short amount of time. This means a large sample can be obtained resulting in the findings being representative and having the ability to be generalized to a large population..

Continuous…• Limitations

1. Controlled observations can lack validity due to the Hawthorne effect/demand characteristics. When participants know they are being watched they may act differently.

2. By being able to observe the flow of behaviour in its own setting studies have greater ecological validity.

3. Like case studies naturalistic observation is often used to generate new ideas. Because it gives the researcher the opportunity to study the total situation it often suggests avenues of enquiry not thought of before.

Continuous….5. These observations are often conducted on a micro (small) scale and may lack a representative sample (biased in relation to age, gender, social class or ethnicity). This may result in the findings lacking the ability to be generalized to wider society.

6. Natural observations are less reliable as other variables cannot be controlled. This makes it difficult for another researcher to repeat the study in exactly the same way.

7. A further disadvantage is that the researcher needs to be trained to be able to recognise aspects of a situation that are psychologically significant and worth further attention.

8. With observations we do not have manipulations of variables (or control over extraneous variables) which means cause and effect relationships cannot be established.

Limitation of Participant Observation

• Limitations1. It can be difficult to get time / privacy for recording. For example, with covert observations researchers can’t take notes openly as this would blow their cover. This means they have to wait until they are alone and reply on their memory. This is a problem as they may forget details and are unlikely to remember direct quotations.

2. If the researcher becomes too involved they may lose objectivity and become bias. There is always the danger that we will “see” what we expect (or want) to see. This is a problem as they could selectively report information instead of noting everything they observe. Thus reducing the

Validity of their data.• Recording of Data .With all observation studies an important

decision the researcher has to make is how to classify and record the data. Usually this will involve a method of sampling. The three main sampling methods are:

• Event sampling. The observer decides in advance what types of behaviour (events) she is interested in and records all occurrences. All other types of behaviour are ignored.

• Time sampling. The observer decides in advance that observation will take place only during specified time periods (e.g. 10 minutes every hour, 1 hour per day) and records the occurrence of the specified behaviour during that period only.

• Instantaneous (target time) sampling. The observer decides in advance the pre-selected moments when observation will take place and records what is happening at that instant. Everything happening before or after is ignored.

References

1. Hennekens CH, Buring JE. Epidemiology in Medicine, Lippincott Williams & Wilkins, 1987.2. Kendall JM. Designing a research project: Randomised Controlled trials and their principles, Emerg Med J. 2003, March;20(2)164-168.3. Hollis S, Campbell F, What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ 1999; 319;670-74.Further Resources1. Pocock SJ. Clinical Trials: A practical approach, Chichester, Wiley, 1984.2. Sibbald B, Roland M, Understanding controlled trials: Why are randomised controlled trials important?, BMJ 1998, 316:201.3. Altman DG, Randomisation. BMJ 1991;302;1481-2.• ‹ Introduction to study designs - cohort studies up

Introduction to study designs - developing a questionnaire ›