System error

Biases in epidemiological studies

FETP India

Competency to be gained from this lecture

Prevent biases that may be avoided and understand the effect of others

Key elements

• Definition of biases• Selection biases• Information biases• Controlling the effect of biases

Misclassification

• Non-differential misclassification Error on outcome status independent from

exposure status Error on exposure status independent from

outcome status

• Differential misclassification (bias) Error influenced by outcome or exposure

status

Biases

Effect of misclassification

• Non-differential misclassification Usually reduces the strength of association

between outcome and exposure (RR or OR closer to 1)

Is not a sufficient reason to dismiss the results of a study reporting an association

• Differential misclassification (bias) May commonly increase the strength of the

association May also reduce the strength of the

associationBiases

Definition of a bias

Systematic error in the protocol of a study that leads to a distortion of measurement

affecting internal validity

How to understand the term systematic?• Error systematically done in the same way

Differential misclassification

• Error of the system Built in misconception of the protocol

Biases

Types of biases

• Selection biases Biases in the way subjects enter a study

• Information biases Biases in the way information is collected

after inclusion in a study

Biases

Level of operation of biases

Systemat ic recruitmentof cases / controls

w ith specifi c exposure status

Systemat ic col lect ion ofinformat ion leaning towards

specifi c exposure status

Col lect ion of informat ionabout exposure

Inclusionon the basis of outcome

Case control

Systemat ic recruitmentof exposed / unexposed

with specifi c r isk of outcome

Systemat ic col lect ion ofinformat ion leaning towards

specifi c outcome status

Col lect ion of informat ionabout outcome

Inclusionon the basis of exposure

Cohort

Type of study

Selection biases

Information biases

Biases

Selection biases

Type of studyInclusion into the

studySources of

selection biases

Case control•On the basis of outcome

•Selection of cases or controls with specific exposure status

Cohort•On the basis of exposure

•Selection of exposed or unexposed with specific risk

Selection biases

Examples of selection biases in a cohort study

• Subjects selected on the basis of exposure• Outcome not independently distributed

among exposed and unexposed Systematic selection of exposed subjects with:

• Higher risk of outcome • Lower risk of outcome

Systematic selection of unexposed subjects with:• Higher risk of outcome • Lower risk of outcome

A b

c d

a B

c d

a b

c D

a b

C d Selection biases

Examples of selection biases in a case control study

• Subjects selected on the basis of outcome• Exposure not independently distributed

among cases and controls Systematic selection of case-patients with:

• Higher frequency of exposure • Lower frequency of exposure

Systematic selection of control-subjects with:• Higher frequency of exposure • Lower frequency of exposure

A b

c d

a b

C d

a b

c D

a B

c d Selection biases

Sources of selection biases

• Surveillance• Screening / diagnosis• Admission to health care facilities • Selective survival• Non-response / loss to follow up

Selection biases

Sources of selection biases

• Surveillance Systematic notification of cases exposed

• Screening / diagnosis• Admission to health care facilities• Selective survival• Non-response / loss to follow up

Selection biases

Sources of selection biases

• Surveillance• Screening / diagnosis

Systematic case search among exposed

• Admission to health care facilities• Selective survival• Non-response / loss to follow up

Selection biases

Sources of selection biases

• Surveillance• Screening / diagnosis• Admission to health care facilities

Systematic admission of:• Case-patients exposed / unexposed• Control-subjects exposed / unexposed

• Selective survival• Non-response / loss to follow up

Selection biases

Sources of selection biases

• Surveillance• Screening / diagnosis• Admission to health care facilities • Selective survival

Systematic inclusion of cases who survived and who may be more or less exposed

• Non-response / loss to follow up

Selection biases

Sources of selection biases

• Surveillance• Screening / diagnosis• Admission to health care facilities • Selective survival• Non-response / loss to follow up

Systematic inclusion of subjects more likely to participate who may be:• More or less exposed• More or less at risk

Selection biases

Information biases

Type of studyCollection of information

Sources of information biases

Case control •About exposure

•Collection of information leaning towards specific exposure status

Cohort •About outcome

•Collection of information leaning towards specific outcome status

Information biases

Examples of information biases in a cohort study

• Subjects selected on the basis of exposure• Outcome not independently measured

among exposed and unexposed Measurement of a higher incidence of outcome:

• Among exposed• Among unexposed

Measurement of a lower incidence of outcome:• Among exposed• Among unexposed

A b

c d

a b

C d

a b

c D

a B

c d Information biases

Examples of information biases in a case control study

• Subjects selected on the basis of outcome• Exposure not independently measured

among cases and controls : Measurement of a higher frequency of exposure:

• Among cases• Among controls

Measurement of a lower frequency of exposure:• Among cases• Among controls

A b

c d

a B

c d

a b

c D

a b

C d Information biases

Sources of information biases

• Recall • Investigator• Data quality• Prevarication

Information biases

Sources of information biases

• Recall Cases may recall exposure more than

controls

• Investigator• Data quality• Prevarication

Information biases

Sources of information biases

• Recall • Investigator

Systematic collection of information supporting expected conclusions• Unconsciously• Consciously

May be examined in the analysis

• Data quality• Prevarication

Information biases

Sources of information biases

• Recall • Investigator• Data quality

Better exposure data available on cases Better outcome data available on exposed

• Prevarication

Information biases

Sources of information biases

• Recall • Investigator• Data quality• Prevarication

Systematic distortion of the truth by subjects

Information biases

Other limitations also referred to as biases

• Intervention biases Non-specific effect of interventions in a trial

• Analysis biases Non-systematic statistical analyses

• Interpretation biases Investigators with pre-conceived ideas

• Publication biases Negative studies do not get published

Checklist to prevent biases when preparing a protocol (1/2)

1. Is the sample representative? Are controls representative of the population

from which cases are drawn?

2. Are the exposure and outcome criteria: Standardized? Specific? Clear? Well measured? Applied consistently by trained field

workers?Controlling biases

Checklist to prevent biases when preparing a protocol (2/2)

3. Could exposure status affect the probability to detect the outcome?

4. Could outcome status affect the probability to recall the exposure?

5. What will be the proportion of response?

6. What will be the proportion of “lost to follow up”?

Controlling biases

Discussing biases critically

• Review critically selection and information biases that are likely to have operated

• Review the potential effect of each of these biases (e.g., under- or overestimation)

• Review what can be done to address these biases a posteriori (e.g., analysis plan)

• Analyze possible summary effect of all these biases on the final results

• Interpret data in light of this summary effect

Controlling biases

Take-home messages

• Have a strong design that prevents biases

• Examine how subjects enter a study• Collect information in a standardized

way• Prevent the biases you can avoid,

understand those you cannot avoid