Evidence Based Evidence Based Chiropractic Chiropractic April 3, 2003April 3, 2003

Research designs & their Research designs & their purposespurposes

Why Chiropractic Research ?Why Chiropractic Research ?

To improve the science of chiropractic To improve the science of chiropractic

To improve the profession of To improve the profession of chiropracticchiropractic

To add to mankind’s store of To add to mankind’s store of knowledge knowledge

The Building Blocks of TheoryThe Building Blocks of Theory

Concepts Concepts Operational DefinitionsOperational DefinitionsPropositionsPropositions

ConceptsConcepts

Abstractions that allow Abstractions that allow classification of observationsclassification of observations

When scalar values can be When scalar values can be assigned, they may become assigned, they may become variablesvariables

Variables must be Variables must be operationally operationally defineddefined

Operational DefinitionsOperational Definitions

Description/delineation of the Description/delineation of the exact procedures for exact procedures for measuring or observing the measuring or observing the phenomenon, event or phenomenon, event or behavior.behavior.

PropositionsPropositions

Propositions state the Propositions state the nature of nature of the relationshipthe relationship between variables between variables (concepts).(concepts).

An An hypothesishypothesis is a statement about is a statement about the expected relationship between the expected relationship between two or more concepts that is based two or more concepts that is based on a theory and that on a theory and that can be testedcan be tested..

Chiropractic PropositionsChiropractic Propositions

AdjustmentAdjustment

SubluxationSubluxation

HealthHealth

Chiropractic Proposition IChiropractic Proposition I

Between Between subluxationsubluxation and and healthhealth::

The The >> the quantity, quality, the quantity, quality, severity of subluxations, theseverity of subluxations, the < < health.health.

Chiropractic Proposition IIChiropractic Proposition II

Between Between adjustmentadjustment and and subluxationsubluxation::

The The >> the quantity, (etc) of the quantity, (etc) of adjustment, the adjustment, the << subluxation. subluxation.

Chiropractic Proposition IIIChiropractic Proposition III

Between Between adjustmentadjustment and and healthhealth::

The The >> the quantity, (etc) of the quantity, (etc) of adjustment, the adjustment, the >> the health. the health.

Chiropractic vs. Spinal Chiropractic vs. Spinal ManipulationManipulation

Chiropractic is a professionChiropractic is a profession Study requires sociologic, historic, Study requires sociologic, historic,

economic and health services economic and health services methodsmethods

Spinal manipulation is a family of Spinal manipulation is a family of treatment procedurestreatment procedures

Study requires epidemiologic and Study requires epidemiologic and physiologic methodsphysiologic methods

What do you mean “It What do you mean “It Works”Works”

PatientPatient– convenient; effective, satisfyingconvenient; effective, satisfying

ClinicianClinician– efficient, effective, safe, satisfyingefficient, effective, safe, satisfying

Clinical ScientistClinical Scientist– quantify effect size, efficacy quantify effect size, efficacy

compared to placebo, no tx, safetycompared to placebo, no tx, safety

Why critical appraisal?Why critical appraisal? Each study contributes to the Evidence BaseEach study contributes to the Evidence Base Rarely is one study considered definitive Rarely is one study considered definitive

‘proof’‘proof’ Lesson: do not draw overarching conclusions Lesson: do not draw overarching conclusions

from one piece of evidencefrom one piece of evidence– Consider possible methodological flawsConsider possible methodological flaws– Consider other explanations, confoundersConsider other explanations, confounders– Appropriate use and interpretation of Appropriate use and interpretation of

statisticsstatistics– Study design appropriate for answering the Study design appropriate for answering the

research questionresearch question– Overall, the evidence base will overcome Overall, the evidence base will overcome

each flaweach flaw

Basic VocabularyBasic Vocabulary

Observational (descriptive) vs. Observational (descriptive) vs. experimentexperiment

cross-sectional vs. longitudinalcross-sectional vs. longitudinal prospective vs. retrospectiveprospective vs. retrospective Subjects/patients/participantsSubjects/patients/participants variables (predictor vs. outcome vs. variables (predictor vs. outcome vs.

confounding)confounding) Sample vs. populationSample vs. population

Basic Study DesignsBasic Study Designs DescriptiveDescriptive

Document & communicate experienceDocument & communicate experience– Share ideas, programs, treatments, unusual events Share ideas, programs, treatments, unusual events – Begin search for explanationsBegin search for explanations

ExplanatoryExplanatoryExamine etiology, cause, efficacy, using the Examine etiology, cause, efficacy, using the

strategy of comparisonsstrategy of comparisons

– ExperimentalExperimentalEvaluate efficacy of therapeutic, educational Evaluate efficacy of therapeutic, educational

or administrative interventionsor administrative interventions Investigator controls allocationInvestigator controls allocation

– ObservationalObservationalSeek etiologies, predictors, better diagnosesSeek etiologies, predictors, better diagnoses Investigator observes natureInvestigator observes nature

Basic Study Designs - ExamplesBasic Study Designs - Examples DescriptiveDescriptive

Case Report or seriesCase Report or seriesPopulation (prevalence)Population (prevalence)Document experiences of a classroom courseDocument experiences of a classroom course

ExplanatoryExplanatory– ExperimentalExperimental

Clinical TrialClinical TrialEducational InterventionEducational Intervention

– ObservationalObservationalCase-ControlCase-ControlFollow-upFollow-upSurveySurvey

Observational StudiesObservational Studies

Not designed to answer “cause & Not designed to answer “cause & effect” questionseffect” questions

So why not just do RCTs?So why not just do RCTs?– Unethical unless less-invasive studies Unethical unless less-invasive studies

indicate plausible associationindicate plausible association– RCT methodology sometimes a difficult ‘fit’RCT methodology sometimes a difficult ‘fit’

Usual care vs. a ‘lab’ settingUsual care vs. a ‘lab’ settingTreating a condition vs. optimizing healthTreating a condition vs. optimizing healthOutcomes measures – how to measure ‘better Outcomes measures – how to measure ‘better

than okay’than okay’Dose/responseDose/response

Clinical Trial – Clinical Trial – an experiment in which the investigator an experiment in which the investigator manipulates or controls an interventionmanipulates or controls an intervention

AdvantagesAdvantages– Gold standard for Gold standard for

evaluating evaluating effectivenesseffectiveness of of a procedurea procedure

– Strongest Strongest evidence of evidence of causalcausal linkages linkages

DisadvantagesDisadvantages– Expensive & time Expensive & time

consumingconsuming– Difficult format for Difficult format for

some questionssome questions– Not good for rare Not good for rare

diseasesdiseases

Case Report – Case Report – Observational. Interesting characteristics in a Observational. Interesting characteristics in a

single patient are described in detailsingle patient are described in detail

AdvantagesAdvantages– May lead to May lead to

formalized research formalized research to ID causes & to ID causes & interventionsinterventions

– May lead to May lead to detection of unusual detection of unusual patterns of diseasepatterns of disease

DisadvantagesDisadvantages– Interesting Interesting

findings may be findings may be due to chancedue to chance

– Cannot Cannot generalize generalize findingsfindings

Case Series – Same advantages & disadvantages, definition = interesting characteristics of a series of patients are described in detail

Case Control –Case Control –Observational. Cases (diseased) & controls Observational. Cases (diseased) & controls (disease-free) are compared to tease out (disease-free) are compared to tease out

differences in their hx.differences in their hx. AdvantagesAdvantages

– Efficient design, esp. Efficient design, esp. for rare diseasesfor rare diseases

– Relatively Relatively inexpensiveinexpensive

– Can use to study Can use to study preliminary preliminary hypotheseshypotheses

DisadvantagesDisadvantages– Depends of quality Depends of quality

of extant recordsof extant records– Difficult to identify Difficult to identify

appropriate control appropriate control groupsgroups

DiseaseNo

Disease

Exposed NotExposed

Exposed NotExposed

Design of a case-control study

Cross-sectional –Cross-sectional –Observational. Characteristics of participants are Observational. Characteristics of participants are

studies at a point in time to look for assoc. studies at a point in time to look for assoc. between disease and possible causes.between disease and possible causes.

AdvantagesAdvantages– Preliminary Preliminary

investigationsinvestigations– Relatively Relatively

inexpensive & easyinexpensive & easy– Estimating Estimating

prevalenceprevalence

DisadvantagesDisadvantages– Temporality of events Temporality of events

unknownunknown– May lead to erroneous May lead to erroneous

conclusion if sequence conclusion if sequence of events is of events is undeterminedundetermined

Cross-sectional studiesCross-sectional studies Surveys, questionnaires, interviewsSurveys, questionnaires, interviews A snapshot in timeA snapshot in time Exposure and Outcome assessed at the Exposure and Outcome assessed at the

same timesame time Cannot determine chronologyCannot determine chronology Useful as early study to investigate Useful as early study to investigate

phenomenaphenomena– A A made-upmade-up example: Surveyed people under example: Surveyed people under

chiropractic care engage in more vigorous chiropractic care engage in more vigorous forms of exercise than do people not under forms of exercise than do people not under chiropractic carechiropractic care

– What can we conclude?What can we conclude?

Prospective cohort –Prospective cohort –Observational. Exposed & unexposed individuals are Observational. Exposed & unexposed individuals are

followed; occurrence of new disease is studied to examine followed; occurrence of new disease is studied to examine whether occurrence is associated with exposurewhether occurrence is associated with exposure

AdvantagesAdvantages– Best observational Best observational

study for:study for: PossiblePossible cause of disease cause of disease Course of diseaseCourse of disease Risk factors for diseaseRisk factors for disease

DisadvantagesDisadvantages– ExpensiveExpensive– Patient attritionPatient attrition– Takes a long timeTakes a long time– Not suited for rare Not suited for rare

or disease with long or disease with long latencylatency

– Does not Does not establishestablish cause because cause because there is no there is no interventionintervention

ExposedNot

Exposed

DevelopDisease

Do NotDevelopDisease

DevelopDisease

Do NotDevelopDisease

Design of a cohort study

Retrospective cohort –Retrospective cohort –Observational. Exposed & unexposed individuals are Observational. Exposed & unexposed individuals are followed retrospectively; occurrence of disease during followed retrospectively; occurrence of disease during

retrospective period is studied to examine whether retrospective period is studied to examine whether occurrence is associated with exposureoccurrence is associated with exposure

AdvantagesAdvantages– Best observational Best observational

study for:study for: PossiblePossible cause of disease cause of disease Course of diseaseCourse of disease Risk factors for diseaseRisk factors for disease

DisadvantagesDisadvantages– Depends on quality Depends on quality

of recordsof records– ExpensiveExpensive– Not suited for rare Not suited for rare

or disease with long or disease with long latencylatency

– Does not Does not establishestablish cause because cause because there is no there is no interventionintervention

Broad topics of researchBroad topics of research: most : most research studies are concerned research studies are concerned

with one or more of the following:with one or more of the following:

Intervention / Therapy / ModalityIntervention / Therapy / Modality Assessment / Diagnosis /ScreeningAssessment / Diagnosis /Screening PrognosisPrognosis CausationCausation

Intervention / Therapy / ModalityIntervention / Therapy / Modality

Testing efficacy of patient careTesting efficacy of patient care– Spinal manipulationSpinal manipulation– Drug therapyDrug therapy– Surgical procedureSurgical procedure

Preferred study design in randomized Preferred study design in randomized controlled trialcontrolled trial

Assessment / Measurement / DiagnosisAssessment / Measurement / Diagnosis

Is a diagnostic test valid (can we trust it?)Is a diagnostic test valid (can we trust it?) Is it reliable (would we get the same Is it reliable (would we get the same

results every time?)results every time?) Preferred study design is cross-sectionalPreferred study design is cross-sectional

– Both the new test and the gold standard test Both the new test and the gold standard test are performed and comparedare performed and compared

PrognosisPrognosis

What is likely to happen to someone What is likely to happen to someone with a certain stage of conditionwith a certain stage of condition

Preferred study design is Preferred study design is longitudinal longitudinal cohortcohort– A special type of cohort study to A special type of cohort study to

investigate prognoses investigate prognoses – Early stage or (+) screening test Early stage or (+) screening test

patients = The Cohortpatients = The Cohort– The cohort is followed up repeatedly forThe cohort is followed up repeatedly for

Incidence (new cases per year)Incidence (new cases per year)Time course of different outcomesTime course of different outcomes

CausationCausation

– Causation is first investigated by Causation is first investigated by determining association (link) between determining association (link) between exposure and outcomeexposure and outcomeSmoking / lung cancerSmoking / lung cancerChiropractic care / optimal healthChiropractic care / optimal healthSeatbelt use / MVC injuriesSeatbelt use / MVC injuries

– Cohort or case-control studyCohort or case-control studyDepends upon how rare the outcome isDepends upon how rare the outcome isCase reports may also provide crucial Case reports may also provide crucial

preliminary informationpreliminary information

Smoking causes lung cancerSmoking causes lung cancer Where is the RCT that gave evidence Where is the RCT that gave evidence

to this?to this? Decades of observational research Decades of observational research

– Observing the same thing in a wide Observing the same thing in a wide variety of settingsvariety of settings

– Some free of some types of bias, others Some free of some types of bias, others free of other types of biasfree of other types of bias

It takes It takes a lota lot of observational data to of observational data to even begin to suggest causation!even begin to suggest causation!– Strength of association – Consistency – Strength of association – Consistency –

Specificity – Temporality – Biologic Specificity – Temporality – Biologic gradient (dose/response) – Biologic gradient (dose/response) – Biologic plausibility – Coherence of evidenceplausibility – Coherence of evidence

SampleSample One is interested in the characteristics One is interested in the characteristics

of a population but must, for practical of a population but must, for practical reasons, estimate them by describing a reasons, estimate them by describing a samplesample

A subset of a populationA subset of a population Selected from the populationSelected from the population

– 100,000 randomly selected US residents100,000 randomly selected US residents– LBP patients recruited for a clinical trialLBP patients recruited for a clinical trial– HMO members randomly selected from HMO members randomly selected from

filesfiles– Students enrolled in the EBC classStudents enrolled in the EBC class

Figure 1.3

Populations and SamplesPopulations and Samples

Who was the intended population Who was the intended population represented by the sample?represented by the sample?

PopulationPopulation– Large groups of people in a defined Large groups of people in a defined

setting or with a certain setting or with a certain characteristiccharacteristicThe general populationThe general populationAdults with low back painAdults with low back painResidents of North CarolinaResidents of North CarolinaMembers of a California HMOMembers of a California HMOChiropractic studentsChiropractic students

RandomizationRandomization

Random (lay definition): having no Random (lay definition): having no specific pattern or objective; lacking specific pattern or objective; lacking causal relationships; haphazardcausal relationships; haphazard

Random (scientific definition): A Random (scientific definition): A selection or assignment process in selection or assignment process in which there is associated with every which there is associated with every legitimate outcome a known legitimate outcome a known probabilityprobability

Random selectionRandom selection

How a sample is drawn from a How a sample is drawn from a population for a studypopulation for a study

Example: randomly draw 100 files Example: randomly draw 100 files from a population of 1000 patientsfrom a population of 1000 patients– vs. taking the first 100 on the listvs. taking the first 100 on the list

Sampling is related to external Sampling is related to external validity & generalizability of resultsvalidity & generalizability of results

Randomly sample so that research Randomly sample so that research participants better represent the participants better represent the larger grouplarger group

Random assignmentRandom assignment How you assign the sample that you How you assign the sample that you

draw to different groups or treatmentsdraw to different groups or treatments Randomly assign 50 of 100 subjects to Randomly assign 50 of 100 subjects to

a group or treatmenta group or treatment Related to study designRelated to study design Related to internal validity – to ensure Related to internal validity – to ensure

that treatment groups are similar to that treatment groups are similar to each other prior to treatmenteach other prior to treatment

Randomization principlesRandomization principles

Protects against selection biasProtects against selection bias Expected degree of baseline Expected degree of baseline

comparability for an unobserved comparability for an unobserved variable is the same as for an variable is the same as for an observed variableobserved variable

Can have one without the otherCan have one without the other

Can randomly assign a nonrandom Can randomly assign a nonrandom sample to treatment versus controlsample to treatment versus control

Can randomly select from a Can randomly select from a population, than nonrandomly population, than nonrandomly (arbitrarily) assign to treatment or (arbitrarily) assign to treatment or controlcontrol

Absence of evidence is not Absence of evidence is not evidence of absence*evidence of absence*

RCT’s are intended to statistically RCT’s are intended to statistically detect a difference between groups detect a difference between groups if if there is onethere is one

What if p>0.05?What if p>0.05?– A ‘negative’ study? A ‘negative’ study? NoNo– ‘‘Evidence’ that the treatments are Evidence’ that the treatments are

equivalent? equivalent? NoNo– Only: “There is no evidence that the Only: “There is no evidence that the

groups are different”groups are different”*Altman and Bland, BMJ 1995;311:485 (19 *Altman and Bland, BMJ 1995;311:485 (19

August).August).

*Whew!**Whew!*

Next weekNext week

Discuss Bland Altman articleDiscuss Bland Altman article Statistical and clinical significanceStatistical and clinical significance Dr. Cyndy Long, BiostatisticianDr. Cyndy Long, Biostatistician

– What can we really say about the What can we really say about the results?results?

Please: email me Please: email me BEFOREBEFORE the the midterm to arrange a make-upmidterm to arrange a make-up