Effectiveness in Review & Oversight of Effectiveness in Review & Oversight of Human Subjects ResearchHuman Subjects Research

Steven Joffe, MD, MPHAssistant Professor of Pediatrics

Human Subjects Research (HSR) Human Subjects Research (HSR) Review & OversightReview & Oversight

• Complex system for ensuring high-quality, safe, respectful research– Investigators, sponsors, institutions, IRBs, DSMBs,

etc.

• Primary responsibility for independent research review & oversight lies with IRB & its staff

Defining Quality Defining Quality (or Effectiveness) in HSR(or Effectiveness) in HSR

– Health care (Institute of Medicine): “the degree to which health services for individuals & populations increase the likelihood of desired health outcomes & are consistent with current professional knowledge”

– Research oversight (by analogy): “the degree to which research review & oversight activities increase the likelihood that HSR will satisfy essential scientific & ethical requirements, & are consistent with current professional knowledge & best practice”

JAMA 1998;280:1000

What Are The “Essential Requirements?”What Are The “Essential Requirements?”

Belmont PrinciplesBelmont Principles

• Respect for persons– Informed consent– Protections for vulnerable

subjects• Beneficence

– Minimization and justification of research risk

• Justice– Fairness in distribution of

burdens and benefits

• Respect for persons– Informed consent– Protections for vulnerable

subjects• Beneficence

– Minimization and justification of research risk

• Justice– Fairness in distribution of

burdens and benefits

Six Substantive RequirementsSix Substantive Requirements

• Social value• Scientific validity• Fair distribution of benefits

& burdens• Favorable risk-benefit ratio• Informed consent• Respect for potential &

enrolled subjects

• Social value• Scientific validity• Fair distribution of benefits

& burdens• Favorable risk-benefit ratio• Informed consent• Respect for potential &

enrolled subjects

JAMA 2000;283:2701

Conceptual Model of Conceptual Model of Quality/EffectivenessQuality/Effectiveness

StructureStructure

ProcessProcess

OutcomeOutcome

JAMA 1988;260:1743

Definitions of TermsDefinitions of Terms

• Structure– Attributes of settings in which review is conducted

• Material & human resources; organizational structure; written policies & procedures

• Process– Activities of review & oversight bodies

• Outcome– Ultimate ends that review and oversight system seeks to

achieve

ExampleExample

Consent form editor on IRB staff

Consent form editor on IRB staff

Editing of consent form for clarity, readability

Editing of consent form for clarity, readability

Understanding among individuals considering

research

Understanding among individuals considering

research

Priorities for the Priorities for the Quality & Effectiveness AgendaQuality & Effectiveness Agenda

1. Define key outcomes

2. Establish relationships among structure, process & outcome

3. Decide what will be assessed, & how

4. In parallel, begin integrating quality improvement & assessment of effectiveness into IRB culture

1. Define Key Outcomes1. Define Key Outcomes

• Six substantive requirements for ethical research– E.g., social value value of individual studies & of

research portfolio

– E.g., informed consent participant understanding, voluntariness of decisions

– E.g., favorable risk-benefit ratio avoidance of preventable research-related harm

JAMA 2000;283:2701

1. Define Key Outcomes1. Define Key Outcomes

• Also:– Procedural & interactional fairness for investigators– Respect for review & oversight system among

investigators– Community & public trust– Efficiency?– Other? Consider stakeholder perspectives

• Substantive & procedural quality

JERHRE 2006;67BMC Med Eth 2008;9:6

1. Define Key Outcomes1. Define Key Outcomes

• Need to decide what the primary objective of the quality/effectiveness agenda is?– Seeking of optimal outcomes?

– Avoidance of bad outcomes?

2. Establish Structure-Process-2. Establish Structure-Process-Outcome RelationshipsOutcome Relationships

• Structure-process– E.g., how do IRB resources or written policies & procedures impact

quality or comprehensiveness of review?

• Structure-outcome– E.g., how do members’ expertise & training impact likelihood that

scientifically poor-quality protocols will be improved or disapproved?

• Process-outcome (most important)– E.g., how does attention to clarity of consent form impact likelihood

that prospective participants will make informed decisions?

3. Decide What Will Be Assessed3. Decide What Will Be Assessed

• Structure?– Easiest to quantify– Easiest to modify– But relationship to outcome must be validated,

and is likely attenuated

3. Decide What Will Be Assessed3. Decide What Will Be Assessed

• Process?– Most direct measure of IRB performance– But first need to validate which processes are

associated with desired outcomes– Proximal vs. distal process

3. Decide What Will Be Assessed3. Decide What Will Be Assessed

• Outcome?– Most concrete object of measurement– But numerous challenges and confounders:• Normative agreement on what counts• Valid measurement instrument• “Case mix”• Multiple causal influences on outcomes; research

review and oversight explains only part of variability

3. Decide What Will Be Assessed3. Decide What Will Be Assessed

• Ultimate goal

– Assess process variables that are known to be associated with outcome• “Best practice”• Evidence-based review and oversight

– Assess selected outcome measures directly• Limited to those most causally related to review and

oversight

4. In Parallel, Integrate QI & 4. In Parallel, Integrate QI & Evaluation into IRB Practice Evaluation into IRB Practice

• Self-evaluation

• Continuous or rapid-cycle QI– Plan, do, study, act

• Root cause/systems analysis– E.g., investigations into TGN1412 disaster

• Peer review

JERHRE 2008;3:25http://www.patientsafety.gov/CogAids/RCA/index.html

http://www.patientensicherheit.ch/de/projekte/londonprotocol_e.pdfBMJ 2006;333:270

What is the Evidence Base Today?What is the Evidence Base Today?

• Not much• Primarily related to informed consent and participant

understanding*

– Use of consent form templates; presence of 3rd party in consent conversation; time to consider decision; use of nurse educators; structured consent process

– Use of real-time tests of comprehension– Formal assessment of decision-making capacity

• Ethical evaluation of approved protocols is methodologically possible

*See last slide for references

SummarySummary

• Effectiveness agenda needs to start with conceptual model

• Long-term project: select outcomes, validate relationships to structure and process, and develop measures of key indicators

• Short-term project: begin to create culture of QI and of evidence-based research oversight

• Limited evidence base exists on which to build

ReferencesReferences

• Lancet 2001;358:1772• J Clin Oncol 1996;14:984• JAMA 2004;291:470• Control Clin Trials 1989;10:83• Pediatr 2007;119:e849• Schizophr Res 2005;80:1• Arch Gen Psychiatry 2000;57:533• Am J Psychiatr 2006;163:1323• Advisory Committee on Human Radiation Experiments. Final

Report. New York:OUP, 1996 (Chapter 15)