Science to Improve the Nation‘s Health Care:

The Case for Comparative Effectiveness

Research

Michael S Lauer, MD, FACC, FAHA

Director, Division of Cardiovascular Sciences

NHLBI/NIH

May 5, 2011

Disclosures

My immediate family and I have NOT

received anything of value related to the

technologies and topics being presented.

I will present my personal views, which are

not necessarily those of NHLBI, NIH, or

DHHS.

I am neither an economist nor a politician.

2

So What is CER?

C = Comparative

Real contest

Existing options

Effectiveness = Outcomes

Clinical outcomes: mortality, morbidity, major

clinical events, costs

Systems outcomes: adherence to guidelines

Research = Science

Observations, experiments, syntheses

3 Mitka M JAMA 2010;304:2229-31

CER Legislation and Health Care Reform

ARRA

Allocation $1.1 billion

Federal Coordinating Council for CER

IOM issued report on priorities June 2009

New Patient-Centered Outcomes Research

Institute (―PCORI‖)

Non-government, non-profit

BOG – 21 members including NIH, AHRQ

Methodology committee

4

5

Screaming Headlines!

―It's hard not to scream when you see how many physicians,

pharmaceutical companies, [and] medical-device

makers…seem to hate science, or at best ignore it. These

days the science that inspires fear and loathing is

‗comparative-effectiveness research.‘‖

Sharon Begley, Newsweek, Feb 28 2009

What‘s Going On?

Scaremongers have morphed effectiveness research into

cost-benefit analysis…Effectiveness research will

―‗threaten…lives‘ as government decides ‗who gets

lifesaving treatment and who doesn't.‘ Sen. Tom

Coburn (a doctor) warned of ‗a Soviet-style Federal

Health Board that will put bureaucrats and politicians in

charge of our nation's health-care system.‘‖

You might attribute Coburn's rant to his small government

ideology, but I say blame his profession—not politics

but medicine. Doctors have long resisted having

science guide their practice.

6 Sharon Begley, Newsweek, Feb 28 2009

What‘s Going On?

―Only a limited amount of evidence is available

about which treatments work best for which

patients and whether the added benefits of

more-effective but more-expensive services

are sufficient to warrant their added costs—

yet current practice tends to adopt more-

expensive treatments even in the absence of

rigorous assessments of their impacts….‖

Peter Orszag

7 Congressional Budget Office 2007

A Story of a Popular Therapy

8 http://explorepahistory.com/cms/pbfiles/Project1/Scheme34/ExplorePAHistory-a0j4x4-a_349.jpg

http://upload.wikimedia.org/wikipedia/commons/0/01/Blood_letting.jpg

Come Down to the Contest…

―Come down to the contest ye Humorists: Let us

take out of the Hospitals or the camps or

elsewhere, 200, or 500 poor People, that have

Fevers etc. Let us divide them in Halfes, let us

cast lots, that one half of them may fall to my

share and the other to yours; I will cure them

without bloodletting…; but do you do as ye

know. We shall see how many Funerals both of

us shall have: But let the reward of the

contention or wager, be 300 Florens, deposited

on both sides: Here your business is decided.‖

9 Van Helmont JA. Oriatrike. London: Lodowick-Loyd, 1662, p.526

The Trial is Reported…

10

―It had been so arranged, that this number was

admitted, alternately, in such a manner that

each of us had one third of the whole. The sick

were indiscriminately received, and were

attended as nearly as possible with the same

care and accommodated with the same

comforts.

Neither Mr Anderson nor I ever once employed

the lancet. He lost two, I four cases; whilst out

of the other third [treated with bloodletting by

the third surgeon] thirty five patients died.”

Milne I and Chalmers I. J Epidemiol Community Health 2002;56:1a

Over 100 Years Later…

11

―During the last

decades we have

certainly bled too

little.‖

William Osler, MD

David Sackett, Gairdner-Wightman Award, March 31, 2009 (www.cebm.net)

Modern Examples of Bloodletting

Thalidomide (birth defects)

Vitamins to prevent cancer/CVD (failed)

PSA for prostate cancer (over-diagnosis)

Anti-arrhythmic drugs (higher death rate)

Hormone replacement (cancer, strokes)

Stents after myocardial infarction

Bone marrow transplantation for breast

cancer (higher death rate)

12 Thank you to Andrew Epstein

Another Story: Stents After Acute MI

13http://www.circulation.or.kr/info/case/200904/fig1.gif

http://www.indiastudychannel.com/attachments/Resources/82666-221135-Coronary%20Angiogram.jpg

Observational Findings

14 Lamas GA et al. Circulation 1995;92:1101-9

They Ran Into a Problem…

When researchers tried to organize a randomized study of the benefits of angioplasty for patients who had suffered a heart attack three days or more before, they ran into a problem. Many doctors were so convinced of the value of this procedure…that they thought it would be unethicalto assign any patients to the control group, which would get all the best medicines for this condition but not the artery-reopening procedure.

But the researchers persisted, with heavy support from the National Heart, Lung, and Blood Institute. After four years of work examining 2,166 patients, they came to an unexpected conclusion….

15 Boston Globe, December 9, 2006

The Findings and Response

16

ACC/AHA STEMI Guideline: ―PCI of a totally occluded infarct artery greater

than 24 hours after STEMI is not recommended in asymptomatic

patients…[who] do not have evidence of severe ischemia.‖

Hochman JS. N Engl J Med 2006;355:2395-407

Antman EM et al. JACC 2008;51:210-47

High-Dose Chemotherapy and ABMT

17

―…We believe that confirmation of these results in a

prospective randomized trial is important before this therapy

can be accepted for widespread use. Many new therapies,

initially promising, fizzle. This treatment should only be

offered at major centers…and, whenever possible, [into]

randomized comparative trials…‖

Peters WP, et al. J Clinical Oncology 1993;11:1132-43

What Actually Happened

18

―… By the time Peters had organized his trial, few women

wanted to participate…[It] meant running the risk of not

getting high-dose chemo, and many had read newspaper

accounts that convinced them that the treatment was their

only chance for survival. Their doctors often agreed. One

transplanter pulled out a copy of Peters' 1993 paper. ‗I don't

see how it's even ethical to do a randomized trial,‘ he said.‖

Brownlee S. Discover Magazine 2002.

What Actually Happened; Large RCT

19

―… From the moment Peters first administered high-dose chemotherapy

until the first clinical trials were concluded, nearly 20 years passed.

During that time, hundreds of physicians practiced the unproven

treatment. An estimated 30,000 breast cancer patients suffered through

high-dose chemotherapy, only a fraction of them as part of a clinical

trial. All told, the nation spent around $3 billion paying for it, while an

estimated 4,000 to 9,000 women died not from their cancer but from the

treatment…‖

Brownlee S. Discover Magazine 2002.

Peters WP et al. J Clinical Oncology 2005;23:2191-2200

Let‘s Look at These Stories

Bloodletting

Made sense, strong professional interest

Observational impressions failed

Stakeholders favored evidence-free medicine

Long time to do a trial, long time to change

Stents after MI and HDC with BMT

Made sense, strong professional interest

Observational data failed

Stakeholders favored evidence-free medicine

Long time to do a trial, short time to change

20

Upbeat Story of Collaboration: NETT

Lung reduction surgery in emphysema (N=1218)

21 National Emphysema Treatment Trial Research Group. N Engl J Med. 2003;348:2059-73

―The National Marfan Foundation does not recommend switching from a

beta blocker to losartan as a way to manage Marfan syndrome until the trial

is completed. This is because we do not know whether losartan is clearly

better than atenolol for taking care of people with Marfan syndrome. Also, we

do not know if people with Marfan syndrome will have unwanted side effects

when they take losartan.‖

Upbeat Story of Collaboration: Marfan Syndrome

http://www.marfan.org/marfan/2408/Atenolol-vs.-Losartan-Clinical-Trial

Success Story: Prevention of MI

23 Yeh RW, et al. N Engl J Med 2010;362:2155-65

Evidence-Based Medicine Works!

24

Better hypertension control

Better cholesterol control

Selby J et al Med Care 2010;48:133-9

Contemporary Challenges…

Criteria for priorities

Public health

Scientific opportunity

Stakeholder interests: may collide

How will do affordable pragmatic trials?

When can we trust observational data?

Implementation: Will knowledge help?

What about personalized medicine?

25 Lauer MS, Collins FS. JAMA 2010;303:2182-3

Stakeholder Interests May Collide

―It is difficult to get a man to

understand something when

his salary depends upon his

not understanding it.‖

Upton Sinclair

26 I, Candidate for Governor: And How I Got Licked (1935), ISBN 0-520-08198-6; repr. University of California Press, 1994, p. 109.

So What is a Practical Trial?

―The hypothesis and study design are

developed specifically to answer the

questions faced by decision makers‖

Decision makers = patients, clinicians,

payers, policy makers

27 Tunis S, Stryer D, Clancy C. JAMA 2003;290:1624-32

What‘s the Lesion?

―Neither of the major sources of funding for

clinical research in the United States—the

NIH and the medical products industry—

has as a primary mission the goal of

ensuring that studies are performed to

address clinical questions important to

decision makers.‖

Sean Tunis, Daniel Stryer, and Carolyn Clancy

28 Tunis S, Stryer D, Clancy C. JAMA 2003;290:1624-32

An Inspiring Story

29

“It started with no funding and skepticism in some quarters

but today GISSI is recognized as an Italian achievement that

has changed cardiology treatment worldwide.”

http://eurheartj.oxfordjournals.org/content/31/9/1023.full

The GISSI Story

Trial (topic) Patients Cost (Euro millions ) Cost/Pt* (E K)

Thrombolysis 12,000 0.4 0.03

Thrombolysis and

sc heparin

13,000 4 0.2

ACE after MI 19,000 6 0.3

PUFA and Vit E 11,000 4 0.2

Statin in HF 7,000 20 3

30

*Per trial, as some were factorial designs

―Logistical Aspects‖

• All Italian cardiologists are invited to participate

• Enrollment of a broad spectrum of patients

• Clinical procedures which mimic clinical practice

• Low costs

• Immediate transfer to clinical practice (investigators see patients)

Tavazzi L et al. Am Heart J 2004;148:222-9

Practical Trial Design Tool

31 Thorpe KE et al. CMAJ 2009;180:E47

What are We Looking For in ―Practical‖

Practical elements

Broad eligibility criteria

Flexible interventions

Typical practitioners

No follow-up visits

Clinical outcomes

Usual compliance

Intent-to-treat

32 Thorpe KE et al. CMAJ 2009;180:E47

An Ongoing Example

33

How Can We Simplify and Improve Trials?

Sites and Coordinating Centers

Fit trials into practice and clinical IT

Limit case report form length

Apply metrics for performance

Operations

Decrease monitoring, use statistical analyses

Research and evaluate processes

Central IRB and standardized contracts

Design: factorial, adaptive, cluster

34 Eisenstein EL, Califf RM. Clinical Trials 2008;5:75. Menikoff J. NEJM 2010;363:1951

Back to Basics: Beware Inferior CER

35

―It was because they were brilliant

observers of humans, not

experimenters upon them, and

observation by itself provides

insufficient evidence of the value of

a treatment.‖

David Sackett

―The principle of science, the

definition, almost, is the following:

The test of all knowledge is the

experiment. Experiment is the

sole judge of truth.‖

Richard Feynman

Inferior CER: Observational data, small trials, surrogates

With thanks to Robert Califf and David Sackett

Back to Basics: Beware Inferior CER

36

―The smaller the studies, the less likely the research

findings are to be true. (Inadequate power)

The greater the financial interests and prejudices, the less

likely the research findings are to be true. (Excess bias) ‖

Ioannidis JPA. PLoS Medicine 2005;2:e124

Likelihood of Truth

37

Design Likelihood of Truth

Large, adequately powered RCT with little

bias and 1:1 pre-study odds

85%

Meta-analysis of small trials 41%

Small, well-performed phase II trial 23%

Large epidemiological study 20%

Discovery-oriented exploratory research 0.1%

Ioannidis JPA. PLoS Medicine 2005;2:e124

How Can You Support National CER?

Do the right thing! Embrace science…

That means:

Do large, simple pragmatic trials

Target evidence-free medicine

If it‘s ―unethical‖ it may be most ethical

Beware of real conflicts of interests

Create a culture in which enrollment in large,

simple trials is the norm, not the exception

Science guides medical practice

38