measuring the impact of resident work hours reform: recent findings and next steps patrick s....

Measuring the Impact of Resident Work Hours Reform: Recent Findings

and Next Steps

Patrick S. Romano, MD MPHProfessor of Medicine and Pediatrics

Division of General Medicineand

Center for Healthcare Policy and ResearchUniversity of California, Davis

May 21, 2009

Disclosure

• Financial support from NHLBI RO1 HL82637 (Kevin Volpp, University of Pennsylvania, PI)

• Have you (or your spouse/partner) had a personal financial relationship in the last 12 months with the manufacturer of the products or services that will be discussed in this CME activity?

NOT APPLICABLE

AcknowledgmentsInvestigative team• Kevin Volpp, MD PhD, University of Pennsylvania• Jeffrey H. Silber, MD PhD, Director, Center for Outcomes

Research, Children’s Hospital of Philadelphia• Amy K. Rosen, PhD, Bedford VA Center of Excellence in

HSR&D• Paul Rosenbaum, PhD (Statistician), Wharton School• Lisa Bellini, MD, Program Director, University of

Pennsylvania

Staff• Orit Even-Shoshan, MS, Anne Canamucio, MS, Tiffany

Behringer, MS, Yanli Wang, Hong Zhou, Liyi Cen, Mike Halenar

Other sources of slides• John Welch, MD (former pediatric resident)• Garth Utter, MD, Department of Surgery

Learning Objectives

• To Explain Recent and Proposed Policy Changes to Limit Resident Work Hours

• To Summarize Recent Evidence Regarding the Impact of ACGME Work Hour Rules Implemented in 2003

• To Discuss Ideas and Methods for Future Research in this Area

The Birth of Residency

• 1889 with the opening of The Johns Hopkins Hospital

• Osler, Halsted, and Kelly (bedside teaching)

• Based on German model

• Room, board, and laundry provided; salary optional until 1965

Core Concepts and Practices

• Graded responsibility, especially for inpatients

• Variable and lengthy training period

• Pyramidal system of promotion

• Restrictive lifestyle (100-120 hrs/week, continuous shifts up to 36 hours)

Libby Zion Case

• 18 year old college student, with known history of depression, taking Nardil® (MAOI), was brought into New York Hospital on October 4, 1984

• Presented with fever, agitation and strange jerking motions of body, with occasional disorientation

• Admitted with diagnosis of “viral syndrome with hysterical symptoms”

Libby Zion Case

• Ordered Demerol® to control her shaking

• Later in evening Libby became more agitated

• The intern was contacted at least twice, ordered physical restraints and Haldol®

• Patient finally fell asleep

• At 6:30 a.m. her temperature was noted to be 107°F

• Emergency measures were attempted

• Patient suffered a cardiac arrest and died

Aftermath

• Sidney Zion’s efforts to change the system

• 1986 Grand Jury does not indict those involved but strongly criticizes system

• Bell Commission established to investigate and make recommendations

• July 1, 1989 Section 405.4 of Title 10 of the New York Codes, Rules and Regulations of the Department of Health go into effect• Work hours: Maximum 80 hrs/week averaged over

4 weeks, 24 consecutive hours (12 in ED), at least 1 scheduled 24 hour break per week

• On-site supervision 24 hrs/day, 7 days/week

The 80 hour work week

“The specific "80-hour week" was actually determined by a colleague on my porch and was based on the following informal reasoning:

(1) there are 168 hours in a week;

(2) it is reasonable for residents to work a 10-hour day for 5 days a week;

(3) it is humane for people to work every fourth night;

(4) subtracting the 50-hour week (10 hours per day x 5 days) from 168 hours leaves 118 hours;

(5) divide 118 by 4 (every fourth night) and add to the 50 hours and, eureka, that equals an 80-hour week.”

Bell JAMA 2007: 298(24):2865-2866

Timeline

• June 1998 - New York State Department of Health found work hours often exceeded regulatory limits

• April 2001 - OSHA is petitioned to regulate work hours nationwide

• November 2001 Representative John Conyers (D-Mich) introduced federal legislation to restrict resident work hours

• In response ACGME announced its guidelines effective July 1, 2003

ACGME Work Hour Restrictions

• Principles, Supervision, Fatigue, Duty Hours, On-Call activities, and Moonlighting

• Limited to 80 hours per week

• One day in seven free of all responsibilities

• 10 hour off-duty period between work periods

• In house call no more frequently than every 3rd night

• In house call must not exceed 24 consecutive hours (with up to 6 extra hours for transition of care)

Resident Education and QOL

• Meta-analysis of studies that assessed a system change designed to counteract the effects of resident work hours, fatigue, or sleep deprivation; included an outcome directly related to residents; and were conducted in the United States.

• 54 articles met inclusion criteria (12 IM, 6 ob/gyn, 7 pediatrics, 25 surgery, 4 other)

• Interventions included night and day float teams, extra cross-coverage, and physician extenders.

Fletcher, Fletcher, JAMAJAMA, 2005, 2005

Resident Education and QOL

• Interventions to reduce resident work hours resulted in mixed effects on both operative experience and on perceived educational quality but generally improved residents’ quality of life (i.e., more sleep, better mood, better family relationships, better satisfaction).

• Interpretation of the outcomes of these studies is hampered by suboptimal study design and the use of nonvalidated instruments. The long-term impact of reducing resident work hours on education remains unknown.

Fletcher, Fletcher, JAMAJAMA, 2005, 2005

N Engl J Med 2004;351:1838-1848

Effect of Reducing Interns’ Work Hours

• Investigated effect of reducing interns’ work hours on serious medical errors in ICU

• Randomized, prospective crossover trial

• Conducted in MICU and CCU, 20 interns on 3 week rotations

• Q3 day “traditional” call versus 4-day schedule without extended shifts >16 hours (7am-3pm on day 1, 7am-10pm "day call" on day 2, 9pm-1pm “night call" on days 3-4)

• Compared rates of serious medical errors (by masked direct observation) made by interns on traditional vs. intervention schedule

Effect of Reducing Interns’ Work Hours

• Traditional work week: 74-92 hours (mean 85)

• Intervention work week: 57-76 hours (mean 65)

• No change in staffing or other personnel

• Randomly assigned order, and spread throughout year

• Interns worked 19.5 hrs/week less (P<0.001), slept 5.8 hrs/week more (P<0.001), and had fewer attentional failures (EOG slow eye movements) during on-call nights (0.33/hr= 2.6/overnight versus 0.69/hr=5.5/overnight; P=0.02) on the intervention schedule

Relationship between work hours and sleep duration on two schedules

Most but not all interns slept more on intervention schedule

Results of RCT

• 2203 patient days, 5888 hours of observation

• During traditional schedule, interns made:• 35.9% more serious medical errors

• 27.8% more intercepted serious medical errors

• 56.6% more nonintercepted serious medical errors

• 20.8% more serious medication errors

• 5.6 times as many serious diagnostic errors

• No difference in procedural errors

• No significant change in other staff errors

• No significant change in # of medications or procedures, tests interpreted, LOS, mortality

Landrigan C et al. N Engl J Med 2004;351:1838-1848

Incidence of Serious Medical Errors

Evaluations of 1988 Libby Zion Laws

• No relative improvements in mortality for AMI, CHF, pneumonia in teaching vs. non-teaching hospitals1

• Significant increases in proportion of patients with at least one complication (35% vs. 22%, p=.002) and in delays in diagnostic tests (17% vs. 1.9%, P<.0001)2

• Compliance poor (60% of surgical residents exceeded 95 hours per week – 1997 audit)

1 Howard, Silber, Jobes JGIM 20052 Laine JAMA 1993

Effect of work hours reform in NY teaching hospitals on smoothed rates of Patient Safety Indicators

Poulose BK, et al., Ann Surg 2005;241:847-860

Benefits versus Harms

• Benefits:• More sleep, better sleep, better quality of life

• Better cognitive performance, fewer errors

• Caveat #1: while duty hour rules reduce total number of hours work per week, 30 hour shifts allowed

• Caveat #2: we don’t know how much more residents are actually sleeping

• Harms:• Less opportunity to observe trajectory of illness

• More frequent hand-offs

• Studies have shown higher rates of significant adverse events when patients are “cross covered” (26% vs. 12%, OR=3.5)1

1Petersen et al, Annals of Int Med 1994 121: 866-872.

First study of impact on mortality:First study of impact on mortality:Inpatient only, different samples by yearInpatient only, different samples by year

Shetty, Ann Intern Med, 2004

0.25% absolute reduction; 3.75% relative reduction

Our Study CohortsVolpp K, et al. JAMA 2007;298(9):975-1001

• All unique patients admitted between July 1, 2000 and June 30, 2005 (3 yrs pre-reform, 2 yrs post-reform)• Principal diagnoses: AMI, CHF, GI bleed, or stroke • DRG classification of general, orthopedic, or vascular

surgery

• VA• 320,685 patients, 131 hospitals• Data from VA Patient Treatment File (PTF) and

Beneficiary Identification Record Locator System (BIRLS), VA Office of Academic Affiliations

• Medicare• 8,529,595 patients, 3321 hospitals• Data from the Medicare Provider Analysis and Treatment

File (MEDPAR), denominator files, Medicare Cost Reports

Effects measured by comparing pre- to post-reform changes in mortality in hospitals of

differing teaching intensity

0

5

10

15

20

Pre Post

Hypothetical Mortality

Data

Non-teaching Teaching A Teaching B

ACGME Reform

Logistic regression used to adjust for patient comorbidities, secular trends, hospital site where treated using “difference in differences”

EFFECT =

Diff A – Diff B

(D-in-D)Diff B

Diff A1.

Diff A2

Medicare - No significant relative change in mortality according to teaching intensity

Volpp KG et al. JAMA; 2007: 298 (9): 975-983.

VA - Significant relative improvement in mortality among medical patients in post-reform year 2


How big were these effects?

Medical patients: Improvement in mortality from pre-1 to post-2 of 0.70 percentage points (11.1%) for hospitals in 75th compared to 25th percentile


VA hospitals much more teaching intensive

• VA Hospitals

• Medicare

Volpp KG et al. JAMA; 2007: 298 (9): 975-983; Volpp KG et al. JAMA 2007; 298(9): 984-992.

Do effects of reform on mortality vary across hospitals (Medicare)?

Why no improvement in quality among Medicare patients?

• Design flaws• 30 hour shifts allow acute sleep deprivation

• Current design does not respect circadian rhythms

• Sleep inertia at night when paged

• Implementation• Compliance likely incomplete; may be worse than in

VA hospitals, given higher work intensity

• Offsetting factors• Worsened continuity

• Higher work intensity

• Sicker patients

Why improvement in some groups but not others?

• VA vs. Medicare• VAs more teaching intensive (“dose response”)

• Better information systems may have mitigated some of the continuity of care (hand-off) problems

• Confounding due to other changes

• Medical vs. surgical• Differences in balance between reduction in fatigue

and continuity?

• Differences in compliance?

• Differences in effort to address discontinuity through structured sign-out, increased attending involvement?

Failure to Rescue:Death among surgical patients with potentially treatable complications

Resident/bed ratio post-reform year 1 * top 10%/25% of severity


Odds ratio† (95% CI) P-value Odds ratio† (95% CI) P-value

Medicare

Combined medical

Highest 10% (vs. bottom 90%)

1.01(0.90, 1.13) 0.86

0.90(0.80, 1.02) 0.09


0.99(0.98, 1.08) 0.81

0.94(0.85, 1.03) 0.17

Combined surgical


0.91(0.80, 1.04) 0.18

1.01(0.88, 1.15) 0.88


0.98(0.86, 1.12) 0.76

1.09(0.95, 1.24) 0.21

Failure to rescue


0.94(0.80, 1.09) 0.40

1.01(0.86, 1.18) 0.92


0.90 (0.79, 1.02) 0.10

1.00(0.88, 1.14) 0.98

Highest risk patients fared no differently than lower risk patients - Medicare

Volpp KG et al. JGIM 2009. In Press.



Odds ratio† (95% CI) P-value Odds ratio† (95% CI) P-value

Medicare

Combined medical


1.63(1.08, 2.46) 0.02

1.35(0.88, 2.07) 0.17


1.44(1.01, 2.05) 0.045

0.99(0.67, 1.43) 0.93

Combined surgical


0.68(0.39, 1.20) 0.19

0.80(0.45, 1.43) 0.45


0.52(0.29, 0.96) 0.04

1.13(0.59, 2.17) 0.71

Failure to rescue


0.67(0.35, 1.30) 0.23

0.64(0.33, 1.24) 0.19


0.86(0.49. 1.51) 0.60

0.82(0.46, 1.48) 0.51

Highest risk patients fared no differently than lower risk patients - VA

Volpp KG et al. JGIM 2009. In Press.

The concept of prolonged stays

Non-teaching (0) Very Minor/Minor (>0 & <.25)

Major (>0.25& <0.6) Very Major (>0.6)

Medicare Combined Medical Patients

40%

60%

80%

pre-3 pre-2 pre-1 post-1 post-2

Year

Per

cen

t P

rolo

ng

ed

VA Combined Medical Patients

40%

60%

80%

pre-3 pre-2 pre-1 post-1 post -2

Year

Per

cen

t P

rolo

ng

ed

Medicare Combined Surgical Patients

40%

60%

80%

pre-3 pre-2 pre-1 post-1 post-2

Year

Per

cen

t P

rolo

ng

ed

VA Combined Surgical Patients

40%

60%

80%

pre-3 pre-2 pre-1 post-1 post -2

Year

Per

cen

t P

rolo

ng

ed

The rate of prolonged stays varies little over time

Silber et al. 2009. Medical Care

Odds of prolonged stay change at similar rates in more vs. less teaching intensive hospitals

Patient categories(Number of Cases

Medicare/VA)RB ratio post-reform year 1

OR (95%CI)RB ratio post-reform year 2

OR (95% CI)

Medical Conditions Medicare VA Medicare VA

Stroke (933,225/25,385) 1.01 (0.92, 1.10) 0.92 (0.66, 1.27) 1.01 (0.92, 1.10) 0.95 (0.69, 1.31)

AMI (970,184/32,170) 1.01 (0.93, 1.10) 0.96 (0.72, 1.29) 1.06 (0.97, 1.15) 0.96 (0.72, 1.28)

GI bleed (763,765/36,035) 1.06 (0.97, 1.16) 1.26 (1.00, 1.58) a 1.09 (1.00, 1.20) 1.08 (0.86, 1.36)

CHF (1,196,294/50,266) 0.99 (0.92, 1.06) 1.11 (0.92, 1.35) 1.02 (0.95, 1.10) 1.18 (0.97, 1.43)

Combined Medical (3,863,468/143,856) 1.01 (0.97, 1.05) 1.07 (0.94, 1.20) 1.04 (0.99, 1.08) 1.05 (0.93, 1.19)

Surgical Conditions

General Surgery (651,515/22,482) 1.09 (0.99, 1.21) 1.07 (0.79, 1.43) 0.94 (0.85, 1.05) 1.02 (0.76, 1.36)

Orthopedic Surgery (1,364,559/32,719) 1.03 (0.96, 1.10) 0.82 (0.61, 1.12) 0.94 (0.88, 1.01) 1.04 (0.77, 1.41)

Vascular Surgery (179,473/11,219) 1.16 (1.00, 1.34) 1.08 (0.66, 1.77) 1.21 (1.04, 1.40) a 1.16 (0.71, 1.91)

Combined Surgical(2,195,547/66,420) 1.04 (0.98, 1.09) 0.94 (0.78, 1.14) 0.96 (0.91, 1.01) 1.00 (0.83, 1.21)

ap<0.05 bp<0.01 cp<0.001 Silber et al. Medical Care 2009.

AHRQ Patient Safety Indicators• Technical composite

• iatrogenic pneumothorax• foreign body left in during procedure• postoperative wound dehiscence• accidental puncture or laceration• postoperative hemorrhage or hematoma

• Continuity of Care composite• postoperative physiologic or metabolic derangement• postoperative pulmonary embolism or deep vein thrombosis

(PE/DVT)• postoperative sepsis

• Collaborative Care composite• postoperative hip fracture• postoperative respiratory failure• selected infections due to medical care

MEDICARE TECHNICAL PSI COMPOSITE UNADJUSTED RATES BY ACACEMIC YEAR AND TEACHING INTENSITY

0

0.5

1

1.5

2

2.5

Pre3 Pre2 Pre1 Post1 Post2

Academic Year

Rat

es (

%)

MEDICARE CONTINUITY OF CARE PSI COMPOSITE UNADJUSTED RATES BY ACACEMIC YEAR AND TEACHING INTENSITY

0

0.5

1

1.5

2

2.5


Academic Year

Rat

es (

%)

MEDICARE COLLABORATIVE CARE PSI COMPOSITE UNADJUSTED RATES BY ACADEMIC YEAR AND TEACHING INTENSITY

0

0.5

1

1.5

2

2.5


Academic Year

Rate

s (

%)

PSI composite rates change at similar rates in hospitals of different teaching intensity

Rosen et al. Medical Care. 2009. In Press.

Technical CareComposite

Continuity of CareComposite

Collaborative CareComposite

VAOdds Ratio

(95% CI)

MedicareOdds Ratio

(95% CI)

VAOdds Ratio (95% CI)

MedicareOdds Ratio

(95% CI)

VAOdds Ratio (95% CI)

MedicareOdds Ratio

(95% CI)

Resident/bed ratio*post1 a

1.09(0.78 - 1.51)

P= 0.62

1.15(1.04 – 1.27)

P=0.01

1.01(0.70 - 1.46)

P= 0.95

1.02( 0.94 -1.11)

P = 0.66

1.18 (0.75 - 1.85)

P= 0.48

0.98( 0.87- 1.11)

P= 0.80

Resident/bed ratio*post2 a

1.05(0.75 -1.45)

P= 0.79

1.09(0.99 – 1.21)

P=0.09

1.39(0.97 - 1.99)

P= 0.08

1.08( 0.99 - 1.17)

P = 0.08

1.60(1.01 - 2.53)

P= 0.04

1.00 (0.89 -1.14)

P= 0.97

Number of cases

795,306 12,426,475 339,504 7,669,946 653,270 11,295,527

Odds of experiencing a PSI generally changed at similar rates in more vs.

less teaching intensive hospitals

Rosen et al. Medical Care. 2009. In Press.

Results Summary

• Good news?• No evidence of worsening of outcomes for a broad

range of measures within either Medicare or VA

• No evidence of harm (or benefit) for high-risk patients

• Question about prolonged length of stay for vascular surgery patients

• Bad news?• No evidence of significant relative improvements

in outcomes except for medical patients in VA in post-reform year 2

Institute of Medicine 2008 report

The Institute of Medicine formed a consensus committee to:

1) synthesize current evidence on medical resident schedules and healthcare safety.

2) develop strategies to enable optimization of work schedules to improve safety in the healthcare work environment. The strategies recommended will take into account the learning and experience that residents must achieve during their training. The recommendations will be structured to optimize both the quality of care and the educational objectives.

What do we not know?

• Hours per week residents are actually working

• How much more sleep residents are actually getting

• Impact on broader range of clinical outcomes

• Longer-term impact on clinical outcomes

• Impact on educational outcomes

• How residents are spending their time

• What approaches have helped programs successfully adapt

• Comparative effectiveness and cost effectiveness of different approaches

• Role of hospital finances

IOM Report on Resident Work Hours. 2008.

IOM Recommendations

• “Safe transportation options”

• Minimize work “that is of limited or no educational value, is extraneous to their program’s goals and objectives, and can be done well by others”

• “Adequate time to conduct thorough evaluations of patients and for reflective learning…”

• Specialty-specific, RRC-set limits on caseload

• Supervisory physician (resident) in house at all times

• Schedule overlap time and facilitate safe handoffs

IOM Recommendations

• 80 hour week

• 5 hour mandatory nap between 10pm-8am if overnight shifts used

• Averaging of days off not allowed; 1 day per week and 5 days per month

• All moonlighting counted against limits

• 10 hours off after day shift; 12 hours off after night shift; 14 hours off after extended shift

Design of R01 HL094593

New study focuses on educational and clinical impact (NHLBI)

• To describe the variety and frequency of program-level behavioral responses to duty hour reform and resident work conditions• qualitative field work at a sample of IM and GS residency

programs

• Mixed-method approach of direct observation and interviews

• 12 hospitals placed in a 2x2 matrix of large versus small size and good versus poor financial balance sheets in FY 2008 by assessing operating margins and fund balances over the previous 5 years

• Direct observation of resident involvement in provision of hospital care (especially rounds, hand-offs)

• Semi-structured 1 on 1 interviews with open ended questions involving residents, nurses, attending physicians, and administrators


• To describe the variety and frequency of program-level behavioral responses to duty hour reform and resident work conditions• national surveys of program directors and residents in Internal

Medicine (IM) and General Surgery (GS)

• Partner with the ABIM, ABS, APDIM, APDS, and ACP

• Surveys of residents to focus on resident-specific issues such as balance between service delivery and education, assessment of work intensity, use of free time, how handoffs are done, actual hours worked and days off, and hours slept.

• Surveys of program directors to focus on use of non-teaching services, hiring hospitalists or physician extenders, work intensity (admissions per resident, # patients covered, hours worked, days off), training (if any) in how to do ‘handoffs’, helpful or problematic attributes of work environment.


• To assess how educational outcomes (board scores) have changed with duty hour reform for residents in different specialties.

• To examine how clinical outcomes (mortality, FTR, PLOS, PSIs) have changed beyond the first two years post-duty hour reform.

• To examine how pre-reform hospital financial health and staffing levels predicted changes in staffing and educational and clinical outcomes.

Variables

Comparison across specialties

Questions and discussion

measuring the impact of resident work hours reform: recent findings and next steps patrick s....

Documents

area slide

daysweek slide

applicable slide

md phd

department of surgery

hour work week

effect work hours

limit resident work