Consensus Conference on Transplant Program Quality and Surveillance

Arlington, VAFeb 13-15, 2012

Consensus Conference on Transplant Program Quality and Surveillance

Co-Chairs

Bertram Kasiske (SRTR) and Maureen McBride (OPTN/UNOS)

Steering Committee

Robert Gaston (AST) Dennis Irwin (Optum Health)Mitch Henry (ASTS) Nancy Metzler (OPTN-TAC)Thomas Hamilton/Karen Tritz (CMS) John Roberts (OPTN-MPSC)Danielle Cornell (ACOT) Alan Reed (OPTN-PAIS)Kenyon Murphy (Patient/Public) Stuart Sweet (OPTN-POC)

FundingHRSA

1) What is the SRTR’s mandate?2) Who uses PSRs and why?3) Are there unintended consequences?4) What can we learn from others?5) What statistical methods should we use?6) How should we adjust for risk?7) What outcomes should we use?8) What data should we collect?

Eight Key Questions

Plenary Session 1 – Monday, Feb. 13Time Topic Presenter1:00 PM Welcome and Introductions Maureen McBride

1:10 PM Overview of Program Specific Reports Bert Kasiske

Current and Future Uses of PSRs:

1:30 PM The OPTN Perspective Alan Reed

1:50 PM Centers for Medicare & Medicaid Services Perspective Thomas Hamilton

2:10 PM The Programs’ Perspective Barry Friedman

2:30 PM The Private Insurers’ Perspective Richard Migliori

2:50 PM The Public’s Perspective David Howard

3:10 PM Break (20 minutes)

Plenary Session 2 – Monday, Feb. 13Time Topic Presenter3:30 PM Unintended Consequences of PSRs Jesse Schold

3:50 PM Analysis of the Effect of PSRs on Center Behaviors Jon Snyder

4:10 PM What Can We Learn From Other Experiences? David Naftel

4:25 PM What Can We Learn From Other Experiences? Douglas Rizzo

4:40 PM Cumulative Sum (CUSUM) Approach David Axelrod

4:55 PM Other Analytical Techniques Nick Salkowski

5:15 Wrap Up Bert Kasiske

Plenary Session 3 – Tuesday, Feb. 14Time Topic Presenter8:00 AM Framing the Questions: Limitations of Current System Ajay Israni

8:20 AM Methods Nick Salkowski

8:50 AM Risk Adjustment Mike Abecassis

9:20 AM Outcomes Larry Hunsicker

9:50 AM Data Stuart Sweet

10:20 AM Break (30 minutes)

10:50 AM General Discussion Bert Kasiske & Maureen McBride

12:00 PM Lunch (1 hour)

Plenary Session 3 – Tuesday, Feb. 14Time Topic Presenter1:00 PM Break-Out Sessions Join Your Assigned Group

3:00 PM Break (20 minutes)

3:20 PM Break-Out Sessions (continued) Join Your Assigned Group

5:00 PM Wrap Up Bert Kasiske

Plenary Session 4 – Wednesday, Feb. 15

Presenter8:30 AM Break-Out Group Reports Danielle Cornell & Ken Murphy

10:15 AM Break (15 minutes)

12:00 PM Conclusion - Lunch Bert Kasiske & Maureen McBride

“Make available to the public timely and accurate program-specific information on the performance of transplant programs. This shall include … risk-adjusted probabilities of receiving a transplant or dying while awaiting a transplant, risk-adjusted graft and patient survival following the transplant, and risk-adjusted overall survival following listing … These data shall include confidence intervals or other measures that provide information on the extent to which chance may influence transplant program-specific results.”

OPTN Final Rule -Page 21- October 20, 1999

The Final Rule

HRSA contract with the SRTR• Produce PSRs no less than every 6 mo.• Post-transplant:

− risk-adjusted graft and patient survival− morbidity & functional impairment, etc.

• Waiting list probability of:− receiving a transplant− dying while waiting− being removed from the waiting list

• Living donor:− profiles (age, sex, ethnicities, comorbidities, etc.)− outcomes (death, re-hospitalization, etc.)

Recommendations: Statistical MethodsI.1. PSRs should be better suited to the needs of users, particularly patients.

I.2. Rather than refitting each model every 6 months, models could be fitted less often, and the time between reporting periods could be used to more carefully review models.

I.3. Although the current Cox proportional hazards models (with inclusion of additional comorbidity parameters) will likely be adequate for “flagging”, consider comparing this method with mixed effects methods.

Recommendations: Statistical Methods

I.4. Consider providing transplant centers with tools like CUSUM and/or PSR forecasting “scorecards” to facilitate Quality Assessment and Performance Improvement (QAPI).

I.5. Consider increasing the observed-to-expected thresholds and/or using sliding-scale P-values to monitor the outcomes of small-volume centers equitably.

Recommendations: Statistical Methods

I.6. Mortality data from the Social Security Administration Death Master File (SSADMF) should continue to be available to the SRTR.

I.7. The SRTR should substitute missing data with values that yield the best outcomes to encourage centers to accurately record data, and should consider including the timeliness and completeness of data submission as a quality indicator.

I.8. Avoid the conversion of continuous data elements to categorical elements, and use splines in instances where continuous linear values are not appropriate.

Recommendations: Adjusting for Risk

II.1. Consider protecting innovation by excluding patients who are in approved protocols from PSR models in identifying underperforming centers.

II.2. Identify centers that manage high-risk patients and donors well.

Recommendations: Adjusting for Risk

II.3. Develop more detailed, reliable, organ-specific data on:

a) coronary heart disease, e.g. revascularizations,

b) peripheral vascular disease, e.g. revascularizations and amputations,

c) diabetes mellitus,d) ZIP code socioeconomic status / race-ethnicity,

e) donor/organ risk, andf) ventricular assist devices.

II.4. Provide more data on waiting list risk and outcomes.

Recommendations: Appropriate Outcomes

III.1. The composite pre-transplant metric (CPM), combining waiting list mortality, transplantation rate, and organ acceptance rate, is potentially useful.

III.2. Life-years after listing is a metric that deserves consideration and study.

Recommendations: Appropriate Outcomes

III.3. Transplant program risk-tolerance may be a potentially useful metric.

III.4. Improve the monitoring and reporting of short-term living donor outcomes.

III.5. Consider providing information on long term outcomes.

Recommendations: Appropriate Outcomes

III.6. Study the utility of reporting on outcomes such as:a) life years from transplantation,b) quality of life,c) serum creatinine as a surrogate for long term

outcomes after kidney transplantation,d) FEV1 as a surrogate for long term outcomes after

lung transplantation,e) acute rejection,f) hospitalization, andg) rates of candidate acceptance for transplantation.

Recommendations: Sufficient Data

IV.1. Provide standard definitions and identify source documents for all data elements.

IV.2. Examine whether data in DonorNet® can be used to provide information on donor and organ quality.

IV.3. Consider reducing data by collecting complete data that are needed for organ allocation and in PSR outcomes models, but otherwise use data sampling strategies for other non-essential data.

Recommendations: Sufficient Data

IV.4. Consider using Medicare claims data to supplement OPTN data.

IV.5. Survey transplant programs to better understand the data collection burden and which data are the most difficult to report.

Recommendations: Sufficient DataIV.6. Assist transplant programs in maintaining their OPTN data by:

a) educating programs about the availability of tools (on the SRTR private sites, Tiedi® export, etc.) to examine missing data and improving the utility of these tools,

b) developing enhancements that allow programs to more easily monitor their performance,

c) allowing programs to more easily make corrections when problems are identified, and

d) offering standardized OPTN training for data entry personnel and a certification process.

Recommendations: Sufficient DataIV.7. Use the OPTN policy development process and follow strict criteria for adding new data elements, including:

a) reasons for each data element,b) how each data element will be used, c) clear definitions,d) source documentation requirements,e) appropriate populations,f) minimizing unproductive data entry categories, e.g.

“other” or “unknown”, andg) understanding cost implications.

Recommendations: Sufficient Data

IV.8. The OPTN should explore the feasibility of building data collection interfaces with electronic medical records.

IV.9. Consider allowing the cost of mandated data entry to be placed on the Medicare Cost Report for reimbursement, and not limit this option to the Candidate Registration Form.

IV.10. Consider providing information about paired exchange.

SRTR Technical Advisory Committee (STAC) Meeting Summary: February 23, 2012• STAC Reviewed the Consensus Conference Recommendations• STAC is continuing the prioritization discussion, but general

themes of the discussion included: Strong support for creation of separate reports for:

• Public Consumption: Focused on easily understood metrics that are meaningful for patients, targeting the types of information patients are most concerned about:

Waiting Time Life-Years from Listing

• Program Consumption Quality Improvement Focus: More Forward-looking metrics rather than

historical performance metrics.

Traditional Adult Graft Survival

SRTR Technical Advisory Committee (STAC) Meeting Summary: February 23, 2012• Quality Assurance:

Need to continue to support OPTN’s MPSC committee and CMS Conditions of Participation

• O/E framework is likely here to stay for the near future.• Recommended studying additional methodologies,

including Bayesian hierarchical modeling strategies (What is the probability

program X is underperforming?) CUSUM charts could be provided for a program’s internal

consumption, but are likely not suited for public reporting.

Methods: Use of hierarchical models with (Bayesian) suggested performance criteria

Christiansen CL, Morris CN. Ann Intern Med. 1997;127:764.

Methods: funnel plots

SRTR Technical Advisory Committee (STAC) Meeting Summary: February 23, 2012• Support for formalizing the SRTR’s risk model development

process to include: Written “manual of operations” for risk model

development. OPTN involvement (expert review) Set model building schedules, perhaps 3-year cycles

31

Example of 3-year Model Building Cycle

Year 1•Heart•Lung

Year 2•Liver•Kidney

Year 3•Pancreas

• Update Cycle Based on Organ Groups.

• During the year, both the post- and pre-transplant models will be rebuilt.

• Could subset the year into semesters and focus on one organ in each semester.

Path Forward• SRTR-STAC discussions are ongoing.• Looking to create a PSR subcommittee of the STAC focusing on

performance assessment methodologies.• Prioritized list will be settled in the coming days with review

by HRSA.• SRTR staff will begin researching alternative methods to

achieve the stated recommendations.

Thank You