ro summit breakout – bruce g. haffty, m.d. 1 abr summit on practice quality improvement...

RO Summit Breakout – Bruce G. Haffty, M.D.

1

ABR SUMMIT ON PRACTICE QUALITY IMPROVEMENT

RO-Breakout Session

August 19, 2006


2

RADIATION ONCOLOGY BREAKOUT SESSION OUTLINE

• Brief Review of ABMS Guidelines• Brief Review of ABR Submission to ABMS on PQI• Timeline for Projects • Discussion of Fundamental Elements of any PQI

Project• Type I Projects• Discussion of Type I and Type II Projects• Type II Projects Developed or Under Development • Reporting and Documentation


3

GENERAL ABMS GUIDELINES FOR PRACTICE PERFORMANCE IMPROVEMENT

1. A program of practice assessment should be phased in, periodically evaluated for its effectiveness, and systematically improved. Diplomates should be kept informed of the development of practice performance assessment.

2. The assessment process should reflect the activities of a diplomate related to patients or patient care.

3. Standards for measurement of clinical practice performance should be based on evidence-based guidelines, explicit expert consensus, or normative peer comparison.


4


4. The assessment process should compare the diplomate’s practice performance to evidence-based guidelines or explicit expert consensus, where available, and to peers. After an initial baseline assessment, diplomates should be asked to develop an implementation plan for how they would improve performance. Diplomates should submit a follow-up assessment of the effect of the improvement plan. Each board should have a plan for what to do with diplomates whose performance does not meet acceptable expectations.

5. Initially each of the six general competencies should be assessed at least once during a board’s repeating maintenance of certification cycle. It is expected that by the end of the second cycle, this should be a continuous process.


5


6. Assessment of patient care initially should focus on a sampling of patients in a practice with a key disease or clinical process (such as asthma, diabetes, pregnancy, immunizations, surgical procedure or processes central to that specialty) at least once per cycle. By the end of the second cycle, each board should move to a more continuous sampling of patients that will enable diplomates to demonstrate, at any point in time, the quality of his/her care for a defined number of consecutive patients or specialty-related key activities.

7. An effective method for boards to consider for assessment and improvement of clinical performance is to be part of a collaborative effort with other practices using shared databases.


6


8. The measurement of practice performance should use proven educational and assessment methodology.

9. Practice assessment should provide performance feedback, improve workflow, improve efficiency of practice, and should not duplicate other assessment efforts.

10. Practice assessment should include appropriate collaboration with specialty societies and other organizations with relevant education and assessment expertise.


7


11. Boards should develop a consistent approach regarding the status of Maintenance of Certification© for diplomates who are not involved in direct patient care.

12. The assessment of physician performance should begin during residency and continue throughout practice. The board’s evaluation of physician performance during residency should be linked to the six general competencies described by the ABMS-ACGME.


8

ABR SUBMISSION TO ABMS-PART IV PRACTICE PERFORMANCE SECTION

Evaluation of Performance in Practice[Assessment of Practice Performance]

Each diplomate is expected to maintain active, professional involvement in radiation oncology. Individuals and groups are expected to understand and participate in the concepts of continuous quality improvement and lifelong learning. The ABR, like several boards, has struggled to identify components of its Practice Performance Program (PPP) with outcomes data and benchmarks in the first cycle of MOC. In


9


conjunction with our specialty societies, our initial efforts have resulted in a sophisticated, physician-directed practice assessment module (PPM) based upon published standards for the field, including measures directly linked to published outcomes data. As well, we have encouraged development of relatively simple, relevant measures of physician practice clearly related to the process of continuous quality improvement. Quality improvement is key for a rapidly developing, technologically oriented medical specialty. Practice performance modules currently in development (with the first now in alpha testing) are based upon common central components of the practice of radiation oncology.


10


1) Practice Performance Assumptions

Self-Assessment Modules (SAMs) are essential elements of an overall PPP in measuring gaps in practice related knowledge and identifying resources for improvement.

Hospital privileges within the standards of JCAHO/VAH accredited hospitals are required for physicians practicing in hospitals because they are one indicator of acceptable performance in practice. Privileges will be tracked at regular intervals over the MOC cycle.

Practice performance modules are specifically designed to be relevant and applicable in both hospital-based and freestanding radiation oncology settings. Freestanding radiation oncology practices can employ AAAHC (Accreditation Association for Ambulatory Health Care) standards. JCAHO also provides volunteer accreditation processes for ambulatory care settings.


11


2) Initial Practice Performance Modules

PPMs are currently in development in response to parameters identified by the ABR. We anticipate at least two basic modules will be available at implementation in 2007, and that several additional PPMs will be proposed for ABR qualification over the next few years. Qualified PPMs will meet basic ABR requirements for both content and design. All modules will include feedback to the diplomate regarding his or her level of performance in comparison to identified standards and his/her peers, as well as suggestions regarding learning opportunities in areas of relative weakness. Specific aspects of quality improvement intended to enhance the individual’s practice performance will be highlighted.


12


The two practice performance modules currently in development (ACR’s R-O Peer® and Chart Rounds) were previously submitted to the ABMS. Alpha testing has been initiated for the ACR’s R-O Peer®. The largest radiation oncology society (ASTRO) is studying plans for timely development of the Chart Rounds PPM. Modules include feedback to the physician, documenting successful completion, as well as identifying areas that should be addressed in the context of continuous improvement. For physicians who do not meet the standards for successful completion of the PPM, feedback will include areas of weakness and recommendations for improvement.


13


3. Meeting Practice Performance Program Requirements

Physicians’ participation in Part IV will be documented through completion of three (3) PPMs during each ten-year MOC interval. It is anticipated that diplomates will document successful completion of one PPM with every three-year period of MOC. The ABR’s MOC website will include entry to an individual diplomate password-protected profile. The profile will document participation in qualified PPMs, including successful completion, areas identified for improvement, and action plans that may be required as fundamental components of quality improvement. In successfully completing the PPM, each


14


diplomate will be required to file an action plan as part of his/her personal profile entry for the Practice Performance Program. Action plans are under development by the ABR, and will allow the diplomate to identify areas that may require improvement based upon the results of the required PPMs. A diplomate who fails to successfully complete a PPM will include in his/her action plan steps to meet the standards of practice performance; successful completion of a subsequent PP module within three years will document improvement. Failure to complete a second PPM will require the diplomate to file an action plan directly with the MOC Coordinating Committee for review and monitoring. The Committee will work with the diplomate to ensure successful documentation of the action plan within an appropriate time interval.


15


4) Performance Assessment Timeline

Year Practice Performance Program (PPP) Timeline 1

2

3 Documentation of completion of one PPM and filingof related Action Plan

4


16



6

7 Documentation of completion of second PPM and related Action Plan. (Any individual unable todocument successful completion of at least one ofthe PPMs will file an Action Plan directly withthe MOC Coordinating Committee.)


17



9 Documentation of completion of third PPM andrelated Action Plan. (Any individual unable todocument successful completion of at least one ofthe PPMs will file an Action Plan directly withthe MOC Coordinating Committee.)

10 (Successful completion of the MOC-CC monitoredAction Plan as may be required; see Year 9 above.)


18

FUNDAMENTALS OF PQI PROJECTS • REFLECT THE ACTIVITIES OF THE DIPLOMATE RELATED TO

PATIENT CARE

• RELATED TO INDIVIDUAL PHYSICIAN PRACTICE

• MEASURABLE ENDPOINTS

• BENCHMARKS, EVIDENCE BASED, CONSENSUS OR PEER COMPARISON

• ACTION PLAN-FOLLOWUP PLAN FOR IMPROVEMENT

• INCORPORATION OF SIX COMPETENCIES (not all competencies need to be addressed for each PQI, but over 10 years the six competencies should be addressed)


19

TYPE I vs TYPE II PROJECTS

TYPE I TYPE II

INDIVIDUAL/INSTITUTIONAL SOCIETY INITIATED

QUALIFIED BY ATTESTATION QUALIFIED BY ABR

SUBJECT TO AUDIT SUBJECT TO AUDIT

UP TO 2 OF 3 PQI MAY BE TYPE I AT LEAST 1 OF 3 MUST BE TYPE II

MUST MEET FUNDAMENTALS

• INDIVIDUAL DIPLOMATE

• PATIENT CARE RELATED


• BENCHMARK OR COMPARISON

• ACTION PLAN

• COMPONENT OF SIX COMPETENCIES

MUST MEET FUNDAMENTALS

• INDIVIDUAL DIPLOMATE

• PATIENT CARE RELATED


• BENCHMARK OR COMPARISON

• ACTION PLAN

• COMPONENT OF SIX COMPETENCIES


20

TYPE I PROJECTS DISCUSSION

• WHAT ABOUT CHART ROUNDS-DOES THAT COUNT?

• WHAT ABOUT A RETROSPECTIVE REVIEW OF MY PRACTICE OUTCOMES?

• WHAT ABOUT PRESS-GANEY OR OTHER PATIENT SURVEYS?

• WHAT ABOUT HOSPITAL OR INSTITUTIONALLY INITIATED QI PROGRAMS?


21

TYPE I PROJECTS DISCUSSION

In my view, any of the above, and other creative programs that we will discuss may be totally reasonable practice quality improvement projects. They should, however, meet the fundamental elements previously discussed, and be formatted in such a way that there is a clear patient care focus, the program is being carried out with respect to the individual diplomats practice, there is a measurable endpoint which is being compared to some norm (benchmark, consensus, or evidence based guideline), with an action plan. I have outlined below a hypothetical “project” and how it might fit the guidelines for discussion.


22

HYPOTHETICAL PHYSICIAN INITIATED TYPE I PQI PROJECT

Review of patients following mastectomy with positive nodes• Physician reviews both his own practice and hospital tumor registry

records in patients treated by mastectomy with node positive disease. (Reflects diplomats activities in patient care)

• He will review his own practice patterns, and referral of patients from the hospital for post-mastectomy radiation (measurable endpoint)

• He finds that in his own practice, 100% of patients with 4 or more nodes were offered radiation therapy and 30% of patients with 1-3 nodes were offered radiation (Patient Care-Measurable endpoint)

• He compares this to ASCO and ASTRO guidelines, with which his practice appears to be consistent (Comparison to Benchmark/ Consensus standard)


23


• He finds, however, that of patients in the hospital tumor registry with 4 or more nodes, only 50% were referred for radiation therapy after mastectomy, which is clearly not consistent with ASCO and ASTRO guidelines (Comparison with standards)

• He also compares his general policies (doses, fields treated, use of bolus, etc.) to published ACR appropriateness criteria for post-mastectomy radiation. (Measurable endpoints –compared to published standards)

• He is in compliance in general, but treated the full axilla in several patients who had only 1-3 nodes involved, which was not in accordance with ACR appropriateness criteria.


24


• His action plan is two-fold. Modify treatment policy in patients with 1-3 nodes involved to not include the full axilla. He will present data in a hospital wide forum on the merits of post-mastectomy radiation in patients with 4 or more nodes, ASCO and ASTRO guidelines.

• Follow-up. He will reexamine in 3 years the referral patterns for patients with 4 or more involved lymph nodes for post-mastectomy radiation.

• He will self-attest to the ABR that he conducted this PQI program and keep a record of his efforts.

• If audited by the ABR, will this program stand up as a legitimate PQI

It should also be noted that the program above incorporates the majority of competencies (patient care, medical knowledge, professionalism, systems based practice, practice based learning, and interpersonal communications).


25

OTHER INSTITUTIONAL OR GROUP TYPE I PROJECTS

Hospital initiated Type I projectsPractice efficiencyPatient satisfactionPatient safetyReferring satisfaction

These programs, typically developed within an institutional “QUALITY IMPROVEMENT” programs, may be ideally suited to qualify for TYPE I initiatives. Provided they meet the essential elements of PQI which include

Related to individual physicians practiceRelated to patient careMeasurable endpointsComparable to benchmark, evidence based standard or peer comparisonAction plan for improvement and follow-up planAddresses at least some of the six competencies


26


Example: As an overall component of their Quality Improvement Program and Marketing Strategy in a competitive environment, a Hospital or Group practice with multiple specialties initiates a multi-specialists-wide program to evaluate practice efficiency/patient satisfaction, employing measurable endpoints such as turn-around time of consults, patient waiting time for consults, time between initial consultation request and visit, patient and referring satisfaction. Radiation oncology is included in this program. At the end of the assessment, the radiation oncologist is


27


informed that they scored well in comparison to other departments, with the exception of patient waiting times. They develop an action plan to improve patient waiting by modifying the way in which they schedule consults. Action plan is also to re-assess patient waiting 6 months after they modify the scheduling to evaluate impact of action plan.

In my view, this addresses the fundamental elements and remains within the scope and spirit of the PQI initiatives. It directly relates to the diplomates practice and patient care, it


28


has measurable endpoints, compares to a standard of peer performance, and includes an action plan and follow-up.

Again, it is easy to also justify how this program addresses most of the competencies and is perhaps more focused than the previous Type I example, with respect to professionalism and interpersonal skills, particularly if patient satisfaction surveys included some measure of the physician-patient interaction.


29

SOCIETY INITIATED TYPE II PROJECTS

ACR RO-PEER

ASTRO-PAAROT

ABS-IN DEVELOPMENT


30

ACR-RO PEER

Developed by the ACR to comply with part IV (PQI) of the MOC process

Two options within plan• OPTION 1

o This option will be available as part of the process of a facility undergoing the current ACR facility accreditation process, as a separate component of the process

• OPTION 2o This will be a remote review of cases submitted, and not a component

of the facility accreditation process.


31

RO-PEER PROGRAM

• Review of randomly selected cases from a submitted list

• Reviewed by trained reviewers/members of a reviewing committee-radiation oncologists

• Pre-determined areas, included H&P, treatment summary, follow-up, physics, etc. will be reviewed

• Compliance in these areas will be evaluated and “scored”

• Acton Plan for improvement will be incorporated

• HIPPA regulations will be adhered to in this process


32

ASTRO’s PAAROT PROGRAM (Performance Assessment for the Advancement of Radiation Oncology Treatment)Recently developed by ASTRO and submitted to ABR for review and comment

• Baseline Performance where 10-15 charts are randomly selected and specific questions are asked related to the diagnosis, staging and treatment.

• Answers will be compared to “ideal” answers, and suggestions for answers that deviate from “ideal” will be made.

• A patient satisfaction survey will be incorporated.• Physician will select one performance indicator to focus on for

practice “improvement”. • Re-measurement will be performed to assess effectiveness of

action plan.


33

REPORTING AND DOCUMENTATON Discussion of automated reporting, validation and documentation of PQI projects

ABR

Society Initiated Gateway -DI employs this model to coordinate CME and SAM documentation for diagnostic diplomats in MOC. Various Societies evidently electronically enter the information into this central gateway for recording and tracking this information. Incorporating PQI into this will be discussed. Whether RO can do something similar with our CME, SAM and PQI will be discussed.

ro summit breakout – bruce g. haffty, m.d. 1 abr summit on practice quality improvement...

Documents