Ongoing Evaluation of Physician Performance:

Developing a Performance Portfolio

Cary Sennett, MD, PhDMedBiquitous Annual Conference

May 15, 2008

Overview

“Performance Portfolio”—what are we talking about?

Design Criteria—what would a portfolio look like?

One—I hope promising—effort (in a bit more detail)

Summary…

Defining terms

Portfolio: A (comprehensive) information set that a physician can use • To establish his/her qualifications• To improve care

A “performance portfolio” to improve care• Must begin with information about practice

performance• Must contain elements that speak to the

capability of those factors that determine practice performance

Elements that “speak to capability”

How my practice is performing depends on• My competence (my knowledge and skill)• How effectively that competence is deployed

“Deployment”—that is, the conversion of capability to results—depends upon environmental factors• “Micro-environment”• “Macro-environment”• And (of course) the patient…

For purposes of our discussion, it may be less important to consider the “macro-environment”

Practice performance• Are we achieving the results that we could?• What limits our ability to do better?

His or her own competence• Do I have the knowledge and skill necessary to deliver excellent

care?• And how can I close critical gaps?

(Micro)-system • Is the (micro-) system in which I function optimally configured to

support my efforts to deploy that competence?• Am I using it effectively?• And, if not, how can I improve it?

My patients• Have I maximized their ability to achieve desired health results?• And, if not, how can I do so more effectively?

So a portfolio should help a physician evaluate

Designing a portfolio (to support practice improvement)

Practice results

Physician knowledge and skill

Systems infrastructure

Patient self-management

Need for broad set of inputs

What “Practice Results?”

Must address the range of results that are relevant to patients (and other customers…)

It may be helpful to think of that range as the IOM does: care that is• Safe• Timely• Effective• Efficient (delivered at appropriate cost)• Equitable• Patient-Centered

But, in any event, it is essential to recognize that performance is multidimensional

What “Knowledge and Skill?”

ACGME (ABMS) competencies Professionalism Patient Care Medical Knowledge Communication and Interpersonal Skill Systems-based Practice Practice-based Learning and Improvement

What “System Infrastructure?”

Information management Patient activation Access and communication with patients Safety and efficiency Consultation and referral Team function Improvement process

How do we get there?

A lot of activity right now—but widely distributed

ABMS Certification/Maintenance of Certification may be a helpful “seed” around which this work can organize

Step on the path—but not the end of the road

Board Certification Professional effort to evaluate competency (capability) of

individual physician

Structured so as to offer a window into other key portfolio elements• Practice results• Practice systems infrastructure• Patient self-management

Designed—and objective is—to support improvement

But essential function is assessment (both formative and summative)

The Structure of Board Certification

Specific reference to ACGME competencies Two elements

• Initial certification• Periodic recertification/Maintenance of Certification

Structure common across 24 ABMS Boards• Assessment of actions against license• Self-assessment of medical knowledge• High stakes secure examination of cognitive

knowledge and judgment• Self-assessment and improvement of practice

performance

Boards vary with respect to implementation—strategies and timelines

Self-Assessment of Practice Performance

Designed to force “a new way of thinking” about quality and QI• Quality is about what the system produces• Quality improvement is not about working harder (or

knowing more)—but about “diagnosing and treating” system problems

Designed to promote adult (experiential) learning

Physicians can receive up to 20 units category I CME credit, as well as credit toward renewing their certificate

The ABIM Practice Improvement Module (PIM™)

Performance Report

Performance Report

Improvement

Patient survey

Impact

PlanDo

Study

Act

Practice survey

Chart reviewApply quality measures to practice

Apply quality measures to practice

Compare performance to guidelines

Compare performance to guidelines

Test a process change aimed at improving

care

Test a process change aimed at improving

care

Examine practice infrastructure and

process

Examine practice infrastructure and

process

Report what was learned

Report what was learned

The PIM as Portfolio

Provides window on practice results

Can link to other information about physician competency (knowledge and skill)

Provides window on systems infrastructure

Potential window on “patient competence”

Increasing robustness

Need broader window on practice performance—more, and more diverse—data

Need tighter and more intentional link to information about individual physician competency

Need mechanism to track (all elements over time)

Need expanded window on “patient competence”

Getting from here to there: “PIM Future” Links to extant datasets—power practice and self-

assessment off of available data

Performance (and personal) “dashboards”: turning data into information that speaks (meaningfully and reliably) to practice performance and to core drivers

Links to—truly integration with—support needed to go from• “I see an opportunity to improve” to• “I’m able to capitalize on that opportunity to improve”

What will it take?

More (more standardized and more available) data from many, many sources

More (much more) research

Collaboration/partnerships that will link assessment capability to improvement capability

Collaboration/partnerships to organize and focus energy and resources

Summary It is possible to conceive of a portfolio that will

support practice improvement

Such a portfolio will need to address a broad ranges of issues

Board certification (Maintenance of Certification) may be an important part of early efforts to build a portfolio

To get where we need to go will require collaboration—which will require much, much more standardization