pbli - evaluation project

7/30/2019 PBLI - Evaluation Project

1/17

Practice Based Learning

and ImprovementStephen J. Kimatian MD

Assistant Professor of Anesthesiology and PediatricsThe Penn State, Milton S. Hershey Medical Center


2/17

PRACTICE-BASED LEARNING

AND IMPROVEMENT

Residents must be able to investigate

and evaluate their patient carepractices, appraise and assimilate

scientific evidence, and improve their

patient care practices.


3/17

Residents are expected to:

analyze practice experience and perform

practice-based improvement activities using a

systematic methodology locate, appraise, and assimilate evidence from

scientific studies related to their patients health

problems

obtain and use information about their ownpopulation of patients and the larger population

from which their patients are drawn


4/17

Residents are expected to:

apply knowledge of study designs and statistical

methods to the appraisal of clinical studies and

other information on diagnostic and therapeuticeffectiveness

use information technology to manage

information, access on-line medical information;

and support their own education facilitate the learning of students and other

health care professionals


5/17

The Mandate

Develop the skills of critical self

examination and self improvement

Become life long learners


6/17

The Question

How do you evaluate yourself if you do

not know what the standard for

competence is? Must have an established standard.

How do you establish the standard for

competence?ACGME competency as applied to practice?

Practice as applied to ACGME competency?


7/17

Perspective

What is a consultant?

Knowledge Skill

Judgment

How do you evaluate?

Open Ended Easy to use but low on detail

Highly Specific Complex and time intensive


8/17

Evaluation Tools

Easy to understand

Intuitive to use

Generate useful information For self evaluation

For evaluation of competence

For focused remediation

Minimize Bias


9/17

The Evaluation Project

Goal to develop an evaluation tool that

maintains a global perspective while

allowing detailed analysis of specificareas of strength or weakness.

A tool that is simple to use

Standardized language for the discussion ofcompetency.


10/17

The Evaluation Project

Phase 1Define Consultant using

discrete statements.

Phase 2 Characterize definingstatements.

Phase 3 - Create a Database (rubrics)

that defines Competency Phase 4 - Manipulate the data base to

create evaluation tools.


11/17

The Project - Phase 1

Define Consultant in discrete statements.

Intravenous Access

Display knowledge of indications for IV access Develop plan for IV placement considering medical and surgical

factors effecting gauge, position, and timing of IV placement.

Communicate with patient throughout procedure showing

respect and compassion

Display knowledge of aseptic technique and universalprecautions

Display proper aseptic technique and universal precautions


12/17

The Project Phase 2

Characterize defining statements Subspecialty (general, peds, cardiac .)

Phase of Care (preop, op,postop,clinical,..)

Core Competency (pt care, pbli, comm..)

Educational Domain (knowledge / skill /judgment)

Training Level (PGY1, PGY2, PGY3.

consultant)


13/17

The Project Phase 2 (cont)

Intravenous Access Display knowledge of indications for IV access

General / Preop / Pt Care / Knowledge / PGY 1

Develop plan for IV placement considering medical and surgicalfactors effecting gauge, position, and timing of IV placement

General / Preop / Pt Care / Judgment / PGY 1

Communicate with patient throughout procedure showingrespect and compassion

General / Preop / Pt Care / Skill / PGY 1

Display knowledge of aseptic technique and universalprecautions

General / Preop / Pt Care / Knowledge / PGY 1

Display proper aseptic technique and universal precautions

General / Preop / Pt Care / Skill / PGY 1


14/17

The Project Phase 3

Create a Database (rubrics) that define CompetencyIV Access Cognitive / Knowledge Skills / Behaviors Attitude / Judgment

Patient Care:

Residents mustbe able to

provide patient

care that is

compassionate,

appropriate, and

effective for the

treatment of

health problems

and thepromotion of

health.

PGY1/2

-Display Knowledge ofindications for IV access

-Display knowledge of

aseptic technique and

universal precautions

PGY1/2 -Communicate with

patient showing respectand compassion

-Display proper aseptic

technique and universal

precautions

PGY1/2

-Develop plan for IVplacement considering

medical and surgical

factors effecting gauge,

position, and timing of IV

placement

CA2

- Display knowledge of

advance IV access

modalities

CA2

-Demonstrate skills needed

to establish IV access in

difficult patient- Effectively Place IV in

uncomplicated patient with

little or no assistance

CA2

CA3 CA3 CA3


15/17

The Project Phase 4

Develop Web based evaluation form

Rotating selection of 10 15 questions

Substandard rating result in automatic expansion of question

Resident performance tracked System tracks which competencies have been evaluated

Comparison of resident self evaluation vs. faculty evaluation

Deficiencies characterized to create remediation plan focused on

specific problems and with identifiable points for reevaluation

Faculty Clinical Teaching and Evaluation tracked Puts evaluations in perspective with norms

Identify Biases / Halo Effect

Part of 360 degree evaluation


16/17

Project Time Line

Phase 1 / 2

Underway for General Anesthesia

Working model by June 03 Subspecialties added by Fall 03

Phase 3 / 4

First meeting with program team 15 April

Database parameters by June 03

Phase 3 Beta testing by Sept 03

Phase 4 Beta testing by Fall 03


17/17

Questions

?

pbli - evaluation project

Documents