pifmanship 101 pifmanship 101 (or how to survive writing/reviewing program information forms) ann m....

PIFmanship 101PIFmanship 101(or how to survive writing/reviewing program

information forms)

Ann M. Dohn, MA Director, GMEAnn M. Dohn, MA Director, GMEDesignated Institutional Official (DIO) &Designated Institutional Official (DIO) &

Nancy Piro, PhDNancy Piro, PhDProgram Manager/Education SpecialistProgram Manager/Education Specialist

Stanford University Medical Center Stanford University Medical Center Department of Graduate Medical EducationDepartment of Graduate Medical Education

DisclosureDisclosure

No conflicts of interest to reportNo conflicts of interest to report

Disclosure #2Disclosure #2

We have made every attempt We have made every attempt to protect the identities of the to protect the identities of the guilty.guilty.

“I see a Site Visit coming in your future…”

“I see a Site Visit coming in your future…”

PIFmanship 101PIFmanship 101

PIF Writing 101PIF Writing 101

Gentle words of wisdom…Gentle words of wisdom…– The first impression of your The first impression of your

program for the site visitor will be program for the site visitor will be your PIF.your PIF.

Be accurateBe accurate

Answer the questions (be terse…be Answer the questions (be terse…be tight)tight)

Have documentation to back up your Have documentation to back up your answersanswers

Start early-it takes months to write a Start early-it takes months to write a good PIFgood PIF

This is not something your mother, This is not something your mother, spouse, best friend, or admin asst. spouse, best friend, or admin asst. can do for you!can do for you!

General InformationGeneral InformationGlossary of ACGME terms : Glossary of ACGME terms :

http://www.acgme.org/acWebsite/about/ab_ACGMEglossary.pdf

Examples of “common information” Examples of “common information” required in the PIFs we provide to our required in the PIFs we provide to our Program Directors & Coordinators:Program Directors & Coordinators:

1) Hospital Statistics:1) Hospital Statistics:– 613 Licensed Beds at SHC613 Licensed Beds at SHC

– 272 Beds at LPCH272 Beds at LPCH 52 Obstetric52 Obstetric

220 Pediatric220 Pediatric

2. Major Participating 2. Major Participating Institutions (Affiliates)Institutions (Affiliates)

1. Children's Health Council [058191]

2. Kaiser Permanente Medical Center (Santa Clara) [050571]

3. Kaiser Santa Teresa [058092]4. Lucile Salter Packard Children's

Hospital at Stanford [050572]5. San Mateo Medical Center

[050585]6. Santa Clara Valley Medical

Center [050438]

7. Veterans Affairs Palo Alto Health Care System [050273]

3. Library Resources3. Library Resources

http://lane.stanford.edu/biomed-resources/index.html?laneNav=3

BackgroundBackground

There are actually 2 PIFS to There are actually 2 PIFS to complete:complete:– The Common PIFThe Common PIF

Addresses the program’s compliance with Addresses the program’s compliance with the the Common Program RequirementsCommon Program Requirements common to all GME programs.common to all GME programs.

The common PIF is electronically The common PIF is electronically generated through the Web Accreditation generated through the Web Accreditation Data System (WebADS)Data System (WebADS)

– The Specialty PIFThe Specialty PIFAddresses compliance with the specialty Addresses compliance with the specialty specific program requirementsspecific program requirements

Word document downloaded from the Word document downloaded from the ACGME SiteACGME Site

Common PIFCommon PIF

Common Program Information FormCommon Program Information Formwww.acgme.org

Use your ACGME assigned User ID & Use your ACGME assigned User ID & passwordpassword

Go to the PIF preparation section (left hand Go to the PIF preparation section (left hand menu)menu)

Update the Common PIF data. Update the Common PIF data.

Most of the data should be updated annually Most of the data should be updated annually by your by your coordinatorcoordinator..

The common PIF contains questions The common PIF contains questions regarding participating sites, faculty CVs, regarding participating sites, faculty CVs, evaluation, and duty hours. evaluation, and duty hours.

Some information is entered only at site visit Some information is entered only at site visit time. time.

Once all of the data is entered and Once all of the data is entered and VALIDATED VALIDATED

PrintPrint

Version ControlVersion Control

Start by keeping track of which Start by keeping track of which version of the PIF you are version of the PIF you are working on…working on…

You must hit “refresh” in You must hit “refresh” in WEBADS to save your WEBADS to save your changeschanges

““Refresh” before printingRefresh” before printing

Common PIF –1Common PIF –1


Data feeds from ACGME Data feeds from ACGME WebADSWebADS

TWO (or THREE) Original TWO (or THREE) Original Signatures RequiredSignatures Required– Program DirectorProgram Director– Core Program Director for Core Program Director for

FellowshipsFellowships– DIODIO

OopsOops

Do you really need to have Do you really need to have original signatures on the face original signatures on the face page?page?– YES!!!!!!YES!!!!!!– (we were cited on this)(we were cited on this)

Common PIF – 2Common PIF – 2


Respond to previous citationsRespond to previous citations– Data feeds from ACGME Data feeds from ACGME

WebADSWebADS– If you had a progress report due, If you had a progress report due,

the data remains in WebADS the data remains in WebADS Automatically populated into your Automatically populated into your next PIF next PIF

OopsOops


Major Changes are:Major Changes are:– Only since last site visitOnly since last site visit– Program Format, e.g.,Program Format, e.g.,

Have you gone from 3 to 4 years?Have you gone from 3 to 4 years?

– Change in Resident Change in Resident Complement?Complement?

– Change in Program Leadership?Change in Program Leadership?– Change in Participating Sites?Change in Participating Sites?

Ah Come On Now…Ah Come On Now…

Too Little (they had a Too Little (they had a new program director)new program director)

Too Much (they hadToo Much (they had an old program an old program

director)director) List the outstanding or special feature of the program List the outstanding or special feature of the program • Stanford’s fellowship program has a long history of training academicians and Stanford’s fellowship program has a long history of training academicians and

future leaders in XX future leaders in XX • and XX and is greatly enhanced by the NIH institutional grants and the tremendous and XX and is greatly enhanced by the NIH institutional grants and the tremendous

resources available at Stanford University. The program has an NIH Institutional resources available at Stanford University. The program has an NIH Institutional Research Training grant (T32) for the past 20+ years, and the division also has an Research Training grant (T32) for the past 20+ years, and the division also has an NIH institutional Digestive Disease Center grant (P30) for the past 10+ years which NIH institutional Digestive Disease Center grant (P30) for the past 10+ years which is currently undergoing re-submission. The Stanford campus has great talents and is currently undergoing re-submission. The Stanford campus has great talents and excellent opportunities for both education and research. The key faculty members excellent opportunities for both education and research. The key faculty members have strong patient care and research activities, and are recognized leaders in their have strong patient care and research activities, and are recognized leaders in their fields both nationally and internationally. The institutional training grant provides fields both nationally and internationally. The institutional training grant provides both salary and tuition support for fellows to initiate and eventually pursue an both salary and tuition support for fellows to initiate and eventually pursue an academic career. In terms of research, one important feature of the fellowship academic career. In terms of research, one important feature of the fellowship program is the tremendous amount of expertise and resources available at Stanford program is the tremendous amount of expertise and resources available at Stanford University. Increasingly, we are encouraging fellows to be co-mentored by faculty University. Increasingly, we are encouraging fellows to be co-mentored by faculty members in other fields and be groomed to be the future leaders in those fields members in other fields and be groomed to be the future leaders in those fields through an inter-disciplinary through an inter-disciplinary

• approach between the and other Divisions or Departments of the School of approach between the and other Divisions or Departments of the School of Medicine. We have a long Medicine. We have a long

• track record of training fellows in health outcomes research, in collaboration with track record of training fellows in health outcomes research, in collaboration with the Stanford Health Policy and Research Center. Fellows pursuing a career in this the Stanford Health Policy and Research Center. Fellows pursuing a career in this field usually also enroll in a formal Master’s degree program during their second field usually also enroll in a formal Master’s degree program during their second and third year of their fellowship. The second area is Biodesign. This is a natural fit and third year of their fellowship. The second area is Biodesign. This is a natural fit for XX since this is a procedure-oriented subspecialty with emerging areas of for XX since this is a procedure-oriented subspecialty with emerging areas of biodesign research activities, such as natural orifice translumenal endoscopic biodesign research activities, such as natural orifice translumenal endoscopic surgery (NOTES), that require the development of new equipment and techniques. surgery (NOTES), that require the development of new equipment and techniques. Stanford has a world renowned biodesign group, that is training leaders in Stanford has a world renowned biodesign group, that is training leaders in biomedical biomedical

• technology and innovation and has been offering a 1-year biodesign fellowship technology and innovation and has been offering a 1-year biodesign fellowship since 2001. We have developed a Biodesign tract in our fellowship program in since 2001. We have developed a Biodesign tract in our fellowship program in which our fellows spend their second year as a biodesign fellow and spending the which our fellows spend their second year as a biodesign fellow and spending the third year in completing a research project in biodesign. Another new program is third year in completing a research project in biodesign. Another new program is modeled after the biodesign program but focuses on operational research is the modeled after the biodesign program but focuses on operational research is the Clinical Excellence Research center. One of our current second year fellows is Clinical Excellence Research center. One of our current second year fellows is being co-mentored by Dr. Arnold Milstein, Director of the Clinical Excellence being co-mentored by Dr. Arnold Milstein, Director of the Clinical Excellence Research Center. Further we are in the process of developing a research tract in in Research Center. Further we are in the process of developing a research tract in in conjunction with the Global Health Program at Stanford and one of our 2012 fellows conjunction with the Global Health Program at Stanford and one of our 2012 fellows will be focusing on this. All these opportunities make the Stanford XX fellowship will be focusing on this. All these opportunities make the Stanford XX fellowship program unique in the country and the world, generating pioneers in new research program unique in the country and the world, generating pioneers in new research areas areas

• and training future research and academic leaders. and training future research and academic leaders. • A major strength of our clinical program is its diverse patient population cared for A major strength of our clinical program is its diverse patient population cared for

by a diverse faculty of basic and …by a diverse faculty of basic and …

Common PIF – 3Common PIF – 3


Participating SitesParticipating Sites– Feeds from ACGME WebADSFeeds from ACGME WebADS– Stanford, for example, is NOT a Stanford, for example, is NOT a

single limited site sponsor as we single limited site sponsor as we have 85 programshave 85 programs

– Only required sites appear on Only required sites appear on the PIFthe PIF

Required means all residents rotate Required means all residents rotate to that siteto that site

– Length of rotationLength of rotationMake sure the length of rotations Make sure the length of rotations adds up to 12 months per year adds up to 12 months per year across all sitesacross all sites

Common PIF - 4Common PIF - 4

Common PIFCommon PIFParticipating SitesParticipating Sites– Program Letters of Agreement Program Letters of Agreement

(PLAs)(PLAs)We provide a PLA template on our We provide a PLA template on our GME Website for Programs to useGME Website for Programs to use

Must be signed and datedMust be signed and dated

The PLA may be between you and The PLA may be between you and yourself…yourself…


Program Director MUST be able to Program Director MUST be able to approve the teaching facultyapprove the teaching faculty

The Program Director MUST The Program Director MUST evaluate the faculty and approve evaluate the faculty and approve continuation as teaching facultycontinuation as teaching faculty

Program Directors MUST comply Program Directors MUST comply with the university’s or medical with the university’s or medical center’s policies on selection, center’s policies on selection, evaluation, promotion, disciplinary evaluation, promotion, disciplinary action and supervision.action and supervision.– e.g., at Stanford we refer them to: e.g., at Stanford we refer them to: The The

GME Housestaff Policies and GME Housestaff Policies and Procedures – on the GME WebsiteProcedures – on the GME Website

http://med.stanford.edu/gme/policy/


The Program Director MUST The Program Director MUST be familiar with ACGME be familiar with ACGME Policies and ProceduresPolicies and Procedures– Institutional RequirementsInstitutional Requirements– Common RequirementsCommon Requirements– Program Specific RequirementsProgram Specific Requirements


Physician Faculty RosterPhysician Faculty Roster– List Alphabetically and List Alphabetically and by Siteby Site– Faculty who Faculty who spend at least 10 spend at least 10

hrs per weekhrs per week in resident in resident educationeducation

– Make sure the data matches Make sure the data matches their CVstheir CVs


Faculty Teaching StaffFaculty Teaching Staff– Hours/week devoted to Hours/week devoted to

educationeducationShould be realistic Should be realistic

– Board certificationBoard certificationIf double boarded include both If double boarded include both boardsboards

If you are sub-specialty program If you are sub-specialty program director or faculty, ensure primary director or faculty, ensure primary boards are includedboards are included

OopsOops

Common PIF- 6Common PIF- 6


Potential Area for a HUGE Potential Area for a HUGE number of citations for Programs:number of citations for Programs:– Must be totally completed (no Must be totally completed (no

BLANKS)BLANKS)– Ensure certification is validEnsure certification is valid– Ensure license is current and hasn’t Ensure license is current and hasn’t

lapsedlapsed– Is your program director board Is your program director board

certified?certified?– Are your faculty board certified?Are your faculty board certified?– How old are the publications?How old are the publications?– How old are the appointments?How old are the appointments?– Etc..Etc..

Are We Confused?Are We Confused?

Concise Summary of Role in Concise Summary of Role in Program:Program:

Dr. Donald Duck is the Chief of Dr. Donald Duck is the Chief of the “Animation” Fellowship the “Animation” Fellowship program. The associate director program. The associate director is Dr. Mickey Mouse the former is Dr. Mickey Mouse the former director. In the capacity of Co-director. In the capacity of Co-Director of the Fellowship, he Director of the Fellowship, he serves to devise and structure serves to devise and structure the clinical training rotations for the clinical training rotations for the fellowsthe fellows


– Concise Summary of Role in Concise Summary of Role in Program:Program:

<Program Director> oversees <Program Director> oversees the operations of the entire the operations of the entire program; supervises trainees program; supervises trainees during patient-care activities, during patient-care activities, mentors trainees’ mentors trainees’ research/projects, provides research/projects, provides professional training, tutorials professional training, tutorials (lectures and seminars); (lectures and seminars); monitors duty hour compliance; monitors duty hour compliance; coordinates evaluation of coordinates evaluation of courses, rotations, trainees and courses, rotations, trainees and faculty.faculty.


Selected BibliographySelected Bibliography– Strict limit of 10Strict limit of 10– Strict limit only Strict limit only the last five years the last five years

-“Publications from the last 5 -“Publications from the last 5 years” excludes any before years” excludes any before 2007…2007…

– Publications should not be “in Publications should not be “in press” if submitted many years press” if submitted many years ago….ago….

– Submitted doesn’t equal a Submitted doesn’t equal a “Publication”“Publication”

If not ABMS certified, explain…If not ABMS certified, explain…

Common PIF - 7 Common PIF - 7


Number of ACGME Approved Number of ACGME Approved PositionsPositions– Note: If you have a resident Note: If you have a resident

making up a maternity leave, you making up a maternity leave, you must ask ACGME for an extra must ask ACGME for an extra slot if you are over your capslot if you are over your cap

Actively enrolled residentsActively enrolled residents– Other than interns, everyone Other than interns, everyone

should have prior years of GMEshould have prior years of GME

Common PIF-10 Common PIF-10


Transferred, Withdrawn or Transferred, Withdrawn or Dismissed ResidentsDismissed Residents– Residents who resign are NOT Residents who resign are NOT

dismisseddismissed– Must have summative evaluations Must have summative evaluations

on transferred traineeson transferred trainees

EvaluationsEvaluations– Yes residents are evaluated Yes residents are evaluated

following each learning experience following each learning experience (ie. rotation)(ie. rotation)

– Yes- evaluations are documented Yes- evaluations are documented and state howand state how

ElectronicallyElectronically

FilesFiles


Methods of EvaluationMethods of Evaluation– Assessment MethodAssessment Method

Direct ObservationDirect Observation

SimulationSimulation

OSCEOSCE

EtcEtc

– EvaluatorEvaluatorFacultyFaculty

NursesNurses

Program DirectorProgram Director

360 (required as of 2007)360 (required as of 2007)

Common PIF -11Common PIF -11

Common PIFCommon PIFEvaluationEvaluation

Describe how evaluators are Describe how evaluators are educated to use the assessment educated to use the assessment methods listed above so that methods listed above so that residents are evaluated fairly and residents are evaluated fairly and consistently.consistently.– The Program Director meets with The Program Director meets with

evaluators annually before new fellows evaluators annually before new fellows start in the training program and start in the training program and reviews, answers questions and reviews, answers questions and discusses the core competencies and discusses the core competencies and core competency performance core competency performance evaluations used in the program and evaluations used in the program and assessment methods with the faculty. assessment methods with the faculty. The electronic assessment system and The electronic assessment system and the scales and their use is also reviewed the scales and their use is also reviewed and discussed in faculty meetings in and discussed in faculty meetings in order to ensure that faculty are fully order to ensure that faculty are fully educated and up to date with the educated and up to date with the assessment methods and processes. assessment methods and processes.


Describe how residents are informed of Describe how residents are informed of the performance criteria on which they the performance criteria on which they will be evaluated.will be evaluated.

– At the beginning of each academic year, the At the beginning of each academic year, the Program Director conducts an orientation to Program Director conducts an orientation to address several key issues related to the address several key issues related to the residency including the performance criteria on residency including the performance criteria on which the residents will be evaluated. During which the residents will be evaluated. During this orientation session, the Program Director this orientation session, the Program Director carefully details the specific evaluation methods carefully details the specific evaluation methods to be used. Both the criteria and methods are to be used. Both the criteria and methods are also documented in the residency handbook. also documented in the residency handbook. Additionally, at the beginning of each block Additionally, at the beginning of each block rotation, the rotation director meets with the rotation, the rotation director meets with the resident and carefully delineates the resident and carefully delineates the expectations and performance criteria on which expectations and performance criteria on which the resident will be evaluated for that specific the resident will be evaluated for that specific rotation.rotation.


Describe the system to ensure that Describe the system to ensure that faculty complete written faculty complete written evaluations of residents in a timely evaluations of residents in a timely manner following each rotation or manner following each rotation or educational experience.educational experience.– At the end of each block rotation, our At the end of each block rotation, our

online resident data collection and online resident data collection and tracking system, <our residency tracking system, <our residency management system>, sends an management system>, sends an automated reminder to the service automated reminder to the service attending(s) to evaluate the resident(s) on attending(s) to evaluate the resident(s) on that particular rotation. Using <the that particular rotation. Using <the system> the Program Coordinator tracks system> the Program Coordinator tracks pending evaluations and follows up with pending evaluations and follows up with faculty as needed to urge them to faculty as needed to urge them to complete their evaluations on time. In the complete their evaluations on time. In the unlikely event that a faculty member is unlikely event that a faculty member is unresponsive to the coordinator's unresponsive to the coordinator's requests to complete evaluations, the requests to complete evaluations, the Program Director contacts the faculty Program Director contacts the faculty member and requests him/her to member and requests him/her to complete the evaluations. complete the evaluations.


Describe the process used to complete Describe the process used to complete and document written semiannual and document written semiannual resident evaluations, including the resident evaluations, including the mechanism for reviewing results (e.g., mechanism for reviewing results (e.g., who meets with the residents and how who meets with the residents and how the results are documented in resident the results are documented in resident files).files).

– Residence performance is evaluated by the Residence performance is evaluated by the teaching faculty at the conclusion of each rotation teaching faculty at the conclusion of each rotation using an electronic evaluation form <the system>. using an electronic evaluation form <the system>. Additionally, each faculty mentor meets with their Additionally, each faculty mentor meets with their resident advisee quarterly and documents a resident advisee quarterly and documents a summary of the meeting to be placed in the summary of the meeting to be placed in the resident’s file. The Program Director meets with resident’s file. The Program Director meets with each resident on an individual basis semiannually each resident on an individual basis semiannually to review the accumulated performance to review the accumulated performance evaluations and mentor notes, provide feedback, evaluations and mentor notes, provide feedback, and update the resident’s learning plan as and update the resident’s learning plan as appropriate. A summary of these meetings is appropriate. A summary of these meetings is documented and placed in the resident’s file. The documented and placed in the resident’s file. The residents are free to review the contents of their residents are free to review the contents of their records at any time.records at any time.


Describe the system used by the Describe the system used by the residents to provide annual confidential residents to provide annual confidential written evaluations of the teaching written evaluations of the teaching faculty (have examples and forms faculty (have examples and forms available for review by site visitor).available for review by site visitor).

– Residents annually confidentially evaluate the Residents annually confidentially evaluate the teaching faculty using <system>. The teaching faculty using <system>. The electronically submitted <system>. evaluation electronically submitted <system>. evaluation forms are anonymous. Paper copies of completed forms are anonymous. Paper copies of completed evaluations without personal identifiers are evaluations without personal identifiers are printed by the Program Coordinator, reviewed by printed by the Program Coordinator, reviewed by the Program Director, and presented to individual the Program Director, and presented to individual faculty members for review and consideration. faculty members for review and consideration.

– Written evaluations are completed electronically Written evaluations are completed electronically and anonymously by using a centralized system and anonymously by using a centralized system used throughout the hospital to provide annual used throughout the hospital to provide annual aggregated data. Fellows are sent e-mail aggregated data. Fellows are sent e-mail reminders about completing evaluations in a reminders about completing evaluations in a timely manner by the Fellowship Coordinator. timely manner by the Fellowship Coordinator. The program coordinator follows up, aggregates The program coordinator follows up, aggregates the results and forwards them to the Program the results and forwards them to the Program Director. The Program Director reviews the Director. The Program Director reviews the results and takes any actions appropriate with results and takes any actions appropriate with individual program faculty.individual program faculty.


Describe the program's (or Department's, Describe the program's (or Department's, if applicable) system for evaluating and if applicable) system for evaluating and providing feedback to the teaching providing feedback to the teaching faculty.faculty.

– Residents annually evaluate the teaching faculty Residents annually evaluate the teaching faculty using the <system>. The electronically submitted using the <system>. The electronically submitted evaluation forms are anonymous. Paper copies of evaluation forms are anonymous. Paper copies of completed evaluations without personal identifiers completed evaluations without personal identifiers are printed by the Program Coordinator, reviewed are printed by the Program Coordinator, reviewed by the Program Director, and presented to by the Program Director, and presented to individual faculty members for review and individual faculty members for review and consideration. If and when the evaluations reveal consideration. If and when the evaluations reveal particular issues with the teaching faculty, the particular issues with the teaching faculty, the Program Director meets with the particular faculty Program Director meets with the particular faculty more urgently to address those issues and more urgently to address those issues and concerns. Additionally, the Program Director concerns. Additionally, the Program Director meets with the all faculty on an annual basis to meets with the all faculty on an annual basis to review resident feedback and to effect any review resident feedback and to effect any necessary changes.necessary changes.

Common PIFCommon PIFProgram EvaluationProgram Evaluation

Describe the approach used for program Describe the approach used for program evaluation.evaluation.

– Residents and faculty annually evaluate the Residents and faculty annually evaluate the program using the program using the <system><system>. The . The electronically submitted evaluation forms are electronically submitted evaluation forms are anonymous. Residents and faculty are also anonymous. Residents and faculty are also encouraged to provide feedback to the encouraged to provide feedback to the Program Director whenever any issue arises Program Director whenever any issue arises or as they see opportunities for improvement. or as they see opportunities for improvement. Additionally, residents and faculty participate Additionally, residents and faculty participate in an Annual Program Improvement Meeting in an Annual Program Improvement Meeting led by the Program Director. Aggregated data led by the Program Director. Aggregated data including the most recent ACGME survey including the most recent ACGME survey results and the resident/faculty program results and the resident/faculty program evaluation results are reviewed and used at evaluation results are reviewed and used at this meeting to improve the program. The this meeting to improve the program. The Program Coordinator keeps minutes during Program Coordinator keeps minutes during this annual meeting and documents any plans this annual meeting and documents any plans to address areas requiring improvement. to address areas requiring improvement. Action planAction plan progress is progress is monitoredmonitored and and documenteddocumented by the Program Director. by the Program Director.

Common PIFCommon PIFProgram EvaluationProgram Evaluation

Describe one example how the program Describe one example how the program used the aggregated results of residents' used the aggregated results of residents' performance and/or other program performance and/or other program evaluation results to improve the evaluation results to improve the program (have the written plan of action program (have the written plan of action available for review by the site visitor).available for review by the site visitor).

– Annual fellowship retreats including Annual fellowship retreats including fellows and faculty are conducted each fellows and faculty are conducted each year in May or June (Whenever yours is year in May or June (Whenever yours is held) ….the agenda for these meetings held) ….the agenda for these meetings includes review of all the aforementioned includes review of all the aforementioned aggregated data as well an open aggregated data as well an open discussion about educational quality and discussion about educational quality and compliance with the program compliance with the program requirements and suggestions for requirements and suggestions for improving the program for incoming improving the program for incoming residents. (See GME Website for more residents. (See GME Website for more information on this and examples of information on this and examples of written action plans)written action plans)

– http://med.stanford.edu/gme/program_directors/eval_prog.html

Common PIF -12Common PIF -12


Describe the improvement Describe the improvement efforts currently undertaken efforts currently undertaken based on feedback from the based on feedback from the ACGME Resident Survey.ACGME Resident Survey.– You MUST review your ACGME You MUST review your ACGME

Annual Survey each year, bring Annual Survey each year, bring the results to your Annual the results to your Annual Program Retreat, address each Program Retreat, address each area of concern (more than 20%) area of concern (more than 20%) or any Duty Hour negative or any Duty Hour negative responseresponse


DUTY HOURSDUTY HOURS– Use the summary from your Use the summary from your

tracking system reporting tracking system reporting functionfunction

– Describe Describe


Briefly describe how the Briefly describe how the faculty provides appropriate faculty provides appropriate supervision of residents in supervision of residents in patient care activities.patient care activities.– Faculty discusses all patient care Faculty discusses all patient care

history and physical findings with history and physical findings with resident. Faculty then confirms resident. Faculty then confirms both the history and physical both the history and physical findings of the patient. The patient findings of the patient. The patient is then discussed in detail, is then discussed in detail, education given regarding the education given regarding the patient illness or complaint, patient illness or complaint, education materials are provided education materials are provided to the resident and finally, a care to the resident and finally, a care plan is developed and provided by plan is developed and provided by the resident and confirmed by the the resident and confirmed by the faculty. faculty.


Other Questions that may be Other Questions that may be included in you Common or included in you Common or Specialty PIFSpecialty PIF

Other PIF QuestionsOther PIF Questions

Skills and CompetenciesSkills and Competencies– Describe how residents are Describe how residents are

informed about their informed about their assignments and duties during assignments and duties during the residency.the residency.

This should be in writing and verbal. This should be in writing and verbal. For example, all fellows are given For example, all fellows are given the program training manual which the program training manual which describes their assignments and describes their assignments and duties during orientation to the duties during orientation to the program. Each fellow signs an program. Each fellow signs an attestation that they have received attestation that they have received and read the manual. The manual is and read the manual. The manual is discussed during orientation with discussed during orientation with the fellows as a group. the fellows as a group.

Just Answer the Just Answer the Question!Question!

Describe Describe howhow fellows are informed about their assignments fellows are informed about their assignments and duties during the fellowship.and duties during the fellowship.

Fellows are informed about their assignments at the Fellows are informed about their assignments at the beginning of the fellowship.beginning of the fellowship.Handouts are provided which describe the goals and Handouts are provided which describe the goals and objectives for each rotation, as well as a reading list of objectives for each rotation, as well as a reading list of recommended articles. fellows are informed about their recommended articles. fellows are informed about their assignments at the beginning of the fellowship. assignments at the beginning of the fellowship. Handouts are provided which describe the goalsHandouts are provided which describe the goalsand objectives for each rotation, as well as a reading and objectives for each rotation, as well as a reading list of recommended articles.list of recommended articles.

The Six Core competencies for each rotation includeThe Six Core competencies for each rotation include1. Patient Care1. Patient Care2. Medical Knowledge2. Medical Knowledge3. Practice-based learning & improvement,3. Practice-based learning & improvement,4. Interpersonal & Communication Skills4. Interpersonal & Communication Skills5. Professionalism5. Professionalism6. Systems-based Practices6. Systems-based Practices

Competencies differ for each rotation and will be Competencies differ for each rotation and will be provided.provided.

Other PIF QuestionsOther PIF Questions

GENERALIZE-GENERALIZE-

Grievance ProceduresGrievance Procedures– Describe how the program Describe how the program

handles complaints or handles complaints or concerns the residents raise.concerns the residents raise.

The program follows the Dispute The program follows the Dispute Resolution (grievance ) procedures Resolution (grievance ) procedures described in the Stanford University described in the Stanford University Medical Center House Staff Policies Medical Center House Staff Policies & Procedures, pages 20-24, & Procedures, pages 20-24, distributed to residents at the distributed to residents at the beginning of their residency and beginning of their residency and available on the available on the GME website

Other PIF QuestionsOther PIF QuestionsDescribe how the program monitors for excessive Describe how the program monitors for excessive service and modifies the program accordingly.service and modifies the program accordingly.

– The residency has several methods to monitor for The residency has several methods to monitor for excessive service and modify the program if required:excessive service and modify the program if required:

Formal rotation evaluations with alerts: Formal rotation evaluations with alerts: Residents complete monthly evaluations using Residents complete monthly evaluations using the system that are reviewed and the system that are reviewed and communicated with the rotation directors.communicated with the rotation directors.Committee on Residency Training and Clinical Committee on Residency Training and Clinical Services: Each class has resident Services: Each class has resident representatives meet with the Program Director representatives meet with the Program Director monthly for a formal agenda to review and monthly for a formal agenda to review and address issues in the residency program.address issues in the residency program.The rotation directors are sent the residents’ The rotation directors are sent the residents’ monthly feedback on a quarterly basis (to monthly feedback on a quarterly basis (to ensure confidentiality for the residents’ ensure confidentiality for the residents’ feedback)feedback)Yearly, the residents have a retreat in which Yearly, the residents have a retreat in which each aspect of the program is reviewed. Area of each aspect of the program is reviewed. Area of concern are brought back to the Committee on concern are brought back to the Committee on Residency Training. The ACGME and GME Residency Training. The ACGME and GME resident survey questions with respect to resident survey questions with respect to excessive service are reviewed.excessive service are reviewed.Informal feedback: the residents are Informal feedback: the residents are encouraged to let the Chief Residents, the encouraged to let the Chief Residents, the Faculty, and the Program Director know Faculty, and the Program Director know immediately when either the service load or the immediately when either the service load or the educational activities are compromised.educational activities are compromised.Weekly meetings of the program director and Weekly meetings of the program director and chief residents to discuss the residents and chief residents to discuss the residents and experiences on individual rotations.experiences on individual rotations.Computerized entry and monitoring of all hours Computerized entry and monitoring of all hours is performed in <the system>.is performed in <the system>.

Duty HoursDuty Hours

““Other”Other”

SPECIALTY PIFSPECIALTY PIF

The specialty specific PIF is located The specialty specific PIF is located on the ACGME website under your on the ACGME website under your Residency Review Committee.Residency Review Committee.– http://acgme.org/acWebsite/navPages/

nav_PDcoord.asp

The specialty PIF contains the The specialty PIF contains the questions regarding the ACGME questions regarding the ACGME core competencies, block diagrams core competencies, block diagrams for your program, a narrative for your program, a narrative description of the program, description of the program, scholarly activity, and often case scholarly activity, and often case logs.logs.

Specialty PIF-Practice-Specialty PIF-Practice-Based Learning & Based Learning &

ImprovementImprovement

Describe one learning activity in Describe one learning activity in which residents engage to identify which residents engage to identify strengths, deficiencies, and limits strengths, deficiencies, and limits in their knowledge and expertise in their knowledge and expertise (self-reflection and self-(self-reflection and self-assessment); set learning and assessment); set learning and improvement goals; identify and improvement goals; identify and perform appropriate learning perform appropriate learning activities to achieve self-identified activities to achieve self-identified goals (life-long learning).goals (life-long learning).

– Programs may use a structured process Programs may use a structured process for reflection in which a faculty advisor for reflection in which a faculty advisor guides the resident in using feedback guides the resident in using feedback and evaluations to inform the self and evaluations to inform the self assessment process. Documentation of assessment process. Documentation of the semi-annual evaluation meetings in the semi-annual evaluation meetings in which this process is demonstrated which this process is demonstrated would provide evidence that this would provide evidence that this requirement is being addressed. requirement is being addressed.

Specialty PIF-Practice-Specialty PIF-Practice-Based Learning & Based Learning &

ImprovementImprovementDescribe one example of a learning Describe one example of a learning activity in which residents engage to activity in which residents engage to develop the skills needed to use develop the skills needed to use information technology to locate, information technology to locate, appraise, and assimilate evidence from appraise, and assimilate evidence from scientific studies and apply it to their scientific studies and apply it to their patients’ health problems. The patients’ health problems. The description should include: a) locating description should include: a) locating information, b) using information information, b) using information technology, c) appraising information, d) technology, c) appraising information, d) assimilating evidence information (from assimilating evidence information (from scientific studies), e) applying scientific studies), e) applying information to patient care.information to patient care.

– An appropriate learning activity could be a An appropriate learning activity could be a structured activity such as a journal club structured activity such as a journal club presentation, critical appraisal of a topic, or presentation, critical appraisal of a topic, or educational prescription with appropriate faculty educational prescription with appropriate faculty oversight and formal assessment of skills. oversight and formal assessment of skills. Additional documentation would be the written Additional documentation would be the written goals and objectives for this learning activity and goals and objectives for this learning activity and how residents are assessed.how residents are assessed.

Specialty PIF-Specialty PIF-ProfessionalismProfessionalism

Describe at least one learning Describe at least one learning activity, other than lecture, by activity, other than lecture, by which residents develop a which residents develop a commitment to carrying out commitment to carrying out professional responsibilities and professional responsibilities and an adherence to ethical an adherence to ethical principles.principles.

– This activity should be structured, This activity should be structured, should demonstrate active faculty should demonstrate active faculty involvement (not just passive role involvement (not just passive role modeling) and timely feedback to modeling) and timely feedback to residents, and should include a residents, and should include a mechanism for collecting evaluations mechanism for collecting evaluations (including routine multi-source (including routine multi-source assessment).assessment).

Specialty Systems-Specialty Systems-Based PracticeBased Practice

Describe an activity that fulfills the Describe an activity that fulfills the requirement for experiential requirement for experiential learning in identifying system learning in identifying system errors.errors.

– Important elements may include identified Important elements may include identified faculty to guide the activity, mechanism to faculty to guide the activity, mechanism to ensure active engagement by each ensure active engagement by each resident, and evidence of experiential resident, and evidence of experiential learning (not just passive presence at learning (not just passive presence at conferences and meetings) in which conferences and meetings) in which residents participate in identifying a residents participate in identifying a system problem or error and contribute to system problem or error and contribute to a potential solution.a potential solution.

– Additional documentation could include Additional documentation could include written goals and objectives for this written goals and objectives for this learning activity and how residents are learning activity and how residents are assessed. Aggregated resident outcomes assessed. Aggregated resident outcomes may be in the form of percentage of may be in the form of percentage of residents that completed a patient safety residents that completed a patient safety or other Systems-Based Practice project or other Systems-Based Practice project by the end of training, annual list of by the end of training, annual list of improvements that resulted from such improvements that resulted from such projects, etc.projects, etc.

PIFPIF

Briefly describe how the faculty Briefly describe how the faculty provides appropriate supervision provides appropriate supervision of residents in patient care of residents in patient care activities.activities.

Remember the new 2011 new Remember the new 2011 new supervision guidelines as you supervision guidelines as you answer this question.answer this question.

Table of ContentsTable of Contents

Common ErrorsCommon Errors

Table of Contents InaccurateTable of Contents Inaccurate

Pages not numbered or not Pages not numbered or not numbered correctlynumbered correctly

Document not spell-checkedDocument not spell-checked

Question asked not answeredQuestion asked not answered

Statistics not added correctlyStatistics not added correctly

% of time for faculty % of time for faculty inconsistent within the PIFinconsistent within the PIF

Type font does not matchType font does not match

Putting It All TogetherPutting It All TogetherREVISE---One complete PIF (common REVISE---One complete PIF (common and specialty) is due to the site visitor at and specialty) is due to the site visitor at least 14 days before the site visit.least 14 days before the site visit.For example, Stanford’s GME needs the For example, Stanford’s GME needs the complete PIF 14-30 days before it is send complete PIF 14-30 days before it is send to the site visitor. (the math means that to the site visitor. (the math means that the program should have the completed the program should have the completed document ready 6 weeks BEFORE your document ready 6 weeks BEFORE your site visit)site visit)Many large institutions have 3 site visits Many large institutions have 3 site visits during a week. This translates to 3 PIFS during a week. This translates to 3 PIFS to review. Help the poor folks out….and to review. Help the poor folks out….and be EARLY!!!! be EARLY!!!! The more time you give GME…the more The more time you give GME…the more time for editing, comments, and changes. time for editing, comments, and changes. You really cannot have too many eyes You really cannot have too many eyes look at one PIF!!look at one PIF!!

Save time in the PIF Site Visit/Internal Review Process

Leverage Inexpensive Technology Leverage Inexpensive Technology Applications e.g.,Applications e.g.,– PDF – PIF Review FunctionalityPDF – PIF Review Functionality– PDF – PDF – ii-annotate or similar-annotate or similar– Microsoft Word Track ChangesMicrosoft Word Track Changes

Documents revisions that are requiredDocuments revisions that are required

Electronic copies can easily be uploaded Electronic copies can easily be uploaded to internal web sites for easy access and to internal web sites for easy access and documentationdocumentation

Enables team reviews – from “anywhere”Enables team reviews – from “anywhere”

Provides Program Directors easy review Provides Program Directors easy review of recommendationsof recommendations

Provides Program Coordinators a Provides Program Coordinators a streamlined way to make many of the streamlined way to make many of the required changesrequired changes

The Land MinesThe Land Mines

As you write your PIF… As you write your PIF… remember to read each remember to read each question carefully.question carefully.– One example does NOT mean One example does NOT mean

severalseveral– ““How” does not mean “we do”How” does not mean “we do”– No “will’s”No “will’s”– No “see attached – “see below”No “see attached – “see below”

And now we are And now we are going “PIF-less”going “PIF-less”

A new processA new process

Somewhat scarySomewhat scary– The unknown makes us anxiousThe unknown makes us anxious

So this is the Stanford So this is the Stanford experience from an ACGME experience from an ACGME pilot of three programs that had pilot of three programs that had PIF-less reviewsPIF-less reviews

Common Program and Common Program and Institutional RequirementsInstitutional Requirements– Policies and procedures for Policies and procedures for

residents’ duty hours and work residents’ duty hours and work environment (institutional and environment (institutional and program)program)

– Moonlighting policyMoonlighting policy– Documentation of internal Documentation of internal

review (date, participants’ titles, review (date, participants’ titles, type of date collected, and date type of date collected, and date of review by GMEC)of review by GMEC)

Documents We Had to Have

Ready for the PIF-Less Site Visit

Documents We Had to Documents We Had to Have Ready for the PIF-Less Have Ready for the PIF-Less

Site VisitSite Visit Common Program RequirementsCommon Program Requirements

Overall educational goals for the programOverall educational goals for the program

Competency-based goals and objectives for Competency-based goals and objectives for each experience at each educational leveleach experience at each educational level

Current Program Letters of Agreement (PLAs)Current Program Letters of Agreement (PLAs)

Files of current residents and most recent Files of current residents and most recent program graduates containing the following:program graduates containing the following:

– Evaluations of residents at the completion of Evaluations of residents at the completion of each assignmenteach assignment

– Evaluations showing use of multiple evaluators Evaluations showing use of multiple evaluators (e.g., faculty, peers, other professional staff, (e.g., faculty, peers, other professional staff, patients, and self-evaluation)patients, and self-evaluation)

– Documentation of residents’ semiannual Documentation of residents’ semiannual evaluations of performance with feedbackevaluations of performance with feedback

– Final (summative) evaluation of residents, Final (summative) evaluation of residents, documenting performance during the final documenting performance during the final period of education and verifying that the period of education and verifying that the resident has demonstrated sufficient resident has demonstrated sufficient competence to enter practice without competence to enter practice without supervision …..cont’dsupervision …..cont’d

Documents We Had to Documents We Had to Have Ready for the PIF-less Site VisitHave Ready for the PIF-less Site Visit

Completed annual written confidential evaluations of Completed annual written confidential evaluations of faculty by the residentsfaculty by the residents

Completed annual written confidential evaluations of the Completed annual written confidential evaluations of the program by the residents program by the residents

Completed annual written confidential evaluations of the Completed annual written confidential evaluations of the program by the facultyprogram by the faculty

Documentation of program evaluation and written Documentation of program evaluation and written improvement planimprovement plan

Documentation of resident duty hoursDocumentation of resident duty hours

If applicable, files of current residents who have If applicable, files of current residents who have transferred into the program including documentation of transferred into the program including documentation of previous experiences and competency-based previous experiences and competency-based performance evaluations.performance evaluations.

If applicable, files of residents who have transferred out of If applicable, files of residents who have transferred out of this program into another program, including this program into another program, including documentation of experiences and summative documentation of experiences and summative competency-based performance evaluations provided to competency-based performance evaluations provided to the receiving program director the receiving program director

Policy for recruitment, appointment, eligibility, selection, Policy for recruitment, appointment, eligibility, selection, and promotion of residentsand promotion of residents

Documents to Have Ready for Documents to Have Ready for the PIF-Less Site Visitthe PIF-Less Site Visit

New 2011 Common Standards New 2011 Common Standards

Transfer protocol handover/transfer and sample educational Transfer protocol handover/transfer and sample educational materials related to handovers/ transfersmaterials related to handovers/ transfers

Sample schedules that inform all members of the health care Sample schedules that inform all members of the health care team of attending physicians and residents currently team of attending physicians and residents currently responsible for each patient’s careresponsible for each patient’s care

Document review of policies for supervision and lines of Document review of policies for supervision and lines of supervisory responsibility supervisory responsibility Protocols defining common Protocols defining common circumstances requiring faculty involvement (care of a circumstances requiring faculty involvement (care of a complex patient, ICU transfer, DNR or other end of life complex patient, ICU transfer, DNR or other end of life decision (by year/level of training)decision (by year/level of training)

Protocol and (completed) sample documents for episodes Protocol and (completed) sample documents for episodes when residents remain on duty beyond scheduled hourswhen residents remain on duty beyond scheduled hours

Policies to ensure that residents have adequate rest Policies to ensure that residents have adequate rest between daily duty periods and after in-house call (showing between daily duty periods and after in-house call (showing differences by level/year of training)differences by level/year of training)

Sample documents offering evidence of resident Sample documents offering evidence of resident participation in QI and Safety Projects participation in QI and Safety Projects

Added documents for some programs Added documents for some programs (Site Visitor will clarify)(Site Visitor will clarify)

– Review of Case/Procedure logs, if Review of Case/Procedure logs, if specified in the program requirementsspecified in the program requirements

– Documentation of conference Documentation of conference attendance, if required by the Review attendance, if required by the Review CommitteeCommittee

– If available, examples of resident If available, examples of resident involvement in quality involvement in quality

– Any additional materials relevant to Any additional materials relevant to the review in the specialty (the review in the specialty (Site Site Visitor will specify)Visitor will specify)

Documents to Have Documents to Have Ready for the Ready for the

PIF-less Site VisitPIF-less Site Visit

New 2011 Common StandardsNew 2011 Common Standards

And now we are And now we are going “PIF- less”going “PIF- less”

Lessons learned from a pilot of Lessons learned from a pilot of the first “PIF-less” in August the first “PIF-less” in August 20112011– Details! Details! – Attention to organizationAttention to organization– Team workTeam work– Time managementTime management

And now we are And now we are going “PIF-less”going “PIF-less”

We had all of the required We had all of the required elements loaded into our elements loaded into our residency management systemresidency management system– PaperlessPaperless– 3 programs3 programs– Site visitors “driving” the data Site visitors “driving” the data

collection by the 2collection by the 2ndnd day day

– And it was truly a great And it was truly a great experience for ALL!!!!!experience for ALL!!!!!

PIFmanship 101PIFmanship 101

And that’s all folks…And that’s all folks…

Stay tuned for PIF-lessmanship 102 next year!

pifmanship 101 pifmanship 101 (or how to survive writing/reviewing program information forms) ann m....

Documents

common pif data

common information

printingcommon pif

pif writing

thiscommon pif

program leadership

common pifrespond

pif preparation section