effective evaluations: a guide for preceptors · astinet al. principles of good practice for...

6/27/2017

1

Effective Evaluations: A Guide for Preceptors

Randy Martin, Pharm.D., BCCCPDirector of Institutional Experiential Education

University of North Texas System College of Pharmacy

Texas Consortium of Experiential ProgramsTexas Pharmacy Association

San Antonio, Texas

• Effective Evaluations: A Guide for Preceptors is accredited by ACPE for pharmacists and technicians.

• Dr. Randy Martin has not disclosed any financial or conflicts of interest in relation to this program.

Learning Objectives• Compare and contrast each level of performance on the TCEP rubric scale and how that translates to

pharmacy students becoming “practice-ready”

• Utilize the TCEP student evaluation rubric to assess student performance at different stages of professional development

• Employ evaluations as teaching tools to elicit student growth and development at the beginning, midpoint, and end of rotation

• Combine TCEP activity rubrics with professional experience to effectively assess students’ progress toward becoming practice ready

• Discuss the role of end-of-rotation exit exams

• Utilize data provide by the student and college to improve preceptor and site performance

6/27/2017

2

Active Learning Ahead!

Instructions

• Evaluate the following Drug Information Responses using the TCEP Standardized Form at your table.

Student #1 – Drug Information Response

6/27/2017

3

Thank you for your question regarding the selection of the appropriate anticoagulation therapy for VTE prophylaxis in patients with severe renal impairment. Patients who are hospitalized and/or critically ill are at significant risk for thromboembolic events due to the presence of one or multiple contributing factors of Virchow’s triad: hypercoagulability, hemodynamic changes, and endothelial injury or dysfunction. Since multiple antithrombotic drugs undergo renal clearance, this can complicate drug selection for patients with renal dysfunction and necessitate greater emphasis on proper drug and dose selection for maintaining the balance between prevention of thrombosis and risk of bleed.Current CHEST Guidelines do not address specific considerations for this subpopulation beyond the recommendation that, in the setting of severe renal impairment (CrCl < 30 mL/min), doses of low-molecular-weight heparins should be reduced (IIC).1 A review of the literature suggests that data is lacking as many studies do not include patients with renal dysfunction, but the most well-established options at this time are unfractionated heparin, which does not require renal adjustment, and enoxaparin, which is the most studied LMWH for this population. Argatroban is an anticoagulant that has been tested specifically with renal impairment and does not require dose adjustment as it undergoes hepatic metabolism, however pharmacokinetic data is lacking.2 Due to a lack of evidence-based guidelines, clinical judgment is key in managing therapeutic efficacy without compromising patient safety. LMWH are widely used and still may be considered in patients with renal impairment, but it should be noted that a meta-analysis examining their use in these patients showed a 2-to-3 fold increase risk in major bleeding events when compared to patients with normal renal function.3 There is insufficient data to suggest that LMWH should be avoided entirely in this population, however if they should be used, dose adjustment is recommended.In summary, although there are a wide array of anticoagulant options for VTE prophylaxis in patients with renal impairment, it is important to assess the pharmacokinetic properties of each drug and the possible consequence of their use under these circumstances. The most studied options at this time are UFH and enoxaparin, but regardless of the specific drug selected, appropriate dose adjustment and close monitoring of relevant labs should be carried out.

References:1. Guyatt, G., Akl, E., Crowther, M., Gutterman, D., Schuunemann, H. (2012). Executive

Summary: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. CHEST, 141(2), 7S – 47S. doi:10.1378/chest.1412S3

2. Grand’Maison, A., Charest, A., Geerts, William. (2005). Anticoagulant use in patients with chronic renal impairment. American Journal of Cardiovascular Drugs, 5.5(Sept-Oct 2005), 291. Retrieved from http://go.galegroup.com.proxy.hsc.unt.edu/ps/i.do?id=GALE%7CA199464065&v=2.1&u=txshracd2595&it=r&p=HRCA&sw=w&asid=3d45643d116fabe8af870e0be57fb70f

3. Lim, W., Dentali, F., Eikelboom, J., Crowther, M. (2006). Meta-Analysis: Low-Molecular-Weight Heparin and Bleeding in Patients with Severe Renal Insufficiency. Ann Intern Med, 144(9), 673 – 684. doi:10.7326/0003-4819-144-9-200605020-00011


6/27/2017

4

Re: Avycaz treatment and renal impairment dosingTo begin with, thanks for the question regarding Avycaz (Ceftazidime/Avibactam), formerly known as NXL104, which is a new cephalosporin beta lactamase inhibitor combination approved in February 2015. It has activity against Gram negatives including some Klebsiella pneumoniae producing carbapenemases, class A enzymes and extended spectrum cephalosporinases that contain Amp C gene (Class C enzymes). It is administered intravenously 2.5mg every 8 hours IV Infusion over 2 hours for 5 to 14 days1.In a Phase III cIAI trial, it was observed that Avycaz was not metabolized by the kidney or liver, but was renally excreted in its unchanged form. Also, patients with baseline CrCl of 30 ml/min to 50 ml/min demonstrated lower clinical response2. In view, some recommendations of renal monitoring and dose adjustment were made.A Phase I, multicenter, open-label Pharmacokinetic Cohort study conducted in hospitalized pediatrics with infection and receiving systemic antibiotics concluded that a single dose of Avycaz was comparable between each of the four (4) age cohorts [Cohort 1: >12yrs to <18yrs, Cohort 2: >6yrs to <12 yrs; Cohort 3: >2 yrs to <6 yrs and Cohort 4: >3months to <2yrs] and also, no new safety concerns were identified3.In summary, patients with impaired kidney function currently taking or about to take Avycaz need their CrCl monitored as well as an adjustment in dosage [CrCl: 30 – 50 ml/min 1.25g Q8h and CrCl < 30ml/min 0.94g Q12h to 24h]4, 5. Besides, the question of usage by patients on dialysis has not been fully explored, as there are currently no research data, clinical trials or studies to further provide the much-anticipated answer. As such, patient on dialysis are not to be given this new medication.

References:1. Chahine, Elias B., Mariette Sourial, and Raquel Ortiz. "Ceftazidime/Avibactam: a new antibiotic

for Gram-Negative infections." The Consultant Pharmacist® 30.12 (2015): 695-705. Doi: 10.4140/TCP.n.2015.695. Review. PubMed PMID: 26671269. Web 24 Feb 2017.

2. Hidalgo, Jose A., Celeste M. Vinluan, and Nishaal Antony. "Ceftazidime/avibactam: a novel cephalosporin/nonbeta-lactam beta-lactamase inhibitor for the treatment of complicated urinary tract infections and complicated intra-abdominal infections." Drug Design, Development and Therapy 10 (2016): 2379. PubMed PMID PMC 4970634. Doi: 10.2147/DDDT.S110946. PMC. Web. 24 Feb 2017.

3. Bradley, John S., et al. "Phase I Study Assessing the Pharmacokinetic Profile, Safety, and Tolerability of a Single Dose of Ceftazidime-Avibactam in Hospitalized Pediatric Patients." Antimicrobial Agents and Chemotherapy 60.10 (2016): 6252-6259. PubMed PMID: PMC5038276. Doi: 10.1128/AAC.00862-16. PMC. Web. 24 Feb 2017.

4. Mosley, Juan F., et al. "Ceftazidime-Avibactam (Avycaz): For the Treatment of Complicated Intra-Abdominal and Urinary Tract Infections." Pharmacy and Therapeutics 41.8 (2016): 479. PubMed PMID: 27504064. PMC4959616. Web 24 Feb 2017.

5. Avycaz (Ceftazidime/Avibactam) Prescribing Information. Cincinnati, Ohio: Forest Pharmaceuticals Inc.; 2016. Available at www.allergan.com/assets/pdf/avycaz_pi. Accessed February 24, 2017


6/27/2017

5

Share at Your Table

• Compare your scores• What is the range of your scores?• What was the lowest? What was the highest?

• Discussion questions:• Did you want to change the grade for Student #1 after reading the response

from Student #2?• Would you have graded the responses differently if you only received one of

them?• Do you want to change the grades after discussing at your table?

CRITERIA SCORE 1 SCORE 2Drug Information QuestionDrug information question clearly defined; Summarized the question or therapeutic dilemma; Identified and retrieved pertinent data and information

Literature SourcesAbility to find and utilize appropriate literature sources; Appropriate use of primary, secondary, tertiary literature sources

Literature Review and EvaluationPerform critical analysis of primary literature; Summarized available data

Conclusion/Answer/Response Reached evidenced-based conclusion; Applies information from critical analysis of literature; Applies therapeutic and pharmacologic knowledge

Format of Presented MaterialWritten - Organized, concise response with logic, grammatically correct, no typographical errorsVerbal - Appropriate language for inquirer; appropriate in speech, clear, concise, and logical in organizationFielded questions effectively; Effectively communicates response to audienceProper citation of referencesProfessionalismSelf-directed and displays independenceConscientious and follows throughAttentive to detailsTimeliness in responseDetermines appropriate response time to drug information inquiry based on acuity of situation

Assessment in Experiential Education

6/27/2017

6

What is “Assessment”?

• Assessment is data collection and analysis

• Assessment seeks to answer the following questions:

• What do students know?• What do students understand?• What can students do with their

knowledge?• Assessment culminates in

quality improvement of the teaching/learning process

Learner-Centered Assessment on College Campuses: shifting the focus from teaching to learning. 2000.J of Higher Ed. 1989; 60(6):644-664.

Principles of Good Practice of Assessing Student Learning• Assessment

• Begins with educational values• Accepts that learning is multidimensional, integrated, and progressive• Has clear, explicit purposes• Pays equal attention to outcomes and the experiences that lead to those outcomes• Should be ongoing, not episodic• Involves representatives from across the educational community• Illuminates questions people care about• Should be integrated into a larger scheme of continuous quality improvement

• Through assessment, educations meet their responsibility to the student, public, and profession

Astin et al. Principles of good practice for assessing student learning. 1992. American Association for Higher Education.

Assessment of the Student

• Preceptor evaluation of the student

• Assignment rubrics

• Direct observations

• Written exit examinations

• Objective structured clinical examinations (OSCEs)

• Student self-evaluation

6/27/2017

7


Active Learning - Self-Assessment

• Reflect on your own process for assessment.

• How do you assess student performance as a preceptor?

• Does your process meet all of the good practices of assessment (right)?

Reflects educational valuesMultidimensional and integratedDetects progressive growthClear purposesIncludes outcomes and experiences Ongoing, not episodicInvolves across the educational

communityAnswers questions we care aboutPromotes continuous quality

improvement

What are our educational values?

• Practice-ready

• Team-ready

6/27/2017

8

What is Practice-ready?Perspective: Organizations/Expert Panels

Hospital Entry-Level Competencies

Professional Practice

Medication Use System

Medication Safety and

Quality

Clinical application

Community Entry-Level CompetenciesLeadership Legal

Considerations

Public Health

Business Management

Patient Care

Communication

Medication Use System

CommunicationProfessionalism

Patient CareProblem Solving

Thompson et al. Am J Pharm Educ. 2012; 76(9):170. 22

What is Practice-ready?Perspective: Practicing Pharmacists

0

5

10

15

20

25

Retail Hospital


What is Practice-ready?Perspective: Pharmacy Practice Managers

0

5

10

15

20

25

Retail Hospital


6/27/2017

9

What are our outcomes?

• Foundational knowledge• Learner

• Essentials for Practice and Care• Patient-centered care• Medication use systems management• Health and wellness• Population-based care

• Approach to Practice and Care• Problem solving• Educator• Patient advocacy• Interprofessional collaboration• Cultural sensitivity• Communication

• Personal and Professional Development

• Self-awareness• Leadership• Innovation and Entrepreneurship• Professionalism

Assessment Road Map

Learning Experiences

CAPE Outcomes

Practice & Team Ready

Why Do We Assess Student Performance?

6/27/2017

10

Assessment of Learning

• Summative assessment• What does the student know?• What can the student do?• What gaps exist?

• Toll gate assessment• Determines progression to the

next phase• High stakes exams• OSCEs

Preceptor Evaluation of the Student

Rating 5(Excellent)

4.5(Very Good)

4 (Good)

3.5 (Minimal

Competency)

3 (Needs

Improvement)

2 (Significant

Deficits Exist)

0 (Unacceptable)

No Opportunity/Not Observed

Student Performance

Student has exceeded expectations and can function independently at all times.

Student has met expectations and requires minimal to no guidenace from preceptor (can perform independently 90% of the time).

Student has met expectations and can complete task in a supervised situation with limited guidance from preceptor (can perform independently ≥ 80% of the time).

Student has met expectations but requires occasional guidance from preceptor (can perform independently ≥ 70% of the time).

Student requires significant guidance from preceptor (can perform independently < 70% of the time).

Student requires significant guidance from preceptor, and preceptor must often complete activity for student (can perform independently < 50% of the time).

Student does not function independently and requires direct supervision by preceptor at all times.

No opportunities exist on this rotation to allow student to demonstrate skills. Demonstration of skills was not observed.


Rating 5(Excellent)

4.5(Very Good)

4 (Good)

3.5 (Minimal

Competency)

3 (Needs

Improvement)

2 (Significant

Deficits Exist)

0 (Unacceptable)


Student Performance









Rotation 1, 2

6/27/2017

11


Rating 5(Excellent)

4.5(Very Good)

4 (Good)

3.5 (Minimal

Competency)

3 (Needs

Improvement)

2 (Significant

Deficits Exist)

0 (Unacceptable)


Student Performance









Rotation 3, 4, 5


Rating 5(Excellent)

4.5(Very Good)

4 (Good)

3.5 (Minimal

Competency)

3 (Needs

Improvement)

2 (Significant

Deficits Exist)

0 (Unacceptable)


Student Performance









Rotation 6, 7, 8

Predicted Progressive Student Performance

3.5 3.53.9

4.1 4.24.5

4.75

ROTATION 1 ROTATION 2 ROTATION 3 ROTATION 4 ROTATION 5 ROTATION 6 ROTATION 7 ROTATION 8

Preceptor Rating Prediction Assuming Progressive Development of APPE Students

Prediction

6/27/2017

12

Prediction vs. Reality of Student Performance

3.5 3.53.9

4.1 4.24.5

4.75

4.7 4.8 4.8 4.74.9

4.7 4.6 4.5

ROTATION 1 ROTATION 2 ROTATION 3 ROTATION 4 ROTATION 5 ROTATION 6 ROTATION 7 ROTATION 8

Rating Prediction Assuming Progressive Development vs. Actual Preceptor Ratings of APPE Students

Prediction Reality


Rating 5(Excellent)

4.5(Very Good)

4 (Good)

3.5 (Minimal

Competency)

3 (Needs

Improvement)

2 (Significant

Deficits Exist)

0 (Unacceptable)


Student Performance










6/27/2017

13

Table Discussion

• “Student has exceeded expectations and can function independentlyat all times.”

• What does independently mean to you?a) The student can do your job better than you canb) The student can do your job as good as you canc) The student can cover you if you are sickd) The student can cover your lunch

• Where should IPPE students perform on the TCEP rubric?

Assessment for LearningFormative Assessment

Assessment for Learning• Formative assessment• Used for improvement rather than

accountability or final decisions• Done throughout the teaching-learning

process• Provide feedback and allow for correction• Can occur at any stage of the learning process• Examples

• Homework• Quizzes• Pre-tests• Self-assessment• Reflections• Feedback Fridays

DiVall et al. Am J Pharm Educ. 2014; 78(9):160.

6/27/2017

14

Key Components of Formative Assessments

• It is a process, not a single activity• It is about learning

• Reduce extrinsic motivators• Its not about the grade!

• Provide feedback to improve student learning• Focus on progress!• Provide faculty/preceptors with information to modify instruction,

activities, and experiences• Allow student self-assessment to improve self-awareness of their own

learning


Elements of Effective Feedback

• Adapted S.M.A.R.T. model• Specific• Measurable & Meaningful• Accurate & Actionable• Respectful• Timely


Feedback should be SPECIFIC

• “You have a lot of room for improvement in patient care activities.”

• Tips• Provide details of your observations concerning performance• Give multiple examples when possible• Separate what went well, and what didn’t go well


6/27/2017

15

Feedback should be MEASURABLE & MEANINGFUL• “You need to be more professional.”

• Tips• Focus attention to actions and examples of behavior• As subjectivity increases, specificity of feedback needs to increase


Feedback should be ACCURATE & ACTIONABLE• “You should be more proactive.”

• Tips• Be open to new information from the student• Be aware of the time frame remaining for improvement• Help prioritize improvement needs for the student (rule of 3)• Provide examples of corrective action


Feedback should be RESPECTFUL

• “I know you already signed on with a retail store, but you need to try harder on this rotation.”

• Tips• Focus feedback on PERFORMANCE, not the PERFORMER• Don’t assign motivations to students’ actions• Balance negative feedback with positive feedback when appropriate


6/27/2017

16

Feedback should be TIMELY

• “I was busy at the beginning of your rotation, so I just graded your week 1 SOAP note and this week’s SOAP note together. Both had huge gaps in monitoring.”

• Tips:• Provide feedback as close to the observation as possible• Allow adequate time for improvement



Instructions

• Assume this is a non-graded activity. Evaluate the following Drug Information Responses. Utilizing the TCEP rubric as a formative assessment, what key feedback would you provide to the student?

6/27/2017

17


Thank you for your question regarding the selection of the appropriate anticoagulation therapy for VTE prophylaxis in patients with severe renal impairment. Patients who are hospitalized and/or critically ill are at significant risk for thromboembolic events due to the presence of one or multiple contributing factors of Virchow’s triad: hypercoagulability, hemodynamic changes, and endothelial injury or dysfunction. Since multiple antithrombotic drugs undergo renal clearance, this can complicate drug selection for patients with renal dysfunction and necessitate greater emphasis on proper drug and dose selection for maintaining the balance between prevention of thrombosis and risk of bleed.Current CHEST Guidelines do not address specific considerations for this subpopulation beyond the recommendation that, in the setting of severe renal impairment (CrCl < 30 mL/min), doses of low-molecular-weight heparins should be reduced (IIC).1 A review of the literature suggests that data is lacking as many studies do not include patients with renal dysfunction, but the most well-established options at this time are unfractionated heparin, which does not require renal adjustment, and enoxaparin, which is the most studied LMWH for this population. Argatroban is an anticoagulant that has been tested specifically with renal impairment and does not require dose adjustment as it undergoes hepatic metabolism, however pharmacokinetic data is lacking.2 Due to a lack of evidence-based guidelines, clinical judgment is key in managing therapeutic efficacy without compromising patient safety. LMWH are widely used and still may be considered in patients with renal impairment, but it should be noted that a meta-analysis examining their use in these patients showed a 2-to-3 fold increase risk in major bleeding events when compared to patients with normal renal function.3 There is insufficient data to suggest that LMWH should be avoided entirely in this population, however if they should be used, dose adjustment is recommended.In summary, although there are a wide array of anticoagulant options for VTE prophylaxis in patients with renal impairment, it is important to assess the pharmacokinetic properties of each drug and the possible consequence of their use under these circumstances. The most studied options at this time are UFH and enoxaparin, but regardless of the specific drug selected, appropriate dose adjustment and close monitoring of relevant labs should be carried out.

References:1. Guyatt, G., Akl, E., Crowther, M., Gutterman, D., Schuunemann, H. (2012). Executive

Summary: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. CHEST, 141(2), 7S – 47S. doi:10.1378/chest.1412S3

2. Grand’Maison, A., Charest, A., Geerts, William. (2005). Anticoagulant use in patients with chronic renal impairment. American Journal of Cardiovascular Drugs, 5.5(Sept-Oct 2005), 291. Retrieved from http://go.galegroup.com.proxy.hsc.unt.edu/ps/i.do?id=GALE%7CA199464065&v=2.1&u=txshracd2595&it=r&p=HRCA&sw=w&asid=3d45643d116fabe8af870e0be57fb70f

3. Lim, W., Dentali, F., Eikelboom, J., Crowther, M. (2006). Meta-Analysis: Low-Molecular-Weight Heparin and Bleeding in Patients with Severe Renal Insufficiency. Ann Intern Med, 144(9), 673 – 684. doi:10.7326/0003-4819-144-9-200605020-00011

6/27/2017

18


Re: Avycaz treatment and renal impairment dosingTo begin with, thanks for the question regarding Avycaz (Ceftazidime/Avibactam), formerly known as NXL104, which is a new cephalosporin beta lactamase inhibitor combination approved in February 2015. It has activity against Gram negatives including some Klebsiella pneumoniae producing carbapenemases, class A enzymes and extended spectrum cephalosporinases that contain Amp C gene (Class C enzymes). It is administered intravenously 2.5mg every 8 hours IV Infusion over 2 hours for 5 to 14 days1.In a Phase III cIAI trial, it was observed that Avycaz was not metabolized by the kidney or liver, but was renally excreted in its unchanged form. Also, patients with baseline CrCl of 30 ml/min to 50 ml/min demonstrated lower clinical response2. In view, some recommendations of renal monitoring and dose adjustment were made.A Phase I, multicenter, open-label Pharmacokinetic Cohort study conducted in hospitalized pediatrics with infection and receiving systemic antibiotics concluded that a single dose of Avycaz was comparable between each of the four (4) age cohorts [Cohort 1: >12yrs to <18yrs, Cohort 2: >6yrs to <12 yrs; Cohort 3: >2 yrs to <6 yrs and Cohort 4: >3months to <2yrs] and also, no new safety concerns were identified3.In summary, patients with impaired kidney function currently taking or about to take Avycaz need their CrCl monitored as well as an adjustment in dosage [CrCl: 30 – 50 ml/min 1.25g Q8h and CrCl < 30ml/min 0.94g Q12h to 24h]4, 5. Besides, the question of usage by patients on dialysis has not been fully explored, as there are currently no research data, clinical trials or studies to further provide the much-anticipated answer. As such, patient on dialysis are not to be given this new medication.

References:1. Chahine, Elias B., Mariette Sourial, and Raquel Ortiz. "Ceftazidime/Avibactam: a new antibiotic

for Gram-Negative infections." The Consultant Pharmacist® 30.12 (2015): 695-705. Doi: 10.4140/TCP.n.2015.695. Review. PubMed PMID: 26671269. Web 24 Feb 2017.

2. Hidalgo, Jose A., Celeste M. Vinluan, and Nishaal Antony. "Ceftazidime/avibactam: a novel cephalosporin/nonbeta-lactam beta-lactamase inhibitor for the treatment of complicated urinary tract infections and complicated intra-abdominal infections." Drug Design, Development and Therapy 10 (2016): 2379. PubMed PMID PMC 4970634. Doi: 10.2147/DDDT.S110946. PMC. Web. 24 Feb 2017.

3. Bradley, John S., et al. "Phase I Study Assessing the Pharmacokinetic Profile, Safety, and Tolerability of a Single Dose of Ceftazidime-Avibactam in Hospitalized Pediatric Patients." Antimicrobial Agents and Chemotherapy 60.10 (2016): 6252-6259. PubMed PMID: PMC5038276. Doi: 10.1128/AAC.00862-16. PMC. Web. 24 Feb 2017.

4. Mosley, Juan F., et al. "Ceftazidime-Avibactam (Avycaz): For the Treatment of Complicated Intra-Abdominal and Urinary Tract Infections." Pharmacy and Therapeutics 41.8 (2016): 479. PubMed PMID: 27504064. PMC4959616. Web 24 Feb 2017.

5. Avycaz (Ceftazidime/Avibactam) Prescribing Information. Cincinnati, Ohio: Forest Pharmaceuticals Inc.; 2016. Available at www.allergan.com/assets/pdf/avycaz_pi. Accessed February 24, 2017

6/27/2017

19

Self-AssessmentDo you ever think about what you think about?

Student #2 Self-AssessmentCRITERIA SCOREDrug Information QuestionDrug information question clearly defined; Summarized the question or therapeutic dilemma; Identified and retrieved pertinent data and information

5

Literature SourcesAbility to find and utilize appropriate literature sources; Appropriate use of primary, secondary, tertiary literature sources

5

Literature Review and EvaluationPerform critical analysis of primary literature; Summarized available data

5Conclusion/Answer/Response Reached evidenced-based conclusion; Applies information from critical analysis of literatureApplies therapeutic and pharmacologic knowledge

5

Format of Presented MaterialWritten - Organized, concise response with logic, grammatically correct, no typographical errorsProper citation of references

5

ProfessionalismSelf-directed and displays independence; Conscientious and follows through; Attentive to details

5Timeliness in responseDetermines appropriate response time to drug information inquiry based on acuity of situation

4.5

Active Learning

• Think about how you think…• Case: Your pharmacy and clinical leadership have decided that

pharmacists will now select perioperative antibiotics and doses. You have been provided with a protocol and algorithm which was developed by your infectious disease pharmacist.

• How will you learn the new program?

• What steps will you take to be ready when the program rolls out next month?

6/27/2017

20

Metacognition

• Awareness and understanding of your own thought processes

• Self-awareness enhances performance

• Inexperienced and incompetent individuals LACK metacognition

• Poor performers significantly overestimate their performance

• Gender differences in metacognition

• Metacognitive learners are strategic in learning process

Identify Strategies

Know How to Use Strategies

Know When & Why to Use Strategies

COGNITION

Hill et al. Am J Pharm Educ. 2005; 69(4):63.

Monitoring Metacognition

• Written reflection• Changes in perspectives, perceptions• Identification of new strengths• Identification of new gaps• Next steps

• Debriefing

• “Walk me through your process.”

Self-Authorship

Following Formulas

• Influenced by Authority

• Follows rules, algorithms, guidelines

Crossroads

• Influenced by Authority

• Engages in self-direction

• Looks to authority for confirmation

Becoming Self-Authored

• Influenced by Self

• Students direct their own lives

• Use authority for confidence building and as a positive role model

Internal Foundations

• Influenced by Self

• Students have a fully self-directed foundation of beliefs and values

JL Johnson, et al. Am J Pharm Ed. 2016;80(10):172.

6/27/2017

21

Reflection and Learning Partnership Model

• Utilize both interactive and written reflection

• Balance challenge and support• Model

• Read and review 3 articles by the “authority”

• Participate in a preceptor-facilitated discussion

• Defend and Devil’s Advocate• Hold self accountable to those that

they agreed with• Written reflection on how the articles

influenced their development and identify opportunities for improvement

Validate the learner as a knower. Mute the voice of authority.

Reinforce “self” as central to growth. Students should bring their perspective into play.

Promote mutual learning and sharing of expertise, knowledge, and authority.

JL Johnson, et al. Am J Pharm Ed. 2016;80(10):172.

Assessment for Program Development

Experiential Program Assessment

• Educational outcomes met• High quality preceptors• Sites are conducive to learning• Adequate diversity of experience

6/27/2017

22

Educational Outcomes

• Center for Advancement of Pharmacy Education (CAPE) 2013 Outcomes

• Accreditation Council for Pharmacy Education (ACPE) Standards• Interprofessional Educational Collaborative (IPEC) Competency

Standards• Pharmacist’s Patient Care Process• Entrustable Professional Activities (EPA)

Entrustable Professional ActivitiesAACP’s answer to Practice & Team Ready

What are EPAs?

• Specific tasks or responsibilities, which they are entrusted to perform without direct supervision

• Previously developed for medical programs to bridge a performance gap between completion of medical school and residency

• Pharmacy EPAs are categorized approximately parallel to CAPE• Both competence in the activity AND level of entrustability are

measured

Haynes et al. Am J Pharm Educ. 2017; 81(1):S2.

6/27/2017

23

Levels of Entrustability

Level Description

I. Observe Only Learner is permitted to observe only. Even with direct supervision, learner is not entrusted to perform the activity or task.

II. Direct Supervision Learner is entrusted to perform the activity or task with direct and proactive supervision. Learner must be observed performing task in order to provide immediate feedback.

III. Reactive Supervision Learner is entrusted to perform the activity or task with indirect and reactive supervision. Learner can perform task without direct supervision but may request assistance. Supervising pharmacist is quickly available on site. Feedback is provided immediately after completion of activity or task.

IV. Intermittent Supervision Learner is entrusted to perform the activity or task with supervision at a distance. Learner can independently perform task. Learner meets with supervising pharmacist at periodic intervals. Feedback is provided regarding overall performance based on sample of work.

V. General Direction Learner is entrusted to independently decide what activities and tasks need to be performed. Learner entrusted to direct and supervise the activities of others. Learner meets with supervising pharmacist at periodic intervals. Feedback is provided regarding overall performance based on broad professional expectations and organizational goals


AACP Core Entrustable Professional Activities Domains

ProfessionalismSelf-awarenessCommunication

Patient Care Provider

Interprofessional Team Member

Population Health

PromoterInformation

MasterPractice Manager Self-Developer


Closing the Loop

6/27/2017

24

Utilizing Assessments to Improve Instruction


Table Activity – Summary Points

6/27/2017

25

On Your Own – Review Your Self-Assessment

• Reflect on your own process for assessment.

• How do you assess student performance as a preceptor?

• Does your process meet all of the good practices of assessment (right)?

Reflects educational valuesMultidimensional and integratedDetects progressive growthClear purposesIncludes outcomes and experiences Ongoing, not episodicInvolves across the educational

communityAnswers questions we care aboutPromotes continuous quality

improvement

Share at Your Table

• What is one thing you already knew that was reinforced today?

• What is one thing you learned today that you did not already know?

• What is one thing you will change about your preceptorship based on what you’ve learned?

Share with the Room

• What is one thing you already knew that was reinforced today?

• What is one thing you learned today that you did not already know?

• What is one thing you will change about your preceptorship based on what you’ve learned?

6/27/2017

26

Questions & Answers

effective evaluations: a guide for preceptors · astinet al. principles of good practice for...

Documents