queen’s university cbme workshop #1 family medicine’s journey 2009-present dr. karen schultz...
TRANSCRIPT
Queen’s UniversityCBME Workshop #1
Family Medicine’s Journey 2009-present
Dr. Karen SchultzProgram Director
February 2015
Presenter Disclosure
Faculty: Karen Schultz
Relationships with commercial interests: No relationshipsPotential for conflict(s) of interest: No conflicts
What is CBME?
“Competency-Based Education is an approach to preparing physicians for practice that is fundamentally oriented to graduate outcome abilities and organized around competencies derived from an analysis of societal and patient needs. It de-emphasizes time-based training and promises greater accountability, flexibility, and learner-centeredness.”
Frank et al, Med Teacher 2010; 32: p.636
What is competence*?
Repeatedly doing:• the right thing• in the right way• at the right time• to the right person• in the right place
*with thanks to the Scottish Doctor work
Our Realities 2009
BQ 12
PK 12
KTI 100
QBOL 16 FTEs: KTI:20; Sites: 0.5/site
Hundreds of preceptors academic and community
4 sites
The expansion sites (BQ, PK, QBOL)
Our original site (KTI)
Time, Effort, Money
Successes to date
• Program– Increased direct observation (22,000 FNs for 140
residents over 2.5 years)– Earlier ID of outlying residentsprogram
modification– Program decisions upheld
1. CB thematic objectives
2. Critical competencies
4. Planned Experiences
5. Assessment System
6. Declarations 8. Distribution
Societal need (current, anticipated)
Accreditation standards
Program specific
Societal need Accreditation
standards
Authentic/WBA Reliable:
multiple expert assessors, multiple data points over time/contexts
Practical For learning
System to: Collect data Collate data Interpret
(standards, patterns of performance, trajectory)
Accrediting bodies
University Regulatory
authorities
9. Ongoing Processes: Change management; Faculty Development; Research; Quality Assurance (evaluation and adaptation)
MAP
Support objectives and competency development
Education theories (Adult learning theory, SDL, Novice to Expert Theory)
Local resources
3. MAP
7. Adjust LP
C B M E Im p lem en tati o n
1. CB thematic objectives
2. Critical competencies
4. Planned Experiences
5. Assessment System
6. Declarations 8. Distribution
Societal need (current, anticipated)
Accreditation standards
Program specific
Societal need Accreditation
standards
Authentic/WBA Reliable:
multiple expert assessors, multiple data points over time/contexts
Practical For learning
System to: Collect data Collate data Interpret
(standards, patterns of performance, trajectory)
Accrediting bodies
University Regulatory
authorities
9. Ongoing Processes: Change management; Faculty Development; Research; Quality Assurance (evaluation and adaptation)
MAP
Support objectives and competency development
Education theories (Adult learning theory, SDL, Novice to Expert Theory)
Local resources
3. MAP
7. Adjust LP
C B M E Im p lem en tati o n
Domains of Clinical Care
• Maternity and Newborn Care• Care of Children and Adolescents• Care of Adults• Care of the Elderly Patient• End of Life Care• Behavioural Medicine• Global Health/ Care of the Vulnerable and Underserved• Surgical and Procedural Skills• Physicianship
1. CB thematic objectives
2. Critical competencies
4. Planned Experiences
5. Assessment System
6. Declarations 8. Distribution
Societal need (current, anticipated)
Accreditation standards
Program specific
Societal need Accreditation
standards
Authentic/WBA Reliable:
multiple expert assessors, multiple data points over time/contexts
Practical For learning
System to: Collect data Collate data Interpret
(standards, patterns of performance, trajectory)
Accrediting bodies
University Regulatory
authorities
9. Ongoing Processes: Change management; Faculty Development; Research; Quality Assurance (evaluation and adaptation)
MAP
Support objectives and competency development
Education theories (Adult learning theory, SDL, Novice to Expert Theory)
Local resources
3. MAP
7. Adjust LP
C B M E Im p lem en tati o n
1. CB thematic objectives
2. Critical competencies
4. Planned Experiences
5. Assessment System
6. Declarations 8. Distribution
Societal need (current, anticipated)
Accreditation standards
Program specific
Societal need Accreditation
standards
Authentic/WBA Reliable:
multiple expert assessors, multiple data points over time/contexts
Practical For learning
System to: Collect data Collate data Interpret
(standards, patterns of performance, trajectory)
Accrediting bodies
University Regulatory
authorities
9. Ongoing Processes: Change management; Faculty Development; Research; Quality Assurance (evaluation and adaptation)
MAP
Support objectives and competency development
Education theories (Adult learning theory, SDL, Novice to Expert Theory)
Local resources
3. MAP
7. Adjust LP
C B M E Im p lem en tati o n
1. CB thematic objectives
2. Critical competencies
4. Planned Experiences
5. Assessment System
6. Declarations 8. Distribution
Societal need (current, anticipated)
Accreditation standards
Program specific
Societal need Accreditation
standards
Authentic/WBA Reliable:
multiple expert assessors, multiple data points over time/contexts
Practical For learning
System to: Collect data Collate data Interpret
(standards, patterns of performance, trajectory)
Accrediting bodies
University Regulatory
authorities
9. Ongoing Processes: Change management; Faculty Development; Research; Quality Assurance (evaluation and adaptation)
MAP
Support objectives and competency development
Education theories (Adult learning theory, SDL, Novice to Expert Theory)
Local resources
3. MAP
7. Adjust LP
C B M E Im p lem en tati o n
1. CB thematic objectives
2. Critical competencies
4. Planned Experiences
5. Assessment System
6. Declarations 8. Distribution
Societal need (current, anticipated)
Accreditation standards
Program specific
Societal need Accreditation
standards
Authentic/WBA Reliable:
multiple expert assessors, multiple data points over time/contexts
Practical For learning
System to: Collect data Collate data Interpret
(standards, patterns of performance, trajectory)
Accrediting bodies
University Regulatory
authorities
9. Ongoing Processes: Change management; Faculty Development; Research; Quality Assurance (evaluation and adaptation)
MAP
Support objectives and competency development
Education theories (Adult learning theory, SDL, Novice to Expert Theory)
Local resources
3. MAP
7. Adjust LP
C B M E Im p lem en tati o n
1. CB thematic objectives
2. Critical competencies
4. Planned Experiences
5. Assessment System
6. Declarations 8. Distribution
Societal need (current, anticipated)
Accreditation standards
Program specific
Societal need Accreditation
standards
Authentic/WBA Reliable:
multiple expert assessors, multiple data points over time/contexts
Practical For learning
System to: Collect data Collate data Interpret
(standards, patterns of performance, trajectory)
Accrediting bodies
University Regulatory
authorities
9. Ongoing Processes: Change management; Faculty Development; Research; Quality Assurance (evaluation and adaptation)
MAP
Support objectives and competency development
Education theories (Adult learning theory, SDL, Novice to Expert Theory)
Local resources
3. MAP
7. Adjust LP
C B M E Im p lem en tati o n
Does
Shows how
Knows how
Knows
Prof
essi
onal
Aut
henti
city
e.g. MCQs
e.g. OSCEs, SPs
e.g. SAMPS
Miller’s Pyramid Acad Med 1990: 65; S63-70
e.g. WBA
Principles of WBA
• Preceptors=your instrumentIf preceptors=instrument then you need:• multiple informed dedicated assessors• directly observing• critical competencies• repeatedly over time and in different settings
Enhanced/Facilitated by
• Faculty Development• Time – direct observation, feedback, documentation
• Practical tools
1. CB thematic objectives
2. Critical competencies
4. Planned Experiences
5. Assessment System
6. Declarations 8. Distribution
Societal need (current, anticipated)
Accreditation standards
Program specific
Societal need Accreditation
standards
Authentic/WBA Reliable:
multiple expert assessors, multiple data points over time/contexts
Practical For learning
System to: Collect data Collate data Interpret
(standards, patterns of performance, trajectory)
Accrediting bodies
University Regulatory
authorities
9. Ongoing Processes: Change management; Faculty Development; Research; Quality Assurance (evaluation and adaptation)
MAP
Support objectives and competency development
Education theories (Adult learning theory, SDL, Novice to Expert Theory)
Local resources
3. MAP
7. Adjust LP
C B M E Im p lem en tati o n
Entrustable Professional Activities
Van der Vleuten’s Utility Index Valid Reliable Educational impact
(assessment FOR learning) Acceptable Cost effective
Domains of Clinical Care
• Maternity and Newborn Care• Care of Children and Adolescents• Care of Adults• Care of the Elderly Patient• End of Life Care• Behavioural Medicine• Global Health/ Care of the Vulnerable and
Underserved• Surgical and Procedural Skills• Physicianship
36EPAs
Our Final Generic EPA Template
36
Descriptors
Different DOCC, EPA ordifferent phasedifferent unique descriptor
Queen’s FM Assessment Toolkit:
-OSCEs-MSF-SOOs-ITERs-Resident Project-Physical Exam sign off-SIM courses
ElectronicEPA FNs
1. CB thematic objectives
2. Critical competencies
4. Planned Experiences
5. Assessment System
6. Declarations 8. Distribution
Societal need (current, anticipated)
Accreditation standards
Program specific
Societal need Accreditation
standards
Authentic/WBA Reliable:
multiple expert assessors, multiple data points over time/contexts
Practical For learning
System to: Collect data Collate data Interpret
(standards, patterns of performance, trajectory)
Accrediting bodies
University Regulatory
authorities
9. Ongoing Processes: Change management; Faculty Development; Research; Quality Assurance (evaluation and adaptation)
MAP
Support objectives and competency development
Education theories (Adult learning theory, SDL, Novice to Expert Theory)
Local resources
3. MAP
7. Adjust LP
C B M E Im p lem en tati o n
QFM Portfolio Assessment Support System (PASS)
Electronic Portfolio
Assessments Reflections Modules Documents
Academic Advisor
Competency Decisions
Guidance/Learning Plans
Requirement completion Role modelling
Principles of CBACompetencies
• Relevant• Articulated
standards
Assessment
• Authentic• Multiple• Compared to
standards• Over time,
settings, different expert assessors
System for interpretation
• Data gathering, compilation and interpretation
• Patterns of performance and trajectory
• Competency development declarations
• Program adjustment
• Distribution to stakeholders
1. CB thematic objectives
2. Critical competencies
4. Planned Experiences
5. Assessment System
6. Declarations 8. Distribution
Societal need (current, anticipated)
Accreditation standards
Program specific
Societal need Accreditation
standards
Authentic/WBA Reliable:
multiple expert assessors, multiple data points over time/contexts
Practical For learning
System to: Collect data Collate data Interpret
(standards, patterns of performance, trajectory)
Accrediting bodies
University Regulatory
authorities
9. Ongoing Processes: Change management; Faculty Development; Research; Quality Assurance (evaluation and adaptation)
MAP
Support objectives and competency development
Education theories (Adult learning theory, SDL, Novice to Expert Theory)
Local resources
3. MAP
7. Adjust LP
C B M E Im p lem en tati o n
1. CB thematic objectives
2. Critical competencies
4. Planned Experiences
5. Assessment System
6. Declarations 8. Distribution
Societal need (current, anticipated)
Accreditation standards
Program specific
Societal need Accreditation
standards
Authentic/WBA Reliable:
multiple expert assessors, multiple data points over time/contexts
Practical For learning
System to: Collect data Collate data Interpret
(standards, patterns of performance, trajectory)
Accrediting bodies
University Regulatory
authorities
9. Ongoing Processes: Change management; Faculty Development; Research; Quality Assurance (evaluation and adaptation)
MAP
Support objectives and competency development
Education theories (Adult learning theory, SDL, Novice to Expert Theory)
Local resources
3. MAP
7. Adjust LP
C B M E Im p lem en tati o n
1. CB thematic objectives
2. Critical competencies
4. Planned Experiences
5. Assessment System
6. Declarations 8. Distribution
Societal need (current, anticipated)
Accreditation standards
Program specific
Societal need Accreditation
standards
Authentic/WBA Reliable:
multiple expert assessors, multiple data points over time/contexts
Practical For learning
System to: Collect data Collate data Interpret
(standards, patterns of performance, trajectory)
Accrediting bodies
University Regulatory
authorities
9. Ongoing Processes: Change management; Faculty Development; Research; Quality Assurance (evaluation and adaptation)
MAP
Support objectives and competency development
Education theories (Adult learning theory, SDL, Novice to Expert Theory)
Local resources
3. MAP
7. Adjust LP
C B M E Im p lem en tati o n
Change Management
Change Management:Factors in moving QFM from Traditional ME to CBME
Create
Urgency
• Accrediting bodies (the stick), our expansion—fell swoop--no pilots• Information--why CBME makes sense (the carrot)• Institutional deliverables
Form a
Powerful
Coalition
• Identify champions and build their excitement, expertise and connections—support their development, support communities/networking
• Support with adequate protected time
Create a
Vision for Chan
ge
• Have a clear strategy/planned path, anticipate trouble spots, identify existing facilitators (e.g. previous generic field notes, DO, faculty advisors)
• Build an organizational structure that supports CBME (formative, summative, competency declarations)
Communic
ate the
Vision
• Multi-faceted Faculty Development early and often for input and buy-in (seek ideas, address concerns, 1::1 with influential people)--Input from the assessors
• Input from the assessed
Kotter’s 8 Steps of Change
Change Management:Factors in moving QFM from Traditional ME to CBME
Remove Obstacles
•Faculty development: build skills and confidence as expert assessors. Program of faculty development. • Resident sessions: optimize self-regulated learning including seeking feedback•Minimize time and effort involved (e.g. structure and intuitiveness of tools, electronic platform) but do provide time/remuneration
Create
short term wins
•Early feedback about positive outcomes. Positive resident feedback is gold!
Build on the
change
•Continuous assessment—for improvement (reflection, identification, modification)(processes and people) and for exploration (research-expert, time, money)
•Continuous communication/multi-faceted faculty development (2 steps forward, 1 step back). This is never done.
Anchor
the change in cultu
re
•Inform colleagues and leaders of progress and successes—collaborate and broaden awareness•Succession planning
Kotter’s 8 Steps of Change
Way Forward
• Ongoing refinement• Continued collaboration/dissemination of
ideas at Queen’s, provincially and nationally with CFPC and RC programs and internationally
• Research
Questions/Comments