welcome!!! disclosures: suzanne cashman – no financial disclosures barbara gottlieb – no...
Post on 25-Dec-2015
225 Views
Preview:
TRANSCRIPT
WELCOME!!!WELCOME!!!
Disclosures:Disclosures:
Suzanne Cashman – no financial disclosuresSuzanne Cashman – no financial disclosures
Barbara Gottlieb – no financial disclosuresBarbara Gottlieb – no financial disclosures
1
Service-LearningService-LearningCCPH Annual CCPH Annual ConferenceConference
Houston, TexasHouston, TexasApril 18, 2012April 18, 2012
Suzanne CashmanSuzanne Cashman
University of Massachusetts Medical SchoolUniversity of Massachusetts Medical School
Barbara GottliebBarbara Gottlieb
Brookside Community Health CenterBrookside Community Health Center
Harvard Medical School and School of Public HealthHarvard Medical School and School of Public Health2
Today’s AgendaToday’s Agenda1:00-1:301:00-1:30 Introductions and plan for the dayIntroductions and plan for the day
Definitions – Service, Service-Definitions – Service, Service-LearningLearning
1:30-2:101:30-2:10 PartnershipsPartnerships
2:10-2:502:10-2:50 ReflectionReflection
BREAKBREAK 5 minutes5 minutes
2:55-3:352:55-3:35 EvaluationEvaluation
3:35-4:003:35-4:00 Summary, Service-learning in Summary, Service-learning in perspective, wrap-up, feedback perspective, wrap-up, feedback 3
Learning objectivesLearning objectives
►Describe the theoretical basis and key Describe the theoretical basis and key components of service-learningcomponents of service-learning
►Articulate the varied ways in which Articulate the varied ways in which service can be viewedservice can be viewed
►Apply the principles of partnership to Apply the principles of partnership to service-learningservice-learning
►Explain the key role of reflection in Explain the key role of reflection in service-learningservice-learning
4
Learning objectivesLearning objectives
►Demonstrate a variety of reflection Demonstrate a variety of reflection modalitiesmodalities
►Understand how to evaluate service-Understand how to evaluate service-learning activities and programslearning activities and programs
►Articulate the place and role of Articulate the place and role of service-learning in the context of service-learning in the context of community-engaged scholarly community-engaged scholarly activitiesactivities
5
I slept and dreamt that life was I slept and dreamt that life was joy. I awoke and saw that life was joy. I awoke and saw that life was
serviceservice. I acted and behold, . I acted and behold, serviceservice was joy. Rabindranath was joy. Rabindranath
TagoreTagore
7
What is service?What is service?
What are some examples of What are some examples of service?service?
8
A type of A type of economic activityeconomic activity that is that is intangible, is not stored and does not result intangible, is not stored and does not result in ownership. A service is consumed at the in ownership. A service is consumed at the point of sale. Services are one of the two point of sale. Services are one of the two key components of economics, the other key components of economics, the other
being goods. Examples of services include being goods. Examples of services include the transfer of goods, such as the postal the transfer of goods, such as the postal service delivering mail, and the use of service delivering mail, and the use of
expertise or experience, such as a person expertise or experience, such as a person visiting a doctor.visiting a doctor.
Basically, when you engage in an unpaid Basically, when you engage in an unpaid activity intended to benefit others, that’s activity intended to benefit others, that’s
serviceservice..9
What makes service What makes service service?service?
►Brainstorming exerciseBrainstorming exercise
10
What is Service-Learning? What is Service-Learning?
► Service-learning is a structured learning Service-learning is a structured learning experience that combines community experience that combines community service with preparation and reflection. service with preparation and reflection. Students engaged in service-learning Students engaged in service-learning provide community service in response to provide community service in response to community-identified concerns and learn community-identified concerns and learn about the context in which service is about the context in which service is provided, the connection between their provided, the connection between their service and their academic coursework, and service and their academic coursework, and their roles as citizens. (CCPH Website)their roles as citizens. (CCPH Website)
12
Service-Learning Service-Learning
►Strives to balance service and learning Strives to balance service and learning objectivesobjectives
►Addresses community concerns and Addresses community concerns and broad determinants of health broad determinants of health
► Integrates community partners Integrates community partners ►Emphasizes reciprocal learning - Emphasizes reciprocal learning -
traditional definitions of "faculty," traditional definitions of "faculty," "teacher" and "learner" are intentionally "teacher" and "learner" are intentionally blurredblurred
13
Service-Learning Service-Learning (cont’d)(cont’d)►Emphasizes reflective practice - Emphasizes reflective practice -
fosters critical thinking and self-fosters critical thinking and self-awarenessawareness Integrates reflection at all stages of Integrates reflection at all stages of
learninglearning
►Develops citizenship skills and Develops citizenship skills and focuses on achieving social changefocuses on achieving social change Provision of health services is not often the Provision of health services is not often the
most important factor; students place their most important factor; students place their roles as health professionals and citizens in roles as health professionals and citizens in a larger societal contexta larger societal context
14
Service-Learning Service-Learning (cont’d)(cont’d)
►Focus is on partnershipsFocus is on partnerships►Quality of service-learning Quality of service-learning
opportunities develop and evolve opportunities develop and evolve over timeover time
15
Service-Learning
Field ExperienceCommunity Service
Volunteerism Practicum, Internship, Clerkship
Furco, Service-Learning. 1996
Recipient ProviderBeneficiary
What Distinguished Service-Learning from Other Forms of Experiential
Education?
Service Focus Learner
16
Distinguishing service-learning with other types of experiential learning
(CCPH, 1999)
Community Community ServiceService
AcademicAcademic
LearningLearningPurposeful Purposeful
Civic learningCivic learning
Clinical Clinical TrainingTraining
MayMay YesYes Generally NotGenerally Not
Volunteer Volunteer WorkWork
YesYes NoNo NoNo
InternshipInternship MayMay YesYes NoNo
Service- Service- learninglearning
YesYes YesYes YesYes
17
Points of Departure: Points of Departure: S-L and Traditional EducationS-L and Traditional Education
► Balance between service and learningBalance between service and learning► Integral involvement of community Integral involvement of community
partners partners ► Emphasizes :Emphasizes :
Importance of addressing community-identified Importance of addressing community-identified concerns and broad determinants of healthconcerns and broad determinants of health
Strength/asset-based approachStrength/asset-based approach Reciprocal learningReciprocal learning Reflective practiceReflective practice Developing citizenship skills and achieving Developing citizenship skills and achieving
social changesocial change 18
Evidence: HPSISNEvidence: HPSISN ► Health Professions Schools in Service to Health Professions Schools in Service to
the Nation (HPSISN) Program, 1994-98the Nation (HPSISN) Program, 1994-98 Three-year grants to 17 health professions Three-year grants to 17 health professions
schoolsschools Program goals:Program goals:
► Strengthen partnerships between health professions Strengthen partnerships between health professions schools and communitiesschools and communities
► Address unmet community needsAddress unmet community needs► Instill ethic of community service and social Instill ethic of community service and social
responsibility in health professions schools, students responsibility in health professions schools, students and facultyand faculty
► Equip next generation of health professionals with Equip next generation of health professionals with community- oriented competencies necessary to community- oriented competencies necessary to practice in a changing health care environment. practice in a changing health care environment.
20
HPSISN Partnership FocusHPSISN Partnership Focus
► School-based health School-based health educationeducation
► Health promotion Health promotion and disease and disease preventionprevention Teenage Teenage
pregnancypregnancy Domestic violenceDomestic violence Oral healthOral health
► Worksite-based Worksite-based health educationhealth education
► CompanionshipCompanionship► Case managementCase management► Mentoring and Mentoring and
tutoringtutoring► Rural access to careRural access to care
21
Findings: StudentsFindings: Students
Transformational learning experiencesTransformational learning experiencesClarification of values, sense of selfClarification of values, sense of self
Taken more seriously when it’s Taken more seriously when it’s requiredrequired
Greater gains when non-clinical:Greater gains when non-clinical:Awareness of determinants of healthAwareness of determinants of health
Sensitivity to diversitySensitivity to diversity
Knowledge of health policy issuesKnowledge of health policy issues
Leadership developmentLeadership development22
Findings: Community Findings: Community PartnersPartners
► Service, economic and social benefits Service, economic and social benefits ► awareness of institutional awareness of institutional
assets/limitationsassets/limitations► High value placed on relationship with High value placed on relationship with
facultyfaculty► Eager to be seen as teachers and expertsEager to be seen as teachers and experts
Campus involvement limitedCampus involvement limited► Benefits of SL outweighed the burdensBenefits of SL outweighed the burdens► Concerns re: communication, logistics, Concerns re: communication, logistics,
needs-based and expert approachesneeds-based and expert approaches
23
Findings: FacultyFindings: Faculty►Stronger relationships associated with:Stronger relationships associated with:
Joint planningJoint planning Partners being offered specific and active Partners being offered specific and active
rolesroles Genuine sense of reciprocityGenuine sense of reciprocity Student preparation and orientationStudent preparation and orientation Single point of contactSingle point of contact Consistent, accessible communicationConsistent, accessible communication
24
Findings: Institutional Findings: Institutional CapacityCapacity
Link to mission and strategic goalsLink to mission and strategic goals Clear definition of service-learningClear definition of service-learning Supportive leadership at all levelsSupportive leadership at all levels Effective institutional structures and Effective institutional structures and
policy policy Investment in faculty developmentInvestment in faculty development Integration of SL into existing coursesIntegration of SL into existing courses Long-term community relationshipsLong-term community relationships Ongoing assessment and improvementOngoing assessment and improvement
25
Lessons LearnedLessons Learned► Service learning is powerful pedagogyService learning is powerful pedagogy► SL can contribute to competencies needed SL can contribute to competencies needed
for health professions practicefor health professions practice► SL can benefit students, faculty, the SL can benefit students, faculty, the
community and community-university community and community-university relationshipsrelationships
► Community can be effective educatorsCommunity can be effective educators► Community assets are often overlookedCommunity assets are often overlooked► SL requires schools to give up “control”SL requires schools to give up “control”
26
2121stst Century Challenges for Century Challenges for EducatorsEducators
► Knowledge explosionKnowledge explosion► Cost, resource constraintsCost, resource constraints► Multiple stakeholders in educationMultiple stakeholders in education► Demand for new paradigmsDemand for new paradigms► Recognition of multiple learning styles and Recognition of multiple learning styles and
“intelligences”“intelligences”► Complex problems, but historical constraints on Complex problems, but historical constraints on
innovationinnovation► Educated consumers/publicEducated consumers/public► Disparities and inequitiesDisparities and inequities► Demand for equity and social justiceDemand for equity and social justice
28
Are we meeting these challengesAre we meeting these challenges??
► Do our teaching methods …Do our teaching methods … promote and develop the thinking skills that our promote and develop the thinking skills that our
students will need to work compassionately, safely students will need to work compassionately, safely and effectively? and effectively?
speak to multiple learning styles and intelligences?speak to multiple learning styles and intelligences? provide the skills and encourage a commitment to provide the skills and encourage a commitment to
life-long learning?life-long learning? promote the exploration of values that are promote the exploration of values that are
consistent with social justice and equity?consistent with social justice and equity? promote altruism and help students learn that promote altruism and help students learn that
they can do well by doing good?they can do well by doing good? promote teamwork, collaboration, communication promote teamwork, collaboration, communication
skills?skills?29
Service-learning Service-learning vsvs traditional traditional learninglearning
TraditionalTraditionalFact-orientedFact-orientedMy-side biasMy-side biasAlgorithmicAlgorithmicGroup thinkGroup thinkStay in comfort zoneStay in comfort zoneAbility to focus may Ability to focus may
obscure the big pictureobscure the big pictureExisting knowledge may Existing knowledge may
compete with compete with acquisition of newacquisition of new
knowledgeknowledge(Ritchhart, Perkins)(Ritchhart, Perkins)
Service-learningService-learningProcess-orientedProcess-orientedDiverse perspectivesDiverse perspectivesDiverse strands of Diverse strands of
informationinformationTake risksTake risksHeightened awareness Heightened awareness
of experienceof experienceBig-picture orientationBig-picture orientationOpenness, self-Openness, self-
awareness, critical awareness, critical thinkingthinking
30
ActivityActivity
►Using the word magnets we’ve Using the word magnets we’ve distributed, compose the following:distributed, compose the following:• A sentence or phrase that describes your A sentence or phrase that describes your
experiences with community-campus experiences with community-campus partnershipspartnerships
• A sentence that describes your dream A sentence that describes your dream partnershippartnership
32
Why partnerships?Why partnerships?
Community-campus partnerships – a Community-campus partnerships – a strategy for social changestrategy for social change
Establish missing but critical connectionsEstablish missing but critical connections
Identify new/better way to solve problemsIdentify new/better way to solve problems
Link complementary skills and resources Link complementary skills and resources of diverse people and organizationsof diverse people and organizations
Plan and carry out comprehensive actions Plan and carry out comprehensive actions that coordinate reinforcing strategies and that coordinate reinforcing strategies and systemssystems 33
Partners: Faculty/CampusPartners: Faculty/Campus
Faculty & campuses involved in partnerships:Faculty & campuses involved in partnerships:
Primary motivators: personal values, belief in Primary motivators: personal values, belief in improved learning, new knowledgeimproved learning, new knowledge
Enhanced relationships - students, communityEnhanced relationships - students, community
Linkage of personal/professional livesLinkage of personal/professional lives
understanding of community issuesunderstanding of community issues
New career and scholarship directionsNew career and scholarship directions
Concerns re: time, controlConcerns re: time, control
34
Partners: CommunitiesPartners: Communities
Communities involved in partnerships:Communities involved in partnerships: Service, economic and social benefits Service, economic and social benefits
awareness of institutional assets/limitationsawareness of institutional assets/limitations
High value placed on relationship with facultyHigh value placed on relationship with faculty
Eager to be seen as teachers and expertsEager to be seen as teachers and experts
Campus involvement limitedCampus involvement limited
Benefits outweigh the burdensBenefits outweigh the burdens
Concerns re: communication, logistics, needs-based and Concerns re: communication, logistics, needs-based and expert approachesexpert approaches
35
The Power of PartnershipsThe Power of Partnerships
► Initial principles adopted 1998Initial principles adopted 1998►Revised 2005-2006 (October 2006)Revised 2005-2006 (October 2006)
Based on practice and feedbackBased on practice and feedback CCPH board discussionCCPH board discussion Community Partner Summit Community Partner Summit
(Wingspread Conference Center, (Wingspread Conference Center, Racine, Wisconsin)Racine, Wisconsin)
36
Principles of Principles of GoodGood Community- Community-Campus PartnershipsCampus Partnerships
► Partnerships form to Partnerships form to serve a specific purpose serve a specific purpose and and may take on new goals over time. may take on new goals over time.
► Partners have agreed upon Partners have agreed upon mission, values, goalsmission, values, goals, , measurable measurable outcomes outcomes and and accountabilityaccountability for the for the partnership. partnership.
► The relationship between partners is characterized by The relationship between partners is characterized by mutual mutual trust, respect, genuineness, and trust, respect, genuineness, and commitmentcommitment. .
► The partnership builds upon identified strengths and The partnership builds upon identified strengths and assets, but also works to address needs and assets, but also works to address needs and increase capacity of all partners. increase capacity of all partners.
► The partnership The partnership balances power balances power among partners among partners and enables resources among partners to be shared. and enables resources among partners to be shared.
37
Principles (cont’d)Principles (cont’d)► Partners make clear and open Partners make clear and open communicationcommunication an an
ongoing priority by striving to understand each ongoing priority by striving to understand each other's needs and self-interests, and developing a other's needs and self-interests, and developing a common language. common language.
► Principles and processes for the partnership are Principles and processes for the partnership are established with the input and agreement of all established with the input and agreement of all partners, especially for partners, especially for decision-making and decision-making and conflict resolutionconflict resolution. .
► There is There is feedback feedback among all stakeholders in the among all stakeholders in the partnership, with the goal of continuously improving partnership, with the goal of continuously improving the partnership and its outcomes. the partnership and its outcomes.
► Partners Partners share the benefits share the benefits of the partnership's of the partnership's accomplishments. accomplishments.
► Partnerships can dissolve and need to plan a process Partnerships can dissolve and need to plan a process for for closureclosure
38
Partnerships - exercisePartnerships - exercise
►Working in pairs – choose one of the Working in pairs – choose one of the principles of partnership and discussprinciples of partnership and discuss how you have seen this principle in actionhow you have seen this principle in action
►What has gone well?What has gone well?►Challenges?Challenges?
(if relevant) – what this principle might look (if relevant) – what this principle might look like early in a partnership, and how it might like early in a partnership, and how it might evolve in a more developed partnershipevolve in a more developed partnership
does this seem like a “core” principle?does this seem like a “core” principle?
39
Kolb’s learning cycleKolb’s learning cycle
► Inventory of Inventory of dominant learning dominant learning stylesstyles
► Cycle of learning Cycle of learning (for all learners)(for all learners)
► Cycle of teaching Cycle of teaching (for educators)(for educators)
41
Challenges in higher Challenges in higher educationeducation
► Do our teaching methods …Do our teaching methods … promote and develop the thinking skills that our promote and develop the thinking skills that our
students will need to work compassionately, students will need to work compassionately, safely and effectively? safely and effectively?
provide the skills and encourage a commitment to provide the skills and encourage a commitment to life-long learning?life-long learning?
promote the exploration of values that are promote the exploration of values that are consistent with social justice and equity?consistent with social justice and equity?
promote altruism and help students learn that promote altruism and help students learn that they can do well by doing good?they can do well by doing good?
promote teamwork, collaboration, communication promote teamwork, collaboration, communication skills?skills? 42
How do learners learn best?How do learners learn best?
► ExperientialExperiential► Deliberate ecology of learnersDeliberate ecology of learners
MentoringMentoring Reciprocal learningReciprocal learning
► Embedded learning Embedded learning GroundedGrounded Values-basedValues-based
► Developmentally appropriateDevelopmentally appropriate► Guided reflection supports all of these areasGuided reflection supports all of these areas
43
It is one thing to absorb a fact, to situate it alongside other facts in a It is one thing to absorb a fact, to situate it alongside other facts in a configuration, and quite another to contemplate that fact at leisure, configuration, and quite another to contemplate that fact at leisure, allowing it to declare its connectivity with other facts, its thematic allowing it to declare its connectivity with other facts, its thematic destiny, its resonance…..destiny, its resonance…..
Sven Birkerts, The Guttenberg ElegiesSven Birkerts, The Guttenberg Elegies 46
ReflectionReflection► Dewey, 1933: ‘‘active, persistent and careful Dewey, 1933: ‘‘active, persistent and careful
consideration of any belief or supposed form of consideration of any belief or supposed form of knowledge in the light of the grounds that knowledge in the light of the grounds that support it and the further conclusion to which it support it and the further conclusion to which it tends” tends”
► Bond, 1985: ‘‘a generic term for those intellectual Bond, 1985: ‘‘a generic term for those intellectual and affective activities in which individuals and affective activities in which individuals engage to explore their experiences in order to engage to explore their experiences in order to lead to a new understanding and appreciation”lead to a new understanding and appreciation”
► Moon, 1999: ‘‘a form of mental processing with a Moon, 1999: ‘‘a form of mental processing with a purpose and/or anticipated outcome that is purpose and/or anticipated outcome that is applied to relatively complex or unstructured applied to relatively complex or unstructured ideas for which there is not an obvious solutionideas for which there is not an obvious solution
47
Reflection definitions - Reflection definitions - themesthemes
► Critical analysis of experienceCritical analysis of experience► Search for deeper meaningSearch for deeper meaning► Active construction of lessons and Active construction of lessons and
meaningsmeanings► Guide for future actionGuide for future action► Affective processingAffective processing► 2 dimensions (Mann, 2 dimensions (Mann, et alet al))
Iterative – new understanding leads to new Iterative – new understanding leads to new action or changed response in future practiceaction or changed response in future practice
Vertical – deepening of understanding and Vertical – deepening of understanding and analysisanalysis
48
Evidence base for Evidence base for reflectionreflection
►Commentary/essaysCommentary/essays►TestimoniesTestimonies►Descriptive studiesDescriptive studies►Studies with measured outcomesStudies with measured outcomes►Comparisons of 2 or more approaches Comparisons of 2 or more approaches
to reflectionto reflection►Comparisons of reflection vs no Comparisons of reflection vs no
reflectionreflection49
ReflectionReflection
►How do you like to reflect?How do you like to reflect?►What modalities have you used in your What modalities have you used in your
lives?lives?►What modalities have you used in your What modalities have you used in your
teaching?teaching?►Do you prefer some over others? Do you prefer some over others?
Why?Why?
51
Mapping S-L ReflectionMapping S-L ReflectionBeforeBefore During During AfterAfter
AloneAlone
With With ClassmateClassmatess
With With CommunitCommunity Partnersy Partners
With With academic academic communitcommunityy
(From: Eyler, Creating your Reflection Map)
52
Mapping S-L ReflectionMapping S-L ReflectionBeforeBefore During During AfterAfter
AloneAlone What are my What are my concerns? What concerns? What challenges do I challenges do I anticipate? Why do I anticipate? Why do I want this want this experience?experience?
Double-entry journal Double-entry journal (personal thoughts (personal thoughts re experience/re experience/
class discussion class discussion responses)responses)
Advocacy letter, Advocacy letter, letter to editor, letter to editor, personal reflections personal reflections on how I’ve changed on how I’ve changed and grownand grown
With With ClassmateClassmatess
With With CommunitCommunity Partnersy Partners
With With academic academic communitcommunityy (From: Eyler, Creating your Reflection Map)
53
Mapping S-L ReflectionMapping S-L ReflectionBeforeBefore During During AfterAfter
AloneAlone
With With ClassmateClassmatess
With With CommunitCommunity Partnersy Partners
With With academic academic communitcommunityy
(From: Eyler, Creating your Reflection Map)
54
Reflection exerciseReflection exercise
ScenarioScenario: Your students are going to : Your students are going to participate in a semester-long project in a participate in a semester-long project in a nursing home. They will take turns doing a nursing home. They will take turns doing a variety of activities, including leading an variety of activities, including leading an exercise class, doing arts and crafts exercise class, doing arts and crafts activities, shadowing and assisting various activities, shadowing and assisting various staff members, and simply sitting and staff members, and simply sitting and visiting with the residents in an unstructured visiting with the residents in an unstructured manner. manner.
55
Reflection exercise: using the grid, design reflection Reflection exercise: using the grid, design reflection activities for each of the boxes that would enhance activities for each of the boxes that would enhance students’ learning from their experiences in the students’ learning from their experiences in the nursing home. Try in particular to think of activities nursing home. Try in particular to think of activities that might benefit the community partners, and that might benefit the community partners, and develop the students’ understanding of their roles develop the students’ understanding of their roles as citizens.as citizens.
BeforeBefore During During AfterAfter
AloneAlone
With With ClassmatesClassmates
With With Community Community PartnersPartners
With With academic academic communitycommunity
(From: Eyler, Creating your Reflection Map)
56
Mapping S-L ReflectionMapping S-L ReflectionBeforeBefore During During AfterAfter
AloneAlone Letter to selfLetter to self Structured journalsStructured journals Summary reflective Summary reflective essayessay
Advocacy letterAdvocacy letter
With With ClassmateClassmatess
Hopes and fears Hopes and fears discussiondiscussion
TheatreTheatre
Role playRole play
Mixed team Mixed team discussiondiscussion
Team presentationTeam presentation
Collage, mural, Collage, mural, video, photo essayvideo, photo essay
With With CommunitCommunity Partnersy Partners
Asset mappingAsset mapping
Planning w/ Planning w/ communitycommunity
Lessons learned, Lessons learned, debriefingdebriefing
Presentation to Presentation to community group, to community group, to school school administrators/administrators/
leadersleaders
With With academic academic communitcommunityy
Review of theory Review of theory and anticipated and anticipated practicepractice
Ways in which Ways in which actual practice actual practice does/does not does/does not reflect theoryreflect theory
Where theory might Where theory might be amendedbe amended
RecommendationsRecommendations
e
(From: Eyler, Creating your Reflection Map)
57
Reflection - How to get Reflection - How to get startedstarted
►Create a culture for reflectionCreate a culture for reflection►Establish clear expectationsEstablish clear expectations►Reflection activitiesReflection activities►System for respondingSystem for responding►System for sharing with other learners System for sharing with other learners ►System for sharing with community System for sharing with community
59
Establishing a culture for Establishing a culture for reflectionreflection
► Explain learning objectivesExplain learning objectives► InspirationInspiration““I celebrate such writing and the impulse to write, the impulse I celebrate such writing and the impulse to write, the impulse
to share some transformative incident that I am privileged to to share some transformative incident that I am privileged to have witnessed. In my own writing, I often feel that I write in have witnessed. In my own writing, I often feel that I write in order to understand what I am thinking. Mysteriously, insight order to understand what I am thinking. Mysteriously, insight comes (when it does come) in the very comes (when it does come) in the very actact of writing, as if of writing, as if only by sitting with pen and pad can we snatch it out of the only by sitting with pen and pad can we snatch it out of the ether.” ether.” Abraham Verghese, forward in Abraham Verghese, forward in Narrative MattersNarrative Matters (2006) (2006)
► ModelingModeling► Feedback/gratification/rewardFeedback/gratification/reward
60
Evaluation of reflection - Evaluation of reflection - issuesissues
►StandardizationStandardization►SafetySafety►Who will see (and respond?) to written Who will see (and respond?) to written
reflections?reflections? Instructors?Instructors? Peers?Peers? Community preceptors?Community preceptors? Others? Who?Others? Who?
61
Principles of assessmentPrinciples of assessment
►Linked to goals and objectivesLinked to goals and objectives Project/activity Project/activity Learning objectives of the courseLearning objectives of the course Individual learning objectives of the studentIndividual learning objectives of the student
►Clear and transparentClear and transparent PurposePurpose
► FeedbackFeedback► Track progress toward goalsTrack progress toward goals
►Learner-centerLearner-center Appropriately timedAppropriately timed Provides basis for change/growthProvides basis for change/growth
62
Domains of assessmentDomains of assessment
►ContentContent Breadth and depthBreadth and depth
►Diversity and creativityDiversity and creativity Depends on structure/non-structure of Depends on structure/non-structure of
reflection activitiesreflection activities►Domains of learningDomains of learning
DescriptiveDescriptive AnalyticAnalytic AffectiveAffective Self-exploration, developmentalSelf-exploration, developmental
63
Assessment planningAssessment planning
► Establish clear goalsEstablish clear goals► Establish clear learning objectives for the courseEstablish clear learning objectives for the course► Allow students to set individual learning objectivesAllow students to set individual learning objectives► Ask – how can reflection enhance the goals and Ask – how can reflection enhance the goals and
objectives above?objectives above?► Design reflection activitiesDesign reflection activities
When? (before, during, after)When? (before, during, after) What?What?
► Schedule for reflection activities and Schedule for reflection activities and feedback/responsefeedback/response
► Communicate to studentsCommunicate to students► Communicate to community partnersCommunicate to community partners 64
68
““Good evaluation” is nothing Good evaluation” is nothing more than “good thinking”more than “good thinking”
It is the systematic collection of information It is the systematic collection of information about activities, characteristics and about activities, characteristics and outcomes of programs, personnel, and outcomes of programs, personnel, and products to use to reduce uncertainties, products to use to reduce uncertainties, improve effectiveness and make decisions. improve effectiveness and make decisions.
Patton, 1997Patton, 1997
69
Why Are You Engaged in Why Are You Engaged in Evaluation?Evaluation?
►What have been some of your…What have been some of your… positive experiences?positive experiences? negative experiences?negative experiences?
70
Why EvaluateWhy Evaluate??
Reduce uncertaintiesReduce uncertainties Measure program and/or student Measure program and/or student
achievementachievement Improve effectivenessImprove effectiveness Demonstrate accountabilityDemonstrate accountability Make programmatic and curriculum Make programmatic and curriculum
decisionsdecisions Build constituencyBuild constituency
71
Why EvaluateWhy Evaluate? (cont’d)? (cont’d)
Hold people accountableHold people accountable Identify areas needing improvementIdentify areas needing improvement Use scarce resources wiselyUse scarce resources wisely Build capacityBuild capacity Influence policyInfluence policy It can be fun…and interestingIt can be fun…and interesting
72
Continuous ImprovementContinuous Improvement
►EvaluationEvaluation Provides opportunity for continuous Provides opportunity for continuous
improvementimprovement Identifies where in the course Identifies where in the course
improvemenet is neededimprovemenet is needed
73
Comparison of Academic Research and Practical Evaluation
Academic Research
Practical Evaluation
Purpose Test hypotheses Improve program/ practice
Method Controlled environment
Context sensitive
Statistics Sophisticated Simpler
74
Evaluation should:Evaluation should:
►Strengthen projectsStrengthen projects►Use multiple approachesUse multiple approaches►Address real issuesAddress real issues►Create a participatory processCreate a participatory process►Allow for flexibilityAllow for flexibility►Build capacityBuild capacity
WKKellogg Foundation, 1998WKKellogg Foundation, 1998
75
Guiding Principles
Ongoing processMeans, not an endCollaboration based on trust and respectSensitive to cultural, socioeconomic, ethnic,
lifestyle, life-span pluralisms Allow for combination of methodologiesRemain flexibleBuild capacity
76
Planning An EvaluationPlanning An Evaluation
Develop a logic modelDevelop a logic model
Enumerate stakeholdersEnumerate stakeholders
Create an evaluation planCreate an evaluation plan
Decide on methods, strategiesDecide on methods, strategies
Specify data collection approaches Specify data collection approaches and indicatorsand indicators
Develop dissemination planDevelop dissemination plan
77
Logic ModelsLogic Models
►WHAT?WHAT? Systematic and visual way to present and Systematic and visual way to present and
share your understanding of the share your understanding of the relationship among resources, activities, relationship among resources, activities, changes/resultschanges/results
►WHY?WHY? Provide stakeholders with road map Provide stakeholders with road map
describing sequence of related events describing sequence of related events connecting need with resultsconnecting need with results
Students
CommunityPartners
Faculty
ServiceRecipients
PLANNING COMPONENT• Funding
• Risk management• Service-learning objectives
•Protocols• Plans for research
and reporting•Principles of Partnership
• Reflection
TEACHING COMPONENT FOR
STUDENTS• Geographic community
•Agency mission / function• Service recipients
• Clinical and interpersonal skills• Role and responsibility
• Risk management• Reflection
ORIENTATIONCOMPONENTFOR AGENCY
• Student role & responsibilities•Agency role & responsibilities
• Student clinical skills &time commitment
• Risk management•Reflection
Inputs
Service-LearningProgram
OutcomesOutputs
Service and Learning
Service report anddissemination
Assessment and evaluationof student learning
Impact of reflectionon program revisions
Publication inprofessional journal
CQI implementationand work toward
sustainability
Evaluation -- Continuous Quality Improvement (CQI) -- Reflection
Logic model for service-learning curriculum (K.Yoder, Ind University)
Health policy activism
79
Logic Model
INPUTS/RESOURCESThe resources required for this program to operate.
ACTIVITIESStrategies you will use or services you will provide to try to achieve your goal.
OUTPUTSBasic data on program, product, or service
OUTCOMESDesired changes in the population as a result of the program activities.
IMPACTChanges that can be expected in organizations, communities, or systems
80
Logic Model
INPUTS/RESOURCESThe resources required for this program to operate.
ACTIVITIESStrategies you will use or services you will provide to try to achieve your goal.
OUTPUTSBasic data on program, product, or service
OUTCOMESDesired changes in the population as a result of the program activities.
IMPACTChanges that can be expected in organizations, communities, or systems
StaffTimeMoneyMaterialsEquipmentTechnologyPartners
Meet with youth group directorsMeet with youthObtain relevant cultural materialsDevelop and implement culturally relevant dating violence prevention program
Collect baseline dataUse data to inform culturally relevant violence prevention programEvaluate programShare results
Youth show increased recognition of dating violenceYouth increase skills to help others in abusive relationshipsYouth report attitudes unaccepting of dating violence
Decreased gender stereotypingReduced rates of dating violence
81
StakeholdersStakeholders
► People who have a “stake” in what will be People who have a “stake” in what will be learned from an evaluation and what will be learned from an evaluation and what will be done with the knowledgedone with the knowledge
► They include:They include: People who manage or work in the People who manage or work in the
program/organizationprogram/organization People who are served or affected by the program, People who are served or affected by the program,
or who work in partnership with the programor who work in partnership with the program People who are in a position to do or to decide People who are in a position to do or to decide
something about the programsomething about the program Who else…?Who else…?
82
StakeholdersStakeholders
►Stakeholders’ information needs and Stakeholders’ information needs and intended uses focus the evaluationintended uses focus the evaluation
►Variety of stakeholders may mean:Variety of stakeholders may mean: more than one focus (policy implications more than one focus (policy implications
vs documentation of local activities)vs documentation of local activities) varied levels of involvementvaried levels of involvement
83
StakeholdersStakeholders
►Who are your stakeholders?Who are your stakeholders?►What do they want and need to know?What do they want and need to know?►How will you make sure that they have How will you make sure that they have
data they want and information they data they want and information they need to know?need to know?
84
Methods, Strategies: Thinking Methods, Strategies: Thinking about dataabout data
►Match the data to the questions – what Match the data to the questions – what kinds of information would be kinds of information would be worthwhile?worthwhile?
►As much as possible, use data that are As much as possible, use data that are being created as a regular part of the being created as a regular part of the programprogram
►Collect and analyze data from multiple Collect and analyze data from multiple perspectivesperspectives
►Keep available resources in mindKeep available resources in mind
85
DataData
►Where might we find them?Where might we find them?►How might we obtain them?How might we obtain them?►What types should we consider?What types should we consider?
Quantitative and qualitativeQuantitative and qualitative►What do we do now that we have What do we do now that we have
them?them?
86
Who can help us collect and Who can help us collect and make sense of data?make sense of data?
Community partnersCommunity partners Student participantsStudent participants College administrative officesCollege administrative offices Faculty colleagues (and their students)Faculty colleagues (and their students) Students who participated in previous Students who participated in previous
programsprograms Campus service-learning centersCampus service-learning centers
87
Outcome IndicatorsOutcome Indicators
►The specific, measurable The specific, measurable characteristics or changes that characteristics or changes that represent achievement of an outcomerepresent achievement of an outcome
► Indicators are measurable and Indicators are measurable and observable and answer the question: observable and answer the question: “How will we know?”“How will we know?”
88
Program Outcome IndicatorsProgram Outcome Indicators
► Short term Short term OutcomesOutcomes Youth show increased Youth show increased
recognition of dating recognition of dating violence and healthy violence and healthy relationshipsrelationships
Youth increase their Youth increase their anger management anger management skillsskills
Youth increase their Youth increase their communication skillscommunication skills
► IndicatorsIndicators Pre/post attitude and Pre/post attitude and
knowledge surveyknowledge survey
Police and school Police and school reports reports
Teacher reportsTeacher reports
Learner Outcome IndicatorsLearner Outcome Indicators
► Short-term OutcomesShort-term Outcomes Learners report/Learners report/
demonstrate greater demonstrate greater cultural humilitycultural humility
--Learners demonstrate--Learners demonstrate
increased advocacy increased advocacy skillsskills
--Learners link theory to --Learners link theory to actionaction
► IndicatorsIndicators Pre/post survey/Pre/post survey/
community faculty community faculty observationsobservations
--Letter to the editor --Letter to the editor pre and postpre and post
--Reflective journals--Reflective journals
90
Creating an Evaluation Plan
What do we want to know?
What will we measure?
How will we measure?
Who will be responsible? When?
How will we use results?
91
Creating an Evaluation PlanWhat do we want to know?
What will we measure?
How will we measure?
Who will be responsible? When?
How will we use results?
Impact on clients/service users
Client/Service user attitudes
Pre and post survey
EvaluatorBeginning of course and at the end
Demonstrate power and influence of student involvement in reaching program goals
92
Closing the LoopClosing the Loop
Collaborate with stakeholders for Collaborate with stakeholders for meaningfulmeaningful::Communication of results (process and Communication of results (process and
outcome)outcome)Decisions based on resultsDecisions based on resultsNew assessment plans emerging from New assessment plans emerging from
resultsresultsReflection on the assessment processReflection on the assessment process
Summary and wrap upSummary and wrap up
►Where are you now? Where do you Where are you now? Where do you want to go?want to go?
►Where does service-learning fit into Where does service-learning fit into the full spectrum of community-the full spectrum of community-engaged scholarship?engaged scholarship?
►Final reflectionsFinal reflections►FeedbackFeedback
93
Categories of the Service-Categories of the Service-Learning JourneyLearning Journey
PartnershipsPartnerships► SustainedSustained► S-L objectivesS-L objectives► Broad preparationBroad preparation► Apply principlesApply principles► EvaluationEvaluationResourcesResources► FundingFunding► SpaceSpace► StaffStaffCurriculumCurriculum► Service objectivesService objectives► Learning objectivesLearning objectives► S-L objectivesS-L objectives► Guided reflectionGuided reflection► EvaluationEvaluation
95
Categories of the Service-Categories of the Service-Learning JourneyLearning Journey (continued) (continued)
ScholarshipScholarship► Community engagedCommunity engaged► Parity Parity Faculty and staff recognitionFaculty and staff recognition► Protected timeProtected time► Promotion/tenurePromotion/tenureCommunity benefitCommunity benefit► CapacityCapacity► ResourcesResources► Improved health outcomesImproved health outcomes
96
Stages in the journeyStages in the journey1 Precontemplative1 Precontemplative
2 Contemplative2 Contemplative
3 Planning3 Planning
4 Early stage implementation4 Early stage implementation
5 Mid-stage implementation5 Mid-stage implementation
6 Well-established6 Well-established
7 Fully institutionalized (locally)7 Fully institutionalized (locally)
8 National/international recognition8 National/international recognition97
Where am I now? Where do I want to Where am I now? Where do I want to be?be?
►Working in small groups….Working in small groups…. Briefly catalogue strengths, resources, Briefly catalogue strengths, resources,
challengeschallenges Look at the service-learning journey and Look at the service-learning journey and
see where you are, where your institution see where you are, where your institution is, for several of the spokes of the Yoder is, for several of the spokes of the Yoder wheelwheel
Think of 2 action steps you can take when Think of 2 action steps you can take when you returnyou return
98
Community-engaged Community-engaged scholarshipscholarship
Where does service-learning fit in?Where does service-learning fit in?
CBPR Community based
research
Volunteerprojects
Community Service
Service-Learning
99
CBPRVolunteer projects
Community service
Community based
research
Selected referencesSelected referencesFurco A. Service-Learning: A Balanced Approach to Furco A. Service-Learning: A Balanced Approach to
Experiential Education Experiential Education Expanding Boundaries Expanding Boundaries 1996.1996.
Eyler J, Giles D, Stenson C, Gray C. At a Glance: Eyler J, Giles D, Stenson C, Gray C. At a Glance: What We Know about the Effects of Service-What We Know about the Effects of Service-Learning 1993-2000. Third Edition. Corporation for Learning 1993-2000. Third Edition. Corporation for National Service.National Service.
Seifer S. Service-learning: Community-Campus Seifer S. Service-learning: Community-Campus Partnerships for Health Professions Education Partnerships for Health Professions Education Acad MedAcad Med 1998;73(3):273-7. 1998;73(3):273-7.
Yoder K. A Framework for Service-Learning in Dental Yoder K. A Framework for Service-Learning in Dental Education. Education. Journal of Dental EducationJournal of Dental Education 2006;70(2):115-122. 2006;70(2):115-122.
http://www.ccph.info/http://www.ccph.info/http://depts.washington.edu/ccph/http://depts.washington.edu/ccph/
servicelearningres.htmlservicelearningres.html102
103
references …references …
► Centers for Disease Control and Prevention. Centers for Disease Control and Prevention. Practical Evaluation of Public Health ProgramsPractical Evaluation of Public Health Programs. . 1998.1998.
► Patton M. Patton M. Utilization-Focused EvaluationUtilization-Focused Evaluation. Sage . Sage Publications. 1997.Publications. 1997.
► WKKellogg Foundation. WKKellogg Foundation. Evaluation HandbookEvaluation Handbook. . 1998.1998.
► WKKellogg Foundation. Logic Model Development WKKellogg Foundation. Logic Model Development Guide. 2001.Guide. 2001.
► http://depts.washington.edu/ccph/http://depts.washington.edu/ccph/►American Evaluation Association American Evaluation Association
www.eval.orgwww.eval.org
references …references …
► Smith CS, Irby DM. The roles of experience and Smith CS, Irby DM. The roles of experience and reflection in ambulatory care education. Acad reflection in ambulatory care education. Acad Med (1997); 72: 32-35.Med (1997); 72: 32-35.
► Mann K, Gordon J, MacLeod A. Reflection and Mann K, Gordon J, MacLeod A. Reflection and reflective practice in health professions reflective practice in health professions education: a systematic review. Adv in Health education: a systematic review. Adv in Health Sci Educ (2009) 14:595–621.Sci Educ (2009) 14:595–621.
► Boenink, A, Oderwald A, de Jonge P, van Tilburg Boenink, A, Oderwald A, de Jonge P, van Tilburg W, Smal J. (2004). Assessing student reflection W, Smal J. (2004). Assessing student reflection inmedical practice. The development of an inmedical practice. The development of an observer-rated instrument: Reliability, validity observer-rated instrument: Reliability, validity and initial experiences. Medical Education and initial experiences. Medical Education (2004); 38:368-377.(2004); 38:368-377.
referencesreferences
► Glaze J. Reflection as a transforming process: Glaze J. Reflection as a transforming process: students advanced nurse practitioners’ students advanced nurse practitioners’ experiences of developing reflective skills as experiences of developing reflective skills as part of an MSc programme. J of Adv Nursing part of an MSc programme. J of Adv Nursing (2001); 34(5): 639-647.(2001); 34(5): 639-647.
► Leung D, Kember D. The relationship between Leung D, Kember D. The relationship between approaches to learning and reflection upon approaches to learning and reflection upon practice. Educational Psychology (2003); 23: practice. Educational Psychology (2003); 23: 61–71.61–71.
► Williams R, Sundelin O, Foster-Seargeant E, Williams R, Sundelin O, Foster-Seargeant E, Norman G. (2000). Assessing the reliability of Norman G. (2000). Assessing the reliability of grading reflective journal writing. J of Physical grading reflective journal writing. J of Physical Therapy Education (2000); 14: 23–26.Therapy Education (2000); 14: 23–26.
top related