![Page 1: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/1.jpg)
Evaluation
![Page 2: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/2.jpg)
HPS is a “change” process that takes place within a school
community
A key consideration is that the change needs to be sustainable
![Page 3: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/3.jpg)
Sustainable change = changing peoples beliefs and values So sustainable change is achieved through values driven practice That is why the HPS Framework is a values-driven model
![Page 4: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/4.jpg)
HPS values
The values underpinning HPS include:
– Te Tiriti o Waitangi
– Whanaungatanga – strengthening relationships
– Kotahitanga – partnership in learning, reciprocity
– Rangatiratanga – uplifting, growing, leadership
![Page 5: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/5.jpg)
New or more information does not lead to sustainable improvement
To improve outcomes for our children and young people, Health Promotion professionals schools and communities don’t simply need new or more information.
For improvements to be sustainable learning needs to be transformational rather than just informational.
![Page 6: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/6.jpg)
This is because…..
Transformational learning changes thinking and behaviours, while informational learning adds new skills and information without a change to belief systems and therefore related behaviours and attitudes.
![Page 7: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/7.jpg)
change their beliefs or values
In our research we have identified that any improvement initiative in schools that focuses on improving outcomes and practices; requires teachers and leaders, as well as students and families/whānau, to change their beliefs or values too rather than to just adopt new skills or knowledge. “It is the purposeful thinking that counts, not the mere doing.”
• Mintzberg, H (2004) Managers not MBAs, Berret-Koehler Publishers, San Francisco
![Page 8: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/8.jpg)
Changing beliefs, values & behaviours - through inquiry ( transformational process)
![Page 9: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/9.jpg)
Elements of enduring professional learning and development in school communities – it must be:
• grounded, that is developed and delivered in the context of the school community’s everyday work while also being informed by key theories of future focussed learning.
• sustainable, that is have immediate application for the school community so that they see the learning as useful and therefore enduring, therefore replacing their previous held beliefs and practices
• spacious, that is responsive to the various viewpoints and diverse backgrounds that the participants bring to the experience
• contagious, that is designed so that all members of the school community have access to the ideas even if they are not having first hand experiences; that is the learning spreads across schools, networks and beyond and is available to new staff through in school mechanisms.
• connected, that is designed in a way that connects across all systems and processes in the school community.
![Page 10: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/10.jpg)
How do we know if we are being successful/achieving the desired outcomes?
![Page 11: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/11.jpg)
Monitoring, review and evaluation at school community level
Monitoring: on-going checks on progress towards outcomes e.g. student attendance data is showing less absences
Review: analysis of processes or actions
Evaluation: detailed analysis and judgment based on both quantitative and qualitative data to see how effective planned actions have been/how successful you have been in achieving the outcomes
![Page 12: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/12.jpg)
A commitment to evaluation requires the development of an evaluation framework which provides:
• Triangulation of data/evidence
• A theory for change, intervention and improvement
• The context (physical, social, cultural, political)
• The process (identifying, planning, acting, monitoring, reporting and evaluating/reflecting on outcomes as a basis for on-going improvement)
• The factors that contribute to health in a school community
• The health actions that improve educational outcomes in the school context
• The outcomes (positive changes in knowledge, attitudes, skills, behaviours, social and physical environment) reflected in participation, engagement, achievement through quality
relationships and improved whānau wellbeing. (WHO 2009)
![Page 13: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/13.jpg)
What data and evidence do we gather to show: Accelerated equity Improved Whānau outcomes Improved educational outcomes (Presence/participation, Engagement, Achievement
through quality relationships)
evidence
data
Decision
![Page 14: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/14.jpg)
Evidence
• Any facts, circumstances or perceptions that can be used as an input for an analysis or decision
• Data are one form of evidence
![Page 15: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/15.jpg)
Data • Known facts or measurements, probably
expressed in some systematic or symbolic way (e.g. as numbers)
• Assessment results, gender, attendance, ethnicity …
• Data are one form of evidence http://www.tki.org.nz/r/governance/consider
![Page 16: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/16.jpg)
Current and Future
Outcomes/ Evaluation
Insufficient evidence on effectiveness of HPS in
New Zealand
Performance largely measured by outputs and
HPS sector perceptions
Evaluations largely focused on effectiveness of
initiatives
HPS logic model developed
Outcome and performance measures of school community and
population levels developed
New HPS framework evaluated
Māori world-views and Kaupapa embedded into HPS framework
evaluation
![Page 17: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/17.jpg)
Our own learning needs
Design tasks
Teaching actions
Evidence used to judge
impact
Students learning needs
Our own learning needs
Design tasks
Teaching actions
Evidence used to judge
impact
Students learning needs
Our own learning needs
Design tasks
Teaching actions
Evidence used to judge
impact
Students learning needs
Our own learning needs
Design tasks
Teaching actions
Evidence used to judge
impact
Students learning needs
School communities
HPS facilitators
HPS Managers
MoE/ MoH HPS Values
Improved educational outcomes
Improved whānau
wellbeing
![Page 18: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/18.jpg)
Monitoring, review and evaluation at all levels
![Page 19: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/19.jpg)
Impact is monitored, reviewed and evaluated at all levels… Impact = the outcomes achieved by an intervention/initiative at each level:
National
Contract
Local
Government
![Page 20: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/20.jpg)
HPS inquiry based processes
adopted in school communities
Long Term
Outcomes
HPS school community logic model
Medium Term
Outcomes
Short Term
Outcomes
Health Promoting Schools inquiry based learning in action stages 1-6
Health
Pro
motin
g S
chools va
lues a
nd p
rincip
les
HPS Inquiry based process embedded in school culture
Ultimate
Outcomes Whānau Ora Equity
School communities achieve at levels that enable
success
Improved Whānau wellbeing
Accelerated equity Improved educational
outcomes
Increased partnership
and collaboration
in school community
School community
has improved access to health,
education and social services
Supportive school
community policies,
practices and environments
School community
has improved knowledge,
skills, attitudes and
behaviours
School community has
improved presence,
engagement achievement
through quality relationships
Quality relationships and engagement in
the school community
Shared intersectoral
understandings, partnerships and
learnings
Quality HPS evaluation and systems in the
school community
![Page 21: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/21.jpg)
Table 4 in Executive summary
• Short term outcomes
![Page 22: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/22.jpg)
HPS local, contract, national and government impact indicators reported
• Process indicators
> Levels of integration (engage, implement and embed)
> Stages within the inquiry cycle
• Outcome indicators
> Presence
> Engagement
> Achievement through quality relationships
![Page 23: Evaluation - hps.tki.org.nzhps.tki.org.nz/content/download/795/3376/file/Planning and Evaluati… · A commitment to evaluation requires the development of an evaluation framework](https://reader035.vdocuments.site/reader035/viewer/2022071117/600379cc9c518c23ec5c1bd6/html5/thumbnails/23.jpg)
So for the facilitator on the ground
• Monitoring, review and evaluation need to be built in as part of the planning and reporting process
• The draft database and draft planning tool is intended to capture this information.