Evaluation

HPS is a “change” process that takes place within a school

community

A key consideration is that the change needs to be sustainable

Sustainable change = changing peoples beliefs and values So sustainable change is achieved through values driven practice That is why the HPS Framework is a values-driven model

HPS values

The values underpinning HPS include:

– Te Tiriti o Waitangi

– Whanaungatanga – strengthening relationships

– Kotahitanga – partnership in learning, reciprocity

– Rangatiratanga – uplifting, growing, leadership

New or more information does not lead to sustainable improvement

To improve outcomes for our children and young people, Health Promotion professionals schools and communities don’t simply need new or more information.

For improvements to be sustainable learning needs to be transformational rather than just informational.

This is because…..

Transformational learning changes thinking and behaviours, while informational learning adds new skills and information without a change to belief systems and therefore related behaviours and attitudes.

change their beliefs or values

In our research we have identified that any improvement initiative in schools that focuses on improving outcomes and practices; requires teachers and leaders, as well as students and families/whānau, to change their beliefs or values too rather than to just adopt new skills or knowledge. “It is the purposeful thinking that counts, not the mere doing.”

• Mintzberg, H (2004) Managers not MBAs, Berret-Koehler Publishers, San Francisco

Changing beliefs, values & behaviours - through inquiry ( transformational process)

Elements of enduring professional learning and development in school communities – it must be:

• grounded, that is developed and delivered in the context of the school community’s everyday work while also being informed by key theories of future focussed learning.

• sustainable, that is have immediate application for the school community so that they see the learning as useful and therefore enduring, therefore replacing their previous held beliefs and practices

• spacious, that is responsive to the various viewpoints and diverse backgrounds that the participants bring to the experience

• contagious, that is designed so that all members of the school community have access to the ideas even if they are not having first hand experiences; that is the learning spreads across schools, networks and beyond and is available to new staff through in school mechanisms.

• connected, that is designed in a way that connects across all systems and processes in the school community.

How do we know if we are being successful/achieving the desired outcomes?

Monitoring, review and evaluation at school community level

Monitoring: on-going checks on progress towards outcomes e.g. student attendance data is showing less absences

Review: analysis of processes or actions

Evaluation: detailed analysis and judgment based on both quantitative and qualitative data to see how effective planned actions have been/how successful you have been in achieving the outcomes

A commitment to evaluation requires the development of an evaluation framework which provides:

• Triangulation of data/evidence

• A theory for change, intervention and improvement

• The context (physical, social, cultural, political)

• The process (identifying, planning, acting, monitoring, reporting and evaluating/reflecting on outcomes as a basis for on-going improvement)

• The factors that contribute to health in a school community

• The health actions that improve educational outcomes in the school context

• The outcomes (positive changes in knowledge, attitudes, skills, behaviours, social and physical environment) reflected in participation, engagement, achievement through quality

relationships and improved whānau wellbeing. (WHO 2009)

What data and evidence do we gather to show: Accelerated equity Improved Whānau outcomes Improved educational outcomes (Presence/participation, Engagement, Achievement

through quality relationships)

evidence

data

Decision

Evidence

• Any facts, circumstances or perceptions that can be used as an input for an analysis or decision

• Data are one form of evidence

Data • Known facts or measurements, probably

expressed in some systematic or symbolic way (e.g. as numbers)

• Assessment results, gender, attendance, ethnicity …

• Data are one form of evidence http://www.tki.org.nz/r/governance/consider

Current and Future

Outcomes/ Evaluation

Insufficient evidence on effectiveness of HPS in

New Zealand

Performance largely measured by outputs and

HPS sector perceptions

Evaluations largely focused on effectiveness of

initiatives

HPS logic model developed

Outcome and performance measures of school community and

population levels developed

New HPS framework evaluated

Māori world-views and Kaupapa embedded into HPS framework

evaluation

Our own learning needs

Design tasks

Teaching actions

Evidence used to judge

impact

Students learning needs

Our own learning needs

Design tasks

Teaching actions

Evidence used to judge

impact

Students learning needs

Our own learning needs

Design tasks

Teaching actions

Evidence used to judge

impact

Students learning needs

Our own learning needs

Design tasks

Teaching actions

Evidence used to judge

impact

Students learning needs

School communities

HPS facilitators

HPS Managers

MoE/ MoH HPS Values

Improved educational outcomes

Improved whānau

wellbeing

Monitoring, review and evaluation at all levels

Impact is monitored, reviewed and evaluated at all levels… Impact = the outcomes achieved by an intervention/initiative at each level:

National

Contract

Local

Government

HPS inquiry based processes

adopted in school communities

Long Term

Outcomes

HPS school community logic model

Medium Term

Outcomes

Short Term

Outcomes

Health Promoting Schools inquiry based learning in action stages 1-6

Health

Pro

motin

g S

chools va

lues a

nd p

rincip

les

HPS Inquiry based process embedded in school culture

Ultimate

Outcomes Whānau Ora Equity

School communities achieve at levels that enable

success

Improved Whānau wellbeing

Accelerated equity Improved educational

outcomes

Increased partnership

and collaboration

in school community

School community

has improved access to health,

education and social services

Supportive school

community policies,

practices and environments

School community

has improved knowledge,

skills, attitudes and

behaviours

School community has

improved presence,

engagement achievement

through quality relationships

Quality relationships and engagement in

the school community

Shared intersectoral

understandings, partnerships and

learnings

Quality HPS evaluation and systems in the

school community

Table 4 in Executive summary

• Short term outcomes

HPS local, contract, national and government impact indicators reported

• Process indicators

> Levels of integration (engage, implement and embed)

> Stages within the inquiry cycle

• Outcome indicators

> Presence

> Engagement

> Achievement through quality relationships

So for the facilitator on the ground

• Monitoring, review and evaluation need to be built in as part of the planning and reporting process

• The draft database and draft planning tool is intended to capture this information.