1

Indicators to measure the effectiveness

of the implementation of the UNECE Strategy for ESD

Expert group Indicators for ESD

drs. R.M. van Raaij, ministry LNV, the Netherlands

2

Background:

Strategy for ESD (Vilnius, March 2005):To facilitate the introduction and promotion of education for sustainable development (ESD) in the UNECE region to the realisation of our common vision.

The Aim of the Strategy:To encourage UNECE member States tot develop and incorporate ESD into their formal education systems, in all relevant subjects, and in non-formal and informal education.

3

Objectives :

1) Ensure that policy, regulatory and operational frameworks support ESD

2) Promotion SD through formal, non-formal and informal learning

3) Equip educators with the competences to include SD in their teaching

4) Ensure that adequate tools and materials for ESD are accessible

5) Promote research on and development of ESD

6) Strengthen cooperation on ESD at all levels within the UNECE region.

4

The process of developing indicators

Establishment of Expert group following Vilnius

Mandate: “to develop indicators to measure the effectiveness of the implementation of the strategy.

- Four meetings scheduled

- Setting up a “framework”

- Translating objectives into questions : what do we need to know, what do we want to know, which data are available, what methodology is available

- State of the art: constructing indicators out of these questions

5

The expert group :

Representatives from: Armenia, Austria, Canada (via e-mail), France, Greece, Italy, Lithuania, Netherlands, Russian Federation, Slovenia, Sweden, United Kingdom, UNESCO, CA Interstate Working group on EE and ESD, ENSI, Ecoforum were designated; Germany will join for the future meetings; UNECE – the Secretariat

Meetings :

September 2005 Ede (Netherlands)

October 2005 Geneva

To come:

March 2006 Vienna

May 2006 The Hague

6

Developing the concepts, take into account :

-Sustainable Development is not a fixed goal, but a developing concept, process oriented

-“Learning” is a broad concept, especially in ESD regarding to non-formal and informal education.

-Education refers to knowledge, but also to attitude, values, skills, competences, behaviour.

-Methodology may be demanding in some respect

-Data are not always available, besides some in the field of formal education

- Education contributes to SD we trust and can be described in an logical chain, We can’t “proof” SD as a result from education alone.

7

We look for an evaluation model that covers :

a) the process of implementation

b) The effectiveness of the implementation (as a qualitative feature of both the process and long-term effects of ESD).

And that helps to:

c) distinguish between different types of indicators in relation to different phases of the process of implementation

8

Background for our work : Evaluation model

Check List……. Y/n……. Y/nTYPE 1

Type 0

Current Situation

(baseline)

Policy Framework

Throughputactivities

Direct / Indirect

Effects,Impact On SD

Outcome TYPE 4 outcome

TYPE 2input

TYPE 3Output

T=0

2015

9

An example:

Policy : to promote recycling of glass

Input : a campaign for collecting used glass

( target groups, Budget, media)

Throughput : TV-spots, posters, brochures

Output : number of people informed ?, awareness of the problem ?, people that can reproduce the message ? …………………..

Outcome: Knowledge ? Attitude ? Behaviour ? Or …..

Expected Effects: more glass collected for recycling !

10

So, different TYPES of indicators

Type 0 : base-line data: showing the existing situation in relation to an issue at a certain point in time

Type 1: Checklist indicators : provide information on initial policy, legislation, regulatory and governance measures taken by a government in order to implement the Strategy

Type 2: Input Indicators : provide information on a broader spectrum of activities taking place in terms of conducting activities on ESD

Type 3: Output Indicators : provide information on the result of these activities

Type 4: Outcome indicators : provide information on the possible impact due to the implementation of the strategy, in particular its qualitative aspect in terms of values, attitudes and choices in favour of SD

11

Translating objectives into indicators:State of the art.

-Construction of the evaluation model-Closely reading of the 6 objectives and text in the Strategy-Formulating the underlying questions (ca. 80)-Look into : required/proposed, Type, Phase of implementation, sources and availability.

-Now : for each objective we construct a group of indicators on underlying questions-To come : further construction and validation

-Example : under Objective 1 in our Table (in your documents

12

Further process:

-First draft of indicators prepared and submitted for comments to Steering Committee (December 2005)

-With regard to the issue of “indigenous people” (not a separate objective) it is proposed that a description on how their knowledge is conserved, used and promoted, be considered as an indicator

-Indicators to be brought in line with UNESCO reporting on the Decade for ESD, and with available data from other educational monitoring systems.

13

At the next meetings the experts :

-will focus on consistency and validation and deletion of duplications in the draft

-Will work further on the grouping of underlying questions in order to develop aggregated indicators

-Will prepare give handout and guide for use of this indicator system

-Prepare for adoption by the Steering Committee in 2006.

14

The expert group gives into consideration :

-With respect of difference between States, each state should consider a baseline report of the situation in 2005, which is useful to measure progress

-Into each country the possibility of a monitoring system should be determined. In national action plans the objectives should be formulated as operational goals.

- observation: Through the nature of the Strategy it selves most of indicators seems to be “checklist” or “input”type ! We are aware, but its not in our mandate to change that.

-This system of evaluation is not for benchmark or reporting only : in line with the spirit of the Strategy we should celebrate progress, share good practices and LEARN !

15

Thank you for your attention