developing and testing performance indicators for the development and transfer of est’s under the...

56
Developing and testing performance Developing and testing performance indicators for the development and indicators for the development and transfer of EST’s transfer of EST’s under the UNFCCC: under the UNFCCC: about process and methodology about process and methodology Bernard MAZIJN (Belgium) Bernard MAZIJN (Belgium) UNFCCC Expert Meeting on Monitoring and UNFCCC Expert Meeting on Monitoring and Evaluating Evaluating Capacity-building in Developing Countries Capacity-building in Developing Countries Rio de Janeiro, 6-7 November 2008 Rio de Janeiro, 6-7 November 2008

Upload: godfrey-little

Post on 23-Dec-2015

213 views

Category:

Documents


0 download

TRANSCRIPT

Developing and testing performance Developing and testing performance indicators for the development and indicators for the development and

transfer of EST’s transfer of EST’s under the UNFCCC: under the UNFCCC:

about process and methodologyabout process and methodology

Bernard MAZIJN (Belgium)Bernard MAZIJN (Belgium)UNFCCC Expert Meeting on Monitoring and UNFCCC Expert Meeting on Monitoring and

EvaluatingEvaluatingCapacity-building in Developing CountriesCapacity-building in Developing Countries

Rio de Janeiro, 6-7 November 2008Rio de Janeiro, 6-7 November 2008

Content of the presentationContent of the presentation

• IntroductionIntroduction• Developing and testing indicators: Developing and testing indicators:

a participative co-design processa participative co-design process• Indicators to monitor and to evaluate Indicators to monitor and to evaluate

effectivenesseffectiveness• Experiences with performance indicators Experiences with performance indicators

inside the conventioninside the convention• ‘‘Unravelling’ the TT Framework Unravelling’ the TT Framework • Next stepsNext steps

Content of the presentationContent of the presentation

• IntroductionIntroduction• Developing and testing indicators: Developing and testing indicators:

a participative co-design processa participative co-design process• Indicators to monitor and to evaluate Indicators to monitor and to evaluate

effectivenesseffectiveness• Experiences with performance indicators Experiences with performance indicators

inside the conventioninside the convention• ‘‘Unravelling’ the TT Framework Unravelling’ the TT Framework • Next stepsNext steps

UNFCCC – Art.4. CommitmentsUNFCCC – Art.4. Commitments

……

§ 5. The developed country Parties and other developed § 5. The developed country Parties and other developed Parties included in Annex II shall take all practicable Parties included in Annex II shall take all practicable steps to promote, facilitate and finance, as steps to promote, facilitate and finance, as appropriate, the transfer of, or access to, appropriate, the transfer of, or access to, environmentally sound technologies and know-how to environmentally sound technologies and know-how to other Parties, particularly developing country Parties, other Parties, particularly developing country Parties, to enable them to implement the provisions of the to enable them to implement the provisions of the Convention. In this process, the developed country Convention. In this process, the developed country Parties shall support the development and Parties shall support the development and enhancement of endogenous capacities and enhancement of endogenous capacities and technologies of developing country Parties. Other technologies of developing country Parties. Other Parties and organizations in a position to do so may Parties and organizations in a position to do so may also assist in facilitating the transfer of such also assist in facilitating the transfer of such technologies.technologies.

……

History of the development and History of the development and transfer of EST’s as seen under transfer of EST’s as seen under

UNFCCCUNFCCC

• Phase 1 (1992-1998): turbulent periodPhase 1 (1992-1998): turbulent period• Phase 2 (1998-2002): consultative roundPhase 2 (1998-2002): consultative round• Phase 3 (2002-2008): ‘common Phase 3 (2002-2008): ‘common

understanding’understanding’• Phase 4 (2008-2012): acceleration on Phase 4 (2008-2012): acceleration on

implementationimplementation• Phase 5 (Post-2012): massive deployment Phase 5 (Post-2012): massive deployment

and diffusionand diffusion

COP 7Establishment of the UNFCCC

Expert Group on Technology Transfer

Development and transfer of EST’sDevelopment and transfer of EST’sunder the UNFCCC: 5 key under the UNFCCC: 5 key

elementselements

• Technology Needs AssessmentsTechnology Needs Assessments• Technology Information Technology Information • Capacity BuildingCapacity Building• Enabling EnvironmentsEnabling Environments• Financial and Institutional Financial and Institutional

Mechanisms Mechanisms

(( cf. link with CB Framework cf. link with CB Framework & CB priorities 2, 8 and 10)& CB priorities 2, 8 and 10)

The overall objective of the The overall objective of the assignmentassignment

• “ … “ … to develop and test a balanced and robust to develop and test a balanced and robust set of performance indicators set of performance indicators that could be that could be usedused by the SBI to monitor and evaluate the by the SBI to monitor and evaluate the effectiveness of the implementation of the effectiveness of the implementation of the technology transfer framework.”technology transfer framework.”

• The outcome of this work can also provide input The outcome of this work can also provide input to the work of the SB’s to the work of the SB’s – (i) on reviewing and assessing the effectiveness of the (i) on reviewing and assessing the effectiveness of the

implementation of Article 4, paragraphs 1(c) and 5, implementation of Article 4, paragraphs 1(c) and 5, – (ii) on considering the role of new financing (ii) on considering the role of new financing

mechanisms and tools for scaling up development and mechanisms and tools for scaling up development and transfer of technology transfer of technology

– (iii) on Long-term Cooperative Action under the (iii) on Long-term Cooperative Action under the Convention Convention

Deliverable Report

Date Task Meeting Main contents

Background paper

July 31st, 2008Basis for the first interim report

Special EGTT Meeting in Ghana

Outline of the approach(incl. Text 1, Text 2 and Text 3)

First interim report

September 30th 2008

Task 1: develop a set of candidate performance indicators

UNFCCC Expert Meeting in Ireland

A long list of indicators(incl. Text 4)

Second interim report

February 28th, 2009

Task 2: Test the set of performance indicators

Selection of key indicators, incl. methodological sheets

Final draft report June 30th, 2009

Task 3: Prepare recommendations for using the indicators

Recommendations for using the indicators

Final report August 31st, 2009 COP 15

Content of the presentationContent of the presentation

• IntroductionIntroduction• Developing and testing indicators: Developing and testing indicators:

a participative co-design processa participative co-design process• Indicators to monitor and to evaluate Indicators to monitor and to evaluate

effectivenesseffectiveness• Experiences with performance indicators Experiences with performance indicators

inside the conventioninside the convention• ‘‘Unravelling’ the TT Framework Unravelling’ the TT Framework • Next stepsNext steps

IISD Bellagio PrinciplesIISD Bellagio Principles

These principles deal with four aspects of assessing These principles deal with four aspects of assessing progress progress

toward sustainable development:toward sustainable development:

• Principle 1Principle 1 deals with the starting point of any deals with the starting point of any assessment - establishing a ‘assessment - establishing a ‘visionvision’ and defining ’ and defining ‘‘clear goalsclear goals’.’.

• Principles 2 through 5Principles 2 through 5 ( (‘holistic perspective’, ‘holistic perspective’, ‘essential elements’, ‘adequate scope’ and ‘practical ‘essential elements’, ‘adequate scope’ and ‘practical focus’) focus’) deal with deal with the content of any assessmentthe content of any assessment. .

• Principles 6 through 8Principles 6 through 8 (‘ (‘openness’, ‘effective openness’, ‘effective communication’ and ‘broad participation’) communication’ and ‘broad participation’) deal with deal with key issues of the process of assessmentkey issues of the process of assessment..

• Principles 9 and 10Principles 9 and 10 (‘ongoing assessment’ and (‘ongoing assessment’ and ‘institutional capacity’) ‘institutional capacity’) deal with deal with the necessity for the necessity for establishing a continuing capacity for establishing a continuing capacity for assessmentassessment..

BACKGROUND

Text 1:UN Normative frame

‘DTT’

INDICATORS

Feedback and selection

presented to the

stakeholders

BACKGROUND

Text 1:UN Normative frame

‘DTT’

INDICATORS

Feedback and selection

presented to the

stakeholders

Rio Declaration, Agenda 21 and UNFCCC (in particular the references to DTT)=

BACKGROUND

Text 1:UN Normative frame

‘DTT’

Text 2: Current situation

analysis

INDICATORS

Feedback and selection

presented to the

stakeholders

BACKGROUND

Text 1:UN Normative frame

‘DTT’

Text 2: Current situation

analysis

INDICATORS

Feedback and selection

presented to the

stakeholders

‘The promise of technology’, ‘Results of the assements by the Expert Group on Technology Transfer’ and ‘Expert Group on Technology Transfer: Five years of work’

plus detailed reports on one of the key elements of the technology transfer framework

=

BACKGROUND

Text 1:Normative frame

‘DTT’

Text 2: Current situation

analysis

Text 3:Vision of the UNFCCC

on DTT

INDICATORS

Feedback and selection

presented to the

stakeholders

BACKGROUND

Text 1:Normative frame

‘DTT’

Text 2: Current situation

analysis

Text 3:Vision of the UNFCCC

on DTT

INDICATORS

Feedback and selection

presented to the

stakeholders

TT Framework (Annexes to 4/CP.7 & 3/CP.13)=

BACKGROUND

Text 1:Normative frame

‘DTT’

Text 2: Current situation

analysis

Text 3:Vision of the UNFCCC

on DTT

INDICATORS

Feedback and selection

presented to the

stakeholders

This is

an ex-post exercise !

BACKGROUND

Text 1:UN Normative frame

‘DTT’

Text 2: Current situation

analysis

Text 3:Vision of the UNFCCC

on DTT

Text 4:Tree structure

INDICATORS

Feedback and selection

presented to the stakeholdders

BACKGROUND

Text 1:UN Normative frame

‘DTT’

Text 2: Current situation

analysis

Text 3:Vision of the UNFCCC

on DTT

Text 4:Tree structure

INDICATORS

Feedback and selection

presented to the stakeholdders

‘unravel’ exercise

BACKGROUND

Text 1:UN Normative frame

‘DTT’

Text 2: Current situation

analysis

Text 3:Vision of the UNFCCC

on DTT

Text 4:Tree structure

INDICATORS

Draft model list indicators

- taking into account the internationally accepted prerequisites

- taking into account existing related work on indicators

Feedback and selection

presented to the

stakeholders

Results

a set of indicators for DTT

with a methodolocical sheet for every indicator in the

proposed model list

including recommendations

Feedback loop to take corrective action

BACKGROUND

Text 1:UN Normative frame,

on DTT

Text 2: Current situation analysis

Text 3:Vision of the UNFCCC

on DTT

Text 4:Tree structure

INDICAT.

Draft model list indicators

-taking into account the internationally accepted prerequisites

-- taking into account existing related work on indicators

Feedback and selection

presented to the

stakeholders

Results

a set of indicatorsfor DTT

with a methodological sheet for every indicator in the proposed list

includingrecommendations

Strategic goals for DTT-policy

Operational subgoals for DTT-policy

Indicators for the evaluation of DTT policy

Policy vision concerning DTT

for each competent authority: IGO’s (UNDP, UNEP, GEF …),

Annex II Parties, Non-Annex I Parties …

See the work of the Executive Committee

under the MLF of the Montréal Protocol

Content of the presentationContent of the presentation

• IntroductionIntroduction• Developing and testing indicators: Developing and testing indicators:

a participative co-design processa participative co-design process• Indicators to monitor and to evaluate Indicators to monitor and to evaluate

effectivenesseffectiveness• Experiences with performance indicators Experiences with performance indicators

inside the conventioninside the convention• ‘‘Unravelling’ the TT Framework Unravelling’ the TT Framework • Next stepsNext steps

Evaluation of a policyEvaluation of a policy

goals inputs outputs

impacts

outcomes

External

factorssociety

Relevance: to what extend are the goals justified in relation to needs?

Effectiveness: to what extend have the expected goals been achieved?

Efficiency: Have the goals been achieved at the lowest cost?

Utility: Are the goals or unexpected effects contributing to a net increase in social welfare?

Relevance

Utility

Effectiveness

Efficiency

What are we looking for?What are we looking for?

• ‘‘A set of measures enabling to monitor A set of measures enabling to monitor performance, to track progress towards the performance, to track progress towards the achievement of objectives, to demonstrate achievement of objectives, to demonstrate results, and to take corrective action.’ results, and to take corrective action.’

• They can be expressed as qualitative or They can be expressed as qualitative or quantitative measures, financial and non-quantitative measures, financial and non-financial metrics, SMART (financial metrics, SMART (Specific, Measurable, Specific, Measurable, Achievable, Relevant, and Time-bound) Achievable, Relevant, and Time-bound) formulated. Quid link with MRV?formulated. Quid link with MRV?

• The indicators will measure ‘means’ (methods to The indicators will measure ‘means’ (methods to achieve objectives) or ‘ends’ (achievement of achieve objectives) or ‘ends’ (achievement of objectives) or a combination at any point along objectives) or a combination at any point along the continuum (inputs, processes, outputs, the continuum (inputs, processes, outputs, outcomes, and impacts) from means to ends. outcomes, and impacts) from means to ends.

Content of the presentationContent of the presentation

• IntroductionIntroduction• Developing and testing indicators: Developing and testing indicators:

a participative co-design processa participative co-design process• Indicators to monitor and to evaluate Indicators to monitor and to evaluate

effectivenesseffectiveness• Experiences with performance Experiences with performance

indicators inside (and outside) the indicators inside (and outside) the conventionconvention

• ‘‘Unravelling’ the TT Framework Unravelling’ the TT Framework • Next stepsNext steps

Monitoring and evaluation Monitoring and evaluation of capacity-buildingof capacity-building

Source of information, inter alea:Source of information, inter alea:

• A proposal for the collection, processing and A proposal for the collection, processing and dissemination of information on monitoring capacity dissemination of information on monitoring capacity and capacity-building activities in developing and capacity-building activities in developing countries countries (Document FCCC/SBI/2005/5) (Document FCCC/SBI/2005/5)

• A possible format for reporting on the regular A possible format for reporting on the regular monitoring of the implementation of the capacity-monitoring of the implementation of the capacity-building framework in accordance with decision building framework in accordance with decision 2/CP.7, paragraph 9.2/CP.7, paragraph 9.(Document (Document FCCC/SBI/2007/5)FCCC/SBI/2007/5)

• Report on the expert workshop on monitoring and Report on the expert workshop on monitoring and evaluating capacity-building in developing countries. evaluating capacity-building in developing countries. (Document (Document FCCC/SBI/2007/33)FCCC/SBI/2007/33)

Priority areas for capacity building

Strategic areas of support

Methodological approaches

Targets Outcomes or expected results

Performance indicators (progress, gap, effectiveness)

Metrics & data sources

Participation in UNCCCC

Human e.g. training for negotiations

Critical mass of negotiations

Capacity to mobilize information

No of negotiators

Attendance at UNFCCC conferences

Institutional Strengthen UNFCCC focal points

Focal points working within 1 year

Capacity to monitor

Existence of the focal points

Dates of establishment

Systemic Enhance enabling environment

Change in public perceptions

Capacity to engage and build consensus

Attitude and perceptions

Submission from Parties & surveys

A proposal for structuring available A proposal for structuring available informationinformation

Comparison with the present Comparison with the present exercise exercise

for the technology transfer for the technology transfer framework framework

The following similarities and differences can be observed:The following similarities and differences can be observed:

• ‘‘Priority areas for capacity building’ = the key themes of the TT-Priority areas for capacity building’ = the key themes of the TT-framework;framework;

• ‘‘Strategic areas of support’: this does not exist in the present exercise;Strategic areas of support’: this does not exist in the present exercise;• ‘‘Methodological approaches’ = the ‘unravelled’ objectives (= Methodological approaches’ = the ‘unravelled’ objectives (=

actions/activities);actions/activities);• ‘‘Targets’: it seems that sometimes real targets are formulated and in Targets’: it seems that sometimes real targets are formulated and in

other proposed targets are more going in the direction of ‘measurement’ other proposed targets are more going in the direction of ‘measurement’ of the indicator;of the indicator;

• ‘‘Outcomes or expected results’: there is no distinction between ‘outputs’ Outcomes or expected results’: there is no distinction between ‘outputs’ and ‘outcomes’; it seems there is no intention to formulate ‘impact’ and ‘outcomes’; it seems there is no intention to formulate ‘impact’ indicators; indicators;

• ‘‘Performance indicators’: the proposed indicators are linked to the Performance indicators’: the proposed indicators are linked to the possibility to report on ‘progress’, ‘gap’ and/or ‘effectiveness’;possibility to report on ‘progress’, ‘gap’ and/or ‘effectiveness’;

• ‘‘Metrics and data sources’: this information will be reflected in the TT-Metrics and data sources’: this information will be reflected in the TT-methodological sheets.methodological sheets.

Other relevant initiatives Other relevant initiatives inside the Conventioninside the Convention

• Reporting on global observing systems for climateReporting on global observing systems for climate• Administrative and financial mattersAdministrative and financial matters• Kyoto ProtocolKyoto Protocol• Climate related risks and extreme eventsClimate related risks and extreme events

In general it can be concluded the experiences are In general it can be concluded the experiences are

rather limited and that processes have been rather limited and that processes have been initiated initiated

but are not well advanced yet in their but are not well advanced yet in their implementation.implementation.

Lessons learntLessons learnt

• Learning curveLearning curve• Framework with causal relationsFramework with causal relations• Performance indicatorsPerformance indicators• Data availabilityData availability• StakeholdersStakeholders• Capacity buildingCapacity building

Content of the presentationContent of the presentation

• IntroductionIntroduction• Developing and testing indicators: Developing and testing indicators:

a participative co-design processa participative co-design process• Indicators to monitor and to evaluate Indicators to monitor and to evaluate

effectivenesseffectiveness• Experiences with performance indicators Experiences with performance indicators

inside the conventioninside the convention• ‘‘Unravelling’ the TT FrameworkUnravelling’ the TT Framework • Next stepsNext steps

BACKGROUND

Text 1:UN Normative frame

‘DTT’

Text 2: Current situation

analysis

Text 3:Vision of the UNFCCC

on DTT

Text 4:Tree structure

INDICATORS

Draft model list indicators

- taking into account the internationally accepted prerequisites

- taking into account existing related work on indicators

Feedback and selection

presented to the

stakeholders

Results

a set of indicators for DTT

with a methodolocical sheet for every indicator in the

proposed model list

including recommendations

Reductionist approach?

What is the rationale for labelling the TT Framework, in particular the key element ‘enabling environments’?

The difference is the starting point:a comprehensive TT Framework adopted as a vision.

The ‘unravel’ exercise: The ‘unravel’ exercise: a step-by-step approacha step-by-step approach

1.1. Objectives for the key theme Objectives for the key theme

2.2. Regrouping by synthesized objectiveRegrouping by synthesized objective

3.3. Re-grouping by stakeholderRe-grouping by stakeholder

4.4. Re-grouping by indicator categoryRe-grouping by indicator category

5.5. Checking against earlier proposalsChecking against earlier proposals(i.e. submissions by Parties)(i.e. submissions by Parties)

6.6. Identification of missing links between Identification of missing links between vision, objectives and effectsvision, objectives and effects

The ‘unravel’ exercise: The ‘unravel’ exercise: a step-by-step approacha step-by-step approach

1.1. Objectives for the key themeObjectives for the key theme

2.2. Regrouping by synthesized objectiveRegrouping by synthesized objective

3.3. Re-grouping by stakeholderRe-grouping by stakeholder

4.4. Re-grouping by indicator categoryRe-grouping by indicator category

5.5. Checking against earlier proposalsChecking against earlier proposals(i.e. submissions by Parties)(i.e. submissions by Parties)

6.6. Identification of missing links between Identification of missing links between vision, objectives and effectsvision, objectives and effects

Objectives for each themeObjectives for each theme

The technology transfer framework - contained The technology transfer framework - contained in the Annex to decision 4/CP.7, in the Annex to decision 4/CP.7, complemented with the set of actions set out in decision complemented with the set of actions set out in decision

3/CP.13, annex I,3/CP.13, annex I,for each theme consists of the following elements:for each theme consists of the following elements:

• in the Annex to decision 4/CP.7, under C. Key themes in the Annex to decision 4/CP.7, under C. Key themes and areas for meaningful and effective actions,and areas for meaningful and effective actions,– definitiondefinition– purposepurpose– implementationimplementation

• in the Annex I to decision 3/CP.13, in the Annex I to decision 3/CP.13, – recommendations for enhancing the implementationrecommendations for enhancing the implementation

In addition there are cross-references in other key theme.In addition there are cross-references in other key theme.

The ‘unravel’ exercise: The ‘unravel’ exercise: a step-by-step approacha step-by-step approach

1.1. Objectives for the key theme Objectives for the key theme

2.2. Regrouping by synthesized objectiveRegrouping by synthesized objective(incl. the formulation of performance (incl. the formulation of performance indicators)indicators)

3.3. Re-grouping by stakeholderRe-grouping by stakeholder

4.4. Re-grouping by indicator categoryRe-grouping by indicator category

5.5. Checking against earlier proposalsChecking against earlier proposals(i.e. submissions by Parties)(i.e. submissions by Parties)

6.6. Identification of missing links between Identification of missing links between vision, objectives and effectsvision, objectives and effects

CB: re-grouping by synthesized CB: re-grouping by synthesized objectiveobjective

• ‘‘to report on Capacity Building Needs and to report on Capacity Building Needs and experiences experiences for DTTfor DTT of the developing country of the developing country Parties’Parties’

• ‘‘to implement/support CB activities to implement/support CB activities for DTTfor DTT in in developing country Parties.developing country Parties.

• ‘‘to establish/strengthen capacity to establish/strengthen capacity for DTTfor DTT in in institutions of developing country Parties’.institutions of developing country Parties’.

• ‘‘to increase/enhance/improve to increase/enhance/improve awareness/knowledge awareness/knowledge on ESTson ESTs in developing in developing country Parties’.country Parties’.

• ‘‘to provide training to provide training on ESTson ESTs in developing country in developing country Parties’Parties’

• ‘‘to develop and implement standards and to develop and implement standards and regulations regulations for ESTsfor ESTs’’

The ‘unravel’ exercise: The ‘unravel’ exercise: a step-by-step approacha step-by-step approach

1.1. Objectives for the key theme Objectives for the key theme

2.2. Regrouping by synthesized objectiveRegrouping by synthesized objective

3.3. Re-grouping by stakeholderRe-grouping by stakeholder

4.4. Re-grouping by indicator categoryRe-grouping by indicator category

5.5. Checking against earlier proposalsChecking against earlier proposals(i.e. submissions by Parties)(i.e. submissions by Parties)

6.6. Identification of missing links between Identification of missing links between vision, objectives and effectsvision, objectives and effects

Input-indicator

Process-indicator

Output-indicators

Outcome-indicators

Total

CB 7 8 11 10 36

An overview of the number of An overview of the number of indicators by category for indicators by category for

capacity-buildingcapacity-building

And, in terms of stakeholders And, in terms of stakeholders for capacity-building...for capacity-building...

… … the 36 indicators are targetting:the 36 indicators are targetting:

• UNFCCC (Secretariat, EGTT, SBSTA, UNFCCC (Secretariat, EGTT, SBSTA, SBI)SBI)

• IGO’sIGO’s• Developed country PartiesDeveloped country Parties• All PartiesAll Parties

The ‘unravel’ exercise: The ‘unravel’ exercise: a step-by-step approacha step-by-step approach

1.1. Objectives for the key theme Objectives for the key theme

2.2. Regrouping by synthesized objectiveRegrouping by synthesized objective

3.3. Re-grouping by stakeholderRe-grouping by stakeholder

4.4. Re-grouping by indicator categoryRe-grouping by indicator category

5.5. Checking against earlier proposalsChecking against earlier proposals(i.e. submissions by Parties)(i.e. submissions by Parties)

6.6. Identification of missing links between Identification of missing links between vision, objectives and effectsvision, objectives and effects

In the document FCCC/SBI/2008/MISC.1 and Add.1, Parties In the document FCCC/SBI/2008/MISC.1 and Add.1, Parties have made proposals of performance indicators, which are have made proposals of performance indicators, which are summarized in FCCC/SBI/2008/7. In Annex IV each of the summarized in FCCC/SBI/2008/7. In Annex IV each of the proposed indicators is discussed. It seems that most/some proposed indicators is discussed. It seems that most/some

of the of the performance indicators by Parties in their submissions are performance indicators by Parties in their submissions are

– or already taken up by making explicit the causal or already taken up by making explicit the causal relationship;relationship;

– or targeting the very short term;or targeting the very short term;– or emphasizing a number of (new) objectives;or emphasizing a number of (new) objectives;– or need to be unravelled in clearly formulated or need to be unravelled in clearly formulated

separate objectives;separate objectives;– or are included and will be checked – later on - or are included and will be checked – later on -

against the SMART-principles.against the SMART-principles.

It is clear that further in the process a check against It is clear that further in the process a check against indicators indicators

already proposed in other experiences inside and outside already proposed in other experiences inside and outside the the

UNFCCC could help to finalize the set of performance UNFCCC could help to finalize the set of performance indicators.indicators.

The ‘unravel’ exercise: The ‘unravel’ exercise: a step-by-step approacha step-by-step approach

1.1. Objectives for the key theme Objectives for the key theme

2.2. Regrouping by synthesized objectiveRegrouping by synthesized objective

3.3. Re-grouping by stakeholderRe-grouping by stakeholder

4.4. Re-grouping by indicator categoryRe-grouping by indicator category

5.5. Checking against earlier proposalsChecking against earlier proposals(i.e. submissions by Parties)(i.e. submissions by Parties)

6.6. Identification of missing links Identification of missing links between vision, objectives and effectsbetween vision, objectives and effects

Capacity-buildingCapacity-building

to buildto buildto developto develop

““processprocess to strengthen existing scientific and to strengthen existing scientific and technical skills/capabilities/institutionstechnical skills/capabilities/institutions

to enhanceto enhanceto improveto improve

to accessto accessto adaptto adapt

in NAII Partiesin NAII Parties in order to enable in order to enable to to managemanage EST’sEST’s

to developto develop

disseminationdissemination to strengthen capacities of NAII to promoteto strengthen capacities of NAII to promoteapplicationapplication of EST’s” of EST’s”

developmentdevelopment

Capacity-BuildingCapacity-Building

• From the goal as spelled out, it can be taken From the goal as spelled out, it can be taken the focus is on ‘strengthening the capabilities in the focus is on ‘strengthening the capabilities in Non-Annex II Parties’, through the support of Non-Annex II Parties’, through the support of ‘existing scientific and technical ‘existing scientific and technical skills/capabilities/institutions’. skills/capabilities/institutions’.

• But CB is a cross-cutting theme through all the But CB is a cross-cutting theme through all the other key themes of the technology transfer other key themes of the technology transfer framework.framework.

• Furthermore, it can be concluded from the Furthermore, it can be concluded from the objectives in the technology transfer framework objectives in the technology transfer framework that the burden lies with different stakeholders.that the burden lies with different stakeholders.

• There is need for streamlining with your There is need for streamlining with your exercise before selecting the final list of exercise before selecting the final list of performance indicators for the key theme performance indicators for the key theme ‘Capacity-Building’ under the technology ‘Capacity-Building’ under the technology transfer framework.transfer framework.

At the EGTT Meeting in At the EGTT Meeting in Dublin, Dublin,

the initial list of indicators the initial list of indicators has been reduced to … has been reduced to …

… … a preliminary list of 4 key a preliminary list of 4 key indicators for CB (1)indicators for CB (1)

• Amount of financial inputs (scholarships, Amount of financial inputs (scholarships, institutional financing, technical assistance) by institutional financing, technical assistance) by developed countries into capacity building for developed countries into capacity building for development and transfer of technology in development and transfer of technology in developing countries.developing countries.(cf. the relation with ‘human capacity’) (cf. the relation with ‘human capacity’) (input-indicator) (Annex II Parties)(input-indicator) (Annex II Parties)

• Reported needs and agreed priorities for Reported needs and agreed priorities for capacity building for development and transfer of capacity building for development and transfer of technologytechnology(cf. the relation with ‘institutional capacity’) (cf. the relation with ‘institutional capacity’) (input-indicator) (Non-Annex I Parties)(input-indicator) (Non-Annex I Parties)

… … a preliminary list of 4 key a preliminary list of 4 key indicators for CB (2)indicators for CB (2)

• Number of participants/experts in DTT Number of participants/experts in DTT training programmes, in particular on the training programmes, in particular on the development of standards and regulations development of standards and regulations (cf. the relation with ‘human capacity’) (cf. the relation with ‘human capacity’) (output-indicator) (Non-Annex I Parties)(output-indicator) (Non-Annex I Parties)

• Number of new and existing national and Number of new and existing national and regional institutions, operating as centers of regional institutions, operating as centers of excellence, on DTT excellence, on DTT (cf. the relation with ‘institutional capacity’) (cf. the relation with ‘institutional capacity’) (output-indicator) (Non-Annex I Parties)(output-indicator) (Non-Annex I Parties)

No indicatorsNo indicatorsrelated to the ‘systemic’ strategic related to the ‘systemic’ strategic

area of support,area of support,because there were to related because there were to related

objectivesobjectives

Content of the presentationContent of the presentation

• IntroductionIntroduction• Developing and testing indicators: Developing and testing indicators:

a participative co-design processa participative co-design process• Indicators to monitor and to evaluate Indicators to monitor and to evaluate

effectivenesseffectiveness• Experiences with performance indicators Experiences with performance indicators

inside the conventioninside the convention• ‘‘Unravelling’ the TT Framework Unravelling’ the TT Framework • Next stepsNext steps

For each of the 5 key themesFor each of the 5 key themesof the TT Framework: of the TT Framework:

testing the preliminary list testing the preliminary list of key indicatorsof key indicators

Deliverable Report

Date Task Meeting Main contents

Background paper

July 31st, 2008Basis for the first interim report

Special EGTT Meeting in Ghana

Outline of the approach(incl. Text 1, Text 2 and Text 3)

First interim report

September 30th 2008

Task 1: develop a set of candidate performance indicators

UNFCCC Expert Meeting in Ireland

A long list of indicators(incl. Text 4)

Second interim report

February 28th, 2009

Task 2: Test the set of performance indicators

Selection of key indicators, incl. methodological sheets

Final draft report June 30th, 2009

Task 3: Prepare recommendations for using the indicators

Recommendations for using the indicators

Final report August 31st, 2009 COP 15

Draft Methodological SheetDraft Methodological Sheet

The next step is to look if the indicators can be formulated The next step is to look if the indicators can be formulated SMART SMART

(Specific, Measurable, Achievable, Relevant, and Time-(Specific, Measurable, Achievable, Relevant, and Time-bound). bound).

Filling in the methodological sheet for each and every Filling in the methodological sheet for each and every indicator is a way of indicator is a way of

checking these requirements.checking these requirements.

• DESCRIPTION OF THE PERFORMANCE INDICATORDESCRIPTION OF THE PERFORMANCE INDICATOR• POLICY RELEVANCEPOLICY RELEVANCE• METHODOLOGICAL DESCRIPTION METHODOLOGICAL DESCRIPTION • ASSESSMENT OF DATAASSESSMENT OF DATA• ALTERNATIVE DEFINITION(S) OR PERFORMANCE ALTERNATIVE DEFINITION(S) OR PERFORMANCE

INDICATOR(S)INDICATOR(S)• REFERENCESREFERENCES• FACTUAL DATA AND EVALUATIONFACTUAL DATA AND EVALUATION

Linking with MRV Linking with MRV (measurable, reportable and (measurable, reportable and

verifiable)verifiable)

• The notion of MRV (measurable, reportable and The notion of MRV (measurable, reportable and verifiable) appears in the sub-paragraphs (i) and (ii) verifiable) appears in the sub-paragraphs (i) and (ii) of paragraph 1(b) of 1/CP.13, the so-called Bali of paragraph 1(b) of 1/CP.13, the so-called Bali Action Plan. There is an exclusive reference to Action Plan. There is an exclusive reference to ‘mitigation (commitments and) actions’ of Parties.‘mitigation (commitments and) actions’ of Parties.

• One could wonder about the link with performance One could wonder about the link with performance indicators. Taking into account the methodological indicators. Taking into account the methodological approach described in this report, the following approach described in this report, the following observations about the communalities can be made:observations about the communalities can be made:– the performance indicators will be formulated in a SMART-the performance indicators will be formulated in a SMART-

way; the ‘M’ stands for ‘measurable’;way; the ‘M’ stands for ‘measurable’;– a methodological sheet will be developed for each a methodological sheet will be developed for each

performance indicator in order to be reportable and performance indicator in order to be reportable and verifiable.verifiable.

Reflections on the discussion Reflections on the discussion during the RJ Expert Meetingduring the RJ Expert Meeting

• Bidirectional cross-linking with other Bidirectional cross-linking with other processes processes under the UNFCCCunder the UNFCCC

• Development of indicators should not be pre-Development of indicators should not be pre-conditioned (cf. conceptual indicators, 2nd conditioned (cf. conceptual indicators, 2nd choice indicators, etc.)choice indicators, etc.)

• Indicators Indicators Monitoring Monitoring Evaluation Evaluation• About indicatorsAbout indicators

– Number?Number?– Targeting who? Related to accountability?Targeting who? Related to accountability?– At macro (= national) level.At macro (= national) level.

• Does size matters?Does size matters?

Thank you.Thank you.