the national evaluation policy framework · the national evaluation policy framework: ... the code...

32
The National Evaluation Policy Framework: Engendering an Evaluation Culture in the Philippines 3 rd M&E Network Forum 06-08 November 2013 Asian Development Bank, Mandaluyong City Deputy Director-General Rolando G. Tungpalan National Economic and Development Authority (NEDA) 1

Upload: buituong

Post on 05-Jul-2018

235 views

Category:

Documents


0 download

TRANSCRIPT

The National Evaluation Policy Framework:Engendering an Evaluation Culture in the Philippines

3rd M&E Network Forum

06-08 November 2013

Asian Development Bank, Mandaluyong City

Deputy Director-General Rolando G. Tungpalan

National Economic and Development Authority (NEDA)

1

Presentation Outline

• Preliminaries

– Public Sector Management

– Policy Milestones

• Country Context

– Results Framework

– Evaluation Culture

– Institutional Mandates

• National Evaluation Policy

– Initial Activities

– Draft Policy2

Preliminaries

3

Public Sector Management (PSM)

Source: Asia Pacific Community of Practice on Managing for Development Results, Framework for Results-Based Public Sector Management. September 2011.

Features of results-oriented

PSM:

a. presence of core result

attributes

b. a focus on common

results

c. interdependency among

the components

d. effective vertical and

horizontal linkages

4

NATIONAL

DEVELOPMENT

GOALS

PLAN FOR RESULTS• Indicators with targets are

specified for each level of

results

BUDGET FOR RESULTS• Budget supports planned

national priorities

IMPLEMENT FOR

RESULTS• Policies, people,

processes oriented to

deliver intended results

MONITOR RESULTS• Defined indicators are used

to track achievement of

agreed results

EVALUATE RESULTS

• Evaluation

methodology enables

assessment of results

and informed decision-

making

1987

EO 230

Reorganizing

NEDA mandates

the agency to

monitor and

evaluate the

implementation

of the PDP

1989 1992 1993 1996 1999 2001

EO 376 established

the Regional

Project Monitoring

and Evaluation

System (RPMES)

for M&E at the sub-

national level

EO 93 further

refined and

streamlined the

roles and

responsibilities

and operating

procedures under

the RPMES

NB Resolution

No. 30 instructed

the ICC to review

all ongoing ODA-

funded projects

with the aim of

improving

absorptive

capacity

RA 8182 (as

amended by RA

8555) mandated

NEDA to conduct

an annual review of

status of all ODA

projects

NB Resolution No.

3 provides for

reporting of project

outcomes and

impacts by ICC and

Implementing

Agencies

Sector Effectiveness

and Efficiency Review

(joint exercise by DBM

and NEDA, through

the 2001 National

Budget Call) assessed

the responsiveness of

programs and

projects to sector

outcome objectives

2007

DBM adopted the

Organizational

Performance Indicator

Framework (OPIF) which

seeks to align goods

and services supported

by the budget with the

government’s desired

outcome objectives

2011

AO 25 provided the

mechanism for the

establishment of a

unified and

integrated Results-

Based Performance

Management System

within the Executive

Branch of the

government.

5

Policy Context

Evaluation

• Definition

• Uses of Evaluation

a. for learning – to improve future projects and programs through

feedback of lessons learned

b. for accountability – disclosing information to stakeholders

Institution Definition

OECD

DAC

An assessment, as systematic and objective as possible, of an on-

going or completed project, program or policy, its design,

implementation and results.

UN

Evaluation Group

(UNEG)

An assessment, as systematic and impartial as possible, of an activity,

project, programme, strategy, policy, topic, theme, sector, operational

area, institutional performance, etc.

6

Country Context

7

GPH Results Framework

Sectoral Goals/Outcomes

Good Governance and Anti-Corruption

Human Development & Poverty Reduction

Economic Development

Security, Justice,and Peace

Climate change

Societal Goals/Outcomes

Organizational Outcomes

Major Final Outputs (Citizen-focused Service and Product Result )

Enab

lers

/ D

rive

rs

Financial Stewardship Internal Process Leadership, Learning, and Growth

OP

IF

Strategic Performance Management System (Individual)

EO 4

3/P

DP

(RM

)P

DP

(RM

)

Res

ult

s/O

utc

om

esO

rgan

izat

ion

al

Man

agem

en

t R

esu

lts

Co

mp

reh

ensi

ve P

erfo

rman

ce In

dic

ato

rs

8

Country Context: Evaluation Culture

Criteria Assessment

Evaluation takes place in many domains. Emerging

Supply of domestic evaluators in different fields. Present

National discourse concerning evaluation. Emerging

Presence of a profession with own societies. Emerging

Institutional arrangements in the government. Present

Institutional arrangements in legislative bodies. Emerging

Pluralism exists (institutions, evaluators) Present

Evaluation takes place within the audit institution. Emerging

Outcome evaluations (not output and process) Emerging

Note: Criteria adapted from Furubu, Rist, Sandahl, 2002, International Atlas of Evaluation.

Assessment of country-context done by the presentor. 9

Country Context: Evaluations

• Evaluations more pronounced at the program/project

level. Usually conducted by development partners

through external evaluators.

• Policy evaluations

• Sectoral evaluations

• Evaluation of Country Assistance Strategies

• Country Level Evaluation (e.g., PD, MDG)

• Development Plans (at the national and regional levels)

10

Country Context: Institutional Mandates

• M, E, M&E conducted at the Implementing Agencies

(IAs), Oversight Agency (OAs), and Inter-Agency

committees (IACs).

IAs OAs IACs

• Internal M&E

system

• Project

Implementation

Officers (PIO)

System

• NEDA (planning)

• DBM (budgeting)

• COA (auditing)

• Office of the

President

• Office of the

Cabinet Secretary

• Investment

Coordination

Committee

• Project Monitoring

Committees under

the RPMES*

11*Regional Project Monitoring and Evaluation System

National Evaluation Policy

Framework

12

National Evaluation Policy

Preliminary Activities by NEDA

• Initial review of existing international evaluation policies

Evaluation Policy

What is it? A policy framework to guide evaluation activities.

Why do we

need it?

Evaluation is a crucial function that informs

decision-making. A set of policy statements on it

will promote and strengthen the practice and use

of evaluations.

13

Source: R.G. Tungpalan, “National Evaluation Policy Framework,” presented at the 2nd M&E Network Forum, 07 November 2012.

National Evaluation Policy

Elements of an Evaluation Policy

14

•evaluation

concepts

•role of evaluation

•guiding principles

•evaluation

standards

•conduct of evaluation (programming,

planning, design, management)

•defined institutional responsibilities

•disclosure and dissemination

protocols

•periodic review of policy

Source: R.G. Tungpalan, “National Evaluation Policy Framework,” presented at the 2nd M&E Network Forum, 07 November 2012.

National Evaluation Policy

• Consultants – Donald Hall and Ruperto Alonzo

• Technical assistance from UNICEF

• Methodology:– Review of other country experiences

• Literature review

• Interviews

– Interview of stakeholders• Oversight agencies – NEDA, DBM, DOF, OP-PMS, COA

• Implementing agencies – DPWH, DILG, DAR, DA, DOH

• Other institutions – House of Representatives, Senate, academe

• Development partners – UNICEF, ADB, USAID, AusAID

15

Consultations on the Policy

(timeline)

16

(based on original 30 Aug 2013 draft)

• Project Implementation Officers (9 Sep 2013)

• National Project Monitoring Committee / RPMES* Forum

(24 Sep 2013)

• Investment Coordinating Committee of the NEDA Board

(04 Oct 2013)

• Public Consultation Forum (18 Oct 2013)

*Regional Project Monitoring and Evaluation System

Consultations on the Policy

(comments)

17

• Broad-based support from all stakeholders.

• Concise policy document (i.e., details spelled out in

separate guideline/s).

• Options to effect the policy (i.e., legislative act, Executive

Order, NEDA Board Resolution, Memorandum Circular).

• Conscious effort on data generation (baselines, etc.)

18

• Greater elaboration on the roles and responsibilities of

institutions and organizations (e.g. participation of

oversight agencies, role of sub-national oversight and

line agencies, and internal arrangements for evaluation

units of IAs)

• Provision of resources in support of the evaluation

function (for the conduct of evaluations and for capacity

building).

Consultations on the Policy

(comments)

National Evaluation Policy Framework*

19

Section Content

1 Intent Emphasizes government’s aim to achieve PDP

societal goals of inclusive growth and poverty

reduction.

Project/program planning and implementation

characterized by evidence-based decisions,

accountability and learning which, in turn, are

supported by systematic, rigorous and impartial

evaluation.

Evaluation policy intended to govern the practice

of evaluation in the country towards more effective

development.

*based on revised draft as of 31 October 2013

20

Section / Content Highlights

2 Objectives

States the objectives of

the policy in support of

evidence-based

project/program

planning and

implementation

through systematic,

rigorous and impartial

evaluations.

Spells out the objectives of the policy:

a) support for evidence-based decisions;

b) promotion of program improvement;

and,

c) promotion of accountability.

National Evaluation Policy Framework

*based on revised draft as of 31 October 2013

21

Section / Content Highlights

3 Coverage

States the

scope of the

Policy

Proposed policy covers all programs/projects

managed by departments and agencies.

Evaluations address, at the minimum the following

questions covering three issues:

a) effectiveness (on achievement of objectives

and on unintended results);

b) operational efficiency (on efficient delivery of

outputs and on operational alternatives);

c) relevance (on alignment with national

priorities, on uniqueness/ coordination, and

on programmatic alternatives).

National Evaluation Policy Framework

*based on revised draft as of 31 October 2013

22

Section / Content Highlights

4 Institutional

Responsibilities

Enumerates the

responsibilities of

implementing and

oversight agencies,

and of the Evaluation

Secretariat.

Implementing agency heads responsible for

the establishment within their agency of a

capable, neutral evaluation unit. The head of the

evaluation unit reports directly to the

implementing agency head.

All project/program proposal put forward by

implementing agencies for approval in the

course of the annual budgeting process or at

other times include an evaluation plan. Proposal

lacking an adequate evaluation plan are not

eligible for approval. Disputes with respect to

evaluation plan adequacy are referred to the

Evaluation Secretariat.

National Evaluation Policy Framework

*based on revised draft as of 31 October 2013

23

Section / Content Highlights

4 Institutional

Responsibilities

Enumerates the

responsibilities of

implementing and

oversight agencies,

and of the Evaluation

Secretariat.

Implementing agencies to ensure that

evaluations are undertaken with due regard for

impartiality and in line with evaluation best

practices.

Implementing agencies to maintain a rolling 5-

year evaluation agenda listing projects/programs

to be evaluated during the current year and the

subsequent 4 years.

Protocols on Reporting, Dissemination and

Use of Evaluations prescribed.

National Evaluation Policy Framework

*based on revised draft as of 31 October 2013

24

Section / Content Highlights

4 Institutional

Responsibilities

Enumerates the

responsibilities of

implementing and

oversight agencies,

and of the Evaluation

Secretariat.

Establishment of a Government of the

Philippines Evaluation Secretariat at NEDA.

Quality Assurance

- Review and assess quality of IAs

evaluation reports and provide feedback to

agencies concerned

- Submit annually to the NEDA Board a

summary of GPH evaluation activities

including an assessment of the overall

quality and comprehensiveness of IA

evaluation

National Evaluation Policy Framework

*based on revised draft as of 31 October 2013

25

Section / Content Highlights

4 Institutional

Responsibilities

Enumerates the

responsibilities of

implementing and

oversight agencies,

and of the Evaluation

Secretariat.

Establishment of a Government of the

Philippines Evaluation Secretariat at NEDA.

Technical Support

- Provide advice and support to IAs in: (a)

preparing evaluation plans; (b) preparing

rolling 5-year evaluation plans; and, (c)

guidance in the conduct/management of

evaluation.

National Evaluation Policy Framework

*based on revised draft as of 31 October 2013

26

Section / Content Highlights

5 Implementation

Tackles the capacity

and financial

considerations in

implementing the

Policy. Also contains

a process for the

review of the Policy.

Department and agencies to allocate in their

annual budgets adequate resources (e.g.,

including for capacity building, external

professional service fees, etc.) to implement

policy provisions. Evaluation Secretariat

adequately supported by resources.

National Evaluation Policy Framework

*based on revised draft as of 31 October 2013

27

Section / Content Highlights

5 Implementation

Tackles the capacity

and financial

considerations in

implementing the

Policy. Also contains

a process for the

review of the Policy.

Evaluation and Amendment

-A formative evaluation of the Policy shall be

done within the second year of

implementation; the findings shall inform

amendments to the Policy as warranted.

-A summative evaluation of the Policy shall

be completed within five years following the

effective date of the policy

National Evaluation Policy Framework

*based on revised draft as of 31 October 2013

28

Section / Content Highlights

6 Ethics

Contains the

standard of ethics to

be observed in

undertaking

evaluation

All those engaged in designing, conducting

and managing evaluation activities shall abide by

the Code of Conduct and Ethical Standards for

Public Officials and Employees (RA 6713).

Evaluators shall respect the right of

implementing agencies and individuals to

provide information in confidence, and ensure

that sensitive data cannot be traced to its source.

Evaluators must ensure that those involved in

evaluations have the opportunity to review and

approve the statements attributed to them.

National Evaluation Policy Framework

*based on revised draft as of 31 October 2013

29

Section / Content Highlights

6 Ethics

Contains the

standard of ethics to

be observed in

undertaking

evaluation

Evaluators shall be sensitive to the cultural,

social and economic environment of all

stakeholders, and conduct themselves in a

manner that is fair and appropriate to this

environment.

Evaluators shall be accountable for their

performance and their products.

National Evaluation Policy Framework

*based on revised draft as of 31 October 2013

30

Section Content/Highlights

Annex A:

Historical Context

Describes the government’s reform initiatives

over time related to performance evaluation,

results orientation and monitoring and evaluation.

Annex B:

Impartiality

Ensure impartiality in order to maximize

objectivity and minimize potential for bias.

Annex C:

Best Practices in

Evaluation

Lays down best practices in evaluation in terms

of (a) evaluation scale, and (b) evaluation design

and execution.

Annex D:

Evaluation

Competencies

Those engaged in designing, conducting and

managing evaluation should demonstrate

competencies on: (a) technical foundations; (b)

leading, managing and delivering evaluations; (c)

communicating and sharing evaluation findings;

and (d) integrity.

National Evaluation Policy Framework

*based on revised draft as of 31 October 2013

Our Take on the Policy

31

• There is a need to step up on capacity

development

• Responsiveness of the statistical system should

be enhanced to support the results framework

• Policy will support organization learning

• Policy enhances public sector management

cycle integration

The National Evaluation Policy Framework:Engendering an Evaluation Culture in the Philippines

3rd M&E Network Forum

06-08 November 2013

Asian Development Bank, Mandaluyong City

Deputy Director-General Rolando G. Tungpalan

National Economic and Development Authority (NEDA)

32