Comments on the Proposed Thematic Project on Evaluation

and Impact Assessment (recommendations 33,38,41)

Viviana Muñoz Tellez

Programme Officer, South Centre

WIPO Open-Ended Forum on Proposed Development Agenda Projects

October 14 2009, Geneva

For the Effective Monitoring, Assessment and Evaluation of Development Activities, DA

recommendations, and Impact of WIPO Activities on Development:

Project is highly useful, but additional mechanisms and measures will be

necessary

1) Requires involvement of many WIPO bodies

2) Scope of Thematic Project is inevitably limited

Guidelines for Thematic Project Approach: CDIP 3 Chair’s Summary

“ the Committee agreed to proceed on the basis of the following guidelines:

(i) each recommendation would be discussed first in order to agree on the activities for implementation;

(ii) recommendations that dealt with similar or identical activities would be brought under one theme, where possible; and

(iii) implementation would be structured in the form of projects and other activities, as appropriate, with the understanding that additional activities may be proposed.”

• CDIP pending discussion of individual recommendations on:

– Cluster D: assessment, evaluation and impact studies and

– Cluster E: Institutional Matters including Mandate and Governance, among others

• CDIP pending discussion and agreement on mechanisms and modalities for the monitoring, assessment, discussion and reporting on the implementation all recommendations, and for coordination with other WIPO bodies

Theme 5 Project in Perspective

Effective Monitoring, Assessment and Evaluation of

1) WIPO Development-related Activities 2) DA recommendations, and

3) Impact of WIPO Activities on Development

requires the involvement and review of the work of many WIPO bodies

WIPO BODIES INVOLVED IN DA MONITORING AND EVALUATION

1) General Assembly: 2007 called upon all the Member States, the Secretariat and other relevant WIPO bodies to ensure the immediate and effective implementation of these proposals.

2) Committee on Development and Intellectual Property (CDIP): Mandate to “b) monitor, assess, discuss, and report on the implementation of all recommendations, and for that purpose it shall coordinate with other WIPO bodies”

3) WIPO norm-setting and other substantive bodies: self-assessment and evaluation

4) Program and Budget Committee: Key Member States oversight role in design and follow up of biennial Program and Budget and Medium-Term Strategic Framework. Setting right benchmarks and indicators for self-assessment, resource management.

5)WIPO Secretariat: Undertake activities as requested / approved by Member States. Self-assessment and self-evaluation.

6) WIPO Oversight Bodies and Mechanisms

1)Internal Oversight:

– Internal Audit and Oversight Division: Reports to DG, independent operation

– WIPO Internal Audit Charter 2007: Internal audit, investigation & inspection

– Results-Based Management Framework (RBM)– WIPO Evaluation Policy: guiding principles for all WIPO oversight

functions, to be revised in 2009:• Self-evaluations• Independent evaluations: Evaluation plans and Annual Evaluation

Reports– WIPO Evaluation Unit: as part of internal oversight

6) WIPO Oversight Bodies and Mechanisms

2) External Oversight:

– Audit Committee: Independent, advisory, external. TORs 2007: Reviews/oversees financial, management, and major activities/projects

– External Auditor – UN Joint Inspection Unit (JIU)

7) UN mechanisms, norms and standards of evaluation:

– United Nations Evaluation Group (UNEG): WIPO is member– Norms and Standards for Evaluation in the UN system 2005

A number of Development Agenda recommendations relate to the Monitoring, Assessment and Evaluation of

1) WIPO Development-related Activities 2) DA recommendations, and

3) Impact of WIPO Activities on Development

including…

33. Develop an effective yearly review and evaluation mechanism for the assessment of all WIPO development-oriented activities, including those related to TA, establishing for that purpose specific indicators and benchmarks, where appropriate.

may include :

- Implementation of all WIPO DA recommendations- Norm-setting activities- Technical Assistance- Substantive programs - Studies

1) OVERALL ASSESSMENT AND EVALUATION

2) ASSESSMENT AND EVALUATION OF TECHNICAL ASSISTANCE

1. …Design, delivery mechanisms and evaluation processes of technical assistance programs shall be country specific.

41. Conduct a review of current WIPO technical assistance activities in the area of cooperation and development.

38. Strengthen WIPO’s capacity to perform objective assessments of the impact of the organization’s activities on development.

3) DEVELOPMENT IMPACT ASSESSMENTS

34. Conduct a study on constraints to IP protection in the informal economy, including the tangible costs and benefits of IP protection in particular in relation to the generation of employment.

35. To undertake, upon request of Member States, new studies to assess the economic, social and cultural impact of the use of IP systems in these States.

37. Upon request and as directed by Member States, WIPO may conduct studies on the protection of IP, to identify the possible links and impacts between IP and development.

3) STUDIES ON THE IMPACT OF IP ON DEVELOPMENT

40. To intensify WIPO cooperation on IP related issues with UN agencies, according to Member States’ orientation, in particular UNCTAD, UNEP, WHO, UNIDO, UNESCO and other relevant international organizations, especially WTO in order to strengthen the coordination for maximum efficiency in undertaking development

programs.

5) STRENGHENING COOPERATION AND COORDINATION WITH OTHER UN AGENCIES

A Thematic Project Approach (Theme 5)can contribute,

but to a limited extent, to the Effective Monitoring, Assessment and Evaluation

of

1) WIPO Development-related Activities 2) DA recommendations, and

3) Impact of WIPO Activities on Development

Selected Objectives

To implement recommendations 33 and 41:• Adapt and strengthen the WIPO RBM framework focused on

WIPO development-related activities and DA recommendations, including technical assistance.

To implement recommendation 33:• Contribute to a monitoring and evaluation culture, in particular

as it relates to development, within WIPO, and the systematic collection and use of performance information to ensure accountability, and support informed decision making by WIPO and its stakeholders, in particular in respect to development-oriented activities.

To implement recommendation 38:3) Increase the capacity and ability for independent and

objective evaluations of the development impact of WIPO’s activities.

Expected Project Outputs and Indicators for Self-evaluation

1) Improve, Adapt and Strengthen existing RBM framework

Outcome Indicators:- Put in place coherent framework- Finalize review of development-oriented activities- No. of staff in key management positions trained in RBM- The framework is used by all programs on a regular basis to monitor their developmentoriented activities and to report to their various stakeholders including CDIP- The framework facilitates the undertaking of evaluations, specifically those focused ondevelopment agenda activities.

Right project focus on integrating the Development Agenda recommendations anddevelopment-oriented activities to the RBM and Evaluation framework, to improve theability for assessment.

Improve design of the Program and Budget, Program Reports and other Reports relatedWIPO development activities and implementation of DA recommendations. I.e.

- Specify links between DA recommendations and program (Annex IX P&B 2009/10)- Improve expected results, program indicators and performance data

Independent evaluations are outside the scope of the project. Separate project required.What evaluation methodologies to be used? how to build capacity for independentevaluation (IAOD and Audit Committee)?

Expected Project Outputs and Indicators for Self-evaluation

2) Develop monitoring and evaluation mechanisms for development related activities and contribute to a monitoring and evaluation culture.. and the systematic collection and use of performance information, to ensure accountability and support decision making by WIPO and its stakeholders in respect to development-oriented activities, Program Performance Reports and other development and Development Agenda specific reports

Outcome Indicators:- No. of stakeholders that have participated in the awareness raising activities ?- Information used not only for accountability purposes but also coordination activities,

lessons learned and knowledge-sharing purposes- Framework is used for decision-making purposes

How to ensure that the performance information collected and used in respect to WIPOdevelopment-activities and implementation of development agenda recommendations isadequate and sufficient?

What methodology will be employed to define the baselines for ensuring that national IPstrategies and innovation strategies are developed in accordance to national needs anddevelopment priorities?

Expected Project Outputs and Indicators for Self-evaluation

3) Review of WIPO Technical assistance work in the area of cooperation for Development

Outcome Indicators ???

What is the current WIPO process for planning, monitoring and evaluating its technical assistance programs and activities?

What is the new element this review (s) will introduce? periodicity?

What is the methodology to monitor and evaluate in a reliable and unbiased manner,whether WIPO technical assistance programs and activities:

- are undertaken at the request of and designed, formulated and implemented in close consultation and cooperation with the Member States concerned, and

- respond to Member States’ specific needs and development priorities

Methodology and process for conducting country level needs assessment? Methodology for the selection of indicators, collection of empirical information and involvement of stakeholders?

Collaboration with other organizations, i.e. WTO, in conducting the review(s)? Independent evaluation of WIPO technical assistance is outside of the scope of this project. Separate project for the independent evaluation of WIPO Development-oriented activities, including technical assistance, is required.

Expected Project Outputs and Indicators for Self-evaluation

4) Increase the capacity and ability for independent and objective

evaluations of the development impact of WIPO’s activities

Outcome indicators:- Capacity to monitor WIPO’s contribution to national IP and innovation strategies in place

- Framework to improve the quality of country-level evaluations is in place

- Ability to provide WIPO/IP contribution to development-impact assessment studies

- Increased demand from countries for these services.

Methodology for ex-ante (planning, design, articulation of intended results and monitoring) and ex-post impact assessment of WIPO activities on development?

How should WIPO coordinate and cooperate with other UN agencies to build its capacity to understand and undertake development impact assessments of its activities?

Key Missing Components

• Role of Member States and Member-State led WIPO bodies (CDIP, PBC) in monitoring the work of the Steering Committee composed of external experts, the DACD and Program Management and Performance Section. Process for consultations - monitoring.

• Role of Oversight Bodies. Key role of Independent Evaluation of the IAOD and of the Audit Committee. Additional project may be required to strengthen the capacity of IAOD and AC to undertake independent evaluation of Development Agenda implementation, WIPO development-oriented activities, including technical assistance and studies, and the impact of WIPO activities such as norm-setting, on development.

• Role of Independent Evaluation as distinct from monitoring and evaluation within the RBM framework.

Key Missing Components• Evaluation methodology to ensure credible, fair and unbiased assessment of :

– WIPO development-oriented activities. Identification of activities to be evaluated (i.e. country-level technical assistance, studies).

– WIPO - role in the development of national innovation and IP strategies, as the basis for WIPO development-oriented work at the country-levels.

– WIPO studies, TORs, including on the linkages and impact of IP on development and those specified in the WIPO Development Agenda recommendations.

– Impact assessment of WIPO activities on development. Include evaluation of norm-setting, ex-ante and ex-post, on development.

• Cooperation with UN agencies in building WIPO capacity to undertake impact assessments of WIPO activities on development.

• Technical assistance review: program level evaluations and independent evaluations. Baselines, information collection, stakeolders, time frame for review, and periodicity. Annual Member state – country level reviews.

Thank you!

Viviana Muñoz Tellez

munoz@southcentre.org