this project is co-financed by the european union and the republic of turkey

Monitoring & Evaluation, 12-16 Dec 2011

1

Bölgesel Rekabet Edebilirlik Operasyonel Programı’nın Uygulanması için Kurumsal Kapasitenin Oluşturulmasına Yönelik Teknik Yardım

Technical Assistance on Institutional Building for the Implementation of RCOP in Turkey

This project is co-financed by the European Union and the Republic of Turkey

MONITORING & EVALUATIONMONITORING & EVALUATIONfor RCOPfor RCOP

Ankara, 12 – 16 December 2011Ankara, 12 – 16 December 2011

Monitoring & Evaluation, 12-16 Dec 2011 2


SECTION I.MONITORING VS VALUATION

• Clarification of definitions



Distinction between evaluation, Distinction between evaluation, monitoring and auditmonitoring and audit

• Evaluation • Assessment of the efficiency, effectiveness, impact, relevance and sustainability of aid policies and actions

• Monitoring • Ongoing analysis of programme/project progress towards achieving planned results with the purpose of improving management decision making

• Audit • Assessment of the legality and regularity of project expenditure and income i.e.

compliance with laws and regulations and with applicable contractual rules and criteria;

whether project funds have been used efficiently and economically i.e. in accordance with sound financial management;

whether project funds have been used effectively i.e. for purposes intended.

Primarily a financial and financial management focus, with the focus of effectiveness being on project results.



Evaluation-monitoring-project Evaluation-monitoring-project managementmanagement

• Monitoring: to be aware of the present situation• Evaluation: to rationally connect the present situation with

the desired future situation• Project management: operations and transactions

(contracting, approval of reports, payments…) but activities leading to these rely on monitoring and to (interim) evaluation outcomes

Potential overlaps of monitoring and project management activities

Segregation of duties, clear definition of responsibilities Cooperation



Distinction between and Distinction between and evaluation monitoringevaluation monitoring

Monitoring•What? Monitoring is an integral part of a day-to-day management.•How? Monitoring embodies the regular tracking of inputs, activities, outputs, reach, outcomes, and impacts of development activities at the project program, sector and national levels•Why?Monitoring provides information by which management can identify and solve implementation problems and assess progress towards project's objectivesEvaluation•What? Evaluation is an assessment that refers to design, implementation and results of completed or on-going project / program / policy. •How? Evaluation should be systematic and objective. Key criteria to be used are: relevance, developmental efficiency, effectiveness, impact and sustainability.•Why?Evaluation should provide credible and useful information to enable the incorporation of lessons




Monitoring•focused on daily management issues•typical questions: “How many?” "When?” “How?” “For how much?” •assess whether activities are implemented effectively and efficientlyEvaluation •addresses strategic questions: “So what?”(impact and sustainability) and “Why?” (relevancy)•analysis is deeper and seeks for actual cause-results relationships•perception of “big picture"




Monitoring•usually means a system•data collected and analyzed more or less frequently•according to a predefine timetable (Performance Measurement Plan)•regularity and continuity of data collection•methodology used to analyze itEvaluation•flexibility in specifying, which aspects of the program should be assessed, when and how.




Monitoring•Part of modern project management•Generates useful information for project manager

• where are bottlenecks?• how are we doing towards our objectives?• are expenses under control?

•Utility is the primary featureEvaluation•Target groups: donors, planners, assistance recipients and wider public

• Have we achieved our goals? • Are our results sustainable?• Have we learned anything for the future?

•Focus on • transparency of evaluator’s approach • revealing cause-effect relationships between subsequent

layers of analysis.




Monitoring•Project management needs rapid assessment methods

• fast feedback• not very expensive

Evaluation•Scrupulous research methodologies

• representative surveys• comprehensive quantitative analyzes



M&E – CM&E – Comparative characteristicsomparative characteristics

Characteristics Evaluation Monitoring

Subject:usually focused on strategic aspects

addresses operational management issues

Character:incidental, flexible subject & methods

continuous, regular, systematic

Primary client:stakeholders and external audience

programme management

Approach: objectivity, transparency utility

Methodology:rigorous research methodologies, sophisticated tools

rapid appraisal methods

Primary focus:focus on relevancy, outcomes, impact and sustainability

focus on operational efficiency and effectiveness

Objectives:to check outcomes / impact, verify developmental hypothesis

to identify and resolve implementation problems

to document successes and lessons learned

to assess progress towards objectives



M&E system diagrammeM&E system diagramme

INTERVENTION

LOGIC SOURCES OF DATA M&E TOOLS FOLLOW-UP

OVERALLOBJECTIVES EXTERNAL REPORTS AND DATA

FINAL EVALUATION, SURVEY,STATISTICAL ANALYSIS

FINAL EVALUATION REPORTPROGRAMMING

BENEFICIARIES OPINIONMID-TERM EVALUATIONBENEFICIARIES SURVEY

MID-TERM EVAL. REPORTREVISING CONTRACTS

PROJECTPURPOSE

EXTERNAL EXPERTS' OPINIONSTRUCTURED INTERVIEW,

FOCUS GROUPSQUARTERLY MONITORING

REPORT

RESULTSQUANTITAIVE DATA ON

PROJECT PROGRESS PIPELINE ANALYSIS MANAGEMENT MEETINGS

IMPLEMENTERS' PERIODIC

REPORTS REVIEW OF PERIODIC REPORTS REPORT APPROVAL PROCESS

ACTIVITIES

PRODUCTS DELIVERED BY

CONTRACTORSREVIEW OF PRODUCTS

AND SERVICES PRODUCT REVIEW REPORT

MEANS &RESOURCES

DIRECT OBSERVATIONS SITE VISITS SITE VISIT REPORT



SECTION II.EVALUATION

• EU Regulations on evaluation• Introductory guide on basic notions



COUNCIL REGULATION COUNCIL REGULATION 1085/2006 establishing IPA1085/2006 establishing IPA

Article 22 Evaluation•Commission shall regularly evaluate

• the results and efficiency of policies and programmes

• effectiveness of programming whether the objectives have been met

•Objectives of Evaluation• to enable the COM to formulate recommendations• results shall feed back into programme design

and resource allocation



COM REGULATION 718/2007COM REGULATION 718/2007implementing 1085/2006implementing 1085/2006

• Title II Common rules for implementation/Chapter V Evaluation and monitoring

• Article 57 Evaluation• Objectives: to improve the quality of

effectiveness and consistency of the assistance strategy and implementation of the programmes.

• Subject: Multi-annual indicative planning documents (ex ante evaluation)

• Forms: Strategic Ex ante Interim Ex post




Article 57 Evaluation (cont’d)•Interim:

At least one during implementation Specifically when the monitoring reveals significant

departure from the goals initially set.•Ex post

the responsibility of the Commission Shall include identifiable IPA component-specific results

•The results of ex ante and interim evaluation shall be taken into account in the programming




CHAPTER III/Implementation/Section 1 Framework for implementation and principlesArticle 78 Implementation principles in the event of participation in Community programmes and agencies (as amended by COM REGULATION 80/2010)•All evaluations shall be carried out by the COM prior to the conferral of management powers on the beneficiary country, •After the conferral

BC is responsible for interim evaluation; COM’s rights to perform any ad-hoc evaluations remains; ex-ante and ex-post remain with COM BC’s right to carry out ex-ante and ex-post as it deems

necessary.




Title III Regional development and human resources development/Chapter III Implementation/Section 3 Evaluation and monitoring Article 166 Evaluation•ex ante

• Beneficiary countries - under the responsibility of the operating structure.• For each operational programme separately. • If duly justified and in agreement with the COM BC may carry out a single ex ante

evaluation covering more than one operational programme.• Shall aim to

• optimise the allocation of budgetary resources under operational programmes• improve programming quality• identify and appraise

• the disparities, gaps and potential for development, • the goals to be achieved, • the results expected, • the quantified targets,• The coherence of the strategy proposed • and the quality of the procedures for implementation, monitoring,

evaluation• Annexed to OP




Article 166 Evaluation (cont’d)•Interim

• During the programming period• Beneficiary countries• Linked to the monitoring of OPs, in particular where

• this reveals a significant departure from the goals initially set or

• proposals are made for the revision of OPs• Results shall be sent to the sectoral monitoring committee for

the OP and to COM• Evaluations shall be carried out by experts or bodies, internal

or external, functionally independent of the authorities• The results shall be published



Purpose and principlesPurpose and principles

Logical elements of purpose of evaluation:•Assessment;•Systematic and objective;•Ongoing or completed project, programme or policy;•Design, implementation and results;•Determine the relevance and fulfillment of objectives, developmental efficiency, effectiveness, impact and sustainability;•Provide information that is credible and useful;•Enabling the incorporation of lessons learned into the decision-making process of both recipients and donors;



Purpose and principlesPurpose and principles

Principles of the approach to evaluation:•Impartiality and independence of the evaluation process from the programming and implementation functions;•Credibility of the evaluation, through use of appropriately skilled and independent experts and the transparency of the evaluation process, including wide dissemination of results;•Participation of stakeholders in the evaluation process, to ensure different perspectives and views are taken into account; •Usefulness of the evaluation findings and recommendations, through timely presentation of relevant, clear and concise information to decision makers.



Evaluation typesEvaluation types

Ex anteArticle 2 of the Financial Regulation•Use of budget appropriations

• principles of sound financial management• quantified objectives must be identified• „mobilisation of Community resources must be

preceded by an evaluation to ensure that the resultant benefits are in proportion to the resources applied.”




Ex ante•Implementation Rules for FR – ex ante evaluation shall identify:

• need to be met in the short or long term;• objectives to be realised;• results expected and the indicators needed to measure

them;• added value of Community involvement;• risks, including fraud;• lessons learned from similar experiences in the past;• volume of appropriations, human resources and

administrative expenditures;• monitoring system to be set up.




Ex ante•Process that supports the preparation of interventions - related to programming ~ quality assurance•Purpose: to gather and analyse information

• define objectives, • ensure that these objectives can be met,• that the instruments used are cost-effective, • reliable later evaluation will be possible

•Independent from planners




Interim•Assessment of the quality of programme implementation based on the first outputs and results

• relevance of the adopted strategy;• newly occured factors having an impact on the

implementation;• objectives have been defined accurately;• indicators are relevant;• so-far effectiveness and extrapolation;• management quality of the project implementation;• reliable collected data, including the monitoring system

•Independent from implementors




• Ex post• After programme completion• Examining long-lasting effects

• impact (more) visible• verification to what extent objectives have been

achieved • Sustainability – estimated character• Anticipated and unexpected effects• Source of information for future programming

excercises



Major tasks for an Evaluation UnitMajor tasks for an Evaluation Unit

• Identifying the need for an evaluation and selecting the topics/themes to be evaluated;

• Designing the evaluation, including preparing the Terms of Reference;

• Drafting tender documents for the evaluation study and selecting the contractor according to the established rules;

• Briefing the contractor and the parties involved, and supporting the evaluation mission;

• Ensuring the production of a high quality evaluation report and of the dissemination of evaluation findings and recommendations.



Tools and key documentsTools and key documents

Primary tools available to support a project evaluation:•Terms of reference for the evaluation mission;•The project’s Logframe matrix - to help assess what has been achieved against plan; •Programme planning documents; •Monitoring reports (internal and external); •IPA Interim Evaluation Reports;•IPA Monitoring reports•Minutes of SMCs and other organisations involved in monitoring



Key documents producedKey documents produced

• Terms of Reference for the evaluation mission,• Final Evaluation Mission Report

• should mirror the structure of the main evaluation criteria

• taking into account • the nature of the project, • the stage at which the evaluation is carried out, • the users for whom the report is prepared.



Evaluation CriteriaEvaluation Criteria

Relevance•@ significance; pertinence•To what extent do we do the right things ? Does it make sense?•~ to the identified problems or real needs to be addressed •The appropriateness of project objectives to

• problems that it was supposed to address• physical and policy environment within which it operated

•The extent to which the objectives of a programme/project are consistent with beneficiaries’ needs, country needs, global priorities.




• Relevance (cont’d)• Relates primarily to programme design

• extent to which correctly address the identified problems or real needs. (Needs to be kept under review throughout the life of the project in case changes occur)

• logic and completeness of the project planning process• internal logic and coherence of the project design• at two points in time: when the project was

designed, and at the time of the evaluation. • Focus

• identification of real problems or needs• correct beneficiaries




Efficiency•How well the various activities transformed the available resources into the intended results (outputs)

• quantity, quality and timeliness.• value-for-money: similar at lower cost in the same time

•Focus: the quality of day-to-day management, i.e:• budget• personnel, information, property, etc;• risk and change• relations/co-ordination with actors• time (deadlines)




Effectiveness•Contribution made by results to achievement of the Project Purpose•Benefits accruing to target groups•Did we achieve our objectives? To what extent did our outputs produce the desired outcomes?•By how far the intended beneficiaries really benefited from the products or services it made available?•Focus:

• whether the planned benefits have been delivered and received;

• appropriateness of the indicators of benefit used;• how unplanned results may have affected the benefits received;




Impact•Effect of the project on its wider environment, and its contribution to the wider policy or sector objectives (as summarised in the project’s Overall Objective).•Outcome: relationship between the project’s purpose and overall objectives•Benefits received by the target beneficiaries had a wider overall effect on larger group•Focus:

• to what extent the planned overall objectives have been achieved, • how far that was directly due to the project;• unplanned impacts’ effects on overall impact;

•Ex: higher standard of living, increased food security, democratic rule of law




Sustainability•Continuation of benefits, effects generated by a programme/project after its termination.•Likelihood of benefits produced to continue to flow after external funding has ended,•Focus:

• Ownership• Policy support and the responsibility of the beneficiary institutions• Institutional capacity, commitment• Economic and financial factors, socio-cultural aspects, gender equality• Technology, environmental aspects• Financial sustainability

•Ex: a micro-credit scheme that is generating enough money for the scheme to operate, cover risks and develop its staff



A Quick reflectionA Quick reflection

• Appropriateness is not about money. It is about what was achieved against what was needed.

• Effectiveness is not about money. It is about how much it was achieved.

• Efficiency is about how much was achieved and how much money was spent.

• Sustainability is not about money. It is about the embedment of what was achieved in the given context (natural, societal, economic…)



Decision optionsDecision options

Look out:•one may be very efficient but not effective!•or very effective but not relevant!•to do things right is fine, but to do the right things is for sure far more important!



Decision optionsDecision options

• Continue project implementation as planned• Re-orient/restructure the project• To stop the project (mid-term evaluation);• Modify the design of future projects or

programmes in light of lessons learned (ex-post evaluation)

• Modify policies, co-operation strategies, and subsequent programming or identification exercises (sector, thematic or cross-sector evaluations)



Performance RatingsPerformance Ratings

• Performance rating in two steps:1. Numerical rating on each criteria for each component of the sector

from –2 to +2:2. Conversion of numerical overall rating into a qualitative rating.

• Qualitative ratings• « Highly satisfactory » The programmes reviewed are expected

to achieve or exceed all the objectives set during their lifetime; most numerical ratings in the range of 1 to +2

• « Satisfactory » The programmes reviewed are expected to largely achieve the objectives; most numerical ratings in the range of 0 to 1

• « Unsatisfactory » The programmes reviewed are not expected to achieve most of the objectives; most numerical ratings in the range of –1 to 0

• « Highly unsatisfactory » The programmes reviewed are not expected to achieve any of the objectives; most numerical ratings in the range of -1 to –2



Quality Assessment of IE ReportsQuality Assessment of IE Reports

• Criteria – Rate – Remarks• A. General: Does the report design appropriately fit the

evaluation?• B. Sound sectoral overview: to what extent are the sector

composition and priorities appropriately described?• C1. Sound analysis: to what extent are the facts and data

adequately analysed?• C2. Sound analysis: to what extent have the indicators of

achievement been adequately considered and have they been used properly where was possible?

• D. Robust Findings in the implementation evaluation: do the Conclusions follow logically from, and are they justified by, the data described in the Sectoral Overview?




• E. Impartial conclusions: does the report provide value judgements based upon the five evaluation criteria of relevance, efficiency, effectiveness, sustainability and impact?

• F. Useful recommendations: to what extent do the Recommendations follow logically from the Conclusions? Are they operational? Do they clearly address the monitoring sector and are they targeted to the different stakeholders?

• G. The executive summary: to what extent is the executive summary a synthesis and does it meet the requirements set out in the template guidelines?

• H. Annexes: to what extent do the Annexes support the analysis in the main text?

• I. Overall style, structure and text design: within the template’s framework, to what extent is the text easily readable and accessible to the various categories of readers so that the main messages are easily detectable?




Verbal Unacceptable

Poor

Sufficient/ adequate

Good

Excellent

Single category

-2 -1 0 1 2

Entire report

<=-15 -14 to -6

-5 to 5 6 to 14

>=15Taking into account the contextual constraints on the evaluation, the overall quality rating of the report is considered to be:



Why Results Are IgnoredWhy Results Are Ignored??

• Implications may not be presented in a way that nonresearchers can understand.

• Results sometimes contradict deeply held beliefs.

• Vested interest in a program.



SECTION III.SECTION III.MONITORINGMONITORING

• EU Regulations on monitoring• Monitoring system and its evaluation



MONITORINGMONITORING

• @ follow up , controlling • A continuing observation • uses systematic collection of relevant and selected data • to provide management and the main stakeholders of a

programme/project with indications• of the extent of progress and achievement of

objectives• of process and impact.




Title II Common rules for implementation/Chapter V Evaluation and monitoring Article 58 Monitoring in the case of decentralised management (IPA Monitoring Committee)

•BC in agreement with NIPAC and COM•to ensure coherence and coordination in the implementation of the IPA components•meeting the objectives set out in the financing agreements and MIIFP•based on the elements given by the SMCs




IPA Monitoring Committee – proposals •to COM, NIPAC and NAO for:

• any actions to ensure the coherence and • crosscomponent corrective measures needed to

ensure the achievement•to SMCs for:

• decisions on any corrective measures to ensure the achievements of programme




IPA Monitoring Committee – organisation •Internal rules of procedure

• in compliance with a monitoring committee mandate established by the COM

• within the institutional, legal and financial framework of the BC

•Members: Commission, NIPAC, NAO, OS, strategic coordinator•COM and NIPAC co-chair•Meetings at least once a year•Intermediate meetings on a thematic basis.




Article 59 Sectoral monitoring committees in the case of decentralised management

•Assist IPA Monitoring Committee•Attached to programmes or components.•May include representatives of civil society,•Objective:

• effectiveness and quality of the implementation• in accordance with the related sectoral and/or financing

agreements.•Proposals

• to the COM and NIPAC, cc. NAO • for decisions on any corrective measures to ensure the

achievements of programme objectives




Sectoral monitoring committee - reports•to IPA monitoring committee on

• progress • by priority axis/ measures or operations;• results achieved, financial implementation

indicators• any aspects of the functioning of the management

and control systems raised by the audit authority, NAO or the competent accrediting officer




Article 61 Annual and final reports on implementation

•Sectoral annual report and sectoral final report by OS• Sectoral annual report: financial year• Sectoral final report: whole period of

implementation•SMC examination to be sent to NIPAC, NAO and COM




Annual and final reports on implementation•NIPAC sends annual and final reports based on synthesis of sectoral reports to COM and NAO•by 31 August each year•progress made in relation to the priorities of MIIFP and OPs•financial implementation of Community assistance.




Title III Regional development and human resources development/Chapter III Implementation/Section 3 Evaluation and monitoring

Article 167 Sectoral monitoring committee - Organisation •To be established by OS for each programme•single SMC may be set up for several programmes within the same component •meeting at least twice a year, at the initiative of BC or COM




Sectoral monitoring committee – Organisation (cont’d) •Internal rules of procedure

• in compliance with a monitoring committee mandate established by the COM

• within the institutional, legal and financial framework of the BC• In agreement with the OS and the IPA monitoring committee

•COM and Head of OS co-chair•composition to be decided by OS, in agreement with COM

• Members: COM, NIPAC, strategic coordinator for RD and HR components, OS

• Civil society and socio-economic partners where appropriate• European Investment Bank in advisory role where EIB makes a

contribution




Sectoral monitoring committee – Tasks •Consider/approve/revise criteria for selecting within six months of the entry into force of the financing agreement•review progress in OP targets based on OS reporting•examine (by reference to the indicators)

• results of implementation, achievement of the targets priority axis and measures

• interim evaluations• sectoral annual and final reports• relevant parts and recommendations of annual audit activity report• any proposal to amend the financing agreement

•propose to OS any revision or examination of the programme (incl. its financial management)




Article 169 Sectoral annual and final reports on implementation

•OS to COM and NIPAC after examination by SMC• sectoral annual report by 30 June each year• sectoral final report six months after the final date of eligibility of

expenditure•COM informs NIPAC and OS on admissibility of SAR within 10 working days from the date of its receipt – formal•COM informs NIPAC and OS on its opinion on admissible

• SAR report within two months from the date of receipt• SFR within a maximum of five months from the date of receipt




Sectoral annual and final reports - content•quantitative and qualitative progress in

• OP, priority axes, measures, operations• against quantified targets

•financial implementation of OP, priority axis, measure:• total expenditure paid out by the final beneficiaries included

in NF’s payment applications • total expenditure committed and paid out by NF

accompanied by computerised forms listing the operations, • total payments received from the COM

•breakdown of allocation by categories as listed in FA




Sectoral annual and final reports – content (cont’d)•steps taken by OS or SMC to ensure quality and effectiveness

• monitoring and evaluation measures, incl. data collection arrangements;

• summary of problems and subsequent measures;• use of technical assistance.

•information and publicity activities;•progress and financing of major projects.



Monitoring conceptsMonitoring concepts

Activity and/or result oriented monitoring?•Activity based monitoring

• identify activities/deliverables and deadlines for each• activities to be completed in a timely and appropriate

manner•Result based monitoring

• Results to be achieved•Analogy to project management and monitoring

• Activities must be completed – legal and financial consequences

• Results must be achieved – effectiveness aspect



ROM and Internal monitoring in EUROM and Internal monitoring in EU

ROM Internal Monitoring

Objectives To provide independent assessment of project performance, with focus on ‘results’

To provide advice and recommendations to project stakeholders

To generate aggregate data for reporting

To support effective and

timely decision making

by project managers

To promote accountability for resource use and

achievement of results

Responsible

COM Project implementing

partners/contractors



ROM and Internal monitoring in EUROM and Internal monitoring in EU

ROM Internal Monitoring

Method Short-visits to project sites by independent experts, on a periodic

basis

Analysis of project records and interviews with stakeholders

Standardised assessment

Project plans, ongoing

data collection, analysis

of data, progress reports

Consultation with

stakeholders

Participation in review meetings



Key principles for information flowKey principles for information flow

• Results information needs at the project, program, and policy levels results information must move both horizontally and vertically• Horizontal: what information is being collected by their own

organization and by other• Vertical: bureaucratical and hierarchical burdens

need to coordinate and collaborate in sharing performance information

• Responsibility at each level needs to be clear for• What data are collected (source)• When data are collected (frequency)• How data are collected (methodology)• Who collects data• Who reports data• For whom data are collected



Needs of Monitoring SystemNeeds of Monitoring System

Ownership

•Demand part : who needs the information?•Needs to be identified•Critical issue: when levels of need and data collecting /maintenance do not fit•Ownership vs enforcement

• Legal• Financial• Technical




Management

•Who, how, and where the system will be managed•Data collection overlaps, duplication of data•Time lags in receiving data•Data maintenance

• Responsible levels• Continuous development and trainings




Credibility

•Report all data—both good and bad• Deliberate non-reporting of information demonstrating failure• Political pressure

•Reliability: the extent to which the data collection approach is stable and consistent across time and space - measurement of the indicators is conducted the same way every time•Validity: indicators should measure actual and intended performance levels•Timeliness

• Frequency (how often are data collected?)• Currency (how recently have data been collected?)• Relevance (are data available frequently enough to support

management decisions?)



Design of a monitoring systemDesign of a monitoring system

1. Analyse project objectives to clarify project design

2. Review implementation procedures to determine information needs at the different levels of the project management structure

3. Review indicators4. Design reporting schemes5. Prepare an implementation plan for the

monitoring system



Analyse project objectivesAnalyse project objectives

• Time factor since original design: chaning project environment and actors

• Logframe approach• analysis of existing situation in light of future desired

situation• planning operational details• Logframe matrix

• A possible tool: project start-up workshop• all stakeholders• review project documents and key assumptions• revise project objectives



Review implementation proceduresReview implementation procedures

• Level of detail of information need and frequency of reporting vary according to the level of management

• Define information needs to the different levels of the management structure

• From project administrators to IPAMC• Workflow to be defined – procedures manual• Problem: users don’t know in advance what they need request

more information than needed users are not aware of what information is available

• Continual review of users requirements through:• planning and review meetings: what is lacking or redundant• systematic dialogue with users on content and format



Review indicatorsReview indicators

To avoid:•selection of too many indicators•selection of overly-complex indicators •over-concentration on progress indicators



Design reporting schemesDesign reporting schemes

• Reports• Based on project administrators’ needs on

daily/weekly/monthly basis• Based on the needs of the hierarchy• Incidental, Interim, Final as per contract

• Meetings• ‘monthly’, ‘Project Review/Implementation Meetings’

• EUD/NAC office vs CA, OBs• Informal, but methodology shall be determined in advance (all

the projects, only problematic ones)• Who represents the organisation?

• Steering Committees



SECTION IV.SECTION IV.MANAGEMENT INFORMATION MANAGEMENT INFORMATION

SYSTEMSYSTEM

• Main concepts and hints• An example: the Hungarian MIS



MISMIS

• All projects (service, supply, works, grant schemes): Similar characteristics

• Follow the same rules: PRAG rules and local• Efficient use of limited resources: No need to

rewrite again and again• Harmonization and Standardization of Monitoring:

Common to all



MISMIS

Objectives

•Projects achieved their objectives•Project implementations follow the rules•Project progressing as planned and performing well•Problems detected earlier

Functions• Follow-up of implementation• Management control tool• Support to beneficiaries• Immediate data availability for the hierarchy and

public



MISMIS

• Common communication platform and channel• Enforces obeying the rules• Provides monitoring system integrity• Creates standards for reporting and data entry• Verifies the declarations; Reports and Visits• Improve the quality of monitoring activities



MISMIS

Progress

•Ensuring project to produce expected outputs:• with planned activities• within the contract budget and duration

• Monitoring the Progress on:• Financial Progress• Technical Progress• Risk Assessment• Performance indicators



MISMIS

Characteristics of the system•Built with Participatory Approach•Comprehensive and decentralized•Open to all actors•Easily Accessible (Web-based)•Transparent System (Any data cannot be deleted or changed without permission)•Up-to-date data collection from the source•Different user levels according to responsibilities including project owners



MISMIS

Some points of consideration•Capacity issues

• Participatory approach during development and testing

• Database maintenance• Need to assign staff specific to MIS (not IT)

•Source code issue• For further upgrades• Public procurement consequences



EMIR – the Hungarian modelEMIR – the Hungarian model

www.emir.gov.hu

this project is co-financed by the european union and the republic of turkey

Documents

training monitoring

project results

project funds

project managementmonitoring

european union

republic of turkey distinction

republic of turkey section

financial management