MONITORING AND EVALUATION&

REPORTING

02 August 2016

PRESENTATION OUTLINE

Monitoring and Evaluation What is it? Monitoring and Evaluation Defined Monitoring and Evaluation: What & Why Project Development Setting of Objectives (1&2) Project Logic Model Program Theory Reporting: Does it matter? Reporting for decision making purposes (1&2) Theory of Change Our roles and responsibilities Conclusion (Theory of Bones) Questions

MONITORING AND EVALUATION: WHAT IS IT?

A sensible answer to this is…“so what”

It is about responsibility and accountability; providing our stakeholders (e.g. programme managers, funders, the public and any interested parties) with information to which extent we are achieving the set targets and goals

Provide substantial evidence on the achievements (build on, replication, best practice) / non-achievement (correction in approach or policy, programme or project)

MONITORING AND EVALUATION DEFINED

MONITORING, is continuous function that uses systematic collection of data on specified indicators to provide management and the main stakeholders of an ongoing development intervention with indications of the extent of progress and achievement of objectives and progress in the use of allocated funds

EVALUATION, is the systematic and objective assessment of an ongoing or completed project, program, or policy, including its design, implementation, and results. The aim is to determine the relevance and fulfilment of objectives, development efficiency, effectiveness, impact, and sustainability. Of key, info should be – credible, useful, enabling for decision-making of both recipients and donors (OECD in Jody Kusek & Ray Rist)

MONITORING AND EVALUATION: WHAT AND WHY?

WHAT? – PROGRAMME / PROJECTThe initiation of a project/programme happen because a

problem that has been identified or a policy decision by Government

WHY MONITOR? - PROGRESSCheck whether the project/programme is on track, identify gaps during implementation process, correct and report Ensure the achievement of objectives

PROJECT DEVELOPMENT

Project Title

Need for the Project (Background)

Problem Statement (Problem identified)

Mandate

Objectives (Aim of what the project should achieve)

Scope (Project coverage and beneficiaries)

Methodology (Process)

Milestones (Activities and timelines)

Risks and Mitigation

Output

PROJECTS: SETTING OF OBJECTIVES (1)

Ask the question “what is the problem?”

Objectives should answer your problem statement

What you want to achieve? (i.e. outcome)

What should the project/programme address?

Objectives should follow the SMART approach (simple, measurable, achievable,

realistic and time-bound)

Select key performance indicators (KPIs)

Design activities (Implementation Plan)

Set the standard indicators

Should take the CREAM approach o Clear (Precise and un-ambiguous)o Relevant (Appropriate to the subject at hand)o Available (available at a reasonable cost)o Adequate(Provide a sufficient basis to assess performance)o Monitorable (Amenable to independent validation)

Should have a direct link to the outcomes

PROJECTS: SETTING OF OBJECTIVES (2)

PROJECTS: PROGRAM LOGIC MODEL

This is planning or design tool to help you think through what it will take to

achieve the outcomes you have in mind.

This is a connection between the activities you develop and the outcomes

you hope to accomplish, it articulates the assumptions about the ability of

certain activities to drive particular changes.

PROJECTS: PROGRAM THEORY (Example Educator Workshop)

INPUT/RESOURCES ACTIVITIES OUTPUT OUTCOME IMPACT

Conduct educator training workshops (maths and science)

Number of training workshops

Number of educators trained

Improved ….. Effect of educator training workshops on learners – yield better results

Team:Funding:Sponsors:Tools:

The action: What is the level of intervention / assistance, the methods or areas

Tangible/quantity outputs: How many learners/teachers, the number of hours.

What is the reasonable change on the output or intervention in the short to medium term

What is the reasonable change on the output in the long term (change in behaviour)

REPORTING: DOES IT MATTER?

Report for compliance purposes according to legislative requirements (PFMA and Treasury Regulations), this include how information should be disclosed

Report to Programme Managers, sponsors/funders for decision making purposes

REPORTING FOR DECISION MAKING PURPOSES (1)

In consideration of the three EEE – (Accountability is with the AO of an institution to ensure adherence to the following in the utilisation of resources, the MD in our case – section 51 of the PFMA and any other official as a delegated authority by the AO at the level of their responsibility – section 57 of the PFMA)

ECONOMIC – Competitive Price (Price bargaining)

EFFICIENCY – Process (did we use the most efficient process to produce/deliver on our outputs/outcomes)

EFFECTIVENESS - Quality (did we produce the desired results within our means)

REPORTING FOR DECISION MAKING PURPOSES (2)

Utilisation of funds

What has been achieved? (outputs –quantitative data)

Is it according to the set objectives? (set standards)

Is it supported by evidence? (outcomes - qualitative data)

Highlights (what was outstanding)

REPORTING: WHAT IS EXPECTED

It is expected that the reports contain valid and project-specific information that is aligned to the project objectives.

It is expected that all the quantitative information in the reports can be validated i.e. your numbers should be supported substantive evidence e.g. attendance registers. This is audited.

It is expected that the qualitative information is relevant to your project. It should contain a good overview or ‘story’ of what happened during the reporting period.

It is expected that the templates be maintained, unless communicated differently. This is audited.

REPORTING: TYPES OF REPORTS (1)

NRF|SAASTA has several reports that is required throughout the year

1. THREE-YEAR ANNUAL PERFORMANCE PLAN (September) – this report is submitted to Corporate and formally published for the public and governmental institutions (DST and Treasury). Note: Once targets have been formally published, it cannot be changed for the succeeding year.

2. ONE-YEAR ANNUAL PERFORMANCE PLAN (March/April) – this report is submitted to the MD for project monitoring and performance management. Note: This report is an internal document.

3. QUARTERLY CONTRACT REPORTS (June/September/November/March) – this report contains the progress information for the contract projects. This report is submitted to Corporate and, then, to DST and the relevant project managers.

REPORTING: TYPES OF REPORTS (2)

4. QUARTERLY UNIT REPORTS (June/September/November/March) – this report is submitted to Monitoring and Evaluation for performance management and consolidation into the BUSINESS PROGRESS REPORT.

5. QUARTERLY PROGRESS REPORT (BUSINESS) (June/September/November/March) – this report compiled based on the unit and contract reports and contains all information for the NRF|SAASTA for the reporting period as aligned with the APP . This report is submitted to Corporate for consolidation into the NRF Performance Report.

6. SCIENCE ENGAGEMENT REPORT (June/September/November/March) – this is a new initiative by NRF. All facilities will now report all science engagement activities through SAASTA. More information on this will be communicated in due course.

REPORTING: PROCEDURES (LINE OF REPORTING)

NRF|SAASTA NRF DST TREASURY

• MANAGERS• PROJECT

COORDINATORS• SUPPORT

FUNCTIONS

• MANAGING DIRECTOR

• MONITORING AND EVALUATION

• NRF CORPORATE GOVERNANCE

REPORTING: PROCEDURES (DATES TO REMEMBERFOR THE 2017 FINANCIAL YEAR)

WHO WHAT TO DATE

Project Coordinators

Contract ReportsAnd

Unit Reports

Unit ManagersTo be decided by unit

ManagersMonitoring and Evaluation UnitManaging Director

Q2 - 19 September 2016Q3 - 18 November 2016Q4 - 20 March 2017

NRF|SAASTA Group ExecutiveQ2 - 29 September 2016Q3 - 30 November 2016Q4 - 05 April 2017

Group ExecutiveCorporate Governance

Q2 - 05 October 2016Q3 - 05 December 2016Q4 - 05 April 2017

THEORY OF CHANGE

What is the change we wanted to see?

Did we make that change? (if not, can we go back to our processes and establish

where the gap is and take a different approach)

Did the change answer our problem? (If not, are we able to say whether there is

need to still continue with the project or discard)

Are we able to show the “before” / “after”

Can we provide evidence of “was” / “now”

(e.g. if we have an outcome as, “educator outcomes in maths and science for

grade 12 improved”- indicator will be, “% of grade 12 learners scoring above 70%

or better in maths and science” baseline could have been 40-60%)

MONITORING AND EVALUATION

OUR ROLES AND RESPONSIBILITIES

CORE FUNCTIONS

SCIENCE AWARENESS

SCIENCE COMMUNICATION

SCIENCE EDUCATION

SUPPORT FUNCTIONS

HUMAN RESOURCES CORPORATE COMMUNICATIONS

FINANCE

SCIENCE ENGAGEMENT

CONCLUSION

PAUSE AND THINK ABOUT THE THEORY OF BONES

“WISHBONE” Those who spend time wishing somebody else would do the work

“JAWBONES” Those who do all the talking, but very little less

“KNUCKLEBONES” Those who are there to knock down everything else everyone tries to do

“BACKBONES” Those who are there to get under the load and do the work

Now, what kind of bone are you?Wilbert Hill

ANY QUESTIONS?

Thank youRe a lebogaRo livhuwa

Ndza nkhensaSiya bonga

EnkosiBaie dankie