revised srep results framework...8 table 1: results framework – scaling up renewable energy...
TRANSCRIPT
April 3, 2012
REVISED SREP RESULTS FRAMEWORK
2
BASIC PRINCIPLES
1. The application of the SREP results framework (in common with all the results
frameworks under the Climate Investment Funds) is based on the following principles:
a) Living document – The revised SREP results framework is a living document to
serve as a basis for moving forward in developing M&E systems for SREP
investment plans and related projects and programs.
b) Field testing – The logic model and results framework comprise a set of
assumptions which need to be further tested in light of on the ground experience
in the pilot countries. MDBs will need to report progress in field testing to the CIF
Administrative Unit on an annual basis. Further revisions of the logic model and
the results framework might be needed in light of the experience gained.
c) National monitoring and evaluations (M&E) systems – The results framework
is designed to operate: (i) within existing national monitoring and evaluation
systems; and (ii) the MDBs’ own managing for development results (MfDR)
approach. The development of parallel structures or processes for SREP
monitoring and evaluation will be avoided. National systems and capacities will
be taken into account when applying the results framework.
d) Flexible and pragmatic approach – The framework will be applied flexibly and
pragmatically taking into account pilot country circumstances. As noted above,
the proposed indicators need to be field tested. Country circumstances need to be
taken into account in selecting relevant indicators and subsequent reporting.
However, it is expected that pilot countries include at least 2 out of the 3 SREP
program outcome indicators in their investment plan results frameworks. The
results framework embraces the CIF principle of learning - a trial-and-error
learning approach is explicitly encouraged.
e) Data collection and reporting standards – In order to be able to aggregate
country-level results at the programmatic level (investment plan), a set of core
indicators1 will be measured using compatible methodologies. This is especially
true for indicators for the core objectives of the SREP: Reduced energy poverty
and increased energy security.
1 The suggested indicators in table 1 are core indicators. Results frameworks of specific projects can comprise many other
indicators but for the purpose of aggregation and comparison the proposed indicators are recommended for the national
M&E systems and the project/program results frameworks.
3
INTRODUCTION
2. In its meeting in November 2010, the joint Meeting of the CTF-SCF Trust Fund
Committees approved the logic model for the Scaling-Up Renewable Energy Program in
Low Income Countries (SREP) as a living document with the understanding that it would
be revised after field testing. The six pilot countries and the multilateral development
banks (MDB) have attempted to apply the approved results framework in developing
investment plans and project/program interventions, but significant difficulties have
emerged. Pilot countries and MDBs have expressed that the approved SREP results
framework is too ambitious and complex and would benefit from major simplification.
The key constraints are:
a) The results chain is unclear; in consequence pilot countries have
difficulties to develop their own results chains.
b) There are too many indicators across multiple levels, creating confusion
over objectives and raising the transaction cost.
c) Most of the indicators do not correspond to the data/statistics that
countries/MDBs collect through existing processes, making it very
difficult and costly to establish baselines.
d) Many indicators do not allow uniform application and aggregation across
all programs, hence making it impossible to report on overall results of
SREP.
3. In line with document CTF-SCF/TFC.7/4 Proposed Measures to Improve the
Operations of the Climate Investment Funds to enhance the performance of the CIF, the
CIF Administrative Unit and the MDBs are proposing a revised SREP logic model and
results framework to the SREP Sub-Committee.2 This proposal is based on (a) an
interpretation of the key SREP objectives; (b) an improved understanding of what is
possible as part of the development and implementation of a SREP investment plan; (c)
recently initiated work on improved energy indicators in support of the Sustainable
Energy for All initiative; and (d) consultations with the MDBs and recipient country
counterparts, including a discussion in the SREP pilot country meeting and SREP Sub-
Committee in March 2012 in Kenya.
4. The main purpose of the proposed results framework is to establish a basis for
future monitoring and evaluation of the impact, outcomes and outputs of SREP-funded
activities. In addition, the proposed results framework is designed to guide pilot
countries and MDBs in further developing their own results frameworks to ensure that
SREP-relevant results and indicators are integrated in their own monitoring and
evaluation (M&E) systems at the country or the project/program level.
2 See CIF. 2011. Proposed Measures to Improve the Operations of the Climate Investment Funds, paragraph 39.
4
5. Section 2 introduces the revised SREP logical model. Based on the logic model,
section 3 outlines the proposed SREP results frameworks with result statements and
indicators. The last section outlines briefly necessary changes in the project/program
documentation to reflect the simplified M&E approach.
THE REVISED SREP LOGIC MODEL
6. The logic model is a diagram intended to demonstrate the cause and effect chain
of results from inputs and activities through to project outputs, program outcomes, and
national/international impacts. The logic model is not intended to show how these results
will be measured through indicators. One of the strengths of the logic model is the
flexibility with which it can be applied to a variety of circumstances and contexts. As
with all results frameworks these logic models should not be seen as a blueprint for
implementation, but rather a framework that can be adjusted as progress is made and
lessons are learnt, especially at the project and country levels of the results chain.
7. The original SREP logic model was approved by the Joint Meeting of the CTF-
SCF Trust Fund Committees in November 2010. It is suggested to change the current
logic model to give greater focus to the key operational objectives of SREP. Other
objectives, if any and co-benefits are incorporated by explicitly stating the assumptions
and proxies underlying them, and would be incorporated in any ex-post evaluation of
SREP or individual country programs.
8. The stated impact objective for SREP is to support low carbon development
pathways by reducing energy poverty and/or increasing energy security. The proposed
outcome objectives for SREP are: i) increased access to clean energy; ii) increased supply
of renewable energy; and iii) maximize leverage of new and additional resources for
renewable energy projects/programs. Because funding to SREP is classified as ‘climate
finance’ by many CIF contributors3, it is proposed that the SREP results framework also
include a measure of the GHG emissions co-benefits associated with an increased supply
of RE at the outcome level. Due to the complexity of this issue, and the link to other CIF
programs, the Sub-Committee will allow further time for consideration of the metric(s) to
be used.
9. SREP will contribute to these results through programs and projects that build
infrastructure, develop capacity, and provide financing. Investments in renewable energy
(RE) infrastructure will increase the supply of electricity and heat from low carbon
sources, thereby supporting low carbon development and increased energy security. It is
assumed that programs/projects will, over time, also help improve the reliability and
economic viability of renewable energy provision at the country level when compared to
conventional energy sources. The outputs in the project/program section are provided as
examples of potential investment areas. Investment plans submitted by the SREP pilot
countries will have to articulate explicitly the expected results chain for envisaged
3 See CIF 2010. SREP Programming Modalities and Operational Guidelines, paragraphs 20-23.
5
projects/programs. A key supporting factor will be the adoption and implementation of
low carbon development plans and/or the enactment of policies, laws and regulations for
the promotion of RE.
6
Figure 1: Logic model – Scaling Up Renewable Energy Program in Low Income Countries (SREP) – REVISED
7
SREP RESULTS FRAMEWORK
10. The following tables contain the expected results flowing from the logic models
and the indicators that are proposed to measure them.
11. The results framework in table 1 summarizes the core elements of the
performance measurement system. It combines the results statements with the indicators.
The first two columns represent the results statements as stated in the logic model. The
results framework outlines the SREP Transformative Impact and the SREP Program
Outcomes. The transformative impact cannot be achieved only by SREP interventions. It
requires a truly national effort to move into a low carbon development pathway by
reducing energy poverty and/or increasing energy access. SREP is an important part of
this bigger change agenda in the SREP pilot countries. However, it is expected that SREP
projects/programs contribute directly to the SREP outcomes: (i) increasing access to
clean energy; (ii) increasing supply of renewable energy (RE); and mobilizing new &
additional resources for renewable energy projects/programs. The framework does not
include project/program outputs, activities, products and services because these are
specific to each project/program. Such an approach emphasizes also the commitment to a
managing for development results (MfDR) approach with emphasis on impact and
outcomes and the requirement to work within the MDBs’ own project/program
management approach.
12. The columns three to six represent the indicators for each result. The
performance indicators together with the baseline and target column are what the
program will use to measure expected results. The targets and baseline are currently
available only for a limited number of indicators. The pilot countries and the MDBs have
to cooperate closely to fill the gaps. Some of these indicators have very different time
frames. Baselines might only be established in the medium-term (1-2 years) and a true
impact reporting is probably not possible for a significant time span (10-15 years). The
sixth column summarizes some assumptions related to the reliability or validity of the
indicators and the difficulties operations might face when addressing these. The last
column briefly outlines the means of verification or data source.
8
Table 1: Results Framework – Scaling Up Renewable Energy Program in Low Income Countries (SREP) - REVISED
Results Explanation of the
result statement
Indicators Baseline Targets Assumptions Means of
verification
SREP Transformative Impact (based on governments long term targets for the sector)
Support low
carbon
development
pathways by
reducing
energy
poverty
and/or
increasing
energy
security
The highest result level
desired by SREP is the
transformation of the
way energy is produced
and distributed/
accessed.
Increased production of
renewable energy (RE)
in low income
countries is expected to
improve energy
security. Although
there are different
definitions of energy
security, an increase in
domestic supply of RE
is generally accepted to
increase a country’s
energy security.
Programs and projects
will focus on providing
access to energy to
poor households.
National measure
of ‘energy
poverty’ such as
the Multi-
dimensional
Energy Poverty
Index (MEPI), or
some equivalent
mutually agreed
measure
MEPI
score
where
available;
where this
does not
yet exist,
work will
be carried
out to
obtain a
score.
Country defined
according to
high level
energy/
development
strategy within
the SREP
implementation
timeframe
The Energy Sector
Management
Assistant Program
(ESMAP) is working
closely with the
International Energy
Agency (IEA) and
UNIDO to improve
the indicators used to
measure energy
poverty at the impact
level. This will be an
iterative process and
the results will be
incorporated into the
SREP results
framework as and
when international
consensus emerges.
Country-based
reporting
using
household
survey data –
(pilot
countries
supported by
the MDBs)
Annual electricity
output from RE in
GWh
Current
electricity
output
from RE in
each pilot
country
Country defined
according to
high level
energy/
development
strategy
Because this indicator
does not take account
of the current status
of energy supply, it
puts the emphasis on
actions taken from
National
statistics
agency or
energy
ministry
9
Results Explanation of the
result statement
Indicators Baseline Targets Assumptions Means of
verification
the present onwards.
SREP Program Outcomes
1.Increased
supply of
renewable
energy
In order to achieve the
transformation to
increased energy
supply and demand
based on RE the
economic viability of
the RE sector will need
to increase. This
means that the sector
will need to grow in
size and provide the
benefit of increased
employment.
Annual electricity
output from RE
as a result of
SREP
interventions
(GWh)4
Current
annual
electricity
output
from RE
(GWh)
Country-
defined
according to
investment plan
It should be possible
to undertake basic
aggregation of GWh
produced across pilot
countries.
National
M&E system
and M&E
framework of
the
implementing
agency
4 It is assumed that there will be GHG emissions co-benefits from increased output from RE. This indicator is primarily focused on grid-connected RE systems.
However, it can also include the electricity generation avoided by demand-side technologies such as solar water heaters. It can include as well mini-grid or off-grid
electricity generation as long as data are readily available.
10
Results Explanation of the
result statement
Indicators Baseline Targets Assumptions Means of
verification
2.Increased
access to
modern
energy
services
SREP aims to improve
access to modern
energy services in two
ways: i) by providing
improved access to
lighting, clean cooking
and other energy
services; ii) by
increasing the supply of
renewable energy to
communities that
already have access,
thereby improving the
quality of access.5
Number of
women and men,
community
services and
businesses
benefiting from
improved access
to electricity and
fuels as a result of
SREP
interventions
Zero Country-
defined
according to
investment plan
Specific energy
access indicators will
be developed
building on the
ongoing work by
ESMAP, leading a
collaborative effort to
define and
operationalize a set of
improved energy
access indicators at
the outcome level
that can be used for
project/program
reporting by
governments and
development
agencies. Such
indicators will seek to
capture disaggregated
data in terms of (i)
electricity / fuels; and
(ii) households /
community services
National
M&E system
and M&E
framework of
the
implementing
agency
5 To be able to claim energy access benefits from increasing centralized RE supply (i.e. grid-supplied electricity) there would need to be a clear demonstration of causality.
11
Results Explanation of the
result statement
Indicators Baseline Targets Assumptions Means of
verification
and businesses. They
will also enable
capturing information
about the
differentiated impact
of energy access on
men and women
The organizations
directly involved in
this work include
GIZ, Practical
Action, UNDP and
the World Bank.
3.New and
additional
resources for
renewable
energy
projects
The SREP will involve
the leveraging of new
and additional
resources for clean
production and
consumption
technologies. This will
occur in the context of
projects where multiple
sources of funding will
be leveraged by SREP
for particular
investments.
Leverage factor:
SREP funding
compared to $
financing from
other sources
(contributions
from MDBs,
governments,
multilaterals and
bilaterals, CSOs,
private sector)
N/a Country-
defined
according to
investment plan
Measurement of
leveraged resources
will be routinely
undertaken and
aggregated across
projects and
countries.
Numbers will be
disaggregated to
indicate
private/commercial
financing.
National
M&E system
and M&E
framework of
the
implementing
agency
12
CONCLUSION
13. The revised results framework is submitted to the SREP Sub-Committee for endorsement
with the understanding that the results framework needs to be flexible to allow for adjustments
based on actual SREP program implementation experience. The revised results framework is
based on the first-hand experiences of the pilot countries and the MDBs in implementing the
original SREP results framework. The investment plan development process in Honduras,
Kenya, Mali and Nepal generated a significant debate about the complexity of the approved
SREP results framework. A preliminary analysis across the investment plans revealed that most
pilot countries do not have the capacity to establish a complex M&E system, which would have
been required under the original results framework. Hence, this proposal was developed with
MDB and pilot country input to simplify the SREP results framework before countries get too
advanced in project/program preparation.
14. The revised SREP results framework reduces the number of indicators from 22 to five.
These five indicators cover two M&E levels – transformative impact (two indicators) and SREP
program outcomes (three indicators). The indicators cover energy, environmental and
development considerations to reflect the expected transformation process in SREP countries.
Although there would be fewer indicators, it will still be necessary to test the practicality of the
results framework, particularly linking projects/programs with higher level country objectives.
15. As project level output/intermediate indicators are specific to each project/program, and
the priorities of each country that this represents, it is proposed that they are not specified by the
SREP results framework. However, project/program documentation will demonstrate how the
output indicators that are selected will help achieve outcomes at the SREP program (country)
level. Each program will be expected to contribute to at least two of the three SREP program
outcomes. It will be either RE and/or access to energy and leveraging of new and additional
resources..
16. It is recommended that project/program documentation explains how the project/program
will contribute to achieving co-benefits at the transformative impact level. For example:
a) GHG emissions co-benefits: GHG emissions are closely related to economic
development and energy provision. It is expected that SREP investments will help
developing countries to continue to grow but at the same time avoiding the GHG
emissions typically associated with economic development– decoupling growth and
fossil fuel use.
b) Health co-benefits: Improved health of women, men and children is also a likely co-
benefit of RE investments, particularly for projects/programs targeting household
cooking access. RE is also generally characterized by decreased air pollution in the form
of particulate emissions when compared to traditional biomass and fossil fuels, resulting
in fewer respiratory health problems, especially for poor women, men and children.
c) Employment co-benefits: It is expected that RE investments will also have some direct
employment co-benefits, both temporary and long-term jobs.
13
17. Co-benefits are also expected at the outcome level:
a) Reliability co-benefits: Increased output from renewable energy is expected to improve
the overall provision and diversification of energy at the country level compared to the
current situation, thereby improving reliability.
b) Economic viability co-benefits: Economies of scale are expected over time in SREP
countries which will contribute to RE cost reductions. However, for achieving economic
viability, it is key to strengthen the enabling environment for renewable energy
production and use. Transformed energy supply and demand to more RE will require an
improved policy and regulatory framework. This will require reforms to be carried out
promoting clean production and consumption technologies and creating a level playing
field for local private sector and small scale renewable energy schemes.
18. It is suggested that project/programs outline in the project/program documentation how
the project/program might trigger positive development benefits beyond the immediate project
outputs. It is expected that key or underlying assumptions about co-benefits are clearly
articulated in the project documents so that ex-post evaluations can assess the effectiveness of
supported interventions. It is also expected that a gender impact indicator will be developed for
each project/program.
19. Pilot countries and MDBs should report back in 12 months after the approval of the
revised SREP results framework on: (i) how the results framework has been integrated in
national M&E systems; and (ii) how individual project/program interventions will be linked with
SREP program outcomes at the country level.
20. For any investment plan that has been endorsed prior to approval of the revised results
framework, the country and the MDBs are requested to review the results framework initially
submitted with the investment plan and to make any revisions that are necessary to align the
plan's results framework with the revised SREP results framework. The country should inform
the SREP Sub-Committee of any revisions that are made.
21. Progress reports, including reporting against the proposed indicators, will be provided to
the SREP Sub-Committee annually.