activity monitoring and evaluation plan (amep)

ACTIVITY MONITORING AND

EVALUATION PLAN (AMEP)

FEBRUARY 18, 2016

This publication was produced for review by the United States Agency for International Development. It was prepared

by Tetra Tech ARD.

This publication was prepared for review by the United States Agency for International

Development under Contract # AID-497-TO-15-00005. It was produced by Tetra Tech ARD.

The period of this contract is from July 2015 to July 2020.

Implemented by: Tetra Tech P.O. Box 1397 Burlington, VT 05402

USAID LESTARI Activity Monitoring and Evaluation Plan P a g e | 1

ACTIVITY MONITORING

AND EVALUATION PLAN

(AMEP)

FEBRUARY 18, 2016

DISCLAIMER

This publication is made possible by the support of the American People through the United

States Agency for International Development (USAID). The contents of this publication are

the sole responsibility of Tetra Tech ARD and do not necessarily reflect the views of USAID

or the United States Government.


TABLE OF CONTENTS

Acronyms ............................................................................................................ 3

1. Introduction .................................................................................................... 7

1.1 Project Background ................................................................................................... 7

1.2. AMEP Development ................................................................................................. 9

2. LESTARI M&E Frameworks ......................................................................... 10

2.1 Situation Model ....................................................................................................... 10

2.2. LESTARI Results Framework and Theories of Change .......................................... 14

2.3 Logical Framework and Performance Indicator Tracking Table (PITT) .................... 25

3. Plan for Activity Monitoring and Evaluation .............................................. 44

3.1 LESTARI Learning Loop: Data Collection, Storage, Quality Assurance, Analysis, and Utilization ............................................................................................................... 44

3.2 Internal Evaluations Plan ......................................................................................... 51

3.3 Schedule for Performance Monitoring Tasks ........................................................... 55

3.4. Monitoring and evaluation structures, functions, and capabilities ........................... 56

Annex I: Performance Indicator Reference Sheets (PIRS) ........................... 59

Annex II: Results Chains ............................................................................... 118

Annex III: Data Quality Assessment Checklists .......................................... 126


ACRONYMS ADS Automated Directives System

AMEP Activity Monitoring and Evaluation Plan

APBD Anggaran Pendapatan dan Belanja Daerah (Local Budget Revenues and

Expenditures)

APL Area Penggunaan Lain (Other Uses Zones)

BAPPEDA Badan Perencanaan Pembangunan Daerah (Regional Development

Planning Agency)

BAPPENAS Badan Perencanaan Pembangunan Nasional (National Development

Planning Agency)

BAU Business as Usual

BIG Badan Informasi Geospasial (Geospatial Information Agency)

BMPs Best Management Practices

C4J Changes for Justice

CAs Conservation Areas

CCAs Community Conservation Agreements

CIFOR Center for International Forestry Research

CMMPs Conservation Management and Monitoring Plans

CMP Conservation Measures Partnership

CO2 Carbon Dioxide

CO2e CO2 equivalent

COP Chief of Party

CSOs Civil Society Organizations

DAK Dana Alokasi Khusus (Special Allocation Fund)

DCOP Deputy Chief of Party

DNPI Dewan Nasional Perubahan Iklim (National Council on Climate Change)

DQ Data Quality


DQAs Data Quality Assessments

FACTS Foreign Assistance Coordination and Tracking Systems

FMU Forest Management Unit (or KPH)

FY Fiscal Year

GAP Good Agricultural Practice

GHG Greenhouse Gas

GIS Geographic Information System

GOI Government of Indonesia

GPS Global Positioning System

Ha Hectare

HCS High Carbon Stock

HCV High Conservation Value

HO Home Office

ICCTF Indonesia Climate Change Trust Fund

ICRAF International Center for Research in Agroforestry/World Agroforestry

Center

IFACS Indonesian Forest and Climate Support

IPCC Intergovernmental Panel on Climate Change

IR Intermediate Result

IUCN International Union for Conservation Nature

KR Key Result

LBA Landscape Baseline Analysis

LCPs Landscape Conservation Plans

LEDS Low Emissions Development Strategy

LOP Life of Project

LULUCF Land Use, Land Use Change, and Forestry

M&E Monitoring & Evaluation

ME & L Monitoring Evaluation & Learning


METT Management Effectiveness Tracking Tool

MIS Management Information System

MOEF Ministry of Environment and Forestry

MOU Memorandum of Understanding

MRV Measuring Reporting Verification

MSF Multi Stakeholder Forum

NGO Non-Governmental Organization

NRM Natural Resource Management

PA Protected Areas

PES Payment for Environmental Services

PIRS Performance Indicator Reference Sheets

PITT Performance Indicator Tracking Table

PPPs Public-Private Partnerships

QCA Qualitative Comparative Analysis

RAN GRK Rencana Aksi Nasional Gas Rumah Kaca (National Action Plan for

Greenhouse Gas Emission)

REDD+ Reducing Emissions from Deforestation and Forest Degradation

(including conservation, sustainable management of forests, and

enhancement of forest carbon stocks)

RF Results Framework

REL Reference Emission Level

RPJM Rencana Pembangunan Jangka Menengah (Mid-Term Development

Plan)

RTRWK Rencana Tata Ruang Wilayah Kabupaten (District Spatial Plan)

RTRWN Rencana Tata Ruang Wilayah Nasional (National Spatial Plan)

RTRWP Rencana Tata Ruang Wilayah Provinsi (Provincial Spatial Plan)

SEAs Strategic Environmental Assessments

SDI Spatial Data Infrastructure

SMART Spatial Monitoring and Reporting Tool


SOP Standard Operating Procedure

SOW Scope of Work

STI Sustainable Travel International

TFA Tropical Forest Alliance

ToC Theory of Change

UN-REDD United Nations-Reducing Emissions from Deforestation and Forest

Degradation

USAID United States Agency for International Development

USG United State Government

WCPA World Commission of Protected Area

WCS Wildlife Conservation Society

WWF World Wildlife Fund


1. INTRODUCTION 1.1 Project Background USAID LESTARI supports the Government of Indonesia to reduce greenhouse gas (GHG)

emissions and conserve biodiversity in carbon rich and biologically significant forest and

mangrove ecosystems. Built on the strong foundation of USAID’s IFACS project, LESTARI

applies a landscape approach to reduce GHG emissions, integrating forest and peatland

conservation with low emissions development (LEDS) on other, already degraded land. This

is achieved through improved land use governance, enhanced protected areas management

and protection of key species, sustainable private sector and industry practices, and

expanded constituencies for conservation among various stakeholders. LESTARI is

implemented under the leadership of Tetra Tech and a consortium of partners including

WWF-Indonesia, Winrock International, Wildlife Conservation Society (WCS), Blue Forests,

Yayasan Sahabat Cipta, PT Hydro South Pole Carbon, Sustainable Travel International

(STI), Michigan State University, and the FIELD Foundation. LESTARI runs from August

2015 through July 2020.

LESTARI activities are targeted in six strategic landscapes on three of Indonesia’s largest

islands, where primary forest cover remains most intact and carbon stocks are greatest. In

northern Sumatra, the Leuser Landscape comprises significant portions of Aceh Selatan,

Gayo Lues, Aceh Tenggara, and Aceh Barat Daya districts, and includes the Aceh portion of

Leuser National Park and Singkil Wildlife Sanctuary. In Central Kalimantan, LESTARI works

in the Katingan-Kahayan Landscape, comprising Pulang Pisau, Katingan, and Gunung Mas

districts; Palangkaraya municipality; and Sebangau and Bukit Baka Bukit Raya National

Parks. LESTARI also works in four landscapes in Papua. Sarmi and Cyclops Landscapes

are located along the northern coast and comprise Sarmi district as well as Jayapura district

and municipality. The Lorentz Lowlands Landscape, comprising Mimika and Asmat districts

plus a large portion of Lorentz National Park, and the Mappi-Bouven Digoel Landscape are

located along Papua’s southern coast. LESTARI is managed from its headquarters in

Jakarta, with offices in each landscape as well as the provincial capitals of Aceh, Central

Kalimantan, and Papua.

Overall Results of LESTARI are:

Goals:

Key Result 1: At least 41% of total CO2-equivalent emissions reduced from land use,

land use change and deforestation averaged across all landscapes within the project

scope;

Key Result 2: At least 8.42 Million hectares of primary or secondary forest (and

peatland), including orangutan habitat, under improved management;


IR 1: Improved Forest Management:

Key Result 3: Management of at least six Conservation Areas (CAs) improved,

resulting in the conservation of valuable orangutan and other key species habitat,

and the reduction in poaching of threatened and endemic species;

Key Result 4: At least ten Public-Private Partnerships (PPPs) promoting low-

emissions conservation oriented development established;

Key Result 5: Funding leveraged from public and private sources, representing co-

investment in project outcomes;

IR 2: Improved Land Use Governance:

Key Result 6: Increased commitment of key private sector, government, and

community stakeholders regarding the positive benefits of conservation and

sustainable use of forests and the species they encompass;

Key Result 7: Policies, laws, regulations, and procedures in support of low emission

development and forest conservation and management increased, promulgated, and

enforced at all levels; and

Key Result 8: Model(s) for successful integration of district, provincial, and national

low emissions development and forest conservation strategies developed and shared

at all levels of government and with other key stakeholders.

To achieve these results, LESTARI will implement 8 Strategic Approaches in three

complementary technical thematic areas:

1. Forest & Land Use Governance & Advocacy;

2. Conservation Co-Management; and

3. Private Sector Engagement

Following a Landscape Approach1 to implementation, each strategic approach will be

nuanced to address the biophysical, social, political, and cultural context of each landscape.

LESTARI will be implemented primarily with congressionally mandated Biodiversity

Conservation funds. All Biodiversity Earmark-funded activities must align with U.S. Agency

for International Development’s (USAID) Biodiversity Policy and Biodiversity Code. The

Code requires that funded activities must a) have an explicit biodiversity objective; b) be

identified based on an analysis of drivers and threats to biodiversity and a corresponding

theory of change; c) must have the intent to positively impact biodiversity in biologically

significant areas; and d) must monitor indicators associated with each stated theory of

change for biodiversity conservation results. To support learning, adaptive management, and

to comply with the Biodiversity Code and ADS 203, Tetra Tech has developed the LESTARI

Activity Monitoring and Evaluation Plan (AMEP) to provide standards and guidelines for

monitoring, evaluating, and communicating our progress toward achievement of the

biodiversity and emissions targets, project results, and higher level USAID and GOI goals.

1 Further information about Landscape Approach, please refer to LESTARI Landscape Baseline Analysis (LBA) document.


1.2. AMEP Development The LESTARI AMEP is a living document, designed to ensure accountability and learning

which will be used to facilitate and promote evidence-based decision making. The document

outlines Monitoring & Evaluation (M&E) systems, protocols, and data collection

methodologies. Throughout implementation, this framework will ensure continuous learning

while improving coordination and collaboration to track, evaluate, and articulate results and

impacts. The LESTARI M&E approach also ensures the adaptive management of project

activities through the use of sound, reliable, and timely data. Furthermore, the approach to

performance management integrates the five Open Standards for the Practice of

Conservation steps (hereafter, Open Standards). This enables the LESTARI AMEP to be

used as not only a monitoring tool but also a management tool that will be referenced by

staff, and used for training and capacity building to ensure comprehension and compliance

with sound performance management.

The design of LESTARI AMEP is further founded upon lessons learned from the IFACS

M&E system, and key recommendations from the IFACS Final Assessment. A key

recommendation was the need to incorporate a Theory of Change (ToC) approach in M&E

design. The assessment found that key critical assumptions about the causal linkages

between activities and higher level results had not been considered due to limited use of

program theory.

Following this recommendation, the LESTARI AMEP incorporates a theory of change

approach to define all building blocks required to bring about positive change whereby multi-

stakeholders’ interests and actions are better aligned with sustainable outcomes. LESTARI’s

theories of change not only illustrate the causal linkages between project interventions, but

also how these interventions address drivers to deforestation to reduce high level threats to

ecosystems, species, and human well-being targets. In addition, performance indicators and

key impact evaluation questions have been selected to monitor progress toward

achievement of outcomes and impacts along each Results Chain. Theories of Change will

also serve as a project management tool to review critical assumptions driving LESTARI

strategic approaches, facilitate early identification of technical challenges, manage

stakeholder expectations, and serve as a framework for gathering evidence.

This document outlines the detailed blueprint designed to monitor and evaluate the project’s

performance. First, the LESTARI Situation Model, Results Framework, Strategic

Approaches, and Theories of Change are described. This is followed by outlines project

indicators and targets which further detailed in Performance Indicator Reference Sheets

(PIRS) formatted to USAID standards. PIRS will provide identification of staff responsible for

different tasks, as well as timelines. The document then describes the LESTARI Learning

Loop which explains how the LESTARI M&E system be integrated into project management

and how project impact will be routinely monitored and assessed. Finally, the

operationalization of M&E system through the establishment of M&E Team and the

development of Management Information System (MIS). The structure and content of this

AMEP is congruent with ADS 203 and includes both USAID Foreign Assistance

Coordination Tracking Systems (FACTS) and custom indicators for monitor performance and

evaluating the theories of change.


2. LESTARI M&E FRAMEWORKS 2.1 Situation Model The LESTARI AMEP was developed following the Conservation Measures Partnership

(CMP) Open Standards for Adaptive Management. Design began with identification of the

project scope (geographic focus), conservation targets (key species and ecosystems, as well

as human well-being targets linked to these species and ecosystems), direct and indirect

threats, and drivers to be included in the LESTARI Situation Model. Upon completion of the

Situation Model, theories of change were developed by analyzing how LESTARI

interventions will address factors within each results chain to reduce key threats and lead to

the conservation of priority targets and emissions reductions. In September 2015, a

participatory M&E Workshop was held with LESTARI Staff, USAID representatives, and

representatives from USAID’s Measuring Impact project to review and further nuance the

Theories of Change for inclusion in the AMEP. This process resulted in the identification of

eight Strategic Approaches which will be implemented within three Technical Themes:

Forest & Land Use Governance & Advocacy, Conservation Co-Management, and Private

Sector Engagement. Throughout the course of the project, LESTARI will lead participatory

reviews of theories of change with partners, staff across all landscapes, and USAID to

assess the validity of initial assumptions and casual linkages, to refine project planning as

necessary, to address gaps and maximize results, and to update the AMEP to support

ongoing learning and adaptive management. Reviews will address key evaluation questions,

including:

To what extent have LESTARI interventions been effective in reducing biodiversity

threats and carbon emissions in targeted areas?

To what extent have LESTARI intervention been effective in improved land-use

governance?

To what extent has the project addressed barriers of political will and buy-in at the

national and regional levels?

To what extent has improved collaboration between government, private sector and

community contributed to improved forest management?


Figure 1. LESTARI Situation Model


Table 1 LESTARI Situation Analysis

Problem Statement

As one of the richest biological countries in the world, Indonesia is considered “mega diverse” country, contributing, along with six other Asian countries, up to 70 percent of the globe’s biological diversity. Indonesia has held Asia’s largest and continuous track of tropical rain forests and 10% of the world’s forests. Important terrestrial ecosystems include mangroves, lowland forests, montane forests, and peat/swamp forests, the protection of which is considered critical for conserving biodiversity and preventing or reducing land based carbon emissions. Healthy intact forest ecosystems are also an important safety net for local communities living nearby, presenting economic opportunities through forest dependent livelihoods that prevent their slide into extreme poverty and improving their resilience to climate and other external shocks.

While at least 30 million people depend directly on Indonesia’s forests and on the ecosystem services they provide, these ecosystems have been and continue to be under extreme threats. Indonesia’s deforestation rate is well known to be among the highest in the world. It was reported 15.79 million Ha of forest cover loss for period 2000-2012 (Margono, 2014). Deforestation is especially critical in carbon rich peat and swamp forests of Sumatra, Kalimantan, and Papua. These forests are also the primary habitat for key species of global importance, particularly those listed on CITES Appendix I (especially rhinos, elephants, tigers, and orangutans) and endemics found in Papua. These key species are threatened by large-scale destruction of tropical forests, habitat loss, unsustainable harvesting, and trafficking.

Deforestation, peatland degradation, and forest fire have put Indonesia among the top three largest emitters of greenhouse gases in the world. The emissions resulting from forest fires and degradation are five times greater than those ensuing from non-forestry emissions. A 2010 report suggests that 85% of Indonesia’s greenhouse gas (GHG) emissions stem from land use activities with 37% due to deforestation and 27% due to peat fires (National Council on Climate Change, 2010). The most recent research on forest fire and emission it has resulted demonstrates that since the beginning of September 2015, Indonesia’s fires have been emitting carbon at a rate of 15-20 million tons per day, or more than the 14 million tons emitted on a daily basis by the whole American economy.

Forest fires also destroy large areas that serve as habitat for biodiversity. Plants and animals are eliminated directly by the fires and those survive will diminish gradually. For example, the fires in 1997-1998 caused the population of orangutans to decline by 33 percent on the island of Borneo (Rijksen et al.,2006).

Threats, Drivers, and Opportunities

Over the last 20 years deforestation has been driven predominantly by agricultural expansion, especially of oil palm plantation. A history of weak governance combined with opportunistic private sector actors has created unsustainable economics based on natural resource exploitation. Land clearing for oil palm plantations is one of the key drivers of deforestation. Palm oil is also an important element of Indonesia’s national development strategy, as it is the world’s largest producer and exporter of crude palm oil. Oil palm plantation area has doubled in Indondesia in the past decade, and pressures for expansion continues as GOI has instituted targets for doubling CPO production to 40 million tons by 2020.

Other commodity-driven pressures on land include unsustainable industrial scale logging and conversion for pulp, rubber, and coffee/cacao plantations. An inequitable system of land use decision-making is threatening forests, as smallholders that cannot compete for access with large private sector actors are forced to encroach onto forested areas.

In regards with key species, expansion of oil palm and pulpwood plantations in


Sumatra was responsible for nearly two-thirds of the destruction of tiger habitat from 2009 to 2011. When rainforests are cleared, it becomes increasingly easier for poachers to track and kill tigers, which are considered prized trophies for any hunters. Used in traditional Chinese medicine and as ornaments, each Sumatran tiger can sell for up to $5,000, offering poachers a greater income than they would earn working in paddy fields or on rubber tree plantations.

The World Wildlife Fund (WWF) estimates that orangutan populations have significantly declined from 230,000 nearly a century ago to 7,500 in Sumatra and 55,000 in Borneo. A significant factor in the decline of orangutan numbers is the fragmentation of their habitat, which makes them vulnerable to wildlife traffickers.2

The LESTARI Situation Model and Analysis illustrate how deforestation and forest degradation in Indonesia is a result of a complex dynamic of political, economic and institutional drivers. Decision-making on land use involves a large array of laws and actors and the process is rarely transparent or consistent. Land use decisions are driven by economic incentives that prioritize extractive industries and agriculture over low emissions and conservation-oriented business models. However, enabling conditions for a shift to more equitable and sustainable forest and land use management are emerging after years of contradictory talk about forest clearing moratoriums while oil palm and industrial forest plantations rapidly expanded. The new President administration is taking clear, firm, and ambitious steps to improve forest and land use governance in a sustainable and equitable manner. Three new ministries have been formed to elevate and reinforce the goals of better governance of natural resources and the rural poor who depend upon them for most livelihoods: Environment and Forestry; Agrarian and Spatial Planning; and Villages, Development of Backward Regions, and Transmigration.

The private sector is also demonstrating substantial leadership toward reducing large-scale deforestation. The US Ambassador recently facilitated the Palm Oil Pledge, signed by CEOs of Asian Agri, Cargill, Golden Agri Recourses, Wilmar, and Indonesian Chamber of Commerce (KADIN). Reaching all the way to their third-party suppliers, the pledge aims to make palm oil supply chains more sustainable. Further, Sinar Mas’ Asia Pulp & Paper has committed to halt deforestation in their concessions and to conserve 1,000,000 hectares of High Conservation Value (HCV) and High Carbon Stock (HCS) landscapes in and around their operations across Sumatra and Kalimantan; it has undergone an independent audit of these areas by Rainforest Alliance. Increasingly, companies are demonstrating that good business integrates sustainable landscape and conservation management into their operations. Civil society, particularly forest-dependent communities, are increasingly advocating for more responsible land and forest governance. This is demonstrated in Central Kalimantan, where communities have become disenchanted with the empty promises of economic benefits from oil palm expansion while suffering from the omnipresent haze of land clearing and peatland fires. Forest-dependent communities across Indonesia are initiating actions to counter this situation—mapping customary (adat) forests, integrating these maps into the OneMap initiative, and negotiating boundaries with concession holders and spatial planners. LESTARI aims to catalyze such positive change, and is strategically designed to do so. LESTARI targets a 41% GHG emissions reduction in its landscapes by 2020. The Government of Indonesia has also committed to reducing GHG emissions by 29% below the business as usual condition (baseline) by 2030 (previously 26% by 2020) unilaterally, and up to 41% below the baseline with bi-

2 http://www.onegreenplanet.org/animalsandnature/deforestation-poaching-and-the-wildlife-trade-in-indonesia/

http://www.onegreenplanet.org/animalsandnature/deforestation-poaching-and-the-wildlife-trade-in-indonesia/


lateral or international assistance. LESTARI will work within the same time frame as the GoI commitments and has adopted the same targets of working towards 41% reduction of Baseline emissions by 2020. As such the project can effectively harmonize efforts with the GoI within the LESTARI landscapes. A thorough understanding of where LESTARI will contribute to the GoI targets is presented in the Landscape Baseline Analysis Plan (LBA). Historical deforestation and degradation in protected areas, production forest and development (non-forest estate zones), as well as on peat and non-peat areas was analysed to indicate where forest loss and degradation is likely to occur in the future, thus targeting LESTARI’s efforts. LESTARI targets of GHG emission reductions are based on the LBA.

Indonesia is also a Party to the Convention on Biological Diversity and as such, has made commitments to achieve the Aichi Biodiversity Targets by 2020. Indonesia has set specific targets for protection of key endangered species such as tigers, rhinos, and orangutan. This clear alignment in goals and interests is expected to give LESTARI strong buy-in and support from all stakeholders and contribute to the project’s successes in protecting Indonesia’s forests.

LESTARI Landscape Approach

LESTARI applies a landscape approach – an integrated land use management framework that seeks to integrate policies and actions across sectors in order to harmonize development and conservation objectives. In other words, the landscape approach aims to ensure that all the uses of land and all the users of that land are being addressed in an integrated way. LESTARI activities are targeted in six strategic landscapes on three of Indonesia’s largest islands, where primary forest cover remains most intact and carbon stocks are greatest. The LESTARI landscapes are largely defined by national parks and forests supporting mega diversity and key species. To achieve both climate change and conservation objectives, LESTARI has targeted area with HCV and HCS. To address key threats, key activities to be implemented under LESTARI will focus on improving spatial planning for better land management, improved collaborative management, alternative funding for forest conservation or payment for environmental services, preventing encroachment, reduce poaching, community conservation agreements, forest education initiatives as well as strengthening local partners’ landscape vision and commitment for low emissions development.

2.2. LESTARI Results Framework and Theories of Change An overarching Results Framework (RF) has been developed (Figure 2a and 2b) to explain

how LESTARI Strategic Approaches will address the drivers of deforestation and forest

degradation, and reduce emissions and threats to biodiversity presented in the Situation

Model. The RF demonstrates causality and the logical linkages between shorter-term,

intermediate, and longer-term results. The Results Framework is founded upon the

overarching theory that if there is increased awareness and advocacy of local

stakeholders (government officials, communities, CSOs and private sector) for

conservation and natural resource management in ways which contribute to their

long term interests; if Conservation Co-Management is actually enhanced to the

benefit of key stakeholders, and if the private sector is engaged, there will be

improved governance and natural resource management in biologically significant


and critical areas3, leading to a 41% reduction in GHG emissions by 2020. Our Results

Framework is then translated into a matrix showing how LESTARI plans to integrate

LESTARI’s Technical Components into 8 strategic approaches that can be nuanced

according to the context of each targeted landscape. This matrix also demonstrates our

theory of change for each Strategic Approach, in accordance with the Results Chains

provided in Annex II: Results Chains.

3 Critical areas: areas identified for conservation because of their high conservation value or their high carbon value


Figure 2a. LESTARI Results Framework with Key Results


Figure 2b. LESTARI diagrammatic Results Framework


Table 2 Theory of Change Matrix

Strategic Approaches

Technical Components and Tasks: (SOW Technical Components: 1. Land Use Governance;

2. CAs & Key Species; 3. Private Sector; 4. Constituencies)

Theory of Change

Technical Theme I: Forest and Land Use Governance and Advocacy

Awareness and Advocacy

C1.1. Increasing government level awareness, appreciation, and advocacy to ensure that government decision-makers embrace a conservation and sustainable use vision for land use planning.

Improved local level awareness, appreciation, and advocacy to promote citizen action is one of enabling conditions for other LESTARI strategic approaches to run effectively. Through the implementation of this strategy, we expect that local stakeholders will adopt and continue desired behavior thus results in the formation of constituent for conservation. We assume that if the citizens demand for healthy ecosystems and have the capacities for advocacy and lobbying, the government will respond by embracing a conservation and sustainable development vision to better address demand from its citizens.

As outlined in the Result Chains for this strategy (Annex II), it is critical for LESTARI to first identify target audience, message, desired behavior, and appropriate media to deliver the message. LESTARI needs to ensure that target audiences receive the messages through a series of awareness and advocacy activities, retain the desire knowledge which will lead to improvement on attitudes and values. Draw on IFACS experience, it is critical to confirm that knowledge and attitude be translated into effective desired behavior. Therefore, during the implementation LESTARI will assess factors that impede change to occur and address them immediately.

C4.1. Improve local level awareness, appreciation, and advocacy. Improve key stakeholders’ awareness, appreciation and advocacy for healthy ecosystems, leveraging increased availability of mobile and internet based technology.

C4.2. Empower media coverage of environmental issues. Strengthen the ability of media and citizen journalists to monitor, document, and publicize the positive and negative consequences of land use decisions, including those made by private sector actors in the landscapes.

Operationalize SEAs & LCPs

C1.2. Operationalizing Strategic Environmental Assessments (SEAs) and Landscape Conservation Plans (LCPs) developed by USAID IFACS within land use plans

Improved land use governance is a cornerstone of achieving improved forest management and biodiversity conservation. Good governance is characterized by the respect for legal certainty, transparency and free flow of information, significant public participation (expression of voice and choice), equality, high accountability, effective and coordinated management of public resources.

LESTARI assumes that SEAs and LCPs initiated under IFACS are effective entry points to improve land use governance. The incorporation of those documents in government planning (notably spatial planning,





Theory of Change

RTRW and related zonation as well as development plans, RPJMD/P) will involve significant public participation. As such a comprehensive and accurate SEA is crucial to developing a spatial plan that incorporates LEDS. In conjunction, LCPs is a tool to give local communities quality information about the conservation value of their land and ensure that government plans consider local conservation needs. Such data can also serve as valuable evidence for communities in their advocacy and lobbying work, which may be an important element in holding decision makers accountable.

LESTARI will operationalize both SEA-LEDS and LCPs through the following:

Facilitate the development or update the SEA-LEDS document that incorporate LCPs

Advocate for district to issue instruction for operationalizing SEA-LEDS

Work through local government and the MSF to integrate high quality SEA-LEDS in spatial plans

Expand impact at district levels by officially incorporating SEA-LEDS recommendations into RPJMs’ planning and budgeting as well as licensing and permitting process

Environmental Governance

C1.3. Improving licensing and permitting processes so they are public and accessible, subject to public oversight, compliant with laws and regulations, and violations are enforced

Land use planning and licensing of natural resource use is the single largest contributor to GHG emissions in the LESTARI landscapes. Environmental governance is a principle strategic approach that contributes to improved planning, permitting, and enforcement of natural resource use within landscapes. It is an interlinked approach that combines introducing citizen-based mechanisms for input on land use, as well as increasing the district willingness to adopt such input, with improved mechanisms for licensing, monitoring, and enforcing the use of natural resources. Multi Stakeholder Forum (MSF) built under IFACS will serve as a bridge to facilitate communication between government and citizen. The strategic approach also links to operationalizing SEA and LCP recommendations.

C1.4. Monitoring and enforcing land use, including increasing district government prosecution and adjudication capacity, building on USAID C4J activities, and capacity to use remote sensing and other geospatial tools, building on the spatial data infrastructure units established previously by USAID IFACS

C4.3. Strengthen citizen based mechanisms for public input on land use. Building on the multistakeholder forums established





Theory of Change

under USAID IFACS, support citizen-based mechanisms for public input and monitoring of land use decisions by key groups, especially local communities, indigenous groups, and women. This task is closely aligned with the Component 1 task to encourage districts to solicit and adopt input from citizen based mechanisms

It is also important to note that of the numerous governance challenges, the limited availability of data which several stakeholder groups deem credible represents one of the most serious. Robust and accurate data is needed to inform well-measured action plans, especially in terms of reforming legal and policy frameworks and improving the capacities of government, civil society, indigenous communities and private sector stakeholders. Therefore, LESTARI will continue to build upon IFACS progress on developing Spatial Data Infrastructure (SDI) units, linked to OneMap, to improve the accuracy, consistency, and availability of land use information especially public transparency and accessibility of natural resource use processes and licenses.

C1.6. Increasing district governments’ willingness to adopt input from citizen-based mechanisms. This is connected to the work with multistakeholder forums under Component 4.

Technical Theme II: Conservation Co-Management

Co-Management

C2.3 Improve co-management adjacent to Conservation Areas by building staff knowledge and implementation of improved collaborative management strategies with adjacent districts and local communities. This task is closely linked to Component 4’s collaborative management task.

Key to project success is building capacity and commitment for co-management among relevant actors in areas adjacent to PAs and critical areas. This is essential for pressure reduction on the PA through more transparent, negotiated roles and responsibilities. Relevant actors will include Ministry of Environment and Forestry Park staff, provincial and district governments, relevant CSOs, and villages living adjacent to and dependent on PA and critical areas resources and services.

LESTARI will facilitate dialogue between key stakeholders and communities, develop the capacity of villages to be able to negotiate with district, provincial and national agreements, comply with, and benefit from co-management agreements to protect their local natural resources.

The strategic approach supported by awareness and advocacy so as to increase the willingness of key stakeholders to develop co-management agreements with communities. It also links to strategic approach green enterprise and innovative financing mechanism whereas the co-management agreements provide incentives for communities to participate in forest protection.

LESTARI will work with leaders that represent both men and women and

C3.2. Pilot innovative financing for critical areas. Mobilize financing opportunities, including payments for ecosystem services, to incentivize sustainable enterprises that meet required criteria for environment and social soundness

C4.4. Improve co-management adjacent to critical areas. Provide technical assistance to establish co-management arrangements between communities living in and near critical areas and the appropriate government entities. Includes defining acquiring clear tenure. Closely linked to Strategic Approach 2. SEAs. See IUCN definition of "co-management".





Theory of Change

other disadvantaged groups to ensure differing gender considerations and viewpoints are addressed through co-management.

Protected Area Management

C2.1. Improve Conservation Area management by determining priority needs and actions for addressing threats specific to each CAs and the species they harbor in consultation with key stakeholders. Once these have been identified, support the implementation of priority actions to address threats and strengthen the protection and/or management of CA and the species they harbor. These may include management planning; improving data, monitoring and information systems; increasing the use of technology to monitor and enforce wildlife encroachment and poaching; enhancing patrols; improving public relations with districts and communities inside or adjacent to CAs; and other capacity building

Key to success in improving conservation areas management capacity is building political leadership; implementing improved management approaches including protection of core areas to combat poaching and deforestation; and adopting M&E systems to assess the effectiveness of management interventions and adapt approaches as needed.

LESTARI will focus on the socialization and application of the METT tool to assess management effectiveness in protected areas embedded in its landscapes. Baseline METT will be conducted in a participatory way so as to enable CA managers taking ownership of METT score and committed to improving their management weaknesses. Awareness and advocacy will be emphasized to increase public pressures on CA authority to improve their management weaknesses.

A training program –with a focus on technical skills and modern tools that managers and PA staff require for effective day-to-day duties –will be delivered based on the outcome of the METT assessments. LESTARI will also equip CAs with the capacity to undertake SMART (Spatial Monitoring and Reporting Tool) as a tool to monitor and enforce wildlife encroachment and poaching.

This strategic approach links to improve co-management adjacent to CAs, especially because threats are coming from area adjacent to CAs. Further assumption is CAs could improve and implement key aspects of management plan if they are adequate financing. Therefore, LESTARI will identify and develop effective innovative financing plans for CAs.

Furthermore, the success of LESTARI in implementing a landscape approach requires effective policy engagement at multiple levels across multiple stakeholders. LESTARI will coordinate with the national level government and provincial level to ensure local impact sustainability,

C2.2. Pilot innovative financing for Conservation Area by identifying and trailing CA sustainable financing and revenue-generating strategies

C1.7. Enhance national level policy coordination Component 2





Theory of Change

particularly since Law 23 of 2014 recentralizes a significant amount of natural resource decision-making to the province.

Technical Theme III. Private Sector Engagement

Green Enterprises

C3.1. Develop public private partnerships for green enterprises. This includes reviewing existing or conducting new market and feasibility studies to explore the potential for ecotourism and other environment-friendly sustainable enterprises that will directly benefit conservation and local communities living within and adjacent to high conservation area forests. Equality important is cultivating relationships with key private sector actors in the project’s landscapes to identify sources of co-investment in the above conservation efforts. Finally, the Contractor should develop and pilot sustainable business models that can have broad impacts, including outside the specified landscapes, and disseminate the results of pilots

As with all other Technical Themes, Private Sector Engagement takes an integrated approach to the development of livelihoods, responsibilities of environmental governance, and shared roles in co-management of natural resources by reducing spatial uncertainty especially for local communities. The capacity of indigenous communities is naturally impacted by the actions of the business sector: when a business company follows good governance practices, this will have a positive impact on indigenous communities, and vice versa. Therefore, LESTARI will facilitate both parties to seek opportunities for cooperation and mutual reinforcement. Through establishing public-private partnerships for green enterprises, and coupled with commitments to safeguards for forest resources (such as through co-management) and improve environmental governance, LESTARI will support alternative livelihoods while reducing pressure on forest resources. C4.5. Improved sustainable livelihoods and alternative

livelihoods for communities adjacent to critical areas. Provide technical assistance to communities living in or near critical areas to improve sustainable natural resources management practices, including those that might lead to eventual third party certification, that enhance productivity and income or adopt locally appropriate alternative livelihoods or green enterprises in order to reduce encroachment. Communities receiving livelihood support must acknowledge and agree to reduce encroachment through formal agreements such as the Community Conservation Agreements (CCAs) developed under the USAID IFACS project





Theory of Change

Private Sector BMP

C3.3. Industry certification, best management practices (BMPs) and Conservation Management and Mitigation Plans (CMMPs). Support private sector-led efforts to make explicit commitments towards zero deforestation and conservation of HCV and other critical areas including support for third party certification, implementation of BMPs and CMMPs

The private sector provides significant opportunities for conservation of biodiversity and reduction of deforestation and degradation within LESTARI landscapes. LESTARI will adapt a suite of tools, expanding the applicability of the existing Agricultural Sustainability Screening Protocol and Sustainability Roadmap developed under the USAID CIRCLE project that allow the private sector to identify key steps to increase their sustainability and transparently monitor and demonstrate the impact of its commodity production on emissions. Subsequently, LESTARI will expand on its work under IFACS to promote BMPs for natural forest concessions and fiber and oil palm plantations leading to HCV forest set-asides, as well as for value chains critical to improved livelihoods. Through co-management agreements, LESTARI will facilitate the establishment of LEDS-based community partnerships with villages adjacent to concessions area. The assumption then if companies apply BMPs, if companies enhance co-management with villages, and if both parties gain benefits from BMPs and co-management implementation, then unsustainable use of natural resource will be discontinued thus emissions will be reduced and biodiversity conserved. Public discourse will be encouraged towards the removal of policy constraints to increasing the value of natural resources to the private sector in ways that are both economically and environmentally sustainable.

PES and REDD Innovative Finance

C1.5.Enhance district readiness to access financing (DAK, REDD+, etc) by building necessary capacity, supporting MRV systems, and informing policies, rules, and systems for the equitable distribution of funds generated from payments for carbon or other ecosystem services

With the creation of Indonesia Climate Change Trust Fund (ICCTF), Global Environment Fund, and the Green Climate Fund, there is potentially increased opportunity for districts to access funds to reduce emissions from deforestation and degradation. An important and more omnipresent source of funding is from government budgets. LESTARI will provide technical assistance to local government to prepare project design documents and safeguards to access REDD+ funding. LESTARI will also work with central and local government officials to include LEDS, forest conservation, and climate change mitigation programs to meet reduced emission commitments, especially through Payment for Environmental Services (PES) and societal valuing of such services in





Theory of Change

general. LESTARI will also identify and engage related ministries to gain more access and support additional financing sources and to explore potential collaboration with similar programs from the government in optimizing the benefits to the communities and environment. If funding can be secured, then impact on emissions reduction could be magnified.


2.3 Logical Framework and Performance Indicator Tracking Table (PITT) Our Logical Framework presents an overview of LESTARI Goals, Strategic Objectives,

indicators, means of verification, critical assumptions, and annual and life of project (LOP)

targets for monitoring progress toward achievement of results and evaluating the validity of

LESTARI’s Theories of Change. Output, outcome, and impact level indicators that provide a

sound benchmark for performance management and reporting on project results have been

selected, including required standard indicators for biodiversity and sustainable landscapes.

Detailed information for each indicator is provided in Annex I: Performance Indicator

Reference Sheets (PIRS).


Table 3 Logical Framework and Targets

Note: Please refer to Performance Indicator Reference Sheets (PIRS) in Annex I for indicator disaggregation

STRATEGIC OBJECTIVES

INDICATOR MEANS OF

VERIFICATION ASSUMPTION TARGETS

B Y1 Y2 Y3 Y4 Y5 LOP

LESTARI Goals

Reduced GHG emissions from land-use sector

1. Percentage reduction in GHG emissions as a result of USG assistance measured using actual emissions compared to REL (OUTCOME/IMPACT)

Note: for the purpose of FACTS reporting, quantity of GHG emissions will be reported in metric tons of CO2equivalent

Comparison of actual emissions to REL following GoI methodology for Land-Based GHG Emission and Sequestration Baseline Calculations. The REDD Abacus SP program is used to calculate land cover changes and related emissions.

Reductions in deforestation and degradation will lead to reductions in GHG emissions.

0% reduction

Not measure

Not measu

re

30% reducti

on

40% reducti

on

41% reducti

on

41% reducti

on

Not measure

Not measu

re

41 M tCO2-eq

68 M tCO2-eq

96 M tCO2-

eq

96 M tCO2-

eq

Key species protected

See #3 See #3 See #3 See #3

See #3 See #3 See #3 See #3 See #3 See #3



INDICATOR MEANS OF



Carbon rich forest, peatland, mangrove ecosystems and the habitat of key species conserved

2. Number of hectares of biological significance and/or natural resources under improved natural resource management as a result of USG assistance (FACTS 4.8.1-26) (OUTCOME/IMPACT)

Sub-national: Co-

management plan in place and under implementation as evidenced by incorporation into government program and budget.

FMUs assisted by LESTARI operational.

Certain area set-aside for conservation as result of SEA-LEDS and/or LCP recommendations into local development plans.

CA: Increased

METT score

Concessions:

CMMP or other conservation plan in place and under implementation as evidenced by incorporation into SOP and company budget.

Improved management will lead to the conservation of key species habitats as well as contribute to reduction in GHG emissions.

0 0 1.7 M 2.5 M 3 M 1.5 M 8.7 M



INDICATOR MEANS OF



Reduced rate of deforestation and degradation

See #1 See #1 See #1 See

#1 See #1 See #1 See #1 See #1 See #1 See #1

Reduced threat on key species

3. Percentage reduction in poaching in focus area (OUTCOME)

Analysis of Spatial Monitoring and Reporting Tool (SMART) data. Comparison of baseline and endline. LESTARI will undertake baseline in Q2 Year 1 to determine number of current poaching incident in focus area.

Reduction in poaching will lead to conservation of key species in focus area

TBD Not measure

10% reducti

on

20% reducti

on

30% reducti

on

40% reducti

on

40% reducti

on

Improved Land Use Governance

Forest & Land Use Governance & Advocacy

4. Number of public policies addressing climate change and/or biodiversity conservation introduced, changed or adopted consistent with citizen input (OUTCOME)

Review the process of policies development, minutes of public consultation meetings, comparison of policies introduced, changed or adopted with citizen input.

When stakeholder’s voices continue to be heard by their government, it will ensure sustainability of MSF and lead to improved land use regulatory framework as prerequisite of improved land use governance

0 3 5 8 8 4 28



INDICATOR MEANS OF



5. Number of sub-national government with improved licensing and permitting mechanism (OUTCOME)

The term “improved” is taken to subsume the five core principles of good governance (transparency, participation, accountability, responsiveness, and timeliness). Standard Operational Procedures (SOP) will be developed to ensure that all licenses and permits follow the government regulation and procedures. Verification will be made through reviewing the implementation of SOP, licensing process, interview licensing officers, business enterprises and community to ensure good governance principles are adhered.

When licensing and permitting mechanism adopt five core principles of good governance, the sustainable use of natural resource can be secured.

0 0 4 4 4 2 14



INDICATOR MEANS OF



6. Number of sub-national government incorporating high quality SEA-LEDS & LCPs into draft spatial plans, zonation regulation or draft mid-term development plan (OUTCOME)

Comparison of SEA-LEDS & LCPs recommendation to spatial plan, zonation regulation and mid-term development plan.

Explicit reference to adopting SEA recommendation of improved-case scenario (not BAU) in spatial plan, zonation regulation and mid-term development plan documents.

Incorporation of SEA-LEDS & LCPs recommendation into government planning documents will lead to improved land use policy, which contributes to improved land use governance.

0 2 4 4 2 2 14

7. Number of Multi Stakeholder Forum (MSF) operational as citizen based mechanisms for public input on land use (OUTCOME)

Review of the process of public consultation, review Bupati decree of MSF, qualitative assessment to learn the effectiveness of MSF

The success of MSF to bridge communication between local government and wider public in land use will lead to more transparent, participatory and accountable land use decisions. Those principles are prerequisite of good governance.

0 2 4 4 4 0 14



INDICATOR MEANS OF



8. Number of community champions engaged in advocacy interventions (OUTPUT)

Review advocacy plans or strategies, recording of press conference, copy of testimony or press release, advocacy campaign materials, record of public consultations.

Advocacy should be understood as a means for individuals and constituencies to shape public agendas, change public policies, and influence other processes that impact their lives. Advocacy interventions are essential aspects of democratic policy making, citizen participation, and oversight of government.

0 50 150 150 150 0 500



INDICATOR MEANS OF



9. Number of people reached by LESTARI communication programs to improve awareness and understanding of LEDS and biodiversity conservation (OUTPUT)

Sign-in sheet from communication trainings and meetings. Review communication products developed by stakeholders. Monitoring social media.

Qualitative assessments (e.g., case study) of increase awareness and understanding.

Increase awareness and understanding, supported by continuous advocacy will lead to increased commitment of key stakeholders regarding the positive benefits of conservation and sustainable use of forests and the species they encompass, which in turn lead to improved land use decision making, thus contributes to improved land use governance.

0 1,000 4,000 5,000 3,500 1,500 15,000



INDICATOR MEANS OF



Improved Forest Management

Conservation Co-Management

10. Number of Conservation Areas (CAs) with at least 70 point in METT scores across LESTARI landscapes (OUTCOME)

METT baseline and endline assessment; enriched by qualitative analysis

To maximize the potential of protected areas, it is important to understand the strengths and weaknesses of their management and the threats that they face. Addressing the weaknesses and reducing threats will lead to increase effectiveness of CAs management.

See PIRS

0 1 2 2 1 6

11. Number of co-management agreements signed that secure community rights and benefits (OUTPUT)

Review the content of co-management agreements, evaluation of the effectiveness of their implementation

Co-management agreement that secure community rights and benefits will lead to reduction in land use conflicts and increase commitment to better forest management

0 0 10 13 12 5 40

12. Number of people receiving USG supported training in natural resources management and/or biodiversity conservation (FACTS 4.8.1-27/OUTPUT)

Sign-in sheets, training reports. Assessments of the effectiveness of trainings and the application of knowledge and skills gained.

Increased capacity to manage natural resources and/or biodiversity conservation will lead to improved land use governance and forest management

0 1,000 2,000 3,000 3,000 1,000 10,000



INDICATOR MEANS OF



Private Sector Engagement

13. Amount of investment mobilized (in USD) for climate change as supported by USG assistance (FACTS 4.8.2-10/OUTPUT)

Finance documents (e.g., APBD, finance reports, etc). Interviews the financers. Assess the use of funds and the issues being addressed with the funds.

LESTARI programs should attract additional funds that are necessary to increase capacities for addressing land use and forest conservation. Such funds represent knowledge and commitment of local stakeholders to, and institutionalizati on of, the issues being addressed with the funds.

0 2 M 4 M 4 M 5 M 5 M 20 M

14. Number of people receiving livelihood co-benefits (monetary or non monetary) (OUTCOME)

Random survey and on-site observations

Economic benefits derived from sustainable use of forest resource will provide incentives for people to be committed to LEDS and forest conservation.

0 0 7,500 7,500 7,500 7,500 30,000



INDICATOR MEANS OF



15. Number of private sector firms that have improved management practices as a result of USG assistance (FACTS 4.6.2-9/OUTCOME)

Review the CMMP/conservation plan document, incorporation into SOP and budget line item allocated by concessionaires to implement the plan. Spot-check to observe field implementation

Implementation of CMMPs/conservation plan will ensure that HCV areas in concessions are managed in sustainable manner.

0 2 2 2 2 2 10

16. Number of new USG-supported public-private partnerships (PPPs) formed (FACTS PPP 5) (OUTPUT)

PPP document signed by both public and private entities

PPP will provide incentives both for governments, private sectors and communities to embrace LEDS and conservation oriented practices.

0 3 5 5 5 2 20


Table 4 LESTARI Targets per Landscape

Note: This is illustrative table. The target per landscape will be updated in LESTARI Annual Work Plans based on annual achievement.

INDICATOR TARGET OVER THE LIFE OF

PROJECT LANDSCAPE

ANNUAL TARGETS

B Y1 Y2 Y3 Y4 Y5 TOTAL

1. Percentage reduction in GHG emissions as a result of USG assistance measured using actual emissions compared to REL (OUTCOME/IMPACT)

Note: for the purpose of FACTS reporting, quantity of GHG emissions will be reported in metric tons of CO2equivalent

41% reduction

Leuser 0% Not measure

Not measure

30% 40% 41% 41%

Not measure

Not measure

5,839,579 7,786,105 7,980,758 7,980,758 M tCO2-eq

Katingan-Kahayan

0% Not measure

Not measure

30% 40% 41% 41%

Not measure

Not measure

40,715,537 54,287,382 55,644,567 55,644,567 M tCO2-eq

Lorentz Lowlands

0% Not measure

Not measure

30% 40% 41% 41%

Not measure

Not measure

12,231,438 16,308,584 16,716,299 16,716,299M tCO2-eq

Mappi-Bouven Digoel

0% Not measure

Not measure

30% 40% 41% 41%

Not measure

Not measure

10,132,229 13,509,638 13,847,379 13,847,379 M tCO2-eq

Sarmi 0% Not measure

Not measure

30% 40% 41% 41%

Not measure

Not measure

1,001,325 1,335,100 1,368,478 1,368,478 M tCO2-eq

Cyclops 0% Not measure

Not measure

30% 40% 41% 41%



PROJECT LANDSCAPE

ANNUAL TARGETS


Not

measure Not

measure 81,488 108,651 111,367 111,367 M

tCO2-eq

2. Number of hectares of biological significance and/or natural resources under improved natural resource management as a result of USG assistance (FACTS 4.8.1-26) (OUTCOME/IMPACT)

8.7 Million

Leuser 0 0 178,271 262,163 314,596 157,298 912,328

Katingan-Kahayan

0 0 416,571 612,604 735,125 367,563 2,131,863

Lorentz Lowlands

0 0 642,592 944,988 1,133,986 566,993 3,288,559

Mappi-Bouven Digoel

0 0 317,644 467,123 560,547 280,274 1,625,588

Sarmi 0 0 139,811 205,605 246,726 123,363 715,505

Cyclops 0 0 5,111 7,517 9,020 4,509 26,157

3. Percentage reduction in poaching in focus area (OUTCOME)

40% reduction

Leuser TBD Not measure

10% 20% 30% 40% 40%

Katingan-Kahayan

TBD Not measure

10% 20% 30% 40% 40%

Lorentz Lowlands

TBD Not measure

10% 20% 30% 40% 40%

Mappi-Bouven Digoel

n/a n/a n/a n/a n/a n/a n/a

Sarmi n/a n/a n/a n/a n/a n/a n/a

Cyclops TBD Not measure

10% 20% 30% 40% 40%



PROJECT LANDSCAPE

ANNUAL TARGETS


4. Number of public policies addressing climate change and/or biodiversity conservation introduced, changed or adopted consistent with citizen input (OUTCOME)

28 policies

Leuser 0 1 1 2 3 1 8

Katingan-Kahayan

0 1 1 2 3 1 8

Lorentz Lowlands

0 0 1 1 1 1 4

Mappi-Bouven Digoel

0 0 1 1 1 1 4

Sarmi 0 0 1 1 0 0 2

Cyclops 0 1 0 1 0 0 2

5. Number of sub-national government with improved licensing and permitting mechanism (OUTCOME)

14 sub-national governments

Leuser 0 0 1 1 1 1 4

Katingan-Kahayan

0 0 1 1 1 1 4

Lorentz Lowlands

0 0 1 1 0 0 2

Mappi-Bouven Digoel

0 0 0 1 1 0 2

Sarmi 0 0 1 0 0 0 1

Cyclops 0 0 0 0 1 0 1

6. Number of sub-national government incorporating high quality SEA-LEDS & LCPs into draft spatial plans, zonation regulation or draft

14 sub-national governments

Leuser 0 0 1 1 1 1 4

Katingan-Kahayan

0 1 1 1 1 0 4



PROJECT LANDSCAPE

ANNUAL TARGETS


mid-term development plan (OUTCOME) Lorentz

Lowlands 0 0 1 1 0 0 2

Mappi-Bouven Digoel

0 0 0 1 0 1 2

Sarmi 0 1 0 0 0 0 1

Cyclops 0 0 1 0 0 0 1

7. Number of Multi Stakeholder Forum (MSF) operational as citizen based mechanisms for public input on land use (OUTCOME)

14 MSFs

Leuser 0 1 1 1 1 0 4

Katingan-Kahayan

0 1 1 1 1 0 4

Lorentz Lowlands

0 0 1 1 0 0 2

Mappi-Bouven Digoel

0 0 0 0 2 0 2

Sarmi 0 0 1 0 0 0 1

Cyclops 0 0 0 1 0 0 1

8. Number of community champions engaged in advocacy interventions (OUTPUT)

500 community champions

Leuser 0 15 40 40 40 0 135

Katingan-Kahayan

0 15 40 40 40 0 135

Lorentz Lowlands

0 5 20 20 20 0 65

Mappi-Bouven Digoel

0 5 20 20 20 0 65



PROJECT LANDSCAPE

ANNUAL TARGETS


Sarmi 0 5 10 10 10 0 35

Cyclops 0 5 20 20 20 0 65

9. Number of people reached by LESTARI communication programs to improve awareness and understanding of LEDS and biodiversity conservation (OUTPUT)

15,000 people

Leuser 0 300 1,500 2,000 1,300 500 5,600

Katingan-Kahayan

0 300 1,500 2,000 1,300 500 5,600

Lorentz Lowlands

0 150 500 500 500 200 1,850

Mappi-Bouven Digoel

0 50 200 300 300 200 1,050

Sarmi 0 50 100 100 0 0 250

Cyclops 0 150 200 100 100 100 650

10. Number of Conservation Areas (CAs) with at least 70 point in METT scores across LESTARI landscapes (OUTCOME)

6 CAs

Leuser See PIRS

Not measure

0 1 0 1 2

Katingan-Kahayan

See PIRS

Not measure

1 0 1 0 2

Lorentz Lowlands

See PIRS

Not measure

0 0 1 0 1

Mappi-Bouven Digoel

n/a n/a n/a n/a n/a n/a n/a

Sarmi n/a n/a n/a n/a n/a n/a n/a

Cyclops See PIRS

Not measure

0 1 0 0 1



PROJECT LANDSCAPE

ANNUAL TARGETS


11. Number of co-management agreements signed that secure community rights and benefits (OUTPUT)

40 agreements

Leuser 0 0 2 4 3 2 11

Katingan-Kahayan

0 0 2 3 4 1 10

Lorentz Lowlands

0 0 2 2 2 1 7

Mappi-Bouven Digoel

0 0 1 1 1 0 3

Sarmi 0 0 1 1 0 0 2

Cyclops 0 0 2 2 2 1 7

12. Number of people receiving USG supported training in natural resources management and/or biodiversity conservation (FACTS 4.8.1-27/OUTPUT)

10,000 people

Leuser 0 350 700 1,100 1,200 400 3,750

Katingan-Kahayan

0 350 700 1,000 1,200 400 3,650

Lorentz Lowlands

0 100 300 600 500 100 1,600

Mappi-Bouven Digoel

0 100 100 100 50 50 400

Sarmi 0 50 100 100 0 0 250

Cyclops 0 50 100 100 50 50 350

13. Amount of investment mobilized (in USD) for climate change as supported by USG assistance (FACTS 4.8.2-10/OUTPUT)

USD 20 Million

Leuser 0 500,000 1,500,000 1,000,000 1,000,000 1,500,000 5,500,000

Katingan-Kahayan

0 1,000,000 1,500,000 1,000,000 1,500,000 1,500,000 6,500,000



PROJECT LANDSCAPE

ANNUAL TARGETS


Lorentz Lowlands

0 500,000 1,000,000 1,200,000 1,500,000 1,000,000 5,200,000

Mappi-Bouven Digoel

0 0 0 600,000 350,000 500,000 1,450,000

Sarmi 0 0 0 200,000 300,000 500,000 1,000,000

Cyclops 0 0 0 0 350,000 0 350,000

14. Number of people receiving livelihood co-benefits (monetary or non monetary) (OUTCOME)

30,000 people

Leuser 0 0 2,500 2,500 3,000 3,000 11,000

Katingan-Kahayan

0 0 2,500 2,500 3,000 3,000 11,000

Lorentz Lowlands

0 0 1,000 1,000 700 700 3,400

Mappi-Bouven Digoel

0 0 200 200 300 300 1,000

Sarmi 0 0 300 300 0 0 600

Cyclops 0 0 1,000 1,000 500 500 3,000

15. Number of private sector firms that have improved management practices as a result of USG assistance (FACTS 4.6.2-9/OUTCOME) 10 private sector

firms

Leuser 0 0 0 0 1 0 1

Katingan-Kahayan

0 2 1 1 0 0 4

Lorentz Lowlands

0 0 0 0 0 1 1

Mappi-Bouven Digoel

0 0 0 1 1 1 3



PROJECT LANDSCAPE

ANNUAL TARGETS


Sarmi 0 0 0 1 0 0 1

Cyclops n/a n/a n/a n/a n/a n/a n/a

16. Number of new USG-supported public-private partnerships (PPPs) formed (FACTS PPP 5) (OUTPUT)

20 PPPs

Leuser 0 1 1 1 1 0 4

Katingan-Kahayan

0 1 1 1 2 1 6

Lorentz Lowlands

0 1 1 1 1 0 4

Mappi-Bouven Digoel

0 0 1 1 1 1 4

Sarmi 0 0 1 0 0 0 1

Cyclops 0 0 0 1 0 0 1


3. PLAN FOR ACTIVITY MONITORING AND EVALUATION 3.1 LESTARI Learning Loop: Data Collection, Storage, Quality Assurance, Analysis, and Utilization Monitoring and evaluation (M&E) and reporting are essential parts of the LESTARI project.

The ability to rapidly collect, review, analyze, and communicate monitoring and evaluation

information is essential for maximizing LESTARI results. To facilitate this process, LESTARI

will develop a tailored Management Information System (MIS). The MIS will serve as an

accessible and transparent reporting tool to guide future activities and learning. It will also

connect the development of activity scopes of work to the LESTARI approval process and

link the outcomes of these activities to results as outlined in this AMEP as well as annual

work plans. To increase the efficiency, the MIS will also include features that allow

coordination of technical activities with necessary administrative and operations support.

Variable levels of MIS access will be provided to USAID and LESTARI partners for reporting,

operations support, and invoicing purposes. Progress captured in the MIS will be used to

facilitate LESTARI Adaptive Management approach.

DESIGN: The first step of the

project’s Learning Loop is design,

which includes building

consensus amongst project staff,

USAID, and partners on the

project’s theory of change, expected results, and

indicators. As described in Section 3.1, this

AMEP is founded upon the results of a

Measuring Impact Workshop, where LESTARI

staff and USAID collaborated in the

development of theories of change and

indicators for each LESTARI Strategic

Approach. The outcomes of this workshop were

presented to regional staff and partners in

breakout sessions during the first Annual Work

Planning Meeting, so that theories of change

could be further refined and used to develop

Landscape Level Work Plans.

Figure 3 LESTARI Adaptive Management Data Cycle


Step 1: DESIGN

USAID/Indonesia Construct Situation Model to understand project context

Develop Theories of Change for each Strategic Approach

Develop a Results Framework to show linkages between

Strategic Approaches and higher-level results

Identify performance indicators and targets.

Identify key evaluation questions.

Measuring Impact

LESTARI Team

Home Office M&E Specialist

Local Partners

COLLECT : Most M&E data will be collected using electronic data

collection form. During project startup, LESTARI Monitoring, Evaluation &

Learning (ME&L) Coordinator, with support from Tetra Tech’s Home

Office M&E Specialist, will use mobile data collection tools to create

customized electronic data collection forms (based on the Year 1 Work

Plan and AMEP) whereas data collected will be put into LESTARI MIS.

These electronic forms will include required fields, skips, and ranges to

improve standardization and data quality, as well as open-ended fields to

capture qualitative data.

It is important to note that changes in the methodology for collecting data may compromise

LESTARI’s ability to compare results over time. However, as the project continues to learn

how to improve data quality, changes to data collection tools or methods may become

necessary. In these cases, the ME&L Coordinator will be able to use the web-based

interface to update data collection forms. However, the ME&L Coordinator must document

these changes over time in revisions to the AMEP. Additional details on our specific data

collections methods for LESTARI indicators are outlined in Annex I: Performance Indicator

Reference Sheets (PIRS).

Step 2: COLLECT

LESTARI Staff (include Landscape M&E Specialists)

Collects data and data documentation for all project activities using

approved data collection forms and following the requirements outlined

in the PIRS

Submits data to ME&L Coordinator as data available

ME&L Coordinator Develops data collection forms

Collects additional qualitative data as necessary

Trains Technical Staff on data collection methods, standards, and

requirements

Trains Technical Staff on how to collect data using data collection

forms

Documents changes to data collection tools/methods resulting from


Step 2: COLLECT

ongoing learning

COP Provides support to the ME&L Coordinator in ensuring that data are

collected and submitted in a timely manner.

HO M&E Specialist Provides initial support and training in data collection form and

development of data collection tools.

REVIEW: Mobile data collection tool will improve data quality through

facilitating real-time reviews by the ME&L Coordinator. The ME&L

Coordinator will receive a notification each time new data are added to MIS

and will then review the quality of data against ADS 203 data quality criteria

(validity, reliability, timeliness, precision and integrity), and reassign the

form to the field-based data collector if corrections are required. If ongoing data quality

issues persist, the ME&L Coordinator will work with the CoP, Tetra Tech’s home office M&E

Specialist as well as USAID/Indonesia to develop new data collection strategies to

strengthen data quality. In addition to this continuous review of data quality, at least two

internal data quality assessments (DQAs) will be conducted by the Home Office M&E

Specialist during the project following the format outlined in Annex III: Data Quality

Assessment Checklists. The internal DQA described here is an internal quality control

mechanism and does not substitute for any formalized, USAID-initiated DQA of project data.

Step 3: REVIEW

Technical Staff & Landscape M&E Specialists

Makes necessary changes to incorrect/incomplete data reviewed by

ME&L Coordinator.

ME&L Coordinator Reviews all data against ADS 203 data quality standards

COP Ensures that at least two (2) internal Data Quality Assessments

(DQAs) are conducted during the project.

TT ARD Home Office M&E Specialist

Conducts two (2) internal DQAs during the project.


MANAGE: Following the quality review, the Jakarta M&E Specialist will assist

ME&L Coordinator to enter disaggregated data into the LESTARI Management

Information System (MIS). For each data point entered, supporting

documentation (e.g., sign in sheets, photographs, meeting agendas, etc.) must

be provided.

Step 4: MANAGE

Jakarta M&E Specialist Inputs data into MIS

Ensures each data point is supported by documentation

Files hard and electronic data documentation

Maintain MIS in day to day basis

ANALYZE: On a quarterly basis, LESTARI’s ME&L Coordinator will

review disaggregated data to assess progress toward targets and identify

implementation challenges. When analyzing LESTARI data, the ME&L

Coordinator will consider both INTERNAL and EXTERNAL factors that

may be influencing results. Internal factors include challenges or

opportunities related directly to project implementation (for example, staff

turnover) while external factors refer to those outside of the control of

LESTARI staff and USAID (for example, negative factors might include political instability or

natural disasters; positive factors might include new opportunities for collaboration). An

important part of the analysis step is not only providing data to explain the “what,” but also to

provide context to explain the “why.” This context is extremely important in informing how

LESTARI may need to adjust activities. It will also be beneficial to future USAID projects, as

they determine what worked in the past, and why.

Step 5: ANALYZE


Work with the ME&L Coordinator to contextualize the results: What

other INTERNAL and EXTERNAL factors (risks and challenges, or

enabling events or opportunities) may be influencing our results?

ME&L Coordinator Assesses progress toward targets

Analyzes quantitative and qualitative data to tell the story of project

successes, challenges, and lessons learned.


REPORT AND COMMUNICATE RESULTS: Once data have been analyzed,

they will be ready for reporting to USAID/Indonesia. The M&E Sections of reports

will include:

Any M&E-related activities (e.g., trainings, data collection activities, internal data quality assessments and findings) conducted that quarter;

An indicator table(s) showing quantitative results achieved that quarter (targets vs. actuals);

A narrative describing overall analysis of project achievements, lessons learned, and implications for implementation;

Project Success Stories, including photographs and quotes from beneficiaries (starting in year two);

Discussion of any data limitations, reporting challenges, and proposed changes to the AMEP.

The findings of any special studies/analyses performed to inform project learning.

While the minimum requirement is an indicator table showing disaggregated actuals against

targets for each indicator, it is sometimes difficult for a reader to easily assess what is/isn’t

working in this format. The goal of the report is to provide a clear picture of what the project

has achieved, how much the project has achieved, and why the project has or has not met

its goals. Whenever possible, the project will use visual representations of data (such as pie

charts, bar graphs, or line graphs) to report project results.

In addition to reporting results to USAID, the ME&L Coordinator will support the development

and dissemination of Success Stories and Lessons Learned with LESTARI partners and

stakeholders at the national and regional levels during annual meetings.

Step 6: REPORT


Provides any additional clarification/information requested by ME&L

Coordinator for inclusion in M&E Sections of Quarterly/Annual Reports

ME&L Coordinator Drafts M&E sections of Quarterly/Annual reports

Assists with development of Success Stories (beginning in Year 2)

COP Reviews M&E Sections of Quarterly/Annual reports for quality prior to

reporting to USAID.


LEARN: A critical step in the adaptive management process is learning.

While the process described above will promote learning throughout all

phases of the project, the ME&L Coordinator will facilitate a formal learning

process at Annual Strategic Reviews. Following annual internal evaluations

conducted by the ME&L Coordinator, Strategic Reviews will seek the

participation of project staff, partners, and USAID to review evaluation data and findings to

address the following questions:

Which activities were successful, and why?

Which activities fell short of their anticipated results? Why?

How could any activity be redesigned to increase efficiencies, effectiveness, scope,

and satisfaction?

How have activities had a positive effect on women and other

vulnerable/disadvantaged groups?

Team members will also review key aspects of the AMEP (theories of change, indicators and

targets) and consider whether these remain aligned with project activities. If the process

reveals misalignment between activities, indicators and expected results; changes in initial

assumptions (or causal logic) supporting the theory of change, unanticipated factors

influencing achievement of results, or changes in scope, elements of the AMEP may need to

be refined or redesigned.

Table 5 Annual Strategic Reviews

Question Response

Decisions Made/

Actions Required

Items Not Resolved from

previous period- why?

Which activities during previous quarter were successful, and why?

What activities were planned but didn’t occur?

If they did not occur, why?

For activities planned but did not occur, when will they be re-programmed?

Are we achieving our performance indicators targets for key performance indicators?

If we are not, why?

What activities can be planned for the following year to meet our performance indicator targets?

Are our target assumptions still valid? If not, why?

Which activities are falling short of their anticipated


Question Response

Decisions Made/

Actions Required

Items Not Resolved from

previous period- why?

results and why?

Is the causal logic of the theories of change still valid? If not, why?

Are our assumptions related to our ability to implement Strategic Approaches and meet our performance indicators targets still valid?

If not, what has changed?

What effects do these changed assumption(s) have on the project?

What is our strategy to address any changed assumption(s)?

Are there any “red-flags” if so; are there any corrective actions that need to be taken?

Step 7: LEARN


Participate in Annual Strategic Reviews

Contribute useful contextual information that contributes to the

learning process

ME&L Coordinator Facilitate learning sessions at Annual Strategic Review

Document and disseminate the findings to project staff

Include the findings in the next Quarterly/Annual Report

COP Ensure that findings from learning forums are included in annual work

plans and are considered in the development of activities.


REFINE: If the learning process results in recommended changes to the

AMEP, the ME&L Coordinator (under the guidance of the COP), will

document any major changes in the Executive Summary of the revised

AMEP along with the rationale for these adjustments. Major revisions could

include changes to frameworks, indicators, or the PIRS. If minor AMEP

elements change, the AMEP will be updated to reflect these changes. While

required changes to the AMEP may become apparent at any point in the project cycle, the

project must continue to follow the standards and methods outlined in the current AMEP until

the revised document is officially approved by USAID. Once approved, the project may then

follow the new AMEP standards and protocols throughout the adaptive management

process.

Step 8: REFINE


Review AMEP elements

Recommend changes to AMEP based on project learning

Recommend changes to Work Plan and/or activity implementation

based on project learning

ME&L Coordinator Review AMEP elements

Recommend changes to AMEP based on project learning

Recommend changes to Work Plan and/or activity implementation

based on project learning

Make necessary changes to the AMEP

COP Review and approve recommended changes to AMEP prior to

submission to USAID

Submit revised AMEP/Work Plan to USAID.

3.2 Internal Evaluations Plan Evaluation is an applied inquiry process for collecting and synthesizing evidence that

culminates in conclusions about the state of affairs, value, merit, worth, significance, or

quality of a program, product, person, policy, proposal, or plan. Conclusions made in

evaluations encompass both an empirical aspect (that something is the case) and a

normative aspect (judgement about the value of something). It is the value feature that

distinguishes evaluation from other types of inquiry, such as basic science research, clinical

epidemiology, investigative journalism, or public polling (Fournier, 2005, p.140)

In accordance with the 2011 USAID Evaluation Policy, LESTARI will use evaluation

methodologies with the dual purposes of accountability to USAID, government partners and

other stakeholders; and learning about key results to improve effectiveness. In the simplest

terms, evaluations planned for LESTARI will answer three questions: What? So what? Now

what? The answers of these questions will not only aim to understand project effectiveness

but also to generate new knowledge for the wider biodiversity and climate change


community. It is worth noting that evaluation for LESTARI is not an end in itself. The purpose

is to inform thought and action. Moving from what, to so what, to now what means moving

from data to interpretation to action.

Regular monitoring of LESTARI performance indicators will provide the project with the

answers of what questions as well as demonstrate accountability. However, several key

impact questions will need to be assessed to answer the so what question and track

changes over time so as to LESTARI can determine its effectiveness. An example for this

case could be: a national park has increased its METT score to 70 point (what), but does this

score reflect on improved biodiversity condition? (so what). If yes, what can we learn to

replicate the success (now what). If not, how can we modify our approach? (now what).

Internal evaluations for LESTARI will be designed in a way that meet the principles of USAID

evaluation policy (i.e., unbiased, relevant, based on the best methods, transparent, high

utility) and address the most important questions about project impact, how LESTARI is

impacting the direct threats and the key species of interest. Tools, methods and design

for evaluating LESTARI performance indicators will be developed and agreed upon in a

consultation with LESTARI COR on a timeframe that allows for LESTARI interventions to

demonstrate impact. These evaluations will be performed during mid and end of project life,

and timed in order to ensure that they are completed before and thus can contribute to

external mid-term and final evaluations. The SOW for the mid-term internal evaluation will be

developed with the USAID COR by the end of the second quarter of Year 2, and the

evaluation will be completed and written-up by the end of Year 2. The SOW for the final

internal evaluation will be developed with the USAID COR by the end of the first quarter of

Year 5, and the evaluation will be completed and written-up by the end of the third quarter of

Year 5. In both cases, adjustments could be made to ensure these internal evaluation

documents are available in time for external evaluations.

Table 6 presents the illustrative evaluation tools, suggested methods and designs relate to

the evaluation questions posed. The proposed key evaluation questions (KEQs) are selected

to assess the validity of assumptions illustrated in LESTARI Results Chains (Annex II).


Table 6 Illustrative summary of tools, methods and design implications for proposed evaluation questions

Key Evaluation Question Suggested tools, methods, and designs Timing of Data Collection

To what extent have LESTARI interventions

been effective in reducing biodiversity threats

(e.g., poaching, fragmentation of key habitat)

and positively impacted populations of key

identified species?

Collecting data from population survey done by WCS,

WWF, or other organization (e.g. BOSF). Perform

statistical analysis to determine a causal relationship

between LESTARI interventions and changes in

populations. When data on population is not available,

LESTARI may want to consider funding population

surveys for targeted key and indicator species (e.g.,

tiger and orangutan).

Mid-term (2017) and End of Project (2020)

Note:

Population survey is available every three years

depend on the key species assessed (e.g. a

WCS population survey for tiger will be

conducted in 2016).

Impact evaluation – detailed land use and land use

change (LULUC) assessments to determine the impact

of LESTARI interventions on key habitat.

To what extent has improved collaboration

between government, private sector and

community contributed to improved forest

management?

Are economic benefits sustained by

LESTARI beneficiaries –government,

private sector and community?

Are economic benefits sufficiently

incentivizing those beneficiaries to

minimize their encroachment of forests?

Cost-benefit analysis Mid-term (2017) and End of Project (2020)

To what extent have LESTARI intervention

been effective in improved land-use

governance?

To what extent has the project addressed

barriers of political will and buy-in at the

national and regional levels?

A combination of tools such as:

Assessing land cover and policy changes

Stories of change – a case study method to determine pathways of success

Bellwether method – an interview method to determine an issue’s position on the policy agenda

Stakeholder analysis – method to determine which stakeholders are invested in the intervention

Stories of change, bellwether method, and

stakeholder analysis will be used in regular basis

(rolling as data available) to monitor the process

of policy influence.

The data collected from the tools will feed into

QCA and Contribution Analysis which will be


Key Evaluation Question Suggested tools, methods, and designs Timing of Data Collection

What changes have been made to land use policy as result of advocacy interventions and strengthening citizen based mechanism input?

Have beliefs, opinions, and behaviours of stakeholders changed as result of advocacy and campaign activities?

Have improved land use governance impacted on reduction in large scale and small scale deforestation and degradation?

Qualitative Comparative Analysis (QCA)4 and other theory-based evaluation design, e.g. contribution

analysis5 will be used to gain a better understanding of project attribution. QCA is a middle way that combines certain features of the qualitative approach (case orientation, interest in complexity) with those of quantitative research (interest in generalization). An example of QCA utilization can be seen further in CIFOR research that aims to compare national REDD+ policy processes in 12 countries

(http://www.cifor.org/library/4278/qualitative-comparative-analysis-qca-an-application-to-compare-national-redd-policy-processes/).

used in mid-term and end of project evaluation.

Knowledge generation: enhance general

understandings and identify generic principles

about effectiveness

Meta-analyses

Lessons learned

Case studies

Mid-term and end of project year

4 QCA is an alternative approach to examining contributory causes that uses a comparative framework perspective. QCA provides a credible causal claim that the intervention is a contributory

cause. QCA can answer the question ‘Did it make a difference?’, but not the question ‘Why has the result occured?’ 5 Contribution analysis is one of theory-based approaches to evaluation. Contribution analysis confirms: a) that the expected result occured; b) that the causal package is sufficient; c) that the

intervention is a necessary part of the causal package.

http://www.cifor.org/library/4278/qualitative-comparative-analysis-qca-an-application-to-compare-national-redd-policy-processes/




3.3 Schedule for Performance Monitoring Tasks Table 7 Key Performance Monitoring and Evaluation Tasks and Timeline

Tasks FY 2015 FY 2016 FY 2017 FY 2018 FY 2019 FY 2020

Responsible

Quarter: Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3

Submit Draft/Final AMEP

● ● M&E

Develop MIS and train staff

● ● M&E/HO M&E

Draft M&E Section of Quarterly Report

● ● ● ● ● ● ● ● ● ● ● ● ● ● M&E

Support development of Success Stories

● ● ● ● ● ● ● ● M&E

Draft M&E Section of Annual Report

● ● ● ● M&E

Conduct Internal Evaluation

● ● ● ● ● M&E

Facilitate M&E Session of Strategic Annual Review

● ● ● ● ● M&E

Revise AMEP ● ● ● ● ● ●

M&E

Draft M&E Session of Final Project Report

● M&E

Assess Data Quality

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● M&E

Conduct internal DQA

● ● ● M&E/HO M&E

Review Internal Performance

● ● ● ● ● ● ● ● ● ● M&E

Hold Adaptive Management Meeting

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● M&E

LESTARI Collects Data

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ALL


3.4. Monitoring and evaluation structures, functions, and capabilities Monitoring, evaluation, and learning are core priorities across LESTARI’s technical and

administrative positions. While the ME&L Coordinator (under the direction of the COP) will

hold primary responsibility for developing and implementing the M&E system, all LESTARI

staff must participate in monitoring and evaluation in order for the system to be successful.

This section provides a general description of the M&E roles and responsibilities for

LESTARI staff. Specific M&E roles and responsibilities for each indicator can be found in the

PIRS in Annex I.

Table 8 M&E Roles and Responsibilities

M&E Contributor Roles and Responsibilities

ME&L Coordinator

Provide professional guidance and hands-on assistance to all LESTARI

staff on the LESTARI AMEP; MIS; USAID ADS 203 guidelines; and data

collection techniques, standards, and best practices.

Provide training and technical assistance to local partners in

performance monitoring and information sharing.

With assistance from the HO M&E Specialist, develop and maintain the

LESTARI MIS, including standardized data collection forms, cloud-based

M&E database, and a Performance Monitoring Dashboard.

Assess the quality of collected data and provide feedback to parties

responsible for data collection.

Draft and submit M&E sections of quarterly and final reports to the COP

for submission to USAID.

Conduct special studies and analyses as necessary to inform project

learning.

Lead Quarterly Results Reviews.

Support development of Success Stories and other communications and

outreach materials.

Work with the DCOP and regional and technical staff to ensure that M&E

data are informing LESTARI management and decision making.

Conduct regular data verification activities.

Conduct annual internal evaluations.

Share internal evaluation findings and facilitate review and feedback

sessions with stakeholders at Annual Strategic Reviews.

Facilitate internal impact evaluations, and dissemination of findings.

Jakarta M&E Specialist

Provide inputs to ME&L Coordinator in the development of LESTARI

AMEP; standardized data collection forms; and MIS.

Maintain MIS operation in daily basis and train staff on its use.

Supports Jakarta technical staff to perform their monitoring tasks and



responsibilities.

Maintain up-to-date hard and electronic files of all data and

documentation.

Work with Landscape M&E Specialists to ensure timely reporting on

LESTARI activities.

Landscape M&E Specialists

In the activity design level, provide hands-on assistance to regional team

to develop activity logic align with project level ToC.

Train LESTARI regional team, grantees, and sub-contractors to

accurately and reliability collect data, ensure that data collection forms

and templates are used correctly.

Verify and validate all data including reports from grantee and

subcontractors for submission to ME&L Coordinator.

Provide inputs to ME&L Coordinator for quarterly and final reports.

Under guidance of ME&L Coordinator, conduct special studies and

analyses as necessary to inform project learning.

Provide inputs to ME&L Coordinator for the development of Success

Stories and other communications and outreach materials.

Work with the regional team to ensure timely reporting on LESTARI

activities.

Under guidance of ME&L Coordinator, conduct regular data verification

activities.

Facilitate review and feedback sessions with regional team at quarterly

strategic reviews.

Regional Coordinators/

Technical Staff

Work with the M&E team to gain consensus on how LESTARI activities

will contribute to indicators and objectives.

Collect data and data documentation (photos, GPS coordinates,

agendas, sign in sheets, etc.) using data collection form.

Provide recommendations to the M&E Team on any needed changes to

the AMEP or M&E process, such as revisions to data collection forms,

methods, indicators, or targets.

Participate in forums (Quarterly Results Reviews; Annual Strategic

Reviews) intended to facilitate learning from project data.

Incorporate findings from learning forums into future activities and work

plans.

DCOP The DCOP will be responsible for ensuring that project activities contribute to indicator targets and expected results, are gender-responsive, and that M&E data and findings are used to inform planning and decision making.

Tetra Tech Home Office M&E Specialist (Robynne

In Year 1, Tetra Tech’s home office M&E Specialist, Ms. Robynne Locke, will be responsible for assisting with development of the AMEP and providing training to the M&E team on data collection form development and AMEP



Locke) operationalization. Thereafter, she will provide training and technical assistance to the M&E team as needed. In addition, she will be responsible for conducting at least one internal DQA within the first two years of the project.


ANNEX I: PERFORMANCE INDICATOR REFERENCE SHEETS (PIRS)

INDICATOR 1

GOAL Reduced GHG emissions from land-use sector

RESULT KR 1. At least 41% of total CO2-equivalent emissions reduced from land use, land use change and deforestation averaged across all landscapes within the project scope

INDICATOR TITLE

Percentage reduction in GHG emissions as a result of USG assistance measured using actual emissions compared to REL

(For the purpose of FACTS reporting, quantitiy of GHG emissions will be reported in metric tons of CO2equivalent)

PERFORMANCE INDICATOR DESCRIPTION

INDICATOR LEVEL Outcome

INDICATOR TYPE Custom Indicator – contributes to FACTS 4.8-7

PRECISE DEFINITION

This indicator reports the percentage of GHG emissions reduced, sequestered, and/or avoided, as a result of USG activities, as compared to a baseline level of GHG emissions. The baseline is the “business-as-usual” reference for GHG emissions that would have occured during the reporting period if there had been no USG intervention. At least 41% of total CO2-equivalent emissions will be reduced from land use, land use change and deforestation averaged across all landscapes within the project scope, based on the use of the IPCC Good Practice Guidance for Land Use, Land-Use Change and Forestry, and taking into consideration appropriate relevant national or subnational methods.

Reduced emission will be achieved through LESTARI direct assistance, originating from reducing forest encroachment and improving peat land management with communities and protected area authorities, developing effective communities conservation agreements and better agricultural practices, engaging the private sector to implement best management practices in their concessions, and governance advocacy in changing licensing and embracing sustainable development visions that will stop deforestation.

It is noted that for the purpose of FACTS reporting, quantity of GHG emissions will be reported in metric tons of CO2equivalent.

UNIT OF MEASURE Percentage (and metric tons of CO2equivalent)

DISAGGREGATIONS Landscape

RATIONALE OR JUSTIFICATION FOR

Indicator used to track progress on reducing emissions from land use sector. The indicator is in line with GoI target to reduce emissions nationally as outlined in


INDICATOR 1

INDICATOR National Action Plan for Greenhouse Gas Emissions (RAN-GRK).

DESIRED DIRECTION

Decreasing

LINKAGE TO LONG TERM OUTCOME OR IMPACT

Reductions in deforestation and degradation will lead to reduction in GHG emissions. Reducing, sequestering, or avoiding GHG emissions will slow the rate of climate change and reduce the impacts.

DATA COLLECTION APPROACH

DATA COLLECTION

METHOD

The data collection methodology for calculating reduced emissions is described in the Landscape Baseline Analysis (LBA) report. LESTARI has adopted the exact same methodology of GoI for calculating land-based emissions in an attempt to gain the maximum level of buy-in from the local stakeholders, especially the government. Detailed methodology can be found in the BAPPENAS Technical Guide for Land-Based GHG Emission and Sequestration Baseline Calculations6 that is followed by the RAN/D-GRK. The methodology applied to LESTARI that works in the forestry and land-use sector is based on:

Living biomass (above ground) GHG emissions

Below ground GHG emissions from the decomposition of organic soils (peat)

GHG emissions from organic soils through land fires

The REDD Abacus SP program developed by World Agroforestry Center (ICRAF) is used to calculate land cover changes and related emissions. This includes below ground emission estimates on peat due to land use.

It is acknowledged in the LBA that the methodology may require revision or adaptation going forward. Any changes in the calculation methods for land-based emissions across LESTARI landscapes will be presented in the LESTARI Landscape Baseline Analysis (LBA) for approval by USAID.

DATA SOURCE The current methodology uses land cover data from Ministry of Environment and Forestry and MODIS hotspot (subject to change).

FREQUENCY AND TIMING OF DATA ACQUISITION

Annual

ESTIMATED COST OF DATA ACQUISITION

Partial cost of remotely sensed imagery allocated to this activity, ground check, staff time for data analysis

INDIVIDUAL RESPONSIBLE

Spatial Planning and GIS Specialist; Deputy Chief of Party

DATA STORAGE LOCATION

LESTARI MIS; with supporting files kept at the LESTARI Office in Jakarta

DATA QUALITY

6 F. Agus, I. Santosa, S. Dewi, P. Setyanto, S. Thamrin, Y. C. Wulan, F. Suryaningrum (eds.). 2013. Pedoman Teknis

Penghitungan Baseline Emisi dan Serapan Gas Rumah Kaca Sektor Berbasis Lahan: Buku I Landasan Ilmiah. Badan

Perencanaan Pembangunan Nasional, Republik Indonesia, Jakarta.


INDICATOR 1

INITIAL DQA TBD

DATE OF FUTURE DQA

Internally by the Tetra Tech M&E team in June 2016

PROCESS FOR FUTURE DQA

The project will employ a system of continuous adaptive management and therefore will audit data on a systematic basis before input into the project’s MIS. For each data point reported to USAID, supporting documentation will be identified and reviewed using the process outlined in USAID’s DQA Template. USAID LESTARI’s ME&L Coordinator will ensure that each data point is supported with documentation and that data are assessed against data integrity standards as outlined in ADS 203.3.11.1.

KNOWN DATA LIMITATIONS

The methodology for estimating emissions from fire is not universally

accepted due to the reliability of data used in calculating emissions. The

intensity and duration of fires, depth of peat, water table depth, drought

conditions and vegetation cover all affect the potential emissions from peat.

With all of these variables, it is unlikely that a single method for calculating

emissions from fire will ever be available.

The baseline data used in the analysis of deforestation and degradation and

associated GHG emissions uses Ministry of Environment and Forestry

(MOEF) wall-to-wall land cover (penutupan lahan) data that has

questionable accuracy in some parts of LESTARI landscapes – especially

Papua. GIS operators have frequently misclassified land use types and this

is the most problematic.

Another issue that needs some effort in rectifying and aligning is the

difference between peatland data (sourced from Soil Research Centre,

Ministry of Agriculture) and land cover (MOEF). For instance, the incidence

of dry land forest types occuring on peat in questionable. It is hard to state

which data set is more accurate. At the scale of LESTARI landscape, this is

not a major issue for the project, but administrations wishing to report on

their site level emissions reductions should seek accurate data. Where

unlikely combinations of land cover types and soils do not align, estimation

of emissions should be treated with caution.

Emissions reductions are calculated using a tool that relies on data of

carbon stocks, averaged across the country. If the government figures for

carbon stocks change (very likely within LOP) or Tier 2 or 3 values are used

to calculate a more accurate REL (baseline) then the tons may go up or

down. So depending on the methodology used the actual amount (metric

tons of CO2e) may go up or down. This makes setting LOP target in exact

figure of metric tons of CO2e is not feasible.

ACTION TAKEN TO

ADDRESS

LIMITATIONS

To calculate emissions from fire, compromises must be made and average

emissions from areas using available monitoring data have been applied to

develop the LESTARI baseline. This is in line with provincial government

approaches. Forest and land fire is a significant issue in Central Kalimantan

and LESTARI has applied methodology adopted by the Central Kalimantan

Province.

Ground check and verification with stakeholders (especially established


INDICATOR 1

Spatial Data Infrastructure groups and networks that have been established

through USAID IFACS) will be undertaken to ensure accuracy of land cover

data.

LESTARI should seek to obtain accurate peat distribution data and redefine

estimated emissions.

LESTARI will use percentage of reduction as project target, but will provide

information in metric tons of CO2-equivalent to serve USAID global reporting

purpose.

DATA ANALYSIS REPORTING AND REVIEW

DATA ANALYSIS Data analysis will be done by Spatial Planning and GIS Specialist with assistance from Winrock following USAID approved methodology

PRESENTATION OF DATA

Tabular by landscape

Landscape Target Achievement % Completion

xx % xx %

xx tCO2-eq xx tCO2-eq

REVIEW OF DATA Annually by ME&L Coordinator

REPORTING OF DATA

Annual Report

BASELINE AND TARGETS

BASELINE DATA SOURCE

LESTARI Landscape Baseline Analysis report. Baseline calculated following GoI methodology for Land-Based GHG Emission and Sequestration Baseline Calculations.

BASELINE NOTES The methodology used to calculate the baseline for LESTARI is based on the national approach. It is expected that this national level methodology will be improved in the future. For the LESTARI project, methodology used must evolve for the simple reason of delivering achievement at the end of the project. The data used for this methodology (land cover based on Landsat Imagery and developed by the MoEF) is normally available to the public at least 2 years after it was captured. Therefore, a methodology that can utilize up-to-date data taken within the last year of the project must be sought.

BASELINE VALUE

LESTARI applied the base years of 2006 – 2013. The table below presents a summary of annual deforestation and degradation rates and land-based GHG emissions per landscape.

Landscape Annual Deforestation

(ha/yr)

Annual Degradation

(ha/yr)

Net Annual GHG emissions/REL

(tCO2-eq)

Leuser 8,039 0 6,531,315

Katingan- 34,136 5,236 42,630,563


INDICATOR 1

Kahayan

Lorentz Lowlands 2,551 15,513 11,407,378

Mappi – Bouven Digoel

1,118 29,275 9,915,365

Sarmi 353 4,560 996,222

Cyclops 144 11 83,249

BASELINE YEAR 2006 - 2013

TARGETS NOTES The target set for LESTARI averaged across the landscapes is a 41% reduction from the baseline emissions by 2020. LESTARI proposes a progressive approach—increasing the reductions of to reach this target by 2020. Annual emissions will be reduced progressively by 10%, 20%, 30%, 40% and 41% each year of the project. But emissions reductions will only start to be reported in the third year of the project.

YEAR TARGETS ACTUAL OBSERVATIONS

FY 2016 Not measured Not measured

FY 2017 Not measured Not measured

FY 2018 30% reduction 41 M tCO2-eq % reduction M tCO2-eq



Targets LOP 41% reduction 96 M tCO2-eq % reduction M tCO2-eq

NOTES ABOUT CHANGES TO THE INDICATOR

LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: November 30th, 2015


INDICATOR 2

GOAL Carbon rich forest, peatland, mangrove ecosystems and the habitat of key species maintained

RESULT KR 2. At least 8.42 million hectares of primary or secondary forest, including orangutan habitat, under improved management

INDICATOR TITLE Number of hectares of biological significance and/or natural resources under improved natural resource management as a result of USG assistance



INDICATOR TYPE FACTS 4.8.1-26

PRECISE DEFINITION

“Improved natural resource management” includes activities that promote enhanced management of natural resources for one or more objectives, such as conserving biodiversity, sustaining soil or water resources, mitigating climate change, and/or promoting sustainable agriculture. Management should be guided by a stakeholder-endorsed process following principles of sustainable NRM and conservation, improved human and institutional capacity for sustainable NRM and conservation, access to better information for decision-making, and/or adoption of sustainable NRM and conservation practices.

For LESTARI, an area is considered under "improved management” when any one of the following occurs:

1. Co-management plan among Provincial and District Government Community, and Forest Management Unit (FMU) are in place and under implementation.

2. FMUs assisted by LESTARI operational.

3. Certain area set-aside for conservation as result of SEA-LEDS and/or LCP recommendations into local development plans.

4. An increased in the Management Effectiveness Tracking Tool (METT) score for Protected Areas (PAs).

5. Conservation Management and Monitoring Plan (CMMP) or other conservation plan within private sector natural resources concessions are in place and under implementation as evidenced by incorporation in business SOP, company budget and monitoring reports (or through sustained certification audits implemented by a third party).

6. Adaptive management is demonstrated; or on-the-ground management impacts are demonstrated (eg. illegal roads closed, snares removed, no-fishing zones demarcated).

Reported as total number of hectares improved during the fiscal year in question, which can include maintained improvement in previously reported hectares and/or new, additional hectares. Improved management should be reported for activities where the LESTARI supported program was plausibly linked to the improvements observed.

UNIT OF MEASURE Hectares

DISAGGREGATIONS

Areas under improved management will be disaggregated by landscape following further disaggregation below:

Biologically significant areas = areas identified as important for biodiversity through national, regional, or global priority-setting processes. For LESTARI, these


INDICATOR 2

areas will include:

Orangutan habitats (note: if orangutan habitats are inside conservation areas, the hectares will be excluded from hectares of conservation areas under improved management to avoid double counting)

HCV in concession

Conservation Areas (e.g., national parks, wildlife reserve)

All other areas = areas with natural resources which are outside of biologically significant areas and targeted for management interventions with non-biodiversity funds. These may include areas characterized by forest production, watersheds, areas with tree crop or agroforestry systems, peatlands (high carbon stock), forested areas within and adjacent to private sector concessions, etc.

Areas under IFACS and new areas under LESTARI.

RATIONALE OR JUSTIFICATION FOR INDICATOR

Measures of this indicator demonstrate progress towards sustainable natural resources governance and institutions, and can inform adaptive management of programs. This indicator is a reliable annual measure that demonstrates the magnitude of USG investments in biodiversity conservation and other natural resource sectors.

DESIRED DIRECTION

Increasing


A spatial indicator is an appropriate measure of the scale of impact of biodiversity conservation and/or NRM interventions. Good management of natural resources is a prerequisite for achieving improved biophysical condition of natural resources and contributes to reduction in deforestation and degradation.


DATA COLLECTION METHOD

Field staff will compile for review implementation of conservation management plans (PA and non-PA), review district and provincial spatial plans and land use policy process (to identify changes in legal status), site-observation. The GIS team will conduct geospatial analysis of remotely sensed imagery and GIS layers.

DATA SOURCE GIS, conservation management plans, site-visit reports, land cover maps from Ministry of Environment and Forestry (MoEF), spatial plans, procurement of remotely sensed imagery (when feasible).


Annual


Partial cost of remotely sensed imagery allocated to this activity, ground check, staff time for data analysis and interpretation, and cost of photocopies for conservation management plans and spatial plans.


Spatial Planning and GIS Specialist; Component Team Leaders



DATA QUALITY


INDICATOR 2

INITIAL DQA TBD

DATE OF FUTURE DQA





Precision is low: “improved management” is a relative term, and narrative is required to explain the quality of this management improved. Equal weight is given to unequal improvements along a continuum: eg. creating, adopting and implementing management plans may each be an improvement over a baseline. Likewise, a small management improvement across a large area may be as important as a large improvement across a small area. Conservation plans may not be well described spatially.

ACTION TAKEN TO ADDRESS LIMITATIONS

Conservation plans will be interpreted by trained LESTARI staff. Locations of all activity areas carried out by partners will be located by LESTARI field staff using GPS and recorded into electronic data form. Benchmark that are being used within the project to gauge success will be articulated clearly and a short narrative to describe the benchmarks that have been reached in the past year will be provided.


DATA ANALYSIS

Field staff will compile for review conservation management plans, maps and district spatial plans. The GIS team will conduct geospatial analysis of remotely sensed imagery and GIS layers. Landscape hectare totals will be aggregated into a tabular format, with accompanying narrative. The link to USG assistance will be established through periodic assessments conducted by LESTARI staff.

For those areas already improved under IFACS, LESTARI will report these areas as project achievement should there is new level of improvement made under LESTARI. Narratives will be provided to describe new level of improvement. Data presentation will be disaggregated between improved areas under IFACS (e.g., concession area, area under Community Conservation and Livelihood Agreements, etc) and new areas under LESTARI.

Analysis will also be made to present data on orangutan habitats. If orangutan habitats are inside conservation areas, the hectares will be excluded from hectares of conservation areas under improved management to avoid double counting.


Tabular

Landscape Type of Area Forest Management Type

Location Target Achievement % Completion

Biologically significant areas (divided into areas under IFACS and new areas


INDICATOR 2

under LESTARI):

Orangutan habitat

HCV in concession

Conservation Areas

All other areas (divided into areas under IFACS and new areas under LESTARI):


REPORTING OF DATA

Annual Report



LESTARI Landscape Baseline Analysis report, METT baseline report, existing spatial planning and project activity progress report to determine the level of current management.

BASELINE NOTES Project activities to improve current management will be started in FY 1, therefore baseline for this indicator is assumed as zero because no improved management yet observed. Analysis of existing condition will include business-as-usual scenario in order to be able to attribute to USG assistance.

BASELINE VALUE 0

BASELINE YEAR 2015

TARGETS NOTES Hectares under improved management will only start to be reported in the second year of the project with the assumption that sufficient time is allowed for impact to occur.


FY 2016 Not measure

FY 2017 1.7 M

FY 2018 2.5 M

FY 2019 3 M

FY 2020 1.5 M

Targets LOP 8.7 M


INDICATOR 2


LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: January 15, 2016


INDICATOR 3

GOAL Reduced threats on key species

RESULT IR 1. Improved forest management

INDICATOR TITLE Percentage reduction in poaching in focus area



INDICATOR TYPE Custom Indicator

PRECISE DEFINITION

Poaching is the illegal hunting, killing or capturing of animals, a practice that occurs in a variety of ways. Poaching can refer to the failure to comply with regulations for legal harvest, resulting in the illegal taking of wildlife that would otherwise be allowable. Examples include: hunting, killing or collecting wildlife that is listed as endangered by IUCN and protected by law such as CITES, taking without a license or permit, use of a prohibited weapon or trap, taking outside of the designated time of day or year, and taking of a prohibited sex or life stage. Poaching can also refer to the taking of animals from a gazzetted wildlife sanctuary, such as a national park, game reserve, or zoo. Wildlife biologists and conservationists consider paoching to have a detrimental effect on biodiversity both within and outside protected areas as wildlife populations decline, species are depleted locally, and the functionality of ecosystem is disturbed.

Focus area are defined as the area within protected areas targeted for regular patrols due to the high level of threats (e.g., accessibility). Patrols are targeted and not all areas need to be patrolled. To monitor a reduction in poaching, LESTARI will use SMART (Spatial Monitoring and Reporting Tool). SMART is designed to improve anti-poaching efforts and overall law enforcement effectiveness in established conservation area management zones. SMART enables the collection, storage, communication, and evaluation of data on: patrol effort (e.g. the time spent on patrols, effective area patrol and the distance of patroling), patrol results (e.g. number of snares removed, arrests made), and threat levels (e.g. the number of paths into the protected areas, areas deforested or number of tress felled illegally).

UNIT OF MEASURE Percentage

DISAGGREGATIONS Conservation Area


Indicator used to track increased commitment of key stakeholders regarding the protection of key species and conservation of their habitat.

DESIRED DIRECTION

Decreasing


When effectively monitored and enforced, reduce in poaching can help to substantially improve protection of wildlife and their habitats.



INDICATOR 3


Review regular patrol reports and patrol maps (monthly or quarterly); undertake in-depth data analysis with an evaluation of various trends in patrol perfromance and threat-levels, as well as an evaluation of the entire patrol management system (annually).

DATA SOURCE Patrol reports; direct observation; annual evaluation


Rolling as data available


Staff labor costs; monitoring and evaluation cost


Biodiversity Conservation Advisor



DATA QUALITY

INITIAL DQA TBD

DATE OF FUTURE DQA





Using the SMART software and establishing a patrol database will not, on its own, improved protection of a conservation area. In addition to the SMART software and database, basic enforcement capacity and infrastructure must be in place.


LESTARI will combine the utilization of SMART with introduction of adaptive patrol management practices through promotion of feedback mechanisms between managers and rangers to discuss patrol efforts and results and set new patrol targets. This intervention is closely linked with METT intervention as to support overall improvement of conservation area management capacity.


DATA ANALYSIS

SMART allows the calculation of poaching incidence, per unit of effort – such as the number of snares removed per kilometer patrolled in high risk areas, and therefore can be used to show a change in paoching incidence even if the number of patrols varies over time. During the first 6 months, SMART patrols and observation in the field will identify areas of high risk, and develop a baseline of the average number of incidents per kilometer traveled on foot patrols. The base


INDICATOR 3

unit will be no. incidents / km measured semi-annually.

Application of effort-related base units is due to the fact that the number of poaching incidents is correlated with patrol effort. The ore one patrols, the more incidents are found. With sustained patrol, however, the assumption is that the incidents will start to decrease as the law is enforced. A semi-annual evaluation to determine trends in poaching levels will be made, as well as semi-annual evaluations of the threat levels within conservation areas to maintain targeting of patrol effort in high risk areas.

The reduction in poaching will only be reported for areas where patroling is programed and implemented on a regular basis. The individual Poaching reduction will be reported individually for each conservation areas and for the project as a whole. If different protected areas are including in the anti-poaching effort facilitated by the project, overall effort in patroling, a weighted average will be applied to report the LESTARI project-wide poaching redcution effort.


Tabular with supporting narrative, as appropriate

Landscape Conservation Area

Target Achievement % of Completion

REVIEW OF DATA Semi-annually by ME&L Coordinator

REPORTING OF DATA

Annual Report



SMART patrol result in Year 1

BASELINE NOTES LESTARI will undertake baseline in Year 1 to determine number of current poaching incident in targeted conservation areas.

BASELINE VALUE TBD

BASELINE YEAR 2015

TARGETS NOTES Numbers of poaching will be reduced progressively by 10%, 20%, 30%, and 40% each year of the project compared to baseline. But achievement will only start to be reported in the second year of the project.


FY 2016 Not measure

FY 2017 10%

FY 2018 20%

FY 2019 30%


INDICATOR 3

FY 2020 40%

Targets LOP 40%


LAST SHEET UPDATED BY: Erlinda Ekaputri, ME&L Coordinator DATE: February 3, 2016


INDICATOR 4

GOAL Reduced rate of deforestation and degradation

RESULT

IR 2. Improved land use governance

KR 6. Increased commitment of key private sector, government, and community stakeholders regarding the positive benefits of conservation and sustainable use of forests and the species they encompass

KR 7. Policies, laws, regulations, and procedures in support of low emission development and forest conservation and management increased, promulgated, and enforced at all levels

INDICATOR TITLE Number of public policies addressing climate change and/or biodiversity conservation introduced, changed or adopted consistent with citizen input



INDICATOR TYPE Custom Indicator – contributes to FACTS 2.4.1-12 and FACTS 4.8.2-28

PRECISE DEFINITION

Public policies include any law, regulation, policy or similar directive that is formally adopted by either the legislative branch or a unit of the executive branch at any level.

Citizen inputs means that the public, citizens and/or civil society organizations have proposed language used in, provided comments incorporated into, or monitored the implementation of the policy.

Introduced refers to draft legislation formally being presented and accepted for consideration by a legislative body.

Changed refers to an existing policy that has been substantively changed.

Adopted refers to the process by which a public policy, whether a completely new policy or one that has been changed, is formally adopted through decree, passage of law or other formal process.

Policies may be district-wide if they cover LESTARI landscapes, e.g., acceleration of community forestry licensing in support of the national goal of 12.7 million hectares. Policies may explicitly target LESTARI landscapes, e.g., collaborative management of Cyclops.

For interpretation of this indicator, a qualitative description will be provided to explain what the number represents, particularly:

What is the title of the measure?

At what stage is it? (e.g., officially introduced, changed, adopted, or implemented?)

How does the measure contribute to climate change mitigation or biodiversity conservation?

What is/are the institution(s) that will be implementing and/or enforcing the measure and at what scale (e.g., national, provincial, district)

UNIT OF MEASURE Number of public policies

DISAGGREGATIONS District; Provincial; and National

Climate change policy and biodiversity policy


INDICATOR 4


The indicator is used by LESTARI to ensure that there is willingness from local government to adopt input from citizen-based mechanisms, or in other word, to ensure that development policies are responsive to the public.

DESIRED DIRECTION

Increasing


Democratic governance concerns are integral to improving policy reform and implementation across a wide range of development sectors. More effective policy change across development sectors requires government policy makers (in both the executive and legislative branches) to consider and incorporate citizen input into the policy development process. Also, an improved enabling environment through legal and policy reform is essential for ensuring that efforts and investments in climate change and/or biodiversity conservation have legal and strategic backing and institutional ownership.



Review project reports, policy documents, tracking policy development process, direct observation, analysis of secondary data and third party observation (i.e. monitoring of public policy by local NGO).

DATA SOURCE Activity reports, policy documentation, assessment reports, policy monitoring reports


Rolling, as data are available


Staff labor costs; policy monitoring cost; photocopies of documents


Decentralized Governance Specialist; Advocacy Campaign Specialist



DATA QUALITY

INITIAL DQA TBD

DATE OF FUTURE DQA




KNOWN DATA There are some limitations to validity since the quality of citizen input is very subjective. Reliability is also of concern since policy reform is not always a linear


INDICATOR 4

LIMITATIONS process. Because the intended result is an improved enabling environment, the number of public policies provides only a partial measure of success, given that effective implementation and enforcement are also critical. Public policies might also not be well-designed or effective.


Narrative is critical for interpreting this indicator. Also if the policies may affect climate or biodiversity indirectly, LESTARI will provide narrative to explain the connection.


DATA ANALYSIS

A citation analysis method will be used to evaluate “uptake and use.” This method involves looking at the extent to which citizen recommendations are visibly “picked up” and used by local governments. Analysis will include triangulation of secondary data and third party observation (when available). Case study will be deployed to assess the progress made along the way in terms of convening stakeholders, drafting, approving, and implementing/enforcing policies.

It should be noted that the policy introduced, changed or adopted may take concern provincial and national regulations. In this case LESTARI will still count it as a single policy.

Data analysis for this indicator will involve assessing a series of milestones. We will count this indicator achieved once the policy is either introduced, changed or adopted. However, the project will still monitor and report the number of policies that are implemented or enforced. Key analytical questions include: how many (percentage) of policies get adopted? Of those that get adopted, what percentage are implemented? How many of the policies that were adopted and implemented had substantial participatory events as part of the development process?

Type of Policy

Introduced Changed Adopted Implemented



Landscape District, Provincial, National

Target Achievement % Completion

REVIEW OF DATA Quarterly by ME&L Coordinator

REPORTING OF DATA

Quarterly and Annual Report



N/A

BASELINE NOTES For this indicator, the baseline is zero

BASELINE VALUE 0


INDICATOR 4

BASELINE YEAR N/A

TARGETS NOTES It is assumed that in each of 14 districts at least two policies (for instance one concerning climate change and one concerning biodiversity) will be influenced as a result of advocacy activities. However, it should be noted that the intended result is an improved enabling environment. Therefore the numbers provide only a partial measure of success. Effective implementation and enforcement are much more critical which in turn could be reflected through achievement of LESTARI key goals: reducing emissions and conserving biodiversity.


FY 2016 3

FY 2017 5

FY 2018 8

FY 2019 8

FY 2020 4

Targets LOP 28




INDICATOR 5


RESULT

IR 2. Improved land use governance

KR 7. Policies, laws, regulations, and procedures in support of low emission

development and forest conservation and management increased,

promulgated, and enforced at all levels

INDICATOR TITLE Number of sub-national government with improved licensing and permitting mechanism




PRECISE DEFINITION

Sub-national government including but not limited to province, district, FMU.

The term “improved” is taken to subsume the five core principles of good governance. First and foremost transparency, inclusivity/participation (e.g., village voice in license approval), accountability (clear focal point of responsibility for legally-approved licenses and their consequences), responsiveness (responding to public or sectoral complaints) and timeliness.

For new licenses, improved also means that all new requests for licenses and permits for a new concession area for natural resources extraction activities (i.e., mining, oil palm plantation and logging/industrial timber plantation) should follow Standard Operational Procedures (SOP) that developed within the tool based on the government regulation and procedures for granting licenses and permits.

Improved licenses and permits including but not limited to Izin Prinsip, Izin Lokasi, AMDAL, and others according to the type of the private sectors that are operating within LESTARI landscapes.

UNIT OF MEASURE Number of sub-national government units

DISAGGREGATIONS Province, District


The indicator is used by LESTARI to track the progress of strengthening governance process in the respective districts regarding the issuance of forest licenses and permits. The indicator also illustrates the extent to which spatial plans policy enforced.

DESIRED DIRECTION

Increasing


Improved licensing and permitting process will ensure the certainty of land use and transparency, reduce potential conflicts between private sectors and communities, and address driver of deforestation and degradation.



Review the maps indicates existing licenses, case study to assess adoption of good governance principles into licensing and permitting mechanism, review the licensing process and SOP, interview government licensing officers and business enterprises, analysis of secondary data and third party observation (i.e. monitoring of forest licensing by local NGO).


DATA SOURCE Maps, SOP of licensing and permitting process, activity reports, monitoring/ assessment reports




Staff labor costs; case study cost; photocopies of documents


Sustainability Screening Specialist; Private Sector Engagement Coordinator



DATA QUALITY

INITIAL DQA TBD

DATE OF FUTURE DQA





Incomplete regulatory rules and the lack of synergy between different plans (the National Spatial Land use Plan (RTRWN), the Provincial Spatial Land use Plans (RTRWP) and the District Spatial Land use Plans (RTRWK)), opens the door to weak implementation of the regulations and corruption. As a result land may be inappropriately zoned or zones may be changed in order to allow logging or other extraction. Without adequate plans or consultation with local communities, the current zoning of land does not provide a basis on which to make licensing decisions, and therefore such decision-making processes are open to manipulation and undue influence by loggers or extractors. Furthermore, there are no official institutions tasked with monitoring the zoning of land and in the absence of accurate data and maps on logging/extraction zones independent monitoring is difficult.


LESTARI will develop capacity of local CSOs to monitor the forestry license activities through research and analysis supported by advocacy and Geographic information System (GIS) analysis. GIS analysis is used to identify any distortion in the areas for which logging/extraction permits have been given or violations in these areas. Spatial Data Infrastructure (SDI) initiated under IFACS will be strengthened to improve the accuracy, consistency, and availaibility of land use information for public use.


DATA ANALYSIS Technical review will be carried out to determine transparency and accessibility of licensing and permitting information. Also to review if new licenses have followed the SOP/screening process. The analysis will be enriched by qualitative


assessment through interview processes toward government officers and business enterprises. LESTARI will track intermediate milestones when counting progress toward meeting these targets.

Sub-national level

Milestones

Review and align licenses and permits with spatial plan

Screening tool developed - Protocol and procedures for public monitoring developed

Communication and consultation with local government and approval received for screening tool and monitoring protocol

Official public socialization & training for the screening tool

Issuance of licenses and permits follow the SOP and good governance principles



Landscape Sub-national level



REPORTING OF DATA




N/A


BASELINE VALUE 0

BASELINE YEAR N/A

TARGETS NOTES Achievement of target will only start to be reported in the second year of the project because a considerable time is needed to develop SOP and enhance local officers capacity to implement SOP.


FY 2016 Not measure

FY 2017 4

FY 2018 4

FY 2019 4


FY 2020 2

Targets LOP 14




INDICATOR 6


RESULT IR 2. Improved land use governance

INDICATOR TITLE Number of sub-national government incorporating high quality SEA-LEDS & LCPs into draft spatial plans, zonation regulation or draft of mid-term development plan




PRECISE DEFINITION

Sub-national government including province and district.

There are different level of RTRW status in each district working with LESTARI: approved or under development. For the districts with approved RTRW’s, they now enter the 5-year review phase in which zoning (inside Kawasan and APL) through Peraturan Zonasi as well as Kawasan Pedesaan are required by Law 26 of 2007 and PP15 of 2010. This provides a unique opportunity for the LESTARI Landscapes to be regulated not only inside Kawasan Hutan but also APL down to the village level. SEA and LCP indicate where this should be and what should be done but to be actionable zoning at a greater level of spatial detail both inside and outside Kawasan Hutan to include APL will be required, especially important in peatland areas.

Therefore, incorporating SEA-LEDS and LCP recommendations will be done through zonation process. LESTARI will assist local government in such zonation. The results will be zonation maps and rules about what can and cannot be done in the zones towards reducing GHGs and increasing biodiversity conservation. Spatial planning enforcement will be supported by the preparation of the zonation maps and raising public awareness of the zonation areas (where and what allowed / not allowed). The measurable impacts will be the formalization of zoning maps down to the village cluster level (by watershed and as far as possible by sub-district) along with budget and institutional allocations to realize this along with public awareness about the maps in support of greater participation and accountability.

For the districts in which the RTRW are under development, incorporation of SEA/LEDS and LCPs recommendations will be achieved when such recommendations explicitly referred in draft RTRW. This will also be the case for draft RPJMD.

High quality SEA-LEDS and LCP means that the recommendations provide improved-case scenario and not business-as-usual.

UNIT OF MEASURE Number of sub-national government

DISAGGREGATIONS Province, District


The indicator is used to measure progress of improved land use governance. Since both documents were initiated under IFACS project and none of the SEA-LEDS supported by IFACS have been incorporated into district spatial plans due to long periods required for approval across the various levels of government, it is crucial to keep monitoring the process and find other potential entry point to maximize impact through incorporating SEA-LEDS & LCPs into mid-term development plan which will secure financing for implementation.


INDICATOR 6

DESIRED DIRECTION

Increasing


Integrating environmental consideration into strategic decision-making can improve the quality of local planning, ensure sustainability, and change the land-use practices thus contributes to emission reduction.



Technical review of SEA-LEDS and LCPs recommendations and draft of spatial plans, draft of district mid-term development plans and zonation maps/policy. Interviews will be conducted with relevant government officers to evaluate recommendations “uptake and use”.

DATA SOURCE SEA document, LCP document, Spatial Plan document, Mid-term development Plan document, assessment report, zonation maps.




Staff labor costs; assessment cost; photocopies of documents


Sustainable Land Use Planning Coordinator; Forest Governance Advisor



DATA QUALITY

INITIAL DQA TBD

DATE OF FUTURE DQA





This indicator measures SEA-LEDS & LCP recommendations incorporated into draft spatial plans and mid-term development plans but does not measure the number or success of recommendations that have been implemented.


Build sustainability within the process to ensure that the recommendations are accepted and then implemented. Building sustainability is increased as a result of specific and targeted public consultation by LESTARI to bring the community and private sector and local government together to arrive at consensus and


INDICATOR 6

commitment to implement the plan. Incorporationg the recommendations into zonation policy will increase the likelihood of implementation.


DATA ANALYSIS

Citation analysis method will be used to evaluate “uptake and use.” This method involves looking at explicit reference to adopting SEA recommendation of improved-case scenario in draft RTRW and draft RPJM documents or for approved RTRW indicated by zonation maps. Because policy improvement involves a multi-step process, LESTARI will track intermediate milestones when counting progress toward meeting these targets.

Sub-national level

Milestones

High quality SEAs developed or updated

Issuance of Bupati/Head of Dinas instructions for incorporation

Incorporation of SEAs into spatial plans, illustrated by zonation maps, explicitly referred in draft RTRW or draft RPJMDs

Enforcement evidenced



Landscape Sub-national Target Achievement % Completion


REPORTING OF DATA




N/A


BASELINE VALUE 0

BASELINE YEAR N/A

TARGETS NOTES N/A


FY 2016 2


INDICATOR 6

FY 2017 4

FY 2018 4

FY 2019 2

FY 2020 2

Targets LOP 14




INDICATOR 7



INDICATOR TITLE Number of Multi Stakeholder Forum (MSF) operational as citizen based mechanisms for public input on land use




PRECISE DEFINITION

A multi-stakeholder forum (MSF) is a working group of stakeholders with a vested interest in the future of forests, land use, Low Emission Development Strategies (LEDS), improving the livelihoods and future of their constituency, and mitigating and adapting to climate change in the district. MSF contains key decision makers and stakeholders from the public, private, and civil societies.

Operational means MSF is institutionalized and the key stakeholder base broadened, managed by a secretariat with local government budgetary support.

MSF will serve as a bridge between citizens and local government and foster trust-based relationships built upon mutual respect. MSF will be institutionalized with the explicit purpose of tightening the relationship between RTRW and RPJM as well as annual budgeting processes.

UNIT OF MEASURE Number of MSF

DISAGGREGATIONS District


The indicator is used to ensure that all interested people have an opportunity to be heard or to influence government decisions that affect improved forest conservation. Also, this indicator is used to measure the extent to which citizen-based mechanisms for public input successfully strengthened as a result of USG assistance.

DESIRED DIRECTION

Increasing


The success of MSF to bridge communication between local government and wider public in land use will lead to more transparent, participatory and accountable land use decisions. Those principles are prerequisite of good governance.



Review reports from regional technical team on MSF engagement, review the process of public consultation, review Bupati decree of MSF (i.e., clear statement of MSF function), qualitative assessment/case study to learn MSF effectiveness

DATA SOURCE Activity reports, MSF plans, LCPs, Bupati Decree, assessment reports




INDICATOR 7




Stakeholder Coordination Specialist; Advocacy Campaign Specialist



DATA QUALITY

INITIAL DQA TBD

DATE OF FUTURE DQA





Activity reports may not always be complete


Will require supporting documentation and observation to verify results


DATA ANALYSIS

Because operationalization of MSF involves a multi-step process, partial completion will be counted by tracking intermediate milestones.

MSF

Milestones

MSF institutionalized - issuance of Bupati Decree

MSF revitalized (key stakeholder base broadened)

MSF functional with secretariat and local budget support

MSF facilitate change in policy / regulation / Bupati Vision etc)



Landscape District Target Achievement % Completion


INDICATOR 7


REPORTING OF DATA




N/A

BASELINE NOTES MSF became dormant after IFACS. While popular, they were not institutionalized. Part of the problem was that MSF had disproportionate government representation. During LESTARI, MSF will be institutionalized and the key stakeholder base broadened, managed by a secretariat with local government budgetary support. Due to absence of this condition, the baseline is assumed as zero

BASELINE VALUE 0

BASELINE YEAR N/A

TARGETS NOTES LESTARI will develop or strengthen MSF in 14 target districts.


FY 2016 2

FY 2017 4

FY 2018 4

FY 2019 4

FY 2020 0

Targets LOP 14




INDICATOR 8



INDICATOR TITLE Number of community champions engaged in advocacy interventions


INDICATOR LEVEL Output


PRECISE DEFINITION

Community champions are individuals chosen to speak on behalf of their communities. The indicator measures community champions in LESTARI program that initiate or participate in advocacy interventions. Advocacy should be understood as a means for individuals, constituencies, or organizations to shape public agendas, change public policies, and influence other processes that impact their lives.

Advocacy does not involve one march, meeting or poster, but a series of strategic, interconnected, integrated activities designed to achieve goal. For LESTARI, advocacy activities could include lobbying, public interest litigation, letter writing campaigns, monitoring licensing and permitting, etc. Advocacy interventions will be strategic, involve a set of actions that are sustained in order to build and direct pressure, be designed to persuade, be targeted and involve alliance building. Community champions in LESTARI could be involved in MSF and use MSF as channel for advocacy.

Successful advocacy efforts result in change. This indicator is highly connected with Indicator #4 (public policies) and Indicator #5 (more transparent and accountable licensing and permitting).

UNIT OF MEASURE Number of individuals

DISAGGREGATIONS Sex (female/male); District; Landscape


The indicator is used to track the progress of strengthening citizen-based mechanisms for public input on land use. It is also used for measuring the extent to which the project is able to build constituency for conservation.

DESIRED DIRECTION

Increasing


Advocacy interventions are essential aspects of democratic policy making, citizen participation, and oversight of all branches of government. These interventions play an important role in giving voice to citizens and historically marginalized groups. Successful advocacy will improve land use decisions.



Review advocacy plans or strategies, recording of press conference, copy of testimony or press release, advocacy campaign materials, record of public consultation. Case study will be conducted to track the process of advocacy and policy change.

DATA SOURCE Advocacy plans or strategies, press conference, press release, advocacy


INDICATOR 8

campaign materials, minutes of public consultation, assessment report.






Communication and Advocacy Specialist



DATA QUALITY

INITIAL DQA TBD

DATE OF FUTURE DQA





Activity reports may not always be complete


Will require supporting documentation and observation to verify results


DATA ANALYSIS Counting number of individuals chosen as spokespersons and engaged in advocacy activities supported by documentation



Landscape District Target Achievement (disaggregated by Female/Male)

% Completion


REPORTING OF Quarterly and Annual Report


INDICATOR 8

DATA



N/A


BASELINE VALUE 0

BASELINE YEAR N/A

TARGETS NOTES Target is determined based on potential CSOs in the district level, population, membership of MSF during IFACS project.


FY 2016 50

FY 2017 150

FY 2018 150

FY 2019 150

FY 2020 0

Targets LOP 500




INDICATOR 9



INDICATOR TITLE Number of people reached by LESTARI communication programs to improve awareness and understanding of LEDS and biodiversity conservation




PRECISE DEFINITION

“Reached by LESTARI communication programs” is defined as individuals who have been reached via LESTARI communication and outreach campaign, such as printed climate change and forest conservation materials, radio campaigns, influential religious leader raising climate change and forest conservation issues to their congregations, social media, video, communication trainings, and exhibitions. People could be reached through passive participation (act as “empty vessels” and receive information) or empowerment participation where primary stakeholders are capable and willing to become involved in the process and take part in advocacy and decision-making.

This indicator is an early step to measure the scale of communication program. While changes in people knowledge and behaviors could be assessed when those people reached turn into champions as indicated by Indicator #8.

UNIT OF MEASURE Number of individuals

DISAGGREGATIONS Sex (female/male); District; Landscape


These results will help to estimate the coverage of LESTARI communication program.

DESIRED DIRECTION

Increasing


Increase awareness and understanding of LEDS and biodiversity, supported by continous advocacy will lead to increase commitment of key stakeholders regarding the positive benefits of conservation and sustainable use of forests and the species they encompass. This in turn leads to improved land use decision making contributes to improved land use governance.


DATA COLLECTION

METHOD

Data collection will vary widely based on the particular activity that reaches people with LESTARI messages/information on climate change and biodiversity conservation issues. For communication training, sign-in sheets will validate the result. For radio campaign, LESTARI will assess the number of people that could potentially be “reached” to this message through population and geographical data. Using the data, the project will estimate the % of people (disaggregated by sex) within that geographical area that likely heard that message. A similar methodology will be used for mass media such as information in newspapers and other forms of print media. The project will also review communication products


INDICATOR 9

developed by stakeholders and deliver case study to assess increase awareness and changes in behaviours.

For social media, the process of gathering data, as well as what data we gather, differs for each channel. In Facebook, LESTARI focus on three main areas: likes, comments and links on posts, and shares. If someone “likes” LESTARI page, we will know that they’re listening to us. When followers add comments to our posts, we’ve made the conversation two-way by engaging them. And when they share our posts with their Facebook friends, we’ve expanded our audience. Gathering this information will help us understand what people are doing on Facebook, and how to tap into it. Twitter is similar—engagement is a more accurate measure of our influence than follower count. Do followers retweet our posts, or share information of relevance and interest to our organization and constituents? Measure engagement against the time we spend on the site to get a sense of the channel’s value. For LESTARI website, we will use Google Analytics to monitor traffic and referral data.

DATA SOURCE Sign-in sheet from trainings, secondary data (population and geographical), communication products (e.g., articles, shermon sheets, etc) developed by stakeholders, activity reports, assessment report, social media monitoring report.




Staff and labor cost, assessment cost, photocopies of activities documentation.


Communication & Advocacy Specialist



DATA QUALITY

INITIAL DQA TBD

DATE OF FUTURE DQA





Calculating direct beneficiaries as a result of communication training is rather easy. However, when LESTARI issues a radio spot or puts information in a local newspaper, measuring the precise number is extremely difficult and imprecise.

ACTION TAKEN TO ADDRESS

LESTARI will provide a rational justification for data counted against this indicator that obtained through mass media such as radio and newspaper information dissemination. LESTARI will access the reach of the message (e.g., the

http://www.google.com/analytics/


INDICATOR 9

LIMITATIONS newspaper’s geographic circulation or how far a radio signal can carry) and determine the population within that area. LESTARI will then apply a percentage of people that are likely reached to that message, using a rather conservative percentage (e.g., 30% of total population within that range).


DATA ANALYSIS

Counting number of individuals reached in LESTARI communication programs. LESTARI will assess the efficacy of outreach and other communication activities using the case study as well as assess the achievement of other related indicators that indicate changes in behaviors.



Landscape District Target Achievement (disaggregated by Female/Male)

% Completion


REPORTING OF DATA




N/A


BASELINE VALUE 0

BASELINE YEAR N/A

TARGETS NOTES Target is determined based on potential population reached per landscape and IFACS lesson learned. LOP target is 15,000. Target for the fourth and fifth year of the project decreased following the activity declines.


FY 2016 1,000

FY 2017 4,000

FY 2018 5,000

FY 2019 3,500

FY 2020 1,500

Targets LOP 15,000


INDICATOR 9




INDICATOR 10


RESULT

IR 1. Improved forest management

KR 3. Management of at least six conservation areas improved, resulting in the

conservation of valuable orangutan and other key species habitat, and the

reduction in poaching of threatened and endemic species

INDICATOR TITLE Number of Conservation Areas (CAs) with at least 70 point in METT scores across LESTARI landscapes




PRECISE DEFINITION

Conservation Areas include National Parks, Wildlife sanctuaries, natural reserves, all areas of high biodiversity under the jurisdiction of the Ministry of Forestry.

The Management Effectiveness Tracking Tool (METT) is a tool developed by the World Wildlife Foundation and The World Bank to track and monitor the effectiveness of protected area management. It provides a rapid and replicable assessment designed to reveal trends, standardize assessment and reporting, and aid in adaptive management. METT consists of two primary sections – datasheets and an assessment form. The datasheets contain contextual information such as size and location, local designation, and budget, as well as a ranking of threats. The assessment form consists of a 30-question scorecard that quantifies performance based on a 4-point scale (0-3). Each question also requires an explanation for qualitative judgments, such as detailing the level of staff knowledge or results from external studies.

Overall METT encompasses 6 key elements (context, planning, inputs, process, outputs, and outcomes) crucial to effective protected area management. METT allows park managers to identify needs, constraints, and priority actions to improve protected area management.

UNIT OF MEASURE Number of CAs

DISAGGREGATIONS CAs, Landscape


METT was selected, not only because it allows the LESTARI team to monitor a portfolio of sites with a single, cost-effective tool, but also because it has been adopted by GoI as a tool to assess the management effectiveness of protected areas across Indonesia.

The ability to manage protected areas effectively relies on a combination of good governance, sufficient capacity, well-trained staff and enough money to pay for essential management activities and equipment. Therefore, METT will provide a quick and simple picture of effectiveness in individual protected areas, through repeated application in sites over time.

DESIRED DIRECTION

Increasing

LINKAGE TO LONG TERM OUTCOME OR

Success in maintaining biodiversity is linked to a well-regulated and managed protected area, where staff are assessing progress and making changes as necessary. The strongest association is with law enforcement, control of access,


INDICATOR 10

IMPACT resource management, monitoring and evaluation, maintenance of equipment, budget management and existence of annual work plans. Increase effectiveness of CAs management will contribute to conservation goal and reduce GHG emissions.



METT baseline and endline survey; monitoring the implementation of CAs management plan

DATA SOURCE Survey report; CAs management plan; direct observation


Annual


Staff labor costs; METT survey cost; photocopies of necessary documents





DATA QUALITY

INITIAL DQA TBD

DATE OF FUTURE DQA





METT has been developed to provide a quick overview of progress in improving the effectiveness of management in individual protected areas, to be filled in by the protected area manager or other relevant site staff. As such it is clear that there are limitations on what it can achieve: it should not for example be regarded as an independent assessment, or as the sole basis for adaptive management.

The Tracking Tool is also too limited to allow a detailed evaluation of outcomes and is really aimed at providing a quick overview of the management steps identified in the WCPA (World Commission of Protected Area) Framework up to and including outputs. Clearly, however good management is, if biodiversity

continues to decline, the protected area objectives are not being met. Therefore the questions on condition assessment have disproportionate importance in the overall Tracking Tool.

ACTION TAKEN TO Additional questions may need to be added to suit local circumstances rather


INDICATOR 10

ADDRESS LIMITATIONS

than modify the Tracking Tool. The Tracking Tool contains a set of questions that have been designed to be easily answered by those managing the protected area without any additional research. However, it is useful to review the results of existing monitoring and to spend sufficient time discussing each aspect

of management being assessed to arrive at a considered judgement. A group

of protected area staff from the reserve, project staff or other agency staff should be involved in answering the questions in the Tracking Tool; where possible additional external experts, local community leaders or others with knowledge and interest in the area and its management should also be involved.


DATA ANALYSIS

Analysis will be done through quantitative (scoring) and qualitative methods. METT assessment is made by assigning a simple score ranging between 0 (poor) to 3 (excellent). A series of four alternative answers are provided against each question to help assessors to make judgements as to the level of score given.

The maximum score of the 30 questions and supplementary questions is 99. A final total of the score from completing the assessment form will be calculated as a percentage of 99 or of the total score from those questions that were relevant to a particular protected area. Thus if a protected area scores 65 out of a maximum score of 87 the percentage can be calculated by dividing 65 by 87 and multiplying by 100 (i.e. 65 ÷ 87 x 100 = 75%). Qualitative judgements will be applied whereas the information come from local staff knowledge (in many cases, staff knowledge will be the most informed and reliable source of knowledge), a reference document, monitoring results or external studies and assessments.



Landscape Conservation Area



REPORTING OF DATA

Annual Report



METT baseline survey undertaken by Ministry of Forestry and Environment (MOFE) for six conservation areas

BASELINE NOTES None

BASELINE VALUE

Conservation Areas METT score

Gunung Leuser National Park 67 %

Sebangau National Park 62 %

Lorentz National Park 52 %

Cycloops Nature Reserve 43 %

Rawa Singkil 55 %

Bukit Baka Bukit Raya National Park 64 %

BASELINE YEAR 2015


INDICATOR 10

TARGETS NOTES LESTARI will assist six conservation areas to improve their METT scores.


FY 2016 Not measure

FY 2017 1

FY 2018 2

FY 2019 2

FY 2020 1

Targets LOP 6




INDICATOR 11



INDICATOR TITLE Number of co-management agreements signed that secure community rights and benefits




PRECISE DEFINITION

LESTARI uses an IUCN definition of the term co-management which includes “a situation in which two or more social actors (parties) negotiate, define and guarantee amongst themselves a fair sharing of the management functions, entitlements, and responsibilities for a given territory, area, or set of natural resources” – in this case for conservation and sustainable NRM utilization.

The co-management agreements referred here will include community based forest management agreements covered in the existing policies, which include Hutan Kemasyarakatan (HKm), Hutan Desa (HD), Hutan Adat (HA), Hutan Tanaman Rakyat (HTR), and other forms of co-management agreements such as partnership agreements between communities with conservation area managers, private sectors, or stakeholders as well as co-management agreements involving multistakeholders (such as the format that is currently initiated in Cyclops). These co-management agreements will ensure communities’ secured access to forest resources.

Co-management agreements signed will cover the process of obtaining permits from social forestry initiatives (HKM, HD, HTR, HA) or the signing of partnership agreements between communities and other stakeholders (conservation area managers, private sectors, etc).

Co-management agreement implemented and monitored means that forest management action plan that will be included in the agreements are implemented and monitored.

UNIT OF MEASURE Number of agreements

DISAGGREGATIONS Landscape; Type of co-management agreement


The indicator is used by LESTARI to assess the extent to which key stakeholders committed to sustainable use of forests. The indicator also illustrates reduction in land use conflicts.

DESIRED DIRECTION

Increasing


Many critical forest areas within LESTARI landscapes are poorly managed and experience a high rate of deforestation and degradation as a result. An effective management, a high level of local social capital, including cohesive communities, is essential for conservation of these important resources.



INDICATOR 11


Review the content of co-management agreements, assessment of the effectiveness of their implementation, spot-check to observe compliance.

DATA SOURCE Assessment reports; document of agreements; direct observation









DATA QUALITY

INITIAL DQA TBD

DATE OF FUTURE DQA





Activity reports may not always be accurate. Community may live in remote sites that are difficult to visit to validate data.


LESTARI field staff to spot-check claims through field visits. Sampling techniques for remote sites. Review agreements to confirm that they include statements of commitment and secure community rights and benefits.


DATA ANALYSIS

The development of co-management agreements involves multiple steps. When counting progress toward meeting this target, LESTARI will track the intermediate milestones. We will count this indicator achieved once the agreement is signed; however, the project will still monitor and report the implementation of agreements.

Hectares of the area managed under co-management agreement will also be reported and contribute to Indicator #2.

Districts

Milestones

Stakeholders and areas for

Dialogue facilitated by

Co-management

Agreements implemented


INDICATOR 11

co-management identified

the project agreements signed

and monitored



Landscape Type of co-management agreement

Target Achievement Ha under co-management agreement

% Completion


REPORTING OF DATA




N/A


BASELINE VALUE 0

BASELINE YEAR N/A

TARGETS NOTES Target is determined based on locations that proposed to LESTARI by Ministry of Environment and Forestry.


FY 2016 0

FY 2017 10

FY 2018 13

FY 2019 12

FY 2020 5

Targets LOP 40




INDICATOR 12



INDICATOR TITLE Number of people receiving USG supported training in natural resources management and/or biodiversity conservation




PRECISE DEFINITION

This indicator will measure the number of individuals trained through learning

activities in natural resources management (NRM) and/or biodiversity

conservation. Training will primarily be traditional classroom training and

workshops (led by designated instructors or “lead persons,” with a learning

objective and defined curricula).

Training topics will include but are not limited to: Best Management Practices,

forest fire management, Good Agricultural Practice (GAP), etc.

This indicator reports against the Biodiversity Fund.

UNIT OF MEASURE Number of people

DISAGGREGATIONS Sex (male/female); District; Landscape; and Affiliation (government, non-governmental organizations, private sector, and community, as appropriate).


Tracking training in NRM and biodiversity conservation provides information about the reach and scale of training and capacity building efforts.

DESIRED DIRECTION

Increasing


Increased capacity to manage forest resources and adapt to climate change will lead to improved land use management and strengthened mitigation and adaptation strategies to overall ecosystem management.



Review of training attendance records, with a proportion of the trainings attended by LESTARI staff. Pre and post survey to seek information about changes in knowledge resulting from the training. Monitor other indicators to justify changes in application of knowledge and skill.

DATA SOURCE Training records/reports, training grantee/subcontractor reports, monitoring reports.


At the end of each training, to be compiled quarterly and annually

ESTIMATED COST Staff labor costs including field staff and transportation to training sites to observe


INDICATOR 12

OF DATA ACQUISITION

and verify the training.


Assigned field staff will report to Landscape M&E Specialist with cc: ME&L Coordinator



DATA QUALITY

INITIAL DQA TBD

DATE OF FUTURE DQA





Training record/report may not always be accurate.


Independent verification through attendance by LESTARI staff.


DATA ANALYSIS Counting number of people participating in LESTARI NRM and/or biodiversity trainings



Landscape District Target Achievement (disaggregated by sex and affiliation)

% Completion


REPORTING OF DATA




N/A

BASELINE NOTES For training, the baseline is zero


INDICATOR 12

BASELINE VALUE 0

BASELINE YEAR N/A

TARGETS NOTES Target is determined based on potential population and IFACS lesson learned


FY 2016 1,000

FY 2017 2,000

FY 2018 3,000

FY 2019 3,000

FY 2020 1,000

Targets LOP 10,000




INDICATOR 13


RESULT


KR 5. Funding leveraged from public and private sources, representing co-

investment in project outcomes

INDICATOR TITLE Amount of investment mobilized (in USD) for climate change as supported by USG assistance




PRECISE DEFINITION

This indicator includes finance mobilized (or leveraged), enabled by USG assistance, for actions, activities, projects or programs that avoid, reduce, or sequester GHGs. Finance may be mobilized from the public sector (e.g., local government) or private sector (e.g., corporate investment) and should help to advance the objectives established by LESTARI program.

For LESTARI, finance mobilized under this indicator also include those allocated for sustainable forest management, spatial planning, and conservation activities in targeted landscapes.

For LESTARI, finance can be mobilized through a variety of instruments and vehicles, including common funding instruments, parallel investments, or in-kind support as a result of USG support. Examples of the types of U.S. assistance (under LESTARI) that could mobilize finance include:

Finance interventions, such as:

Grants (or in-kind support) for technical assistance

Equity or investment shares through PPP scheme

Investments made possible by policy interventions and technical assistance interventions, such as:

Facilitate organizations/local governments to access financing from ICCTF (Indonesia Climate Change Trust Fund)

Market value chains for cocoa that link farmers to wider market

Regulatory policy support for transparent and fair permitting and approvals

Regulatory policy support for Payment for Environmental Services (PES)

Co-management interventions

Examples of what mobilized funds may support include improving the enabling environment for mitigation actions, funding the costs of climate change and conservation activities advanced by the program, monitoring climate change progress or outcomes, or sensitizing stakeholders to climate risks and land use issues and opportunities addressed through the program.

USD amount will be determined on a quarterly basis for all Indonesian rupiah values within that quarter using a currency converter (http://www.oanda.com/currency/converter/). This will be printed and filed with the indicator documentation.

http://www.oanda.com/currency/converter/


INDICATOR 13

UNIT OF MEASURE US Dollars

DISAGGREGATIONS

Domain of interest: Sustainable Landscapes and Biodiversity

Source of funds (public, private) dissagregated further into domestic and international sources of investment

District; Landscape


Indicator used to track amount of investment leveraged the project could reach under initiatives fitting together public and private entities

DESIRED DIRECTION

Increasing


LESTARI programs are intended to be catalytic and to have sustainable benefits. The mobilization of additional financial resources can be an important indicator for assessing the success of LESTARI in catalyzing resources needed for transformational change. This indicator can also help to provide a baseline of data needed to test hypotheses as to the most effective strategies, techniques, or necessary capacities for mobilizing the funds required to address climate change and biodiversity conservation, leading to lessons learning over time.



LESTARI will gather data about the amount of finance mobilized from all types of potential activities intervention (please see the examples under precise definition section).

Documentation should include a rationale for how LESTARI support has facilitated the mobilization of additional resources and include information such as source of funds and use of funds.

DATA SOURCE Financial records/statements verify amounts leveraged and documentation that prices out labor and material per period of time that was mobilized to enhance LESTARI objectives in climate change and biodiversity conservation.




Staff labor costs; photocopies of documentation


Landscape M&E Specialists



DATA QUALITY

INITIAL DQA TBD


INDICATOR 13

DATE OF FUTURE DQA





Proprietary information – some organizations providing funding may consider some information on their funding support proprietary.

Precision – measuring funding leveraged does not necessarily indicate the magnitude of impact or results achieved.


Accept self-reporting (with narrative caveats) and validate with estimates of public or private sector investments using documented similar investments for which data are known.


DATA ANALYSIS

Use standard audit methods on available data, calculate descriptive statistics - Disaggregation by domain of interest - Comparison with targets. When LESTARI intervention resulting on government allocating budget for certain program (e.g., fire prevention) in APBD, we will make a comparison between current APBD and previous APBD to allow justification of achievement.



Landscape District Target Achievement (disaggregated by domain of interest and source of funds – public, private – domestic vs international)

% Completion


REPORTING OF DATA




N/A

BASELINE NOTES The baseline is zero. The project is at a starting point and no activity has been developed by this time.

BASELINE VALUE 0


INDICATOR 13

BASELINE YEAR N/A

TARGETS NOTES Target is determined based on potential investment that can be leveraged in each landscape.


FY 2016 2,000,000

FY 2017 4,000,000

FY 2018 4,000,000

FY 2019 5,000,000

FY 2020 5,000,000

Targets LOP 20,000,000




INDICATOR 14



INDICATOR TITLE Number of people receiving livelihood co-benefits (monetary or non monetary)



INDICATOR TYPE Custom Indicator – contributes to FACTS 4.8.2-36

PRECISE DEFINITION

This indicator identifies the number of people in LESTARI landscapes where sustainable landscapes activities as well as conservation and sustainable use of forest resources activities are implemented, who have received livelihood co-benefits associated with these activities. People included in the metric should be part of populations or households identified by a project with a documented relationship to the project. Beneficiaries should be reasonably assumed to have received a documented benefit or service enabled by USG assistance.

Beneficiaries may include, but are not limited to: members of a household with an increased income or a newly secured land title, members of a cooperative who have increased sales due to increased market access, or even children attending a school renovated with payments for ecosystem services.

Examples of monetary benefits may include, but are not limited to: increased income due to government policies related to climate change mitigation, payments for avoided emissions or carbon sequestration, monetary benefits from ecosystem services, public-private partnership, etc, facilitated by LESTARI.

Examples of non-monetary benefits may include, but are not limited to: access to services, access to markets, land titling or registration, increased access to environmental services, protection of traditional livelihoods and customary rights, or increased productivity from climate-smart agricultural practices, etc, facilitated by LESTARI.

Number is specific to each year, not cumulative

UNIT OF MEASURE Number of people

DISAGGREGATIONS Sex (male/female); Landscape; District

Type of activity: Sustainable landscape; Biodiversity


Reducing emission from deforestation and degradation can stimulate a win-win situation, by delivering carbon sinks and at the same time guaranteeing the conservation of biodiversity and livelihoods for forest-dependent communities. It is important to not only monitor emission reduced or avoided, but also to keep track of biodiversity and the livelihoods of local people. This measure demonstrates project reach.

DESIRED DIRECTION

Increasing


This indicator links sustainable natural resources management to economic growth objective. When people receive tangible monetary or non-monetary benefits from natural resource management or conservation, they are more likely


INDICATOR 14

to value and support these activities into the future, well after the project ends, creating a sustainable impact.



Beneficiary interview, random survey and on-sight observations to approximate impact across households or population

DATA SOURCE Project and subcontractor/grantee records; survey reports


Annually


Staff labor costs; survey costs


Landscape M&E Specialist



DATA QUALITY

INITIAL DQA TBD

DATE OF FUTURE DQA





This indicator is not intended to capture the quality of co-benefits received or the degree to which livelihoods are improved.


During the quantitative survey, the survey team will also solicit qualitative assessment to capture performance and other relevant assumptions into the calculations.


DATA ANALYSIS

The analysis will involve quantitative (statistical) and qualitative analysis. Qualitative will enrich explanation of project impact. Analysis includes gender and location of those receiving benefits, which will more closely assess are those benefits equitably distributed between men and women.

Individuals receiving benefits from more than one sustainable landscapes activity, or receiving multiple benefits from a single activity, should be counted once per fiscal year.


INDICATOR 14



Landscape District Type of Activity (Sustainable Landscape; Biodiversity)

Target Achievement (disaggregated by sex)

% Completion


REPORTING OF DATA

Annual Report



N/A

BASELINE NOTES N/A

BASELINE VALUE 0

BASELINE YEAR N/A

TARGETS NOTES Target achievement for this indicator will only start to be reported in the second year of the project to allow impact to realize.


FY 2016 0

FY 2017 7,500

FY 2018 7,500

FY 2019 7,500

FY 2020 7,500

Targets LOP 30,000




INDICATOR 15



INDICATOR TITLE Number of private sector firms that have improved management practices as a result of USG assistance




PRECISE DEFINITION

This indicator measures the number of firms receiving USG assistance that improved their management practices. There are numerous private sector companies in LESTARI landscapes, ranging from large, multi-national, publicly traded Tropical Forest Alliance (TFA) members to smaller Indonesian privately held firms. Firms include forestry concenssions, oil palm and fiber plantations, mining and agroforestry/commodity producers, and their buyers. Companies, especially those involved in international trade, are under growing pressure to establish and meet sustainability and reduced emissions standards. Most companies lack the tools to assess potential GHG impacts, costs, and efficacy of these emerging goals, nor do they necessarily incorporate best management practices (BMPs) to meet these goals.

The project will facilitate development (or update) of Conservation Management and Monitoring Plans (CMMPs) and support their implementation. These plans are used as a means to guide the management and monitoring of HCV and HCS areas in concessions that include Best Management Practices (BMPs). Improved management practices is achieved when companies incorporate BMPs in their Standard Operating Procedures (SOP), allocate budget for implementation, and implement some or all the action plans.

UNIT OF MEASURE Number of firms

DISAGGREGATIONS Landscape


Indicator used to track private sectors commitment regarding the positive benefits of conservation and sustainable use of forests.

DESIRED DIRECTION

Increasing


Adoption and implementation of BMPs will ensure that HCV and HCS areas in concessions are managed in perpetuity thus result on reduction in GHG emissions.



Review the CMMP document, SOP, and budget line item allocated by concessionaires to implement the plan. Spot-check to observe field implementation.

DATA SOURCE CMMP document, SOP, direct observation

FREQUENCY AND Rolling as data available


INDICATOR 15

TIMING OF DATA ACQUISITION


Staff labor costs; photocopies of documentation; field visit cost


Private Sector Engagement Coordinator



DATA QUALITY

INITIAL DQA TBD

DATE OF FUTURE DQA





Concessionaires may refuse to release their financial reports


LESTARI field staff to spot-check concessionaires’ claims of implementation through field visits and site observations


DATA ANALYSIS

Triangulation of findings will be made through compilation of CMMP, SOP, budget documents review, direct observation, and interview concessionaires managers and staff. When counting progress toward meeting this target, LESTARI will track the intermediate milestones. LESTARI will also report the number of hectares under improved management practices.

Achievement of this indicator will contribute to Indicator #2.

Milestones

MoU with private sector

BMP package agreed upon

Adoption of BMP activities and SOP finalized

Budget for BMP activities incorporated into SOP

Training for BMP provided to company staff

BMP activities implemented

BMP monitoring and evaluation conducted

PRESENTATION OF Tabular with supporting narrative, as appropriate


INDICATOR 15

DATA Landscape Target Achievement

(number of private firms and hectare under improved management practices)

% Completion


REPORTING OF DATA




N/A

BASELINE NOTES The baseline is zero. The Project is at a starting point and no activity has been developed by this time.

BASELINE VALUE 0

BASELINE YEAR N/A

TARGETS NOTES Target is determined based on potential private firms to collaborate in each landscape.


FY 2016 2

FY 2017 1

FY 2018 3

FY 2019 2

FY 2020 2

Targets LOP 10




INDICATOR 16


RESULT


KR 4. At least ten public-private partnerships (PPPs) promoting low-emissions

conservation oriented development established

INDICATOR TITLE Number of new USG-supported public-private partnerships (PPPs) formed



INDICATOR TYPE FACTS PPP 5

PRECISE DEFINITION

Number of public-private partnerships (PPPs) formed during the reporting year. Partnerships can be long or short in duration (length is not a criteria for measurement). A partnership with multiple partners should only be counted as a single partnership. However, an institution may form more than one partnership with the same entity and each partnership should be counted separately.

A partnership is considered “formed” when there is a clear agreement, written and signed, to work together to achieve a common objective. Public-private partnership will be understood as a formal agreement between public and private entities with the purpose to provide and /or fulfill a public service in a schema defined to allow to the private enterprise to reach return on investment. Private partners could be for-profit enterprises, NGOs, private company, a community group, or a state-owned enterprise which seeks to make a profit (even if unsuccessfully).

Public-private partnership also include partnerships between LESTARI (on behalf of USAID) and the private sector entities in its landscape.

LESTARI will work directly with businesses and business associations from the onset in the creation of the PPPs to foster their support and to broaden impact to increase economic benefits while reducing GHG emissions and deforestation.

PPP scheme will require some amount of leveraged funding from the private sector that contributes to low emissions conservation oriented activities. The ratio of funding between two parties will be determined during the process of developing agreement. Though it is expected that there is one to one match of funding.

Investment mobilized through PPP scheme will contribute to Indicator #13 (investment mobilized).

UNIT OF MEASURE Number of new partnerships. The indicator should only measure the new partnerships in the reporting year.

DISAGGREGATIONS Landscape; Type of PPP


Indicator used to track capacity of public and private entities to reinforce the legal frameworks for facilitating PPP, establish partnerships and support implementation of these kind of partnerships. The indicator is also used to measure sustainability of efforts and reinforce initiative for resilience and local benefits.

DESIRED DIRECTION

Increasing


INDICATOR 16


PPP will provide incentives for governments, private sectors and communities to embrace LEDS and conservation oriented practices.



Review of partnership signed, implementation documentation, activities reports and proofs of disbursements

DATA SOURCE PPP agreement, activities reports and proofs of disbursements


Annually


Staff labor costs; photocopies of documentation


Private Sector Engagement Coordinator



DATA QUALITY

INITIAL DQA TBD

DATE OF FUTURE DQA





None


None


DATA ANALYSIS Compilation of data - Disaggregation by type of PPP - Comparison with targets - Analyze on improvements and impacts on climate change and biodiversity conservation aspects.

PRESENTATION OF Tabular with supporting narrative, as appropriate


INDICATOR 16

DATA Landscape Target Achievement

(disaggregated type of PPP)

% Completion


REPORTING OF DATA

Annual Report



N/A

BASELINE NOTES N/A

BASELINE VALUE 0

BASELINE YEAR N/A

TARGETS NOTES Target is determined based on potential PPP in each landscape


FY 2016 3

FY 2017 5

FY 2018 5

FY 2019 5

FY 2020 2

Targets LOP 20




ANNEX II: RESULTS CHAINS


ANNEX III: DATA QUALITY ASSESSMENT CHECKLISTS This Data Quality Assessment (DQA) Checklist is provided as a recommended tool that an

operating unit (OU) may use to complete its DQAs. If the OU prefers or has successfully

used a different tool for conducting and documenting its DQAs in the past, they are free to

continue the use of that tool instead. The checklist below is intended to assist in assessing

each of the five aspects of data quality and provide a convenient manner in which to

document the OU’s DQA findings.

USAID Mission or Operating Unit Name:

Title of Performance Indicator:

[Indicator should be copied directly from the Performance Indicator Reference Sheet]

Linkage to Foreign Assistance Standardized Project Structure, if applicable (i.e. Project Area,

Element, etc.):

Result This Indicator Measures [For USAID only] (i.e., Specify the Development Objective,

Intermediate Result, or Project Purpose, etc.):

Data Source(s):

[Information can be copied directly from the Performance Indicator Reference Sheet]

Partner or Contractor Who Provided the Data:

[It is recommended that this checklist is completed for each partner that contributes data to an

indicator– it should state in the contract or grant that it is the prime’s responsibility to ensure

the data quality of sub-contractors or sub grantees.]


Period for Which the Data Are Being Reported:

Is This Indicator a Standard or Custom

Indicator?

____ Standard Foreign Assistance

Indicator

____ Custom (created by the OU; not

standard)

Data Quality Assessment methodology:

[Describe here or attach to this checklist the methods and procedures for assessing the

quality of the indicator data. E.g. Reviewing data collection procedures and documentation,

interviewing those responsible for data analysis, checking a sample of the data for errors,

etc.]

Date(s) of Assessment:

Assessment Team Members:

USAID Mission/OU Verification of DQA

Team Leader Officer approval

X_______________________________________

YES NO COMMENTS

VALIDITY – Data should clearly and adequately represent the intended result.

1 Does the information collected measure what it is supposed to

measure? (E.g. A valid measure of overall nutrition is healthy

variation in diet; Age is not a valid measure of overall health.)

2 Do results collected fall within a plausible range?


3 Is there reasonable assurance that the data collection

methods being used do not produce systematically biased

data (e.g. consistently over- or under-counting)?

4 Are sound research methods being used to collect the data?

RELIABILITY – Data should reflect stable and consistent data collection processes and

analysis methods over time.

1 When the same data collection method is used to

measure/observe the same thing multiple times, is the same

result produced each time? (E.g. A ruler used over and over

always indicates the same length for an inch.)

2 Are data collection and analysis methods documented in

writing and being used to ensure the same procedures are

followed each time?

TIMELINESS – Data should be available at a useful frequency, should be current, and should

be timely enough to influence management decision making.

1 Are data available frequently enough to inform project

management decisions?

2 Are the data reported the most current practically available?

3 Are the data reported as soon as possible after collection?

PRECISION – Data have a sufficient level of detail to permit management decision making;

e.g. the margin of error is less than the anticipated change.

1 Is the margin of error less than the expected change being

measured? (E.g. If a change of only 2% is expected and the

margin of error in a survey used to collect the data is +/- 5%,

then the tool is not precise enough to detect the change.)

2 Has the margin of error been reported along with the data?

(Only applicable to results obtained through statistical

samples.)

3 Is the data collection method/tool being used to collect the

data fine-tuned or exact enough to register the expected

change? (E.g. A yardstick may not be a precise enough tool to

measure a change of a few millimeters.)

INTEGRITY – Data collected should have safeguards to minimize the risk of transcription

error or data manipulation.

1 Are procedures or safeguards in place to minimize data

transcription errors?


3 Is there independence in key data collection, management,

and assessment procedures?

3 Are mechanisms in place to prevent unauthorized changes to

the data?

SUMMARY

Based on the assessment relative to the five standards, what is the overall conclusion

regarding the quality of the data?

Significance of limitations (if any):

Actions needed to address limitations prior to the next DQA (given level of USG control

over data):

IF NO DATA ARE AVAILABLE FOR THE INDICATOR COMMENTS

If no recent relevant data are available for this indicator, why not?

What concrete actions are now being taken to collect and report these

data as soon as possible?

When will data be reported?

RECOMMENDATIONS FOR CONDUCTING DATA QUALITY ASSESSMENTS

1. Data Quality (DQ) assessor should make sure that they understand the precise definition of the indicator by checking the Performance Indicator Reference Sheet. Please address any issues of ambiguity before the DQA is conducted.

2. DQ assessor should have a copy of the methodology for data collection in hand before assessing the indicator. For USAID Missions, this information should be in the AMEP’s Performance Indicator Reference Sheets for each indicator. Each indicator should have a written description of how the data being assessed are supposed to be collected.


3. Each implementing partner should have a copy of the method of data collection in their files and documented evidence that they are collecting the data according to the methodology.

4. DQ assessor should record the names and titles of all individuals involved in the assessment.

5. Does the implementing partner have documented evidence that they have verified the data that has been reported? Partners should be able to provided USAID with documents (process/person conducting the verification/field visit dates/persons met/activities visited, etc.) which demonstrates that they have verified the data that was reported. Note: Verification by the partners should be an ongoing process.

6. The DQ assessor should be able to review the implementing partner files/records against the methodology for data collection laid out in the AMEP (for USAID Missions only). Any data quality concerns should be documented.

7. The DQ should include a summary of significant limitations found. A plan of action, including timelines and responsibilities, for addressing the limitations should be made.

LESTARI

Wisma GKBI, 12th Floor, #1210

Jl. Jend. Sudirman No. 28, Jakarta 10210, Indonesia.

Phone: +62-21 574 0565

Fax: +62-21 574 0566

Email: [email protected]