ALINe working paper

!

!1

! !

Working Paper 11 Rethinking Environmental Monitoring and Assessment in Agricultural Research and Development Fiona Marshall and Erik Millstone, with Timothy Karpouzoglou and Sally Brooks

September 2013

ALINe working paper

!

!2

! !

About ALINe

The Agricultural Learning and Impacts Network (ALINe) is an initiative between the Institute of Development Studies, UK and other partners, with funding from the Bill & Melinda Gates Foundation (BMGF). ALINe combines practical experience and technical

expertise with cutting-edge academic research in order to learn from farmer experience, share good practice and pilot new approaches to monitoring and evaluation. ALINe utilises a people-centred performance measurement that is responsive to poor farmers, offering the potential to re-incentivise and re-align the aid system. ALINe has multiple clients in the public, private and third sectors and a portfolio currently in 9 countries with emphasis in Sub-Saharan Africa and South Asia. Visit our website: www.aline.org.uk

About IDS

The Institute of Development Studies is one of the world's leading charities for research, teaching and communications on international development. Founded in 1966, the Institute enjoys an international reputation based on the quality of its work and the rigour with which it applies academic skills to real

world challenges. Its purpose is to understand and explain the world, and to try to change it – to influence as well as to inform. IDS hosts five dynamic research programmes, five popular postgraduate courses, and a family of world-class web-based knowledge services. These three spheres are integrated in a unique combination – as a development knowledge hub, IDS is connected into and is a convenor of networks throughout the world. The Institute is home to approximately 80 researchers, 50 knowledge services staff, 50 support staff and about 150 students at any one time. But the IDS community extends far beyond, encompassing an extensive network of partners, former staff and students across the development community worldwide. Visit our website: www.ids.ac.uk

1

Rethinking!Environmental!Monitoring!and!Assessment!in!Agricultural!Research!and!Development!Fiona!Marshall,!Erik!Millstone,!Timothy!Karpouzoglou!and!Sally!Brooks!!ALINe!Working!Paper!©!Institute!of!Development!Studies!2011!All!rights!reserved.!Reproduction,!copy,!transmission,!or!translation!of!any!part!of!this!publication!may!be!made!only!under!the!following!conditions:!•!!with!the!prior!permission!of!the!publisher;!or!•!!with!a!licence!from!the!Copyright!Licensing!Agency!Ltd.,!90!Tottenham!Court!Road,!London!W1P!9HE,!UK,!or!from!another!national!licensing!agency;!or!•!!under!the!terms!set!out!below.!!This!publication!is!copyright,!but!may!be!reproduced!by!any!method!without!fee!for!teaching!or!nonprofit!purposes,!but!not!for!resale.!Formal!permission!is!required!for!all!such!uses,!but!normally!will!be!granted!immediately.!For!copying!in!any!other!circumstances,!or!for!reuse!in!other!publications,!or!for!translation!or!adaptation,!prior!written!permission!must!be!obtained!from!the!publisher!and!a!fee!may!be!payable.!

2

!!!!Available!from:!Communications!Unit,!Institute!of!Development!Studies,!Brighton!BN1!9RE,!UK!Tel:!+44!(0)!1273!915637!!Fax:!+44!(0)!1273!621202!E_mail:!bookshop@ids.ac.uk!Web:!www.ids.ac.uk/ids/bookshop!!IDS!is!a!charitable!company!limited!by!guarantee!and!

registered!in!England!(No.!877338)!

ALINe working paper

!

!3

! !

Rethinking Environmental Monitoring and Assessment in Agricultural Research and Development Fiona Marshall and Erik Millstone, with Timothy Karpouzoglou and Sally Brooks

Fiona Marshall is a Professor of Environment and Development at the Science and Technology Policy Research Unit (SPRU), School of Business, Management and Economics at the University of Sussex. Fiona is a specialist in environment and development, with over 20 years experience in the design, management and implementation of interdisciplinary research and engagement initiatives. Originally trained as a tropical agricultural ecologist, she works across the natural and social sciences, examining the interface between environmental change, health and social justice, largely in the context of food systems in the south. She has also led a number of initiatives concerned with the role of agricultural science and technology in meeting the needs of the poor.

Erik Millstone is a Professor of Science Policy. Erik gained a first degree in Physics, followed by three postgraduate degrees in Philosophy. From 1971 to 1973 Erik held four temporary and part-time posts teaching philosophy at three universities and one polytechnic. He was then appointed as a lecturer at the University of Sussex in History and Social Studies of Science. He joined SPRU in 1987, becoming a senior lecturer in 1995, a reader in 2001 and a professor in 2005. Since 1974 he has been researching into the causes and consequences of scientific and technological change in the food and agricultural sectors.

ALINe working paper

!

!4

! !ARDAP

BMGF ESP DAs EIA M&E MEL MoA OA P/FTCs POs SAA SEAES SMSs

Appropriate Rural Development Agriculture Programme Bill and Melinda Gates Foundation Environmental Sustainability Planning Development Agents Environmental Impact Assessment Monitoring and Evaluation Monitoring, Evaluation and Learning Ministry of Agriculture Oxfam America Pastoralist/Farmer Training Centres Program Officers Sasakawa Africa Association Strengthening the Ethiopian Agricultural Extension System Subject Matter Specialists

Abbreviations and Acronyms

ALINe working paper 51 !

!

!

!Author notes 3 Abbreviations and Acronyms 4 1 Introduction 6 2 The evolution of environmental assessments in agricultural development 6 agendas 2.1 Eco-efficiency 7 2.2 Eco—systems services 8 2.3 Environmental Entitlements - livelihoods, capitals, capabilities 8 2.4 Political ecology 8 2.5 Resilience and adaptive capacity 9 2.6 Sustainable Intensification 9 3 Key challenges to assessing the environmental impacts of agricultural 10 development interventions and their implications for poverty reduction goals 3.1 Understanding cause-effect relationships 10 3.2 Biophysical impacts may be indirect and vary across time and 10 space 3.3 Synergies and trade-offs between environmental change and poverty 10

reduction 3.4 Environmental effects may be difficult to anticipate, and their 10

significance may be contested 3.5 Project interventions may affect peoples’ ability to cope with 11

environmental change 4 Monitoring and evaluation of agricultural development projects and 11 practices 4.1 The Swedish International Development Cooperation Agency 11 4.2 CGIAR 2011 Measuring the Environmental Impacts of Agricultural 12

Research 4.3 Bill & Melinda Gates Foundation’s Environmental Sustainability 13 Planning

5 Empirical findings: Reflections from Pilot study in West Kenya and Ethiopia 14 5.1 Institutional barriers to the adoption of environmental M&E 15 5.2 Cross – scale analysis: Interactions between agricultural and 16

ecological changes across spatial and temporal scales 5.3 Integrative Approaches - Linking environmental M&E with 17 ‘contextually specific outcomes’

5.4 Integrating critical moments of learning in the M&E process 18 5.5 Temporal dimensions 19 6 References 20

Contents

ALINe working paper

!

!6

1. Introduction This paper focuses on identifying conditions under which environmental impacts are recognised, and where an appreciation of their relationship with poverty reduction goals can be integrated into relevant project monitoring and evaluation. In this context, emergent features of monitoring and evaluation strategies and frameworks are discussed, and opportunities for refinement proposed. An increasing body of evidence suggests that interventions aimed at increasing the productivity of agricultural systems may have important unforeseen implications for the environment. These may affect the sustainability of the agricultural system itself and the livelihoods that depend upon them. There may also be significant impacts for ecosystems and the services they provide across a range of temporal and spatial scales. The need to recognise interactions between agricultural interventions and environmental impacts has also been incorporated into the design of some development and innovation trajectories. However, this has rarely contributed to plans for monitoring and evaluation of those projects and programmes. Only since 2003 have several influential research and development funding agencies stipulated that environmental considerations need to be addressed in the design of agricultural development projects, and incorporated into plans for monitoring and evaluation. This paper provides a thematic chronology of several ways in which understanding of the intended and unintended environmental consequences of agricultural development initiatives have evolved. The paper then provides examples of key challenges in assessing the environmental impacts of agricultural development interventions. Initiatives led by three international funding agencies are discussed in this context, and suggestions are made for enhancements. This paper also includes insights from pilot studies of two regionally based agricultural development projects in Ethiopia and Kenya. These support a discussion regarding institutional arrangements that deter or support the successful integration of environmental assessments into project planning and implementation. The case studies illustrate many of the issues raised in the literature review analysis – including core challenges for environmental monitoring evaluation and learning (MEL) initiatives, the implications of widespread oversights in MEL frameworks and opportunities for intervention. 2. The evolution of environmental assessments in agricultural development

agendas Informed by lessons from the original ‘green revolution’, attempts have been made over the past three decades to conceptualise and implement more context responsive research and innovation methodologies. In the 1980s for example, Glaeser and colleagues attempted to integrate their understanding of agricultural technologies alongside an appreciation of environmental consequences and poverty alleviation. They produced an innovative integrating framework called ‘ecodevelopment’, which they presented as an alternative to the ‘high input-labour saving’ strategy of the Green Revolution (Glaeser et al, 1987). Their strategy explicitly addressed the basic needs of the poor, aiming to promote self-reliance and environmental sustainability. This had much in common with Conway’s conception of agricultural systems as ‘agro-ecosystems’. Agro-ecosystems were understood not just in terms of productivity but also stability, sustainability and equity, where ‘sustainability’ was interpreted as “the ability of a system to maintain productivity in spite of a major disturbance caused by intensive stress or a large perturbation”, (Conway, 1987). A range of practical agricultural research, innovation and diffusion projects, and ultimately a broadening of the agenda for monitoring and evaluation followed these major theoretical advances. The CGIAR’s impact assessment agenda lagged behind, but sequentially corresponded to, earlier developments in the theoretical literature and in the design of programmes and projects. Initially it broadened from plant breeding to ‘farming systems research’ (e.g. Pardey et al, 2006). It was not until the early 1990s that attention within the CGIAR’s impact assessment agendas extended to include considerations of gender, environment and health. A 2001 review of the development of Impact Assessment Research in the CGIAR, 1970-1999, indicated (Figure 1) how the CGIAR’s impact assessment agenda had broadened in a step-wise manner during the last 30 years of the 20th century (Pingali, 2001).

ALINe working paper

!

!7

Source: Pingali, 2001 Progress in understanding the complex interactions between agriculture, environmental change and sustainable development in marginal environments, occurred in parallel to this. These included rain-fed agricultural areas in much of Sub Saharan Africa and parts of India, which were largely neglected by the Green Revolution. For example, in India, work on arid areas characterised by extreme poverty and food insecurity developed from a focus on soils and hydrology in particular watersheds, into a ‘watershed plus’ framework. This sought to integrate understandings of agricultural production with environmental conservation, social justice and local democracy. (Turton, 2000; Brooks and Loevinsohn, 2011) The initiatives were built on cooperative relationships, established over many years, between NGOs well positioned to innovate, and government bodies keen to diffuse successful initiatives to other communities in similar agro-ecological environments (Kerr, 2002). Elsewhere in India and Sub-Saharan Africa, mainstream agricultural development programmes neglected agro-ecologies characterised as ‘complex, diverse and risk prone’. This led to the birth of the ‘farmer first’ movement, which drew upon developments in agro-ecology and anthropological studies of local farming systems. The ‘farmer first’ movement generated a set of participatory methodologies founded on a critique of established power-knowledge relations between policy actors, formal crop science, public agricultural extension and small holder farmers (Richards, 1985; Conway, 1987; Chambers et al., 1989, Scoones and Thompson, 1994, Scoones and Thompson, 2009). The primary insight of the farmer first movement has been that the perspectives of the intended beneficiaries must effectively inform agricultural development projects. Furthermore that this should be obtained through active engagement with representatives of those beneficiaries. It is proposed that in the absence of this approach, projects are at serious risk of being ineffective or, at worst, counter-productive. Work that aspired to enrich understandings of the socio-economic, institutional and ecological aspects of agricultural development was not, however, confined to studies of marginal lands or of the communities that inhabited them. There have been multiple attempts to develop theoretical and analytical frameworks with which to understand the role of agricultural development in the global south. These have included interactions with and relevance to critical issues such as environmental sustainability and poverty reduction. Many aspects of those approaches have been seen as compatible or complimentary, and various aspects of them have influenced the design and implementation of innovative agricultural research and development projects. The remainder of this section provides a review, which is neither comprehensive nor exhaustive, of several distinctive and influential approaches. The purpose is to illustrate the range of ways in which scholars have tried to understand agricultural development as connected to numerous economic, ecological, institutional and cultural considerations. The frameworks included in this section are discussed in approximate chronological order of their emergence. !

Eco-efficiency 2.1.!The term ‘eco-efficiency’ was coined by the World Business Council for Sustainable Development, as the articulation of a business response to the Rio Earth Summit and Agenda 21, (Schmidheiny, 1992). Eco-efficiency was defined as ‘creating more goods and services with ever less use of resources, waste and pollution’, (Lehni et

ALINe working paper

!

!8

al, 2000). It has been encapsulated in several ways, including the ‘ecological rucksack’ (or weight of natural resources used to produce a product), ‘ecological footprint’ (or land area required to provide services), and ‘MIPS’ (material input per unit of service), (Hukkinen, 2001:212). Eco-efficiency differs from more traditional notions of efficient agriculture because it means increasing agronomic production with fewer resources while sustaining the productive potential of land (Park et al., 2009; Lal, 2010:124). Various interpretations of ‘eco-efficiency’ have been criticised. Firstly, if environmental impacts are expressed in terms of mass consumption of resources, there is a supposition that environmental impacts are ‘universally commensurable’; regardless of where and when the impact takes place and who is affected. (Hukkinen, 2001:212). Secondly, advocates have struggled to integrate social factors within the eco-efficiency concept. Thirdly, despite the departure from conventional definitions of efficiency, the concept of eco-efficiency nevertheless presupposes optimisation. However, conditions of climate change and uncertainty may require investments in what might be deemed ‘inefficiencies’, because they contribute to long-term resilience and enhanced adaptive capacity. Korhonen and Seager argued that in practice: “…eco-efficiency optimization rarely results in improved diversity or adaptability and consequently may have perverse consequences to sustainability by eroding the resilience of production systems.” (Korhonen and Seager, 2008:411) In complex, qualitative, uncertain and dynamic co-evolving economic–ecological systems eco-efficiency might actually increase risk, vulnerability and unsustainability. It is important to invest in system diversity, adaptability, and flexibility and preserve the capacity of the production system to create more resilience (ibid).”

Eco-systems services 2.2.

Ecosystem services are the benefits that people obtain from ecosystems and the processes by which the environment produces them. Since the 2005 Millennium Ecosystems Assessment, the ‘ecosystem services’ approach has been increasingly applied as a framework to aid understanding of the human use and management of natural resources. In agricultural systems where the goal has been to maximize the yield outputs (the provisioning service in the system), increasing attention has been given to the trade-offs, both in terms of impacts on other ecosystem services and their valuation. These include pest and disease management, water quality regulation and flood control or preserving habitats or biodiversity. The important value of the ecosystem services framework is that it is designed in such a way that it can take account of on-farm as well as off-farm impacts on ecosystem services as a result of particular agricultural production practices. The framework provides the opportunity for both marketed and non-marketed values to be included within an integrated framework, and it is assumed that they may all be amenable to cost-benefit evaluations allowing analyses of trade-offs and synergies across diverse factors, (Bennett, 2011:37). In this rapidly evolving area of research, efforts are being made to understand the complex interactions between ecosystem services and constituents of human well-being, including security and health. There is also increasing emphasis on enhancing understanding of how ecosystems can be sustainably managed in a way that contributes to poverty alleviation and inclusive/sustainable growth.

Environmental Entitlements - livelihoods, capitals, capabilities 2.3.!A related group of approaches drew on the ‘entitlements’ approach pioneered by Sen, (Sen, 1981). The focus is on social institutions facilitating access to natural resources at the community level, characterized in terms of ‘environmental entitlements’ and on the related concept of ‘sustainable livelihoods’ (SL), (Chambers & Conway, 1992; Leach et al., 1999; Scoones, 1998; Scoones, 2009). The strength of the environmental entitlement framework is the focus on local complexity, heterogeneity and inequality and the role of micro- and meso- level social institutions and practices in maintaining (or transforming) these relationships. The sustainable livelihoods framework is less complex and nuanced but its language of ‘capitals’ (natural, social, human, physical and financial) has facilitated its adoption by some development economists and donor institutions, notably the UK government’s Department for International Development where it was mainstreamed from 1997. While their strengths lie in local (household, community) level analysis, these frameworks have been most often criticised for under-playing the macro-scale and structural factors.

Political ecology 2.4.!Another group of frameworks comes from political ecology and geography, in particular the work of Piers Blaikie and colleagues (Blaikie et al., 1994, Wisner et al., 2004). In contrast to SL and similar approaches, these frameworks analyze vulnerability to environmental hazards and disasters. They provide important insights by

ALINe working paper

!

!9

linking macro-, meso- and micro-scale causes and effects - with key terms including root causes, dynamics pressures, local conditions and cumulative vulnerability. However, this cross-scale analysis is applied only to the social/political dynamics rather than to the interconnected social and ecological dimensions of human-environment systems (cf. Turner et al., 2003). In general, these frameworks have been applied to single disasters rather than on-going environmental changes, and the synergies and trade-offs associated with agricultural production systems. However, the concept of cumulative vulnerability (which highlights cumulative effects of interactions between exposure, vulnerability and structural inequality over time) is relevant and may be incorporated into a future framework concerned with agricultural interventions.

Resilience and adaptive capacity 2.5.!Within the last ten years a different body of literature has focussed on the links between vulnerability, resilience and climate change adaptation in innovative ways. This builds on the work of Holling and Folke on complex socio-ecological systems (Holling, 1973; Folke, 2006). Key features of that work include emphases on non-linearity, adaptability and transformability of systems. (Thompson et al, 2007). The focus on complex, non-linear system dynamics recognises problems of uncertainty and change. It represents a shift from a focus on stability and efficiency towards a concern with learning, recovery and flexibility. Resilience, rather than stability, is deemed a crucial property of complex, coupled human-environment systems. (Turner et al., 2003) There is a growing and related literature on issues such as vulnerability and adaptation to climate change, understood in political economy terms, and expressed in the emergence of terms such as ‘fair adaptation’ (Adger, 2006; Adger et al, 2003, GallopÌn, 2006) and ‘differential vulnerability’ to climate change (Kasperson and Kasperson, 2001). This literature highlights cross-scale dynamics, synergies, trade-offs, mismatches and conflicts, but also acknowledges the possibility of transformation to alternative, more stable states, e.g. from mixed farming to rangeland in drought-prone areas affected by climate change, (Adger et al, 2008; O’Brien et al, 2008, Jones and Thornton, 2009). Unlike much of the ecosystem services literature, these authors understand those interactions as potentially non-linear and unpredictable. A debate has emerged between ‘eco-efficiency’ and ‘resilience’. While extending conventional definitions of efficiency, the eco-efficiency school often remains wedded to assumptions of linearity, balance and optimization. By contrast, much of the resilience literature emphasizes the need to ‘invest in inefficiency’ in the short-term to enhance capacity and protect (biological, social, institutional, technical) diversity over the long-term. In a 2011 review of those theoretical and empirical debates on agricultural development and food security under conditions of rapid environmental and socio-economic change, Brooks and Loevinsohn identify: “…four features of innovation systems more likely to build, sustain or enhance food security in conditions of rapid environmental change:

(i) Recognition of the multifunctionality of agriculture; (ii) Access to diversity as the basis for flexibility and resilience; (iii) Concern for enhancing capacity of decision makers at all levels and

Continuity of effort [by accountable and responsive institutions] aimed at securing well-being for those who depend on agriculture.”

Sustainable Intensification 2.6.!The concept of sustainable intensification incorporates elements of the concept of eco-efficiency. (UN FAO, 2011). It has been defined by the Food and Agricultural Organisation of the United Nations (FAO) as: “…producing more from the same area of land while reducing negative environmental impacts and increasing contributions to natural capital and the flow of environmental services.” (UN FAO, 2011). In the UK, Pretty’s innovative work in this field influenced the approach adopted by the Foresight: Future of Global Food and Framing report, where ‘sustainable intensification’ was defined “…as producing more output from the same area of land while reducing the negative impacts and at the same time increasing contributions to natural capital and the flow of environmental services.” (Pretty et al., 2011). While the idea of ‘sustainable intensification’ has become fashionable, multiple definitions are in circulation, and there is little clarity about how such approaches might be implemented in practice. These approaches also seem to confine their attention to supply-side issues to the neglect of issues concerning, for example, the extent and source of effective demand or issues of consumption, nutrition and safety.

ALINe working paper

!

!10

3. Key challenges to assessing the environmental impacts of agricultural development interventions and their implications for poverty reduction goals

There have been several attempts to introduce and implement systems to monitor and evaluate the implications of agricultural development interventions for environmental sustainability and poverty reduction. These have been in the context of questions about the extent to which agricultural development projects represent an effective use of scarce resources, and about the consequences of such projects for the welfare of the intended beneficiaries and for environmental sustainability. Initially, these initiatives concentrated on issues such as pollution and resource depletion, and then widened to include issues such as bio-diversity and climate change. In reviewing these initiatives a number of characteristic challenges can be recognised, which can be summarised as follows:

Understanding cause-effect relationships 3.1.

The most straightforward and intuitively obvious ways in which environmental issues can be appraised in agricultural development projects has been in respect of their ‘impact’ on the biophysical environment. It is important to appreciate that the impact of interventions will typically not occur in systems that are in equilibrium but in systems that are already changing, through a variety of processes. It may be relatively straightforward to identify ex post a correlation between a project intervention and an environmental effect. However, this may not definitively tell us whether the intervention was wholly or partly responsible for that change without an adequate understanding of cause-effect relationships. Consequently, researchers need to gather base-line data ex ante and try to locate a suitable comparator.

Biophysical impacts may be indirect and vary across time and space 3.2.!For example, forest degradation at the site of an agricultural development initiative may influence water quantity and quality downstream, affecting agricultural production outside a project area. In addition, interventions may cause impacts via behavioural changes associated with use of a new technology rather than directly by the technology itself. Consequently, care needs to be taken during the scoping phase of any assessment of environmental impacts to identify potentially relevant consequences and pathways of causation.

Synergies and trade-offs between environmental change and poverty reduction 3.3.!Improved environmental outcomes may sometimes be associated with poverty reduction (for example as improvements in soil fertility give rise to greater yields and higher incomes for farmers). However, in other circumstances improving some environmental outcomes may be associated with damaging effects on poverty reduction (for example, if pastoralists are excluded from areas in which they wish their stock to graze to protect wildlife). Social and economic differences within agricultural communities may also mean that some members of those communities, particularly the very poor, are far more vulnerable to environmental changes than others. This may be, for example, because the poorest communities depend more on environmental resources to construct and sustain their livelihoods. Consequently, potential impacts on the most vulnerable groups should not be discounted; they should be included in any assessment of the impact of projects, before, during and after.

Environmental effects may be difficult to anticipate, and their significance may be 3.4.contested

!There are frequently different views as to which kinds of environmental effects might occur as a result of agricultural interventions, but also about which of those are sufficiently important (and measurable) that they should be included in a monitoring and evaluation strategy. This arises in part because different stakeholders possess different kinds of knowledge about the relationship between agricultural intervention and environmental sustainability. It is also because different stakeholders possess different values about which aspects of environmental sustainability are important. This means that close attention should be paid to the processes by which the scoping phase of any assessment is decided, to ensure that different values and kinds of knowledge are considered. Consequently, deliberations on the scope of impact assessments should involve relevant groups of stakeholders (from local experts to intended beneficiaries). Without their participation, key knowledge, aspirations and interests would be missing. Participants should also endeavour to notice and learn from unanticipated developments that they may have to respond or adapt to.

ALINe working paper

!

!11

Project interventions may affect peoples’ ability to cope with environmental change 3.5. Sometimes the more significant environmental dimension of a project intervention will concern not so much the question of its biophysical ‘impact’ but rather whether a particular intervention will help or hinder peoples’ ability to cope with environmental changes, such as those occurring as a consequence of anthropogenic climate change. An example may be reduced diversity in cropping systems, potentially resulting in reduced resilience in the face of extreme weather events associated with climate change. Even though we may not be able accurately to predict the direction of environmental change, it can be useful and important to ask what the possible implications of an intervention might be - under a range of feasible background changes. 4. Monitoring and evaluation of agricultural development projects and

practices Several research and policy initiatives have tried to develop systematic assessment frameworks for considering the environmental effects of agricultural development projects. The following discussion examines three such initiatives. The Swedish International Development Cooperation Agency (SIDA) was selected as an example of an unusually comprehensive tool which was not developed specifically for agricultural development interventions, but that has many relevant features. We then discuss two agriculture-specific initiatives one from the Consultative Group on International Agricultural Research (CGIAR), and the other from the Bill and Melinda Gates Foundation (BMGF). These were selected because they are relatively new but major initiatives with potential for further development and widespread impact. In reviewing these initiatives we have paid particular attention to features that will help to the address the core challenges that we have introduced above. Drawing on the literature review above, we also focus on the following, often neglected features of MEL initiatives: Integrative approaches: Does the MEL initiative integrate environmental and social factors within broader notions of sustainability and, in particular, include socially differentiated analysis? Cross-scale analysis: Does the initiative consider interactions, trade-offs and synergies within and between spatial and temporal scales? Dealing with uncertainty? Is the initiative able to recognize and respond to uncertainty? Uncertainly in this case is seen as distinct from simple known risks with predictable outcomes that can be identified at the outset of the project. We may know the probability of them happening or their outcome but not both. Alternatively, they could involve issues that take us completely by surprise. Participatory approaches: Is there scope for participation of diverse stakeholders throughout the process, especially the beneficiaries? Capacity building? Is there scope for building on local adaptive capacity? Reflexivity? Is there scope for the process of environmental M&E to prompt a re-evaluation of the agricultural intervention (or does it only serve the purpose of identifying mitigation measures)? Our analysis of each of the donor initiatives in relation to these key features in summarized in Table 1.

The Swedish International Development Cooperation Agency 4.1.!The Swedish International Development Cooperation Agency (SIDA) provides a particularly comprehensive approach to the M&E of environmental consequences of development projects. It is not uniquely targeted at agricultural projects but is directly applicable to them, and incorporates features that would help to address each of the core challenges that we have identified in relation to agricultural development interventions. The SIDA approach emerged in 2003 with the publication of a set of guidelines for M&E1. Applying SIDA’s guidelines is mandatory for all its projects, and they are interesting in part because they provide a suitable balance between !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!1!! SIDA!(2003)!Guidelines)for)the)Review)of)Environmental)Impact)Assessments:)Sustainable)Development?,!available!from!http://www2.sida.se/?d=118&a=2532&language=en_US!accessed!15!July!2009!

ALINe working paper

!

!12

specificity and flexibility. They emphasise that questions about environmental impacts often deserve to be asked, and that the appropriate answers influence subsequent responses, which should be proportionate to those answers. The SIDA guidelines do not require a detailed analysis of any and all possible environmental consequences of every agricultural intervention, but they stipulate that as projects are being designed, a few essential environment-related questions should be posed and answered. SIDA’s approach is framed by a set of assumptions, for example that all SIDA-supported projects should ensure that the development to which they plan to contribute would be sustainable economically, socially and ecologically. It emphasises cross-scale interactions and impacts and the economic significance of environmental impacts. It also pre-supposes a precautionary approach, which takes seriously problems of uncertainties that characterise and qualify claims and considers both risks and benefits. Furthermore, it emphasises the importance of ensuring the participation of representatives of the projects’ intended beneficiaries in deliberations about potential impacts (positive and negative) and over the choice of indicators to be monitored and the dimensions of evaluation. In other words, SIDA recognises that members of the communities designated as intended beneficiaries of SIDA-supported projects may well have relevant information about, and understandings of, their own environments and ways of living with them that experts from industrialised countries may well not possess. The suggestion is not that M&E plans should only focus on changes deemed locally to be ‘risks’ or ‘benefits’, since impacts on remote or global-level environmental parameters may be low or absent from local agendas, but rather that any M&E plan that had no regard for those environmental parameters considered important by local beneficiary communities would be seriously incomplete. Local communities may also be able to contribute by identifying opportunities as well as risks that external experts might not initially appreciate. In respect of all SIDA-funded agricultural development projects, the project team must indicate which environmental impacts the initiative may have, beneficial and harmful, direct and indirect. A preliminary EIA may be sufficient to indicate that the impacts will either be negligible or beneficial. If however, adverse impacts can reasonably be anticipated, it is appropriate to conduct a more thorough EIA. That deliberation may include issues of how adverse impacts could be diminished, and how environmental benefits could be enhanced. Responsibility for conducting EIAs and incorporating environmental considerations into the planning and conduct of M&E lies with the project team. The responsibility of SIDA’s project officers is to check that those responsibilities are being met.

CGIAR 2011 Measuring the Environmental Impacts of Agricultural Research 4.2. In the autumn of 2011, the CGIAR’s Independent Science and Partnership Council issued a report entitled Measuring the Environmental Impacts of Agricultural Research: Theory and Applications to CGIAR Research (Renkow, 2011). A distinctive feature of this document is that it was oriented specifically at assessing ex post the environmental impact of CGIAR research; its focus did not extend to ex ante assessments. In other words, it confined its attention to retrospective inquiries into the impacts of agricultural research projects and programmes, and did not include a case for prospective consideration of such impacts in the initial design and early planning of projects or programmes. While some CGIAR projects have been framed and designed by reference to environmental considerations, they have always been a minority. There is no general requirement for CGIAR institutes, programmes or projects to consider potential environmental impacts in their design, planning or M&E systems. A further distinctive feature of the CGIAR’s 2011 report is that it presupposes that environmental consequences need to be evaluated not just in bio-physical terms, but that those parameters should be converted into monetary values, in large part to enable the magnitude of trade-offs between increased productivity on the one hand and environmental harm on the other to be estimated (CGIAR, 2011). The authors of the report treat the monetisation of environmental changes as conceptually sound if sometimes problematic in practice. They typically do not argue that money values cannot be assigned to environmental consequences, but rather that a highly diverse set of competing monetary values can be assigned by different individuals and groups, and by utilising different methods, and that there are objective ways of choosing between those competing valuations (ibid). The first chapter in the CGIAR report is an overview and proposed analytical framework for further research (Renkow, 2011). Renkow identifies and discusses five key elements of environmental impact assessment (EIA) as biophysical measurement, scaling, valuation, counterfactual development and attribution. Reviewing progress with regards to these five elements, Renkow argues that more progress has been made in developing tools for valuation than in addressing the other elements. While that might be an accurate comparative judgement, it could also be a pessimistic one. Firstly, valuations of environmental consequences of agricultural activities and initiatives typically confine their attention to estimating their significance to the agricultural economy, but struggle

ALINe working paper

!

!13

to cope with, for example, impacts on non-agricultural biodiversity. Secondly, since issues of monetary valuation remain irredeemably contested, progress on the other elements may therefore be characterised as even more rudimentary. Issues of attribution and the development and endorsement of counterfactuals are linked. To attribute an environmental change to a prior intervention implies being able to reliably assert the truth of a counterfactual claim to the effect that if the intervention had not occurred the environmental change would not have occurred. It is often notoriously difficult to establish causal relationships between agricultural interventions and subsequent environmental changes with any certainty, especially if the subsequent environmental changes emerge slowly or cumulatively. In cases with high-levels of complexity it may be more appropriate to seek to understand contribution to system changes, rather than direct attribution. The approach to issues of scale is another distinctive feature of the analysis provided in the CGIAR report. Renkow initially invokes a dichotomy between ‘on-site’ production effects and 'off-site' environmental effects. It is suggested that the former are already covered by ‘standard’ economic impact assessments, but that measurement and assessment of larger scale off-site impacts had been discounted or ignored. The claim that local, on-farm effects are routinely included in conventional impact assessments implies that only environmental changes that affect the economics of farming are counted, whilst others are discounted or ignored. The CGIAR approach as described here does not appear to address the challenges of an integrative approach. The framework provided in this particular CGIAR publication focuses exclusively on assessment of off-site environmental impacts and does not incorporate social or environmental impacts at farm level. This contrasts with the SIDA initiative or indeed earlier efforts by other agencies such as DFID.

Bill & Melinda Gates Foundation’s Environmental Sustainability Planning 4.3. In 2011, the Bill & Melinda Gates Foundation (BMGF) set out a distinctive approach characterized as Environmental Sustainability Planning (or ESP) for BMGF grants in agricultural development (BMGF 2011). The approach shares with SIDA the assumption that environmental impacts can and should be appraised during early-stage project planning and in M&E systems, rather than retrospectively, as suggested by the CGIAR approach. The overarching purpose is to design grants, or components of grants, such as specific activities or MLE indicators, with the deliberate intention to address the sustainability of the approach (Kate Schneider pers com). Schneider also reports that use of the ESP tool is intended to contribute to building the evidence base for the positive environmental contributions of investments. For BMGF, the addition of the ESP concept reflects the belief that agricultural productivity must account for changes (positive or negative) that occur to the social, economic, and environmental support systems that underpin the well-being and wealth of rural societies. The types of environmental parameters that are included, as candidates for relevance and appraisal appear to reflect systems’ framing that recognizes complex ecosystem interactions and interdependencies. The use of ESP is recommended, but is not a requirement, for all agricultural projects. Like the SIDA approach, the BMGF framework is structured in clearly delineated stages, with an initial brief assessment to determine whether or not the project could have identifiable eco-system effects, and if so, the scope and depth of subsequent review. Where impacts on environmental sustainability are deemed relevant, a systemic approach is applied, with the system understood in comprehensive terms, scaling across both space and time. The approach also aspires to being symmetrical between risks and opportunities. It also recognizes uncertainties, and issues of spatial and temporal scale and the possibility of transgressing thresholds. A ‘risk assessment’ matrix, in which activities are to be considered in terms of their baseline resource context and potential impacts, underpins the assessment. Levels of risk are described according to scale, temporality, reversibility, etc., if a range of potential impacts (positive or negative) were to occur. However, the risk assessment is more a probabilistic assessment of risk –and not an analysis of uncertainty. While cross-scale issues are in the checklist, it is less clear what the tools and specific guidance are for this. There are, however, external technical assistance resources in place to work with POs and (potential) grantees to assess alternatives, evaluate trade-offs, and help guide decisions around the best pragmatic actions to include in the grant design (within activities or MLE). These include dedicated crop-environment briefs composed by crop experts to guide assessments and known best practices. Additionally, a team of faculty and PhD students at one of the leading Agricultural and Natural Resource Economics programs provides proposal reviews, works with (potential) grantees to optimize the

ALINe working paper

!

!14

activities for the primary objective of the grant and environmental sustainability and design MLE plans to monitor environmental impacts (positive or negative). One other area in which there appears to be opportunity for further development concerns the breadth of participation. The main responsibility for ESP, and for using the tool-kit provided, lies with the Program Officers (POs) who are direct employees of the BMGF. Whilst grantees may be required to help with the completion of the ESP, and experts may be recruited to assist in selecting environmental criteria, there is limited scope for the direct involvement of beneficiaries in the process. While consultation with identified beneficiaries is a general requirement, their engagement in deliberations about possible environmental impacts and their implications from the outset (for example during the establishment of a theory of change for the project) is not an explicit requirement. The BMGF ESP is explicitly designed to guide a conversation between the PO and the (potential) grantee. The grantees are the ones generally working with the tools - often as a structured framework to guide discussion, such as in a workshop planning session. They are designed to be flexible and to guide (potential) grantees in their internal conversations, their discussions with the PO, and to conduct an assessment and make decisions about how to incorporate the results into the grant design. External technical assistance is available to work with POs and/or grantees, but is more likely to work directly with the grantee (Schneider pers com). This would seem to be an important area for further development - such that practical experiential knowledge can become central to the process. Whilst the SIDA approach recognizes community adaptive capacity as a key indicator of sustainability, this, and other elements of capacity building are not highlighted in the ESP approach. Furthermore, there may be scope to develop the reflexivity of the approach. The ESP tools appear unlikely to lead the PO to question the intervention vis-à-vis alternatives (or the zero alternative). This is in contrast to the SIDA approach for which identification and assessment of alternatives to the project is a step in the EIA process with the documentation stating the following ‘Note the importance of being open for completely different solutions than those assumed in the project proposal.’ One particularly helpful indication provided by the BMGF document, however, concerns the practical demands that implementing the ESP might pose. The BMGF ESP documents explain that the workload of POs has been a primary concern that has guided the design of the ESP tools, and emphasize the development of a tool that need not be onerous to use. While extending deliberations about which environmental parameters may be important to the local communities where interventions are taking place, they may add to the work required of grantees and POs. The marginal extra cost in terms of time and effort could well, however, pay a handsome dividend in terms of diminished harm and opportunities for synergistic benefits. 5. Empirical findings: Reflections from Pilot study in West Kenya and Ethiopia A pilot study of two regionally-based agricultural development projects in Africa was conducted to examine a) how and why environmental considerations were taken into account in project planning and M&E activities, b) how different stakeholders perceived or prioritised environmental impacts, c) to consider the institutional barriers for the incorporation of environmental issues into monitoring and evaluation, and d) to explore the extent to which the case study examples demonstrate the importance of the six, often neglected, elements of environmental monitoring and evaluation identified in section 4 above (integrative approaches; cross scale analysis; dealing with uncertainty; participation; capacity building; reflexivity). The two projects to which visits were made were:

1) Strengthening the Ethiopian Agricultural Extension System (SEAES):

An agricultural research and development initiative operated jointly by the Sasakawa Africa Association (SAA) and Oxfam America (OA), working in close collaboration with the Ministry of Agriculture and with funding from the Bill and Melinda Gates Foundation (BMGF), SAA and OA are currently working together to strengthen the agricultural extension system in Ethiopia. Following a series of diagnostic studies carried out by the International Food Policy Research Institute (IFPRI) in 2010, the key areas of focus on extension service improvements that are followed by SEAES, include strengthening farmer driven orientation across all levels of extension, broadening the range of extension services offered, equipping Pastoralist/Farmer Training Centres (P/FTCs) with necessary facilities and

ALINe working paper

!

!15

equipment such as new farming tools and machines and educational resources, and improving Development Agents (DAs) knowledge and capabilities (Ling et al., 2011).

The government of Ethiopia has a target of establishing 15,000-18,000 P/FTCs throughout the country, while the SEAES project aims to work with 215 P/FTCs and the train 645 DAs. In order to align the work of SAA and OA, and periodically assess progress of the SEAES, a Joint Monitoring Evaluation and Learning (MEL) framework has been developed through a collaborative process bringing together key stakeholders from the MoA, SAA and OA (Ling et al., 2011).

The study team conducted interviews with project stakeholders involved in different ways with the SEAES project. Interviews with stakeholders from the MoA and OA took place in Addis Ababa, while the views of those involved in project execution and delivery were gathered during a series of field trips to the P/FTCs, at regional, kebele and farm levels. DAs, farmers and agronomists involved in extension service delivery were also interviewed.

2) Appropriate Rural Development Agriculture Programme (ARDAP):

ARDAP is a local Kenyan non-governmental organization based and working in Busia County, close to the Kenya-Uganda border. Established in 2000, ARDAP has developed a niche in advocating, sharing and exchanging information for sustainable agriculture in Western Kenya (ARDAP, 2009). Working with smallholder farmers in Busia County, ARDAP works to improve food security, build capacity for sustainable livelihoods, improve local seeds, and contribute to economic and social development along with environmental conservation. Environmental concerns thus have a central role in the work of ARDAP as exemplified by the ARDAP core mission of ‘safeguarding and enhancing ecological health for improved livelihoods through research and extensions services, and the adoption of technologies by local communities to provide food security’ (ARDAP, 2009: 3).

The way ARDAP operates at its headquarters in Butula Division, Busia County, is by using its facilities as a farmer learning and development centre, with field demonstration plots from which farmers can learn directly about the added value of adopting specific seeds or farming practices. During fieldwork, discussions and interviews took place with ARDAP members, village residents, and representatives of farming associations that are working in partnership with ARDAP.

Institutional barriers to the adoption of environmental M&E 5.1.

Much of the literature on environment and M&E does not recognise the significance of the institutional context in influencing key decisions about M&E in practice. In this respect, the case studies illustrate a number of institutional factors that play a role in shaping decisions concerning environmental M&E. In the case of ARDAP, a focus on the ‘livelihoods’ framework appeared to determine how project staff perceived what activities were deemed important to focus on. In an interview with a project worker of the GORTA project (a soil and water conservation project), it was highlighted for instance, that project implementation sites were chosen on the basis of the project's ‘need’ to target poorer farmers as a first priority. In the ARDAP project reports, there is frequent reference to terms such as improvement of ‘livelihoods’, ‘community mobilisation’ and ‘farmer capacity building’, that share many conceptual similarities with the work of Sen and subsequent sustainable livelihoods literature (see also previous section 2.3 on environmental entitlements and description of sustainable livelihoods). As discussed in the literature review above, cumulative effects appear to play a less important role in this approach. This raises the question of whether underpinning philosophies and disciplinary frameworks of human-environment interactions can to some extent influence how project workers, in the specific context of M&E, understand environmental effects. An exciting opportunity arising from these findings would be to look at complementarities between the uses of an existing framework with others. For example, the ‘ecosystem services approach’ may perform better in terms of integrating environmental effects from a more systematic or cross-scale point of view (see previous section on ecosystem service approach 2.2). An obvious extension of the current analysis would be to further interrogate the complementary nature of several frameworks, including those presented in Section 2, in the context of some detailed project case study examples. This might lead to a re-casting of environment impacts within the M&E process of agricultural development projects, and potentially stimulate a dialogue concerning additional complementary theoretical and practical insights, which may support future initiatives.

ALINe working paper

!

!16

In the case of the SEAES, that has addressed project M&E fairly systematically, stakeholders expressed concerns of a different nature. Even though there is an M&E put in place, during interviews project stakeholders expressed their concerns about not having one specifically for the environment. The substance of the argument was summarised by one of the SEAES M&E specialists: ‘On environmental assessment, we have all the ingredients, know where we are going but we don’t have the mechanisms to monitor impact’. This exemplified a situation where attention was often directed to the lack of ‘tools’ and ‘methodologies’ to conduct environmental M&E. However, it should be noted, that as environmental aspects are not made explicit in the articulation of SEAES objectives, it therefore follows that there is no specific plan for the integration of such methodologies. This creates an unfavourable institutional environment for SEAES to be able to extend its mandate beyond the formally stated components of strengthening the agricultural extension system. Whilst it is perhaps possible to envisage a situation whereby the scope of M&E is redefined to include the necessary tools for addressing environmental aspects, the research suggests that one also has to consider the underlying politics of conducting environmental M&E in the first place. For some agricultural practitioners in Ethiopia there is resistance within the policy community for integrating EIA into project activities, fearing that the social and environmental impacts may be too serious and could lead to ‘termination’ of a planned project activity. Given the current policy enthusiasm to stimulate production in the Ethiopian agricultural sector, such a statement may not be so far from truth. The MoA has set some very ambitious targets for increasing agricultural outputs in the coming years. This was encapsulated in a comment from one of our respondents ”Shoot for 100 but lets be happy with 50, the ministry is already thinking about millions!” It is important to consider how prevailing policy frameworks and narratives shape the scope of environmental M&E and, therefore, to anticipate that political constellations, both old and new, may not always be conducive for mainstreaming environmental M&E.

Cross – scale analysis: Interactions between agricultural and ecological changes 5.2.across spatial and temporal scales

The two case studies reflected very different views regarding the ‘scale’ over which environmental impacts are deemed important and are incorporated into M&E. In the Kenyan case, some environmental effects were taken into account. However, they were framed in very local terms, reflecting a rather limited account of the kinds of environmental effects, which ideally should be included in an environmental monitoring and evaluation strategy. The ARDAP project emphasises delivering ‘on farm’ improvements intended to replenish and enhance soil fertility2. Measures included terracing3, water harvesting, establishment of tree nurseries and vegetable gardens in ‘demonstration plots’ with smallholder farmers. Other activities revolved around improving crop productivity at individual farm level. However, given the strong emphasis on individual farm improvements (and the implicit focus on crop yield improvements in particular), there was little consideration of cumulative environmental effects that are likely to arise in the event that agricultural productivity is substantially enhanced. Field extension officers acknowledged that applying increasing amounts of fertiliser is likely to have wider environmental impacts, but the M&E plan did not include indicators to measure such effects. A soil health expert working for Ethiopia’s Agricultural Transformation Agency (ATA) explained how agricultural extension practitioners are not thinking seriously about fertilizer use because of a widely shared view that farmers use ‘so little’ fertiliser in the first place. Because of this logic, there is a growing sense of uncertainty around larger environmental impacts associated with fertilizer use. One respondent stated ‘we know the numerator (fertilizer imports), but not the denominator (i.e. area of land cultivated with fertiliser use)’. In SEAES, there is a different emphasis in terms of scale considerations. Discussions with project partners from Oxfam America and the Ministry of Agriculture in Ethiopia, suggests that at a national scale, the interest in monitoring environmental impacts is becoming stronger. One respondent summarised current policy thinking on environment in Ethiopia as such:

”Largely we continue to implement projects (meaning agricultural development projects) that look at productivity, but often fail to look at what happens to the soil and the environment. But we are beginning

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!2!Deforestation!and!poor!land!stewardship!within!the!Sio_Malaba_Malakisi!sub_catchment!(part!of!Western!Kenya)!has!accelerated!

soil!erosion!and!topsoil!loss!leading!to!reduced!soil!fertility!and!reduced!crop!yields!The!Republic!of!Kenya.,!(2010)!Programme)for)agriculture)and)livelihoods)in)Western)Communities)in)Kenya)!3!A!method!of!shaping!land!to!control!erosion!on!slopes!of!rolling!land!used!for!cropping!and!other!purposes!

ALINe working paper

!

!17

to notice that rain is declining, as well as the fertility of the soil is diminishing. To add to the environmental impacts, agricultural development carries social impacts too”.

A number of practical challenges to implementation are presented at regional and sub-regional levels that pose further obstacles to ensuring the effectiveness of environmental M & E. Although Ethiopia's 2011 Climate Resilient Green Economy Plan (CRGEP)4 encapsulates some of the current policy enthusiasm to prioritise environmental concerns, the type of expertise needed to mainstream a better assessment of environmental impacts within the country, particularly in the context of agricultural extension work, is largely missing. This becomes apparent at the kebeles, where the Ethiopian government has established farmer-training centres (FTCs). Here dialogue between farmers and DAs on environmental monitoring has a much more marginal role than the CRGEP national discussions seem to suggest. On the one hand, the focus at the kebele is on the infrastructural improvements (such as providing desks, computers and demonstration plots and equipment) required for training farmers. These interventions are pursued particularly by OA that is providing a large part of the funding for the FTCs. On the other hand, as was explained by DAs, FTCs are primarily perceived as sites of knowledge exchange on new agricultural technologies. Both these positions however, appear far more detached from environmental concerns than the CRGEP plan might suggest.

Integrative Approaches - Linking environmental M&E with ‘contextually specific 5.3.outcomes’

A key question, which arises from the case studies, is whether the current approach to the assessment of environmental impacts provides sufficient scope for the delivery of contextually appropriate outcomes. This requires consideration of environmental impacts beyond those that are directly measurable by established agronomic techniques. The extent to which this can become possible depends partly upon whether farmers’ views can feed into M&E practice. In Ethiopia, this does not seem to be the case at the moment, since the main source of information in the current agricultural extension M&E system usually comes from agronomic expertise, which has become institutionalised at different policy levels. To elaborate on the issue of expertise further, it is important to take a closer inspection of the roles of Subject Matter Specialists (SMSs)5. From the perspective of an SMS respondent working on soil and water conservation at the Debre Leibanos woreda, the ‘environment’ is simply addressed as part of a ‘checklist’ of targets (mostly of an agronomic nature). It emerges from the fieldwork, that the majority of these targets are ‘quantitative’ (e.g. numbers of trees planted in the year, number of gullies built to prevent soil erosion etc.) and have been developed on the basis of limited consultation with farmers. In fact, as it was explained by SMSs, the majority of these targets are indeed proposed and reviewed at the national level by the MoA. In this reporting structure then, the SEAES project has proposed an important innovation that attempts to bring farmers' voices into the M&E system. This is achieved primarily through periodically gathering qualitative and quantified perceptual data that incorporates farmer feedback and satisfaction with a range of extension services, including individual technologies, trainings and demonstration events, and also with the performance of DAs and SMSs functioning at kebele and woreda level. However, this new system is still in an initial pilot stage and as such has not influenced wider practices across the extension system that relies on established quantitative methods. This research highlighted that the way that SEAES is currently framed, reinforces the separation between those activities that are meant to deliver poverty alleviation outcomes linked to agricultural extension6, and environmental improvements based largely on agronomic expertise and quantitative targets. This can result in important interactions being overlooked in the M&E process. For example, in Debre Lleibanos woreda, a central !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!4!!Part!of!this!thinking!is!reflected!in!Ethiopia’s!‘Climate!Resilient!Green!Economy!Plan’!(CRGEP),!prepared!in!advance!of!the!COP17!climate!conference!in!Durban,!South!Africa.!CRGEP!retains!a!strong!focus!on!the!agricultural!sector,!and!has!prioritised!a!set!of!initiatives!to!reduce!soil!based!emissions!from!agriculture!and!limit!agricultural!expansion!into!forested!areas!(Federal!Democratic!Republic!of!Ethiopia.,!2011).!!Given!that!Ethiopia!is!an!‘anchor!country’!for!the!BMGF!and!for!several!others!donors!based!in!Addis!(including!Oxfam!America!and!Sasakawa!Africa!Association),!a!discussion!is!on_going!on!‘high!level’!coordination!and!alignment!between!donors!and!government!on!agricultural!and!environmental!protection.!!5!! Subject!Matter!Specialists!are!based!at!the!woreda!level,!in!the!offices!of!Agriculture.!They!have!an!important!role!in!technically!

supporting!the!work!of!Development!Agents!(DAs).!SMS!also!have!a!role!in!reporting!back!up!to!the!Regional!Bureau!of!Agriculture!and!from!there!on!to!the!Ministry!of!Agriculture!(MoA)!on!the!actual!situation!of!extension!service!delivery!on!the!ground.!Different!types!of!specialisms!are!covered,!including!agricultural!science,!soil!and!water!conservation,!natural!resource!management.!!6!!Note!that!the!SEAES!is!a!combination!of!two!partner!projects:!OA!and!SAA.!OA!focuses!on!agricultural!system!(such!as!infrastructure!improvements!at!the!FTCs)!and!SAA!on!delivery!of!‘innovative’!technologies!to!the!farmers!through!the!extension!system.!A!key!focus!of!the!M&E!is!to!be!able!to!link!these!two!initiatives.!!!

ALINe working paper

!

!18

Ethiopian highland zone, loss of rainfall (due to loss of the short, ‘belg’ rainy season), was an important livelihood dimension for the farmers because it had an impact on the diversity as well as quantity of crops that farmers in the region were able to produce. Farmers also considered deforestation important, primarily because the loss of tree coverage also meant that farmers became vulnerable to the risks of flooding and soil erosion. It remains to be seen whether the new M&E system that is being introduced will capture such concerns.

Integrating critical moments of learning in the M&E process 5.4.!Our study demonstrates a number of missed opportunities to address environmental effects arising from agricultural development interventions in an integrated, participatory and systematic manner. Recognition of the importance of environmental factors is not entirely absent, but there is very little clarity on what can be achieved in practice to include environmental concerns and to integrate them with other objectives. This is partly explained by the fact that the linkages between environmental effects and a particular agricultural intervention may not always be immediately apparent or readily understood by practitioners engaged directly in a project. Consequently, critical ‘moments for learning’ on environmental effects can be missed from M&E despite their importance in enhancing the contributions of a project. In the Kenyan case study, it emerged that ARDAP was working closely with farmers and learnt to recognise the high priority accorded by them to various environmental factors, such as soil erosion. ARDAP was later able to accommodate these factors, at least to some extent, in their plans and activities. The missed opportunity lies in the fact that this learning was never incorporated into ARDAP’s M&E reporting structure. It was rather only communicated to us informally during interviews. As a result, activities with significant environmental impacts (e.g. the free supply of chemical pesticides and fertilisers to farmers) were absent from any form of M&E activity. Similarly, information about fertiliser impacts did not appear to be incorporated into the reports shared with ARDAP's funders. In the Ethiopian case as well, a far larger initiative implemented through governmental structures, there was even less evidence that such learning is incorporated into their project activities. Clearly, this is not to dismiss the added value for farmers from the supply of fertilisers. Rather, it is to bring attention to the rather unclear protocol for accounting for their environmental impacts in the case studies described. In contrast, there was considerable emphasis on ‘teaching’ farmers, as opposed to learning from farmers' experiences with different practices. A field extension officer of ARDAP explained that, for instance, ‘the environmental benefits of the technologies7 are known to us. We are more interested in making sure farmers know how to use them’. However this is an important omission because farmer appraisals can actually yield surprising insights. For example, farmers in Bumutiru village, one of the pilot villages for the ARDAP project, outlined a number of factors that may influence their ability to cope with environmental change after the introduction of the new seeds (see section 3.5). In particular, they noted changing rainfall patterns and a very pragmatic concern about how new seed varieties would perform under conditions of climate change. Although these issues were deemed important by farmers, and to some extent recognised by ARDAP extension workers, it was felt that these concerns didn’t ‘fit-in’ with what ARDAP is meant to deliver. In ARDAP's GORTA project, a more successful learning approach was realised. The adoption of a participatory approach early on in the process became a successful strategy for improving the learning of ARDAP project officers on how to ‘steer’ their activities towards delivering positive environmental gains that are likely to be relevant to farmers. As a field extension officer for this project explained, ‘we only realised that the main cause for low crop production in the villages was soil erosion (associated with high incidence of flooding in the rainy season and the consequent removal of the top soil), after we conducted interviews with farmers and talked to village elders’. They further explained that assistance to farmers in safeguarding farms from soil erosion subsequently became a core focus of the GORTA project, meeting its core objective around tackling land degradation for the concerned villages. Although there are other complex dimensions to the conduct of environmental M&E that were perhaps not touched upon here, the case study suggests that there are critical moments of learning on environmental impacts of agricultural research projects that appear to be largely undocumented by project stakeholders. Another moment for project officers to incorporate learning from the beneficiaries was in the context of involving local youths in the building of water conservation structures (e.g. ‘terracing’). As a senior member of the team explained, ‘we realized early on that when one thing is done for you (i.e. the building of terraces) you will not know how to do it yourself in the future. Instead, we decided to pay the local youths to do the job, as opposed to hiring

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!7!!Referring!to!the!ability!of!ISFM!(seeds!combined!with!rhizobium!bacteria)!to!improve!soil!fertility!while!increasing!crop!yields!for!staple!crops!such!as!soy!and!cow!beans.!!

ALINe working paper

!

!19

labour from other villages’. Reviewing their approach thus helped them re-evaluate priorities, while simultaneously improving the ability of farmers to cope with land degradation. As one field officer noted, ‘initially we wanted to scale-up and cover as many villages as possible. But there were things that were unknown to us. We didn’t know that the work would be so tedious.’ Some of the project activities such as the building of terraces and water harvesting tanks, for example, proved to be much more labour intensive than anticipated. This learning acquired by project officers led to adopting a ‘softer’ approach to scaling-up, focusing much more on building effective engagement with a smaller number of villages before extending the remit of operations. This suggests that the learning obtained from farmers is actually valued quite strongly even though it may not be documented. As one member stated ‘now we are learning from the lessons learnt and turning them into opportunities’. A next step here, and an opportunity in terms of future appraisals of environmental effects, is to document this learning in a more systematic way and to integrate it into the wider M&E process.

Temporal dimensions 5.5.!Equally important are learning processes taking place beyond the funding period of an agricultural intervention. Generally, the level of involvement of project staff beyond the end of a project tends to become more limited. As it was highlighted during an interview with a coordinator of a number of large internationally funded agricultural projects in Ethiopia, ‘a lot of post-project work is based on assumptions. To really get a sense of what is happening we would need to re-visit project sites for 5 years or more. But we don’t do that, there are no funds for this type of work’. This is an important shortcoming, since for some environmental impacts, such as those associated with climate change (i.e. changing rainfall patterns), it may be difficult to evaluate their long term significance within the time frame of an agricultural intervention. In this case, post-project engagement may turn out to be particularly important. For instance, in Debre Lebanos, a woreda that the Ethiopian SEAES programme is currently focusing on, farmers often discussed how changing rainfall and temperature increase is gradually contributing to the loss of the short (‘belg’) rainy season and further explained how this has meant that they have to rely on only one rainy season in the summer (‘kiremt’) instead of two. This is a growing area of concern because crops are now receiving less rain and, as a consequence, it has become more difficult for farmers to grow fodder crops for livestock or to diversify their cropping system. Similarly, in Bumutiru village, West Kenya, one of the sites of the ARDAP GLP project, one of the village elders said, ‘With farming it takes time. We cannot know precisely whether GLP is helping us increase farming productivity unless we follow results over a number of harvest seasons. We depend entirely on the climate’. Key insights on such issues need to be consolidated at the design stage and during the implementation process so that projects can be modified in light of these considerations. They must also be considered after a project ends, so that information on how beneficiaries’ ability to cope with long-term environmental changes is also taken into consideration. This links to issues around ‘seasonality’ in rainfed agriculture and the impacts climate change is causing, such as more erratic rainfall patterns. Insights suggest that there is a need to think about climate change as an opportunity for improving agricultural systems, and not only as a barrier or distraction from an agricultural intervention’s key objectives. In addition, it is likely that there are missed opportunities for learning from farming communities about adaptation to climate change on the basis of previous adaptation to local environmental change.

ALINe working paper

!

!20

6. References Adger, W.N. (2006). ‘Vulnerability’, Global Environmental Change, Vol.16, pp.268-281 Adger, W. N., Eakin, H. & Winkels, A. (2008). ‘Nested and teleconnected vulnerabilities to environmental change’, Frontiers in Ecology and the Environment, Vol 7, pp.150-157 Adger, W. N., Huq, S., Brown, K., Conway, D. & Hulme, M. (2003). ‘Adaptation to climate change in the developing world’, Progress in Development Studies, Vol 3, p.179 Agyeman, J., Bullard, R. D. & Evans, B. (2002). ‘Exploring the Nexus: Bringing Together Sustainability, Environmental Justice and Equity’, Space and Polity, Vol 6, pp.77-90 Anderson, R.S., Levy, E. & Morrison, B.M. (1991). Rice Science and Development Politics: Research Strategies and IRRI’s Technologies Confront Asian Diversity (1950-1980), Oxford, Clarendon Press Arce, A. (2003). ‘Value contestations in development interventions: community development and sustainable livelihoods approaches’, Community Development Journal, Vol. 38, p.199 ARDAP. (2009) Annual and Financial Report for the Year 2009, ARDAP ARDAP. (2011) Upscaling Green Legumes for replenishing soil fertility and enhancing incomes and livelihoods of small holder farmers, ARDAP Bennett, J. (2011). ‘Advancing ex-post impact assessment of environmental impacts of CGIAR research: conceptual issues, applications and the way forward’ in Measuring the Environmental Impacts of Agricultural Research: Theory and Applications to CGIAR Research, Rome: CGIAR Blaikie, P., Cannon, T., Davis, I. & Wisner, B. (1994). ‘At risk: natural hazards, people's vulnerability, and disasters’, Psychology Press BMGF (2011) ‘User’s Manual Environmental Sustainability Planning (ESP) for BMGF Grants in Agricultural Development’, Version 1.4 (11 April 2011), BMGF Brooks, S. & Loevinsohn, M.E. (2011). ‘Shaping agricultural innovation systems sensitive to food insecurity and climate change’, Natural Resources Forum, Vol. 5(3), pp.185-200 CGIAR (2011). Measuring the impacts of agricultural research: theory and applications to CGIAR research, Rome, CGIAR Independent Science and Partnership Council, available as http://www.sciencecouncil.cgiar.org/fileadmin/templates/ispc/documents/Publications/EIAStudy2011James.pdf Chambers, R. & Conway, G. (1992). ‘Sustainable rural livelihoods: practical concepts for the 21st century’, IDS Discussion Paper 293. Brighton: Institute of Development Studies at the University of Sussex Chambers, R., Pacey, A. & Thrupp, L.A. (eds.) (1989). Farmer First: Farmer Innovation and Agricultural Research, London: Intermediate Technology Publications CIAT (2009). Annual Report 2009: An Eco-efficiency Imperative for Tropical Agriculture. Cali: CIAT Conway, G. R. (1987). ‘The Properties of Agroecosystems’, Agricultural Systems, Vol. 24, pp.95-117 Eakin, H. & Wehbe, M.B. (2009). ‘Linking local vulnerability to system sustainability in a resilience framework: two cases from Latin America’, Climatic Change, Vol. 93, pp.355–377 Devereux, S. & Longhurst, R. (2010) ‘Incorporating Seasonality into Agricultural Project Design and Learning’, IDS Bulletin, Vol. 41(6), pp.88-95 Ericksen, P. J. (2008). ‘Conceptualizing food systems for global environmental change research’, Global Environmental Change, Vol. 18, pp.234-245

ALINe working paper

!

!21

Federal Democratic Republic of Ethiopia. (2011) Ethiopia's Climate-Resilient Green Economy: Green economy strategy, Addis Ababa Fitzhugh, H. & Brader, L. (2002). Core Funding for Systemwide and Ecoregional Programmes: Draft submitted to the CGIAR on 31st March 2002, Washington DC: CGIAR Folke, C. (2006). ‘Resilience: the emergence of a perspective for socio-ecological systems analyses’, Global Environmental Change, 16, 253-267 GallopÌn, G.C. (2006). ‘Linkages between vulnerability, resilience, and adaptive capacity’, Global Environmental Change, Vol. 16, pp.293-303 Glaeser, B. (1987). ‘Agriculture between the Green Revolution and Eco-development: Which way to go?’ in Glaeser, B. (ed.) The Green Revolution Revisited: Critique and Alternatives. London: Allen and Unwin Glaeser, B. (1995). Environment, development, agriculture: Integrated policy through human ecology, London, UCL Press Griffin, K. (1979). The Political Economy of Agrarian Change: An Essay on the Green Revolution, New York, New York, Macmillan Press Haber, W. (2011). ‘An Ecosystem View into the Twenty-first Century’, in: Schwarz, A. & Jax, K. (eds.). Ecology Revisited, Netherlands, Springer Hazell, P. B.R. (2009). ‘Transforming Agriculture: The Green Revolution in Asia’, in Spielman, D.J. & Pandya-Lorch, R. (eds.) Millions Fed; Proven Successes in Agricultural Development. Washington DC: IFPRI Holling, C.S. (1973). ‘Resilience and stability of ecological systems’, Annual Review of Ecology and Systematics, Vol. 4, pp.1-23 Hukkinen, J. (2001). ‘Eco-efficiency as abandonment of nature’, Ecological Economics, Vol. 38, pp.311-315 IAASTD (2009). Agriculture at a Crossroads: Synthesis Report - International Assessment of Agricultural Knowledge, Science and Technology for Development, Washington DC, Island Press Jones, P.G. & Thornton, P.K. (2009). ‘Croppers to livestock keepers: livelihood transitions to 2050 in Africa due to climate change’, Environmental Science & Policy, Vol.12, pp.427-437 Kasperson, R.E. & Kasperson, J.X. (2001). Climate change, vulnerability and social justice. Stockholm Environment Institute, Stockholm Keating, B. A., Carberry, P.S., Bindraban, P.S., Asseng, S., Meinke, H. & Dixon, J. 2010. ‘Eco-efficient agriculture: Concepts, challenges, and opportunities’, Crop Science, Vol. 50, pp.S-109-S-119 Kerr, J. (2002). ‘Watershed development, environmental services, and poverty alleviation in India’, World Development, Vol. 30, pp.1387-1400 Korhonen, J. & Seager, T. P. 2008. Beyond eco-efficiency: a resilience perspective. Business Strategy and the Environment, 17, 411-419 Lal, R. (2010). ‘Enhancing eco-efficiency in agro-ecosystems through soil carbon sequestration’, Crop Science, Vol. 50, p.S-120 Leach, M., Mearns, R. & Scoones, I. (1999). ‘Environmental entitlements: dynamics and institutions in community-based natural resource management’, World Development, Vol. 27, pp.225-247 Lehni, M., Pepper, J. & World Business Council for Sustainable Development (2000). Eco-efficiency: creating more value with less impact, WBSCD

ALINe working paper

!

!22

Ling, A., Van Hemelrijk, A. & Dirirsa, B., (2011) Joint MEL Framework for Strengthening Agricultural Extension in Ethiopia MoA, SAA, OA Loevinsohn, M.E. (1987). ‘Insecticide Use And Increased Mortality In Rural Central Luzon, Philippines’, The Lancet, Vol. 329, pp.1359-1362 MEA (2005). ‘Ecosystems and Human Well-being: Synthesis’. Island Press, Washington DC O’Brien, K., Sygna, L., Leichenko, R., Adger, W.N., Barnett, J., Mitchell, T., Schipper, L., Tanner, T., Vogel, C. & Mortreux, C. (2008). ‘Disaster Risk Reduction, Climate Change Adaptation and Human Security: A Commissioned Report for the Norwegian Ministry of Foreign Affairs’, GECHS Report 2008:3, Oslo: University of Oslo Oasa, E. K. 1987. The Political Economy of International Agricultural Research: A Review of the CGIAR’s response to criticism of the Green Revolution. In: Glaeser, B. (ed.) The Green Revolution Revisited: Critique and Alternatives. London: Allen and Unwin Park, S., Howden, S. & Crimp, S. (2009). Is eco-efficiency enough to tackle food security under a changing climate?, Conference Material for CGIAR Science Forum 16-17 June 2009, Netherlands, CGIAR Science Forum Oxfam America. Strengthening the Ethiopian Agricultural Extension System (SEAES), Oxfam America Pearse, A. (1980). Seeds of Plenty, Seeds of Want: Social and Economic Implications of the Green Revolution, Oxford, Clarendon Press Power, A. G. (2010). ‘Ecosystem services and agriculture: tradeoffs and synergies’, Philosophical Transactions of the Royal Society B: Biological Sciences, Vol. 365, p.2959 Pretty, J., Toulmin, C. & Williams, S. 2011. ‘Sustainable intensification in African agriculture’, International Journal of Agricultural Sustainability, Vol.9, pp.5-24 Renkow, M. (2011). ‘Assessing the environmental impacts of CGIAR research: toward an analytical framework’, in Measuring the Environmental Impacts of Agricultural Research: Theory and Applications to CGIAR Research. Rome: CGIAR Independent Science and Partnership Council Renting, H., Rossing, W., Groot, J., Van der Ploeg, J., Laurent, C., Perraud, D., Stobbelaar, D. J. & Van Ittersum, M. (2009). ‘Exploring multifunctional agriculture. A review of conceptual approaches and prospects for an integrative transitional framework’, Journal of environmental management, Vol. 90, pp.S112-S123 Richards, P. (1985). Indigenous Agricultural Revolution, London, Hutchinson The Republic of Kenya. (2010) Programme for agriculture and livelihoods in Western Communities in Kenya (PALWECO), The Republic of Kenya Schmidheiny, S. (1992). Changing course: A global business perspective on development and the environment, MIT Press Scoones, I. (1998). ‘Sustainable rural livelihoods: a framework for analysis’, IDS Working Paper 72, Brighton, Institute of Development Studies at the University of Sussex Scoones, I. (2009). ‘Livelihoods perspectives and rural development’, Journal of Peasant Studies, Vol. 36, pp.171-196 Scoones, I. & Thompson, J. (1994). Beyond Farmer First: Rural Peoples' Knowledge, Agricultural Research and Extension Practice, London: IT Publications Scoones, I. & Thompson, J. (2009). Farmer First Revisited, London, Practical Action Sen, A.K. (1981). Poverty and Famines, Oxford, Oxford University Press

ALINe working paper

!

!23

Stevenson, J., Byerlee, D., Villoria, N., Kelley, T. & Maredia41, M. (2011). ‘Agricultural technology, global land use and deforestation: a review and new estimates of the impact of crop research’, in Measuring the Environmental Impacts of Agricultural Research: Theory and Applications to CGIAR Research, Rome: CGIAR Swallow, B.M., Sang, J.K., Nyabenge, M., Bundotich, D.K., Duraiappah, A.K. & Yatich, T.B. (2009). ‘Tradeoffs, synergies and traps among ecosystem services in the Lake Victoria basin of East Africa’, Environmental Science & Policy, Vol. 12, pp.504-519 Swift, M.J., Izac, A. & van Noordwijk, M. (2004). ‘Biodiversity and ecosystem services in agricultural landscapes--are we asking the right questions?’ Agriculture, Ecosystems & Environment, Vol. 104, pp.113-134 Tansley, A.G. (1935). ‘The use and abuse of vegetational concepts and terms’, Ecology, Vol. 16, pp.284-307 Thompson, J., Millstone, E., Ely, A., Marshall, F., Shah, E. & Stagl, S. (2007). Agriculture. Agrifood system dynamics: pathways to sustainability in an era of uncertainty, IDS/SPRU/ University of Sussex Thornton, P.K., Jones, P.G., Eriksen, P.J. & Challinor, A.J. (2011). Agriculture and food systems in sub-Saharan Africa in a 4°C+ world Philosophical Transactions of the Royal Society, 369, 117-136 Tittonell, P., Vanlauwe, B., Leffelaar, P., Rowe, E.C. & Giller, K.E. 2005. ‘Exploring diversity in soil fertility management of smallholder farms in western Kenya: I. Heterogeneity at region and farm scale’, Agriculture, Ecosystems & Environment, Vol. 110, pp.149-165 Turner, B. L., Kasperson, R. E., Matson, P. A., McCarthy, J. J., Corell, R. W., Christensen, L., Eckley, N., Kasperson, J. X., Luers, A., Martello, M. L., Polsky, C., Pulsipher, A. & Schiller, A. 2003. A framework for vulnerability analysis in sustainability science. Proceedings of the National Academy of Sciences, 100, 8074-8079 Turton, C. (2000). ‘Sustainable livelihoods and project design in India’, ODI Working Paper 127, London, Overseas Development Institute UN FAO (2011). Save and Grow: A policymaker’s guide to the sustainable intensification of smallholder crop production, available at http://www.fao.org/ag/save-and-grow/index_en.html van der Werf, H.M.G. & Petit, J. (2002). ‘Evaluation of the environmental impact of agriculture at the farm level: a comparison and analysis of 12 indicator-based methods’, Agriculture, Ecosystems & Environment, Vol. 93, pp.131-145 Vanlauwe, B. & Giller, K.E. (2006). ‘Popular myths around soil fertility management in sub-Saharan Africa’, Agriculture, Ecosystems & Environment, Vol. pp.116, 34-46 Wesonga, M., Masinde, A., Musumba, B. & Obiero, B., (2009) GORTA Project Baseline Survey Report, Busia, Kenya White, R., Tunstall, D. & Henninger, N. (2002). An ecosystem approach to drylands: building support for new development policies, Washington, DC, World Resources Institute Wisner, B., Blaikie, P., Cannon, T. & Davis, I. (2004). ‘At risk: natural hazards, people's vulnerability and disasters’, Psychology Press (Second edition). Zandstra, H. & Taylor, P. (2006). ‘Farming systems research: a retrospective consideration of the importance of ecological scale and stakeholder participation’, Ottawa: CRDI

ALINe working paper

!

!24

!

ALINe (Agricultural Learning and Impacts Network) 3 Dean Trench Street London SW1P 3HB T: +44 207 148 0914 E: info@aline.org.uk W: www.aline.org.uk