1

On Cooperation, Compliance and Performance in International Water Management Thomas Bernauer, ETH Zurich Tobias Siegfried, Columbia University Authors: Thomas Bernauer ETH Zurich, Center for Comparative and International Studies (CIS), Seilergraben 49, SEI - G.9, 8092 Zurich, Switzerland, Phone: +41-44-632-6466, Fax: +41-44-632-1289, bernauer@ir.gess.ethz.ch Tobias Siegfried Earth Institute, Columbia University in the City of New York, 500 W. 120th St. Room 842A, New York, NY 10027, USA, Phone: +1-212-854-1695, ts2293@columbia.edu Correspondence: Thomas Bernauer

Acknowledgements We are grateful to Andrey Yakovlev, Uzbek Hydrometeorological Service, and the Global Runoff Data Center, Federal Institute of Hydrology (BfG), Koblenz, Germany, for providing the hydrological data. We are also grateful to Christoph Schär, Jean Fried, Detlef Sprinz, Gabi Ruoff, Vally Koubi, Oran Young, Gary Goertz, Aaron Wolf, Thomas Hinkebein, Nils Petter Gleditsch and Peter Bauer for valuable comments on earlier versions of this paper.

2

On Cooperation, Compliance and Performance in International Water Management

Abstract Many case studies and some large-N research have shown that upstream-downstream cooperation in international river basins occurs quite frequently. We argue that such findings are blind on one eye because they focus solely on policy-output or compliance with international agreements. We present a policy performance metric (PER) that allows for a more substantive assessment of success or failure in international water management. To test its empirical relevance, we apply this metric to the Naryn / Syr Darya basin, a major international river system in Central Asia. Management of the Toktogul reservoir, the main reservoir in the Naryn / Syr Darya basin, was internationalized in 1991 when the Soviet Union collapsed. Compliance with an international agreement, concluded in 1998, is quite high. This agreement establishes an international trade-off among water releases for upstream hydropower-production in winter and water releases for downstream irrigation in summer. Our analysis shows that the performance of this agreement over time is very low and highly variable. The principal policy-implication is that the management system in place is in urgent need of reform. The more general message is that many international upstream-downstream water agreements may rest on “shallow” or unstable cooperation, and that solving such problems is usually a matter of decades. Keywords: International cooperation, compliance, performance, water management, Syr Darya, Toktogul dam

3

1. Introduction Cooperation, when defined as a dependent variable in causal explanations of international water management, is usually measured in binary terms (agreement, treaty, international institution; yes/no). Examples can be found in the many qualitative case studies on international water management (e.g., Marty 2001, Bernauer 1996) and the few large-N quantitative studies that exist on the subject (e.g., Durth 1996, Brochmann and Gleditsch 2006, Dombrowsky 2005, Tir and Ackerman 2004, Wolf et al. 2005, Yoffe et al. 2004). Many case studies also include some assessment of how substantive international cooperation is. But the criteria against which the “depth” or substance of cooperation is measured differ across studies, and assessments are usually qualitative (see Bernauer 2002, 1995). Moreover, most assessments rely on non-causal criteria. The most common approach is to describe, over time, the development of a particular problem targeted by a cooperative effort (e.g. pollution) and to assess compliance with international obligations. This is usually done without systematic analysis of whether changes in environmental outcomes and in compliance levels have, ceteris paribus, been affected by international cooperation. Coding of the contents of international agreements for purposes of measuring the “depth” or substance of cooperation in large-N analysis is still in its infancy (Mitchell and Rothman 2006, Dombrowsky 2005; Conca et al. 2006). Another approach has been to code cooperative and conflictive events among riparian countries (e.g., Wolf et al. 2005, Brochmann and Gleditsch 2006). This approach offers only indirect insights into the depth of cooperation. International water management efforts are to some extent directly included in the codings of cooperative events. Moreover, deep cooperation may often be accompanied by conflict events. More cooperative than conflictive events may thus tell us little about whether international cooperation performs well in terms of problem solving. Another line of research uses environmental parameters as proxies for cooperation. Sigman (2004) and Kuhn and Bernauer (2006), for example, examine whether trade ties promote international efforts to clean up water pollution. Since environmental outcomes are measured without causal reference to international cooperation (cleaner transboundary water is simply assumed to indicate more cooperation), this approach does not offer direct insights into how successful cooperation is. Substantial progress has been made in recent years in measuring the performance (or “depth”) of international cooperation. In the first part of this paper we outline a methodology for estimating the performance of international cooperation, building on work by Helm and Sprinz (2000), Hovi, Sprinz, and Underdal (2003), Sprinz and Helm (2000), and Underdal (1992). Our policy performance metric (PER) is a time-dependent function of the outcome that should ideally be reached (optimum), the performance of a given policy at the time of measurement (actual performance), and the outcome that would have occurred in the absence of this policy (counterfactual performance). The advantages of this measurement concept are: first, it makes explicit reference to optimal performance and thus problem solving; second, it focuses explicitly on the causal relationship between international policies and outcomes; third, it can be used to assess international policy performance at specific points in time in contexts marked by rather little data, but also to assess performance dynamics over time in contexts where more data exist; fourth, cooperative efforts can be disaggregated with reference to particular objectives, policy performance can then be measured for each objective separately and can then be aggregated or not.

4

The broader relevance of our approach is twofold. First, it addresses an ongoing debate in International Relations about situation structures and their effects on international cooperation (Mitchell and Kielbach 2001). In respect to international water policy, this debate has concentrated on the difficulties of overcoming upstream-downstream settings where preferences of the countries involved are often antagonistic. Recent quantitative and qualitative research (e.g., Wolf et al. 2005, Marty 2001) suggests that upstream-downstream cooperation is quite frequent. However, the empirical evidence remains controversial. For example, Brochmann and Gleditsch (2006) find contradictory effects of upstream-downstream settings on international cooperation. International cooperation in water issues, according to their analysis, is more likely in upstream-downstream than in other settings in the time period 1820-2001, but the effect is insignificant in the sample period of 1975-2001. When cooperation is measured by treaty signing (instead of treaty existence), the effects are largely insignificant. Brochmann and Gleditsch also find that upstream-downstream settings produce both more cooperative and more conflictive events. The latter result tells us that such settings lead to more interaction, but it does not tell us whether such settings facilitate or hinder cooperation. Other, more process-oriented studies show that compensation or issue-linkages to offset upstream-downstream asymmetries are often difficult to construct and that cooperation, if it emerges at all, remains “shallow”. Tir and Ackerman (2004) conclude that international water treaties are less likely in upstream-downstream settings. Bernauer (1996, see also Bernauer and Moser 1996) shows that it took the riparian countries of the river Rhine several decades to reduce (upstream-downstream) water pollution, and that driving forces other than international cooperation have been key. If upstream-downstream cooperation is very difficult among highly developed democratic countries, we should expect even greater difficulties in achieving similar levels of cooperation in less fortunate regions of the world. Second, our approach concentrates on problem solving in that it relates policy-outcomes to specific notions of what should ideally be achieved and what would have happened without cooperation. It thus produces a more accurate and policy-relevant diagnosis. This, in turn, provides a better foundation for finding ways and means of making cooperation more effective. To demonstrate the empirical relevance of our concept, we examine international water management in the Naryn / Syr Darya basin, a major international river system in Central Asia. The analysis focuses on the Toktogul reservoir, the main reservoir in the Naryn / Syr Darya basin, and its downstream effects. The principal policy challenge in this case has been to design and implement an international trade-off among water releases for upstream hydropower-production in winter and water releases for downstream irrigation in summer. We start by examining institutional outcomes and the compliance-related behavior of riparian countries. This analysis shows that a detailed agreement was concluded in 1998, and that compliance with this agreement is high. We then apply our measurement concept, bearing in mind possible endogeneity problems associated with assessments of compliance – compliance may be high only because international obligations are weak and cooperation is therefore “shallow”. In other words, this second step allows us to assess whether good news about compliance is also good news about cooperation (Downs, Rocke, and Barsoom 1996). This analysis shows that the 1998 agreement is characterized by low performance and high variability. The principal policy-implication is that, even though compliance is high, the management system in place is in urgent need of reform. We discuss some options for improvement. The more general message of our analysis is that many international upstream-downstream water

5

agreements may indeed rest on “shallow” cooperation, and that no quick fixes to such problems can be expected.

2. Measuring Performance

Our starting point is a simple formula suggested by Sprinz and Helm (2000, see also Underdal 1992). It is defined as

i

AP CPPER

OP CP

!=

! (1)

where AP captures actual performance, CP stands for counterfactual performance, and OP for optimal performance.1 The subscript i denotes the ith criteria with regard to which PER is estimated. In international water management, such criteria may relate to hydropower production, irrigation water provision, water quality, or water provision for ecosystems functions.2

PER can be estimated in relation to any public demand addressed by a public policy. In effect, this equation captures the extent to which a given problem has actually been solved (AP - CP) relative to the problem solving potential (OP - CP). The first term alone would only tell us that the relevant policy or regime has had some effect. Only by adding the second term (and OP in particular) do we gain information on the extent to which the problem has been solved. Moreover, adding the second term (OP – CP) facilitates comparisons across policies within and across policy-domains, and over time: provided we distinguish between maximizing (CP AP OP! ! ) and minimizing (CP AP OP! ! ) cases it sets a lower and upper bound and (with some exceptions) standardizes PER values between 0 and 1.

A more complex version of the above formula has been developed by Siegfried and Bernauer (2006). It solves some conceptual problems in the simple formula (e.g., overcompliance scenarios and inefficiencies associated with them) and allows for the measurement of performance and its variation over time. It starts with the following definition

( )( ) ( )

( ) ( )*

1AP t OP t

PER tCP t OP t

!= !

! (2)

where PER*(t) is a measure of policy performance at time t . PER*(t) measures performance relative to optimal performance OP at a specific observation time t . If we use the notation

( ) ( ) ( )APt AP t OP t! = " and ( ) ( ) ( )CP

t CP t OP t! = " , then Equation (2) becomes

( )( )

( )*

1AP

CP

tPER t

t

!

!= " (3)

by the definition of the absolute value and its properties. If CP(t)<AP(t)<OP(t) or CP(t)>AP(t)>OP(t), it is easy to see that the two performance measures as defined by Equations

1 The names of the variables we use differ from the original. 2 For purposes of simplification, the subscript is dropped in the following parts.

6

(1) and (2) are equal, i.e. PER*(t)=PER. Note that PER*(t) is symmetric around OP(t) and that, according to Equation (3), PER*(t) is defined as long as ( ) 0

CPt! " .

Estimating performance over time means that we must look at AP, CP and OP in terms of times-series data. In our context, we regard the times series AP(t), CP(t) and OP(t) (as well as the derived δAP(t) and δCP(t)) as finite realizations of underlying stochastic processes. In the subsequent analysis, we restrict our focus to stationary processes. We use the expected value as well as the variance of PER*(t) to characterize policy performance over time. Siegfried and Bernauer (2006) show that the expected value of ( )*

PER t can be approximated by

*

2

11 Cov( , )AP

CP CP

AP CPPER

!

! !

µ! !

µ µ= " + (4)

where Cov( , )AP CP

! ! denotes the covariance and AP

!µ as well as

CP!

µ the mean of the times series

( )APt! and ( )CP

t! .

The variance of ( )*PER t can be approximated by

*

2 22 22

2 4 4 3

4 2Cov( , )Cov( , )AP CPAP AP

CP CP CP CP

AP CPAP CP

PER

! !! !

! ! ! !

µ "" ! ! µ! !"

µ µ µ µ= # # # (5)

In Equations (4) and (5), AP

!µ ,

CP!

µ , 2

AP!

" , 2

CP!

" and Cov( , )AP CP

! ! have to be estimated empirically from available data.

3. International Water Management in the Syr Darya Basin

For purposes of demonstrating the empirical applicability of our approach we focus on the Syr Darya river basin. First, this case involves a clearly circumscribed upstream-downstream conflict. Second, the main conflict and cooperation issue involves water allocation. This facilitates measurement of the parameters in our metric and also enables us to systematically compare the results from a compliance-based and a performance-based assessment of international cooperation. Such comparison would be more difficult (but still feasible) for multi-dimensional international cooperation problems (see Rieckermann et al. 2006). Third, we were able to obtain new data for the Syr Darya case, providing us with an opportunity to systematically assess performance of this management system for the first time. Fourth, the Syr Darya case is interesting because it allows us to study the transition from a top-down domestic water management system (in Soviet times) to a more horizontal, international water management system. The Syr Darya river originates as the Naryn river in the mountains of Kyrgyzstan (see Figure 1). It then flows through Uzbekistan and Tajikistan and ends in the Aral Sea in Kazakhstan. Its total length is around 2’800 km. Around 20 million people inhabit this river catchment, which covers an area of around 250’000 km2. The river is mainly fed by snowmelt and water from glaciers. The natural run-off pattern, with annual flow ranges of 23.5 – 51 km3 (around 40 km3 in the past

7

few years) is characterized by a spring / summer flood. The latter usually starts in April and peaks in June. Around 90% of the Syr Darya’s mean annual flow is nowadays regulated by storage reservoirs. Approximately 75% of the run-off comes from Kyrgyzstan (Dukhovny and Sokolov 2005). Water abstraction from the Syr Darya basin is mainly for irrigated farming.

Figure 1: Naryn and Syr Darya catchment

The run-off of the Naryn / Syr Darya, as measured at the Uch Kurgan gauge station, i.e. at the foot of the Naryn / Syr Darya cascade shortly after the river enters Uzbekistan from Kyrgyzstan, varies strongly over time. As shown in Figure 2, it is characterized by four distinct periods. When the run-off was natural (1933–1974), i.e. determined entirely by seasonal and climatic variability, mean flow was around 390 m3/s, with a high variability in summer. A substantial change in flow patterns occurred with the commissioning of the Toktogul dam in 1974.3 This event marks the beginning of the first river management period (1974 – 1990) in our analysis. This period was characterized by centralized management by the USSR of the Toktogul reservoir and the river basin as a whole. The Toktogul dam is by far the largest storage facility in the Aral Sea basin. It has 14 km3 effective capacity, 8.7 km3 firm yield, and a full capacity of ca. 19.5 km3. The reservoir area is around 280 km2, its length around 65 km.4 Hydropower capacity of the Toktogul power plant is 1’200 MW, i.e. the second biggest in the Aral Sea basin (Antipova et al. 2002). After the commissioning of the dam, a general attenuation of peak downstream flows was observed (see Figure 2). Moreover, an overall decline of monthly flow variability occurred. This decline was most pronounced in the summer months. During this first management period, the system was oriented primarily towards water provision for irrigated agriculture (above all, cotton production) in Uzbekistan and Kazakhstan. The timing of winter and summer flow releases did not change substantially compared to the natural runoff 3 Some smaller reservoirs downstream of Uch Kurgan, notably the Kairakkum and Chardara reservoirs, had been put in place earlier. 4 In the whole Naryn / Syr Darya basin, total usable reservoir capacity is around 27 km3.

8

pattern. This is indicated by seasonal ratios r of inflow vs. outflow that oscillate around r = 1 (see inflow/outflow ratios in Figure 3 in 1980 – 1990).

Figure 2: Mean monthly flow of the Naryn / Syr Darya river at the Uch Kurgan gauge. Data Sources: Global Runoff Data Center (GRDC) and Andrey Yakovlev, Uzbek Hydrometeorological Service.

In the early 1980s, a water management organization for the Naryn / Syr Darya was set up in Tashkent, Uzbekistan. Its mandate was to operate and maintain all head water structures with a discharge of more than 10 m3/s. This management system and its infrastructure was fully funded from the federal budget of the USSR. In consultation with the governments of the riparian republics and based on forecasts by the Central Asia Hydromet Service, the ministry of water resources (Minvodgoz) in Moscow defined annually (based on a multi-year master plan) how much water was to be released for irrigation during the growing season (April to September). The aforementioned organization was responsible for implementing the water allocations and maintaining the infrastructure. It also had the authority to increase or reduce allocations to each Soviet republic by up to 10%. The electricity produced at Toktogul during that period went into the Central Asian Energy Pool (CAEP) and was thus shared among the riparian republics. In exchange, the neighboring republics supplied coal, oil, and natural gas to Kyrgyzstan in winter to cover increased Kyrgyz energy demand during the colder months. The fossil fuel was used primarily in thermal power plants in Bishkek and Osh (Cai, McKinney, and Lasdon 2002).

The collapse of the Soviet Union in 1991 led to the breakdown of centralized water resources management and water-energy trade-off arrangements and caused disputes between the newly independent states over water allocation. The second river management period, as depicted in

9

Figure 2, commenced. Coal, oil, natural gas, and electricity supplies to Kyrgyzstan declined dramatically between 1991 and 1997, and so did the thermal and electric power output of Kyrgyz thermal power plants (TPP).5 Consumers turned to electricity. This, in turn, increased winter demand for electricity by more than 100%. Purchases of energy from abroad were (and still are) difficult because the government was (for political and administrative reasons) unable to increase and collect appropriate energy tariffs. Moreover, financial contributions from Moscow and the former republics in the basin for the maintenance of the reservoir ceased. In response to the sharp drop in thermal power output and rising winter demand for electricity, Kyrgyzstan switched the operation of the Toktogul reservoir from an irrigation to an electric power production mode. Since winter 1993, the flow peaks no longer occur in summer but rather in winter. This has led to the opening up of a gap between the summer inflow/outflow ratios r and their winter counterparts (see Figure 3). The main political problem since 1991 has been of an antagonistic upstream-downstream nature. Upstream interests deriving from temporal water demands are diametrically opposed to downstream water demands and interests. Kyrgyzstan uses very little water consumptively, i.e. for irrigation. But it is interested in producing hydro-electricity at the Toktogul power plant, particularly in winter when energy demand is higher. Kyrgyzstan has no fossil fuel sources of its own. Thus it cannot rely on domestic fossil fuel for electricity production and thermal energy. Its interest in hydropower production has become ever stronger as the downstream countries have cut back their energy supplies to Kyrgyzstan. Kyrgyzstan also views electricity production as a potential export commodity. Hence it is eager to store water in spring to autumn and release this water in winter to spring for energy production. Conversely, downstream Uzbekistan and Kazakhstan, by far the largest consumers of irrigation water in the river basin, are interested in obtaining much more water during the growing season (April to September) than in the non-growing season (October to March). They are also interested in electricity produced upstream through water releases during the growing season for operating irrigation pumps. Moreover, downstream countries prefer low water releases in winter because high flows in winter may cause floods: ice in the river bed reduces water flow capacity (Savoskul et al. 2003).

The principal problem to be solved therefore pertains to coordinating the management of the Naryn / Syr Darya cascade of reservoirs that are located entirely in Kyrgyzstan, and in particular the handling of trade-offs between consumptive water use for downstream irrigation purposes and non-consumptive use for upstream energy production in Kyrgyzstan.

5 Thermal power output from Kyrgyz TPPs in 1991 - 1997 declined from 5.8·106 Gcal to 2.8·106, electric power output from 3.9 to 1.6 M kWh.

10

Figure 3: Ratios r of inflow to outflow from the Toktogul reservoir, averaged over three months. The switch from cooperative to non-cooperative water resources management is characterized by the opening up of a gap between winter (months 1-3 and 10-12) and summer (months 4-6 and 7-9) inflow/outflow ratios from 1991 onwards. The pronounced peaks in the summer month ratios characterize years of above average summer runoff.

Non-cooperative management during the second period led to winter spills from the river, which have damaged infrastructure and land resources in downstream Uzbekistan. Moreover, they have reduced the potential for water releases for irrigation during the vegetation period. Ever since 1991, the riparian countries have been struggling to re-establish an effective management scheme (Savoskul et al. 2003).

International negotiations focusing on the management of the Toktogul reservoir began shortly after the demise of the USSR. In February 1992 the five newly independent riparian countries of the Naryn / Syr Darya basin set up the Inter-State Commission for Water Coordination (ICWC). They agreed to keep the water allocation principles of the former USSR in place until a new system could be established, albeit without the funding for the infrastructure that had formerly come from Moscow. The most important hydraulic structures, and in particular the biggest reservoirs in the basin (including the Toktogul), were not put under the control of the ICWC. That is, they were de facto nationalized by the newly independent countries.

This period of unilateralism continued until March 1998, when under the aegis of the Executive Committee of the Central Asian Economic Community and assisted by USAID, Kazakhstan, Kyrgyzstan, and Uzbekistan signed an agreement. This agreement marks the beginning of Period 3, as defined in Figure 2. In 1999 Tajikistan joined this agreement.6 The agreed release schedule is shown in Table 1.

6 http://ocid.nacse.org/cgi-bin/qml/tfdd/treaties.qml

11

Month 1 2 3 4 5 6 7 8 9

q [m3/s] 495 490 300 230 270 500 650 600 190

Table 1: Release schedule of Toktogul reservoir as established in the 1998 treaty. No values were defined for the months of October to December.

The 1998 agreement includes a general framework agreement and a specific barter agreement on energy-water exchanges in 1998. The second agreement holds that in the growing season (April 1 – October 1), Kyrgyzstan will supply 2.2 M kWh of electricity to Kazakhstan and Uzbekistan (1.1 M kWh each). Kazakhstan and Uzbekistan, in exchange, agree to deliver specific amounts of electricity, natural gas, fuel oil, and coal to Kyrgyzstan in specific months under conditions set forth in bilateral agreements concluded already in 1997. Compensation can also be carried out in the form of “other products” (labor and services are mentioned) or money. Kyrgyzstan agreed to cut its energy consumption by 10% against 1997 levels. The framework agreement, also concluded in March 19987, holds that these exchanges will subsequently be defined annually through negotiations. In other words, the water management system put in place in 1998 holds that during the vegetation period Kyrgyzstan releases more water than it needs for its own hydro-power demand, and that the energy surplus is distributed to Kazakhstan and Uzbekistan. In the non-growing period (October 1 – April 1) Uzbekistan and Kazakhstan supply Kyrgyzstan with energy resources in amounts that are approximately equivalent to the electricity they receive from Kyrgyzstan during the growing season. The exact amounts of water and energy are defined annually through negotiations among the countries. Typically, Kyrgyzstan has been scheduled to release around 6.5 km3 of water during the vegetation period and transfer around 2.2 M kWh of electricity to Uzbekistan and Kazakhstan.

4. Compliance From the viewpoint of policy-output, the 1998 agreement is good news. Its design follows the pattern of other international upstream-downstream water agreements in that it seeks to cope with strong asymmetries of interests through economic exchanges. In contrast to cases with similarly strong upstream-downstream asymmetries, for example the river Rhine chloride case or the Colorado salination case (Bernauer 1996, Marty 2000), where it took the riparian countries several decades to arrive at an international agreement, the 1998 agreement for the Syr Darya was established already seven years after the collapse of the Soviet Union.

In moving beyond the simple “agreement, yes/no” and “how long did it take to come to an agreement” measurement of international cooperation, compliance is an obvious candidate. Measuring compliance tells us to what extent the riparian countries have fulfilled their obligations under the 1998 agreement. In the Syr Darya case, this requires an assumption. The release schedule, as shown in Table 1, was set for the year 1998, with schedules for subsequent years to be negotiated annually. Because no information could be obtained on whether the 1998

7 http://ocid.nacse.org/cgi-bin/qml/tfdd/treaties.qml

12

schedule was changed in the following years we assume that the 1998 schedule constitutes the benchmark for compliance from 1998 onward. To assess compliance, we computed ratios of actual water releases from the Toktogul reservoir (three month averages) and the targets for the respective months as defined in the 1998 agreement. Figure 4 shows the results. The average compliance scores (1 = perfect compliance) were 1.6 in 1980-90, 1.1 in 1991-97, and 0.9 in 1998-2006. The overall picture we obtain from this assessment is that compliance is high, particularly in spring to autumn, and somewhat lower (by 25%) in winter. We will return to the latter issue in the concluding section of this paper.

Figure 4: Compliance with the 1998 agreement. Compliance c in months 1-3 (January to March) is defined as the 1998 target for these months divided by the actual water release; compliance in months 4-6 and 7-9 is defined as the actual water release divided by the 1998 target. These definitions are based on the assumption that exceeding the target in the growing season is better for downstream countries than exceeding the target in winter. We also show the results for the years before 1998 to point out the general trend, though the 1998 agreement was, of course, not in force before 1998.

As noted by Downs, Rocke, and Barsoom (1996), even high levels of compliance in international regulatory regimes may not necessarily be good news about international cooperation. The problem is that measuring cooperation in terms of compliance suffers from endogeneity and selection problems. States often define treaty commitments so that meeting them requires little or no effort above and beyond what the states concerned would do in the absence of the respective international commitments. Consequently, a low level of compliance could still implicate very substantial international cooperation if the states involved have engaged in very ambitious commitments. Conversely, high levels of compliance could implicate very shallow cooperation if commitments merely register the status quo ante of state behavior. To deal with this problem, the empirical application of our measurement concept for policy performance in the following

13

section uses OP and CP rather than the 1998 treaty targets as benchmarks against which actual state behavior is compared.. 5. Performance

In the period of centralized water management under USSR rule (Period 1, 1974 - 1990), mean annual flow was reduced from 388 m3/s to 311 m3/s, mainly due to the filling of the Toktogul reservoir. The characteristics of the yearly averages do not differ substantially from the natural flow. There still is a summer discharge peak and winter low flow, but due to the filling of the reservoir the summer peak is less pronounced. This characteristic flow pattern changes after the breakdown of centralized governance (Period 2). Figure 5 shows average runoff data per month for the natural runoff period and the three management periods.

Figure 5: Monthly long-term average flows at the Uch Kurgan gauge (based on data from GRDC and Andrey Yakovlev). The data on flow variability for the corresponding months and periods is shown in the Appendix. The monthly data µ(optim.) are calculated optimal releases from the Naryn / Syr Darya cascade. Optimization was carried out with a coupled hydrologic-agronomic-economic model on the basin scale by Cai, McKinney, and Lasdon (2003).

As discussed above, the increased hydropower demand in upstream Kyrgyzstan led to a pronounced increase of reservoir water releases in the winter months. The somewhat reduced monthly variability in flow characterizes the unilateral upstream management of the Syr Darya run-off. With the implementation of the 1998 agreement in Period 3, monthly flows seem to reflect the trade-offs made in that agreement. Average flow is now 396 m3/s, with a considerable decline in monthly variability compared to the prior period. We approach the measurement of performance in two steps. First, we apply the simple performance formula (equation 1) to long term average runoff data. Second, we use the more complex performance metric.

Whereas AP(t) is clearly defined in terms of the water releases in Period 3 (1998-2006), CP and OP need to be constructed. We define the period of breakdown of the centralized management

14

system in 1991–1997 (Period 2), i.e. the period where there was no international agreement, as counterfactual performance, i.e. CP(t). The assumption is that the riparian countries would have continued to behave as they did in 1991-1997 had the 1998 agreement not been reached. Another approach to measuring CP could be to assume unconstrained (by the downstream countries or actors from outside the basin) maximization by Kyrgyzstan of hydropower production to cover domestic energy needs and export excess energy to obtain foreign currency, and to carry out a simulation-optimization (from the Kyrgyz perspective) on that basis. Discussions with experts on the region led us to the conclusion that such a scenario would have been very unlikely, and that the scenario of CP in terms of Kyrgyz behavior along the lines observed in 1991-97 would have been more likely in the absence of the 1998 agreement. As to the definition of OP, we have three options: the natural runoff regime, runoff under Soviet rule, estimates from optimization models. The natural runoff, OPN(t), is arguably the most problematic measure of the three since it is quite difficult to see why no regulation of river flow should be Pareto-improving on a properly operated dam (in our case the Toktogul dam). However, we use this measure of OP for purposes of comparison. The second measure for OP assumes that centralized management in Soviet times (Period 1, 1980-19908) was optimal (OPS(t)) because up- and downstream interests were addressed through an integrated water-energy exchange system. Interviews with experts on the region confirmed that the exchanges of water and energy under the Soviet management system worked relatively well, both in terms of providing water for irrigation downstream and facilitating energy production upstream. The disadvantage of the second measure of OP is that from the perspective of the long-term Aral Sea problem and local economic and environmental interests there, Period 1 was certainly not optimal9. We thus employ also a third notion of optimality, OPC(t), which emphasizes sustainability of natural resources management at the basin scale. µ(optim.) in Figure 5 (see also Appendix) is not observed but is the result of a simulation-optimization approach that we denote as OPC(t). This simulation was undertaken by Cai, McKinney, and Lasdon (2003) and McKinney, Cai, and Lasdon (1999). It considers risk minimization in water supply, environmental conservation of soil and water resources, spatial and temporal equity in water allocation and economic efficiency in the development of future water infrastructure. The full optimization scenario operates under the assumption of long-term average precipitation in the basin and determines monthly reservoir releases, infrastructure development, irrigated crop patterns and area with the objective to maximize the resulting sum of irrigation and ecological benefits and hydropower profits.

We start with a simple estimation of PER based on equation (1). The results are shown in Figure 6. For all three measures of OP and for almost all months of the year, performance is very low.

8 We start with 1980 because 1974-1979 were years of reservoir filling. We expect a strong trend effect in reservoir outflows during the latter years - this would introduce errors in our performance estimation. 9 Young (2001, 2003) argues that definitions of the optimum with reference to which performance is assessed must not necessarily be based on objective notions, but can depend on understandings of the nature of the problem and the options available for solving the problem.

15

Figure 6: PER1 is based on OPN(t), PER2 on OPS(t), and PER3 on OPS(t). The calculation is based on long term averages for each month – see Table 1, Appendix.

We now move to the more complex approach and use the following notation to distinguish the scaling of PER*(t): ( )*

SOP

PER t is calculated with respect to OPS(t) and ( )*

COP

PER t with respect

to OPC(t). We restrict the estimation to OPS(t) and OPC(t) since OPN(t) is a rather problematic measure for optimum performance. To compute the performance PER*(t) of the international management system installed in 1998 we use monthly averaged flow values for OPS(t) and CP(t) (see Table 2). This is necessary for two reasons. First, comparing individual hydrological years with differing resource endowments (i.e., inflow as well as reservoir levels) and demand (for electricity as well as irrigation water) is problematic. Doing so would lead to an arbitrary comparison of reservoir outflows between years that are not necessarily comparable with respect to the above mentioned state variables. Second, the individual periods have different lengths. Hence, they cannot be compared directly.

Month 1 2 3 4 5 6 7 8 9 10 11 12

ì (OP S (t) ) [m3/s] 245 251 231 281 457 593 790 583 213 176 203 248

ì (CP(t) ) [m3/s] 497 487 454 338 310 377 402 322 197 242 352 502

Table 2: Mean monthly flows for Period 1 (1980-1990, µ(OPS(t)), and Period 2 (1991-1997), µ(CP(t)). As to µ(OPS(t)), we do not take into account the initial years of reservoir filling (1974-1979). µ(OPC(t)) is shown in the Appendix.

The calculation of PER*(t) based on OPS(t) may be problematic. The underlying assumption is that demand for irrigation water and hydroelectric power has not changed in 1980-2006. This assumption could, for example, be violated by the fact that in Uzbekistan the irrigated area grew

16

from 63.5 10! ha 10 in 1980 to 6

4.4 10! ha in 1998, i.e. by more than 25%. Demand in Kyrgyzstan for hydroelectric power has grown substantially as well.11 We could have addressed this problem by scaling µ(OPS(t)) according to changes in demand for irrigation water and hydroelectric power. We did not do so because there would be a high degree of arbitrariness in such an approach. In particular, the very notion of optimality may loose sense after such scaling since the latter does not take into account inter-seasonal shifts of optimal water allocation. In other words, optimal allocation is not a linear function of the quantity of water available. Note that such a problem does not apply to ( )*

COP

PER t because the according measure of OP reflects recent up-

and downstream demand constraints.

Figure 7: PER*(t) during Period 3 with respect to the two notions of optimality.

The temporal development of ( )*

SOP

PER t and ( )*

COP

PER t is shown in Figure 7. With respect to

both notions of optimality, performance of the 1998 regime has been poor. The figure shows that extremely negative values of ( )*

SOP

PER t start to occur from 2002 onwards, usually in

September. This can be explained by the fact that in this month, ( )( ) ( )( )SCP t OP tµ µ! , i.e.,

the denominator of ( )*

SOP

PER t is small and the difference between actual performance and the

monthly averaged performance of Period 1, i.e. ( ) ( )( )SAP t OP tµ! , is large.

10 See http://www.fao.org/ag/AGL/aglw/aquastat/countries/uzbekistan/index.stm. 11 A similar argument applies to μ(CP(t)).

17

Tables 3 and 4 show the overall results of our performance estimation. To calculate *PER and

*

2

PER! , we need to estimate the sample means !

( )AP!

µ•

, !( )CP

!µ

•, the variances !

2

( )AP

!"•

, !2

( )CP

!"•

and the covariances

Cov!

(!AP

,!CP

)•( )

. ( )• is a placeholder for OPS(t) and OPC(t). The estimated

values for the mean and variance are shown in Table 3. For the covariances, we obtain

Cov!

(!AP

,!CP

)S

= 9640.5 m6 /s2 with respect to OPS(t) and Cov!

(!AP

,!CP

)C

= 1250.1 m6 /s2 with

respect to OPC(t).

( )Abs

APS! ( )

Abs

CPS! ( )

Abs

APC! ( )

Abs

CPC!

unit

µ 259.9 171.3 157.7 85.2 [m3/s]

σ2 15314.4 6932.9 5900.2 3755.7 [m6/s2]

Table 3: Estimated sample means and variances. The times series AP(t) and OPS(t) have been truncated to 7 years for the sample estimations of the mean, variance and covariance values12.

We can now calculate performance and its variance. The results are shown in Table 4.

*

PER *

2

PER!

OPS -0.24 0.64

OPC -0.71 0.92

Table 4: Average regime performance and variance with reference to OPS and OPC. The calculations are based on Equations (4) and (5) and Table 3.

These calculations confirm the visual impression from Figure 7 that the performance of the international management regime for the Naryn / Syr Darya is very low.

6. Conclusion The theoretical literature stipulates that international environmental cooperation in upstream-downstream settings is very difficult (e.g. Mitchell and Kielbach 2001, Bernauer 1995, 1996). But the empirical evidence for this claim remains controversial. Many qualitative case studies

12 OPC as given in Cai, McKinney et al. (2003) is provided as monthly averaged series of values. In the calculations based on this computed optimum, we assume that the monthly values of OPC do not change over the period of assessment.

18

and some quantitative research show that upstream-downstream water cooperation occurs quite frequently. For example, Brochmann and Gleditsch (2006) find no significant negative effects of upstream-downstream settings on the likelihood of cooperation. Such empirical findings are indeed surprising because they suggests that upstream-downstream asymmetries can be overcome through compensation payments and issue-linkages offered by downstream countries in exchange for concessions by upstream countries at reasonably low transactions costs. We argue, however, that such findings may be overly optimistic because they rely on definitions of the dependent variable (cooperation) that do not really capture the substance or depth of cooperation. Our skepticism derives from the argument by Downs, Rocke, and Barsoom (1996) and other authors that international cooperation, as measured by the existence of treaties and compliance with international commitments, may often be more “shallow” than it looks at first sight. In the first part of this paper we presented a measurement concept that seeks to capture the depth or substance of cooperation. Assessing substantively the performance of international water management is important from an academic and a practical viewpoint. Developing and testing generalizable explanations for success and failure in international water management must rely on an accurate measurement of the dependent variable (success/failure). Moreover, helping policy-makers understand whether or not a given water management system performs well is usually the first step towards improving policies and institutions.

To demonstrate its empirical relevance, we applied our metric to the Naryn / Syr Darya basin. We observed that an international agreement was concluded in 1998, already 7 years after the USSR collapsed. This is very fast in international comparison. Moreover, we observed that compliance with this agreement is quite impressive. The agreement establishes an international trade-off between water releases for upstream hydropower-production in winter and water releases for downstream irrigation in summer. When we applied our metric the initially positive picture changed entirely. We observed that the performance over time of the water management system established in 1998 is very low and highly variable.

The principal policy-implication of our findings is that the management system in place for the Syr Darya is in urgent need of reform. Its poor performance notwithstanding, conflicts over water allocation among the riparian countries have in the past few years been muted by high levels of precipitation upstream. The compliance analysis (see above) and Figure 8 (below) show that, due to high precipitation and thus high inflows into the Toktogul reservoir in recent years, excessive water releases from the reservoir in winter have not come at the cost of lower releases in spring to autumn. But as soon as an extended period of low precipitation sets in (due to climate change or for other reasons), seasonal trade-offs will become manifest again and the conflict is likely to heat up very quickly. Systematic analysis of the reasons for poor performance of the existing management system is beyond the scope of this paper. However, the most apparent reasons relate to economic crisis and domestic political instability in the riparian countries. These problems have made it hard to establish credible long-term commitments. Most notably, as long as Kyrgyzstan does not receive credible commitments from the downstream countries that higher levels of water release from the Toktogul dam in spring to autumn (but not in winter) will be followed by energy deliveries by downstream countries in winter, the incentive for Kyrgyzstan to release large amounts of water for energy production in winter and lower amounts in summer will dominate.

19

Figure 8: Water volume of the Toktogul reservoir, in million m3, at the end of March (end of winter), end of October (end of the growing season), and as average of the respective year.

“Structural” (i.e., engineering) solutions to the problem have been proposed and, to a minor degree, already undertaken. Uzbekistan has built several small reservoirs on its territory to retain (excessive) water releases from Kyrgyzstan in winter for use for irrigation in spring to autumn. But there are topographical limits to this solution. Another solution would be to reactivate long-time plans to build a new reservoir upstream of the Toktogul reservoir. Releases from this new reservoir could serve to produce electricity for Kyrgyzstan in winter, and the released water could be retained further downstream in the Toktogul reservoir for release in spring to autumn for irrigation in the downstream countries. Such a solution might work if foreign investors could be attracted to this project, an unlikely prospect for the time being. It is quite obvious, however, that water-energy exchanges among the three riparian countries would be more cost-efficient than hydrological decoupling. International efforts should thus focus on establishing long-term hydrological forecasting systems for the Syr Darya basin. A revised water-energy exchange mechanism, which builds on such forecasting, should include multi-year targets for the management of the Toktogul reservoir. To solve the time-inconsistency problem in this upstream-downstream exchange, guarantees by advanced industrialized countries or international organizations could be established.

The more general message from our analysis is that many international upstream-downstream water agreements may involve more “shallow” or unstable cooperation than is evident at first glance. Indeed, a closer look at a range of prominent cases – e.g., the rivers Rhine, Danube, Colorado – suggests that solving upstream-downstream problems is often a matter of decades. It usually goes hand in hand with growing income and intensifying political and economic ties among riparian countries. In the river Rhine case, for instance, it took half a century to set up a system of international funding for pollution reduction to deal with salination problems. And by the time this exchange was established the problem had been largely solved independently of international cooperation – the main sources of salination, coal and potash mines, were closed for economic reasons. Similarly, efforts to clean up the Danube and other rivers in Europe and North

20

America have developed together with growing income, trade interdependence, and democratization. A quite common pattern, at least in respect to water quality issues, seems to be that domestic public demand for stricter environmental policies grows with income and political and civil liberties (democracy). Stricter domestic standards (and thus lower pollution) tend to foster international cooperation in this area as well. For instance, when country A adopts higher water quality standards and commissions water treatment plants, these measures usually apply to areas near the national border too. As a result, country A’s water flowing into neighboring country B is bound to be of higher quality. To the extent such domestic processes develop in parallel in two or more riparian countries, this will facilitate international cooperation on water quality issues. Further research should study interactions between policy processes at domestic and international levels in order to establish whether policy-output and policy-outcomes in international water management are driven by primarily international cooperation or rather by domestic processes that converge into higher standards internationally.

Appendix

Optim.

month ! ó! ó ! ó ! ó !

1 150.9 27.7 183.9 74.5 478.5 101.1 590.0 55.3 357.7

2 152.2 24.8 196.5 68.9 464.2 113.0 561.8 78.6 426.2

3 178.8 28.6 192.9 49.9 428.9 122.1 465.8 52.9 323.4

4 318.4 92.5 265.7 94.7 350.2 115.6 367.0 79.6 426.2

5 672.2 190.5 443.6 189.7 348.0 120.2 286.8 52.0 452.8

6 987.3 325.8 532.1 205.2 450.1 152.6 270.6 73.8 468.0

7 807.9 258.8 638.5 210.1 481.0 174.5 324.3 78.2 494.7

8 518.1 138.4 518.4 149.6 354.1 79.5 316.6 40.3 490.9

9 289.3 71.2 184.9 96.7 198.5 89.2 228.1 93.0 441.4

10 231.0 49.3 146.5 72.4 234.5 67.7 313.7 86.8 300.6

11 219.0 44.4 143.8 89.0 343.5 51.9 439.4 84.9 304.4

12 176.3 26.8 181.9 80.9 479.7 82.3 590.6 53.0 418.6

Overall 388 307 311 215 384 139 396 141 409

natural runoff regime Period 3Period 1 Period 2

Table 1: Means and standard deviations of monthly flows under different management systems. The bottom row displays overall means and standard deviations for the duration of the management periods. Units are m3/s for µ and σ. The last column shows data from Cai, McKinney et al. (2003).

21

References Bernauer, T. 1995. The Effect of International Environmental Institutions: How We

Might Learn More. International Organization 49/2:351-377. Bernauer, T. 1996. Protecting the Rhine River Against Chloride Pollution. In: R. O.

Keohane and M. A. Levy (eds.). Institutions for Environmental Aid: Pitfalls and Promise. Cambridge, Mass: MIT Press: 201–233.

Bernauer, T., and P. Moser. 1996. Reducing Pollution of the River Rhine: The Influence of International Cooperation. Journal of Environment and Development 5/4: 391-417.

Brochmann, M., and N. P. Gleditsch. 2006. Conflict, Cooperation, and Good Governance in International River Basins. Paper prepared for the workshop ‘Governance and the Global Water System: Institutions, Actors, Scales of Water Governance. Facing the Challenges of Global Change’, 20-23 June 2006, Bonn, Germany.

Cai, X. M., D. C. McKinney, and L. S. Lasdon. 2002. A Framework for Sustainability Analysis in Water Resources Management and Application to the Syr Darya Basin. Water Resources Research 38/6: 21/1-21/14.

Cai, X. M., D. McKinney, and L. S. Lasdon. 2003. Integrated Hydrologic-Agronomic-Economic Model for River Basin Management. Journal of Water Resources Planning and Management 129/1:4-17.

Conca, K. and F. Wu. 2006. Global Regime Formation or Complex Institution Building? Principled Content of International River Agreements. International Studies Quarterly 50: 263-285.

Dombrowsky, Ines. 2005. Conflict, Cooperation, and Institutions in International Water Management. Ph.D. dissertation. Martin-Luther-Universität Halle-Wittenberg, Germany, Faculty of Economics.

Downs, G. W., D. M. Rocke and P. N. Barsoom, 1996. Is the Good News About Compliance Good News About Cooperation? International Organization 50/3: 379–406.

Dukhovny, V., and V. Sokolov. 2005. Integrated Water Resources Management, Experience and Lessons Learned form Central Asia: Towards the Fourth World Water Forum. Tashkent: Inter-State Commission for Water Coordination in the Aral Sea Basin.

Durth, R., 1996. Grenzüberschreitende Umweltprobleme und regionale Integration: Zur Politischen Oekonomie von Oberlauf-Unterlauf-Problemen an internationalen Flüssen. Baden-Baden: Nomos Verlag.

Helm, C., and D. Sprinz. 2000. Measuring the Effectiveness of International Environmental Regimes. Journal of Conflict Resolution 45/5:630-652.

Hovi, J., D. Sprinz, and A. Underdal. 2003. Regime Effectiveness and the Oslo-Potsdam Solution: A Rejoinder to Oran Young. Global Environmental Politics 3/3:105-107.

Hovi, J., D. Sprinz, and A. Underdal. 2003. The Oslo-Potsdam Solution to Measuring Regime Effectiveness: Critique, Response, and the Road Ahead. Global Environmental Politics 3/3:74-96.

22

Kuhn, P., and T. Bernauer. 2006. Does Trade Promote Effective International Environmental Problem Solving? Manuscript, ETH Zurich, Center for Comparative and International Studies.

Marty, F. 2001. The Management of International Rivers - Problems, Politics and Institutions. Frankfurt: Peter Lang.

McKinney, D., X. M. Cai, and L. S. Lasdon. 1999. Integrated Water Resources Management Model for the Syr Darya Basin. Published by EPIC, Almaty, Kazhakhstan: US Agency for International Development.

Mitchell, R. B., and P. M. Kielbach. 2001. Situation Structure and Institutional Design: Reciprocity, Coercion, and Exchange. International Organization 55/4:891-917.

Mitchell, R. B., and St. B. Rothman. 2006. Creating Large-N Datasets from Qualitative Information: Lessons from International Environmental Agreements. Prepared for the 2006 Annual Meeting of the American Political Science Association, Philadelphia, September 1-3.

Rieckermann, J., H. Daebel, M. Ronteltap, and T. Bernauer. 2006. Assessing the Performance of International Water Management at Lake Titicaca. Aquatic Sciences (forthcoming).

Savoskul, O. et al. 2003. Water, Climate, Food, and Environment in the Syr Darya Basin. Manuscript, available at: www.weap21.org/downloads/AdaptSyrDarya.pdf.

Sigman, H. 2004. Does Trade Promote Environmental Coordination? Pollution in International Rivers. Contributions to Economic Analysis & Policy 3: Article 2. Available at <http://www.bepress.com/bejeap/contributions/vol3/iss2/art2> 2005, 23 May.

Sprinz, D. 2005. Regime Effectiveness: The Next Wave of Research. Paper prepared for the Conference on the Human Dimensions of Global Environmental Change, Berlin.

Tir, J., and J. T. Ackerman. 2004. To Share or Not to Share: Politics of Cooperation Between Riparian States. Paper presented at the Annual Convention of the International Studies Association. Montreal, 17-20 March.

Underdal, A. 1992. The Concept of Regime "Effectiveness". Cooperation and Conflict 27/3:227-240.

Wolf, A. T., A. Kramer, A. Carius, and G. D. Dabelko. 2005. Chapter 5: Managing Water Conflict and Cooperation. In: State of the World 2005: Redefining Global Security. Washington, D.C.: WorldWatch Institute.

Yoffe, S. B. et al. 2004. Geography of International Water Conflict and Cooperation: Data Sets and Applications. Water Resources Research 40/5:1-12.

Young, O. 2001. Inferences and Indices: Evaluating the Effectiveness of International Environmental Regimes. Global Environmental Politics 1/1:99-121.

Young, O. 2003. Determining Regime Effectiveness: A Commentary on the Oslo-Potsdam Solution. Global Environmental Politics 3/3:97-104.