Measuring Peace: Comparability, Commensurability and Complementarity using Bottom-Up Indicators

Pamina Firchow, George Mason UniversityRoger Mac Ginty, University of Manchester

- The ability to get off at the bus stop nearest home without being attacked

- Being able to trust the police to investigate crimes- Being able to leave your house without possessions being stolen- The freedom to move around in your neighbourhood

- Sample indicators of change chosen in focus groups in selected sub-Saharan countries

Introduction

This article takes, as its starting point, the apparent dissatisfaction among many scholars, practitioners and policymakers with currently available peace indicators (Brück 2014, Gleditsch 2014). This point is important since how peace is measured can impact policy (Guelke 2014; McCartney and Nolan 2014). Political Scientists have demonstrated the powerful influence of monitoring and ranking in international relations (Kelley and Simmons 2015). There have been a number of initiatives seeking to extend the reach and accuracy of indicators (Wilton Park 2014; Ioannou 2015; Merry 2011), although considerable inertia persists. This article discusses one area of innovation – the development of bottom-up indicators that operate at the local level in conflict-affected societies. An interest in bottom-up indicators is in keeping with the micro, local and sociological ‘turns’ that have influenced the study of peace and conflict in recent years (Justino et al. 2013; Mac Ginty and Richmond 2013; Brewer 2010). Some international organizations and donor states have shown an interest in bottom-up research methodologies (Irmer 2009; Narayan 2000). There has been significant momentum in recent years towards a next generation of peace and conflict studies that focuses on disaggregated data and analysis to test and compare domestic factors that lead to violent conflict as well as peace and stability (Raleigh et al. 2010).

1

The principal aim of the article is to engage in conceptual and practical scoping of the potential added value that bottom-up indicators might have in augmenting more traditional top-down indicators and making these more inclusive of marginalized populations and ‘hard to see’ events and processes (Merry 2011). The article can be read as a commentary on many currently available indicators, especially their difficulty in capturing the sub-state level, and measures of positive peace. The article draws on data from an on-going research project, the Everyday Peace Indicators Project (EPI), demonstrating the potential for comparison of these heterogeneous indicators, and exploring the possibilities for complementarity with two different measures of peace: macro indices of conflict, peace and development, and databases that geographically and temporally disaggregate conflict events. While the Everyday Peace Indicators serve as our case study, it is our intention that the article prompts a more general discussion on the issues of complementarity, comparability and commensurability between different types and levels of datasets and indicators.

In order to compare top-down and bottom-up indicators, we examine the EPIs alongside four general indices of conflict, peace and development: the Human Development Index (HDI) and the Global Peace Index (GPI); the geo-temporal datasets we chose are the Uppsala Conflict Data Program’s Geo-referenced Event Dataset (UCDP GED) and the Armed Conflict Location & Event Data Project (ACLED). All of these are well-established and commonly used by scholars and policy-makers when measuring peace and development. By comparison with the Everyday Peace Indicators, we show how bottom-up and top-down indicators can be compatible with other indicators and how national-level indicators can benefit from finer-grained analysis provided by bottom-up tools. We are aware that comparing different indicator systems is not comparing like with like. The EPIs are more comparable with the GPI and HDI indicators given that they intend to measure peace and development using pre-established indicators (albeit on different scales), while UCDP and ACLED focus on the counting of events. It is also worth noting that each system will have a different intellectual heritage based on different priorities and biases. Although the different indicator systems were not established to be directly comparable to one another, it is legitimate to investigate

2

the extent to which they can be compared and how they can strengthen each other.

Importantly, we find that using these measurement tools allows us to make some meaningful empirical conclusions. From the EPI indicators collected in twelve pilots across four countries, we found three main theoretically relevant findings based on empirical data. First, overall, people primarily use security related indicators to define peace. These are closely followed by indicators relating to social cohesion, or indicators that stress relationships and community cohesion, and social activities, such as parties or community gatherings. Interestingly, people chose few human rights related indicators when asked to choose indicators they use to determine whether they are more or less peaceful in their communities in our sub-Saharan pilots. Second, reinforcing findings from previous studies, we found that those communities that were further away in time from violent conflict had a higher amount of positive peace indicators than those closer to violent conflict and crime.1 This finding suggests that the post-conflict areas we studied are dynamic and that perceptions mirror circumstances. Third, the community-identified indicators suggest that the relationship between citizen security and peacebuilding requires further exploration, especially in light of continuing problems of crime and insecurity in ‘post-conflict’ areas. This suggests the need for prudence when thinking of end dates for measuring political violence.

This article proceeds with a section outlining the Everyday Peace Indicators Project, which provides the basis for our analysis of locally sourced, bottom-up indicators of peace and change. The second section illustrates the gulf between top-down and bottom-up worldviews in order to show the scale of the task facing those interested in seeking points of complementarity between different levels of analysis. It goes on to examine the potential for comparability of the EPI indicators and the possibilities for commensurability of this kind of local level data. Finally, it discusses the potential for complementarity of the EPI indicators with other systems to augment top-down indicators with finer grained analysis. We use data from our South African and Ugandan pilots to give examples and showcase our findings throughout the paper. Our focus is on in-country (domestic) peace and often highly-localized versions of that, but it is worth noting 1 For another study with similar findings, see Vinck & Pham 2008

3

that contemporary civil wars and authoritarianism are highly internationalized.

In terms of defining key concepts, we regard top-down and bottom-up as a short-hand for the actual research methodologies, and the epistemologies that lie behind them. Clearly the concepts are more complex than the simple binary suggested by the two terms. We identify bottom-up research as being inductive, localized, interested in granularity, and possibly community-sourced. Top-down approaches are identified as large-scale, often reliant on aggregated and secondary data, with indicators developed by outsiders and less interested in local-level granularity. Positive and negative peace are again a convenient short-hand and mask a complex story. Popularly associated with Johann Galtung, the concepts were actually developed by Quincy Wright in the 1950s and rest on the notion of structural violence or violence that is embedded in social and economic relations (Regan 2014: 346). Thus negative peace refers to the absence of direct violence but the continuation of indirect forms of violence such as discrimination, patriarchy, poverty and preventable disease. Positive peace envisages a situation in which these indirect harms are dealt with, allowing people to reach their potential.

The Everyday Peace Indicators Project

The Everyday Peace Indicators Project was inspired by projects from the ecological sciences that were concerned with the exclusions commonly found in top-down research (Miller 2005; Parkins et al 2001; Kreutzmann 2001). The project sought to construct community-level indicators of peace and change through the community sourcing of indicators. So rather than using pre-existing datasets or indicators as a starting point, we began by asking people to reflect on the conditions of peace in their own communities and to identify indicators that would help them track changes. The everyday indicators discussed in this article provide rich empirical data about what people use in their daily lives to measure their own peace and safety. Thus, for example, people mentioned seemingly anecdotal indicators that came from their everyday experiences: the ability to walk safely from a bus stop; the ability to sleep at home at night without fear of rebel attack; the ability to access public services.

4

The Everyday Peace Indicators Project operates in three local communities in each of South Africa, South Sudan, Uganda and Zimbabwe. The funder specified the African continent as the focus of the study. The communities were chosen to reflect a mix of urban and rural localities, localities that experience extensive or light-touch peacebuilding intervention, and localities that experienced violent conflict in the recent past and less recent past. The communities were chosen to be indicative of post-civil war localities, but there is no pretense that they are representative of all post-civil war societies. Communitiy selection took place in collaboration with our local NGO partners, who were able to advise on issues of access and safety. Once the communities were chosen, community members were selected to participate through convenience sampling and an effort was made to be representative of the community at large.2 Community members were split up into groups of women, men and youth and discussed general questions surrounding how they determine peace and safety in their communities. Focus groups were facilitated by our partner NGOs and by locals who spoke the language of the community and could identify with community members – in some instances additional translation was necessary, but in all cases there was one team member present who understood the primary language of the focus group. Once these focus groups were conducted and recorded, transcribed and translated, long-lists of indicators were generated by the EPI research team for each locality. These long-lists were presented to verification focus groups in each locality that were composed of representatives of the previous three focus groups (men, women and youth). The verification focus groups then decided on a ranked short-list of indicators that would eventually be fed into the surveys.3 We also analyzed selected indicators to determine whether there might be any trends in terms of selection by a particular group (women, men or youth), however we found no clear trend except that the majority of indicators that were chosen came from the adult groups.4 2 Focus groups consisted of a mix of different geographic and economic sectors of the community, as well as a mix of participants based on their level of involvement in the community and wartime experiences. The reason we chose to convenience sample for our focus groups was to ensure accurate representation of the community at large.3 Please see Appendix 1 for all shortlist indicators by community and country.4 There were some outliers such as a high prevalence of final indicators chosen from the youth group in Nimule, South Sudan and a significantly higher prevalence of indicators chosen by women in Atiak, Uganda. When disaggregating for men,

5

The indicators identified in the focus groups were rendered into a series of Likert scale questions. We then used mixed mode surveys (face-to-face, telephone, mobile phones) according to on-the-ground conditions. The EPI longitudinal, panel survey was then repeated longitudinally so as to help construct a timeline of change.5 The research presented in this article will focus on the first phase of the project – the participatory selection process of indicators of peace and safety in these communities, with some preliminary results from the surveys. The indicators selected by communities tell a narrative of the issues that matter to communities, often expressed in a vernacular that does not precisely match commonly-used academic or policy categories. As such, they provide useful insights into the value, but also the problems, of using bottom-up peace indicators and the prospects of such indicators being complementary with established top-down indicators and how these can be strengthened using bottom-up approaches.

A gulf in analyses and narratives

This section illustrates the gulf often found between top-down and bottom-up analyses in relation to peace and conflict, and thus reflects the difficulty of effecting complementarity between data from different sources. A journal article cannot do justice to the different discursive frames, levels of analysis and methodological approaches that can fall under the top-down and bottom up headings. The broad-brush headings of top-down and bottom-up also contain much dynamism and diversity. Despite this, it is possible to highlight substantially different worldviews in the level of analysis of peace and conflict (Mac Ginty and Firchow 2016). To illustrate this gulf in a very brief way, we juxtapose the top-down narratives from major international peacebuilding policy documents published between 1992-2012, and the highly localized narratives produced by the EPI focus groups and indicator selections.6 however, they did not have significantly more indicators chosen than any other group in any of the pilots. These locally-selected indicators were then rendered into a survey that was run among the wider community in that locality.5 For more on the EPI surveys, please see the working paper “Including hard to access populations using mobile phone surveys and participatory indicators” (revise and resubmit with Sociological Methods and Research)6 To ascertain the ‘narrative’ of international peacebuilding, we coded major reports by international peacebuilding actors using Nvivo including: The Agenda for Peace (1992), The Brahimi Report (2000), The Responsibility to Protect (2001), A More

6

Taken together, the major peacebuilding policy documents published by international organizations can be considered as an official transcript of international peacebuilding policy. It is clear that the documents are often reactive to world events (and perceived failures by the international community) and that the earlier documents (pre-2001) are more focused on the role of the United Nations in the maintenance of international peace and interstate war. An Nvivo word count analysis shows that usage of the word ‘peace’ declines after 2001 and indeed it is not listed in the top ten words in Responsibility to Protect (2001), In Larger Freedom (2005) or the World Development Report (2011). The reports after 2001 become much more heavily securitized and the word ‘security’ is prominently displayed in Responsibility to Protect (2001), A More Secure World (2004), In Larger Freedom (2005) and the World Development Report (2011).

Table 1. Top Ten Words in International Peacebuilding Policy Documents, 1992-2012

Interestingly few of the most frequently used words in the reports are reflected in the highly localized discussions and conversations about peace and safety in the Everyday Peace Indicators Project. This is despite the fact that the international policy documents were often aimed at ‘fragile’ societies such as those covered in the EPI project. In the community sourced short and long lists of indicators, we found localized ‘anecdotal’ indicators such as the prevalence of barking dogs (a sign of prowlers), the ability for boda-boda cyclists to move about freely and the possibility to urinate outside at night (signs that there is freedom of movement). Indicators were locally contextualized and vernacular. More pronouncedly, however, it was clear that people used a myriad of factors to determine their peace and safety and didn’t focus exclusively on negative peace indicators. They had ontological notions of peace and security that considered multiple aspects of life and struggled to compartmentalize different aspects of security. Indicators and discussions of peacefulness included issues of security and war and economic development, but also focused on freedom, social cohesion, gender violence, human rights and traditional practices. In addition, these indicators tell stories of the precariousness of life in conflict related contexts. For example, the fact that people in

Secure World (2004), In Larger Freedom (2005), Aftermath of Conflict (2009), New Deal (2011), World Development Report (2011), Governance for Peace (2012)

7

Uganda observe the presence of boda-boda cyclists to assess their relative peacefulness or safety demonstrates how post-conflict networks in these contexts work since many boda-boda cyclists are ex-combatants. Fear to urinate outside at night reflects a clear preoccupation with crime and night-time insecurity as well as possible sexual assaults.

While this has been a very brief illustration of the gulf between top-down and bottom-up worldviews, hopefully, the point is clear: international and local indicators and narratives often do not match. The differences between emic and etic perspectives can have serious political and social consequences with international peacebuilding interventions failing to deal with the needs and expectations of local populations (Roberts 2011). There is some evidence of a ‘local turn’ in the statements and policies of international organizations with more attention being paid to local, inclusive and participatory approaches to peacebuilding, governance and stabilization. The most recent report reviewed, Governance for Peace (2012), contains much usage of words such as “local,” “social,” and “support”. Yet taken as a whole, the international peacebuilding corpus uses a different vernacular to that used by individuals in the Everyday Peace Indicators Project focus groups. Where the themes overlapped, for example on security or governance, the international reports tend to use these as generalized categories while focus group participants used them in localized and sometimes personal ways.

Community-identified indicators can be dismissed as idiosyncratic, anecdotal and hyper localized. Some of the micro-narratives they represent threw up issues that sometimes are discussed only on the margins of policy and academic debates on post-conflict societies: the social impact of alcohol, religious faith, and superstitions. Yet the majority of indicators chosen across the different research sites could be associated with basic needs and aspirations such as safety from violence or access to public services. This encouraged us to think about ways in which the community chosen indicators might operate in conjunction with existing indicator systems and think about the potential for larger trends and comparisons.

Commensurability of EPI Indicators

8

Before considering if the Everyday Peace Indicators can be complementary to other, more established, indicators, it is worth considering the difficulties, and potential, facing commensurability across the EPIs. Because of the inevitably distinct nature of the EPIs – where each indicator is uniquely chosen by a community to reflect its particular measurement of peace and safety – the potential for direct comparison is somewhat limited. For example, a pressing issue in one community may be the violence associated with gang feuds, while in another area it might be a poor harvest or the influx of a large number of internally displaced people. These issues may not resonate in other communities. However, despite the localized differences, there are a few possibilities for comparison and commensuration. First, it was clear that many of the locally-chosen indicators fell into certain categories that could be found across the pilots in different localities and different countries. Therefore, it was advantageous to classify the indicators in these categories, to not only maximize the potential for commensuration, but also to be able to identify trends. Second, indicators or iterations of similar indicators were repeated across communities and countries. As we observed this development, we were careful to word survey questions identically across the pilots. If an indicator was similar, but differed slightly, we included two questions: the original indicator question from that community and the similar question that had been seen in another community. These two opportunities for comparison – indicator categories and repeat indicators – are now discussed in more detail below.

Indicator Categories

In order to identify trends and compare across communities and countries, a series of categories were created in order to classify the indicators (See Table 2). These categories reflect those of positive peace (education, employment, health, human rights) as well as those of negative peace (crime, daily security, security forces), but also include more community level elements such as indicators of cohesion and interdependence, freedom of speech and religion, leadership, social life, dispute resolution and concerns with infrastructure and economic development. The categories were chosen inductively by the grouping of indicators according to a category. In other words, indicators were analyzed to determine what category or dimension they would most appropriately belong to. Our assignment of indicators

9

to categories was done qualitatively and according to a codebook that outlines and defines each category. 7

Table 2. Categories for the Everyday Peace Indicators Project

Repeat Indicators

We analyzed repeat indicators for both our short and long lists of indicators. In terms of the short list of indicators, the most frequently repeated indicators were related to basic needs such as having a safe place to sleep (or being able to sleep at home), having access to healthcare, being able to walk in the street (or going to the store) and going to school (paying school fees) (see Table 3). Repetition of indicators happened across pilots and a repeat indicator in one community did not mean the corresponding repeat indicator was in the same region or even in the same country. Eleven of the repeat indicators were repeated across two communities, but seven were repeated across three communities, five across four communities, four across five communities with the most frequently cited indicator across communities being “walking in the street/going to the shop” out of a total of twenty-eight repeat indicators. It is be possible to directly compare data from those communities that share an indicator.

Table 3. Top Ten Repeat Everyday Peace Indicators (Short List)

Indicator Trends Across EPI PilotsThis section gives a flavor of the indicators most commonly chosen as part of the EPI project. Although daily and ontological security (Mitzen 2006) (i.e. feeling safe, allowing children to play outside, sleeping at home, etc.) was the most prominent category of indicator across countries, reflecting the most persistent category in all four countries, there was disparate attention to security and negative peace indicators across pilots. Not surprisingly, when looking at secondary and tertiary categories as well as breaking down categories by community, indicators reflected the level of overall security in each particular community.

7 The coding tool for classification can be found on our website: everydaypeaceindicators.org

10

Reflecting the malleable and sometimes elusive nature of what peace means to individuals, there was significant variance in the different communities about how they defined peace and what environmental indicators they used in order to recognize it. Importantly, we found that those communities that were further away in time from violent conflict tended to have a higher level of positive peace indicators than those closer to violent political conflict and crime. This finding suggests that the post-conflict areas we studied are dynamic and that perceptions mirror circumstances (although not necessarily individual events). It also reinforces findings from previous studies using local level data. Patrick Vinck and Phuong Pham compare data from the Eastern DRC and Northern Uganda to demonstrate that the further a country is from conflict, the more development-related needs will take priority, while security takes precedence in places where conflict is ongoing (Vinck and Phuong 2008). Also, quite logically, those more urban communities that had more contact with crime had higher levels of indicators in the crime/security category. The findings resonate with Maslow’s basic needs conceptualization – a notion that has played a significant role in the development of peace and conflict theory (Burton 1993). As people become temporally removed from conflict and pre-occupations with imminent security threats, they can turn their attention to more ‘social’ issues (Maslow 1943).

Table 4. Indicator Categories Uganda

If we take a closer look at the three communities studied in Uganda, for example, although overall the daily security indicators were most prevalent in two areas, cohesion and interdependence were high for all three. If you disaggregate the two rural communities of Odek and Atiak from the urban community of Kasubi, you find that the rural communities had a higher percentage of indicators related to infrastructure, education (although marginally on this issue), routine social practices, health and food/agriculture (and a relatively low level of concern with security – 8.7% of indicators for Odek and 23.5% for Atiak) whereas the urban community of Kasubi was highly concerned with issues of security (42.3% of indicators). This reflects the reality of an urban post-conflict context of economic and war-related IDPs living in a densely populated space that has had a recent influx of Dinka refugees from South Sudan – this has in turn created conflicts with native Ugandan Acholis. This reflection of the reality on the ground and proximity to conflict is also reflected at the national level. In Uganda, the security categories contained only 25.8% of indicators and in

11

Zimbabwe 26.7%. Routine social practices and development feature prominently as indicator categories in Zimbabwe and Uganda. These countries are several years post-conflict and the communities (except perhaps Kasubi) are experiencing relative calm. Therefore it makes sense that the rest of the indicator categories were related to positive peace issues such as infrastructure, economic development, education, freedom/human rights, cohesion/interdependence and food/agriculture. Importantly, however, communities and countries experiencing higher levels of violent conflict chose higher levels of negative peace indicators. South Africa’s top category was daily security (28.4%) and all three security categories constituted 54.1% of the indicators chosen in all three South African pilots. This was a little less prevalent in South Sudan where daily security represented 23.4% of indicators and security/crime represented 10.4%, security/forces 9.1% for a total of 42.9% (42.9%) of indicators falling into security related categories.The community-identified indicators suggest that the relationship between citizen security and peacebuilding requires further exploration, especially in the light of continuing problems of crime and insecurity in ‘post-conflict’ areas such as South Africa. This suggests the need for prudence when thinking of end dates for political violence. A categorization difficulty arises because in a number of cases violence that could be primarily designated as ‘political’ has morphed into violence that is better described as ‘criminal’. Certainly many of the focus group transcripts, and the choices of indicators, reflect community pre-occupations with crime and fear of crime. Only a few of the transcripts make explicit links between prevailing crime (and fear of crime) and the fore-going conflict. Thus, for example, former combatants, formerly abducted persons or the availability of weapons are rarely (if ever) mentioned in relation to crime.

Figure 1. Communities Most often Define Peace in Terms of Security

An Argument for Complementarity

We now move on to discuss the potential for bottom-up indicators, such as the Everyday Peace Indicators, to complement existing indicator systems. It is worth noting that the EPIs are not alone in experimenting with bottom-up indicators. As increasing numbers of organizations investigate community-sourced and alternative approaches to gauging opinion, there is a necessity to think through the interstices of indicator systems and facilitate a dialogue between top-down, bottom-up and intermediate perspectives. The Uppsala

12

Conflict Data Program Georeferenced Event Dataset (UCDP GED), the Armed Conflict Location & Event Data Project (ACLED), the Human Development Index (HDI), and the Global Peace Index (GPI) have been chosen as points of comparison because they measure some element of peace, positive or negative, and are used among peace researchers and practitioners to gauge levels of peace and safety in countries and communities. The teams behind the UCDP GED, ACLED and the HDI have also been transparent on their methodologies, giving external parties an opportunity to scrutinize the assumptions and mechanics behind their data gathering, coding and analysis (Klugman, Rodríguez, Choi 2011; Raleigh, Linke, Hegre & Karlsen 2010; Eck 2012). The Global Peace Index has been less transparent on their measurement strategies (access to their data and information on their coding tools does not seem to be available) and so we must be less confident on their usefulness as a point of comparison.

While we make an effort to compare the EPIs to these systems, and scope out the prospects for complementarity, it is important to note that we are not arguing to replace existing measurement tools with our own and instead see the utility in complementarity or a side-by-side analysis of different tools. The genesis, units and levels of analysis, and means of categorization of the EPIs are different from existing indicators and methods of capturing peace or conflict. What we can anticipate, however, are forms of complementarity through which bottom-up and top-down indicators can work alongside each other. This emphasis on augmentation, rather than championing one methodological vantage point over another, is in keeping with attempts to see conflict in a comprehensive way. It is also in keeping with the ethos of many top-down indicators that have a track record of working together or alongside each other. Our suggestion is that this complementarity can be further enhanced with reference to bottom-up indicators. The layering and sequencing of indicators should be able to render more complex and nuanced representations of conflict-affected societies. This added value may come in three ways.

First, bottom-up indicators may be able to reinforce or contradict the validity of international policy and academic priorities. If communities in conflict affected areas identify issues of concern that are not on the international or national policy radar then that suggests that international and local priorities are mismatched. There may be many

13

reasons for this, one of which may be that international actors (admittedly a heterogeneous category) have limited means of accessing localized narratives and priorities.

Second, bottom-up indicators may be able to add more textured and finer-grained detail that is not available at the national level. By their nature, national-level indicators rely on abstraction and generalization. Their advantage is in their economy of being able to aggregate a possibly diverse story into comprehensible data. Bottom-up indicators, depending on their nature and the categorizations they employ, may be able to supplement the generalizations of top-down indicators with the particularity of localism thereby covering all the content areas of interest encompassing a concept like peace (Adcock and Collier 2001). This could have a significant impact on the approach and conceptual perspective of existing measures of peace, potentially transforming the way indicators are currently used to measure peace and other difficult to measure concepts.

Third, bottom-up indicators can reveal sub-state variations in data. One aspect of national-level data is that it flattens out sub-state experiences to produce a generalized account of a phenomenon. Community-level, bottom-up data can show how a national account of peace or conflict can be unrepresentative of localized experiences. Of course, the extent to which community-level surveys or indicators can help to augment the national-level picture is very much dependent on the number of localities included in any study, and the extent to which they may be representative.

In thinking about comparison and complementarity of the indices and datasets described above with the EPIs, three further observations are worth making. The first pertains to units of analysis. While the EPIs are restricted to the inclusion of community actors in creating their own indicators, the other four sets of measures concentrate on conflict events (UCDP GED and ACLED) or states (HDI and GPI). It is also worth noting that the emphasis on states risks reinforcing formal and institutionalist worldviews that often are not shared by inhabitants in conflict-affected societies where the state may be an absent, remote or occasional phenomenon (Kabamba 2010). The second point is that three of the sets of indicators (UCDP GED, ACLED and GPI) betray a bias towards negative conceptualizations of peace. That is, in many of

14

their measures, peace is measured in terms of direct violence (e.g., battlefield casualties) or a societal ‘deficiency’ (per capita prison population). The HDI, with its focus on more sociological data, more easily occupies the realms of positive peace (to the extent that they are designed to capture peace at all). The third point is that the established indicators tend to rely on aggregate data, often collected by others (for example, national statistical agencies or through media reports) and in top-down ways. The potential pitfalls of relying on such sources has been well-discussed elsewhere (Barron and Sharpe 2005: 46-49).

It is precisely for the reasons discussed above that a system that is more participatory, locally sourced and fieldwork based is necessary to complement these existing systems. Having made these preliminary points, it is worth delving deeper and drawing out some of the differences between established indicators and bottom-up approaches as evidenced through the EPI project.

Capturing the sub-state dimension: Geo-referenced Event Datasets

To illustrate points of comparability and complementarity between different datasets and levels of analysis, we draw on EPI data from South Africa. A first point relates to the ability of peace and conflict datasets and indicators to ‘see’ sub-state conflict. The prevalence of sub-state violent conflict is well known (Black 2013), as is its potential to spread. The UCDP GED and ACLED are examples of efforts by scholars to record conflict events at the individual level. Clearly apartheid-era South Africa was the scene of politically motivated violence. While it is impossible not to put high crime levels in a political context, and to identify some legacies of apartheid in continuing violence, it is clear that contemporary violence in South Africa can be regarded as primarily criminal.

The UCDP GED disaggregates conflict events geographically and temporally at an individual level and includes events corresponding to years where the conflicts did not exceed the 25 battle-related deaths required for inclusion in the aggregate conflict datasets. UCDP GED is available primarily for Africa for the period of 1989 to the present and provides fatality data for events of organized violence as well as the

15

actor and conflict ID associated with that event.8 It relies heavily on media sources to document its data, although it does cross-check news reports with other sources such as UN and INGO reports (Eck 2012). The UCDP GED has strict guidelines for the inclusion of actors in its various categories of organized violence. In terms of its intrastate conflict category “actors must be the government of a state or an organized group with a name” and for non-state actors, the requirement is relaxed and allowance is made for “communal groups that engage in conflict with each other” (Eck 2012: 135). At face value, either category includes the gang warfare that is prevalent in many South African townships and suburbs and is a direct consequence of the legacy of apartheid and the deeply entrenched culture of violence that was produced by decades of repressive racial policing, violent crime and social conflict (Kynoch 2005). However, the EPI and UCDP GED data diverge in relation to this kind of intrastate conflict. The UCDP GED data does not pickup any conflict data in this area after 1999, instead only documenting violence associated directly with the consequences of apartheid and disregarding chronic violence and conditions that may persist. It does this even though local level research often points to the chronic nature of violent conflict and the absence of firmly identifiable start and terminal points (Steenkamp 2009).

ACLED, another widely used event dataset based at the University of Sussex, documents all reported political violence events in over 50 developing countries and includes events that occur within civil wars and periods of instability from 1997-2014. It is, therefore, not limited to conflict that is defined as civil war or other limited categories like the UCDP GED. In fact, ACLED gathers data on South Africa to the present day and in its most recent report at the time of writing (May 2015) classified the country in its most urgent “escalating” category of the conflict trajectory. Yet, ACLED still limits its events data to “political violence” therefore excluding potentially related criminal violence that does not have an explicit “purpose of pursuing a political agenda.” This means that that South Africa has been categorized as “no change” with a few intervals in the second ACLED category of “warning” until

8 UCDP GED data for GED Asia, Afghanistan, India and Indonesia was still incomplete at the time of writing. UCDP defines organized violence as state-based armed conflict, non-state conflict and one-sided violence. See http://ucdp.uu.se/ged/data/ucdp-ged-points-v-1-5-codebook.pdf for more information.

16

March of 2015, which it attributes to service delivery protests leading up to the 2016 elections. Nevertheless, ACLED does include nonviolent events such as headquarter establishment, protesting and rebel presence/recruitment (Raleigh et al. 2010). This gives us an indication of potential change, but does not reflect the experience of individuals and communities on the ground. It gives us little context of the daily concerns, perceptions and experiences that ebb and flow with the variance of conflict and peace.

The EPIs, on the other hand, reflect concerns about daily security, crime and discrimination in Hanover Park and Atlantis, two communities in the Cape Flats outside of Cape Town, an area where many of South Africa’s colored and black Africans were relocated during apartheid. These areas have deeply entrenched and powerful gangs that began to gain power in the 1980s nurtured by the instability created by social dislocation and related organized crime, drug trade and other illicit activities (Pinnock 1984).

The EPI indicators emerging from Atlantis and Hanover Park tell a story of concerns about daily security and crime and a disappointment with peacebuilding efforts in their communities (See Table 4). They demonstrate unease among residents with the categorization of gang violence simply as criminal violence and accentuate the racial dimensions still prevalent in people’s perceptions of their peace and safety (Jensen 2010; Lemanski 2004). For example, an indicator from the community-sourced short list in Hanover Park demonstrates that people do not only feel safer when there is a police presence in the community, but they feel even more at ease when that police presence includes coloured police officers – something which is clearly important to a community that identifies itself primarily as coloured (94% of inhabitants according to the 2011 census) and something that reflects the historical dimensions of the relationship these communities have had with the South African police. Thus, the EPIs reflect a concern with the lasting legacies of political violence in post-conflict contexts (Gibson 2003).

In addition, the preliminary survey data from the first round of surveys suggests that people feel highly unsafe in their community. When asked in Hanover Park whether it is safe to allow children to play away from home, 82% of respondents chose “Never” or “Rarely.” When

17

asked if they were able to walk freely at night, 85% of respondents answered “Never” or “Rarely” and when asked whether there was a presence of colored police officers, 74% responded “Never” or “Rarely.” Only 16% of respondents “Always” or “Often” felt safe to go to the Terminus and only 14% “often” or “always” felt safe attending festivals (whereas 70% responded that people “Often” and “Always” had parties – an indication of instability according to the focus group discussions).

Table 5. Everyday Peace Indicators for Hanover Park and Atlantis (Short List)

Another important distinction that sets the EPI data apart from geo-referenced, events datasets is that EPI focuses on the people’s perceptions of their sense of peace and safety in their community. Although these are often based on concrete events, perceptions are not always predictable and are influenced by multiple factors in addition to actual events. Also, importantly, perceptions of events might be more fundamental to response and event prediction than the actual events themselves. Therefore, collecting data on perceptions of peace and safety might give us additional tools to not only predict, but also measure the effectiveness of certain interventions in a community.

Geo-referenced, events datasets such as ACLED and the UCDP-GED are useful for counting sub-level fatalities and action events related to political violence and civil war. Their unit of analysis is an event that occurred at any given location according to their specifications. This gives us important information regarding the amount of activity related to violence and politics that is happening in any given context and can help to measure the effectiveness of peacebuilding and peacekeeping activities. However, these measures are limited to activities that are considered political and therefore often omit important events and processes that are directly related to the escalation of conflict. The databases are also limited to certain events; in other words, something has to happen so that it can be recorded in their systems. This restricts any kind of measurement of positive peace since these would not be considered events according to their criteria, which means that beyond the counting of political events, there is little detail about how stable or unstable an area is. In fact, political events are not always indicators of

18

a deterioration in peacefulness, although they can be an indicator of increased political participation and democracy. Finally, these databases are also highly dependent on media sources, which are often unreliable (especially in conflict affected contexts) and consequently measurements occur at a distance without any real interaction with actors or communities on the ground.

Complementing Macro Indices Having examined the potential for bottom-up indices to complement event datasets (specifically, UCDP GED and ACLED) we now turn our attention to the potential for complementarity with macro indices (specifically HDI and GPI). The inability of many national-level databases and indices to capture sub-state dynamics, or a more multifaceted and positive understanding of peace, is evidenced by the HDI and GPI mechanisms (Raleigh et al. 2010). The HDI was created by the United Nations Development Program to emphasize that people and their capabilities should be the ultimate criteria for assessing the development of a country, not economic growth alone. In 2013, the HDI measured development in 187 states using three indicators: health, knowledge and income. The HDI is widely considered a measure of peace since economic and social development are often taken as proxy indicators for peace, and indeed an immense academic and policy literature stresses the correlation between conflict on the one hand and economic decline and bad governance on the other (World Bank 2011: 2). Yet, using this approach does not allow for the kind of micro and mezzo analyses that sometimes are necessary when measuring peace on the ground. Consider the example of Sri Lanka, a state that has suffered sustained civil war with the attendant problems of social dislocation, inter-ethnic discrimination and the distortion of the economy (Holt 2013). Notwithstanding these problems, it is possible to find measures of peacebuilding in Sri Lanka that report success, even from time periods when the conflict was demonstrably worsening. The United Nations Development Program indicators show Sri Lanka experiencing a steady increase in its Human Development Index (HDI) score from 1980 onwards (UNDP 2011). This is despite severe intra and inter-ethnic conflict with high casualty levels, a campaign of violence by the Liberation Tigers of Tamil Eelam that deterred much foreign investment, the direction of much government spending towards military purposes, and the 2004 tsunami that left an estimated 34,000 dead. At no stage, and despite these national

19

traumas, does the HDI figure fall. The key point is that national-level statistics can mask sub-state variations (for example, violence in a particular area of a nation state). Problems may also arise if government agencies are tempted to fabricate or massage official statistics, or rely too heavily on estimates or outdated benchmarks (IMF 2013: 10).

Relatedly, the Global Peace Index (GPI) is a more recent attempt to measure peace, one that has conceivably had more of an impact on international organizations than on the academic study of war and peace (Gleditsch et al 2014). Nevertheless, it is one of the few measures that purports to measure peace directly, even if many of its indicators are more about conflict than they are about peace. Like HDI, the GPI measures peacefulness at a national level based on twenty-two indicators. Data is collected to measure these indicators from the Uppsala Conflict Data Program, the Economist Intelligence Unit, the United Nations, the International Center for Prison Studies, the International Institute for Strategic Studies, the Stockholm International Peace Research Institute and the Bonn International Center for Conversion. The indicators are mainly measures of conflict, violence and war - thus heavily focused on negative peace. Clearly, the GPI measures peace at a much more macro level than the EPI does, but measuring peace at 30,000 feet with high-level statistical analysis and the use of several other data sources does not allow for an accurate snapshot of peace at the local level. For example, the GPI classifies Uganda and South Africa in the same category of “peacefulness,” ranking them 110/162 and 122/162 respectfully for 2014 (162 being least peaceful), whereas Zimbabwe was given a score of 148/162 and South Sudan a score of 160/162. The initial survey data from the EPI pilots suggested a much more variegated and localized picture within these countries that top-down indicators were unable to capture.

Even the national measurements of peace rely in large part on (sometimes questionable) national statistics and chosen indicators are decided by researchers.9 This means that what might be an accurate reflection of peace for one community or group may not be an

9 The GPI, for example, selected its indicators in 2007 with the help of an expert panel. The indicators are revisited on an annual basis, although – as far as we can tell - the original list has not changed since 2007.

20

accurate reflection of peace for another. Take the GPI’s indicator of “Security officers & police” – a police presence may or may not be a sign of peace to individuals depending on their race, socio-economic or immigration status, or if they have recently experienced violence. In the EPI example of Hanover Park, clearly the presence of coloured police officers was a sign of stability for community members, but an increase in the presence of security officers and police in Mbare, Zimbabwe may be a sign of instability since this may signal a repressive state. Therefore, more inclusive and participatory measures are necessary to allow communities to participate and tell policymakers and elites what these implications of the data actually are.

Finally, it is interesting to note again that communities that were further away in time from violent conflict produced indicators that were more inline with the HDI. In other words, their indicators were more predominantly focused on socio-economic factors such as education, food/agriculture, cohesion/interdependence and infrastructure. For example, only 25% (23.5%) of indicators in Atiak, Uganda fell into any kind of security category. In contrast, in Hanover Park 60% (56.5%) of indicators were security related – more inline with the kinds of indicators used by the GPI. This means that for communities, the variability in the definitions of peace are highly correlated with the distance in time from violent conflict. It also underscores the importance of national level classifications that are usually taken to apply uniformly across a nation state. South Africa is often classified as ‘post-conflict’, ‘post-transition’, post-authoritarian’ or, more specifically, ‘post-apartheid’ yet the local-level data suggests significant on-going violence and security concerns.

Conclusion

This article has considered the interface between top-down and bottom-up indicators of peace. In common with many other scholars and practitioners, it has expressed a concern at the capabilities of some indicator systems to capture, accurately and fully, the nature of peace and conflict. This concern reflects more than a desire for methodological rightness. Instead it is aware that major policy assumptions and decisions are based on indicator systems that give a partial, rather than full, picture of the reality on the ground.  Geo-referenced data collection and national level indices can be very useful in measuring events to tell us more about the fluctuation of peace and conflict at the state and sub-state levels. However, they do little to tell us what actual implications these events have on the relative peace and stability of a community, nor do they tell us about how these

21

events have been perceived by communities, which may be a more accurate predictor of future events. More inclusive measures are necessary to allow communities to participate and tell policymakers and elites what these events actually mean.

Our consideration of the possibilities that bottom-up indicators might bring to existing top-down indicators led to two conclusions. First, we noted that all indicators tell a narrative. In the case of community-chosen indicators the narrative is one of the issues that community members feel are important to them. In the case of top-down indicators, the narrative may be chosen deliberately to tell policymakers or academics about key trends and categories in society that are deemed to be important. By comparing these two sets of indicators we can see the extent to which the priorities identified by top-down and bottom-up indicators reinforce or contradict one another. For example, our finding that communities that were further away in time from violent conflict tended to have a higher level of positive peace indicators than those closer to violent political conflict and crime, indicates the possible variance in priorities depending on context.

A second conclusion is that bottom-up indicators can provide detail and texture that top-down indicators are unable to capture. This capturing of detail could be said about any qualitative research that operates at the local level, for example, gathering life histories through extended interviews or ethnographically-influenced community observation. The advantage of the approach taken by the Everyday Peace Indicators Project is that the research has multiple layers or stages that, hopefully, can reinforce the validity of findings and thus make them reliable as comparators for top-down indicators, as well as capture local level change. Thus, through the community indicator selection and vetting process, we are able to produce indicators that have robust validity among the community itself. Top-down measurements could adopt the EPI approach in order to more accurately create indicators to measure peace, reconciliation or other hard to measure concepts.

The localized nature of the Everyday Peace Indicators means that they do not have the extensive reach of national-level indicators, but we are able to rectify this to some extent through categorization of indicators.

22

Arguably, the localized nature of the indicators also gives us the layers and nuance that is necessary to examine conditions on-the-ground. Therefore, bottom-up indicators, like the Everyday Peace Indicators, provide sub-state capture that is not available from state-level indicators. The transnational and highly localized nature of much violent conflict is well known, so it makes sense to have at our disposal tools that can see all of these levels. Moreover, by having the indicators chosen (and indeed termed in the words of local populations) then we avoid the risk of overwriting local perceptions of peace (or the lack of it) with national or international versions.

The key point of this article is need for a plurality of vantage points from which to view peace or conflict. Such phenomena are necessarily complex and unlikely to be rendered accurately through a single methodological, ontological and epistemological lens.

ANNEX 1: EVERYDAY PEACE INDICATORS BY COMMUNITY (Shortlist)

23

References

Adcock, R. and D. Collier (2001). Measurement validity: A shared standard for qualitative and quantitative research. American Political Science Association Vol. 95, No. 03, pp. 529-546.

AfroBarometerhttp://www.afrobarometer.org/survey-and-methods/questionairresQuestionnaire South Africa 2011http://www.afrobarometer.org/files/documents/questionnaires/saf_r5_questionnaire.pdf

Barron, P. and J. Sharpe (2005) Counting conflicts: Using newspaper reports to understand violence in Indonesia, World Bank Social Development Working paper, no. 25. Washington DC: World Bank.

Black, N. (2013) When have violent civil conflicts spread? Introducing a dataset of substate conflict contagion. Journal of Peace Research, Vol. 50 no. 6: 751-59.

Brewer, J. (2010) Peace Processes: A sociological approaches. Cambridge: Polity.

Brück, T. (2014) Measuring peace, security and conflict: the challenges ahead for the SIPRI Yearbook, SIPRI Newsletter, 13 June. Available at: http://www.sipri.org/media/newsletter/essay/brueck_june13

Burton, J. (1993) Conflict: Basic needs theory. Basingstoke: Palgrave.

Crime Stats South Africa 2014http://www.crimestatssa.com/precinct.php?id=1076

Cunningham, D.E., K.S. Gleditsch and I. Salehyau (2013) ‘Non-state actors in civil wars: A new dataset’, Conflict Management and Peace Science 30(5): 516-31.

Eck, Kristine. 2012. In data we trust? A comparison of UCDP GED and ACLED conflict events datasets. Cooperation and Conflict, vol. 47 no. 1: 124-141.

24

Gibson, J.L. 2003. The Legacy of Apartheid: Racial differences in the legitimacy of democratic institutions and processes in the new South Africa. Comparative Political Studies 36, 7: 772-800.

Gleditsch, Nils Petter, J. Nordkvelle and H. Strand. Peace Research – Just the Study of War? Journal of Peace Research, Vol. 51 (2): 145-158.

Guelke, A. 2012. Politics in Deeply Divided Societies. Cambridge: Polity.

Guelke, A. 2014. Brief reflections on measuring peace. Shared Space: A research journal of peace, conflict and community relations in Northern Ireland 18: 105-112.

Gulu District Local Government Statistical Abstract 2012/2013http://www.ubos.org/onlinefiles/uploads/ubos/2009_HLG_%20Abstract_printed/CIS+UPLOADS/Higher%20Local%20Government%20Statistical%20Abstracts_2014/Gulu.pdf

Holt, S. (2013) ‘The limits of formal metrics during conflict and post-conflict transition: Exploring opportunities for qualitative assessment in Sri Lanka’, Journal of Intervention and Statebuilding 7(4): 431-52.

Ioannou, M. (2015) ‘Measuring Peace: What’s the score?’ UNDP

Irmer, C. (2009). A systems approach and the Interagency Conflict Assessment frame- work. The Cornwallis Group XIV, Analysis of Social Conflict and Counter-Insurgency (pp. 168–175).

Jensen, S. 2006. Capetonia Back Streets: Territorializing young men. Ethnography 7, 3: 275-301.

Jensen, S. (2010) The security and development nexus in Cape Town: War on gangs, counterinsurgency and citizenship. Security Dialogue 4(1): 77-97.

Jensen, S., 2014. Intimate Connections: Gangs and the political economy of urbanization in South Africa. In J.M. Hazen and D. Rodgers eds., Global Gangs: Street violence across the world. Minneapolis: University of Minnesota Press, pp. 29-48.

25

Justino, P., T. Brück, and P. Verwimp eds., (2013) A Micro Level Perspective on the Dynamics of Conflict, Violence and Development. Oxford: Oxford University Press.

Kabamba, P. 2010. “Heart of Darkness”: Current images of the DRC and their theoretical underpinnings. Anthropological Theory, vol. 10, no. 3: 265-301.

Kelley, J. G., & Simmons, B. A. (2015). Politics by number: indicators as social pressure in international relations. American journal of political science, 59(1), 55-70.

Klugman, J., Rodríguez, F., & Choi, H. J. (2011). The HDI 2011: New controversies, old critiques. UNDP Human Development Reports Research Paper.

Kreutzmann, H. (2001). Development indicators for mountain regions. Mountain Research and Development, 21(2), 132–139.

Kynoch, Gary. 2005. Crime, Conflict and Politics in Transitional-Era South Africa. African Affairs, Vol. 104, No. 416: 493-514.

Lemanski, C. 2004. A new apartheid? The spatial implications of fear of crime in Cape Town, South Africa. Environment and Urbanization 16(2): 101-112.

Lujala, P., J.K. Rød and N. Thieme (2007) ‘Fighting Over Oil: Introducing a new dataset’, Conflict Management and Peace Science 24: 239-56.

Luyt, R. and D. Foster eds., 2001. Hegemonic Masculine Conceptualisation in gang culture. South African Journal of Psychology 31(3): 1-11.

Mac Ginty, R. and O. Richmond (2013) ‘The Local Turn in Peace Building: A critical agenda for peace’, Third World Quarterly 34(5): 763-83.

Mac Ginty, R. and P. Firchow. (2016). Everyday Narratives in World Politics. Politics doi: 10.1177/0263395715622967

Maslow, A. H. (1943) ‘A Theory of Human Motivation,’ Psychological

26

Review 50: 370–396.

McCartney, C. and P. Nolan, 2014. Tracking progress in building peaceful and shared societies. Shared Space: A research journal of peace, conflict and community relations in Northern Ireland 18: 4-12.

Merry, S. E. (2011). Measuring the World. Current Anthropology, 52(S3).

Miller, C. A. (2005). New civic epistemologies of quantification: Making sense of indicators of local and global sustainability. Science, Technology and Human Values, 30(3), 403–432.

Mitzen, J. 2006. Ontological Security in World Politics: State identity and the security dilemma. European Journal of International Relations 12(3): 34-70.

Narayan, D. (2000) Can Anyone Hear Us? Voices of the poor. Washington, D.C.: World Bank.

Parkins, J., Stedman, R., & Varghese, J. (2001). Moving towards local-level indicators of sustainability in forest-based communities: A mixed-method approach. Social Indicators Research, 56(1), 43–72.

Pinnock, D. 1984. The Brotherhoods: Street-gang culture and state control. Cape Town: David Philips.

Podobnikar, T. (2007) Production of integrated digital terrain model from multiple datasets of different quality. International Journal of Geographical Information Science 19(1): 69-89.

Raleigh, C., Linke, A., Hegre, H., & Karlsen, J. (2010). Introducing ACLED: An armed conflict location and event dataset special data feature. Journal of Peace Research, 47(5), 651-660.

27

Regan, P.M. (2014) ‘Bringing Peace Back In: Presidential address to the peace research society 2013’, Conflict Management and Peace Science 31(4): 345-56.

Roberts, D. (2011) Liberal Peacebuilding and Global Governance: Beyond the metropolis. London: Routledge.

Steenkamp, C. (2009) Violence and Post-war reconstruction: Managing insecurity in the aftermath of peace accords. London: IB Tauris.

Tseng, Y.H., P.W. Liu and F.C. Chou (2008) ‘Modeling urban landscapes with integrated datasets’, Journal of the Chinese Institute of Engineers 31(5): 853-9.

Vinck, P., & Pham, P. (2008). Ownership and participation in transitional justice mechanisms: a sustainable human development perspective from eastern DRC. International Journal of Transitional Justice, 2(3), 398-411.

Wilton Park (2014) Conference Report: Measuring Peace Consolidation, 15-17 October 2014. Available at: https://www.wiltonpark.org.uk/wp-content/uploads/WP1355-Report.pdf.

UCDP GED code-bookhttp://ucdp.uu.se/ged/data/ucdp-ged-points-v-1-5-codebook.pdf

UCDP GEDhttp://www.ucdp.uu.se/ged/#__utma=1.1969803115.1409923317.1425565024.1426078186.7&__utmb=1.20.10.1426078186&__utmc=1&__utmx=-&__utmz=1.1426078186.7.6.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)&__utmv=-&__utmk=160120381

28